Diving into AI as a Jaded Technologist

Having worked in the software engineering industry professionally for coming up on 15 years, I've seen quite a few hype cycles and fads come and go in various states. Blockchain, NFTs, Microservices, Gatsbyjs, SSG - are but just some that have rocketed themselves into the minds of us geeks more recently. Generally, the popularity and relative impact and value provided by each of these topics can be quite disconnected.

The best way I understand them is with the Gartner Hype Cycle:

My understanding of the above is that a new shiny thing comes around and everyone get's incredibly excited about its possibilities. New startups form out of nowhere as eager and wide-eyed entrepreneurs rush to get the advantage of being early adopters. Money starts getting thrown around as investors don't want to miss out on The Next Big Thing™.

This time is super exciting, but then we start entering the trough of disillusionment as a lot of promises cannot be kept. Reality sinks in and once we get past the honeymoon phase, we realize this new thing isn't going to solve all of our problems and hard work is still needed to be successful.

Eventually, most of the startups falter and disappear but some persist and make it back up the slope of enlightenment and we eventually land into the plateau of productivity.

This is where I usually get hung up on it. All too often I view newer technologies as never really getting out of the trough of disillusionment and not worth my time. Heck, I've been known to espouse platitudes around the notion that "Boring tech is sexy tech" - to which I still greatly agree!

But the more I see what's going on with AI, and its actual lasting power being adopted by real businesses, the more I believe we're actually getting out of that trough.

Time for me to get on board. Let's hope I'm not too late.

What am I doing about it?

Github Copilot

We've been using Github Copilot ever since it launched over at Seed Health. It's been a nice-to-have but not a huge game changer for our workflows. The autocomplete works well enough and it can generate code within our IDEs, but it gets hung up on existing code bases and tends to behave like a newer engineer - defaulting to refactoring code instead of truly understanding the existing constraints and making an informed decision on how to proceed.

AI-focused IDEs

The market is being flooded with tailored developer experiences that provide code autocompletion, in-application chats to provide guidance on how to proceed with a particular problem and even entire prompts and workflows that automatically implement the desired changes in your files all from a single prompt.

• Cursor appears to currently have the most traction and integrates incredibly well with a wide suite of premium models out there. I've used this sparingly as I haven't personally committed much money into building out my own projects (ie this website).
• Windsurf by Codeium is a recent upstart at this and really impressed me as I've used it to release several new features in this blog, entirely via its "Cascade" feature. It worked less well making tweaks to existing parts of the website but that could just be my relative inexperience with prompts.
• Continue is an excellent plugin for VS Code that provides chat, code autocomplete and more. Being able to do this in my existing IDE is nice given my familiarity with it. I also like it because it allows me to use my own local models...

Self Hosted Models via Ollama

I've held off diving too deep into some of the premium services out there due to my lack of "justification" in needing to spend money for my personal projects. This lead me into playing around with Ollama on my existing Gaming PC. It has a abundance of CPU power, RAM and a strong-enough GPU to run models locally.

This flexibility has been a game changer as I can truly play around with the systems and their capabilities without it costing an arm and a leg (aside from power costs and my inherent geek desire to upgrade my computer to be more and more powerful).

I can point Continue.dev to my local models and it works great, while also installing Open WebUI and making it publicly accesible - meaning I can run my own prompts and queries on any device even when I'm not at home. This probably deserves its own blog post or two - in due time.

What's next?

This is the fun part - basically anything. I can continue using this (silly) website to push my practical understanding and use cases for AI and LLMs for software engineering practices.

I've also begun doing courses at Google Cloud Skills Boost. This has already been great to give me foundational knowledge of GenAI as well as an excuse to learn more about GCP and what it has to offer.

Not only is this fun, but it's also a good hedge against what every software engineer fears the most - becoming stagnant in an ever changing industry.

Cover image: This blog's cover was generated using AI.