Building Agentic AI ... Fancy.

how about a simple SQL Agent?

I’m getting to the point where I’m a little burnt out on hearing about AI this and AI that. Work, LinkedIn, YouTube, Substack … it’s a never-ending glut of AI-generated content talking about AI. I get the sense that the closest most developers get to AI is asking Cusor to puke out the next code chunk.

Classic.

I’ve never been one to make a mountain out of a mole hill. I find myself adrift in the middle ground of the AI purgatory we find ourselves in as writers of code.

There seem to be two deeply entrenched cabals of software engineers.

• AI is for the weak; humans forever.

• Vibe code your way to glory.

The truth is that the middle is most likely slanted towards the AI is the future group. You cannot turn back the clock once you’ve opened Pandora’s box. Will there always be a bright future for intelligent and industrious writers of code and systems? Absolutely.

Is AI here to stay and an extreme force multiplier in the creation and execution of software of all kinds? Absolutely. Old dogs die hard, and the old guard is having a hard time swallowing the new reality.

---

Let’s bring it all back down to earth.

I am no AI expert, but I’ve probably done more than most, including fine-tuning my own LLM. I don’t get overly excited about building AI, insofar as it’s actually helpful and interesting problems to solve.

I would guess that there is a fair number of Data Engineers and others who may be overwhelmed by the fast pace of development in AI and LLMs, and feel a little left behind and lost. I assure you, there is hope.

Just remember, all those liars on LinkedIn and Reddit trying to be AI smart, they breathe the same air and drink the same water you and I do. I’m going to prove that to you by showing how easy it is for you to build an AI SQL Agent.

Forget all the too-bright suckers, confusing acronyms; you, my friend, are capable of building “Agentic AI.” Don’t listen to the haters. Let’s do this.

---

Agentic AI

Let’s start with a simple concept: what is Agentic AI, and those Agents you hear about regarding LLMs? Let’s do what any normal person would do, and ask AI (ChatGPT) what it thinks Agentic AI is.

This is a decent response, I think.

Since I am a Grug-brained developer, I would summarize it like a good old Midwest-born, half Bigfoot, rascal that I am.

• LLM → knows a little about everything (question -? answer)

• Agent → takes it a step further and can DO A THING

An AI Agent will do something, something that a LLM (a binary model blob in memory somewhere) needs help doing. It’s about system interaction, not just answering a question.

Maybe an AI Agent searches the internet for recent blog posts, polls an API for a result, or creates a file for you like a cursor. Generally speaking, an Agent is going to be good at DOING a particular thing.

One of those things ChatGPT will NOT do for you would be to access, query, and summarize the results of some data sitting in a database … not just a database … YOUR database.

Let’s build an AI SQL Agent.

---

How Agentic AI can be helpful.

Imagine you are a CTO somewhere, let’s say SanFran, that beacon of light … oh wait … I mean darkness … where the Illuminati tech bros plan the demise of humankind. I digress.

Let’s pretend you’re a CTO and you have a bunch of people in the C-suite and elsewhere constantly asking about sales numbers. How much have we sold? Who’s sold the most? What were the numbers last month? How are we doing on sales so far this month?

No one uses the crusty Dashboards that were set up; one, they don’t know how to use them, and they are half-broken and slow much of the time. Then you get a brilliant idea. All that sales data is sitting in a database. Why not connect a SQL Agent to it, adopt Agentic AI, and have people interact with an LLM to get the answers they want?

You fall asleep that night dreaming of being paraded around the company parking lot riding a white stallion, crowned with laurels, like a new Caesar.

---

Building a SQL Agent as an introduction to Agentic AI.

Ok, so hopefully after that pretend discussion above, you can see the use of what we are trying to do (As a side note, an Agent is only as helpful as the problem it solves for you … aka … what it does).

In our case, it seems helpful because we don’t have direct access to a Dashboard; in fact, we can delete the Dashboard, the person who made it, the pipeline that filled it, and the compute that ran it. Delete. Delete. Delete.

We can replace it with a Sales SQL Agent ready 24/7 to answer whatever you need about sales numbers.

Let’s talk about it at a high level: which components are needed?

- Some UI / App (probably web based)
- Web Server to serve that web app
- LLM (let's just someone else's like OpenAI)
- SQL Database with our data
- Code to glue it together (let's use LangChain)

It’s really not that complicated, and that’s the point I’m trying to make. Pretty much every single thing on the list above is a solved problem. Tools like LangChain make building Chat workflows super easy.

I think what throws most people off at first about building AI Agents, and was annoying for me as well … was that it includes the Full Stack. Have you ever wanted to be a full-stack engineer? Well, this is your chance.

I have all the code available on GitHub; it’s wrapped in a Docker Compose command so that anyone can run it. You just need an OpenAI API key. Here is the result first.

Look, it ain’t rocket science, friend. My guess is that, as far as SQL Agents go (putting an LLM on top of your database(s), it would not be much more complicated than this.)

You might have too …

• Provide more context with DDL and JOINS to the LLM

• Provide specific instructions about the data in the tables, etc.

But you will be surprised how well general LLMs are writing SQL queries against any data and answering most questions with ease.

To show you that there is no magic here, let’s talk more about this “architecture,” aka the different technology pieces involved here. Remember, this is what high-paid consultants call “Agentic AI” and charge obscene amounts of money to build.

Let’s look at our Docker Compose file for this project, remember, it’s all on GitHub, and see at a high level the different components. You will see two things you’ve seen a thousand times.

• Backend

• Frontend

Who would have thought?

Let’s also look at two Dockerfiles: one for the backend and one for the frontend. We could call these “services” ehh???

You are clever folk; there are about a thousand ways to deploy a service, a million SaaS options under the sun, or you could wing it and just run docker-compose up on a bare-metal instance.

I mean, if you’re looking for rocket science, you ain’t going to find it here, my friend.

Can I be honest with you?

There are some positives to the AI arms race for Developers. So many companies and frameworks are vying for the hearts of the programmers and AI developers that they have made it SUPER easy to build Agents and every kind of LLM workflow possible.

This means less work for us. I mean, look at our SQL Agent code. Flipping langchain comes with a SQLDatabaseToolKit for crying out loud.

I mean, good Lord, could it be any easier to make an SQL Agent???

Look, all I’m trying to do here is make a point. So keep your spicy comments to yourself. If you spend all day on YouTube and LinkedIn, you might conclude that you’ve been left in the AI Race dust and have no hope for the future.

You see job postings for “AI Engineering,” building “ChatBots,” and “AI Agents,” and your eyes glaze over.

I’m not trying to downplay good AI Engineers and the difficult task of putting an Agentic Agent(s) into production. But I do want to let you in on a secret.

The complex parts aren’t what you think.

• How to integrate Agent LLMs into more “complicated” tools

  • say some internal specialized API or systems calls chained together

• How to set up the infrastructure

  • backend

  • frontend

• How to keep costs down

• How to keep latency down

• How to apply governance and security

• How to find impactful product-driven use cases

• Fast iteration and development

It isn’t actually hard to use tools like LangChain to do anything LLM-related. Backends have been around forever, services, web apps, front ends … all that stuff is old hat.

What is new? RAGs, unstructured documents, tying various datasets together, governance of LLMs (input and output), managing AI costs, tightly coupled infrastructure, and deploying multiple Agents.

These are simply classic Software problems. Remember, all this code is on GitHub. I encourage you to clone it and play around with it.