How to stay relevant in this AI world?

This week I went for a coffee with my friend Drazen Zaric, ex-colleague of mine from almost 10 years ago at Nordeus, one of the best companies I have worked for.

Drazen is one of the most talented data/ML/AI engineers I have ever met, and he is back in the AI game after a couple of months building his own product, so if you are hiring for AI engineers that have a product-first mindset, and need to catch a big fish (well, who doesn’t?) hit him on LinkedIn.

We both agreed how impressive it is the speed at which current AI systems are evolving, especially the ones used for assisted coding, like Claude Code, Cursor and friends.

For example

This week OpenAI (and previously Google) claimed to have built an AI system that scores gold in the International Mathematical Olympiad. That is freaking nuts! And I am saying this not because I am an AI influencer claiming that AI has killed Mathematicians, but as someone who 20 years ago participated in the IMO and could not solve even one problem.

Drazen then asked me about the bootcamps I teach at the moment, who are my students and what we built in each cohort.

Then he sipped a bit of coffee and asked the question:

“Given the speed at which AI coding assistants are improving, do you think that people will still enrol in your AI engineering bootcamps in 5 years?”

And this made me think a lot.

Not only about the future of my AI teaching and consulting business, but the future of the AI engineer job (your job), and the future of any other job in this world.

Let me share with you my thoughts and some pieces of advice. I will take some tangents here and there because this week I am on holiday, and this newsletter is just me dumping my stream of thoughts on paper in a very raw manner, before I go back to the water with this little one.

LLMs alone cannot do it

LLMs alone are impressive, but they cannot do much unless you provide the right context to them. In other words, everything an LLM can output is either

• baked in its weights (aka memories built during training), or

• provided in the input context window.

Everything else does not exist for an LLM, and cannot be used by any LLM to craft a good/useful answer.

Now, LLMs have been trained on vast amounts of data, and have memorized a lot of information.

This is the reason why any decent LLM will solve the “give me the capital of country X” task, out of the box. They just memorized that information.

Now, for tasks that involve private information (like your company’s client purchase history), LLMs alone cannot do it. Instead, you need to expand the prompt with contextual information that you can fetch using from external data sources, either with classic RAG or with more advanced tool calling. This step is doable, and the problem can be solved.

Finally, for tasks that require private information and are very complex, for example:

“Hey LLM! Design, build and deploy an agentic trading system for the stock market”

No LLM or workflow of LLMs will solve it. And this is because this is a very complex problem and the context you provide is not enough.

In other words, you don’t have/provide enough context to help the system lead you to the solution.

What about agentic workflows?

Agentic workflows try to solve a complex task by breaking it down into smaller subtasks, and applying the usual (input, LLM) → output mapping again and again, providing each time the right input, to the best LLM for that step of the workflow.

This sounds like a great idea, but it has one little problem.

Every mapping (input, LLM) → output can introduce errors, and errors compound.

Which means that after a few iterations, your agent is likely far from the right track, and the system gets back to square 0, plus the time/compute you spent for that.

The same applies to any real-world business/tech problems that require multi-step reasoning and action.

There are many places where things go wrong, but you are not capable to identify these moments, and the system just does not work.

So, what does the LLM need?

The expert human-in-the-loop

LLMs and agentic systems, even the best ones, need high-quality human feedback, either at training time, or at inference time.

• At training time → This is what AI researchers at OpenAI, Google, Anthropic… They craft either the training dataset, or algorithm that generates the training dataset, and the training steps that, together with a few million dollars and weeks of compute, give a final product that works for a range of tasks.

  For example, an LLM system that can solve problems from the International Mathematical Olympiad.

• At inference time → This is what you need to do, for example, when you use Claude Code or Cursor Agent, to code, or when you as an AI engineer need to build an agentic workflow to speed up a human-labour intense task at your company, like researching potentially lucrative trading ideas.

Unless you, or your agentic workflow, provide high-signal context at each step, chances are the assistant will not speed you up. Oftentimes it will just take you on a never ending loop around the solution, but it will never hit it.

For example, when I am using Cursor to code in Python, I code 10x faster.

Why?

Because I know Python very well, and can filter out the output that Cursor spits out. Often I get answers that seem correct and work, but are very verbose and excessively complex.

Remember

Code that works but is excessively complex is no free lunch. The time you save today is just a fraction of the time that someone else in your team (maybe you?) will spend tomorrow understanding how to fix it.

On the other hand, when I use Cursor while building in Rust, a language I love but I am no expert in, the assistant is actually something that slows me down. And this is because I cannot provide good enough context and feedback to guide the assistant towards the subspace of correct solutions. So I easily hit partial solutions which are either excessively complex, or just wrong. And errors keep on compounding.

So having said all this, the next and final (obvious) question is …

How can I/you become an expert human in the loop?

To become an expert in AI, you need to understand how to build AI systems from A to Z. Because only when you know exactly what needs to be built, you can correctly ask the LLM what to write, and fix it when it does not.

Human expertise PLUS LLMs is acceleration.

My 2 cents

10 years ago it was way more than enough to know good Python to land and keep a data science/ML job. Nowadays, this is just the first requisite of a long list. This is because the game is no longer at the coding level, but way above.

Companies need to know WHAT needs to be built, so they can then build it 10x faster using AI tooling.

I work as an AI developer and consultant, and project after project I see this happening. Design and clarity, plus dev skills, is what companies look for.

For example

Let’s take an AI system, like the Agentic Trading System Marius and I will build (with you?) in our bootcamp starting in October.

This monster is a conglomerate of hardware and software, that goes from the bare metal computers where we will deploy the whole thing, to the final system output showing on your monitor.

If you peel this onion you will find many layers:

• Computing nodes (aka machines) with and without GPU acceleration.

• Operating systems running on each of these machines, which are typically Ubuntu, or even better, Talos Linux.

• A Kubernetes control plane to manage this fleet of nodes.

• A toolbox layer with monitoring and observability, gitops, logging, tracing, storage and networking.

• A collection of containers with business logic, like LLM servers or Python scripts that orchestrate agentic workflows that we need to build, test, and deploy easily and regularly.

Now, each of these layers is handled by different people when you work in a company.

For example

Marius is the infra guy who goes from the bare metal to the Kubernetes cluster, and I am the ML engineer who knows how to talk to Kubernetes, and how to build on top of it the software that moves the business metrics of our client.

And the thing is, AI tools accelerate each person working in each layer of the cake.

But you need to know how to cook each of these layers, and how to ensemble them to produce a system that works in production.

This is where the game is played now.

This is what has and will have value 5/10/15 years from now.

This is what I can teach you, if you want of course.

Gift 🎁

As a subscriber to the Real World ML Newsletter you can enrol in the LLMOps bootcamp with an exclusive 35% discount if you enrol in the following 5 days. After that, the price will go up.

Talk to you next week,

From Menorca

With love

Pau