My recipe for solving real world problems with LLMs
Part 1.5 of many
Today my throat is sore and my nose is running.
And the number one suspect I have behind this half-day of sickness is the Chief Technical Officer of Kai & Nil's Crypto Algorithmic Trading Fund.
And that is Mr Nil.
Nil spends half of his day at the fund developing quantitative trading strategies, and the other half at the kindergarten. Where he probably caught this cold.
Look, I am not blaming Nil.
I (and you, remember?) are working with him, as the latest/shiniest AI engineer hire in the fund. And we want him, and the CEO (Kai) to be happy.
So we need to deliver some results this week, and the clock is ticking.
This is often the case when you work as a freelance/contractor AI engineer.
This has been my life for the last 6 years working as a contractor.
And what I am showing you today (and next Wednesday) is my strategy to getting up and running with a new project, so that you get buy in in the first week, and get yourself some time.
Real world example
Last week I (and you, remember?) landed an AI engineer job at Kai & Nil's Crypto Algorithmic Trading Fund, the coolest trading firm in the world.
And we started designing an LLM system that can parse crypto sentiment
As I advised you in the previous newsletter, we kicked off the project by having a looong coffee session with Nil, who is both
the end-user of the system we want to build (super important to engage them from day 1), and
the guy who knows the best the infrastructure the company uses.
After our coffee session, we drew the diagram of the system we want to build, that looks like this:
To build this thing we essentially need to do 2 things:
Thing #1 → Come up with a good LLM/Agent model/prompt/workflow that can parse crypto sentiment from news in real-time.
Thing #2 → Integrate this LLM/bot into a real-time pipeline that integrates with the rest of the system, which essentially means Apache Kafka.
My advice
Thing #1 is the sexy part of the project, and the one you might be tempted to do first.
However, bear in mind that it is also the part that requires more experimentation, and hence it is very hard to estimate how long it will take.
Because of this, I recommend you leave Thing #1 for next week, and focus on Thing #2 this week.
That is, we will start by building a minimally working pipeline that parse (not accurately yet), and we will mock the LLM responses.
So we have something to show by Friday, and we can start working on Thing #1 next week.
That is your first easy win.
See you on Wednesday
On Wednesday, I will show you how to structure the repo, and get up and running with the project.
Believe me, I tried. But my body and brains are not feeling well today.
Do you want to learn Real World ML/AI with me, live?
I love building AI products.
And I love sharing everything I learn along the way.
This is what I do for a living.
This is what my courses are all about.
In the last year I have built, together with 417 students and for more than 184 live coding hours, FOUR Real Time ML Systems to predict crypto prices. From scratch, every single time.
And in the next cohort of the course, starting in February, we will build a credit card fraud detection system from the ground up.
If you are interested, you can still grab the massive 50% early bird price and have lifetime access to all 4 past cohorts and all future cohorts of this course, without paying a penny more.
If you are into LLM engineering, and you are tired of building small demos, I have another course. A new one, that I will be teaching with Marius Rugan, one of the best infrastructure engineers I have found in my (rather long, because I am a bit old) career
Look, I won't lie to you.
My courses are not easy.
You need to show up.
17 sessions in a row. 3 times a week. 3 hours each time.
But hey, no pain, no gain?
And if (for whatever reason you cannot attend live) you can follow at your own pace.
We are a Discord community with almost 800 hard-core members.
Do you dare to take the challenge?
Do you want to join us?
Talk to you next week,
From Ostružnica,
Pau