It's one of the most common questions I get about coding agents.
Are we ready yet to use open source models to build real world projects?
I'm here to tell you that yes, we are ready.
And I'm going to show you.
In just about 10 minutes time, we're going to be looking at real world products like this and this
built by coding agents using open source models.
And we'll be comparing it to what Claude can do with the same brief.
And I'm cramming so many interesting things into this one video.
Instead of using Claude Code, we're going to be using Pi, which is a super popular coding agent.
Just recently become really popular.
We're going to be using GLM 5.2, the latest groundbreaking model from ZAI.
And we're going to be using DeepSeq v4.
And we're going to be using dev containers for extra safety.
A lot going on.
Now, this website is one of the most important resources for an AI engineer.
And I'm going to link to it in the description.
It's artificialanalysis.ai with tons of information, a lot of data about different models.
In particular, if you scroll down, there is this chart intelligence versus cost.
It's a bit buried, but it's a really valuable chart.
And people on my AI Engineer CoreTrack course know this chart well.
Each dot represents a different model.
The x-axis is how much does it cost to run the model for a task.
More to the right means it's more expensive.
the y-axis is how intelligent is the model. A smarter model goes higher up. So you're ideally
looking to fit in that green quadrant on the top left. That is where models go if they are cheap
and they're also smart. And today we're going to try out two models that are kind of on the
rim of this line here. They're kind of at this frontier of being smart but cheap. We're going
to look at DeepSeq v4 Pro. This is one that has a great reputation for being really low cost,
and we'll see. And we're also going to look at GLM 5.2 from z.ai, and that has a reputation for
being even smarter than this diagram might lead you to believe, and we'll judge that for ourselves.
So GLM 5.2, which the community is super excited about, and DeepSeq v4, let's put them to the test.
So Pi is an example of a coding agent like a Claude code or more like an open code because it's one of these coding agents that are designed to not be tied to any LLM but to allow you to plug in any open source model.
And in particular, Pi has a great reputation because it's very lightweight.
It is minimal.
It's easy to extend it in different directions.
But what you get out of the box is very simple.
And it just has a simple prompt and a simple approach to context management.
And it lets you take it in whatever direction you want.
And that's exactly what we're going to do.
Okay, the moment has arrived to put this to the test.
And I would say whether or not you're technical, you're going to want to watch this just to
get a sense for how it all works, how Pi works, and how these models perform. We are going to put
GLM and DeepSeek to the test. We're going to do it right now. We're going to build out an
interesting project. And here it is. I've got a folder here, which is almost empty. It just has
a few things, including an agents.md file. And that describes what we'd like to build. We would
like to build a personal CRM system, a personal Salesforce kind of platform, just for you, just
for me, our very own version of it, like a Salesforce, a pipe drive that I cover in some
of my other courses, that kind of thing. But we're going to want to build the whole thing from
scratch. We want a coding agent to do that. Just a simple one, mind. And I've got the business
requirements here written from a business person's point of view. There is one small technical
section in here, but otherwise it's about the look and feel. It's about what's in scope and
not in scope and what the CRM can do from a business person's point of view, just a little
bit of technical guidance to keep the model on rails. But the most important thing we do that is
absolutely mandatory when working with open source models is this. We lay out the project in terms of
phases, bite-sized phases that can each be implemented a step at a time. For each phase,
we explain the features in business terms, and then we give success criteria, ways that the
coding agent can verify that it has passed this phase, it's done, it can move on to the next.
And then at the end, we give overall success criteria. I've got six phases and then final
success criteria at the end. And it cannot declare victory on this project until all these success
criteria are met. And I know I go on about it a lot, but success criteria are so crucial to making
sure that you keep the coding agent on track. And importantly, you also have to give it the skills
it needs to be able to test thoroughly, like bring up the screens in a browser window and check it.
That way it's got the feedback loop and it can keep iterating until the success criteria are met.
So the agents.md file has all of the instructions for our coding agent, for Py to follow, but the
readme has instructions for you and me. This tells us what we need to do to get Py up and running.
And it's super simple because I'm using something called dev containers built into VS Code.
so that we can easily run everything in a Docker container.
That means that we're sandboxing everything, we're nice and secure,
and also makes it really easy to set up.
So follow these instructions.
It includes having an open router key in a .emv file.
And then when you're ready, we're going to bring this up in a dev container.
So I'm going to do the Ctrl-Shift-P on a PC or Command-Shift-P on a Mac.
And I'm going to say dev containers reopen in container.
and this is going to now launch a dev container.
I'm going to close this chat window here.
This is launching a Linux container,
which is going to be running installing Pi right now.
And we now have Pi installed
and we're ready to try this out.
And now as explained in the readme,
I'm running a command that will add a skill
to our coding agent.
It's a skill called agent browser
from the great people of Vercel.
And it's a skill which is designed
to give any coding agent the ability
to run a Chrome-based browser behind the scenes.
It's asking me which agent I would like to install it for.
I'm going to come all the way down and select Pi.
Find Pi in this list.
There it is.
And press the space bar and then enter.
Install it for the project.
Yes, we're done.
We've just installed the agent browser skill for Pi.
And now I'm going to run the command to start Pi.
As described in the readme, I just call Pi dash dash models.
and I pass in the name of the two models through OpenRouter that I want to use. And this is your
first sighting of Pi. This is Pi running. It's very simple and we're about to use it to try
building a sales CRM platform of our very own. The first thing I show you is that if you hold
down control P, you can toggle the different models and the models show on the bottom right
here. You can see it says DeepSeek V4 Pro. I can do control P and it will flip to GLM 5.2.
These are the two models that I started it with, so we can switch between them at any point.
And we're going to use both. Of course, we're going to use both.
The other thing to show you is that Shift-Tab chooses the different reasoning levels.
We can toggle High to Extra High to Thinking Off back to High and Extra High.
And we're going to go on Extra High.
We're going to have Deep Seek V4 Pro and also Control-P.
We're going to have GLM 5.2 on Extra High mode.
And now all I have to do wonderfully is say, look, please build the entire project.
As described in agents.md, don't stop until the success criteria are met and the server is running and ready for me to test.
I can kick that off and off it goes.
You can see down here, it tells you how much of the context is being used.
It's got 0.7% of the 1 million context.
percent of the 1 million context window for glm 5.2 and it will also tell you how much it's spent
and we've already spent 0.9 of a cent uh already and we've just spent our first cent on this and
We'll be keeping a very close watch on the spend, but it is off.
It's going to keep going.
It's going to go through each of the different stages of build,
and it's going to check the success criteria at every point.
Well, it's been going away for about five minutes.
Lots has been going on.
We've now spent 10 cents.
But meanwhile, I'm going to do something cool.
I've got another VS Code window running over here.
But this is, in fact, another dev container on exactly the same project.
I've cloned it separately to a different directory.
It's called now CRM DeepSeek, which is a clue as to what I'm about to do.
I am, of course, I'm going to make the screen a little bit bigger for you.
I am, of course, now set to use DeepSeek instead of GLM 5.2.
And I'm going to use the same exact command.
Please build the entire project.
So I'm now going to set off in this other screen the same project in parallel.
Off it goes with DeepSeek V4 Pro running, also on extra high reasoning.
Look at it go. And if I just shrink this window, you'll see that the other one, GLM 5.2, is also on the go.
I have both coding agents, both using Pi, both models on the go at the same time, building out the same project so we can see how they do, both in terms of cost and speed, and also, of course, in terms of the final product.
And this is now one second later for you, but it's like an hour and a half later for me, and both of them have finished.
And wonderfully, you can go into OpenRouter and look at the logs and see the messages going to GLM 5.2 and DeepSeek V4 Pro.
And you can also look at things like the activity charts and see the usage and explore what was going on.
We look at the past three hours and we can see everything that was happening and use that to figure out how much did we spend on each.
And this page in the OpenRouter activity screen shows you everything that went on.
You can see the activity there. In the end, if I back out things when I ran something a second time,
for GLM 5.2, it cost $4.15 to run this. $4.15, and it took an hour and a quarter. For DeepSeek,
it was cheaper. It was $2.56 in total, and it was a little bit faster. It took just about one hour,
a bit faster as well. So DeepSeek was faster and cheaper. What were the results like?
And here is DeepSeek's implementation of a personal CRM. Your personal CRM, like a sales force. You've got a dashboard with deals one and revenue. It's got some dummy data in here to show it off. Something is overdue here. You've got organizations, the companies that you have with an edit and delete, a Dell button, that's all right. Contacts and deals listed out here. Contacts, the people you know at these companies.
deals, which are the deals that are at various stages of the pipeline shown in the stage here.
And the pipeline itself is a Kanban style board of deals moving from new all the way through to
one or lost. And you can drag them across to see what happens when you do that. And you should be
able to add and add and search throughout. Let's search for someone in here. There we go.
The search works too. It is a functioning platform. It's backed by SQLite database.
This is a working but simple sales CRM. And you've got to admit, it's pretty impressive.
But you know what's more impressive? GLM 5.2's version is more impressive. Let's switch over
to it. This is the dashboard. And come on, immediately you see this is better. This is
better. We've got these kind of hot metrics being surfaced at the top. We've got deals one and
revenue, which just is like a better form chart with more rich data. We've got these tasks here
that could be checked off. We've got recent activity and then pipeline by stage at the
bottom and an open board link there. It's really nice. We've got these icons here. We can switch
to organizations. This looks similar, but of course we've got proper icons, not a Dell button.
We've got contacts as before, deals here with this really nice badges. And then this is the
pipeline. It's just sharper. Let's go back over here again to the pipeline. Compare this is a bit
more bland. This is really gorgeous, a really nice, sharp pipeline. And of course, we will still no
doubt have the ability to drag and drop. Everything works with the same stages and won and lost.
This is a pretty impressive screen, a really nice product from GLM 5.2. But I know what you're
thinking. I know you so well. You're thinking, okay, Ed, we get it. This is impressive. GLM 5.2
did a decent job, but how would Opus have done on the same thing? How would Opus 4.8, the strongest
model on the planet, at least that's available to us all, how would that do with this? And how would
it do if it was on ultra code mode and allowed to do its absolute best? Well, of course, I've given
that a try too. And it's running on this tab right here. It took an hour and it would have cost $21.27
if I'd used API directly rather than just using a usage of my Claude AI account. And let's go over
to it. Here we go. This is the dashboard. You can see it's quite similar. It has a chart. It has the
same called out kinds of numbers and a checklist here, but it is a bit nicer, right? Look at the
dark background here it looks a little bit more professional but only slightly only by a hair
and clicking through the tables you'll see a very similar construct clicking through them here it's
just slightly better perhaps and the pipeline here we have another nice pipeline here is a kanban board
but check this out when i drag and drop again you get this dotted line thing it's just a little bit
better. And so in summary, GLM 5.2 was neither the cheapest nor the best, but it was quite close on
both fronts. It was low cost and it was really good. And so I would say, yes, this is a credible
alternative to frontier models that can be used for your own projects. It's not super cheap. You're
still spending north of $4 for a product like this, but there's a lot of capability here. I didn't
even show you. You can click into organizations and edit them and then click here to go over to
the contact screen, edit a log activity here in the pipeline. You can click on cards to bring them
up. There's tons of functionality and it built all of this first time one shot and it met the
success criteria and worked right away. And if you'd like to give this a try yourself and use
Pi, there are details in the description below and I'd love to see what you come up with. And if you
enjoyed this video, then please do like and subscribe. That's the way I know you're really
there. And it's the way the YouTube algorithm knows you're really there as well. And either way,
I hope to see you very soon for another video.