So looks like we just got flashed by Google DeepMind once again because they just dropped
Gemini 3.7 flash today. Yes, their newest model, which is once again a flash model, is here.
And this is only after three weeks since Gemini 3.6 flash. And what they are saying is that Gemini
3.7 flash is their most intelligent workhorse model. And this is coming three weeks after 3.6
flash because of developer feedback and algorithmic innovations and that is why they're saying that
gemini 3.7 flash model is you know out with so much improvement because if you take a look at
the benchmarks this model is way better than a gemini 3.6 flash but i think this is also because
we know internally that they're planning on canceling gemini 3.5 pro completely and working
on gemini 4 at the moment so maybe the improvements they made with the pro model that they're not
going to be dropping anymore maybe they repackaged it as a flash model because this model is better
than 3.6 and if the timeline is three weeks plus they're planning on canceling the 3.5 pro it's
quite likely that is the case we'll just ignore that and put that into the side we don't know
when we're getting a new pro model from the google deep mine team but this model across many of the
benchmarks we will see that yes it is stronger than the 3.6 flash model for example when it comes
to code quality production code quality the model is producing 43.6 percent versus the flash model
34.4 percent and then sonnet 5 is at 42.7 percent and the terra model is at 41.3 percent so one thing
to remember since this is a flash model you're not going to see them compare this to opus 5 or any of
the other stronger models the reason being because this is not their strongest tier and the 3.1 pro
model when they finally decide to upgrade it to Gemini 4 Pro probably then we'll see it being
compared to Opus 5 or GPT 5.6 Soul but anyways we can see that this model is an improvement from
3.6 Flash and I would say that's basically the biggest result or the biggest update that we saw
with the new model because it does not really like change up things a lot it's still like not
becoming the number one model or the number one Flash model even on some benchmarks DeepSeek
version for flash it's actually cheaper and more intelligent than this model but let's just take a
look at the benchmarks that they have published on long horizon software engineering this model
is a little bit behind gpd 5.6 tera which sits at 69.6 percent and then 65.3 percent is the flash
model the older flash model 3.6 sits at 48.6 percent sonnet 5 sits at 53.8 percent and the
new player that is finally being included on benchmarks which even the google deep mine team
is considering with their launch is musepark 1.2 which is sitting at 54.9 so welcome meta to the
benchmark charts because now we're starting to see it appear on more and more benchmarks and if we
also take a look at web development this model is getting an elo score of 1588 versus their old
model is 1538 so not a crazy difference but compared to everything else out there this is
number one. When I say everything else out there, once again, compared to all the mid-tier models,
this is beating all of them when it comes to web development. Now, this model is probably going to
be used in enterprises a lot just because of Google's footprint in the enterprise space.
But we are seeing this model achieve on the automation bench 30.4%. And GPT 5.6 Terra sits
at 23.6%. So yes, this model is stronger than the other Flash models out there. But as I said,
they haven't included DeepSeq version for Flash because if they do, in some areas that model is
actually quite better than the 3.7 Flash model. Before we continue, if you're building AI agents
or just messing around with them, Arcade is worth knowing about. It's the runtime that lets your
agent actually do things instead of just talking about them. Because that's the gap right now.
The models are smart enough. Your agent can figure out exactly what needs to happen in your email,
your Slack, your CRM. It just can't go in and do it. And the reason isn't intelligence, it's
permissions. Something has to prove the agent is allowed to act on behalf of a specific person
in a specific account. That's the messy part everyone runs into, and it's the part ArcGate
actually handles for you. So instead of your agent using one shared login for everybody,
it acts as whoever is actually signed in with exactly the access that person has.
If they can't see something, the agent can't either, and you never have to touch any of that setup yourself.
Then there's the tools.
Arcade has thousands of them already built for Gmail, Google Drive, Slack, Notion, Salesforce,
most of the apps people already work in, and they are built specifically for AI to use,
so the agent gets it right the first time instead of guessing and failing and retrying.
It also keeps a record of everything, what the agent did for who and where,
which matters a lot the moment other people start using the thing you built.
And it works with whatever you're already using. Any model, any framework, cloud, cursor, chat GPT,
doesn't matter. So you're not just giving an AI a list of tools and hoping it works,
you're giving it a place where it can safely take real actions in real apps. It's free to start and
the link is in the description. Thank you once again for Arcade for sponsoring today's video.
Now let's get back into the video. Now one thing to note is that the Gemini 3.7 flash model through
the end of this year so end of 2026 they have a cheap pricing model that they're placing on the
model 75 cents per 1 million input tokens and 3 dollars and 75 cents for 1 million output tokens
so it's a competitive price but this is only for the next six months because after those six months
are done the model's pricing is actually you know a little bit more expensive and now they show it
at the bottom over here you can see that after starting January of 2027 it will become a dollar
and 50 per input and seven dollars and 50 per output so yeah it's still cheap compared to the
other frontier labs but it's not as cheap as for example muse spark when the pricing is updated
or even deep seek version for flash but across these benchmarks we can see that this model
is better in many areas that they highlighted at the top but then they also have some other areas
like long video understanding which this model excels at 85.4% versus 78.9% for the Terra model
and then the old model was also pretty good at that 84.2%. Then long context performance the
model is at 97% and this model the GPT-6 Terra one it sits at 93.5%. So yeah this model in summary
it is better so it's not all negative but it's not all like you know that positive where you are
super excited for Google DeepMind because as I said they're probably still holding off their
biggest release for Gemini 4 lineup. Now one thing people might have missed in their charts because
these charts sometimes are so messy to read and understand but the 3.7 flash is worse than GPT 5.6
Luna which is the model over here which is achieving a higher score on this benchmark
deep software engineering for about three times the cost. So yeah this is kind of interesting
because yeah the cost for gemini 3.7 flash is a little bit more than what it looks like
now some people are a little bit upset and they're like oh disgraceful google left out
so opus and fable because google is incredibly behind but i think one thing we got to remember
guys is that this model is a flash model
to be a pro model or it's not trying to compete with opus or soul or fable five level models yet
that is probably going to be gemini 4 so when gemini 4 comes out then i think it's okay for
us to criticize them if they don't include Sol, Opus, and Fable in their benchmark charts because
for now I think what they have done is pretty accurate. One lab that I would have liked to see
or one model for example I would have liked to see on that chart would be DeepSeq version 4 Flash
because that would kind of spoil their release because that model is way cheaper compared to
the Gemini 3.7 Flash model lineup. Now this model jumped from number 19 to 8 on the web development
area and we see it over here now and a couple of models that are ahead of it are Opus 5 obviously,
Kimi K3, Quen 3.8 Max, Cloud Opus 5, Grok 4.6 which is a model that came yesterday which was a big win
for SpaceX, Fable 5 and 5.6 Sol. So yeah this model is trying to compete in the web development
but it's still behind all of these models which is you know expected because it's a flash model
it's not really a pro model. Today OpenAI has also launched a waitlist for 5.6 SOL ultra fast mode.
Now this is possible because of the fast chips that they have access to now through Cerebris
and GPT 5.6 SOL with this chip kind of running it is able to achieve an ultra fast mode that
generates up to 750 output tokens per second which is about 14 times faster than the standard mode.
So we're getting a really fast version of GPT 5.6 Soul. And now this is supposed to be used for
live or near production workloads like, you know, real-time voice, support, commerce.
What this allows GPT 5.6 Soul to do is kind of be really fast in critical situations when people
might be interacting with the AI agent, like financial research, security response, support,
I think is going to be a big area where this new ultra fast mode will be kind of implemented.
business and developer agents maybe yeah but i think like support or near production workloads
like real-time voice i see this model really excelling at that and obviously this is still
a waitlist mode we don't know how many people are going to get access to this how successful it is
or what the pricing is i don't know if the pricing has changed because there's no information in their
actual you know blog post but as i mentioned a couple of areas where they mentioned that this
is going to be really important customer support and voice commerce live research and experimentation
financial research and security incident response and reliability but yeah i think this partnership
is going to be important for open ai going forward but as i said 14 times the speed what does that
mean for cost we don't know yet but that's it for today's video make sure you guys are subscribed
to the channel follow our new newsletter as well at universeofai.beehive.com as well subscribe to
the main channel, World of AI, and support us on X by following the universe of AIZ as well.
Until then, I'll see you guys in the next video.