OpenAI released GPT 6.1 SOL on September 29.
Some say this model slashed costs by 50% for their $200 tier.
Others say it unceremoniously cut their subscription usage by 50% while calling it an efficiency boost.
Why are there two completely opposite reactions to the same announcement?
Because cheaper, and, more expensive, happened on the same day.
Depending on whether you pay per token or per month.
This isn't about whether the new model is good.
It's about OpenAI saying two contradictory things to different users.
Today, we won't discuss if 6.1 is worth using.
We'll clarify three things.
First, why can the bills for models with the same list price differ by 10 times?
Second, which official data matches independent evaluations, and which do not?
Third, how should you determine if it is right for you?
Okay, let's start with the first one.
GPT-6.1 is not the flagship model.
It is an upgraded version of the GPT-6 SOL released on September 22nd.
OpenAI's current product line looks like this.
At the top, GPT-6 Ultra is the most powerful and expensive.
In the middle, GPT-6 SOL is slightly weaker and cheaper.
At the bottom, GPT-6 Lite is the weakest and cheapest.
The official positioning for 6.1 SOL is this.
Providing Near Ultra Intelligence at 1-5th the price of Ultra.
The pricing is as follows.
Ultra is $10 input, $50 output
6.1 SOL is $2 input, $10 output
Light is $0.10 input, $0.50 output
If you calculate Ultra divided by SOL, it is exactly 5 times
This is what the official statement means by 1 fifth the price
But that is just the unit price
The real cost is a completely different story
Why?
On September 29th, someone on X posted a chart with a headline meaning that for the same $2
$10.00 price tag, the bill can differ by 10 times. What were they comparing?
GPT 6.1 SOL at $2 input, $10 output. Claude Sonnet 5.5 at $2 input, $10 output.
The prices are identical, but the per-task cost calculated by artificial analysis shows 5.5
is $7.60, while 6.1 SOL is $0.72. A difference of 10.6 times. Why? Because Sonnet 5.5 is
很 wordy
它映出了一共193,000典的登場在本項上
那是最高的一種的模式測試驗
它是60%更多的 Opus 5.5
和7倍的GPT6Ultra
6.1 SOL
A發現它的登場是10%-30%的高度
但它是10%的一共1%的上一共1%
它是比方的一页价值,
但是因为它说少
你费同一项供应价
但一项供应价格更
而另一项供应价格更多
当然,这些人会比较不同
这就是第一个我们必须理解
在这些人的价格上不符定在这些价格上
与其中的价格出现
我们看看这些 data
2从公司的公司
2从公司的公司
4个人数在 total
第一
75.2 vs 74.1
There is a chart on OpenAI's release page for SWE Bench.
It measures AI's ability to complete complex software engineering tasks in real codebases.
O1 has five reasoning levels, low, medium, high, ultra, and max.
The numbers are, low is 70.0% at $0.17.
Medium is 73.2% at $0.42.
High is 75.2% at $0.65.
Ultra is 74.9% at $0.79
Max is 71.9% at $1.57
You see, the high level is the best
The max level actually regressed
And it costs 2.4 times more than the high level
Compared to Claude 3.5 Sonnet's best at 74.1% for $4.43
O1's high level scores higher and is 6.8 times cheaper
The official claim says it's, approaching, Sonnet
but the numbers show it surpasses Sonnet.
The official statement downplayed the result.
The second number, 71.9% equals 71.9%.
This is O1's ultra and max levels.
The scores are identical, both 71.9%, but max costs double.
Official data tells you, don't set it to max.
This is also true on the Artificial Analysis Leaderboard.
their coding agent index shows the ultra level is three points higher than max
higher reasoning intensity isn't always better
the third number 52 versus 58
from artificial analysis's intelligence index
this is their comprehensive score from actual evaluations
the scores are
claude 3.5 opus at 58
sonnet 3.5 at 56
extra at 53
flash at 53
and 01 at 52.01 is the lowest.
But the cost is also the lowest.
Per task, it's 72 cents.
Sonnet is $3.26, and Opus 3.5 is $5.98.
Claude 3.5 Sonnet is $7.60.
So the question isn't which is best,
but whether your task needs a 58 or if a 52 is enough.
The fourth number, 646 vs 643 vs 728.
This is the big code bench leaderboard score
Not an official list, but an independent one
The results are like this
01-646 points
Previous version, 643 points
GPT40, 728 points
01 is only 3 points higher than the previous version
And it lags behind 40 by 82 points
They outright say it moved a lot on their own benchmarks
But barely moved on independent ones
The term used is
Benchmarking to the test on their own list
Cognition's Frontier Code 1.1 Extended is the same.
01 is 60.4% and the previous version was 60.7%, almost flat.
The official and independent leaderboards tell two different stories.
OK, pause.
We now know four numbers.
First, it beats Sonnet on some tasks.
Second, Max is more expensive and worse.
Third, the comprehensive score is the lowest of the bunch, but the cost is also the lowest.
四、インデピンチマーク インプーブメント
What do these four numbers tell you when put together?
6.1 Sonnet is not a cheap version of Opus.
It is an efficiency model for specific tasks.
It is not an all-rounder.
It is specialized.
So the question arises.
How do you decide if it's right for you?
We divide scenarios into three categories.
Code fixing, batch tasks, and repetitive work.
If you have to fix bugs daily or run massive repetitive knowledge tasks.
6.1 sonnet is perfect for this
a developer conducted an experiment
they put 105 bugs into 2 code repositories
opus scored 45 and cost 33 dollars
6.1 sonnet scored 44 and cost 6 dollars and 56 cents
opus 3.5 scored 41.7 and cost 58 dollars
6.1 sonnet nearly matched opus at one fifth the cost
however they noted n equals 1
meaning they only did this experiment once
On the Click, Benchmark for White Collar Knowledge Work
6.1 Sonnet scored 80.0, costing 18 cents
Claud 3.5 scored 81.4, costing 89
For these tasks, using 6.1 sonnet is more than enough.
Don't use Macs.
The second category, Creative Visuals, Games, and Complex Reasoning.
如果您做3D scenes, visual effects, or game design, tasks needing imagination,
6.1 Sonnet may not be enough.
BridgeBench asked it to render a sunset seascape.
The conclusion was it looked like a sunset in a different price bracket,
and the sea was barely rendered.
Someone posted a tweet that got over 1,200 likes.
OPUS 3.5 is much better than GPT 6.1.
Someone else made a comparison video.
With the same, high effort setting, the conclusion was that
OPUS 3 won and 50 Jae-on in quality, bugs, and color schemes.
SONNET is only better in token efficiency.
Others have also used logic questions to test it.
6.1 SONNET set to max failed.
OPUS 3.5 set to medium passed easily.
For these tasks, stick to CLOD or OPUS.
The third category, scientific research.
The company admitted this themselves.
On the Terminal Bench Science benchmark, 6.1 SONNET is 57.0, while OPUS is 68.1%.
the official word is that the most challenging scientific research tasks should use opus
for these tasks use opus and don't try to save a few bucks
alright now you know 6.1 sonnet isn't a cheap opus
it's an efficiency tool for specific tasks
if your task aligns it's great if not it's a trap
and there's one more thing you need to know
On September 29, the same day, OpenAI's TBO posted a tweet.
The quota for the new Pro 200 plan was reduced from 20x to 10x.
Old users temporarily retained the 20x limit.
On the same day, they launched Pro 500.
$500 per month for a 25x quota.
API users saw it as getting cheaper.
Subscription users saw it as getting more expensive.
So this isn't just a new model release.
This is OpenAI repricing for different user segments.
do you understand now why some people are cheering while others are cursing
because they are looking at two different things
one last thing if you ask me if 6.15 is worth using
my answer is don't ask if it's worth it ask if it's suitable
it is suitable for code repair and batch tasks
use the higher tier and turn on max
for creative and complex reasoning
current tests show it can't beat claude
Continue using Claude or Opus
As for scientific research, the official guide already told you
One more thing about using Opus
If you are a subscriber, your quota has been cut
You need to recalculate if it's enough for you
If it's not enough, either upgrade to Pro500
Or switch to API and pay by token
This isn't a question of which model is stronger
This is a question of OpenAI's repricing
That's all for today's content
If you liked it, remember to like, follow, and share
This is my motivation to keep creating. See you next time!