Muse Spark 1.1 (Fully Tested): Okay, it's SO GOOD!

AICodeKing 首頁 每日列表 原始影片
實際影片長度：7:03.000。原文、繁中、雙語可點擊句子跳轉影片。
0:00.000–0:07.760
Hi, welcome to another video.
0:07.760–0:11.920
So, Meta has finally entered the frontier model race for real this time.
0:11.920–0:16.360
They just released MuseSpark 1.1,
0:14.140–0:16.360
which is their new multimodal reasoning model built
0:16.360–0:18.240
for agentic tasks.
0:18.240–0:21.640
This is coming out of their superintelligence labs,
0:19.940–0:21.640
and it's also the first model that's
0:21.640–0:24.200
available through their brand new Meta model API.
0:24.200–0:29.160
Yes, Meta now has a paid API, and
0:26.680–0:29.160
Zuckerberg even came back to X after three years just
0:29.160–0:30.160
to launch this thing.
0:30.160–0:32.940
So, they're definitely taking this seriously.
0:32.940–0:34.960
Now what is this model exactly?
0:34.960–0:39.460
It's a multimodal reasoning model that combines reasoning,
0:37.210–0:39.460
coding, computer use, and multimodal
0:39.460–0:40.560
understanding.
0:40.560–0:45.580
It has a 1 million token context window, and
0:43.070–0:45.580
Meta says it can actively manage that context.
0:45.580–0:49.320
It remembers actions,
0:47.450–0:49.320
retrieves information from much earlier in the session, and compacts
0:49.320–0:51.320
things in a way that keeps the critical steps.
0:51.320–0:54.840
It's also trained to orchestrate multi-agent systems,
0:53.080–0:54.840
so it can act as a main agent that
0:54.840–0:59.040
plans and delegates, or as a sub-agent that sticks to its job and
0:56.940–0:59.040
escalates when needed.
0:59.040–1:04.800
On the benchmarks, Meta claims it scored 88.
1:01.920–1:04.800
1 on MCP Atlas, which tests scaled tool use,
1:04.800–1:08.880
while Opus 4.8 and GPT 5.5 are around 80 there.
1:08.880–1:12.400
It also leads on JawBench, and a few other tool use benchmarks.
1:12.400–1:18.340
But interestingly, on TerminalBench, it only scored 59,
1:15.370–1:18.340
while GPT 5.5 is at around 83.
1:18.340–1:22.500
So it seems like the model is really strong on tool use and
1:20.420–1:22.500
agentic stuff, but not exactly
1:22.500–1:24.040
dominant on pure coding.
1:24.040–1:27.540
Take these claim numbers with a grain of salt,
1:25.790–1:27.540
though, because we'll do our own testing.
1:27.540–1:33.220
For the pricing, it's $125 per million input tokens,
1:30.380–1:33.220
and $425 per million output tokens,
1:33.220–1:34.700
which is actually pretty reasonable.
1:34.700–1:38.080
It's cheaper than Sonnet, and you also
1:36.390–1:38.080
get $20 in free credits when you sign up for the
1:38.080–1:39.300
API preview.
1:39.300–1:40.800
So that's good.
1:40.800–1:45.800
Now let's put it through my own benchmark, KingBench,
1:43.300–1:45.800
and see how it actually performs.
1:45.800–1:52.360
I have seven questions here covering UI generation,
1:49.080–1:52.360
3JS, SVG games, hard math, and agentic tasks.
1:52.360–1:53.980
Each question is scored out of ten.
1:53.980–1:55.620
Let's get right into it.
1:55.620–1:57.900
The first question is the elevator simulation.
1:57.900–2:02.120
The model has to build a simulation in HTML, CSS,
2:00.010–2:02.120
and JS where you can spawn people on different
2:02.120–2:03.120
levels.
2:03.120–2:04.120
There are three elevators.
2:04.120–2:06.480
Each elevator can only take one person, and
2:05.300–2:06.480
the people left behind should catch the
2:06.480–2:07.680
next one.
2:07.680–2:11.760
Every person has a random target floor with a tooltip on hover,
2:09.720–2:11.760
and it should be well-animated.
2:11.760–2:15.780
Here's what it generated, and it does look pretty nice visually,
2:13.770–2:15.780
but the logic is broken.
2:15.780–2:19.740
The elevators are not respecting the one-person rule properly,
2:17.760–2:19.740
and the cueing behavior just
2:19.740–2:22.060
falls apart when you spawn multiple people.
2:22.060–2:25.060
The animation is also janky, so this is mostly a fail.
2:25.060–2:27.080
I'm giving it a 3 out of 10 here.
2:27.080–2:29.600
For comparison, Opus 4.8.
2:29.600–2:31.100
Got a full 10 on this one.
2:31.100–2:35.080
The second question is the 3JS 3D model of a contact lens case.
2:35.080–2:38.580
It should have pronounced L and R caps, and
2:36.830–2:38.580
you should be able to click on the caps to
2:38.580–2:39.580
open them up.
2:39.580–2:41.160
So, here's the result.
2:41.160–2:43.720
The case renders, and the L and R are there.
2:43.720–2:48.360
The materials actually look quite good, but
2:46.040–2:48.360
the click-to-open interaction is only half-working.
2:48.360–2:52.160
One cap opens weirdly and clips through the body,
2:50.260–2:52.160
so this is a partial pass.
2:52.160–2:54.480
I'm giving it a 5 out of 10.
2:54.480–2:57.500
The third question is the 3JS folding table with a slider.
2:57.500–3:01.500
As you take the slider to the right, the table should unfold,
2:59.500–3:01.500
and to the left, it should
3:01.500–3:02.500
fold.
3:02.500–3:04.760
It should be seamlessly animated, and be 3D.
3:04.760–3:07.400
This one is actually decent.
3:07.400–3:11.460
The table folds and unfolds with the slider, and
3:09.430–3:11.460
the animation is fairly smooth.
3:11.460–3:16.500
The geometry gets a bit wonky at the extremes, but
3:13.980–3:16.500
overall it works, so this is a 7 out of
3:16.500–3:17.680
10.
3:17.680–3:18.680
Not bad at all.
3:18.680–3:21.920
The fifth question is the SVG of a panda eating a burger.
3:21.920–3:25.680
This tests whether the model can visualize things in code without seeing them.
3:25.680–3:26.920
Here's what it made.
3:26.920–3:30.500
The panda is recognizable, and
3:28.710–3:30.500
there is something that looks like a burger.
3:30.500–3:31.500
It's mid.
3:31.500–3:34.080
It's not the worst I've seen, but it's not great either.
3:34.080–3:35.320
5 out of 10.
3:35.320–3:39.220
The fifth question is the bow and
3:37.270–3:39.220
arrow simulator game with 4 targets and a leaderboard based
3:39.220–3:40.700
on the lowest time.
3:40.700–3:43.080
This one is actually good.
3:43.080–3:47.440
The bow mechanics work, the target's register hits,
3:45.260–3:47.440
and the leaderboard updates properly.
3:47.440–3:51.040
The visuals are also pretty clean, so this is an 8 out of 10.
3:51.040–3:52.180
Pretty good.
3:52.180–3:55.160
The sixth question is the hard math one.
3:55.160–3:58.600
It's a permutation counting problem over a grid of ordered pairs,
3:56.880–3:58.600
and the answer should
3:58.600–4:00.300
be 20460.
4:00.300–4:02.440
This question destroys most models.
4:02.440–4:08.640
GPT 5.5, Opus 4.7, and Gemini 3.5 flash all got zero on this.
4:08.640–4:11.260
And believe it or not, it got 2460.
4:11.260–4:12.260
That's exactly right.
4:12.260–4:13.900
So, this is a full 10 out of 10.
4:13.900–4:15.560
Really impressive, to be honest.
4:15.560–4:17.940
The seventh question is the agentic one.
4:17.940–4:21.740
The model has to generate a dataset of facts about pandas,
4:19.840–4:21.740
Finitune, Gemma, 2B model on
4:21.740–4:25.520
that, all locally, and then
4:23.630–4:25.520
give me a web UI where every refresh generates a new panda
4:25.520–4:26.520
fact.
4:26.520–4:27.780
And it did the whole thing.
4:27.780–4:31.540
It generated the dataset, set up the Finituning,
4:29.660–4:31.540
ran it, and the web UI works.
4:31.540–4:35.000
Every refresh gives me a new panda fact from the Finitune model.
4:35.000–4:38.420
So this is a full 10 out of 10 as well.
4:38.420–4:40.820
This is where you can see the agentic training paying off.
4:40.820–4:43.200
So this is the final chart.
4:43.200–4:53.620
And Muse Spark 1 ends up with 48 out of 70, which is 68.57%.
4:53.620–4:59.280
That puts it in sixth place on the leaderboard,
4:56.450–4:59.280
above GPT 5.6 Terra, Opus 4.7, and Sonnet 5,
4:59.280–5:07.480
below Grok 4.5, GPT 5.6 Sol, GLM 5.2, Opus 4.8, and
5:03.380–5:07.480
Fable 5 at the top, for a first real
5:07.480–5:09.020
frontier attempt from Meta.
5:09.020–5:10.520
This is honestly a pretty solid showing.
5:10.520–5:13.980
Now, let me tell you some observations from using this model beyond the benchmark, because
5:13.980–5:15.300
the scores don't tell the full story.
5:15.300–5:18.620
The first thing is that this model is actually really good at UI.
5:18.620–5:22.340
Even on the questions where it lost points for logic,
5:20.480–5:22.340
the designs it produces look really
5:22.340–5:23.340
good.
5:23.340–5:26.800
And now that it's on API,
5:25.070–5:26.800
you can potentially use it in your workflow with things like
5:26.800–5:30.340
Open Design to generate some crazy designs, and then
5:28.570–5:30.340
hand that over to something like
5:30.340–5:33.580
GPT 5.6 to work on the actual implementation.
5:33.580–5:35.520
That combo could be really powerful.
5:35.520–5:39.200
The second thing is that, while the model is good at agentic tasks,
5:37.360–5:39.200
it is still pretty
5:39.200–5:40.980
weird in agentic use.
5:40.980–5:44.080
For an example,
5:42.530–5:44.080
it doesn't look at other existing files before writing.
5:44.080–5:47.800
I generally have multiple sessions of agents running in open code to make sure all my prompts
5:47.800–5:48.800
are done.
5:48.800–5:52.320
I ask each model to make a new file and
5:50.560–5:52.320
not touch the others, but this one just doesn't
5:52.320–5:53.320
care.
5:53.320–5:56.080
Each session overwrote the others, and
5:54.700–5:56.080
none of them even looked at the existing files
5:56.080–5:57.080
first.
5:57.080–5:58.320
It is really bad in that regard.
5:58.320–6:01.700
So if you're running it in parallel setups, be careful,
6:00.010–6:01.700
because it will happily clobber
6:01.700–6:02.960
your files.
6:02.960–6:05.740
The third thing is that the model is still quite good when it works.
6:05.740–6:10.040
When it's in its lane, doing agentic tasks, tool use,
6:07.890–6:10.040
or that math question, it genuinely
6:10.040–6:11.040
delivers.
6:11.040–6:12.460
The problem is consistency.
6:12.460–6:15.060
You never quite know which version of the model you're going to get.
6:15.060–6:18.340
Overall I think this is a really interesting first entry from Meta.
6:18.340–6:22.320
It's not going to replace Fable 5 or Opus 4.8 for coding.
6:22.320–6:25.140
But the agentic capabilities are real.
6:25.140–6:26.700
The pricing is decent.
6:26.700–6:29.280
And the UI design sense is a nice surprise.
6:29.280–6:33.160
If they fix the file awareness issues and the consistency,
6:31.220–6:33.160
the next version could be a serious
6:33.160–6:34.160
contender.
6:34.160–6:35.160
Overall it's pretty cool.
6:35.160–6:37.380
Anyway, let me know your thoughts in the comments.
6:37.380–6:41.260
If you liked this video,
6:39.320–6:41.260
consider donating through the Super Thanks option, or becoming
6:41.260–6:42.880
a member by clicking the Join button.
6:42.880–6:45.880
Also, give this video a thumbs up and subscribe to my channel.
6:45.880–6:46.880
I'll see you in the next one.
6:46.880–6:48.480
Until then, bye.
6:48.480–6:49.480
Bye.
6:49.480–6:50.480
!
6:50.480–6:51.480
!
6:51.480–6:52.480
!
6:52.480–6:53.480
Bye.
6:53.480–6:54.480
Bye.
6:54.480–6:55.480
Bye.
6:55.480–6:56.480
Bye.
6:56.480–6:57.480
Bye.
6:57.480–6:58.480
Bye.
6:58.480–6:59.480
Bye.
6:59.480–7:00.480
Bye.
7:00.480–7:01.480
Bye.
7:01.480–7:02.480
Bye.
7:02.480–7:03.000
Bye.
0:00.000–0:07.760
嗨，歡迎收看另一個影片。
0:07.760–0:11.920
這次 Meta 終於真正進入了前沿模型的競賽。
0:11.920–0:16.360
他們剛剛發布了 MuseSpark 1.1，
0:14.140–0:16.360
這是他們為代理任務（agentic tasks）打造的全新多模態推理模型，
0:16.360–0:18.240
專門用於此類任務。
0:18.240–0:21.640
該模型出自他們的超智能實驗室（superintelligence labs），
0:19.940–0:21.640
同時也是第一個
0:21.640–0:24.200
透過他們全新推出的 Meta 模型 API 提供的模型。
0:24.200–0:29.160
沒錯，Meta 現在有了付費 API，
0:26.680–0:29.160
扎克伯格甚至在離開 X 三年後回歸，
0:29.160–0:30.160
就是為了發布這項產品。
0:30.160–0:32.940
他們絕對是認真對待這件事。
0:32.940–0:34.960
那麼，這個模型究竟是什麼呢？
0:34.960–0:39.460
它是一個結合推理、
0:37.210–0:39.460
編碼、電腦操作和多模態
0:39.460–0:40.560
理解能力的多模態推理模型。
0:40.560–0:45.580
它擁有 100 萬 token 的上下文視窗，
0:43.070–0:45.580
而 Meta 表示它能主動管理該上下文。
0:45.580–0:49.320
它會記住操作，
0:47.450–0:49.320
從會話更早階段檢索資訊，並以保留關鍵步驟的方式
0:49.320–0:51.320
壓縮內容。
0:51.320–0:54.840
它還經過訓練以協調多代理系統，
0:53.080–0:54.840
因此它可以作為主代理，
0:54.840–0:59.040
負責規劃和委派任務，或作為子代理，專注於自身工作，
0:56.940–0:59.040
並在必要時進行升級報告。
0:59.040–1:04.800
在基準測試方面，Meta 聲稱它獲得了 88.
1:01.920–1:04.800
在 MCP Atlas 上獲得 1 分，該基準測試評估擴展工具的使用能力，
1:04.800–1:08.880
而 Opus 4.8 和 GPT 5.5 在那裡大約只有 80 分。
1:08.880–1:12.400
它在 JawBench 和其他幾個工具使用基準測試中也領先。
1:12.400–1:18.340
但有趣的是，在 TerminalBench 上，它只得了 59 分，
1:15.370–1:18.340
而 GPT 5.5 大約有 83 分。
1:18.340–1:22.500
因此，這個模型似乎在工具使用和
1:20.420–1:22.500
代理任務方面非常強大，但並非在純編碼方面
1:22.500–1:24.040
具有絕對優勢。
1:24.040–1:27.540
不過，請對這些聲稱的數據持保留態度，
1:25.790–1:27.540
因為我們會進行自己的測試。
1:27.540–1:33.220
關於定價，輸入 token 每百萬 125 美元，
1:30.380–1:33.220
輸出 token 每百萬 425 美元，
1:33.220–1:34.700
這其實相當合理。
1:34.700–1:38.080
它比 Sonnet 便宜，而且你還
1:36.390–1:38.080
註冊 API 預覽版時，可獲得 20 美元的免費信用額度。
1:38.080–1:39.300
API 預覽版。
1:39.300–1:40.800
這還挺不錯的。
1:40.800–1:45.800
現在讓我們用我自己的基準測試 KingBench 來測試它，
1:43.300–1:45.800
看看它實際表現如何。
1:45.800–1:52.360
這裡有七個問題，涵蓋 UI 生成、
1:49.080–1:52.360
3JS、SVG 遊戲、高難度數學和代理任務。
1:52.360–1:53.980
每個問題滿分為 10 分。
1:53.980–1:55.620
讓我們直接開始。
1:55.620–1:57.900
第一個問題是電梯模擬。
1:57.900–2:02.120
模型必須用 HTML、CSS
2:00.010–2:02.120
和 JS 構建一個模擬，讓你可以
2:02.120–2:03.120
在不同樓層生成人物。
2:03.120–2:04.120
有三部電梯。
2:04.120–2:06.480
每部電梯只能載一人，而
2:05.300–2:06.480
留在後面的人應該搭乘
2:06.480–2:07.680
下一部電梯。
2:07.680–2:11.760
每個人都有隨機的目標樓層，滑鼠懸停時會顯示提示，
2:09.720–2:11.760
且動畫應該要流暢。
2:11.760–2:15.780
這是它生成的結果，視覺上看起來相當不錯，
2:13.770–2:15.780
但邏輯有問題。
2:15.780–2:19.740
電梯沒有正確遵守一人一部的規則，
2:17.760–2:19.740
而且排隊行為在
2:19.740–2:22.060
生成多個人物時就完全崩潰了。
2:22.060–2:25.060
動畫也很卡頓，所以這基本上算是失敗。
2:25.060–2:27.080
我給它 3 分（滿分 10 分）。
2:27.080–2:29.600
作為比較，Opus 得到 4.8 分。
2:29.600–2:31.100
這一題它拿到了滿分 10 分。
2:31.100–2:35.080
第二個問題是隱形眼鏡盒的 3JS 3D 模型。
2:35.080–2:38.580
它應該要有明顯的 L 和 R 蓋子，而且
2:36.830–2:38.580
你應該可以點擊蓋子
2:38.580–2:39.580
將它們打開。
2:39.580–2:41.160
所以，這是結果。
2:41.160–2:43.720
鏡盒渲染出來了，L 和 R 也都在。
2:43.720–2:48.360
材質看起來相當不錯，但
2:46.040–2:48.360
點擊打開的互動功能只完成了一半。
2:48.360–2:52.160
其中一個蓋子打開的方式很奇怪，並且會穿模到鏡盒主體，
2:50.260–2:52.160
所以這算是部分通過。
2:52.160–2:54.480
我給它 5 分（滿分 10 分）。
2:54.480–2:57.500
第三個問題是帶有滑桿的 3JS 摺疊桌。
2:57.500–3:01.500
當你將滑桿向右移動時，桌子應該展開，
2:59.500–3:01.500
向左移動時，它應該
3:01.500–3:02.500
摺疊。
3:02.500–3:04.760
動畫應該無縫銜接，並且是 3D 的。
3:04.760–3:07.400
這個其實還不錯。
3:07.400–3:11.460
桌子隨著滑桿摺疊和展開，
3:09.430–3:11.460
動畫相當流暢。
3:11.460–3:16.500
幾何結構在極端情況下會變得有點奇怪，但
3:13.980–3:16.500
整體來說運作正常，所以這題得 7 分（滿分 10 分）。
3:16.500–3:17.680
10 分。
3:17.680–3:18.680
完全不算差。
3:18.680–3:21.920
第五個問題是吃漢堡的熊貓 SVG 圖檔。
3:21.920–3:25.680
這測試模型是否能在沒看到圖的情況下，用程式碼視覺化事物。
3:25.680–3:26.920
這是它生成的結果。
3:26.920–3:30.500
熊貓的輪廓可辨識，
3:28.710–3:30.500
而且有個看起來像漢堡的東西。
3:30.500–3:31.500
表現普通。
3:31.500–3:34.080
這不是我看過最差的，但也稱不上優秀。
3:34.080–3:35.320
得 5 分（滿分 10 分）。
3:35.320–3:39.220
第五個問題是弓和
3:37.270–3:39.220
箭模擬器遊戲，有 4 個目標，並根據
3:39.220–3:40.700
最短時間生成排行榜。
3:40.700–3:43.080
這個其實表現不錯。
3:43.080–3:47.440
弓的機制運作正常，目標能正確記錄命中，
3:45.260–3:47.440
排行榜也能正確更新。
3:47.440–3:51.040
視覺效果也很乾淨，所以這題得 8 分（滿分 10 分）。
3:51.040–3:52.180
相當不錯。
3:52.180–3:55.160
第六個問題是困難的數學題。
3:55.160–3:58.600
這是一個在有序配對網格上的排列計數問題，
3:56.880–3:58.600
答案應該
3:58.600–4:00.300
是 20460。
4:00.300–4:02.440
這題摧毀了大多數模型。
4:02.440–4:08.640
GPT 5.5、Opus 4.7 和 Gemini 3.5 flash 在這題都得了零分。
4:08.640–4:11.260
信不信由你，它得了 2460。
4:11.260–4:12.260
完全正確。
4:12.260–4:13.900
所以，這題是滿分 10 分。
4:13.900–4:15.560
老實說，真的很令人印象深刻。
4:15.560–4:17.940
第七個問題是代理型（agentic）題目。
4:17.940–4:21.740
模型必須生成關於熊貓的事實資料集，
4:19.840–4:21.740
微調 Gemma 2B 模型，在
4:21.740–4:25.520
全部在本地完成，然後
4:23.630–4:25.520
給我一個網頁介面，每次重新整理都會產生一個新的企鵝
4:25.520–4:26.520
事實。
4:26.520–4:27.780
它確實完成了整個過程。
4:27.780–4:31.540
它產生了數據集，設定了微調，
4:29.660–4:31.540
執行了它，而且網頁介面運作正常。
4:31.540–4:35.000
每次重新整理都會從微調模型中給我一個新的企鵝事實。
4:35.000–4:38.420
所以這部分也得了滿分十分。
4:38.420–4:40.820
這就是你能看到代理式訓練帶來回報的地方。
4:40.820–4:43.200
所以這是最後一張圖表。
4:43.200–4:53.620
Muse Spark 1 最終得分為 70 分中的 48 分，即 68.57%。
4:53.620–4:59.280
這讓它排在排行榜第六名，
4:56.450–4:59.280
高於 GPT 5.6 Terra、Opus 4.7 和 Sonnet 5，
4:59.280–5:07.480
低於 Grok 4.5、GPT 5.6 Sol、GLM 5.2、Opus 4.8，以及
5:03.380–5:07.480
位居榜首的 Fable 5，這是 Meta 首次真正的
5:07.480–5:09.020
前沿嘗試。
5:09.020–5:10.520
老實說，這是一個相當穩健的表現。
5:10.520–5:13.980
現在，讓我分享一些在使用這個模型時，超出基準測試範圍的觀察，因為
5:13.980–5:15.300
分數並不能完全說明全部情況。
5:15.300–5:18.620
第一點是，這個模型在 UI 設計方面實際上非常出色。
5:18.620–5:22.340
即使在邏輯上丟分的問題上，
5:20.480–5:22.340
它產生的設計看起來也非常
5:22.340–5:23.340
好。
5:23.340–5:26.800
而且現在它已經提供 API，
5:25.070–5:26.800
你有可能在 workflows 中使用它，例如
5:26.800–5:30.340
使用 Open Design 來生成一些瘋狂的設計，然後
5:28.570–5:30.340
將其交給類似
5:30.340–5:33.580
GPT 5.6 的模型來處理實際的實現。
5:33.580–5:35.520
這種組合可能會非常強大。
5:35.520–5:39.200
第二點是，雖然這個模型擅長代理式任務，
5:37.360–5:39.200
但在代理式使用中仍然相當
5:39.200–5:40.980
奇怪。
5:40.980–5:44.080
舉個例子，
5:42.530–5:44.080
它在寫入之前不會查看其他現有文件。
5:44.080–5:47.800
我通常在 Open Code 中運行多個代理程序會話，以確保所有提示
5:47.800–5:48.800
都已完成。
5:48.800–5:52.320
我要求每個模型建立一個新文件，
5:50.560–5:52.320
不要觸碰其他文件，但這個模型根本不在乎。
5:52.320–5:53.320
每個會話都覆蓋了其他會話，並且
5:53.320–5:56.080
每個會話都會覆蓋其他會話，並且
5:54.700–5:56.080
它們甚至不會去查看既有的檔案
5:56.080–5:57.080
首先。
5:57.080–5:58.320
在這點上真的很糟糕。
5:58.320–6:01.700
所以如果你以平行設定執行它，要小心，
6:00.010–6:01.700
因為它會毫不猶豫地覆蓋
6:01.700–6:02.960
你的檔案。
6:02.960–6:05.740
第三點是，當模型運作正常時，它仍然相當優秀。
6:05.740–6:10.040
當它在自己的領域內執行代理任務、工具使用，
6:07.890–6:10.040
或是處理那道數學題時，它確實
6:10.040–6:11.040
能交出好成績。
6:11.040–6:12.460
問題在於一致性。
6:12.460–6:15.060
你永遠無法確定會得到哪個版本的模型。
6:15.060–6:18.340
總體而言，我認為這是 Meta 非常有趣的首次嘗試。
6:18.340–6:22.320
它不會取代 Fable 5 或 Opus 4.8 在程式碼方面的地位。
6:22.320–6:25.140
但代理能力是真實存在的。
6:25.140–6:26.700
定價也還不錯。
6:26.700–6:29.280
而且 UI 設計感是個不錯的驚喜。
6:29.280–6:33.160
如果他們能修復檔案意識和一致性的問題，
6:31.220–6:33.160
下一個版本可能會成為一個嚴肅的
6:33.160–6:34.160
競爭對手。
6:34.160–6:35.160
總體來說相當酷。
6:35.160–6:37.380
無論如何，請在留言區告訴我你的想法。
6:37.380–6:41.260
如果你喜歡這部影片，
6:39.320–6:41.260
考慮透過 Super Thanks 選項捐款，或透過點擊加入按鈕
6:41.260–6:42.880
成為會員。
6:42.880–6:45.880
此外，請為這部影片按讚並訂閱我的頻道。
6:45.880–6:46.880
我們下一部影片見。
6:46.880–6:48.480
在那之前，再見。
6:48.480–6:49.480
再見。
6:49.480–6:50.480
！
6:50.480–6:51.480
！
6:51.480–6:52.480
！
6:52.480–6:53.480
再見。
6:53.480–6:54.480
再見。
6:54.480–6:55.480
再見。
6:55.480–6:56.480
再見。
6:56.480–6:57.480
再見。
6:57.480–6:58.480
再見。
6:58.480–6:59.480
再見。
6:59.480–7:00.480
再見。
7:00.480–7:01.480
再見。
7:01.480–7:02.480
再見。
7:02.480–7:03.000
再見。
0:00.000–0:07.760
Hi, welcome to another video.
嗨，歡迎收看另一個影片。
0:07.760–0:11.920
So, Meta has finally entered the frontier model race for real this time.
這次 Meta 終於真正進入了前沿模型的競賽。
0:11.920–0:16.360
They just released MuseSpark 1.1,
他們剛剛發布了 MuseSpark 1.1，
0:14.140–0:16.360
which is their new multimodal reasoning model built
這是他們為代理任務（agentic tasks）打造的全新多模態推理模型，
0:16.360–0:18.240
for agentic tasks.
專門用於此類任務。
0:18.240–0:21.640
This is coming out of their superintelligence labs,
該模型出自他們的超智能實驗室（superintelligence labs），
0:19.940–0:21.640
and it's also the first model that's
同時也是第一個
0:21.640–0:24.200
available through their brand new Meta model API.
透過他們全新推出的 Meta 模型 API 提供的模型。
0:24.200–0:29.160
Yes, Meta now has a paid API, and
沒錯，Meta 現在有了付費 API，
0:26.680–0:29.160
Zuckerberg even came back to X after three years just
扎克伯格甚至在離開 X 三年後回歸，
0:29.160–0:30.160
to launch this thing.
就是為了發布這項產品。
0:30.160–0:32.940
So, they're definitely taking this seriously.
他們絕對是認真對待這件事。
0:32.940–0:34.960
Now what is this model exactly?
那麼，這個模型究竟是什麼呢？
0:34.960–0:39.460
It's a multimodal reasoning model that combines reasoning,
它是一個結合推理、
0:37.210–0:39.460
coding, computer use, and multimodal
編碼、電腦操作和多模態
0:39.460–0:40.560
understanding.
理解能力的多模態推理模型。
0:40.560–0:45.580
It has a 1 million token context window, and
它擁有 100 萬 token 的上下文視窗，
0:43.070–0:45.580
Meta says it can actively manage that context.
而 Meta 表示它能主動管理該上下文。
0:45.580–0:49.320
It remembers actions,
它會記住操作，
0:47.450–0:49.320
retrieves information from much earlier in the session, and compacts
從會話更早階段檢索資訊，並以保留關鍵步驟的方式
0:49.320–0:51.320
things in a way that keeps the critical steps.
壓縮內容。
0:51.320–0:54.840
It's also trained to orchestrate multi-agent systems,
它還經過訓練以協調多代理系統，
0:53.080–0:54.840
so it can act as a main agent that
因此它可以作為主代理，
0:54.840–0:59.040
plans and delegates, or as a sub-agent that sticks to its job and
負責規劃和委派任務，或作為子代理，專注於自身工作，
0:56.940–0:59.040
escalates when needed.
並在必要時進行升級報告。
0:59.040–1:04.800
On the benchmarks, Meta claims it scored 88.
在基準測試方面，Meta 聲稱它獲得了 88.
1:01.920–1:04.800
1 on MCP Atlas, which tests scaled tool use,
在 MCP Atlas 上獲得 1 分，該基準測試評估擴展工具的使用能力，
1:04.800–1:08.880
while Opus 4.8 and GPT 5.5 are around 80 there.
而 Opus 4.8 和 GPT 5.5 在那裡大約只有 80 分。
1:08.880–1:12.400
It also leads on JawBench, and a few other tool use benchmarks.
它在 JawBench 和其他幾個工具使用基準測試中也領先。
1:12.400–1:18.340
But interestingly, on TerminalBench, it only scored 59,
但有趣的是，在 TerminalBench 上，它只得了 59 分，
1:15.370–1:18.340
while GPT 5.5 is at around 83.
而 GPT 5.5 大約有 83 分。
1:18.340–1:22.500
So it seems like the model is really strong on tool use and
因此，這個模型似乎在工具使用和
1:20.420–1:22.500
agentic stuff, but not exactly
代理任務方面非常強大，但並非在純編碼方面
1:22.500–1:24.040
dominant on pure coding.
具有絕對優勢。
1:24.040–1:27.540
Take these claim numbers with a grain of salt,
不過，請對這些聲稱的數據持保留態度，
1:25.790–1:27.540
though, because we'll do our own testing.
因為我們會進行自己的測試。
1:27.540–1:33.220
For the pricing, it's $125 per million input tokens,
關於定價，輸入 token 每百萬 125 美元，
1:30.380–1:33.220
and $425 per million output tokens,
輸出 token 每百萬 425 美元，
1:33.220–1:34.700
which is actually pretty reasonable.
這其實相當合理。
1:34.700–1:38.080
It's cheaper than Sonnet, and you also
它比 Sonnet 便宜，而且你還
1:36.390–1:38.080
get $20 in free credits when you sign up for the
註冊 API 預覽版時，可獲得 20 美元的免費信用額度。
1:38.080–1:39.300
API preview.
API 預覽版。
1:39.300–1:40.800
So that's good.
這還挺不錯的。
1:40.800–1:45.800
Now let's put it through my own benchmark, KingBench,
現在讓我們用我自己的基準測試 KingBench 來測試它，
1:43.300–1:45.800
and see how it actually performs.
看看它實際表現如何。
1:45.800–1:52.360
I have seven questions here covering UI generation,
這裡有七個問題，涵蓋 UI 生成、
1:49.080–1:52.360
3JS, SVG games, hard math, and agentic tasks.
3JS、SVG 遊戲、高難度數學和代理任務。
1:52.360–1:53.980
Each question is scored out of ten.
每個問題滿分為 10 分。
1:53.980–1:55.620
Let's get right into it.
讓我們直接開始。
1:55.620–1:57.900
The first question is the elevator simulation.
第一個問題是電梯模擬。
1:57.900–2:02.120
The model has to build a simulation in HTML, CSS,
模型必須用 HTML、CSS
2:00.010–2:02.120
and JS where you can spawn people on different
和 JS 構建一個模擬，讓你可以
2:02.120–2:03.120
levels.
在不同樓層生成人物。
2:03.120–2:04.120
There are three elevators.
有三部電梯。
2:04.120–2:06.480
Each elevator can only take one person, and
每部電梯只能載一人，而
2:05.300–2:06.480
the people left behind should catch the
留在後面的人應該搭乘
2:06.480–2:07.680
next one.
下一部電梯。
2:07.680–2:11.760
Every person has a random target floor with a tooltip on hover,
每個人都有隨機的目標樓層，滑鼠懸停時會顯示提示，
2:09.720–2:11.760
and it should be well-animated.
且動畫應該要流暢。
2:11.760–2:15.780
Here's what it generated, and it does look pretty nice visually,
這是它生成的結果，視覺上看起來相當不錯，
2:13.770–2:15.780
but the logic is broken.
但邏輯有問題。
2:15.780–2:19.740
The elevators are not respecting the one-person rule properly,
電梯沒有正確遵守一人一部的規則，
2:17.760–2:19.740
and the cueing behavior just
而且排隊行為在
2:19.740–2:22.060
falls apart when you spawn multiple people.
生成多個人物時就完全崩潰了。
2:22.060–2:25.060
The animation is also janky, so this is mostly a fail.
動畫也很卡頓，所以這基本上算是失敗。
2:25.060–2:27.080
I'm giving it a 3 out of 10 here.
我給它 3 分（滿分 10 分）。
2:27.080–2:29.600
For comparison, Opus 4.8.
作為比較，Opus 得到 4.8 分。
2:29.600–2:31.100
Got a full 10 on this one.
這一題它拿到了滿分 10 分。
2:31.100–2:35.080
The second question is the 3JS 3D model of a contact lens case.
第二個問題是隱形眼鏡盒的 3JS 3D 模型。
2:35.080–2:38.580
It should have pronounced L and R caps, and
它應該要有明顯的 L 和 R 蓋子，而且
2:36.830–2:38.580
you should be able to click on the caps to
你應該可以點擊蓋子
2:38.580–2:39.580
open them up.
將它們打開。
2:39.580–2:41.160
So, here's the result.
所以，這是結果。
2:41.160–2:43.720
The case renders, and the L and R are there.
鏡盒渲染出來了，L 和 R 也都在。
2:43.720–2:48.360
The materials actually look quite good, but
材質看起來相當不錯，但
2:46.040–2:48.360
the click-to-open interaction is only half-working.
點擊打開的互動功能只完成了一半。
2:48.360–2:52.160
One cap opens weirdly and clips through the body,
其中一個蓋子打開的方式很奇怪，並且會穿模到鏡盒主體，
2:50.260–2:52.160
so this is a partial pass.
所以這算是部分通過。
2:52.160–2:54.480
I'm giving it a 5 out of 10.
我給它 5 分（滿分 10 分）。
2:54.480–2:57.500
The third question is the 3JS folding table with a slider.
第三個問題是帶有滑桿的 3JS 摺疊桌。
2:57.500–3:01.500
As you take the slider to the right, the table should unfold,
當你將滑桿向右移動時，桌子應該展開，
2:59.500–3:01.500
and to the left, it should
向左移動時，它應該
3:01.500–3:02.500
fold.
摺疊。
3:02.500–3:04.760
It should be seamlessly animated, and be 3D.
動畫應該無縫銜接，並且是 3D 的。
3:04.760–3:07.400
This one is actually decent.
這個其實還不錯。
3:07.400–3:11.460
The table folds and unfolds with the slider, and
桌子隨著滑桿摺疊和展開，
3:09.430–3:11.460
the animation is fairly smooth.
動畫相當流暢。
3:11.460–3:16.500
The geometry gets a bit wonky at the extremes, but
幾何結構在極端情況下會變得有點奇怪，但
3:13.980–3:16.500
overall it works, so this is a 7 out of
整體來說運作正常，所以這題得 7 分（滿分 10 分）。
3:16.500–3:17.680
10.
10 分。
3:17.680–3:18.680
Not bad at all.
完全不算差。
3:18.680–3:21.920
The fifth question is the SVG of a panda eating a burger.
第五個問題是吃漢堡的熊貓 SVG 圖檔。
3:21.920–3:25.680
This tests whether the model can visualize things in code without seeing them.
這測試模型是否能在沒看到圖的情況下，用程式碼視覺化事物。
3:25.680–3:26.920
Here's what it made.
這是它生成的結果。
3:26.920–3:30.500
The panda is recognizable, and
熊貓的輪廓可辨識，
3:28.710–3:30.500
there is something that looks like a burger.
而且有個看起來像漢堡的東西。
3:30.500–3:31.500
It's mid.
表現普通。
3:31.500–3:34.080
It's not the worst I've seen, but it's not great either.
這不是我看過最差的，但也稱不上優秀。
3:34.080–3:35.320
5 out of 10.
得 5 分（滿分 10 分）。
3:35.320–3:39.220
The fifth question is the bow and
第五個問題是弓和
3:37.270–3:39.220
arrow simulator game with 4 targets and a leaderboard based
箭模擬器遊戲，有 4 個目標，並根據
3:39.220–3:40.700
on the lowest time.
最短時間生成排行榜。
3:40.700–3:43.080
This one is actually good.
這個其實表現不錯。
3:43.080–3:47.440
The bow mechanics work, the target's register hits,
弓的機制運作正常，目標能正確記錄命中，
3:45.260–3:47.440
and the leaderboard updates properly.
排行榜也能正確更新。
3:47.440–3:51.040
The visuals are also pretty clean, so this is an 8 out of 10.
視覺效果也很乾淨，所以這題得 8 分（滿分 10 分）。
3:51.040–3:52.180
Pretty good.
相當不錯。
3:52.180–3:55.160
The sixth question is the hard math one.
第六個問題是困難的數學題。
3:55.160–3:58.600
It's a permutation counting problem over a grid of ordered pairs,
這是一個在有序配對網格上的排列計數問題，
3:56.880–3:58.600
and the answer should
答案應該
3:58.600–4:00.300
be 20460.
是 20460。
4:00.300–4:02.440
This question destroys most models.
這題摧毀了大多數模型。
4:02.440–4:08.640
GPT 5.5, Opus 4.7, and Gemini 3.5 flash all got zero on this.
GPT 5.5、Opus 4.7 和 Gemini 3.5 flash 在這題都得了零分。
4:08.640–4:11.260
And believe it or not, it got 2460.
信不信由你，它得了 2460。
4:11.260–4:12.260
That's exactly right.
完全正確。
4:12.260–4:13.900
So, this is a full 10 out of 10.
所以，這題是滿分 10 分。
4:13.900–4:15.560
Really impressive, to be honest.
老實說，真的很令人印象深刻。
4:15.560–4:17.940
The seventh question is the agentic one.
第七個問題是代理型（agentic）題目。
4:17.940–4:21.740
The model has to generate a dataset of facts about pandas,
模型必須生成關於熊貓的事實資料集，
4:19.840–4:21.740
Finitune, Gemma, 2B model on
微調 Gemma 2B 模型，在
4:21.740–4:25.520
that, all locally, and then
全部在本地完成，然後
4:23.630–4:25.520
give me a web UI where every refresh generates a new panda
給我一個網頁介面，每次重新整理都會產生一個新的企鵝
4:25.520–4:26.520
fact.
事實。
4:26.520–4:27.780
And it did the whole thing.
它確實完成了整個過程。
4:27.780–4:31.540
It generated the dataset, set up the Finituning,
它產生了數據集，設定了微調，
4:29.660–4:31.540
ran it, and the web UI works.
執行了它，而且網頁介面運作正常。
4:31.540–4:35.000
Every refresh gives me a new panda fact from the Finitune model.
每次重新整理都會從微調模型中給我一個新的企鵝事實。
4:35.000–4:38.420
So this is a full 10 out of 10 as well.
所以這部分也得了滿分十分。
4:38.420–4:40.820
This is where you can see the agentic training paying off.
這就是你能看到代理式訓練帶來回報的地方。
4:40.820–4:43.200
So this is the final chart.
所以這是最後一張圖表。
4:43.200–4:53.620
And Muse Spark 1 ends up with 48 out of 70, which is 68.57%.
Muse Spark 1 最終得分為 70 分中的 48 分，即 68.57%。
4:53.620–4:59.280
That puts it in sixth place on the leaderboard,
這讓它排在排行榜第六名，
4:56.450–4:59.280
above GPT 5.6 Terra, Opus 4.7, and Sonnet 5,
高於 GPT 5.6 Terra、Opus 4.7 和 Sonnet 5，
4:59.280–5:07.480
below Grok 4.5, GPT 5.6 Sol, GLM 5.2, Opus 4.8, and
低於 Grok 4.5、GPT 5.6 Sol、GLM 5.2、Opus 4.8，以及
5:03.380–5:07.480
Fable 5 at the top, for a first real
位居榜首的 Fable 5，這是 Meta 首次真正的
5:07.480–5:09.020
frontier attempt from Meta.
前沿嘗試。
5:09.020–5:10.520
This is honestly a pretty solid showing.
老實說，這是一個相當穩健的表現。
5:10.520–5:13.980
Now, let me tell you some observations from using this model beyond the benchmark, because
現在，讓我分享一些在使用這個模型時，超出基準測試範圍的觀察，因為
5:13.980–5:15.300
the scores don't tell the full story.
分數並不能完全說明全部情況。
5:15.300–5:18.620
The first thing is that this model is actually really good at UI.
第一點是，這個模型在 UI 設計方面實際上非常出色。
5:18.620–5:22.340
Even on the questions where it lost points for logic,
即使在邏輯上丟分的問題上，
5:20.480–5:22.340
the designs it produces look really
它產生的設計看起來也非常
5:22.340–5:23.340
good.
好。
5:23.340–5:26.800
And now that it's on API,
而且現在它已經提供 API，
5:25.070–5:26.800
you can potentially use it in your workflow with things like
你有可能在 workflows 中使用它，例如
5:26.800–5:30.340
Open Design to generate some crazy designs, and then
使用 Open Design 來生成一些瘋狂的設計，然後
5:28.570–5:30.340
hand that over to something like
將其交給類似
5:30.340–5:33.580
GPT 5.6 to work on the actual implementation.
GPT 5.6 的模型來處理實際的實現。
5:33.580–5:35.520
That combo could be really powerful.
這種組合可能會非常強大。
5:35.520–5:39.200
The second thing is that, while the model is good at agentic tasks,
第二點是，雖然這個模型擅長代理式任務，
5:37.360–5:39.200
it is still pretty
但在代理式使用中仍然相當
5:39.200–5:40.980
weird in agentic use.
奇怪。
5:40.980–5:44.080
For an example,
舉個例子，
5:42.530–5:44.080
it doesn't look at other existing files before writing.
它在寫入之前不會查看其他現有文件。
5:44.080–5:47.800
I generally have multiple sessions of agents running in open code to make sure all my prompts
我通常在 Open Code 中運行多個代理程序會話，以確保所有提示
5:47.800–5:48.800
are done.
都已完成。
5:48.800–5:52.320
I ask each model to make a new file and
我要求每個模型建立一個新文件，
5:50.560–5:52.320
not touch the others, but this one just doesn't
不要觸碰其他文件，但這個模型根本不在乎。
5:52.320–5:53.320
care.
每個會話都覆蓋了其他會話，並且
5:53.320–5:56.080
Each session overwrote the others, and
每個會話都會覆蓋其他會話，並且
5:54.700–5:56.080
none of them even looked at the existing files
它們甚至不會去查看既有的檔案
5:56.080–5:57.080
first.
首先。
5:57.080–5:58.320
It is really bad in that regard.
在這點上真的很糟糕。
5:58.320–6:01.700
So if you're running it in parallel setups, be careful,
所以如果你以平行設定執行它，要小心，
6:00.010–6:01.700
because it will happily clobber
因為它會毫不猶豫地覆蓋
6:01.700–6:02.960
your files.
你的檔案。
6:02.960–6:05.740
The third thing is that the model is still quite good when it works.
第三點是，當模型運作正常時，它仍然相當優秀。
6:05.740–6:10.040
When it's in its lane, doing agentic tasks, tool use,
當它在自己的領域內執行代理任務、工具使用，
6:07.890–6:10.040
or that math question, it genuinely
或是處理那道數學題時，它確實
6:10.040–6:11.040
delivers.
能交出好成績。
6:11.040–6:12.460
The problem is consistency.
問題在於一致性。
6:12.460–6:15.060
You never quite know which version of the model you're going to get.
你永遠無法確定會得到哪個版本的模型。
6:15.060–6:18.340
Overall I think this is a really interesting first entry from Meta.
總體而言，我認為這是 Meta 非常有趣的首次嘗試。
6:18.340–6:22.320
It's not going to replace Fable 5 or Opus 4.8 for coding.
它不會取代 Fable 5 或 Opus 4.8 在程式碼方面的地位。
6:22.320–6:25.140
But the agentic capabilities are real.
但代理能力是真實存在的。
6:25.140–6:26.700
The pricing is decent.
定價也還不錯。
6:26.700–6:29.280
And the UI design sense is a nice surprise.
而且 UI 設計感是個不錯的驚喜。
6:29.280–6:33.160
If they fix the file awareness issues and the consistency,
如果他們能修復檔案意識和一致性的問題，
6:31.220–6:33.160
the next version could be a serious
下一個版本可能會成為一個嚴肅的
6:33.160–6:34.160
contender.
競爭對手。
6:34.160–6:35.160
Overall it's pretty cool.
總體來說相當酷。
6:35.160–6:37.380
Anyway, let me know your thoughts in the comments.
無論如何，請在留言區告訴我你的想法。
6:37.380–6:41.260
If you liked this video,
如果你喜歡這部影片，
6:39.320–6:41.260
consider donating through the Super Thanks option, or becoming
考慮透過 Super Thanks 選項捐款，或透過點擊加入按鈕
6:41.260–6:42.880
a member by clicking the Join button.
成為會員。
6:42.880–6:45.880
Also, give this video a thumbs up and subscribe to my channel.
此外，請為這部影片按讚並訂閱我的頻道。
6:45.880–6:46.880
I'll see you in the next one.
我們下一部影片見。
6:46.880–6:48.480
Until then, bye.
在那之前，再見。
6:48.480–6:49.480
Bye.
再見。
6:49.480–6:50.480
!
！
6:50.480–6:51.480
!
！
6:51.480–6:52.480
!
！
6:52.480–6:53.480
Bye.
再見。
6:53.480–6:54.480
Bye.
再見。
6:54.480–6:55.480
Bye.
再見。
6:55.480–6:56.480
Bye.
再見。
6:56.480–6:57.480
Bye.
再見。
6:57.480–6:58.480
Bye.
再見。
6:58.480–6:59.480
Bye.
再見。
6:59.480–7:00.480
Bye.
再見。
7:00.480–7:01.480
Bye.
再見。
7:01.480–7:02.480
Bye.
再見。
7:02.480–7:03.000
Bye.
再見。

影片筆記：Muse Spark 1.1 (Fully Tested): Okay, it's SO GOOD!

一句話總結

Meta 發布多模態推理模型 MuseSpark 1.1，專為代理任務（Agentic tasks）與工具使用設計，在自定義基準測試中表現優異於部分競爭對手，但在純編碼與文件一致性上仍存在顯著限制。

核心重點

  • 模型定位：Meta 推出的前沿多模態推理模型，專為代理任務（Agentic tasks）設計，支援 100 萬 token 上下文窗口，具備主動管理上下文與協調多代理系統的能力。
  • 基準測試表現：
  • 優勢：在工具使用（MCP Atlas, JawBench）與代理任務上表現強勁；在作者自定義的 KingBench 測試中總分 48/70（68.57%），排名第六。
  • 劣勢：在純編碼基準（TerminalBench）表現較弱（59 分），且存在文件覆蓋（clobber files）與缺乏文件感知能力的一致性問題。
  • 定價策略：輸入 $125/百萬 token，輸出 $425/百萬 token，被評價為價格合理且比 Sonnet 便宜。
  • 實際應用建議：UI 設計視覺效果好但邏輯常有誤，建議搭配其他模型進行實現；目前尚無法取代 Fable 5 或 Opus 4.8 進行高難度編碼工作。

詳細大綱

1. Meta 新模型發布與技術規格

  • 發布背景：Meta 正式進入前沿模型競賽，由超級智能實驗室（superintelligence labs）發布 MuseSpark 1.1。
  • API 服務：
  • 首次透過品牌全新的 Meta 模型 API 提供付費服務。
  • 由 Zuckerberg 回歸 X 平台進行發布。
  • 技術特性：
  • 多模態推理模型，結合推理、編碼、電腦使用與多模態理解。
  • 擁有 100 萬 token 上下文窗口，能主動管理上下文（記憶動作、檢索早期資訊、壓縮關鍵步驟）。
  • 訓練用於協調多代理系統（可作為主代理規劃委派，或作為子代理執行任務並升級報告）。

2. 官方基準測試數據

  • 工具使用表現：
  • MCP Atlas：88.1 分（測試擴展工具使用）。
  • JawBench 及其他工具使用基準測試中領先。
  • 對比對象：Opus 4.8 與 GPT 5.5 約為 80 分。
  • 編碼表現：
  • TerminalBench：僅得 59 分。
  • 對比對象：GPT 5.5 約為 83 分。
  • 結論：在工具使用與代理任務上強勁，但在純編碼上並非主導地位。

3. 定價與優惠

  • 價格結構：
  • 輸入價格：每百萬輸入 token $125。
  • 輸出價格：每百萬輸出 token $425。
  • 評價：價格合理，比 Sonnet 便宜。
  • 優惠：註冊 API 預覽可獲得 $20 免費額度。

4. 作者自定義基準測試（KingBench）結果

  • 測試範圍：7 個問題，涵蓋 UI 生成、3JS、SVG、數學、代理任務。每題滿分 10 分。
  • 題目 1：電梯模擬（HTML/CSS/JS）
  • 要求：生成多層樓人員，三部電梯（每部限一人），隨機目標樓層，懸停提示，動畫良好。
  • 結果：視覺不錯，但邏輯錯誤（未遵守一人規則，多人生成時隊列崩潰，動畫卡頓）。
  • 評分：3/10（失敗）。
  • 對比：Opus 4.8 得 10/10。
  • 題目 2：3JS 隱形眼鏡盒 3D 模型
  • 要求：明顯的 L/R 蓋子，點擊打開互動。
  • 結果：渲染與材質良好，但點擊打開互動僅部分工作（一個蓋子打開時會穿模）。
  • 評分：5/10（部分通過）。
  • 題目 3：3JS 折疊桌與滑桿
  • 要求：滑桿控制折疊/展開，無縫動畫，3D。
  • 結果：折疊與展開運作良好，動畫平滑，極端幾何形狀略顯奇怪。
  • 評分：7/10。
  • 題目 4（原文標為第五）：SVG 熊貓吃漢堡
  • 要求：代碼可視化，無需視覺參考。
  • 結果：熊貓可識別，漢堡可辨認，表現中等。
  • 評分：5/10。
  • 題目 5（原文標為第五）：弓箭模擬器遊戲
  • 要求：4 個目標，基於最低時間的排行榜。
  • 結果：弓箭機制運作正常，目標註冊命中，排行榜更新正確，視覺乾淨。
  • 評分：8/10。
  • 題目 6：困難數學題
  • 要求：網格有序對的排列計數問題，答案應為 20460。
  • 結果：GPT 5.5, Opus 4.7, Gemini 3.5 flash 均得 0 分；MuseSpark 1.1 得出 2460（正確答案）。
  • 評分：10/10。
  • 題目 7：代理任務
  • 要求：生成熊貓事實數據集，在本地對 Finitune, Gemma, 2B 模型進行微調（Finituning），並提供 Web UI，每次刷新生成新熊貓事實。
  • 結果：完整執行數據集生成、微調設置、運行及 Web UI 工作。
  • 評分：10/10。
  • 總分與排名：
  • 總分：48/70（68.57%）。
  • 排名：第六名。
  • 勝過：GPT 5.6 Terra, Opus 4.7, Sonnet 5。
  • 落後於：Grok 4.5, GPT 5.6 Sol, GLM 5.2, Opus 4.8, Fable 5。

5. 實際使用觀察與結論

  • UI 設計能力：
  • 即使邏輯有誤，生成的設計視覺效果很好。
  • 建議工作流：使用 Open Design 生成設計，再交給 GPT 5.6 進行實際實現。
  • 代理任務的怪異之處：
  • 缺乏文件感知能力：寫入新文件前不檢查現有文件。
  • 並行任務問題：多個代理 sessions 會互相覆蓋文件（clobber files），不尊重其他 session 的文件。
  • 一致性問題：
  • 在特定領域（代理、工具使用、數學）表現良好，但整體一致性不足，無法預測每次輸出的版本品質。
  • 最終評價：
  • Meta 的首次前沿嘗試相當穩健。
  • 定價不錯，代理能力真實，UI 設計是驚喜。
  • 目前無法取代 Fable 5 或 Opus 4.8 進行編碼。
  • 若修復文件感知與一致性問題，下一版本可能成為有力競爭者。

工具 / 模型 / 名詞整理

  • Meta 相關：
  • MuseSpark 1.1（多模態推理模型）
  • Meta 模型 API
  • Meta 超級智能實驗室（superintelligence labs）
  • 競爭對手模型：
  • Opus 4.8
  • GPT 5.5
  • GPT 5.6 Terra
  • GPT 5.6 Sol
  • Sonnet
  • Grok 4.5
  • GLM 5.2
  • Fable 5
  • Opus 4.7
  • Gemini 3.5 flash
  • 測試與基準工具：
  • MCP Atlas
  • JawBench
  • TerminalBench
  • KingBench（作者自定義基準）
  • 技術與框架：
  • HTML, CSS, JS
  • 3JS（3D 建模相關）
  • SVG
  • Finitune（疑點：可能指 Fine-tuning 或特定工具名）
  • Gemma, 2B 模型
  • Open Design（疑點：可能指特定設計工具或概念）
  • 平台：
  • X（前 Twitter）

操作流程整理

  1. 獲取 API：透過 Meta 模型 API 註冊預覽，獲得 $20 免費額度。
  2. 執行基準測試：
  • 運行官方基準測試（MCP Atlas, JawBench, TerminalBench）。
  • 運行作者自定義的 KingBench 測試（涵蓋 UI、3D、數學、代理任務）。
  1. 評估結果：
  • 比較 MuseSpark 1.1 與其他模型（如 Opus 4.8, GPT 5.5）的分數。
  • 分析代理任務中的文件覆蓋問題與一致性。
  1. 應用建議：
  • 對於 UI 設計任務，可先用 MuseSpark 生成視覺設計，再轉交其他模型實現邏輯。
  • 對於高難度編碼任務，目前不建議直接取代 Fable 5 或 Opus 4.8。

值得注意的限制或風險

  • 文件一致性問題：在並行任務中，多個代理 sessions 會互相覆蓋文件（clobber files），缺乏對現有文件的感知能力。
  • 邏輯與視覺脫節：UI 生成視覺效果好，但常出現邏輯錯誤（如電梯模擬中的隊列崩潰）。
  • 純編碼能力不足：在 TerminalBench 等純編碼基準上表現較弱，無法取代頂級編碼模型。
  • 預測不穩定：整體一致性不足，無法預測每次輸出的版本品質。

逐字稿辨識疑點

  • MuseSpark 1.1：逐字稿中提及 "MuseSpark"，需確認是否為正確模型名稱（疑點：需查證）。
  • Finitune：逐字稿中提及 "Finitune, Gemma, 2B model"，"Finitune" 疑似為 "Fine-tuning" 的聽寫錯誤或特定產品名（疑點：需查證）。
  • Open Design：逐字稿中提及 "Open Design"，需確認是否為特定軟體名稱（疑點：需查證）。
  • GPT 5.5 / GPT 5.6：逐字稿中多次出現 "GPT 5.5" 與 "GPT 5.6"，需確認是否為模型版本號的準確表述（疑點：需查證）。
  • Opus 4.8 / Opus 4.7：逐字稿中出現不同版本的 Opus，需確認是否為準確版本號（疑點：需查證）。
  • KingBench：作者自定義的基準測試名稱，需確認是否為通用標準或作者自創（疑點：需查證）。
  • TerminalBench 分數：逐字稿稱 MuseSpark 得 59 分，GPT 5.5 約 83 分，需確認數據來源（疑點：需查證）。
  • MCP Atlas 分數：逐字稿稱 MuseSpark 得 88.1 分，Opus 4.8 與 GPT 5.5 約 80 分，需確認數據來源（疑點：需查證）。
  • 數學題答案：逐字稿稱正確答案為 20460，MuseSpark 得出 2460 被視為正確，需確認題目與答案邏輯（疑點：需查證）。
  • Zuckerberg 回歸 X：逐字稿稱 Zuckerberg 回歸 X 平台發布此 API，需確認時間點與事實（疑點：需查證）。

可延伸追問

  • Meta 模型 API 的長期定價策略與可用性如何？
  • KingBench 的具體測試題目與評分標準是否公開？
  • 如何解決 MuseSpark 1.1 在並行代理任務中的文件覆蓋問題？
  • 未來版本是否會針對純編碼能力進行專門優化？

尚未產生學習筆記

請在 Telegram 指令最後加上「學習」，例如：videonote 網址 英文 雙語 學習