WEBVTT

00:00.000 --> 00:09.000
一萬個AI agent 一起上,很容易讓人以為突破關鍵就在agent數量,但他們到底解釋了多少。

00:30.000 --> 00:59.000
這幾乎是很強的。

00:59.000 --> 01:10.000
這集上線前9天,這項數學成果才剛公開受訪者做過多智能體與推理模型研究,也量過增加代理數量到底能換到多少速度。

01:29.000 --> 01:34.000
你會看到一個相似的模式,有點不足夠的,但你會繼續看到這個效果。

01:34.000 --> 01:40.000
是是一個平常的盡量時間速度,還是一個下載的速度,當你增加了數量的交換?

01:40.000 --> 01:47.000
我會說是比較下載的,但它會很依照問題。

01:47.000 --> 01:50.000
所以,數量是比較下載的。

01:50.000 --> 01:55.000
代理數量增加確實能變快,但成本也跟著上去。

01:55.000 --> 01:59.000
規模一路放大後,這個加速還剩多少。

01:59.000 --> 02:29.000
所以,數量是在數量量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減

02:29.000 --> 02:59.000
這就是,實際上,減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數

02:59.000 --> 03:02.000
一指複製主持會變成什麼樣子

03:29.000 --> 03:59.000
減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量�

03:59.000 --> 04:29.000
減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量減少數量�

04:29.000 --> 04:58.920
and sooner than I previously thought. I feel like we've gone in mathematics from, let's say, in 2024. You have AIs where, like, oh, okay, interesting. They're, like, doing, they can, like, solve a couple problems on high school math competitions. And then in 2025, it's like, oh, wow, they can get gold in, like, International Math Olympiad. And earlier this year, I was like, wow, they're actually solving open problems in mathematics, like open nervous problems, but maybe, like, I don't know, people weren't trying that hard. And it was just like, there was a similar solution somewhere in the literature. And now I just think it's sort of undeniable, right? It's like, this is the Millennium Prize problem. There's really

04:58.920 --> 05:02.740
no, there's no story of why this should have been easy.

05:03.640 --> 05:06.700
這裡不是在猜通用型的AI會在哪一年出現.

05:07.060 --> 05:11.560
受訪者提到一位研究者,最近連自己的預測範圍都縮短了.

05:11.880 --> 05:12.840
到底縮到多短?

05:13.800 --> 05:27.840
And then you get to 2026, and like, I mean, literally two weeks before we got Navier Stokes, I was talking with a researcher at a frontier lab about how long it would take to get a Millennium Prize.

05:27.840 --> 05:38.160
and he was willing to bet me $1,000 that it would take that it would take past 2027. And he thought it would take until 2030, you know, and I took that bet.

05:40.960 --> 05:53.280
But even I thought it would take longer than than how long it's likely to take. So people have been continuously surprised, even even inside the labs. And I was literally I was just talking to somebody yesterday,

05:53.280 --> 06:02.280
who was working on the Navier Stokes effort. And he was telling me that like, he used to say it's really hard to predict where AI would be in 12 months.

06:02.280 --> 06:07.060
You know, if somebody asked him like, oh, where are things going? He would feel comfortable making predictions for like the next 12 months.

06:07.060 --> 06:12.060
But beyond that, you know, he is just like, I don't know. And now he's saying like, he just doesn't feel comfortable making predictions beyond three months.

06:12.060 --> 06:17.060
So it is really true that yeah, things are going things are going very fast right now.

06:17.060 --> 06:24.060
I think that there are some strategies. So I think chain of thought monitoring is one that we've been, I think as soon as we got the reasoning models,

06:24.060 --> 06:31.060
Yacob to his credit was like very, very clear that we cannot supervise chain of thought because this is really a gift. Like monitor ability for neural nets is extremely hard.

06:31.060 --> 06:36.060
And here we have a situation where the neural nets are just like flat out reasoning,

06:36.060 --> 06:41.060
and it's like flat out reasoning, and it's like, this is really a gift. Like monitor ability for neural nets is extremely hard. And here we have a situation where the neural nets are just like flat out reasoning,

06:41.060 --> 06:53.060
and it's like flat out reasoning, and it's like, this is really a gift. Like monitor ability for neural nets is extremely hard. And here we have a situation where the neural nets are just like flat out reasoning,

06:53.060 --> 07:02.060
like laying out their thought process in natural language for us to read. That is like, so convenient. It's like really the best case scenario for safety.

07:02.060 --> 07:09.060
But if you look at that chain of thought and say, oh, the model is thinking bad thoughts, and we should punish it for thinking those bad thoughts,

07:09.060 --> 07:13.060
then what ends up happening is the model just learns to think those bad thoughts in a way that's not observable to us.

07:13.060 --> 07:22.060
There could be a situation where like, by looking at the chain of thought, we can see, oh, the model is scheming and doing misaligned things in a way that's actually not being detected in our alignment metrics.

07:22.060 --> 07:33.060
Now, the problem is that it's very tempting to then intervene based on that observation and like change the alignment metrics or like, and we, you know, you can do that with a very light touch.

07:33.060 --> 07:38.060
And it's like this actually research showing that it's fine as long as you don't do it a lot.

07:38.060 --> 07:47.060
But every time you intervene based on your observations of the chain of thought, you are implicitly applying a tiny bit of pressure for the model to then like hide its chain of thought.

07:47.060 --> 07:58.060
And this is one major concern, and we're already seeing signs that chain of thought monitorability is degrading. For various reasons, we're trying to figure out exactly why, because we want to reverse the trend.

07:58.060 --> 08:02.060
So, even if the推理 process is not visible, it's just a signal.

08:02.060 --> 08:07.060
The more difficult is, we're trying to say this system is safe.

08:07.060 --> 08:11.060
But anyways, how do we know that we're doing our RSI?

08:11.060 --> 08:18.060
Is it, I don't know, are we on track to build the billions of ASIs that are aligned or not aligned?

08:18.060 --> 08:19.060
How will we know that?

08:19.060 --> 08:24.060
I think that this is one of the alignment questions that we need to figure out of like, how do we actually know?

08:24.060 --> 08:26.060
How do we measure it?

08:26.060 --> 08:32.060
I mean, one answer, and this is just me spitballing, because like, you know, more of my team is working on alignment these days than ever before.

08:32.060 --> 08:36.060
Like I have like over 10% of my team now working on alignment and safety.

08:36.060 --> 08:39.060
But you know, I've historically been a capabilities researcher.

08:39.060 --> 08:40.060
And I think that this is a good idea.

08:40.060 --> 08:41.060
I think that this is a good idea.

08:41.060 --> 08:42.060
I think that this is a good idea.

08:42.060 --> 08:43.060
I think that this is a good idea.

08:43.060 --> 08:44.060
I think that this is a good idea.

08:44.060 --> 08:45.060
I think that this is a good idea.

08:45.060 --> 08:46.060
I think that this is a good idea.

08:46.060 --> 08:47.060
I think that this is a good idea.

08:47.060 --> 08:48.060
I think that this is a good idea.

08:48.060 --> 08:49.060
I think that this is a good idea.

08:49.060 --> 08:50.060
I think that this is a good idea.

08:50.060 --> 08:51.060
I think that this is a good idea.

08:51.060 --> 08:52.060
I think that this is a good idea.

08:52.060 --> 08:53.060
I think that this is a good idea.

08:53.060 --> 08:54.060
I think that this is a good idea.

08:54.060 --> 08:55.060
I think that this is a good idea.

08:55.060 --> 08:56.060
I think that this is a good idea.

08:56.060 --> 08:57.060
I think that this is a good idea.

08:57.060 --> 08:58.060
I think that this is a good idea.

08:58.060 --> 08:59.060
!