When sharing this paper, Google's Subham Sahu wrote that for an agent to achieve recursive self-improvement, it must learn to dream. History is the event of its dreaming.
Well, this sounds like science fiction, but what this paper actually does is more practical than the term dreaming might imply.
为什么人们想想想?
因为在夜中的一天已经过了
但在夜中,脑子转弄和维持这些感觉
我们不需要在一起的一天
但我们可能会发现新的关系
或是改变 how我们做的事情
提供了一种相似的想法
中文字幕提供
What if, instead of running four processes at once, I ran eight?
If there has been no improvement for several consecutive steps, should I have stopped sooner?
It doesn't regenerate the code or rerun all the experiments.
Many results have already happened and are hung on the history of exploration.
What the AI does is try different exploration methods over and over within that past world.
Then, it brings the better strategies found in the dream back to reality.
在前面我们问了什么是正常的性改良的性改良的性改良的性改良
在这个节目中有一个真实的性改良的性改良
我们可以实现实在一起
好,我们把这个想法放在一旦的想法
看看什么是在一旦的日子
很多自动设计的设计已经看得到自己的性改良
像一个渠道的渠道经统制定的性改良
一个渠道经济的渠道经济的性改良
然后把整体的性改良的性改良
to continue iterating.
But there are actually two completely different jobs mixed up here.
Imagine a research team with two people.
One performs the experiments, writes the code, runs tests, and gets results,
while the other doesn't necessarily get their hands dirty.
They are responsible for deciding what to do next,
such as which path is worth continuing, which should be stopped,
and where resources should be invested.
Systems like AlphaEvo make it very easy to get the feeling that as the program gets stronger generation by generation, it seems like the AI is already improving itself.
But if we zoom out a little, we see a problem.
While the underlying code can change, and which results to keep can change, the second type of work, how to explore the next steps, is often still pre-designed.
在某些程中的解釋,
像是什么解釋的解釋,
在什么解釋,
在什么解釋的解釋,
在什么解釋的解釋,
和什么解釋的解釋。
这些解釋的解釋。
这些选择的解釋只需要一件很简单的问题,
像,在哪一種旅行的旅行?
我們要開始新的旅行?
我們要開始新的旅行?
我們要開始旅行?
我們要停止嗎?
我們要看數字,
最重要的數字是解釋整個整個文竅
你可以想像這個森林中的中間
這個森林在中間的天空間
每個旅行的核心是一位它其實在這裡面
而是一位它們已經訪問
而是已經知道的
所以問是,
如果我去看二新措施的新措施,
我可以把 AI的手法通過一次從初開始?
以前,那是一種方法,
因為你只能知道如果新措施是好的
其實再次再次回應
但JRSI的答案是…
不必
因為它已經在一旦走過它
而且在一旦的核能被消失
新措施可以完全走過這個核能先
它可能會往左邊,它可能會往右邊,
或它可能會只要三步步,
只要他在旁邊的旁邊
有所遭遇的確實在
它可以直接向前提出的確實效果
這關於是記憶體的記憶體
而是把記憶體從一個記憶體
到一個記憶體的記憶體
現在,有一個很重要的理解
Dream RSI does not train a new world model first and then have it approximate reality.
For Dream RSI, the history that has already played out serves as the playback simulator itself.
Thus, its dreams are quite conservative.
It isn't predicting events that haven't happened, but re-entering events that truly took place.
Strictly speaking, this world has a clear boundary.
It can only dream of places it has actually experienced.
它不可以知道的結果的失敗的棘子從浦淨空中的深空
這就是其他優點和其他優點
優點是真正的實行行動作已經有所謂
不需要賠優點的費用上網搜尋
支持一個大量的廣告
的廣告發展規模式
如果第一回合的實行行動作出現
沒有高質的路程度在歷史上
寫的絕對未來的絕對不合作出現
所以,它真的连避免是这个
reality provides一部分的世界
dreams in this world
find better exploration strategies
and new strategies
return to reality
to traverse new nodes
which then expand
the next round of dreams
this is the evolving world
mentioned in the paper's title
the world isn't built once
and left unchanged
it expands continuously
alongside real exploration
and the exploration strategy
它是更多趣的
比例的一句詳細的詳細的詳細
数字幕提供了一篇文章的文章
其实试试了一个简单的简单
我们已经知道了哪些选择了
为什么不让我们来说这些文章
和试试这些文章的文章
进行的文章
However, the results weren't good.
Let's look at Figure 5.
After adding the guidance, performance actually worsened, regardless of whether RSI was originally used.
This is worth pondering, because remembering the past and utilizing the past are not the same thing.
Compressing history into a single best thing to do is equivalent to imposing a predetermined shared direction on all subsequent exploration.
It could be helpful, but it might also constrain the search space too early, excluding many unconventional paths that should have been preserved.
Re is different. It doesn't tell the AI up front which experience is most important.
其实过程的措断过程
导致了解试
在游戏研究,游戏游戏,游戏游戏
Cernal Engineering
有很多数字
但我认为只有几个人
是值得记得
第一次是 Cost
在 Lowell的游戏探索
一极最有趣的极章
是一个极限的极限
在162次
但是,这极章不能被取得到
在适量
DreamRSI使用 Gemini 1.5 Flash
而 SimpleTex uses GPT-4.0
VGG16 and LayerNorm show a better sense of when enough is enough,
essentially achieving similar performance with fewer generations.
Meanwhile, Conv and CX show a better sense of when not to stop,
meaning they push performance further within a similar budget.
This leads to what I consider the most elegant behavioral chart in the entire paper.
Let's look at this chart together.
The strategy learned by Dream RSI isn't a simple rule like always do less or always branch out more.
When performance improvements are steady early on, it reduces the number of attempts.
But when performance hits a plateau, it expands its exploration again.
In other words, it has learned a dynamic exploration rhythm.
Don't waste your budget casting a wide net when things are going well,
but don't get stuck on the same path when you hit a wall.
This is more important than the 162 token count because it shows that SI ultimately changes
not just how much compute a single task uses, but how compute is allocated for the next exploration.
This is why the authors place it within the context of recursive self-improvement.
However, we must clearly define the boundaries.
The underlying coding agent remains unchanged, the model remains the same,
the evaluator is untouched, and the task objectives are fixed.
这个RSI只改了一种适量的方法
为继续追求更好的效果
所以,如果我们承设这个
作为Google的AI开始重新的脑子
这会是完全的诈论
如果我们只是看它为一个费用的适量
这会设定的这些研究竟实在做什么
而是这些设计的 rules
现在成为了更加努力
在此,有人在X上已经问了
最重要的问题
Would you call this true RSI?
This is also the question I wanted to ask most after reading this paper.
Since we spent our entire last episode trying to answer what counts as true recursive self-improvement,
we can bring back the L1 to L5 chart from last time and use it as a yardstick to measure this RSI.
One of the most important viewpoints from that paper was that performance improvement and recursive self-improvement are not the same thing.
A system can perform tasks increasingly well, but if the method of self-improvement is hard-coded from the outside, it is still one step away from true recursion.
Taking AlphaDev as an example, its most intuitive ability is to continuously search for better programs and algorithms.
The main object of improvement is the result itself.
You get better at finding better answers, while Darwin-inspired agents take a step further.
它不只是改變 code
它讓它們的效果
它們的效果是改變的
所以它們的效果是沒有其他人的答案
但是它們的答案
但是在那些人的答案
從前來的
法案上的解決策定
而是哪些人的解決策定
而哪些人的解決策定
是哪些人的解決策定
還不算是不一樣的
自動的解決策定
在其中一個人的解決策定
但它們的解決策定
where the next generation of agents should come from
have not yet entered the recursion.
DAMRSI happens to tread exactly on this boundary.
What DAM leaves outside the self-modification loop
is precisely the mechanism
of candidate solution maintenance and selection,
essentially where the next generation continues from.
What JRSI does is precisely turn this layer,
which nodes continue to branch,
how many branches to create, and when to stop,
into an object that can be replaced and improved.
每周一周的一周可以繼續探索和新增長新增長。
The real difference is that Dream RSI lets the new strategy learned in the previous round
decide where the tree grows next and how resources are allocated.
And the new exploration history feeds back into the replay simulator, which in turn modifies
the next version of the strategy.
So if we look at it through the L1-L5 framework from that paper last time, Dream RSI already
exhibits a very clear L5 characteristic.
The object being modified now includes the method used to generate the next round of improvements itself.
But I must add a very important caveat.
It is an extremely narrow L5.
It does not autonomously modify the underlying model, change the evaluator,
redefine the task, or decide which capability I should improve next.
It performs recursive improvement on the exploration mechanism within the boundaries of fixed goals,
fixed evaluation rules, and a fixed underlying agent.
在前面的一段影片,我们其实分析了两种选择的选择
第一个是非常简单
Baseline
真正的問題是否有效果的效果是否有效果的效果。
所以, DreamRSI's L5有兩種模型。
它是很短的,但在於它的確實證據,它並不夠不夠。
A more accurate statement is not that true general RSI has arrived, but rather that we
have finally seen a very clean, local example where the improvement mechanism itself has
entered an improvement loop.
This also makes its relationship with AV training particularly interesting.
Last time, we discussed how AV training precipitates the results of post-training experiments into
a set of persistent research strategies and exploration logs, which are then modified
by the meta-agent to guide the subsequent executive agents.
Doesn't that sound already very close to JRSI?
Neither is satisfied with just modifying the final model or the final answer.
They have begun to make the research process itself
something that can be accumulated, modified, and inherited.
However, the way both utilize history is not the same.
AVIL training is more like letting a meta-agent read past evidence
to directly diagnose problems and then modify the policy.
但它加了一段階段
不停地寫一项新的方法
根据一项新的方法
它寫了几个选择选择选择
它们进入了一个草莓的
实际的方法
使用了解决定
它们是更多的模式
这些是更多的模式
在一个模式的模式
而是更多的模式
新的模式进入了一个棺材
这带来我们回到的模式
这些人其实特别的模式
所以这些设计可以再次重新的一种程度
在这一步,
Memory会不会只能够认为了一种程度的一种程度
但会开始开始成为一种程度的一种程度
可以再次重新的程度
这一种程度是一种程度的问题
为什么我们继续叫这个DREAM?
我们先前提到,DREAM-RSI
不像是一种程度的一种程度
但它拥有同一种意识度
实际上是太贵的
所以可以把遍遍遍遍了一環境的環境
可以用了一環境的環境
JRSI的思考方法是一種的
但它是不同的
它不斷勉強化了一整個世界的世界
但卻不斷勉強化了一整個世界的世界
和一整個世界的世界
所以我認為這真是真是真是
世界的世界是在這裡的世界上
改變了一種世界
過去我們曾經比我們更多多的世界
但是JRSI开始在想要如何我自己的改变
在前面的改变
和什么会发生的一种选择
当然,今天JRSI仍然很远
是一种超级的超级性能改变
它的模式 hasn't changed
它的目标 haven't changed
和它的规律不改变
甚至是一种想法它们可以持续
是一种未来的经历它们终于实在
但…
Something that used to be set in stone has started to shift.
It's not about what the next answer should be,
but rather how I should find a way to improve myself next time.
When the answer to this question also starts to be modified and verified by the machine itself
to be used in the next round,
the most noteworthy part of recursive self-improvement may have just begun.
Alright, that's all for today's content.
Please like, share, and subscribe to my channel.
如果你想支持我更加,请加入我的连接
售价值一杯咖啡每月不仅仅支持我
但也帮我继续創更多的内容关系与 AI和技术
好的,谢谢观看,我会见到你在下一集
Yay