start	end	text
0	11580	When sharing this paper, Google's Subham Sahu wrote that for an agent to achieve recursive self-improvement, it must learn to dream. History is the event of its dreaming.
12240	20280	Well, this sounds like science fiction, but what this paper actually does is more practical than the term dreaming might imply.
20280	22160	为什么人们想想想?
22800	25800	因为在夜中的一天已经过了
25800	30360	但在夜中,脑子转弄和维持这些感觉
30360	33720	我们不需要在一起的一天
33720	35840	但我们可能会发现新的关系
35840	38940	或是改变 how我们做的事情
43040	45240	提供了一种相似的想法
45240	75220	中文字幕提供
75240	79160	What if, instead of running four processes at once, I ran eight?
79160	83920	If there has been no improvement for several consecutive steps, should I have stopped sooner?
83920	88440	It doesn't regenerate the code or rerun all the experiments.
88440	93320	Many results have already happened and are hung on the history of exploration.
93320	99560	What the AI does is try different exploration methods over and over within that past world.
99560	104200	Then, it brings the better strategies found in the dream back to reality.
134200	140200	在前面我们问了什么是正常的性改良的性改良的性改良的性改良
140200	143200	在这个节目中有一个真实的性改良的性改良
143200	144880	我们可以实现实在一起
144880	147520	好,我们把这个想法放在一旦的想法
147520	149520	看看什么是在一旦的日子
149520	153920	很多自动设计的设计已经看得到自己的性改良
153920	157880	像一个渠道的渠道经统制定的性改良
157880	160320	一个渠道经济的渠道经济的性改良
160320	162560	然后把整体的性改良的性改良
162560	163760	to continue iterating.
163940	167920	But there are actually two completely different jobs mixed up here.
168440	170960	Imagine a research team with two people.
171440	176460	One performs the experiments, writes the code, runs tests, and gets results,
176980	179760	while the other doesn't necessarily get their hands dirty.
180220	182900	They are responsible for deciding what to do next,
183320	186720	such as which path is worth continuing, which should be stopped,
187060	188880	and where resources should be invested.
188880	199020	Systems like AlphaEvo make it very easy to get the feeling that as the program gets stronger generation by generation, it seems like the AI is already improving itself.
199200	201740	But if we zoom out a little, we see a problem.
202340	211760	While the underlying code can change, and which results to keep can change, the second type of work, how to explore the next steps, is often still pre-designed.
241760	242760	在某些程中的解釋,
242760	244760	像是什么解釋的解釋,
244760	245760	在什么解釋,
245760	246760	在什么解釋的解釋,
246760	247760	在什么解釋的解釋,
247760	249760	和什么解釋的解釋。
249760	251760	这些解釋的解釋。
251760	255760	这些选择的解釋只需要一件很简单的问题,
255760	258760	像,在哪一種旅行的旅行?
258760	260760	我們要開始新的旅行?
260760	261760	我們要開始新的旅行?
261760	263760	我們要開始旅行?
263760	264760	我們要停止嗎?
264760	266760	我們要看數字,
266760	269760	最重要的數字是解釋整個整個文竅
269760	271760	你可以想像這個森林中的中間
271760	273760	這個森林在中間的天空間
273760	277760	每個旅行的核心是一位它其實在這裡面
277760	278760	而是一位它們已經訪問
278760	280760	而是已經知道的
280760	281760	所以問是,
281760	285920	如果我去看二新措施的新措施,
285920	289920	我可以把 AI的手法通過一次從初開始?
289920	293000	以前,那是一種方法,
293000	296120	因為你只能知道如果新措施是好的
296120	297960	其實再次再次回應
297960	300160	但JRSI的答案是…
300160	301520	不必
301520	303920	因為它已經在一旦走過它
303920	306560	而且在一旦的核能被消失
306560	310920	新措施可以完全走過這個核能先
310920	314420	它可能會往左邊,它可能會往右邊,
314420	317220	或它可能會只要三步步,
317220	319320	只要他在旁邊的旁邊
319320	320760	有所遭遇的確實在
320760	324880	它可以直接向前提出的確實效果
325680	329160	這關於是記憶體的記憶體
329160	332220	而是把記憶體從一個記憶體
332220	334520	到一個記憶體的記憶體
334520	336920	現在,有一個很重要的理解
336920	343580	Dream RSI does not train a new world model first and then have it approximate reality.
343840	349500	For Dream RSI, the history that has already played out serves as the playback simulator itself.
349780	351680	Thus, its dreams are quite conservative.
351680	357580	It isn't predicting events that haven't happened, but re-entering events that truly took place.
357820	360760	Strictly speaking, this world has a clear boundary.
361280	364380	It can only dream of places it has actually experienced.
364380	368620	它不可以知道的結果的失敗的棘子從浦淨空中的深空
368620	372220	這就是其他優點和其他優點
372220	375580	優點是真正的實行行動作已經有所謂
375580	379020	不需要賠優點的費用上網搜尋
379020	380780	支持一個大量的廣告
380780	382860	的廣告發展規模式
382860	386860	如果第一回合的實行行動作出現
386860	389900	沒有高質的路程度在歷史上
389900	393220	寫的絕對未來的絕對不合作出現
393220	395980	所以,它真的连避免是这个
395980	398980	reality provides一部分的世界
398980	400640	dreams in this world
400640	402740	find better exploration strategies
402740	404220	and new strategies
404220	405540	return to reality
405540	407040	to traverse new nodes
407040	408440	which then expand
408440	409780	the next round of dreams
409780	411860	this is the evolving world
411860	413380	mentioned in the paper's title
413380	415420	the world isn't built once
415420	416600	and left unchanged
416600	418460	it expands continuously
418460	420080	alongside real exploration
420080	421960	and the exploration strategy
421960	424960	它是更多趣的
424960	427960	比例的一句詳細的詳細的詳細
427960	429960	数字幕提供了一篇文章的文章
429960	433960	其实试试了一个简单的简单
433960	436960	我们已经知道了哪些选择了
436960	439960	为什么不让我们来说这些文章
439960	441960	和试试这些文章的文章
441960	443901	进行的文章
443901	447101	However, the results weren't good.
447101	449101	Let's look at Figure 5.
449101	455501	After adding the guidance, performance actually worsened, regardless of whether RSI was originally used.
455501	461981	This is worth pondering, because remembering the past and utilizing the past are not the same thing.
462141	470301	Compressing history into a single best thing to do is equivalent to imposing a predetermined shared direction on all subsequent exploration.
470561	478781	It could be helpful, but it might also constrain the search space too early, excluding many unconventional paths that should have been preserved.
478781	483941	Re is different. It doesn't tell the AI up front which experience is most important.
513941	516221	其实过程的措断过程
516221	517381	导致了解试
517381	519821	在游戏研究,游戏游戏,游戏游戏
519821	520941	Cernal Engineering
520941	522941	有很多数字
522941	524781	但我认为只有几个人
524781	525741	是值得记得
525741	527541	第一次是 Cost
527541	530061	在 Lowell的游戏探索
530061	532381	一极最有趣的极章
532381	533901	是一个极限的极限
533901	536061	在162次
536061	538021	但是,这极章不能被取得到
538021	538781	在适量
538781	541741	DreamRSI使用 Gemini 1.5 Flash
541741	544281	而 SimpleTex uses GPT-4.0
574281	580081	VGG16 and LayerNorm show a better sense of when enough is enough,
580081	584081	essentially achieving similar performance with fewer generations.
584081	589881	Meanwhile, Conv and CX show a better sense of when not to stop,
589881	593481	meaning they push performance further within a similar budget.
593481	598881	This leads to what I consider the most elegant behavioral chart in the entire paper.
598881	600681	Let's look at this chart together.
600681	609341	The strategy learned by Dream RSI isn't a simple rule like always do less or always branch out more.
609641	614261	When performance improvements are steady early on, it reduces the number of attempts.
614741	618721	But when performance hits a plateau, it expands its exploration again.
619381	622981	In other words, it has learned a dynamic exploration rhythm.
623541	627021	Don't waste your budget casting a wide net when things are going well,
627021	630241	but don't get stuck on the same path when you hit a wall.
630241	636641	This is more important than the 162 token count because it shows that SI ultimately changes
636641	643121	not just how much compute a single task uses, but how compute is allocated for the next exploration.
643121	648321	This is why the authors place it within the context of recursive self-improvement.
648321	651281	However, we must clearly define the boundaries.
651281	655601	The underlying coding agent remains unchanged, the model remains the same,
655601	659281	the evaluator is untouched, and the task objectives are fixed.
659281	662281	这个RSI只改了一种适量的方法
662281	664561	为继续追求更好的效果
664561	666561	所以,如果我们承设这个
666561	669761	作为Google的AI开始重新的脑子
669761	672201	这会是完全的诈论
672201	675961	如果我们只是看它为一个费用的适量
675961	679321	这会设定的这些研究竟实在做什么
679321	680841	而是这些设计的 rules
680841	683721	现在成为了更加努力
683721	686681	在此,有人在X上已经问了
686681	688161	最重要的问题
688161	690661	Would you call this true RSI?
690661	694901	This is also the question I wanted to ask most after reading this paper.
694901	701881	Since we spent our entire last episode trying to answer what counts as true recursive self-improvement,
702301	708641	we can bring back the L1 to L5 chart from last time and use it as a yardstick to measure this RSI.
709101	716981	One of the most important viewpoints from that paper was that performance improvement and recursive self-improvement are not the same thing.
716981	727281	A system can perform tasks increasingly well, but if the method of self-improvement is hard-coded from the outside, it is still one step away from true recursion.
727281	735121	Taking AlphaDev as an example, its most intuitive ability is to continuously search for better programs and algorithms.
735121	738541	The main object of improvement is the result itself.
738541	744121	You get better at finding better answers, while Darwin-inspired agents take a step further.
744121	745661	它不只是改變 code
745661	747321	它讓它們的效果
747321	749361	它們的效果是改變的
749361	752421	所以它們的效果是沒有其他人的答案
752421	754461	但是它們的答案
754461	756461	但是在那些人的答案
756461	757621	從前來的
757621	758961	法案上的解決策定
758961	760961	而是哪些人的解決策定
760961	762461	而哪些人的解決策定
762461	764361	是哪些人的解決策定
764361	766121	還不算是不一樣的
766121	767461	自動的解決策定
767461	769461	在其中一個人的解決策定
769461	771961	但它們的解決策定
771961	774621	where the next generation of agents should come from
774621	776521	have not yet entered the recursion.
776521	780161	DAMRSI happens to tread exactly on this boundary.
780161	783221	What DAM leaves outside the self-modification loop
783221	784681	is precisely the mechanism
784681	787341	of candidate solution maintenance and selection,
787341	790181	essentially where the next generation continues from.
790181	793301	What JRSI does is precisely turn this layer,
793301	794841	which nodes continue to branch,
794841	797341	how many branches to create, and when to stop,
797341	799801	into an object that can be replaced and improved.
829801	834081	每周一周的一周可以繼續探索和新增長新增長。
834081	839701	The real difference is that Dream RSI lets the new strategy learned in the previous round
839701	844141	decide where the tree grows next and how resources are allocated.
844141	849841	And the new exploration history feeds back into the replay simulator, which in turn modifies
849841	851841	the next version of the strategy.
851841	857581	So if we look at it through the L1-L5 framework from that paper last time, Dream RSI already
857581	860661	exhibits a very clear L5 characteristic.
860661	865961	The object being modified now includes the method used to generate the next round of improvements itself.
865961	868661	But I must add a very important caveat.
868661	871161	It is an extremely narrow L5.
871161	875961	It does not autonomously modify the underlying model, change the evaluator,
875961	880161	redefine the task, or decide which capability I should improve next.
880161	885161	It performs recursive improvement on the exploration mechanism within the boundaries of fixed goals,
885161	888161	fixed evaluation rules, and a fixed underlying agent.
888161	891761	在前面的一段影片,我们其实分析了两种选择的选择
891761	893201	第一个是非常简单
923201	923901	Baseline
923901	928901	真正的問題是否有效果的效果是否有效果的效果。
928901	932901	所以, DreamRSI's L5有兩種模型。
932901	938901	它是很短的,但在於它的確實證據,它並不夠不夠。
938901	944781	A more accurate statement is not that true general RSI has arrived, but rather that we
944781	950541	have finally seen a very clean, local example where the improvement mechanism itself has
950541	952161	entered an improvement loop.
952161	956801	This also makes its relationship with AV training particularly interesting.
956801	962161	Last time, we discussed how AV training precipitates the results of post-training experiments into
962161	966941	a set of persistent research strategies and exploration logs, which are then modified
966941	970561	by the meta-agent to guide the subsequent executive agents.
970921	974261	Doesn't that sound already very close to JRSI?
974521	978321	Neither is satisfied with just modifying the final model or the final answer.
978761	981241	They have begun to make the research process itself
981241	984421	something that can be accumulated, modified, and inherited.
984741	988361	However, the way both utilize history is not the same.
988581	992501	AVIL training is more like letting a meta-agent read past evidence
992501	995901	to directly diagnose problems and then modify the policy.
995901	997901	但它加了一段階段
997901	999901	不停地寫一项新的方法
999901	1001901	根据一项新的方法
1001901	1003901	它寫了几个选择选择选择
1003901	1005901	它们进入了一个草莓的
1005901	1007901	实际的方法
1007901	1009901	使用了解决定
1009901	1011901	它们是更多的模式
1011901	1013901	这些是更多的模式
1013901	1015901	在一个模式的模式
1015901	1017901	而是更多的模式
1017901	1019901	新的模式进入了一个棺材
1019901	1021901	这带来我们回到的模式
1021901	1023901	这些人其实特别的模式
1053901	1056621	所以这些设计可以再次重新的一种程度
1056621	1057561	在这一步,
1057941	1061821	Memory会不会只能够认为了一种程度的一种程度
1061821	1064201	但会开始开始成为一种程度的一种程度
1064201	1065461	可以再次重新的程度
1065461	1068761	这一种程度是一种程度的问题
1068761	1072421	为什么我们继续叫这个DREAM?
1072421	1074961	我们先前提到,DREAM-RSI
1074961	1077861	不像是一种程度的一种程度
1077861	1080061	但它拥有同一种意识度
1080061	1083021	实际上是太贵的
1083021	1087061	所以可以把遍遍遍遍了一環境的環境
1087061	1088181	可以用了一環境的環境
1088181	1091261	JRSI的思考方法是一種的
1091261	1093461	但它是不同的
1093461	1096541	它不斷勉強化了一整個世界的世界
1096541	1099981	但卻不斷勉強化了一整個世界的世界
1099981	1101981	和一整個世界的世界
1101981	1105021	所以我認為這真是真是真是
1105021	1106581	世界的世界是在這裡的世界上
1106581	1108461	改變了一種世界
1108461	1111021	過去我們曾經比我們更多多的世界
1111021	1116101	但是JRSI开始在想要如何我自己的改变
1116101	1117021	在前面的改变
1117021	1119741	和什么会发生的一种选择
1119741	1122941	当然,今天JRSI仍然很远
1122941	1126141	是一种超级的超级性能改变
1126141	1127821	它的模式 hasn't changed
1127821	1129341	它的目标 haven't changed
1129341	1131821	和它的规律不改变
1131821	1134421	甚至是一种想法它们可以持续
1134421	1137181	是一种未来的经历它们终于实在
1137181	1138741	但…
1138741	1142361	Something that used to be set in stone has started to shift.
1143021	1145241	It's not about what the next answer should be,
1145761	1149261	but rather how I should find a way to improve myself next time.
1149421	1154581	When the answer to this question also starts to be modified and verified by the machine itself
1154581	1155881	to be used in the next round,
1156361	1160701	the most noteworthy part of recursive self-improvement may have just begun.
1160901	1162861	Alright, that's all for today's content.
1163241	1166141	Please like, share, and subscribe to my channel.
1166141	1169441	如果你想支持我更加,请加入我的连接
1169441	1172341	售价值一杯咖啡每月不仅仅支持我
1172341	1176141	但也帮我继续創更多的内容关系与 AI和技术
1176141	1178381	好的,谢谢观看,我会见到你在下一集
1178381	1179021	Yay