{ "text": "When sharing this paper, Google's Subham Sahu wrote that for an agent to achieve recursive self-improvement, it must learn to dream.\nHistory is the event of its dreaming.\nWell, this sounds like science fiction, but what this paper actually does is more practical than the term dreaming might imply.\nWhy do humans dream?\nBecause what happened during the day is already over,\nbut at night, the brain replays and recombines those experiences.\nWe don't need to actually relive the same day,\nbut we might discover new connections in these replays,\nor even change how we make decisions the next day.\nNow, a research team from Google and Google DeepMind has brought a very similar idea into AI.\nThe paper is called Dream RSI, Recursive Self-Improvement Based on Evolutionary Worlds.\nThe system it proposes explores a real environment during the day, writing actual code, compiling, and running tests.\nBut in the next round, it doesn't repeat all those expensive processes.\nInstead, it turns past explorations into a dream world that can be revisited.\nIn this world, it can ask, what if I hadn't kept digging down this path, but had changed direction earlier?\nWhat if, instead of running four processes at once, I ran eight?\nIf there has been no improvement for several consecutive steps, should I have stopped sooner?\nIt doesn't regenerate the code or rerun all the experiments.\nMany results have already happened and are embedded in the history of exploration.\nWhat the AI does is try different exploration methods over and over within that past world.\nThen it brings the better strategies found in the dream back to reality.\nNew strategies produce new exploration history.\nNew history becomes new dreams, and new dreams modify the exploration method for the next round.\nThis is why the authors call it RSI.\nBut the really interesting question here isn't the dreaming metaphor itself, but what actually changes after it wakes up?\nAnd this is exactly where Dream RSI sets itself apart from AlphaCode, Godel Machines, and AI Scientist.\nIn the last episode, we asked what counts as true recursive self-improvement,\nand in this episode, a real system has emerged that we can actually examine.\nAll right, let's put the dreams aside for a moment and look at what it's doing during the day.\nMany automated discovery systems already look like they are self-evolving,\nLike a coding agent that keeps generating candidate programs,\nan evaluator that scores them,\nand then the system keeping the better results to continue iterating.\nBut there are actually two completely different jobs mixed up here.\nImagine a research team with two people.\nOne performs the experiments, writes the code, runs tests, and gets results,\nwhile the other doesn't necessarily get their hands dirty.\nThey are responsible for deciding what to do next,\nsuch as which path is worth continuing, which should be stopped, and where resources should\nbe invested. Systems like AlphaEvo make it very easy to get the feeling that as the program gets\nstronger generation by generation, it seems like the AI is already improving itself. But if we zoom\nout a little, we see a problem. While the underlying code can change and which results to keep can\nchange, the second type of work, how to explore the next steps, is often still pre-designed. For\nFor example, from which node to continue, how many branches to open simultaneously,\nand after how many attempts to abandon a direction.\nThese rules usually don't evolve along with the system.\nWhat about Dream RSI?\nIt specifically targets the person responsible for deciding what to do next.\nDream RSI separates this layer, and its real-world exploration continuously grows a discovery\ntree.\nEvery node on the tree corresponds to a real attempt that has taken place, such as what\nsolution the model generated in what context, whether the code could run, and what results\nthe evaluator gave.\nAll of this has been recorded.\nThe outer exploration strategy only needs to answer a few very simple questions, like\nfrom which node should we continue now?\nShould we start a new path?\nHow many directions should we pursue in parallel?\nWhen should we stop?\nLet's look at figure 2, which is the most important figure for understanding the entire paper.\nYou can think of this tree in the middle as the maze that the AI actually walked through\nyesterday.\nEvery node that is lit up is a place it actually visited, and the results are already known.\nSo the question is, if I want to compare two new exploration strategies today, must I make\nthe AI walk through the maze twice from the beginning?\nIn the past, that was basically the only way.\nBecause you could only know if a new strategy was good by actually running it again.\nBut Dream RSI's answer is, not necessarily.\nSince it has already walked through it yesterday, and what happened at each node has been saved,\na new strategy can completely walk through this historical tree first.\nIt might go left first, it might turn right sooner, or it might just take three steps\nand stop.\nAs long as he visits nodes that have truly occurred, the simulator can directly reveal\nthe actual results from that time.\nThe key shift here isn't just remembering history, but rather turning history from a\nlog into a re-enterable event.\nNow, there is a common misunderstanding here.\nDream RSI does not train a new world model first and then have it approximate reality.\nFor Dream RSI, the history that has already played out serves as the playback simulator\nitself.\nThus, its dreams are quite conservative.\nIt isn't predicting events that haven't happened, but re-entering events that truly took place.\nStrictly speaking, this world has a clear boundary.\nIt can only dream of places it has actually experienced.\nIt cannot know the results of unexplored branches out of thin air.\nThis is precisely both its advantage and its limitation.\nThe advantage is that real executions that have already occurred don't require paying\nfor expensive online exploration again, supporting a large volume of rapid offline strategy comparisons.\nThe limitation is that if the first round of real exploration is poor, with almost no high-quality\npaths in the history tree, the subsequent dreams won't conjure up a good event out of nothing.\nSo the way it truly loops is this. Reality provides a portion of the world. Dreams in this\nworld find better exploration strategies, and new strategies return to reality to traverse\nnew nodes, which then expand the next round of dreams.\nThis is the evolving world mentioned in the paper's title.\nThe world isn't built once and left unchanged.\nIt expands continuously alongside real exploration, and the exploration strategy evolves with\nthe world.\nThis is much more interesting than simply summarizing historical experience into a few sentences.\nThe authors actually tested another, seemingly simpler method.\nSince we already know which directions succeeded or failed, why not let a model summarize the\nlessons and inject these suggestions into the next round of prompts?\nHowever, the results weren't good.\nLet's look at Figure 5.\nAfter adding the guidance, performance actually worsened, regardless of whether RSI was originally used.\nThis is worth pondering, because remembering the past and utilizing the past are not the same thing.\nCompressing history into a single best thing to do is equivalent to imposing a predetermined shared direction on all subsequent exploration.\nIt could be helpful, but it might also constrain the search space too early, excluding many unconventional paths that should have been preserved.\nDream RSI is different. It doesn't tell the AI up front which experience is most important.\nIt simply preserves that world, allowing different strategies to explore it for themselves.\nI think the most interesting takeaway from this paper isn't that AI can learn from history,\nbut rather that perhaps truly valuable experience isn't about summarizing the past into a single instruction,\nbut about preserving a past that can be re-experienced.\nIf Dream RSI only ends up saving a few API calls, then it's not actually that interesting.\nWhat's truly worth looking at is whether it changes the actual exploration process that follows.\nThe authors conducted experiments in algorithmic engineering, mathematical optimization, and GPU.\nKernel engineering.\nThere are a lot of numbers, but I think only a few are worth remembering.\nThe first set is cost.\nIn the Lowell algorithm discovery task, one of the most striking figures in the paper is a maximum call gap of about 162 times.\nHowever, this figure cannot be taken in isolation.\nDream RSI uses Gemini 1.5 Flash, while Simpletext uses GPT-40.\nA much cleaner comparison is the head-to-head control between Dream RSI and recursive fixed exploration using the same base model.\nHowever, I am more concerned with another question.\nIf we search so much less, will the results get worse?\nThe paper does not show this kind of simple trade-off.\nOn some tasks, it achieves comparable performance with fewer generations,\nwhile on others, it achieves even higher performance within a similar budget.\nCuril BCH is a great example to illustrate these two different types of gains.\nVGG16 and LayerNorm show a better sense of when enough is enough,\nessentially achieving similar performance with fewer generations.\nMeanwhile, Conv and CX show a better sense of when not to stop, meaning they push performance further within a similar budget.\nThis leads to what I consider the most elegant behavioral chart in the entire paper.\nLet's look at this chart together.\nThe strategy learned by Dream RSI isn't a simple rule like always do less or always branch out more.\nWhen performance improvements are steady early on, it reduces the number of attempts.\nBut when performance hits a plateau, it expands its exploration again.\nIn other words, it has learned a dynamic exploration rhythm.\nDon't waste your budget casting a wide net when things are going well, but don't get\nstuck on the same path when you hit a wall.\nThis is more important than the 162 token count because it shows that RSI ultimately changes\nnot just how much compute a single task uses, but how compute is allocated for the next\nexploration.\nThis is why the authors place it within the context of recursive self-improvement.\nHowever, we must clearly define the boundaries.\nThe underlying coding agent remains unchanged, the model remains the same, the evaluator\nis untouched, and the task objectives are fixed.\nThis RSI only modifies the outer strategy for how to continue seeking better results.\nTherefore, if we interpret this as Google's AI starting to rewrite its own brain, that\nwould be a complete exaggeration.\nConversely, if we just view it as a cost-saving scheduler, that underestimates what this paper truly achieves, as the scheduling rules themselves have now become subjects for improvement.\nIn fact, someone on X has already asked the most critical question.\nWould you call this true RSI?\nThis is also the question I wanted to ask most after reading this paper.\nSince we spent our entire last episode trying to answer what counts as true recursive self-improvement, we can bring back the L1 to L5 chart from last time and use it as a yardstick to measure this RSI.\nOne of the most important viewpoints from that paper was that performance improvement and recursive self-improvement are not the same thing.\nA system can perform tasks increasingly well, but if the method of self-improvement is hard-coded from the outside, it is still one step away from true recursion.\nTaking AlphaDev as an example, its most intuitive ability is to continuously search for better programs and algorithms.\nThe main object of improvement is the result itself.\nYou get better at finding better answers, while Darwin-inspired agents take a step further.\nIt doesn't just change code. It allows the implementation of the coding agent itself to be modified.\nSo the object of improvement is no longer just the answer, but the agent that generates the answer.\nHowever, in that RSI paper from last time, rules regarding which candidate solutions are kept and which candidate solutions the next generation evolves from were still not part of the self-modification loop.\nIn other words, the agent can change,\nbut the rules determining where the next generation of agents should come from\nhave not yet entered the recursion.\nDream RSI happens to tread exactly on this boundary.\nWhat Dream RSI leaves outside the self-modification loop\nis precisely the mechanism of candidate solution maintenance and selection,\nessentially where the next generation continues from.\nWhat Dream RSI does is precisely turn this layer,\nwhich nodes continue to branch, how many branches to create, and when to stop,\ninto an object that can be replaced and improved.\nIt didn't modify the underlying agent, but instead changed the outer layer,\nthat is, how the next round of improvement should be sought,\nhow many branches to continue opening from historical nodes, and when to stop.\nThese elements, which originally belonged to the system designer,\nhave been transformed into an exploration strategy\nthat can be evaluated, modified, and then carried over to the next round.\nThe key is that this new strategy doesn't just stay in a dream.\nIt directly governs the real exploration of the next round.\nWith a fixed strategy, each round can still continue to explore and grow new nodes.\nThe real difference is that Dream RSI lets the new strategy learned in the previous round\ndecide where the tree grows next and how resources are allocated.\nAnd the new exploration history feeds back into the replay simulator,\nwhich in turn modifies the next version of the strategy.\nSo, if we look at it through the L1-L5 framework from that paper last time, Dream RSI already\nexhibits a very clear L5 characteristic.\nThe object being modified now includes the method used to generate the next round of\nimprovements itself.\nBut I must add a very important caveat.\nIt is an extremely narrow L5.\nIt does not autonomously modify the underlying model, change the evaluator, redefine the\ntask, or decide which capability I should improve next.\nIt performs recursive improvement on the exploration mechanism within the boundaries of fixed goals,\nfixed evaluation rules, and a fixed underlying agent.\nIn the last episode, we actually distinguished between two types of recursion.\nThe first is very simple.\nIs the mechanism you modified actually used again in the next round?\nThis is called structural recursion.\nDream RSI is very clear on this point.\nThe new exploration strategy enters the next round of online exploration directly.\nBut the harder one is the second type.\ndoes using it actually make the successor system better at improving?\nThis is effective recursion.\nDream RSI provides a fairly clean contrast here.\nIt uses the same underlying model, the same evaluator,\nand the same initial strategy as the fixed exploration baseline.\nby whether the subsequent exploration strategy has been improved through dreaming. So Dream RSI's\nL5 has two dimensions. It is very narrow in scope, but in terms of the clarity of its recursive\nevidence, it is not weak. Therefore, a more accurate statement is not that true general RSI\nhas arrived, but rather that we have finally seen a very clean, local example where the improvement\nThis also makes its relationship with AV training particularly interesting.\nparticularly interesting. Last time, we discussed how AV training precipitates the results of\npost-training experiments into a set of persistent research strategies and exploration logs,\nwhich are then modified by the meta-agent to guide the subsequent executive agents.\nDoesn't that sound already very close to Dream RSI?\nNeither is satisfied with just modifying the final model or the final answer.\nThey have begun to make the research process itself something that can be accumulated,\nmodified, and inherited.\nHowever, the way both utilize history is not the same.\nAVIL training is more like letting a meta-agent read past evidence to directly diagnose problems\nand then modify the policy. But it adds an extra step. Instead of just inferring a new strategy\nbased on history, it generates multiple candidate exploration strategies, puts them into a historical\nreplay world to actually run them, and uses the results to choose which one enters reality.\nSee, this is more like modifying a plan after a post-mortem review, whereas the other is more\nlike putting multiple new plans back into a sandbox for simulation. This brings us right\nback to the figure. The authors actually specifically tried the more direct path of\nsummarizing history into semantic guidance, but the results were worse. Therefore, the most special\nthing about JRSI is not just that it preserves history, but that it lets candidate strategies\nre-experience history and then uses the outcomes of that experience to make a choice. Perhaps a\ntruly powerful self-improving system in the future will not choose just one of these, but might\nsimultaneously distill experience into high-level strategies while retaining a sufficiently rich\nset of historical events so these strategies can be retested at any time. At that point,\nmemory will no longer just be the model knowing what it has done before,\nbut will start to become a past that can be rerun. Speaking of which, there is one final question.\nWhy did the authors choose to call this dream? As we mentioned earlier, Dream RSI does not train\na world model in the traditional sense, but it borrows the same intuition. Real-world interaction\nis too expensive, so can we turn past experiences into an internal environment for continued learning?\nDream RSI borrows this line of thinking, but the object it simulates is different. It does not attempt to\nlearn a complete external world, but instead preserves how it explored this world in the past\nand what happened during each exploration. So I think the truly interesting part of this paper\nis that the world here has quietly changed its target.\nIn the past, we were more concerned\nwith how the world would change.\nBut Dream RSI starts to care about\nhow I personally sought out changes in the past\nand what would happen if I used a different way of searching.\nOf course, today's Dream RSI is still very far\nfrom being a superintelligence\nthat can improve itself indefinitely.\nIts model hasn't changed, its goals haven't changed,\nand the rules evaluating it haven't changed either.\nEven the dreams it can have are limited to the past it has truly experienced.\nBut something that used to be set in stone has started to shift.\nIt's not about what the next answer should be,\nbut rather how I should find a way to improve myself next time.\nWhen the answer to this question also starts to be modified and verified by the machine itself\nto be used in the next round,\nthe most noteworthy part of recursive self-improvement may have just begun.\nAlright, that's all for today's content. Please like, share, and subscribe to my channel.\nIf you'd like to support me further, feel free to join my membership. The cost of a cup of coffee\neach month not only supports me, but also helps me continue creating more content related to AI\nand technology. Alright, thanks for watching, and I'll see you in the next one. Yay!", "language": "multilingual", "duration": 1179.5505, "segments": [ { "start": 0.0, "end": 9.12, "text": "When sharing this paper, Google's Subham Sahu wrote that for an agent to achieve recursive self-improvement, it must learn to dream.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 9.64, "end": 11.58, "text": "History is the event of its dreaming.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 12.22, "end": 20.28, "text": "Well, this sounds like science fiction, but what this paper actually does is more practical than the term dreaming might imply.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 20.8, "end": 22.18, "text": "Why do humans dream?", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 22.8, "end": 25.8, "text": "Because what happened during the day is already over,", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 26.22, "end": 30.32, "text": "but at night, the brain replays and recombines those experiences.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 30.94, "end": 33.74, "text": "We don't need to actually relive the same day,", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 34.14, "end": 36.84, "text": "but we might discover new connections in these replays,", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 37.08, "end": 39.56, "text": "or even change how we make decisions the next day.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 39.56, "end": 45.88, "text": "Now, a research team from Google and Google DeepMind has brought a very similar idea into AI.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 46.24, "end": 51.76, "text": "The paper is called Dream RSI, Recursive Self-Improvement Based on Evolutionary Worlds.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 51.76, "end": 59.02, "text": "The system it proposes explores a real environment during the day, writing actual code, compiling, and running tests.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 59.46, "end": 62.86, "text": "But in the next round, it doesn't repeat all those expensive processes.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 63.2, "end": 68.1, "text": "Instead, it turns past explorations into a dream world that can be revisited.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 68.3, "end": 74.96, "text": "In this world, it can ask, what if I hadn't kept digging down this path, but had changed direction earlier?", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 75.16, "end": 79.16, "text": "What if, instead of running four processes at once, I ran eight?", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 79.16, "end": 83.74, "text": "If there has been no improvement for several consecutive steps, should I have stopped sooner?", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 83.92, "end": 87.84, "text": "It doesn't regenerate the code or rerun all the experiments.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 88.56, "end": 92.52, "text": "Many results have already happened and are embedded in the history of exploration.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0, "text_original": "Many results have already happened and are hung on the history of exploration.", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 93.32, "end": 98.92, "text": "What the AI does is try different exploration methods over and over within that past world.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 99.62, "end": 103.96, "text": "Then it brings the better strategies found in the dream back to reality.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 104.28, "end": 106.68, "text": "New strategies produce new exploration history.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 106.68, "end": 112.18, "text": "New history becomes new dreams, and new dreams modify the exploration method for the next round.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 112.46, "end": 114.94, "text": "This is why the authors call it RSI.", "language": "en", "chunk": 0, "local_language_run": 0, "language_run": 0 }, { "start": 117.203, "end": 124.733, "text": "But the really interesting question here isn't the dreaming metaphor itself, but what actually changes after it wakes up?", "language": "en", "chunk": 0, "local_language_run": 1, "language_run": 1 }, { "start": 127.026, "end": 134.756, "text": "And this is exactly where Dream RSI sets itself apart from AlphaCode, Godel Machines, and AI Scientist.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 135.196, "end": 139.896, "text": "In the last episode, we asked what counts as true recursive self-improvement,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 140.196, "end": 144.716, "text": "and in this episode, a real system has emerged that we can actually examine.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 144.956, "end": 149.156, "text": "All right, let's put the dreams aside for a moment and look at what it's doing during the day.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 149.556, "end": 153.916, "text": "Many automated discovery systems already look like they are self-evolving,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 154.416, "end": 157.836, "text": "Like a coding agent that keeps generating candidate programs,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 158.276, "end": 160.056, "text": "an evaluator that scores them,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 160.336, "end": 163.756, "text": "and then the system keeping the better results to continue iterating.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 163.876, "end": 167.896, "text": "But there are actually two completely different jobs mixed up here.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 168.416, "end": 170.936, "text": "Imagine a research team with two people.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 171.436, "end": 176.456, "text": "One performs the experiments, writes the code, runs tests, and gets results,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 176.456, "end": 179.756, "text": "while the other doesn't necessarily get their hands dirty.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 180.316, "end": 182.896, "text": "They are responsible for deciding what to do next,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 182.896, "end": 188.376, "text": "such as which path is worth continuing, which should be stopped, and where resources should", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 188.376, "end": 193.976, "text": "be invested. Systems like AlphaEvo make it very easy to get the feeling that as the program gets", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 193.976, "end": 200.056, "text": "stronger generation by generation, it seems like the AI is already improving itself. But if we zoom", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 200.056, "end": 205.896, "text": "out a little, we see a problem. While the underlying code can change and which results to keep can", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 205.896, "end": 212.116, "text": "change, the second type of work, how to explore the next steps, is often still pre-designed. For", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 212.116, "end": 217.476, "text": "For example, from which node to continue, how many branches to open simultaneously,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 217.476, "end": 220.676, "text": "and after how many attempts to abandon a direction.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 220.676, "end": 223.496, "text": "These rules usually don't evolve along with the system.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 223.496, "end": 225.496, "text": "What about Dream RSI?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2, "text_original": "What about JRSI?", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 225.496, "end": 230.736, "text": "It specifically targets the person responsible for deciding what to do next.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 230.736, "end": 237.676, "text": "Dream RSI separates this layer, and its real-world exploration continuously grows a discovery", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2, "text_original": "JRSI separates this layer, and its real-world exploration continuously grows a discovery", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 237.676, "end": 238.676, "text": "tree.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 238.676, "end": 243.516, "text": "Every node on the tree corresponds to a real attempt that has taken place, such as what", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 243.516, "end": 248.536, "text": "solution the model generated in what context, whether the code could run, and what results", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 248.536, "end": 250.276, "text": "the evaluator gave.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 250.276, "end": 251.656, "text": "All of this has been recorded.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 251.656, "end": 256.896, "text": "The outer exploration strategy only needs to answer a few very simple questions, like", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 256.896, "end": 259.076, "text": "from which node should we continue now?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 259.076, "end": 261.116, "text": "Should we start a new path?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 261.116, "end": 264.116, "text": "How many directions should we pursue in parallel?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 264.116, "end": 265.116, "text": "When should we stop?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 265.116, "end": 270.316, "text": "Let's look at figure 2, which is the most important figure for understanding the entire paper.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 270.316, "end": 274.416, "text": "You can think of this tree in the middle as the maze that the AI actually walked through", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 274.416, "end": 275.416, "text": "yesterday.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 275.416, "end": 280.836, "text": "Every node that is lit up is a place it actually visited, and the results are already known.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 280.836, "end": 286.976, "text": "So the question is, if I want to compare two new exploration strategies today, must I make", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 286.976, "end": 290.116, "text": "the AI walk through the maze twice from the beginning?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 290.116, "end": 293.416, "text": "In the past, that was basically the only way.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 293.416, "end": 298.136, "text": "Because you could only know if a new strategy was good by actually running it again.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 298.136, "end": 301.736, "text": "But Dream RSI's answer is, not necessarily.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2, "text_original": "But JRSI's answer is, not necessarily.", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 301.736, "end": 306.856, "text": "Since it has already walked through it yesterday, and what happened at each node has been saved,", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 306.856, "end": 311.196, "text": "a new strategy can completely walk through this historical tree first.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 311.196, "end": 316.496, "text": "It might go left first, it might turn right sooner, or it might just take three steps", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 316.496, "end": 317.696, "text": "and stop.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 317.696, "end": 322.776, "text": "As long as he visits nodes that have truly occurred, the simulator can directly reveal", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 322.776, "end": 325.916, "text": "the actual results from that time.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 325.916, "end": 331.456, "text": "The key shift here isn't just remembering history, but rather turning history from a", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 331.456, "end": 334.496, "text": "log into a re-enterable event.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 334.496, "end": 337.836, "text": "Now, there is a common misunderstanding here.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 337.836, "end": 343.836, "text": "Dream RSI does not train a new world model first and then have it approximate reality.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 343.836, "end": 349.176, "text": "For Dream RSI, the history that has already played out serves as the playback simulator", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 349.176, "end": 350.176, "text": "itself.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 350.176, "end": 352.336, "text": "Thus, its dreams are quite conservative.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 352.336, "end": 357.936, "text": "It isn't predicting events that haven't happened, but re-entering events that truly took place.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 357.936, "end": 361.356, "text": "Strictly speaking, this world has a clear boundary.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 361.356, "end": 364.736, "text": "It can only dream of places it has actually experienced.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 364.736, "end": 368.936, "text": "It cannot know the results of unexplored branches out of thin air.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 368.936, "end": 372.496, "text": "This is precisely both its advantage and its limitation.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 372.496, "end": 376.816, "text": "The advantage is that real executions that have already occurred don't require paying", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 376.816, "end": 382.736, "text": "for expensive online exploration again, supporting a large volume of rapid offline strategy comparisons.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 382.736, "end": 388.616, "text": "The limitation is that if the first round of real exploration is poor, with almost no high-quality", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 388.616, "end": 393.156, "text": "paths in the history tree, the subsequent dreams won't conjure up a good event out of nothing.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 393.376, "end": 400.316, "text": "So the way it truly loops is this. Reality provides a portion of the world. Dreams in this", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 400.316, "end": 406.256, "text": "world find better exploration strategies, and new strategies return to reality to traverse", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 406.256, "end": 410.196, "text": "new nodes, which then expand the next round of dreams.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 410.196, "end": 413.936, "text": "This is the evolving world mentioned in the paper's title.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 413.936, "end": 417.136, "text": "The world isn't built once and left unchanged.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 417.136, "end": 422.856, "text": "It expands continuously alongside real exploration, and the exploration strategy evolves with", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 422.856, "end": 423.856, "text": "the world.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 423.856, "end": 430.076, "text": "This is much more interesting than simply summarizing historical experience into a few sentences.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 430.076, "end": 434.476, "text": "The authors actually tested another, seemingly simpler method.", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 434.476, "end": 439.636, "text": "Since we already know which directions succeeded or failed, why not let a model summarize the", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 439.636, "end": 443.196, "text": "lessons and inject these suggestions into the next round of prompts?", "language": "en", "chunk": 0, "local_language_run": 2, "language_run": 2 }, { "start": 444.611, "end": 446.841, "text": "However, the results weren't good.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 447.461, "end": 448.781, "text": "Let's look at Figure 5.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 449.281, "end": 455.461, "text": "After adding the guidance, performance actually worsened, regardless of whether RSI was originally used.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 455.701, "end": 461.981, "text": "This is worth pondering, because remembering the past and utilizing the past are not the same thing.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 462.141, "end": 470.301, "text": "Compressing history into a single best thing to do is equivalent to imposing a predetermined shared direction on all subsequent exploration.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 470.301, "end": 478.781, "text": "It could be helpful, but it might also constrain the search space too early, excluding many unconventional paths that should have been preserved.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 479.061, "end": 483.881, "text": "Dream RSI is different. It doesn't tell the AI up front which experience is most important.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2, "text_original": "Re is different. It doesn't tell the AI up front which experience is most important.", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 484.201, "end": 489.281, "text": "It simply preserves that world, allowing different strategies to explore it for themselves.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 489.281, "end": 494.481, "text": "I think the most interesting takeaway from this paper isn't that AI can learn from history,", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 494.921, "end": 501.361, "text": "but rather that perhaps truly valuable experience isn't about summarizing the past into a single instruction,", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 501.841, "end": 505.041, "text": "but about preserving a past that can be re-experienced.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 505.201, "end": 510.761, "text": "If Dream RSI only ends up saving a few API calls, then it's not actually that interesting.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 511.301, "end": 516.141, "text": "What's truly worth looking at is whether it changes the actual exploration process that follows.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 516.141, "end": 519.781, "text": "The authors conducted experiments in algorithmic engineering, mathematical optimization, and GPU.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 519.981, "end": 520.921, "text": "Kernel engineering.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 521.521, "end": 525.561, "text": "There are a lot of numbers, but I think only a few are worth remembering.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 525.881, "end": 527.541, "text": "The first set is cost.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 528.061, "end": 535.921, "text": "In the Lowell algorithm discovery task, one of the most striking figures in the paper is a maximum call gap of about 162 times.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 536.181, "end": 538.521, "text": "However, this figure cannot be taken in isolation.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 539.141, "end": 544.121, "text": "Dream RSI uses Gemini 1.5 Flash, while Simpletext uses GPT-40.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 544.121, "end": 551.381, "text": "A much cleaner comparison is the head-to-head control between Dream RSI and recursive fixed exploration using the same base model.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 551.561, "end": 554.041, "text": "However, I am more concerned with another question.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 554.561, "end": 557.881, "text": "If we search so much less, will the results get worse?", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 558.041, "end": 561.161, "text": "The paper does not show this kind of simple trade-off.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 561.541, "end": 565.361, "text": "On some tasks, it achieves comparable performance with fewer generations,", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 565.861, "end": 570.061, "text": "while on others, it achieves even higher performance within a similar budget.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 570.061, "end": 574.941, "text": "Curil BCH is a great example to illustrate these two different types of gains.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 574.941, "end": 580.381, "text": "VGG16 and LayerNorm show a better sense of when enough is enough,", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 580.381, "end": 584.201, "text": "essentially achieving similar performance with fewer generations.", "language": "en", "chunk": 1, "local_language_run": 1, "language_run": 2 }, { "start": 585.36, "end": 593.35, "text": "Meanwhile, Conv and CX show a better sense of when not to stop, meaning they push performance further within a similar budget.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 594.13, "end": 598.81, "text": "This leads to what I consider the most elegant behavioral chart in the entire paper.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 599.05, "end": 600.51, "text": "Let's look at this chart together.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 601.27, "end": 609.33, "text": "The strategy learned by Dream RSI isn't a simple rule like always do less or always branch out more.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 609.33, "end": 614.25, "text": "When performance improvements are steady early on, it reduces the number of attempts.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 614.25, "end": 619.51, "text": "But when performance hits a plateau, it expands its exploration again.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 619.51, "end": 623.81, "text": "In other words, it has learned a dynamic exploration rhythm.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 623.81, "end": 628.11, "text": "Don't waste your budget casting a wide net when things are going well, but don't get", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 628.11, "end": 630.53, "text": "stuck on the same path when you hit a wall.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 630.53, "end": 636.71, "text": "This is more important than the 162 token count because it shows that RSI ultimately changes", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2, "text_original": "This is more important than the 162 token count because it shows that SI ultimately changes", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 636.71, "end": 642.09, "text": "not just how much compute a single task uses, but how compute is allocated for the next", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 642.09, "end": 643.49, "text": "exploration.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 643.49, "end": 647.95, "text": "This is why the authors place it within the context of recursive self-improvement.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 647.95, "end": 651.49, "text": "However, we must clearly define the boundaries.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 651.49, "end": 656.53, "text": "The underlying coding agent remains unchanged, the model remains the same, the evaluator", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 656.53, "end": 659.61, "text": "is untouched, and the task objectives are fixed.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 659.61, "end": 664.87, "text": "This RSI only modifies the outer strategy for how to continue seeking better results.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 664.87, "end": 670.45, "text": "Therefore, if we interpret this as Google's AI starting to rewrite its own brain, that", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 670.45, "end": 672.53, "text": "would be a complete exaggeration.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 672.53, "end": 683.71, "text": "Conversely, if we just view it as a cost-saving scheduler, that underestimates what this paper truly achieves, as the scheduling rules themselves have now become subjects for improvement.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 684.01, "end": 688.17, "text": "In fact, someone on X has already asked the most critical question.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 688.77, "end": 690.65, "text": "Would you call this true RSI?", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 690.93, "end": 694.91, "text": "This is also the question I wanted to ask most after reading this paper.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 694.91, "end": 708.65, "text": "Since we spent our entire last episode trying to answer what counts as true recursive self-improvement, we can bring back the L1 to L5 chart from last time and use it as a yardstick to measure this RSI.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 709.15, "end": 716.97, "text": "One of the most important viewpoints from that paper was that performance improvement and recursive self-improvement are not the same thing.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 716.97, "end": 727.27, "text": "A system can perform tasks increasingly well, but if the method of self-improvement is hard-coded from the outside, it is still one step away from true recursion.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 727.45, "end": 734.93, "text": "Taking AlphaDev as an example, its most intuitive ability is to continuously search for better programs and algorithms.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 735.33, "end": 738.31, "text": "The main object of improvement is the result itself.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 738.83, "end": 744.01, "text": "You get better at finding better answers, while Darwin-inspired agents take a step further.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 744.01, "end": 749.13, "text": "It doesn't just change code. It allows the implementation of the coding agent itself to be modified.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 749.61, "end": 754.29, "text": "So the object of improvement is no longer just the answer, but the agent that generates the answer.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 754.53, "end": 767.31, "text": "However, in that RSI paper from last time, rules regarding which candidate solutions are kept and which candidate solutions the next generation evolves from were still not part of the self-modification loop.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 768.03, "end": 770.29, "text": "In other words, the agent can change,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 770.61, "end": 774.49, "text": "but the rules determining where the next generation of agents should come from", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 774.49, "end": 776.35, "text": "have not yet entered the recursion.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 776.35, "end": 779.95, "text": "Dream RSI happens to tread exactly on this boundary.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2, "text_original": "DAM RSI happens to tread exactly on this boundary.", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 780.23, "end": 782.93, "text": "What Dream RSI leaves outside the self-modification loop", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2, "text_original": "What DAM leaves outside the self-modification loop", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 782.93, "end": 786.79, "text": "is precisely the mechanism of candidate solution maintenance and selection,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 787.33, "end": 790.03, "text": "essentially where the next generation continues from.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 790.19, "end": 792.95, "text": "What Dream RSI does is precisely turn this layer,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2, "text_original": "What JRSI does is precisely turn this layer,", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 793.29, "end": 797.01, "text": "which nodes continue to branch, how many branches to create, and when to stop,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 797.01, "end": 799.63, "text": "into an object that can be replaced and improved.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 799.77, "end": 804.13, "text": "It didn't modify the underlying agent, but instead changed the outer layer,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 804.49, "end": 807.11, "text": "that is, how the next round of improvement should be sought,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 807.11, "end": 811.29, "text": "how many branches to continue opening from historical nodes, and when to stop.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 811.45, "end": 814.61, "text": "These elements, which originally belonged to the system designer,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 814.99, "end": 817.43, "text": "have been transformed into an exploration strategy", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 817.43, "end": 821.53, "text": "that can be evaluated, modified, and then carried over to the next round.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 821.67, "end": 825.09, "text": "The key is that this new strategy doesn't just stay in a dream.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 825.09, "end": 828.51, "text": "It directly governs the real exploration of the next round.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 828.83, "end": 833.95, "text": "With a fixed strategy, each round can still continue to explore and grow new nodes.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 834.53, "end": 839.57, "text": "The real difference is that Dream RSI lets the new strategy learned in the previous round", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 839.57, "end": 843.79, "text": "decide where the tree grows next and how resources are allocated.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 844.01, "end": 848.07, "text": "And the new exploration history feeds back into the replay simulator,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 848.49, "end": 851.39, "text": "which in turn modifies the next version of the strategy.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 851.39, "end": 857.59, "text": "So, if we look at it through the L1-L5 framework from that paper last time, Dream RSI already", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 857.59, "end": 860.67, "text": "exhibits a very clear L5 characteristic.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 860.67, "end": 864.89, "text": "The object being modified now includes the method used to generate the next round of", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 864.89, "end": 866.25, "text": "improvements itself.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 866.25, "end": 868.95, "text": "But I must add a very important caveat.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 868.95, "end": 871.33, "text": "It is an extremely narrow L5.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 871.33, "end": 876.61, "text": "It does not autonomously modify the underlying model, change the evaluator, redefine the", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 876.61, "end": 880.25, "text": "task, or decide which capability I should improve next.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 880.25, "end": 885.03, "text": "It performs recursive improvement on the exploration mechanism within the boundaries of fixed goals,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 885.37, "end": 888.13, "text": "fixed evaluation rules, and a fixed underlying agent.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 888.29, "end": 891.69, "text": "In the last episode, we actually distinguished between two types of recursion.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 892.05, "end": 893.07, "text": "The first is very simple.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 893.51, "end": 896.47, "text": "Is the mechanism you modified actually used again in the next round?", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 896.61, "end": 898.89, "text": "This is called structural recursion.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 899.13, "end": 901.09, "text": "Dream RSI is very clear on this point.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 901.09, "end": 905.25, "text": "The new exploration strategy enters the next round of online exploration directly.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 905.49, "end": 907.25, "text": "But the harder one is the second type.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 907.25, "end": 910.85, "text": "does using it actually make the successor system better at improving?", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 911.13, "end": 912.97, "text": "This is effective recursion.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 913.45, "end": 916.67, "text": "Dream RSI provides a fairly clean contrast here.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 917.09, "end": 920.27, "text": "It uses the same underlying model, the same evaluator,", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 920.51, "end": 923.89, "text": "and the same initial strategy as the fixed exploration baseline.", "language": "en", "chunk": 1, "local_language_run": 2, "language_run": 2 }, { "start": 925.581, "end": 931.801, "text": "by whether the subsequent exploration strategy has been improved through dreaming. So Dream RSI's", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 931.801, "end": 937.241, "text": "L5 has two dimensions. It is very narrow in scope, but in terms of the clarity of its recursive", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 937.241, "end": 942.861, "text": "evidence, it is not weak. Therefore, a more accurate statement is not that true general RSI", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 942.861, "end": 949.161, "text": "has arrived, but rather that we have finally seen a very clean, local example where the improvement", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 949.161, "end": 954.801, "text": "This also makes its relationship with AV training particularly interesting.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2, "text_original": "mechanism itself has entered an improvement loop. This also makes its relationship with AV training", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 954.801, "end": 960.421, "text": "particularly interesting. Last time, we discussed how AV training precipitates the results of", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 960.421, "end": 965.601, "text": "post-training experiments into a set of persistent research strategies and exploration logs,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 965.601, "end": 970.561, "text": "which are then modified by the meta-agent to guide the subsequent executive agents.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 970.941, "end": 974.181, "text": "Doesn't that sound already very close to Dream RSI?", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2, "text_original": "Doesn't that sound already very close to JRSI?", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 974.541, "end": 978.321, "text": "Neither is satisfied with just modifying the final model or the final answer.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 978.761, "end": 982.821, "text": "They have begun to make the research process itself something that can be accumulated,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 983.261, "end": 984.421, "text": "modified, and inherited.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 984.741, "end": 988.361, "text": "However, the way both utilize history is not the same.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 988.581, "end": 994.221, "text": "AVIL training is more like letting a meta-agent read past evidence to directly diagnose problems", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 994.221, "end": 1000.341, "text": "and then modify the policy. But it adds an extra step. Instead of just inferring a new strategy", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1000.341, "end": 1006.341, "text": "based on history, it generates multiple candidate exploration strategies, puts them into a historical", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1006.341, "end": 1011.481, "text": "replay world to actually run them, and uses the results to choose which one enters reality.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1011.701, "end": 1016.901, "text": "See, this is more like modifying a plan after a post-mortem review, whereas the other is more", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1016.901, "end": 1021.561, "text": "like putting multiple new plans back into a sandbox for simulation. This brings us right", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1021.561, "end": 1026.061, "text": "back to the figure. The authors actually specifically tried the more direct path of", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1026.061, "end": 1031.541, "text": "summarizing history into semantic guidance, but the results were worse. Therefore, the most special", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1031.541, "end": 1036.701, "text": "thing about JRSI is not just that it preserves history, but that it lets candidate strategies", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1036.701, "end": 1041.861, "text": "re-experience history and then uses the outcomes of that experience to make a choice. Perhaps a", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1041.861, "end": 1047.381, "text": "truly powerful self-improving system in the future will not choose just one of these, but might", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1047.381, "end": 1052.801, "text": "simultaneously distill experience into high-level strategies while retaining a sufficiently rich", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1052.801, "end": 1057.581, "text": "set of historical events so these strategies can be retested at any time. At that point,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1057.941, "end": 1061.821, "text": "memory will no longer just be the model knowing what it has done before,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1061.821, "end": 1068.761, "text": "but will start to become a past that can be rerun. Speaking of which, there is one final question.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1069.461, "end": 1075.841, "text": "Why did the authors choose to call this dream? As we mentioned earlier, Dream RSI does not train", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1075.841, "end": 1081.841, "text": "a world model in the traditional sense, but it borrows the same intuition. Real-world interaction", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1081.841, "end": 1087.961, "text": "is too expensive, so can we turn past experiences into an internal environment for continued learning?", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1089.001, "end": 1094.821, "text": "Dream RSI borrows this line of thinking, but the object it simulates is different. It does not attempt to", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2, "text_original": "JRSI borrows this line of thinking, but the object it simulates is different. It does not attempt to", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 1094.821, "end": 1099.881, "text": "learn a complete external world, but instead preserves how it explored this world in the past", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1099.881, "end": 1105.001, "text": "and what happened during each exploration. So I think the truly interesting part of this paper", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1105.001, "end": 1108.321, "text": "is that the world here has quietly changed its target.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1108.561, "end": 1110.241, "text": "In the past, we were more concerned", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1110.241, "end": 1111.741, "text": "with how the world would change.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1112.201, "end": 1114.181, "text": "But Dream RSI starts to care about", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2, "text_original": "But JRSI starts to care about", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 1114.181, "end": 1116.741, "text": "how I personally sought out changes in the past", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1116.741, "end": 1119.621, "text": "and what would happen if I used a different way of searching.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1119.781, "end": 1122.941, "text": "Of course, today's Dream RSI is still very far", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2, "text_original": "Of course, today's JRSI is still very far", "correction_applied": true, "correction_source": "llm_correct_a_presegmentation" }, { "start": 1122.941, "end": 1124.461, "text": "from being a superintelligence", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1124.461, "end": 1126.141, "text": "that can improve itself indefinitely.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1126.441, "end": 1129.321, "text": "Its model hasn't changed, its goals haven't changed,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1129.641, "end": 1132.181, "text": "and the rules evaluating it haven't changed either.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1132.821, "end": 1137.061, "text": "Even the dreams it can have are limited to the past it has truly experienced.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1137.761, "end": 1142.361, "text": "But something that used to be set in stone has started to shift.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1143.021, "end": 1145.241, "text": "It's not about what the next answer should be,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1145.741, "end": 1149.261, "text": "but rather how I should find a way to improve myself next time.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1149.261, "end": 1154.581, "text": "When the answer to this question also starts to be modified and verified by the machine itself", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1154.581, "end": 1155.881, "text": "to be used in the next round,", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1156.341, "end": 1160.721, "text": "the most noteworthy part of recursive self-improvement may have just begun.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1160.721, "end": 1166.141, "text": "Alright, that's all for today's content. Please like, share, and subscribe to my channel.", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1166.301, "end": 1170.781, "text": "If you'd like to support me further, feel free to join my membership. The cost of a cup of coffee", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1170.781, "end": 1175.321, "text": "each month not only supports me, but also helps me continue creating more content related to AI", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 }, { "start": 1175.321, "end": 1178.961, "text": "and technology. Alright, thanks for watching, and I'll see you in the next one. Yay!", "language": "en", "chunk": 2, "local_language_run": 0, "language_run": 2 } ], "segmented_multilingual_transcription": { "status": "STABLE_SEGMENTED_MULTILINGUAL_OK", "created_at": "2026-09-21T23:52:52", "manifest": "/Users/sagawa/AI_WORK/video_notes/url/20260921_234513__谷歌这次_真的不一样了/segmented_work/chunk_manifest.json", "audio": "/Users/sagawa/AI_WORK/video_notes/url/20260921_234513__谷歌这次_真的不一样了/audio_16k_mono.wav", "audio_sha256": "0557ef7b5697e8f6ede8f68c851235c0039f7ba05282476c71890c001d26f9ef", "audio_duration": 1179.5505, "model": "/Users/sagawa/AI_WORK/cache/huggingface/hub/models--mlx-community--whisper-large-v3-turbo/snapshots/a4aaeec0636e6fef84abdcbe3544cb2bf7e9f6fb", "language_profile": "/Users/sagawa/AI_WORK/video_notes/url/20260921_234513__谷歌这次_真的不一样了/multilingual_work/multilingual_language_profile.json", "dominant_languages": [ "en" ], "suppressed_languages": [], "chunk_count": 3, "completed_chunks": 3, "resumed_chunks": 0, "merged_segment_count": 263, "detected_languages": [ "en" ], "transcript_languages": [ "en" ], "missing_languages": [], "unexpected_languages": [], "language_sequence": [ "en" ], "speech_region_seconds": 1173.327, "speech_covered_seconds": 1126.3200000000002, "speech_coverage_ratio": 0.9599369996599415, "tail_repeat_run": 1, "tail_repeat_text": "When sharing this paper, Google's Subham Sahu wrote that for an agent to achieve recursive self-improvement, it must learn to dream.", "chunk_needs_review": [], "qc_ok": true, "qc_errors": [], "complete_review_artifact": true }, "presegmentation_correction": { "applied": true, "correction_count": 14, "deterministic_fix_count": 0 } }