{
  "text": "Hello,大家好,我是Tommy。今天很高興跟大家分享最新的前置內容。\n那上一次我們講過Deepseek V4.1 Flash,怎麼降低KP Cache的存儲開銷。\n那麼這一次我們繼續看它怎麼減少脫離計算和緩存成本。\n那核心呢就兩件事。第一個就是重新計算上下文的時候呢,只計算前一半的層。\n那麼後面一半是直接附用已有的結果。\n那第二個呢就是一部分短期緩存不再長期保存,而是在需要時重新計算。\n那最终呢\n计算量降低约一半\n持久化缓存降到原来的1%\n同时性能基本不受影响\n那么这里呢\n最关键的是两个问题\n那第一个就是\n为什么只计算一半的层就够了\n第二个呢\n为什么重新计算\n反而比把缓存一直存下来更划算\n那下面我们就直接来看它的具体设计\n上一集那个编码agent\n修一个仓库里的bug\n跑了三个小时\n上下文堆到50万token\n上一集算的是它要存多少\n40层只存4份KV\n每个Token890字节\n这一集算的是它要读多少\n上一集结尾留了一句\n4份里解码器那一份\n还是从编码器投影来的\n这意味着读入新内容时\n只用跑前20层\n后20层不用算\n这一招叫CED\n全称因果编码器解码器\n这一集就讲它\n它把40层分成前后两半\n前20层叫编码器\n负责读上下文\n后20层叫解码器负责生成\n解码器要用的KV\n从编码器的输出投影出来\n所以读入的时候\n后20层可以不跑\n为什么能这么干\n代价是什么\n就是这一集的内容\n先看读入这一步为什么贵\nAgent每掉一次工具\n工具的返回都要接到上下文后面\n模型再把新内容读一遍\n也就是上一集说过的prefill\n有缓存的时候\n只读新增的那一段\n缓存一旦没命中\n整段50万都要重算\n一次任务几十次工具调用\n就是几十次读入\n读一次\n40层每一层都要算\n上下文越长\n这比算力越大\n这还只是算力\n50万token的KV还得留着\n等下一轮\n甚至等第二天用户回来接着聊\n所以它要从显存落到硬盘上\nV4的线上部署里\n这份缓存要在硬盘上留72小时以上\n显存装不装的下\n硬盘存不存的起\n机器之间搬不搬的动\n三处都是压力\n算力和硬盘两笔成本\n问题来了\n独入这一步\nV4.1凭什么算力砍一半\n硬盘砍到1%\n先说算力\n2024年有篇论文叫Yoko\n意思是你只缓存一次\n它把模型前后两半分开\n后半层不再各自算KV\n直接用前半层算好的那一份\n后半层在独入阶段\n就可以整个跳过\n代价是后半层共用同一份KV\n每一层没法有自己的一份\n能存的信息就少了\nCD顺着这个思路改了两处\n解码器要用的全局KV\n拿编码器第20层的输出\n乘一个投影矩阵得到\n第一处改动\n投影矩阵可以逐层不同\n每层能有自己的一份KV\n容量比YOKO共用一份大\n第二处\n滑动窗口那条局部分支\n不投影\n每一层还是自己算\n局部信息的计算深度保住了\n在V4.1里\n配合上一集讲的共用\n真正做投影的是解码器那一个负层\n其余层沿用它\n独入阶段编码器20层算完\n解码器要的全局KV顺手投影出来\n后20层根本不用跑\n算力从40层变20层\n激活参数从160亿降到80亿\n生成阶段才用满160亿\n这里有个缺口要补\n滑动窗口KV每一层自己算\n可独入阶段解码器根本没跑\n解码器20层的滑动窗口KV就是空的\n生成第一个Token时\n解码器要用到前面128个Token的窗口KV\n手里却是空的\n想精确补上\n就得让解码器\n把前面那些token重新算一遍\n算多少\n窗口是一层套一层的\n解码器第一层的一个位置\n只看它前面128个\n第二层的同一个位置\n看的是第一层算出的那128个\n而那128个\n各自又看过更前面的128个\n所以第二层实际牵扯到前面256个\n一层多128个\n到第二层就是20乘128\n2560个\n也就是说\n要把第20层的窗口KV算准\n得从2560个Token之前开始\n把解码器重新跑一遍\nAgent的场景是多轮短消息\n每轮新内容可能就几百个Token\n为了补窗口再多跑2560个\n这比开销不小\n这个版本的做法叫\n滑动窗口有界回放\n不回退2560个\n只回退末尾128个\n把这128个Token的编码器输出\n送进解码器跑一遍\n开销是128个Token乘20层\n跟50万笔可以忽略\n窗口也只在这128个里面看\n再往前的一律不看\n因为再往前的token没经过解码器\n没有东西可看\n回放段里的第一个token\n窗口里就只有他自己\n越往后能看到的越多\n到末尾正好是完整的128个\n算出来的滑动窗口KV只拿来生成\n不存进缓存\n为什么敢\n因为窗口的有效视野远比理论知晓\n有篇叫PowerAttention的工作专门测过这件事\n理论上第20层的视野\n能到2560个token之外\n实际上远处的信息\n一层一层传过来\n早就衰减到可以忽略\n远处的东西\n本来就归全局分支管\n窗口只负责眼前这一段\n这样补出来的KV\n和完整跑一遍\n不完全一样\n报告说实测几乎不掉点\n为了保险\n后训练时\n把同样的回放模拟进去\n模型是在这种近似下练出来的\n顺便说一句\n这两集提到的前变量\n稀疏注意力\n后训练里的这种训练感知\n都出自DeepSeek这套技术战\n我做了一门DeepSeek专题的实战课\n从MLA MOE ROPE这些核心机制\n讲到GRPO对齐训练\n再到VLLM SGLAN固鼓鼠\n共计11张配项目代码\n想把这两集背后的底子补齐\n链接在简介和置顶评论里\n算力说完\n说硬盘\n这一半省得更巧\n先看V4是怎么存的\n落到硬盘的缓存\n叫持久化缓存\n分两类\n全局KV整段存\n只要新请求的开头和缓存里的一样\n就整段复用\n滑动窗口KV不整段存\n只在两个位置各存128个\n一段提示词的末尾\n一段回答的末尾\n为的是重新生成和多轮接续\n两类各自淘汰\n最久没被用到的先删\n硬盘配的够大\n两类都能留72小时以上\n就这样\n滑动窗口KV还是占了持久化缓存将近一半\n它没有压缩\n每个位置128个Token\n每一层各一份的8比特KV原样存\n多轮对话每轮都要存一份\n轮次多每轮短分数就多\n问题处在访问模式\n全局KV是常委复用\n一天以后用户回来还会命中\n滑动窗口KV只在一次绘画里\n几分钟之内被用到\n绘画一结束或者进到下一轮\n它就再也不会被用到\n把一份分钟级的东西放进保留72小时的缓存\n又贵又没用\nV4的报告提过干脆不存\n要用再算\n叫做Zero SWA Catching\n可精确重算\n要按层数乘128个token跑一遍完整前向\n线上扛不住\n这个版本改了两条\n第一条\n滑动窗口KV不再进硬盘\n改放进一个内存池\n所有机器各拿出10%的内存拼成一个池子\n注意是机器的内存不是显存\n池子总量比硬盘小得多\n但有效期只有几分钟\n过期立刻腾给新绘画\n真实负载下这种高周转\n够服务绝大多数正在进行的绘画\n全局KV照旧流硬盘\n保证72小时\n第二条池子小必然有未命中\n未命中指的是全局KV硬盘上还在\n滑动窗口KV在池子里已经过期\n编码器也是20层\n精确补齐同样要2560个\n现在只回放128个\n补出来的窗口KV是近似的\n缓存到哪里为止\n回放就从哪里开始\n起点不同\n补出来的KV就略有不同\n报告说这对回答质量几乎没有影响\n还用了一个说法\n把灾难性的未命中\n变成了优雅又便宜的降级\n正因为兜底便宜\n才敢把滑动窗口KV从硬盘上拿掉\n两种回放别混\n解码器回放每次独入都做\n补的是解码器的窗口KV\n因为解码器从来不跑独入\n编码器回放只在未命中识做\n补的是编码器的窗口KV\n因为硬盘上不再存它\n两个都只回放末尾128个\n都基于同一个判断\n窗口的记忆很短\n1%就是这么算出来的\n不存滑动窗口KV\n19化缓存少一半\n剩下的全局KV\n上一集讲过\n是上一代的四分之一\n二分之一乘四分之一\n八分之一\n把三层存储放在一起看就清楚了\n显存里放正在用的全局KV\n每个Token890字节\n机器内存里放几分钟内会用的\n滑动窗口KV\n过期就扔\n硬盘上放几天内可能会用的全局KV\n留72小时\n三层各管一个时间尺度\n每一层只存它该存的\n回到那个50万Token的绘画\n看两个时刻\n第一个时刻\n工具返回一段新内容\n读入直跑编码器20层\n激活80亿参数\n解码器要的全局KV顺手投影出来\n滑动窗口回放128个\n开始生成\n第二个时刻\n第二天用户回来接着聊\n硬盘上取出来的只有全局KV400多兆\n滑动窗口那份现算还是128个\n两处近似\n报告给的结论是性能与基线相当\n读入的计算量减半\n部署上还把视觉编码读入生成分成三段\n各自扩缩容\n读入密集的agent负载可以单独加机器\n多轮短消息的agent绘画\n就是报告点名要服务的场景\n局限一共三条\n第一条\n有界回放是近似\n尤其是缓存和新内容接缝的地方\n报告承认还没充分验证\n接下来要加压力测试\n也要在真实负载里继续观察\n后两条是我自己的判断\n第二条\n收益跟场景绑定\n多轮每轮短的agent绘画收益最大\n一次性为一份几十万token的长文档\n省的主要是算力那一半\n硬盘那一半的好处基本用不上\n第三条\n10%的内存池\n几分钟的有效期\n72小时的硬盘保留\n都是按Deepseek自家的线上负载掉出来的\n换一种负载分布\n未命中率和回放开销都要重新测\n回到开头的问题\n算力砍一半\n靠CED解码器的全局KV\n从编码器投影\n独入只跑20层\n硬盘砍到1%\n靠不存滑动窗口KV\n为民众就回放128个补出来\n再乘上上一集全局KV的1%\n两件事背后是同一个判断\n滑动窗口的记忆很短\n短到可以现算\n上一集的共用\n加上这一集的回放\n长上下文的存和算一起压了下来\n成绩还没有掉\n如果这一集让你有点收获\n欢迎点赞转发\n关注\n评论区也聊聊\n换你来做部署\n滑动窗口KV\n你会存硬盘保命中率\n还是不存\n靠回放节省硬盘\n好\n我们下期继续",
  "language": "zh",
  "duration": 722.372813,
  "segments": [
    {
      "start": 0.0,
      "end": 3.84,
      "text": "Hello,大家好,我是Tommy。今天很高興跟大家分享最新的前置內容。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 4.1,
      "end": 9.22,
      "text": "那上一次我們講過Deepseek V4.1 Flash,怎麼降低KP Cache的存儲開銷。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 9.22,
      "end": 13.32,
      "text": "那麼這一次我們繼續看它怎麼減少脫離計算和緩存成本。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 13.56,
      "end": 19.46,
      "text": "那核心呢就兩件事。第一個就是重新計算上下文的時候呢,只計算前一半的層。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 19.46,
      "end": 22.52,
      "text": "那麼後面一半是直接附用已有的結果。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 22.78,
      "end": 28.92,
      "text": "那第二個呢就是一部分短期緩存不再長期保存,而是在需要時重新計算。",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.25998384406767694,
      "no_speech_prob": 1.020244479177368e-11,
      "compression_ratio": 1.3109919571045576
    },
    {
      "start": 28.92,
      "end": 29.88,
      "text": "那最终呢",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 29.88,
      "end": 31.66,
      "text": "计算量降低约一半",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 31.66,
      "end": 34.02,
      "text": "持久化缓存降到原来的1%",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 34.02,
      "end": 36.3,
      "text": "同时性能基本不受影响",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 36.3,
      "end": 37.2,
      "text": "那么这里呢",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 37.2,
      "end": 38.86,
      "text": "最关键的是两个问题",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 38.86,
      "end": 39.86,
      "text": "那第一个就是",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 39.86,
      "end": 42.64,
      "text": "为什么只计算一半的层就够了",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 42.64,
      "end": 43.58,
      "text": "第二个呢",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 43.58,
      "end": 45.0,
      "text": "为什么重新计算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 45.0,
      "end": 47.8,
      "text": "反而比把缓存一直存下来更划算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 47.8,
      "end": 50.52,
      "text": "那下面我们就直接来看它的具体设计",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 50.52,
      "end": 52.72,
      "text": "上一集那个编码agent",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 52.72,
      "end": 54.24,
      "text": "修一个仓库里的bug",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 54.24,
      "end": 55.2,
      "text": "跑了三个小时",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 55.2,
      "end": 56.92,
      "text": "上下文堆到50万token",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13888399396623885,
      "no_speech_prob": 1.7504988777949926e-11,
      "compression_ratio": 1.29874213836478
    },
    {
      "start": 56.92,
      "end": 59.2,
      "text": "上一集算的是它要存多少",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 59.2,
      "end": 61.58,
      "text": "40层只存4份KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 61.58,
      "end": 63.22,
      "text": "每个Token890字节",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 63.22,
      "end": 65.32,
      "text": "这一集算的是它要读多少",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 65.32,
      "end": 66.98,
      "text": "上一集结尾留了一句",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 66.98,
      "end": 68.86,
      "text": "4份里解码器那一份",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 68.86,
      "end": 71.0,
      "text": "还是从编码器投影来的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 71.0,
      "end": 72.86,
      "text": "这意味着读入新内容时",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 72.86,
      "end": 74.3,
      "text": "只用跑前20层",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 74.3,
      "end": 75.52,
      "text": "后20层不用算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 75.52,
      "end": 77.1,
      "text": "这一招叫CED",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 77.1,
      "end": 79.2,
      "text": "全称因果编码器解码器",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 79.2,
      "end": 80.6,
      "text": "这一集就讲它",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 80.6,
      "end": 83.02,
      "text": "它把40层分成前后两半",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 83.02,
      "end": 84.62,
      "text": "前20层叫编码器",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 84.62,
      "end": 86.06,
      "text": "负责读上下文",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09258116208589993,
      "no_speech_prob": 1.575479850746131e-11,
      "compression_ratio": 1.3633093525179856
    },
    {
      "start": 86.06,
      "end": 88.86,
      "text": "后20层叫解码器负责生成",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 88.86,
      "end": 90.64,
      "text": "解码器要用的KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 90.64,
      "end": 92.76,
      "text": "从编码器的输出投影出来",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 92.76,
      "end": 94.0,
      "text": "所以读入的时候",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 94.0,
      "end": 95.34,
      "text": "后20层可以不跑",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 95.34,
      "end": 96.66,
      "text": "为什么能这么干",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 96.66,
      "end": 97.76,
      "text": "代价是什么",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 97.76,
      "end": 99.22,
      "text": "就是这一集的内容",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 99.22,
      "end": 101.54,
      "text": "先看读入这一步为什么贵",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 101.54,
      "end": 103.36,
      "text": "Agent每掉一次工具",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 103.36,
      "end": 106.06,
      "text": "工具的返回都要接到上下文后面",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 106.06,
      "end": 107.84,
      "text": "模型再把新内容读一遍",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 107.84,
      "end": 109.82,
      "text": "也就是上一集说过的prefill",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 109.82,
      "end": 111.42,
      "text": "有缓存的时候",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 111.42,
      "end": 113.04,
      "text": "只读新增的那一段",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 113.04,
      "end": 114.7,
      "text": "缓存一旦没命中",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1101282634092181,
      "no_speech_prob": 1.5679894882492107e-11,
      "compression_ratio": 1.3475409836065573
    },
    {
      "start": 114.7,
      "end": 116.52,
      "text": "整段50万都要重算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 116.52,
      "end": 118.88,
      "text": "一次任务几十次工具调用",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 118.88,
      "end": 120.24,
      "text": "就是几十次读入",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 120.24,
      "end": 121.1,
      "text": "读一次",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 121.1,
      "end": 122.88,
      "text": "40层每一层都要算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 122.88,
      "end": 124.08,
      "text": "上下文越长",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 124.08,
      "end": 125.26,
      "text": "这比算力越大",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 125.26,
      "end": 127.02,
      "text": "这还只是算力",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 127.02,
      "end": 129.44,
      "text": "50万token的KV还得留着",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 129.44,
      "end": 130.28,
      "text": "等下一轮",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 130.28,
      "end": 132.36,
      "text": "甚至等第二天用户回来接着聊",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 132.36,
      "end": 134.76,
      "text": "所以它要从显存落到硬盘上",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 134.76,
      "end": 136.26,
      "text": "V4的线上部署里",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 136.26,
      "end": 139.18,
      "text": "这份缓存要在硬盘上留72小时以上",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 139.18,
      "end": 141.06,
      "text": "显存装不装的下",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 141.06,
      "end": 142.5,
      "text": "硬盘存不存的起",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 142.5,
      "end": 143.92,
      "text": "机器之间搬不搬的动",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10598150065525826,
      "no_speech_prob": 1.4967360684181585e-11,
      "compression_ratio": 1.3257328990228012
    },
    {
      "start": 143.92,
      "end": 145.36,
      "text": "三处都是压力",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 145.36,
      "end": 147.6,
      "text": "算力和硬盘两笔成本",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 147.6,
      "end": 148.54,
      "text": "问题来了",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 148.54,
      "end": 149.52,
      "text": "独入这一步",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 149.52,
      "end": 151.84,
      "text": "V4.1凭什么算力砍一半",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 151.84,
      "end": 152.92,
      "text": "硬盘砍到1%",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 152.92,
      "end": 154.12,
      "text": "先说算力",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 154.12,
      "end": 156.12,
      "text": "2024年有篇论文叫Yoko",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 156.12,
      "end": 157.88,
      "text": "意思是你只缓存一次",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 157.88,
      "end": 160.2,
      "text": "它把模型前后两半分开",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 160.2,
      "end": 162.4,
      "text": "后半层不再各自算KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 162.4,
      "end": 164.64,
      "text": "直接用前半层算好的那一份",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 164.64,
      "end": 166.44,
      "text": "后半层在独入阶段",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 166.44,
      "end": 167.64,
      "text": "就可以整个跳过",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 167.64,
      "end": 170.32,
      "text": "代价是后半层共用同一份KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 170.32,
      "end": 172.14,
      "text": "每一层没法有自己的一份",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 172.14,
      "end": 173.84,
      "text": "能存的信息就少了",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09203361712004009,
      "no_speech_prob": 1.5322109900295366e-11,
      "compression_ratio": 1.2911392405063291
    },
    {
      "start": 173.92,
      "end": 176.36,
      "text": "CD顺着这个思路改了两处",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 176.36,
      "end": 178.38,
      "text": "解码器要用的全局KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 178.38,
      "end": 180.66,
      "text": "拿编码器第20层的输出",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 180.66,
      "end": 182.4,
      "text": "乘一个投影矩阵得到",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 182.4,
      "end": 183.44,
      "text": "第一处改动",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 183.44,
      "end": 185.38,
      "text": "投影矩阵可以逐层不同",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 185.38,
      "end": 187.26,
      "text": "每层能有自己的一份KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 187.26,
      "end": 189.34,
      "text": "容量比YOKO共用一份大",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 189.34,
      "end": 190.24,
      "text": "第二处",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 190.24,
      "end": 192.1,
      "text": "滑动窗口那条局部分支",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 192.1,
      "end": 192.7,
      "text": "不投影",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 192.7,
      "end": 194.14,
      "text": "每一层还是自己算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 194.14,
      "end": 196.32,
      "text": "局部信息的计算深度保住了",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 196.32,
      "end": 197.8,
      "text": "在V4.1里",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 197.8,
      "end": 199.4,
      "text": "配合上一集讲的共用",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 199.4,
      "end": 202.04,
      "text": "真正做投影的是解码器那一个负层",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 202.04,
      "end": 203.32,
      "text": "其余层沿用它",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10801135063171387,
      "no_speech_prob": 1.8635178469783575e-11,
      "compression_ratio": 1.329153605015674
    },
    {
      "start": 203.32,
      "end": 205.82,
      "text": "独入阶段编码器20层算完",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 205.82,
      "end": 208.52,
      "text": "解码器要的全局KV顺手投影出来",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 208.52,
      "end": 210.24,
      "text": "后20层根本不用跑",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 210.24,
      "end": 212.34,
      "text": "算力从40层变20层",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 212.34,
      "end": 214.92,
      "text": "激活参数从160亿降到80亿",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 214.92,
      "end": 217.16,
      "text": "生成阶段才用满160亿",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 217.16,
      "end": 218.58,
      "text": "这里有个缺口要补",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 218.58,
      "end": 220.98,
      "text": "滑动窗口KV每一层自己算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 220.98,
      "end": 223.18,
      "text": "可独入阶段解码器根本没跑",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 223.18,
      "end": 226.3,
      "text": "解码器20层的滑动窗口KV就是空的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 226.3,
      "end": 227.8,
      "text": "生成第一个Token时",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 227.8,
      "end": 231.2,
      "text": "解码器要用到前面128个Token的窗口KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 231.2,
      "end": 232.36,
      "text": "手里却是空的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10543540528583042,
      "no_speech_prob": 1.3423428167624873e-11,
      "compression_ratio": 1.4148936170212767
    },
    {
      "start": 232.36,
      "end": 233.82,
      "text": "想精确补上",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 233.82,
      "end": 234.8,
      "text": "就得让解码器",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 234.8,
      "end": 237.34,
      "text": "把前面那些token重新算一遍",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 237.34,
      "end": 237.8,
      "text": "算多少",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 237.8,
      "end": 239.82,
      "text": "窗口是一层套一层的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 239.82,
      "end": 241.76,
      "text": "解码器第一层的一个位置",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 241.76,
      "end": 243.62,
      "text": "只看它前面128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 243.62,
      "end": 245.24,
      "text": "第二层的同一个位置",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 245.24,
      "end": 247.9,
      "text": "看的是第一层算出的那128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 247.9,
      "end": 249.2,
      "text": "而那128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 249.2,
      "end": 251.5,
      "text": "各自又看过更前面的128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 251.5,
      "end": 254.72,
      "text": "所以第二层实际牵扯到前面256个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 254.72,
      "end": 256.56,
      "text": "一层多128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 256.56,
      "end": 259.12,
      "text": "到第二层就是20乘128",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 259.12,
      "end": 260.48,
      "text": "2560个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 260.48,
      "end": 261.54,
      "text": "也就是说",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11524285826572152,
      "no_speech_prob": 1.5390128407788417e-11,
      "compression_ratio": 1.472
    },
    {
      "start": 261.54,
      "end": 264.02,
      "text": "要把第20层的窗口KV算准",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 264.02,
      "end": 266.5,
      "text": "得从2560个Token之前开始",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 266.5,
      "end": 268.3,
      "text": "把解码器重新跑一遍",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 268.3,
      "end": 270.54,
      "text": "Agent的场景是多轮短消息",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 270.54,
      "end": 272.74,
      "text": "每轮新内容可能就几百个Token",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 272.74,
      "end": 275.26,
      "text": "为了补窗口再多跑2560个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 275.26,
      "end": 276.34,
      "text": "这比开销不小",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 276.34,
      "end": 277.9,
      "text": "这个版本的做法叫",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 277.9,
      "end": 279.8,
      "text": "滑动窗口有界回放",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 279.8,
      "end": 281.92,
      "text": "不回退2560个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 281.92,
      "end": 283.94,
      "text": "只回退末尾128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 283.94,
      "end": 286.5,
      "text": "把这128个Token的编码器输出",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 286.5,
      "end": 287.98,
      "text": "送进解码器跑一遍",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 287.98,
      "end": 290.7,
      "text": "开销是128个Token乘20层",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.09610757570008974,
      "no_speech_prob": 1.3957343093784491e-11,
      "compression_ratio": 1.3114186851211074
    },
    {
      "start": 290.7,
      "end": 292.38,
      "text": "跟50万笔可以忽略",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 292.38,
      "end": 295.34,
      "text": "窗口也只在这128个里面看",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 295.34,
      "end": 296.94,
      "text": "再往前的一律不看",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 296.94,
      "end": 299.36,
      "text": "因为再往前的token没经过解码器",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 299.36,
      "end": 300.32,
      "text": "没有东西可看",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 300.32,
      "end": 302.28,
      "text": "回放段里的第一个token",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 302.28,
      "end": 303.78,
      "text": "窗口里就只有他自己",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 303.78,
      "end": 305.68,
      "text": "越往后能看到的越多",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 305.68,
      "end": 308.2,
      "text": "到末尾正好是完整的128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 308.2,
      "end": 311.34,
      "text": "算出来的滑动窗口KV只拿来生成",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 311.34,
      "end": 312.38,
      "text": "不存进缓存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 312.38,
      "end": 313.52,
      "text": "为什么敢",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 313.52,
      "end": 316.54,
      "text": "因为窗口的有效视野远比理论知晓",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 316.54,
      "end": 319.72,
      "text": "有篇叫PowerAttention的工作专门测过这件事",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10846803991834103,
      "no_speech_prob": 1.52078922371901e-11,
      "compression_ratio": 1.2604790419161678
    },
    {
      "start": 319.72,
      "end": 321.58,
      "text": "理论上第20层的视野",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 321.58,
      "end": 323.68,
      "text": "能到2560个token之外",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 323.68,
      "end": 325.26,
      "text": "实际上远处的信息",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 325.26,
      "end": 326.3,
      "text": "一层一层传过来",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 326.3,
      "end": 327.98,
      "text": "早就衰减到可以忽略",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 327.98,
      "end": 329.06,
      "text": "远处的东西",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 329.06,
      "end": 330.74,
      "text": "本来就归全局分支管",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 330.74,
      "end": 332.6,
      "text": "窗口只负责眼前这一段",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 332.6,
      "end": 334.04,
      "text": "这样补出来的KV",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 334.04,
      "end": 335.34,
      "text": "和完整跑一遍",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 335.34,
      "end": 336.16,
      "text": "不完全一样",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 336.16,
      "end": 338.1,
      "text": "报告说实测几乎不掉点",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 338.1,
      "end": 339.12,
      "text": "为了保险",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 339.12,
      "end": 340.02,
      "text": "后训练时",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 340.02,
      "end": 341.76,
      "text": "把同样的回放模拟进去",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 341.76,
      "end": 344.18,
      "text": "模型是在这种近似下练出来的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 344.18,
      "end": 345.44,
      "text": "顺便说一句",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 345.44,
      "end": 347.58,
      "text": "这两集提到的前变量",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 347.58,
      "end": 348.52,
      "text": "稀疏注意力",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.1048318844921184,
      "no_speech_prob": 1.8052000866353168e-11,
      "compression_ratio": 1.2448979591836735
    },
    {
      "start": 348.52,
      "end": 350.36,
      "text": "后训练里的这种训练感知",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 350.36,
      "end": 352.52,
      "text": "都出自DeepSeek这套技术战",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 352.52,
      "end": 355.04,
      "text": "我做了一门DeepSeek专题的实战课",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 355.04,
      "end": 358.86,
      "text": "从MLA MOE ROPE这些核心机制",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 358.86,
      "end": 360.82,
      "text": "讲到GRPO对齐训练",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 360.82,
      "end": 363.3,
      "text": "再到VLLM SGLAN固鼓鼠",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 363.3,
      "end": 365.38,
      "text": "共计11张配项目代码",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 365.38,
      "end": 367.48,
      "text": "想把这两集背后的底子补齐",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 367.48,
      "end": 369.78,
      "text": "链接在简介和置顶评论里",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 369.78,
      "end": 371.02,
      "text": "算力说完",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 371.02,
      "end": 371.74,
      "text": "说硬盘",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 371.74,
      "end": 373.4,
      "text": "这一半省得更巧",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 373.4,
      "end": 375.52,
      "text": "先看V4是怎么存的",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.13569524379375855,
      "no_speech_prob": 1.6624500387418806e-11,
      "compression_ratio": 1.1344262295081968
    },
    {
      "start": 375.52,
      "end": 377.34,
      "text": "落到硬盘的缓存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 377.34,
      "end": 378.64,
      "text": "叫持久化缓存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 378.64,
      "end": 379.5,
      "text": "分两类",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 379.5,
      "end": 381.54,
      "text": "全局KV整段存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 381.54,
      "end": 384.38,
      "text": "只要新请求的开头和缓存里的一样",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 384.38,
      "end": 385.6,
      "text": "就整段复用",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 385.6,
      "end": 387.88,
      "text": "滑动窗口KV不整段存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 387.88,
      "end": 390.32,
      "text": "只在两个位置各存128个",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 390.32,
      "end": 391.98,
      "text": "一段提示词的末尾",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 391.98,
      "end": 393.3,
      "text": "一段回答的末尾",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 393.3,
      "end": 395.7,
      "text": "为的是重新生成和多轮接续",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 395.7,
      "end": 397.12,
      "text": "两类各自淘汰",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 397.12,
      "end": 398.9,
      "text": "最久没被用到的先删",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 398.9,
      "end": 400.38,
      "text": "硬盘配的够大",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 400.38,
      "end": 402.44,
      "text": "两类都能留72小时以上",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 402.44,
      "end": 403.3,
      "text": "就这样",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11165316899617513,
      "no_speech_prob": 1.520311133929031e-11,
      "compression_ratio": 1.32
    },
    {
      "start": 403.3,
      "end": 406.94,
      "text": "滑动窗口KV还是占了持久化缓存将近一半",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 406.94,
      "end": 408.06,
      "text": "它没有压缩",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 408.06,
      "end": 409.94,
      "text": "每个位置128个Token",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 409.94,
      "end": 412.74,
      "text": "每一层各一份的8比特KV原样存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 412.74,
      "end": 415.12,
      "text": "多轮对话每轮都要存一份",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 415.12,
      "end": 417.94,
      "text": "轮次多每轮短分数就多",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 417.94,
      "end": 419.96,
      "text": "问题处在访问模式",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 419.96,
      "end": 422.18,
      "text": "全局KV是常委复用",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 422.18,
      "end": 424.52,
      "text": "一天以后用户回来还会命中",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 424.52,
      "end": 427.1,
      "text": "滑动窗口KV只在一次绘画里",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 427.1,
      "end": 428.68,
      "text": "几分钟之内被用到",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 428.68,
      "end": 430.96,
      "text": "绘画一结束或者进到下一轮",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 430.96,
      "end": 432.46,
      "text": "它就再也不会被用到",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10093643105095201,
      "no_speech_prob": 1.3842881702030851e-11,
      "compression_ratio": 1.279742765273312
    },
    {
      "start": 432.46,
      "end": 436.66,
      "text": "把一份分钟级的东西放进保留72小时的缓存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 436.66,
      "end": 437.9,
      "text": "又贵又没用",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 437.9,
      "end": 440.32,
      "text": "V4的报告提过干脆不存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 440.32,
      "end": 441.26,
      "text": "要用再算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 441.26,
      "end": 444.02,
      "text": "叫做Zero SWA Catching",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 444.02,
      "end": 445.0,
      "text": "可精确重算",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 445.0,
      "end": 448.42,
      "text": "要按层数乘128个token跑一遍完整前向",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 448.42,
      "end": 449.58,
      "text": "线上扛不住",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 449.58,
      "end": 451.44,
      "text": "这个版本改了两条",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 451.44,
      "end": 452.46,
      "text": "第一条",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 452.46,
      "end": 454.56,
      "text": "滑动窗口KV不再进硬盘",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 454.56,
      "end": 456.1,
      "text": "改放进一个内存池",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 456.1,
      "end": 460.1,
      "text": "所有机器各拿出10%的内存拼成一个池子",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.10403958073368778,
      "no_speech_prob": 1.20270980674686e-11,
      "compression_ratio": 1.1331168831168832
    },
    {
      "start": 460.1,
      "end": 462.6,
      "text": "注意是机器的内存不是显存",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 462.6,
      "end": 464.78,
      "text": "池子总量比硬盘小得多",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 464.78,
      "end": 466.32,
      "text": "但有效期只有几分钟",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 466.32,
      "end": 468.16,
      "text": "过期立刻腾给新绘画",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 468.16,
      "end": 470.86,
      "text": "真实负载下这种高周转",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 470.86,
      "end": 472.98,
      "text": "够服务绝大多数正在进行的绘画",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 472.98,
      "end": 475.16,
      "text": "全局KV照旧流硬盘",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 475.16,
      "end": 476.5,
      "text": "保证72小时",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 476.5,
      "end": 480.0,
      "text": "第二条池子小必然有未命中",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 480.0,
      "end": 483.48,
      "text": "未命中指的是全局KV硬盘上还在",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 483.48,
      "end": 486.36,
      "text": "滑动窗口KV在池子里已经过期",
      "chunk": 0,
      "language": "zh",
      "avg_logprob": -0.11575499773025513,
      "no_speech_prob": 1.443272064360901e-11,
      "compression_ratio": 1.2981132075471697
    },
    {
      "start": 501.753,
      "end": 503.173,
      "text": "编码器也是20层",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 503.173,
      "end": 506.073,
      "text": "精确补齐同样要2560个",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 506.073,
      "end": 508.333,
      "text": "现在只回放128个",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 508.333,
      "end": 511.053,
      "text": "补出来的窗口KV是近似的",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 511.053,
      "end": 512.333,
      "text": "缓存到哪里为止",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 512.333,
      "end": 514.173,
      "text": "回放就从哪里开始",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 514.173,
      "end": 515.373,
      "text": "起点不同",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 515.373,
      "end": 517.353,
      "text": "补出来的KV就略有不同",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 517.353,
      "end": 520.373,
      "text": "报告说这对回答质量几乎没有影响",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 520.373,
      "end": 521.833,
      "text": "还用了一个说法",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 521.833,
      "end": 523.713,
      "text": "把灾难性的未命中",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 523.713,
      "end": 526.053,
      "text": "变成了优雅又便宜的降级",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 526.053,
      "end": 527.893,
      "text": "正因为兜底便宜",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 527.893,
      "end": 530.753,
      "text": "才敢把滑动窗口KV从硬盘上拿掉",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10747444495726168,
      "no_speech_prob": 1.785294134748483e-11,
      "compression_ratio": 1.2610169491525425
    },
    {
      "start": 530.753,
      "end": 532.673,
      "text": "两种回放别混",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 532.673,
      "end": 535.013,
      "text": "解码器回放每次独入都做",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 535.013,
      "end": 537.073,
      "text": "补的是解码器的窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 537.073,
      "end": 539.193,
      "text": "因为解码器从来不跑独入",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 539.193,
      "end": 541.753,
      "text": "编码器回放只在未命中识做",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 541.753,
      "end": 543.653,
      "text": "补的是编码器的窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 543.653,
      "end": 545.333,
      "text": "因为硬盘上不再存它",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 545.333,
      "end": 548.073,
      "text": "两个都只回放末尾128个",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 548.073,
      "end": 549.453,
      "text": "都基于同一个判断",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 549.453,
      "end": 550.973,
      "text": "窗口的记忆很短",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 550.973,
      "end": 553.533,
      "text": "1%就是这么算出来的",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 553.533,
      "end": 555.033,
      "text": "不存滑动窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 555.033,
      "end": 556.613,
      "text": "19化缓存少一半",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 556.613,
      "end": 558.433,
      "text": "剩下的全局KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 558.433,
      "end": 559.393,
      "text": "上一集讲过",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12686762423128695,
      "no_speech_prob": 1.5151510254773903e-11,
      "compression_ratio": 1.4157303370786516
    },
    {
      "start": 559.393,
      "end": 561.073,
      "text": "是上一代的四分之一",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 561.073,
      "end": 562.853,
      "text": "二分之一乘四分之一",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 562.853,
      "end": 564.113,
      "text": "八分之一",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 564.113,
      "end": 566.673,
      "text": "把三层存储放在一起看就清楚了",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 566.673,
      "end": 569.273,
      "text": "显存里放正在用的全局KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 569.273,
      "end": 571.373,
      "text": "每个Token890字节",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 571.373,
      "end": 574.093,
      "text": "机器内存里放几分钟内会用的",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 574.093,
      "end": 575.113,
      "text": "滑动窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 575.113,
      "end": 575.973,
      "text": "过期就扔",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 575.973,
      "end": 579.253,
      "text": "硬盘上放几天内可能会用的全局KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 579.253,
      "end": 580.433,
      "text": "留72小时",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 580.433,
      "end": 582.733,
      "text": "三层各管一个时间尺度",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 582.733,
      "end": 584.613,
      "text": "每一层只存它该存的",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 584.613,
      "end": 586.853,
      "text": "回到那个50万Token的绘画",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 586.853,
      "end": 587.853,
      "text": "看两个时刻",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 587.853,
      "end": 588.913,
      "text": "第一个时刻",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12841022343925348,
      "no_speech_prob": 1.2313447136924616e-11,
      "compression_ratio": 1.386759581881533
    },
    {
      "start": 588.913,
      "end": 590.933,
      "text": "工具返回一段新内容",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 590.933,
      "end": 593.293,
      "text": "读入直跑编码器20层",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 593.293,
      "end": 594.813,
      "text": "激活80亿参数",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 594.813,
      "end": 597.973,
      "text": "解码器要的全局KV顺手投影出来",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 597.973,
      "end": 600.193,
      "text": "滑动窗口回放128个",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 600.193,
      "end": 601.113,
      "text": "开始生成",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 601.113,
      "end": 602.213,
      "text": "第二个时刻",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 602.213,
      "end": 603.973,
      "text": "第二天用户回来接着聊",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 603.973,
      "end": 607.433,
      "text": "硬盘上取出来的只有全局KV400多兆",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 607.433,
      "end": 610.753,
      "text": "滑动窗口那份现算还是128个",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 610.753,
      "end": 611.973,
      "text": "两处近似",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 611.973,
      "end": 614.893,
      "text": "报告给的结论是性能与基线相当",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 614.893,
      "end": 616.613,
      "text": "读入的计算量减半",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12659840329864322,
      "no_speech_prob": 1.408399438740382e-11,
      "compression_ratio": 1.2411347517730495
    },
    {
      "start": 616.613,
      "end": 621.273,
      "text": "部署上还把视觉编码读入生成分成三段",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 621.273,
      "end": 622.573,
      "text": "各自扩缩容",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 622.573,
      "end": 625.573,
      "text": "读入密集的agent负载可以单独加机器",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 625.573,
      "end": 627.453,
      "text": "多轮短消息的agent绘画",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 627.453,
      "end": 629.713,
      "text": "就是报告点名要服务的场景",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 629.713,
      "end": 631.253,
      "text": "局限一共三条",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 631.253,
      "end": 632.173,
      "text": "第一条",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 632.173,
      "end": 633.933,
      "text": "有界回放是近似",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 633.933,
      "end": 636.653,
      "text": "尤其是缓存和新内容接缝的地方",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 636.653,
      "end": 638.833,
      "text": "报告承认还没充分验证",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 638.833,
      "end": 640.593,
      "text": "接下来要加压力测试",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 640.593,
      "end": 642.733,
      "text": "也要在真实负载里继续观察",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 642.733,
      "end": 644.773,
      "text": "后两条是我自己的判断",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 644.773,
      "end": 645.773,
      "text": "第二条",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.11346972430193866,
      "no_speech_prob": 1.4595399541739162e-11,
      "compression_ratio": 1.2618296529968454
    },
    {
      "start": 645.773,
      "end": 647.533,
      "text": "收益跟场景绑定",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 647.533,
      "end": 650.953,
      "text": "多轮每轮短的agent绘画收益最大",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 650.953,
      "end": 653.933,
      "text": "一次性为一份几十万token的长文档",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 653.933,
      "end": 655.893,
      "text": "省的主要是算力那一半",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 655.893,
      "end": 658.193,
      "text": "硬盘那一半的好处基本用不上",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 658.193,
      "end": 659.333,
      "text": "第三条",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 659.333,
      "end": 661.033,
      "text": "10%的内存池",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 661.033,
      "end": 662.513,
      "text": "几分钟的有效期",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 662.513,
      "end": 664.353,
      "text": "72小时的硬盘保留",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 664.353,
      "end": 667.753,
      "text": "都是按Deepseek自家的线上负载掉出来的",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 667.753,
      "end": 669.073,
      "text": "换一种负载分布",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 669.073,
      "end": 671.793,
      "text": "未命中率和回放开销都要重新测",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 671.793,
      "end": 673.753,
      "text": "回到开头的问题",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 673.753,
      "end": 674.913,
      "text": "算力砍一半",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.1261424500009288,
      "no_speech_prob": 1.371135496808229e-11,
      "compression_ratio": 1.2327044025157232
    },
    {
      "start": 674.913,
      "end": 677.633,
      "text": "靠CED解码器的全局KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 677.633,
      "end": 678.873,
      "text": "从编码器投影",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 678.873,
      "end": 680.313,
      "text": "独入只跑20层",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 680.313,
      "end": 681.793,
      "text": "硬盘砍到1%",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 681.793,
      "end": 683.633,
      "text": "靠不存滑动窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 683.633,
      "end": 685.973,
      "text": "为民众就回放128个补出来",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 685.973,
      "end": 688.853,
      "text": "再乘上上一集全局KV的1%",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 688.853,
      "end": 691.193,
      "text": "两件事背后是同一个判断",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 691.193,
      "end": 692.933,
      "text": "滑动窗口的记忆很短",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 692.933,
      "end": 694.353,
      "text": "短到可以现算",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 694.353,
      "end": 695.693,
      "text": "上一集的共用",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 695.693,
      "end": 697.193,
      "text": "加上这一集的回放",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 697.193,
      "end": 700.253,
      "text": "长上下文的存和算一起压了下来",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 700.253,
      "end": 701.253,
      "text": "成绩还没有掉",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 701.253,
      "end": 703.393,
      "text": "如果这一集让你有点收获",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 703.393,
      "end": 704.873,
      "text": "欢迎点赞转发",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.12767846805533184,
      "no_speech_prob": 1.5808378911463805e-11,
      "compression_ratio": 1.297029702970297
    },
    {
      "start": 704.873,
      "end": 705.353,
      "text": "关注",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 705.353,
      "end": 706.773,
      "text": "评论区也聊聊",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 706.773,
      "end": 707.813,
      "text": "换你来做部署",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 707.813,
      "end": 708.913,
      "text": "滑动窗口KV",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 708.913,
      "end": 710.693,
      "text": "你会存硬盘保命中率",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 710.693,
      "end": 711.533,
      "text": "还是不存",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 711.533,
      "end": 713.053,
      "text": "靠回放节省硬盘",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 713.053,
      "end": 713.713,
      "text": "好",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    },
    {
      "start": 713.713,
      "end": 714.933,
      "text": "我们下期继续",
      "chunk": 1,
      "language": "zh",
      "avg_logprob": -0.10559028915212124,
      "no_speech_prob": 1.379008886248645e-11,
      "compression_ratio": 1.0148148148148148
    }
  ],
  "segmented_transcription": {
    "status": "SEGMENTED_TRANSCRIPT_OK",
    "created_at": "2026-09-18T14:27:44",
    "manifest": "/Users/sagawa/AI_WORK/video_notes/url/20260918_142716__DeepSeek_V4.1_40层模型_读入只跑20层_CED架构与KV_Cache优化/segmented_work/chunk_manifest.json",
    "audio": "/Users/sagawa/AI_WORK/video_notes/url/20260918_142716__DeepSeek_V4.1_40层模型_读入只跑20层_CED架构与KV_Cache优化/audio_16k_mono.wav",
    "audio_sha256": "dfdf995759dfa47dee17ee46f2cbfdd987cd15ef8ff0372ff484d6dab5a7989e",
    "audio_duration": 722.372813,
    "model": "/Users/sagawa/AI_WORK/cache/huggingface/hub/models--mlx-community--whisper-large-v3-turbo/snapshots/a4aaeec0636e6fef84abdcbe3544cb2bf7e9f6fb",
    "language_hint": "zh",
    "condition_on_previous_text": false,
    "chunk_count": 2,
    "resumed_chunks": 0,
    "merged_segment_count": 358,
    "cleaned_segment_count": 358,
    "removed_hallucination_segment_count": 0,
    "final_segment_end": 714.933,
    "global_repeat_run": 1,
    "global_repeat_text": "Hello,大家好,我是Tommy。今天很高興跟大家分享最新的前置內容。",
    "tail_repeat_run": 1,
    "tail_repeat_text": "都是按Deepseek自家的线上负载掉出来的",
    "hallucination_region_count": 0,
    "hallucination_regions": [],
    "qc_ok": true,
    "qc_errors": []
  }
}