1
00:00:00,000 --> 00:00:03,840
Hello,大家好,我是Tommy。今天很高興跟大家分享最新的前置內容。

2
00:00:04,100 --> 00:00:09,220
那上一次我們講過Deepseek V4.1 Flash,怎麼降低KP Cache的存儲開銷。

3
00:00:09,220 --> 00:00:13,320
那麼這一次我們繼續看它怎麼減少脫離計算和緩存成本。

4
00:00:13,560 --> 00:00:19,460
那核心呢就兩件事。第一個就是重新計算上下文的時候呢,只計算前一半的層。

5
00:00:19,460 --> 00:00:22,520
那麼後面一半是直接附用已有的結果。

6
00:00:22,780 --> 00:00:28,920
那第二個呢就是一部分短期緩存不再長期保存,而是在需要時重新計算。

7
00:00:28,920 --> 00:00:29,880
那最终呢

8
00:00:29,880 --> 00:00:31,660
计算量降低约一半

9
00:00:31,660 --> 00:00:34,020
持久化缓存降到原来的1%

10
00:00:34,020 --> 00:00:36,300
同时性能基本不受影响

11
00:00:36,300 --> 00:00:37,200
那么这里呢

12
00:00:37,200 --> 00:00:38,860
最关键的是两个问题

13
00:00:38,860 --> 00:00:39,860
那第一个就是

14
00:00:39,860 --> 00:00:42,640
为什么只计算一半的层就够了

15
00:00:42,640 --> 00:00:43,580
第二个呢

16
00:00:43,580 --> 00:00:45,000
为什么重新计算

17
00:00:45,000 --> 00:00:47,800
反而比把缓存一直存下来更划算

18
00:00:47,800 --> 00:00:50,520
那下面我们就直接来看它的具体设计

19
00:00:50,520 --> 00:00:52,720
上一集那个编码agent

20
00:00:52,720 --> 00:00:54,240
修一个仓库里的bug

21
00:00:54,240 --> 00:00:55,200
跑了三个小时

22
00:00:55,200 --> 00:00:56,920
上下文堆到50万token

23
00:00:56,920 --> 00:00:59,200
上一集算的是它要存多少

24
00:00:59,200 --> 00:01:01,580
40层只存4份KV

25
00:01:01,580 --> 00:01:03,220
每个Token890字节

26
00:01:03,220 --> 00:01:05,320
这一集算的是它要读多少

27
00:01:05,320 --> 00:01:06,980
上一集结尾留了一句

28
00:01:06,980 --> 00:01:08,860
4份里解码器那一份

29
00:01:08,860 --> 00:01:11,000
还是从编码器投影来的

30
00:01:11,000 --> 00:01:12,860
这意味着读入新内容时

31
00:01:12,860 --> 00:01:14,300
只用跑前20层

32
00:01:14,300 --> 00:01:15,520
后20层不用算

33
00:01:15,520 --> 00:01:17,100
这一招叫CED

34
00:01:17,100 --> 00:01:19,200
全称因果编码器解码器

35
00:01:19,200 --> 00:01:20,600
这一集就讲它

36
00:01:20,600 --> 00:01:23,020
它把40层分成前后两半

37
00:01:23,020 --> 00:01:24,620
前20层叫编码器

38
00:01:24,620 --> 00:01:26,060
负责读上下文

39
00:01:26,060 --> 00:01:28,860
后20层叫解码器负责生成

40
00:01:28,860 --> 00:01:30,640
解码器要用的KV

41
00:01:30,640 --> 00:01:32,760
从编码器的输出投影出来

42
00:01:32,760 --> 00:01:34,000
所以读入的时候

43
00:01:34,000 --> 00:01:35,340
后20层可以不跑

44
00:01:35,340 --> 00:01:36,660
为什么能这么干

45
00:01:36,660 --> 00:01:37,760
代价是什么

46
00:01:37,760 --> 00:01:39,220
就是这一集的内容

47
00:01:39,220 --> 00:01:41,540
先看读入这一步为什么贵

48
00:01:41,540 --> 00:01:43,360
Agent每掉一次工具

49
00:01:43,360 --> 00:01:46,060
工具的返回都要接到上下文后面

50
00:01:46,060 --> 00:01:47,840
模型再把新内容读一遍

51
00:01:47,840 --> 00:01:49,820
也就是上一集说过的prefill

52
00:01:49,820 --> 00:01:51,420
有缓存的时候

53
00:01:51,420 --> 00:01:53,040
只读新增的那一段

54
00:01:53,040 --> 00:01:54,700
缓存一旦没命中

55
00:01:54,700 --> 00:01:56,520
整段50万都要重算

56
00:01:56,520 --> 00:01:58,880
一次任务几十次工具调用

57
00:01:58,880 --> 00:02:00,240
就是几十次读入

58
00:02:00,240 --> 00:02:01,100
读一次

59
00:02:01,100 --> 00:02:02,880
40层每一层都要算

60
00:02:02,880 --> 00:02:04,080
上下文越长

61
00:02:04,080 --> 00:02:05,260
这比算力越大

62
00:02:05,260 --> 00:02:07,020
这还只是算力

63
00:02:07,020 --> 00:02:09,440
50万token的KV还得留着

64
00:02:09,440 --> 00:02:10,280
等下一轮

65
00:02:10,280 --> 00:02:12,360
甚至等第二天用户回来接着聊

66
00:02:12,360 --> 00:02:14,760
所以它要从显存落到硬盘上

67
00:02:14,760 --> 00:02:16,260
V4的线上部署里

68
00:02:16,260 --> 00:02:19,180
这份缓存要在硬盘上留72小时以上

69
00:02:19,180 --> 00:02:21,060
显存装不装的下

70
00:02:21,060 --> 00:02:22,500
硬盘存不存的起

71
00:02:22,500 --> 00:02:23,920
机器之间搬不搬的动

72
00:02:23,920 --> 00:02:25,360
三处都是压力

73
00:02:25,360 --> 00:02:27,600
算力和硬盘两笔成本

74
00:02:27,600 --> 00:02:28,540
问题来了

75
00:02:28,540 --> 00:02:29,520
独入这一步

76
00:02:29,520 --> 00:02:31,840
V4.1凭什么算力砍一半

77
00:02:31,840 --> 00:02:32,920
硬盘砍到1%

78
00:02:32,920 --> 00:02:34,120
先说算力

79
00:02:34,120 --> 00:02:36,120
2024年有篇论文叫Yoko

80
00:02:36,120 --> 00:02:37,880
意思是你只缓存一次

81
00:02:37,880 --> 00:02:40,200
它把模型前后两半分开

82
00:02:40,200 --> 00:02:42,400
后半层不再各自算KV

83
00:02:42,400 --> 00:02:44,640
直接用前半层算好的那一份

84
00:02:44,640 --> 00:02:46,440
后半层在独入阶段

85
00:02:46,440 --> 00:02:47,640
就可以整个跳过

86
00:02:47,640 --> 00:02:50,320
代价是后半层共用同一份KV

87
00:02:50,320 --> 00:02:52,140
每一层没法有自己的一份

88
00:02:52,140 --> 00:02:53,840
能存的信息就少了

89
00:02:53,920 --> 00:02:56,360
CD顺着这个思路改了两处

90
00:02:56,360 --> 00:02:58,380
解码器要用的全局KV

91
00:02:58,380 --> 00:03:00,660
拿编码器第20层的输出

92
00:03:00,660 --> 00:03:02,400
乘一个投影矩阵得到

93
00:03:02,400 --> 00:03:03,440
第一处改动

94
00:03:03,440 --> 00:03:05,380
投影矩阵可以逐层不同

95
00:03:05,380 --> 00:03:07,260
每层能有自己的一份KV

96
00:03:07,260 --> 00:03:09,340
容量比YOKO共用一份大

97
00:03:09,340 --> 00:03:10,240
第二处

98
00:03:10,240 --> 00:03:12,100
滑动窗口那条局部分支

99
00:03:12,100 --> 00:03:12,700
不投影

100
00:03:12,700 --> 00:03:14,140
每一层还是自己算

101
00:03:14,140 --> 00:03:16,320
局部信息的计算深度保住了

102
00:03:16,320 --> 00:03:17,800
在V4.1里

103
00:03:17,800 --> 00:03:19,400
配合上一集讲的共用

104
00:03:19,400 --> 00:03:22,040
真正做投影的是解码器那一个负层

105
00:03:22,040 --> 00:03:23,320
其余层沿用它

106
00:03:23,320 --> 00:03:25,820
独入阶段编码器20层算完

107
00:03:25,820 --> 00:03:28,520
解码器要的全局KV顺手投影出来

108
00:03:28,520 --> 00:03:30,240
后20层根本不用跑

109
00:03:30,240 --> 00:03:32,340
算力从40层变20层

110
00:03:32,340 --> 00:03:34,920
激活参数从160亿降到80亿

111
00:03:34,920 --> 00:03:37,160
生成阶段才用满160亿

112
00:03:37,160 --> 00:03:38,580
这里有个缺口要补

113
00:03:38,580 --> 00:03:40,980
滑动窗口KV每一层自己算

114
00:03:40,980 --> 00:03:43,180
可独入阶段解码器根本没跑

115
00:03:43,180 --> 00:03:46,300
解码器20层的滑动窗口KV就是空的

116
00:03:46,300 --> 00:03:47,800
生成第一个Token时

117
00:03:47,800 --> 00:03:51,200
解码器要用到前面128个Token的窗口KV

118
00:03:51,200 --> 00:03:52,360
手里却是空的

119
00:03:52,360 --> 00:03:53,820
想精确补上

120
00:03:53,820 --> 00:03:54,800
就得让解码器

121
00:03:54,800 --> 00:03:57,340
把前面那些token重新算一遍

122
00:03:57,340 --> 00:03:57,800
算多少

123
00:03:57,800 --> 00:03:59,820
窗口是一层套一层的

124
00:03:59,820 --> 00:04:01,760
解码器第一层的一个位置

125
00:04:01,760 --> 00:04:03,620
只看它前面128个

126
00:04:03,620 --> 00:04:05,240
第二层的同一个位置

127
00:04:05,240 --> 00:04:07,900
看的是第一层算出的那128个

128
00:04:07,900 --> 00:04:09,200
而那128个

129
00:04:09,200 --> 00:04:11,500
各自又看过更前面的128个

130
00:04:11,500 --> 00:04:14,720
所以第二层实际牵扯到前面256个

131
00:04:14,720 --> 00:04:16,560
一层多128个

132
00:04:16,560 --> 00:04:19,120
到第二层就是20乘128

133
00:04:19,120 --> 00:04:20,480
2560个

134
00:04:20,480 --> 00:04:21,540
也就是说

135
00:04:21,540 --> 00:04:24,020
要把第20层的窗口KV算准

136
00:04:24,020 --> 00:04:26,500
得从2560个Token之前开始

137
00:04:26,500 --> 00:04:28,300
把解码器重新跑一遍

138
00:04:28,300 --> 00:04:30,540
Agent的场景是多轮短消息

139
00:04:30,540 --> 00:04:32,740
每轮新内容可能就几百个Token

140
00:04:32,740 --> 00:04:35,260
为了补窗口再多跑2560个

141
00:04:35,260 --> 00:04:36,340
这比开销不小

142
00:04:36,340 --> 00:04:37,900
这个版本的做法叫

143
00:04:37,900 --> 00:04:39,800
滑动窗口有界回放

144
00:04:39,800 --> 00:04:41,920
不回退2560个

145
00:04:41,920 --> 00:04:43,940
只回退末尾128个

146
00:04:43,940 --> 00:04:46,500
把这128个Token的编码器输出

147
00:04:46,500 --> 00:04:47,980
送进解码器跑一遍

148
00:04:47,980 --> 00:04:50,700
开销是128个Token乘20层

149
00:04:50,700 --> 00:04:52,380
跟50万笔可以忽略

150
00:04:52,380 --> 00:04:55,340
窗口也只在这128个里面看

151
00:04:55,340 --> 00:04:56,940
再往前的一律不看

152
00:04:56,940 --> 00:04:59,360
因为再往前的token没经过解码器

153
00:04:59,360 --> 00:05:00,320
没有东西可看

154
00:05:00,320 --> 00:05:02,280
回放段里的第一个token

155
00:05:02,280 --> 00:05:03,780
窗口里就只有他自己

156
00:05:03,780 --> 00:05:05,680
越往后能看到的越多

157
00:05:05,680 --> 00:05:08,200
到末尾正好是完整的128个

158
00:05:08,200 --> 00:05:11,340
算出来的滑动窗口KV只拿来生成

159
00:05:11,340 --> 00:05:12,380
不存进缓存

160
00:05:12,380 --> 00:05:13,520
为什么敢

161
00:05:13,520 --> 00:05:16,540
因为窗口的有效视野远比理论知晓

162
00:05:16,540 --> 00:05:19,720
有篇叫PowerAttention的工作专门测过这件事

163
00:05:19,720 --> 00:05:21,580
理论上第20层的视野

164
00:05:21,580 --> 00:05:23,680
能到2560个token之外

165
00:05:23,680 --> 00:05:25,260
实际上远处的信息

166
00:05:25,260 --> 00:05:26,300
一层一层传过来

167
00:05:26,300 --> 00:05:27,980
早就衰减到可以忽略

168
00:05:27,980 --> 00:05:29,060
远处的东西

169
00:05:29,060 --> 00:05:30,740
本来就归全局分支管

170
00:05:30,740 --> 00:05:32,600
窗口只负责眼前这一段

171
00:05:32,600 --> 00:05:34,040
这样补出来的KV

172
00:05:34,040 --> 00:05:35,340
和完整跑一遍

173
00:05:35,340 --> 00:05:36,160
不完全一样

174
00:05:36,160 --> 00:05:38,100
报告说实测几乎不掉点

175
00:05:38,100 --> 00:05:39,120
为了保险

176
00:05:39,120 --> 00:05:40,020
后训练时

177
00:05:40,020 --> 00:05:41,760
把同样的回放模拟进去

178
00:05:41,760 --> 00:05:44,180
模型是在这种近似下练出来的

179
00:05:44,180 --> 00:05:45,440
顺便说一句

180
00:05:45,440 --> 00:05:47,580
这两集提到的前变量

181
00:05:47,580 --> 00:05:48,520
稀疏注意力

182
00:05:48,520 --> 00:05:50,360
后训练里的这种训练感知

183
00:05:50,360 --> 00:05:52,520
都出自DeepSeek这套技术战

184
00:05:52,520 --> 00:05:55,040
我做了一门DeepSeek专题的实战课

185
00:05:55,040 --> 00:05:58,860
从MLA MOE ROPE这些核心机制

186
00:05:58,860 --> 00:06:00,820
讲到GRPO对齐训练

187
00:06:00,820 --> 00:06:03,300
再到VLLM SGLAN固鼓鼠

188
00:06:03,300 --> 00:06:05,380
共计11张配项目代码

189
00:06:05,380 --> 00:06:07,480
想把这两集背后的底子补齐

190
00:06:07,480 --> 00:06:09,780
链接在简介和置顶评论里

191
00:06:09,780 --> 00:06:11,020
算力说完

192
00:06:11,020 --> 00:06:11,740
说硬盘

193
00:06:11,740 --> 00:06:13,400
这一半省得更巧

194
00:06:13,400 --> 00:06:15,520
先看V4是怎么存的

195
00:06:15,520 --> 00:06:17,340
落到硬盘的缓存

196
00:06:17,340 --> 00:06:18,640
叫持久化缓存

197
00:06:18,640 --> 00:06:19,500
分两类

198
00:06:19,500 --> 00:06:21,540
全局KV整段存

199
00:06:21,540 --> 00:06:24,380
只要新请求的开头和缓存里的一样

200
00:06:24,380 --> 00:06:25,600
就整段复用

201
00:06:25,600 --> 00:06:27,880
滑动窗口KV不整段存

202
00:06:27,880 --> 00:06:30,320
只在两个位置各存128个

203
00:06:30,320 --> 00:06:31,980
一段提示词的末尾

204
00:06:31,980 --> 00:06:33,300
一段回答的末尾

205
00:06:33,300 --> 00:06:35,700
为的是重新生成和多轮接续

206
00:06:35,700 --> 00:06:37,120
两类各自淘汰

207
00:06:37,120 --> 00:06:38,900
最久没被用到的先删

208
00:06:38,900 --> 00:06:40,380
硬盘配的够大

209
00:06:40,380 --> 00:06:42,440
两类都能留72小时以上

210
00:06:42,440 --> 00:06:43,300
就这样

211
00:06:43,300 --> 00:06:46,940
滑动窗口KV还是占了持久化缓存将近一半

212
00:06:46,940 --> 00:06:48,060
它没有压缩

213
00:06:48,060 --> 00:06:49,940
每个位置128个Token

214
00:06:49,940 --> 00:06:52,740
每一层各一份的8比特KV原样存

215
00:06:52,740 --> 00:06:55,120
多轮对话每轮都要存一份

216
00:06:55,120 --> 00:06:57,940
轮次多每轮短分数就多

217
00:06:57,940 --> 00:06:59,960
问题处在访问模式

218
00:06:59,960 --> 00:07:02,180
全局KV是常委复用

219
00:07:02,180 --> 00:07:04,520
一天以后用户回来还会命中

220
00:07:04,520 --> 00:07:07,100
滑动窗口KV只在一次绘画里

221
00:07:07,100 --> 00:07:08,680
几分钟之内被用到

222
00:07:08,680 --> 00:07:10,960
绘画一结束或者进到下一轮

223
00:07:10,960 --> 00:07:12,460
它就再也不会被用到

224
00:07:12,460 --> 00:07:16,660
把一份分钟级的东西放进保留72小时的缓存

225
00:07:16,660 --> 00:07:17,900
又贵又没用

226
00:07:17,900 --> 00:07:20,320
V4的报告提过干脆不存

227
00:07:20,320 --> 00:07:21,260
要用再算

228
00:07:21,260 --> 00:07:24,020
叫做Zero SWA Catching

229
00:07:24,020 --> 00:07:25,000
可精确重算

230
00:07:25,000 --> 00:07:28,420
要按层数乘128个token跑一遍完整前向

231
00:07:28,420 --> 00:07:29,580
线上扛不住

232
00:07:29,580 --> 00:07:31,440
这个版本改了两条

233
00:07:31,440 --> 00:07:32,460
第一条

234
00:07:32,460 --> 00:07:34,560
滑动窗口KV不再进硬盘

235
00:07:34,560 --> 00:07:36,100
改放进一个内存池

236
00:07:36,100 --> 00:07:40,100
所有机器各拿出10%的内存拼成一个池子

237
00:07:40,100 --> 00:07:42,600
注意是机器的内存不是显存

238
00:07:42,600 --> 00:07:44,780
池子总量比硬盘小得多

239
00:07:44,780 --> 00:07:46,320
但有效期只有几分钟

240
00:07:46,320 --> 00:07:48,160
过期立刻腾给新绘画

241
00:07:48,160 --> 00:07:50,860
真实负载下这种高周转

242
00:07:50,860 --> 00:07:52,980
够服务绝大多数正在进行的绘画

243
00:07:52,980 --> 00:07:55,160
全局KV照旧流硬盘

244
00:07:55,160 --> 00:07:56,500
保证72小时

245
00:07:56,500 --> 00:08:00,000
第二条池子小必然有未命中

246
00:08:00,000 --> 00:08:03,480
未命中指的是全局KV硬盘上还在

247
00:08:03,480 --> 00:08:06,360
滑动窗口KV在池子里已经过期

248
00:08:21,753 --> 00:08:23,173
编码器也是20层

249
00:08:23,173 --> 00:08:26,073
精确补齐同样要2560个

250
00:08:26,073 --> 00:08:28,333
现在只回放128个

251
00:08:28,333 --> 00:08:31,053
补出来的窗口KV是近似的

252
00:08:31,053 --> 00:08:32,333
缓存到哪里为止

253
00:08:32,333 --> 00:08:34,173
回放就从哪里开始

254
00:08:34,173 --> 00:08:35,373
起点不同

255
00:08:35,373 --> 00:08:37,353
补出来的KV就略有不同

256
00:08:37,353 --> 00:08:40,373
报告说这对回答质量几乎没有影响

257
00:08:40,373 --> 00:08:41,833
还用了一个说法

258
00:08:41,833 --> 00:08:43,713
把灾难性的未命中

259
00:08:43,713 --> 00:08:46,053
变成了优雅又便宜的降级

260
00:08:46,053 --> 00:08:47,893
正因为兜底便宜

261
00:08:47,893 --> 00:08:50,753
才敢把滑动窗口KV从硬盘上拿掉

262
00:08:50,753 --> 00:08:52,673
两种回放别混

263
00:08:52,673 --> 00:08:55,013
解码器回放每次独入都做

264
00:08:55,013 --> 00:08:57,073
补的是解码器的窗口KV

265
00:08:57,073 --> 00:08:59,193
因为解码器从来不跑独入

266
00:08:59,193 --> 00:09:01,753
编码器回放只在未命中识做

267
00:09:01,753 --> 00:09:03,653
补的是编码器的窗口KV

268
00:09:03,653 --> 00:09:05,333
因为硬盘上不再存它

269
00:09:05,333 --> 00:09:08,073
两个都只回放末尾128个

270
00:09:08,073 --> 00:09:09,453
都基于同一个判断

271
00:09:09,453 --> 00:09:10,973
窗口的记忆很短

272
00:09:10,973 --> 00:09:13,533
1%就是这么算出来的

273
00:09:13,533 --> 00:09:15,033
不存滑动窗口KV

274
00:09:15,033 --> 00:09:16,613
19化缓存少一半

275
00:09:16,613 --> 00:09:18,433
剩下的全局KV

276
00:09:18,433 --> 00:09:19,393
上一集讲过

277
00:09:19,393 --> 00:09:21,073
是上一代的四分之一

278
00:09:21,073 --> 00:09:22,853
二分之一乘四分之一

279
00:09:22,853 --> 00:09:24,113
八分之一

280
00:09:24,113 --> 00:09:26,673
把三层存储放在一起看就清楚了

281
00:09:26,673 --> 00:09:29,273
显存里放正在用的全局KV

282
00:09:29,273 --> 00:09:31,373
每个Token890字节

283
00:09:31,373 --> 00:09:34,093
机器内存里放几分钟内会用的

284
00:09:34,093 --> 00:09:35,113
滑动窗口KV

285
00:09:35,113 --> 00:09:35,973
过期就扔

286
00:09:35,973 --> 00:09:39,253
硬盘上放几天内可能会用的全局KV

287
00:09:39,253 --> 00:09:40,433
留72小时

288
00:09:40,433 --> 00:09:42,733
三层各管一个时间尺度

289
00:09:42,733 --> 00:09:44,613
每一层只存它该存的

290
00:09:44,613 --> 00:09:46,853
回到那个50万Token的绘画

291
00:09:46,853 --> 00:09:47,853
看两个时刻

292
00:09:47,853 --> 00:09:48,913
第一个时刻

293
00:09:48,913 --> 00:09:50,933
工具返回一段新内容

294
00:09:50,933 --> 00:09:53,293
读入直跑编码器20层

295
00:09:53,293 --> 00:09:54,813
激活80亿参数

296
00:09:54,813 --> 00:09:57,973
解码器要的全局KV顺手投影出来

297
00:09:57,973 --> 00:10:00,193
滑动窗口回放128个

298
00:10:00,193 --> 00:10:01,113
开始生成

299
00:10:01,113 --> 00:10:02,213
第二个时刻

300
00:10:02,213 --> 00:10:03,973
第二天用户回来接着聊

301
00:10:03,973 --> 00:10:07,433
硬盘上取出来的只有全局KV400多兆

302
00:10:07,433 --> 00:10:10,753
滑动窗口那份现算还是128个

303
00:10:10,753 --> 00:10:11,973
两处近似

304
00:10:11,973 --> 00:10:14,893
报告给的结论是性能与基线相当

305
00:10:14,893 --> 00:10:16,613
读入的计算量减半

306
00:10:16,613 --> 00:10:21,273
部署上还把视觉编码读入生成分成三段

307
00:10:21,273 --> 00:10:22,573
各自扩缩容

308
00:10:22,573 --> 00:10:25,573
读入密集的agent负载可以单独加机器

309
00:10:25,573 --> 00:10:27,453
多轮短消息的agent绘画

310
00:10:27,453 --> 00:10:29,713
就是报告点名要服务的场景

311
00:10:29,713 --> 00:10:31,253
局限一共三条

312
00:10:31,253 --> 00:10:32,173
第一条

313
00:10:32,173 --> 00:10:33,933
有界回放是近似

314
00:10:33,933 --> 00:10:36,653
尤其是缓存和新内容接缝的地方

315
00:10:36,653 --> 00:10:38,833
报告承认还没充分验证

316
00:10:38,833 --> 00:10:40,593
接下来要加压力测试

317
00:10:40,593 --> 00:10:42,733
也要在真实负载里继续观察

318
00:10:42,733 --> 00:10:44,773
后两条是我自己的判断

319
00:10:44,773 --> 00:10:45,773
第二条

320
00:10:45,773 --> 00:10:47,533
收益跟场景绑定

321
00:10:47,533 --> 00:10:50,953
多轮每轮短的agent绘画收益最大

322
00:10:50,953 --> 00:10:53,933
一次性为一份几十万token的长文档

323
00:10:53,933 --> 00:10:55,893
省的主要是算力那一半

324
00:10:55,893 --> 00:10:58,193
硬盘那一半的好处基本用不上

325
00:10:58,193 --> 00:10:59,333
第三条

326
00:10:59,333 --> 00:11:01,033
10%的内存池

327
00:11:01,033 --> 00:11:02,513
几分钟的有效期

328
00:11:02,513 --> 00:11:04,353
72小时的硬盘保留

329
00:11:04,353 --> 00:11:07,753
都是按Deepseek自家的线上负载掉出来的

330
00:11:07,753 --> 00:11:09,073
换一种负载分布

331
00:11:09,073 --> 00:11:11,793
未命中率和回放开销都要重新测

332
00:11:11,793 --> 00:11:13,753
回到开头的问题

333
00:11:13,753 --> 00:11:14,913
算力砍一半

334
00:11:14,913 --> 00:11:17,633
靠CED解码器的全局KV

335
00:11:17,633 --> 00:11:18,873
从编码器投影

336
00:11:18,873 --> 00:11:20,313
独入只跑20层

337
00:11:20,313 --> 00:11:21,793
硬盘砍到1%

338
00:11:21,793 --> 00:11:23,633
靠不存滑动窗口KV

339
00:11:23,633 --> 00:11:25,973
为民众就回放128个补出来

340
00:11:25,973 --> 00:11:28,853
再乘上上一集全局KV的1%

341
00:11:28,853 --> 00:11:31,193
两件事背后是同一个判断

342
00:11:31,193 --> 00:11:32,933
滑动窗口的记忆很短

343
00:11:32,933 --> 00:11:34,353
短到可以现算

344
00:11:34,353 --> 00:11:35,693
上一集的共用

345
00:11:35,693 --> 00:11:37,193
加上这一集的回放

346
00:11:37,193 --> 00:11:40,253
长上下文的存和算一起压了下来

347
00:11:40,253 --> 00:11:41,253
成绩还没有掉

348
00:11:41,253 --> 00:11:43,393
如果这一集让你有点收获

349
00:11:43,393 --> 00:11:44,873
欢迎点赞转发

350
00:11:44,873 --> 00:11:45,353
关注

351
00:11:45,353 --> 00:11:46,773
评论区也聊聊

352
00:11:46,773 --> 00:11:47,813
换你来做部署

353
00:11:47,813 --> 00:11:48,913
滑动窗口KV

354
00:11:48,913 --> 00:11:50,693
你会存硬盘保命中率

355
00:11:50,693 --> 00:11:51,533
还是不存

356
00:11:51,533 --> 00:11:53,053
靠回放节省硬盘

357
00:11:53,053 --> 00:11:53,713
好

358
00:11:53,713 --> 00:11:54,933
我们下期继续