{
  "text": "开放了千万3.8RGB的模型权重\n很多人看到RGB之后的第一个问题就是\n这东西\n我的Mac到底能不能跑呢\n有人看到量化以后\n只需要十几GB的内存\n觉得自己的24GBMac应该也没有什么问题\n结果模型下载完以后\n要么加载失败\n要么刚开始生成\n系统内存就直接爆掉\n有人花了半天时间把模型跑起来\n最后发现生成速度只有每秒八九个Token\n打开MTP之后不仅没有加速\n反而更慢了\n所以今天这支影片\n我会从零开始\n带你在Mac上完整部署一次\n千万3.8RGB\n我们会先搞清楚RGB到底代表什么\n它和Cloud GPT这些B原模型大概处于什么水平\n然后算清楚模型运行时到底会占用多少内存\n接下来我会带你选择正确的MLX量化版本\n使用Hugging Face的并行下载工具\n把模型下载到本地\n先跑出一个没有任何加速的基础速度\n最后再安装DFlesh 2\n在下载模型之前\n我们首先得搞清楚一件事\n前往3.827B里面的这个27B到底是什么意思\n这里的B是Billing\n也就是实义\n27B代表这个语言的模型部分\n大约拥有271个参数\n你可以把参数理解成\n有模型训练完成以后保留下来的271个数字\n模型每生成一个token都要读取这些数字进行计算\n然后判断下一个token最有可能是什么\n如果想象的更具体一点\n你可以把它理解成一台拥有270亿个旋钮的巨大机器\n训练模型的过程就是不断调整这270亿个旋钮\n而我们现在进行的本地推理\n就是把这些调整好的旋钮加载到内存里\n然后用它们来生成答案\n这里还有一个非常重要的概念\n也就是说\n千问3.827B是一个Distance模型\n也就是稠密模型\n而不是MROE混合专家模型\n有些模型名称可能写着总参数35B\n但是后面还有一个A3B\n所谓的A3B呢\n这个A3B的意思就是\n它虽然一共有350亿参数\n但每次生成Token的时候\n大约只有30亿参数会被激活\n前往3.827B不一样\n它没有A几B的后缀\n它的语言模型拥有270亿参数\n每生成一个Token\n那么就代表着\n基本要经过这270亿参数的计算\n所以它的总参数大约是27B\n激活参数同样也是27B\n这也是为什么同样叫做30B左右的模型\n千万3.827B可能比一些35B A3B的MOE模型大很多\n当然稠密模型也不是只有缺点\n它的优势是所有能力都集中在统一套参数里\n通常在输出稳定性\n复杂任务和知识调用上会更加均衡\n如果可以把本地运行的模型简单付成三个类别\n3B到8B通常属于轻量模型\n这类模型速度很快\n但复杂任务比较容易走偏\n14B到30B可以看成本地大模型目前最实用的高质量区间\n能力已经足以处理比较复杂的编程文章生成和工具调用\n以及agent任务\n70B以上的稠密模型能力通常会更加稳定\n但需要的内存和内存带宽也会明显上升\n对于大部分个人用户来说\n部署成本其实是比较高的\n所以千万3.8RGB\n基本属于个人电脑\n还能够比较现实的部署\n同时能力又足够强的高端本地模型\n那它跟市面上的B原模型相比怎么样呢\n首先我们不能直接拿RGB\n去和Cloud和GPT的比较参数量\n因为大部分B原模型\n不会公开完整参数量和激活参数\n即使参数量一样\n训练数据模型架构后训练和推理预算不同\n最终能力也可能差很多\n比较稳妥的说法是\n千问3.8RGB已经进入了可以和B原旗舰模型\n在部分任务上正面竞争的范围\n但它还不能全面替代B原旗舰\n以千问官方测试为例\n可以看到\n在Batch Pro里\n千万3.8\nRTB是61.7\nOPLA 4.6 MAX是53.4\nTerminal Bench则是73对78.2\nHLE是30.8对40\n那么这组数据就可以说明\n它在某些编程和工具调用任务上\n可以接近甚至超过B1\n但是复杂开放式推理\n知识广度和常融稳定性呢\nB原旗舰通常还是更强一些\n而且这是签问官方公布的测试\n它不是不同模型可能使用了不同的harness\n所以呢\n这也包含官方自己的测试集\n可以作为一部分定位参考\n但不能因为某一个分数更高\n就是说签问已经全面超越了cloud\n对我来说\n最准确的定位是\n它不是B原旗舰的完整替代品\n但已经是一款可以真正承担生产任务的高端本地模型\n接下来我们来算一下它到底需要多少内存\n很多人看到27B会直接认为它需要27G的内存\n这个说法只在非常特定的条件下才大致成立\n如果使用BF16一个参数需要两个字节\n270一个参数乘以两个字节的话\n光语言模型原始权重就要大约54G\n这还没有算计算系统\n缓存和退理过程中的临时内存\n所以64G Mac虽然理论上可以放下BF16权重\n但运行语量其实非常有限\n量化的作用呢\n就是使用更少的位数保存这些参数\n8bit理论权重大约27G\n4bit理论上大约13.5G\n实际MLX版本还包含量化参数\n词表和配置\n8bit纯文字版约28.6g\n带视觉能力约29.5g\n4bit版本大约16g\n那么从这些地方的这些数字\n只代表模型权重\n并不是模型启动后的总内存占用\n每次运行时\n内存里至少还要放进五类东西\n第一类也就是最大的一类\n就是模型群众\n运行8bit版本光这一向就已经接近29G\n而且启动后基本都要进入Mac统一内存\n第二类是KVKISH和线性注意力机\n模型需要保存已经处理过的上下纹\n否则每生成一个Token都需要重新计算前纹\nKVKISH和上下纹长度直接下纹\n前往原生支持约26万Token\n但支持不代表你的Mac适合直接开满\n粗略计算\n32K上下纹\n仅标准注意力\n64K约4G\n128K约8G\n256K就要16G\n此外还没包含线性注意力状态\n如果平常只是聊天或者处理普通代码任务\n先从16K和32K开始\n通常就已经足够了\n第三类是MLX和META运行时使用的计算空间\n包括临时激活值和计算缓冲区\n那么它不是固定数字\n会随输入程度和并发变化\n尤其是在第一次读取大段Prompt\n也就是Prefield阶段\n内存占用可能突然上升\n第四类是视觉编码器\n那么只做文字写代码或运行Agent\n可以选择Text Only\n需要读取图片或视频才选择完整多模态版本\n第五类是D Flash 2草稿模型\n它不是普通开关\n还需要一个配套的草稿模型\n本身呢\n它大约21个参数\n并额外占用缓存和缓冲区\n所以最终的内存帐\n不是27B等于27G\n而是模型权重加Cache\n加运行缓冲加系统占用\n再加DFlex2草稿模型\n如果想运行8比特版本\n那么48G统一内存\n可以作为比较现实的起点\n64G会更加舒服\n32G不是绝对加载不了\n但余量非常小\n如果Mac只有24G或32G的话\n我建议更加直接使用4bit版本是比较好的\n模型文件大约16G\n给系统Cache和D-Flash 2留下的空间会宽松很多\nMac运行本地模型常见的了线\nGGUF加LamaCPP1GMLX\n这次选择MLX\n因为它针对苹果电脑\n有比较特殊的一些优化\n而且后面的DFLASH2工具\n可以直接运行MLX权重\n我这次选择\n千万3.8RGB的MLX 8bit版本\n如果内存比较小\n把8bit换成对应的4bit仓库即可\n其他流程基本是一样的\n模型可以直接去HarkinFace下载\n那么下载的时候\n不建议打开浏览器\n一个文件一个文件点击\n大模型通常拆成多个权重封片\n浏览器下载非常容易中断\n也可能漏掉配置文件\n那么这里更推荐用HarkinFace官方的HF命令行工具\n使用新版的HUB\n使用XNET后端\n它可以支持并行下载\n就造成你的transfer已经逐步被淘汰了\n那么我们这里首先安装最新版Hagenface CLI\n接下来开启高性能模式\n然后把模型下载到指定目录\n那么高性能模式会提高并发\n尽可能利用网络和CPU资源\n模型比较大\n下载前记得磁盘至少还有几十G的空余空间\n下载完成之后\n我们先别着急安装D-Flash 2\n先跑出没有加速的基础版本\n建议给本地模型单独创建\nPython虚拟环境\n避免不同的项目依赖会互相污染\n创建环境激活环境\n以及安装MXM和MX\n第一次测试不要开超长的上下文\n不要开D-Flash 2\n同时也不要修改一大堆采样参数\n只指定模型路径\n固定Prompt\n生成几百个Token\n运行最基础的生成命令\n运行完成之后呢\n我们可以把模型加载时间\nPrompt处理速度\n守Token等待时间和生成Token速度\n这四个数字记录下来\n如果没有基础数据\n后面看到每每20几个Token\n也不知道到底快了多少\n基础9变27是3倍\n那么基础20变22可能就不值得增加内存和复杂度\n确认基础模型稳定之后呢\n我们就可以安装D-Flash 2\nD-Flash 2本质上是一种推测解码技术\n正常情况下千万3.827B每生成一个Token的话\n都要让完整270亿参数计算一次\n那么生成10个Token就要连续做10次完整的推理\nDflash2让一个大约20亿参数的草稿模型一次性并行预测接下来的一组Token\n然后在27B主模型一次检查这一整组答案\n如果草稿模型猜的比较准\n主模型一次计算就可以接受多个Token\n原本连续走五六四的流程\n现在可能只需要一次验证\n草稿模型只负责猜\n最终决定权依然在27B主模型手上\n贪心解码下最终输出\n可以基础模型一致\n随机采样则保留主模型原本的分布\n在Mac上\n比较直接的实现是MX的D Spark\n那么这个虽然名字写着D Spark\n但它同时支持D Spark D Flash和D Flash 2\n在用的时候对于千万3.8 27B Ultra模式会直接使用D Flash 2\n安装命令也非常简单\n屏幕上的安装命令\n安装完成以后不要凭感觉判断有没有变快\n直接运行项目自带的Benchmark\n使用三轮测试\n第一次运行会自动下载匹配的DFlex2草稿模型\n并根据这台Mac的实际成本测试\n合适的推理长度\nM1 M3 M4 M5的最佳参数呢\n它是不一定一样的\n如果你想直接生成内容的话\n建议用这个命令\n可以使用MXD Spark Generate\n默认AUTO模式会为\n签问3.8RCB匹配DFLASH2\n再用完全相同的Prompt测试\n对比基础速度\nDFLASH2速度接受长度和内存占用\n那么比下来之后\n项目在特定的M4 Pro测试中\n8bit的基础约8.4Token每秒\nDFLASH2平均约30.5Token每秒\n不同任务约2.8-4倍\n但这不是你的Mac一定能达到的速度\n因为DFLASH2不是打开以后它就一定变快\n如果草稿接受率低的话\n主模型会不断拒绝和重新生成\n草稿计算反而成为额外的负担\n对于只有几十个Token的短回答\n模型刚加载完成和校准任务就结束了\n实际体感提升可能不是很明显\nDFLASH2还会增加内存占用\n所以呢\n如果巴比特主\n所以如果8bit主模型以及频繁swap\n那么再加载草稿模型\n最终速度反而可能更慢\n所以要用自己最常执行的任务去测试一下\n写代码就测真实代码修改\n生成口播稿就连续输出一两千的token\n运行agent就测试完整工具调用\n最后一步是把模型启动成API\nMLXDspark可以兼容OpenAI\n那么和Astorbit格式的本地服务\n启动命令是MLXDsparkServe\n默认是开在本级8080端口去提供接口的\n客户端BaseURL设置成127.0.18080\n设置好之后\n只在自己电脑使用\n可以不设置真正的API Key\n需要局域网访问的时候\n要调整监听地址\n检查防火墙\n但不要把无建权接口直接暴露到公网\n到这里我们完成了整套部署流程\n第一\n27B 代表约270亿参数\n而且它是全量激活的稠密模型\n第二\nRGB不等于只需RGB内存\n把比特权重接近R9G\n运行时还要加Cache缓冲区系统和草稿模型\n那么第三\n它属于本地模型高端档\n部分编程和Agent任务接近B原起见\n但不是全面的替代\n第四\n不要一上来\n就打开所有加速箱\n先用MLX跑出基础速度\n再用相同Prompt做A-B测试\n决定是否长期使用D-Flash 2\n如果真实任务从每秒9个Token\n能够提高到2-30个的话\n而且内存稳定\n那么它就值得长期去打开\n如果频繁Swap或者主要都是短问答的话\n4bit基础模型可能更舒服一些\n本地部署大模型的价值不是省API费用\n当它变成Mac上的本地服务\n它就可以接入代码Langraph\n自动剪辑系统\n知识库\n以及agent工作流\n真正好的本地布置方案\n不是参数最大\n也不是测试数字最高\n而是在自己的设备上\n质量足够好\n速度能够接受\n内存保持稳定\n并且真的进入日常工作流\n才能长期去使用\n影片领誘到的模型地址安装命令和测试命令\n我都会整理在下方\n如果希望下一期测试4bit 6bit和8bit的品质差距的话\n以及DFlex 2速度也可以留言给我\n喜欢这支影片的话\n记得帮我按赞订阅加分享哦\n这是我持续创作的最大动力\n我们下次再见",
  "language": "multilingual",
  "duration": 1125.935625,
  "segments": [
    {
      "start": 0.0,
      "end": 3.14,
      "text": "开放了千万3.8RGB的模型权重",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 3.14,
      "end": 6.46,
      "text": "很多人看到RGB之后的第一个问题就是",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 6.46,
      "end": 8.12,
      "text": "这东西",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 8.12,
      "end": 11.52,
      "text": "我的Mac到底能不能跑呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 11.52,
      "end": 14.24,
      "text": "有人看到量化以后",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 14.24,
      "end": 16.06,
      "text": "只需要十几GB的内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 16.06,
      "end": 20.92,
      "text": "觉得自己的24GBMac应该也没有什么问题",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 20.92,
      "end": 24.06,
      "text": "结果模型下载完以后",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 24.06,
      "end": 25.78,
      "text": "要么加载失败",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 25.78,
      "end": 27.9,
      "text": "要么刚开始生成",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 27.9,
      "end": 30.36,
      "text": "系统内存就直接爆掉",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 30.36,
      "end": 34.86,
      "text": "有人花了半天时间把模型跑起来",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 34.86,
      "end": 38.6,
      "text": "最后发现生成速度只有每秒八九个Token",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 38.6,
      "end": 41.7,
      "text": "打开MTP之后不仅没有加速",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 41.7,
      "end": 42.98,
      "text": "反而更慢了",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 42.98,
      "end": 45.52,
      "text": "所以今天这支影片",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 45.52,
      "end": 47.66,
      "text": "我会从零开始",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 47.66,
      "end": 50.84,
      "text": "带你在Mac上完整部署一次",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 50.84,
      "end": 52.7,
      "text": "千万3.8RGB",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 52.7,
      "end": 56.1,
      "text": "我们会先搞清楚RGB到底代表什么",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 56.1,
      "end": 60.62,
      "text": "它和Cloud GPT这些B原模型大概处于什么水平",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 60.62,
      "end": 65.88,
      "text": "然后算清楚模型运行时到底会占用多少内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 65.88,
      "end": 71.12,
      "text": "接下来我会带你选择正确的MLX量化版本",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 71.12,
      "end": 73.56,
      "text": "使用Hugging Face的并行下载工具",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 73.56,
      "end": 76.46,
      "text": "把模型下载到本地",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 76.46,
      "end": 81.5,
      "text": "先跑出一个没有任何加速的基础速度",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 81.5,
      "end": 84.12,
      "text": "最后再安装DFlesh 2",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 84.12,
      "end": 86.26,
      "text": "在下载模型之前",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 86.26,
      "end": 88.46,
      "text": "我们首先得搞清楚一件事",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 88.46,
      "end": 93.64,
      "text": "前往3.827B里面的这个27B到底是什么意思",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 93.64,
      "end": 97.58,
      "text": "这里的B是Billing",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 97.58,
      "end": 99.02,
      "text": "也就是实义",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 99.02,
      "end": 102.82,
      "text": "27B代表这个语言的模型部分",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 102.82,
      "end": 106.14,
      "text": "大约拥有271个参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 106.14,
      "end": 108.48,
      "text": "你可以把参数理解成",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 108.48,
      "end": 113.78,
      "text": "有模型训练完成以后保留下来的271个数字",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 113.78,
      "end": 118.6,
      "text": "模型每生成一个token都要读取这些数字进行计算",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 118.6,
      "end": 122.66,
      "text": "然后判断下一个token最有可能是什么",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 122.66,
      "end": 124.72,
      "text": "如果想象的更具体一点",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 124.72,
      "end": 130.9,
      "text": "你可以把它理解成一台拥有270亿个旋钮的巨大机器",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 130.9,
      "end": 135.78,
      "text": "训练模型的过程就是不断调整这270亿个旋钮",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 135.78,
      "end": 138.74,
      "text": "而我们现在进行的本地推理",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 138.74,
      "end": 142.14,
      "text": "就是把这些调整好的旋钮加载到内存里",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 142.14,
      "end": 144.4,
      "text": "然后用它们来生成答案",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 144.4,
      "end": 147.22,
      "text": "这里还有一个非常重要的概念",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 147.22,
      "end": 149.0,
      "text": "也就是说",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 149.0,
      "end": 152.1,
      "text": "千问3.827B是一个Distance模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 152.1,
      "end": 153.82,
      "text": "也就是稠密模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 153.82,
      "end": 156.72,
      "text": "而不是MROE混合专家模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 156.72,
      "end": 161.84,
      "text": "有些模型名称可能写着总参数35B",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 161.84,
      "end": 164.16,
      "text": "但是后面还有一个A3B",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 164.16,
      "end": 165.98,
      "text": "所谓的A3B呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 165.98,
      "end": 167.38,
      "text": "这个A3B的意思就是",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 167.38,
      "end": 169.98,
      "text": "它虽然一共有350亿参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 169.98,
      "end": 171.78,
      "text": "但每次生成Token的时候",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 171.78,
      "end": 174.58,
      "text": "大约只有30亿参数会被激活",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 174.58,
      "end": 177.66,
      "text": "前往3.827B不一样",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 177.66,
      "end": 179.72,
      "text": "它没有A几B的后缀",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 179.72,
      "end": 182.98,
      "text": "它的语言模型拥有270亿参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 182.98,
      "end": 185.02,
      "text": "每生成一个Token",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 185.02,
      "end": 187.8,
      "text": "那么就代表着",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 187.8,
      "end": 190.74,
      "text": "基本要经过这270亿参数的计算",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 190.74,
      "end": 193.72,
      "text": "所以它的总参数大约是27B",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 193.72,
      "end": 198.16,
      "text": "激活参数同样也是27B",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 198.16,
      "end": 203.32,
      "text": "这也是为什么同样叫做30B左右的模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 203.32,
      "end": 208.82,
      "text": "千万3.827B可能比一些35B A3B的MOE模型大很多",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 208.82,
      "end": 211.86,
      "text": "当然稠密模型也不是只有缺点",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 211.86,
      "end": 216.14,
      "text": "它的优势是所有能力都集中在统一套参数里",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 216.14,
      "end": 217.92,
      "text": "通常在输出稳定性",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 217.92,
      "end": 221.06,
      "text": "复杂任务和知识调用上会更加均衡",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 221.06,
      "end": 224.86,
      "text": "如果可以把本地运行的模型简单付成三个类别",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 224.86,
      "end": 227.66,
      "text": "3B到8B通常属于轻量模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 227.66,
      "end": 229.22,
      "text": "这类模型速度很快",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 229.22,
      "end": 231.6,
      "text": "但复杂任务比较容易走偏",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 231.6,
      "end": 237.68,
      "text": "14B到30B可以看成本地大模型目前最实用的高质量区间",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 237.68,
      "end": 242.4,
      "text": "能力已经足以处理比较复杂的编程文章生成和工具调用",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 242.4,
      "end": 243.72,
      "text": "以及agent任务",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 243.72,
      "end": 250.86,
      "text": "70B以上的稠密模型能力通常会更加稳定",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 250.86,
      "end": 255.06,
      "text": "但需要的内存和内存带宽也会明显上升",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 255.06,
      "end": 257.4,
      "text": "对于大部分个人用户来说",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 257.4,
      "end": 259.84,
      "text": "部署成本其实是比较高的",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 259.84,
      "end": 261.96,
      "text": "所以千万3.8RGB",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 261.96,
      "end": 264.54,
      "text": "基本属于个人电脑",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 264.54,
      "end": 266.9,
      "text": "还能够比较现实的部署",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 266.9,
      "end": 270.9,
      "text": "同时能力又足够强的高端本地模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 270.9,
      "end": 274.0,
      "text": "那它跟市面上的B原模型相比怎么样呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 274.0,
      "end": 277.44,
      "text": "首先我们不能直接拿RGB",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 277.44,
      "end": 280.02,
      "text": "去和Cloud和GPT的比较参数量",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 280.02,
      "end": 281.8,
      "text": "因为大部分B原模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 281.8,
      "end": 285.08,
      "text": "不会公开完整参数量和激活参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 285.08,
      "end": 287.72,
      "text": "即使参数量一样",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 287.72,
      "end": 291.8,
      "text": "训练数据模型架构后训练和推理预算不同",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 291.8,
      "end": 294.4,
      "text": "最终能力也可能差很多",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 294.4,
      "end": 296.24,
      "text": "比较稳妥的说法是",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 296.24,
      "end": 300.68,
      "text": "千问3.8RGB已经进入了可以和B原旗舰模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 300.68,
      "end": 302.92,
      "text": "在部分任务上正面竞争的范围",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 302.92,
      "end": 306.94,
      "text": "但它还不能全面替代B原旗舰",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 306.94,
      "end": 311.26,
      "text": "以千问官方测试为例",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 311.26,
      "end": 312.98,
      "text": "可以看到",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 312.98,
      "end": 316.54,
      "text": "在Batch Pro里",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 316.54,
      "end": 317.62,
      "text": "千万3.8",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 317.62,
      "end": 319.08,
      "text": "RTB是61.7",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 319.08,
      "end": 321.46,
      "text": "OPLA 4.6 MAX是53.4",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 321.46,
      "end": 325.62,
      "text": "Terminal Bench则是73对78.2",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 325.62,
      "end": 328.82,
      "text": "HLE是30.8对40",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 328.82,
      "end": 332.26,
      "text": "那么这组数据就可以说明",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 332.26,
      "end": 335.06,
      "text": "它在某些编程和工具调用任务上",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 335.06,
      "end": 337.1,
      "text": "可以接近甚至超过B1",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 337.1,
      "end": 339.12,
      "text": "但是复杂开放式推理",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 339.12,
      "end": 341.36,
      "text": "知识广度和常融稳定性呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 341.36,
      "end": 344.04,
      "text": "B原旗舰通常还是更强一些",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 344.04,
      "end": 348.32,
      "text": "而且这是签问官方公布的测试",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 348.32,
      "end": 352.18,
      "text": "它不是不同模型可能使用了不同的harness",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 352.18,
      "end": 354.72,
      "text": "所以呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 354.72,
      "end": 357.04,
      "text": "这也包含官方自己的测试集",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 357.04,
      "end": 360.36,
      "text": "可以作为一部分定位参考",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 360.36,
      "end": 363.08,
      "text": "但不能因为某一个分数更高",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 363.08,
      "end": 365.32,
      "text": "就是说签问已经全面超越了cloud",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 365.32,
      "end": 366.44,
      "text": "对我来说",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 366.44,
      "end": 367.58,
      "text": "最准确的定位是",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 367.58,
      "end": 370.66,
      "text": "它不是B原旗舰的完整替代品",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 370.66,
      "end": 375.66,
      "text": "但已经是一款可以真正承担生产任务的高端本地模型",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 375.66,
      "end": 378.96,
      "text": "接下来我们来算一下它到底需要多少内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 378.96,
      "end": 383.68,
      "text": "很多人看到27B会直接认为它需要27G的内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 383.68,
      "end": 388.18,
      "text": "这个说法只在非常特定的条件下才大致成立",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 388.18,
      "end": 394.16,
      "text": "如果使用BF16一个参数需要两个字节",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 394.16,
      "end": 396.88,
      "text": "270一个参数乘以两个字节的话",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 396.88,
      "end": 401.48,
      "text": "光语言模型原始权重就要大约54G",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 401.48,
      "end": 403.64,
      "text": "这还没有算计算系统",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 403.64,
      "end": 406.62,
      "text": "缓存和退理过程中的临时内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 406.62,
      "end": 410.54,
      "text": "所以64G Mac虽然理论上可以放下BF16权重",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 410.54,
      "end": 413.1,
      "text": "但运行语量其实非常有限",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 413.1,
      "end": 415.9,
      "text": "量化的作用呢",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 415.9,
      "end": 419.04,
      "text": "就是使用更少的位数保存这些参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 419.04,
      "end": 422.34,
      "text": "8bit理论权重大约27G",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 422.34,
      "end": 426.1,
      "text": "4bit理论上大约13.5G",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 426.1,
      "end": 430.08,
      "text": "实际MLX版本还包含量化参数",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 430.08,
      "end": 431.4,
      "text": "词表和配置",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 431.4,
      "end": 434.14,
      "text": "8bit纯文字版约28.6g",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 434.14,
      "end": 436.42,
      "text": "带视觉能力约29.5g",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 436.42,
      "end": 438.82,
      "text": "4bit版本大约16g",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 438.82,
      "end": 442.12,
      "text": "那么从这些地方的这些数字",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 442.12,
      "end": 444.1,
      "text": "只代表模型权重",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 444.1,
      "end": 447.28,
      "text": "并不是模型启动后的总内存占用",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 447.28,
      "end": 449.14,
      "text": "每次运行时",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 449.14,
      "end": 451.2,
      "text": "内存里至少还要放进五类东西",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 451.2,
      "end": 453.0,
      "text": "第一类也就是最大的一类",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 453.0,
      "end": 455.04,
      "text": "就是模型群众",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 455.04,
      "end": 459.46,
      "text": "运行8bit版本光这一向就已经接近29G",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 459.46,
      "end": 462.66,
      "text": "而且启动后基本都要进入Mac统一内存",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 462.66,
      "end": 467.06,
      "text": "第二类是KVKISH和线性注意力机",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 467.06,
      "end": 470.16,
      "text": "模型需要保存已经处理过的上下纹",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 470.16,
      "end": 476.7,
      "text": "否则每生成一个Token都需要重新计算前纹",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 476.7,
      "end": 480.0,
      "text": "KVKISH和上下纹长度直接下纹",
      "language": "zh",
      "chunk": 0,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 480.44,
      "end": 483.74,
      "text": "前往原生支持约26万Token",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 483.74,
      "end": 487.8,
      "text": "但支持不代表你的Mac适合直接开满",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 487.8,
      "end": 489.76,
      "text": "粗略计算",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 489.76,
      "end": 491.22,
      "text": "32K上下纹",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 491.22,
      "end": 492.62,
      "text": "仅标准注意力",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 492.62,
      "end": 494.26,
      "text": "64K约4G",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 494.26,
      "end": 496.6,
      "text": "128K约8G",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 496.6,
      "end": 499.12,
      "text": "256K就要16G",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 499.12,
      "end": 502.76,
      "text": "此外还没包含线性注意力状态",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 502.76,
      "end": 506.92,
      "text": "如果平常只是聊天或者处理普通代码任务",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 506.92,
      "end": 509.16,
      "text": "先从16K和32K开始",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 509.16,
      "end": 511.1,
      "text": "通常就已经足够了",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 511.1,
      "end": 515.54,
      "text": "第三类是MLX和META运行时使用的计算空间",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 515.54,
      "end": 518.84,
      "text": "包括临时激活值和计算缓冲区",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 518.84,
      "end": 521.66,
      "text": "那么它不是固定数字",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 521.66,
      "end": 524.02,
      "text": "会随输入程度和并发变化",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 524.02,
      "end": 527.86,
      "text": "尤其是在第一次读取大段Prompt",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 527.86,
      "end": 529.46,
      "text": "也就是Prefield阶段",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 529.46,
      "end": 532.02,
      "text": "内存占用可能突然上升",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 532.02,
      "end": 536.02,
      "text": "第四类是视觉编码器",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 536.02,
      "end": 540.88,
      "text": "那么只做文字写代码或运行Agent",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 540.88,
      "end": 542.3,
      "text": "可以选择Text Only",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 542.3,
      "end": 546.26,
      "text": "需要读取图片或视频才选择完整多模态版本",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 546.26,
      "end": 549.5,
      "text": "第五类是D Flash 2草稿模型",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 549.5,
      "end": 551.02,
      "text": "它不是普通开关",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 551.02,
      "end": 553.22,
      "text": "还需要一个配套的草稿模型",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 553.22,
      "end": 555.34,
      "text": "本身呢",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 555.34,
      "end": 557.02,
      "text": "它大约21个参数",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 557.02,
      "end": 561.48,
      "text": "并额外占用缓存和缓冲区",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 561.48,
      "end": 564.6,
      "text": "所以最终的内存帐",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 564.6,
      "end": 566.38,
      "text": "不是27B等于27G",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 566.38,
      "end": 568.22,
      "text": "而是模型权重加Cache",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 568.22,
      "end": 571.04,
      "text": "加运行缓冲加系统占用",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 571.04,
      "end": 573.84,
      "text": "再加DFlex2草稿模型",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 573.84,
      "end": 577.88,
      "text": "如果想运行8比特版本",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 577.88,
      "end": 580.1,
      "text": "那么48G统一内存",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 580.1,
      "end": 581.82,
      "text": "可以作为比较现实的起点",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 581.82,
      "end": 584.24,
      "text": "64G会更加舒服",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 584.24,
      "end": 587.3,
      "text": "32G不是绝对加载不了",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 587.3,
      "end": 588.78,
      "text": "但余量非常小",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 588.78,
      "end": 591.22,
      "text": "如果Mac只有24G或32G的话",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 591.22,
      "end": 596.98,
      "text": "我建议更加直接使用4bit版本是比较好的",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 596.98,
      "end": 599.98,
      "text": "模型文件大约16G",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 599.98,
      "end": 605.0,
      "text": "给系统Cache和D-Flash 2留下的空间会宽松很多",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 605.0,
      "end": 607.58,
      "text": "Mac运行本地模型常见的了线",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 607.58,
      "end": 610.52,
      "text": "GGUF加LamaCPP1GMLX",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 610.52,
      "end": 613.32,
      "text": "这次选择MLX",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 613.32,
      "end": 614.82,
      "text": "因为它针对苹果电脑",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 614.82,
      "end": 618.24,
      "text": "有比较特殊的一些优化",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 618.24,
      "end": 620.0,
      "text": "而且后面的DFLASH2工具",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 620.0,
      "end": 621.88,
      "text": "可以直接运行MLX权重",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 621.88,
      "end": 625.64,
      "text": "我这次选择",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 625.64,
      "end": 628.88,
      "text": "千万3.8RGB的MLX 8bit版本",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 628.88,
      "end": 630.86,
      "text": "如果内存比较小",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 630.86,
      "end": 634.1,
      "text": "把8bit换成对应的4bit仓库即可",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 634.1,
      "end": 637.6,
      "text": "其他流程基本是一样的",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 637.6,
      "end": 641.36,
      "text": "模型可以直接去HarkinFace下载",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 641.36,
      "end": 642.96,
      "text": "那么下载的时候",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 642.96,
      "end": 644.62,
      "text": "不建议打开浏览器",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 644.62,
      "end": 646.52,
      "text": "一个文件一个文件点击",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 646.52,
      "end": 649.94,
      "text": "大模型通常拆成多个权重封片",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 649.94,
      "end": 652.5,
      "text": "浏览器下载非常容易中断",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 652.5,
      "end": 654.52,
      "text": "也可能漏掉配置文件",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 654.52,
      "end": 659.64,
      "text": "那么这里更推荐用HarkinFace官方的HF命令行工具",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 659.64,
      "end": 662.58,
      "text": "使用新版的HUB",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 662.58,
      "end": 664.9,
      "text": "使用XNET后端",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 664.9,
      "end": 666.8,
      "text": "它可以支持并行下载",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 666.8,
      "end": 671.76,
      "text": "就造成你的transfer已经逐步被淘汰了",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 671.76,
      "end": 675.68,
      "text": "那么我们这里首先安装最新版Hagenface CLI",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 675.68,
      "end": 679.34,
      "text": "接下来开启高性能模式",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 679.34,
      "end": 682.56,
      "text": "然后把模型下载到指定目录",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 682.56,
      "end": 685.96,
      "text": "那么高性能模式会提高并发",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 685.96,
      "end": 688.2,
      "text": "尽可能利用网络和CPU资源",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 688.2,
      "end": 689.26,
      "text": "模型比较大",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 689.26,
      "end": 692.56,
      "text": "下载前记得磁盘至少还有几十G的空余空间",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 692.56,
      "end": 694.0,
      "text": "下载完成之后",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 694.0,
      "end": 696.56,
      "text": "我们先别着急安装D-Flash 2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 696.56,
      "end": 700.16,
      "text": "先跑出没有加速的基础版本",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 700.16,
      "end": 703.16,
      "text": "建议给本地模型单独创建",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 703.16,
      "end": 704.66,
      "text": "Python虚拟环境",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 704.66,
      "end": 708.88,
      "text": "避免不同的项目依赖会互相污染",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 708.88,
      "end": 711.88,
      "text": "创建环境激活环境",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 711.88,
      "end": 714.8,
      "text": "以及安装MXM和MX",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 714.8,
      "end": 718.42,
      "text": "第一次测试不要开超长的上下文",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 718.42,
      "end": 720.52,
      "text": "不要开D-Flash 2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 720.52,
      "end": 724.94,
      "text": "同时也不要修改一大堆采样参数",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 724.94,
      "end": 727.3,
      "text": "只指定模型路径",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 727.3,
      "end": 728.32,
      "text": "固定Prompt",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 728.32,
      "end": 729.96,
      "text": "生成几百个Token",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 729.96,
      "end": 732.12,
      "text": "运行最基础的生成命令",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 732.12,
      "end": 734.2,
      "text": "运行完成之后呢",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 734.2,
      "end": 737.18,
      "text": "我们可以把模型加载时间",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 737.18,
      "end": 738.94,
      "text": "Prompt处理速度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 738.94,
      "end": 741.66,
      "text": "守Token等待时间和生成Token速度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 741.66,
      "end": 743.54,
      "text": "这四个数字记录下来",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 743.54,
      "end": 746.04,
      "text": "如果没有基础数据",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 746.04,
      "end": 747.76,
      "text": "后面看到每每20几个Token",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 747.76,
      "end": 749.78,
      "text": "也不知道到底快了多少",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 749.78,
      "end": 752.02,
      "text": "基础9变27是3倍",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 752.02,
      "end": 757.38,
      "text": "那么基础20变22可能就不值得增加内存和复杂度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 757.38,
      "end": 762.6,
      "text": "确认基础模型稳定之后呢",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 762.6,
      "end": 765.48,
      "text": "我们就可以安装D-Flash 2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 765.48,
      "end": 769.96,
      "text": "D-Flash 2本质上是一种推测解码技术",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 769.96,
      "end": 774.16,
      "text": "正常情况下千万3.827B每生成一个Token的话",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 774.16,
      "end": 777.58,
      "text": "都要让完整270亿参数计算一次",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 777.58,
      "end": 783.3,
      "text": "那么生成10个Token就要连续做10次完整的推理",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 783.3,
      "end": 792.3,
      "text": "Dflash2让一个大约20亿参数的草稿模型一次性并行预测接下来的一组Token",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 792.3,
      "end": 798.06,
      "text": "然后在27B主模型一次检查这一整组答案",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 798.86,
      "end": 800.9,
      "text": "如果草稿模型猜的比较准",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 800.9,
      "end": 804.2,
      "text": "主模型一次计算就可以接受多个Token",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 804.2,
      "end": 807.26,
      "text": "原本连续走五六四的流程",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 807.26,
      "end": 809.62,
      "text": "现在可能只需要一次验证",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 809.62,
      "end": 812.46,
      "text": "草稿模型只负责猜",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 812.46,
      "end": 816.0,
      "text": "最终决定权依然在27B主模型手上",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 816.0,
      "end": 817.76,
      "text": "贪心解码下最终输出",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 817.76,
      "end": 819.88,
      "text": "可以基础模型一致",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 819.88,
      "end": 822.72,
      "text": "随机采样则保留主模型原本的分布",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 822.72,
      "end": 824.34,
      "text": "在Mac上",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 824.34,
      "end": 827.32,
      "text": "比较直接的实现是MX的D Spark",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 827.32,
      "end": 831.22,
      "text": "那么这个虽然名字写着D Spark",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 831.22,
      "end": 834.28,
      "text": "但它同时支持D Spark D Flash和D Flash 2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 834.28,
      "end": 842.04,
      "text": "在用的时候对于千万3.8 27B Ultra模式会直接使用D Flash 2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 842.04,
      "end": 847.22,
      "text": "安装命令也非常简单",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 847.22,
      "end": 853.4,
      "text": "屏幕上的安装命令",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 853.4,
      "end": 857.06,
      "text": "安装完成以后不要凭感觉判断有没有变快",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 857.06,
      "end": 859.5,
      "text": "直接运行项目自带的Benchmark",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 859.5,
      "end": 861.72,
      "text": "使用三轮测试",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 861.72,
      "end": 867.16,
      "text": "第一次运行会自动下载匹配的DFlex2草稿模型",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 867.16,
      "end": 870.34,
      "text": "并根据这台Mac的实际成本测试",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 870.34,
      "end": 872.58,
      "text": "合适的推理长度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 872.58,
      "end": 875.78,
      "text": "M1 M3 M4 M5的最佳参数呢",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 875.78,
      "end": 877.44,
      "text": "它是不一定一样的",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 877.44,
      "end": 880.16,
      "text": "如果你想直接生成内容的话",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 880.16,
      "end": 882.5,
      "text": "建议用这个命令",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 882.5,
      "end": 884.8,
      "text": "可以使用MXD Spark Generate",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 884.8,
      "end": 887.84,
      "text": "默认AUTO模式会为",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 887.84,
      "end": 890.84,
      "text": "签问3.8RCB匹配DFLASH2",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 890.84,
      "end": 894.32,
      "text": "再用完全相同的Prompt测试",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 894.32,
      "end": 895.66,
      "text": "对比基础速度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 895.66,
      "end": 899.1,
      "text": "DFLASH2速度接受长度和内存占用",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 899.1,
      "end": 900.98,
      "text": "那么比下来之后",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 900.98,
      "end": 903.36,
      "text": "项目在特定的M4 Pro测试中",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 903.36,
      "end": 906.16,
      "text": "8bit的基础约8.4Token每秒",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 906.16,
      "end": 909.16,
      "text": "DFLASH2平均约30.5Token每秒",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 909.16,
      "end": 913.04,
      "text": "不同任务约2.8-4倍",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 913.04,
      "end": 917.22,
      "text": "但这不是你的Mac一定能达到的速度",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 917.22,
      "end": 922.0,
      "text": "因为DFLASH2不是打开以后它就一定变快",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 922.0,
      "end": 923.98,
      "text": "如果草稿接受率低的话",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 923.98,
      "end": 926.98,
      "text": "主模型会不断拒绝和重新生成",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 926.98,
      "end": 930.08,
      "text": "草稿计算反而成为额外的负担",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 930.08,
      "end": 933.44,
      "text": "对于只有几十个Token的短回答",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 933.44,
      "end": 937.44,
      "text": "模型刚加载完成和校准任务就结束了",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 937.44,
      "end": 939.98,
      "text": "实际体感提升可能不是很明显",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 939.98,
      "end": 943.0,
      "text": "DFLASH2还会增加内存占用",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 943.0,
      "end": 944.46,
      "text": "所以呢",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 944.46,
      "end": 945.936,
      "text": "如果巴比特主",
      "language": "zh",
      "chunk": 1,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 945.936,
      "end": 948.656,
      "text": "所以如果8bit主模型以及频繁swap",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 948.656,
      "end": 950.736,
      "text": "那么再加载草稿模型",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 950.736,
      "end": 953.056,
      "text": "最终速度反而可能更慢",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 953.056,
      "end": 956.736,
      "text": "所以要用自己最常执行的任务去测试一下",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 956.736,
      "end": 959.296,
      "text": "写代码就测真实代码修改",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 959.296,
      "end": 963.356,
      "text": "生成口播稿就连续输出一两千的token",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 963.356,
      "end": 966.696,
      "text": "运行agent就测试完整工具调用",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 966.696,
      "end": 971.456,
      "text": "最后一步是把模型启动成API",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 971.456,
      "end": 975.036,
      "text": "MLXDspark可以兼容OpenAI",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 975.036,
      "end": 978.636,
      "text": "那么和Astorbit格式的本地服务",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 978.636,
      "end": 983.096,
      "text": "启动命令是MLXDsparkServe",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 983.096,
      "end": 988.396,
      "text": "默认是开在本级8080端口去提供接口的",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 988.396,
      "end": 992.396,
      "text": "客户端BaseURL设置成127.0.18080",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 994.156,
      "end": 995.996,
      "text": "设置好之后",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 995.996,
      "end": 997.516,
      "text": "只在自己电脑使用",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 997.516,
      "end": 1000.316,
      "text": "可以不设置真正的API Key",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1000.316,
      "end": 1002.116,
      "text": "需要局域网访问的时候",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1002.116,
      "end": 1003.716,
      "text": "要调整监听地址",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1003.716,
      "end": 1005.596,
      "text": "检查防火墙",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1005.596,
      "end": 1009.596,
      "text": "但不要把无建权接口直接暴露到公网",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1009.596,
      "end": 1013.196,
      "text": "到这里我们完成了整套部署流程",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1013.196,
      "end": 1013.796,
      "text": "第一",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1013.796,
      "end": 1016.636,
      "text": "27B 代表约270亿参数",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1016.636,
      "end": 1019.596,
      "text": "而且它是全量激活的稠密模型",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1019.596,
      "end": 1020.196,
      "text": "第二",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1020.196,
      "end": 1023.456,
      "text": "RGB不等于只需RGB内存",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1023.456,
      "end": 1025.236,
      "text": "把比特权重接近R9G",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1025.236,
      "end": 1029.416,
      "text": "运行时还要加Cache缓冲区系统和草稿模型",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1029.416,
      "end": 1032.116,
      "text": "那么第三",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1032.116,
      "end": 1035.016,
      "text": "它属于本地模型高端档",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1035.016,
      "end": 1038.116,
      "text": "部分编程和Agent任务接近B原起见",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1038.116,
      "end": 1039.996,
      "text": "但不是全面的替代",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1039.996,
      "end": 1041.076,
      "text": "第四",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1041.076,
      "end": 1042.616,
      "text": "不要一上来",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1042.616,
      "end": 1044.516,
      "text": "就打开所有加速箱",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1044.516,
      "end": 1046.976,
      "text": "先用MLX跑出基础速度",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1046.976,
      "end": 1049.436,
      "text": "再用相同Prompt做A-B测试",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1049.436,
      "end": 1052.596,
      "text": "决定是否长期使用D-Flash 2",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1052.596,
      "end": 1055.716,
      "text": "如果真实任务从每秒9个Token",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1055.716,
      "end": 1057.776,
      "text": "能够提高到2-30个的话",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1057.776,
      "end": 1059.356,
      "text": "而且内存稳定",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1059.356,
      "end": 1061.856,
      "text": "那么它就值得长期去打开",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1061.856,
      "end": 1065.996,
      "text": "如果频繁Swap或者主要都是短问答的话",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1065.996,
      "end": 1068.736,
      "text": "4bit基础模型可能更舒服一些",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1068.736,
      "end": 1073.036,
      "text": "本地部署大模型的价值不是省API费用",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1073.036,
      "end": 1075.936,
      "text": "当它变成Mac上的本地服务",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1075.936,
      "end": 1078.696,
      "text": "它就可以接入代码Langraph",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1078.696,
      "end": 1079.936,
      "text": "自动剪辑系统",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1079.936,
      "end": 1080.776,
      "text": "知识库",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1080.776,
      "end": 1083.916,
      "text": "以及agent工作流",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1083.916,
      "end": 1087.556,
      "text": "真正好的本地布置方案",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1087.556,
      "end": 1088.876,
      "text": "不是参数最大",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1088.876,
      "end": 1090.636,
      "text": "也不是测试数字最高",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1090.636,
      "end": 1092.116,
      "text": "而是在自己的设备上",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1092.116,
      "end": 1093.076,
      "text": "质量足够好",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1093.076,
      "end": 1094.316,
      "text": "速度能够接受",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1094.316,
      "end": 1096.236,
      "text": "内存保持稳定",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1096.236,
      "end": 1099.156,
      "text": "并且真的进入日常工作流",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1099.156,
      "end": 1100.756,
      "text": "才能长期去使用",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1100.756,
      "end": 1104.196,
      "text": "影片领誘到的模型地址安装命令和测试命令",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1104.196,
      "end": 1106.416,
      "text": "我都会整理在下方",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1106.416,
      "end": 1113.436,
      "text": "如果希望下一期测试4bit 6bit和8bit的品质差距的话",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1113.436,
      "end": 1116.856,
      "text": "以及DFlex 2速度也可以留言给我",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1116.856,
      "end": 1119.056,
      "text": "喜欢这支影片的话",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1119.056,
      "end": 1122.776,
      "text": "记得帮我按赞订阅加分享哦",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1122.776,
      "end": 1124.956,
      "text": "这是我持续创作的最大动力",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    },
    {
      "start": 1124.956,
      "end": 1125.916,
      "text": "我们下次再见",
      "language": "zh",
      "chunk": 2,
      "local_language_run": 0,
      "language_run": 0
    }
  ],
  "segmented_multilingual_transcription": {
    "status": "STABLE_SEGMENTED_MULTILINGUAL_REVIEW",
    "created_at": "2026-09-02T12:07:06",
    "manifest": "/Users/sagawa/AI_WORK/video_notes/url/20260902_114229__不用5090_Mac也能实现Token自由_Qwen3.8_27B本地实测/segmented_work/chunk_manifest.json",
    "audio": "/Users/sagawa/AI_WORK/video_notes/url/20260902_114229__不用5090_Mac也能实现Token自由_Qwen3.8_27B本地实测/audio_16k_mono.wav",
    "audio_sha256": "c9a44dc71e2416c98955bc287bca8b7e1f68264fea3e26b566c8f7a72f18aac8",
    "audio_duration": 1125.935625,
    "model": "/Users/sagawa/AI_WORK/cache/huggingface/hub/models--mlx-community--whisper-large-v3-turbo/snapshots/a4aaeec0636e6fef84abdcbe3544cb2bf7e9f6fb",
    "language_profile": "/Users/sagawa/AI_WORK/video_notes/url/20260902_114229__不用5090_Mac也能实现Token自由_Qwen3.8_27B本地实测/multilingual_work/multilingual_language_profile.json",
    "dominant_languages": [
      "zh",
      "en"
    ],
    "suppressed_languages": [
      "pt",
      "ru"
    ],
    "chunk_count": 3,
    "completed_chunks": 3,
    "resumed_chunks": 0,
    "merged_segment_count": 377,
    "detected_languages": [
      "zh"
    ],
    "transcript_languages": [
      "zh"
    ],
    "missing_languages": [
      "en"
    ],
    "unexpected_languages": [],
    "language_sequence": [
      "zh"
    ],
    "speech_region_seconds": 1125.9359999999997,
    "speech_covered_seconds": 1122.9159999999997,
    "speech_coverage_ratio": 0.9973177871566412,
    "tail_repeat_run": 1,
    "tail_repeat_text": "开放了千万3.8RGB的模型权重",
    "chunk_needs_review": [],
    "qc_ok": false,
    "qc_errors": [
      "missing dominant languages: en"
    ],
    "complete_review_artifact": true
  }
}