{ "text": "最近最火的国产开源模型\n竟然不是DeepSeek V4\n而是阿里云千问3.8-27B这个小模型\n我在自己的Mac Studio上呢\n实测了一周啊\n结论先说啊\n智力是顶级\n速度是瓶颈啊\n但有个办法能够提速50%\n现在开源模型的军备竞赛呢\n已经卷到万亿的参数了\n那么DeepSeek V4呢\n是1.6万亿的\n但是阿里云的Qwen3.8-27B\n竟然只是一个只有\n200多亿参数的模型啊\n那么跟主流的万亿参数相比呢\n它的体量上差了几十倍\n却能够排到大模型智能的前列\n现在最主流的模型评测网站呢\n是Artificial Analysis\n他把600多个主流的模型都放到\n一套intelligence index里面来做测试\n你会看到啊\n这个图表当中\nQwen3.8-27B呢\n它的智能指数呢是52\n跟GPT5.6Luna是相同的\n而且距离DeepSeek V4 Pro\n这个1.6万亿的模型呢\n分数是非常接近的\n那么这么小的模型竟然\n能够跟万亿模型媲美的话\n那岂不是我们就可以把它\n装在我们的个人电脑上\n然后在个人电脑上跑\n一个最前沿的AI模型了吗\n甚至有很多人啊\n已经把它跑起来了\n在英伟达5090这样的消费级显卡上\n跑出了每秒70多个token啊\n那么每秒70个token是什么概念呢\n它其实已经接近了我们主流\n使用云端AI模型的推理速度了\n抱着很强的好奇心啊\n我也在自己个人的Mac Studio的\n工作站上来部署使用\n实际体验了一下\n首先呢我选择的是苹果平台上的\n一个开源推理框架叫oMLX\n那这个推理框架啊\n它能够充分利用苹果的MLX指令集\n尽量达到最优的效果\noMLX安装好以后啊\n就可以直接从海外的Hugging Face\n或者国内阿里云的魔搭社区中\n下载Qwen3.8-27B的这个量化版本\n支持4比特和支持8比特的\n如果你想要速度啊\n就选4比特的版本\n模型呢只有15GB大小\n而8比特呢是27.5G\n其实都不大\n那么把模型安装好以后啊\n就可以把它跑起来了\n这个时候啊\n我就可以跟它对话了\n整体上还是很流畅的呀\n当然刚开始对话的时候呢\n预填充的阶段还是有点慢\n但是预填充好了之后啊\n回答就非常的流畅了\n那我可以用oMLX跑一下评测\n你就可以看到它的预填充啊\n生成代码呀\n以及生成token的解码速度\n那我分别测试了4K16K\n还有32K的提示词的长度\n那么预填充的速度啊\n每秒有200多个token\n其实并不快\n那么解码速度每秒是不到30个\n说实话啊\n看到这个数字我心里咯噔一下\n传说破灭了吗\n那这个速度确实\n没办法直接用来跑Agent\n只适合用来做知识库问答呀\n客服系统啊\n这种类型的应用\n当然了它占的内存确实不多啊\n峰值内存都不到20GB\n所以真正的瓶颈是我这台苹果的\nMac Studio M4工作站的内存带宽不够\n所以导致了token生成的速度受限\n那么有没有办法解决这个问题\n来加快token的生成速度呢\n答案是可以用DFlash2\n原理很简单\n给他配一个很小的草稿模型\n让他每次呢先快速的生成做预测\n然后主模型呢只负责来做验证\n那token的生成速度就\n可以大幅度提高了\n啊我在OMX里面呢直接\n下载了这个草稿模型很小\n只有3.5GB\n那么设置打开DFlash2\n未配置DFlash2\n好那么这样40-50个token在实际环境\n当中可以真正的使用了吗\n我现在用非常干净的PI-Agent来尝试\n运行Qwen 27B的4-bit的量化版本\n让它来执行一个知识库查询\n因为我的知识库有2,000多篇文档啊\n那这个查询的Skill相对来说\n它已经有一定的复杂度了\n跑下来可以看到\n它能够很稳定的把任务给执行完毕\n结果呢也令我很满意\n速度呢确实很慢\n需要很长的等待时间\n但已经能干活了\n那么实际执行任务的时候呢\n平均每秒有30多个token\n35个左右吧\n虽然慢啊\n但是它其实已经完全是一个\n可以在本地跑Agent的模型了\n我又做了一些更多的测试\n慢终归是慢\n但胜在稳定啊\n任务给出的幻觉也很小\n其实呢\n我在我自己这台Mac的工作站上呢\n常用的是一个Qwen 3.6 35B-A3B的模型\n它是350亿的参数\n但它是一个MoE的专家模型\n所以每次推理呢\n它只激活30亿的参数\n那这个模型的token\n生成速度就非常快\n我同样用oMLX来给你跑一下测试\n你可以看到啊\n它预填充速度每秒\n达到了1,400-1,900多个token\n加载速度非常的快\n然后呢它的生成的速度啊\n每秒有80多个到100多个token\n这个速度已经达到了我们平常\n使用云端主流模型的速度了\n但是用Pi-Agent呢\n跑复杂任务的时候啊\n你会发现它速度虽然很快\n但是幻觉也非常的明显\n它的任务的质量啊\n明显不如Qwen3.8-27B的模型稳啊\n最后我总结一下吧\n在我自己的Mac Studio工作站上\n我本地主力跑的是Qwen-3.6 35B\n但它的幻觉确实比较明显\n那么把它换到Qwen3.8-27B之后呢\n几乎感觉不到什么幻觉了\n那么在本地的私有化的部署场景下\n已经完全达到了可用的状态\n另外还要补充一点啊\n我的Mac Studio的内存带宽确实不太够\n像我有朋友呢\n在英伟达5090的显卡上\n用这个模型呢\n实实在在跑出了每秒70多个token\n那达到这个速度就可以完全\n在本地执行Agent的任务了\n那我觉得呢\nQwen3.8-27B呢\n的确是第一款真正\n能够在本地私有化部署\n又达到了主流模型\n能力的一个模型了\n当然它的智力还没有\n达到Opus4.8这个级别啊\n但评测分数已经站到第一梯队了\n所以你想本地跑AI\n现在可以尝试一下\n在苹果上用oMLX加上Qwen3.8-27B\n好然后再加上DFlash2\n照着我的视频顺序装就可以了\n一言以蔽之啊\n本地跑AI的临界点真的已经到了\n关注范凯说AI\n我们下期再见", "language": "zh", "duration": 415.0, "segments": [ { "id": 0, "start": 0.2, "end": 2.3, "text": "最近最火的国产开源模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 1, "start": 2.3, "end": 3.9, "text": "竟然不是DeepSeek V4", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 2, "start": 3.9, "end": 7.566, "text": "而是阿里云千问3.8-27B这个小模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 3, "start": 7.733, "end": 9.5, "text": "我在自己的Mac Studio上呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 4, "start": 9.5, "end": 10.666, "text": "实测了一周啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 5, "start": 10.766, "end": 11.733, "text": "结论先说啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 6, "start": 11.733, "end": 12.766, "text": "智力是顶级", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 7, "start": 12.766, "end": 14.0, "text": "速度是瓶颈啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 8, "start": 14.0, "end": 16.8, "text": "但有个办法能够提速50%", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 9, "start": 17.0, "end": 19.1, "text": "现在开源模型的军备竞赛呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 10, "start": 19.1, "end": 21.066, "text": "已经卷到万亿的参数了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 11, "start": 21.066, "end": 22.333, "text": "那么DeepSeek V4呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 12, "start": 22.333, "end": 23.666, "text": "是1.6万亿的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 13, "start": 23.666, "end": 26.133, "text": "但是阿里云的Qwen3.8-27B", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 14, "start": 26.266, "end": 27.8, "text": "竟然只是一个只有", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 15, "start": 27.8, "end": 29.8, "text": "200多亿参数的模型啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 16, "start": 29.966, "end": 32.5, "text": "那么跟主流的万亿参数相比呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 17, "start": 32.533, "end": 34.966, "text": "它的体量上差了几十倍", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 18, "start": 35.133, "end": 37.7, "text": "却能够排到大模型智能的前列", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 19, "start": 38.4, "end": 40.6, "text": "现在最主流的模型评测网站呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 20, "start": 40.6, "end": 42.266, "text": "是Artificial Analysis", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 21, "start": 42.666, "end": 45.1, "text": "他把600多个主流的模型都放到", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 22, "start": 45.1, "end": 48.266, "text": "一套intelligence index里面来做测试", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 23, "start": 48.466, "end": 49.533, "text": "你会看到啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 24, "start": 49.733, "end": 51.133, "text": "这个图表当中", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 25, "start": 51.266, "end": 53.2, "text": "Qwen3.8-27B呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 26, "start": 53.2, "end": 55.1, "text": "它的智能指数呢是52", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 27, "start": 55.1, "end": 57.666, "text": "跟GPT5.6Luna是相同的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 28, "start": 57.666, "end": 60.1, "text": "而且距离DeepSeek V4 Pro", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 29, "start": 60.166, "end": 62.033, "text": "这个1.6万亿的模型呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 30, "start": 62.1, "end": 63.933, "text": "分数是非常接近的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 31, "start": 64.266, "end": 66.2, "text": "那么这么小的模型竟然", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 32, "start": 66.2, "end": 68.466, "text": "能够跟万亿模型媲美的话", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 33, "start": 68.466, "end": 70.333, "text": "那岂不是我们就可以把它", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 34, "start": 70.333, "end": 71.866, "text": "装在我们的个人电脑上", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 35, "start": 71.866, "end": 73.7, "text": "然后在个人电脑上跑", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 36, "start": 73.7, "end": 75.766, "text": "一个最前沿的AI模型了吗", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 37, "start": 76.3, "end": 77.266, "text": "甚至有很多人啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 38, "start": 77.266, "end": 78.5, "text": "已经把它跑起来了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 39, "start": 78.5, "end": 81.9, "text": "在英伟达5090这样的消费级显卡上", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 40, "start": 81.9, "end": 84.2, "text": "跑出了每秒70多个token啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 41, "start": 84.4, "end": 87.066, "text": "那么每秒70个token是什么概念呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 42, "start": 87.1, "end": 89.366, "text": "它其实已经接近了我们主流", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 43, "start": 89.366, "end": 92.366, "text": "使用云端AI模型的推理速度了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 44, "start": 92.7, "end": 93.9, "text": "抱着很强的好奇心啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 45, "start": 93.9, "end": 96.933, "text": "我也在自己个人的Mac Studio的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 46, "start": 96.933, "end": 98.4, "text": "工作站上来部署使用", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 47, "start": 98.4, "end": 99.666, "text": "实际体验了一下", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 48, "start": 99.866, "end": 102.266, "text": "首先呢我选择的是苹果平台上的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 49, "start": 102.266, "end": 105.466, "text": "一个开源推理框架叫oMLX", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 50, "start": 105.733, "end": 107.0, "text": "那这个推理框架啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 51, "start": 107.0, "end": 110.3, "text": "它能够充分利用苹果的MLX指令集", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 52, "start": 110.6, "end": 112.366, "text": "尽量达到最优的效果", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 53, "start": 112.8, "end": 114.533, "text": "oMLX安装好以后啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 54, "start": 114.666, "end": 117.566, "text": "就可以直接从海外的Hugging Face", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 55, "start": 117.766, "end": 120.233, "text": "或者国内阿里云的魔搭社区中", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 56, "start": 120.3, "end": 123.466, "text": "下载Qwen3.8-27B的这个量化版本", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 57, "start": 123.733, "end": 126.133, "text": "支持4比特和支持8比特的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 58, "start": 126.166, "end": 127.4, "text": "如果你想要速度啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 59, "start": 127.4, "end": 128.733, "text": "就选4比特的版本", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 60, "start": 128.933, "end": 131.066, "text": "模型呢只有15GB大小", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 61, "start": 131.1, "end": 133.666, "text": "而8比特呢是27.5G", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 62, "start": 133.666, "end": 134.866, "text": "其实都不大", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 63, "start": 135.2, "end": 136.866, "text": "那么把模型安装好以后啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 64, "start": 136.866, "end": 138.166, "text": "就可以把它跑起来了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 65, "start": 138.166, "end": 138.866, "text": "这个时候啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 66, "start": 138.866, "end": 140.333, "text": "我就可以跟它对话了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 67, "start": 140.5, "end": 142.266, "text": "整体上还是很流畅的呀", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 68, "start": 142.4, "end": 144.366, "text": "当然刚开始对话的时候呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 69, "start": 144.366, "end": 146.666, "text": "预填充的阶段还是有点慢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 70, "start": 146.666, "end": 148.5, "text": "但是预填充好了之后啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 71, "start": 148.533, "end": 150.5, "text": "回答就非常的流畅了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 72, "start": 150.9, "end": 153.733, "text": "那我可以用oMLX跑一下评测", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 73, "start": 153.766, "end": 156.066, "text": "你就可以看到它的预填充啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 74, "start": 156.066, "end": 157.1, "text": "生成代码呀", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 75, "start": 157.1, "end": 159.066, "text": "以及生成token的解码速度", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 76, "start": 159.2, "end": 161.933, "text": "那我分别测试了4K16K", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 77, "start": 161.933, "end": 164.266, "text": "还有32K的提示词的长度", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 78, "start": 164.4, "end": 165.933, "text": "那么预填充的速度啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 79, "start": 165.933, "end": 167.533, "text": "每秒有200多个token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 80, "start": 167.533, "end": 168.533, "text": "其实并不快", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 81, "start": 168.533, "end": 171.133, "text": "那么解码速度每秒是不到30个", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 82, "start": 171.366, "end": 172.766, "text": "说实话啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 83, "start": 172.766, "end": 174.733, "text": "看到这个数字我心里咯噔一下", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 84, "start": 174.9, "end": 176.466, "text": "传说破灭了吗", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 85, "start": 176.733, "end": 177.766, "text": "那这个速度确实", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 86, "start": 177.766, "end": 179.5, "text": "没办法直接用来跑Agent", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 87, "start": 179.766, "end": 181.8, "text": "只适合用来做知识库问答呀", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 88, "start": 181.8, "end": 182.666, "text": "客服系统啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 89, "start": 182.666, "end": 183.8, "text": "这种类型的应用", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 90, "start": 184.4, "end": 186.966, "text": "当然了它占的内存确实不多啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 91, "start": 186.966, "end": 188.9, "text": "峰值内存都不到20GB", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 92, "start": 189.5, "end": 192.466, "text": "所以真正的瓶颈是我这台苹果的", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 93, "start": 192.466, "end": 195.6, "text": "Mac Studio M4工作站的内存带宽不够", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 94, "start": 195.7, "end": 198.2, "text": "所以导致了token生成的速度受限", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 95, "start": 198.533, "end": 200.666, "text": "那么有没有办法解决这个问题", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 96, "start": 200.666, "end": 202.966, "text": "来加快token的生成速度呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 97, "start": 203.133, "end": 205.066, "text": "答案是可以用DFlash2", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 98, "start": 205.333, "end": 206.333, "text": "原理很简单", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 99, "start": 206.666, "end": 208.733, "text": "给他配一个很小的草稿模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 100, "start": 208.733, "end": 211.533, "text": "让他每次呢先快速的生成做预测", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 101, "start": 211.766, "end": 214.0, "text": "然后主模型呢只负责来做验证", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 102, "start": 214.0, "end": 215.5, "text": "那token的生成速度就", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 103, "start": 215.5, "end": 217.1, "text": "可以大幅度提高了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 104, "start": 217.333, "end": 219.566, "text": "啊我在OMX里面呢直接", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 105, "start": 219.566, "end": 222.033, "text": "下载了这个草稿模型很小", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 106, "start": 222.033, "end": 223.5, "text": "只有3.5GB", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 107, "start": 223.733, "end": 226.1, "text": "那么设置打开DFlash2", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 108, "start": 226.1, "end": 235.533, "text": "未配置DFlash2", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 109, "start": 235.733, "end": 238.766, "text": "好那么这样40-50个token在实际环境", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 110, "start": 238.766, "end": 241.666, "text": "当中可以真正的使用了吗", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 111, "start": 241.8, "end": 245.566, "text": "我现在用非常干净的PI-Agent来尝试", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 112, "start": 245.566, "end": 249.166, "text": "运行Qwen 27B的4-bit的量化版本", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 113, "start": 249.3, "end": 251.466, "text": "让它来执行一个知识库查询", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 114, "start": 251.7, "end": 254.8, "text": "因为我的知识库有2,000多篇文档啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 115, "start": 254.866, "end": 257.033, "text": "那这个查询的Skill相对来说", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 116, "start": 257.033, "end": 258.766, "text": "它已经有一定的复杂度了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 117, "start": 258.866, "end": 260.766, "text": "跑下来可以看到", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 118, "start": 261.066, "end": 264.366, "text": "它能够很稳定的把任务给执行完毕", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 119, "start": 264.533, "end": 266.266, "text": "结果呢也令我很满意", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 120, "start": 266.5, "end": 267.966, "text": "速度呢确实很慢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 121, "start": 267.966, "end": 269.6, "text": "需要很长的等待时间", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 122, "start": 269.7, "end": 271.1, "text": "但已经能干活了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 123, "start": 271.333, "end": 273.0, "text": "那么实际执行任务的时候呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 124, "start": 273.0, "end": 274.9, "text": "平均每秒有30多个token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 125, "start": 275.133, "end": 276.4, "text": "35个左右吧", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 126, "start": 276.5, "end": 277.4, "text": "虽然慢啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 127, "start": 277.4, "end": 280.3, "text": "但是它其实已经完全是一个", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 128, "start": 280.3, "end": 282.966, "text": "可以在本地跑Agent的模型了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 129, "start": 282.966, "end": 285.166, "text": "我又做了一些更多的测试", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 130, "start": 285.166, "end": 287.066, "text": "慢终归是慢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 131, "start": 287.066, "end": 288.533, "text": "但胜在稳定啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 132, "start": 288.566, "end": 290.433, "text": "任务给出的幻觉也很小", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 133, "start": 290.6, "end": 291.1, "text": "其实呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 134, "start": 291.1, "end": 293.933, "text": "我在我自己这台Mac的工作站上呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 135, "start": 293.966, "end": 298.866, "text": "常用的是一个Qwen 3.6 35B-A3B的模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 136, "start": 299.066, "end": 300.766, "text": "它是350亿的参数", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 137, "start": 300.9, "end": 303.1, "text": "但它是一个MoE的专家模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 138, "start": 303.1, "end": 304.533, "text": "所以每次推理呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 139, "start": 304.566, "end": 306.466, "text": "它只激活30亿的参数", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 140, "start": 306.566, "end": 307.8, "text": "那这个模型的token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 141, "start": 307.8, "end": 309.266, "text": "生成速度就非常快", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 142, "start": 309.5, "end": 312.9, "text": "我同样用oMLX来给你跑一下测试", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 143, "start": 313.2, "end": 314.1, "text": "你可以看到啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 144, "start": 314.1, "end": 315.866, "text": "它预填充速度每秒", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 145, "start": 315.866, "end": 318.933, "text": "达到了1,400-1,900多个token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 146, "start": 319.4, "end": 321.0, "text": "加载速度非常的快", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 147, "start": 321.266, "end": 323.1, "text": "然后呢它的生成的速度啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 148, "start": 323.1, "end": 325.8, "text": "每秒有80多个到100多个token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 149, "start": 325.966, "end": 327.866, "text": "这个速度已经达到了我们平常", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 150, "start": 327.866, "end": 330.033, "text": "使用云端主流模型的速度了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 151, "start": 330.033, "end": 331.633, "text": "但是用Pi-Agent呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 152, "start": 331.633, "end": 332.966, "text": "跑复杂任务的时候啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 153, "start": 332.966, "end": 334.8, "text": "你会发现它速度虽然很快", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 154, "start": 334.8, "end": 336.9, "text": "但是幻觉也非常的明显", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 155, "start": 337.1, "end": 338.466, "text": "它的任务的质量啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 156, "start": 338.466, "end": 342.033, "text": "明显不如Qwen3.8-27B的模型稳啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 157, "start": 342.033, "end": 343.233, "text": "最后我总结一下吧", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 158, "start": 343.233, "end": 345.433, "text": "在我自己的Mac Studio工作站上", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 159, "start": 345.433, "end": 349.5, "text": "我本地主力跑的是Qwen-3.6 35B", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 160, "start": 349.5, "end": 351.666, "text": "但它的幻觉确实比较明显", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 161, "start": 351.666, "end": 354.8, "text": "那么把它换到Qwen3.8-27B之后呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 162, "start": 354.8, "end": 356.7, "text": "几乎感觉不到什么幻觉了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 163, "start": 356.8, "end": 359.966, "text": "那么在本地的私有化的部署场景下", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 164, "start": 360.033, "end": 362.966, "text": "已经完全达到了可用的状态", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 165, "start": 363.5, "end": 364.9, "text": "另外还要补充一点啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 166, "start": 364.966, "end": 368.066, "text": "我的Mac Studio的内存带宽确实不太够", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 167, "start": 368.366, "end": 369.366, "text": "像我有朋友呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 168, "start": 369.366, "end": 371.666, "text": "在英伟达5090的显卡上", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 169, "start": 371.666, "end": 372.666, "text": "用这个模型呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 170, "start": 372.666, "end": 375.833, "text": "实实在在跑出了每秒70多个token", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 171, "start": 375.966, "end": 377.833, "text": "那达到这个速度就可以完全", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 172, "start": 377.833, "end": 379.866, "text": "在本地执行Agent的任务了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 173, "start": 379.966, "end": 381.066, "text": "那我觉得呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 174, "start": 381.066, "end": 382.8, "text": "Qwen3.8-27B呢", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 175, "start": 382.8, "end": 384.833, "text": "的确是第一款真正", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 176, "start": 384.833, "end": 386.9, "text": "能够在本地私有化部署", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 177, "start": 386.9, "end": 388.6, "text": "又达到了主流模型", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 178, "start": 388.6, "end": 390.1, "text": "能力的一个模型了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 179, "start": 390.233, "end": 391.466, "text": "当然它的智力还没有", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 180, "start": 391.466, "end": 393.466, "text": "达到Opus4.8这个级别啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 181, "start": 393.5, "end": 396.3, "text": "但评测分数已经站到第一梯队了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 182, "start": 396.3, "end": 397.966, "text": "所以你想本地跑AI", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 183, "start": 397.966, "end": 399.7, "text": "现在可以尝试一下", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 184, "start": 399.833, "end": 403.5, "text": "在苹果上用oMLX加上Qwen3.8-27B", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 185, "start": 403.766, "end": 405.6, "text": "好然后再加上DFlash2", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 186, "start": 405.866, "end": 408.433, "text": "照着我的视频顺序装就可以了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 187, "start": 408.5, "end": 409.8, "text": "一言以蔽之啊", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 188, "start": 409.8, "end": 412.833, "text": "本地跑AI的临界点真的已经到了", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 189, "start": 412.9, "end": 414.2, "text": "关注范凯说AI", "source": "video_source_subtitle", "authority": "video_source_subtitle" }, { "id": 190, "start": 414.2, "end": 415.0, "text": "我们下期再见", "source": "video_source_subtitle", "authority": "video_source_subtitle" } ], "transcript_selection": { "schema": "videonote.transcript_selection.v1", "status": "SELECTED", "requested_language": "zh", "duration": 415.0, "reference_path": "/Users/sagawa/AI_WORK/video_notes/url/20260902_153052__Qwen_3.8-27B_真有传说中那么强吗_我在_Mac_上跑完_Agent_后_得到了一个反直觉答案/reference_subtitle.vtt", "reference_kind": "manual", "reference_language": "zh-Hans", "selected_source": "video_source_subtitle", "source_caption_kind": "manual", "source_caption_processing": "verbatim_direct", "youtube_overlap_clamps": [], "general_high_confidence_repairs": [], "decision_reason": "complete_source_subtitle_fast_path_no_whisper", "whisper_rejected": false, "source_profile": { "segment_count": 191, "duration_coverage": 0.954851 } } }