{"text": "你敢信吗?微软最新一篇论文把一个2.47GB的语音识别模型硬塞到了670Mbps,准确率几乎没掉。故事是这样的,一直以来,做端侧语音识别有四道坎,模型小,速度快,延迟低,还得准。这四个几乎不可能同时满足,要么云端调API泄露隐私,要么本地跑的卡成PPT。微软团队拉通测了50多种配置,结果挖到一个反常识的事实,P处理跑分最高的模型,到了流处理场景直接崩。QWN3ASRP处理只有5.9%的词错率,切到2.4秒分块后,直接标到10.45%,几乎翻倍。真正的赢家是英伟达的Nemotron,这个模型最大的特点是缓存感知架构,天生为实时识别而生。它能记住前面5.6秒的历史信息,每次只处理0.56秒的新音频。从P处理切到10时,准确率几乎没掉。选完模型只是开始,微软把整个推理管线用ONEX ROM10重写了一遍,把模型拆成编码器、解码器、Joyner三块独立优化。最关键的一招是Int 4K Quant量化,不是简单的四舍五入,而是按权重重要性加权重建。最终结果,模型从2.47G压到670米,体积少了73%,平均词错率8.2%,相比FP32只退化了0.17个百分点。CPU上跑得比实时快6倍多,算法延迟0.56秒。这套方案已经在微软Foundry Local平台开源,端测语音识别这事儿真要起飞了。", "segments": [{"id": 0, "seek": 0, "start": 0.0, "end": 7.94, "text": "你敢信吗?微软最新一篇论文把一个2.47GB的语音识别模型硬塞到了670Mbps,准确率几乎没掉。", "tokens": [50365, 2166, 40205, 17665, 14769, 30, 39152, 17819, 107, 8661, 12560, 2257, 20878, 229, 7422, 118, 17174, 16075, 20182, 17, 13, 14060, 8769, 1546, 5233, 255, 18034, 5233, 228, 18453, 41908, 39823, 38114, 105, 13331, 252, 21381, 21, 5867, 44, 65, 1878, 11, 6336, 228, 38114, 106, 44866, 6336, 254, 2930, 236, 10062, 29327, 1543, 50762], "temperature": 0, "avg_logprob": -0.0946997090389854, "compression_ratio": 1.1453900709219857, "no_speech_prob": 1.690506762352939e-11}, {"id": 1, "seek": 0, "start": 9.16, "end": 15.18, "text": "故事是这样的,一直以来,做端侧语音识别有四道坎,模型小,速度快,延迟低,还得准。", "tokens": [50823, 43045, 6973, 1541, 21209, 1546, 11, 34448, 3588, 6912, 11, 10907, 11957, 107, 3254, 100, 5233, 255, 18034, 5233, 228, 18453, 2412, 19425, 6025, 14872, 236, 11, 41908, 39823, 7322, 11, 31217, 13127, 10251, 11, 17527, 114, 3316, 253, 41377, 11, 14852, 5916, 6336, 228, 1543, 51124], "temperature": 0, "avg_logprob": -0.0946997090389854, "compression_ratio": 1.1453900709219857, "no_speech_prob": 1.690506762352939e-11}, {"id": 2, "seek": 0, "start": 15.700000000000001, "end": 21.46, "text": "这四个几乎不可能同时满足,要么云端调API泄露隐私,要么本地跑的卡成PPT。", "tokens": [51150, 5562, 19425, 7549, 6336, 254, 2930, 236, 1960, 16657, 13089, 15729, 15868, 94, 37236, 11, 4275, 6656, 1369, 239, 11957, 107, 8897, 225, 4715, 40, 6847, 226, 18594, 110, 10673, 238, 20083, 11, 4275, 6656, 8802, 10928, 32585, 1546, 32681, 11336, 17755, 51, 1543, 51438], "temperature": 0, "avg_logprob": -0.0946997090389854, "compression_ratio": 1.1453900709219857, "no_speech_prob": 1.690506762352939e-11}, {"id": 3, "seek": 2146, "start": 21.46, "end": 30.060000000000002, "text": "微软团队拉通测了50多种配置,结果挖到一个反常识的事实,P处理跑分最高的模型,到了流处理场景直接崩。", "tokens": [50365, 39152, 17819, 107, 3919, 95, 10034, 253, 29717, 19550, 11038, 233, 2289, 2803, 6392, 39810, 38846, 34719, 11, 45641, 9319, 8501, 244, 4511, 20182, 22138, 11279, 5233, 228, 1546, 6973, 24726, 11, 47, 1787, 226, 13876, 32585, 6627, 8661, 12979, 1546, 41908, 39823, 11, 21381, 27854, 1787, 226, 13876, 50255, 50218, 43297, 49378, 102, 1543, 50795], "temperature": 0, "avg_logprob": -0.059509656368157804, "compression_ratio": 1.1442622950819672, "no_speech_prob": 1.0708121889191347e-11}, {"id": 4, "seek": 2146, "start": 30.76, "end": 38.019999999999996, "text": "QWN3ASRP处理只有5.9%的词错率,切到2.4秒分块后,直接标到10.45%,几乎翻倍。", "tokens": [50830, 48, 54, 45, 18, 3160, 49, 47, 1787, 226, 13876, 35244, 20, 13, 24, 4, 1546, 5233, 235, 29900, 44866, 11, 23632, 4511, 17, 13, 19, 20956, 6627, 47734, 13547, 11, 43297, 162, 3921, 4511, 3279, 13, 8465, 8923, 6336, 254, 2930, 236, 42716, 35477, 1543, 51193], "temperature": 0, "avg_logprob": -0.059509656368157804, "compression_ratio": 1.1442622950819672, "no_speech_prob": 1.0708121889191347e-11}, {"id": 5, "seek": 2146, "start": 39.28, "end": 45.540000000000006, "text": "真正的赢家是英伟达的Nemotron,这个模型最大的特点是缓存感知架构,天生为实时识别而生。", "tokens": [51256, 6303, 15789, 1546, 5266, 95, 5155, 1541, 27869, 7384, 253, 9830, 122, 1546, 45, 443, 310, 2044, 11, 15368, 41908, 39823, 8661, 39156, 17682, 12579, 1541, 38109, 241, 39781, 9709, 6498, 7360, 114, 7360, 226, 11, 6135, 8244, 13992, 24726, 15729, 5233, 228, 18453, 11070, 8244, 1543, 51569], "temperature": 0, "avg_logprob": -0.059509656368157804, "compression_ratio": 1.1442622950819672, "no_speech_prob": 1.0708121889191347e-11}, {"id": 6, "seek": 4554, "start": 45.54, "end": 50.06, "text": "它能记住前面5.6秒的历史信息,每次只处理0.56秒的新音频。", "tokens": [50365, 11284, 8225, 34756, 21632, 8945, 8833, 20, 13, 21, 20956, 1546, 5014, 228, 45399, 17665, 26460, 11, 23664, 9487, 14003, 1787, 226, 13876, 15, 13, 18317, 20956, 1546, 12560, 18034, 39752, 1543, 50591], "temperature": 0, "avg_logprob": -0.09698933332394331, "compression_ratio": 1.1671159029649596, "no_speech_prob": 1.461501059063508e-11}, {"id": 7, "seek": 4554, "start": 50.58, "end": 53.12, "text": "从P处理切到10时,准确率几乎没掉。", "tokens": [50617, 35630, 47, 1787, 226, 13876, 23632, 4511, 3279, 15729, 11, 6336, 228, 38114, 106, 44866, 6336, 254, 2930, 236, 10062, 29327, 1543, 50744], "temperature": 0, "avg_logprob": -0.09698933332394331, "compression_ratio": 1.1671159029649596, "no_speech_prob": 1.461501059063508e-11}, {"id": 8, "seek": 4554, "start": 54.48, "end": 62.22, "text": "选完模型只是开始,微软把整个推理管线用ONEX ROM10重写了一遍,把模型拆成编码器、解码器、Joyner三块独立优化。", "tokens": [50812, 2215, 231, 14128, 41908, 39823, 36859, 45213, 11, 39152, 17819, 107, 16075, 27662, 7549, 33597, 13876, 23131, 16853, 123, 9254, 14248, 55, 41678, 3279, 12624, 5676, 247, 2289, 2257, 3330, 235, 11, 16075, 41908, 39823, 6852, 228, 11336, 38109, 244, 23230, 223, 34386, 1231, 17278, 23230, 223, 34386, 1231, 41, 939, 1193, 10960, 47734, 18637, 105, 24409, 7384, 246, 23756, 1543, 51199], "temperature": 0, "avg_logprob": -0.09698933332394331, "compression_ratio": 1.1671159029649596, "no_speech_prob": 1.461501059063508e-11}, {"id": 9, "seek": 4554, "start": 62.379999999999995, "end": 68.52, "text": "最关键的一招是Int 4K Quant量化,不是简单的四舍五入,而是按权重重要性加权重建。", "tokens": [51207, 8661, 28053, 23049, 106, 1546, 2257, 6852, 249, 1541, 25597, 1017, 42, 26968, 26748, 23756, 11, 7296, 11249, 222, 47446, 1546, 19425, 10256, 235, 21001, 14028, 11, 11070, 1541, 26613, 4422, 225, 12624, 24928, 21686, 9990, 4422, 225, 12624, 34157, 1543, 51514], "temperature": 0, "avg_logprob": -0.09698933332394331, "compression_ratio": 1.1671159029649596, "no_speech_prob": 1.461501059063508e-11}, {"id": 10, "seek": 4554, "start": 69.8, "end": 74.44, "text": "最终结果,模型从2.47G压到670米,体积少了73%,", "tokens": [51578, 8661, 10115, 230, 45641, 9319, 11, 41908, 39823, 35630, 17, 13, 14060, 38, 5014, 233, 4511, 21, 5867, 27742, 11, 29485, 8204, 107, 15686, 2289, 33396, 8923, 51810], "temperature": 0, "avg_logprob": -0.09698933332394331, "compression_ratio": 1.1671159029649596, "no_speech_prob": 1.461501059063508e-11}, {"id": 11, "seek": 7444, "start": 74.44, "end": 79.16, "text": "平均词错率8.2%,相比FP32只退化了0.17个百分点。", "tokens": [50365, 16716, 14872, 229, 5233, 235, 29900, 44866, 23, 13, 17, 8923, 15106, 11706, 45882, 11440, 14003, 46361, 23756, 2289, 15, 13, 7773, 7549, 31906, 6627, 12579, 1543, 50601], "temperature": 0, "avg_logprob": -0.09813115031448837, "compression_ratio": 0.9508928571428571, "no_speech_prob": 1.4006240611763587e-11}, {"id": 12, "seek": 7444, "start": 79.66, "end": 83.08, "text": "CPU上跑得比实时快6倍多,算法延迟0.56秒。", "tokens": [50626, 34, 8115, 5708, 32585, 5916, 11706, 24726, 15729, 10251, 21, 35477, 6392, 11, 19497, 11148, 17527, 114, 3316, 253, 15, 13, 18317, 20956, 1543, 50797], "temperature": 0, "avg_logprob": -0.09813115031448837, "compression_ratio": 0.9508928571428571, "no_speech_prob": 1.4006240611763587e-11}, {"id": 13, "seek": 7444, "start": 83.42, "end": 88.14, "text": "这套方案已经在微软Foundry Local平台开源,端测语音识别这事儿真要起飞了。", "tokens": [50814, 5562, 1881, 245, 9249, 28899, 49161, 3581, 39152, 17819, 107, 37, 554, 627, 22755, 16716, 15433, 18937, 47402, 11, 11957, 107, 11038, 233, 5233, 255, 18034, 5233, 228, 18453, 5562, 6973, 22933, 6303, 4275, 9147, 11808, 252, 2289, 1543, 51050], "temperature": 0, "avg_logprob": -0.09813115031448837, "compression_ratio": 0.9508928571428571, "no_speech_prob": 1.4006240611763587e-11}], "language": "zh"}