1
00:00:00,520 --> 00:00:07,160
大家好,今天我們給大家講解Voice Agent的這個適時語音識別ASR的一個原理。

2
00:00:07,940 --> 00:00:22,780
我們從頭到尾把它捋一遍,要知道它背後的原理是什麼。當然具體的這個技術實現,肯定是由對應的這個供應商已經實現了。但是我們了解原理的話,對我們理解Voice Agent是有幫助的。

3
00:00:23,300 --> 00:00:23,800
好

4
00:00:24,320 --> 00:00:25,860
那么ASR呢

5
00:00:26,100 --> 00:00:28,920
其实我们一般会说ICTT

6
00:00:29,180 --> 00:00:30,720
就是语音转文字

7
00:00:31,480 --> 00:00:35,320
但是更专业的说法可以说ASR

8
00:00:35,580 --> 00:00:37,880
并且在这个voice agent里面

9
00:00:38,140 --> 00:00:39,420
一定要用

10
00:00:40,440 --> 00:00:41,460
实施语音

11
00:00:41,720 --> 00:00:42,500
实施语音的转换

12
00:00:42,740 --> 00:00:45,060
你整段语音的转文本

13
00:00:45,300 --> 00:00:47,100
对于这个voice agent意义不大

14
00:00:47,360 --> 00:00:49,140
因为你无法做这些

15
00:00:49,660 --> 00:00:50,940
打断了这个处理

16
00:00:51,700 --> 00:00:51,960
好

17
00:00:52,780 --> 00:00:55,780
那么他不是听懂了整句在翻译

18
00:00:55,780 --> 00:00:58,820
而是把音频声音切成一帧帧的逐帧

19
00:00:58,820 --> 00:01:01,900
拆这个什么音再把它串成

20
00:01:01,900 --> 00:01:05,020
这个再串成这个字

21
00:01:05,020 --> 00:01:06,080
是这么来的

22
00:01:06,080 --> 00:01:13,240
然后第一步把声音变成模型能吃的东西

23
00:01:13,240 --> 00:01:16,020
比如说声波到一串数字

24
00:01:16,020 --> 00:01:20,020
每秒采集16000个点

25
00:01:20,020 --> 00:01:22,740
气压变化变成一串整数

26
00:01:22,740 --> 00:01:24,520
第二部分呢

27
00:01:24,520 --> 00:01:25,740
就是切成一帧一帧的

28
00:01:25,740 --> 00:01:27,120
每帧20秒

29
00:01:27,120 --> 00:01:28,200
这么短的时间内

30
00:01:28,200 --> 00:01:29,740
声音特征基本不变

31
00:01:29,740 --> 00:01:30,680
那就是20毫秒

32
00:01:30,680 --> 00:01:32,260
20毫秒切变成一帧

33
00:01:32,260 --> 00:01:36,720
所以我们说之前说了一个40毫秒

34
00:01:36,720 --> 00:01:37,600
其实就是两帧嘛

35
00:01:37,600 --> 00:01:42,880
第三个是每帧算出频谱指纹

36
00:01:42,880 --> 00:01:43,920
不看波形

37
00:01:43,920 --> 00:01:46,000
看频率成分

38
00:01:46,000 --> 00:01:47,280
人耳听的就是这个

39
00:01:47,280 --> 00:01:50,300
其实这个是声学的里面

40
00:01:50,300 --> 00:01:51,360
最基本的原理

41
00:01:51,360 --> 00:01:51,900
这个呢

42
00:01:51,900 --> 00:01:52,220
其实

43
00:01:52,220 --> 00:01:54,000
我们不用过去顺究

44
00:01:54,000 --> 00:01:55,100
我们简单了解就行了

45
00:01:55,100 --> 00:01:57,180
为本要转成平谱了

46
00:01:57,180 --> 00:01:58,380
同一个R

47
00:01:58,380 --> 00:02:00,260
你说和我说

48
00:02:00,260 --> 00:02:01,580
大声说和小声说

49
00:02:01,580 --> 00:02:02,520
波形完全不同

50
00:02:02,520 --> 00:02:06,480
但平谱成分分布是相似的

51
00:02:06,480 --> 00:02:07,720
换成平谱模型

52
00:02:07,720 --> 00:02:09,440
要认的东西就稳定多了

53
00:02:09,440 --> 00:02:10,220
说白就说

54
00:02:10,220 --> 00:02:11,120
你说R

55
00:02:11,120 --> 00:02:12,160
我说R

56
00:02:12,160 --> 00:02:12,540
是吧

57
00:02:12,540 --> 00:02:13,540
那两个声音大小

58
00:02:13,540 --> 00:02:14,320
可能是不同的

59
00:02:14,320 --> 00:02:15,320
波形是不同的

60
00:02:15,320 --> 00:02:16,880
所以用通过波形是不行的

61
00:02:16,880 --> 00:02:18,360
要把它换成平谱

62
00:02:18,360 --> 00:02:20,960
模型才能更好的识别

63
00:02:20,960 --> 00:02:22,060
好

64
00:02:22,060 --> 00:02:25,400
第二步就是升学模型逐针猜

65
00:02:25,400 --> 00:02:26,680
这一针是什么音

66
00:02:26,680 --> 00:02:29,360
比如说你好

67
00:02:29,360 --> 00:02:30,120
对吧

68
00:02:30,120 --> 00:02:30,440
为例

69
00:02:30,440 --> 00:02:33,260
那模型对每一针独立给出一个猜测

70
00:02:33,260 --> 00:02:35,620
他不知道这个字

71
00:02:35,620 --> 00:02:37,720
只知道这一针最像什么

72
00:02:37,720 --> 00:02:39,480
那每一针猜测

73
00:02:39,480 --> 00:02:39,980
比如理

74
00:02:39,980 --> 00:02:40,740
这个是空格

75
00:02:40,740 --> 00:02:42,220
空的空白的

76
00:02:42,220 --> 00:02:43,200
理理理

77
00:02:43,200 --> 00:02:45,200
空白好好好

78
00:02:45,200 --> 00:02:46,060
空白

79
00:02:46,060 --> 00:02:47,980
灰色是空白的符号

80
00:02:47,980 --> 00:02:50,020
意思是这一针没有内容输出

81
00:02:50,020 --> 00:02:50,460
有了它

82
00:02:50,460 --> 00:02:51,540
模型就不用纠结

83
00:02:51,540 --> 00:02:53,140
一个音到底跨几帧

84
00:02:53,140 --> 00:02:55,560
说了长就多图几帧

85
00:02:55,560 --> 00:02:57,440
同样字交给下一步收拾

86
00:02:57,440 --> 00:02:59,120
所以说我们可以看到

87
00:02:59,120 --> 00:02:59,960
可能是这么个情况

88
00:02:59,960 --> 00:03:02,320
这个是每一帧的情况

89
00:03:02,320 --> 00:03:02,880
每一帧

90
00:03:02,880 --> 00:03:04,520
每一帧我们说了20毫秒

91
00:03:04,520 --> 00:03:06,360
第三步就解码

92
00:03:06,360 --> 00:03:09,140
合并重复去掉空白

93
00:03:09,140 --> 00:03:10,580
比如说我们可以看到

94
00:03:10,580 --> 00:03:11,780
有理理理好好好

95
00:03:11,780 --> 00:03:14,780
那肯定是合并成理好就行了

96
00:03:14,780 --> 00:03:18,080
然后再把这些空白的地方

97
00:03:18,080 --> 00:03:19,420
再把它去掉就变成理好

98
00:03:19,420 --> 00:03:20,880
其实就是用户就说了个理好

99
00:03:20,880 --> 00:03:22,640
好好好

100
00:03:22,640 --> 00:03:23,480
你好

101
00:03:25,640 --> 00:03:27,680
那么连续的这个

102
00:03:28,480 --> 00:03:29,560
空白的哈

103
00:03:30,080 --> 00:03:31,120
这段没人说话

104
00:03:31,120 --> 00:03:32,600
所以很多流色ASR

105
00:03:32,600 --> 00:03:34,400
根本不用单独跑一个VED

106
00:03:35,040 --> 00:03:37,600
直接读升学模型空白输出就行

107
00:03:37,600 --> 00:03:38,760
判断依据识别

108
00:03:38,760 --> 00:03:39,760
本身天然一致

109
00:03:40,240 --> 00:03:41,600
那你说白了就是有

110
00:03:41,600 --> 00:03:43,920
你这个针里面有这么多空白的符号

111
00:03:43,920 --> 00:03:45,600
那肯定就是你没说话

112
00:03:45,600 --> 00:03:46,320
对吧

113
00:03:46,480 --> 00:03:49,000
那这个就相当于做了一个VED的判断

114
00:03:49,560 --> 00:03:50,680
就是你没说话

115
00:03:51,880 --> 00:03:56,320
这就是断句原理那张图里第二步的最常见的实验方式之一

116
00:03:57,520 --> 00:03:58,360
空白符嘛

117
00:03:58,360 --> 00:03:59,240
那不就断句了吗

118
00:03:59,240 --> 00:04:00,200
我就认为你没说话

119
00:04:01,440 --> 00:04:02,920
比如说你这里面

120
00:04:02,920 --> 00:04:04,360
如果你这空白的地方

121
00:04:04,640 --> 00:04:05,600
并且时间呢

122
00:04:06,560 --> 00:04:07,560
这个比如说

123
00:04:08,520 --> 00:04:09,920
这个超过300毫秒了

124
00:04:09,920 --> 00:04:10,840
那我们认为

125
00:04:11,520 --> 00:04:12,600
你这句话说完了

126
00:04:13,040 --> 00:04:13,560
这个意思

127
00:04:15,560 --> 00:04:19,480
那么为什么牛市的识别会边缩边改呢

128
00:04:19,480 --> 00:04:20,840
对吧

129
00:04:20,840 --> 00:04:22,760
为什么他会边说边改

130
00:04:22,760 --> 00:04:26,200
比如他听到0.3秒

131
00:04:26,200 --> 00:04:27,520
离

132
00:04:27,520 --> 00:04:30,440
听到0.7秒

133
00:04:30,440 --> 00:04:30,900
离好

134
00:04:30,900 --> 00:04:32,880
听到1.2秒

135
00:04:32,880 --> 00:04:33,440
离好吗

136
00:04:33,440 --> 00:04:36,480
那么在这个时候

137
00:04:36,480 --> 00:04:37,680
他静音不是很久

138
00:04:37,680 --> 00:04:38,220
也就是说

139
00:04:38,220 --> 00:04:39,480
他是低于300毫秒的

140
00:04:40,160 --> 00:04:43,160
其实这句话就没说完

141
00:04:43,160 --> 00:04:45,420
离好这时候也没说完

142
00:04:45,420 --> 00:04:47,940
但是到这个时候

143
00:04:47,940 --> 00:04:49,020
可能后面的静音

144
00:04:49,020 --> 00:04:50,260
已经超过300毫秒了

145
00:04:50,860 --> 00:04:51,660
就final了

146
00:04:51,780 --> 00:04:53,380
就静音够久定稿了

147
00:04:56,260 --> 00:04:58,980
我们来看右边这段话

148
00:05:08,220 --> 00:05:09,420
我们看右边的解释

149
00:05:09,860 --> 00:05:12,460
流失模型只看过去看不到未来

150
00:05:13,420 --> 00:05:15,100
比如像理解的模型whisper

151
00:05:15,380 --> 00:05:16,380
拿到整段音频

152
00:05:16,380 --> 00:05:19,120
可以前后互相印证

153
00:05:19,120 --> 00:05:20,220
牛4必须在

154
00:05:20,920 --> 00:05:22,220
这个前后互相印证是什么意思

155
00:05:22,220 --> 00:05:24,480
就是他有一段完整的音频

156
00:05:24,480 --> 00:05:27,520
他已经把一个逐字稿全部练完了

157
00:05:27,720 --> 00:05:29,720
那你当然知道前后有什么

158
00:05:30,420 --> 00:05:32,380
语言他会把它改掉的

159
00:05:32,380 --> 00:05:33,820
但牛4必须在

160
00:05:34,180 --> 00:05:36,880
我想聊聊乒乓的时候就输出

161
00:05:37,180 --> 00:05:39,040
而后面是球还是球拍

162
00:05:39,040 --> 00:05:39,920
根本没有发生

163
00:05:40,180 --> 00:05:41,740
他也不知道你是聊乒乓球

164
00:05:41,740 --> 00:05:43,040
还是聊乒乓球猪

165
00:05:43,240 --> 00:05:44,380
还是聊乒乓球技术

166
00:05:44,380 --> 00:05:45,480
还是聊乒乓球拍

167
00:05:46,000 --> 00:05:46,500
对吧

168
00:05:47,780 --> 00:05:49,840
那这个牛市的就需要解决这种问题

169
00:05:50,340 --> 00:05:52,640
所以他边拆边改,改的过程呢

170
00:05:54,180 --> 00:05:54,960
就是这个

171
00:05:55,720 --> 00:05:57,760
就是我们前面说的这两段对吧

172
00:06:01,100 --> 00:06:01,860
就是in term

173
00:06:02,380 --> 00:06:03,920
那么确定不了的就是final的

174
00:06:04,940 --> 00:06:08,260
那么前面这一部分他就逐字就是在屏幕上了

175
00:06:09,040 --> 00:06:09,540
是吧

176
00:06:09,540 --> 00:06:10,820
用户才能看到他的听

177
00:06:11,340 --> 00:06:12,620
那我们经常看到这个

178
00:06:12,880 --> 00:06:14,920
比如说你在用豆包的经常就说你

179
00:06:14,920 --> 00:06:16,520
你问他问题干什么

180
00:06:16,520 --> 00:06:17,120
就是他

181
00:06:17,400 --> 00:06:18,600
或者说别的这种

182
00:06:19,680 --> 00:06:20,480
语音的这种工具

183
00:06:20,480 --> 00:06:22,520
你会发现他的字在逐步在出是吧

184
00:06:22,760 --> 00:06:23,680
就是这个意思

185
00:06:24,520 --> 00:06:26,200
他在编出还帮你改

186
00:06:27,120 --> 00:06:30,000
中文尤其是没有词的编辑

187
00:06:30,000 --> 00:06:31,080
同音字扎堆

188
00:06:31,280 --> 00:06:31,560
对吧

189
00:06:31,560 --> 00:06:32,720
他同音字特别多

190
00:06:32,920 --> 00:06:34,240
全靠上下文区分

191
00:06:34,360 --> 00:06:36,440
而上下文恰恰是流是最缺的部分

192
00:06:36,600 --> 00:06:38,160
这也就是中文识别了

193
00:06:38,280 --> 00:06:39,160
既然用阿里云

194
00:06:39,160 --> 00:06:40,320
而不是用deepgram

195
00:06:40,400 --> 00:06:41,080
那么deepgram

196
00:06:41,080 --> 00:06:42,360
他是对英文字是特别好

197
00:06:42,360 --> 00:06:43,680
包括Yenion Live这些

198
00:06:43,680 --> 00:06:48,080
所以你中文的话你还是用阿里云、火山、腾讯

199
00:06:48,080 --> 00:06:49,040
像这些是吧

200
00:06:49,040 --> 00:06:50,320
他肯定中文制作的好一些

201
00:06:50,320 --> 00:06:51,760
好

202
00:06:51,760 --> 00:06:58,400
那我们上面讲了学会之后怎么用

203
00:06:58,400 --> 00:06:59,080
推理

204
00:06:59,080 --> 00:07:01,440
那他是怎么学会的

205
00:07:01,440 --> 00:07:04,700
他为什么能识别呢

206
00:07:04,700 --> 00:07:07,160
为什么他能识别出来了

207
00:07:07,160 --> 00:07:08,640
背后的原理是什么

208
00:07:08,640 --> 00:07:11,040
第一个就是为他配好的

209
00:07:11,040 --> 00:07:12,460
对的数据

210
00:07:12,460 --> 00:07:14,040
训练数据标注预料

211
00:07:29,192 --> 00:07:31,492
有东北画,对吧,天津画

212
00:07:31,752 --> 00:07:33,792
等等,比如说上海画

213
00:07:34,312 --> 00:07:36,352
那你有这么多这个

214
00:07:36,872 --> 00:07:38,412
方言,普通话

215
00:07:38,912 --> 00:07:39,952
等等这些画

216
00:07:40,192 --> 00:07:41,232
你录给他

217
00:07:42,512 --> 00:07:44,032
他去训练这个数据

218
00:07:45,072 --> 00:07:46,352
这个是和其他的

219
00:07:46,852 --> 00:07:48,912
这个模型是一样的,你得有训练数据

220
00:07:50,692 --> 00:07:55,312
让他自己找规律,然后就训练嘛,就推理嘛,这个是为数据嘛,然后这个就是推理

221
00:07:55,552 --> 00:07:56,072
反向

222
00:07:56,072 --> 00:07:59,192
反向传播掉参数

223
00:07:59,192 --> 00:08:01,452
反复对照这几百万条

224
00:08:01,452 --> 00:08:03,472
没有人告诉他

225
00:08:03,472 --> 00:08:04,652
你这个音长什么样

226
00:08:04,652 --> 00:08:07,372
规律是他自己从数据里总结出来的

227
00:08:07,372 --> 00:08:08,612
学出来是两样东西

228
00:08:08,612 --> 00:08:09,432
升线模型

229
00:08:09,432 --> 00:08:11,312
认这一针是什么音

230
00:08:11,312 --> 00:08:13,352
语言模型

231
00:08:13,352 --> 00:08:15,492
同音字里挑哪个更适合上下文

232
00:08:15,492 --> 00:08:17,212
说白了

233
00:08:17,212 --> 00:08:18,512
就你的这个模型

234
00:08:18,512 --> 00:08:20,452
给他的数据越多

235
00:08:20,452 --> 00:08:21,252
训练的越

236
00:08:21,252 --> 00:08:23,992
找的规律越好

237
00:08:23,992 --> 00:08:24,592
对吧

238
00:08:24,592 --> 00:08:26,192
你的参数越多

239
00:08:26,192 --> 00:08:27,872
那当然你的这个识别

240
00:08:27,872 --> 00:08:28,972
可能越准

241
00:08:28,972 --> 00:08:33,112
那么学会了就可以放话

242
00:08:33,112 --> 00:08:33,532
对吧

243
00:08:33,532 --> 00:08:35,612
和推理

244
00:08:35,612 --> 00:08:37,272
那这一段

245
00:08:37,272 --> 00:08:39,132
从没听过的录音

246
00:08:39,132 --> 00:08:41,092
换一个没听过的人来说

247
00:08:41,092 --> 00:08:41,852
他也认得出

248
00:08:41,852 --> 00:08:45,412
这就是上边言那个逐针猜的

249
00:08:45,412 --> 00:08:45,992
用的模型

250
00:08:45,992 --> 00:08:47,452
逐针猜嘛

251
00:08:47,452 --> 00:08:48,612
对吧

252
00:08:48,612 --> 00:08:49,572
他要逐针猜

253
00:08:49,572 --> 00:08:50,852
比如说你说

254
00:08:50,852 --> 00:08:52,372
我要去打乒乓

255
00:08:52,372 --> 00:08:53,792
他会猜出乒乓球

256
00:08:53,792 --> 00:08:56,872
乒乓球球拍 乒乓球桌 乒乓球技术

257
00:08:57,632 --> 00:08:58,652
这属于这个

258
00:08:59,932 --> 00:09:01,732
我们说这个ASR用的对应的

259
00:09:01,992 --> 00:09:03,512
背后用的这种声学模型

260
00:09:03,772 --> 00:09:04,032
对吧

261
00:09:04,552 --> 00:09:05,832
或者说这个语言模型

262
00:09:06,332 --> 00:09:08,132
它就具备了这个能力了

263
00:09:08,392 --> 00:09:09,152
它也能猜

264
00:09:09,412 --> 00:09:10,172
能猜你

265
00:09:10,432 --> 00:09:10,952
说什么

266
00:09:11,712 --> 00:09:14,532
就像教小孩真智 你不会给他

267
00:09:14,792 --> 00:09:15,812
讲字形

268
00:09:16,312 --> 00:09:18,112
学理论 就是反复指的说

269
00:09:18,372 --> 00:09:19,132
这是礼

270
00:09:19,912 --> 00:09:20,412
是吧

271
00:09:20,672 --> 00:09:21,952
礼子多了

272
00:09:21,952 --> 00:09:24,252
他以后就会见过了字体语言的粗

273
00:09:24,252 --> 00:09:26,052
模型学的不是规则是见的多

274
00:09:26,052 --> 00:09:27,072
所以训练时

275
00:09:27,332 --> 00:09:28,872
见的少的东西

276
00:09:28,872 --> 00:09:31,432
专业术语人民方言他有容易出错

277
00:09:31,932 --> 00:09:34,492
这也是供应商为什么提供热词表

278
00:09:35,012 --> 00:09:36,552
所以说就算你的中文呢

279
00:09:36,552 --> 00:09:38,852
当然我们用中文的这种

280
00:09:39,352 --> 00:09:40,392
模型更好对吧

281
00:09:40,392 --> 00:09:41,672
英文就用英文的模型

282
00:09:42,432 --> 00:09:42,952
是吧

283
00:09:43,192 --> 00:09:44,732
你别的语言当然也是

284
00:09:45,752 --> 00:09:47,032
这个所以说是有区别的

285
00:09:47,032 --> 00:09:47,552
这么你

286
00:09:47,812 --> 00:09:49,592
你在你的这个voice in的这个项目的时候

287
00:09:49,592 --> 00:09:51,652
你在挑选来一家供应商的时候

288
00:09:51,652 --> 00:09:53,652
那肯定是根据你的业务场景来的

289
00:09:53,652 --> 00:09:55,492
你是什么语言为主

290
00:09:55,492 --> 00:09:57,112
比如说你是粤语的

291
00:09:57,112 --> 00:10:01,672
那当然你要考虑他这个模型对粤语的识别的怎么样

292
00:10:01,672 --> 00:10:03,012
识别好你才用它

293
00:10:03,012 --> 00:10:05,572
而不是说随便拿一个provider

294
00:10:05,572 --> 00:10:06,912
就行不行了

295
00:10:06,912 --> 00:10:09,652
然后我们理解了原理之后

296
00:10:09,652 --> 00:10:14,772
你对去挑选你的ASR的对应的服务

297
00:10:14,772 --> 00:10:16,292
甚至你在做本地的

298
00:10:16,292 --> 00:10:18,012
比如我们有欧拉玛去接本地的模型

299
00:10:18,012 --> 00:10:19,152
你挑哪个模型

300
00:10:19,152 --> 00:10:20,732
那都是有帮助的

301
00:10:20,732 --> 00:10:24,232
好那么关于语音识别的原理的

302
00:10:24,232 --> 00:10:26,192
这个分享了

303
00:10:26,192 --> 00:10:27,932
我们就讲到这里

304
00:10:27,932 --> 00:10:28,732
谢谢大家观看