1
00:00:00,760 --> 00:00:09,720
大家好,今天我們給大家講解這個Voice Agent的一個總體架構啊,以及它的這個完整流程。

2
00:00:10,500 --> 00:00:20,480
我們首先了解一下什麼是Voice Agent。你看這個Voice,Voice就是語音的意思,就是語音智能體。也就是說不僅有語音,還有智能體。

3
00:00:22,020 --> 00:00:29,960
那最簡單的比如說你用豆包的書,或者用LGBT的書,你跟它對話。那這個就是最基本的一個語音智能體。

4
00:00:30,760 --> 00:00:42,280
它裡面有哪些關鍵技術呢?有Web RC,有STD,有TTS,有IIM,這幾個關鍵的技術。這幾個名詞呢,希望大家都去查一下。

5
00:00:42,800 --> 00:00:55,600
那首先,Web RC解決什麼?Web RC解決就是說,音視頻的問題。你比如說你的聲音怎麼採集的,你聲音怎麼播放的,聲音怎麼傳輸的,這個是由Web RC來解決的。

6
00:00:56,120 --> 00:00:58,420
那麼STD是什麼?STD就是

7
00:00:58,920 --> 00:01:00,200
語音轉文字。

8
00:01:00,460 --> 00:01:01,240
也就是說

9
00:01:01,740 --> 00:01:07,120
你語音採集了以後,需要把它轉成文字,這個文字再交給大語音模型。

10
00:01:07,640 --> 00:01:08,660
也就是LM

11
00:01:09,160 --> 00:01:10,200
決定說什麼?

12
00:01:10,440 --> 00:01:15,560
也就是說,LM是把文字拿到以後,它會加工,會思考,會返回。

13
00:01:15,820 --> 00:01:16,840
返回完以後,

14
00:01:17,100 --> 00:01:18,120
這不有文字了嗎?

15
00:01:18,380 --> 00:01:19,400
文字在

16
00:01:19,660 --> 00:01:21,460
第一個,它首先會發到

17
00:01:22,220 --> 00:01:27,100
這個端上,也就是比如,一般我們叫前端,比如網頁裡面,或者手機裡面。

18
00:01:28,120 --> 00:01:34,000
然後不經意發,有可能要播語音,這個播語音就要把這個文字再轉成語音,這個過程叫TTS。

19
00:01:35,020 --> 00:01:37,580
所以說,這裡面有幾個關鍵概念,叫Web RC

20
00:01:37,840 --> 00:01:38,620
STD

21
00:01:38,860 --> 00:01:39,640
TTS

22
00:01:39,900 --> 00:01:40,660
LM

23
00:01:41,420 --> 00:01:41,680
對吧

24
00:01:42,200 --> 00:01:43,980
好,我們來看一下它總體架構

25
00:01:44,760 --> 00:01:45,260
首先是

26
00:01:46,040 --> 00:01:48,340
前端,前端呢,它有一個

27
00:01:48,860 --> 00:01:49,880
比如說可能是瀏覽器

28
00:01:49,880 --> 00:01:51,680
有可能是App

29
00:01:51,920 --> 00:01:53,200
甚至是一個硬件啊,對吧

30
00:01:53,460 --> 00:01:55,000
好,那麼麥克風採集

31
00:01:55,520 --> 00:01:56,280
那音頻

32
00:01:56,800 --> 00:01:59,860
同時音頻可能會播放,然後屏幕上可能有文字

33
00:02:00,120 --> 00:02:01,140
大家用這個

34
00:02:01,400 --> 00:02:03,960
用豆包的時候,你在跟它對話的時候,它不經常有這個

35
00:02:05,240 --> 00:02:07,800
會有文字顯示嗎?另外一個它會播放語音

36
00:02:08,820 --> 00:02:10,100
那麼音頻會上行

37
00:02:10,360 --> 00:02:11,120
到後端

38
00:02:11,640 --> 00:02:12,160
後端呢

39
00:02:12,660 --> 00:02:15,480
它這裡面呢,它會比如說,它一般是Web RC

40
00:02:16,500 --> 00:02:17,520
處理啊,處理完

41
00:02:18,040 --> 00:02:18,800
以後呢,它會

42
00:02:19,060 --> 00:02:20,840
把這個聲音提取出來

43
00:02:21,100 --> 00:02:22,640
然後把這個聲音呢,丟給

44
00:02:22,900 --> 00:02:23,660
STT

45
00:02:23,920 --> 00:02:24,680
STT呢

46
00:02:24,940 --> 00:02:25,460
呃

47
00:02:25,960 --> 00:02:28,520
有幾種,一種是你本地的這種服務,對吧

48
00:02:28,780 --> 00:02:30,060
你可能從歐拉馬來接入

49
00:02:30,320 --> 00:02:32,360
第二種呢,有可能比如說你是阿里的

50
00:02:32,620 --> 00:02:32,880
對吧

51
00:02:33,140 --> 00:02:34,420
也有可能是騰訊的

52
00:02:34,680 --> 00:02:36,720
也有可能是火山引擎的

53
00:02:36,980 --> 00:02:38,000
也可能是MiniMax

54
00:02:38,260 --> 00:02:38,520
是吧

55
00:02:39,540 --> 00:02:40,560
STT,MiniMax

56
00:02:40,820 --> 00:02:41,320
目前

57
00:02:41,840 --> 00:02:42,360
不一定有

58
00:02:42,600 --> 00:02:43,880
然後你可能Deepgram

59
00:02:44,140 --> 00:02:45,420
有可能是ElevenLab

60
00:02:45,680 --> 00:02:47,220
這裡面就有中英文

61
00:02:47,480 --> 00:02:49,780
這裡面我們主要以中英文給大家解釋就行了

62
00:02:50,300 --> 00:02:52,600
那麼由它轉動文字又返回給

63
00:02:52,860 --> 00:02:57,200
這個後端,後端這塊它又會掉的這個IMO,也就是掉的大元模型

64
00:02:57,460 --> 00:02:59,760
這裡面選擇就多了,你比如說你可以用Deepseek

65
00:03:00,020 --> 00:03:01,300
可以用這個

66
00:03:02,060 --> 00:03:04,120
MiniMax,可以用這個Kimi

67
00:03:04,380 --> 00:03:06,940
可以用這個OpenAI,對吧

68
00:03:07,440 --> 00:03:08,460
你可以用

69
00:03:08,720 --> 00:03:10,520
這個很多很多模型啊,都可以用

70
00:03:11,020 --> 00:03:11,540
那

71
00:03:11,800 --> 00:03:13,580
丟給大元模型,大元模型會

72
00:03:13,840 --> 00:03:15,620
對你的畫進行加工

73
00:03:15,880 --> 00:03:17,160
加工完以後

74
00:03:17,420 --> 00:03:18,960
他比如說你今天問了一個

75
00:03:19,220 --> 00:03:21,000
舉個例子,什麼是人工智能對吧

76
00:03:21,260 --> 00:03:22,280
他會肯定回答

77
00:03:23,060 --> 00:03:24,580
一堆這個內容嘛,然後

78
00:03:25,100 --> 00:03:25,860
他返回完

79
00:03:26,120 --> 00:03:27,920
返回他以後,然後後端呢

80
00:03:28,680 --> 00:03:28,940
又

81
00:03:29,200 --> 00:03:29,960
緊接著

82
00:03:30,220 --> 00:03:32,260
調這個TTS模型,這個TTS模型也有

83
00:03:32,520 --> 00:03:34,580
比如火山引擎呢,阿里騰訊呢

84
00:03:34,840 --> 00:03:37,660
對吧,Deepgram,ElevenLav,他都會把

85
00:03:38,160 --> 00:03:39,180
文字再轉成語音

86
00:03:39,440 --> 00:03:40,720
轉成語音呢,他會有一個

87
00:03:40,980 --> 00:03:44,560
實實的流,這個實實流呢,再通過WebSEE技術再傳給前端

88
00:03:44,820 --> 00:03:46,100
前端把聲音播放出來

89
00:03:46,620 --> 00:03:48,660
同時文字部分呢,他也會到前端

90
00:03:49,180 --> 00:03:49,940
那這時候呢

91
00:03:50,960 --> 00:03:51,480
這時候呢

92
00:03:51,740 --> 00:03:53,260
就完成了整個的

93
00:03:54,300 --> 00:03:55,320
完成了整個的這個

94
00:03:55,580 --> 00:03:56,600
voice agent的一個流程

95
00:03:57,360 --> 00:03:59,160
那麼為什麼我們又稱為agent呢

96
00:03:59,920 --> 00:04:00,180
就是

97
00:04:00,940 --> 00:04:02,220
不僅是voice,為什麼agent呢

98
00:04:02,220 --> 00:04:05,300
原因就是因為這裡面有R,就是大語言模型的介入

99
00:04:05,540 --> 00:04:06,320
它是一個智能的

100
00:04:07,080 --> 00:04:08,360
所以說我們就要

101
00:04:08,880 --> 00:04:09,900
voice agent

102
00:04:10,160 --> 00:04:11,440
你既然是agent,那麼

103
00:04:11,700 --> 00:04:15,020
他可能也可以做RE,也就是檢索增加生成

104
00:04:15,280 --> 00:04:15,780
比如說

105
00:04:16,040 --> 00:04:17,060
我經常問的問題

106
00:04:17,320 --> 00:04:18,600
那麼我們可以把這個問題

107
00:04:18,860 --> 00:04:21,940
問的問題以及回答的過程把它傳到相量信息庫裡面

108
00:04:22,180 --> 00:04:25,260
那你問題問多了以後,那麼以後他在

109
00:04:25,780 --> 00:04:28,080
你再問他問題的,他就更懂你了,對吧

110
00:04:28,340 --> 00:04:31,660
更懂你了,知道你以前問過,他會知道怎麼再去回答你

111
00:04:32,180 --> 00:04:32,940
這個就是

112
00:04:33,460 --> 00:04:36,020
接入這個大語言模型的好處

113
00:04:36,260 --> 00:04:38,580
這個你還在這部分進行加強

114
00:04:38,820 --> 00:04:39,600
就這部分加強

115
00:04:40,360 --> 00:04:40,880
好

116
00:04:41,140 --> 00:04:42,660
那下面這邊有解釋

117
00:04:43,440 --> 00:04:45,740
好,我們看到總體的架構,我們再看一下這個

118
00:04:46,000 --> 00:04:48,560
還有完整流程,那這個裡面當然就更詳細了

119
00:04:48,820 --> 00:04:51,880
比如說瀏覽器這邊設計到麥克風採集

120
00:04:52,140 --> 00:04:52,660
對吧

121
00:04:54,180 --> 00:04:54,960
重採樣

122
00:04:55,220 --> 00:04:55,720
分針

123
00:04:56,240 --> 00:04:58,020
這個技術名稱我們不需要長那麼細

124
00:04:58,280 --> 00:04:58,800
暫時

125
00:04:59,300 --> 00:05:00,580
然後二金字上行

126
00:05:00,840 --> 00:05:01,600
對吧

127
00:05:01,860 --> 00:05:02,620
然後

128
00:05:02,880 --> 00:05:03,900
這邊還會分發

129
00:05:04,940 --> 00:05:07,740
那麼這邊有VAD檢測,這個VAD是幹什麼

130
00:05:08,260 --> 00:05:09,540
你跟這個

131
00:05:10,060 --> 00:05:10,560
AI

132
00:05:10,820 --> 00:05:12,360
對話的時候你不經常可能會

133
00:05:12,620 --> 00:05:13,900
打斷他嗎?比如說

134
00:05:14,140 --> 00:05:15,420
他正在回答你某一個

135
00:05:15,680 --> 00:05:17,480
某一段話的時候你突然間打斷他了

136
00:05:17,740 --> 00:05:19,520
那如果是有打斷的話

137
00:05:19,780 --> 00:05:20,300
對吧

138
00:05:20,540 --> 00:05:21,320
有打斷的話

139
00:05:21,580 --> 00:05:22,080
那麼

140
00:05:22,860 --> 00:05:24,380
這時候會丟給LM

141
00:05:24,640 --> 00:05:25,160
對吧

142
00:05:25,420 --> 00:05:26,180
會切這個句子

143
00:05:26,440 --> 00:05:27,460
然後又貼貼的再返回

144
00:05:27,460 --> 00:05:28,220
對吧

145
00:05:28,480 --> 00:05:30,020
如果不打斷的話

146
00:05:30,280 --> 00:05:31,040
不打的話他就

147
00:05:31,560 --> 00:05:32,840
他就整個就是

148
00:05:33,860 --> 00:05:35,140
朝下去走

149
00:05:35,400 --> 00:05:36,160
這個意思

150
00:05:36,680 --> 00:05:37,440
整個朝下去走

151
00:05:37,700 --> 00:05:38,460
直接從這

152
00:05:38,720 --> 00:05:39,240
TDS

153
00:05:39,740 --> 00:05:40,000
這走

154
00:05:40,260 --> 00:05:41,280
然後就返回

155
00:05:41,540 --> 00:05:42,560
再返回到前端

156
00:05:43,340 --> 00:05:44,100
然後進行播放

157
00:05:44,360 --> 00:05:45,120
播放這個聲音

158
00:05:45,640 --> 00:05:46,660
中間發生的過程就是

159
00:05:46,920 --> 00:05:47,680
STT code端

160
00:05:47,940 --> 00:05:48,200
你要走

161
00:05:48,460 --> 00:05:49,220
STT的服務

162
00:05:49,480 --> 00:05:49,740
對吧

163
00:05:49,980 --> 00:05:50,500
你比如說

164
00:05:51,020 --> 00:05:51,780
阿里的呀

165
00:05:52,040 --> 00:05:52,540
Deepgram呀

166
00:05:52,800 --> 00:05:53,060
之類的

167
00:05:53,320 --> 00:05:53,820
那麼

168
00:05:54,600 --> 00:05:55,620
這個如果是Irm的話

169
00:05:55,880 --> 00:05:56,640
你可能接入OpenAI

170
00:05:56,900 --> 00:05:56,900


171
00:05:57,160 --> 00:05:57,920
也可能接入Deepseek

172
00:05:58,180 --> 00:05:58,940
有可能接入Milimax

173
00:05:59,200 --> 00:05:59,460
Kimi

174
00:05:59,720 --> 00:06:00,220
都可以

175
00:06:00,480 --> 00:06:00,740
好

176
00:06:01,000 --> 00:06:01,260
那麼

177
00:06:01,500 --> 00:06:02,280
TDS這塊

178
00:06:02,540 --> 00:06:03,560
通過SSE的流

179
00:06:03,820 --> 00:06:04,320
來進行播放

180
00:06:04,580 --> 00:06:05,600
那麼SSE什麼呢

181
00:06:05,860 --> 00:06:07,900
這個是AI裡面的一個技術

182
00:06:08,160 --> 00:06:09,700
就是一個流式的

183
00:06:10,220 --> 00:06:10,980
返回的

184
00:06:11,240 --> 00:06:11,500
一個

185
00:06:11,740 --> 00:06:12,520
處理技術

186
00:06:13,020 --> 00:06:14,300
就是文字的一個流式

187
00:06:14,560 --> 00:06:15,840
同時還有音頻的

188
00:06:16,620 --> 00:06:17,380
最後到

189
00:06:17,380 --> 00:06:20,700
網頁這段它有音頻的一個流式

190
00:06:21,480 --> 00:06:23,780
所以這個是更詳細的一個完整流程

191
00:06:24,040 --> 00:06:25,820
這裡面涉及到好些技術

192
00:06:26,600 --> 00:06:26,860
好技術

193
00:06:27,100 --> 00:06:27,880
大家這些技術

194
00:06:28,140 --> 00:06:28,900
我們會一步步拆解

195
00:06:29,160 --> 00:06:29,660
給大家講的

196
00:06:30,180 --> 00:06:30,440
那麼

197
00:06:30,700 --> 00:06:32,220
我們對於一個

198
00:06:32,480 --> 00:06:34,280
總體的架構以及完整流程

199
00:06:34,540 --> 00:06:35,300
有所了解以後

200
00:06:35,820 --> 00:06:36,580
你才能搞懂

201
00:06:36,840 --> 00:06:38,620
我們後續要給大家講的

202
00:06:39,400 --> 00:06:39,900
比如說

203
00:06:41,960 --> 00:06:44,000
從Web端到Server端的整個的一個

204
00:06:44,780 --> 00:06:46,820
Voice Agent的一個整個流程

205
00:06:46,820 --> 00:06:48,100
那麼我們這個例子

206
00:06:48,360 --> 00:06:49,120
寫的不是很大

207
00:06:49,380 --> 00:06:50,660
但是通過這個例子

208
00:06:50,920 --> 00:06:51,180
你

209
00:06:51,420 --> 00:06:53,220
能對Voice Agent有一個

210
00:06:53,740 --> 00:06:55,260
完整的流程的一個理解

211
00:06:55,520 --> 00:06:56,040
並且

212
00:06:56,300 --> 00:06:57,820
它是正式辦理能跑起來的一個項目

213
00:06:58,340 --> 00:07:00,140
我們首先是把理論先搞清楚

214
00:07:00,900 --> 00:07:01,660
這裡面

215
00:07:01,920 --> 00:07:02,700
我們可以看到

216
00:07:02,940 --> 00:07:03,980
這裡面我們涉及的各種服務

217
00:07:04,220 --> 00:07:05,000
比如

218
00:07:05,260 --> 00:07:05,760
STTS

219
00:07:06,020 --> 00:07:06,540
IM

220
00:07:06,780 --> 00:07:07,040
這些

221
00:07:07,300 --> 00:07:09,100
大家都可以去採用不同的

222
00:07:09,600 --> 00:07:10,380
Provider

223
00:07:10,620 --> 00:07:11,400
有著供應商的服務

224
00:07:11,660 --> 00:07:12,680
不一定說是非要某一間的

225
00:07:12,940 --> 00:07:13,180
對吧

226
00:07:13,960 --> 00:07:14,460
!

227
00:07:14,460 --> 00:07:17,020
比如說如果說我們英文的我建議大家用Deepgram

228
00:07:17,280 --> 00:07:18,040
Eleven Lab

229
00:07:18,300 --> 00:07:20,600
如果說是中文的建議大家用

230
00:07:21,120 --> 00:07:23,940
阿里的騰訊的火車引擎的像這些

231
00:07:24,180 --> 00:07:25,220
包括Millimax像這些

232
00:07:25,720 --> 00:07:27,000
這樣的話是比較好的

233
00:07:27,260 --> 00:07:27,780
因為

234
00:07:28,020 --> 00:07:29,060
肯定是

235
00:07:30,080 --> 00:07:30,840
各自的模型

236
00:07:31,100 --> 00:07:34,180
它對中英文支持的出來的效果是不一樣的

237
00:07:34,940 --> 00:07:37,240
當然同時它還支持方言的

238
00:07:37,500 --> 00:07:38,520
比如說粵語

239
00:07:39,040 --> 00:07:39,800
對吧

240
00:07:40,320 --> 00:07:40,560
等等

241
00:07:40,820 --> 00:07:42,100
像這些方言的四川話

242
00:07:42,620 --> 00:07:43,120
普通話

243
00:07:43,900 --> 00:07:45,440
像方言普通話都支持的

244
00:07:46,460 --> 00:07:47,220
所以說這個

245
00:07:47,480 --> 00:07:48,760
你要不同的去

246
00:07:49,020 --> 00:07:48,760
測

247
00:07:49,020 --> 00:07:49,520
另外還有一個

248
00:07:49,780 --> 00:07:50,560
VED打斷

249
00:07:50,800 --> 00:07:51,580
VED打斷的

250
00:07:51,840 --> 00:07:52,340
不同的

251
00:07:53,120 --> 00:07:53,880
比如STT

252
00:07:54,140 --> 00:07:54,400
就是

253
00:07:54,640 --> 00:07:55,160
它叫

254
00:07:55,420 --> 00:07:55,680
實際上叫

255
00:07:55,920 --> 00:07:56,960
AS2技術

256
00:07:57,460 --> 00:07:58,480
不同的

257
00:07:59,520 --> 00:08:00,280
廠商提供出來的

258
00:08:00,540 --> 00:08:01,560
質量也是不一樣的

259
00:08:01,820 --> 00:08:02,840
這個都去接

260
00:08:03,360 --> 00:08:04,380
然後我們這裡面當然就接了

261
00:08:04,640 --> 00:08:05,140
有阿里的

262
00:08:05,400 --> 00:08:05,920
有Deepgram

263
00:08:05,920 --> 00:08:06,440
這些

264
00:08:06,940 --> 00:08:09,240
已經有兩個比較好用的給大家接進來了

265
00:08:09,500 --> 00:08:10,520
告訴大家怎麼去接的

266
00:08:11,800 --> 00:08:12,060
好

267
00:08:12,320 --> 00:08:16,420
那麼關於它的這個總體的架構儀完整流程的我們就先講到這裡謝謝大家觀看