WEBVTT
Kind: captions
Language: zh

00:00:01.100 --> 00:00:02.733
大家好我是小木头

00:00:03.266 --> 00:00:03.733
OpenAI

00:00:03.733 --> 00:00:06.366
近期又发布了一系列的新模型

00:00:06.533 --> 00:00:07.433
其中有一款

00:00:07.433 --> 00:00:10.733
是专门做实时的同传或实时翻译

00:00:11.033 --> 00:00:12.333
似乎在 AI 时代

00:00:12.333 --> 00:00:15.000
要做同声传译简单了许多

00:00:15.333 --> 00:00:16.000
本期视频呢

00:00:16.000 --> 00:00:17.633
我们就来实际地体验一下

00:00:17.633 --> 00:00:19.366
这款实时翻译的模型

00:00:19.466 --> 00:00:21.200
看看在同声传译上

00:00:21.200 --> 00:00:22.800
它的质量究竟如何

00:00:23.466 --> 00:00:25.366
我们首先来看个演示吧

00:00:25.966 --> 00:00:26.733
我将模型呢

00:00:26.733 --> 00:00:28.966
封装了一个小小的 Chrome 插件

00:00:29.166 --> 00:00:31.100
AI 把它命名为了 Babelfish

00:00:31.266 --> 00:00:32.766
插件的作用是

00:00:32.766 --> 00:00:36.766
监听当前所浏览的页面中的音频

00:00:36.866 --> 00:00:39.400
将音频呢转换成期望的目标语言

00:00:39.900 --> 00:00:42.366
这个转换当然是语音层面的转换

00:00:42.866 --> 00:00:45.766
我就以过去我所发布的一期视频为例

00:00:45.766 --> 00:00:47.166
这是一个中文视频

00:00:47.366 --> 00:00:50.300
我期望将目标语言呢翻译成英文

00:00:51.500 --> 00:00:52.500
点击 start

00:00:53.866 --> 00:00:57.500
开始监听当前浏览页面中的音频数据

00:00:58.000 --> 00:00:59.466
那我们就来播放一段语音

00:00:59.466 --> 00:01:01.033
看看它的转换效果

00:01:01.633 --> 00:01:04.333
let me introduce its open source design

00:01:04.333 --> 00:01:07.866
the op design in a recent video

00:01:08.633 --> 00:01:12.233
i introduced closed design if you're interested

00:01:15.066 --> 00:01:15.566
you can

00:01:15.566 --> 00:01:18.733
re watch that video here i'll briefly go over its feature

00:01:20.200 --> 00:01:22.333
in cloude design we can create curves

00:01:24.733 --> 00:01:25.533
create blades

00:01:25.533 --> 00:01:27.466
and develop things based on templates

00:01:28.800 --> 00:01:29.566
unfortunately

00:01:29.566 --> 00:01:33.400
clod design is a closed source product for various reasons

00:01:33.400 --> 00:01:35.066
many friends can't use it properly

00:01:37.066 --> 00:01:39.733
in this video we'll introduce an open souls design

00:01:39.733 --> 00:01:41.933
op design is the star of today's shed

00:01:43.933 --> 00:01:46.666
this project is currently very popular and in a short time

00:01:46.666 --> 00:01:48.833
it has gathered nearly twenty thousand stars

00:01:48.833 --> 00:01:51.033
i'll put the project's link in the video description

00:01:51.733 --> 00:01:52.533
interested friends

00:01:52.533 --> 00:01:54.000
please get it from the description

00:01:55.566 --> 00:01:58.066
this is a local first open source clothes design

00:01:58.066 --> 00:02:00.766
let's look at its main core features first

00:02:00.766 --> 00:02:01.933
it's open source

00:02:02.700 --> 00:02:04.400
好了演示就先到这里

00:02:04.500 --> 00:02:06.166
大家觉得质量如何呢

00:02:06.633 --> 00:02:08.400
刚才听到的英文的语音

00:02:08.400 --> 00:02:10.766
完全是通过 OpenAI 的实时翻译

00:02:10.900 --> 00:02:13.400
将我这个视频当中的中文音频

00:02:13.400 --> 00:02:14.300
翻译出来

00:02:15.033 --> 00:02:16.533
如果已经有使用过的朋友

00:02:16.533 --> 00:02:17.900
欢迎在评论区来留言

00:02:17.900 --> 00:02:21.333
你对这个实时模型翻译的效果的体验

00:02:21.500 --> 00:02:23.666
以及在应用中的最佳实践

00:02:23.733 --> 00:02:25.100
看看有没有什么经验

00:02:25.100 --> 00:02:27.066
可以互相分享和学习的

00:02:28.166 --> 00:02:30.266
接下来我们来做两件事情

00:02:30.266 --> 00:02:30.733
首先呢

00:02:30.733 --> 00:02:33.300
是把这个模型做一番清晰的介绍

00:02:33.500 --> 00:02:34.866
接下来再来看看

00:02:34.866 --> 00:02:37.366
我们用它来构建的这款 Chrome 插件

00:02:37.366 --> 00:02:38.566
究竟是怎么回事

00:02:39.266 --> 00:02:41.800
OpenAI 实际上是在 5 月 7 日

00:02:42.233 --> 00:02:44.833
在实时 API 下放了三个新模型

00:02:45.233 --> 00:02:48.400
gpt-realtime-2、gpt-realtime-translate、

00:02:48.400 --> 00:02:49.800
gpt-realtime-whisper

00:02:50.433 --> 00:02:51.833
在官方的

00:02:51.933 --> 00:02:54.166
公告中列出了这三款模型

00:02:54.500 --> 00:02:57.800
gpt-realtime-translate 是我们今天分享的主角

00:02:58.133 --> 00:03:02.100
它做一件事就是把语音同步地翻译

00:03:02.233 --> 00:03:04.700
70 多种输入语言自动检测

00:03:04.700 --> 00:03:06.933
13 种输出语言任选

00:03:06.933 --> 00:03:07.933
中文、英文、

00:03:07.933 --> 00:03:08.766
西班牙语、

00:03:08.766 --> 00:03:09.366
葡萄牙语、

00:03:09.366 --> 00:03:10.966
法语、德语等等

00:03:11.466 --> 00:03:12.600
计费层面来讲

00:03:12.600 --> 00:03:14.800
0.034 美元每分钟

00:03:14.866 --> 00:03:17.566
听一小时英文 audiobook 大约两美元

00:03:17.900 --> 00:03:20.500
比人类同传便宜两个数量级

00:03:21.566 --> 00:03:23.833
但它跟之前的通用语音模型

00:03:23.833 --> 00:03:26.866
加 prompt 让它翻译并不是一回事

00:03:27.133 --> 00:03:29.666
根据 OpenAI 的 Cookbook

00:03:29.666 --> 00:03:30.866
他们说得很清晰

00:03:30.866 --> 00:03:34.433
这个模型是用同传译员的语料训练的

00:03:34.700 --> 00:03:35.933
具备两个特征

00:03:36.433 --> 00:03:39.300
第一,它会等到上下文够了才开口

00:03:39.466 --> 00:03:41.833
不会人家一开口它就抢着翻

00:03:42.300 --> 00:03:44.800
这对语序差异大的语言特别重要

00:03:45.100 --> 00:03:46.200
英语和日语、

00:03:46.300 --> 00:03:49.366
英语和中文谓词、动词位置不一样

00:03:49.700 --> 00:03:51.566
等不够上下文就乱翻

00:03:51.566 --> 00:03:52.533
这是不行的

00:03:53.300 --> 00:03:54.900
如果大家实际地测试一下

00:03:54.900 --> 00:03:56.200
应该能感觉到

00:03:56.400 --> 00:03:59.600
相较于原始的 YouTube 的音频

00:04:00.566 --> 00:04:02.133
实时翻译的音频呢

00:04:02.133 --> 00:04:06.133
在段句子之间呢可能会有一定的延迟

00:04:06.500 --> 00:04:09.533
这也是在上下文等待中产生的效果

00:04:11.333 --> 00:04:14.533
那第二个特征是它能边收音边吐译音

00:04:15.000 --> 00:04:18.333
它并不是 speech to text 转文字再翻译

00:04:18.333 --> 00:04:22.100
再通过 text to speech 这种三段的拼接

00:04:22.133 --> 00:04:25.033
而是端到端的 speech to speech

00:04:25.033 --> 00:04:27.900
做到真正的低延迟连续语音

00:04:28.466 --> 00:04:30.766
API 形态走 realtime endpoint

00:04:30.766 --> 00:04:32.500
浏览器用 WebRTC

00:04:32.500 --> 00:04:34.166
服务端用 WebSocket

00:04:34.600 --> 00:04:36.766
OpenAI Cookbook 有一个 demo

00:04:36.766 --> 00:04:38.366
就是浏览器 tab 翻译

00:04:39.233 --> 00:04:41.233
目前我所分享的这个项目呢

00:04:41.233 --> 00:04:43.200
就是刚才咱们介绍的这个 demo

00:04:43.466 --> 00:04:45.866
我会将链接也放在视频描述中

00:04:46.533 --> 00:04:48.333
那么在刚才我们的演示中

00:04:48.333 --> 00:04:50.300
所用到的 Chrome 插件呢

00:04:50.333 --> 00:04:53.100
就是现在我所分享的这个代码仓库

00:04:53.933 --> 00:04:55.333
这是一个 Chrome 插件

00:04:55.333 --> 00:04:57.100
它包含了三个组件

00:04:57.333 --> 00:04:59.133
一个是 service worker

00:04:59.366 --> 00:05:01.266
一个是 offscreen document

00:05:01.466 --> 00:05:04.066
还包含了一部分 content script

00:05:04.233 --> 00:05:07.233
总共是一个不到 600 行的 TypeScript

00:05:07.233 --> 00:05:08.633
代码非常简单

00:05:08.766 --> 00:05:10.266
感兴趣的朋友可以克隆下来

00:05:10.266 --> 00:05:11.666
自己尝试一下

00:05:11.900 --> 00:05:13.833
大家要做的就是几步

00:05:13.833 --> 00:05:15.333
首先呢克隆项目

00:05:15.600 --> 00:05:17.166
我们通过 npm install、

00:05:17.166 --> 00:05:19.700
npm run build 完成安装与打包

00:05:20.100 --> 00:05:24.600
接下来来到 Chrome 浏览器管理插件

00:05:25.566 --> 00:05:27.233
在插件这里

00:05:27.800 --> 00:05:30.800
首先要在右上角启用开发者模式

00:05:30.933 --> 00:05:32.733
然后点击 Load unpacked

00:05:33.100 --> 00:05:37.066
在 openai-realtime-translate 目录当中选择 dist

00:05:38.233 --> 00:05:40.733
这个目录呢就是打包后生成的目录

00:05:41.533 --> 00:05:43.066
要使用这个插件也很简单

00:05:43.066 --> 00:05:44.733
在扩展这里点击它

00:05:45.000 --> 00:05:46.200
第一次使用的时候呢

00:05:46.200 --> 00:05:47.400
需要配个 API key

00:05:47.733 --> 00:05:50.033
大家去申请一个自己的 OpenAI 的 API key

00:05:50.033 --> 00:05:51.433
可以配置进来保存

00:05:51.933 --> 00:05:54.900
然后任意打开一个标签页

00:05:55.333 --> 00:05:57.600
比如播放一段 YouTube 视频

00:05:58.166 --> 00:05:59.066
那我们现在呢

00:05:59.066 --> 00:06:00.600
再来做一番演示

00:06:01.133 --> 00:06:03.633
比如我想将我这个语音呢

00:06:03.633 --> 00:06:05.400
实时地转换成日语

00:06:07.066 --> 00:06:08.433
来听听效果吧

00:06:21.733 --> 00:06:23.333
so when you take crowd design

00:06:23.333 --> 00:06:24.833
don't need that are you a swaggy

00:06:24.833 --> 00:06:26.033
Johnny Kino to all

00:06:26.066 --> 00:06:28.233
you're Gonna open source a setting of coach Christmas

00:06:28.900 --> 00:06:30.800
open designer Genji Deno

00:06:30.800 --> 00:06:33.066
open source a system that she's only him to not

00:06:33.200 --> 00:06:34.800
he could tell the project to all go stay

00:06:34.800 --> 00:06:37.233
must have to wait for another design yeah

00:06:37.466 --> 00:06:39.333
hope that she's on KTT skinnable

00:06:39.600 --> 00:06:40.933
open course design and open course design and all

00:06:42.166 --> 00:06:43.733
细心的朋友可能注意到了

00:06:43.733 --> 00:06:45.433
刚才我将视频暂停了

00:06:45.433 --> 00:06:46.733
但是呢语音输出还

00:06:46.733 --> 00:06:48.400
在持续了一小段时间

00:06:48.800 --> 00:06:50.500
刚才我们在介绍模型的时候

00:06:50.500 --> 00:06:51.666
也聊到这一块

00:06:52.400 --> 00:06:54.900
模型会收集一段必要的上下文

00:06:54.900 --> 00:06:55.833
再做翻译

00:06:56.033 --> 00:06:59.100
因此呢在这个翻译和语音输出上

00:06:59.100 --> 00:07:01.033
就会有一点小小的延迟

00:07:01.533 --> 00:07:02.966
这也是它的特点吧

00:07:03.233 --> 00:07:04.866
来看看开销情况吧

00:07:04.866 --> 00:07:07.900
我大概前前后后在视频录制前的准备

00:07:07.900 --> 00:07:09.666
到视频中的实际分享

00:07:09.666 --> 00:07:12.600
大概在语音的输出上

00:07:12.600 --> 00:07:15.066
花了可能有半小时

00:07:15.866 --> 00:07:19.233
那么目前的开销是 0.9 美元

00:07:19.866 --> 00:07:21.400
那现在就对咱们今天的分享

00:07:21.400 --> 00:07:23.000
做一番简单的总结

00:07:23.100 --> 00:07:25.400
首先呢这个 Chrome 插件

00:07:25.600 --> 00:07:28.866
目前只是抓取的浏览器 Tab 中的音频

00:07:29.266 --> 00:07:31.400
像 Zoom 桌面客户端、

00:07:31.400 --> 00:07:33.966
Spotify 桌面、本地视频播放器等等呢

00:07:33.966 --> 00:07:34.900
是抓不到的

00:07:35.066 --> 00:07:38.100
不过同样的逻辑或者是代码呢

00:07:38.100 --> 00:07:40.533
可以应用到不同的应用场景中

00:07:40.666 --> 00:07:41.866
比如可以覆盖到

00:07:41.866 --> 00:07:43.966
我们刚才介绍的这些桌面应用

00:07:45.800 --> 00:07:48.133
至于具体的技术瓶颈在哪里

00:07:48.200 --> 00:07:51.166
是否操作系统支持音频的捕获呢

00:07:51.266 --> 00:07:53.500
这个就是我们在技术或工程层面

00:07:53.500 --> 00:07:54.700
需要去考虑的了

00:07:55.133 --> 00:07:57.600
第二点是 gpt-realtime-translate

00:07:57.600 --> 00:07:59.166
这个实时翻译模型

00:07:59.166 --> 00:08:01.800
它不支持自定义的提示词、

00:08:01.866 --> 00:08:03.866
术语表、配音选择

00:08:04.000 --> 00:08:06.900
如果你的场景对专有名词敏感

00:08:08.800 --> 00:08:11.400
它会按一般的翻译习惯来处理

00:08:12.233 --> 00:08:13.966
另外一点很重要的是

00:08:14.100 --> 00:08:15.300
session 没有关掉

00:08:15.300 --> 00:08:16.133
它会一直跑

00:08:16.133 --> 00:08:17.100
一直计费

00:08:18.733 --> 00:08:20.266
一小时大概两美元

00:08:20.266 --> 00:08:22.333
4 小时大概就是 8 美元

00:08:22.533 --> 00:08:24.766
因此呢大家用完记得要 stop

00:08:25.966 --> 00:08:27.333
在插件层面

00:08:27.333 --> 00:08:30.333
大家就记得点击 start translating、restart

00:08:31.000 --> 00:08:32.766
如果大家不再使用了呢

00:08:32.933 --> 00:08:34.366
点击 stop 停掉它

00:08:34.366 --> 00:08:36.866
否则呢这个 session 一直会开着

00:08:37.733 --> 00:08:39.600
整个扩展从想法到能用

00:08:39.600 --> 00:08:42.266
目前在 AI 时代编写起来非常的快

00:08:42.300 --> 00:08:44.133
给到它足够的参考文档、

00:08:44.133 --> 00:08:46.966
API 手册很快就搞定了

00:08:46.966 --> 00:08:48.700
你可以把它分叉下来

00:08:48.900 --> 00:08:50.300
加自己想要的功能

00:08:50.766 --> 00:08:52.866
比如把字幕导出到 SRT、

00:08:53.000 --> 00:08:54.866
加 hotkey 切换语言、

00:08:55.066 --> 00:08:59.333
把译音导进 OBS 做直播同传等等这

00:08:59.333 --> 00:09:00.966
都是直接的扩展点

00:09:02.166 --> 00:09:05.066
OpenAI 把同传从需要专业人员的服务

00:09:05.066 --> 00:09:06.800
变成了一个 API 调用

00:09:07.066 --> 00:09:09.233
这会不会是同传这个职业

00:09:09.566 --> 00:09:10.333
也是跨语言

00:09:10.333 --> 00:09:13.433
内容消费方式的一个明显的转折点呢

00:09:13.533 --> 00:09:14.666
大家怎么看呢

00:09:15.533 --> 00:09:16.966
那感兴趣的朋友

00:09:16.966 --> 00:09:19.666
或者在这方面已经有应用经验的朋友

00:09:19.866 --> 00:09:21.333
欢迎在评论区留言

00:09:21.466 --> 00:09:22.933
那今天的分享就到这里

00:09:22.933 --> 00:09:24.033
感谢大家收看

00:09:24.033 --> 00:09:25.100
我们下期再见