WEBVTT
Kind: captions
Language: en

00:00:00.240 --> 00:00:02.869 align:start position:0%
 
Can<00:00:00.640><c> the</c><00:00:00.880><c> open</c><00:00:01.199><c> AI</c><00:00:01.680><c> whisper</c><00:00:02.159><c> model</c><00:00:02.560><c> do</c>

00:00:02.869 --> 00:00:02.879 align:start position:0%
Can the open AI whisper model do
 

00:00:02.879 --> 00:00:05.910 align:start position:0%
Can the open AI whisper model do
streaming<00:00:03.360><c> ASR?</c><00:00:04.480><c> My</c><00:00:04.720><c> name</c><00:00:04.799><c> is</c><00:00:05.040><c> Bai.</c><00:00:05.520><c> I'm</c><00:00:05.759><c> a</c>

00:00:05.910 --> 00:00:05.920 align:start position:0%
streaming ASR? My name is Bai. I'm a
 

00:00:05.920 --> 00:00:07.909 align:start position:0%
streaming ASR? My name is Bai. I'm a
machine<00:00:06.160><c> learning</c><00:00:06.480><c> engineer</c><00:00:06.960><c> and</c><00:00:07.200><c> a</c><00:00:07.440><c> PhD</c><00:00:07.759><c> in</c>

00:00:07.909 --> 00:00:07.919 align:start position:0%
machine learning engineer and a PhD in
 

00:00:07.919 --> 00:00:09.990 align:start position:0%
machine learning engineer and a PhD in
natural<00:00:08.240><c> language</c><00:00:08.559><c> processing.</c><00:00:09.280><c> And</c><00:00:09.519><c> today</c><00:00:09.760><c> I</c>

00:00:09.990 --> 00:00:10.000 align:start position:0%
natural language processing. And today I
 

00:00:10.000 --> 00:00:11.990 align:start position:0%
natural language processing. And today I
will<00:00:10.160><c> answer</c><00:00:10.400><c> this</c><00:00:10.639><c> question.</c><00:00:11.440><c> So</c><00:00:11.599><c> if</c><00:00:11.840><c> you</c>

00:00:11.990 --> 00:00:12.000 align:start position:0%
will answer this question. So if you
 

00:00:12.000 --> 00:00:13.830 align:start position:0%
will answer this question. So if you
haven't<00:00:12.160><c> seen</c><00:00:12.400><c> the</c><00:00:12.559><c> whisper</c><00:00:12.960><c> model,</c><00:00:13.360><c> it</c><00:00:13.519><c> is</c><00:00:13.679><c> a</c>

00:00:13.830 --> 00:00:13.840 align:start position:0%
haven't seen the whisper model, it is a
 

00:00:13.840 --> 00:00:16.150 align:start position:0%
haven't seen the whisper model, it is a
speechtoext<00:00:14.559><c> model.</c><00:00:15.200><c> It</c><00:00:15.440><c> is</c><00:00:15.599><c> trained</c><00:00:15.920><c> on</c>

00:00:16.150 --> 00:00:16.160 align:start position:0%
speechtoext model. It is trained on
 

00:00:16.160 --> 00:00:19.830 align:start position:0%
speechtoext model. It is trained on
about<00:00:16.640><c> 100</c><00:00:17.039><c> languages</c><00:00:17.760><c> on</c><00:00:18.240><c> 680,000</c><00:00:19.359><c> hours</c><00:00:19.680><c> of</c>

00:00:19.830 --> 00:00:19.840 align:start position:0%
about 100 languages on 680,000 hours of
 

00:00:19.840 --> 00:00:21.910 align:start position:0%
about 100 languages on 680,000 hours of
data.<00:00:20.480><c> The</c><00:00:20.720><c> model</c><00:00:20.960><c> architecture</c><00:00:21.520><c> is</c><00:00:21.680><c> a</c>

00:00:21.910 --> 00:00:21.920 align:start position:0%
data. The model architecture is a
 

00:00:21.920 --> 00:00:24.230 align:start position:0%
data. The model architecture is a
encoder<00:00:22.400><c> decoder</c><00:00:22.880><c> transformer</c><00:00:23.439><c> model</c><00:00:23.760><c> and</c><00:00:24.000><c> it</c>

00:00:24.230 --> 00:00:24.240 align:start position:0%
encoder decoder transformer model and it
 

00:00:24.240 --> 00:00:27.269 align:start position:0%
encoder decoder transformer model and it
comes<00:00:24.560><c> in</c><00:00:25.119><c> five</c><00:00:25.439><c> different</c><00:00:25.680><c> sizes</c><00:00:26.640><c> and</c><00:00:26.880><c> it</c><00:00:27.039><c> is</c>

00:00:27.269 --> 00:00:27.279 align:start position:0%
comes in five different sizes and it is
 

00:00:27.279 --> 00:00:29.269 align:start position:0%
comes in five different sizes and it is
growing<00:00:27.599><c> in</c><00:00:27.840><c> popularity</c><00:00:28.400><c> recently</c><00:00:28.960><c> because</c>

00:00:29.269 --> 00:00:29.279 align:start position:0%
growing in popularity recently because
 

00:00:29.279 --> 00:00:31.830 align:start position:0%
growing in popularity recently because
it<00:00:29.519><c> is</c><00:00:29.679><c> robust</c><00:00:30.160><c> to</c><00:00:30.400><c> noise</c><00:00:30.640><c> and</c><00:00:30.880><c> accents.</c><00:00:31.679><c> So</c>

00:00:31.830 --> 00:00:31.840 align:start position:0%
it is robust to noise and accents. So
 

00:00:31.840 --> 00:00:33.510 align:start position:0%
it is robust to noise and accents. So
the<00:00:32.079><c> question</c><00:00:32.239><c> of</c><00:00:32.399><c> the</c><00:00:32.559><c> day</c><00:00:32.719><c> is</c><00:00:33.040><c> can</c><00:00:33.280><c> this</c>

00:00:33.510 --> 00:00:33.520 align:start position:0%
the question of the day is can this
 

00:00:33.520 --> 00:00:35.830 align:start position:0%
the question of the day is can this
model<00:00:33.760><c> do</c><00:00:34.079><c> streaming</c><00:00:34.559><c> speech</c><00:00:34.880><c> recognition.</c>

00:00:35.830 --> 00:00:35.840 align:start position:0%
model do streaming speech recognition.
 

00:00:35.840 --> 00:00:37.430 align:start position:0%
model do streaming speech recognition.
So<00:00:36.160><c> first</c><00:00:36.320><c> of</c><00:00:36.399><c> all</c><00:00:36.640><c> what</c><00:00:36.800><c> do</c><00:00:36.960><c> we</c><00:00:37.120><c> mean</c><00:00:37.200><c> by</c>

00:00:37.430 --> 00:00:37.440 align:start position:0%
So first of all what do we mean by
 

00:00:37.440 --> 00:00:40.310 align:start position:0%
So first of all what do we mean by
streaming<00:00:37.840><c> ASR?</c><00:00:38.879><c> Well,</c><00:00:39.360><c> automated</c><00:00:40.000><c> speech</c>

00:00:40.310 --> 00:00:40.320 align:start position:0%
streaming ASR? Well, automated speech
 

00:00:40.320 --> 00:00:42.549 align:start position:0%
streaming ASR? Well, automated speech
recognition<00:00:40.800><c> can</c><00:00:41.040><c> come</c><00:00:41.200><c> in</c><00:00:41.360><c> two</c><00:00:41.600><c> forms</c><00:00:42.160><c> batch</c>

00:00:42.549 --> 00:00:42.559 align:start position:0%
recognition can come in two forms batch
 

00:00:42.559 --> 00:00:44.549 align:start position:0%
recognition can come in two forms batch
and<00:00:42.800><c> streaming.</c><00:00:43.520><c> Batch</c><00:00:43.920><c> means</c><00:00:44.160><c> that</c><00:00:44.320><c> the</c>

00:00:44.549 --> 00:00:44.559 align:start position:0%
and streaming. Batch means that the
 

00:00:44.559 --> 00:00:46.310 align:start position:0%
and streaming. Batch means that the
model<00:00:44.800><c> needs</c><00:00:45.040><c> to</c><00:00:45.360><c> take</c><00:00:45.520><c> a</c><00:00:45.760><c> bunch</c><00:00:45.920><c> of</c><00:00:46.079><c> speech</c>

00:00:46.310 --> 00:00:46.320 align:start position:0%
model needs to take a bunch of speech
 

00:00:46.320 --> 00:00:48.549 align:start position:0%
model needs to take a bunch of speech
and<00:00:46.640><c> produce</c><00:00:46.960><c> a</c><00:00:47.120><c> bunch</c><00:00:47.280><c> of</c><00:00:47.440><c> text.</c><00:00:48.239><c> But</c>

00:00:48.549 --> 00:00:48.559 align:start position:0%
and produce a bunch of text. But
 

00:00:48.559 --> 00:00:51.910 align:start position:0%
and produce a bunch of text. But
streaming<00:00:49.039><c> ASR</c><00:00:50.000><c> is</c><00:00:50.320><c> when</c><00:00:50.640><c> the</c><00:00:50.800><c> model</c><00:00:51.200><c> needs</c><00:00:51.520><c> to</c>

00:00:51.910 --> 00:00:51.920 align:start position:0%
streaming ASR is when the model needs to
 

00:00:51.920 --> 00:00:54.950 align:start position:0%
streaming ASR is when the model needs to
produce<00:00:52.480><c> an</c><00:00:52.719><c> output</c><00:00:53.199><c> as</c><00:00:53.600><c> the</c><00:00:54.079><c> speaker</c><00:00:54.480><c> is</c>

00:00:54.950 --> 00:00:54.960 align:start position:0%
produce an output as the speaker is
 

00:00:54.960 --> 00:00:57.670 align:start position:0%
produce an output as the speaker is
saying<00:00:55.280><c> things</c><00:00:55.920><c> with</c><00:00:56.239><c> a</c><00:00:56.480><c> delay</c><00:00:56.879><c> of</c><00:00:57.199><c> not</c><00:00:57.440><c> more</c>

00:00:57.670 --> 00:00:57.680 align:start position:0%
saying things with a delay of not more
 

00:00:57.680 --> 00:01:00.150 align:start position:0%
saying things with a delay of not more
than<00:00:57.840><c> a</c><00:00:58.079><c> few</c><00:00:58.239><c> seconds.</c><00:00:59.039><c> So</c><00:00:59.359><c> for</c><00:00:59.440><c> example,</c><00:00:59.920><c> if</c>

00:01:00.150 --> 00:01:00.160 align:start position:0%
than a few seconds. So for example, if
 

00:01:00.160 --> 00:01:02.150 align:start position:0%
than a few seconds. So for example, if
you're<00:01:00.399><c> listening</c><00:01:00.719><c> to</c><00:01:00.879><c> a</c><00:01:01.039><c> live</c><00:01:01.359><c> broadcast,</c>

00:01:02.150 --> 00:01:02.160 align:start position:0%
you're listening to a live broadcast,
 

00:01:02.160 --> 00:01:04.950 align:start position:0%
you're listening to a live broadcast,
you<00:01:02.399><c> don't</c><00:01:02.559><c> want</c><00:01:02.960><c> to</c><00:01:03.920><c> first</c><00:01:04.400><c> record</c><00:01:04.720><c> the</c>

00:01:04.950 --> 00:01:04.960 align:start position:0%
you don't want to first record the
 

00:01:04.960 --> 00:01:06.710 align:start position:0%
you don't want to first record the
entire<00:01:05.360><c> broadcast</c><00:01:05.840><c> and</c><00:01:06.080><c> then</c><00:01:06.240><c> produce</c><00:01:06.560><c> the</c>

00:01:06.710 --> 00:01:06.720 align:start position:0%
entire broadcast and then produce the
 

00:01:06.720 --> 00:01:09.350 align:start position:0%
entire broadcast and then produce the
subtitles,<00:01:07.760><c> you</c><00:01:07.920><c> want</c><00:01:08.080><c> it</c><00:01:08.320><c> to</c><00:01:08.479><c> appear</c><00:01:08.880><c> at</c><00:01:09.119><c> most</c>

00:01:09.350 --> 00:01:09.360 align:start position:0%
subtitles, you want it to appear at most
 

00:01:09.360 --> 00:01:12.390 align:start position:0%
subtitles, you want it to appear at most
a<00:01:09.600><c> few</c><00:01:09.840><c> seconds</c><00:01:10.320><c> after</c><00:01:11.040><c> it</c><00:01:11.200><c> is</c><00:01:11.439><c> said.</c><00:01:12.159><c> And</c>

00:01:12.390 --> 00:01:12.400 align:start position:0%
a few seconds after it is said. And
 

00:01:12.400 --> 00:01:14.950 align:start position:0%
a few seconds after it is said. And
generally<00:01:12.880><c> you'll</c><00:01:13.200><c> expect</c><00:01:13.680><c> streaming</c><00:01:14.159><c> ASR</c><00:01:14.720><c> to</c>

00:01:14.950 --> 00:01:14.960 align:start position:0%
generally you'll expect streaming ASR to
 

00:01:14.960 --> 00:01:17.350 align:start position:0%
generally you'll expect streaming ASR to
be<00:01:15.200><c> a</c><00:01:15.520><c> few</c><00:01:15.760><c> percentage</c><00:01:16.240><c> points</c><00:01:16.880><c> lower</c>

00:01:17.350 --> 00:01:17.360 align:start position:0%
be a few percentage points lower
 

00:01:17.360 --> 00:01:19.830 align:start position:0%
be a few percentage points lower
accuracy<00:01:18.000><c> than</c><00:01:18.240><c> batch</c><00:01:18.560><c> ASR</c><00:01:19.280><c> because</c><00:01:19.600><c> the</c>

00:01:19.830 --> 00:01:19.840 align:start position:0%
accuracy than batch ASR because the
 

00:01:19.840 --> 00:01:22.230 align:start position:0%
accuracy than batch ASR because the
model<00:01:20.159><c> does</c><00:01:20.400><c> not</c><00:01:20.560><c> have</c><00:01:20.880><c> access</c><00:01:21.280><c> to</c><00:01:21.759><c> all</c><00:01:22.000><c> the</c>

00:01:22.230 --> 00:01:22.240 align:start position:0%
model does not have access to all the
 

00:01:22.240 --> 00:01:24.630 align:start position:0%
model does not have access to all the
context<00:01:22.960><c> only</c><00:01:23.200><c> the</c><00:01:23.439><c> point</c><00:01:23.759><c> up</c><00:01:24.000><c> until</c><00:01:24.400><c> the</c>

00:01:24.630 --> 00:01:24.640 align:start position:0%
context only the point up until the
 

00:01:24.640 --> 00:01:27.030 align:start position:0%
context only the point up until the
current<00:01:24.960><c> word.</c><00:01:25.600><c> The</c><00:01:25.840><c> question</c><00:01:26.159><c> of</c><00:01:26.479><c> how</c><00:01:26.640><c> to</c><00:01:26.799><c> do</c>

00:01:27.030 --> 00:01:27.040 align:start position:0%
current word. The question of how to do
 

00:01:27.040 --> 00:01:29.350 align:start position:0%
current word. The question of how to do
streaming<00:01:27.439><c> ASR</c><00:01:28.159><c> came</c><00:01:28.320><c> up</c><00:01:28.720><c> when</c><00:01:28.960><c> I</c><00:01:29.200><c> was</c>

00:01:29.350 --> 00:01:29.360 align:start position:0%
streaming ASR came up when I was
 

00:01:29.360 --> 00:01:32.069 align:start position:0%
streaming ASR came up when I was
building<00:01:29.680><c> the</c><00:01:29.920><c> voice</c><00:01:30.240><c> writer.</c><00:01:31.119><c> This</c><00:01:31.280><c> is</c><00:01:31.360><c> an</c><00:01:31.600><c> AI</c>

00:01:32.069 --> 00:01:32.079 align:start position:0%
building the voice writer. This is an AI
 

00:01:32.079 --> 00:01:33.749 align:start position:0%
building the voice writer. This is an AI
writing<00:01:32.479><c> assistant</c><00:01:32.880><c> that</c><00:01:33.119><c> works</c><00:01:33.360><c> in</c><00:01:33.520><c> two</c>

00:01:33.749 --> 00:01:33.759 align:start position:0%
writing assistant that works in two
 

00:01:33.759 --> 00:01:35.830 align:start position:0%
writing assistant that works in two
steps.<00:01:34.320><c> In</c><00:01:34.479><c> the</c><00:01:34.640><c> first</c><00:01:34.799><c> step,</c><00:01:35.200><c> you</c><00:01:35.439><c> just</c><00:01:35.600><c> speak</c>

00:01:35.830 --> 00:01:35.840 align:start position:0%
steps. In the first step, you just speak
 

00:01:35.840 --> 00:01:37.830 align:start position:0%
steps. In the first step, you just speak
your<00:01:36.079><c> thoughts</c><00:01:36.400><c> without</c><00:01:36.960><c> worrying</c><00:01:37.439><c> too</c><00:01:37.600><c> much</c>

00:01:37.830 --> 00:01:37.840 align:start position:0%
your thoughts without worrying too much
 

00:01:37.840 --> 00:01:39.590 align:start position:0%
your thoughts without worrying too much
about<00:01:38.000><c> the</c><00:01:38.240><c> grammar.</c><00:01:38.799><c> And</c><00:01:39.040><c> in</c><00:01:39.200><c> the</c><00:01:39.360><c> second</c>

00:01:39.590 --> 00:01:39.600 align:start position:0%
about the grammar. And in the second
 

00:01:39.600 --> 00:01:41.670 align:start position:0%
about the grammar. And in the second
step,<00:01:40.000><c> the</c><00:01:40.240><c> AI</c><00:01:40.640><c> corrects</c><00:01:40.960><c> the</c><00:01:41.200><c> grammar</c><00:01:41.439><c> for</c>

00:01:41.670 --> 00:01:41.680 align:start position:0%
step, the AI corrects the grammar for
 

00:01:41.680 --> 00:01:44.149 align:start position:0%
step, the AI corrects the grammar for
you.<00:01:42.400><c> I've</c><00:01:42.720><c> been</c><00:01:42.880><c> finding</c><00:01:43.119><c> it</c><00:01:43.360><c> super</c><00:01:43.680><c> helpful</c>

00:01:44.149 --> 00:01:44.159 align:start position:0%
you. I've been finding it super helpful
 

00:01:44.159 --> 00:01:45.590 align:start position:0%
you. I've been finding it super helpful
for<00:01:44.400><c> writing</c><00:01:44.720><c> all</c><00:01:44.880><c> kinds</c><00:01:45.119><c> of</c><00:01:45.280><c> things,</c>

00:01:45.590 --> 00:01:45.600 align:start position:0%
for writing all kinds of things,
 

00:01:45.600 --> 00:01:47.910 align:start position:0%
for writing all kinds of things,
including<00:01:46.079><c> emails,</c><00:01:46.720><c> blog</c><00:01:46.960><c> posts,</c><00:01:47.360><c> and</c><00:01:47.600><c> Slack</c>

00:01:47.910 --> 00:01:47.920 align:start position:0%
including emails, blog posts, and Slack
 

00:01:47.920 --> 00:01:50.550 align:start position:0%
including emails, blog posts, and Slack
messages.<00:01:48.799><c> You</c><00:01:49.040><c> can</c><00:01:49.119><c> try</c><00:01:49.360><c> it</c><00:01:49.520><c> out</c><00:01:49.680><c> for</c><00:01:49.920><c> free</c><00:01:50.320><c> in</c>

00:01:50.550 --> 00:01:50.560 align:start position:0%
messages. You can try it out for free in
 

00:01:50.560 --> 00:01:52.550 align:start position:0%
messages. You can try it out for free in
the<00:01:50.720><c> link</c><00:01:50.960><c> here.</c><00:01:51.200><c> And</c><00:01:51.360><c> I'll</c><00:01:51.520><c> also</c><00:01:51.840><c> post</c><00:01:52.159><c> a</c><00:01:52.320><c> link</c>

00:01:52.550 --> 00:01:52.560 align:start position:0%
the link here. And I'll also post a link
 

00:01:52.560 --> 00:01:54.550 align:start position:0%
the link here. And I'll also post a link
in<00:01:52.720><c> the</c><00:01:52.880><c> description</c><00:01:53.200><c> of</c><00:01:53.360><c> the</c><00:01:53.600><c> video.</c><00:01:54.240><c> Now,</c>

00:01:54.550 --> 00:01:54.560 align:start position:0%
in the description of the video. Now,
 

00:01:54.560 --> 00:01:56.870 align:start position:0%
in the description of the video. Now,
back<00:01:54.720><c> to</c><00:01:54.960><c> the</c><00:01:55.280><c> video.</c><00:01:56.000><c> First</c><00:01:56.159><c> of</c><00:01:56.320><c> all,</c><00:01:56.560><c> why</c><00:01:56.720><c> is</c>

00:01:56.870 --> 00:01:56.880 align:start position:0%
back to the video. First of all, why is
 

00:01:56.880 --> 00:01:59.590 align:start position:0%
back to the video. First of all, why is
this<00:01:57.040><c> even</c><00:01:57.360><c> difficult?</c><00:01:58.159><c> Why</c><00:01:58.320><c> is</c><00:01:58.560><c> it</c><00:01:59.119><c> not</c><00:01:59.360><c> so</c>

00:01:59.590 --> 00:01:59.600 align:start position:0%
this even difficult? Why is it not so
 

00:01:59.600 --> 00:02:01.590 align:start position:0%
this even difficult? Why is it not so
easy<00:01:59.920><c> to</c><00:02:00.159><c> use</c><00:02:00.320><c> the</c><00:02:00.560><c> Whisper</c><00:02:01.040><c> model</c><00:02:01.280><c> for</c>

00:02:01.590 --> 00:02:01.600 align:start position:0%
easy to use the Whisper model for
 

00:02:01.600 --> 00:02:04.709 align:start position:0%
easy to use the Whisper model for
streaming<00:02:02.000><c> ASR?</c><00:02:03.200><c> The</c><00:02:03.439><c> issue</c><00:02:03.680><c> is</c><00:02:03.920><c> Whisper</c><00:02:04.479><c> is</c>

00:02:04.709 --> 00:02:04.719 align:start position:0%
streaming ASR? The issue is Whisper is
 

00:02:04.719 --> 00:02:07.830 align:start position:0%
streaming ASR? The issue is Whisper is
trained<00:02:05.119><c> to</c><00:02:05.759><c> process</c><00:02:06.399><c> input</c><00:02:06.799><c> audio</c><00:02:07.200><c> of</c><00:02:07.520><c> 30</c>

00:02:07.830 --> 00:02:07.840 align:start position:0%
trained to process input audio of 30
 

00:02:07.840 --> 00:02:10.389 align:start position:0%
trained to process input audio of 30
seconds<00:02:08.160><c> long.</c><00:02:08.879><c> If</c><00:02:09.119><c> the</c><00:02:09.360><c> audio</c><00:02:09.679><c> is</c><00:02:10.000><c> less</c><00:02:10.239><c> than</c>

00:02:10.389 --> 00:02:10.399 align:start position:0%
seconds long. If the audio is less than
 

00:02:10.399 --> 00:02:12.630 align:start position:0%
seconds long. If the audio is less than
30<00:02:10.720><c> seconds,</c><00:02:11.200><c> you</c><00:02:11.360><c> can</c><00:02:11.520><c> pad</c><00:02:11.760><c> it</c><00:02:11.920><c> to</c><00:02:12.080><c> be</c><00:02:12.239><c> longer.</c>

00:02:12.630 --> 00:02:12.640 align:start position:0%
30 seconds, you can pad it to be longer.
 

00:02:12.640 --> 00:02:15.030 align:start position:0%
30 seconds, you can pad it to be longer.
So,<00:02:12.959><c> this</c><00:02:13.200><c> is</c><00:02:13.360><c> not</c><00:02:13.599><c> too</c><00:02:13.840><c> much</c><00:02:14.000><c> of</c><00:02:14.080><c> a</c><00:02:14.319><c> problem.</c>

00:02:15.030 --> 00:02:15.040 align:start position:0%
So, this is not too much of a problem.
 

00:02:15.040 --> 00:02:17.750 align:start position:0%
So, this is not too much of a problem.
But<00:02:15.280><c> what</c><00:02:15.520><c> if</c><00:02:15.840><c> your</c><00:02:16.160><c> audio</c><00:02:16.560><c> is</c><00:02:16.879><c> longer</c><00:02:17.280><c> than</c><00:02:17.520><c> 30</c>

00:02:17.750 --> 00:02:17.760 align:start position:0%
But what if your audio is longer than 30
 

00:02:17.760 --> 00:02:20.070 align:start position:0%
But what if your audio is longer than 30
seconds?<00:02:18.720><c> There</c><00:02:18.879><c> is</c><00:02:19.040><c> no</c><00:02:19.280><c> way</c><00:02:19.440><c> to</c><00:02:19.599><c> feed</c><00:02:19.840><c> into</c>

00:02:20.070 --> 00:02:20.080 align:start position:0%
seconds? There is no way to feed into
 

00:02:20.080 --> 00:02:22.470 align:start position:0%
seconds? There is no way to feed into
Whisper<00:02:20.640><c> anything</c><00:02:21.120><c> longer</c><00:02:21.440><c> than</c><00:02:21.680><c> 30</c><00:02:21.920><c> seconds.</c>

00:02:22.470 --> 00:02:22.480 align:start position:0%
Whisper anything longer than 30 seconds.
 

00:02:22.480 --> 00:02:24.309 align:start position:0%
Whisper anything longer than 30 seconds.
So,<00:02:22.720><c> the</c><00:02:22.959><c> first</c><00:02:23.280><c> thing</c><00:02:23.520><c> you</c><00:02:23.760><c> might</c><00:02:23.920><c> think</c><00:02:24.080><c> of</c>

00:02:24.309 --> 00:02:24.319 align:start position:0%
So, the first thing you might think of
 

00:02:24.319 --> 00:02:26.550 align:start position:0%
So, the first thing you might think of
is<00:02:24.800><c> what</c><00:02:24.959><c> if</c><00:02:25.120><c> you</c><00:02:25.360><c> just</c><00:02:25.599><c> split</c><00:02:25.920><c> it</c><00:02:26.080><c> up</c><00:02:26.239><c> into</c>

00:02:26.550 --> 00:02:26.560 align:start position:0%
is what if you just split it up into
 

00:02:26.560 --> 00:02:28.790 align:start position:0%
is what if you just split it up into
chunks<00:02:26.959><c> of</c><00:02:27.200><c> 30</c><00:02:27.520><c> seconds</c><00:02:27.760><c> at</c><00:02:27.920><c> a</c><00:02:28.080><c> time</c><00:02:28.480><c> and</c>

00:02:28.790 --> 00:02:28.800 align:start position:0%
chunks of 30 seconds at a time and
 

00:02:28.800 --> 00:02:31.670 align:start position:0%
chunks of 30 seconds at a time and
process<00:02:29.280><c> each</c><00:02:29.599><c> chunk</c><00:02:30.000><c> one</c><00:02:30.239><c> at</c><00:02:30.400><c> a</c><00:02:30.640><c> time.</c><00:02:31.280><c> Well,</c>

00:02:31.670 --> 00:02:31.680 align:start position:0%
process each chunk one at a time. Well,
 

00:02:31.680 --> 00:02:33.670 align:start position:0%
process each chunk one at a time. Well,
the<00:02:31.920><c> first</c><00:02:32.160><c> problem</c><00:02:32.400><c> is</c><00:02:32.879><c> you</c><00:02:33.120><c> might</c><00:02:33.280><c> split</c><00:02:33.519><c> it</c>

00:02:33.670 --> 00:02:33.680 align:start position:0%
the first problem is you might split it
 

00:02:33.680 --> 00:02:35.750 align:start position:0%
the first problem is you might split it
in<00:02:33.840><c> the</c><00:02:34.000><c> middle</c><00:02:34.160><c> of</c><00:02:34.239><c> the</c><00:02:34.400><c> word.</c><00:02:35.200><c> Like</c><00:02:35.440><c> if</c><00:02:35.599><c> you</c>

00:02:35.750 --> 00:02:35.760 align:start position:0%
in the middle of the word. Like if you
 

00:02:35.760 --> 00:02:37.589 align:start position:0%
in the middle of the word. Like if you
split<00:02:36.080><c> it</c><00:02:36.480><c> while</c><00:02:36.800><c> in</c><00:02:37.040><c> the</c><00:02:37.200><c> middle</c><00:02:37.360><c> of</c><00:02:37.440><c> the</c>

00:02:37.589 --> 00:02:37.599 align:start position:0%
split it while in the middle of the
 

00:02:37.599 --> 00:02:40.070 align:start position:0%
split it while in the middle of the
word,<00:02:38.080><c> this</c><00:02:38.400><c> word</c><00:02:38.720><c> will</c><00:02:39.120><c> probably</c><00:02:39.599><c> not</c><00:02:39.920><c> be</c>

00:02:40.070 --> 00:02:40.080 align:start position:0%
word, this word will probably not be
 

00:02:40.080 --> 00:02:42.150 align:start position:0%
word, this word will probably not be
recognized<00:02:40.560><c> correctly.</c><00:02:41.360><c> The</c><00:02:41.599><c> second</c><00:02:41.840><c> problem</c>

00:02:42.150 --> 00:02:42.160 align:start position:0%
recognized correctly. The second problem
 

00:02:42.160 --> 00:02:44.630 align:start position:0%
recognized correctly. The second problem
is<00:02:42.480><c> the</c><00:02:42.720><c> latency</c><00:02:43.200><c> will</c><00:02:43.519><c> be</c><00:02:43.680><c> really</c><00:02:43.920><c> high.</c><00:02:44.400><c> So</c>

00:02:44.630 --> 00:02:44.640 align:start position:0%
is the latency will be really high. So
 

00:02:44.640 --> 00:02:46.790 align:start position:0%
is the latency will be really high. So
imagine<00:02:45.040><c> if</c><00:02:45.280><c> you're</c><00:02:45.599><c> processing</c><00:02:46.080><c> audio,</c>

00:02:46.790 --> 00:02:46.800 align:start position:0%
imagine if you're processing audio,
 

00:02:46.800 --> 00:02:49.110 align:start position:0%
imagine if you're processing audio,
you're<00:02:47.040><c> waiting</c><00:02:47.280><c> for</c><00:02:47.680><c> audio</c><00:02:48.160><c> to</c><00:02:48.400><c> fill</c><00:02:48.640><c> up</c>

00:02:49.110 --> 00:02:49.120 align:start position:0%
you're waiting for audio to fill up
 

00:02:49.120 --> 00:02:52.630 align:start position:0%
you're waiting for audio to fill up
these<00:02:49.599><c> 30</c><00:02:49.920><c> seconds</c><00:02:50.640><c> and</c><00:02:50.879><c> only</c><00:02:51.280><c> then</c><00:02:52.000><c> will</c><00:02:52.319><c> you</c>

00:02:52.630 --> 00:02:52.640 align:start position:0%
these 30 seconds and only then will you
 

00:02:52.640 --> 00:02:55.110 align:start position:0%
these 30 seconds and only then will you
process<00:02:53.200><c> this</c><00:02:53.519><c> entire</c><00:02:54.000><c> chunk</c><00:02:54.560><c> and</c><00:02:54.720><c> then</c><00:02:54.959><c> the</c>

00:02:55.110 --> 00:02:55.120 align:start position:0%
process this entire chunk and then the
 

00:02:55.120 --> 00:02:57.509 align:start position:0%
process this entire chunk and then the
latency<00:02:56.080><c> um</c><00:02:56.160><c> for</c><00:02:56.319><c> for</c><00:02:56.560><c> the</c><00:02:56.720><c> first</c><00:02:56.879><c> word</c><00:02:57.200><c> will</c>

00:02:57.509 --> 00:02:57.519 align:start position:0%
latency um for for the first word will
 

00:02:57.519 --> 00:03:00.070 align:start position:0%
latency um for for the first word will
be<00:02:57.599><c> up</c><00:02:57.840><c> to</c><00:02:58.080><c> 30</c><00:02:58.319><c> seconds.</c><00:02:58.959><c> Fortunately,</c><00:02:59.920><c> there</c>

00:03:00.070 --> 00:03:00.080 align:start position:0%
be up to 30 seconds. Fortunately, there
 

00:03:00.080 --> 00:03:02.630 align:start position:0%
be up to 30 seconds. Fortunately, there
is<00:03:00.319><c> an</c><00:03:00.480><c> open</c><00:03:00.720><c> source</c><00:03:01.120><c> report</c><00:03:01.519><c> called</c><00:03:02.159><c> Whisper</c>

00:03:02.630 --> 00:03:02.640 align:start position:0%
is an open source report called Whisper
 

00:03:02.640 --> 00:03:05.110 align:start position:0%
is an open source report called Whisper
Streaming<00:03:03.280><c> that</c><00:03:03.599><c> does</c><00:03:03.840><c> this</c><00:03:04.000><c> for</c><00:03:04.239><c> you</c><00:03:04.640><c> and</c><00:03:04.879><c> it</c>

00:03:05.110 --> 00:03:05.120 align:start position:0%
Streaming that does this for you and it
 

00:03:05.120 --> 00:03:08.550 align:start position:0%
Streaming that does this for you and it
turns<00:03:05.440><c> the</c><00:03:05.680><c> Whisper</c><00:03:06.239><c> model</c><00:03:06.800><c> into</c><00:03:07.599><c> a</c><00:03:08.159><c> streaming</c>

00:03:08.550 --> 00:03:08.560 align:start position:0%
turns the Whisper model into a streaming
 

00:03:08.560 --> 00:03:11.350 align:start position:0%
turns the Whisper model into a streaming
ASR<00:03:09.120><c> system.</c><00:03:09.920><c> There</c><00:03:10.159><c> are</c><00:03:10.319><c> instructions</c><00:03:10.959><c> on</c>

00:03:11.350 --> 00:03:11.360 align:start position:0%
ASR system. There are instructions on
 

00:03:11.360 --> 00:03:13.509 align:start position:0%
ASR system. There are instructions on
how<00:03:11.680><c> to</c><00:03:11.920><c> set</c><00:03:12.080><c> this</c><00:03:12.319><c> up.</c><00:03:12.720><c> So,</c><00:03:12.959><c> let's</c><00:03:13.200><c> try</c><00:03:13.360><c> it</c>

00:03:13.509 --> 00:03:13.519 align:start position:0%
how to set this up. So, let's try it
 

00:03:13.519 --> 00:03:16.790 align:start position:0%
how to set this up. So, let's try it
out.<00:03:14.159><c> So,</c><00:03:14.319><c> I</c><00:03:14.560><c> run</c><00:03:14.800><c> this</c><00:03:15.040><c> command</c><00:03:15.440><c> to</c><00:03:16.000><c> start</c><00:03:16.480><c> a</c>

00:03:16.790 --> 00:03:16.800 align:start position:0%
out. So, I run this command to start a
 

00:03:16.800 --> 00:03:19.110 align:start position:0%
out. So, I run this command to start a
model<00:03:17.120><c> running</c><00:03:17.680><c> um</c><00:03:17.920><c> it's</c><00:03:18.159><c> using</c><00:03:18.400><c> a</c><00:03:18.560><c> whisper</c>

00:03:19.110 --> 00:03:19.120 align:start position:0%
model running um it's using a whisper
 

00:03:19.120 --> 00:03:21.110 align:start position:0%
model running um it's using a whisper
small<00:03:19.360><c> model</c><00:03:19.840><c> and</c><00:03:20.000><c> then</c><00:03:20.159><c> I</c><00:03:20.319><c> go</c><00:03:20.480><c> to</c><00:03:20.640><c> this</c><00:03:20.800><c> tab</c>

00:03:21.110 --> 00:03:21.120 align:start position:0%
small model and then I go to this tab
 

00:03:21.120 --> 00:03:23.910 align:start position:0%
small model and then I go to this tab
and<00:03:21.760><c> run</c><00:03:22.000><c> another</c><00:03:22.319><c> command</c><00:03:22.879><c> and</c><00:03:23.200><c> I</c><00:03:23.440><c> just</c><00:03:23.599><c> start</c>

00:03:23.910 --> 00:03:23.920 align:start position:0%
and run another command and I just start
 

00:03:23.920 --> 00:03:26.630 align:start position:0%
and run another command and I just start
speaking.<00:03:24.720><c> So</c><00:03:25.280><c> you</c><00:03:25.519><c> can</c><00:03:25.680><c> pretty</c><00:03:26.000><c> quickly</c><00:03:26.480><c> see</c>

00:03:26.630 --> 00:03:26.640 align:start position:0%
speaking. So you can pretty quickly see
 

00:03:26.640 --> 00:03:29.990 align:start position:0%
speaking. So you can pretty quickly see
that<00:03:26.959><c> it's</c><00:03:27.680><c> um</c><00:03:28.159><c> picking</c><00:03:28.480><c> it</c><00:03:28.720><c> up</c><00:03:29.280><c> and</c><00:03:29.599><c> it's</c>

00:03:29.990 --> 00:03:30.000 align:start position:0%
that it's um picking it up and it's
 

00:03:30.000 --> 00:03:32.550 align:start position:0%
that it's um picking it up and it's
pretty<00:03:30.319><c> quick</c><00:03:30.560><c> and</c><00:03:30.879><c> responsive</c><00:03:31.599><c> and</c><00:03:31.920><c> it</c><00:03:32.239><c> even</c>

00:03:32.550 --> 00:03:32.560 align:start position:0%
pretty quick and responsive and it even
 

00:03:32.560 --> 00:03:35.030 align:start position:0%
pretty quick and responsive and it even
gives<00:03:33.040><c> timestamps.</c><00:03:33.840><c> So</c><00:03:34.319><c> um</c><00:03:34.400><c> it's</c><00:03:34.640><c> pretty</c><00:03:34.799><c> cool</c>

00:03:35.030 --> 00:03:35.040 align:start position:0%
gives timestamps. So um it's pretty cool
 

00:03:35.040 --> 00:03:37.509 align:start position:0%
gives timestamps. So um it's pretty cool
demo.<00:03:35.599><c> Um</c><00:03:36.080><c> now</c><00:03:36.319><c> let's</c><00:03:36.640><c> talk</c><00:03:36.799><c> about</c><00:03:37.040><c> how</c><00:03:37.280><c> this</c>

00:03:37.509 --> 00:03:37.519 align:start position:0%
demo. Um now let's talk about how this
 

00:03:37.519 --> 00:03:40.789 align:start position:0%
demo. Um now let's talk about how this
works.<00:03:38.959><c> So</c><00:03:39.120><c> we</c><00:03:39.360><c> start</c><00:03:39.599><c> with</c><00:03:39.760><c> this</c><00:03:40.000><c> audio</c><00:03:40.319><c> file</c>

00:03:40.789 --> 00:03:40.799 align:start position:0%
works. So we start with this audio file
 

00:03:40.799 --> 00:03:43.589 align:start position:0%
works. So we start with this audio file
and<00:03:41.040><c> what</c><00:03:41.280><c> we</c><00:03:41.440><c> do</c><00:03:41.680><c> is</c><00:03:42.239><c> we</c><00:03:42.480><c> feed</c><00:03:42.879><c> chunks</c><00:03:43.280><c> of</c>

00:03:43.589 --> 00:03:43.599 align:start position:0%
and what we do is we feed chunks of
 

00:03:43.599 --> 00:03:46.309 align:start position:0%
and what we do is we feed chunks of
increasing<00:03:44.239><c> size</c><00:03:44.640><c> into</c><00:03:44.959><c> whisper.</c><00:03:45.840><c> The</c><00:03:46.080><c> size</c>

00:03:46.309 --> 00:03:46.319 align:start position:0%
increasing size into whisper. The size
 

00:03:46.319 --> 00:03:48.390 align:start position:0%
increasing size into whisper. The size
of<00:03:46.480><c> these</c><00:03:46.720><c> chunks</c><00:03:47.200><c> is</c><00:03:47.519><c> configured</c><00:03:48.000><c> by</c><00:03:48.159><c> the</c>

00:03:48.390 --> 00:03:48.400 align:start position:0%
of these chunks is configured by the
 

00:03:48.400 --> 00:03:50.470 align:start position:0%
of these chunks is configured by the
minimum<00:03:48.879><c> chunk</c><00:03:49.200><c> size</c><00:03:49.440><c> parameter</c><00:03:50.000><c> which</c><00:03:50.319><c> by</c>

00:03:50.470 --> 00:03:50.480 align:start position:0%
minimum chunk size parameter which by
 

00:03:50.480 --> 00:03:52.550 align:start position:0%
minimum chunk size parameter which by
default<00:03:50.799><c> is</c><00:03:51.040><c> 1</c><00:03:51.360><c> second.</c><00:03:51.920><c> So</c><00:03:52.159><c> in</c><00:03:52.400><c> each</c>

00:03:52.550 --> 00:03:52.560 align:start position:0%
default is 1 second. So in each
 

00:03:52.560 --> 00:03:54.869 align:start position:0%
default is 1 second. So in each
iteration,<00:03:53.200><c> we</c><00:03:53.599><c> increase</c><00:03:53.840><c> the</c><00:03:54.159><c> size</c><00:03:54.400><c> of</c><00:03:54.640><c> this</c>

00:03:54.869 --> 00:03:54.879 align:start position:0%
iteration, we increase the size of this
 

00:03:54.879 --> 00:03:57.830 align:start position:0%
iteration, we increase the size of this
buffer<00:03:55.360><c> by</c><00:03:55.680><c> 1</c><00:03:55.920><c> second</c><00:03:56.400><c> and</c><00:03:56.720><c> feed</c><00:03:57.280><c> all</c><00:03:57.519><c> of</c><00:03:57.680><c> it</c>

00:03:57.830 --> 00:03:57.840 align:start position:0%
buffer by 1 second and feed all of it
 

00:03:57.840 --> 00:04:00.309 align:start position:0%
buffer by 1 second and feed all of it
into<00:03:58.159><c> whisper.</c><00:03:59.120><c> And</c><00:03:59.280><c> this</c><00:03:59.519><c> process</c><00:03:59.840><c> continues</c>

00:04:00.309 --> 00:04:00.319 align:start position:0%
into whisper. And this process continues
 

00:04:00.319 --> 00:04:02.550 align:start position:0%
into whisper. And this process continues
until<00:04:00.799><c> we</c><00:04:01.040><c> hit</c><00:04:01.200><c> a</c><00:04:01.439><c> end</c><00:04:01.599><c> of</c><00:04:01.760><c> sentence</c><00:04:02.080><c> marker</c>

00:04:02.550 --> 00:04:02.560 align:start position:0%
until we hit a end of sentence marker
 

00:04:02.560 --> 00:04:05.110 align:start position:0%
until we hit a end of sentence marker
like<00:04:02.799><c> a</c><00:04:03.040><c> period</c><00:04:03.360><c> or</c><00:04:03.599><c> a</c><00:04:03.760><c> question</c><00:04:04.000><c> mark.</c><00:04:04.879><c> Then</c>

00:04:05.110 --> 00:04:05.120 align:start position:0%
like a period or a question mark. Then
 

00:04:05.120 --> 00:04:07.190 align:start position:0%
like a period or a question mark. Then
we<00:04:05.360><c> move</c><00:04:05.519><c> the</c><00:04:05.680><c> buffer</c><00:04:06.080><c> forward</c><00:04:06.480><c> and</c><00:04:06.720><c> start</c><00:04:06.959><c> the</c>

00:04:07.190 --> 00:04:07.200 align:start position:0%
we move the buffer forward and start the
 

00:04:07.200 --> 00:04:09.670 align:start position:0%
we move the buffer forward and start the
process<00:04:07.599><c> again.</c><00:04:08.159><c> So</c><00:04:08.400><c> each</c><00:04:08.640><c> piece</c><00:04:08.879><c> of</c><00:04:09.040><c> audio</c><00:04:09.439><c> is</c>

00:04:09.670 --> 00:04:09.680 align:start position:0%
process again. So each piece of audio is
 

00:04:09.680 --> 00:04:11.990 align:start position:0%
process again. So each piece of audio is
processed<00:04:10.080><c> by</c><00:04:10.319><c> whisper</c><00:04:10.720><c> multiple</c><00:04:11.200><c> times</c><00:04:11.760><c> as</c>

00:04:11.990 --> 00:04:12.000 align:start position:0%
processed by whisper multiple times as
 

00:04:12.000 --> 00:04:14.309 align:start position:0%
processed by whisper multiple times as
many<00:04:12.239><c> times</c><00:04:12.480><c> as</c><00:04:12.640><c> it</c><00:04:12.879><c> takes</c><00:04:13.200><c> until</c><00:04:13.599><c> we</c><00:04:13.840><c> hit</c><00:04:14.080><c> the</c>

00:04:14.309 --> 00:04:14.319 align:start position:0%
many times as it takes until we hit the
 

00:04:14.319 --> 00:04:16.069 align:start position:0%
many times as it takes until we hit the
end<00:04:14.400><c> of</c><00:04:14.560><c> the</c><00:04:14.720><c> sentence.</c><00:04:15.360><c> The</c><00:04:15.599><c> reasons</c><00:04:15.920><c> for</c>

00:04:16.069 --> 00:04:16.079 align:start position:0%
end of the sentence. The reasons for
 

00:04:16.079 --> 00:04:18.390 align:start position:0%
end of the sentence. The reasons for
this<00:04:16.239><c> is</c><00:04:16.560><c> whisper</c><00:04:16.959><c> is</c><00:04:17.120><c> trained</c><00:04:17.440><c> on</c><00:04:17.680><c> sentences.</c>

00:04:18.390 --> 00:04:18.400 align:start position:0%
this is whisper is trained on sentences.
 

00:04:18.400 --> 00:04:20.629 align:start position:0%
this is whisper is trained on sentences.
So,<00:04:18.560><c> it</c><00:04:18.799><c> gives</c><00:04:18.959><c> the</c><00:04:19.280><c> best</c><00:04:19.519><c> result</c><00:04:20.079><c> when</c><00:04:20.400><c> the</c>

00:04:20.629 --> 00:04:20.639 align:start position:0%
So, it gives the best result when the
 

00:04:20.639 --> 00:04:22.950 align:start position:0%
So, it gives the best result when the
start<00:04:20.880><c> of</c><00:04:21.040><c> the</c><00:04:21.199><c> audio</c><00:04:21.680><c> aligns</c><00:04:22.240><c> with</c><00:04:22.560><c> the</c><00:04:22.800><c> start</c>

00:04:22.950 --> 00:04:22.960 align:start position:0%
start of the audio aligns with the start
 

00:04:22.960 --> 00:04:25.270 align:start position:0%
start of the audio aligns with the start
of<00:04:23.120><c> a</c><00:04:23.280><c> sentence.</c><00:04:23.919><c> And</c><00:04:24.080><c> that's</c><00:04:24.320><c> why</c><00:04:24.560><c> the</c><00:04:24.800><c> buffer</c>

00:04:25.270 --> 00:04:25.280 align:start position:0%
of a sentence. And that's why the buffer
 

00:04:25.280 --> 00:04:27.590 align:start position:0%
of a sentence. And that's why the buffer
only<00:04:25.520><c> moves</c><00:04:25.919><c> forward</c><00:04:26.560><c> when</c><00:04:26.960><c> the</c><00:04:27.280><c> previous</c>

00:04:27.590 --> 00:04:27.600 align:start position:0%
only moves forward when the previous
 

00:04:27.600 --> 00:04:29.670 align:start position:0%
only moves forward when the previous
sentence<00:04:27.919><c> is</c><00:04:28.160><c> complete</c><00:04:28.560><c> and</c><00:04:29.040><c> we're</c><00:04:29.360><c> beginning</c>

00:04:29.670 --> 00:04:29.680 align:start position:0%
sentence is complete and we're beginning
 

00:04:29.680 --> 00:04:32.070 align:start position:0%
sentence is complete and we're beginning
a<00:04:29.840><c> new</c><00:04:30.080><c> sentence.</c><00:04:30.800><c> This</c><00:04:31.040><c> way,</c><00:04:31.280><c> Whisper</c><00:04:31.840><c> never</c>

00:04:32.070 --> 00:04:32.080 align:start position:0%
a new sentence. This way, Whisper never
 

00:04:32.080 --> 00:04:33.990 align:start position:0%
a new sentence. This way, Whisper never
needs<00:04:32.320><c> to</c><00:04:32.560><c> start</c><00:04:32.880><c> transcribing</c><00:04:33.520><c> from</c><00:04:33.840><c> the</c>

00:04:33.990 --> 00:04:34.000 align:start position:0%
needs to start transcribing from the
 

00:04:34.000 --> 00:04:35.510 align:start position:0%
needs to start transcribing from the
middle<00:04:34.160><c> of</c><00:04:34.320><c> a</c><00:04:34.479><c> sentence,</c><00:04:34.880><c> which</c><00:04:35.120><c> would</c><00:04:35.360><c> give</c>

00:04:35.510 --> 00:04:35.520 align:start position:0%
middle of a sentence, which would give
 

00:04:35.520 --> 00:04:37.590 align:start position:0%
middle of a sentence, which would give
suboptimal<00:04:36.240><c> results.</c><00:04:37.040><c> In</c><00:04:37.360><c> many</c>

00:04:37.590 --> 00:04:37.600 align:start position:0%
suboptimal results. In many
 

00:04:37.600 --> 00:04:39.990 align:start position:0%
suboptimal results. In many
applications,<00:04:38.479><c> it</c><00:04:38.720><c> is</c><00:04:38.880><c> useful</c><00:04:39.199><c> to</c><00:04:39.440><c> have</c><00:04:39.759><c> some</c>

00:04:39.990 --> 00:04:40.000 align:start position:0%
applications, it is useful to have some
 

00:04:40.000 --> 00:04:42.310 align:start position:0%
applications, it is useful to have some
results<00:04:40.479><c> available</c><00:04:41.040><c> as</c><00:04:41.280><c> soon</c><00:04:41.440><c> as</c><00:04:41.680><c> possible,</c>

00:04:42.310 --> 00:04:42.320 align:start position:0%
results available as soon as possible,
 

00:04:42.320 --> 00:04:44.629 align:start position:0%
results available as soon as possible,
even<00:04:42.560><c> if</c><00:04:42.800><c> they're</c><00:04:43.040><c> not</c><00:04:43.280><c> completely</c><00:04:43.840><c> accurate.</c>

00:04:44.629 --> 00:04:44.639 align:start position:0%
even if they're not completely accurate.
 

00:04:44.639 --> 00:04:46.629 align:start position:0%
even if they're not completely accurate.
In<00:04:44.800><c> the</c><00:04:44.960><c> voice</c><00:04:45.280><c> writer,</c><00:04:45.759><c> I</c><00:04:46.000><c> am</c><00:04:46.160><c> showing</c><00:04:46.400><c> the</c>

00:04:46.629 --> 00:04:46.639 align:start position:0%
In the voice writer, I am showing the
 

00:04:46.639 --> 00:04:48.950 align:start position:0%
In the voice writer, I am showing the
incomplete<00:04:47.199><c> results</c><00:04:47.600><c> in</c><00:04:47.840><c> gray</c><00:04:48.400><c> whereas</c><00:04:48.720><c> the</c>

00:04:48.950 --> 00:04:48.960 align:start position:0%
incomplete results in gray whereas the
 

00:04:48.960 --> 00:04:51.189 align:start position:0%
incomplete results in gray whereas the
confirmed<00:04:49.440><c> results</c><00:04:49.759><c> are</c><00:04:49.919><c> in</c><00:04:50.160><c> black.</c><00:04:50.960><c> The</c>

00:04:51.189 --> 00:04:51.199 align:start position:0%
confirmed results are in black. The
 

00:04:51.199 --> 00:04:52.629 align:start position:0%
confirmed results are in black. The
difference<00:04:51.440><c> is</c><00:04:51.680><c> the</c><00:04:51.919><c> gray</c><00:04:52.160><c> part</c><00:04:52.320><c> of</c><00:04:52.479><c> the</c>

00:04:52.629 --> 00:04:52.639 align:start position:0%
difference is the gray part of the
 

00:04:52.639 --> 00:04:55.189 align:start position:0%
difference is the gray part of the
sentence<00:04:53.199><c> is</c><00:04:53.520><c> unconfirmed.</c><00:04:54.479><c> So</c><00:04:54.639><c> it</c><00:04:54.960><c> may</c>

00:04:55.189 --> 00:04:55.199 align:start position:0%
sentence is unconfirmed. So it may
 

00:04:55.199 --> 00:04:57.110 align:start position:0%
sentence is unconfirmed. So it may
change<00:04:55.600><c> as</c><00:04:55.840><c> the</c><00:04:56.080><c> model</c><00:04:56.320><c> gets</c><00:04:56.800><c> new</c>

00:04:57.110 --> 00:04:57.120 align:start position:0%
change as the model gets new
 

00:04:57.120 --> 00:04:59.670 align:start position:0%
change as the model gets new
information.<00:04:58.240><c> But</c><00:04:58.400><c> the</c><00:04:58.720><c> black</c><00:04:58.960><c> part</c><00:04:59.199><c> is</c><00:04:59.440><c> the</c>

00:04:59.670 --> 00:04:59.680 align:start position:0%
information. But the black part is the
 

00:04:59.680 --> 00:05:01.749 align:start position:0%
information. But the black part is the
confirmed<00:05:00.320><c> results</c><00:05:01.040><c> and</c><00:05:01.360><c> they</c><00:05:01.600><c> are</c>

00:05:01.749 --> 00:05:01.759 align:start position:0%
confirmed results and they are
 

00:05:01.759 --> 00:05:04.629 align:start position:0%
confirmed results and they are
permanent.<00:05:02.880><c> How</c><00:05:03.120><c> this</c><00:05:03.360><c> works</c><00:05:03.680><c> is</c><00:05:04.000><c> using</c><00:05:04.400><c> an</c>

00:05:04.629 --> 00:05:04.639 align:start position:0%
permanent. How this works is using an
 

00:05:04.639 --> 00:05:07.510 align:start position:0%
permanent. How this works is using an
algorithm<00:05:05.360><c> called</c><00:05:05.840><c> local</c><00:05:06.160><c> agreement</c><00:05:06.960><c> with</c><00:05:07.280><c> n</c>

00:05:07.510 --> 00:05:07.520 align:start position:0%
algorithm called local agreement with n
 

00:05:07.520 --> 00:05:09.990 align:start position:0%
algorithm called local agreement with n
equals<00:05:07.919><c> 2.</c><00:05:08.560><c> This</c><00:05:08.800><c> means</c><00:05:08.960><c> that</c><00:05:09.199><c> for</c><00:05:09.360><c> a</c><00:05:09.520><c> token</c><00:05:09.840><c> to</c>

00:05:09.990 --> 00:05:10.000 align:start position:0%
equals 2. This means that for a token to
 

00:05:10.000 --> 00:05:11.990 align:start position:0%
equals 2. This means that for a token to
be<00:05:10.160><c> confirmed,</c><00:05:10.720><c> it</c><00:05:10.960><c> needs</c><00:05:11.120><c> to</c><00:05:11.280><c> be</c><00:05:11.440><c> generated</c>

00:05:11.990 --> 00:05:12.000 align:start position:0%
be confirmed, it needs to be generated
 

00:05:12.000 --> 00:05:14.710 align:start position:0%
be confirmed, it needs to be generated
in<00:05:12.240><c> two</c><00:05:12.560><c> consecutive</c><00:05:13.199><c> audio</c><00:05:13.600><c> buffers.</c><00:05:14.479><c> Let's</c>

00:05:14.710 --> 00:05:14.720 align:start position:0%
in two consecutive audio buffers. Let's
 

00:05:14.720 --> 00:05:17.110 align:start position:0%
in two consecutive audio buffers. Let's
give<00:05:14.960><c> an</c><00:05:15.039><c> example.</c><00:05:16.000><c> Let's</c><00:05:16.240><c> say</c><00:05:16.400><c> in</c><00:05:16.720><c> the</c><00:05:16.960><c> first</c>

00:05:17.110 --> 00:05:17.120 align:start position:0%
give an example. Let's say in the first
 

00:05:17.120 --> 00:05:19.590 align:start position:0%
give an example. Let's say in the first
step,<00:05:17.520><c> the</c><00:05:17.680><c> worst</c><00:05:18.000><c> per</c><00:05:18.240><c> model</c><00:05:18.560><c> outputs</c><00:05:19.199><c> the</c>

00:05:19.590 --> 00:05:19.600 align:start position:0%
step, the worst per model outputs the
 

00:05:19.600 --> 00:05:22.310 align:start position:0%
step, the worst per model outputs the
three<00:05:19.840><c> tokens</c><00:05:20.479><c> if</c><00:05:20.720><c> you</c><00:05:20.880><c> like</c><00:05:21.440><c> and</c><00:05:21.759><c> nothing</c><00:05:22.080><c> is</c>

00:05:22.310 --> 00:05:22.320 align:start position:0%
three tokens if you like and nothing is
 

00:05:22.320 --> 00:05:24.469 align:start position:0%
three tokens if you like and nothing is
confirmed<00:05:22.639><c> at</c><00:05:22.880><c> this</c><00:05:23.120><c> step.</c><00:05:23.919><c> In</c><00:05:24.080><c> the</c><00:05:24.240><c> second</c>

00:05:24.469 --> 00:05:24.479 align:start position:0%
confirmed at this step. In the second
 

00:05:24.479 --> 00:05:26.629 align:start position:0%
confirmed at this step. In the second
step,<00:05:24.960><c> the</c><00:05:25.199><c> model</c><00:05:25.520><c> produces</c><00:05:25.919><c> more</c><00:05:26.160><c> tokens,</c>

00:05:26.629 --> 00:05:26.639 align:start position:0%
step, the model produces more tokens,
 

00:05:26.639 --> 00:05:28.950 align:start position:0%
step, the model produces more tokens,
but<00:05:26.880><c> only</c><00:05:27.120><c> the</c><00:05:27.360><c> first</c><00:05:27.600><c> two</c><00:05:27.840><c> tokens</c><00:05:28.400><c> agree</c><00:05:28.800><c> with</c>

00:05:28.950 --> 00:05:28.960 align:start position:0%
but only the first two tokens agree with
 

00:05:28.960 --> 00:05:30.950 align:start position:0%
but only the first two tokens agree with
the<00:05:29.199><c> previous</c><00:05:29.440><c> step.</c><00:05:29.919><c> So</c><00:05:30.160><c> those</c><00:05:30.479><c> two</c><00:05:30.720><c> are</c>

00:05:30.950 --> 00:05:30.960 align:start position:0%
the previous step. So those two are
 

00:05:30.960 --> 00:05:33.350 align:start position:0%
the previous step. So those two are
confirmed.<00:05:31.919><c> In</c><00:05:32.160><c> the</c><00:05:32.400><c> third</c><00:05:32.560><c> and</c><00:05:32.800><c> the</c><00:05:32.960><c> fourth</c>

00:05:33.350 --> 00:05:33.360 align:start position:0%
confirmed. In the third and the fourth
 

00:05:33.360 --> 00:05:35.990 align:start position:0%
confirmed. In the third and the fourth
steps,<00:05:34.000><c> more</c><00:05:34.240><c> tokens</c><00:05:34.639><c> are</c><00:05:34.960><c> generated.</c><00:05:35.600><c> But</c><00:05:35.759><c> at</c>

00:05:35.990 --> 00:05:36.000 align:start position:0%
steps, more tokens are generated. But at
 

00:05:36.000 --> 00:05:38.310 align:start position:0%
steps, more tokens are generated. But at
each<00:05:36.240><c> step,</c><00:05:36.800><c> it</c><00:05:37.039><c> does</c><00:05:37.199><c> not</c><00:05:37.440><c> confirm</c><00:05:38.000><c> any</c>

00:05:38.310 --> 00:05:38.320 align:start position:0%
each step, it does not confirm any
 

00:05:38.320 --> 00:05:41.990 align:start position:0%
each step, it does not confirm any
tokens<00:05:38.880><c> until</c><00:05:39.520><c> it</c><00:05:39.840><c> is</c><00:05:40.479><c> at</c><00:05:40.560><c> least</c><00:05:41.120><c> generated</c><00:05:41.680><c> in</c>

00:05:41.990 --> 00:05:42.000 align:start position:0%
tokens until it is at least generated in
 

00:05:42.000 --> 00:05:44.310 align:start position:0%
tokens until it is at least generated in
two<00:05:42.160><c> consecutive</c><00:05:42.800><c> chunks.</c><00:05:43.600><c> So</c><00:05:43.759><c> anything</c><00:05:44.080><c> in</c>

00:05:44.310 --> 00:05:44.320 align:start position:0%
two consecutive chunks. So anything in
 

00:05:44.320 --> 00:05:46.950 align:start position:0%
two consecutive chunks. So anything in
the<00:05:44.479><c> gray</c><00:05:44.720><c> part</c><00:05:45.120><c> is</c><00:05:45.440><c> possible</c><00:05:45.840><c> to</c><00:05:46.080><c> change</c><00:05:46.800><c> and</c>

00:05:46.950 --> 00:05:46.960 align:start position:0%
the gray part is possible to change and
 

00:05:46.960 --> 00:05:49.270 align:start position:0%
the gray part is possible to change and
given<00:05:47.360><c> new</c><00:05:47.520><c> information.</c><00:05:48.160><c> For</c><00:05:48.320><c> example,</c><00:05:49.039><c> the</c>

00:05:49.270 --> 00:05:49.280 align:start position:0%
given new information. For example, the
 

00:05:49.280 --> 00:05:53.270 align:start position:0%
given new information. For example, the
word<00:05:49.520><c> view</c><00:05:50.160><c> might</c><00:05:50.479><c> change</c><00:05:50.800><c> to</c><00:05:51.360><c> video</c><00:05:52.160><c> once</c><00:05:52.639><c> the</c>

00:05:53.270 --> 00:05:53.280 align:start position:0%
word view might change to video once the
 

00:05:53.280 --> 00:05:55.510 align:start position:0%
word view might change to video once the
model<00:05:53.680><c> hears</c><00:05:54.000><c> the</c><00:05:54.240><c> rest</c><00:05:54.400><c> of</c><00:05:54.560><c> the</c><00:05:54.800><c> sentence.</c>

00:05:55.510 --> 00:05:55.520 align:start position:0%
model hears the rest of the sentence.
 

00:05:55.520 --> 00:05:57.029 align:start position:0%
model hears the rest of the sentence.
But<00:05:55.680><c> anything</c><00:05:56.000><c> in</c><00:05:56.160><c> the</c><00:05:56.320><c> black</c><00:05:56.560><c> part</c><00:05:56.800><c> is</c>

00:05:57.029 --> 00:05:57.039 align:start position:0%
But anything in the black part is
 

00:05:57.039 --> 00:06:00.150 align:start position:0%
But anything in the black part is
permanent.<00:05:57.600><c> So</c><00:05:58.000><c> even</c><00:05:58.240><c> if</c><00:05:58.400><c> the</c><00:05:58.639><c> model</c><00:05:59.039><c> wants</c><00:05:59.360><c> to</c>

00:06:00.150 --> 00:06:00.160 align:start position:0%
permanent. So even if the model wants to
 

00:06:00.160 --> 00:06:02.310 align:start position:0%
permanent. So even if the model wants to
change<00:06:00.479><c> it</c><00:06:00.639><c> to</c><00:06:00.880><c> something</c><00:06:01.120><c> else</c><00:06:01.600><c> after</c><00:06:02.080><c> more</c>

00:06:02.310 --> 00:06:02.320 align:start position:0%
change it to something else after more
 

00:06:02.320 --> 00:06:05.189 align:start position:0%
change it to something else after more
iterations<00:06:03.039><c> it</c><00:06:03.280><c> cannot</c><00:06:03.600><c> be</c><00:06:03.759><c> changed</c><00:06:04.000><c> anymore.</c>

00:06:05.189 --> 00:06:05.199 align:start position:0%
iterations it cannot be changed anymore.
 

00:06:05.199 --> 00:06:07.110 align:start position:0%
iterations it cannot be changed anymore.
One<00:06:05.520><c> last</c><00:06:05.759><c> thing</c><00:06:05.919><c> that</c><00:06:06.240><c> this</c><00:06:06.479><c> model</c><00:06:06.800><c> does</c><00:06:06.960><c> when</c>

00:06:07.110 --> 00:06:07.120 align:start position:0%
One last thing that this model does when
 

00:06:07.120 --> 00:06:09.430 align:start position:0%
One last thing that this model does when
it<00:06:07.280><c> generates</c><00:06:07.600><c> a</c><00:06:07.759><c> new</c><00:06:07.919><c> sentence</c><00:06:08.479><c> is</c><00:06:08.880><c> it</c><00:06:09.120><c> feeds</c>

00:06:09.430 --> 00:06:09.440 align:start position:0%
it generates a new sentence is it feeds
 

00:06:09.440 --> 00:06:12.390 align:start position:0%
it generates a new sentence is it feeds
the<00:06:09.759><c> previous</c><00:06:10.240><c> sentence</c><00:06:11.039><c> into</c><00:06:11.520><c> the</c><00:06:11.840><c> model</c><00:06:12.080><c> as</c>

00:06:12.390 --> 00:06:12.400 align:start position:0%
the previous sentence into the model as
 

00:06:12.400 --> 00:06:15.029 align:start position:0%
the previous sentence into the model as
prompt<00:06:12.800><c> tokens.</c><00:06:13.600><c> So</c><00:06:13.840><c> this</c><00:06:14.080><c> is</c><00:06:14.240><c> something</c><00:06:14.639><c> that</c>

00:06:15.029 --> 00:06:15.039 align:start position:0%
prompt tokens. So this is something that
 

00:06:15.039 --> 00:06:17.510 align:start position:0%
prompt tokens. So this is something that
you<00:06:15.280><c> can</c><00:06:15.440><c> do</c><00:06:15.600><c> in</c><00:06:15.840><c> the</c><00:06:16.080><c> whisper</c><00:06:16.479><c> format</c><00:06:16.880><c> is</c><00:06:17.280><c> you</c>

00:06:17.510 --> 00:06:17.520 align:start position:0%
you can do in the whisper format is you
 

00:06:17.520 --> 00:06:19.749 align:start position:0%
you can do in the whisper format is you
can<00:06:17.680><c> give</c><00:06:17.840><c> it</c><00:06:18.080><c> a</c><00:06:18.240><c> bunch</c><00:06:18.400><c> of</c><00:06:18.880><c> prompt</c><00:06:19.280><c> tokens</c>

00:06:19.749 --> 00:06:19.759 align:start position:0%
can give it a bunch of prompt tokens
 

00:06:19.759 --> 00:06:21.990 align:start position:0%
can give it a bunch of prompt tokens
before<00:06:20.160><c> you</c><00:06:20.479><c> start</c><00:06:20.800><c> generating</c><00:06:21.600><c> and</c><00:06:21.759><c> this</c>

00:06:21.990 --> 00:06:22.000 align:start position:0%
before you start generating and this
 

00:06:22.000 --> 00:06:23.670 align:start position:0%
before you start generating and this
tends<00:06:22.240><c> to</c><00:06:22.400><c> improve</c><00:06:22.639><c> the</c><00:06:22.880><c> accuracy</c><00:06:23.360><c> a</c><00:06:23.600><c> little</c>

00:06:23.670 --> 00:06:23.680 align:start position:0%
tends to improve the accuracy a little
 

00:06:23.680 --> 00:06:26.309 align:start position:0%
tends to improve the accuracy a little
bit<00:06:24.080><c> because</c><00:06:24.479><c> more</c><00:06:24.800><c> context</c><00:06:25.199><c> is</c><00:06:25.440><c> always</c><00:06:25.680><c> good.</c>

00:06:26.309 --> 00:06:26.319 align:start position:0%
bit because more context is always good.
 

00:06:26.319 --> 00:06:28.710 align:start position:0%
bit because more context is always good.
So<00:06:26.560><c> in</c><00:06:26.800><c> summary,</c><00:06:27.360><c> the</c><00:06:27.759><c> algorithm</c><00:06:28.400><c> can</c><00:06:28.560><c> be</c>

00:06:28.710 --> 00:06:28.720 align:start position:0%
So in summary, the algorithm can be
 

00:06:28.720 --> 00:06:31.510 align:start position:0%
So in summary, the algorithm can be
summarized<00:06:29.280><c> in</c><00:06:29.680><c> three</c><00:06:30.000><c> basic</c><00:06:30.319><c> ideas.</c><00:06:31.280><c> The</c>

00:06:31.510 --> 00:06:31.520 align:start position:0%
summarized in three basic ideas. The
 

00:06:31.520 --> 00:06:34.309 align:start position:0%
summarized in three basic ideas. The
first<00:06:31.680><c> idea</c><00:06:32.080><c> is</c><00:06:32.639><c> we</c><00:06:32.880><c> feed</c><00:06:33.280><c> longer</c><00:06:33.600><c> and</c><00:06:33.840><c> longer</c>

00:06:34.309 --> 00:06:34.319 align:start position:0%
first idea is we feed longer and longer
 

00:06:34.319 --> 00:06:37.350 align:start position:0%
first idea is we feed longer and longer
consecutive<00:06:34.960><c> audio</c><00:06:35.360><c> buffers</c><00:06:36.000><c> into</c><00:06:36.319><c> whisper</c>

00:06:37.350 --> 00:06:37.360 align:start position:0%
consecutive audio buffers into whisper
 

00:06:37.360 --> 00:06:39.990 align:start position:0%
consecutive audio buffers into whisper
and<00:06:37.600><c> then</c><00:06:38.160><c> we</c><00:06:38.400><c> emit</c><00:06:38.800><c> tokens</c><00:06:39.440><c> as</c><00:06:39.680><c> soon</c><00:06:39.840><c> as</c>

00:06:39.990 --> 00:06:40.000 align:start position:0%
and then we emit tokens as soon as
 

00:06:40.000 --> 00:06:43.029 align:start position:0%
and then we emit tokens as soon as
they're<00:06:40.319><c> confirmed</c><00:06:40.880><c> by</c><00:06:41.280><c> two</c><00:06:41.520><c> iterations.</c><00:06:42.800><c> And</c>

00:06:43.029 --> 00:06:43.039 align:start position:0%
they're confirmed by two iterations. And
 

00:06:43.039 --> 00:06:45.270 align:start position:0%
they're confirmed by two iterations. And
finally,<00:06:43.759><c> we</c><00:06:44.080><c> scroll</c><00:06:44.479><c> forward</c><00:06:44.800><c> the</c><00:06:44.960><c> audio</c>

00:06:45.270 --> 00:06:45.280 align:start position:0%
finally, we scroll forward the audio
 

00:06:45.280 --> 00:06:48.150 align:start position:0%
finally, we scroll forward the audio
buffer<00:06:45.759><c> whenever</c><00:06:46.160><c> a</c><00:06:46.400><c> sentence</c><00:06:46.800><c> is</c><00:06:47.039><c> completed.</c>

00:06:48.150 --> 00:06:48.160 align:start position:0%
buffer whenever a sentence is completed.
 

00:06:48.160 --> 00:06:50.230 align:start position:0%
buffer whenever a sentence is completed.
So,<00:06:48.400><c> it's</c><00:06:48.639><c> a</c><00:06:48.880><c> pretty</c><00:06:49.120><c> simple</c><00:06:49.440><c> algorithm</c><00:06:50.000><c> that</c>

00:06:50.230 --> 00:06:50.240 align:start position:0%
So, it's a pretty simple algorithm that
 

00:06:50.240 --> 00:06:52.390 align:start position:0%
So, it's a pretty simple algorithm that
you<00:06:50.479><c> can</c><00:06:50.720><c> apply</c><00:06:51.039><c> to</c><00:06:51.360><c> any</c><00:06:51.680><c> speech</c><00:06:52.000><c> to</c><00:06:52.160><c> text</c>

00:06:52.390 --> 00:06:52.400 align:start position:0%
you can apply to any speech to text
 

00:06:52.400 --> 00:06:54.230 align:start position:0%
you can apply to any speech to text
model<00:06:52.880><c> that</c><00:06:53.199><c> does</c><00:06:53.360><c> not</c><00:06:53.520><c> support</c><00:06:53.840><c> streaming</c>

00:06:54.230 --> 00:06:54.240 align:start position:0%
model that does not support streaming
 

00:06:54.240 --> 00:06:56.469 align:start position:0%
model that does not support streaming
and<00:06:54.720><c> basically</c><00:06:55.199><c> turn</c><00:06:55.360><c> it</c><00:06:55.600><c> into</c><00:06:55.840><c> a</c><00:06:56.000><c> streaming</c>

00:06:56.469 --> 00:06:56.479 align:start position:0%
and basically turn it into a streaming
 

00:06:56.479 --> 00:06:58.710 align:start position:0%
and basically turn it into a streaming
model.<00:06:57.280><c> If</c><00:06:57.440><c> you</c><00:06:57.600><c> like,</c><00:06:58.080><c> you</c><00:06:58.240><c> can</c><00:06:58.400><c> check</c><00:06:58.560><c> out</c>

00:06:58.710 --> 00:06:58.720 align:start position:0%
model. If you like, you can check out
 

00:06:58.720 --> 00:07:00.710 align:start position:0%
model. If you like, you can check out
all<00:06:58.960><c> the</c><00:06:59.199><c> details</c><00:06:59.520><c> in</c><00:06:59.840><c> this</c><00:07:00.000><c> paper,</c><00:07:00.479><c> which</c>

00:07:00.710 --> 00:07:00.720 align:start position:0%
all the details in this paper, which
 

00:07:00.720 --> 00:07:02.870 align:start position:0%
all the details in this paper, which
I'll<00:07:00.960><c> link</c><00:07:01.120><c> in</c><00:07:01.280><c> the</c><00:07:01.440><c> description.</c><00:07:02.319><c> One</c><00:07:02.560><c> of</c><00:07:02.720><c> the</c>

00:07:02.870 --> 00:07:02.880 align:start position:0%
I'll link in the description. One of the
 

00:07:02.880 --> 00:07:05.029 align:start position:0%
I'll link in the description. One of the
limitations<00:07:03.680><c> comes</c><00:07:04.000><c> from</c><00:07:04.240><c> the</c><00:07:04.479><c> fact</c><00:07:04.720><c> that</c>

00:07:05.029 --> 00:07:05.039 align:start position:0%
limitations comes from the fact that
 

00:07:05.039 --> 00:07:07.510 align:start position:0%
limitations comes from the fact that
whisper<00:07:05.840><c> was</c><00:07:06.080><c> not</c><00:07:06.319><c> really</c><00:07:06.639><c> designed</c><00:07:07.039><c> to</c><00:07:07.199><c> be</c><00:07:07.360><c> a</c>

00:07:07.510 --> 00:07:07.520 align:start position:0%
whisper was not really designed to be a
 

00:07:07.520 --> 00:07:09.749 align:start position:0%
whisper was not really designed to be a
streaming<00:07:07.919><c> model.</c><00:07:08.479><c> And</c><00:07:08.720><c> because</c><00:07:08.960><c> of</c><00:07:09.120><c> this,</c><00:07:09.520><c> it</c>

00:07:09.749 --> 00:07:09.759 align:start position:0%
streaming model. And because of this, it
 

00:07:09.759 --> 00:07:12.469 align:start position:0%
streaming model. And because of this, it
assumes<00:07:10.240><c> that</c><00:07:10.639><c> each</c><00:07:11.120><c> audio</c><00:07:11.599><c> buffer</c><00:07:12.080><c> has</c><00:07:12.319><c> to</c>

00:07:12.469 --> 00:07:12.479 align:start position:0%
assumes that each audio buffer has to
 

00:07:12.479 --> 00:07:14.469 align:start position:0%
assumes that each audio buffer has to
start<00:07:12.720><c> at</c><00:07:12.880><c> the</c><00:07:13.039><c> beginning</c><00:07:13.280><c> of</c><00:07:13.440><c> a</c><00:07:13.599><c> sentence.</c>

00:07:14.469 --> 00:07:14.479 align:start position:0%
start at the beginning of a sentence.
 

00:07:14.479 --> 00:07:16.469 align:start position:0%
start at the beginning of a sentence.
Therefore,<00:07:15.039><c> if</c><00:07:15.280><c> you</c><00:07:15.520><c> have</c><00:07:15.680><c> a</c><00:07:15.919><c> sentence</c><00:07:16.240><c> that</c>

00:07:16.469 --> 00:07:16.479 align:start position:0%
Therefore, if you have a sentence that
 

00:07:16.479 --> 00:07:18.390 align:start position:0%
Therefore, if you have a sentence that
is<00:07:16.639><c> quite</c><00:07:16.880><c> long,</c><00:07:17.280><c> then</c><00:07:17.520><c> the</c><00:07:17.759><c> beginning</c><00:07:18.000><c> of</c><00:07:18.160><c> the</c>

00:07:18.390 --> 00:07:18.400 align:start position:0%
is quite long, then the beginning of the
 

00:07:18.400 --> 00:07:20.469 align:start position:0%
is quite long, then the beginning of the
sentence<00:07:18.960><c> has</c><00:07:19.199><c> to</c><00:07:19.360><c> be</c><00:07:19.440><c> fed</c><00:07:19.759><c> into</c><00:07:20.000><c> the</c><00:07:20.240><c> model</c>

00:07:20.469 --> 00:07:20.479 align:start position:0%
sentence has to be fed into the model
 

00:07:20.479 --> 00:07:22.629 align:start position:0%
sentence has to be fed into the model
and<00:07:20.720><c> processed</c><00:07:21.199><c> many</c><00:07:21.520><c> times.</c><00:07:22.319><c> This</c><00:07:22.479><c> is</c>

00:07:22.629 --> 00:07:22.639 align:start position:0%
and processed many times. This is
 

00:07:22.639 --> 00:07:25.430 align:start position:0%
and processed many times. This is
inefficient<00:07:23.440><c> and</c><00:07:23.840><c> not</c><00:07:24.160><c> necessary</c><00:07:24.880><c> if</c><00:07:25.199><c> the</c>

00:07:25.430 --> 00:07:25.440 align:start position:0%
inefficient and not necessary if the
 

00:07:25.440 --> 00:07:27.270 align:start position:0%
inefficient and not necessary if the
model<00:07:25.759><c> was</c><00:07:26.000><c> trained</c><00:07:26.319><c> from</c><00:07:26.560><c> the</c><00:07:26.800><c> beginning</c><00:07:27.039><c> to</c>

00:07:27.270 --> 00:07:27.280 align:start position:0%
model was trained from the beginning to
 

00:07:27.280 --> 00:07:30.309 align:start position:0%
model was trained from the beginning to
do<00:07:27.440><c> streaming</c><00:07:27.919><c> ASR.</c><00:07:29.039><c> Now,</c><00:07:29.360><c> if</c><00:07:29.599><c> we</c><00:07:29.840><c> look</c><00:07:29.919><c> at</c><00:07:30.160><c> an</c>

00:07:30.309 --> 00:07:30.319 align:start position:0%
do streaming ASR. Now, if we look at an
 

00:07:30.319 --> 00:07:32.230 align:start position:0%
do streaming ASR. Now, if we look at an
architecture<00:07:30.960><c> that</c><00:07:31.280><c> was</c><00:07:31.680><c> designed</c>

00:07:32.230 --> 00:07:32.240 align:start position:0%
architecture that was designed
 

00:07:32.240 --> 00:07:34.070 align:start position:0%
architecture that was designed
specifically<00:07:32.800><c> to</c><00:07:33.039><c> do</c><00:07:33.280><c> streaming</c><00:07:33.759><c> speech</c>

00:07:34.070 --> 00:07:34.080 align:start position:0%
specifically to do streaming speech
 

00:07:34.080 --> 00:07:36.070 align:start position:0%
specifically to do streaming speech
recognition,<00:07:35.039><c> it</c><00:07:35.280><c> looks</c><00:07:35.520><c> a</c><00:07:35.759><c> little</c><00:07:35.840><c> bit</c>

00:07:36.070 --> 00:07:36.080 align:start position:0%
recognition, it looks a little bit
 

00:07:36.080 --> 00:07:38.070 align:start position:0%
recognition, it looks a little bit
different.<00:07:36.880><c> Here</c><00:07:37.120><c> is</c><00:07:37.280><c> a</c><00:07:37.440><c> model</c><00:07:37.680><c> that</c><00:07:37.919><c> was</c>

00:07:38.070 --> 00:07:38.080 align:start position:0%
different. Here is a model that was
 

00:07:38.080 --> 00:07:40.870 align:start position:0%
different. Here is a model that was
proposed<00:07:38.479><c> in</c><00:07:38.720><c> 2021.</c><00:07:39.919><c> At</c><00:07:40.160><c> each</c><00:07:40.319><c> step,</c><00:07:40.639><c> it</c>

00:07:40.870 --> 00:07:40.880 align:start position:0%
proposed in 2021. At each step, it
 

00:07:40.880 --> 00:07:43.749 align:start position:0%
proposed in 2021. At each step, it
predicts<00:07:41.199><c> a</c><00:07:41.360><c> token</c><00:07:42.000><c> and</c><00:07:42.240><c> it</c><00:07:42.479><c> has</c><00:07:42.880><c> access</c><00:07:43.280><c> to</c><00:07:43.520><c> a</c>

00:07:43.749 --> 00:07:43.759 align:start position:0%
predicts a token and it has access to a
 

00:07:43.759 --> 00:07:46.309 align:start position:0%
predicts a token and it has access to a
fixed<00:07:44.080><c> amount</c><00:07:44.400><c> of</c><00:07:44.720><c> past</c><00:07:45.039><c> context</c><00:07:45.599><c> and</c><00:07:46.000><c> future</c>

00:07:46.309 --> 00:07:46.319 align:start position:0%
fixed amount of past context and future
 

00:07:46.319 --> 00:07:49.110 align:start position:0%
fixed amount of past context and future
context.<00:07:47.360><c> During</c><00:07:47.759><c> training,</c><00:07:48.319><c> this</c><00:07:48.560><c> rule</c><00:07:48.880><c> is</c>

00:07:49.110 --> 00:07:49.120 align:start position:0%
context. During training, this rule is
 

00:07:49.120 --> 00:07:51.110 align:start position:0%
context. During training, this rule is
enforced<00:07:49.599><c> by</c><00:07:49.840><c> having</c><00:07:50.080><c> an</c><00:07:50.319><c> intention</c><00:07:50.720><c> mask</c>

00:07:51.110 --> 00:07:51.120 align:start position:0%
enforced by having an intention mask
 

00:07:51.120 --> 00:07:53.189 align:start position:0%
enforced by having an intention mask
that<00:07:51.360><c> is</c><00:07:51.520><c> mostly</c><00:07:51.919><c> zeros,</c><00:07:52.560><c> meaning</c><00:07:52.800><c> that</c><00:07:52.960><c> the</c>

00:07:53.189 --> 00:07:53.199 align:start position:0%
that is mostly zeros, meaning that the
 

00:07:53.199 --> 00:07:55.189 align:start position:0%
that is mostly zeros, meaning that the
model<00:07:53.520><c> cannot</c><00:07:53.919><c> use</c><00:07:54.160><c> information</c><00:07:54.639><c> that</c><00:07:54.960><c> is</c>

00:07:55.189 --> 00:07:55.199 align:start position:0%
model cannot use information that is
 

00:07:55.199 --> 00:07:57.990 align:start position:0%
model cannot use information that is
outside<00:07:55.759><c> of</c><00:07:56.000><c> this</c><00:07:56.240><c> fixed</c><00:07:56.639><c> window.</c><00:07:57.520><c> And</c><00:07:57.680><c> during</c>

00:07:57.990 --> 00:07:58.000 align:start position:0%
outside of this fixed window. And during
 

00:07:58.000 --> 00:08:00.070 align:start position:0%
outside of this fixed window. And during
prediction,<00:07:58.639><c> the</c><00:07:58.879><c> model</c><00:07:59.120><c> predicts</c><00:07:59.440><c> a</c><00:07:59.599><c> token</c>

00:08:00.070 --> 00:08:00.080 align:start position:0%
prediction, the model predicts a token
 

00:08:00.080 --> 00:08:03.189 align:start position:0%
prediction, the model predicts a token
given<00:08:00.400><c> a</c><00:08:00.639><c> limited</c><00:08:01.039><c> amount</c><00:08:01.360><c> of</c><00:08:01.680><c> fixed</c><00:08:02.080><c> context.</c>

00:08:03.189 --> 00:08:03.199 align:start position:0%
given a limited amount of fixed context.
 

00:08:03.199 --> 00:08:05.270 align:start position:0%
given a limited amount of fixed context.
To<00:08:03.440><c> make</c><00:08:03.520><c> the</c><00:08:03.759><c> next</c><00:08:03.919><c> prediction,</c><00:08:04.639><c> the</c><00:08:04.879><c> entire</c>

00:08:05.270 --> 00:08:05.280 align:start position:0%
To make the next prediction, the entire
 

00:08:05.280 --> 00:08:07.350 align:start position:0%
To make the next prediction, the entire
receptive<00:08:05.840><c> field</c><00:08:06.319><c> moves</c><00:08:06.560><c> forward</c><00:08:06.879><c> by</c><00:08:07.199><c> one</c>

00:08:07.350 --> 00:08:07.360 align:start position:0%
receptive field moves forward by one
 

00:08:07.360 --> 00:08:09.589 align:start position:0%
receptive field moves forward by one
chunk,<00:08:07.919><c> but</c><00:08:08.160><c> the</c><00:08:08.400><c> size</c><00:08:08.720><c> of</c><00:08:08.879><c> the</c><00:08:09.120><c> receptive</c>

00:08:09.589 --> 00:08:09.599 align:start position:0%
chunk, but the size of the receptive
 

00:08:09.599 --> 00:08:11.830 align:start position:0%
chunk, but the size of the receptive
field<00:08:09.919><c> is</c><00:08:10.160><c> fixed.</c><00:08:10.879><c> This</c><00:08:11.120><c> way,</c><00:08:11.280><c> the</c><00:08:11.599><c> beginning</c>

00:08:11.830 --> 00:08:11.840 align:start position:0%
field is fixed. This way, the beginning
 

00:08:11.840 --> 00:08:13.749 align:start position:0%
field is fixed. This way, the beginning
of<00:08:12.000><c> the</c><00:08:12.160><c> sentence</c><00:08:12.560><c> will</c><00:08:12.800><c> not</c><00:08:12.960><c> be</c><00:08:13.120><c> processed</c>

00:08:13.749 --> 00:08:13.759 align:start position:0%
of the sentence will not be processed
 

00:08:13.759 --> 00:08:16.150 align:start position:0%
of the sentence will not be processed
multiple<00:08:14.240><c> times.</c><00:08:15.120><c> However,</c><00:08:15.599><c> this</c><00:08:15.840><c> is</c><00:08:16.000><c> not</c>

00:08:16.150 --> 00:08:16.160 align:start position:0%
multiple times. However, this is not
 

00:08:16.160 --> 00:08:17.990 align:start position:0%
multiple times. However, this is not
possible<00:08:16.479><c> with</c><00:08:16.720><c> the</c><00:08:16.879><c> whisper</c><00:08:17.280><c> model</c><00:08:17.680><c> because</c>

00:08:17.990 --> 00:08:18.000 align:start position:0%
possible with the whisper model because
 

00:08:18.000 --> 00:08:20.230 align:start position:0%
possible with the whisper model because
it<00:08:18.319><c> requires</c><00:08:18.960><c> modifying</c><00:08:19.520><c> the</c><00:08:19.680><c> architecture</c>

00:08:20.230 --> 00:08:20.240 align:start position:0%
it requires modifying the architecture
 

00:08:20.240 --> 00:08:22.230 align:start position:0%
it requires modifying the architecture
and<00:08:20.479><c> retraining</c><00:08:20.960><c> the</c><00:08:21.199><c> model</c><00:08:21.440><c> and</c><00:08:21.680><c> we</c><00:08:21.840><c> do</c><00:08:22.000><c> not</c>

00:08:22.230 --> 00:08:22.240 align:start position:0%
and retraining the model and we do not
 

00:08:22.240 --> 00:08:24.710 align:start position:0%
and retraining the model and we do not
have<00:08:22.400><c> access</c><00:08:22.720><c> to</c><00:08:23.039><c> whisper's</c><00:08:23.599><c> training</c><00:08:23.840><c> data.</c>

00:08:24.710 --> 00:08:24.720 align:start position:0%
have access to whisper's training data.
 

00:08:24.720 --> 00:08:26.550 align:start position:0%
have access to whisper's training data.
That's<00:08:24.960><c> it</c><00:08:25.120><c> for</c><00:08:25.360><c> this</c><00:08:25.520><c> video</c><00:08:25.759><c> and</c><00:08:26.000><c> I</c><00:08:26.240><c> hope</c><00:08:26.400><c> you</c>

00:08:26.550 --> 00:08:26.560 align:start position:0%
That's it for this video and I hope you
 

00:08:26.560 --> 00:08:28.869 align:start position:0%
That's it for this video and I hope you
enjoyed<00:08:26.960><c> my</c><00:08:27.199><c> explanation</c><00:08:27.680><c> of</c><00:08:28.080><c> how</c><00:08:28.400><c> Whisper</c>

00:08:28.869 --> 00:08:28.879 align:start position:0%
enjoyed my explanation of how Whisper
 

00:08:28.879 --> 00:08:31.430 align:start position:0%
enjoyed my explanation of how Whisper
can<00:08:29.120><c> be</c><00:08:29.360><c> turned</c><00:08:29.680><c> into</c><00:08:29.840><c> a</c><00:08:30.080><c> streaming</c><00:08:30.479><c> model.</c><00:08:31.199><c> If</c>

00:08:31.430 --> 00:08:31.440 align:start position:0%
can be turned into a streaming model. If
 

00:08:31.440 --> 00:08:33.269 align:start position:0%
can be turned into a streaming model. If
you<00:08:31.599><c> enjoyed</c><00:08:31.919><c> this</c><00:08:32.080><c> video,</c><00:08:32.560><c> please</c><00:08:32.800><c> leave</c><00:08:33.039><c> a</c>

00:08:33.269 --> 00:08:33.279 align:start position:0%
you enjoyed this video, please leave a
 

00:08:33.279 --> 00:08:35.269 align:start position:0%
you enjoyed this video, please leave a
comment,<00:08:33.839><c> hit</c><00:08:34.080><c> the</c><00:08:34.240><c> subscribe</c><00:08:34.640><c> button,</c><00:08:35.039><c> and</c>

00:08:35.269 --> 00:08:35.279 align:start position:0%
comment, hit the subscribe button, and
 

00:08:35.279 --> 00:08:37.350 align:start position:0%
comment, hit the subscribe button, and
ring<00:08:35.519><c> the</c><00:08:35.680><c> bell</c><00:08:36.000><c> icon</c><00:08:36.240><c> to</c><00:08:36.560><c> stay</c><00:08:36.800><c> notified</c><00:08:37.200><c> when</c>

00:08:37.350 --> 00:08:37.360 align:start position:0%
ring the bell icon to stay notified when
 

00:08:37.360 --> 00:08:41.919 align:start position:0%
ring the bell icon to stay notified when
I<00:08:37.599><c> release</c><00:08:37.839><c> future</c><00:08:38.159><c> videos.</c><00:08:39.120><c> Goodbye.</c>