Transcript of 20230302 170602.m4a (at 2023-03-02 1711.02).txt

Hey friend, your use case sounds really interesting.
Actually that's why I created this app initially.
I really love riding around on my bike in the city
and doing voice memo debriefs about whatever.
I also like to do it walking around outside.
And as you say, the trouble with that is wind distortion.
Full stop.
On a day where it's not too windy, it's not too bad.
These models can totally pull the text from it.
But the more distortion you have,
the more of a disaster it is.
And I don't know anything about the multilingual case,
but for English, I definitely find that
small is more than enough if you have good quality audio.
Medium, you might wanna use it
if there's some kind of distortion
that's causing errors in the small.
But if you have really good quality audio,
even tiny is enough.
I mean, it won't get some sort of rare words.
So small is basically good enough for English anyway.
Aligning with what you said,
I remember seeing in the whisper paper
that the performance actually decreases,
the word error rate increases from the medium
to the large model in the multilingual case,
which is kind of interesting.
So basically medium, I think, is all you really need.
I think doing large, running large locally
is probably a waste of time.
But this doesn't apply to the OpenAI API case
because they're running their own sort of special model.
It's very fast.
Plus they're kind of going to be retraining it
so continuing to improve it over time.
So obviously there's that, which is cool.
I think basically I did extensive research
and experiment with this,
with trying to clean up audio for the transcription.
And there's basically no way to do it.
Like if you have a medium to bad level of noise
that the transcription models can still work with,
you're fine.
Just go with that.
But in that case,
there's no point actually trying to denoise the sound first.
That just seems to reduce the signal
and it actually increases the word error rate.
So just give them the raw distorted, windy audio
and the models will do the best they can.
You can't actually improve it, I found.
I tried all kinds of different ways to process it
and none of it actually improved it,
including like the best possible denoiser I could find,
which is the Facebook research denoiser.
So my conclusion was that, okay,
I found a sort of a fundamental physical limit
and I think using denoising is really only good for humans.
Like if you want to listen to the audio again,
you don't want to hear all that wind probably.
And for medium to bad,
but not extreme levels of wind distortion
or other kinds of noise distortion,
you can use a denoiser like the Facebook research one
and that will totally or nearly totally
kind of reduce all that noise.
But I basically decided that the only way
to kind of get better quality audio
or to get better quality transcripts,
if you're doing it outside on a windy day,
is not to go with software enhancement
because it doesn't do anything, it doesn't achieve anything.
I tried everything possible
and nothing produced results in the extreme distortion case.
So what I decided is that's basically a limit,
physical limit and so the best way to do it I think
is to change your microphone setup,
have some sort of baffle around it,
maybe one of those little microphones
that can clip close to your throat or,
I don't know, I'm no expert,
but I think basically you can find a way
to get better quality, less distorted audio
outside by changing the microphone setup,
adding one of those wind baffles or something.
And I think that's basically all you can do essentially.
And then once you have reduced
a lot of that noise distortion,
even if the resulting audio is still distorted,
if it's not too bad, the models can be okay with it.

End of Transcript (created with WisprNote)