Spaces:

sanchit-gandhi

/

whisper-jax

like 1.91k

Running

I guess it doesn't download and convert YouTube video instead of pulling down the audio from YouTube by YouTube API or ydl tool, and then convert to the 16K audio using ffmpeg, this would be much much faster.

Yeah but ffmpeg takes 30mins to convert 1hr audio from URL to 16K in my M1 or ditialocean cpu. I haven’t tried youtube-dl but I guess it will either return audio URL or would download it which would again take time.

It is a little bit strange that ffmpeg takes that long time to convert. I tested a 75 mins long mp3 audio in my M1 and it only costs 7 seconds to convert the mp3 to the wav:

/opt/homebrew/bin/ffmpeg -i /path-to-audio/foo.mp3 -ar 16000 -ac 1 /path-to-audio/foo-converted.wav

---------my ffmpeg info----------------------------------
/opt/homebrew/bin/ffmpeg ─╯
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

That’s weird, here is audio, can you try this: https://rr2---sn-4g5edn6r.googlevideo.com/videoplayback?expire=1684935984&ei=0MBtZOuGBM2n1gK31KqgDg&ip=23.88.39.196&id=o-AP2x11_F2XblyaZfCSBgqWQ4LjkV3DbTuHtKHWwCKCgc&itag=599&source=youtube&requiressl=yes&mh=Lp&mm=31%2C26&mn=sn-4g5edn6r%2Csn-f5f7lnl6&ms=au%2Conr&mv=u&mvi=2&pl=27&spc=qEK7BweDWxPwlAcY3mal7H5IMssIcKE&vprv=1&svpuc=1&mime=audio%2Fmp4&gir=yes&clen=14255290&dur=3704.557&lmt=1684667444904932&mt=1684913713&fvip=3&keepalive=yes&fexp=24007246%2C51000023&c=ANDROID&txp=5432434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRAIgVdkvU6baVgar8WZn2MXcyqL1GwBXXRdLnKHz2YywmDECIA7pTVkK3dYDnAtYFm80IYACSnXqfU1OtNHH1CXYWAq7&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl&lsig=AG3C_xAwRAIgN8yl7v3f0pt6rLQ4DlNDtwItjX6jPyK3F-hjEPLFr98CIBbYR5au9Qz-Zgb03MSCoxz8-xvui0SV7PmrXEROVrqm

From the Github source in line 179;

ydl_opts = {"outtmpl": filename, "format": "worstvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best"}

This format string is like a set of instructions for downloading the YouTube video. It tells the code to prioritize the worst video quality that is available in MP4 format (worstvideo[ext=mp4]). At the same time, it wants the best audio quality in M4A format (bestaudio[ext=m4a]). By specifying this format, the code can quickly download the video with a focus on obtaining good audio quality while not spending extra time and bandwidth on getting the highest video quality.

Hope that helps!

That does help, Thanks! When I posted this, I didn’t know yt-dlp and later learned about it. But now knowing it has worstvideo and bestaudio methods, it made sense.

Thanks for your help here @epireve !