Package: rtasr
Rate That ASR (RTASR)🏆 Run benchmarks against the most common ASR tools on the market.Results[!IMPORTANT]Deepgram benchmark results have been updated with the latest Nova 2 model.WER & WRRwer= Word Error Rate,mer= Match Error Rate,wil= Word Information Lost,wrr= Word Recognition RateDataset:FleursDERder= Diarization Error Rate,miss= missed detection,confusion= incorrect detection,fa= false alarmDataset:VoxConverse[!NOTE]Click on the images to get a bigger display.Dataset:AMI Corpus[!NOTE]Click on the images to get a bigger display.InstallationLast stable versionpipinstallrtasrFrom sourcegitclonehttps://github.com/Wordcab/rtasrcdrtasr

pipinstall.CommandsThe CLI is available through thertasrcommand.rtasr--helpList datasets, metrics and providers# List everythingrtasrlist# List only datasetsrtasrlist-tdatasets# List only metricsrtasrlist-tmetrics# List only providersrtasrlist-tprovidersDatasets downloadAvailable datasets are:ami:AMI Corpusvoxconverse:VoxConversertasrdownload-d<dataset>ASR TranscriptionProvidersImplemented ASR providers are:assemblyai:AssemblyAIaws:AWS Transcribeazure:Azure Speechdeepgram:Deepgramgoogle:Google Cloud Speech-to-Textrevai:RevAIspeechmatics:Speechmaticswordcab:WordcabRun transcriptionRun ASR transcription on a given dataset with a given provider.rtasrtranscription-d<dataset>-p<provider>Multiple providersYou can specify as many providers as you want:rtasrtranscription-d<dataset>-p<provider1><provider2><provider3>...Choose dataset splitYou can specify the dataset split to use:rtasrtranscription-d<dataset>-p<provider>-s<split>If not specified, all the available splits will be used.CachingBy default, the transcription results are cached in the~/.cache/rtasr/transcriptiondirectory for each provider.If you don't want to use the cache, use the--no-cacheflag.rtasrtranscription-d<dataset>-p<provider>--no-cacheNote: the cache is used to avoid running the same file twice. By removing the cache, you will run the transcription on the whole dataset again. We aren't responsible for any extra costs.Debug modeUse the--debugflag to run only one file by split for each provider.rtasrtranscription-d<dataset>-p<provider>--debugEvaluationTheevaluationcommand allows you to run an evaluation on the transcription results.If you don't specify the split, the evaluation will be run on the whole dataset.Run DER evaluationrtasrevaluation-mder-d<dataset>-s<split>Run WER evaluationrtasrevaluation-mwer-d<dataset>-s<split>Plot resultsTo get the plots of the evaluation results, use theplotcommand.If you don't specify the split, the plots will be generated for all the available splits.Plot DER resultsrtasrplot-mder-d<dataset>-s<split>Plot WER resultsrtasrplot-mwer-d<dataset>-s<split>Dataset lengthTo get the total length of a dataset, use theaudio-lengthcommand.
This command allow you to get the number of minutes of audio for each split of a dataset.If you don't specify the split, the total length of the dataset will be returned
for all the available splits.rtasraudio-length-d<dataset>-s<split>ContributingBe sure to havehatchinstalled.QualityRun quality checks:hatch run quality:checkRun quality formatting:hatch run quality:formatTestingRun tests:hatch run tests:run