wasertech commited on
Commit
6efde26
1 Parent(s): 0a1ebba

update custom message

Browse files
Files changed (1) hide show
  1. constants.py +4 -4
constants.py CHANGED
@@ -97,20 +97,20 @@ are ranked based on their average WER scores, from lowest to highest.
97
  For more details on the individual datasets and how models are evaluated to give the ESB score, refer to the [ESB paper](https://arxiv.org/abs/2210.13352).
98
  """
99
 
100
- CUSTOM_MESSAGE = """## Legend:
101
  This space is a fork of the original [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard). It aims to demonstrate how the CommonVoice Test Set provides a relatively accurate approximation of the average WER/CER (Word Error Rate/Character Error Rate) at a significantly lower computational cost.
102
 
103
  #### Why is this useful?
104
  This opens way the to achieve standardized test set for most languages, enabling us to programmatically select a reasonably effective model for any language supported by CommonVoice.
105
 
 
 
106
  Columns `Model`, `RTF`, and `Average WER` were sourced from [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) using the version from September 7, 2023.
107
 
108
- Models are sorted by consistancy in theire results across testsets. (by increasing order of absolute delta between average WER and CommonVoice WER)
109
 
110
  ### Results
111
  The CommonVoice Test provides a Word Error Rate (WER) within a 20-point margin of the average WER.
112
 
113
  While not perfect, this indicates that CommonVoice can be a useful tool for quickly identifying a suitable ASR model for a wide range of languages in a programmatic manner. However, it's important to note that it is not sufficient as the sole criterion for choosing the most appropriate architecture. Further considerations may be needed depending on the specific requirements of your ASR application.
114
-
115
- For more context, [here](https://gist.github.com/wasertech/400ca3dd61f2d6f7f4f5495afbb32ef3) is the output of my ASR server when running without any specified model to load for various languages. It tries to score the most suitable model for any given language. Since metrics are mostly self-reported, sometimes in different format, it consistently picks an unadequate model.
116
  """
 
97
  For more details on the individual datasets and how models are evaluated to give the ESB score, refer to the [ESB paper](https://arxiv.org/abs/2210.13352).
98
  """
99
 
100
+ CUSTOM_MESSAGE = """## Using CommonVoice to approximate average WER for open domain transcriptions
101
  This space is a fork of the original [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard). It aims to demonstrate how the CommonVoice Test Set provides a relatively accurate approximation of the average WER/CER (Word Error Rate/Character Error Rate) at a significantly lower computational cost.
102
 
103
  #### Why is this useful?
104
  This opens way the to achieve standardized test set for most languages, enabling us to programmatically select a reasonably effective model for any language supported by CommonVoice.
105
 
106
+ For more context, [here](https://gist.github.com/wasertech/400ca3dd61f2d6f7f4f5495afbb32ef3) is the output of my ASR server when running without any specified model to load for various languages. It tries to score the most suitable model for any given language. Since metrics are mostly self-reported, sometimes in different format, it consistently picks an unadequate model.
107
+
108
  Columns `Model`, `RTF`, and `Average WER` were sourced from [hf-audio/open_asr_leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) using the version from September 7, 2023.
109
 
110
+ Models are sorted by consistancy in their results across testsets. (by increasing order of absolute delta between average WER and CommonVoice WER)
111
 
112
  ### Results
113
  The CommonVoice Test provides a Word Error Rate (WER) within a 20-point margin of the average WER.
114
 
115
  While not perfect, this indicates that CommonVoice can be a useful tool for quickly identifying a suitable ASR model for a wide range of languages in a programmatic manner. However, it's important to note that it is not sufficient as the sole criterion for choosing the most appropriate architecture. Further considerations may be needed depending on the specific requirements of your ASR application.
 
 
116
  """