Update model card

README.md

@@ -13,41 +13,139 @@ model-index:
   results: []
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+# wav2vec2-common_voice_13_0-eo-3, an Esperanto speech recognizer
 
-# wav2vec2-common_voice_13_0-eo-3
-
-This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the MOZILLA-FOUNDATION/COMMON_VOICE_13_0 - EO dataset.
+This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the [mozilla-foundation/common_voice_13_0](https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0) Esperanto dataset.
 It achieves the following results on the evaluation set:
+
 - Loss: 0.2191
 - Cer: 0.0208
 - Wer: 0.0687
 
 ## Model description
 
-More information needed
+See [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53).
 
 ## Intended uses & limitations
 
-More information needed
+Speech recognition for Esperanto. The base model was pretrained and finetuned on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16KHz.
 
 ## Training and evaluation data
 
-More information needed
+The training split was set to `train[:15000]` while the eval split was set to `validation[:1500]`.
 
 ## Training procedure
 
+I used [`run_speech_recognition_ctc.py`](https://github.com/huggingface/transformers/tree/main/examples/pytorch/speech-recognition) with the following `train.json` file passed to it:
+
+```json
+{
+  "dataset_name": "mozilla-foundation/common_voice_13_0",
+  "model_name_or_path": "facebook/wav2vec2-large-xlsr-53",
+  "dataset_config_name": "eo",
+  "output_dir": "./wav2vec2-common_voice_13_0-eo-3",
+  "train_split_name": "train[:15000]",
+  "eval_split_name": "validation[:1500]",
+  "eval_metrics": ["cer", "wer"],
+  "overwrite_output_dir": true,
+  "preprocessing_num_workers": 8,
+  "num_train_epochs": 100,
+  "per_device_train_batch_size": 8,
+  "gradient_accumulation_steps": 4,
+  "gradient_checkpointing": true,
+  "learning_rate": 3e-5,
+  "warmup_steps": 500,
+  "evaluation_strategy": "steps",
+  "text_column_name": "sentence",
+  "length_column_name": "input_length",
+  "save_steps": 1000,
+  "eval_steps": 1000,
+  "layerdrop": 0.1,
+  "save_total_limit": 3,
+  "freeze_feature_encoder": true,
+  "chars_to_ignore": "-!\"'(),.:;=?_`¨«¸»ʼ‑–—‘’“”„…‹›♫？",
+  "chars_to_substitute": {
+    "przy": "pŝe",
+    "byn": "bin",
+    "cx": "ĉ",
+    "sx": "ŝ",
+    "fi": "fi",
+    "fl": "fl",
+    "ǔ": "ŭ",
+    "ñ": "nj",
+    "á": "a",
+    "é": "e",
+    "ü": "ŭ",
+    "y": "j",
+    "qu": "ku"
+  },
+  "fp16": true,
+  "group_by_length": true,
+  "push_to_hub": true,
+  "do_train": true,
+  "do_eval": true
+}
+```
+
+I went through the dataset to find non-speech characters, and these were placed in `chars_to_ignore`. In addition, there were character sequences that could be transcribed to Esperanto phonemes, and these were placed as a dictionary in `chars_to_substitute`. This required adding such an argument to the program:
+
+```py
+def dict_field(default=None, metadata=None):
+    return field(default_factory=lambda: default, metadata=metadata)
+
+@dataclass
+class DataTrainingArguments:
+  ...
+    chars_to_substitute: Optional[Dict[str, str]] = dict_field(
+        default=None,
+        metadata={"help": "A dict of characters to replace."},
+    )
+
+```
+
+Then I copied `remove_special_characters` to do the actual substitution:
+
+```py
+    def remove_special_characters(batch):
+        text = batch[text_column_name]
+        if chars_to_ignore_regex is not None:
+            text = re.sub(chars_to_ignore_regex, "", batch[text_column_name])
+        batch["target_text"] = text.lower() + " "
+        return batch
+
+    def substitute_characters(batch):
+        text: str = batch["target_text"]
+        if data_args.chars_to_substitute is not None:
+            for k, v in data_args.chars_to_substitute.items():
+                text.replace(k, v)
+        batch["target_text"] = text.lower()
+        return batch
+
+    with training_args.main_process_first(desc="dataset map special characters removal"):
+        raw_datasets = raw_datasets.map(
+            remove_special_characters,
+            remove_columns=[text_column_name],
+            desc="remove special characters from datasets",
+        )
+
+    with training_args.main_process_first(desc="dataset map special characters substitute"):
+        raw_datasets = raw_datasets.map(
+            substitute_characters,
+            desc="substitute special characters in datasets",
+        )
+```
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-06
+- learning_rate: 3e-05
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - gradient_accumulation_steps: 4
 - total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- layerdrop: 0.1
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 500
 - num_epochs: 100
@@ -96,13 +194,13 @@ The following hyperparameters were used during training:
 | 0.0341        | 78.93 | 37000 | 0.0208 | 0.2170          | 0.0688 |
 | 0.032         | 81.07 | 38000 | 0.0209 | 0.2157          | 0.0686 |
 | 0.0318        | 83.33 | 39000 | 0.0209 | 0.2166          | 0.0685 |
-| 0.0325        | 85.47 | 40000 | 0.2172 | 0.0209          | 0.0687 |
-| 0.0316        | 87.6  | 41000 | 0.2181 | 0.0208          | 0.0678 |
-| 0.0302        | 89.73 | 42000 | 0.2171 | 0.0208          | 0.0679 |
-| 0.0318        | 91.87 | 43000 | 0.2179 | 0.0211          | 0.0702 |
-| 0.0314        | 94.0  | 44000 | 0.2186 | 0.0208          | 0.0690 |
-| 0.0309        | 96.13 | 45000 | 0.2193 | 0.0210          | 0.0696 |
-| 0.031         | 98.27 | 46000 | 0.2191 | 0.0208          | 0.0686 |
+| 0.0325        | 85.47 | 40000 | 0.0209 | 0.2172          | 0.0687 |
+| 0.0316        | 87.6  | 41000 | 0.0208 | 0.2181          | 0.0678 |
+| 0.0302        | 89.73 | 42000 | 0.0208 | 0.2171          | 0.0679 |
+| 0.0318        | 91.87 | 43000 | 0.0211 | 0.2179          | 0.0702 |
+| 0.0314        | 94.0  | 44000 | 0.0208 | 0.2186          | 0.0690 |
+| 0.0309        | 96.13 | 45000 | 0.0210 | 0.2193          | 0.0696 |
+| 0.031         | 98.27 | 46000 | 0.0208 | 0.2191          | 0.0686 |
 
 
 ### Framework versions