Upload README.md

README.md

@@ -0,0 +1,110 @@
+---
+language: en
+datasets:
+- timit_asr
+tags:
+- audio
+- automatic-speech-recognition
+- speech
+license: apache-2.0
+---
+
+# Wav2Vec2-Large-LV60-TIMIT
+
+Fine-tuned [facebook/wav2vec2-large-lv60](https://huggingface.co/facebook/wav2vec2-large-lv60)
+on the [timit_asr dataset](https://huggingface.co/datasets/timit_asr).
+When using this model, make sure that your speech input is sampled at 16kHz.
+
+## Usage
+
+The model can be used directly (without a language model) as follows:
+
+```python
+import soundfile as sf
+import torch
+from datasets import load_dataset
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+
+model_name = "elgeish/wav2vec2-large-lv60-timit-asr"
+processor = Wav2Vec2Processor.from_pretrained(model_name)
+model = Wav2Vec2ForCTC.from_pretrained(model_name)
+model.eval()
+
+dataset = load_dataset("timit_asr", split="test").shuffle().select(range(10))
+char_translations = str.maketrans({"-": " ", ",": "", ".": "", "?": ""})
+
+def prepare_example(example):
+    example["speech"], _ = sf.read(example["file"])
+    example["text"] = example["text"].translate(char_translations)
+    example["text"] = " ".join(example["text"].split())  # clean up whitespaces
+    example["text"] = example["text"].lower()
+    return example
+
+dataset = dataset.map(prepare_example, remove_columns=["file"])
+inputs = processor(dataset["speech"], sampling_rate=16000, return_tensors="pt", padding="longest")
+
+with torch.no_grad():
+    predicted_ids = torch.argmax(model(inputs.input_values).logits, dim=-1)
+predicted_ids[predicted_ids == -100] = processor.tokenizer.pad_token_id  # see fine-tuning script
+predicted_transcripts = processor.tokenizer.batch_decode(predicted_ids)
+
+for reference, predicted in zip(dataset["text"], predicted_transcripts):
+    print("reference:", reference)
+    print("predicted:", predicted)
+    print("--")
+```
+
+Here's the output:
+
+```
+reference: the emblem depicts the acropolis all aglow
+predicted: the amblum depicts the acropolis all a glo
+--
+reference: don't ask me to carry an oily rag like that
+predicted: don't ask me to carry an oily rag like that
+--
+reference: they enjoy it when i audition
+predicted: they enjoy it when i addition
+--
+reference: set aside to dry with lid on sugar bowl
+predicted: set aside to dry with a litt on shoogerbowl
+--
+reference: a boring novel is a superb sleeping pill
+predicted: a bor and novel is a suberb sleeping peel
+--
+reference: only the most accomplished artists obtain popularity
+predicted: only the most accomplished artists obtain popularity
+--
+reference: he has never himself done anything for which to be hated which of us has
+predicted: he has never himself done anything for which to be hated which of us has
+--
+reference: the fish began to leap frantically on the surface of the small lake
+predicted: the fish began to leap frantically on the surface of the small lake
+--
+reference: or certain words or rituals that child and adult go through may do the trick
+predicted: or certain words or rituals that child an adult go through may do the trick
+--
+reference: are your grades higher or lower than nancy's
+predicted: are your grades higher or lower than nancies
+--
+```
+
+## Fine-Tuning Script
+
+You can find the script used to produce this model
+[here](https://github.com/elgeish/transformers/blob/8ee49e09c91ffd5d23034ce32ed630d988c50ddf/examples/research_projects/wav2vec2/finetune_large_lv60_timit_asr.sh).
+
+**Note:** This model can be fine-tuned further;
+[trainer_state.json](https://huggingface.co/elgeish/wav2vec2-large-lv60-timit-asr/blob/main/trainer_state.json)
+shows useful details, namely the last state (this checkpoint):
+
+```json
+{
+    "epoch": 29.51,
+    "eval_loss": 25.424150466918945,
+    "eval_runtime": 182.9499,
+    "eval_samples_per_second": 9.183,
+    "eval_wer": 0.1351704233095107,
+    "step": 8500
+}
+```