1 ---
2 language: eo
3 datasets:
4 - common_voice
5 metrics:
6 - wer
7 tags:
8 - audio
9 - automatic-speech-recognition
10 - speech
11 - xlsr-fine-tuning-week
12 license: apache-2.0
13 model-index:
14 - name: XLSR Wav2Vec2 Esperanto by Charles Pierse
15 results:
16 - task:
17 name: Speech Recognition
18 type: automatic-speech-recognition
19 dataset:
20 name: Common Voice eo
21 type: common_voice
22 args: eo
23 metrics:
24 - name: Test WER
25 type: wer
26 value: 12.31
27 ---
28
29 # Wav2Vec2-Large-XLSR-53-eo
30
31 Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on esperanto using the [Common Voice](https://huggingface.co/datasets/common_voice) dataset.
32
33 When using this model, make sure that your speech input is sampled at 16kHz.
34
35 ## Usage
36
37 The model can be used directly (without a language model) as follows:
38
39 ```python
40 import torch
41 import torchaudio
42 from datasets import load_dataset
43 from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
44
45 test_dataset = load_dataset("common_voice", "eo", split="test[:2%]")
46 processor = Wav2Vec2Processor.from_pretrained("cpierse/wav2vec2-large-xlsr-53-esperanto")
47 model = Wav2Vec2ForCTC.from_pretrained("cpierse/wav2vec2-large-xlsr-53-esperanto")
48
49 resampler = torchaudio.transforms.Resample(48_000, 16_000)
50
51 # Preprocessing the datasets.
52 # We need to read the aduio files as arrays
53 def speech_file_to_array_fn(batch):
54 speech_array, sampling_rate = torchaudio.load(batch["path"])
55 batch["speech"] = resampler(speech_array).squeeze().numpy()
56 return batch
57
58 test_dataset = test_dataset.map(speech_file_to_array_fn)
59 inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
60
61 with torch.no_grad():
62 logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
63
64 predicted_ids = torch.argmax(logits, dim=-1)
65
66 print("Prediction:", processor.batch_decode(predicted_ids))
67 print("Reference:", test_dataset["sentence"][:2])
68 ```
69
70
71 ## Evaluation
72
73 The model can be evaluated as follows on the Esperanto test data of Common Voice.
74
75
76 ```python
77 import torch
78 import torchaudio
79 from datasets import load_dataset, load_metric
80 from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
81 import re
82 import jiwer
83
84 def chunked_wer(targets, predictions, chunk_size=None):
85 if chunk_size is None: return jiwer.wer(targets, predictions)
86 start = 0
87 end = chunk_size
88 H, S, D, I = 0, 0, 0, 0
89 while start < len(targets):
90 chunk_metrics = jiwer.compute_measures(targets[start:end], predictions[start:end])
91 H = H + chunk_metrics["hits"]
92 S = S + chunk_metrics["substitutions"]
93 D = D + chunk_metrics["deletions"]
94 I = I + chunk_metrics["insertions"]
95 start += chunk_size
96 end += chunk_size
97 return float(S + D + I) / float(H + S + D)
98
99 test_dataset = load_dataset("common_voice", "eo", split="test") #TODO: replace {lang_id} in your language code here. Make sure the code is one of the *ISO codes* of [this](https://huggingface.co/languages) site.
100 wer = load_metric("wer")
101
102 processor = Wav2Vec2Processor.from_pretrained("cpierse/wav2vec2-large-xlsr-53-esperanto")
103 model = Wav2Vec2ForCTC.from_pretrained("cpierse/wav2vec2-large-xlsr-53-esperanto")
104 model.to("cuda")
105
106 chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�\„\«\(\»\)\’\']'
107 resampler = torchaudio.transforms.Resample(48_000, 16_000)
108
109 # Preprocessing the datasets.
110 # We need to read the aduio files as arrays
111 def speech_file_to_array_fn(batch):
112 batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
113 speech_array, sampling_rate = torchaudio.load(batch["path"])
114 batch["speech"] = resampler(speech_array).squeeze().numpy()
115 return batch
116
117 test_dataset = test_dataset.map(speech_file_to_array_fn)
118
119 # Preprocessing the datasets.
120 # We need to read the aduio files as arrays
121 def evaluate(batch):
122 inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
123
124 with torch.no_grad():
125 logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
126
127 pred_ids = torch.argmax(logits, dim=-1)
128 batch["pred_strings"] = processor.batch_decode(pred_ids)
129 return batch
130
131 result = test_dataset.map(evaluate, batched=True, batch_size=8)
132
133 print("WER: {:2f}".format(100 * chunked_wer(predictions=result["pred_strings"], targets=result["sentence"],chunk_size=2000)))
134 ```
135
136 **Test Result**: 12.31 %
137
138
139 ## Training
140
141 The Common Voice `train`, `validation` datasets were used for training.
142