Create README.md

README.md

@@ -0,0 +1,235 @@
+---
+language: ar
+datasets:
+- arabic_speech_corpus
+- mozilla-foundation/common_voice_6_1
+metrics:
+- wer
+tags:
+- audio
+- automatic-speech-recognition
+- speech
+- xlsr-fine-tuning-week
+license: apache-2.0
+model-index:
+- name: muzamil47-wav2vec2-large-xlsr-53-arabic
+  results:
+  - task: 
+      name: Automatic Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: Common Voice 6.1 (Arabic)
+      type: mozilla-foundation/common_voice_6_1
+      config: ar
+    metrics:
+       - name: Test WER
+         type: wer
+         value: 53.54
+---
+
+# Wav2Vec2-Large-XLSR-53-Arabic
+
+Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Arabic using the [Common Voice](https://huggingface.co/datasets/common_voice). 
+When using this model, make sure that your speech input is sampled at 16kHz.
+
+## Usage
+
+The model can be used directly (without a language model) as follows:
+
+```python
+import librosa
+import torch
+from lang_trans.arabic import buckwalter
+
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+
+asr_model = "muzamil47/wav2vec2-large-xlsr-53-arabic-demo"
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+
+def load_file_to_data(file, srate=16_000):
+    batch = {}
+    speech, sampling_rate = librosa.load(file, sr=srate)
+    batch["speech"] = speech
+    batch["sampling_rate"] = sampling_rate
+    return batch
+
+
+max_length = 128000
+processor = Wav2Vec2Processor.from_pretrained(asr_model)
+model = Wav2Vec2ForCTC.from_pretrained(asr_model).to(device)
+
+
+def predict(data):
+    features = processor(data["speech"], sampling_rate=data["sampling_rate"], return_tensors="pt", padding=True)
+    input_values = features.input_values.to(device)
+    try:
+        attention_mask = features.attention_mask.to(device)
+    except:
+        attention_mask = None
+    with torch.no_grad():
+        predicted = torch.argmax(model(input_values, attention_mask=attention_mask).logits, dim=-1)
+
+    data["predicted"] = processor.tokenizer.decode(predicted[0])
+    print(data["predicted"])
+    print("predicted:", buckwalter.untrans(data["predicted"]))
+    return data
+
+predict(load_file_to_data("common_voice_ar_19058307.mp3"))
+```
+**Output Result**:
+```shell
+reference: هل يمكنني التحدث مع المسؤول هنا
+predicted: هل يمكنني التحدث مع المسؤول هنا
+```
+
+## Evaluation
+
+The model can be evaluated as follows on the Arabic test data of Common Voice. 
+
+```python
+import torch
+import torchaudio
+from datasets import load_dataset
+from lang_trans.arabic import buckwalter
+from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
+
+asr_model = "muzamil47/wav2vec2-large-xlsr-53-arabic-demo"
+
+dataset = load_dataset("common_voice", "ar", split="test[:10]")
+
+resamplers = {  # all three sampling rates exist in test split
+    48000: torchaudio.transforms.Resample(48000, 16000),
+    44100: torchaudio.transforms.Resample(44100, 16000),
+    32000: torchaudio.transforms.Resample(32000, 16000),
+}
+
+def prepare_example(example):
+    speech, sampling_rate = torchaudio.load(example["path"])
+    example["speech"] = resamplers[sampling_rate](speech).squeeze().numpy()
+    return example
+
+dataset = dataset.map(prepare_example)
+processor = Wav2Vec2Processor.from_pretrained(asr_model)
+model = Wav2Vec2ForCTC.from_pretrained(asr_model).eval()
+
+def predict(batch):
+    inputs = processor(batch["speech"], sampling_rate=16000, return_tensors="pt", padding=True)
+    with torch.no_grad():
+        predicted = torch.argmax(model(inputs.input_values).logits, dim=-1)
+    predicted[predicted == -100] = processor.tokenizer.pad_token_id  # see fine-tuning script
+    batch["predicted"] = processor.tokenizer.batch_decode(predicted)
+    return batch
+
+dataset = dataset.map(predict, batched=True, batch_size=1, remove_columns=["speech"])
+
+for reference, predicted in zip(dataset["sentence"], dataset["predicted"]):
+    print("reference:", reference)
+    print("predicted:", buckwalter.untrans(predicted))
+    print("--")
+
+```
+**Output Results**:
+```shell
+reference: ما أطول عودك!
+predicted: ما اطول عودك
+
+reference: ماتت عمتي منذ سنتين.
+predicted: ما تتعمتي منذو سنتين
+
+reference: الألمانية ليست لغة سهلة.
+predicted: الالمانية ليست لغة سهلة
+
+reference: طلبت منه أن يبعث الكتاب إلينا.
+predicted: طلبت منه ان يبعث الكتاب الينا
+
+reference: .السيد إيتو رجل متعلم
+predicted: السيد ايتو رجل متعلم
+
+reference: الحمد لله.
+predicted: الحمذ لللا
+
+reference: في الوقت نفسه بدأت الرماح والسهام تقع بين الغزاة
+predicted: في الوقت نفسه ابدات الرماح و السهام تقع بين الغزاء
+
+reference: لا أريد أن أكون ثقيلَ الظِّل ، أريد أن أكون رائعًا! !
+predicted: لا اريد ان اكون ثقيل الظل اريد ان اكون رائع
+
+reference: خذ مظلة معك في حال أمطرت.
+predicted: خذ مظلة معك في حال امطرت
+
+reference: .ركب توم السيارة
+predicted: ركب توم السيارة
+```
+
+The model evaluation **(WER)** on the Arabic test data of Common Voice. 
+
+```python
+import re
+
+import torch
+import torchaudio
+from datasets import load_dataset, load_metric
+from transformers import set_seed, Wav2Vec2ForCTC, Wav2Vec2Processor
+
+set_seed(42)
+
+test_dataset = load_dataset("common_voice", "ar", split="test")
+
+processor = Wav2Vec2Processor.from_pretrained("muzamil47/wav2vec2-large-xlsr-53-arabic-demo")
+model = Wav2Vec2ForCTC.from_pretrained("muzamil47/wav2vec2-large-xlsr-53-arabic-demo")
+model.to("cuda")
+
+chars_to_ignore_regex = '[\,\؟\.\!\-\;\\:\'\"\☭\«\»\؛\—\ـ\_\،\“\%\‘\”\�]'
+
+resampler = torchaudio.transforms.Resample(48_000, 16_000)
+
+
+# Preprocessing the datasets. We need to read the aduio files as arrays
+def speech_file_to_array_fn(batch):
+    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
+    batch["sentence"] = re.sub('[a-z]','',batch["sentence"])
+    batch["sentence"] = re.sub("[إأٱآا]", "ا", batch["sentence"])
+    noise = re.compile(""" ّ    | # Tashdid
+                             َ    | # Fatha
+                             ً    | # Tanwin Fath
+                             ُ    | # Damma
+                             ٌ    | # Tanwin Damm
+                             ِ    | # Kasra
+                             ٍ    | # Tanwin Kasr
+                             ْ    | # Sukun
+                             ـ     # Tatwil/Kashida
+                         """, re.VERBOSE)
+    batch["sentence"] = re.sub(noise, '', batch["sentence"])
+    speech_array, sampling_rate = torchaudio.load(batch["path"])
+    batch["speech"] = resampler(speech_array).squeeze().numpy()
+    return batch
+
+test_dataset = test_dataset.map(speech_file_to_array_fn)
+
+
+def evaluate(batch):
+    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
+
+    with torch.no_grad():
+         logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
+
+    pred_ids = torch.argmax(logits, dim=-1)
+    batch["pred_strings"] = processor.batch_decode(pred_ids)
+    return batch
+
+result = test_dataset.map(evaluate, batched=True, batch_size=8)
+
+wer = load_metric("wer")
+print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
+
+```
+
+**Test Result**: 53.54
+
+
+## Training
+
+The Common Voice `train`, `validation` datasets were used for training.
+
+The script used for training can be found [here](https://huggingface.co/kmfoda/wav2vec2-large-xlsr-arabic/tree/main)