metadata
language: ka
datasets:
- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
widget:
- label: Common Voice sample 566
src: >-
https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-georgian/resolve/main/sample566.flac
- label: Common Voice sample 95
src: >-
https://huggingface.co/m3hrdadfi/wav2vec2-large-xlsr-georgian/resolve/main/sample95.flac
model-index:
- name: XLSR Wav2Vec2 Georgian by Mehrdad Farahani
results:
- task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ka
type: common_voice
args: ka
metrics:
- name: Test WER
type: wer
value: 54
Wav2Vec2-Large-XLSR-53 Georgian
Fine-tuned facebook/wav2vec2-large-xlsr-53 in Georgian using Common Voice. When using this model, make sure that your speech input is sampled at 16kHz.
Usage
The model can be used directly (without a language model) as follows:
!pip install git+https://github.com/huggingface/datasets.git
!pip install git+https://github.com/huggingface/transformers.git
!pip install torchaudio
!pip install librosa
!pip install jiwer
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import librosa
import pandas as pd
import numpy as np
import random
import os
import string
import six
import re
import IPython.display as ipd
# Loading the datasets
dataset = load_dataset("common_voice", "ka", split="test")
print(dataset)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
processor = Wav2Vec2Processor.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-georgian")
model = Wav2Vec2ForCTC.from_pretrained("m3hrdadfi/wav2vec2-large-xlsr-georgian").to(device)
# Preprocessing the datasets.
chars_to_ignore_regex = f"""[{"".join([
",", "?", ".", "!", "-", ";", ":", '""', "%", "'", '"', "๏ฟฝ",
"#", "!", "?", "ยซ", "ยป", "(", ")", "ุ", ",", "?", ".", "!", "-", ";", ":", '"',
"โ", "%", "โ", "๏ฟฝ", "โ", "โฆ", "_", "โ", 'โ', 'โ'
])}]"""
def remove_special_characters(text, chars_to_ignore):
text = re.sub(chars_to_ignore, '', text).lower() + " "
return text
def normalizer(batch, chars_to_ignore):
text = batch["sentence"]
text = remove_special_characters(text, chars_to_ignore)
batch["sentence"] = text
return batch
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
speech_array, sampling_rate = torchaudio.load(batch["path"])
speech_array = speech_array.squeeze().numpy()
speech_array = librosa.resample(np.asarray(speech_array), sampling_rate, 16_000)
batch["speech"] = speech_array
return batch
def predict(batch):
features = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
input_values = features.input_values.to(device)
attention_mask = features.attention_mask.to(device)
with torch.no_grad():
logits = model(input_values, attention_mask=attention_mask).logits
pred_ids = torch.argmax(logits, dim=-1)
batch["predicted"] = processor.batch_decode(pred_ids)[0]
return batch
dataset = dataset.map(normalizer, fn_kwargs={"chars_to_ignore": chars_to_ignore_regex})
dataset = dataset.map(speech_file_to_array_fn, remove_columns=list(set(dataset.column_names) - set(['sentence', 'path'])))
result = dataset.map(predict)
Prediction
max_items = np.random.randint(0, len(result), 20).tolist()
for i in max_items:
reference, predicted = result["sentence"][i], result["predicted"][i]
print("reference:", reference)
print("predicted:", predicted)
print('---')
reference: แแแแแแแกแขแ แแชแแฃแแ แชแแแขแ แ แฅแแแแฅแ แแแแจแแ
predicted: แแแแแแแกแขแ แแชแแฃแแ แชแแแขแ แ แฅแแแแฅแ แแแแจแแ
---
reference: แแแแแแแ แแแแแแแขแแก แแฏแแฎแจแ
predicted: แแแแแแ แแแแแแแขแแก แแฏแแฎแจแ
---
reference: แแฆแกแแแแจแแแแแ แ แแ แกแแแฆแแ แ แฌแแ แแแแแแแแก แแแ แแแแแแ แขแแแกแ แแ แฏแแ แฏ แฐแแ แแกแแแแก แแจแแแแ แแแแแแฃแ แแฃแแขแก
predicted: แแฆแกแแแแจแแแแแแ แ แกแแแฆแ แ แแฌแแ แแแแแแแแก แแแ แแแแแ แแแแก แแ แฏแแ แฉแฎแแ แแกแแแแก แแจแแแแ แแแแแแฃแ แแฃแแแก
---
reference: แแแ แซแแแแแแแ แฌแแ แแแแแชแแ แฅแแ แแฃแ แแแแแ
predicted: แแแ แซแแแแแแแ แฌแแ แแ แแแชแแ แฅแแ แแฃแ แแแแแ
---
reference: แแฆแแแ แแฃแแแ แแแแแกแ แแ แแแ แแแก แแแแขแแแแแแก แกแแแฆแแแ แแ
predicted: แแฆแแแ แแฃแแแ แแแแแกแ แแ แแแ แแแก แแแแแแแแแแก แกแแแฆแแแ แแ
---
reference: แแฅ แแแ แแแแฌแแแแก แกแแแฎแแขแแ แ แแแแแแแแแจแ แกแแแแช แกแแชแแชแฎแแแก แแแแแแแ แแฌแแแแ แแแแแแแแแฃแ แแแฆแแแฌแแแแแก
predicted: แแฅ แแแ แแแแกแฌแ แแแแก แกแแแฎแแขแ แ แแแแแแแ แแจแแกแ แแ แชแแชแแชแฎแแแก แแแแแแแ แแฌแงแแแแ แแ แแแแแแฃแแแแแ แแแงแแแฌแแแแแแก
---
reference: แแแแ แแกแ แแแแฎแแแแแ แจแแแแแแแแแแแแแ แแ แแแฅแขแแ แแก แแแฎแแแ แแแแ แกแแ แแฃแแ แแแแแแแแก แแแแแก แแแแแแแแ
predicted: แแแแ แแก แแแ แฎแแแแแ แจแแแฃแแแแแแ แแแแ แแ แแแฅแขแแ แแก แแแฎแแแ แแแแช แกแแ แแฃแ แ แแแแแแแแก แแแแแก แแแแแแแแแ
---
reference: แแแ แซแแแ แขแงแแแแแแ แแแญแ แแแแก แฌแแแแแฆแแแแ
predicted: แแแแ แซแแขแ แขแงแแแแแแ แแแญแ แแแแก แฌแแแแแฆแแแ
---
reference: แกแแแแแกแก แแฆแแแกแแแแแแแ แแ แแแกแแแแแแแ แแแแ แกแแ แแแแแ แแฅแแก
predicted: แกแแแแแก แแฆแแแกแแแแแแแ แแ แแแกแแแแแ แแแแแ แกแแ แแแแแ แแฅแแก
---
reference: แแแ แแแแแแ แแแแก แฅแแแแฅแแก แฉแ แแแแแแฆแแแกแแแแแ แแแฌแแแจแ
predicted: แแแ แแแแแแ แแแแก แฅแแแแฅแแก แฉแ แแแแ แแฆแแแกแแแแแ แแแฌแแแจแ
---
Evaluation
wer = load_metric("wer")
print("WER: {:.2f}".format(100 * wer.compute(predictions=result["predicted"], references=result["sentence"])))
Test Result:
- WER: 54.00%
Training & Report
The Common Voice train
, validation
datasets were used for training.
You can see the training states here
The script used for training can be found here