Model Card for wav2vec2-large-xlsr-persian-fine-tuned

Model Details

Model Description

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on Persian language data from the Mozilla Common Voice Dataset. The model is fine-tuned for automatic speech recognition (ASR) tasks.

Developed by: Alireza Dastmalchi Saei
Funded by: -
Shared by: -
Model type: wav2vec2
Language(s) (NLP): Persian
License: MIT
Finetuned from model: wav2vec2-large-xlsr-53

Model Sources

Repository: Model Repository
Paper: -
Demo: -

Uses

Direct Use

This model can be used directly for transcribing Persian speech to text but it needs to be further fine-tuned with data.

Downstream Use

The model can be fine-tuned further for specific ASR tasks or integrated into larger speech-processing pipelines.

Out-of-Scope Use

The model is not suitable for languages other than Persian and may not perform well on noisy audio or speech with heavy accents not represented in the training data.

Bias, Risks, and Limitations

The model is trained on a dataset that may not cover all variations of the Persian language, leading to potential biases in recognizing less represented dialects or accents.

Recommendations

Users should be aware of the biases, risks, and limitations. Further fine-tuning on diverse datasets is recommended to mitigate these biases.

How to Get Started with the Model

from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import torch
import torchaudio

# Load processor and model
processor = Wav2Vec2Processor.from_pretrained("AlirezaSaei/wav2vec2-large-xlsr-persian-fine-tuned")
model = Wav2Vec2ForCTC.from_pretrained("AlirezaSaei/wav2vec2-large-xlsr-persian-fine-tuned")

# Load audio file
audio_input, _ = torchaudio.load("path_to_audio.wav")

# Preprocess and predict
inputs = processor(audio_input, sampling_rate=16000, return_tensors="pt", padding=True)
logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)

print("Transcription:", transcription)

Training Details

Training Data

The model is fine-tuned on the Mozilla Common Voice Dataset. The training data includes Persian speech samples, with lengths filtered between 4 and 6 seconds for training and up to 15 seconds for testing.

Training Procedure

The audio is resampled from 48000 Hz to 16000 Hz. The tokenizer, feature extractor, and processor are defined using the Wav2Vec2CTCTokenizer, Wav2Vec2FeatureExtractor, and Wav2Vec2Processor classes.

Training Hyperparameters

Training regime: fp16 mixed precision
Batch Size: 12
Num Epochs: 5
Learning Rate: 1e-4
Gradient Accumulation Steps: 2
Warmup Steps: 1000

Speeds, Sizes, Times

Training Files: 2217
Testing Files: 5212
Training Time (minutes): 19.67
Total Parameters: 315,479,720
Trainable Parameters: 311,269,544
WER: 1.0

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model is evaluated on a subset of the Mozilla Common Voice Dataset.

Factors

Evaluation is disaggregated by different lengths of audio samples.

Metrics

Word Error Rate (WER) is used as the evaluation metric. It measures the percentage of words that are incorrectly predicted.

Results

The model achieves a WER of 1.0 on the test data.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: Colab T4 GPU

Technical Specifications

Model Architecture and Objective

The model uses the Wav2Vec2 architecture, which is designed for automatic speech recognition.

Compute Infrastructure

Hardware

Colab T4 GPU

Software

Python Notebook (.ipynb)

Model Card Contact

For further information, contact me.