Ananya21wh/ATC_Model

Training Configuration The model was fine-tuned using the Hugging Face transformers library with the following training arguments:

from transformers import TrainingArguments

training_args = TrainingArguments( output_dir="./wav2vec2-librispeech", # Directory to save the model overwrite_output_dir=False, # Do not overwrite existing model outputs model="facebook/wav2vec2-large-lv60", # Base pre-trained Wav2Vec2 model dataset_config_name="clean", # Use the "clean" subset of the dataset train_split_name="train.100", # Training split from the dataset eval_split_name="validation", # Evaluation split from the dataset per_device_train_batch_size=4, # Batch size per GPU/TPU gradient_accumulation_steps=1, # Steps to accumulate gradients num_train_epochs=3, # Number of training epochs learning_rate=3e-4, # Learning rate warmup_steps=500, # Warm-up steps for the scheduler evaluation_strategy="steps", # Evaluate at specific steps save_steps=500, # Save checkpoint every 500 steps eval_steps=5000, # Perform evaluation every 5000 steps logging_steps=1, # Log every step save_total_limit=3, # Limit the number of saved checkpoints group_by_length=True, # Group sequences of similar lengths chars_to_ignore_regex='[,?.!-;:"]', # Characters to ignore in text processing do_train=True, # Enable training do_eval=True, # Enable evaluation fp16=True, # Use 16-bit floating point precision gradient_checkpointing=True, # Enable gradient checkpointing report_to="all" # Report metrics to all available trackers )

Fine-Tuning Details Base Model: facebook/wav2vec2-large-lv60 Dataset: LibriSpeech, subset: clean Training split: train.100 Validation split: validation Batch Size: 4 per device Epochs: 3 Learning Rate: 3e-4 with 500 warm-up steps Mixed Precision: Enabled (fp16) Gradient Accumulation: 1 step Gradient Checkpointing: Enabled to save memory Save Frequency: Every 500 steps, keeping a maximum of 3 checkpoints Ignored Characters: [ , ? . ! - ; : "] in text processing Additional Notes The model was fine-tuned on GPUs for improved performance. The group_by_length parameter was used to optimize data loading by grouping audio samples of similar lengths during training. Logs and metrics were reported using wandb