---
language:
- he
tags:
- automatic-speech-recognition
- robust-speech-event
- he
- generated_from_trainer
model-index:
- name: wav2vec2-xls-r-300m-hebrew
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-hebrew

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the private dataset with stats:

| split  |size | n_samples | duration(hrs)|   |
|---|---|---|---|---|
|train|4.19gb| 20306  | 28  |   |
|dev  |1.05gb|  5076 |  7 |   |


It achieves the following results on the evaluation set:
- Loss: 0.5438
- Wer: 0.1773

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 100.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| No log        | 3.15  | 1000  | 0.5203          | 0.4333 |
| 1.4284        | 6.31  | 2000  | 0.4816          | 0.3951 |
| 1.4284        | 9.46  | 3000  | 0.4315          | 0.3546 |
| 1.283         | 12.62 | 4000  | 0.4278          | 0.3404 |
| 1.283         | 15.77 | 5000  | 0.4090          | 0.3054 |
| 1.1777        | 18.93 | 6000  | 0.3893          | 0.3006 |
| 1.1777        | 22.08 | 7000  | 0.3968          | 0.2857 |
| 1.0994        | 25.24 | 8000  | 0.3892          | 0.2751 |
| 1.0994        | 28.39 | 9000  | 0.4061          | 0.2690 |
| 1.0323        | 31.54 | 10000 | 0.4114          | 0.2507 |
| 1.0323        | 34.7  | 11000 | 0.4021          | 0.2508 |
| 0.9623        | 37.85 | 12000 | 0.4032          | 0.2378 |
| 0.9623        | 41.01 | 13000 | 0.4148          | 0.2374 |
| 0.9077        | 44.16 | 14000 | 0.4350          | 0.2323 |
| 0.9077        | 47.32 | 15000 | 0.4515          | 0.2246 |
| 0.8573        | 50.47 | 16000 | 0.4474          | 0.2180 |
| 0.8573        | 53.63 | 17000 | 0.4649          | 0.2171 |
| 0.8083        | 56.78 | 18000 | 0.4455          | 0.2102 |
| 0.8083        | 59.94 | 19000 | 0.4587          | 0.2092 |
| 0.769         | 63.09 | 20000 | 0.4794          | 0.2012 |
| 0.769         | 66.25 | 21000 | 0.4845          | 0.2007 |
| 0.7308        | 69.4  | 22000 | 0.4937          | 0.2008 |
| 0.7308        | 72.55 | 23000 | 0.4920          | 0.1895 |
| 0.6927        | 75.71 | 24000 | 0.5179          | 0.1911 |
| 0.6927        | 78.86 | 25000 | 0.5202          | 0.1877 |
| 0.6622        | 82.02 | 26000 | 0.5266          | 0.1840 |
| 0.6622        | 85.17 | 27000 | 0.5351          | 0.1854 |
| 0.6315        | 88.33 | 28000 | 0.5373          | 0.1811 |
| 0.6315        | 91.48 | 29000 | 0.5331          | 0.1792 |
| 0.6075        | 94.64 | 30000 | 0.5390          | 0.1779 |
| 0.6075        | 97.79 | 31000 | 0.5459          | 0.1773 |


### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2.dev0
- Tokenizers 0.11.0