---
language:
- pt
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_7_0
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
license: apache-2.0
model-index:
- name: wav2vec2-xls-r-pt-cv7-from-bp400h
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 7
      type: mozilla-foundation/common_voice_7_0
      args: pt
    metrics:
       - name: Test WER
         type: wer
         value: 12.13
       - name: Test CER
         type: cer
         value: 3.68
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Robust Speech Event - Dev Data
      type: speech-recognition-community-v2/dev_data
      args: sv
    metrics:
       - name: Test WER
         type: wer
         value: 28.23
       - name: Test CER
         type: cer
         value: 12.58
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-pt-cv7-from-bp400h

This model is a fine-tuned version of [lgris/bp_400h_xlsr2_300M](https://huggingface.co/lgris/bp_400h_xlsr2_300M) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1535
- Wer: 0.1254

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 5000

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.4991        | 0.13  | 100  | 0.1774          | 0.1464 |
| 0.4655        | 0.26  | 200  | 0.1884          | 0.1568 |
| 0.4689        | 0.39  | 300  | 0.2282          | 0.1672 |
| 0.4662        | 0.52  | 400  | 0.1997          | 0.1584 |
| 0.4592        | 0.65  | 500  | 0.1989          | 0.1663 |
| 0.4533        | 0.78  | 600  | 0.2004          | 0.1698 |
| 0.4391        | 0.91  | 700  | 0.1888          | 0.1642 |
| 0.4655        | 1.04  | 800  | 0.1921          | 0.1624 |
| 0.4138        | 1.17  | 900  | 0.1950          | 0.1602 |
| 0.374         | 1.3   | 1000 | 0.2077          | 0.1658 |
| 0.4064        | 1.43  | 1100 | 0.1945          | 0.1596 |
| 0.3922        | 1.56  | 1200 | 0.2069          | 0.1665 |
| 0.4226        | 1.69  | 1300 | 0.1962          | 0.1573 |
| 0.3974        | 1.82  | 1400 | 0.1919          | 0.1553 |
| 0.3631        | 1.95  | 1500 | 0.1854          | 0.1573 |
| 0.3797        | 2.08  | 1600 | 0.1902          | 0.1550 |
| 0.3287        | 2.21  | 1700 | 0.1926          | 0.1598 |
| 0.3568        | 2.34  | 1800 | 0.1888          | 0.1534 |
| 0.3415        | 2.47  | 1900 | 0.1834          | 0.1502 |
| 0.3545        | 2.6   | 2000 | 0.1906          | 0.1560 |
| 0.3344        | 2.73  | 2100 | 0.1804          | 0.1524 |
| 0.3308        | 2.86  | 2200 | 0.1741          | 0.1485 |
| 0.344         | 2.99  | 2300 | 0.1787          | 0.1455 |
| 0.309         | 3.12  | 2400 | 0.1773          | 0.1448 |
| 0.312         | 3.25  | 2500 | 0.1738          | 0.1440 |
| 0.3066        | 3.38  | 2600 | 0.1727          | 0.1417 |
| 0.2999        | 3.51  | 2700 | 0.1692          | 0.1436 |
| 0.2985        | 3.64  | 2800 | 0.1732          | 0.1430 |
| 0.3058        | 3.77  | 2900 | 0.1754          | 0.1402 |
| 0.2943        | 3.9   | 3000 | 0.1691          | 0.1379 |
| 0.2813        | 4.03  | 3100 | 0.1754          | 0.1376 |
| 0.2733        | 4.16  | 3200 | 0.1639          | 0.1363 |
| 0.2592        | 4.29  | 3300 | 0.1675          | 0.1349 |
| 0.2697        | 4.42  | 3400 | 0.1618          | 0.1360 |
| 0.2538        | 4.55  | 3500 | 0.1658          | 0.1348 |
| 0.2746        | 4.67  | 3600 | 0.1674          | 0.1325 |
| 0.2655        | 4.8   | 3700 | 0.1655          | 0.1319 |
| 0.2745        | 4.93  | 3800 | 0.1665          | 0.1316 |
| 0.2617        | 5.06  | 3900 | 0.1600          | 0.1311 |
| 0.2674        | 5.19  | 4000 | 0.1623          | 0.1311 |
| 0.237         | 5.32  | 4100 | 0.1591          | 0.1315 |
| 0.2669        | 5.45  | 4200 | 0.1584          | 0.1295 |
| 0.2476        | 5.58  | 4300 | 0.1572          | 0.1285 |
| 0.2445        | 5.71  | 4400 | 0.1580          | 0.1271 |
| 0.2207        | 5.84  | 4500 | 0.1567          | 0.1269 |
| 0.2289        | 5.97  | 4600 | 0.1536          | 0.1260 |
| 0.2438        | 6.1   | 4700 | 0.1530          | 0.1260 |
| 0.227         | 6.23  | 4800 | 0.1544          | 0.1249 |
| 0.2256        | 6.36  | 4900 | 0.1543          | 0.1254 |
| 0.2184        | 6.49  | 5000 | 0.1535          | 0.1254 |

### Framework versions

- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


---
language: pt
datasets:
- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0
---

# bp500-base100k_voxpopuli: Wav2vec 2.0 with Brazilian Portuguese (BP) Dataset

This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the following datasets:

- [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz): contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus.
- [Common Voice 7.0](https://commonvoice.mozilla.org/pt):  is a project proposed by Mozilla Foundation with the goal to create a wide open dataset in different languages. In this project, volunteers donate and validate speech using the [oficial site](https://commonvoice.mozilla.org/pt).
- [Lapsbm](https://github.com/falabrasil/gitlab-resources): "Falabrasil - UFPA" is a dataset used by the Fala Brasil group to benchmark ASR systems in Brazilian Portuguese. Contains 35 speakers (10 females), each one pronouncing 20 unique sentences, totalling  700 utterances in Brazilian Portuguese. The audios were recorded in 22.05 kHz without environment control.
- [Multilingual Librispeech (MLS)](https://arxiv.org/abs/2012.03411): a massive dataset available in many languages. The MLS is based on audiobook recordings in public domain like [LibriVox](https://librivox.org/). The dataset contains a total of 6k hours of transcribed data in many languages. The set in Portuguese [used in this work](http://www.openslr.org/94/) (mostly Brazilian variant) has approximately 284 hours of speech, obtained from 55 audiobooks read by 62 speakers.
- [Multilingual TEDx](http://www.openslr.org/100): a collection of audio recordings from TEDx talks in 8 source languages. The Portuguese set (mostly Brazilian Portuguese variant) contains 164 hours of transcribed speech.
- [Sidney](https://igormq.github.io/datasets/) (SID): contains 5,777 utterances recorded by 72 speakers (20 women) from 17 to 59 years old with fields such as place of birth, age, gender, education, and occupation;
- [VoxForge](http://www.voxforge.org/): is a project with the goal to build open datasets for acoustic models. The corpus contains approximately 100 speakers and 4,130 utterances of Brazilian Portuguese, with sample rates varying from 16kHz to 44.1kHz.

These datasets were combined to build a larger Brazilian Portuguese dataset. All data was used for training except Common Voice dev/test sets, that were used for validation/test respectively. We also made test sets for all the gathered datasets.

| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          | 94.0h  |    -- |  5.4h |
| Common Voice                   | 37.8h  |  8.9h |  9.5h |
| LaPS BM                        | 0.8h   |    -- |  0.1h |
| MLS                            | 161.0h |    -- |  3.7h |
| Multilingual TEDx (Portuguese) | 148.9h |    -- |  1.8h |
| SID                            | 7.2h   |    -- |  1.0h |
| VoxForge                       | 3.9h   |    -- |  0.1h |
| Total                          | 453.6h |  8.9h | 21.6h |

The original model was fine-tuned using [fairseq](https://github.com/pytorch/fairseq). This notebook uses a converted version of the original one. The link to the original fairseq model is available [here](https://drive.google.com/file/d/10iESR5AQxuxF5F7w3wLbpc_9YMsYbY9H/view?usp=sharing).

#### Summary

|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| bp\_500-base100k_voxpopuli (demonstration below)              | 0.142 | 0.201 | 0.052 | 0.224 | 0.102 | 0.317 | 0.048 | 0.155 |
| bp\_500-base100k_voxpopuli + 4-gram (demonstration below)       | 0.099 | 0.149 | 0.047 | 0.192 | 0.115 | 0.371 | 0.127 | 0.157 |

#### Transcription examples

| Text                                                                                | Transcription                                                                                |
|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|
|qual o instagram dele|**qualo** **está** **gramedele**|
|o capitão foi expulso do exército porque era doido|o **capitãl** foi **exposo** do exército porque era doido|
|também por que não|também **porque** não|
|não existe tempo como o presente|não existe tempo como *o* presente|
|eu pulei para salvar rachel|eu pulei para salvar **haquel**|
|augusto cezar passos marinho|augusto **cesa** **passoesmarinho**|

## Demonstration

```python
MODEL_NAME = "lgris/bp500-base100k_voxpopuli"
```

### Imports and dependencies

```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```

```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```

### Helpers

```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```

```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```

```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```

### Model

```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```

### Download datasets

```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```

```python
%cd bp_dataset
```

    /content/bp_dataset

### Tests

```python
stt = STT(MODEL_NAME)
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.1419179499917191

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.20079950312040154

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.052780934343434324

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.22413887199364113

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.1019041538671034

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.31711268778273327

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.04826433982683982

### Tests with LM

```python
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```

### Cetuc

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.099518615112877

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.1488912889506362

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.047080176767676764

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.19220291966887196

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.11535498771650306

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.3707890073539895

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.12682088744588746


---
language: pt
datasets:
- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0
---

# sid10-xlsr: Wav2vec 2.0 with Sidney Dataset

This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the [Sidney](https://igormq.github.io/datasets/) dataset.

In this notebook the model is tested against other available Brazilian Portuguese datasets.

| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          |        |    -- |  5.4h |
| Common Voice                   |        |    -- |  9.5h |
| LaPS BM                        |        |    -- |  0.1h |
| MLS                            |        |    -- |  3.7h |
| Multilingual TEDx (Portuguese) |        |    -- |  1.8h |
| SID                            |   7.2h |    -- |  1.0h |
| VoxForge                       |        |    -- |  0.1h |
| Total                          |    7.2h|    -- | 21.6h |

#### Summary

|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| sid\_10 (demonstration below) |0.186 | 0.327 | 0.207 | 0.505 | 0.124 | 0.835 | 0.472 | 0.379|
| sid\_10 + 4-gram (demonstration below) |0.096 | 0.223 | 0.115 | 0.432 | 0.101 | 0.791 | 0.348 | 0.301|

## Demonstration

```python
MODEL_NAME = "lgris/sid10-xlsr"
```

### Imports and dependencies

```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```

```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```

### Helpers

```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```

```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```

```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```

### Model

```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        attention_mask = features.attention_mask.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values, attention_mask=attention_mask).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```

### Download datasets

```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```

### Tests

```python
stt = STT(MODEL_NAME)
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.18623689076557778

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.3279775395502392

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.20780303030303032

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.5056711598536057

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.1247776617710105

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.8350609256842175

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.47242153679653687

### Tests with LM

```python
# !find -type f -name "*.wav" -delete
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.09677271347353278

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.22363215674470321

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.1154924242424242

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.4322369152606427

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.10080313085145765

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.7911789829264236

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.34786255411255407


---
language: pt
tags:
- speech
license: apache-2.0
---

# DistilXLSR-53 for BP
[DistilXLSR-53 for BP: DistilHuBERT applied to Wav2vec XLSR-53 for Brazilian Portuguese](https://github.com/s3prl/s3prl/tree/master/s3prl/upstream/distiller)

The base model pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.

**Note**: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model **speech recognition**, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more in-detail explanation of how to fine-tune the model.
Paper: [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://arxiv.org/abs/2110.01900)
Authors: Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee

**Note 2**: The XLSR-53 model was distilled using [Brazilian Portuguese Datasets](https://huggingface.co/lgris/bp400-xlsr) for test purposes. The dataset is quite small to perform such task (the performance might not be so good as the [original work](https://arxiv.org/abs/2110.01900)).

**Abstract**

Self-supervised speech representation learning methods like wav2vec 2.0 and Hidden-unit BERT (HuBERT) leverage unlabeled speech data for pre-training and offer good representations for numerous speech processing tasks. Despite the success of these methods, they require large memory and high pre-training costs, making them inaccessible for researchers in academia and small companies. Therefore, this paper introduces DistilHuBERT, a novel multi-task learning framework to distill hidden representations from a HuBERT model directly. This method reduces HuBERT's size by 75% and 73% faster while retaining most performance in ten different tasks. Moreover, DistilHuBERT required little training time and data, opening the possibilities of pre-training personal and on-device SSL models for speech.

# Usage
See [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more information on how to fine-tune the model.


---
language:
- pt
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-1b-cv8
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 8
      type: mozilla-foundation/common_voice_8_0
      args: pt
    metrics:
       - name: Test WER
         type: wer
         value: 17.70
       - name: Test CER
         type: cer
         value: 5.21
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Robust Speech Event - Dev Data
      type: speech-recognition-community-v2/dev_data
      args: sv
    metrics:
       - name: Test WER
         type: wer
         value: 45.68
       - name: Test CER
         type: cer
         value: 18.67
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-1b-cv8

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - PT dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2007
- Wer: 0.1838

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 7.5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 30.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 2.1172        | 0.32  | 500   | 1.2852          | 0.9783 |
| 1.4152        | 0.64  | 1000  | 0.6434          | 0.6105 |
| 1.4342        | 0.96  | 1500  | 0.4844          | 0.3989 |
| 1.4657        | 1.29  | 2000  | 0.5080          | 0.4490 |
| 1.4961        | 1.61  | 2500  | 0.4764          | 0.4264 |
| 1.4515        | 1.93  | 3000  | 0.4519          | 0.4068 |
| 1.3924        | 2.25  | 3500  | 0.4472          | 0.4132 |
| 1.4524        | 2.57  | 4000  | 0.4455          | 0.3939 |
| 1.4328        | 2.89  | 4500  | 0.4369          | 0.4069 |
| 1.3456        | 3.22  | 5000  | 0.4234          | 0.3774 |
| 1.3725        | 3.54  | 5500  | 0.4387          | 0.3789 |
| 1.3812        | 3.86  | 6000  | 0.4298          | 0.3825 |
| 1.3282        | 4.18  | 6500  | 0.4025          | 0.3703 |
| 1.3326        | 4.5   | 7000  | 0.3917          | 0.3502 |
| 1.3028        | 4.82  | 7500  | 0.3889          | 0.3582 |
| 1.293         | 5.14  | 8000  | 0.3859          | 0.3496 |
| 1.321         | 5.47  | 8500  | 0.3875          | 0.3576 |
| 1.3165        | 5.79  | 9000  | 0.3927          | 0.3589 |
| 1.2701        | 6.11  | 9500  | 0.4058          | 0.3621 |
| 1.2718        | 6.43  | 10000 | 0.4211          | 0.3916 |
| 1.2683        | 6.75  | 10500 | 0.3968          | 0.3620 |
| 1.2643        | 7.07  | 11000 | 0.4128          | 0.3848 |
| 1.2485        | 7.4   | 11500 | 0.3849          | 0.3727 |
| 1.2608        | 7.72  | 12000 | 0.3770          | 0.3474 |
| 1.2388        | 8.04  | 12500 | 0.3774          | 0.3574 |
| 1.2524        | 8.36  | 13000 | 0.3789          | 0.3550 |
| 1.2458        | 8.68  | 13500 | 0.3770          | 0.3410 |
| 1.2505        | 9.0   | 14000 | 0.3638          | 0.3403 |
| 1.2254        | 9.32  | 14500 | 0.3770          | 0.3509 |
| 1.2459        | 9.65  | 15000 | 0.3592          | 0.3349 |
| 1.2049        | 9.97  | 15500 | 0.3600          | 0.3428 |
| 1.2097        | 10.29 | 16000 | 0.3626          | 0.3347 |
| 1.1988        | 10.61 | 16500 | 0.3740          | 0.3269 |
| 1.1671        | 10.93 | 17000 | 0.3548          | 0.3245 |
| 1.1532        | 11.25 | 17500 | 0.3394          | 0.3140 |
| 1.1459        | 11.58 | 18000 | 0.3349          | 0.3156 |
| 1.1511        | 11.9  | 18500 | 0.3272          | 0.3110 |
| 1.1465        | 12.22 | 19000 | 0.3348          | 0.3084 |
| 1.1426        | 12.54 | 19500 | 0.3193          | 0.3027 |
| 1.1278        | 12.86 | 20000 | 0.3318          | 0.3021 |
| 1.149         | 13.18 | 20500 | 0.3169          | 0.2947 |
| 1.114         | 13.5  | 21000 | 0.3224          | 0.2986 |
| 1.1249        | 13.83 | 21500 | 0.3227          | 0.2921 |
| 1.0968        | 14.15 | 22000 | 0.3033          | 0.2878 |
| 1.0851        | 14.47 | 22500 | 0.2996          | 0.2863 |
| 1.0985        | 14.79 | 23000 | 0.3011          | 0.2843 |
| 1.0808        | 15.11 | 23500 | 0.2932          | 0.2759 |
| 1.069         | 15.43 | 24000 | 0.2919          | 0.2750 |
| 1.0602        | 15.76 | 24500 | 0.2959          | 0.2713 |
| 1.0369        | 16.08 | 25000 | 0.2931          | 0.2754 |
| 1.0573        | 16.4  | 25500 | 0.2920          | 0.2722 |
| 1.051         | 16.72 | 26000 | 0.2855          | 0.2632 |
| 1.0279        | 17.04 | 26500 | 0.2850          | 0.2649 |
| 1.0496        | 17.36 | 27000 | 0.2817          | 0.2585 |
| 1.0516        | 17.68 | 27500 | 0.2961          | 0.2635 |
| 1.0244        | 18.01 | 28000 | 0.2781          | 0.2589 |
| 1.0099        | 18.33 | 28500 | 0.2783          | 0.2565 |
| 1.0016        | 18.65 | 29000 | 0.2719          | 0.2537 |
| 1.0157        | 18.97 | 29500 | 0.2621          | 0.2449 |
| 0.9572        | 19.29 | 30000 | 0.2582          | 0.2427 |
| 0.9802        | 19.61 | 30500 | 0.2707          | 0.2468 |
| 0.9577        | 19.94 | 31000 | 0.2563          | 0.2389 |
| 0.9562        | 20.26 | 31500 | 0.2592          | 0.2382 |
| 0.962         | 20.58 | 32000 | 0.2539          | 0.2341 |
| 0.9541        | 20.9  | 32500 | 0.2505          | 0.2288 |
| 0.9587        | 21.22 | 33000 | 0.2486          | 0.2302 |
| 0.9146        | 21.54 | 33500 | 0.2461          | 0.2269 |
| 0.9215        | 21.86 | 34000 | 0.2387          | 0.2228 |
| 0.9105        | 22.19 | 34500 | 0.2405          | 0.2222 |
| 0.8949        | 22.51 | 35000 | 0.2316          | 0.2191 |
| 0.9153        | 22.83 | 35500 | 0.2358          | 0.2180 |
| 0.8907        | 23.15 | 36000 | 0.2369          | 0.2168 |
| 0.8973        | 23.47 | 36500 | 0.2323          | 0.2120 |
| 0.8878        | 23.79 | 37000 | 0.2293          | 0.2104 |
| 0.8818        | 24.12 | 37500 | 0.2302          | 0.2132 |
| 0.8919        | 24.44 | 38000 | 0.2262          | 0.2083 |
| 0.8473        | 24.76 | 38500 | 0.2257          | 0.2040 |
| 0.8516        | 25.08 | 39000 | 0.2246          | 0.2031 |
| 0.8451        | 25.4  | 39500 | 0.2198          | 0.2000 |
| 0.8288        | 25.72 | 40000 | 0.2199          | 0.1990 |
| 0.8465        | 26.05 | 40500 | 0.2165          | 0.1972 |
| 0.8305        | 26.37 | 41000 | 0.2128          | 0.1957 |
| 0.8202        | 26.69 | 41500 | 0.2127          | 0.1937 |
| 0.8223        | 27.01 | 42000 | 0.2100          | 0.1934 |
| 0.8322        | 27.33 | 42500 | 0.2076          | 0.1905 |
| 0.8139        | 27.65 | 43000 | 0.2054          | 0.1880 |
| 0.8299        | 27.97 | 43500 | 0.2026          | 0.1868 |
| 0.7937        | 28.3  | 44000 | 0.2045          | 0.1872 |
| 0.7972        | 28.62 | 44500 | 0.2025          | 0.1861 |
| 0.809         | 28.94 | 45000 | 0.2026          | 0.1858 |
| 0.813         | 29.26 | 45500 | 0.2013          | 0.1838 |
| 0.7718        | 29.58 | 46000 | 0.2010          | 0.1837 |
| 0.7929        | 29.9  | 46500 | 0.2008          | 0.1840 |

### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.3.dev0
- Tokenizers 0.11.0


---
language:
- gn
license: apache-2.0
tags:
- automatic-speech-recognition
- generated_from_trainer
- gn
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: wav2vec2-xls-r-300m-gn-cv8-3
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-gn-cv8-3

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9517
- Wer: 0.8542

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 5000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 19.9125       | 5.54   | 100  | 5.4279          | 1.0    |
| 3.8031        | 11.11  | 200  | 3.3070          | 1.0    |
| 3.3783        | 16.65  | 300  | 3.2450          | 1.0    |
| 3.3472        | 22.22  | 400  | 3.2424          | 1.0    |
| 3.2714        | 27.76  | 500  | 3.1100          | 1.0    |
| 3.2367        | 33.32  | 600  | 3.1091          | 1.0    |
| 3.1968        | 38.86  | 700  | 3.1013          | 1.0    |
| 3.2004        | 44.43  | 800  | 3.1173          | 1.0    |
| 3.1656        | 49.97  | 900  | 3.0682          | 1.0    |
| 3.1563        | 55.54  | 1000 | 3.0457          | 1.0    |
| 3.1356        | 61.11  | 1100 | 3.0139          | 1.0    |
| 3.086         | 66.65  | 1200 | 2.8108          | 1.0    |
| 2.954         | 72.22  | 1300 | 2.3238          | 1.0    |
| 2.6125        | 77.76  | 1400 | 1.6461          | 1.0    |
| 2.3296        | 83.32  | 1500 | 1.2834          | 0.9744 |
| 2.1345        | 88.86  | 1600 | 1.1091          | 0.9693 |
| 2.0346        | 94.43  | 1700 | 1.0273          | 0.9233 |
| 1.9611        | 99.97  | 1800 | 0.9642          | 0.9182 |
| 1.9066        | 105.54 | 1900 | 0.9590          | 0.9105 |
| 1.8178        | 111.11 | 2000 | 0.9679          | 0.9028 |
| 1.7799        | 116.65 | 2100 | 0.9007          | 0.8619 |
| 1.7726        | 122.22 | 2200 | 0.9689          | 0.8951 |
| 1.7389        | 127.76 | 2300 | 0.8876          | 0.8593 |
| 1.7151        | 133.32 | 2400 | 0.8716          | 0.8542 |
| 1.6842        | 138.86 | 2500 | 0.9536          | 0.8772 |
| 1.6449        | 144.43 | 2600 | 0.9296          | 0.8542 |
| 1.5978        | 149.97 | 2700 | 0.8895          | 0.8440 |
| 1.6515        | 155.54 | 2800 | 0.9162          | 0.8568 |
| 1.6586        | 161.11 | 2900 | 0.9039          | 0.8568 |
| 1.5966        | 166.65 | 3000 | 0.8627          | 0.8542 |
| 1.5695        | 172.22 | 3100 | 0.9549          | 0.8824 |
| 1.5699        | 177.76 | 3200 | 0.9332          | 0.8517 |
| 1.5297        | 183.32 | 3300 | 0.9163          | 0.8338 |
| 1.5367        | 188.86 | 3400 | 0.8822          | 0.8312 |
| 1.5586        | 194.43 | 3500 | 0.9217          | 0.8363 |
| 1.5429        | 199.97 | 3600 | 0.9564          | 0.8568 |
| 1.5273        | 205.54 | 3700 | 0.9508          | 0.8542 |
| 1.5043        | 211.11 | 3800 | 0.9374          | 0.8542 |
| 1.4724        | 216.65 | 3900 | 0.9622          | 0.8619 |
| 1.4794        | 222.22 | 4000 | 0.9550          | 0.8363 |
| 1.4843        | 227.76 | 4100 | 0.9577          | 0.8465 |
| 1.4781        | 233.32 | 4200 | 0.9543          | 0.8440 |
| 1.4507        | 238.86 | 4300 | 0.9553          | 0.8491 |
| 1.4997        | 244.43 | 4400 | 0.9728          | 0.8491 |
| 1.4371        | 249.97 | 4500 | 0.9543          | 0.8670 |
| 1.4825        | 255.54 | 4600 | 0.9636          | 0.8619 |
| 1.4187        | 261.11 | 4700 | 0.9609          | 0.8440 |
| 1.4363        | 266.65 | 4800 | 0.9567          | 0.8593 |
| 1.4463        | 272.22 | 4900 | 0.9581          | 0.8542 |
| 1.4117        | 277.76 | 5000 | 0.9517          | 0.8542 |

### Framework versions

- Transformers 4.16.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.11.0


---
language:
- pt
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: sew-tiny-portuguese-cv
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 6
      type: common_voice
      args: pt
    metrics:
       - name: Test WER
         type: wer
         value: 30.02
       - name: Test CER
         type: cer
         value: 10.34
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Robust Speech Event - Dev Data
      type: speech-recognition-community-v2/dev_data
      args: sv
    metrics:
       - name: Test WER
         type: wer
         value: 56.46
       - name: Test CER
         type: cer
         value: 22.94
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# sew-tiny-portuguese-cv

This model is a fine-tuned version of [lgris/sew-tiny-pt](https://huggingface.co/lgris/sew-tiny-pt) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5110
- Wer: 0.2842

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 40000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| No log        | 4.92   | 1000  | 0.8468          | 0.6494 |
| 3.4638        | 9.85   | 2000  | 0.4978          | 0.3815 |
| 3.4638        | 14.78  | 3000  | 0.4734          | 0.3417 |
| 0.9904        | 19.7   | 4000  | 0.4577          | 0.3344 |
| 0.9904        | 24.63  | 5000  | 0.4376          | 0.3170 |
| 0.8849        | 29.55  | 6000  | 0.4225          | 0.3118 |
| 0.8849        | 34.48  | 7000  | 0.4354          | 0.3080 |
| 0.819         | 39.41  | 8000  | 0.4434          | 0.3004 |
| 0.819         | 44.33  | 9000  | 0.4710          | 0.3132 |
| 0.7706        | 49.26  | 10000 | 0.4497          | 0.3064 |
| 0.7706        | 54.19  | 11000 | 0.4598          | 0.3100 |
| 0.7264        | 59.11  | 12000 | 0.4271          | 0.3013 |
| 0.7264        | 64.04  | 13000 | 0.4333          | 0.2959 |
| 0.6909        | 68.96  | 14000 | 0.4554          | 0.3019 |
| 0.6909        | 73.89  | 15000 | 0.4444          | 0.2888 |
| 0.6614        | 78.81  | 16000 | 0.4734          | 0.3081 |
| 0.6614        | 83.74  | 17000 | 0.4820          | 0.3058 |
| 0.6379        | 88.67  | 18000 | 0.4416          | 0.2950 |
| 0.6379        | 93.59  | 19000 | 0.4614          | 0.2974 |
| 0.6055        | 98.52  | 20000 | 0.4812          | 0.3018 |
| 0.6055        | 103.45 | 21000 | 0.4700          | 0.3018 |
| 0.5823        | 108.37 | 22000 | 0.4726          | 0.2999 |
| 0.5823        | 113.3  | 23000 | 0.4979          | 0.2887 |
| 0.5597        | 118.23 | 24000 | 0.4813          | 0.2980 |
| 0.5597        | 123.15 | 25000 | 0.4968          | 0.2972 |
| 0.542         | 128.08 | 26000 | 0.5331          | 0.3059 |
| 0.542         | 133.0  | 27000 | 0.5046          | 0.2978 |
| 0.5185        | 137.93 | 28000 | 0.4882          | 0.2922 |
| 0.5185        | 142.85 | 29000 | 0.4945          | 0.2938 |
| 0.499         | 147.78 | 30000 | 0.4971          | 0.2913 |
| 0.499         | 152.71 | 31000 | 0.4948          | 0.2873 |
| 0.4811        | 157.63 | 32000 | 0.4924          | 0.2918 |
| 0.4811        | 162.56 | 33000 | 0.5128          | 0.2911 |
| 0.4679        | 167.49 | 34000 | 0.5098          | 0.2892 |
| 0.4679        | 172.41 | 35000 | 0.4966          | 0.2863 |
| 0.456         | 177.34 | 36000 | 0.5033          | 0.2839 |
| 0.456         | 182.27 | 37000 | 0.5114          | 0.2875 |
| 0.4453        | 187.19 | 38000 | 0.5154          | 0.2859 |
| 0.4453        | 192.12 | 39000 | 0.5102          | 0.2847 |
| 0.4366        | 197.04 | 40000 | 0.5110          | 0.2842 |

### Framework versions

- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


---
language:
- pt
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
model-index:
- name: WavLM-large-CORAA-pt
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# WavLM-large-CORAA-pt

This model is a fine-tuned version of [microsoft/wavlm-large](https://huggingface.co/microsoft/wavlm-large) on [CORAA dataset](https://github.com/nilc-nlp/CORAA).
It achieves the following results on the evaluation set:
- Loss: 0.6144
- Wer: 0.3840

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 40000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| No log        | 0.04  | 1000  | 1.9230          | 0.9960 |
| 5.153         | 0.08  | 2000  | 1.3733          | 0.8444 |
| 5.153         | 0.13  | 3000  | 1.1992          | 0.7362 |
| 1.367         | 0.17  | 4000  | 1.1289          | 0.6957 |
| 1.367         | 0.21  | 5000  | 1.0357          | 0.6470 |
| 1.1824        | 0.25  | 6000  | 1.0216          | 0.6201 |
| 1.1824        | 0.29  | 7000  | 0.9338          | 0.6036 |
| 1.097         | 0.33  | 8000  | 0.9149          | 0.5760 |
| 1.097         | 0.38  | 9000  | 0.8885          | 0.5541 |
| 1.0254        | 0.42  | 10000 | 0.8678          | 0.5366 |
| 1.0254        | 0.46  | 11000 | 0.8349          | 0.5323 |
| 0.9782        | 0.5   | 12000 | 0.8230          | 0.5155 |
| 0.9782        | 0.54  | 13000 | 0.8245          | 0.5049 |
| 0.9448        | 0.59  | 14000 | 0.7802          | 0.4990 |
| 0.9448        | 0.63  | 15000 | 0.7650          | 0.4900 |
| 0.9092        | 0.67  | 16000 | 0.7665          | 0.4796 |
| 0.9092        | 0.71  | 17000 | 0.7568          | 0.4795 |
| 0.8764        | 0.75  | 18000 | 0.7403          | 0.4615 |
| 0.8764        | 0.8   | 19000 | 0.7219          | 0.4644 |
| 0.8498        | 0.84  | 20000 | 0.7180          | 0.4502 |
| 0.8498        | 0.88  | 21000 | 0.7017          | 0.4436 |
| 0.8278        | 0.92  | 22000 | 0.6992          | 0.4395 |
| 0.8278        | 0.96  | 23000 | 0.7021          | 0.4329 |
| 0.8077        | 1.0   | 24000 | 0.6892          | 0.4265 |
| 0.8077        | 1.05  | 25000 | 0.6940          | 0.4248 |
| 0.7486        | 1.09  | 26000 | 0.6767          | 0.4202 |
| 0.7486        | 1.13  | 27000 | 0.6734          | 0.4150 |
| 0.7459        | 1.17  | 28000 | 0.6650          | 0.4152 |
| 0.7459        | 1.21  | 29000 | 0.6559          | 0.4078 |
| 0.7304        | 1.26  | 30000 | 0.6536          | 0.4088 |
| 0.7304        | 1.3   | 31000 | 0.6537          | 0.4025 |
| 0.7183        | 1.34  | 32000 | 0.6462          | 0.4008 |
| 0.7183        | 1.38  | 33000 | 0.6381          | 0.3973 |
| 0.7059        | 1.42  | 34000 | 0.6266          | 0.3930 |
| 0.7059        | 1.46  | 35000 | 0.6280          | 0.3921 |
| 0.6983        | 1.51  | 36000 | 0.6248          | 0.3897 |
| 0.6983        | 1.55  | 37000 | 0.6275          | 0.3872 |
| 0.6892        | 1.59  | 38000 | 0.6199          | 0.3852 |
| 0.6892        | 1.63  | 39000 | 0.6180          | 0.3842 |
| 0.691         | 1.67  | 40000 | 0.6144          | 0.3840 |

### Framework versions

- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


---
language:
- gn
license: apache-2.0
tags:
- automatic-speech-recognition
- generated_from_trainer
- gn
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
model-index:
- name: wav2vec2-xls-r-gn-cv7
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 7
      type: mozilla-foundation/common_voice_7_0
      args: pt
    metrics:
       - name: Validation WER
         type: wer
         value: 73.02
       - name: Validation CER
         type: cer
         value: 17.79
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-gn-cv7

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7197
- Wer: 0.7434

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 13000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| 3.4669        | 6.24   | 100   | 3.3003          | 1.0    |
| 3.3214        | 12.48  | 200   | 3.2090          | 1.0    |
| 3.1619        | 18.73  | 300   | 2.6322          | 1.0    |
| 1.751         | 24.97  | 400   | 1.4089          | 0.9803 |
| 0.7997        | 31.24  | 500   | 0.9996          | 0.9211 |
| 0.4996        | 37.48  | 600   | 0.9879          | 0.8553 |
| 0.3677        | 43.73  | 700   | 0.9543          | 0.8289 |
| 0.2851        | 49.97  | 800   | 1.0627          | 0.8487 |
| 0.2556        | 56.24  | 900   | 1.0933          | 0.8355 |
| 0.2268        | 62.48  | 1000  | 0.9191          | 0.8026 |
| 0.1914        | 68.73  | 1100  | 0.9582          | 0.7961 |
| 0.1749        | 74.97  | 1200  | 1.0502          | 0.8092 |
| 0.157         | 81.24  | 1300  | 0.9998          | 0.7632 |
| 0.1505        | 87.48  | 1400  | 1.0076          | 0.7303 |
| 0.1278        | 93.73  | 1500  | 0.9321          | 0.75   |
| 0.1078        | 99.97  | 1600  | 1.0383          | 0.7697 |
| 0.1156        | 106.24 | 1700  | 1.0302          | 0.7763 |
| 0.1107        | 112.48 | 1800  | 1.0419          | 0.7763 |
| 0.091         | 118.73 | 1900  | 1.0694          | 0.75   |
| 0.0829        | 124.97 | 2000  | 1.0257          | 0.7829 |
| 0.0865        | 131.24 | 2100  | 1.2108          | 0.7368 |
| 0.0907        | 137.48 | 2200  | 1.0458          | 0.7697 |
| 0.0897        | 143.73 | 2300  | 1.1504          | 0.7895 |
| 0.0766        | 149.97 | 2400  | 1.1663          | 0.7237 |
| 0.0659        | 156.24 | 2500  | 1.1320          | 0.7632 |
| 0.0699        | 162.48 | 2600  | 1.2586          | 0.7434 |
| 0.0613        | 168.73 | 2700  | 1.1815          | 0.8158 |
| 0.0598        | 174.97 | 2800  | 1.3299          | 0.75   |
| 0.0577        | 181.24 | 2900  | 1.2035          | 0.7171 |
| 0.0576        | 187.48 | 3000  | 1.2134          | 0.7434 |
| 0.0518        | 193.73 | 3100  | 1.3406          | 0.7566 |
| 0.0524        | 199.97 | 3200  | 1.4251          | 0.75   |
| 0.0467        | 206.24 | 3300  | 1.3533          | 0.7697 |
| 0.0428        | 212.48 | 3400  | 1.2463          | 0.7368 |
| 0.0453        | 218.73 | 3500  | 1.4532          | 0.7566 |
| 0.0473        | 224.97 | 3600  | 1.3152          | 0.7434 |
| 0.0451        | 231.24 | 3700  | 1.2232          | 0.7368 |
| 0.0361        | 237.48 | 3800  | 1.2938          | 0.7171 |
| 0.045         | 243.73 | 3900  | 1.4148          | 0.7434 |
| 0.0422        | 249.97 | 4000  | 1.3786          | 0.7961 |
| 0.036         | 256.24 | 4100  | 1.4488          | 0.7697 |
| 0.0352        | 262.48 | 4200  | 1.2294          | 0.6776 |
| 0.0326        | 268.73 | 4300  | 1.2796          | 0.6974 |
| 0.034         | 274.97 | 4400  | 1.3805          | 0.7303 |
| 0.0305        | 281.24 | 4500  | 1.4994          | 0.7237 |
| 0.0325        | 287.48 | 4600  | 1.4330          | 0.6908 |
| 0.0338        | 293.73 | 4700  | 1.3091          | 0.7368 |
| 0.0306        | 299.97 | 4800  | 1.2174          | 0.7171 |
| 0.0299        | 306.24 | 4900  | 1.3527          | 0.7763 |
| 0.0287        | 312.48 | 5000  | 1.3651          | 0.7368 |
| 0.0274        | 318.73 | 5100  | 1.4337          | 0.7368 |
| 0.0258        | 324.97 | 5200  | 1.3831          | 0.6908 |
| 0.022         | 331.24 | 5300  | 1.3556          | 0.6974 |
| 0.021         | 337.48 | 5400  | 1.3836          | 0.7237 |
| 0.0241        | 343.73 | 5500  | 1.4352          | 0.7039 |
| 0.0229        | 349.97 | 5600  | 1.3904          | 0.7105 |
| 0.026         | 356.24 | 5700  | 1.4131          | 0.7171 |
| 0.021         | 362.48 | 5800  | 1.5426          | 0.6974 |
| 0.0191        | 368.73 | 5900  | 1.5960          | 0.7632 |
| 0.0227        | 374.97 | 6000  | 1.6240          | 0.7368 |
| 0.0204        | 381.24 | 6100  | 1.4301          | 0.7105 |
| 0.0175        | 387.48 | 6200  | 1.5554          | 0.75   |
| 0.0183        | 393.73 | 6300  | 1.6044          | 0.7697 |
| 0.0183        | 399.97 | 6400  | 1.5963          | 0.7368 |
| 0.016         | 406.24 | 6500  | 1.5679          | 0.7829 |
| 0.0178        | 412.48 | 6600  | 1.5928          | 0.7697 |
| 0.014         | 418.73 | 6700  | 1.7000          | 0.7632 |
| 0.0182        | 424.97 | 6800  | 1.5340          | 0.75   |
| 0.0148        | 431.24 | 6900  | 1.9274          | 0.7368 |
| 0.0148        | 437.48 | 7000  | 1.6437          | 0.7697 |
| 0.0173        | 443.73 | 7100  | 1.5468          | 0.75   |
| 0.0109        | 449.97 | 7200  | 1.6083          | 0.75   |
| 0.0167        | 456.24 | 7300  | 1.6732          | 0.75   |
| 0.0139        | 462.48 | 7400  | 1.5097          | 0.7237 |
| 0.013         | 468.73 | 7500  | 1.5947          | 0.7171 |
| 0.0128        | 474.97 | 7600  | 1.6260          | 0.7105 |
| 0.0166        | 481.24 | 7700  | 1.5756          | 0.7237 |
| 0.0127        | 487.48 | 7800  | 1.4506          | 0.6908 |
| 0.013         | 493.73 | 7900  | 1.4882          | 0.7368 |
| 0.0125        | 499.97 | 8000  | 1.5589          | 0.7829 |
| 0.0141        | 506.24 | 8100  | 1.6328          | 0.7434 |
| 0.0115        | 512.48 | 8200  | 1.6586          | 0.7434 |
| 0.0117        | 518.73 | 8300  | 1.6043          | 0.7105 |
| 0.009         | 524.97 | 8400  | 1.6508          | 0.7237 |
| 0.0108        | 531.24 | 8500  | 1.4507          | 0.6974 |
| 0.011         | 537.48 | 8600  | 1.5942          | 0.7434 |
| 0.009         | 543.73 | 8700  | 1.8121          | 0.7697 |
| 0.0112        | 549.97 | 8800  | 1.6923          | 0.7697 |
| 0.0073        | 556.24 | 8900  | 1.7096          | 0.7368 |
| 0.0098        | 562.48 | 9000  | 1.7052          | 0.7829 |
| 0.0088        | 568.73 | 9100  | 1.6956          | 0.7566 |
| 0.0099        | 574.97 | 9200  | 1.4909          | 0.7171 |
| 0.0075        | 581.24 | 9300  | 1.6307          | 0.7697 |
| 0.0077        | 587.48 | 9400  | 1.6196          | 0.7961 |
| 0.0088        | 593.73 | 9500  | 1.6119          | 0.7566 |
| 0.0085        | 599.97 | 9600  | 1.4512          | 0.7368 |
| 0.0086        | 606.24 | 9700  | 1.5992          | 0.7237 |
| 0.0109        | 612.48 | 9800  | 1.4706          | 0.7368 |
| 0.0098        | 618.73 | 9900  | 1.3824          | 0.7171 |
| 0.0091        | 624.97 | 10000 | 1.4776          | 0.6974 |
| 0.0072        | 631.24 | 10100 | 1.4896          | 0.7039 |
| 0.0087        | 637.48 | 10200 | 1.5467          | 0.7368 |
| 0.007         | 643.73 | 10300 | 1.5493          | 0.75   |
| 0.0076        | 649.97 | 10400 | 1.5706          | 0.7303 |
| 0.0085        | 656.24 | 10500 | 1.5748          | 0.7237 |
| 0.0075        | 662.48 | 10600 | 1.5081          | 0.7105 |
| 0.0068        | 668.73 | 10700 | 1.4967          | 0.6842 |
| 0.0117        | 674.97 | 10800 | 1.4986          | 0.7105 |
| 0.0054        | 681.24 | 10900 | 1.5587          | 0.7303 |
| 0.0059        | 687.48 | 11000 | 1.5886          | 0.7171 |
| 0.0071        | 693.73 | 11100 | 1.5746          | 0.7171 |
| 0.0048        | 699.97 | 11200 | 1.6166          | 0.7237 |
| 0.0048        | 706.24 | 11300 | 1.6098          | 0.7237 |
| 0.0056        | 712.48 | 11400 | 1.5834          | 0.7237 |
| 0.0048        | 718.73 | 11500 | 1.5653          | 0.7171 |
| 0.0045        | 724.97 | 11600 | 1.6252          | 0.7237 |
| 0.0068        | 731.24 | 11700 | 1.6794          | 0.7171 |
| 0.0044        | 737.48 | 11800 | 1.6881          | 0.7039 |
| 0.008         | 743.73 | 11900 | 1.7393          | 0.75   |
| 0.0045        | 749.97 | 12000 | 1.6869          | 0.7237 |
| 0.0047        | 756.24 | 12100 | 1.7105          | 0.7303 |
| 0.0057        | 762.48 | 12200 | 1.7439          | 0.7303 |
| 0.004         | 768.73 | 12300 | 1.7871          | 0.7434 |
| 0.0061        | 774.97 | 12400 | 1.7812          | 0.7303 |
| 0.005         | 781.24 | 12500 | 1.7410          | 0.7434 |
| 0.0056        | 787.48 | 12600 | 1.7220          | 0.7303 |
| 0.0064        | 793.73 | 12700 | 1.7141          | 0.7434 |
| 0.0042        | 799.97 | 12800 | 1.7139          | 0.7368 |
| 0.0049        | 806.24 | 12900 | 1.7211          | 0.7434 |
| 0.0044        | 812.48 | 13000 | 1.7197          | 0.7434 |

### Framework versions

- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


---
language: pt
datasets:
- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0
---

# bp500-xlsr: Wav2vec 2.0 with Brazilian Portuguese (BP) Dataset

This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the following datasets:

- [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz): contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus;
- [Common Voice 7.0](https://commonvoice.mozilla.org/pt):  is a project proposed by Mozilla Foundation with the goal to create a wide open dataset in different languages. In this project, volunteers donate and validate speech using the [oficial site](https://commonvoice.mozilla.org/pt);
- [Lapsbm](https://github.com/falabrasil/gitlab-resources): "Falabrasil - UFPA" is a dataset used by the Fala Brasil group to benchmark ASR systems in Brazilian Portuguese. Contains 35 speakers (10 females), each one pronouncing 20 unique sentences, totalling  700 utterances in Brazilian Portuguese. The audios were recorded in 22.05 kHz without environment control;
- [Multilingual Librispeech (MLS)](https://arxiv.org/abs/2012.03411): a massive dataset available in many languages. The MLS is based on audiobook recordings in public domain like [LibriVox](https://librivox.org/). The dataset contains a total of 6k hours of transcribed data in many languages. The set in Portuguese [used in this work](http://www.openslr.org/94/) (mostly Brazilian variant) has approximately 284 hours of speech, obtained from 55 audiobooks read by 62 speakers;
- [VoxForge](http://www.voxforge.org/): is a project with the goal to build open datasets for acoustic models. The corpus contains approximately 100 speakers and 4,130 utterances of Brazilian Portuguese, with sample rates varying from 16kHz to 44.1kHz.

These datasets were combined to build a larger Brazilian Portuguese dataset. All data was used for training except Common Voice dev/test sets, that were used for validation/test respectively. We also made test sets for all the gathered datasets.

| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          |  93.9h |    -- |  5.4h |
| Common Voice                   |  37.6h |  8.9h |  9.5h |
| LaPS BM                        |   0.8h |    -- |  0.1h |
| MLS                            | 161.0h |    -- |  3.7h |
| Multilingual TEDx (Portuguese) | 144.2h |    -- |  1.8h |
| SID                            |   5.0h |    -- |  1.0h |
| VoxForge                       |   2.8h |    -- |  0.1h |
| Total                          | 437.2h |  8.9h | 21.6h |

The original model was fine-tuned using [fairseq](https://github.com/pytorch/fairseq). This notebook uses a converted version of the original one. The link to the original fairseq model is available [here](https://drive.google.com/file/d/1J8aR1ltDLQFe-dVrGuyxoRm2uyJjCWgf/view?usp=sharing).

#### Summary

|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| bp\_500 (demonstration below)              | 0.051 | 0.136 | 0.032 | 0.118 | 0.095 | 0.248 | 0.082 | 0.108 |
| bp\_500 + 4-gram (demonstration below)       | 0.032 | 0.097 | 0.022 | 0.114 | 0.125 | 0.246 | 0.065 | 0.100 |

#### Transcription examples

| Text                                                                                | Transcription                                                                                |
|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|
|não há um departamento de mediadores independente das federações e das agremiações|não há um **dearamento** de mediadores independente das federações e das **agrebiações**|
|mas que bodega|**masque** bodega|
|a cortina abriu o show começou|a cortina abriu o **chô** começou|
|por sorte havia uma passadeira|**busote avinhoa** **passadeiro**|
|estou maravilhada está tudo pronto|**stou** estou maravilhada está tudo pronto|

## Demonstration

```python
MODEL_NAME = "lgris/bp500-xlsr"
```

### Imports and dependencies

```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```

```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```

### Helpers

```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```

```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```

```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```

### Model

```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        attention_mask = features.attention_mask.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values, attention_mask=attention_mask).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```

### Download datasets

```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```

```python
%cd bp_dataset
```

    /content/bp_dataset

### Tests

```python
stt = STT(MODEL_NAME)
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.05159097808687998

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.13659981509705973

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.03196969696969697

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.1178481066463896

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.09544588416964224

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.24868046340420813

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.08246076839826841

### Tests with LM

```python
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```

### Cetuc

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.03222801788375573

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.09713866021093655

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.022310606060606065

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.11408590958696524

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.12502797252979136

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.24603179403904793

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.06542207792207791


---
language:
- gn
license: apache-2.0
tags:
- automatic-speech-recognition
- generated_from_trainer
- gn
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: wav2vec2-xls-r-300m-gn-cv8-4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-gn-cv8-4

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5805
- Wer: 0.7545

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 13000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| 9.2216        | 16.65  | 300   | 3.2771          | 1.0    |
| 3.1804        | 33.32  | 600   | 2.2869          | 1.0    |
| 1.5856        | 49.97  | 900   | 0.9573          | 0.8772 |
| 1.0299        | 66.65  | 1200  | 0.9044          | 0.8082 |
| 0.8916        | 83.32  | 1500  | 0.9478          | 0.8056 |
| 0.8451        | 99.97  | 1800  | 0.8814          | 0.8107 |
| 0.7649        | 116.65 | 2100  | 0.9897          | 0.7826 |
| 0.7185        | 133.32 | 2400  | 0.9988          | 0.7621 |
| 0.6595        | 149.97 | 2700  | 1.0607          | 0.7749 |
| 0.6211        | 166.65 | 3000  | 1.1826          | 0.7877 |
| 0.59          | 183.32 | 3300  | 1.1060          | 0.7826 |
| 0.5383        | 199.97 | 3600  | 1.1826          | 0.7852 |
| 0.5205        | 216.65 | 3900  | 1.2148          | 0.8261 |
| 0.4786        | 233.32 | 4200  | 1.2710          | 0.7928 |
| 0.4482        | 249.97 | 4500  | 1.1943          | 0.7980 |
| 0.4149        | 266.65 | 4800  | 1.2449          | 0.8031 |
| 0.3904        | 283.32 | 5100  | 1.3100          | 0.7928 |
| 0.3619        | 299.97 | 5400  | 1.3125          | 0.7596 |
| 0.3496        | 316.65 | 5700  | 1.3699          | 0.7877 |
| 0.3277        | 333.32 | 6000  | 1.4344          | 0.8031 |
| 0.2958        | 349.97 | 6300  | 1.4093          | 0.7980 |
| 0.2883        | 366.65 | 6600  | 1.3296          | 0.7570 |
| 0.2598        | 383.32 | 6900  | 1.4026          | 0.7980 |
| 0.2564        | 399.97 | 7200  | 1.4847          | 0.8031 |
| 0.2408        | 416.65 | 7500  | 1.4896          | 0.8107 |
| 0.2266        | 433.32 | 7800  | 1.4232          | 0.7698 |
| 0.224         | 449.97 | 8100  | 1.5560          | 0.7903 |
| 0.2038        | 466.65 | 8400  | 1.5355          | 0.7724 |
| 0.1948        | 483.32 | 8700  | 1.4624          | 0.7621 |
| 0.1995        | 499.97 | 9000  | 1.5808          | 0.7724 |
| 0.1864        | 516.65 | 9300  | 1.5653          | 0.7698 |
| 0.18          | 533.32 | 9600  | 1.4868          | 0.7494 |
| 0.1689        | 549.97 | 9900  | 1.5379          | 0.7749 |
| 0.1624        | 566.65 | 10200 | 1.5936          | 0.7749 |
| 0.1537        | 583.32 | 10500 | 1.6436          | 0.7801 |
| 0.1455        | 599.97 | 10800 | 1.6401          | 0.7673 |
| 0.1437        | 616.65 | 11100 | 1.6069          | 0.7673 |
| 0.1452        | 633.32 | 11400 | 1.6041          | 0.7519 |
| 0.139         | 649.97 | 11700 | 1.5758          | 0.7545 |
| 0.1299        | 666.65 | 12000 | 1.5559          | 0.7545 |
| 0.127         | 683.32 | 12300 | 1.5776          | 0.7596 |
| 0.1264        | 699.97 | 12600 | 1.5790          | 0.7519 |
| 0.1209        | 716.65 | 12900 | 1.5805          | 0.7545 |

### Framework versions

- Transformers 4.16.1
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0


---
language:
- pt
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: sew-tiny-portuguese-cv8
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 8
      type: mozilla-foundation/common_voice_8_0
      args: pt
    metrics:
       - name: Test WER
         type: wer
         value: 33.71
       - name: Test CER
         type: cer
         value: 10.69
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Robust Speech Event - Dev Data
      type: speech-recognition-community-v2/dev_data
      args: sv
    metrics:
       - name: Test WER
         type: wer
         value: 52.79
       - name: Test CER
         type: cer
         value: 20.98
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# sew-tiny-portuguese-cv8

This model is a fine-tuned version of [lgris/sew-tiny-pt](https://huggingface.co/lgris/sew-tiny-pt) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4082
- Wer: 0.3053

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 40000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| No log        | 1.93  | 1000  | 2.9134          | 0.9767 |
| 2.9224        | 3.86  | 2000  | 2.8405          | 0.9789 |
| 2.9224        | 5.79  | 3000  | 2.8094          | 0.9800 |
| 2.8531        | 7.72  | 4000  | 2.7439          | 0.9891 |
| 2.8531        | 9.65  | 5000  | 2.7057          | 1.0159 |
| 2.7721        | 11.58 | 6000  | 2.7235          | 1.0709 |
| 2.7721        | 13.51 | 7000  | 2.5931          | 1.1035 |
| 2.6566        | 15.44 | 8000  | 2.2171          | 0.9884 |
| 2.6566        | 17.37 | 9000  | 1.2399          | 0.8081 |
| 1.9558        | 19.31 | 10000 | 0.9045          | 0.6353 |
| 1.9558        | 21.24 | 11000 | 0.7705          | 0.5533 |
| 1.4987        | 23.17 | 12000 | 0.7068          | 0.5165 |
| 1.4987        | 25.1  | 13000 | 0.6641          | 0.4718 |
| 1.3811        | 27.03 | 14000 | 0.6043          | 0.4470 |
| 1.3811        | 28.96 | 15000 | 0.5532          | 0.4268 |
| 1.2897        | 30.89 | 16000 | 0.5371          | 0.4101 |
| 1.2897        | 32.82 | 17000 | 0.5924          | 0.4150 |
| 1.225         | 34.75 | 18000 | 0.4949          | 0.3894 |
| 1.225         | 36.68 | 19000 | 0.5591          | 0.4045 |
| 1.193         | 38.61 | 20000 | 0.4927          | 0.3731 |
| 1.193         | 40.54 | 21000 | 0.4922          | 0.3712 |
| 1.1482        | 42.47 | 22000 | 0.4799          | 0.3662 |
| 1.1482        | 44.4  | 23000 | 0.4846          | 0.3648 |
| 1.1201        | 46.33 | 24000 | 0.4770          | 0.3623 |
| 1.1201        | 48.26 | 25000 | 0.4530          | 0.3426 |
| 1.0892        | 50.19 | 26000 | 0.4523          | 0.3527 |
| 1.0892        | 52.12 | 27000 | 0.4573          | 0.3443 |
| 1.0583        | 54.05 | 28000 | 0.4488          | 0.3353 |
| 1.0583        | 55.98 | 29000 | 0.4295          | 0.3285 |
| 1.0319        | 57.92 | 30000 | 0.4321          | 0.3220 |
| 1.0319        | 59.85 | 31000 | 0.4244          | 0.3236 |
| 1.0076        | 61.78 | 32000 | 0.4197          | 0.3201 |
| 1.0076        | 63.71 | 33000 | 0.4230          | 0.3208 |
| 0.9851        | 65.64 | 34000 | 0.4090          | 0.3127 |
| 0.9851        | 67.57 | 35000 | 0.4088          | 0.3133 |
| 0.9695        | 69.5  | 36000 | 0.4123          | 0.3088 |
| 0.9695        | 71.43 | 37000 | 0.4017          | 0.3090 |
| 0.9514        | 73.36 | 38000 | 0.4184          | 0.3086 |
| 0.9514        | 75.29 | 39000 | 0.4075          | 0.3043 |
| 0.944         | 77.22 | 40000 | 0.4082          | 0.3053 |

### Framework versions

- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


---
language: pt
datasets:
- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0
---

# lapsbm1-xlsr: Wav2vec 2.0 with LaPSBM Dataset

This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the [LaPS BM](https://github.com/falabrasil/gitlab-resources) dataset.

In this notebook the model is tested against other available Brazilian Portuguese datasets.

| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          |        |    -- |  5.4h |
| Common Voice                   |        |    -- |  9.5h |
| LaPS BM                        |  0.8h  |    -- |  0.1h |
| MLS                            |        |    -- |  3.7h |
| Multilingual TEDx (Portuguese) |        |    -- |  1.8h |
| SID                            |        |    -- |  1.0h |
| VoxForge                       |        |    -- |  0.1h |
| Total                          |        |    -- | 21.6h |

#### Summary

|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| lapsbm1\_100 (demonstration below) | 0.111 | 0.418 | 0.145 | 0.299 | 0.562 | 0.580 | 0.469 | 0.369 |
| lapsbm1\_100 + 4-gram (demonstration below) | 0.061 | 0.305 | 0.089 | 0.201 | 0.452 | 0.525 | 0.381 | 0.287 |

## Demonstration

```python
MODEL_NAME = "lgris/lapsbm1-xlsr"
```

### Imports and dependencies

```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```

```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```

### Helpers

```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```

```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```

```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```

### Model

```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        attention_mask = features.attention_mask.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values, attention_mask=attention_mask).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```

### Download datasets

```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```

### Tests

```python
stt = STT(MODEL_NAME)
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.11147816967489037

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.41880890234535906

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.1451893939393939

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.29958960206171104

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.5626767414610376

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.5807549973642049

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.4693479437229436

### Tests with LM

```python
# !find -type f -name "*.wav" -delete
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```

#### CETUC

```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```
    CETUC WER: 0.06157628194513477

#### Common Voice

```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```
    CV WER: 0.3051714756833442

#### LaPS

```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```
    Laps WER: 0.0893623737373737

#### MLS

```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```
    MLS WER: 0.20062044237806004

#### SID

```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```
    Sid WER: 0.4522665618175908

#### TEDx

```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```
    TEDx WER: 0.5256707813182246

#### VoxForge

```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```
    VoxForge WER: 0.38106331168831165


---
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
model-index:
- name: wav2vec2-large-xlsr-coraa-portuguese-cv7
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xlsr-coraa-portuguese-cv7

This model is a fine-tuned version of [Edresson/wav2vec2-large-xlsr-coraa-portuguese](https://huggingface.co/Edresson/wav2vec2-large-xlsr-coraa-portuguese) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1777
- Wer: 0.1339

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 5000

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.4779        | 0.13  | 100  | 0.2620          | 0.2020 |
| 0.4505        | 0.26  | 200  | 0.2339          | 0.1998 |
| 0.4285        | 0.39  | 300  | 0.2507          | 0.2109 |
| 0.4148        | 0.52  | 400  | 0.2311          | 0.2101 |
| 0.4072        | 0.65  | 500  | 0.2278          | 0.1899 |
| 0.388         | 0.78  | 600  | 0.2193          | 0.1898 |
| 0.3952        | 0.91  | 700  | 0.2108          | 0.1901 |
| 0.3851        | 1.04  | 800  | 0.2121          | 0.1788 |
| 0.3496        | 1.17  | 900  | 0.2154          | 0.1776 |
| 0.3063        | 1.3   | 1000 | 0.2095          | 0.1730 |
| 0.3376        | 1.43  | 1100 | 0.2129          | 0.1801 |
| 0.3273        | 1.56  | 1200 | 0.2132          | 0.1776 |
| 0.3347        | 1.69  | 1300 | 0.2054          | 0.1698 |
| 0.323         | 1.82  | 1400 | 0.1986          | 0.1724 |
| 0.3079        | 1.95  | 1500 | 0.2005          | 0.1701 |
| 0.3029        | 2.08  | 1600 | 0.2159          | 0.1644 |
| 0.2694        | 2.21  | 1700 | 0.1992          | 0.1678 |
| 0.2733        | 2.34  | 1800 | 0.2032          | 0.1657 |
| 0.269         | 2.47  | 1900 | 0.2056          | 0.1592 |
| 0.2869        | 2.6   | 2000 | 0.2058          | 0.1616 |
| 0.2813        | 2.73  | 2100 | 0.1868          | 0.1584 |
| 0.2616        | 2.86  | 2200 | 0.1841          | 0.1550 |
| 0.2809        | 2.99  | 2300 | 0.1902          | 0.1577 |
| 0.2598        | 3.12  | 2400 | 0.1910          | 0.1514 |
| 0.24          | 3.25  | 2500 | 0.1971          | 0.1555 |
| 0.2481        | 3.38  | 2600 | 0.1853          | 0.1537 |
| 0.2437        | 3.51  | 2700 | 0.1897          | 0.1496 |
| 0.2384        | 3.64  | 2800 | 0.1842          | 0.1495 |
| 0.2405        | 3.77  | 2900 | 0.1884          | 0.1500 |
| 0.2372        | 3.9   | 3000 | 0.1950          | 0.1548 |
| 0.229         | 4.03  | 3100 | 0.1928          | 0.1477 |
| 0.2047        | 4.16  | 3200 | 0.1891          | 0.1472 |
| 0.2102        | 4.29  | 3300 | 0.1930          | 0.1473 |
| 0.199         | 4.42  | 3400 | 0.1914          | 0.1456 |
| 0.2121        | 4.55  | 3500 | 0.1840          | 0.1437 |
| 0.211         | 4.67  | 3600 | 0.1843          | 0.1403 |
| 0.2072        | 4.8   | 3700 | 0.1836          | 0.1428 |
| 0.2224        | 4.93  | 3800 | 0.1747          | 0.1412 |
| 0.1974        | 5.06  | 3900 | 0.1813          | 0.1416 |
| 0.1895        | 5.19  | 4000 | 0.1869          | 0.1406 |
| 0.1763        | 5.32  | 4100 | 0.1830          | 0.1394 |
| 0.2001        | 5.45  | 4200 | 0.1775          | 0.1394 |
| 0.1909        | 5.58  | 4300 | 0.1806          | 0.1373 |
| 0.1812        | 5.71  | 4400 | 0.1784          | 0.1359 |
| 0.1737        | 5.84  | 4500 | 0.1778          | 0.1353 |
| 0.1915        | 5.97  | 4600 | 0.1777          | 0.1349 |
| 0.1921        | 6.1   | 4700 | 0.1784          | 0.1359 |
| 0.1805        | 6.23  | 4800 | 0.1757          | 0.1348 |
| 0.1742        | 6.36  | 4900 | 0.1771          | 0.1341 |
| 0.1709        | 6.49  | 5000 | 0.1777          | 0.1339 |

### Framework versions

- Transformers 4.16.1
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0