```
text = """
Latvijos krepšinio legenda Valdis Valteris pirmadienį socialiniame tinkle pasidalino statistika, kurios viršūnėje yra Arvydas Sabonis.
1982 metais TSRS rinktinėje debiutavęs 222 cm ūgio vidurio puolėjas su raudona apranga sužaidė 52 rungtynes, per kurias rinko po 15,6 taško. Tai pats aukščiausias rezultatyvumo vidurkis tarp visų sovietų komandai atstovavusių žaidėjų, skaičiuojant tuos, kurie sužaidė ne mažiau nei 50 rungtynių. Antras šioje rikiuotėje kitas buvęs Kauno „Žalgirio“ krepšininkas Rimas Kurtinaitis. Jis debiutavo TSRS rinktinėje vėliau nei Sabas, – 1984 metais, bet irgi sužaidė 52 mačus. R.Kurtinaitis pelnė po 15 taškų. 25-ių rezultatyviausių žaidėjų sąrašu pasidalinęs latvis V.Valteris, pelnęs po 13,8 taško, yra trečias.
Ketvirtas yra iš Kazachstano kilęs Valerijus Tichonenka, pelnęs po 13,7 taško per 79 rungtynes. Rezultatyviausią visų laikų TSRS rinktinės penketą uždaro Modestas Paulauskas. Lietuvos krepšinio legenda pelnė po 13,6 taško per 84 mačus.
Dešimtuke taip pat yra Oleksandras Volkovas (po 13,5 taško), Sergejus Belovas (12,7), Anatolijus Myškinas (po 12,3), Vladimiras Tkačenka (11,7) ir Aleksandras Salnikovas (11,4). Dvyliktas šiame sąraše yra Valdemaras Chomičius, vidutiniškai rinkęs po 10 taškų, o keturioliktas dar vienas buvęs žalgirietis Sergejus Jovaiša (po 9,8 taško). Šarūno Marčiulionio rezultatyvumo vidurkis turėjo būti aukštesnis, bet jis sužaidė mažiau nei 50 rungtynių. Kaip žinia, Lietuvai išsilaisvinus ir atkūrus Nepriklausomybę, visi minėti mūsų šalies krepšininkai, išskyrus karjerą jau baigusį M.Paulauską, užsivilko žalią aprangą ir atstovavo savo tėvynei.
A.Sabonis pagal rezultatyvumo vidurkį yra pirmas – jis Lietuvos rinktinei pelnė po 20 taškų. Antras pagal taškų vidurkį yra Artūras Karnišovas, rinkęs po 18,2 taško ir pelnęs iš viso daugiausiai taškų atstovaujant Lietuvos rinktinei (1453).
Tarp žaidėjų, kurie sužaidė bent po 50 oficialių rungtynių Lietuvos rinktinėje, trečią vietą užima Ramūnas Šiškauskas (po 12,9), ketvirtąją Linas Kleiza (po 12,7 taško), o penktas – Saulius Štombergas (po 11,1 taško). Daugiausiai rungtynių Lietuvos rinktinėje sužaidęs ir daugiausiai olimpinių medalių (3) su ja laimėjęs Gintaras Einikis rinko po 9,6 taško, o pirmajame trejete pagal rungtynių skaičių ir pelnytus taškus esantis Šarūnas Jasikevičius pelnė po 9,9 taško.
"""
text = ' '.join(text.strip().split())
```


The summary can be obtained by:


```python
my_pipeline(text)[0]["generated_text"]
```


Output from above would be:


Lietuvos krepšinio federacijos (LKF) prezidento Arvydo Sabonio rezultatyvumo vidurkis yra aukščiausias tarp visų Sovietų Sąjungos rinktinėje atstovavusių žaidėjų, skaičiuojant tuos, kurie sužaidė bent po 50 oficialių rungtynių.


### BibTeX entry and citation info


```bibtex
@misc{stankevičius2021generating,
      title={Generating abstractive summaries of Lithuanian news articles using a transformer model},
      author={Lukas Stankevičius and Mantas Lukoševičius},
      year={2021},
      eprint={2105.03279},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


model name: t5-base-lithuanian-news-summaries-175


model author: LukasStankevicius


task: summarization


tags: pytorch, jax, t5, text2text-generation, lt, arxiv:2105.03279, transformers, Lithuanian, summarization, license:apache-2.0, autonlp_compatible


language: ["ru", "en"]
tags:


- russian
- fill-mask
- pretraining
- embeddings
- masked-lm
- tiny
license: mit
widget:
- text: "Миниатюрная модель для [MASK] разных задач."


Копия модели https://huggingface.co/cointegrated/rubert-tiny. Чисто для теста!


model name: test


model author: k0t1k


task: fill-mask


tags: pytorch, bert, pretraining, ru, en, transformers, russian, fill-mask, embeddings, masked-lm, tiny, license:mit, autonlp_compatible, infinity_compatible


language:


- en
thumbnail: https://github.com/karanchahal/distiller/blob/master/distiller.jpg
tags:
- question-answering
license: apache-2.0
datasets:
- squad
metrics:
- squad


# DistilBERT with a second step of distillation


## Model description


This model replicates the "DistilBERT (D)" model from Table 2 of the [DistilBERT paper](https://arxiv.org/pdf/1910.01108.pdf). In this approach, a DistilBERT student is fine-tuned on SQuAD v1.1, but with a BERT model (also fine-tuned on SQuAD v1.1) acting as a teacher for a second step of task-specific distillation.


In this version, the following pre-trained models were used:


* Student: `distilbert-base-uncased`
* Teacher: `lewtun/bert-base-uncased-finetuned-squad-v1`


## Training data


This model was trained on the SQuAD v1.1 dataset which can be obtained from the `datasets` library as follows:


```python
from datasets import load_dataset
squad = load_dataset('squad')
```


## Training procedure


## Eval results


|                  | Exact Match | F1   |
|------------------|-------------|------|
| DistilBERT paper | 79.1        | 86.9 |
| Ours             | 78.4        | 86.5 |


The scores were calculated using the `squad` metric from `datasets`.


### BibTeX entry and citation info


```bibtex
@misc{sanh2020distilbert,
      title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
      author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
      year={2020},
      eprint={1910.01108},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


model name: distilbert-base-uncased-finetuned-squad-d5716d28


model author: BatuhanYilmaz


task: question-answering


tags: pytorch, distilbert, fill-mask, en, dataset:squad, arxiv:1910.01108, transformers, question-answering, license:apache-2.0, autonlp_compatible, infinity_compatible


license: mit
tags:


- generated_from_keras_callback
model-index:
- name: dummy-model
results: []


<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->


# dummy-model


This model is a fine-tuned version of [camembert-base](https://huggingface.co/camembert-base) on an unknown dataset.
It achieves the following results on the evaluation set:


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- optimizer: None
- training_precision: float32


### Training results


### Framework versions


- Transformers 4.15.0
- TensorFlow 2.7.0
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: dummy-model


model author: BatuhanYilmaz


task: fill-mask


tags: tf, camembert, fill-mask, transformers, license:mit, generated_from_keras_callback, model-index, autonlp_compatible


## language: el


# GreekSocialBERT


## Model description


A Greek language model based on [GreekBERT](https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1)


## Training data


The training data is a corpus of 458,293 documents collected from Greek social media accounts.


The training corpus has been collected and provided by [Palo LTD](http://www.paloservices.com/)


## Eval results


### BibTeX entry and citation info


```bibtex
@Article{info12080331,
AUTHOR = {Alexandridis, Georgios and Varlamis, Iraklis and Korovesis, Konstantinos and Caridakis, George and Tsantilas, Panagiotis},
TITLE = {A Survey on Sentiment Analysis and Opinion Mining in Greek Social Media},
JOURNAL = {Information},
VOLUME = {12},
YEAR = {2021},
NUMBER = {8},
ARTICLE-NUMBER = {331},
URL = {https://www.mdpi.com/2078-2489/12/8/331},
ISSN = {2078-2489},
DOI = {10.3390/info12080331}
}
```


model name: greeksocialbert-base-greek-uncased-v1


model author: gealexandri


tags: pytorch, tf, bert, fill-mask, el, transformers, autonlp_compatible, infinity_compatible


# PaloBERT


A Greek language model based on [RoBERTa](https://arxiv.org/abs/1907.11692)


The training data is a corpus of 458,293 documents collected from Greek social media accounts. It also contains a GTP-2 tokenizer trained from scratch on the same corpus.


```bibtex

@Article{info12080331,
AUTHOR = {Alexandridis, Georgios and Varlamis, Iraklis and Korovesis, Konstantinos and Caridakis, George and Tsantilas, Panagiotis},
TITLE = {A Survey on Sentiment Analysis and Opinion Mining in Greek Social Media},
JOURNAL = {Information},
VOLUME = {12},
YEAR = {2021},
NUMBER = {8},
ARTICLE-NUMBER = {331},
URL = {https://www.mdpi.com/2078-2489/12/8/331},
ISSN = {2078-2489},
DOI = {10.3390/info12080331}
}
```


model name: palobert-base-greek-uncased-v1


task: fill-mask


tags: pytorch, tf, roberta, fill-mask, el, arxiv:1907.11692, transformers, autonlp_compatible, infinity_compatible


widget:


- text: "dem har sökt upp de för att prata."
example_title: "de/dem exempel 1"
- text: "Jag såg de komma runt hörnet och gå i riktning mot dem byggnaderna."
example_title: "de/dem exempel 2"


## DeFormer


DeFormer är en modell som har tränats på att skilja mellan `de` och `dem` i svenska meningar. Modellen kan testas direkt i panelerna till höger under **Hosted Inference API** genom att skriva in en mening och trycka på **Compute**.


**Instruktioner (VIKTIGT):**
Använd endast de/dem med små bokstäver vid testning. I träningen av modellen gjordes alla "De" och "Dem" om till gemener. Avsluta meningen med skiljetecken (punkt, frågetecken, osv) för bäst möjliga resultat.


## Träningsdata


DeFormer har tränats på meningar från Europarlamentet och svenskspråkiga Wikimedia. Dessa hämtades från [OPUS](https://opus.nlpl.eu/). Källorna valdes ut för att de antogs ha ett korrekt språkbruk.


Endast meningar innehållandes `de` eller `dem` -- eller bägge två -- behölls i konstruktionen av träningsdataset. I tabellen nedan återfinns beskrivande statistik över antalet meningar som behölls från respektive dataset, samt frekvenser över förekomsten av `de/dem`.


| Datakälla                                                                                | Meningar    |  # De   | # Dem   | De/Dem ratio |
| -----------                                                                                | ----------- | ------- | ------- | ------------ |
| [Europaparl sv.txt.gz](https://opus.nlpl.eu/download.php?f=Europarl/v8/mono/sv.txt.gz)           | 500660      | 465977      | 54331       | 8.57x        |
| [JRC-Acquis raw.sv.gz](https://opus.nlpl.eu/download.php?f=JRC-Acquis/mono/JRC-Acquis.raw.sv.gz) | 417951      | 408576      | 17028       | 23.99x       |
| [Wikimedia sv.txt.gz](https://opus.nlpl.eu/download.php?f=wikimedia/v20210402/mono/sv.txt.gz)    | 630601      | 602393      | 38852       | 15.48x       |
| **Total**                                                                                | **1549212** | **1476946** | **110211**  | **13.40x**   |


Vid träningen av DeFormer introducerades slumpmässiga substitioner, där `de` eller `dem` byttes ut mot den motsatta formen. Modellen utmanades sedan att klassificera huruvida ett givet ord tillhörde ett av följande kategorier


1. **`ord`** (alla bakgrundsord som inte är de/dem tillhör denna kategori)
1. **`DE`**
1. **`DEM`**


Innan observationerna skickades in till modellträning byttes `de` ut mot `dem` med 47 procent sannolikhet, medan `dem` byttes till `de` i 40 procent av fallen.


## Träffsäkerhet/Accuracy


DeFormer utvärderades på ett valideringsset bestående av 31200 meningar från samma datakälla (svenska wiki + europaparlamentet + JRC) som modellen tränats på. Slumpmässiga fel introducerades för att utmana modellen. 47 procent av förekommande `de` i ursprungsmeningarna ändrades till `dem`, medan 40 procent av förekommande `dem` ändrades till `de`. Tabellen nedan visar att DeFormer är väldigt träffsäker. De få "felaktiga" prediktioner som modellen outputtar är nästan samtliga `de/dem som`-konstruktioner med bisatser. Majoriteten av dessa är egentligen inte att anse som felaktiga, eftersom [båda formerna är accepterade](https://www4.isof.se/cgi-bin/srfl/visasvar.py?sok=dem%20som&svar=79718&log_id=705355).


|             | Accuracy    |
| ----------- | ----------- |
| de          | 99.9\%      |
| dem         | 98.6\%      |


model name: deformer


model author: Lauler


task: token-classification


tags: pytorch, bert, token-classification, transformers, autonlp_compatible, infinity_compatible


## Lawformer


### Introduction


This repository provides the source code and checkpoints of the paper "Lawformer: A Pre-trained Language Model forChinese Legal Long Documents". You can download the checkpoint from the [huggingface model hub](https://huggingface.co/xcjthu/Lawformer) or from [here](https://data.thunlp.org/legal/Lawformer.zip).


### Easy Start


We have uploaded our model to the huggingface model hub. Make sure you have installed transformers.


```python
>>> from transformers import AutoModel, AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("hfl/chinese-roberta-wwm-ext")
>>> model = AutoModel.from_pretrained("thunlp/Lawformer")
>>> inputs = tokenizer("任某提起诉讼，请求判令解除婚姻关系并对夫妻共同财产进行分割。", return_tensors="pt")
>>> outputs = model(**inputs)
```


### Cite


If you use the pre-trained models, please cite this paper:


```
@article{xiao2021lawformer,
  title={Lawformer: A Pre-trained Language Model forChinese Legal Long Documents},
  author={Xiao, Chaojun and Hu, Xueyu and Liu, Zhiyuan and Tu, Cunchao and Sun, Maosong},
  year={2021}
}
```


model name: Lawformer


model author: thunlp


task: fill-mask


tags: pytorch, longformer, fill-mask, transformers, autonlp_compatible


tags:


- espnet
- audio
- automatic-speech-recognition
language: en
datasets:
- an4
license: cc-by-4.0


## ESPnet2 ASR model


### `Fhrozen/test_an4`


This model was trained by Fhrozen using an4 recipe in [espnet](https://github.com/espnet/espnet/).


### Demo: How to use in ESPnet2


```bash
cd espnet
git checkout b8df4c928e132acff78d196988bdb68a66987952
pip install -e .
cd egs2/an4/asr1
./run.sh --skip_data_prep false --skip_train true --download_model Fhrozen/test_an4
```


<!-- Generated by scripts/utils/show_asr_result.sh -->


# RESULTS


## Environments


- date: `Wed Oct 20 00:00:46 JST 2021`
- python version: `3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0]`
- espnet version: `espnet 0.10.4a1`
- pytorch version: `pytorch 1.9.0`
- Git hash: `b8df4c928e132acff78d196988bdb68a66987952`
  - Commit date: `Tue Oct 19 07:48:11 2021 -0400`


## asr_train_raw_en_bpe30


### WER


### CER


### TER


## ASR config


<details><summary>expand</summary>


```
config: null
print_config: false
log_level: INFO
dry_run: false
iterator_type: sequence
output_dir: exp/asr_train_raw_en_bpe30
ngpu: 0
seed: 0
num_workers: 1
num_att_plot: 3
dist_backend: nccl
dist_init_method: env://
dist_world_size: null
dist_rank: null
local_rank: null
dist_master_addr: null
dist_master_port: null
dist_launcher: null
multiprocessing_distributed: false
unused_parameters: false
sharded_ddp: false
cudnn_enabled: true
cudnn_benchmark: false
cudnn_deterministic: true
collect_stats: false
write_collected_feats: false
max_epoch: 40
patience: null
val_scheduler_criterion:
- valid
- loss
early_stopping_criterion:
- valid
- loss
- min
best_model_criterion:
-   - train
    - loss
    - min
-   - valid
    - loss
    - min
-   - train
    - acc
    - max
-   - valid
    - acc
    - max
keep_nbest_models:
- 10
grad_clip: 5.0
grad_clip_type: 2.0
grad_noise: false
accum_grad: 1
no_forward_run: false
resume: true
train_dtype: float32
use_amp: false
log_interval: null
use_tensorboard: true
use_wandb: false
wandb_project: null
wandb_id: null
wandb_entity: null
wandb_name: null
wandb_model_log_interval: -1
detect_anomaly: false
pretrain_path: null
init_param: []
ignore_init_mismatch: false
freeze_param: []
num_iters_per_epoch: null
batch_size: 20
valid_batch_size: null
batch_bins: 1000000
valid_batch_bins: null
train_shape_file:
- exp/asr_stats_raw_en_bpe30/train/speech_shape
- exp/asr_stats_raw_en_bpe30/train/text_shape.bpe
valid_shape_file:
- exp/asr_stats_raw_en_bpe30/valid/speech_shape
- exp/asr_stats_raw_en_bpe30/valid/text_shape.bpe
batch_type: folded
valid_batch_type: null
fold_length:
- 80000
- 150
sort_in_batch: descending
sort_batch: descending
multiple_iterator: false
chunk_length: 500
chunk_shift_ratio: 0.5
num_cache_chunks: 1024
train_data_path_and_name_and_type:
-   - dump/raw/train_nodev/wav.scp
    - speech
    - sound
-   - dump/raw/train_nodev/text
    - text
    - text
valid_data_path_and_name_and_type:
-   - dump/raw/train_dev/wav.scp
    - speech
    - sound
-   - dump/raw/train_dev/text
    - text
    - text
allow_variable_data_keys: false
max_cache_size: 0.0
max_cache_fd: 32
valid_max_cache_size: null
optim: adadelta
optim_conf: {}
scheduler: null
scheduler_conf: {}
token_list:
- <blank>
- <unk>
- ▁
- T
- E
- O
- R
- Y
- A
- H
- U
- S
- I
- F
- B
- L
- P
- D
- G
- M
- C
- V
- X
- J
- K
- Z
- W
- N
- Q
- <sos/eos>
init: null
input_size: null
ctc_conf:
    dropout_rate: 0.0
    ctc_type: builtin
    reduce: true
    ignore_nan_grad: true
model_conf:
    ctc_weight: 0.5
    ignore_id: -1
    lsm_weight: 0.0
    length_normalized_loss: false
    report_cer: true
    report_wer: true
    sym_space: <space>
    sym_blank: <blank>
    extract_feats_in_collect_stats: true
use_preprocessor: true
token_type: bpe
bpemodel: data/en_token_list/bpe_unigram30/bpe.model
non_linguistic_symbols: null
cleaner: null
g2p: null
speech_volume_normalize: null
rir_scp: null
rir_apply_prob: 1.0
noise_scp: null
noise_apply_prob: 1.0
noise_db_range: '13_15'
frontend: default
frontend_conf:
    fs: 16k
specaug: null
specaug_conf: {}
normalize: global_mvn
normalize_conf:
    stats_file: exp/asr_stats_raw_en_bpe30/train/feats_stats.npz
preencoder: null
preencoder_conf: {}
encoder: rnn
encoder_conf: {}
postencoder: null
postencoder_conf: {}
decoder: rnn
decoder_conf: {}
required:
- output_dir
- token_list
version: 0.10.4a1
distributed: false
```


</details>


## LM config


```
  config: conf/train_lm.yaml
print_config: false
log_level: INFO
dry_run: false
iterator_type: sequence
output_dir: exp/lm_train_lm_en_bpe30
ngpu: 0
seed: 0
num_workers: 1
num_att_plot: 3
dist_backend: nccl
dist_init_method: env://
dist_world_size: null
dist_rank: null
local_rank: null
dist_master_addr: null
dist_master_port: null
dist_launcher: null
multiprocessing_distributed: false
unused_parameters: false
sharded_ddp: false
cudnn_enabled: true
cudnn_benchmark: false
cudnn_deterministic: true
collect_stats: false
write_collected_feats: false
max_epoch: 40
patience: null
val_scheduler_criterion:
- valid
- loss
early_stopping_criterion:
- valid
- loss
- min
best_model_criterion:
-   - valid
    - loss
    - min
keep_nbest_models: 1
grad_clip: 5.0
grad_clip_type: 2.0
grad_noise: false
accum_grad: 1
no_forward_run: false
resume: true
train_dtype: float32
use_amp: false
log_interval: null
use_tensorboard: true
use_wandb: false
wandb_project: null
wandb_id: null
wandb_entity: null
wandb_name: null
wandb_model_log_interval: -1
detect_anomaly: false
pretrain_path: null
init_param: []
ignore_init_mismatch: false
freeze_param: []
num_iters_per_epoch: null
batch_size: 256
valid_batch_size: null
batch_bins: 1000000
valid_batch_bins: null
train_shape_file:
- exp/lm_stats_en_bpe30/train/text_shape.bpe
valid_shape_file:
- exp/lm_stats_en_bpe30/valid/text_shape.bpe
batch_type: folded
valid_batch_type: null
fold_length:
- 150
sort_in_batch: descending
sort_batch: descending
multiple_iterator: false
chunk_length: 500
chunk_shift_ratio: 0.5
num_cache_chunks: 1024
train_data_path_and_name_and_type:
-   - dump/raw/lm_train.txt
    - text
    - text
valid_data_path_and_name_and_type:
-   - dump/raw/train_dev/text
    - text
    - text
allow_variable_data_keys: false
max_cache_size: 0.0
max_cache_fd: 32
valid_max_cache_size: null
optim: adam
optim_conf:
    lr: 0.1
scheduler: null
scheduler_conf: {}
token_list:
- <blank>
- <unk>
- ▁
- T
- E
- O
- R
- Y
- A
- H
- U
- S
- I
- F
- B
- L
- P
- D
- G
- M
- C
- V
- X
- J
- K
- Z
- W
- N
- Q
- <sos/eos>
init: null
model_conf:
    ignore_id: 0
use_preprocessor: true
token_type: bpe
bpemodel: data/en_token_list/bpe_unigram30/bpe.model
non_linguistic_symbols: null
cleaner: null
g2p: null
lm: seq_rnn
lm_conf:
    unit: 650
    nlayers: 2
required:
- output_dir
- token_list
version: 0.10.4a1
distributed: false
```


model name: test_an4


model author: Fhrozen


task: automatic-speech-recognition


tags: en, dataset:an4, espnet, audio, automatic-speech-recognition, license:cc-by-4.0


tags:


- generated_from_trainer
model-index:
- name: fine-tune-Pegasus-large
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fine-tune-Pegasus-large


This model is a fine-tuned version of [google/pegasus-large](https://huggingface.co/google/pegasus-large) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 11.0526


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 6.35e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 1
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.1
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: fine-tune-Pegasus-large


model author: SophieTr


task: text2text-generation


tags: pytorch, pegasus, text2text-generation, transformers, generated_from_trainer, model-index, autonlp_compatible


tags:


- generated_from_trainer
model-index:
- name: results
results: []


# results


This model is a fine-tuned version of [sshleifer/distill-pegasus-xsum-16-4](https://huggingface.co/sshleifer/distill-pegasus-xsum-16-4) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 2.4473


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 7.2378        | 0.51  | 100  | 7.1853          |
| 7.2309        | 1.01  | 200  | 6.6342          |
| 6.4796        | 1.52  | 300  | 6.3206          |
| 6.2691        | 2.02  | 400  | 6.0184          |
| 5.7382        | 2.53  | 500  | 5.5754          |
| 4.9922        | 3.03  | 600  | 4.5178          |
| 3.6031        | 3.54  | 700  | 2.8579          |
| 2.5203        | 4.04  | 800  | 2.4718          |
| 2.2563        | 4.55  | 900  | 2.4128          |
| 2.1425        | 5.05  | 1000 | 2.3767          |
| 2.004         | 5.56  | 1100 | 2.3982          |
| 2.0437        | 6.06  | 1200 | 2.3787          |
| 1.9407        | 6.57  | 1300 | 2.3952          |
| 1.9194        | 7.07  | 1400 | 2.3964          |
| 1.758         | 7.58  | 1500 | 2.4056          |
| 1.918         | 8.08  | 1600 | 2.4101          |
| 1.9162        | 8.59  | 1700 | 2.4085          |
| 1.8983        | 9.09  | 1800 | 2.4058          |
| 1.6939        | 9.6   | 1900 | 2.4050          |


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0+cu111
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: results


model author: SophieTr


task: text2text-generation


This is the model so far before time out


model name: distil-pegasus-reddit


tags: pytorch, pegasus, text2text-generation, transformers, autonlp_compatible


language: pt
tags:


- speech
license: apache-2.0


# DistilXLSR-53 for BP


[DistilXLSR-53 for BP: DistilHuBERT applied to Wav2vec XLSR-53 for Brazilian Portuguese](https://github.com/s3prl/s3prl/tree/master/s3prl/upstream/distiller)


The base model pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.


**Note**: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model **speech recognition**, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more in-detail explanation of how to fine-tune the model.
Paper: [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://arxiv.org/abs/2110.01900)
Authors: Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee


**Note 2**: The XLSR-53 model was distilled using [Brazilian Portuguese Datasets](https://huggingface.co/lgris/bp400-xlsr) for test purposes. The dataset is quite small to perform such task (the performance might not be so good as the [original work](https://arxiv.org/abs/2110.01900)).


**Abstract**


Self-supervised speech representation learning methods like wav2vec 2.0 and Hidden-unit BERT (HuBERT) leverage unlabeled speech data for pre-training and offer good representations for numerous speech processing tasks. Despite the success of these methods, they require large memory and high pre-training costs, making them inaccessible for researchers in academia and small companies. Therefore, this paper introduces DistilHuBERT, a novel multi-task learning framework to distill hidden representations from a HuBERT model directly. This method reduces HuBERT's size by 75% and 73% faster while retaining most performance in ten different tasks. Moreover, DistilHuBERT required little training time and data, opening the possibilities of pre-training personal and on-device SSL models for speech.


# Usage


See [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more information on how to fine-tune the model.


model name: distilxlsr_bp_16-24


model author: lgris


task: feature-extraction


tags: pytorch, wav2vec2, feature-extraction, pt, arxiv:2110.01900, transformers, speech, license:apache-2.0


language: pt
datasets:


- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0


# bp500-base10k_voxpopuli: Wav2vec 2.0 with Brazilian Portuguese (BP) Dataset


This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the following datasets:


- [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz): contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus.
- [Common Voice 7.0](https://commonvoice.mozilla.org/pt):  is a project proposed by Mozilla Foundation with the goal to create a wide open dataset in different languages. In this project, volunteers donate and validate speech using the [oficial site](https://commonvoice.mozilla.org/pt).
- [Lapsbm](https://github.com/falabrasil/gitlab-resources): "Falabrasil - UFPA" is a dataset used by the Fala Brasil group to benchmark ASR systems in Brazilian Portuguese. Contains 35 speakers (10 females), each one pronouncing 20 unique sentences, totalling  700 utterances in Brazilian Portuguese. The audios were recorded in 22.05 kHz without environment control.
- [Multilingual Librispeech (MLS)](https://arxiv.org/abs/2012.03411): a massive dataset available in many languages. The MLS is based on audiobook recordings in public domain like [LibriVox](https://librivox.org/). The dataset contains a total of 6k hours of transcribed data in many languages. The set in Portuguese [used in this work](http://www.openslr.org/94/) (mostly Brazilian variant) has approximately 284 hours of speech, obtained from 55 audiobooks read by 62 speakers.
- [Multilingual TEDx](http://www.openslr.org/100): a collection of audio recordings from TEDx talks in 8 source languages. The Portuguese set (mostly Brazilian Portuguese variant) contains 164 hours of transcribed speech.
- [Sidney](https://igormq.github.io/datasets/) (SID): contains 5,777 utterances recorded by 72 speakers (20 women) from 17 to 59 years old with fields such as place of birth, age, gender, education, and occupation;
- [VoxForge](http://www.voxforge.org/): is a project with the goal to build open datasets for acoustic models. The corpus contains approximately 100 speakers and 4,130 utterances of Brazilian Portuguese, with sample rates varying from 16kHz to 44.1kHz.


These datasets were combined to build a larger Brazilian Portuguese dataset. All data was used for training except Common Voice dev/test sets, that were used for validation/test respectively. We also made test sets for all the gathered datasets.


| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          | 94.0h  |    -- |  5.4h |
| Common Voice                   | 37.8h  |  8.9h |  9.5h |
| LaPS BM                        | 0.8h   |    -- |  0.1h |
| MLS                            | 161.0h |    -- |  3.7h |
| Multilingual TEDx (Portuguese) | 148.9h |    -- |  1.8h |
| SID                            | 7.2h   |    -- |  1.0h |
| VoxForge                       | 3.9h   |    -- |  0.1h |
| Total                          | 453.6h |  8.9h | 21.6h |


The original model was fine-tuned using [fairseq](https://github.com/pytorch/fairseq). This notebook uses a converted version of the original one. The link to the original fairseq model is available [here](https://drive.google.com/file/d/19kkENi8uvczmw9OLSdqnjvKqBE53cl_W/view?usp=sharing).


#### Summary


|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| bp\_500-base10k_voxpopuli (demonstration below)              | 0.120 | 0.249 | 0.039 | 0.227 | 0.169 | 0.349 | 0.116 | 0.181 |
| bp\_500-base10k_voxpopuli + 4-gram (demonstration below)       |  0.074 | 0.174 | 0.032 | 0.182 | 0.181 | 0.349 | 0.111 | 0.157 |


#### Transcription examples


| Text                                                                                | Transcription                                                                                |
|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|
|suco de uva e água misturam bem|suco **deúva** e água **misturão** bem|
|culpa do dinheiro|**cupa** do dinheiro|
|eu amo shooters call of duty é o meu favorito|eu **omo** **shúters cofedete** é meu favorito|
|você pode explicar por que isso acontece|você pode explicar *por* que isso **ontece**|
|no futuro você desejará ter começado a investir hoje|no futuro você desejará **a** ter começado a investir hoje|


## Demonstration


```python
MODEL_NAME = "lgris/bp500-base10k_voxpopuli"
```


### Imports and dependencies


```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```


```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```


### Helpers


```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```


```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```


```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```


### Model


```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```


### Download datasets


```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```


```python
%cd bp_dataset
```


### Tests


```python
stt = STT(MODEL_NAME)
```


#### CETUC


```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```


CETUC WER: 0.12096759949218888


#### Common Voice


```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```


CV WER: 0.24977003159495725


#### LaPS


```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```


Laps WER: 0.039769570707070705


#### MLS


```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```


MLS WER: 0.2269637077788063


#### SID


```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```


Sid WER: 0.1691680138494731


#### TEDx


```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```


TEDx WER: 0.34908555859018014


#### VoxForge


```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```


VoxForge WER: 0.11649350649350651


### Tests with LM


```python
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```


### Cetuc


CETUC WER: 0.07499558425787961


#### Common Voice


CV WER: 0.17442648452610307


#### LaPS


Laps WER: 0.032774621212121206


#### MLS


MLS WER: 0.18213620321569274


#### SID


Sid WER: 0.18102544972868206


#### TEDx


TEDx WER: 0.3491402028105601


#### VoxForge


VoxForge WER: 0.11189529220779222


model name: bp500-base10k_voxpopuli


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, dataset:mls, dataset:cetuc, dataset:lapsbm, dataset:voxforge, dataset:tedx, dataset:sid, arxiv:2012.03411, transformers, audio, speech, portuguese-speech-corpus, PyTorch, license:apache-2.0


language: pt
datasets:


- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0


# commonvoice100-xlsr: Wav2vec 2.0 with Common Voice Dataset


This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the [Common Voice 7.0](https://commonvoice.mozilla.org/pt) dataset.


In this notebook the model is tested against other available Brazilian Portuguese datasets.


| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          |        |    -- |  5.4h |
| Common Voice                   |  37.8h |    -- |  9.5h |
| LaPS BM                        |        |    -- |  0.1h |
| MLS                            |        |    -- |  3.7h |
| Multilingual TEDx (Portuguese) |        |    -- |  1.8h |
| SID                            |        |    -- |  1.0h |
| VoxForge                       |        |    -- |  0.1h |
| Total                          |        |    -- | 21.6h |


#### Summary


|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| commonvoice\_100 (demonstration below) |0.088 | 0.126 | 0.121 | 0.173 | 0.177 | 0.424 | 0.145 | 0.179 |
| commonvoice\_100 + 4-gram (demonstration below) |0.057 | 0.095 | 0.076 | 0.138 | 0.146 | 0.382 | 0.130 | 0.146|


## Demonstration


```python
MODEL_NAME = "lgris/commonvoice100-xlsr"
```


### Imports and dependencies


```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```


```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```


### Helpers


```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```


```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```


```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```


### Model


```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        attention_mask = features.attention_mask.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values, attention_mask=attention_mask).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```


### Download datasets


```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```


### Tests


```python
stt = STT(MODEL_NAME)
```


#### CETUC


```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```


CETUC WER: 0.08868880057404624


#### Common Voice


```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```


CV WER: 0.12601035333655114


#### LaPS


```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```


Laps WER: 0.12149621212121209


#### MLS


```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```


MLS WER: 0.173594387890256


#### SID


```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```


Sid WER: 0.1775290775992294


#### TEDx


```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```


TEDx WER: 0.4245704568241374


#### VoxForge


```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```


VoxForge WER: 0.14541801948051947


### Tests with LM


```python
# !find -type f -name "*.wav" -delete
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```


#### CETUC


CETUC WER: 0.05764220069547976


#### Common Voice


CV WER: 0.09569130510737103


#### LaPS


Laps WER: 0.07688131313131312


#### MLS


MLS WER: 0.13814768877494732


#### SID


Sid WER: 0.14652459944499036


#### TEDx


TEDx WER: 0.38196090002435623


#### VoxForge


VoxForge WER: 0.13054112554112554


model name: bp-commonvoice100-xlsr


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, dataset:mls, dataset:cetuc, dataset:lapsbm, dataset:voxforge, dataset:tedx, dataset:sid, transformers, audio, speech, portuguese-speech-corpus, PyTorch, license:apache-2.0


language:


- pt
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
model-index:
- name: sew-tiny-portuguese-cv7
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 7
type: mozilla-foundation/common_voice_7_0
args: pt
metrics:
    - name: Test WER
type: wer
value: 28.90
    - name: Test CER
type: cer
value: 09.41
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: sv
metrics:
    - name: Test WER
type: wer
value: 47.27
    - name: Test CER
type: cer
value: 19.62


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# sew-tiny-portuguese-cv7


This model is a fine-tuned version of [lgris/sew-tiny-pt](https://huggingface.co/lgris/sew-tiny-pt) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4232
- Wer: 0.2745


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 40000
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| No log        | 2.6    | 1000  | 1.0034          | 0.7308 |
| 4.1307        | 5.19   | 2000  | 0.6274          | 0.4721 |
| 4.1307        | 7.79   | 3000  | 0.5541          | 0.4130 |
| 1.3117        | 10.39  | 4000  | 0.5302          | 0.3880 |
| 1.3117        | 12.99  | 5000  | 0.5082          | 0.3644 |
| 1.2047        | 15.58  | 6000  | 0.4818          | 0.3539 |
| 1.2047        | 18.18  | 7000  | 0.4822          | 0.3477 |
| 1.14          | 20.78  | 8000  | 0.4781          | 0.3428 |
| 1.14          | 23.38  | 9000  | 0.4840          | 0.3401 |
| 1.0818        | 25.97  | 10000 | 0.4613          | 0.3251 |
| 1.0818        | 28.57  | 11000 | 0.4569          | 0.3257 |
| 1.0451        | 31.17  | 12000 | 0.4494          | 0.3132 |
| 1.0451        | 33.77  | 13000 | 0.4560          | 0.3201 |
| 1.011         | 36.36  | 14000 | 0.4687          | 0.3174 |
| 1.011         | 38.96  | 15000 | 0.4397          | 0.3122 |
| 0.9785        | 41.56  | 16000 | 0.4605          | 0.3173 |
| 0.9785        | 44.16  | 17000 | 0.4380          | 0.3064 |
| 0.9458        | 46.75  | 18000 | 0.4372          | 0.3048 |
| 0.9458        | 49.35  | 19000 | 0.4426          | 0.3039 |
| 0.9126        | 51.95  | 20000 | 0.4317          | 0.2962 |
| 0.9126        | 54.54  | 21000 | 0.4345          | 0.2960 |
| 0.8926        | 57.14  | 22000 | 0.4365          | 0.2948 |
| 0.8926        | 59.74  | 23000 | 0.4306          | 0.2940 |
| 0.8654        | 62.34  | 24000 | 0.4303          | 0.2928 |
| 0.8654        | 64.93  | 25000 | 0.4351          | 0.2915 |
| 0.8373        | 67.53  | 26000 | 0.4340          | 0.2909 |
| 0.8373        | 70.13  | 27000 | 0.4279          | 0.2907 |
| 0.83          | 72.73  | 28000 | 0.4214          | 0.2867 |
| 0.83          | 75.32  | 29000 | 0.4256          | 0.2849 |
| 0.8062        | 77.92  | 30000 | 0.4281          | 0.2826 |
| 0.8062        | 80.52  | 31000 | 0.4398          | 0.2865 |
| 0.7846        | 83.12  | 32000 | 0.4218          | 0.2812 |
| 0.7846        | 85.71  | 33000 | 0.4227          | 0.2791 |
| 0.7697        | 88.31  | 34000 | 0.4200          | 0.2767 |
| 0.7697        | 90.91  | 35000 | 0.4285          | 0.2791 |
| 0.7539        | 93.51  | 36000 | 0.4238          | 0.2777 |
| 0.7539        | 96.1   | 37000 | 0.4288          | 0.2757 |
| 0.7413        | 98.7   | 38000 | 0.4205          | 0.2748 |
| 0.7413        | 101.3  | 39000 | 0.4241          | 0.2761 |
| 0.7348        | 103.89 | 40000 | 0.4232          | 0.2745 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


model name: sew-tiny-portuguese-cv7


model author: lgris


task: automatic-speech-recognition


tags: pytorch, tensorboard, sew, automatic-speech-recognition, pt, dataset:mozilla-foundation/common_voice_7_0, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language:


- pt
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_7_0
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
model-index:
- name: wav2vec2-xls-r-pt-cv7
results: []


# wav2vec2-xls-r-pt-cv7


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - PT dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7280
- Wer: 0.4698


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 7.5e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- training_steps: 40000
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| 1.7142        | 10.42  | 2000  | 0.8951          | 0.7325 |
| 1.5703        | 20.83  | 4000  | 0.8202          | 0.6916 |
| 1.5028        | 31.25  | 6000  | 0.8049          | 0.6988 |
| 1.4598        | 41.66  | 8000  | 0.7391          | 0.6347 |
| 1.3511        | 52.08  | 10000 | 0.6785          | 0.5926 |
| 1.3271        | 62.5   | 12000 | 0.7235          | 0.6334 |
| 1.2039        | 72.91  | 14000 | 0.6596          | 0.5693 |
| 1.1816        | 83.33  | 16000 | 0.6696          | 0.5633 |
| 1.1137        | 93.75  | 18000 | 0.6595          | 0.5650 |
| 1.0593        | 104.17 | 20000 | 0.6639          | 0.5315 |
| 1.0082        | 114.58 | 22000 | 0.6718          | 0.5234 |
| 0.9576        | 125.0  | 24000 | 0.6703          | 0.5224 |
| 0.9187        | 135.42 | 26000 | 0.6574          | 0.5002 |
| 0.8745        | 145.83 | 28000 | 0.6852          | 0.4990 |
| 0.8418        | 156.25 | 30000 | 0.6893          | 0.4877 |
| 0.7657        | 166.66 | 32000 | 0.7027          | 0.4842 |
| 0.749         | 177.08 | 34000 | 0.7161          | 0.4794 |
| 0.7182        | 187.5  | 36000 | 0.7300          | 0.4795 |
| 0.6766        | 197.91 | 38000 | 0.7279          | 0.4715 |
| 0.6634        | 208.33 | 40000 | 0.7280          | 0.4698 |


### Framework versions


model name: wav2vec2-xls-r-pt-cv7


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:mozilla-foundation/common_voice_7_0, transformers, license:apache-2.0, mozilla-foundation/common_voice_7_0, generated_from_trainer, robust-speech-event, model-index


language:


- pt
license: apache-2.0
tags:
- automatic-speech-recognition
- generated_from_trainer
- robust-speech-event
- pt
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-pt-cv
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 6
type: common_voice
args: pt
metrics:
    - name: Test WER
type: wer
value: 24.29
    - name: Test CER
type: cer
value: 7.51
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: sv
metrics:
    - name: Test WER
type: wer
value: 55.72
    - name: Test CER
type: cer
value: 21.82


# wav2vec2-large-xls-r-300m-pt-cv


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3418
- Wer: 0.3581


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 5
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 10.9035       | 0.2   | 100  | 4.2750          | 1.0    |
| 3.3275        | 0.41  | 200  | 3.0334          | 1.0    |
| 3.0016        | 0.61  | 300  | 2.9494          | 1.0    |
| 2.1874        | 0.82  | 400  | 1.4355          | 0.8721 |
| 1.09          | 1.02  | 500  | 0.9987          | 0.7165 |
| 0.8251        | 1.22  | 600  | 0.7886          | 0.6406 |
| 0.6927        | 1.43  | 700  | 0.6753          | 0.5801 |
| 0.6143        | 1.63  | 800  | 0.6300          | 0.5509 |
| 0.5451        | 1.84  | 900  | 0.5586          | 0.5156 |
| 0.5003        | 2.04  | 1000 | 0.5493          | 0.5027 |
| 0.3712        | 2.24  | 1100 | 0.5271          | 0.4872 |
| 0.3486        | 2.45  | 1200 | 0.4953          | 0.4817 |
| 0.3498        | 2.65  | 1300 | 0.4619          | 0.4538 |
| 0.3112        | 2.86  | 1400 | 0.4570          | 0.4387 |
| 0.3013        | 3.06  | 1500 | 0.4437          | 0.4147 |
| 0.2136        | 3.27  | 1600 | 0.4176          | 0.4124 |
| 0.2131        | 3.47  | 1700 | 0.4281          | 0.4194 |
| 0.2099        | 3.67  | 1800 | 0.3864          | 0.3949 |
| 0.1925        | 3.88  | 1900 | 0.3926          | 0.3913 |
| 0.1709        | 4.08  | 2000 | 0.3764          | 0.3804 |
| 0.1406        | 4.29  | 2100 | 0.3787          | 0.3742 |
| 0.1342        | 4.49  | 2200 | 0.3645          | 0.3693 |
| 0.1305        | 4.69  | 2300 | 0.3463          | 0.3625 |
| 0.1298        | 4.9   | 2400 | 0.3418          | 0.3581 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: wav2vec2-large-xls-r-300m-pt-cv


model author: lgris


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language:


- pt
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_7_0
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_7_0
license: apache-2.0
model-index:
- name: wav2vec2-xls-r-pt-cv7-from-bp400h
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 7
type: mozilla-foundation/common_voice_7_0
args: pt
metrics:
    - name: Test WER
type: wer
value: 12.13
    - name: Test CER
type: cer
value: 3.68
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: sv
metrics:
    - name: Test WER
type: wer
value: 28.23
    - name: Test CER
type: cer
value: 12.58


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-pt-cv7-from-bp400h


This model is a fine-tuned version of [lgris/bp_400h_xlsr2_300M](https://huggingface.co/lgris/bp_400h_xlsr2_300M) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1535
- Wer: 0.1254


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 5000


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.4991        | 0.13  | 100  | 0.1774          | 0.1464 |
| 0.4655        | 0.26  | 200  | 0.1884          | 0.1568 |
| 0.4689        | 0.39  | 300  | 0.2282          | 0.1672 |
| 0.4662        | 0.52  | 400  | 0.1997          | 0.1584 |
| 0.4592        | 0.65  | 500  | 0.1989          | 0.1663 |
| 0.4533        | 0.78  | 600  | 0.2004          | 0.1698 |
| 0.4391        | 0.91  | 700  | 0.1888          | 0.1642 |
| 0.4655        | 1.04  | 800  | 0.1921          | 0.1624 |
| 0.4138        | 1.17  | 900  | 0.1950          | 0.1602 |
| 0.374         | 1.3   | 1000 | 0.2077          | 0.1658 |
| 0.4064        | 1.43  | 1100 | 0.1945          | 0.1596 |
| 0.3922        | 1.56  | 1200 | 0.2069          | 0.1665 |
| 0.4226        | 1.69  | 1300 | 0.1962          | 0.1573 |
| 0.3974        | 1.82  | 1400 | 0.1919          | 0.1553 |
| 0.3631        | 1.95  | 1500 | 0.1854          | 0.1573 |
| 0.3797        | 2.08  | 1600 | 0.1902          | 0.1550 |
| 0.3287        | 2.21  | 1700 | 0.1926          | 0.1598 |
| 0.3568        | 2.34  | 1800 | 0.1888          | 0.1534 |
| 0.3415        | 2.47  | 1900 | 0.1834          | 0.1502 |
| 0.3545        | 2.6   | 2000 | 0.1906          | 0.1560 |
| 0.3344        | 2.73  | 2100 | 0.1804          | 0.1524 |
| 0.3308        | 2.86  | 2200 | 0.1741          | 0.1485 |
| 0.344         | 2.99  | 2300 | 0.1787          | 0.1455 |
| 0.309         | 3.12  | 2400 | 0.1773          | 0.1448 |
| 0.312         | 3.25  | 2500 | 0.1738          | 0.1440 |
| 0.3066        | 3.38  | 2600 | 0.1727          | 0.1417 |
| 0.2999        | 3.51  | 2700 | 0.1692          | 0.1436 |
| 0.2985        | 3.64  | 2800 | 0.1732          | 0.1430 |
| 0.3058        | 3.77  | 2900 | 0.1754          | 0.1402 |
| 0.2943        | 3.9   | 3000 | 0.1691          | 0.1379 |
| 0.2813        | 4.03  | 3100 | 0.1754          | 0.1376 |
| 0.2733        | 4.16  | 3200 | 0.1639          | 0.1363 |
| 0.2592        | 4.29  | 3300 | 0.1675          | 0.1349 |
| 0.2697        | 4.42  | 3400 | 0.1618          | 0.1360 |
| 0.2538        | 4.55  | 3500 | 0.1658          | 0.1348 |
| 0.2746        | 4.67  | 3600 | 0.1674          | 0.1325 |
| 0.2655        | 4.8   | 3700 | 0.1655          | 0.1319 |
| 0.2745        | 4.93  | 3800 | 0.1665          | 0.1316 |
| 0.2617        | 5.06  | 3900 | 0.1600          | 0.1311 |
| 0.2674        | 5.19  | 4000 | 0.1623          | 0.1311 |
| 0.237         | 5.32  | 4100 | 0.1591          | 0.1315 |
| 0.2669        | 5.45  | 4200 | 0.1584          | 0.1295 |
| 0.2476        | 5.58  | 4300 | 0.1572          | 0.1285 |
| 0.2445        | 5.71  | 4400 | 0.1580          | 0.1271 |
| 0.2207        | 5.84  | 4500 | 0.1567          | 0.1269 |
| 0.2289        | 5.97  | 4600 | 0.1536          | 0.1260 |
| 0.2438        | 6.1   | 4700 | 0.1530          | 0.1260 |
| 0.227         | 6.23  | 4800 | 0.1544          | 0.1249 |
| 0.2256        | 6.36  | 4900 | 0.1543          | 0.1254 |
| 0.2184        | 6.49  | 5000 | 0.1535          | 0.1254 |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-pt-cv7-from-bp400h


model author: lgris


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, pt, dataset:mozilla-foundation/common_voice_7_0, transformers, mozilla-foundation/common_voice_7_0, generated_from_trainer, robust-speech-event, license:apache-2.0, model-index


language: pt
datasets:


- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0


# bp500-base100k_voxpopuli: Wav2vec 2.0 with Brazilian Portuguese (BP) Dataset


This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the following datasets:


- [CETUC](http://www02.smt.ufrj.br/~igor.quintanilha/alcaim.tar.gz): contains approximately 145 hours of Brazilian Portuguese speech distributed among 50 male and 50 female speakers, each pronouncing approximately 1,000 phonetically balanced sentences selected from the [CETEN-Folha](https://www.linguateca.pt/cetenfolha/) corpus.
- [Common Voice 7.0](https://commonvoice.mozilla.org/pt):  is a project proposed by Mozilla Foundation with the goal to create a wide open dataset in different languages. In this project, volunteers donate and validate speech using the [oficial site](https://commonvoice.mozilla.org/pt).
- [Lapsbm](https://github.com/falabrasil/gitlab-resources): "Falabrasil - UFPA" is a dataset used by the Fala Brasil group to benchmark ASR systems in Brazilian Portuguese. Contains 35 speakers (10 females), each one pronouncing 20 unique sentences, totalling  700 utterances in Brazilian Portuguese. The audios were recorded in 22.05 kHz without environment control.
- [Multilingual Librispeech (MLS)](https://arxiv.org/abs/2012.03411): a massive dataset available in many languages. The MLS is based on audiobook recordings in public domain like [LibriVox](https://librivox.org/). The dataset contains a total of 6k hours of transcribed data in many languages. The set in Portuguese [used in this work](http://www.openslr.org/94/) (mostly Brazilian variant) has approximately 284 hours of speech, obtained from 55 audiobooks read by 62 speakers.
- [Multilingual TEDx](http://www.openslr.org/100): a collection of audio recordings from TEDx talks in 8 source languages. The Portuguese set (mostly Brazilian Portuguese variant) contains 164 hours of transcribed speech.
- [Sidney](https://igormq.github.io/datasets/) (SID): contains 5,777 utterances recorded by 72 speakers (20 women) from 17 to 59 years old with fields such as place of birth, age, gender, education, and occupation;
- [VoxForge](http://www.voxforge.org/): is a project with the goal to build open datasets for acoustic models. The corpus contains approximately 100 speakers and 4,130 utterances of Brazilian Portuguese, with sample rates varying from 16kHz to 44.1kHz.


These datasets were combined to build a larger Brazilian Portuguese dataset. All data was used for training except Common Voice dev/test sets, that were used for validation/test respectively. We also made test sets for all the gathered datasets.


| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          | 94.0h  |    -- |  5.4h |
| Common Voice                   | 37.8h  |  8.9h |  9.5h |
| LaPS BM                        | 0.8h   |    -- |  0.1h |
| MLS                            | 161.0h |    -- |  3.7h |
| Multilingual TEDx (Portuguese) | 148.9h |    -- |  1.8h |
| SID                            | 7.2h   |    -- |  1.0h |
| VoxForge                       | 3.9h   |    -- |  0.1h |
| Total                          | 453.6h |  8.9h | 21.6h |


The original model was fine-tuned using [fairseq](https://github.com/pytorch/fairseq). This notebook uses a converted version of the original one. The link to the original fairseq model is available [here](https://drive.google.com/file/d/10iESR5AQxuxF5F7w3wLbpc_9YMsYbY9H/view?usp=sharing).


#### Summary


|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| bp\_500-base100k_voxpopuli (demonstration below)              | 0.142 | 0.201 | 0.052 | 0.224 | 0.102 | 0.317 | 0.048 | 0.155 |
| bp\_500-base100k_voxpopuli + 4-gram (demonstration below)       | 0.099 | 0.149 | 0.047 | 0.192 | 0.115 | 0.371 | 0.127 | 0.157 |


#### Transcription examples


| Text                                                                                | Transcription                                                                                |
|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|
|qual o instagram dele|**qualo** **está** **gramedele**|
|o capitão foi expulso do exército porque era doido|o **capitãl** foi **exposo** do exército porque era doido|
|também por que não|também **porque** não|
|não existe tempo como o presente|não existe tempo como *o* presente|
|eu pulei para salvar rachel|eu pulei para salvar **haquel**|
|augusto cezar passos marinho|augusto **cesa** **passoesmarinho**|


## Demonstration


```python
MODEL_NAME = "lgris/bp500-base100k_voxpopuli"
```


### Imports and dependencies


```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```


```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```


### Helpers


```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```


```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```


```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```


### Model


```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```


### Download datasets


```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```


```python
%cd bp_dataset
```


### Tests


```python
stt = STT(MODEL_NAME)
```


#### CETUC


```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```


CETUC WER: 0.1419179499917191


#### Common Voice


```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```


CV WER: 0.20079950312040154


#### LaPS


```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```


Laps WER: 0.052780934343434324


#### MLS


```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```


MLS WER: 0.22413887199364113


#### SID


```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```


Sid WER: 0.1019041538671034


#### TEDx


```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```


TEDx WER: 0.31711268778273327


#### VoxForge


```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```


VoxForge WER: 0.04826433982683982


### Tests with LM


```python
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```


### Cetuc


CETUC WER: 0.099518615112877


#### Common Voice


CV WER: 0.1488912889506362


#### LaPS


Laps WER: 0.047080176767676764


#### MLS


MLS WER: 0.19220291966887196


#### SID


Sid WER: 0.11535498771650306


#### TEDx


TEDx WER: 0.3707890073539895


#### VoxForge


VoxForge WER: 0.12682088744588746


model name: bp500-base100k_voxpopuli


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, dataset:mls, dataset:cetuc, dataset:lapsbm, dataset:voxforge, dataset:tedx, dataset:sid, arxiv:2012.03411, transformers, audio, speech, portuguese-speech-corpus, PyTorch, license:apache-2.0


language: pt
datasets:


- common_voice
- mls
- cetuc
- lapsbm
- voxforge
- tedx
- sid
metrics:
- wer
tags:
- audio
- speech
- wav2vec2
- pt
- portuguese-speech-corpus
- automatic-speech-recognition
- speech
- PyTorch
license: apache-2.0


# sid10-xlsr: Wav2vec 2.0 with Sidney Dataset


This is a the demonstration of a fine-tuned Wav2vec model for Brazilian Portuguese using the [Sidney](https://igormq.github.io/datasets/) dataset.


In this notebook the model is tested against other available Brazilian Portuguese datasets.


| Dataset                        |  Train | Valid |  Test |
|--------------------------------|-------:|------:|------:|
| CETUC                          |        |    -- |  5.4h |
| Common Voice                   |        |    -- |  9.5h |
| LaPS BM                        |        |    -- |  0.1h |
| MLS                            |        |    -- |  3.7h |
| Multilingual TEDx (Portuguese) |        |    -- |  1.8h |
| SID                            |   7.2h |    -- |  1.0h |
| VoxForge                       |        |    -- |  0.1h |
| Total                          |    7.2h|    -- | 21.6h |


#### Summary


|  | CETUC          | CV             | LaPS           | MLS            | SID            | TEDx           | VF             | AVG            |
|----------------------|---------------|----------------|----------------|----------------|----------------|----------------|----------------|----------------|
| sid\_10 (demonstration below) |0.186 | 0.327 | 0.207 | 0.505 | 0.124 | 0.835 | 0.472 | 0.379|
| sid\_10 + 4-gram (demonstration below) |0.096 | 0.223 | 0.115 | 0.432 | 0.101 | 0.791 | 0.348 | 0.301|


## Demonstration


```python
MODEL_NAME = "lgris/sid10-xlsr"
```


### Imports and dependencies


```python
%%capture
!pip install torch==1.8.2+cu111 torchvision==0.9.2+cu111 torchaudio===0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html
!pip install datasets
!pip install jiwer
!pip install transformers
!pip install soundfile
!pip install pyctcdecode
!pip install https://github.com/kpu/kenlm/archive/master.zip
```


```python
import jiwer
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
from pyctcdecode import build_ctcdecoder
import torch
import re
import sys
```


### Helpers


```python
chars_to_ignore_regex = '[\,\?\.\!\;\:\"]'  # noqa: W605

def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = speech.squeeze(0).numpy()
    batch["sampling_rate"] = 16_000
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    batch["target"] = batch["sentence"]
    return batch
```


```python
def calc_metrics(truths, hypos):
    wers = []
    mers = []
    wils = []
    for t, h in zip(truths, hypos):
        try:
            wers.append(jiwer.wer(t, h))
            mers.append(jiwer.mer(t, h))
            wils.append(jiwer.wil(t, h))
        except: # Empty string?
            pass
    wer = sum(wers)/len(wers)
    mer = sum(mers)/len(mers)
    wil = sum(wils)/len(wils)
    return wer, mer, wil
```


```python
def load_data(dataset):
    data_files = {'test': f'{dataset}/test.csv'}
    dataset = load_dataset('csv', data_files=data_files)["test"]
    return dataset.map(map_to_array)
```


### Model


```python
class STT:

    def __init__(self,
                 model_name,
                 device='cuda' if torch.cuda.is_available() else 'cpu',
                 lm=None):
        self.model_name = model_name
        self.model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
        self.processor = Wav2Vec2Processor.from_pretrained(model_name)
        self.vocab_dict = self.processor.tokenizer.get_vocab()
        self.sorted_dict = {
            k.lower(): v for k, v in sorted(self.vocab_dict.items(),
                                            key=lambda item: item[1])
        }
        self.device = device
        self.lm = lm
        if self.lm:
            self.lm_decoder = build_ctcdecoder(
                list(self.sorted_dict.keys()),
                self.lm
            )

    def batch_predict(self, batch):
        features = self.processor(batch["speech"],
                                  sampling_rate=batch["sampling_rate"][0],
                                  padding=True,
                                  return_tensors="pt")
        input_values = features.input_values.to(self.device)
        attention_mask = features.attention_mask.to(self.device)
        with torch.no_grad():
            logits = self.model(input_values, attention_mask=attention_mask).logits
        if self.lm:
            logits = logits.cpu().numpy()
            batch["predicted"] = []
            for sample_logits in logits:
                batch["predicted"].append(self.lm_decoder.decode(sample_logits))
        else:
            pred_ids = torch.argmax(logits, dim=-1)
            batch["predicted"] = self.processor.batch_decode(pred_ids)
        return batch
```


### Download datasets


```python
%%capture
!gdown --id 1HFECzIizf-bmkQRLiQD0QVqcGtOG5upI
!mkdir bp_dataset
!unzip bp_dataset -d bp_dataset/
```


### Tests


```python
stt = STT(MODEL_NAME)
```


#### CETUC


```python
ds = load_data('cetuc_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CETUC WER:", wer)
```


CETUC WER: 0.18623689076557778


#### Common Voice


```python
ds = load_data('commonvoice_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("CV WER:", wer)
```


CV WER: 0.3279775395502392


#### LaPS


```python
ds = load_data('lapsbm_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Laps WER:", wer)
```


Laps WER: 0.20780303030303032


#### MLS


```python
ds = load_data('mls_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("MLS WER:", wer)
```


MLS WER: 0.5056711598536057


#### SID


```python
ds = load_data('sid_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("Sid WER:", wer)
```


Sid WER: 0.1247776617710105


#### TEDx


```python
ds = load_data('tedx_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("TEDx WER:", wer)
```


TEDx WER: 0.8350609256842175


#### VoxForge


```python
ds = load_data('voxforge_dataset')
result = ds.map(stt.batch_predict, batched=True, batch_size=8)
wer, mer, wil = calc_metrics(result["sentence"], result["predicted"])
print("VoxForge WER:", wer)
```


VoxForge WER: 0.47242153679653687


### Tests with LM


```python
# !find -type f -name "*.wav" -delete
!rm -rf ~/.cache
!gdown --id 1GJIKseP5ZkTbllQVgOL98R4yYAcIySFP  # trained with wikipedia
stt = STT(MODEL_NAME, lm='pt-BR-wiki.word.4-gram.arpa')
# !gdown --id 1dLFldy7eguPtyJj5OAlI4Emnx0BpFywg  # trained with bp
# stt = STT(MODEL_NAME, lm='pt-BR.word.4-gram.arpa')
```


#### CETUC


CETUC WER: 0.09677271347353278


#### Common Voice


CV WER: 0.22363215674470321


#### LaPS


Laps WER: 0.1154924242424242


#### MLS


MLS WER: 0.4322369152606427


#### SID


Sid WER: 0.10080313085145765


#### TEDx


TEDx WER: 0.7911789829264236


#### VoxForge


VoxForge WER: 0.34786255411255407


model name: bp-sid10-xlsr


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, dataset:mls, dataset:cetuc, dataset:lapsbm, dataset:voxforge, dataset:tedx, dataset:sid, transformers, audio, speech, portuguese-speech-corpus, PyTorch, license:apache-2.0


language: pt
tags:


- speech
license: apache-2.0


# DistilXLSR-53 for BP


[DistilXLSR-53 for BP: DistilHuBERT applied to Wav2vec XLSR-53 for Brazilian Portuguese](https://github.com/s3prl/s3prl/tree/master/s3prl/upstream/distiller)


The base model pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.


**Note**: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model **speech recognition**, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more in-detail explanation of how to fine-tune the model.
Paper: [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://arxiv.org/abs/2110.01900)
Authors: Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee


**Note 2**: The XLSR-53 model was distilled using [Brazilian Portuguese Datasets](https://huggingface.co/lgris/bp400-xlsr) for test purposes. The dataset is quite small to perform such task (the performance might not be so good as the [original work](https://arxiv.org/abs/2110.01900)).


**Abstract**


Self-supervised speech representation learning methods like wav2vec 2.0 and Hidden-unit BERT (HuBERT) leverage unlabeled speech data for pre-training and offer good representations for numerous speech processing tasks. Despite the success of these methods, they require large memory and high pre-training costs, making them inaccessible for researchers in academia and small companies. Therefore, this paper introduces DistilHuBERT, a novel multi-task learning framework to distill hidden representations from a HuBERT model directly. This method reduces HuBERT's size by 75% and 73% faster while retaining most performance in ten different tasks. Moreover, DistilHuBERT required little training time and data, opening the possibilities of pre-training personal and on-device SSL models for speech.


# Usage


See [this blog](https://huggingface.co/blog/fine-tune-wav2vec2-english) for more information on how to fine-tune the model.


model name: distilxlsr_bp_12-16


task: feature-extraction


tags: pytorch, wav2vec2, feature-extraction, pt, arxiv:2110.01900, transformers, speech, license:apache-2.0


language:


- pt
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-1b-cv8
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 8
type: mozilla-foundation/common_voice_8_0
args: pt
metrics:
    - name: Test WER
type: wer
value: 17.70
    - name: Test CER
type: cer
value: 5.21
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: sv
metrics:
    - name: Test WER
type: wer
value: 45.68
    - name: Test CER
type: cer
value: 18.67


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-1b-cv8


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - PT dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2007
- Wer: 0.1838


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 7.5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 30.0
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 2.1172        | 0.32  | 500   | 1.2852          | 0.9783 |
| 1.4152        | 0.64  | 1000  | 0.6434          | 0.6105 |
| 1.4342        | 0.96  | 1500  | 0.4844          | 0.3989 |
| 1.4657        | 1.29  | 2000  | 0.5080          | 0.4490 |
| 1.4961        | 1.61  | 2500  | 0.4764          | 0.4264 |
| 1.4515        | 1.93  | 3000  | 0.4519          | 0.4068 |
| 1.3924        | 2.25  | 3500  | 0.4472          | 0.4132 |
| 1.4524        | 2.57  | 4000  | 0.4455          | 0.3939 |
| 1.4328        | 2.89  | 4500  | 0.4369          | 0.4069 |
| 1.3456        | 3.22  | 5000  | 0.4234          | 0.3774 |
| 1.3725        | 3.54  | 5500  | 0.4387          | 0.3789 |
| 1.3812        | 3.86  | 6000  | 0.4298          | 0.3825 |
| 1.3282        | 4.18  | 6500  | 0.4025          | 0.3703 |
| 1.3326        | 4.5   | 7000  | 0.3917          | 0.3502 |
| 1.3028        | 4.82  | 7500  | 0.3889          | 0.3582 |
| 1.293         | 5.14  | 8000  | 0.3859          | 0.3496 |
| 1.321         | 5.47  | 8500  | 0.3875          | 0.3576 |
| 1.3165        | 5.79  | 9000  | 0.3927          | 0.3589 |
| 1.2701        | 6.11  | 9500  | 0.4058          | 0.3621 |
| 1.2718        | 6.43  | 10000 | 0.4211          | 0.3916 |
| 1.2683        | 6.75  | 10500 | 0.3968          | 0.3620 |
| 1.2643        | 7.07  | 11000 | 0.4128          | 0.3848 |
| 1.2485        | 7.4   | 11500 | 0.3849          | 0.3727 |
| 1.2608        | 7.72  | 12000 | 0.3770          | 0.3474 |
| 1.2388        | 8.04  | 12500 | 0.3774          | 0.3574 |
| 1.2524        | 8.36  | 13000 | 0.3789          | 0.3550 |
| 1.2458        | 8.68  | 13500 | 0.3770          | 0.3410 |
| 1.2505        | 9.0   | 14000 | 0.3638          | 0.3403 |
| 1.2254        | 9.32  | 14500 | 0.3770          | 0.3509 |
| 1.2459        | 9.65  | 15000 | 0.3592          | 0.3349 |
| 1.2049        | 9.97  | 15500 | 0.3600          | 0.3428 |
| 1.2097        | 10.29 | 16000 | 0.3626          | 0.3347 |
| 1.1988        | 10.61 | 16500 | 0.3740          | 0.3269 |
| 1.1671        | 10.93 | 17000 | 0.3548          | 0.3245 |
| 1.1532        | 11.25 | 17500 | 0.3394          | 0.3140 |
| 1.1459        | 11.58 | 18000 | 0.3349          | 0.3156 |
| 1.1511        | 11.9  | 18500 | 0.3272          | 0.3110 |
| 1.1465        | 12.22 | 19000 | 0.3348          | 0.3084 |
| 1.1426        | 12.54 | 19500 | 0.3193          | 0.3027 |
| 1.1278        | 12.86 | 20000 | 0.3318          | 0.3021 |
| 1.149         | 13.18 | 20500 | 0.3169          | 0.2947 |
| 1.114         | 13.5  | 21000 | 0.3224          | 0.2986 |
| 1.1249        | 13.83 | 21500 | 0.3227          | 0.2921 |
| 1.0968        | 14.15 | 22000 | 0.3033          | 0.2878 |
| 1.0851        | 14.47 | 22500 | 0.2996          | 0.2863 |
| 1.0985        | 14.79 | 23000 | 0.3011          | 0.2843 |
| 1.0808        | 15.11 | 23500 | 0.2932          | 0.2759 |
| 1.069         | 15.43 | 24000 | 0.2919          | 0.2750 |
| 1.0602        | 15.76 | 24500 | 0.2959          | 0.2713 |
| 1.0369        | 16.08 | 25000 | 0.2931          | 0.2754 |
| 1.0573        | 16.4  | 25500 | 0.2920          | 0.2722 |
| 1.051         | 16.72 | 26000 | 0.2855          | 0.2632 |
| 1.0279        | 17.04 | 26500 | 0.2850          | 0.2649 |
| 1.0496        | 17.36 | 27000 | 0.2817          | 0.2585 |
| 1.0516        | 17.68 | 27500 | 0.2961          | 0.2635 |
| 1.0244        | 18.01 | 28000 | 0.2781          | 0.2589 |
| 1.0099        | 18.33 | 28500 | 0.2783          | 0.2565 |
| 1.0016        | 18.65 | 29000 | 0.2719          | 0.2537 |
| 1.0157        | 18.97 | 29500 | 0.2621          | 0.2449 |
| 0.9572        | 19.29 | 30000 | 0.2582          | 0.2427 |
| 0.9802        | 19.61 | 30500 | 0.2707          | 0.2468 |
| 0.9577        | 19.94 | 31000 | 0.2563          | 0.2389 |
| 0.9562        | 20.26 | 31500 | 0.2592          | 0.2382 |
| 0.962         | 20.58 | 32000 | 0.2539          | 0.2341 |
| 0.9541        | 20.9  | 32500 | 0.2505          | 0.2288 |
| 0.9587        | 21.22 | 33000 | 0.2486          | 0.2302 |
| 0.9146        | 21.54 | 33500 | 0.2461          | 0.2269 |
| 0.9215        | 21.86 | 34000 | 0.2387          | 0.2228 |
| 0.9105        | 22.19 | 34500 | 0.2405          | 0.2222 |
| 0.8949        | 22.51 | 35000 | 0.2316          | 0.2191 |
| 0.9153        | 22.83 | 35500 | 0.2358          | 0.2180 |
| 0.8907        | 23.15 | 36000 | 0.2369          | 0.2168 |
| 0.8973        | 23.47 | 36500 | 0.2323          | 0.2120 |
| 0.8878        | 23.79 | 37000 | 0.2293          | 0.2104 |
| 0.8818        | 24.12 | 37500 | 0.2302          | 0.2132 |
| 0.8919        | 24.44 | 38000 | 0.2262          | 0.2083 |
| 0.8473        | 24.76 | 38500 | 0.2257          | 0.2040 |
| 0.8516        | 25.08 | 39000 | 0.2246          | 0.2031 |
| 0.8451        | 25.4  | 39500 | 0.2198          | 0.2000 |
| 0.8288        | 25.72 | 40000 | 0.2199          | 0.1990 |
| 0.8465        | 26.05 | 40500 | 0.2165          | 0.1972 |
| 0.8305        | 26.37 | 41000 | 0.2128          | 0.1957 |
| 0.8202        | 26.69 | 41500 | 0.2127          | 0.1937 |
| 0.8223        | 27.01 | 42000 | 0.2100          | 0.1934 |
| 0.8322        | 27.33 | 42500 | 0.2076          | 0.1905 |
| 0.8139        | 27.65 | 43000 | 0.2054          | 0.1880 |
| 0.8299        | 27.97 | 43500 | 0.2026          | 0.1868 |
| 0.7937        | 28.3  | 44000 | 0.2045          | 0.1872 |
| 0.7972        | 28.62 | 44500 | 0.2025          | 0.1861 |
| 0.809         | 28.94 | 45000 | 0.2026          | 0.1858 |
| 0.813         | 29.26 | 45500 | 0.2013          | 0.1838 |
| 0.7718        | 29.58 | 46000 | 0.2010          | 0.1837 |
| 0.7929        | 29.9  | 46500 | 0.2008          | 0.1840 |


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.3.dev0
- Tokenizers 0.11.0


model name: wav2vec2-xls-r-1b-cv8


model author: lgris


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, pt, dataset:common_voice, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, generated_from_trainer, robust-speech-event, model-index


language:


- gn
license: apache-2.0
tags:
- automatic-speech-recognition
- generated_from_trainer
- gn
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: wav2vec2-xls-r-300m-gn-cv8-3
results: []


# wav2vec2-xls-r-300m-gn-cv8-3


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.9517
- Wer: 0.8542


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- training_steps: 5000
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 19.9125       | 5.54   | 100  | 5.4279          | 1.0    |
| 3.8031        | 11.11  | 200  | 3.3070          | 1.0    |
| 3.3783        | 16.65  | 300  | 3.2450          | 1.0    |
| 3.3472        | 22.22  | 400  | 3.2424          | 1.0    |
| 3.2714        | 27.76  | 500  | 3.1100          | 1.0    |
| 3.2367        | 33.32  | 600  | 3.1091          | 1.0    |
| 3.1968        | 38.86  | 700  | 3.1013          | 1.0    |
| 3.2004        | 44.43  | 800  | 3.1173          | 1.0    |
| 3.1656        | 49.97  | 900  | 3.0682          | 1.0    |
| 3.1563        | 55.54  | 1000 | 3.0457          | 1.0    |
| 3.1356        | 61.11  | 1100 | 3.0139          | 1.0    |
| 3.086         | 66.65  | 1200 | 2.8108          | 1.0    |
| 2.954         | 72.22  | 1300 | 2.3238          | 1.0    |
| 2.6125        | 77.76  | 1400 | 1.6461          | 1.0    |
| 2.3296        | 83.32  | 1500 | 1.2834          | 0.9744 |
| 2.1345        | 88.86  | 1600 | 1.1091          | 0.9693 |
| 2.0346        | 94.43  | 1700 | 1.0273          | 0.9233 |
| 1.9611        | 99.97  | 1800 | 0.9642          | 0.9182 |
| 1.9066        | 105.54 | 1900 | 0.9590          | 0.9105 |
| 1.8178        | 111.11 | 2000 | 0.9679          | 0.9028 |
| 1.7799        | 116.65 | 2100 | 0.9007          | 0.8619 |
| 1.7726        | 122.22 | 2200 | 0.9689          | 0.8951 |
| 1.7389        | 127.76 | 2300 | 0.8876          | 0.8593 |
| 1.7151        | 133.32 | 2400 | 0.8716          | 0.8542 |
| 1.6842        | 138.86 | 2500 | 0.9536          | 0.8772 |
| 1.6449        | 144.43 | 2600 | 0.9296          | 0.8542 |
| 1.5978        | 149.97 | 2700 | 0.8895          | 0.8440 |
| 1.6515        | 155.54 | 2800 | 0.9162          | 0.8568 |
| 1.6586        | 161.11 | 2900 | 0.9039          | 0.8568 |
| 1.5966        | 166.65 | 3000 | 0.8627          | 0.8542 |
| 1.5695        | 172.22 | 3100 | 0.9549          | 0.8824 |
| 1.5699        | 177.76 | 3200 | 0.9332          | 0.8517 |
| 1.5297        | 183.32 | 3300 | 0.9163          | 0.8338 |
| 1.5367        | 188.86 | 3400 | 0.8822          | 0.8312 |
| 1.5586        | 194.43 | 3500 | 0.9217          | 0.8363 |
| 1.5429        | 199.97 | 3600 | 0.9564          | 0.8568 |
| 1.5273        | 205.54 | 3700 | 0.9508          | 0.8542 |
| 1.5043        | 211.11 | 3800 | 0.9374          | 0.8542 |
| 1.4724        | 216.65 | 3900 | 0.9622          | 0.8619 |
| 1.4794        | 222.22 | 4000 | 0.9550          | 0.8363 |
| 1.4843        | 227.76 | 4100 | 0.9577          | 0.8465 |
| 1.4781        | 233.32 | 4200 | 0.9543          | 0.8440 |
| 1.4507        | 238.86 | 4300 | 0.9553          | 0.8491 |
| 1.4997        | 244.43 | 4400 | 0.9728          | 0.8491 |
| 1.4371        | 249.97 | 4500 | 0.9543          | 0.8670 |
| 1.4825        | 255.54 | 4600 | 0.9636          | 0.8619 |
| 1.4187        | 261.11 | 4700 | 0.9609          | 0.8440 |
| 1.4363        | 266.65 | 4800 | 0.9567          | 0.8593 |
| 1.4463        | 272.22 | 4900 | 0.9581          | 0.8542 |
| 1.4117        | 277.76 | 5000 | 0.9517          | 0.8542 |


### Framework versions


- Transformers 4.16.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.11.0


model name: wav2vec2-xls-r-300m-gn-cv8-3


model author: lgris


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, gn, dataset:mozilla-foundation/common_voice_8_0, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language:


- pt
license: apache-2.0
tags:
- generated_from_trainer
- pt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: sew-tiny-portuguese-cv
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 6
type: common_voice
args: pt
metrics:
    - name: Test WER
type: wer
value: 30.02
    - name: Test CER
type: cer
value: 10.34
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: sv
metrics:
    - name: Test WER
type: wer
value: 56.46
    - name: Test CER
type: cer
value: 22.94


# sew-tiny-portuguese-cv


This model is a fine-tuned version of [lgris/sew-tiny-pt](https://huggingface.co/lgris/sew-tiny-pt) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5110
- Wer: 0.2842


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- training_steps: 40000
- mixed_precision_training: Native AMP


### Training results