```python
from transformers.pipelines import pipeline

model_name = "phiyodr/bert-base-finetuned-squad2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
inputs = {
    'question': 'What discipline did Winkelmann create?',
    'context': 'Johann Joachim Winckelmann was a German art historian and archaeologist. He was a pioneering Hellenist who first articulated the difference between Greek, Greco-Roman and Roman art. "The prophet and founding hero of modern archaeology", Winckelmann was one of the founders of scientific archaeology and first applied the categories of style on a large, systematic basis to the history of art. '
}
nlp(inputs)
```


## Training procedure


```
{
	"base_model": "bert-base-uncased",
	"do_lower_case": True,
	"learning_rate": 3e-5,
	"num_train_epochs": 4,
	"max_seq_length": 384,
	"doc_stride": 128,
	"max_query_length": 64,
	"batch_size": 96
}
```


## Eval results


- Data: [dev-v2.0.json](https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json)
- Script: [evaluate-v2.0.py](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/) (original script from [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/README.md))


```
{
  "exact": 70.3950138970774,
  "f1": 73.90527661873521,
  "total": 11873,
  "HasAns_exact": 71.4574898785425,
  "HasAns_f1": 78.48808186475087,
  "HasAns_total": 5928,
  "NoAns_exact": 69.33557611438184,
  "NoAns_f1": 69.33557611438184,
  "NoAns_total": 5945
}
```


model name: bert-base-finetuned-squad2


model author: phiyodr


task: question-answering


tags: pytorch, jax, bert, question-answering, en, dataset:squad2, arxiv:1810.04805, arxiv:1806.03822, transformers, autonlp_compatible, infinity_compatible


language: en
tags:


- pytorch
- question-answering
datasets:
- squad2
metrics:
- exact
- f1
widget:
- text: "What discipline did Winkelmann create?"
context: "Johann Joachim Winckelmann was a German art historian and archaeologist. He was a pioneering Hellenist who first articulated the difference between Greek, Greco-Roman and Roman art. The prophet and founding hero of modern archaeology, Winckelmann was one of the founders of scientific archaeology and first applied the categories of style on a large, systematic basis to the history of art."


# bert-large-finetuned-squad2


## Model description


This model is based on **[bert-large-uncased](https://huggingface.co/bert-large-uncased)** and was finetuned on **[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/)**. The corresponding papers you can found [here (model)](https://arxiv.org/abs/1810.04805) and [here (data)](https://arxiv.org/abs/1806.03822).


## How to use


```python
from transformers.pipelines import pipeline

model_name = "phiyodr/bert-large-finetuned-squad2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
inputs = {
    'question': 'What discipline did Winkelmann create?',
    'context': 'Johann Joachim Winckelmann was a German art historian and archaeologist. He was a pioneering Hellenist who first articulated the difference between Greek, Greco-Roman and Roman art. "The prophet and founding hero of modern archaeology", Winckelmann was one of the founders of scientific archaeology and first applied the categories of style on a large, systematic basis to the history of art. '
}
nlp(inputs)
```


```
{
	"base_model": "bert-large-uncased",
	"do_lower_case": True,
	"learning_rate": 3e-5,
	"num_train_epochs": 4,
	"max_seq_length": 384,
	"doc_stride": 128,
	"max_query_length": 64,
	"batch_size": 96
}
```


```
{
  "exact": 76.22336393497852,
  "f1": 79.72527570261339,
  "total": 11873,
  "HasAns_exact": 76.19770580296895,
  "HasAns_f1": 83.21157193271408,
  "HasAns_total": 5928,
  "NoAns_exact": 76.24894869638352,
  "NoAns_f1": 76.24894869638352,
  "NoAns_total": 5945
}
```


model name: bert-large-finetuned-squad2


# roberta-large-finetuned-squad2


This model is based on [roberta-large](https://huggingface.co/roberta-large) and was finetuned on [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/). The corresponding papers you can found [here (model)](https://arxiv.org/abs/1907.11692) and [here (data)](https://arxiv.org/abs/1806.03822).


```python
from transformers.pipelines import pipeline

model_name = "phiyodr/roberta-large-finetuned-squad2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
inputs = {
    'question': 'What discipline did Winkelmann create?',
    'context': 'Johann Joachim Winckelmann was a German art historian and archaeologist. He was a pioneering Hellenist who first articulated the difference between Greek, Greco-Roman and Roman art. "The prophet and founding hero of modern archaeology", Winckelmann was one of the founders of scientific archaeology and first applied the categories of style on a large, systematic basis to the history of art. '
}
nlp(inputs)
```


## Training procedure


```
{
	"base_model": "roberta-large",
	"do_lower_case": True,
	"learning_rate": 3e-5,
	"num_train_epochs": 4,
	"max_seq_length": 384,
	"doc_stride": 128,
	"max_query_length": 64,
	"batch_size": 96
}
```


## Eval results


```
{
  "exact": 84.38473848227069,
  "f1": 87.89711571225455,
  "total": 11873,
  "HasAns_exact": 80.9885290148448,
  "HasAns_f1": 88.02335608157898,
  "HasAns_total": 5928,
  "NoAns_exact": 87.77123633305298,
  "NoAns_f1": 87.77123633305298,
  "NoAns_total": 5945
}
```


model name: roberta-large-finetuned-squad2


model author: phiyodr


task: question-answering


tags: pytorch, jax, roberta, question-answering, en, dataset:squad2, arxiv:1907.11692, arxiv:1806.03822, transformers, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: bert-finetuned-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: conll2003
type: conll2003
args: conll2003
metrics:
    - name: Precision
type: precision
value: 0.9378727634194831
    - name: Recall
type: recall
value: 0.9527095254123191
    - name: F1
type: f1
value: 0.9452329270328937
    - name: Accuracy
type: accuracy
value: 0.9866515570730559


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-finetuned-ner


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the conll2003 dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0619
- Precision: 0.9379
- Recall: 0.9527
- F1: 0.9452
- Accuracy: 0.9867


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.088         | 1.0   | 1756 | 0.0625          | 0.9203    | 0.9399 | 0.9300 | 0.9835   |
| 0.0383        | 2.0   | 3512 | 0.0614          | 0.9348    | 0.9460 | 0.9404 | 0.9858   |
| 0.0209        | 3.0   | 5268 | 0.0619          | 0.9379    | 0.9527 | 0.9452 | 0.9867   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: bert-finetuned-ner


model author: stefan-jo


task: token-classification


tags: pytorch, tensorboard, bert, token-classification, dataset:conll2003, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


language:


- ro
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_7_0
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-1b-ro
results: []


# wav2vec2-xls-r-1b-ro


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - RO dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1113
- Wer: 0.4770
- Cer: 0.0306


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 3e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 50.0
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|
| 0.7844        | 1.67  | 1500  | 0.3412          | 0.8600 | 0.0940 |
| 0.7272        | 3.34  | 3000  | 0.1926          | 0.6409 | 0.0527 |
| 0.6924        | 5.02  | 4500  | 0.1413          | 0.5722 | 0.0401 |
| 0.6327        | 6.69  | 6000  | 0.1252          | 0.5366 | 0.0371 |
| 0.6363        | 8.36  | 7500  | 0.1235          | 0.5741 | 0.0389 |
| 0.6238        | 10.03 | 9000  | 0.1180          | 0.5542 | 0.0362 |
| 0.6018        | 11.71 | 10500 | 0.1192          | 0.5694 | 0.0369 |
| 0.583         | 13.38 | 12000 | 0.1216          | 0.5772 | 0.0385 |
| 0.5643        | 15.05 | 13500 | 0.1195          | 0.5419 | 0.0371 |
| 0.5399        | 16.72 | 15000 | 0.1240          | 0.5224 | 0.0370 |
| 0.5529        | 18.39 | 16500 | 0.1174          | 0.5555 | 0.0367 |
| 0.5246        | 20.07 | 18000 | 0.1097          | 0.5047 | 0.0339 |
| 0.4936        | 21.74 | 19500 | 0.1225          | 0.5189 | 0.0382 |
| 0.4629        | 23.41 | 21000 | 0.1142          | 0.5047 | 0.0344 |
| 0.4463        | 25.08 | 22500 | 0.1168          | 0.4887 | 0.0339 |
| 0.4671        | 26.76 | 24000 | 0.1119          | 0.5073 | 0.0338 |
| 0.4359        | 28.43 | 25500 | 0.1206          | 0.5479 | 0.0363 |
| 0.4225        | 30.1  | 27000 | 0.1122          | 0.5170 | 0.0345 |
| 0.4038        | 31.77 | 28500 | 0.1159          | 0.5032 | 0.0343 |
| 0.4271        | 33.44 | 30000 | 0.1116          | 0.5126 | 0.0339 |
| 0.3867        | 35.12 | 31500 | 0.1101          | 0.4937 | 0.0327 |
| 0.3674        | 36.79 | 33000 | 0.1142          | 0.4940 | 0.0330 |
| 0.3607        | 38.46 | 34500 | 0.1106          | 0.5145 | 0.0327 |
| 0.3651        | 40.13 | 36000 | 0.1172          | 0.4921 | 0.0317 |
| 0.3268        | 41.81 | 37500 | 0.1093          | 0.4830 | 0.0310 |
| 0.3345        | 43.48 | 39000 | 0.1131          | 0.4760 | 0.0314 |
| 0.3236        | 45.15 | 40500 | 0.1132          | 0.4864 | 0.0317 |
| 0.312         | 46.82 | 42000 | 0.1124          | 0.4861 | 0.0315 |
| 0.3106        | 48.49 | 43500 | 0.1116          | 0.4745 | 0.0306 |


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2.dev0
- Tokenizers 0.11.0


model name: wav2vec2-xls-r-1b-ro


model author: ubamba98


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, ro, dataset:common_voice, transformers, license:apache-2.0, mozilla-foundation/common_voice_7_0, generated_from_trainer, robust-speech-event, model-index


language:


- ro
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-CV8-ro
results: []


# wav2vec2-xls-r-300m-CV8-ro


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - RO dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1578
- Wer: 0.6040
- Cer: 0.0475


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 50.0
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|
| 2.9736        | 3.62  | 500  | 2.9508          | 1.0    | 1.0    |
| 1.3293        | 7.25  | 1000 | 0.3330          | 0.8407 | 0.0862 |
| 0.956         | 10.87 | 1500 | 0.2042          | 0.6872 | 0.0602 |
| 0.9509        | 14.49 | 2000 | 0.2184          | 0.7088 | 0.0652 |
| 0.9272        | 18.12 | 2500 | 0.2312          | 0.7211 | 0.0703 |
| 0.8561        | 21.74 | 3000 | 0.2158          | 0.6838 | 0.0631 |
| 0.8258        | 25.36 | 3500 | 0.1970          | 0.6844 | 0.0601 |
| 0.7993        | 28.98 | 4000 | 0.1895          | 0.6698 | 0.0577 |
| 0.7525        | 32.61 | 4500 | 0.1845          | 0.6453 | 0.0550 |
| 0.7211        | 36.23 | 5000 | 0.1781          | 0.6274 | 0.0531 |
| 0.677         | 39.85 | 5500 | 0.1732          | 0.6188 | 0.0514 |
| 0.6517        | 43.48 | 6000 | 0.1691          | 0.6177 | 0.0503 |
| 0.6326        | 47.1  | 6500 | 0.1619          | 0.6045 | 0.0479 |


### Framework versions


model name: wav2vec2-xls-r-300m-CV8-ro


model author: ubamba98


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, ro, dataset:common_voice, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, generated_from_trainer, robust-speech-event, model-index


# Load the Model


```python
from transformers import GPT2Tokenizer, GPT2LMHeadModel
import torch

# start and end tokens for generation
START_TKN = "<|startoftext|>"
END_TKN = "<|endoftext|>"

# fine tuned on onion dataset w/ distilgpt2
tokenizer = GPT2Tokenizer.from_pretrained("distilgpt2")
model = GPT2LMHeadModel.from_pretrained("distilgpt2")

# use gpu if available
device = "cpu"
  if torch.cuda.is_available():
    device = "cuda"

model = model.to(device)

# get 70th epoch (decent results)
epoch = 70
modelpath = f'distilgpt2_onion_{epoch}.pt'

# load model
model.load_state_dict(torch.load(modelpath))
```


model name: distilgpt2-onion


model author: ejjaffe


tags:


- conversational


# Rick Morty DialoGPT Model


model name: DiabloGPT-medium-rickmorty


model author: codealtgeek


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


End-2-End with english


model name: End2End-asr


model author: cjcu


## language: fr


# tf-allociné


A french sentiment analysis model, based on [CamemBERT](https://camembert-model.fr/), and finetuned on a large-scale dataset scraped from [Allociné.fr](http://www.allocine.fr/) user reviews.


## Results


| Validation Accuracy | Validation F1-Score | Test Accuracy | Test F1-Score |
|--------------------:| -------------------:| -------------:|--------------:|
|               97.39 |               97.36 |         97.44 |         97.34 |


The dataset and the evaluation code are available on [this repo](https://github.com/TheophileBlard/french-sentiment-analysis-with-bert).


## Usage


```python
from transformers import AutoTokenizer, TFAutoModelForSequenceClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("tblard/tf-allocine")
model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine")

nlp = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer)

print(nlp("Alad'2 est clairement le meilleur film de l'année 2018.")) # POSITIVE
print(nlp("Juste whoaaahouuu !")) # POSITIVE
print(nlp("NUL...A...CHIER ! FIN DE TRANSMISSION.")) # NEGATIVE
print(nlp("Je m'attendais à mieux de la part de Franck Dubosc !")) # NEGATIVE
```


## Author


Théophile Blard – :email: theophile.blard@gmail.com


If you use this work (code, model or dataset), please cite as:


> Théophile Blard, French sentiment analysis with BERT, (2020), GitHub repository, <https://github.com/TheophileBlard/french-sentiment-analysis-with-bert>


model name: tf-allocine


model author: tblard


task: text-classification


tags: tf, camembert, text-classification, fr, transformers


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- squad_v2
model_index:
- name: bert-base-uncased-finetuned-squad
results:
  - task:
name: Question Answering
type: question-answering
dataset:
name: squad_v2
type: squad_v2
args: squad_v2


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-base-uncased-finetuned-squad


This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the squad_v2 dataset.
It achieves the following results on the evaluation set:


- Loss: 1.3242


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.0199        | 1.0   | 8157  | 1.0224          |
| 0.7507        | 2.0   | 16314 | 1.1812          |
| 0.5448        | 3.0   | 24471 | 1.3242          |


### Framework versions


- Transformers 4.9.2
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


model name: bert-base-uncased-finetuned-squad


model author: bengul


task: question-answering


tags: pytorch, tensorboard, bert, question-answering, dataset:squad_v2, transformers, license:apache-2.0, generated_from_trainer, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
model-index:
- name: wav2vec2-large-xls-r-300m-dementianet
results: []


# wav2vec2-large-xls-r-300m-dementianet


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 1.3430
- Accuracy: 0.4062


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 22
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.3845        | 3.33  | 40   | 1.3556          | 0.3125   |
| 1.3659        | 6.67  | 80   | 1.3602          | 0.3125   |
| 1.3619        | 10.0  | 120  | 1.3569          | 0.3125   |
| 1.3575        | 13.33 | 160  | 1.3509          | 0.3125   |
| 1.3356        | 16.67 | 200  | 1.3599          | 0.3125   |
| 1.3166        | 20.0  | 240  | 1.3430          | 0.4062   |


### Framework versions


- Transformers 4.14.1
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: wav2vec2-large-xls-r-300m-dementianet


model author: shreyasgite


tags: pytorch, tensorboard, wav2vec2, transformers, license:apache-2.0, generated_from_trainer, model-index


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
model-index:
- name: wav2vec2-large-xls-r-300m-dm32
results: []


# wav2vec2-large-xls-r-300m-dm32


- Loss: 0.5688
- Accuracy: 0.7917


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 2.41  | 34   | 0.6769          | 0.6458   |
| No log        | 4.83  | 68   | 0.6864          | 0.5208   |
| No log        | 7.28  | 102  | 0.6596          | 0.6042   |
| 0.7106        | 9.69  | 136  | 0.6208          | 0.6875   |
| 0.7106        | 12.14 | 170  | 0.6152          | 0.6875   |
| 0.7106        | 14.55 | 204  | 0.6167          | 0.6875   |
| 0.6464        | 16.97 | 238  | 0.5782          | 0.7708   |
| 0.6464        | 19.41 | 272  | 0.6011          | 0.7292   |
| 0.6464        | 21.83 | 306  | 0.5688          | 0.7917   |


### Framework versions


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: wav2vec2-large-xls-r-300m-dm32


model author: shreyasgite


- generated_from_trainer
metrics:
- accuracy
model-index:
- name: wav2vec2-large-xls-r-300m-sanitycheck
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-large-xls-r-300m-sanitycheck


- Loss: 0.0092
- Accuracy: 1.0


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 12
- mixed_precision_training: Native AMP


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.14  | 8    | 0.8034          | 0.4737   |
| No log        | 2.29  | 16   | 0.6803          | 0.5263   |
| No log        | 3.43  | 24   | 0.4867          | 1.0      |
| 0.5907        | 4.57  | 32   | 0.1781          | 0.9474   |
| 0.5907        | 5.71  | 40   | 0.2168          | 0.9474   |
| 0.5907        | 6.86  | 48   | 0.2403          | 0.9474   |
| 0.5907        | 8.0   | 56   | 0.0143          | 1.0      |
| 0.0932        | 9.14  | 64   | 0.0124          | 1.0      |
| 0.0932        | 10.29 | 72   | 0.0089          | 1.0      |
| 0.0932        | 11.43 | 80   | 0.0092          | 1.0      |


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: wav2vec2-large-xls-r-300m-sanitycheck


tags:


- conversational


# DialoGPT chat bot model using discord messages as data


model name: DialoGPT-medium-Mona


model author: S34NtheGuy


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


model name: DialoGPT-small-Harry282


model name: DialoGPT-small-pikamew362


model name: DialoGPT-medium-Glass_Of_Water


model name: DialoGPT-small-wetterlettuce


model name: DialoGPT-small-cursedryno


model name: DialoGPT-small-MJOLNIR_Soul


# Steven Universe DialoGPT Model


model name: DialoGPT-small-steven_universe


model author: rmicheal48


# [Deep Chimpact](https://www.drivendata.org/competitions/82/competition-wildlife-video-depth-estimation/page/390/)


> Depth Estimation for Wildlife Conservation (1st place solution)


## Overview


Healthy natural ecosystems have wide-ranging benefits from public health to the economy to agriculture. In order to protect the Earth's natural resources, conservationists need to be able to monitor species population sizes and population change. Camera traps are widely used in conservation research to capture images and videos of wildlife without human interference. Using statistical models for distance sampling, the frequency of animal sightings can be combined with the distance of each animal from the camera to estimate a species' full population size.


However, getting distances from camera trap footage currently entails an extremely manual, time-intensive process. It takes a researcher more than **10 minutes** on average to label distance for every **1 minute** of video - that’s a lot of time when you have a million videos! This also creates a bottleneck for critical information that conservationists can use to **monitor wildlife populations**.


> Your goal in this challenge is to use machine learning to automatically estimate the distance between a camera trap and an animal in a series of camera trap videos. You will be given a series of timestamps indicating when animals are visible in each camera trap video. To complete the challenge, you will predict the distance between the animal and the camera at each point in time.


Along the way, keep an eye out for some sneaky leopards hunting at night, baby chimpanzees getting piggy-back rides, and diva elephants that can't get enough of the limelight. By contributing to this challenge, you can help advance cutting-edge methods for keeping these animal populations (and humans) healthy and safe!


model name: deep-chimpact


model author: awsaf49


tags: keras


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- covid_qa_deepset
metrics:
- squad_v2  # Example: wer. Use metric id from https://hf.co/metrics
widget:
- text: "What is COVID-19?"
context: "Coronavirus disease 2019 (COVID-19) is a contagious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The first known case was identified in Wuhan, China, in December 2019.[7] The disease has since spread worldwide, leading to an ongoing pandemic."
- text: "Where was COVID-19 first discovered?"
context: "The first known infections from SARS-CoV-2 were discovered in Wuhan, China. The original source of viral transmission to humans remains unclear, as does whether the virus became pathogenic before or after the spillover event."
- text: "What is Post-COVID syndrome?"
context: "Long COVID, also known as post-COVID-19 syndrome, post-acute sequelae of COVID-19 (PASC), or chronic COVID syndrome (CCS) is a condition characterized by long-term sequelae appearing or persisting after the typical convalescence period of COVID-19. Long COVID can affect nearly every organ system, with sequelae including respiratory system disorders, nervous system and neurocognitive disorders, mental health disorders, metabolic disorders, cardiovascular disorders, gastrointestinal disorders, malaise, fatigue, musculoskeletal pain, and anemia. A wide range of symptoms are commonly reported, including fatigue, headaches, shortness of breath, anosmia (loss of smell), parosmia (distorted smell), muscle weakness, low fever and cognitive dysfunction."


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilbert-base-uncased-finetuned-squad


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the covid_qa_deepset dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0976


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 0.2502        | 1.0   | 3880  | 0.1824          |
| 0.2007        | 2.0   | 7760  | 0.1250          |
| 0.1338        | 3.0   | 11640 | 0.0976          |


### Framework versions


- Transformers 4.13.0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: covid_qa_distillbert


model author: shainahub


task: question-answering


tags: pytorch, tensorboard, distilbert, question-answering, dataset:covid_qa_deepset, transformers, license:apache-2.0, generated_from_trainer, autonlp_compatible, infinity_compatible


# Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020


Models and predictions for submission to TRAC - 2020 Second Workshop on Trolling, Aggression and Cyberbullying.


Our trained models as well as evaluation metrics during traing are available at: https://databank.illinois.edu/datasets/IDB-8882752#
We also make a few of our models available in HuggingFace's models repository at https://huggingface.co/socialmediaie/, these models can be further fine-tuned on your dataset of choice.


Our approach is described in our paper titled:


> Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).


The source code for training this model and more details can be found on our code repository: https://github.com/socialmediaie/TRAC2020


NOTE: These models are retrained for uploading here after our submission so the evaluation measures may be slightly different from the ones reported in the paper.


If you plan to use the dataset please cite the following resources:


* Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).
* Mishra, Shubhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. “Trained Models for Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020.” University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-8882752_V1.


```
@inproceedings{Mishra2020TRAC,
author = {Mishra, Sudhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
booktitle = {Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020)},
title = {{Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
year = {2020}
}

@data{illinoisdatabankIDB-8882752,
author = {Mishra, Shubhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
doi = {10.13012/B2IDB-8882752_V1},
publisher = {University of Illinois at Urbana-Champaign},
title = {{Trained models for Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
url = {https://doi.org/10.13012/B2IDB-8882752{\_}V1},
year = {2020}
}
```


## Usage


The models can be used via the following code:


```python
from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
import torch
from pathlib import Path
from scipy.special import softmax
import numpy as np
import pandas as pd

TASK_LABEL_IDS = {
    "Sub-task A": ["OAG", "NAG", "CAG"],
    "Sub-task B": ["GEN", "NGEN"],
    "Sub-task C": ["OAG-GEN", "OAG-NGEN", "NAG-GEN", "NAG-NGEN", "CAG-GEN", "CAG-NGEN"]
}

model_version="databank" # other option is hugging face library
if model_version == "databank":
    # Make sure you have downloaded the required model file from https://databank.illinois.edu/datasets/IDB-8882752
    # Unzip the file at some model_path (we are using: "databank_model")
    model_path = next(Path("databank_model").glob("./*/output/*/model"))
    # Assuming you get the following type of structure inside "databank_model"
    # 'databank_model/ALL/Sub-task C/output/bert-base-multilingual-uncased/model'
    lang, task, _, base_model, _ = model_path.parts
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(model_path)
else:
    lang, task, base_model = "ALL", "Sub-task C", "bert-base-multilingual-uncased"
    base_model = f"socialmediaie/TRAC2020_{lang}_{lang.split()[-1]}_{base_model}"
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(base_model)

# For doing inference set model in eval mode
model.eval()
# If you want to further fine-tune the model you can reset the model to model.train()

task_labels = TASK_LABEL_IDS[task]

sentence = "This is a good cat and this is a bad dog."
processed_sentence = f"{tokenizer.cls_token} {sentence}"
tokens = tokenizer.tokenize(sentence)
indexed_tokens = tokenizer.convert_tokens_to_ids(tokens)
tokens_tensor = torch.tensor([indexed_tokens])

with torch.no_grad():
  logits, = model(tokens_tensor, labels=None)
logits

preds = logits.detach().cpu().numpy()
preds_probs = softmax(preds, axis=1)
preds = np.argmax(preds_probs, axis=1)
preds_labels = np.array(task_labels)[preds]
print(dict(zip(task_labels, preds_probs[0])), preds_labels)
"""You should get an output as follows:

({'CAG-GEN': 0.06762535,
  'CAG-NGEN': 0.03244293,
  'NAG-GEN': 0.6897794,
  'NAG-NGEN': 0.15498641,
  'OAG-GEN': 0.034373745,
  'OAG-NGEN': 0.020792078},
 array(['NAG-GEN'], dtype='<U8'))

"""

```


model name: TRAC2020_IBEN_B_bert-base-multilingual-uncased


model author: socialmediaie


task: text-classification


tags: pytorch, jax, bert, text-classification, transformers, infinity_compatible


model name: TRAC2020_ALL_B_bert-base-multilingual-uncased


model name: TRAC2020_IBEN_A_bert-base-multilingual-uncased


model name: TRAC2020_HIN_B_bert-base-multilingual-uncased


# Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020


Models and predictions for submission to TRAC - 2020 Second Workshop on Trolling, Aggression and Cyberbullying.


Our trained models as well as evaluation metrics during traing are available at: https://databank.illinois.edu/datasets/IDB-8882752#
We also make a few of our models available in HuggingFace's models repository at https://huggingface.co/socialmediaie/, these models can be further fine-tuned on your dataset of choice.


Our approach is described in our paper titled:


> Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).


The source code for training this model and more details can be found on our code repository: https://github.com/socialmediaie/TRAC2020


NOTE: These models are retrained for uploading here after our submission so the evaluation measures may be slightly different from the ones reported in the paper.


If you plan to use the dataset please cite the following resources:


* Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).
* Mishra, Shubhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. “Trained Models for Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020.” University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-8882752_V1.


```
@inproceedings{Mishra2020TRAC,
author = {Mishra, Sudhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
booktitle = {Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020)},
title = {{Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
year = {2020}
}

@data{illinoisdatabankIDB-8882752,
author = {Mishra, Shubhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
doi = {10.13012/B2IDB-8882752_V1},
publisher = {University of Illinois at Urbana-Champaign},
title = {{Trained models for Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
url = {https://doi.org/10.13012/B2IDB-8882752{\_}V1},
year = {2020}
}
```


## Usage


The models can be used via the following code:


```python
from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
import torch
from pathlib import Path
from scipy.special import softmax
import numpy as np
import pandas as pd

TASK_LABEL_IDS = {
    "Sub-task A": ["OAG", "NAG", "CAG"],
    "Sub-task B": ["GEN", "NGEN"],
    "Sub-task C": ["OAG-GEN", "OAG-NGEN", "NAG-GEN", "NAG-NGEN", "CAG-GEN", "CAG-NGEN"]
}

model_version="databank" # other option is hugging face library
if model_version == "databank":
    # Make sure you have downloaded the required model file from https://databank.illinois.edu/datasets/IDB-8882752
    # Unzip the file at some model_path (we are using: "databank_model")
    model_path = next(Path("databank_model").glob("./*/output/*/model"))
    # Assuming you get the following type of structure inside "databank_model"
    # 'databank_model/ALL/Sub-task C/output/bert-base-multilingual-uncased/model'
    lang, task, _, base_model, _ = model_path.parts
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(model_path)
else:
    lang, task, base_model = "ALL", "Sub-task C", "bert-base-multilingual-uncased"
    base_model = f"socialmediaie/TRAC2020_{lang}_{lang.split()[-1]}_{base_model}"
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(base_model)

# For doing inference set model in eval mode
model.eval()
# If you want to further fine-tune the model you can reset the model to model.train()

task_labels = TASK_LABEL_IDS[task]

sentence = "This is a good cat and this is a bad dog."
processed_sentence = f"{tokenizer.cls_token} {sentence}"
tokens = tokenizer.tokenize(sentence)
indexed_tokens = tokenizer.convert_tokens_to_ids(tokens)
tokens_tensor = torch.tensor([indexed_tokens])

with torch.no_grad():
  logits, = model(tokens_tensor, labels=None)
logits

preds = logits.detach().cpu().numpy()
preds_probs = softmax(preds, axis=1)
preds = np.argmax(preds_probs, axis=1)
preds_labels = np.array(task_labels)[preds]
print(dict(zip(task_labels, preds_probs[0])), preds_labels)
"""You should get an output as follows:

({'CAG-GEN': 0.06762535,
  'CAG-NGEN': 0.03244293,
  'NAG-GEN': 0.6897794,
  'NAG-NGEN': 0.15498641,
  'OAG-GEN': 0.034373745,
  'OAG-NGEN': 0.020792078},
 array(['NAG-GEN'], dtype='<U8'))

"""

```


model name: TRAC2020_HIN_A_bert-base-multilingual-uncased


model author: socialmediaie


task: text-classification


tags: pytorch, jax, bert, text-classification, transformers, infinity_compatible


model name: TRAC2020_ENG_B_bert-base-uncased


model name: TRAC2020_IBEN_C_bert-base-multilingual-uncased


Models and predictions for submission to TRAC - 2020 Second Workshop on Trolling, Aggression and Cyberbullying


```python
from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
import torch
from pathlib import Path
from scipy.special import softmax
import numpy as np
import pandas as pd

TASK_LABEL_IDS = {
    "Sub-task A": ["OAG", "NAG", "CAG"],
    "Sub-task B": ["GEN", "NGEN"],
    "Sub-task C": ["OAG-GEN", "OAG-NGEN", "NAG-GEN", "NAG-NGEN", "CAG-GEN", "CAG-NGEN"]
}

model_version="databank" # other option is hugging face library
if model_version == "databank":
    # Make sure you have downloaded the required model file from https://databank.illinois.edu/datasets/IDB-8882752
    # Unzip the file at some model_path (we are using: "databank_model")
    model_path = next(Path("databank_model").glob("./*/output/*/model"))
    # Assuming you get the following type of structure inside "databank_model"
    # 'databank_model/ALL/Sub-task C/output/bert-base-multilingual-uncased/model'
    lang, task, _, base_model, _ = model_path.parts
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(model_path)
else:
    lang, task, base_model = "ALL", "Sub-task C", "bert-base-multilingual-uncased"
    base_model = f"socialmediaie/{lang}_{lang.split()[-1]}_{base_model}"
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(base_model)

# For doing inference set model in eval mode
model.eval()

task_labels = TASK_LABEL_IDS[task]

sentence = "This is a good cat and this is a bad dog."
processed_sentence = f"{tokenizer.cls_token} {sentence}"
tokens = tokenizer.tokenize(sentence)
indexed_tokens = tokenizer.convert_tokens_to_ids(tokens)
tokens_tensor = torch.tensor([indexed_tokens])

with torch.no_grad():
  logits, = model(tokens_tensor, labels=None)
logits

preds = logits.detach().cpu().numpy()
preds_probs = softmax(preds, axis=1)
preds = np.argmax(preds_probs, axis=1)
preds_labels = np.array(task_labels)[preds]
print(dict(zip(task_labels, preds_probs[0])), preds_labels)
"""You should get an output as follows:

({'CAG-GEN': 0.06762535,
  'CAG-NGEN': 0.03244293,
  'NAG-GEN': 0.6897794,
  'NAG-NGEN': 0.15498641,
  'OAG-GEN': 0.034373745,
  'OAG-NGEN': 0.020792078},
 array(['NAG-GEN'], dtype='<U8'))

"""

```


model name: TRAC2020_ALL_C_bert-base-multilingual-uncased


# Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020


Models and predictions for submission to TRAC - 2020 Second Workshop on Trolling, Aggression and Cyberbullying.


Our trained models as well as evaluation metrics during traing are available at: https://databank.illinois.edu/datasets/IDB-8882752#
We also make a few of our models available in HuggingFace's models repository at https://huggingface.co/socialmediaie/, these models can be further fine-tuned on your dataset of choice.


Our approach is described in our paper titled:


> Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).


The source code for training this model and more details can be found on our code repository: https://github.com/socialmediaie/TRAC2020


NOTE: These models are retrained for uploading here after our submission so the evaluation measures may be slightly different from the ones reported in the paper.


If you plan to use the dataset please cite the following resources:


* Mishra, Sudhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. "Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020." In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020).
* Mishra, Shubhanshu, Shivangi Prasad, and Shubhanshu Mishra. 2020. “Trained Models for Multilingual Joint Fine-Tuning of Transformer Models for Identifying Trolling, Aggression and Cyberbullying at TRAC 2020.” University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-8882752_V1.


```
@inproceedings{Mishra2020TRAC,
author = {Mishra, Sudhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
booktitle = {Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (TRAC-2020)},
title = {{Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
year = {2020}
}

@data{illinoisdatabankIDB-8882752,
author = {Mishra, Shubhanshu and Prasad, Shivangi and Mishra, Shubhanshu},
doi = {10.13012/B2IDB-8882752_V1},
publisher = {University of Illinois at Urbana-Champaign},
title = {{Trained models for Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020}},
url = {https://doi.org/10.13012/B2IDB-8882752{\_}V1},
year = {2020}
}
```


## Usage


The models can be used via the following code:


```python
from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
import torch
from pathlib import Path
from scipy.special import softmax
import numpy as np
import pandas as pd

TASK_LABEL_IDS = {
    "Sub-task A": ["OAG", "NAG", "CAG"],
    "Sub-task B": ["GEN", "NGEN"],
    "Sub-task C": ["OAG-GEN", "OAG-NGEN", "NAG-GEN", "NAG-NGEN", "CAG-GEN", "CAG-NGEN"]
}

model_version="databank" # other option is hugging face library
if model_version == "databank":
    # Make sure you have downloaded the required model file from https://databank.illinois.edu/datasets/IDB-8882752
    # Unzip the file at some model_path (we are using: "databank_model")
    model_path = next(Path("databank_model").glob("./*/output/*/model"))
    # Assuming you get the following type of structure inside "databank_model"
    # 'databank_model/ALL/Sub-task C/output/bert-base-multilingual-uncased/model'
    lang, task, _, base_model, _ = model_path.parts
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(model_path)
else:
    lang, task, base_model = "ALL", "Sub-task C", "bert-base-multilingual-uncased"
    base_model = f"socialmediaie/TRAC2020_{lang}_{lang.split()[-1]}_{base_model}"
    tokenizer = AutoTokenizer.from_pretrained(base_model)
    model = AutoModelForSequenceClassification.from_pretrained(base_model)

# For doing inference set model in eval mode
model.eval()
# If you want to further fine-tune the model you can reset the model to model.train()

task_labels = TASK_LABEL_IDS[task]

sentence = "This is a good cat and this is a bad dog."
processed_sentence = f"{tokenizer.cls_token} {sentence}"
tokens = tokenizer.tokenize(sentence)
indexed_tokens = tokenizer.convert_tokens_to_ids(tokens)
tokens_tensor = torch.tensor([indexed_tokens])

with torch.no_grad():
  logits, = model(tokens_tensor, labels=None)
logits

preds = logits.detach().cpu().numpy()
preds_probs = softmax(preds, axis=1)
preds = np.argmax(preds_probs, axis=1)
preds_labels = np.array(task_labels)[preds]
print(dict(zip(task_labels, preds_probs[0])), preds_labels)
"""You should get an output as follows:

({'CAG-GEN': 0.06762535,
  'CAG-NGEN': 0.03244293,
  'NAG-GEN': 0.6897794,
  'NAG-NGEN': 0.15498641,
  'OAG-GEN': 0.034373745,
  'OAG-NGEN': 0.020792078},
 array(['NAG-GEN'], dtype='<U8'))

"""

```


model name: TRAC2020_ALL_A_bert-base-multilingual-uncased


model author: socialmediaie


task: text-classification


tags: pytorch, jax, bert, text-classification, transformers, infinity_compatible


model name: TRAC2020_HIN_C_bert-base-multilingual-uncased


model name: TRAC2020_ENG_A_bert-base-uncased


model name: TRAC2020_ENG_C_bert-base-uncased


language:


- hi
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
model-index:
- name: wav2vec2-large-xls-r-300m-hindi
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-large-xls-r-300m-hindi


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7049
- Wer: 0.3200


model name: wav2vec2-large-xls-r-300m-hindi


model author: ravirajoshi


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, hi, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


- mr
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
model-index:
- name: wav2vec2-large-xls-r-300m-marathi
results: []


# wav2vec2-large-xls-r-300m-marathi


- Loss: 0.5656
- Wer: 0.2156


model name: wav2vec2-large-xls-r-300m-marathi-lm-boosted


tags: pytorch, wav2vec2, automatic-speech-recognition, mr, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


model name: wav2vec2-large-xls-r-300m-marathi


model name: wav2vec2-large-xls-r-300m-hindi-lm-boosted


language: fi
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Finnish by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice fi
type: common_voice
args: fi
metrics:
    - name: Test WER
type: wer
value: 55.097365


# Wav2Vec2-Large-XLSR-53-Finnish


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Finnish using the [Common Voice](https://huggingface.co/datasets/common_voice)
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "fi", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlsr-finnish")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlsr-finnish")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
\\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Finnish test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "fi", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlsr-finnish")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlsr-finnish")
model.to("cuda")

chars_to_ignore_regex = '[\\\\,\\\\?\\\\.\\\\!\\\\-\\\\;\\\\:\\\\"\\\\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\tbatch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
\\tinputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

\\twith torch.no_grad():
\\t\\tlogits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
    pred_ids = torch.argmax(logits, dim=-1)

\\tbatch["pred_strings"] = processor.batch_decode(pred_ids)
\\treturn batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**:
The WER is 55.097365


## Training


The Common Voice `train` and `validation` datasets were used for training.
The script used for training can be found here
https://colab.research.google.com/drive/16AyzqMWU_aWNe3IA-NxrhskB1WLPHG-Q?usp=sharing


model name: wav2vec2-large-xlsr-finnish


model author: birgermoell


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, fi, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: et
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Estonian by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice Estonian
type: common_voice
args: et
metrics:
    - name: Test WER
type: wer
value: 36.951816


# Wav2Vec2-Large-XLSR-53-Estonian


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Luganda using the [Common Voice](https://huggingface.co/datasets/common_voice)
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "et", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlrs-estonian")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlrs-estonian")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


The model can be evaluated as follows on the Luganda test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "fi", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlrs-estonian")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlrs-estonian")
model.to("cuda")

chars_to_ignore_regex = '[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\,\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\?\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\-\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\;\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tinputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\twith torch.no_grad():
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
    pred_ids = torch.argmax(logits, dim=-1)

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["pred_strings"] = processor.batch_decode(pred_ids)
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\treturn batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**:
WER: 36.951816


The Common Voice `train` and `validation` datasets were used for training.
The script used for training can be found here
https://colab.research.google.com/drive/1VcWT92vBCwVn-5d-mkYxhgILPr11OHfR?usp=sharing


model name: wav2vec2-large-xlrs-estonian


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, et, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: lg
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Luganda by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice Luganda
type: common_voice
args: lg
metrics:
    - name: Test WER
type: wer
value: 48.31


# Wav2Vec2-Large-XLSR-53-Luganda


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "lg", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-luganda")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-luganda")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "fi", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-luganda")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-luganda")
model.to("cuda")

chars_to_ignore_regex = '[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\,\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\?\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\-\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\;\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\tbatch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
\\\\\\\\\\\\\\\\tinputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

\\\\\\\\\\\\\\\\twith torch.no_grad():
\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
    pred_ids = torch.argmax(logits, dim=-1)

\\\\\\\\\\\\\\\\tbatch["pred_strings"] = processor.batch_decode(pred_ids)
\\\\\\\\\\\\\\\\treturn batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**:
WER: 48.314356


## Training


The Common Voice `train` and `validation` datasets were used for training.
The script used for training can be found here
https://colab.research.google.com/drive/1ZeII36LZ5IpBrTV7kBaTVfhDqygznlmC?usp=sharing


model name: wav2vec2-luganda


model author: birgermoell


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, lg, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: sv
datasets:


- common_voice
- NST Swedish ASR Database
- P4
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
license: cc0-1.0
model-index:
- name: Wav2vec 2.0 large VoxRex Swedish
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice
type: common_voice
args: sv-SE
metrics:
    - name: Test WER
type: wer
value: 9.914


# Wav2vec 2.0 large VoxRex Swedish (C)


Experiment with LM model.


**Disclaimer:** This is a work in progress. See [VoxRex](https://huggingface.co/KBLab/wav2vec2-large-voxrex) for more details.


**Update 2022-01-10:** Updated to VoxRex-C version.


Finetuned version of KBs [VoxRex large](https://huggingface.co/KBLab/wav2vec2-large-voxrex) model using Swedish radio broadcasts, NST and Common Voice data. Evalutation without a language model gives the following: WER for NST + Common Voice test set (2% of total sentences) is **2.5%**. WER for Common Voice test set is **8.49%** directly and **7.37%** with a 4-gram language model.


When using this model, make sure that your speech input is sampled at 16kHz.


# Performance\*


![Comparison](comparison.png "Comparison")


*Chart shows performance without the additional 20k steps of Common Voice fine-tuning


## Training


This model has been fine-tuned for 120000 updates on NST + CommonVoice<del> and then for an additional 20000 updates on CommonVoice only. The additional fine-tuning on CommonVoice hurts performance on the NST+CommonVoice test set somewhat and, unsurprisingly, improves it on the CommonVoice test set. It seems to perform generally better though [citation needed]</del>.


![WER during training](chart_1.svg "WER")


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").
processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-large-voxrex-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-large-voxrex-swedish")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


model name: lm-swedish


model author: birgermoell


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, sv, dataset:common_voice, dataset:NST Swedish ASR Database, dataset:P4, transformers, audio, speech, license:cc0-1.0, model-index


license: apache-2.0


tags:


- token-classification


datasets:


- wikiann


metrics:


- precision
- recall
- f1
- accuracy


model-index:


- name: ner-swedish-wikiann

  results:

  - task:

    name: Token Classification

    type: token-classification

    dataset:

    name: wikiann

    type: wikiann

    metrics:

    - name: Precision

      type: precision

      value: 0.8331921416757433
    - name: Recall

      type: recall

      value: 0.84243586083126
    - name: F1

      type: f1

      value: 0.8377885044416501
    - name: Accuracy

      type: accuracy

      value: 0.91930707459758


<!-- This model card has been generated automatically according to the information the Trainer had access to. You

should probably proofread and complete it, then remove this comment. -->


# ner-swedish-wikiann


This model is a fine-tuned version of [nordic-roberta-wiki](hhttps://huggingface.co/flax-community/nordic-roberta-wiki) trained for NER on the wikiann dataset.


eval F1-Score: **83,78**


test F1-Score: **83,76**


## Model Usage


```python

from transformers import AutoTokenizer, AutoModelForTokenClassification

from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("birgermoell/ner-swedish-wikiann")

model = AutoModelForTokenClassification.from_pretrained("birgermoell/ner-swedish-wikiann")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)

example = "Jag heter Per och jag jobbar på KTH"

nlp(example)

```


<!--
## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:

- learning_rate: 4.9086903597787154e-05

- train_batch_size: 32

- eval_batch_size: 16

- seed: 42

- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08

- lr_scheduler_type: linear

- num_epochs: 5.0

- mixed_precision_training: Native AMP

### Training results

It achieves the following results on the evaluation set:

- Loss: 0.3156

- Precision: 0.8332
from transformers import AutoTokenizer, AutoModelForTokenClassification

from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("birgermoell/ner-swedish-wikiann")

model = AutoModelForTokenClassification.from_pretrained("birgermoell/ner-swedish-wikiann")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)

example = "Jag heter Per och jag jobbar på KTH"

nlp(example)

- F1: 0.8378

- Accuracy: 0.9193

It achieves the following results on the test set:

- Loss: 0.3023

- Precision: 0.8301

- Recall: 0.8452

- F1: 0.8376

- Accuracy: 0.92

### Framework versions

- Transformers 4.6.1

- Pytorch 1.8.1+cu101

- Datasets 1.6.2

- Tokenizers 0.10.2
 -->


model name: ner-swedish-wikiann


model author: birgermoell


task: token-classification


tags: pytorch, roberta, token-classification, dataset:wikiann, transformers, license:apache-2.0, model-index, autonlp_compatible, infinity_compatible


language:


- sv
datasets:
- oscar
tags:
- summarization
- translation
license: apache-2.0


[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html)
Pretraining Dataset: [C4](https://huggingface.co/datasets/oscar)
Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)
Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*


## Abstract


Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.
![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67)


## Model series


This model is part of a series of models training on TPU with Flax Jax during Huggingface Flax/Jax challenge.


## Gpt models


## Swedish Gpt


## Swedish gpt wiki


# Nordic gpt wiki


## Dansk gpt wiki


## Norsk gpt wiki


## Roberta models


## Nordic Roberta Wiki


## Swe Roberta Wiki Oscar


## Roberta Swedish Scandi


## Roberta Swedish


## Swedish T5 model


model name: t5-base-swedish


model author: birgermoell


task: translation


tags: pytorch, jax, tensorboard, t5, feature-extraction, sv, dataset:oscar, arxiv:1910.10683, transformers, summarization, translation, license:apache-2.0, autonlp_compatible


language: sv
license: cc-by-4.0
tags:


- translate
datasets:
- mc4
widget:
- text: Meningen med livet är <mask>


# Svensk Roberta


## Description


Swedish Roberta model trained on the MC4 dataset. The model performance needs to be assessed


## Model series


## Gpt models


## Swedish Gpt


## Swedish gpt wiki


# Nordic gpt wiki


## Dansk gpt wiki


## Norsk gpt wiki


## Roberta models


## Nordic Roberta Wiki


## Swe Roberta Wiki Oscar


## Roberta Swedish Scandi


## Roberta Swedish


## Swedish T5 model


model name: roberta-swedish-scandi


model author: birgermoell


task: feature-extraction


tags: pytorch, jax, tensorboard, roberta, feature-extraction, sv, dataset:mc4, transformers, license:cc-by-4.0, translate, infinity_compatible


language:


- sv-SE
license: apache-2.0
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-common_voice-tr-demo
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-common_voice-tr-demo


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the COMMON_VOICE - SV-SE dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5528
- Wer: 0.3811


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 15.0
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| No log        | 0.74  | 100  | 3.4444          | 1.0    |
| No log        | 1.47  | 200  | 2.9421          | 1.0    |
| No log        | 2.21  | 300  | 2.2802          | 1.0137 |
| No log        | 2.94  | 400  | 0.9683          | 0.7611 |
| 3.7264        | 3.68  | 500  | 0.7941          | 0.6594 |
| 3.7264        | 4.41  | 600  | 0.6695          | 0.5751 |
| 3.7264        | 5.15  | 700  | 0.6507          | 0.5314 |
| 3.7264        | 5.88  | 800  | 0.5731          | 0.4927 |
| 3.7264        | 6.62  | 900  | 0.5723          | 0.4580 |
| 0.4592        | 7.35  | 1000 | 0.5913          | 0.4479 |
| 0.4592        | 8.09  | 1100 | 0.5562          | 0.4423 |
| 0.4592        | 8.82  | 1200 | 0.5566          | 0.4292 |
| 0.4592        | 9.56  | 1300 | 0.5492          | 0.4303 |
| 0.4592        | 10.29 | 1400 | 0.5665          | 0.4331 |
| 0.2121        | 11.03 | 1500 | 0.5610          | 0.4084 |
| 0.2121        | 11.76 | 1600 | 0.5703          | 0.4014 |
| 0.2121        | 12.5  | 1700 | 0.5669          | 0.3898 |
| 0.2121        | 13.24 | 1800 | 0.5586          | 0.3962 |
| 0.2121        | 13.97 | 1900 | 0.5656          | 0.3897 |
| 0.1326        | 14.71 | 2000 | 0.5565          | 0.3813 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu113
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: wav2vec2-common_voice-tr-demo


model author: birgermoell


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, sv-SE, dataset:common_voice, transformers, license:apache-2.0, common_voice, generated_from_trainer, model-index


language: sv
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Swedish by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice sv-SE
type: common_voice
args: sv-SE
metrics:
    - name: Test WER
type: wer
value: 36.91


# Wav2Vec2-Large-XLSR-53-Swedish


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Swedish using the [Common Voice](https://huggingface.co/datasets/common_voice). The training data amounts to 402 MB.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-swedish-common-voice")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-swedish-common-voice")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the {language} test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "sv-SE", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-swedish-common-voice")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-swedish-common-voice")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
	batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
	inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

	with torch.no_grad():
		logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
	batch["pred_strings"] = processor.batch_decode(pred_ids)
	return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 36.91 %


## Training


The Common Voice `train`, `validation` datasets were used for training.


The script used for training can be found [here](https://colab.research.google.com/drive/1KkD4PeZwnIwxxxOP1bUE7XTZMK7-SzRj?usp=sharing)


model name: wav2vec2-swedish-common-voice


model author: birgermoell


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, sv, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: hu
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Hugarian by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice hu
type: common_voice
args: hu\
metrics:
    - name: Test WER
type: wer
value: 46.97


# Wav2Vec2-Large-XLSR-53-Hungarian


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Hungarian using the [Common Voice](https://huggingface.co/datasets/common_voice)
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "hu", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlsr-hungarian")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlsr-hungarian")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Hungarian test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "hu", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/wav2vec2-large-xlsr-hungarian")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/wav2vec2-large-xlsr-hungarian")
model.to("cuda")

chars_to_ignore_regex = '[\\,\\?\\.\\!\\-\\;\\:\\"\\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\tbatch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
\tinputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

\twith torch.no_grad():
\t\tlogits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
    pred_ids = torch.argmax(logits, dim=-1)

\tbatch["pred_strings"] = processor.batch_decode(pred_ids)
\treturn batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 46.97 %


## Training


The Common Voice `train` and `validation` datasets were used for training.
The script used for training can be found [here](https://colab.research.google.com/drive/1c8LS-RP-RMukvXkpqJ9kLXRWmRKFjevs?usp=sharing)


model name: wav2vec2-large-xlsr-hungarian


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, hu, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: sv
widget:


- text: "Jag är en svensk språkmodell."


## Model series


This model is part of a series of models training on TPU with Flax Jax during Huggingface Flax/Jax challenge.


## Gpt models


## Swedish Gpt


## Swedish gpt wiki


# Nordic gpt wiki


## Dansk gpt wiki


## Norsk gpt wiki


## Roberta models


## Nordic Roberta Wiki


## Swe Roberta Wiki Oscar


## Roberta Swedish Scandi


## Roberta Swedish


## Swedish T5 model


# GPT-svenska-wikipedia


A swedish GPT2 style model trained using Flax CLM pipeline on the Swedish
part of the wiki40b dataset and the Oscar dataset.
https://huggingface.co/datasets/wiki40b


The model was trained for around 22600 steps (42 hours) as part of the Huggingface Jax/Flax challenge with the following loss and learning rate
Loss: 3.1715331077575684, Learning Rate: 0.0024816440418362617)


The model could likely be trained for longer.


## Data cleaning and preprocessing


The data was cleaned and preprocessed using the following script. Make sure to install depencies for beam_runner to make the dataset work.


```python
from datasets import load_dataset
def load_and_clean_wiki():
    dataset = load_dataset('wiki40b', 'sv', beam_runner='DirectRunner', split="train")
    #dataset = load_dataset('wiki40b', 'sv', beam_runner='DirectRunner')
    dataset = dataset.remove_columns(['wikidata_id', 'version_id'])
    filtered_dataset = dataset.map(filter_wikipedia)
    # filtered_dataset[:3]
    # print(filtered_dataset[:3])
    return filtered_dataset

def filter_wikipedia(batch):
    batch["text"] = " ".join(batch["text"].split("\
_START_SECTION_\
"))
    batch["text"] = " ".join(batch["text"].split("\
_START_ARTICLE_\
"))
    batch["text"] = " ".join(batch["text"].split("\
_START_ARTICLE_\
"))
    batch["text"] = " ".join(batch["text"].split("\
_START_PARAGRAPH_\
"))
    batch["text"] = " ".join(batch["text"].split("_NEWLINE_"))
    batch["text"] = " ".join(batch["text"].split("\xa0"))
    return batch
```


## Training script


The following training script was used to train the model.


```bash
./run_clm_flax.py     --output_dir="${MODEL_DIR}"     --model_type="gpt2"     --config_name="${MODEL_DIR}"     --tokenizer_name="${MODEL_DIR}"     --dataset_name="wiki40b"     --dataset_config_name="sv"     --do_train --do_eval     --block_size="512"     --per_device_train_batch_size="64"     --per_device_eval_batch_size="64"     --learning_rate="5e-3" --warmup_steps="1000"     --adam_beta1="0.9" --adam_beta2="0.98" --weight_decay="0.01"     --overwrite_output_dir     --num_train_epochs="20"     --logging_steps="500"     --save_steps="1000"     --eval_steps="2500"     --push_to_hub
```


model name: swedish-gpt


model author: birgermoell


task: text-generation


tags: pytorch, jax, tensorboard, gpt2, text-generation, sv, transformers


language: et
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: common-voice-vox-populi-swedish by Birger Moell
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice Vox Populi Swedish
type: common_voice
args: et
metrics:
    - name: Test WER
type: wer
value: 36.951816


# common-voice-vox-populi-swedish


Fine-tuned [facebook/wav2vec2-large-sv-voxpopuli](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Swedish using the [Common Voice](https://huggingface.co/datasets/common_voice)
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("birgermoell/birgermoell/common-voice-vox-populi-swedish")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/common-voice-vox-populi-swedish")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Swedish test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "sv-SE", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("birgermoell/common-voice-vox-populi-swedish")
model = Wav2Vec2ForCTC.from_pretrained("birgermoell/common-voice-vox-populi-swedish")
model.to("cuda")

chars_to_ignore_regex = '[\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\,\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\?\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\!\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\-\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\;\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tspeech_array, sampling_rate = torchaudio.load(batch["path"])
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["speech"] = resampler(speech_array).squeeze().numpy()
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\treturn batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tinputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\twith torch.no_grad():
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tlogits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
    pred_ids = torch.argmax(logits, dim=-1)

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tbatch["pred_strings"] = processor.batch_decode(pred_ids)
\\\\\\\\\\\\\\\\\\\\

```


**Test Result**:
WER: 22.684600


model name: swedish-common-voice-vox-voxpopuli


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, et, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


widget:


- text: "Var kan jag hitta någon <mask> talar engelska?"


Swedish RoBERTa


## Model series


This model is part of a series of models training on TPU with Flax Jax during Huggingface Flax/Jax challenge.


## Gpt models


## Swedish Gpt


## Swedish gpt wiki


# Nordic gpt wiki


## Dansk gpt wiki


## Norsk gpt wiki


## Roberta models


## Nordic Roberta Wiki


## Swe Roberta Wiki Oscar


## Roberta Swedish Scandi


## Roberta Swedish


## Swedish T5 model


model name: roberta-swedish


model author: birgermoell


task: fill-mask


tags: pytorch, jax, tensorboard, roberta, fill-mask, transformers, autonlp_compatible, infinity_compatible


language:


- sv-SE
license: apache-2.0
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
model-index:
- name: wav2vec2-speechdat
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-speechdat


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the COMMON_VOICE - SV-SE dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4578
- Wer: 0.2927


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 15.0
- mixed_precision_training: Native AMP


### Training results