| Model | Layers | Hidden Size | Params | AVG NER micro-f1 (DaNE-testset) | Average Inference Time (Sec/Epoch) | Download |
| --- | --- | --- | --- | ---  | --- | --- |
| Ælæctra Uncased | 12 | 256 | 13.7M | 78.03 (SD = 1.28) | 10.91 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| Ælæctra Cased | 12 | 256 | 14.7M | 80.08 (SD = 0.26) | 10.92 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| DaBERT | 12 | 768 | 110M | 84.89 (SD = 0.64) | 43.03 | [Link for model](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1) |
| mBERT Uncased | 12 | 768 | 167M | 80.44 (SD = 0.82) | 72.10 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip) |
| mBERT Cased | 12 | 768 | 177M | 83.79 (SD = 0.91) | 70.56 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) |


On [DaNE](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020), Ælæctra scores slightly worse than both cased and uncased Multilingual BERT (Devlin et al., 2019) and Danish BERT (Danish BERT, 2019/2020), however, Ælæctra is less than one third the size, and uses significantly fewer computational resources to pretrain and instantiate. For a full description of the evaluation and specification of the model read the thesis: 'Ælæctra - A Step Towards More Efficient Danish Natural Language Processing'.


### Pretraining


To pretrain Ælæctra it is recommended to build a Docker Container from the [Dockerfile](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/). Next, simply follow the [pretraining notebooks](https://github.com/MalteHB/Ælæctra/tree/master/infrastructure/Dockerfile/)


The pretraining was done by utilizing a single NVIDIA Tesla V100 GPU with 16 GiB, endowed by the Danish data company [KMD](https://www.kmd.dk/). The pretraining took approximately 4 days and 9.5 hours for both the cased and uncased model


### Fine-tuning


To fine-tune any Ælæctra model follow the [fine-tuning notebooks](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/)


### References


Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. ArXiv:2003.10555 [Cs]. http://arxiv.org/abs/2003.10555


Danish BERT. (2020). BotXO. https://github.com/botxo/nordic_bert (Original work published 2019)


Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv:1810.04805 [Cs]. http://arxiv.org/abs/1810.04805


Hvingelby, R., Pauli, A. B., Barrett, M., Rosted, C., Lidegaard, L. M., & Søgaard, A. (2020). DaNE: A Named Entity Resource for Danish. Proceedings of the 12th Language Resources and Evaluation Conference, 4597–4604. https://www.aclweb.org/anthology/2020.lrec-1.565


Strømberg-Derczynski, L., Baglini, R., Christiansen, M. H., Ciosici, M. R., Dalsgaard, J. A., Fusaroli, R., Henrichsen, P. J., Hvingelby, R., Kirkedal, A., Kjeldsen, A. S., Ladefoged, C., Nielsen, F. Å., Petersen, M. L., Rystrøm, J. H., & Varab, D. (2020). The Danish Gigaword Project. ArXiv:2005.03521 [Cs]. http://arxiv.org/abs/2005.03521


#### Acknowledgements


As the majority of this repository is build upon [the works](https://github.com/google-research/electra) by the team at Google who created ELECTRA, a HUGE thanks to them is in order.


A Giga thanks also goes out to the incredible people who collected The Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020).


Furthermore, I would like to thank my supervisor [Riccardo Fusaroli](https://github.com/fusaroli) for the support with the thesis, and a special thanks goes out to [Kenneth Enevoldsen](https://github.com/KennethEnevoldsen) for his continuous feedback.


Lastly, i would like to thank KMD, my colleagues from KMD, and my peers and co-students from Cognitive Science for encouriging me to keep on working hard and holding my head up high!


#### Contact


For help or further information feel free to connect with the author Malte Højmark-Bertelsen on [hjb@kmd.dk](mailto:hjb@kmd.dk?subject=[GitHub]%20Ælæctra) or any of the following platforms:


[](https://twitter.com/malteH_B)
[](https://www.linkedin.com/in/malte-h%C3%B8jmark-bertelsen-9a618017b/)
[](https://www.instagram.com/maltemusen/)


<br />


</details>


model name: aelaectra-danish-electra-small-uncased


model author: Maltehb


tags: pytorch, electra, pretraining, da, dataset:DAGW, arxiv:2003.10555, arxiv:1810.04805, arxiv:2005.03521, transformers, ælæctra, danish, ELECTRA-Small, replaced token detection, license:mit, co2_eq_emissions, infinity_compatible


language: "da"
tags:


- ælæctra
- pytorch
- danish
- ELECTRA-Small
- replaced token detection
license: "mit"
datasets:
- DAGW
widget:
- text: "Chili Jensen, som bor på Danmarksgade 12, køber chilifrugter fra Netto."
metrics:
- f1


# Ælæctra - Finetuned for Named Entity Recognition on the [DaNE dataset](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020) by Malte Højmark-Bertelsen.


**Ælæctra** is a Danish Transformer-based language model created to enhance the variety of Danish NLP resources with a more efficient model compared to previous state-of-the-art (SOTA) models.


Ælæctra was pretrained with the ELECTRA-Small (Clark et al., 2020) pretraining approach by using the Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020) and evaluated on Named Entity Recognition (NER) tasks. Since NER only presents a limited picture of Ælæctra's capabilities I am very interested in further evaluations. Therefore, if you employ it for any task, feel free to hit me up your findings!


Ælæctra was, as mentioned, created to enhance the Danish NLP capabilties and please do note how this GitHub still does not support the Danish characters "*Æ, Ø and Å*" as the title of this repository becomes "*-l-ctra*". How ironic.🙂


Here is an example on how to load the finetuned Ælæctra-uncased model for Named Entity Recognition in [PyTorch](https://pytorch.org/) using the [🤗Transformers](https://github.com/huggingface/transformers) library:


```python
from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("Maltehb/-l-ctra-danish-electra-small-uncased-ner-dane")
model = AutoModelForTokenClassification.from_pretrained("Maltehb/-l-ctra-danish-electra-small-uncased-ner-dane")
```


### Evaluation of current Danish Language Models


Ælæctra, Danish BERT (DaBERT) and multilingual BERT (mBERT) were evaluated:


| Model | Layers | Hidden Size | Params | AVG NER micro-f1 (DaNE-testset) | Average Inference Time (Sec/Epoch) | Download |
| --- | --- | --- | --- | ---  | --- | --- |
| Ælæctra Uncased | 12 | 256 | 13.7M | 78.03 (SD = 1.28) | 10.91 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| Ælæctra Cased | 12 | 256 | 14.7M | 80.08 (SD = 0.26) | 10.92 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| DaBERT | 12 | 768 | 110M | 84.89 (SD = 0.64) | 43.03 | [Link for model](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1) |
| mBERT Uncased | 12 | 768 | 167M | 80.44 (SD = 0.82) | 72.10 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip) |
| mBERT Cased | 12 | 768 | 177M | 83.79 (SD = 0.91) | 70.56 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) |


On [DaNE](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020) without the *MISC-tag*, Ælæctra scores slightly worse than both cased and uncased Multilingual BERT (Devlin et al., 2019) and Danish BERT (Danish BERT, 2019/2020), however, Ælæctra is less than one third the size, and uses significantly fewer computational resources to pretrain and instantiate.


### Pretraining


To pretrain Ælæctra it is recommended to build a Docker Container from the [Dockerfile](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/). Next, simply follow the [pretraining notebooks](https://github.com/MalteHB/Ælæctra/tree/master/infrastructure/Dockerfile/)


The pretraining was done by utilizing a single NVIDIA Tesla V100 GPU with 16 GiB, endowed by the Danish data company [KMD](https://www.kmd.dk/). The pretraining took approximately 4 days and 9.5 hours for both the cased and uncased model


### Fine-tuning


To fine-tune any Ælæctra model follow the [fine-tuning notebooks](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/)


### References


Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. ArXiv:2003.10555 [Cs]. http://arxiv.org/abs/2003.10555


Danish BERT. (2020). BotXO. https://github.com/botxo/nordic_bert (Original work published 2019)


Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv:1810.04805 [Cs]. http://arxiv.org/abs/1810.04805


Hvingelby, R., Pauli, A. B., Barrett, M., Rosted, C., Lidegaard, L. M., & Søgaard, A. (2020). DaNE: A Named Entity Resource for Danish. Proceedings of the 12th Language Resources and Evaluation Conference, 4597–4604. https://www.aclweb.org/anthology/2020.lrec-1.565


Strømberg-Derczynski, L., Baglini, R., Christiansen, M. H., Ciosici, M. R., Dalsgaard, J. A., Fusaroli, R., Henrichsen, P. J., Hvingelby, R., Kirkedal, A., Kjeldsen, A. S., Ladefoged, C., Nielsen, F. Å., Petersen, M. L., Rystrøm, J. H., & Varab, D. (2020). The Danish Gigaword Project. ArXiv:2005.03521 [Cs]. http://arxiv.org/abs/2005.03521


#### Acknowledgements


As the majority of this repository is build upon [the works](https://github.com/google-research/electra) by the team at Google who created ELECTRA, a HUGE thanks to them is in order.


A Giga thanks also goes out to the incredible people who collected The Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020).


Furthermore, I would like to thank my supervisor [Riccardo Fusaroli](https://github.com/fusaroli) for the support with the thesis, and a special thanks goes out to [Kenneth Enevoldsen](https://github.com/KennethEnevoldsen) for his continuous feedback.


Lastly, i would like to thank KMD, my colleagues from KMD, and my peers and co-students from Cognitive Science for encouriging me to keep on working hard and holding my head up high!


#### Contact


For help or further information feel free to connect with the author Malte Højmark-Bertelsen on [hjb@kmd.dk](mailto:hjb@kmd.dk?subject=[GitHub]%20ÆlæctraUncasedNER) or any of the following platforms:


[](https://twitter.com/malteH_B)
[](https://www.linkedin.com/in/malte-h%C3%B8jmark-bertelsen-9a618017b/)
[](https://www.instagram.com/maltemusen/)


<br />


</details>


model name: aelaectra-danish-electra-small-uncased-ner-dane


model author: Maltehb


task: token-classification


tags: pytorch, tf, electra, token-classification, da, dataset:DAGW, arxiv:2003.10555, arxiv:1810.04805, arxiv:2005.03521, transformers, ælæctra, danish, ELECTRA-Small, replaced token detection, license:mit, autonlp_compatible, infinity_compatible


language: "da"
co2_eq_emissions: 4009.5
tags:


- ælæctra
- pytorch
- danish
- ELECTRA-Small
- replaced token detection
license: "mit"
datasets:
- DAGW
metrics:
- f1


# Ælæctra - A Step Towards More Efficient Danish Natural Language Processing


**Ælæctra** is a Danish Transformer-based language model created to enhance the variety of Danish NLP resources with a more efficient model compared to previous state-of-the-art (SOTA) models. Initially a cased and an uncased model are released. It was created as part of a Cognitive Science bachelor's thesis.


Ælæctra was pretrained with the ELECTRA-Small (Clark et al., 2020) pretraining approach by using the Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020) and evaluated on Named Entity Recognition (NER) tasks. Since NER only presents a limited picture of Ælæctra's capabilities I am very interested in further evaluations. Therefore, if you employ it for any task, feel free to hit me up your findings!


Ælæctra was, as mentioned, created to enhance the Danish NLP capabilties and please do note how this GitHub still does not support the Danish characters "*Æ, Ø and Å*" as the title of this repository becomes "*-l-ctra*". How ironic.🙂


Here is an example on how to load both the cased and the uncased Ælæctra model in [PyTorch](https://pytorch.org/) using the [🤗Transformers](https://github.com/huggingface/transformers) library:


```python
from transformers import AutoTokenizer, AutoModelForPreTraining

tokenizer = AutoTokenizer.from_pretrained("Maltehb/-l-ctra-danish-electra-small-cased")
model = AutoModelForPreTraining.from_pretrained("Maltehb/-l-ctra-danish-electra-small-cased")
```


```python
from transformers import AutoTokenizer, AutoModelForPreTraining

tokenizer = AutoTokenizer.from_pretrained("Maltehb/-l-ctra-danish-electra-small-uncased")
model = AutoModelForPreTraining.from_pretrained("Maltehb/-l-ctra-danish-electra-small-uncased")
```


### Evaluation of current Danish Language Models


Ælæctra, Danish BERT (DaBERT) and multilingual BERT (mBERT) were evaluated:


| Model | Layers | Hidden Size | Params | AVG NER micro-f1 (DaNE-testset) | Average Inference Time (Sec/Epoch) | Download |
| --- | --- | --- | --- | ---  | --- | --- |
| Ælæctra Uncased | 12 | 256 | 13.7M | 78.03 (SD = 1.28) | 10.91 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| Ælæctra Cased | 12 | 256 | 14.7M | 80.08 (SD = 0.26) | 10.92 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| DaBERT | 12 | 768 | 110M | 84.89 (SD = 0.64) | 43.03 | [Link for model](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1) |
| mBERT Uncased | 12 | 768 | 167M | 80.44 (SD = 0.82) | 72.10 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip) |
| mBERT Cased | 12 | 768 | 177M | 83.79 (SD = 0.91) | 70.56 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) |


On [DaNE](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020), Ælæctra scores slightly worse than both cased and uncased Multilingual BERT (Devlin et al., 2019) and Danish BERT (Danish BERT, 2019/2020), however, Ælæctra is less than one third the size, and uses significantly fewer computational resources to pretrain and instantiate. For a full description of the evaluation and specification of the model read the thesis: 'Ælæctra - A Step Towards More Efficient Danish Natural Language Processing'.


### Pretraining


To pretrain Ælæctra it is recommended to build a Docker Container from the [Dockerfile](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/). Next, simply follow the [pretraining notebooks](https://github.com/MalteHB/Ælæctra/tree/master/infrastructure/Dockerfile/)


The pretraining was done by utilizing a single NVIDIA Tesla V100 GPU with 16 GiB, endowed by the Danish data company [KMD](https://www.kmd.dk/). The pretraining took approximately 4 days and 9.5 hours for both the cased and uncased model


### Fine-tuning


To fine-tune any Ælæctra model follow the [fine-tuning notebooks](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/)


### References


Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. ArXiv:2003.10555 [Cs]. http://arxiv.org/abs/2003.10555


Danish BERT. (2020). BotXO. https://github.com/botxo/nordic_bert (Original work published 2019)


Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv:1810.04805 [Cs]. http://arxiv.org/abs/1810.04805


Hvingelby, R., Pauli, A. B., Barrett, M., Rosted, C., Lidegaard, L. M., & Søgaard, A. (2020). DaNE: A Named Entity Resource for Danish. Proceedings of the 12th Language Resources and Evaluation Conference, 4597–4604. https://www.aclweb.org/anthology/2020.lrec-1.565


Strømberg-Derczynski, L., Baglini, R., Christiansen, M. H., Ciosici, M. R., Dalsgaard, J. A., Fusaroli, R., Henrichsen, P. J., Hvingelby, R., Kirkedal, A., Kjeldsen, A. S., Ladefoged, C., Nielsen, F. Å., Petersen, M. L., Rystrøm, J. H., & Varab, D. (2020). The Danish Gigaword Project. ArXiv:2005.03521 [Cs]. http://arxiv.org/abs/2005.03521


#### Acknowledgements


As the majority of this repository is build upon [the works](https://github.com/google-research/electra) by the team at Google who created ELECTRA, a HUGE thanks to them is in order.


A Giga thanks also goes out to the incredible people who collected The Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020).


Furthermore, I would like to thank my supervisor [Riccardo Fusaroli](https://github.com/fusaroli) for the support with the thesis, and a special thanks goes out to [Kenneth Enevoldsen](https://github.com/KennethEnevoldsen) for his continuous feedback.


Lastly, i would like to thank KMD, my colleagues from KMD, and my peers and co-students from Cognitive Science for encouriging me to keep on working hard and holding my head up high!


#### Contact


For help or further information feel free to connect with the author Malte Højmark-Bertelsen on [hjb@kmd.dk](mailto:hjb@kmd.dk?subject=[GitHub]%20Ælæctra) or any of the following platforms:


[](https://twitter.com/malteH_B)
[](https://www.linkedin.com/in/malte-h%C3%B8jmark-bertelsen-9a618017b/)
[](https://www.instagram.com/maltemusen/)


<br />


</details>


model name: aelaectra-danish-electra-small-cased


model author: Maltehb


tags: pytorch, tf, electra, pretraining, da, dataset:DAGW, arxiv:2003.10555, arxiv:1810.04805, arxiv:2005.03521, transformers, ælæctra, danish, ELECTRA-Small, replaced token detection, license:mit, co2_eq_emissions, infinity_compatible


language: da
tags:


- danish
- bert
- masked-lm
- Certainly
license: cc-by-4.0
datasets:
- common_crawl
- wikipedia
- dindebat.dk
- hestenettet.dk
- danish OpenSubtitles
pipeline_tag: fill-mask
widget:
- text: "København er [MASK] i Danmark."


# Danish BERT (version 2, uncased) by [Certainly](https://certainly.io/) (previously known as BotXO).


All credit goes to [Certainly](https://certainly.io/) (previously known as BotXO), who developed Danish BERT. For data and training details see their [GitHub repository](https://github.com/certainlyio/nordic_bert) or [this article](https://www.certainly.io/blog/danish-bert-model/). You can also visit their [organization page](https://huggingface.co/Certainly) on Hugging Face.


It is both available in TensorFlow and Pytorch format.


The original TensorFlow version can be downloaded using [this link](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1).


Here is an example on how to load Danish BERT in PyTorch using the [🤗Transformers](https://github.com/huggingface/transformers) library:


```python
from transformers import AutoTokenizer, AutoModelForPreTraining

tokenizer = AutoTokenizer.from_pretrained("Maltehb/danish-bert-botxo")
model = AutoModelForPreTraining.from_pretrained("Maltehb/danish-bert-botxo")

```


model name: danish-bert-botxo


task: fill-mask


tags: pytorch, jax, bert, pretraining, da, dataset:common_crawl, dataset:wikipedia, dataset:dindebat.dk, dataset:hestenettet.dk, dataset:danish OpenSubtitles, transformers, danish, masked-lm, Certainly, license:cc-by-4.0, fill-mask, autonlp_compatible, infinity_compatible


- danish
- bert
- masked-lm
- botxo
license: cc-by-4.0
datasets:
- common_crawl
- wikipedia
- dindebat.dk
- hestenettet.dk
- danish OpenSubtitles
widget:
- text: "Chili Jensen, som bor på Danmarksgade 12, køber chilifrugter fra Netto."


# Danish BERT (version 2, uncased) by [Certainly](https://certainly.io/) (previously known as BotXO) finetuned for Named Entity Recognition on the [DaNE dataset](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020) by Malte Højmark-Bertelsen.


Humongous amounts of credit needs to go to [Certainly](https://certainly.io/) (previously known as BotXO), for pretraining the Danish BERT. For data and training details see their [GitHub repository](https://github.com/certainlyio/nordic_bert) or [this article](https://www.certainly.io/blog/danish-bert-model/). You can also visit their [organization page](https://huggingface.co/Certainly) on Hugging Face.


It is both available in TensorFlow and Pytorch format.
The original TensorFlow version can be downloaded using [this link](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1).


Here is an example on how to load Danish BERT for token classification in PyTorch using the [🤗Transformers](https://github.com/huggingface/transformers) library:


```python
from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("Maltehb/danish-bert-botxo-ner-dane")
model = AutoModelForTokenClassification.from_pretrained("Maltehb/danish-bert-botxo-ner-dane")

```


### References


Danish BERT. (2020). BotXO. https://github.com/botxo/nordic_bert (Original work published 2019)


Hvingelby, R., Pauli, A. B., Barrett, M., Rosted, C., Lidegaard, L. M., & Søgaard, A. (2020). DaNE: A Named Entity Resource for Danish. Proceedings of the 12th Language Resources and Evaluation Conference, 4597–4604. https://www.aclweb.org/anthology/2020.lrec-1.565


#### Contact


For help or further information feel free to connect with the author Malte Højmark-Bertelsen on [hjb@kmd.dk](mailto:hjb@kmd.dk?subject=[GitHub]%20DanishBERTUncasedNER) or any of the following platforms:


[](https://twitter.com/malteH_B)
[](https://www.linkedin.com/in/malte-h%C3%B8jmark-bertelsen-9a618017b/)
[](https://www.instagram.com/maltemusen/)


<br />


</details>


model name: danish-bert-botxo-ner-dane


model author: Maltehb


task: token-classification


tags: pytorch, tf, jax, bert, token-classification, da, dataset:common_crawl, dataset:wikipedia, dataset:dindebat.dk, dataset:hestenettet.dk, dataset:danish OpenSubtitles, transformers, danish, masked-lm, botxo, license:cc-by-4.0, autonlp_compatible, infinity_compatible


language: "da"
tags:


- ælæctra
- pytorch
- danish
- ELECTRA-Small
- replaced token detection
license: "mit"
datasets:
- DAGW
widget:
- text: "Chili Jensen, som bor på Danmarksgade 12, køber chilifrugter fra Netto."
metrics:
- f1


# Ælæctra - Finetuned for Named Entity Recognition on the [DaNE dataset](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020) by Malte Højmark-Bertelsen.


**Ælæctra** is a Danish Transformer-based language model created to enhance the variety of Danish NLP resources with a more efficient model compared to previous state-of-the-art (SOTA) models.


Ælæctra was pretrained with the ELECTRA-Small (Clark et al., 2020) pretraining approach by using the Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020) and evaluated on Named Entity Recognition (NER) tasks. Since NER only presents a limited picture of Ælæctra's capabilities I am very interested in further evaluations. Therefore, if you employ it for any task, feel free to hit me up your findings!


Ælæctra was, as mentioned, created to enhance the Danish NLP capabilties and please do note how this GitHub still does not support the Danish characters "*Æ, Ø and Å*" as the title of this repository becomes "*-l-ctra*". How ironic.🙂


Here is an example on how to load the finetuned Ælæctra-cased model for Named Entity Recognition in [PyTorch](https://pytorch.org/) using the [🤗Transformers](https://github.com/huggingface/transformers) library:


```python
from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("Maltehb/-l-ctra-danish-electra-small-cased-ner-dane")
model = AutoModelForTokenClassification.from_pretrained("Maltehb/-l-ctra-danish-electra-small-cased-ner-dane")
```


### Evaluation of current Danish Language Models


Ælæctra, Danish BERT (DaBERT) and multilingual BERT (mBERT) were evaluated:


| Model | Layers | Hidden Size | Params | AVG NER micro-f1 (DaNE-testset) | Average Inference Time (Sec/Epoch) | Download |
| --- | --- | --- | --- | ---  | --- | --- |
| Ælæctra Uncased | 12 | 256 | 13.7M | 78.03 (SD = 1.28) | 10.91 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| Ælæctra Cased | 12 | 256 | 14.7M | 80.08 (SD = 0.26) | 10.92 | [Link for model](https://www.dropbox.com/s/cag7prs1nvdchqs/%C3%86l%C3%A6ctra.zip?dl=0) |
| DaBERT | 12 | 768 | 110M | 84.89 (SD = 0.64) | 43.03 | [Link for model](https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1) |
| mBERT Uncased | 12 | 768 | 167M | 80.44 (SD = 0.82) | 72.10 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip) |
| mBERT Cased | 12 | 768 | 177M | 83.79 (SD = 0.91) | 70.56 | [Link for model](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip) |


On [DaNE](https://danlp.alexandra.dk/304bd159d5de/datasets/ddt.zip) (Hvingelby et al., 2020) without the *MISC-tag*, Ælæctra scores slightly worse than both cased and uncased Multilingual BERT (Devlin et al., 2019) and Danish BERT (Danish BERT, 2019/2020), however, Ælæctra is less than one third the size, and uses significantly fewer computational resources to pretrain and instantiate.


### Pretraining


To pretrain Ælæctra it is recommended to build a Docker Container from the [Dockerfile](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/). Next, simply follow the [pretraining notebooks](https://github.com/MalteHB/Ælæctra/tree/master/infrastructure/Dockerfile/)


The pretraining was done by utilizing a single NVIDIA Tesla V100 GPU with 16 GiB, endowed by the Danish data company [KMD](https://www.kmd.dk/). The pretraining took approximately 4 days and 9.5 hours for both the cased and uncased model


### Fine-tuning


To fine-tune any Ælæctra model follow the [fine-tuning notebooks](https://github.com/MalteHB/Ælæctra/tree/master/notebooks/fine-tuning/)


### References


Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. ArXiv:2003.10555 [Cs]. http://arxiv.org/abs/2003.10555


Danish BERT. (2020). BotXO. https://github.com/botxo/nordic_bert (Original work published 2019)


Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv:1810.04805 [Cs]. http://arxiv.org/abs/1810.04805


Hvingelby, R., Pauli, A. B., Barrett, M., Rosted, C., Lidegaard, L. M., & Søgaard, A. (2020). DaNE: A Named Entity Resource for Danish. Proceedings of the 12th Language Resources and Evaluation Conference, 4597–4604. https://www.aclweb.org/anthology/2020.lrec-1.565


Strømberg-Derczynski, L., Baglini, R., Christiansen, M. H., Ciosici, M. R., Dalsgaard, J. A., Fusaroli, R., Henrichsen, P. J., Hvingelby, R., Kirkedal, A., Kjeldsen, A. S., Ladefoged, C., Nielsen, F. Å., Petersen, M. L., Rystrøm, J. H., & Varab, D. (2020). The Danish Gigaword Project. ArXiv:2005.03521 [Cs]. http://arxiv.org/abs/2005.03521


#### Acknowledgements


As the majority of this repository is build upon [the works](https://github.com/google-research/electra) by the team at Google who created ELECTRA, a HUGE thanks to them is in order.


A Giga thanks also goes out to the incredible people who collected The Danish Gigaword Corpus (Strømberg-Derczynski et al., 2020).


Furthermore, I would like to thank my supervisor [Riccardo Fusaroli](https://github.com/fusaroli) for the support with the thesis, and a special thanks goes out to [Kenneth Enevoldsen](https://github.com/KennethEnevoldsen) for his continuous feedback.


Lastly, i would like to thank KMD, my colleagues from KMD, and my peers and co-students from Cognitive Science for encouriging me to keep on working hard and holding my head up high!


#### Contact


For help or further information feel free to connect with the author Malte Højmark-Bertelsen on [hjb@kmd.dk](mailto:hjb@kmd.dk?subject=[GitHub]%20ÆlæctraCasedNER) or any of the following platforms:


[](https://twitter.com/malteH_B)
[](https://www.linkedin.com/in/malte-h%C3%B8jmark-bertelsen-9a618017b/)
[](https://www.instagram.com/maltemusen/)


<br />


</details>


model name: aelaectra-danish-electra-small-cased-ner-dane


model author: Maltehb


task: token-classification


tags: pytorch, tf, electra, token-classification, da, dataset:DAGW, arxiv:2003.10555, arxiv:1810.04805, arxiv:2005.03521, transformers, ælæctra, danish, ELECTRA-Small, replaced token detection, license:mit, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_keras_callback
model-index:
- name: ms29315/distilbert-base-uncased-finetuned-cola
results: []


<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->


# ms29315/distilbert-base-uncased-finetuned-cola


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:


- Train Loss: 0.3100
- Validation Loss: 0.5090
- Epoch: 0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- optimizer: {'name': 'Adam', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 2670, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32


### Training results


| Train Loss | Validation Loss | Epoch |
|:----------:|:---------------:|:-----:|
| 0.3100     | 0.5090          | 0     |


### Framework versions


- Transformers 4.15.0
- TensorFlow 2.7.0
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: distilbert-base-uncased-finetuned-cola


model author: ms29315


task: text-classification


tags: tf, tensorboard, distilbert, text-classification, transformers, license:apache-2.0, generated_from_keras_callback, model-index, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: distilgpt2-finetuned-wikitext2
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilgpt2-finetuned-wikitext2


This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 3.6424


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 3.7608        | 1.0   | 2334 | 3.6655          |
| 3.6335        | 2.0   | 4668 | 3.6455          |
| 3.6066        | 3.0   | 7002 | 3.6424          |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: distilgpt2-finetuned-wikitext2


model author: nakamura196


task: text-generation


tags: pytorch, tensorboard, gpt2, text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index


tags:


- conversational


#Harry Potter DialoGPT-medium Model


model name: DialoGPT-medium-harrypotterbot


model author: minsiam


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


#Harry Potter DialoGPT Model


model name: DialoGPT-small-harrypotterbot


## language: zh


model name: bert-base-chinese-jinyong


model author: yechen


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, zh, transformers, autonlp_compatible, infinity_compatible


model name: bert-large-chinese


model name: bert-base-chinese


tags: pytorch, tf, fill-mask, zh, transformers, autonlp_compatible


model name: question-answering-chinese


task: question-answering


tags: pytorch, tf, jax, bert, question-answering, zh, transformers, autonlp_compatible, infinity_compatible


this project was created to use in wav2vec


model name: testtest1


model author: Niciu


tags:


## language: multilingual
thumbnail: "url to a thumbnail used in social sharing"
tags: exbert
license: apache-2.0


# XLM-RoBERTa-large-sag


## Model description


This is a model based on the [XLM-RoBERTa large](https://huggingface.co/xlm-roberta-large) topology (provided by Facebook, see original [paper](https://arxiv.org/abs/1911.02116)) with additional training on two sets of medicine-domain texts:


* about 250.000 text reviews on medicines (1000-tokens-long in average) collected from the site irecommend.ru;
* the raw part of the [RuDReC corpus](https://github.com/cimm-kzn/RuDReC) (about 1.4 million texts, see [paper](https://arxiv.org/abs/2004.03659)).


The XLM-RoBERTa-large calculations for one epoch on this data were performed using one Nvidia Tesla v100 and the Huggingface Transformers library.


## BibTeX entry and citation info


If you have found our results helpful in your work, feel free to cite our publication as:


```
@article{sboev2021analysis,
  title={An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets},
  author={Sboev, Alexander and Sboeva, Sanna and Moloshnikov, Ivan and Gryaznov, Artem and Rybka, Roman and Naumov, Alexander and Selivanov, Anton and Rylkov, Gleb and Ilyin, Viacheslav},
  journal={arXiv preprint arXiv:2105.00059},
  year={2021}
}
```


model name: xlm-roberta-large-sag


model author: sagteam


task: fill-mask


tags: pytorch, xlm-roberta, fill-mask, multilingual, arxiv:1911.02116, arxiv:2004.03659, transformers, exbert, license:apache-2.0, autonlp_compatible


# pharm-relation-extraction


Model trained to recognize 4 types of relationships between significant pharmacological entities in russian-language reviews: ADR–Drugname, Drugname–Diseasename, Drugname–SourceInfoDrug, Diseasename–Indication. The input of the model is a review text and a pair of entities, between which it is required to determine the fact of a relationship and one of the 4 types of relationship, listed above.


## Data


Proposed model is trained on a subset of 908 reviews of the [Russian Drug Review Corpus (RDRS)](https://arxiv.org/pdf/2105.00059.pdf). The subset contains the pairs of entities marked with the 4 listed types of relationships:


- ADR-Drugname — the relationship between the drug and its side effects
- Drugname-SourceInfodrug — the relationship between the medication and the source of information about it (e.g., “was advised at the pharmacy”, e.g., “was advised at the pharmacy”, “the doctor recommended it”);
- Drugname-Diseasname — the relationship between the drug and the disease
- Diseasename-Indication — the connection between the illness and its symptoms (e.g., “cough”, “fever 39 degrees”)
Also, this subset contains pairs of the same entity types between which there is no relationship: for example, a drug and an unrelated side effect that appeared after taking another drug; in other words, this side effect is related to another drug.


## Model topology and training


Proposed model is based on the  [XLM-RoBERTA-large](https://arxiv.org/abs/1911.02116) topology. After the additional training as a language model on corpus of unmarked drug reviews, this model was trained as a classification model on 80% of the texts from subset of the corps described above.


## How to use


See section "How to use" in [our git repository for the model](https://github.com/sag111/Relation_Extraction)


## Results


Here are the accuracy, estimated by the f1 score metric for the recognition of relationships on the best fold.


| ADR–Drugname  | Drugname–Diseasename | Drugname–SourceInfoDrug | Diseasename–Indication |
| ------------- | -------------------- | ----------------------- | ---------------------- |
| 0.955         | 0.892                | 0.922                   | 0.891                  |


## Citation info


```
@article{sboev2021extraction,
  title={Extraction of the Relations between Significant Pharmacological Entities in Russian-Language Internet Reviews on Medications},
  author={Sboev, Alexander and Selivanov, Anton and Moloshnikov, Ivan and Rybka, Roman and Gryaznov, Artem and Sboeva, Sanna and Rylkov, Gleb},
  year={2021},
  publisher={Preprints}
}
```


model name: pharm-relation-extraction


model author: sagteam


task: text-classification


tags: pytorch, xlm-roberta, text-classification, arxiv:2105.00059, arxiv:1911.02116, transformers


# COVID-twitter-XLM-Roberta-large


## Model description


This is a model based on the [XLM-RoBERTa large](https://huggingface.co/xlm-roberta-large) topology (provided by Facebook, see original [paper](https://arxiv.org/abs/1911.02116)) with additional training on a corpus of unmarked tweets.


For more details, please see, our [GitHub repository](https://github.com/sag111/COVID-19-tweets-Russia).


## Training data


We formed a corpus of unlabeled twitter messages.


The data on keyword "covid" was expanded with texts containing  other words often occurred in hashtags on the Covid-19 pandemic: "covid", "stayhome", and "coronavirus" (hereinafter, these are translations of Russian words into English).


Separately, messages were collected from Twitter users from large regions of Russia. The search was provided using different word forms of 58 manually selected keywords on Russian related to the topic of coronavirus infection (including: "PCR", "pandemic", "self-isolation", etc.).


The unlabeled corpus includes all unique Russian-language tweets from the collected data (>1M tweets). Since modern language models are usually multilingual, about 1M more tweets in other languages were added to this corpus using filtering procedures described above. Thus, in the unlabeled part of the collected data, there were about 2 million messages.


### BibTeX entry and citation info


Our GitHub repository: https://github.com/sag111/COVID-19-tweets-Russia


If you have found our results helpful in your work, feel free to cite our publication and this repository as:


```
coming soon
```


model name: covid-twitter-xlm-roberta-large


model author: sagteam


task: fill-mask


tags: pytorch, xlm-roberta, fill-mask, arxiv:1911.02116, transformers, autonlp_compatible


##An MT5ForConditionalGeneration trained on 3 tasks from PAN Profiling Hate Speech Spreaders on Twitter dataset (ES):


* topic attribution - topics were assigned with BertTopic library using embeddings from `Hate-speech-CNERG/dehatebert-mono-spanish` bert model (train and test sets from the PAN task)
* hate speech identification (train set from the PAN task)


in order to generate tone of comment use prefix **hater classification:**


model name: PAN_twitter_hate_speech_2021_ES_MT5


model author: PaulAdversarial


task: text2text-generation


tags: pytorch, mt5, text2text-generation, transformers, autonlp_compatible


##A T5ForConditionalGeneration trained on 3 tasks from PAN Profiling Hate Speech Spreaders on Twitter dataset (EN):


* author attribution (train and test sets from the PAN task)
* topic attribution - topics were assigned with BertTopic library using embeddings from `cardiffnlp/bertweet-base-hate` Roberta model (train and test sets from the PAN task)
* hate speech identification (train set from the PAN task)


model name: T5_PAN_Hate_Speech_Twitter_topic_author_ishatespeach


tags: pytorch, jax, t5, text2text-generation, transformers, autonlp_compatible


A T5ForConditionalGeneration trained on 2 tasks from PAN Profiling Hate Speech Spreaders on Twitter dataset (EN):


* topic attribution - topics were assigned with BertTopic library using embeddings from `cardiffnlp/bertweet-base-hate` Roberta model (train and test sets from the PAN task)
* hate speech identification (train set from the PAN task)


model name: T5_PAN_Hate_Speech_Twitter_topic_ishatespeach


Testing


model name: test2


model author: luisperez123


tags:


- conversational


This is Chandler.
Chandler is your friend too.


model name: DialoGPT-small-chandler


model author: Leonel


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language: ar
widget:


- text: "ممتاز"
- text: "أنا حزين"
- text: "لا شيء"


# Model description


This model is an Arabic language sentiment analysis pretrained model.
The model is built on top of the CAMelBERT_msa_sixteenth BERT-based model.
We used the HARD dataset of hotels review to fine tune the model.
The dataset original labels based on a five-star rating were modified to a 3 label data:


- POSITIVE: for ratings > 3 stars
- NEUTRAL: for a 3 star rating
- NEGATIVE: for ratings < 3 stars


This first prototype was trained on 3 epochs for 1 hours using Colab and a TPU acceleration.


# Examples


Here are some examples in Arabic to test :


- Excellent -> ممتاز(Happy)
- I'am sad -> أنا حزين (Sad)
- Nothing -> لا شيء (Neutral)


# Contact


If you have questions or improvement remarks, feel free to contact me on my LinkedIn profile: https://www.linkedin.com/in/yahya-ghrab/


model name: Sentiment_Analysis_CAMelBERT_msa_sixteenth_HARD


model author: Yah216


task: text-classification


tags: tf, bert, text-classification, ar, transformers, infinity_compatible


tags:


- conversational


# AlchemyBot DialoGPT Model


model name: DialoGPT-small-AlchemyBot


model author: pandyaved98


task: conversational


# GameBot DialoGPT Model


model name: DialoGPT-small-GameBot


- asteroid
- audio
- ConvTasNet
- audio-to-audio
datasets:
- libri1mix
- enh_single
license: cc-by-sa-4.0


## Asteroid model `mhu-coder/ConvTasNet_Libri1Mix_enhsingle`


Imported from [Zenodo](https://zenodo.org/record/4301955#.X9cj98Jw0bY)


### Description:


This model was trained by Mathieu Hu using the librimix/ConvTasNet recipe in
[Asteroid](https://github.com/asteroid-team/asteroid).
It was trained on the `enh_single` task of the Libri1Mix dataset.


### Training config:


```yaml
data:
    n_src: 1
    sample_rate: 16000
    segment: 3
    task: enh_single
    train_dir: data/wav16k/min/train-100
    valid_dir: data/wav16k/min/dev
filterbank:
    kernel_size: 16
    n_filters: 512
    stride: 8
main_args:
    exp_dir: exp/train_convtasnet_f34664b9
    help: None
masknet:
    bn_chan: 128
    hid_chan: 512
    mask_act: relu
    n_blocks: 8
    n_repeats: 3
    n_src: 1
    skip_chan: 128
optim:
    lr: 0.001
    optimizer: adam
    weight_decay: 0.0
positional arguments:
training:
    batch_size: 2
    early_stop: True
    epochs: 200
    half_lr: True
    num_workers: 4
```


### Results:


```yaml
si_sdr: 13.938355526049932
si_sdr_imp: 10.488574220190232
sdr: 14.567380104207393
sdr_imp: 11.064717304994337
sir: inf
sir_imp: nan
sar: 14.567380104207393
sar_imp: 11.064717304994337
stoi: 0.9201010933251715
stoi_imp: 0.1241812697846321
```


### License notice:


This work "ConvTasNet_Libri1Mx_enhsingle" is a derivative of [CSR-I (WSJ0) Complete](https://catalog.ldc.upenn.edu/LDC93S6A)
by [LDC](https://www.ldc.upenn.edu/), used under [LDC User Agreement for
Non-Members](https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf) (Research only).
"ConvTasNet_Libri1Mix_enhsingle" is licensed under [Attribution-ShareAlike 3.0 Unported](https://creativecommons.org/licenses/by-sa/3.0/)
by Mathieu Hu.


model name: ConvTasNet_Libri1Mix_enhsingle


model author: mhu-coder


task: audio-to-audio


tags: pytorch, dataset:libri1mix, dataset:enh_single, asteroid, audio, ConvTasNet, audio-to-audio, license:cc-by-sa-4.0


## license: mit


How to clone :


```
git lfs install
git clone https://huggingface.co/Rajan/Nepali_Word2Vec
```


model name: Nepali_Word2Vec


model author: Rajan


tags: license:mit


# NepaliBERT(Phase 1)


NEPALIBERT is a state-of-the-art language model for Nepali based on the BERT model. The model is trained using a masked language modeling (MLM).


# Loading the model and tokenizer


1. clone the model repo


```
git lfs install
git clone https://huggingface.co/Rajan/NepaliBERT
```


2. Loading the Tokenizer


```
from transformers import BertTokenizer
vocab_file_dir = './NepaliBERT/'
tokenizer = BertTokenizer.from_pretrained(vocab_file_dir,
                                        strip_accents=False,
                                         clean_text=False )
```


3. Loading the model:


```
from transformers import BertForMaskedLM
model = BertForMaskedLM.from_pretrained('./NepaliBERT')
```


The easiest way to check whether our language model is learning anything interesting is via the `FillMaskPipeline`.


Pipelines are simple wrappers around tokenizers and models, and the 'fill-mask' one will let you input a sequence containing a masked token (here, [mask]) and return a list of the most probable filled sequences, with their probabilities.


```
from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model=model,
    tokenizer=tokenizer
)
```


For more info visit the [GITHUB🤗](https://github.com/R4j4n/NepaliBERT)


model name: NepaliBERT


task: fill-mask


tags: pytorch, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


language:


- fi
license: apache-2.0
tags:
- generated_from_trainer
- automatic-speech-recognition
- robust-speech-event
model-index:
- name: wav2vec2-xlsr-fi-lm-1B
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xlsr-fi-lm-1B


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the common voice train/dev/other datasets.
It achieves the following results on the evaluation set without language model:


- Loss: 0.1853
- Wer: 0.2205


With language model:


- Wer: 0.1026


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.8158        | 0.67  | 400  | 0.4835          | 0.6310 |
| 0.5679        | 1.33  | 800  | 0.4806          | 0.5538 |
| 0.6055        | 2.0   | 1200 | 0.3888          | 0.5083 |
| 0.5353        | 2.67  | 1600 | 0.3258          | 0.4365 |
| 0.4883        | 3.33  | 2000 | 0.3313          | 0.4204 |
| 0.4513        | 4.0   | 2400 | 0.2924          | 0.3904 |
| 0.3753        | 4.67  | 2800 | 0.2593          | 0.3608 |
| 0.3478        | 5.33  | 3200 | 0.2832          | 0.3551 |
| 0.3796        | 6.0   | 3600 | 0.2495          | 0.3402 |
| 0.2556        | 6.67  | 4000 | 0.2342          | 0.3106 |
| 0.229         | 7.33  | 4400 | 0.2181          | 0.2812 |
| 0.205         | 8.0   | 4800 | 0.2041          | 0.2523 |
| 0.1654        | 8.67  | 5200 | 0.2015          | 0.2416 |
| 0.152         | 9.33  | 5600 | 0.1942          | 0.2294 |
| 0.1569        | 10.0  | 6000 | 0.1853          | 0.2205 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


model name: wav2vec2-xlsr-fi-lm-1B


model author: RASMUS


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, fi, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language: et
datasets:


- mozilla-foundation/common_voice_8_0
metrics:
- wer
- cer
tags:
- generated_from_trainer
- mozilla-foundation/common_voice_8_0
- audio
- automatic-speech-recognition
- speech
- robust-speech-event
model-index:
- name: XLS-R 1B Wav2Vec2 Estonian by Rasmus Toivanen
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 8
type: mozilla-foundation/common_voice_8_0
args: et
metrics:
    - name: Test WER
type: wer
value: 20.12
    - name: Test CER
type: cer
value: 3.82
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: et
metrics:
    - name: Test WER
type: wer
value: 40.77
    - name: Test CER
type: cer
value: 12.32


# wav2vec2-xlsr-et-lm-1B


This model was finetuned with mozilla_foundation/common_voice_8_0 et with train+other+validation splits.
It achieves the following results on the test set:
(Loss reported with last eval step at step 2000/2040 during training)


- Loss: 0.2150
- Wer: 0.2012


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.00005
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 1
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: wav2vec2-xlsr-1b-et


model author: RASMUS


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, et, dataset:mozilla-foundation/common_voice_8_0, transformers, generated_from_trainer, mozilla-foundation/common_voice_8_0, audio, speech, robust-speech-event, model-index


language: ru
datasets:


- mozilla-foundation/common_voice_8_0
metrics:
- wer
- cer
tags:
- generated_from_trainer
- mozilla-foundation/common_voice_8_0
- audio
- automatic-speech-recognition
- speech
- robust-speech-event
model-index:
- name: XLS-R 1B Wav2Vec2 Russian by Rasmus Toivanen
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 8
type: mozilla-foundation/common_voice_8_0
args: ru
metrics:
    - name: Test WER
type: wer
value: 10.83
    - name: Test CER
type: cer
value: 2.41
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Dev Data
type: speech-recognition-community-v2/dev_data
args: ru
metrics:
    - name: Test WER
type: wer
value: 37.71
    - name: Test CER
type: cer
value: 12.98


# wav2vec2-xlsr-1b-ru


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1352
- Wer: 0.0971


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 0.5462        | 0.35  | 500   | 0.4027          | 0.3575 |
| 0.498         | 0.69  | 1000  | 0.2588          | 0.2513 |
| 0.4279        | 1.04  | 1500  | 0.2265          | 0.2204 |
| 0.4099        | 1.38  | 2000  | 0.2189          | 0.1979 |
| 0.4688        | 1.73  | 2500  | 0.2100          | 0.1920 |
| 0.2241        | 2.07  | 3000  | 0.1980          | 0.1767 |
| 0.2056        | 2.42  | 3500  | 0.2020          | 0.1683 |
| 0.3423        | 2.76  | 4000  | 0.1862          | 0.1606 |
| 0.2478        | 3.11  | 4500  | 0.1787          | 0.1563 |
| 0.3079        | 3.45  | 5000  | 0.1759          | 0.1555 |
| 0.2477        | 3.8   | 5500  | 0.1713          | 0.1423 |
| 0.1718        | 4.14  | 6000  | 0.1695          | 0.1391 |
| 0.1675        | 4.49  | 6500  | 0.1677          | 0.1372 |
| 0.1631        | 4.83  | 7000  | 0.1652          | 0.1333 |
| 0.1429        | 5.18  | 7500  | 0.1605          | 0.1308 |
| 0.1505        | 5.52  | 8000  | 0.1612          | 0.1245 |
| 0.1385        | 5.87  | 8500  | 0.1487          | 0.1225 |
| 0.1285        | 6.22  | 9000  | 0.1526          | 0.1201 |
| 0.1153        | 6.56  | 9500  | 0.1464          | 0.1172 |
| 0.1159        | 6.91  | 10000 | 0.1505          | 0.1143 |
| 0.1061        | 7.25  | 10500 | 0.1444          | 0.1106 |
| 0.1016        | 7.6   | 11000 | 0.1427          | 0.1075 |
| 0.1125        | 7.94  | 11500 | 0.1386          | 0.1045 |
| 0.0937        | 8.29  | 12000 | 0.1403          | 0.1022 |
| 0.1059        | 8.63  | 12500 | 0.1406          | 0.1022 |
| 0.0857        | 8.98  | 13000 | 0.1372          | 0.0992 |
| 0.0901        | 9.32  | 13500 | 0.1380          | 0.0977 |
| 0.0913        | 9.67  | 14000 | 0.1352          | 0.0971 |


### Framework versions


model name: wav2vec2-xlsr-1b-ru


model author: RASMUS


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, ru, dataset:mozilla-foundation/common_voice_8_0, transformers, generated_from_trainer, mozilla-foundation/common_voice_8_0, audio, speech, robust-speech-event, model-index


language: fi
datasets:


- mozilla-foundation/common_voice_7_0
metrics:
- wer
- cer
tags:
- generated_from_trainer
- mozilla-foundation/common_voice_7_0
- audio
- automatic-speech-recognition
- speech
- robust-speech-event
model-index:
- name: XLS-R 1B Wav2Vec2 Finnish by Rasmus Toivanen
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 7
type: mozilla-foundation/common_voice_7_0
args: fi
metrics:
    - name: Test WER
type: wer
value: 10.96
    - name: Test CER
type: cer
value: 2.81


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xlsr-fi-train-aug-lm-1B


This model was trained from scratch on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1499
- Wer: 0.1955


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 4
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.6473        | 0.29  | 400  | 0.2857          | 0.3825 |
| 0.6039        | 0.58  | 800  | 0.2459          | 0.3476 |
| 0.4757        | 0.87  | 1200 | 0.2338          | 0.3274 |
| 0.4473        | 1.15  | 1600 | 0.2246          | 0.3128 |
| 0.4322        | 1.44  | 2000 | 0.1962          | 0.2805 |
| 0.3961        | 1.73  | 2400 | 0.2070          | 0.2797 |
| 0.3642        | 2.02  | 2800 | 0.1790          | 0.2473 |
| 0.3561        | 2.31  | 3200 | 0.1769          | 0.2375 |
| 0.282         | 2.6   | 3600 | 0.1672          | 0.2263 |
| 0.2978        | 2.89  | 4000 | 0.1636          | 0.2192 |
| 0.2722        | 3.17  | 4400 | 0.1637          | 0.2102 |
| 0.2924        | 3.46  | 4800 | 0.1506          | 0.2021 |
| 0.2631        | 3.75  | 5200 | 0.1499          | 0.1955 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


model name: wav2vec2-xlsr-fi-train-aug-lm-1B


model author: RASMUS


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, fi, dataset:mozilla-foundation/common_voice_7_0, transformers, generated_from_trainer, mozilla-foundation/common_voice_7_0, audio, speech, robust-speech-event, model-index


language: fi
datasets:


- mozilla-foundation/common_voice_7_0
metrics:
- wer
- cer
tags:
- generated_from_trainer
- mozilla-foundation/common_voice_7_0
- audio
- automatic-speech-recognition
- speech
model-index:
- name: XLS-R 1B Wav2Vec2 Finnish by Rasmus Toivanen
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 7
type: mozilla-foundation/common_voice_7_0
args: fi


# wav2vec2-xlsr-fi-train-aug-lm-1B


- Loss: 0.1499
- Wer: 0.1955


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


### Framework versions


model name: wav2vec2-xlsr-fi-train-aug-bigLM-1B


tags: pytorch, wav2vec2, automatic-speech-recognition, fi, dataset:mozilla-foundation/common_voice_7_0, transformers, generated_from_trainer, mozilla-foundation/common_voice_7_0, audio, speech, model-index


tags:


- image-classification
- pytorch
- huggingpics
metrics:
- accuracy


model-index:


- name: pollution
results:
  - task:
name: Image Classification
type: image-classification
metrics:
    - name: Accuracy
type: accuracy
value: 0.7129629850387573


# pollution


Autogenerated by HuggingPics🤗🖼


Create your own image classifier for **anything** by running [the demo on Google Colab](https://colab.research.google.com/github/nateraw/huggingpics/blob/main/HuggingPics.ipynb).


Report any issues with the demo at the [github repo](https://github.com/nateraw/huggingpics).


## Example Images


#### air pollution


![air pollution](images/air_pollution_new.jpg)


#### land pollution


![land pollution](images/land_pollution.jpg)


#### water pollution


![water pollution](images/water_pollution.jpg)


model name: pollution


model author: sreeramajay


task: image-classification


tags: pytorch, tensorboard, vit, image-classification, transformers, huggingpics, model-index


tags:


- conversational


# My Awesome Model


model name: DialoGPT-got-skippy


model author: madbuda


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


model name: DialoGPT-medium-skippy


## license: cc


model name: Owl


model author: AlexaMerens


tags: license:cc


## language: zh


model name: bert-base-chinese


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, zh, transformers, autonlp_compatible, infinity_compatible


language:


- he
- en


tags:


- translation


## license: apache-2.0


### he-en


* source group: Hebrew
* target group: English
* OPUS readme: [heb-eng](https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/heb-eng/README.md)
* model: transformer
* source language(s): heb
* target language(s): eng
* model: transformer
* pre-processing: normalization + SentencePiece (spm32k,spm32k)
* download original weights: [opus-2020-10-04.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/heb-eng/opus-2020-10-04.zip)
* test set translations: [opus-2020-10-04.test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/heb-eng/opus-2020-10-04.test.txt)
* test set scores: [opus-2020-10-04.eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/heb-eng/opus-2020-10-04.eval.txt)


## Benchmarks


| testset               | BLEU  | chr-F |
|-----------------------|-------|-------|
| Tatoeba-test.heb.eng 	| 52.0 	| 0.670 |


### System Info:


- hf_name: he-en
- source_languages: heb
- target_languages: eng
- opus_readme_url: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/heb-eng/README.md
- original_repo: Tatoeba-Challenge
- tags: ['translation']
- languages: ['he', 'en']
- src_constituents: ('Hebrew', {'heb'})
- tgt_constituents: ('English', {'eng'})
- src_multilingual: False
- tgt_multilingual: False
- long_pair: heb-eng
- prepro:  normalization + SentencePiece (spm32k,spm32k)
- url_model: https://object.pouta.csc.fi/Tatoeba-MT-models/heb-eng/opus-2020-10-04.zip
- url_test_set: https://object.pouta.csc.fi/Tatoeba-MT-models/heb-eng/opus-2020-10-04.test.txt
- src_alpha3: heb
- tgt_alpha3: eng
- chrF2_score: 0.67
- bleu: 52.0
- brevity_penalty: 0.9690000000000001
- ref_len: 73560.0
- src_name: Hebrew
- tgt_name: English
- train_date: 2020-10-04 00:00:00
- src_alpha2: he
- tgt_alpha2: en
- prefer_old: False
- short_pair: he-en
- helsinki_git_sha: 61fd6908b37d9a7b21cc3e27c1ae1fccedc97561
- transformers_git_sha: d99ed7ad618037ae878f0758157ed0764bd7f935
- port_machine: LM0-400-22516.local
- port_time: 2020-10-15-16:31


model name: opus-mt-he-en


model author: tiedeman


task: translation


tags: pytorch, rust, marian, text2text-generation, he, en, transformers, translation, license:apache-2.0, autonlp_compatible


- en
- he


### en-he


* source group: English
* target group: Hebrew
* OPUS readme: [eng-heb](https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-heb/README.md)
* model: transformer
* source language(s): eng
* target language(s): heb
* model: transformer
* pre-processing: normalization + SentencePiece (spm32k,spm32k)
* download original weights: [opus-2020-10-04.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-heb/opus-2020-10-04.zip)
* test set translations: [opus-2020-10-04.test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-heb/opus-2020-10-04.test.txt)
* test set scores: [opus-2020-10-04.eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/eng-heb/opus-2020-10-04.eval.txt)


| testset               | BLEU  | chr-F |
|-----------------------|-------|-------|
| Tatoeba-test.eng.heb 	| 37.9 	| 0.602 |


- hf_name: en-he
- source_languages: eng
- target_languages: heb
- opus_readme_url: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-heb/README.md
- original_repo: Tatoeba-Challenge
- tags: ['translation']
- languages: ['en', 'he']
- src_constituents: ('English', {'eng'})
- tgt_constituents: ('Hebrew', {'heb'})
- src_multilingual: False
- tgt_multilingual: False
- long_pair: eng-heb
- prepro:  normalization + SentencePiece (spm32k,spm32k)
- url_model: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-heb/opus-2020-10-04.zip
- url_test_set: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-heb/opus-2020-10-04.test.txt
- src_alpha3: eng
- tgt_alpha3: heb
- chrF2_score: 0.602
- bleu: 37.9
- brevity_penalty: 1.0
- ref_len: 60359.0
- src_name: English
- tgt_name: Hebrew
- train_date: 2020-10-04 00:00:00
- src_alpha2: en
- tgt_alpha2: he
- prefer_old: False
- short_pair: en-he
- helsinki_git_sha: 61fd6908b37d9a7b21cc3e27c1ae1fccedc97561
- transformers_git_sha: d99ed7ad618037ae878f0758157ed0764bd7f935
- port_machine: LM0-400-22516.local
- port_time: 2020-10-15-16:31


model name: opus-mt-en-he


tags: pytorch, rust, marian, text2text-generation, en, he, transformers, translation, license:apache-2.0, autonlp_compatible


tags:


- conversational


model name: DialoGPT-small-rick


model author: Tejasvb


task: conversational


model name: DialogGPT-small-rick


tags: conversational


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: distilbert-base-uncased-finetuned-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: conll2003
type: conll2003
args: conll2003
metrics:
    - name: Precision
type: precision
value: 0.9266592920353982
    - name: Recall
type: recall
value: 0.9371294328224634
    - name: F1
type: f1
value: 0.9318649535569274
    - name: Accuracy
type: accuracy
value: 0.9838117781625813


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilbert-base-uncased-finetuned-ner


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the conll2003 dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0620
- Precision: 0.9267
- Recall: 0.9371
- F1: 0.9319
- Accuracy: 0.9838


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.2462        | 1.0   | 878  | 0.0714          | 0.9052    | 0.9223 | 0.9137 | 0.9803   |
| 0.0535        | 2.0   | 1756 | 0.0615          | 0.9188    | 0.9331 | 0.9259 | 0.9827   |
| 0.0315        | 3.0   | 2634 | 0.0620          | 0.9267    | 0.9371 | 0.9319 | 0.9838   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: distilbert-base-uncased-finetuned-ner


model author: Shenyancheng


task: token-classification


tags: pytorch, tensorboard, distilbert, token-classification, dataset:conll2003, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model_index:
- name: bert-base-uncased-finetuned-sst2
results:
  - dataset:
name: glue
type: glue
args: sst2
metric:
name: Accuracy
type: accuracy
value: 0.926605504587156


# bert-base-uncased-finetuned-sst2


This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2716
- Accuracy: 0.9266


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.1666        | 1.0   | 2105  | 0.2403          | 0.9232   |
| 0.1122        | 2.0   | 4210  | 0.2716          | 0.9266   |
| 0.0852        | 3.0   | 6315  | 0.3150          | 0.9232   |
| 0.056         | 4.0   | 8420  | 0.3209          | 0.9163   |
| 0.0344        | 5.0   | 10525 | 0.3740          | 0.9243   |


### Framework versions


- Transformers 4.9.2
- Pytorch 1.8.1
- Datasets 1.11.0
- Tokenizers 0.10.1


model name: bert-base-uncased-finetuned-sst2


model author: ikevin98


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
model_index:
name: bert-base-uncased-sst2-distilled


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-base-uncased-sst2-distilled


This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unkown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2676
- Accuracy: 0.9025


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.3797        | 1.0   | 2105  | 0.2512          | 0.9002   |
| 0.3036        | 2.0   | 4210  | 0.2643          | 0.8933   |
| 0.2609        | 3.0   | 6315  | 0.2831          | 0.8956   |
| 0.2417        | 4.0   | 8420  | 0.2676          | 0.9025   |
| 0.2305        | 5.0   | 10525 | 0.2740          | 0.9025   |


### Framework versions


- Transformers 4.9.1
- Pytorch 1.8.1
- Datasets 1.11.0
- Tokenizers 0.10.1


model name: bert-base-uncased-sst2-distilled


model author: ikevin98


task: text-classification


tags: pytorch, bert, text-classification, transformers, license:apache-2.0, generated_from_trainer, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
model_index:
name: bert-base-uncased-finetuned-sst2-sst2-membership


# bert-base-uncased-finetuned-sst2-sst2-membership


This model is a fine-tuned version of [ikevin98/bert-base-uncased-finetuned-sst2](https://huggingface.co/ikevin98/bert-base-uncased-finetuned-sst2) on an unkown dataset.
It achieves the following results on the evaluation set:


- Loss: 1.3100
- Accuracy: 1.0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.5125        | 1.0   | 3813 | 1.3100          | 1.0      |


### Framework versions


model name: bert-base-uncased-finetuned-sst2-sst2-membership


model author: ikevin98


task: text-classification


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- accuracy
model_index:
name: bert-base-uncased-sst2-membership-attack


# bert-base-uncased-sst2-membership-attack


- Loss: 0.6296
- Accuracy: 0.8681


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.6921        | 1.0   | 3813  | 0.6263          | 0.8360   |
| 0.6916        | 2.0   | 7626  | 0.6296          | 0.8681   |
| 0.6904        | 3.0   | 11439 | 0.6105          | 0.8406   |
| 0.6886        | 4.0   | 15252 | 0.6192          | 0.8200   |
| 0.6845        | 5.0   | 19065 | 0.6250          | 0.7798   |


### Framework versions


- Transformers 4.9.2
- Pytorch 1.8.1
- Datasets 1.11.0
- Tokenizers 0.10.1


model name: bert-base-uncased-sst2-membership-attack


model author: ikevin98


task: text-classification


GPT model developed in [Language Models are Few-Shot Butlers](https://arxiv.org/abs/2104.07972).


model name: lm-butlers-gpt


model author: vmicheli


task: text-generation


tags: pytorch, jax, gpt2, text-generation, arxiv:2104.07972, transformers


language: de
tags:


- exbert
- German


# German Medical BERT


This is a fine-tuned model on the Medical domain for the German language and based on German BERT. This model has only been trained to improve on-target tasks (Masked Language Model). It can later be used to perform a downstream task of your needs, while I performed it for the NTS-ICD-10 text classification task.


## Overview


**Language model:** bert-base-german-cased


**Language:** German


**Fine-tuning:** Medical articles (diseases, symptoms, therapies, etc..)


**Eval data:** NTS-ICD-10 dataset (Classification)


**Infrastructure:** Google Colab


## Details


- We fine-tuned using Pytorch with Huggingface library on Colab GPU.
- With standard parameter settings for fine-tuning as mentioned in the original BERT paper.
- Although had to train for up to 25 epochs for classification.


## Performance (Micro precision, recall, and f1 score for multilabel code classification)


|Models|P|R|F1|
|:------|:------|:------|:------|
|German BERT|86.04|75.82|80.60|
|German MedBERT-256 (fine-tuned)|87.41|77.97|82.42|
|German MedBERT-512 (fine-tuned)|87.75|78.26|82.73|


## Author


Manjil Shrestha: `shresthamanjil21 [at] gmail.com`


## Related Paper: [Report](https://opus4.kobv.de/opus4-rhein-waal/frontdoor/index/index/searchtype/collection/id/16225/start/0/rows/10/doctypefq/masterthesis/docId/740)


Get in touch:
[LinkedIn](https://www.linkedin.com/in/manjil-shrestha-038527b4/)


model name: German-MedBERT


model author: smanjil


task: fill-mask


tags: pytorch, jax, bert, fill-mask, de, transformers, exbert, German, autonlp_compatible, infinity_compatible


language: fr
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: wav2vec2-large-xlsr-53-French by Mehdi Hosseini Moghadam
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice fr
type: common_voice
args: fr
metrics:
    - name: Test WER
type: wer
value:  34.856015


# wav2vec2-large-xlsr-53-French


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in French using the [Common Voice](https://huggingface.co/datasets/common_voice)


When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "fr", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-French")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-French")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


## Evaluation


The model can be evaluated as follows on the French test data of Common Voice.


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "fr", split="test[:10%]")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-French")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-French")

model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():

      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  34.856015 %


## Training


10% of the Common Voice `train`, `validation` datasets were used for training.


## Testing


10% of the Common Voice `Test` dataset were used for training.


model name: wav2vec2-large-xlsr-53-French


model author: MehdiHosseiniMoghadam


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, fr, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: sv-SE


datasets:


- common_voice


tags:


- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week


license: apache-2.0


model-index:


- name: wav2vec2-large-xlsr-53-Swedish by Mehdi Hosseini Moghadam

  results:

  - task:

    name: Speech Recognition

    type: automatic-speech-recognition

    dataset:

    name: Common Voice sv-SE

    type: common_voice

    args: sv-SE

    metrics:

    - name: Test WER

      type: wer

      value:  41.388337


# wav2vec2-large-xlsr-53-Swedish


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Swedish using the [Common Voice](https://huggingface.co/datasets/common_voice)


## Usage


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Swedish")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Swedish")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

	speech_array, sampling_rate = torchaudio.load(batch["path"])

	batch["speech"] = resampler(speech_array).squeeze().numpy()

	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


## Evaluation


The model can be evaluated as follows on the Swedish test data of Common Voice.


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "sv-SE", split="test")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Swedish")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Swedish")

model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

	batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

	speech_array, sampling_rate = torchaudio.load(batch["path"])

	batch["speech"] = resampler(speech_array).squeeze().numpy()

	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

	inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

	with torch.no_grad():

		logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

	batch["pred_strings"] = processor.batch_decode(pred_ids)

	return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  41.388337 %


## Training


The Common Voice `train`, `validation` datasets were used for training.


model name: wav2vec2-large-xlsr-53-Swedish


model author: MehdiHosseiniMoghadam


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, sv-SE, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: ka


datasets:


- common_voice


tags:


license: apache-2.0


model-index:


- name: wav2vec2-large-xlsr-53-Georgian by Mehdi Hosseini Moghadam

  results:

  - task:

    name: Speech Recognition

    type: automatic-speech-recognition

    dataset:

    name: Common Voice ka

    type: common_voice

    args: ka

    metrics:

    - name: Test WER

      type: wer

      value:  60.504024


# wav2vec2-large-xlsr-53-Georgian


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Georgian using the [Common Voice](https://huggingface.co/datasets/common_voice)


When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ka", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Georgian")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Georgian")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


## Evaluation


The model can be evaluated as follows on the Georgian test data of Common Voice.


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "ka", split="test")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Georgian")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Georgian")

model.to("cuda")

chars_to_ignore_regex = '[\\,\\?\\.\\!\\-\\;\\:\\"\\“]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():

      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  60.504024 %


## Training


model name: wav2vec2-large-xlsr-53-Georgian


model author: MehdiHosseiniMoghadam


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, ka, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: cs
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: wav2vec2-large-xlsr-53-Czech by Mehdi Hosseini Moghadam
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice cs
type: common_voice
args: cs
metrics:
    - name: Test WER
type: wer
value:  27.047806


# wav2vec2-large-xlsr-53-Czech


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Czech using the [Common Voice](https://huggingface.co/datasets/common_voice)


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "cs", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Czech")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Czech")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


The model can be evaluated as follows on the Czech test data of Common Voice.


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "cs", split="test")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Czech")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Czech")

model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():

      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  27.047806 %


The Common Voice `train`, `validation` datasets were used for training.


model name: wav2vec2-large-xlsr-53-Czech


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, cs, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: de
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: wav2vec2-large-xlsr-53-German by Mehdi Hosseini Moghadam
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice de
type: common_voice
args: de
metrics:
    - name: Test WER
type: wer
value:  25.284593


# wav2vec2-large-xlsr-53-German


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in German using the [Common Voice](https://huggingface.co/datasets/common_voice)


When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "de", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-German")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-German")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


## Evaluation


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "de", split="test[:15%]")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-German")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-German")

model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():

      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  25.284593 %


## Training


10% of the Common Voice `train`, `validation` datasets were used for training.


## Testing


15% of the Common Voice `Test` dataset were used for training.


model name: wav2vec2-large-xlsr-53-German


model author: MehdiHosseiniMoghadam


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, de, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: nl
datasets:


- common_voice
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: wav2vec2-large-xlsr-53-Dutch by Mehdi Hosseini Moghadam
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice nl
type: common_voice
args: nl
metrics:
    - name: Test WER
type: wer
value:  26.494162


# wav2vec2-large-xlsr-53-Dutch


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Dutch using the [Common Voice](https://huggingface.co/datasets/common_voice)


## Usage


```python

import torch

import torchaudio

from datasets import load_dataset

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "nl", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Dutch")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Dutch")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():

  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))

print("Reference:", test_dataset["sentence"][:2])

```


## Evaluation


The model can be evaluated as follows on the Dutch test data of Common Voice.


```python

import torch

import torchaudio

from datasets import load_dataset, load_metric

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

import re

test_dataset = load_dataset("common_voice", "nl", split="test")

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Dutch")

model = Wav2Vec2ForCTC.from_pretrained("MehdiHosseiniMoghadam/wav2vec2-large-xlsr-53-Dutch")

model.to("cuda")

chars_to_ignore_regex = '[\\\\,\\\\?\\\\.\\\\!\\\\-\\\\;\\\\:\\\\"\\\\“]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def speech_file_to_array_fn(batch):

  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()

  speech_array, sampling_rate = torchaudio.load(batch["path"])

  batch["speech"] = resampler(speech_array).squeeze().numpy()

  return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.

# We need to read the aduio files as arrays

def evaluate(batch):

    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():

      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)

    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))

```


**Test Result**:  26.494162 %


## Training


The Common Voice `train`, `validation` datasets were used for training.


model name: wav2vec2-large-xlsr-53-Dutch


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, nl, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


Gggg


model name: Ges


model author: Mervtttt


tags:


- spacy
- token-classification
language:
- hr
model-index:
- name: hr_hroberta_pipeline
results:
  - task:
name: POS
type: token-classification
metrics:
    - name: POS Accuracy
type: accuracy
value: 0.9341816254
  - task:
name: SENTER
type: token-classification
metrics:
    - name: SENTER Precision
type: precision
value: 0.9666666667
    - name: SENTER Recall
type: recall
value: 0.9682804674
    - name: SENTER F Score
type: f_score
value: 0.9674728941
  - task:
name: UNLABELED_DEPENDENCIES
type: token-classification
metrics:
    - name: Unlabeled Dependencies Accuracy
type: accuracy
value: 0.8833197442
  - task:
name: LABELED_DEPENDENCIES
type: token-classification
metrics:
    - name: Labeled Dependencies Accuracy
type: accuracy
value: 0.8833197442


| Feature | Description |
| --- | --- |
| **Name** | `hr_hroberta_pipeline` |
| **Version** | `0.0.1` |
| **spaCy** | `>=3.1.3,<3.2.0` |
| **Default Pipeline** | `transformer`, `morphologizer`, `tagger`, `parser` |
| **Components** | `transformer`, `morphologizer`, `tagger`, `parser` |
| **Vectors** | 0 keys, 0 unique vectors (0 dimensions) |
| **Sources** | n/a |
| **License** | n/a |
| **Author** | [n/a]() |


### Label Scheme


<details>