```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-lt-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-lt-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-lt-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, lt, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: tr


datasets: wikipedia


## license: apache-2.0


# distilbert-base-tr-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-tr-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-tr-cased")

```


### How to cite


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: distilbert-base-tr-cased


tags: pytorch, distilbert, fill-mask, tr, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-es-zh-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-es-zh-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-es-zh-cased")

```


model name: distilbert-base-en-es-zh-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: es


datasets: wikipedia


## license: apache-2.0


# bert-base-es-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-es-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-es-cased")

```


### How to cite


## Contact


model name: bert-base-es-cased


tags: pytorch, tf, jax, bert, fill-mask, es, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# bert-base-en-ar-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ar-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-ar-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-ar-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-zh-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-zh-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-zh-cased")

```


### How to cite


## Contact


model name: bert-base-en-zh-cased


language: multilingual


## license: apache-2.0


# bert-base-en-fr-zh-ja-vi-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-zh-ja-vi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-zh-ja-vi-cased")

```


model name: bert-base-en-fr-zh-ja-vi-cased


model author: Geotrend


task: fill-mask


# bert-base-en-pl-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-pl-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-pl-cased")

```


model name: bert-base-en-pl-cased


language: nl


datasets: wikipedia


# bert-base-nl-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-nl-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-nl-cased")

```


### How to cite


## Contact


model name: bert-base-nl-cased


tags: pytorch, tf, jax, bert, fill-mask, nl, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: zh


## license: apache-2.0


# distilbert-base-zh-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-zh-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-zh-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-zh-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, zh, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# bert-base-en-fr-de-no-da-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-de-no-da-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-de-no-da-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-fr-de-no-da-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


## license: apache-2.0


# bert-base-en-fr-de-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-de-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-de-cased")

```


model name: bert-base-en-fr-de-cased


model author: Geotrend


task: fill-mask


language: multilingual


datasets: wikipedia


# distilbert-base-en-hi-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-hi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-hi-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-hi-cased


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: lt


## license: apache-2.0


# distilbert-base-lt-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-lt-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-lt-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-lt-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, lt, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: uk


datasets: wikipedia


# bert-base-uk-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-uk-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-uk-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-uk-cased


tags: pytorch, tf, jax, bert, fill-mask, uk, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: fr


license: apache-2.0


widget:


- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."


# bert-base-fr-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-fr-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-fr-cased")

```


model name: bert-base-fr-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, fr, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: hi


datasets: wikipedia


## license: apache-2.0


# bert-base-hi-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-hi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-hi-cased")

```


### How to cite


## Contact


model name: bert-base-hi-cased


tags: pytorch, tf, jax, bert, fill-mask, hi, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-ja-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-ja-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-ja-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-ja-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# bert-base-en-fr-zh-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-zh-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-zh-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-fr-zh-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-ru-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ru-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-ru-cased")

```


model name: bert-base-en-ru-cased


model author: Geotrend


task: fill-mask


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-no-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-no-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-no-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-no-cased


language: bg


# distilbert-base-bg-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-bg-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-bg-cased")

```


model name: distilbert-base-bg-cased


tags: pytorch, distilbert, fill-mask, bg, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-fr-es-pt-it-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-es-pt-it-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-es-pt-it-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-fr-es-pt-it-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# bert-base-en-es-it-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-es-it-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-es-it-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-es-it-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: pt


# distilbert-base-pt-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-pt-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-pt-cased")

```


model name: distilbert-base-pt-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, pt, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


## license: apache-2.0


# bert-base-en-no-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-no-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-no-cased")

```


### How to cite


## Contact


model name: bert-base-en-no-cased


language: vi


# distilbert-base-vi-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-vi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-vi-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-vi-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, vi, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: da


datasets: wikipedia


## license: apache-2.0


# distilbert-base-da-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-da-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-da-cased")

```


### How to cite


## Contact


model name: distilbert-base-da-cased


tags: pytorch, distilbert, fill-mask, da, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."
- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# bert-base-10lang-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


This model handles the following languages: english, french, spanish, german, chinese, arabic, russian, portuguese, italian, and urdu. It produces the same representations as [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) while being 22.5% smaller in size.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-10lang-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-10lang-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-10lang-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: en


datasets: wikipedia


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-cased


tags: pytorch, tf, jax, bert, fill-mask, en, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


## license: apache-2.0


# distilbert-base-en-el-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-el-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-el-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-el-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


# distilbert-base-en-fr-es-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-es-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-es-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-fr-es-cased


language: ro


## license: apache-2.0


# bert-base-ro-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ro-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-ro-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-ro-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, ro, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: tr


datasets: wikipedia


# bert-base-tr-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-tr-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-tr-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-tr-cased


tags: pytorch, tf, jax, bert, fill-mask, tr, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: ur


## license: apache-2.0


# bert-base-ur-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ur-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-ur-cased")

```


model name: bert-base-ur-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, ur, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-es-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-es-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-es-cased")

```


### How to cite


## Contact


model name: bert-base-en-es-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


## license: apache-2.0


# distilbert-base-en-sw-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-sw-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-sw-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-sw-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# distilbert-base-en-ru-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-ru-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-ru-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-ru-cased


license: apache-2.0


widget:


# bert-base-en-hi-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-hi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-hi-cased")

```


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: bert-base-en-hi-cased


model author: Geotrend


task: fill-mask


language: hi


datasets: wikipedia


## license: apache-2.0


# distilbert-base-hi-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-hi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-hi-cased")

```


### How to cite


## Contact


model name: distilbert-base-hi-cased


tags: pytorch, distilbert, fill-mask, hi, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: ro


# distilbert-base-ro-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-ro-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-ro-cased")

```


model name: distilbert-base-ro-cased


tags: pytorch, distilbert, fill-mask, ro, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# bert-base-en-ja-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ja-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-ja-cased")

```


model name: bert-base-en-ja-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-fr-ar-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-ar-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-ar-cased")

```


### How to cite


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: distilbert-base-en-fr-ar-cased


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-de-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-de-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-de-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-de-cased


model author: Geotrend


task: fill-mask


datasets: wikipedia


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-tr-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-tr-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-tr-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-tr-cased


language: multilingual


## license: apache-2.0


# distilbert-base-en-lt-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-lt-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-lt-cased")

```


model name: distilbert-base-en-lt-cased


model author: Geotrend


task: fill-mask


language: ur


datasets: wikipedia


# distilbert-base-ur-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-ur-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-ur-cased")

```


model name: distilbert-base-ur-cased


tags: pytorch, distilbert, fill-mask, ur, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


# distilbert-base-en-zh-hi-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-zh-hi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-zh-hi-cased")

```


### How to cite


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: distilbert-base-en-zh-hi-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


## license: apache-2.0


# bert-base-en-fr-es-de-zh-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-es-de-zh-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-es-de-zh-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-fr-es-de-zh-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


# distilbert-base-en-it-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-it-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-it-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-it-cased


model author: Geotrend


task: fill-mask


language: multilingual


datasets: wikipedia


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."
- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# bert-base-15lang-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


The measurements below have been computed on a [Google Cloud n1-standard-1 machine (1 vCPU, 3.75 GB)](https://cloud.google.com/compute/docs/machine-types#n1_machine_type):


|             Model               | Num parameters |   Size   |  Memory  | Loading time |
| ------------------------------- | -------------- | -------- | -------- | ------------ |
| bert-base-multilingual-cased    |   178 million  |  714 MB  | 1400 MB  |    4.2 sec   |
| Geotrend/bert-base-15lang-cased |   141 million  |  564 MB  | 1098 MB  |    3.1 sec   |


Handled languages: en, fr, es, de, zh, ar, ru, vi, el, bg, th, tr, hi, ur and sw.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-15lang-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-15lang-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-15lang-cased


model author: Geotrend


task: fill-mask


language: el


datasets: wikipedia


## license: apache-2.0


# distilbert-base-el-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-el-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-el-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-el-cased


tags: pytorch, distilbert, fill-mask, el, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-ur-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ur-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-ur-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-ur-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: fr


datasets: wikipedia


## license: apache-2.0


# distilbert-base-fr-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-fr-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-fr-cased")

```


model name: distilbert-base-fr-cased


tags: pytorch, distilbert, fill-mask, fr, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-uk-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-uk-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-uk-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-uk-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: ja


datasets: wikipedia


## license: apache-2.0


# distilbert-base-ja-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-ja-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-ja-cased")

```


model name: distilbert-base-ja-cased


tags: pytorch, distilbert, fill-mask, ja, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: it


# bert-base-it-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-it-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-it-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-it-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, it, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: ar


datasets: wikipedia


## license: apache-2.0


# distilbert-base-ar-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-ar-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-ar-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-ar-cased


tags: pytorch, distilbert, fill-mask, ar, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."
- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# bert-base-25lang-cased


Handled languages: en, fr, es, de, zh, ar, ru, vi, el, bg, th, tr, hi, ur, sw, nl, uk, ro, pt, it, lt, no, pl, da and ja.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-25lang-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-25lang-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-25lang-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: it


datasets: wikipedia


## license: apache-2.0


# distilbert-base-it-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-it-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-it-cased")

```


model name: distilbert-base-it-cased


tags: pytorch, distilbert, fill-mask, it, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-bg-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-bg-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-bg-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-bg-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-fr-it-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-it-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-it-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-fr-it-cased


language: multilingual


# bert-base-en-nl-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-nl-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-nl-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-nl-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-nl-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-nl-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-nl-cased")

```


model name: distilbert-base-en-nl-cased


language: multilingual


# bert-base-en-fr-da-ja-vi-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-da-ja-vi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-da-ja-vi-cased")

```


model name: bert-base-en-fr-da-ja-vi-cased


language: bg


# bert-base-bg-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-bg-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-bg-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-bg-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, bg, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# bert-base-en-el-ru-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-el-ru-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-el-ru-cased")

```


model name: bert-base-en-el-ru-cased


language: zh


# bert-base-zh-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-zh-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-zh-cased")

```


### How to cite


## Contact


model name: bert-base-zh-cased


tags: pytorch, tf, jax, bert, fill-mask, zh, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-el-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-el-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-el-cased")

```


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: bert-base-en-el-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# bert-base-en-es-zh-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-es-zh-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-es-zh-cased")

```


### How to cite


## Contact


model name: bert-base-en-es-zh-cased


language: en


# distilbert-base-en-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, en, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: da


datasets: wikipedia


## license: apache-2.0


# bert-base-da-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-da-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-da-cased")

```


### How to cite


## Contact


model name: bert-base-da-cased


tags: pytorch, tf, jax, bert, fill-mask, da, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-ur-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-ur-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-ur-cased")

```


model name: distilbert-base-en-ur-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# bert-base-en-fr-uk-el-ro-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-uk-el-ro-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-uk-el-ro-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-fr-uk-el-ro-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


# distilbert-base-en-fr-uk-el-ro-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-uk-el-ro-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-uk-el-ro-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-fr-uk-el-ro-cased


model author: Geotrend


task: fill-mask


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-th-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-th-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-th-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-th-cased


language: multilingual


# distilbert-base-en-fr-da-ja-vi-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-da-ja-vi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-da-ja-vi-cased")

```


model name: distilbert-base-en-fr-da-ja-vi-cased


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


# distilbert-base-en-fr-nl-ru-ar-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-nl-ru-ar-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-nl-ru-ar-cased")

```


model name: distilbert-base-en-fr-nl-ru-ar-cased


model author: Geotrend


task: fill-mask


language: ar


datasets: wikipedia


license: apache-2.0


widget:


- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# bert-base-ar-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ar-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-ar-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-ar-cased


tags: pytorch, tf, jax, bert, fill-mask, ar, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."
- text: "تقع سويسرا في [MASK] أوروبا"
- text: "إسمي محمد وأسكن في [MASK]."


# distilbert-base-25lang-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


Handled languages: en, fr, es, de, zh, ar, ru, vi, el, bg, th, tr, hi, ur, sw, nl, uk, ro, pt, it, lt, no, pl, da and ja.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-25lang-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-25lang-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-25lang-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-fr-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-cased")

```


### How to cite


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: distilbert-base-en-fr-cased


language: multilingual


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-de-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-de-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-de-cased")

```


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: bert-base-en-de-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-ar-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-ar-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-ar-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


model name: distilbert-base-en-ar-cased


language: multilingual


license: apache-2.0


widget:


# bert-base-en-th-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-th-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-th-cased")

```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-th-cased


model author: Geotrend


task: fill-mask


language: no


datasets: wikipedia


## license: apache-2.0


# bert-base-no-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-no-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-no-cased")

```


### How to cite


## Contact


model name: bert-base-no-cased


tags: pytorch, tf, jax, bert, fill-mask, no, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: sw


# distilbert-base-sw-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-sw-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-sw-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-sw-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, sw, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


## license: apache-2.0


# bert-base-en-fr-ar-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-ar-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-ar-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-fr-ar-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-bg-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-bg-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-bg-cased")

```


model name: bert-base-en-bg-cased


model author: Geotrend


task: fill-mask


language: multilingual


datasets: wikipedia


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."
- text: "Paris est la [MASK] de la France."
- text: "Paris est la capitale de la [MASK]."
- text: "L'élection américaine a eu [MASK] en novembre 2020."


# bert-base-en-fr-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-fr-cased


## license: apache-2.0


# distilbert-base-en-fr-zh-ja-vi-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-zh-ja-vi-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-zh-ja-vi-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-en-fr-zh-ja-vi-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# bert-base-en-fr-lt-no-pl-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-lt-no-pl-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-lt-no-pl-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-fr-lt-no-pl-cased


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: nl


## license: apache-2.0


# distilbert-base-nl-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-nl-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-nl-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


model name: distilbert-base-nl-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, nl, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# bert-base-en-da-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-da-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-da-cased")

```


### How to cite


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-en-da-cased


language: th


## license: apache-2.0


# distilbert-base-th-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-th-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-th-cased")

```


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: distilbert-base-th-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, th, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# bert-base-en-lt-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-lt-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-lt-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-lt-cased


## license: apache-2.0


# bert-base-en-it-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-it-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-it-cased")

```


model name: bert-base-en-it-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# distilbert-base-en-el-ru-cased


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-el-ru-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-el-ru-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


## Contact


model name: distilbert-base-en-el-ru-cased


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: pl


## license: apache-2.0


# bert-base-pl-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-pl-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-pl-cased")

```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: bert-base-pl-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, pl, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


# bert-base-en-fr-es-pt-it-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-es-pt-it-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-es-pt-it-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-en-fr-es-pt-it-cased


license: apache-2.0


widget:


- text: "Google generated 46 billion [MASK] in revenue."
- text: "Paris is the capital of [MASK]."
- text: "Algiers is the largest city in [MASK]."


# bert-base-en-vi-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-vi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-vi-cased")

```


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


model name: bert-base-en-vi-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: de


datasets: wikipedia


## license: apache-2.0


# distilbert-base-de-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-de-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-de-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


### How to cite


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-de-cased


model author: Geotrend


task: fill-mask


tags: pytorch, distilbert, fill-mask, de, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


## license: apache-2.0


# bert-base-en-es-pt-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-es-pt-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-es-pt-cased")

```


model name: bert-base-en-es-pt-cased


language: pt


# bert-base-pt-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-pt-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-pt-cased")

```


### How to cite


## Contact


model name: bert-base-pt-cased


tags: pytorch, tf, jax, bert, fill-mask, pt, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


# bert-base-en-fr-it-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-it-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-it-cased")

```


model name: bert-base-en-fr-it-cased


model author: Geotrend


task: fill-mask


language: ru


datasets: wikipedia


## license: apache-2.0


# distilbert-base-ru-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-ru-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-ru-cased")

```


model name: distilbert-base-ru-cased


tags: pytorch, distilbert, fill-mask, ru, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: vi


# bert-base-vi-cased


We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages.


Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy.


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-vi-cased")
model = AutoModel.from_pretrained("Geotrend/bert-base-vi-cased")

```


### How to cite


```bibtex
@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


## Contact


model name: bert-base-vi-cased


model author: Geotrend


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, vi, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


language: multilingual


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-da-cased


We are sharing smaller versions of [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) that handle a custom number of languages.


Our versions give exactly the same representations produced by the original model which preserves the original accuracy.


For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf).


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-da-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-da-cased")

```


To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers).


```bibtex
@inproceedings{smallermdistilbert,
  title={Load What You Need: Smaller Versions of Mutlilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}
```


Please contact amine@geotrend.fr for any question, feedback or request.


model name: distilbert-base-en-da-cased


tags: pytorch, distilbert, fill-mask, multilingual, dataset:wikipedia, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


# distilbert-base-en-fr-de-no-da-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-de-no-da-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-de-no-da-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-fr-de-no-da-cased


model author: Geotrend


task: fill-mask


language: multilingual


datasets: wikipedia


## license: apache-2.0


# distilbert-base-en-es-pt-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-es-pt-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-es-pt-cased")

```


model name: distilbert-base-en-es-pt-cased


# distilbert-base-en-fr-de-cased


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-fr-de-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-fr-de-cased")

```


model name: distilbert-base-en-fr-de-cased


# distilbert-base-en-es-cased


## How to use


```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-en-es-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-en-es-cased")

```


### How to cite


## Contact


model name: distilbert-base-en-es-cased


model author: Geotrend


task: fill-mask


## Hugging Face's logo


language:


- en
- zu
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# mbart50-large-zu-en-mt


## Model description


**mbart50-large-zu-en-mt** is a **machine translation** model from isiZulu to English based on a fine-tuned facebook/mbart-large-50 model.  It establishes a **strong baseline** for automatically translating texts from isiZulu to English.


#### Limitations and bias


This model is limited by its training dataset. This may not generalize well for all use cases in different domains.


## Training data


Specifically, this model is a *mbart-large-50* model that was fine-tuned on JW300 isiZulu corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt). The model was trained using Xhosa(xh_ZA) as the language since the pre-trained model does not initially support isiZulu. Thus, you need to use the xh_ZA for language code when evaluating the model.


## Training procedure


This model was trained on NVIDIA V100 GPU


## Eval results on Test set (BLEU score)


Fine-tuning mbart50-large achieves **33.56 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info


By David Adelani


model name: mbart50-large-zu-en-mt


model author: masakhane


task: text2text-generation


tags: pytorch, mbart, text2text-generation, en, zu, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


## Hugging Face's logo


language:


- fr
- fon
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# mbart50-large-fon-fr-mt


## Model description


**mbart50-large-fon-fr-mt** is a **machine translation** model from Fon to French based on a fine-tuned facebook/mbart-large-50 model.  It establishes a **strong baseline** for automatically translating texts from Fon to French.


#### Limitations and bias


## Training data


Specifically, this model is a *mbart-large-50* model that was fine-tuned on JW300 Fon corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt). The model was trained using Swahili(sw_KE) as the language since the pre-trained model does not initially support Fon. Thus, you need to use the sw_KE for language code when evaluating the model.


## Training procedure


Fine-tuning mbart50-large achieves ** 7.22 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info


By David Adelani


model name: mbart50-large-fon-fr-mt


model author: masakhane


task: text2text-generation


tags: pytorch, mbart, text2text-generation, fr, fon, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


## Hugging Face's logo


language:


- fr
- ee
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# mt5-base-ewe-fr-mt


## Model description


**mt5-base-ewe-fr-mt** is a **machine translation** model from Ewe to French based on a fine-tuned google/mt5-base model.  It establishes a **strong baseline** for automatically translating texts from Ewe to French.


#### Limitations and bias


## Training data


Specifically, this model is a *mt5-base* model that was fine-tuned on JW300 Ewe corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt). The model was trained using Swahili(sw_KE) as the language since the pre-trained model does not initially support Ewe. Thus, you need to use the sw_KE for language code when evaluating the model.


## Training procedure


Fine-tuning mt5-base achieves **8.68 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info


By David Adelani


model name: mt5-base-ewe-fr-mt


model author: masakhane


task: text2text-generation


tags: pytorch, mt5, text2text-generation, fr, ee, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


## Hugging Face's logo


language:


# byt5-base-ewe-fr-mt


**byt5-base-ewe-fr-mt** is a **machine translation** model from Ewe to French based on a fine-tuned google/byt5-base model.  It establishes a **strong baseline** for automatically translating texts from Ewe to French.


This model is limited by its training dataset. This may not generalize well for all use cases in different domains.


Specifically, this model is a *byt5-base* model that was fine-tuned on JW300 Ewe corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt). The model was trained using Swahili(sw_KE) as the language since the pre-trained model does not initially support Ewe. Thus, you need to use the sw_KE for language code when evaluating the model.


This model was trained on NVIDIA V100 GPU


## Eval results on Test set (BLEU score)


Fine-tuning byt5-base achieves **11.41 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


model name: byt5-base-ewe-fr-mt


tags: pytorch, t5, text2text-generation, fr, ee, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


- fr
- mos
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# m2m100_418M-mos-fr-mt


## Model description


**m2m100_418M-mos-fr-mt** is a **machine translation** model from Mossi to French based on a fine-tuned facebook/m2m100_418M model.  It establishes a **baseline** for automatically translating texts from Mossi to French.


#### Limitations and bias


## Training data


Specifically, this model is a *m2m100_418M* model that was fine-tuned on JW300 Mossi corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt).


## Training procedure


Fine-tuning m2m100_418M achieves **6.07 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info


By David Adelani


model name: m2m100_418M-mos-fr-mt


model author: masakhane


task: text2text-generation


tags: pytorch, m2m_100, text2text-generation, fr, mos, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


## Hugging Face's logo


language:


- fr
- fon
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# byt5-base-fon-fr-mt


## Model description


**byt5-base-fon-fr-mt** is a **machine translation** model from Fon to French based on a fine-tuned google/byt5-base model.  It establishes a **strong baseline** for automatically translating texts from Fon to French.


#### Limitations and bias


## Training data


Specifically, this model is a *byt5-base* model that was fine-tuned on JW300 Fon corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt). The model was trained using Swahili(sw_KE) as the language since the pre-trained model does not initially support Fon. Thus, you need to use the sw_KE for language code when evaluating the model.


## Training procedure


Fine-tuning byt5-base achieves **5.15 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info


By David Adelani


model name: byt5-base-fon-fr-mt


model author: masakhane


task: text2text-generation


tags: pytorch, t5, text2text-generation, fr, fon, dataset:JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt), transformers, autonlp_compatible


## Hugging Face's logo


language:


- en
- zu
datasets:
- JW300 + [LAFAND](https://github.com/masakhane-io/lafand-mt)


# m2m100_418M-zu-en-mt


## Model description


**m2m100_418M-zu-en-mt** is a **machine translation** model from isiZulu to English based on a fine-tuned facebook/m2m100_418M model.  It establishes a **baseline** for automatically translating texts from isiZulu to English.


#### Limitations and bias


This model is limited by its training dataset. This may not generalize well for all use cases in different domains.


## Training data


Specifically, this model is a *m2m100_418M* model that was fine-tuned on JW300 isiZulu corpus and [LAFAND](https://github.com/masakhane-io/lafand-mt).


## Training procedure


This model was trained on NVIDIA V100 GPU


## Eval results on Test set (BLEU score)


Fine-tuning m2m100_418M achieves **29.74 BLEU** on [LAFAND test set](https://github.com/masakhane-io/lafand-mt)


### BibTeX entry and citation info