tags: pytorch, tf, jax, roberta, fill-mask, bn, transformers, MaskedLM, Bengali, RoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


language:


- te
tags:
- MaskedLM
- Telugu
- DistilBERT
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Telugu DistilBERT


## Model description


This is a DistilBERT language model pre-trained on ~2 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


## Intended uses & limitations


#### How to use


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-distilbert')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-distilbert')
text = "మీరు ఎలా ఉన్నారు"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


#### Limitations and bias


The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html).


model name: indic-transformers-te-distilbert


model author: neuralspace-reverie


task: fill-mask


tags: pytorch, tf, distilbert, fill-mask, te, transformers, MaskedLM, Telugu, DistilBERT, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


- hi
tags:
- MaskedLM
- Hindi
- BERT
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Hindi BERT


This is a BERT language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-bert')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-bert')
text = "आपका स्वागत हैं"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


model name: indic-transformers-hi-bert


tags: pytorch, tf, jax, bert, fill-mask, hi, transformers, MaskedLM, Hindi, BERT, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


language:


- te
tags:
- MaskedLM
- Telugu
- RoBERTa
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Telugu RoBERTa


## Model description


This is a RoBERTa language model pre-trained on ~2 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


## Intended uses & limitations


#### How to use


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-roberta')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-roberta')
text = "మీరు ఎలా ఉన్నారు"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 14, 768]
```


#### Limitations and bias


model name: indic-transformers-te-roberta


model author: neuralspace-reverie


task: fill-mask


tags: pytorch, tf, jax, roberta, fill-mask, te, transformers, MaskedLM, Telugu, RoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


- te
tags:
- MaskedLM
- Telugu
- BERT
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Telugu BERT


This is a BERT language model pre-trained on ~1.6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-bert')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-bert')
text = "మీరు ఎలా ఉన్నారు"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


model name: indic-transformers-te-bert


tags: pytorch, tf, jax, bert, fill-mask, te, transformers, MaskedLM, Telugu, BERT, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


language:


- te
tags:
- MaskedLM
- Telugu
- XLMRoBERTa
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Telugu XLMRoBERTa


## Model description


This is a XLMRoBERTa language model pre-trained on ~1.6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


## Intended uses & limitations


#### How to use


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-xlmroberta')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-xlmroberta')
text = "మీరు ఎలా ఉన్నారు"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


#### Limitations and bias


The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html).


model name: indic-transformers-te-xlmroberta


model author: neuralspace-reverie


task: fill-mask


tags: pytorch, tf, xlm-roberta, fill-mask, te, transformers, MaskedLM, Telugu, XLMRoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible


- hi
tags:
- MaskedLM
- Hindi
- RoBERTa
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Hindi RoBERTa


This is a RoBERTa language model pre-trained on ~10 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-roberta')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-roberta')
text = "आपका स्वागत हैं"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 11, 768]
```


model name: indic-transformers-hi-roberta


tags: pytorch, tf, jax, roberta, fill-mask, hi, transformers, MaskedLM, Hindi, RoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


language:


- bn
tags:
- MaskedLM
- Bengali
- XLMRoBERTa
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Bengali XLMRoBERTa


## Model description


This is a XLMRoBERTa language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


## Intended uses & limitations


#### How to use


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-xlmroberta')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-xlmroberta')
text = "আপনি কেমন আছেন?"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


#### Limitations and bias


model name: indic-transformers-bn-xlmroberta


model author: neuralspace-reverie


task: fill-mask


tags: pytorch, tf, xlm-roberta, fill-mask, bn, transformers, MaskedLM, Bengali, XLMRoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible


- bn
tags:
- MaskedLM
- Bengali
- DistilBERT
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Bengali DistilBERT


This is a DistilBERT language model pre-trained on ~6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-distilbert')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-distilbert')
text = "আপনি কেমন আছেন?"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


model name: indic-transformers-bn-distilbert


tags: pytorch, tf, distilbert, fill-mask, bn, transformers, MaskedLM, Bengali, DistilBERT, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


language:


- hi
tags:
- MaskedLM
- Hindi
- DistilBERT
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Hindi DistilBERT


## Model description


This is a DistilBERT language model pre-trained on ~10 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/).
This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training.


## Intended uses & limitations


#### How to use


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-distilbert')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-distilbert')
text = "आपका स्वागत हैं"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


#### Limitations and bias


The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html).


model name: indic-transformers-hi-distilbert


model author: neuralspace-reverie


task: fill-mask


tags: pytorch, tf, distilbert, fill-mask, hi, transformers, MaskedLM, Hindi, DistilBERT, Question-Answering, Token Classification, Text Classification, autonlp_compatible, infinity_compatible


- hi
tags:
- MaskedLM
- Hindi
- XLMRoBERTa
- Question-Answering
- Token Classification
- Text Classification


# Indic-Transformers Hindi XLMRoBERTa


```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-xlmroberta')
model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-xlmroberta')
text = "आपका स्वागत हैं"
input_ids = tokenizer(text, return_tensors='pt')['input_ids']
out = model(input_ids)[0]
print(out.shape)
# out = [1, 5, 768]
```


model name: indic-transformers-hi-xlmroberta


tags: pytorch, tf, xlm-roberta, fill-mask, hi, transformers, MaskedLM, Hindi, XLMRoBERTa, Question-Answering, Token Classification, Text Classification, autonlp_compatible


language:


- ca
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
- generated_from_trainer
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
model-index:
- name: wav2vec2-xls-r-300m-ca-lm
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: mozilla-foundation/common_voice_8_0 ca
type: mozilla-foundation/common_voice_8_0
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.06771703090587865
    - name: Test CER
type: cer
value: 0.021007777843712293
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: projecte-aina/parlament_parla ca
type: projecte-aina/parlament_parla
args: clean
metrics:
    - name: Test WER
type: wer
value: 0.05565360630662431
    - name: Test CER
type: cer
value: 0.018594390167034354
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: collectivat/tv3_parla ca
type: collectivat/tv3_parla
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.1353312545713516
    - name: Test CER
type: cer
value: 0.08684635913340556
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Catalan Dev Data
type: speech-recognition-community-v2/dev_data
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.2604515843400164
    - name: Test CER
type: cer
value: 0.15056890012642224


# wav2vec2-xls-r-300m-ca-lm


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - CA, the [tv3_parla](https://huggingface.co/datasets/collectivat/tv3_parla) and [parlament_parla](https://huggingface.co/datasets/projecte-aina/parlament_parla) datasets.
It achieves the following results on the evaluation set (for the three datasets and without the LM):


- Loss: 0.2472
- Wer: 0.1499


## Model description


Please check the original [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) Model card. This is just a finetuned version of that model.


## Intended uses & limitations


As any model trained on crowdsourced data, this model can show the biases and particularities of the data and model used to train this model. Moreover, since this is a speech recognition model, it may underperform for some lower-resourced dialects for the catalan language.


## Training and evaluation data


More information needed


## Training procedure


The data is preprocessed to remove characters not on the catalan alphabet. Moreover, numbers are verbalized using code provided by [@ccoreilly](https://github.com/ccoreilly), which can be found on the text/ folder or [here](https://github.com/CollectivaT-dev/catotron-cpu/blob/master/text/numbers_ca.py).


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 7.5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 18.0
- mixed_precision_training: Native AMP


### Training results


Check the Tensorboard tab to check the training profile and evaluation results along training. The model was evaluated on the test splits for each of the datasets used during training.


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 6.2099        | 0.09  | 500   | 3.4125          | 1.0    |
| 2.9961        | 0.18  | 1000  | 2.9224          | 1.0    |
| 2.2147        | 0.26  | 1500  | 0.6521          | 0.5568 |
| 1.3017        | 0.35  | 2000  | 0.3153          | 0.2761 |
| 1.1196        | 0.44  | 2500  | 0.2444          | 0.2367 |
| 1.0712        | 0.53  | 3000  | 0.2324          | 0.2132 |
| 1.052         | 0.62  | 3500  | 0.2173          | 0.2032 |
| 1.2813        | 2.13  | 4000  | 0.3326          | 0.2099 |
| 1.2365        | 2.4   | 4500  | 0.3224          | 0.2003 |
| 1.2193        | 2.66  | 5000  | 0.3198          | 0.1957 |
| 1.2072        | 2.93  | 5500  | 0.3063          | 0.1933 |
| 1.213         | 3.2   | 6000  | 0.3051          | 0.1980 |
| 1.2074        | 3.46  | 6500  | 0.3012          | 0.1879 |
| 1.1918        | 3.73  | 7000  | 0.2947          | 0.1829 |
| 1.1893        | 4.0   | 7500  | 0.2895          | 0.1807 |
| 1.1751        | 4.26  | 8000  | 0.2878          | 0.1776 |
| 1.1628        | 4.53  | 8500  | 0.2835          | 0.1731 |
| 1.1577        | 4.79  | 9000  | 0.2816          | 0.1761 |
| 1.1448        | 5.06  | 9500  | 0.2757          | 0.1740 |
| 1.1407        | 5.33  | 10000 | 0.2768          | 0.1798 |
| 1.1401        | 5.59  | 10500 | 0.2780          | 0.1816 |
| 1.1333        | 5.86  | 11000 | 0.2748          | 0.1750 |
| 1.1571        | 6.13  | 11500 | 0.2808          | 0.1708 |
| 1.1505        | 6.39  | 12000 | 0.2726          | 0.1692 |
| 1.1519        | 6.66  | 12500 | 0.2749          | 0.1654 |
| 1.136         | 6.93  | 13000 | 0.2765          | 0.1643 |
| 1.1326        | 7.19  | 13500 | 0.2706          | 0.1668 |
| 1.1342        | 7.46  | 14000 | 0.2665          | 0.1638 |
| 1.1286        | 7.72  | 14500 | 0.2669          | 0.1636 |
| 1.1243        | 7.99  | 15000 | 0.2619          | 0.1623 |
| 1.1173        | 8.26  | 15500 | 0.2652          | 0.1604 |
| 1.1129        | 8.52  | 16000 | 0.2610          | 0.1598 |
| 1.1091        | 8.79  | 16500 | 0.2608          | 0.1584 |
| 1.1053        | 9.06  | 17000 | 0.2633          | 0.1664 |
| 1.1004        | 9.32  | 17500 | 0.2594          | 0.1662 |
| 1.0995        | 9.59  | 18000 | 0.2623          | 0.1569 |
| 1.0964        | 9.86  | 18500 | 0.2624          | 0.1597 |
| 1.09          | 10.12 | 19000 | 0.2577          | 0.1578 |
| 1.089         | 10.39 | 19500 | 0.2574          | 0.1531 |
| 1.0864        | 10.66 | 20000 | 0.2556          | 0.1546 |
| 1.0806        | 10.92 | 20500 | 0.2548          | 0.1583 |
| 1.0842        | 11.19 | 21000 | 0.2550          | 0.1542 |
| 1.0805        | 11.45 | 21500 | 0.2561          | 0.1524 |
| 1.0722        | 11.72 | 22000 | 0.2540          | 0.1566 |
| 1.0763        | 11.99 | 22500 | 0.2549          | 0.1572 |
| 1.0835        | 12.25 | 23000 | 0.2586          | 0.1521 |
| 1.0883        | 12.52 | 23500 | 0.2583          | 0.1519 |
| 1.0888        | 12.79 | 24000 | 0.2551          | 0.1582 |
| 1.0933        | 13.05 | 24500 | 0.2628          | 0.1537 |
| 1.0799        | 13.32 | 25000 | 0.2600          | 0.1508 |
| 1.0804        | 13.59 | 25500 | 0.2620          | 0.1475 |
| 1.0814        | 13.85 | 26000 | 0.2537          | 0.1517 |
| 1.0693        | 14.12 | 26500 | 0.2560          | 0.1542 |
| 1.0724        | 14.38 | 27000 | 0.2540          | 0.1574 |
| 1.0704        | 14.65 | 27500 | 0.2548          | 0.1626 |
| 1.0729        | 14.92 | 28000 | 0.2548          | 0.1601 |
| 1.0724        | 15.18 | 28500 | 0.2511          | 0.1512 |
| 1.0655        | 15.45 | 29000 | 0.2498          | 0.1490 |
| 1.0608        | 15.98 | 30000 | 0.2487          | 0.1481 |
| 1.0541        | 16.52 | 31000 | 0.2468          | 0.1504 |
| 1.0584        | 17.05 | 32000 | 0.2467          | 0.1493 |
| 1.0507        | 17.58 | 33000 | 0.2481          | 0.1517 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.18.3
- Tokenizers 0.11.0


# Thanks


Want to thank both [@ccoreilly](https://github.com/ccoreilly) and [@gullabi](https://github.com/gullabi) who have contributed with their own resources and knowledge into making this model possible.


model name: wav2vec2-xls-r-300m-ca-lm


model author: PereLluis13


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, ca, dataset:mozilla-foundation/common_voice_8_0, dataset:collectivat/tv3_parla, dataset:projecte-aina/parlament_parla, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, collectivat/tv3_parla, projecte-aina/parlament_parla, generated_from_trainer, robust-speech-event, model-index


language: ca
datasets:


- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Catalan XLSR Wav2Vec Large 53 #TODO: replace {human_readable_name} with a name of your model as it should appear on the leaderboard. It could be something like `Elgeish XLSR Wav2Vec2 Large 53`
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ca
type: common_voice
args: ca #TODO:
metrics:
    - name: Test WER
type: wer
value: 8.11


# Wav2Vec2-Large-XLSR-53-ca


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on catalan using the [Common Voice](https://huggingface.co/datasets/common_voice) dataset.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ca", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("PereLluis13/Wav2Vec2-Large-XLSR-53-catalan")
model = Wav2Vec2ForCTC.from_pretrained("PereLluis13/Wav2Vec2-Large-XLSR-53-catalan")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the catalan test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "ca", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("PereLluis13/Wav2Vec2-Large-XLSR-53-catalan")
model = Wav2Vec2ForCTC.from_pretrained("PereLluis13/Wav2Vec2-Large-XLSR-53-catalan")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\;\:\"\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)
import jiwer

# Chunk WER computation due to memory issues, taken from https://huggingface.co/pcuenq/wav2vec2-large-xlsr-53-es
def chunked_wer(targets, predictions, chunk_size=None):
    if chunk_size is None: return jiwer.wer(targets, predictions)
    start = 0
    end = chunk_size
    H, S, D, I = 0, 0, 0, 0
    while start < len(targets):
        chunk_metrics = jiwer.compute_measures(targets[start:end], predictions[start:end])
        H = H + chunk_metrics["hits"]
        S = S + chunk_metrics["substitutions"]
        D = D + chunk_metrics["deletions"]
        I = I + chunk_metrics["insertions"]
        start += chunk_size
        end += chunk_size
    return float(S + D + I) / float(H + S + D)

print("WER: {:2f}".format(100 * chunked_wer(result["sentence"], result["pred_strings"], chunk_size=4000)))
```


**Test Result**: 8.11 %


## Training


The Common Voice `train`, `validation` datasets were used for training. At the second epoch training was halted due to a memory issue, and was continued with lower batch size, but acc. gradient steps were scaled to keep it at 32 batch size during all training. Then the model was trained for an additional 10 epochs where half the male samples were pitched up.


The script used for training can be found [here](https://github.com/huggingface/transformers/blob/master/examples/research_projects/wav2vec2/run_common_voice.py). Slight modifications were done in order to speed up the ordering by length during training, which can be found [here](https://discuss.huggingface.co/t/spanish-asr-fine-tuning-wav2vec2/4586/6). Another version trained for catalan can be found [here](https://huggingface.co/ccoreilly/wav2vec2-large-xlsr-catala), which may be better than this one since it was trained with extra data and for longer time. Whoever, since it used different splits that include part of the Common Voice test set, this version can be used to get a baseline on the Common Voice dataset.


model name: Wav2Vec2-Large-XLSR-53-catalan


model author: PereLluis13


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, ca, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language:


- ca
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
- generated_from_trainer
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
model-index:
- name: wav2vec2-xls-r-300m-ca
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: mozilla-foundation/common_voice_8_0 ca
type: mozilla-foundation/common_voice_8_0
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.13170091241317552
    - name: Test CER
type: cer
value: 0.03356726205534543
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: projecte-aina/parlament_parla ca
type: projecte-aina/parlament_parla
args: clean
metrics:
    - name: Test WER
type: wer
value: 0.08048005647723261
    - name: Test CER
type: cer
value: 0.02240912911020065
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: collectivat/tv3_parla ca
type: collectivat/tv3_parla
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.23320629787889285
    - name: Test CER
type: cer
value: 0.10439216202089989
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: speech-recognition-community-v2/dev_data ca
type: speech-recognition-community-v2/dev_data
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.3199671115046487
    - name: Test CER
type: cer
value: 0.15820020687277325


# wav2vec2-xls-r-300m-ca


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - CA, the [tv3_parla](https://huggingface.co/datasets/collectivat/tv3_parla) and [parlament_parla](https://huggingface.co/datasets/projecte-aina/parlament_parla) datasets.
It achieves the following results on the evaluation set (for the three datasets):


- Loss: 0.2472
- Wer: 0.1499


## Model description


Please check the original [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) Model card. This is just a finetuned version of that model.


## Intended uses & limitations


As any model trained on crowdsourced data, this model can show the biases and particularities of the data and model used to train this model. Moreover, since this is a speech recognition model, it may underperform for some lower-resourced dialects for the catalan language.


## Training and evaluation data


More information needed


## Training procedure


The data is preprocessed to remove characters not on the catalan alphabet. Moreover, numbers are verbalized using code provided by [@ccoreilly](https://github.com/ccoreilly), which can be found on the text/ folder or [here](https://github.com/CollectivaT-dev/catotron-cpu/blob/master/text/numbers_ca.py).


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 7.5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 18.0
- mixed_precision_training: Native AMP


### Training results


Check the Tensorboard tab to check the training profile and evaluation results along training. The model was evaluated on the test splits for each of the datasets used during training.


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 6.2099        | 0.09  | 500   | 3.4125          | 1.0    |
| 2.9961        | 0.18  | 1000  | 2.9224          | 1.0    |
| 2.2147        | 0.26  | 1500  | 0.6521          | 0.5568 |
| 1.3017        | 0.35  | 2000  | 0.3153          | 0.2761 |
| 1.1196        | 0.44  | 2500  | 0.2444          | 0.2367 |
| 1.0712        | 0.53  | 3000  | 0.2324          | 0.2132 |
| 1.052         | 0.62  | 3500  | 0.2173          | 0.2032 |
| 1.2813        | 2.13  | 4000  | 0.3326          | 0.2099 |
| 1.2365        | 2.4   | 4500  | 0.3224          | 0.2003 |
| 1.2193        | 2.66  | 5000  | 0.3198          | 0.1957 |
| 1.2072        | 2.93  | 5500  | 0.3063          | 0.1933 |
| 1.213         | 3.2   | 6000  | 0.3051          | 0.1980 |
| 1.2074        | 3.46  | 6500  | 0.3012          | 0.1879 |
| 1.1918        | 3.73  | 7000  | 0.2947          | 0.1829 |
| 1.1893        | 4.0   | 7500  | 0.2895          | 0.1807 |
| 1.1751        | 4.26  | 8000  | 0.2878          | 0.1776 |
| 1.1628        | 4.53  | 8500  | 0.2835          | 0.1731 |
| 1.1577        | 4.79  | 9000  | 0.2816          | 0.1761 |
| 1.1448        | 5.06  | 9500  | 0.2757          | 0.1740 |
| 1.1407        | 5.33  | 10000 | 0.2768          | 0.1798 |
| 1.1401        | 5.59  | 10500 | 0.2780          | 0.1816 |
| 1.1333        | 5.86  | 11000 | 0.2748          | 0.1750 |
| 1.1571        | 6.13  | 11500 | 0.2808          | 0.1708 |
| 1.1505        | 6.39  | 12000 | 0.2726          | 0.1692 |
| 1.1519        | 6.66  | 12500 | 0.2749          | 0.1654 |
| 1.136         | 6.93  | 13000 | 0.2765          | 0.1643 |
| 1.1326        | 7.19  | 13500 | 0.2706          | 0.1668 |
| 1.1342        | 7.46  | 14000 | 0.2665          | 0.1638 |
| 1.1286        | 7.72  | 14500 | 0.2669          | 0.1636 |
| 1.1243        | 7.99  | 15000 | 0.2619          | 0.1623 |
| 1.1173        | 8.26  | 15500 | 0.2652          | 0.1604 |
| 1.1129        | 8.52  | 16000 | 0.2610          | 0.1598 |
| 1.1091        | 8.79  | 16500 | 0.2608          | 0.1584 |
| 1.1053        | 9.06  | 17000 | 0.2633          | 0.1664 |
| 1.1004        | 9.32  | 17500 | 0.2594          | 0.1662 |
| 1.0995        | 9.59  | 18000 | 0.2623          | 0.1569 |
| 1.0964        | 9.86  | 18500 | 0.2624          | 0.1597 |
| 1.09          | 10.12 | 19000 | 0.2577          | 0.1578 |
| 1.089         | 10.39 | 19500 | 0.2574          | 0.1531 |
| 1.0864        | 10.66 | 20000 | 0.2556          | 0.1546 |
| 1.0806        | 10.92 | 20500 | 0.2548          | 0.1583 |
| 1.0842        | 11.19 | 21000 | 0.2550          | 0.1542 |
| 1.0805        | 11.45 | 21500 | 0.2561          | 0.1524 |
| 1.0722        | 11.72 | 22000 | 0.2540          | 0.1566 |
| 1.0763        | 11.99 | 22500 | 0.2549          | 0.1572 |
| 1.0835        | 12.25 | 23000 | 0.2586          | 0.1521 |
| 1.0883        | 12.52 | 23500 | 0.2583          | 0.1519 |
| 1.0888        | 12.79 | 24000 | 0.2551          | 0.1582 |
| 1.0933        | 13.05 | 24500 | 0.2628          | 0.1537 |
| 1.0799        | 13.32 | 25000 | 0.2600          | 0.1508 |
| 1.0804        | 13.59 | 25500 | 0.2620          | 0.1475 |
| 1.0814        | 13.85 | 26000 | 0.2537          | 0.1517 |
| 1.0693        | 14.12 | 26500 | 0.2560          | 0.1542 |
| 1.0724        | 14.38 | 27000 | 0.2540          | 0.1574 |
| 1.0704        | 14.65 | 27500 | 0.2548          | 0.1626 |
| 1.0729        | 14.92 | 28000 | 0.2548          | 0.1601 |
| 1.0724        | 15.18 | 28500 | 0.2511          | 0.1512 |
| 1.0655        | 15.45 | 29000 | 0.2498          | 0.1490 |
| 1.0608        | 15.98 | 30000 | 0.2487          | 0.1481 |
| 1.0541        | 16.52 | 31000 | 0.2468          | 0.1504 |
| 1.0584        | 17.05 | 32000 | 0.2467          | 0.1493 |
| 1.0507        | 17.58 | 33000 | 0.2481          | 0.1517 |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.18.3
- Tokenizers 0.11.0


# Thanks


Want to thank both [@ccoreilly](https://github.com/ccoreilly) and [@gullabi](https://github.com/gullabi) who have contributed with their own resources and knowledge into making this model possible.


model name: wav2vec2-xls-r-300m-ca


model author: PereLluis13


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, ca, dataset:mozilla-foundation/common_voice_8_0, dataset:collectivat/tv3_parla, dataset:projecte-aina/parlament_parla, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, collectivat/tv3_parla, projecte-aina/parlament_parla, generated_from_trainer, robust-speech-event, model-index


language:


- ca
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
- generated_from_trainer
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
model-index:
- name: wav2vec2-xls-r-1b-ca
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: mozilla-foundation/common_voice_8_0 ca
type: mozilla-foundation/common_voice_8_0
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.11030639657300516
    - name: Test CER
type: cer
value: 0.028405630530040634
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: projecte-aina/parlament_parla ca
type: projecte-aina/parlament_parla
args: clean
metrics:
    - name: Test WER
type: wer
value: 0.06483115660665961
    - name: Test CER
type: cer
value: 0.020212863746191828
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: collectivat/tv3_parla ca
type: collectivat/tv3_parla
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.17917773414943988
    - name: Test CER
type: cer
value: 0.08872589572206396
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Catalan Dev Data
type: speech-recognition-community-v2/dev_data
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.27126683954209097
    - name: Test CER
type: cer
value: 0.14213308815078726


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-1b-ca


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - CA, the [tv3_parla](https://huggingface.co/datasets/collectivat/tv3_parla) and [parlament_parla](https://huggingface.co/datasets/projecte-aina/parlament_parla) datasets.


## Model description


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training results


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 10.0
- mixed_precision_training: Native AMP


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.3
- Tokenizers 0.11.0


# Thanks


model name: wav2vec2-xls-r-1b-ca


model author: PereLluis13


task: automatic-speech-recognition


language:


- ca
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
- generated_from_trainer
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
- collectivat/tv3_parla
- projecte-aina/parlament_parla
model-index:
- name: wav2vec2-xls-r-1b-ca-lm
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: mozilla-foundation/common_voice_8_0 ca
type: mozilla-foundation/common_voice_8_0
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.060722669958130644
    - name: Test CER
type: cer
value: 0.019180697705166526
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: projecte-aina/parlament_parla ca
type: projecte-aina/parlament_parla
args: clean
metrics:
    - name: Test WER
type: wer
value: 0.05139820371024042
    - name: Test CER
type: cer
value: 0.020163620128164722
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: collectivat/tv3_parla ca
type: collectivat/tv3_parla
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.11207991684952073
    - name: Test CER
type: cer
value: 0.0732119307305963
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Robust Speech Event - Catalan Dev Data
type: speech-recognition-community-v2/dev_data
args: ca
metrics:
    - name: Test WER
type: wer
value: 0.22870153690468661
    - name: Test CER
type: cer
value: 0.1359039190897598


# wav2vec2-xls-r-1b-ca-lm


model name: wav2vec2-xls-r-1b-ca-lm


language: el #TODO: replace {lang_id} in your language code here. Make sure the code is one of the *ISO codes* of [this](https://huggingface.co/languages) site.
datasets:


- common_voice #TODO: remove if you did not use the common voice dataset
- CSS10
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Greek XLSR Wav2Vec2 Large 53 - CV + CSS10 #TODO: replace {human_readable_name} with a name of your model as it should appear on the leaderboard. It could be something like `Elgeish XLSR Wav2Vec2 Large 53`
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice el #TODO: replace {lang_id} in your language code here. Make sure the code is one of the *ISO codes* of [this](https://huggingface.co/languages) site.
type: common_voice
args: el #TODO: replace {lang_id} in your language code here. Make sure the code is one of the *ISO codes* of [this](https://huggingface.co/languages) site.
metrics:
    - name: Test WER
type: wer
value: 20.89 #TODO (IMPORTANT): replace {wer_result_on_test} with the WER error rate you achieved on the common_voice test set. It should be in the format XX.XX (don't add the % sign here). **Please** remember to fill out this value after you evaluated your model, so that your model appears on the leaderboard. If you fill out this model card before evaluating your model, please remember to edit the model card afterward to fill in your value


# Wav2Vec2-Large-XLSR-53-greek


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on greek using the [Common Voice](https://huggingface.co/datasets/common_voice) and [CSS10](https://github.com/Kyubyong/css10) datasets.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "el", split="test")

processor = Wav2Vec2Processor.from_pretrained("PereLluis13/wav2vec2-large-xlsr-53-greek")
model = Wav2Vec2ForCTC.from_pretrained("PereLluis13/wav2vec2-large-xlsr-53-greek")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the greek test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "el", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("PereLluis13/wav2vec2-large-xlsr-53-greek")
model = Wav2Vec2ForCTC.from_pretrained("PereLluis13/wav2vec2-large-xlsr-53-greek")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�]'

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 20.89 %


## Training


The Common Voice `train`, `validation`, and CSS10 datasets were used for training, added as `extra` split to the dataset. The sampling rate and format of the CSS10 files is different, hence the function `speech_file_to_array_fn` was changed to:


```
    def speech_file_to_array_fn(batch):
        try:
            speech_array, sampling_rate = sf.read(batch["path"] + ".wav")
        except:
            speech_array, sampling_rate = librosa.load(batch["path"], sr = 16000, res_type='zero_order_hold')
            sf.write(batch["path"] + ".wav", speech_array, sampling_rate, subtype='PCM_24')
        batch["speech"] = speech_array
        batch["sampling_rate"] = sampling_rate
        batch["target_text"] = batch["text"]
        return batch
```


As suggested by [Florian Zimmermeister](https://github.com/flozi00).


The script used for training can be found in [run_common_voice.py](examples/research_projects/wav2vec2/run_common_voice.py), still pending of PR. The only changes are to `speech_file_to_array_fn`. Batch size was kept at 32 (using `gradient_accumulation_steps`) using one of the [OVH](https://www.ovh.com/) machines, with a V100 GPU (thank you very much [OVH](https://www.ovh.com/)). The model trained for 40 epochs, the first 20 with the `train+validation` splits, and then `extra` split was added with the data from CSS10 at the 20th epoch.


model name: wav2vec2-large-xlsr-53-greek


model author: PereLluis13


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, el, dataset:common_voice, dataset:CSS10, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


# HateSpeechDetection


## pipeline_tag: text-classification


The model is used for classifying a text as Hatespeech or Normal. The model is trained using data from Twitter, specifically Kenyan related tweets. To maximize on the limited dataset, text augmentation was done.


The dataset is available here: https://github.com/sgich/HateSpeechDetection


Using a pre-trained "bert-base-uncased" transformer model, adding a dropout layer, a linear output layer and adding 10 common emojis that may be related to either Hate or Normal Speech. Then the model was tuned on a dataset of Kenyan/Kenyan-related scraped tweets with the purpose of performing text classification of "Normal Speech" or "Hate Speech" based on the text. This model was the result of realizing that majority of similar models did not cater for the African context where the target groups are not based on race and/or religious affiliation but mostly tribal differences which has proved fatal in the past.


The model can be improved greatly by using a large and representative dataset and optimization of the model to a better degree.


model name: bert_case_uncased_KenyaHateSpeech


model author: sgich


task: text-classification


tags: pytorch, text-classification


tags:


- conversational


# Evan Model


model name: DialoGPT-evan


model author: jchen


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


# Evelynn DialoGPT Model


model name: DialoGPT-small-evelynn


model author: Stabley


language: en
tags:


- bart
- seq2seq
- summarization
license: apache-2.0
datasets:
- samsum
widget:
- text: |
Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
model-index:
- name: bart-base-samsum
results:
  - task:
name: Abstractive Text Summarization
type: abstractive-text-summarization
dataset:
name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
type: samsum
metrics:
    - name: Validation ROGUE-1
type: rogue-1
value: 46.6619
    - name: Validation ROGUE-2
type: rogue-2
value: 23.3285
    - name: Validation ROGUE-L
type: rogue-l
value: 39.4811
    - name: Test ROGUE-1
type: rogue-1
value: 44.9932
    - name: Test ROGUE-2
type: rogue-2
value: 21.7286
    - name: Test ROGUE-L
type: rogue-l
value: 38.1921


## `bart-base-samsum`


This model was obtained by fine-tuning `facebook/bart-base` on Samsum dataset.


## Usage


```python
from transformers import pipeline

summarizer = pipeline("summarization", model="lidiya/bart-base-samsum")
conversation = '''Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
'''
summarizer(conversation)
```


## Training procedure


- Colab notebook: https://colab.research.google.com/drive/1RInRjLLso9E2HG_xjA6j8JO3zXzSCBRF?usp=sharing


## Results


| key | value |
| --- | ----- |
| eval_rouge1 | 46.6619 |
| eval_rouge2 | 23.3285 |
| eval_rougeL | 39.4811 |
| eval_rougeLsum | 43.0482 |
| test_rouge1 | 44.9932 |
| test_rouge2 | 21.7286 |
| test_rougeL | 38.1921 |
| test_rougeLsum | 41.2672 |


model name: bart-base-samsum


model author: lidiya


task: summarization


tags: pytorch, bart, text2text-generation, en, dataset:samsum, transformers, seq2seq, summarization, license:apache-2.0, model-index, autonlp_compatible


- bart
- seq2seq
- summarization
license: apache-2.0
datasets:
- samsum
widget:
- text: |
Hannah: Hey, do you have Betty's number?
Amanda: Lemme check
Amanda: Sorry, can't find it.
Amanda: Ask Larry
Amanda: He called her last time we were at the park together
Hannah: I don't know him well
Amanda: Don't be shy, he's very nice
Hannah: If you say so..
Hannah: I'd rather you texted him
Amanda: Just text him 🙂
Hannah: Urgh.. Alright
Hannah: Bye
Amanda: Bye bye
model-index:
- name: bart-large-xsum-samsum
results:
  - task:
name: Abstractive Text Summarization
type: abstractive-text-summarization
dataset:
name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
type: samsum
metrics:
    - name: Validation ROGUE-1
type: rogue-1
value: 54.3921
    - name: Validation ROGUE-2
type: rogue-2
value: 29.8078
    - name: Validation ROGUE-L
type: rogue-l
value: 45.1543
    - name: Test ROGUE-1
type: rogue-1
value: 53.3059
    - name: Test ROGUE-2
type: rogue-2
value: 28.355
    - name: Test ROGUE-L
type: rogue-l
value: 44.0953


## `bart-large-xsum-samsum`


This model was obtained by fine-tuning `facebook/bart-large-xsum` on [Samsum](https://huggingface.co/datasets/samsum) dataset.


```python
from transformers import pipeline

summarizer = pipeline("summarization", model="lidiya/bart-large-xsum-samsum")
conversation = '''Hannah: Hey, do you have Betty's number?
Amanda: Lemme check
Amanda: Sorry, can't find it.
Amanda: Ask Larry
Amanda: He called her last time we were at the park together
Hannah: I don't know him well
Amanda: Don't be shy, he's very nice
Hannah: If you say so..
Hannah: I'd rather you texted him
Amanda: Just text him 🙂
Hannah: Urgh.. Alright
Hannah: Bye
Amanda: Bye bye
'''
summarizer(conversation)
```


- Colab notebook: https://colab.research.google.com/drive/1dul0Sg-TTMy9xZCJzmDRajXbyzDwtYx6?usp=sharing


| key | value |
| --- | ----- |
| eval_rouge1 | 54.3921 |
| eval_rouge2 | 29.8078 |
| eval_rougeL | 45.1543 |
| eval_rougeLsum | 49.942 |
| test_rouge1 | 53.3059 |
| test_rouge2 | 28.355 |
| test_rougeL | 44.0953 |
| test_rougeLsum | 48.9246 |


model name: bart-large-xsum-samsum


tags:


- conversational


# Discord Model Medium 7 epochs


model name: DialoGPT-medium-Discord-1.0


model author: transfaeries


task: conversational


# Discord


model name: DialoGPT-Discord


tags: conversational


# Twilight Model Medium 13 epochs


model name: Twilight-Sparkle-GPT


# Discord Model


model name: DialoGPT-small-Discord-1.0


language: en
tags:


- t5-small
datasets:
- Gigaword
- DUC2004


## Multi-Summary Reinforcement Learning for Unsupervised Length-Controllable Summarization


This model (MSRP) is trained on Gigaword data.


### How to use


```python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer("anonsubms/msrp")
model = AutoModel("anonsubms/msrp")
tar_len = "8"
doc = tar_len+": Replace me by a news article"
encoded_input = tokenizer(doc, return_tensors='pt')
summary = model(**encoded_input)
```


model name: msrp


model author: anonsubms


task: text2text-generation


tags: pytorch, t5, text2text-generation, en, dataset:Gigaword, dataset:DUC2004, transformers, t5-small, autonlp_compatible


tags:


- generated_from_trainer
datasets:
- jnlpba
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: biobert-base-cased-v1.2-finetuned-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: jnlpba
type: jnlpba
args: jnlpba
metrics:
    - name: Precision
type: precision
value: 0.7150627220423177
    - name: Recall
type: recall
value: 0.8300729927007299
    - name: F1
type: f1
value: 0.7682875335686659
    - name: Accuracy
type: accuracy
value: 0.90497239665345


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# biobert-base-cased-v1.2-finetuned-ner


This model is a fine-tuned version of [dmis-lab/biobert-base-cased-v1.2](https://huggingface.co/dmis-lab/biobert-base-cased-v1.2) on the jnlpba dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3655
- Precision: 0.7151
- Recall: 0.8301
- F1: 0.7683
- Accuracy: 0.9050


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.257         | 1.0   | 1160 | 0.2889          | 0.7091    | 0.8222 | 0.7615 | 0.9021   |
| 0.1962        | 2.0   | 2320 | 0.3009          | 0.7154    | 0.8259 | 0.7667 | 0.9048   |
| 0.158         | 3.0   | 3480 | 0.3214          | 0.7098    | 0.8228 | 0.7621 | 0.9031   |
| 0.131         | 4.0   | 4640 | 0.3385          | 0.7174    | 0.8292 | 0.7692 | 0.9055   |
| 0.1081        | 5.0   | 5800 | 0.3655          | 0.7151    | 0.8301 | 0.7683 | 0.9050   |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.9.1+cu102
- Datasets 1.13.2
- Tokenizers 0.10.3


model name: biobert-base-cased-v1.2-finetuned-ner


model author: sciarrilli


task: token-classification


tags: pytorch, bert, token-classification, dataset:jnlpba, transformers, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: distilbert-base-uncased-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.5301312348234369


# distilbert-base-uncased-cola


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 1.2715
- Matthews Correlation: 0.5301


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5216        | 1.0   | 535  | 0.5124          | 0.4104               |
| 0.3456        | 2.0   | 1070 | 0.5700          | 0.4692               |
| 0.2362        | 3.0   | 1605 | 0.7277          | 0.4844               |
| 0.1818        | 4.0   | 2140 | 0.7553          | 0.5007               |
| 0.1509        | 5.0   | 2675 | 0.9406          | 0.4987               |
| 0.1017        | 6.0   | 3210 | 0.9475          | 0.5387               |
| 0.0854        | 7.0   | 3745 | 1.0933          | 0.5317               |
| 0.051         | 8.0   | 4280 | 1.1719          | 0.5358               |
| 0.0512        | 9.0   | 4815 | 1.2296          | 0.5321               |
| 0.0308        | 10.0  | 5350 | 1.2715          | 0.5301               |


### Framework versions


- Transformers 4.12.3
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: distilbert-base-uncased-cola


model author: sciarrilli


task: text-classification


tags: pytorch, tensorboard, distilbert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


tags:


- conversational


#Rick DialoGPT model


model name: DialoGPT-medium-Rick


model author: MrDuckerino


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


## license: afl-3.0


model name: dall-e


model author: cerulianx


tags: license:afl-3.0


Test for First Model


model name: bedrooms


model author: shelly


Hyperparameters: 1 epoch, max_len_dict, and 1e-5 learning rate


model name: sgd-t5-tod


model author: SoLID


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, autonlp_compatible


## Schema Guided Dialogue Output Plan Constructor


model name: sgd-output-plan-constructor


language: ja
thumbnail: https://github.com/rinnakk/japanese-gpt2/blob/master/rinna.png
tags:


- ja
- japanese
- gpt2
- text-generation
- lm
- nlp
license: mit
datasets:
- cc100
- wikipedia
widget:
- text: "生命、宇宙、そして万物についての究極の疑問の答えは"


# japanese-gpt2-xsmall


![rinna-icon](./rinna.png)


This repository provides an extra-small-sized Japanese GPT-2 model. The model was trained using code from Github repository [rinnakk/japanese-pretrained-models](https://github.com/rinnakk/japanese-pretrained-models) by [rinna Co., Ltd.](https://corp.rinna.co.jp/)


# How to use the model


*NOTE:* Use `T5Tokenizer` to initiate the tokenizer.


```
from transformers import T5Tokenizer, GPT2LMHeadModel

tokenizer = T5Tokenizer.from_pretrained("rinna/japanese-gpt2-small")
tokenizer.do_lower_case = True  # due to some bug of tokenizer config loading

model = GPT2LMHeadModel.from_pretrained("rinna/japanese-gpt2-small")
```


# Model architecture


A 6-layer, 512-hidden-size transformer-based language model.


# Training


The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz) and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective on 8\\*V100 GPUs for around 4 days. It reaches around 28 perplexity on a chosen validation set from CC-100.


# Tokenization


The model uses a [sentencepiece](https://github.com/google/sentencepiece)-based tokenizer, the vocabulary was trained on the Japanese Wikipedia using the official sentencepiece training script.


# Licenese


[The MIT license](https://opensource.org/licenses/MIT)


model name: japanese-gpt2-xsmall


model author: rinna


task: text-generation


tags: pytorch, tf, gpt2, text-generation, ja, dataset:cc100, dataset:wikipedia, transformers, japanese, lm, nlp, license:mit


# japanese-gpt2-small


![rinna-icon](./rinna.png)


This repository provides a small-sized Japanese GPT-2 model. The model was trained using code from Github repository [rinnakk/japanese-pretrained-models](https://github.com/rinnakk/japanese-pretrained-models) by [rinna Co., Ltd.](https://corp.rinna.co.jp/)


# How to use the model


A 12-layer, 768-hidden-size transformer-based language model.


The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz) and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective on 8\\*V100 GPUs for around 15 days. It reaches around 21 perplexity on a chosen validation set from CC-100.


model name: japanese-gpt2-small


language: ja
thumbnail: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png
tags:


- ja
- japanese
- gpt
- text-generation
- lm
- nlp
license: mit
datasets:
- cc100
- wikipedia
widget:
- text: "西田幾多郎は、"


# japanese-gpt-1b


This repository provides a 1.3B-parameter Japanese GPT model. The model was trained by [rinna Co., Ltd.](https://corp.rinna.co.jp/)


```
import torch
from transformers import T5Tokenizer, AutoModelForCausalLM

tokenizer = T5Tokenizer.from_pretrained("rinna/japanese-gpt-1b")
model = AutoModelForCausalLM.from_pretrained("rinna/japanese-gpt-1b")

if torch.cuda.is_available():
    model = model.to("cuda")

text = "西田幾多郎は、"
token_ids = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt")

with torch.no_grad():
    output_ids = model.generate(
        token_ids.to(model.device),
        max_length=100,
        min_length=100,
        do_sample=True,
        top_k=500,
        top_p=0.95,
        pad_token_id=tokenizer.pad_token_id,
        bos_token_id=tokenizer.bos_token_id,
        eos_token_id=tokenizer.eos_token_id,
        bad_word_ids=[[tokenizer.unk_token_id]]
    )

output = tokenizer.decode(output_ids.tolist()[0])
print(output)
# sample output: 西田幾多郎は、その主著の「善の研究」などで、人間の内面に自然とその根源があると指摘し、その根源的な性格は、この西田哲学を象徴しているとして、カントの「純粋理性批判」と「判断力批判」を対比して捉えます。それは、「人が理性的存在であるかぎりにおいて、人はその当人に固有な道徳的に自覚された善悪の基準を持っている」とするもので、この理性的な善悪の観念を否定するのがカントの
```


# Model architecture


A 24-layer, 2048-hidden-size transformer-based language model.


# Training


The model was trained on [Japanese C4](https://huggingface.co/datasets/allenai/c4), [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz) and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective. It reaches around 14 perplexity on a chosen validation set from the same data.


# Tokenization


The model uses a [sentencepiece](https://github.com/google/sentencepiece)-based tokenizer. The vocabulary was first trained on a selected subset from the training data using the official sentencepiece training script, and then augmented with emojis and symbols.


# Licenese


model name: japanese-gpt-1b


model author: rinna


task: text-generation


tags: pytorch, gpt2, text-generation, ja, dataset:cc100, dataset:wikipedia, transformers, japanese, gpt, lm, nlp, license:mit


language: ja
thumbnail: https://github.com/rinnakk/japanese-gpt2/blob/master/rinna.png
tags:


- ja
- japanese
- gpt2
- text-generation
- lm
- nlp
license: mit
datasets:
- cc100
- wikipedia
widget:
- text: "生命、宇宙、そして万物についての究極の疑問の答えは"


# japanese-gpt2-medium


![rinna-icon](./rinna.png)


This repository provides a medium-sized Japanese GPT-2 model. The model was trained using code from Github repository [rinnakk/japanese-pretrained-models](https://github.com/rinnakk/japanese-pretrained-models) by [rinna Co., Ltd.](https://corp.rinna.co.jp/)


# How to use the model


*NOTE:* Use `T5Tokenizer` to initiate the tokenizer.


```
from transformers import T5Tokenizer, AutoModelForCausalLM

tokenizer = T5Tokenizer.from_pretrained("rinna/japanese-gpt2-medium")
tokenizer.do_lower_case = True  # due to some bug of tokenizer config loading

model = AutoModelForCausalLM.from_pretrained("rinna/japanese-gpt2-medium")
```


# Model architecture


A 24-layer, 1024-hidden-size transformer-based language model.


# Training


The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz) and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective on 8\\*V100 GPUs for around 30 days. It reaches around 18 perplexity on a chosen validation set from the same data.


# Tokenization


The model uses a [sentencepiece](https://github.com/google/sentencepiece)-based tokenizer, the vocabulary was trained on the Japanese Wikipedia using the official sentencepiece training script.


# Licenese


[The MIT license](https://opensource.org/licenses/MIT)


model name: japanese-gpt2-medium


model author: rinna


task: text-generation


tags: pytorch, tf, jax, gpt2, text-generation, ja, dataset:cc100, dataset:wikipedia, transformers, japanese, lm, nlp, license:mit


- ja
- japanese
- roberta
- masked-lm
- nlp
license: mit
datasets:
- cc100
- wikipedia
mask_token: "[MASK]"
widget:
- text: "[CLS]4年に1度[MASK]は開かれる。"


# japanese-roberta-base


![rinna-icon](./rinna.png)


This repository provides a base-sized Japanese RoBERTa model. The model was trained using code from Github repository [rinnakk/japanese-pretrained-models](https://github.com/rinnakk/japanese-pretrained-models) by [rinna Co., Ltd.](https://corp.rinna.co.jp/)


# How to load the model


```
from transformers import T5Tokenizer, RobertaForMaskedLM

tokenizer = T5Tokenizer.from_pretrained("rinna/japanese-roberta-base")
tokenizer.do_lower_case = True  # due to some bug of tokenizer config loading

model = RobertaForMaskedLM.from_pretrained("rinna/japanese-roberta-base")
```


# How to use the model for masked token prediction


## Note 1: Use `[CLS]`


To predict a masked token, be sure to add a `[CLS]` token before the sentence for the model to correctly encode it, as it is used during the model training.


## Note 2: Use `[MASK]` after tokenization


A) Directly typing `[MASK]` in an input string and B) replacing a token with `[MASK]` after tokenization will yield different token sequences, and thus different prediction results. It is more appropriate to use `[MASK]` after tokenization (as it is consistent with how the model was pretrained). However, the Huggingface Inference API only supports typing `[MASK]` in the input string and produces less robust predictions.


## Note 3: Provide `position_ids` as an argument explicitly


When `position_ids` are not provided for a `Roberta*` model, Huggingface's `transformers` will automatically construct it but start from `padding_idx` instead of `0` (see [issue](https://github.com/rinnakk/japanese-pretrained-models/issues/3) and function `create_position_ids_from_input_ids()` in Huggingface's [implementation](https://github.com/huggingface/transformers/blob/master/src/transformers/models/roberta/modeling_roberta.py)), which unfortunately does not work as expected with `rinna/japanese-roberta-base` since the `padding_idx` of the corresponding tokenizer is not `0`. So please be sure to constrcut the `position_ids` by yourself and make it start from position id `0`.


## Example


Here is an example by to illustrate how our model works as a masked language model. Notice the difference between running the following code example and running the Huggingface Inference API.


```
# original text
text = "4年に1度オリンピックは開かれる。"

# prepend [CLS]
text = "[CLS]" + text

# tokenize
tokens = tokenizer.tokenize(text)
print(tokens)  # output: ['[CLS]', '▁4', '年に', '1', '度', 'オリンピック', 'は', '開かれる', '。']

# mask a token
masked_idx = 5
tokens[masked_idx] = tokenizer.mask_token
print(tokens)  # output: ['[CLS]', '▁4', '年に', '1', '度', '[MASK]', 'は', '開かれる', '。']

# convert to ids
token_ids = tokenizer.convert_tokens_to_ids(tokens)
print(token_ids)  # output: [4, 1602, 44, 24, 368, 6, 11, 21583, 8]

# convert to tensor
import torch
token_tensor = torch.LongTensor([token_ids])

# provide position ids explicitly
position_ids = list(range(0, token_tensor.size(1)))
print(position_ids)  # output: [0, 1, 2, 3, 4, 5, 6, 7, 8]
position_id_tensor = torch.LongTensor([position_ids])

# get the top 10 predictions of the masked token
with torch.no_grad():
    outputs = model(input_ids=token_tensor, position_ids=position_id_tensor)
    predictions = outputs[0][0, masked_idx].topk(10)

for i, index_t in enumerate(predictions.indices):
    index = index_t.item()
    token = tokenizer.convert_ids_to_tokens([index])[0]
    print(i, token)

"""
0 総会
1 サミット
2 ワールドカップ
3 フェスティバル
4 大会
5 オリンピック
6 全国大会
7 党大会
8 イベント
9 世界選手権
"""
```


# Model architecture


A 12-layer, 768-hidden-size transformer-based masked language model.


# Training


The model was trained on [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz) and [Japanese Wikipedia](https://dumps.wikimedia.org/jawiki/) to optimize a masked language modelling objective on 8*V100 GPUs for around 15 days. It reaches ~3.9 perplexity on a dev set sampled from CC-100.


# Tokenization


# Licenese


model name: japanese-roberta-base


model author: rinna


task: fill-mask


tags: pytorch, tf, roberta, fill-mask, ja, dataset:cc100, dataset:wikipedia, transformers, japanese, masked-lm, nlp, license:mit, autonlp_compatible, infinity_compatible


language: en
tags:


- sagemaker
- bart
- summarization
license: apache-2.0
datasets:
- samsum
model-index:
- name: bart-large-cnn-samsum
results:
  - task:
name: Abstractive Text Summarization
type: abstractive-text-summarization
dataset:
name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
type: samsum
metrics:
    - name: Validation ROGUE-1
type: rogue-1
value: 42.621
    - name: Validation ROGUE-2
type: rogue-2
value: 21.9825
    - name: Validation ROGUE-L
type: rogue-l
value: 33.034
    - name: Test ROGUE-1
type: rogue-1
value: 41.3174
    - name: Test ROGUE-2
type: rogue-2
value: 20.8716
    - name: Test ROGUE-L
type: rogue-l
value: 32.1337
widget:
- text: |
Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face


## `bart-large-cnn-samsum`


This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
For more information look at:


- [🤗 Transformers Documentation: Amazon SageMaker](https://huggingface.co/transformers/sagemaker.html)
- [Example Notebooks](https://github.com/huggingface/notebooks/tree/master/sagemaker)
- [Amazon SageMaker documentation for Hugging Face](https://docs.aws.amazon.com/sagemaker/latest/dg/hugging-face.html)
- [Python SDK SageMaker documentation for Hugging Face](https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html)
- [Deep Learning Container](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers)


## Hyperparameters


    {
    "dataset_name": "samsum",
    "do_eval": true,
    "do_predict": true,
    "do_train": true,
    "fp16": true,
    "learning_rate": 5e-05,
    "model_name_or_path": "facebook/bart-large-cnn",
    "num_train_epochs": 3,
    "output_dir": "/opt/ml/model",
    "per_device_eval_batch_size": 4,
    "per_device_train_batch_size": 4,
    "predict_with_generate": true,
    "sagemaker_container_log_level": 20,
    "sagemaker_job_name": "huggingface-pytorch-training-2021-09-08-06-40-19-182",
    "sagemaker_program": "run_summarization.py",
    "sagemaker_region": "us-west-2",
    "sagemaker_submit_directory": "s3://sagemaker-us-west-2-847380964353/huggingface-pytorch-training-2021-09-08-06-40-19-182/source/sourcedir.tar.gz",
    "seed": 7


## Usage


    from transformers import pipeline
    summarizer = pipeline("summarization", model="philschmid/bart-large-cnn-samsum")
    conversation = '''Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
    Philipp: Sure you can use the new Hugging Face Deep Learning Container.
    Jeff: ok.
    Jeff: and how can I get started?
    Jeff: where can I find documentation?
    Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
    '''
    nlp(conversation)


## Results


| key | value |
| --- | ----- |
| eval_rouge1 | 42.059 |
| eval_rouge2 | 21.5509 |
| eval_rougeL | 32.4083 |
| eval_rougeLsum | 39.0015 |
| test_rouge1 | 40.8656 |
| test_rouge2 | 20.3517 |
| test_rougeL | 31.2268 |
| test_rougeLsum | 37.9301 |


model name: bart-large-cnn-samsum


model author: jackieliu930


task: summarization


tags: pytorch, bart, text2text-generation, en, dataset:samsum, transformers, sagemaker, summarization, license:apache-2.0, model-index, autonlp_compatible


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: albert_gpt2_Full_summarization_cnndm
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# albert_gpt2_Full_summarization_cnndm


This model is a fine-tuned version of [](https://huggingface.co/) on the cnn_dailymail dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 3.0
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.12.0.dev0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: albert_gpt2_Full_summarization_cnndm


model author: Ayham


task: text2text-generation


tags: pytorch, tensorboard, encoder-decoder, text2text-generation, dataset:cnn_dailymail, transformers, generated_from_trainer, model-index, autonlp_compatible


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: distilbert_gpt2_summarization_cnndm
results: []


# distilbert_gpt2_summarization_cnndm


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- Transformers 4.12.0.dev0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: distilbert_gpt2_summarization_cnndm


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: xlmroberta_large_gpt2_summarization_cnndm
results: []


# xlmroberta_large_gpt2_summarization_cnndm


### Training results


### Framework versions


model name: xlmroberta_large_gpt2_summarization_cnndm


model author: Ayham


task: text2text-generation


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: bert_distilgpt2_summarization_cnn_dailymail
results: []


# bert_distilgpt2_summarization_cnn_dailymail


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: bert_distilgpt2_summarization_cnn_dailymail


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: roberta_distilgpt2_summarization_cnn_dailymail
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# roberta_distilgpt2_summarization_cnn_dailymail


This model is a fine-tuned version of [](https://huggingface.co/) on the cnn_dailymail dataset.


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 3.0
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0


model name: roberta_distilgpt2_summarization_cnn_dailymail


model author: Ayham


task: text2text-generation


tags: pytorch, tensorboard, encoder-decoder, text2text-generation, dataset:cnn_dailymail, transformers, generated_from_trainer, model-index, autonlp_compatible


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: albert_distilgpt2_summarization_cnn_dailymail
results: []


# albert_distilgpt2_summarization_cnn_dailymail


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


model name: albert_distilgpt2_summarization_cnn_dailymail


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: albert_large_gpt2_summarization_cnndm
results: []


# albert_large_gpt2_summarization_cnndm


### Training results


### Framework versions


- Transformers 4.12.0.dev0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: albert_gpt2_summarization_cnndm


model author: Ayham


task: text2text-generation


tags:


- generated_from_trainer
datasets:
- xsum
model-index:
- name: bert_gpt2_summarization_xsum
results: []


# bert_gpt2_summarization_xsum


This model is a fine-tuned version of [](https://huggingface.co/) on the xsum dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- Transformers 4.12.0.dev0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: bert_gpt2_summarization_xsum


tags: pytorch, tensorboard, encoder-decoder, text2text-generation, dataset:xsum, transformers, generated_from_trainer, model-index, autonlp_compatible


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: roberta_gpt2_summarization_cnn_dailymail
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# roberta_gpt2_summarization_cnn_dailymail


This model is a fine-tuned version of [](https://huggingface.co/) on the cnn_dailymail dataset.


This model uses RoBerta encoder and GPT2 decoder and fine-tuned on the summarization task. It got Rouge scores as follows:


Rouge1= 35.886


Rouge2= 16.292


RougeL= 23.499


To use its API:


from transformers import RobertaTokenizerFast, GPT2Tokenizer, EncoderDecoderModel


model = EncoderDecoderModel.from_pretrained("Ayham/roberta_gpt2_summarization_cnn_dailymail")


input_tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base')


output_tokenizer = GPT2Tokenizer.from_pretrained("gpt2")


article = """Your Input Text"""


input_ids = input_tokenizer(article, return_tensors="pt").input_ids


output_ids = model.generate(input_ids)


print(output_tokenizer.decode(output_ids[0], skip_special_tokens=True))


More information needed


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 3.0
- mixed_precision_training: Native AMP


### Training results


### Framework versions


model name: roberta_gpt2_summarization_cnn_dailymail


model author: Ayham


task: text2text-generation


tags: pytorch, tensorboard, encoder-decoder, text2text-generation, dataset:cnn_dailymail, transformers, generated_from_trainer, model-index, autonlp_compatible


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: bert_gpt2_summarization_cnndm
results: []


# bert_gpt2_summarization_cnndm


## Model description


## Intended uses & limitations


## Training and evaluation data


More information needed


## Training procedure


### Training hyperparameters


- Transformers 4.12.0.dev0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: bert_gpt2_summarization_cnndm


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: xlnet_distilgpt2_summarization_cnn_dailymail
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# xlnet_distilgpt2_summarization_cnn_dailymail


This model is a fine-tuned version of [](https://huggingface.co/) on the cnn_dailymail dataset.


### Training results


### Framework versions


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: xlnet_distilgpt2_summarization_cnn_dailymail


model author: Ayham


task: text2text-generation


tags:


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: xlnet_gpt2_summarization_cnn_dailymail
results: []


# xlnet_gpt2_summarization_cnn_dailymail


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


model name: xlnet_gpt2_summarization_cnn_dailymail


- generated_from_trainer
datasets:
- cnn_dailymail
model-index:
- name: distilbert_distilgpt2_summarization_cnn_dailymail
results: []


# distilbert_distilgpt2_summarization_cnn_dailymail


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 3.0
- mixed_precision_training: Native AMP


### Training results


### Framework versions


model name: distilbert_distilgpt2_summarization_cnn_dailymail


model author: Ayham


task: text2text-generation


tags: pytorch, tensorboard, encoder-decoder, text2text-generation, dataset:cnn_dailymail, transformers, generated_from_trainer, model-index, autonlp_compatible


tags:


- conversational


# My Hero Academia DialoGPT Model


model name: DialoGPT-medium-myheroacademia


model author: Shinx


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language: en
license: apache-2.0
datasets:


- bookcorpus
- wikipedia


# BERT large model (cased) whole word masking


Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.


Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.


The training is identical -- each masked WordPiece token is predicted independently.


Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by
the Hugging Face team.


## Model description


BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it
was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of
publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it
was pretrained with two objectives:


- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run
the entire masked sentence through the model and has to predict the masked words. This is different from traditional
recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like
GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the
sentence.
- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes
they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to
predict if the two sentences were following each other or not.


This way, the model learns an inner representation of the English language that can then be used to extract features
useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard
classifier using the features produced by the BERT model as inputs.


This model has the following configuration:


- 24-layer
- 1024 hidden dimension
- 16 attention heads
- 336M parameters.


## Intended uses & limitations


You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to
be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for
fine-tuned versions on a task that interests you.


Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)
to make decisions, such as sequence classification, token classification or question answering. For tasks such as text
generation you should look at model like GPT2.


### How to use


You can use this model directly with a pipeline for masked language modeling:


```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-large-cased-whole-word-masking')
>>> unmasker("Hello I'm a [MASK] model.")
[
   {
      "sequence":"[CLS] Hello I'm a fashion model. [SEP]",
      "score":0.1474294513463974,
      "token":4633,
      "token_str":"fashion"
   },
   {
      "sequence":"[CLS] Hello I'm a magazine model. [SEP]",
      "score":0.05430116504430771,
      "token":2435,
      "token_str":"magazine"
   },
   {
      "sequence":"[CLS] Hello I'm a male model. [SEP]",
      "score":0.039395421743392944,
      "token":2581,
      "token_str":"male"
   },
   {
      "sequence":"[CLS] Hello I'm a former model. [SEP]",
      "score":0.036936815828084946,
      "token":1393,
      "token_str":"former"
   },
   {
      "sequence":"[CLS] Hello I'm a professional model. [SEP]",
      "score":0.03663451969623566,
      "token":1848,
      "token_str":"professional"
   }
]
```


Here is how to use this model to get the features of a given text in PyTorch:


```python
from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained('bert-large-cased-whole-word-masking')
model = BertModel.from_pretrained("bert-large-cased-whole-word-masking")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```


and in TensorFlow:


```python
from transformers import BertTokenizer, TFBertModel
tokenizer = BertTokenizer.from_pretrained('bert-large-cased-whole-word-masking')
model = TFBertModel.from_pretrained("bert-large-cased-whole-word-masking")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model(encoded_input)
```


### Limitations and bias


Even if the training data used for this model could be characterized as fairly neutral, this model can have biased
predictions:


```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='bert-large-cased-whole-word-masking')
>>> unmasker("The man worked as a [MASK].")
[
   {
      "sequence":"[CLS] The man worked as a carpenter. [SEP]",
      "score":0.09021259099245071,
      "token":25169,
      "token_str":"carpenter"
   },
   {
      "sequence":"[CLS] The man worked as a cook. [SEP]",
      "score":0.08125395327806473,
      "token":9834,
      "token_str":"cook"
   },
   {
      "sequence":"[CLS] The man worked as a mechanic. [SEP]",
      "score":0.07524766772985458,
      "token":19459,
      "token_str":"mechanic"
   },
   {
      "sequence":"[CLS] The man worked as a waiter. [SEP]",
      "score":0.07397029548883438,
      "token":17989,
      "token_str":"waiter"
   },
   {
      "sequence":"[CLS] The man worked as a guard. [SEP]",
      "score":0.05848982185125351,
      "token":3542,
      "token_str":"guard"
   }
]

>>> unmasker("The woman worked as a [MASK].")
[
   {
      "sequence":"[CLS] The woman worked as a maid. [SEP]",
      "score":0.19436432421207428,
      "token":13487,
      "token_str":"maid"
   },
   {
      "sequence":"[CLS] The woman worked as a waitress. [SEP]",
      "score":0.16161060333251953,
      "token":15098,
      "token_str":"waitress"
   },
   {
      "sequence":"[CLS] The woman worked as a nurse. [SEP]",
      "score":0.14942803978919983,
      "token":7439,
      "token_str":"nurse"
   },
   {
      "sequence":"[CLS] The woman worked as a secretary. [SEP]",
      "score":0.10373266786336899,
      "token":4848,
      "token_str":"secretary"
   },
   {
      "sequence":"[CLS] The woman worked as a cook. [SEP]",
      "score":0.06384387612342834,
      "token":9834,
      "token_str":"cook"
   }
]
```


This bias will also affect all fine-tuned versions of this model.


## Training data


The BERT model was pretrained on [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038
unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and
headers).


## Training procedure


### Preprocessing


The texts are lowercased and tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are
then of the form:


```
[CLS] Sentence A [SEP] Sentence B [SEP]
```


With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in
the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a
consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two
"sentences" has a combined length of less than 512 tokens.


The details of the masking procedure for each sentence are the following:


- 15% of the tokens are masked.
- In 80% of the cases, the masked tokens are replaced by `[MASK]`.
- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace.
- In the 10% remaining cases, the masked tokens are left as is.


### Pretraining


The model was trained on 4 cloud TPUs in Pod configuration (16 TPU chips total) for one million steps with a batch size
of 256. The sequence length was limited to 128 tokens for 90% of the steps and 512 for the remaining 10%. The optimizer
used is Adam with a learning rate of 1e-4, \\(\beta_{1} = 0.9\\) and \\(\beta_{2} = 0.999\\), a weight decay of 0.01,
learning rate warmup for 10,000 steps and linear decay of the learning rate after.


## Evaluation results


When fine-tuned on downstream tasks, this model achieves the following results:


Model                                    | SQUAD 1.1 F1/EM | Multi NLI Accuracy
---------------------------------------- | :-------------: | :----------------:
BERT-Large, Cased (Whole Word Masking)   | 92.9/86.7       | 86.46


### BibTeX entry and citation info


```bibtex
@article{DBLP:journals/corr/abs-1810-04805,
  author    = {Jacob Devlin and
               Ming{-}Wei Chang and
               Kenton Lee and
               Kristina Toutanova},
  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
               Understanding},
  journal   = {CoRR},
  volume    = {abs/1810.04805},
  year      = {2018},
  url       = {http://arxiv.org/abs/1810.04805},
  archivePrefix = {arXiv},
  eprint    = {1810.04805},
  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


model name: bert-large-cased-whole-word-masking


task: fill-mask


tags: pytorch, tf, jax, bert, fill-mask, en, dataset:bookcorpus, dataset:wikipedia, arxiv:1810.04805, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


tags:


- token-classification
library_name: generic


# Token Classification repository template


This is a template repository for token classification to support generic inference with Hugging Face Hub generic Inference API. There are two required steps


1. Specify the requirements by defining a `requirements.txt` file.
1. Implement the `pipeline.py` `__init__` and `__call__` methods. These methods are called by the Inference API. The `__init__` method should load the model and preload all the elements needed for inference (model, processors, tokenizers, etc.). This is only called once. The `__call__` method performs the actual inference. Make sure to follow the same input/output specifications defined in the template for the pipeline to work.


Example repos


* https://huggingface.co/osanseviero/en_core_web_sm/blob/main/pipeline.py


## How to start


First create a repo in https://hf.co/new.
Then clone this template and push it to your repo.


```
git clone https://huggingface.co/templates/token-classification
cd token-classification
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: token-classification


model author: templates


task: token-classification


tags: generic, token-classification


- structured-data-classification
library_name: generic


# Structured Data Classification repository template


This is a template repository for structured data classification to support generic inference with Hugging Face Hub generic Inference API. There are two required steps


* https://huggingface.co/osanseviero/wine-quality


```
git clone https://huggingface.co/templates/structured-data-classification
cd structured-data-classification
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: structured-data-classification


task: structured-data-classification


tags: generic, structured-data-classification


tags:


- audio-to-audio
library_name: generic


# Audio to Audio repository template


This is a template repository for Audio to Audio to support generic inference with Hugging Face Hub generic Inference API. Examples of Audio to Audio are Source Separation and Speech Enhancement. There are two required steps:


Example repos


* https://huggingface.co/osanseviero/ConvTasNet_Libri1Mix_enhsingle_16k


## How to start


```
git clone https://huggingface.co/templates/audio-to-audio
cd audio-to-audio
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: audio-to-audio


model author: templates


task: audio-to-audio


tags: generic, audio-to-audio


- image-classification
library_name: generic


# Image Classification repository template


This is a template repository for image classification to support generic inference with Hugging Face Hub generic Inference API. There are two required steps


1. Specify the requirements by defining a `requirements.txt` file.
1. Implement the `pipeline.py` `__init__` and `__call__` methods. These methods are called by the Inference API. The `__init__` method should load the model and preload all the elements needed for inference (model, processors, tokenizers, etc.). This is only called once. The `__call__` method performs the actual inference. Make sure to follow the same input/output specifications defined in the template for the pipeline to work.


* https://huggingface.co/osanseviero/fastai_cat_vs_dog/tree/main


First create a repo in https://hf.co/new.
Then clone this template and push it to your repo.


```
git clone https://huggingface.co/templates/image-classification
cd image-classification
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: image-classification


task: image-classification


tags: generic, image-classification


tags:


- text-classification
library_name: generic


# Text Classification repository template


This is a template repository for Text Classification to support generic inference with Hugging Face Hub generic Inference API. There are two required steps:


## How to start


```
git clone https://huggingface.co/templates/text-classification
cd text-classification
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: text-classification


model author: templates


task: text-classification


tags: generic, text-classification


- text-to-image
library_name: generic


# Text To Image repository template


This is a template repository for text to image to support generic inference with Hugging Face Hub generic Inference API. There are two required steps


Example repos


* https://huggingface.co/osanseviero/BigGAN-deep-128/blob/main/pipeline.py


```
git clone https://huggingface.co/templates/text-to-image
cd text-to-image
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: text-to-image


task: text-to-image


tags: generic, text-to-image


tags:


- automatic-speech-recognition
library_name: generic


# Automatic Speech Recognition repository template


This is a template repository for Automatic Speech Recognition to support generic inference with Hugging Face Hub generic Inference API. There are two required steps:


* https://huggingface.co/osanseviero/pyctcdecode_asr


## How to start


```
git clone https://huggingface.co/templates/automatic-speech-recognition
cd automatic-speech-recognition
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: automatic-speech-recognition


model author: templates


task: automatic-speech-recognition


tags: generic, automatic-speech-recognition


- feature-extraction
library_name: generic


# Feature Extraction repository template


This is a template repository for feature extraction to support generic inference with Hugging Face Hub generic Inference API. There are two required steps


1. Specify the requirements by defining a `requirements.txt` file.
1. Implement the `pipeline.py` `__init__` and `__call__` methods. These methods are called by the Inference API. The `__init__` method should load the model and preload all the elements needed for inference (model, processors, tokenizers, etc.). This is only called once. The `__call__` method performs the actual inference. Make sure to follow the same input/output specifications defined in the template for the pipeline to work.


Example repos


* https://huggingface.co/osanseviero/fasttext_english


First create a repo in https://hf.co/new.
Then clone this template and push it to your repo.