| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.3116        | 1.0   | 6547  | 0.2818          | 0.8849   |
| 0.2467        | 2.0   | 13094 | 0.2532          | 0.9001   |
| 0.1858        | 3.0   | 19641 | 0.3194          | 0.9112   |
| 0.1449        | 4.0   | 26188 | 0.4338          | 0.9103   |
| 0.0584        | 5.0   | 32735 | 0.5752          | 0.9052   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: albert-base-v2-finetuned-qnli


model author: anirudh21


task: text-classification


tags: pytorch, tensorboard, albert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: distilbert-base-uncased-finetuned-mrpc
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: mrpc
metrics:
    - name: Accuracy
type: accuracy
value: 0.8455882352941176
    - name: F1
type: f1
value: 0.8958677685950412


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilbert-base-uncased-finetuned-mrpc


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3830
- Accuracy: 0.8456
- F1: 0.8959


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
| No log        | 1.0   | 230  | 0.3826          | 0.8186   | 0.8683 |
| No log        | 2.0   | 460  | 0.3830          | 0.8456   | 0.8959 |
| 0.4408        | 3.0   | 690  | 0.3835          | 0.8382   | 0.8866 |
| 0.4408        | 4.0   | 920  | 0.5036          | 0.8431   | 0.8919 |
| 0.1941        | 5.0   | 1150 | 0.5783          | 0.8431   | 0.8930 |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: distilbert-base-uncased-finetuned-mrpc


model author: anirudh21


task: text-classification


tags: pytorch, tensorboard, distilbert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: distilbert-base-uncased-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.5633802816901409


# distilbert-base-uncased-finetuned-wnli


- Loss: 0.6883
- Accuracy: 0.5634


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.0   | 40   | 0.6883          | 0.5634   |
| No log        | 2.0   | 80   | 0.6934          | 0.5634   |
| No log        | 3.0   | 120  | 0.6960          | 0.5211   |
| No log        | 4.0   | 160  | 0.6958          | 0.5634   |
| No log        | 5.0   | 200  | 0.6964          | 0.5634   |


### Framework versions


model name: distilbert-base-uncased-finetuned-wnli


model author: anirudh21


task: text-classification


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: distilbert-base-uncased-finetuned-sst2
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: sst2
metrics:
    - name: Accuracy
type: accuracy
value: 0.908256880733945


# distilbert-base-uncased-finetuned-sst2


- Loss: 0.4028
- Accuracy: 0.9083


| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.188         | 1.0   | 4210  | 0.3127          | 0.9037   |
| 0.1299        | 2.0   | 8420  | 0.3887          | 0.9048   |
| 0.0845        | 3.0   | 12630 | 0.4028          | 0.9083   |
| 0.0691        | 4.0   | 16840 | 0.3924          | 0.9071   |
| 0.052         | 5.0   | 21050 | 0.5047          | 0.9002   |


model name: distilbert-base-uncased-finetuned-sst2


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: albert-xxlarge-v2-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.5070422535211268


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# albert-xxlarge-v2-finetuned-wnli


This model is a fine-tuned version of [albert-xxlarge-v2](https://huggingface.co/albert-xxlarge-v2) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6970
- Accuracy: 0.5070


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.0   | 13   | 0.8066          | 0.4366   |
| No log        | 2.0   | 26   | 0.6970          | 0.5070   |
| No log        | 3.0   | 39   | 0.7977          | 0.4507   |
| No log        | 4.0   | 52   | 0.7906          | 0.4930   |
| No log        | 5.0   | 65   | 0.8459          | 0.4366   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: albert-xxlarge-v2-finetuned-wnli


model author: anirudh21


task: text-classification


tags: pytorch, tensorboard, albert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: bert-base-uncased-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.5633802816901409


# bert-base-uncased-finetuned-wnli


This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6854
- Accuracy: 0.5634


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.0   | 40   | 0.6854          | 0.5634   |
| No log        | 2.0   | 80   | 0.6983          | 0.3239   |
| No log        | 3.0   | 120  | 0.6995          | 0.5352   |
| No log        | 4.0   | 160  | 0.6986          | 0.5634   |
| No log        | 5.0   | 200  | 0.6996          | 0.5634   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.0
- Tokenizers 0.10.3


model name: bert-base-uncased-finetuned-wnli


model author: anirudh21


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


# PolitBERT


## Background


This model was created to specialize on political speeches, interviews and press briefings of English-speaking politicians.


## Training


The model was initialized using the pre-trained weights of BERTBASE and trained for 20 epochs on the standard MLM task with default parameters.
The used learning rate was 5e-5 with a linearly decreasing schedule and AdamW.
The used batch size is 8 per GPU while beeing trained on two Nvidia GTX TITAN X.
The rest of the used configuration is the same as in `AutoConfig.from_pretrained('bert-base-uncased')`.
As a tokenizer the default tokenizer of BERT was used (`BertTokenizer.from_pretrained('bert-base-uncased')`)


## Dataset


PolitBERT was trained on the following dataset, which has been split up into single sentences:
<https://www.kaggle.com/mauricerupp/englishspeaking-politicians>


## Usage


To predict a missing word of a sentence, the following pipeline can be applied:


```
from transformers import pipeline, BertTokenizer, AutoModel

fill_mask = pipeline("fill-mask",
                     model=AutoModel.from_pretrained('maurice/PolitBERT'),
                     tokenizer=BertTokenizer.from_pretrained('bert-base-uncased'))

print(fill_mask('Donald Trump is a [MASK].'))
```


## Training Results


Evaluation Loss:
![evalloss](evalloss_BERT.png)
Training Loss:
![evalloss](loss_BERT.png)
Learning Rate Schedule:
![evalloss](LR_BERT.png)


model name: PolitBERT


model author: maurice


task: fill-mask


tags: pytorch, jax, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


language: zh
widget:


- text: "今天是下雨天"
- text: "走向森林"


# EasternFantasyNoval


# Overview


- **Language model**: GPT2-Medium
- **Model size**: 1.2GiB
- **Language**: Chinese


model name: EasternFantasyNoval


model author: mymusise


task: text-generation


tags: tf, gpt2, text-generation, zh, transformers


model name: AIXX


# gpt2-medium-chinese


- **Language model**: GPT2-Medium
- **Model size**: 1.2GiB
- **Language**: Chinese
- **Training data**: [wiki2019zh_corpus](https://github.com/brightmart/nlp_chinese_corpus)
- **Source code**: [gpt2-quickly](https://github.com/mymusise/gpt2-quickly)


# Example


```python
from transformers import BertTokenizer, TFGPT2LMHeadModel
from transformers import TextGenerationPipeline

tokenizer = BertTokenizer.from_pretrained("mymusise/EasternFantasyNoval")
model = TFGPT2LMHeadModel.from_pretrained("mymusise/EasternFantasyNoval")

text_generator = TextGenerationPipeline(model, tokenizer)
print(text_generator("今日", max_length=64, repetition_penalty=1.3, do_sample=True, top_k=10))
print(text_generator("跨越山丘", max_length=64, repetition_penalty=1.3, do_sample=True, top_k=10))
```


输出


```text
[{'generated_text': '今日 ， 他 的 作 品 也 在 各 种 报 刊 发 表 。 201 1 年 ， 他 开 设 了 他 的 网 页 版 《 the dear 》 。 此 外 ， 他 还 在 各 种 电 视 节 目 中 出 现 过 。 2017 年 1 月 ， 他 被 任'}]
[{'generated_text': '跨越山丘 ， 其 中 有 三 分 之 二 的 地 区 被 划 入 山 区 。 最 高 峰 是 位 于 山 脚 上 的 大 岩 （ ） 。 其 中 的 山 脚 下 有 一 处 有 名 为 的 河 谷 ， 因 其 高 度 在 其 中 ， 而 得 名 。'}]
```


[Try it on colab](https://colab.research.google.com/github/mymusise/gpt2-quickly/blob/main/examples/gpt2_medium_chinese.ipynb)


model name: gpt2-medium-chinese


language: zh
widget:


- text: "今天是下雨天"
- text: "走向森林"


CPM


CPM(Chinese Pre-Trained Language Models), which has 2.6B parameters, made by the research team of Beijing Zhiyuan Institute of artificial intelligence and Tsinghua University @TsinghuaAI.


[repo: CPM-Generate](https://github.com/TsinghuaAI/CPM-Generate)


The One Thing You Need to Know is this model is not uploaded by official, the conver script is [here](https://github.com/mymusise/CPM-TF2Transformer/blob/main/transfor_CMP.ipynb)


# Overview


- **Language model**: CPM
- **Model size**: 2.6B parameters
- **Language**: Chinese


# How to use


How to use this model directly from the 🤗/transformers library:


```python
from transformers import XLNetTokenizer, TFGPT2LMHeadModel
import jieba

# add spicel process
class XLNetTokenizer(XLNetTokenizer):
    translator = str.maketrans(" \n", "\u2582\u2583")

    def _tokenize(self, text, *args, **kwargs):
        text = [x.translate(self.translator) for x in jieba.cut(text, cut_all=False)]
        text = " ".join(text)
        return super()._tokenize(text, *args, **kwargs)

    def _decode(self, *args, **kwargs):
        text = super()._decode(*args, **kwargs)
        text = text.replace(' ', '').replace('\u2582', ' ').replace('\u2583', '\n')
        return text

tokenizer = XLNetTokenizer.from_pretrained('mymusise/CPM-GPT2-FP16')
model = TFGPT2LMHeadModel.from_pretrained("mymusise/CPM-GPT2-FP16")
```


How to generate text


```python
from transformers import TextGenerationPipeline

text_generater = TextGenerationPipeline(model, tokenizer)

texts = [
    '今天天气不错',
    '天下武功, 唯快不',
    """
    我们在火星上发现了大量的神奇物种。有神奇的海星兽，身上是粉色的，有5条腿；有胆小的猫猫兽，橘色，有4条腿；有令人恐惧的蜈蚣兽，全身漆黑，36条腿；有纯洁的天使兽，全身洁白无瑕，有3条腿；有贪吃的汪汪兽，银色的毛发，有5条腿；有蛋蛋兽，紫色，8条腿。

    请根据上文，列出一个表格，包含物种名、颜色、腿数量。
    |物种名|颜色|腿数量|
    |亚古兽|金黄|2|
    |海星兽|粉色|5|
    |猫猫兽|橘色|4|
    |蜈蚣兽|漆黑|36|
    """
]

for text in texts:
    token_len = len(tokenizer._tokenize(text))
    print(text_generater(text, max_length=token_len + 15, top_k=1, use_cache=True, prefix='')[0]['generated_text'])
    print(text_generater(text, max_length=token_len + 15, do_sample=True, top_k=5)[0]['generated_text'])
```


You can try it on [colab](https://colab.research.google.com/github/mymusise/CPM-TF2Transformer/blob/main/demo-fp16.ipynb)


model name: CPM-GPT2-FP16


model author: mymusise


task: text-generation


language: zh
widget:


- text: "今天是下雨天"
- text: "走向森林"


# EasternFantasyNoval


model name: EasternFantasyNoval-small


- text: "天下熙熙，"
- text: "天气不错，"


CPM-Generate-distill


[repo: CPM-Generate](https://github.com/TsinghuaAI/CPM-Generate)
The One Thing You Need to Know is this model is not uploaded by official, the conver script is [here](https://github.com/mymusise/CPM-TF2Transformer/blob/main/transfor_CMP.ipynb)


And the `CPM-Generate-distill` is the distill model of `CPM`.


# How to use


```python
from transformers import XLNetTokenizer, TFGPT2LMHeadModel
from transformers import TextGenerationPipeline
import jieba
# add spicel process
class XLNetTokenizer(XLNetTokenizer):
    translator = str.maketrans(" \n", "\u2582\u2583")
    def _tokenize(self, text, *args, **kwargs):
        text = [x.translate(self.translator) for x in jieba.cut(text, cut_all=False)]
        text = " ".join(text)
        return super()._tokenize(text, *args, **kwargs)
    def _decode(self, *args, **kwargs):
        text = super()._decode(*args, **kwargs)
        text = text.replace(' ', '').replace('\u2582', ' ').replace('\u2583', '\n')
        return text

tokenizer = XLNetTokenizer.from_pretrained('mymusise/CPM-Generate-distill')
model = TFGPT2LMHeadModel.from_pretrained("mymusise/CPM-Generate-distill")

text_generater = TextGenerationPipeline(model, tokenizer)

print(text_generater("天下熙熙，", max_length=15, top_k=1, use_cache=True, prefix=''))
```


model name: CPM-Generate-distill


tags: pytorch, tf, jax, gpt2, text-generation, zh, transformers


CPM


# Overview


```python
from transformers import XLNetTokenizer, TFGPT2LMHeadModel
import jieba

# add spicel process
class XLNetTokenizer(XLNetTokenizer):
    translator = str.maketrans(" \n", "\u2582\u2583")

    def _tokenize(self, text, *args, **kwargs):
        text = [x.translate(self.translator) for x in jieba.cut(text, cut_all=False)]
        text = " ".join(text)
        return super()._tokenize(text, *args, **kwargs)

    def _decode(self, *args, **kwargs):
        text = super()._decode(*args, **kwargs)
        text = text.replace(' ', '').replace('\u2582', ' ').replace('\u2583', '\n')
        return text

tokenizer = XLNetTokenizer.from_pretrained('mymusise/CPM-GPT2')
model = TFGPT2LMHeadModel.from_pretrained("mymusise/CPM-GPT2")
```


How to generate text


model name: CPM-GPT2


model author: mymusise


task: text-generation


tags: tf, gpt2, text-generation, zh, transformers


Language Detection Model for Nepali, English, Hindi and Spanish
Model fine tuned on xlm-roberta-large


model name: xlm-roberta-large-language-detection


model author: Manishl7


task: text-classification


tags: pytorch, roberta, text-classification, transformers, infinity_compatible


tags:


- conversational


# Chandler Bot DialoGPT model


model name: DialoGPT-small-ChandlerBot


model author: CyberMuffin


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


license: mit
language:


- sk
tags:
- generated_from_trainer
datasets:
- wikiann
metrics:
- precision
- recall
- f1
- accuracy
inference: false
widget:
- text: "Zuzana Čaputová sa narodila 21. júna 1973 v Bratislave."
example_title: "Named Entity Recognition"
model-index:
- name: slovakbert-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: wikiann
type: wikiann
args: sk
metrics:
    - name: Precision
type: precision
value: 0.9327115256495669
    - name: Recall
type: recall
value: 0.9470124013528749
    - name: F1
type: f1
value: 0.9398075632132469
    - name: Accuracy
type: accuracy
value: 0.9785228256835333


# Named Entity Recognition based on SlovakBERT


This model is a fine-tuned version of [gerulata/slovakbert](https://huggingface.co/gerulata/slovakbert) on the Slovak wikiann dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1600
- Precision: 0.9327
- Recall: 0.9470
- F1: 0.9398
- Accuracy: 0.9785


## Intended uses & limitations


Supported classes: LOCATION, PERSON, ORGANIZATION


```
from transformers import pipeline

ner_pipeline = pipeline(task='ner', model='crabz/slovakbert-ner')
input_sentence = "Minister financií a líder mandátovo najsilnejšieho hnutia OĽaNO Igor Matovič upozorňuje, že následky tretej vlny budú na Slovensku veľmi veľké."
classifications = ner_pipeline(input_sentence)
```


with `displaCy`:


```
import spacy
from spacy import displacy

ner_map = {0: '0', 1: 'B-OSOBA', 2: 'I-OSOBA', 3: 'B-ORGANIZÁCIA', 4: 'I-ORGANIZÁCIA', 5: 'B-LOKALITA', 6: 'I-LOKALITA'}

entities = []
for i in range(len(classifications)):
    if classifications[i]['entity'] != 0:
        if ner_map[classifications[i]['entity']][0] == 'B':
            j = i + 1
            while j < len(classifications) and ner_map[classifications[j]['entity']][0] == 'I':
                j += 1
            entities.append((ner_map[classifications[i]['entity']].split('-')[1], classifications[i]['start'],
                             classifications[j - 1]['end']))

nlp = spacy.blank("en")  # it should work with any language

doc = nlp(input_sentence)

ents = []
for ee in entities:
    ents.append(doc.char_span(ee[1], ee[2], ee[0]))

doc.ents = ents

options = {"ents": ["OSOBA", "ORGANIZÁCIA", "LOKALITA"],
           "colors": {"OSOBA": "lightblue", "ORGANIZÁCIA": "lightcoral", "LOKALITA": "lightgreen"}}
displacy_html = displacy.render(doc, style="ent", options=options)

```


Minister financií a líder mandátovo najsilnejšieho hnutia

    OĽaNO
    ORGANIZÁCIA


    Igor Matovič
    OSOBA

 upozorňuje, že následky tretej vlny budú na

    Slovensku
    LOKALITA

 veľmi veľké.


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.2342        | 1.0   | 625  | 0.1233          | 0.8891    | 0.9076 | 0.8982 | 0.9667   |
| 0.1114        | 2.0   | 1250 | 0.1079          | 0.9118    | 0.9269 | 0.9193 | 0.9725   |
| 0.0817        | 3.0   | 1875 | 0.1093          | 0.9173    | 0.9315 | 0.9243 | 0.9747   |
| 0.0438        | 4.0   | 2500 | 0.1076          | 0.9188    | 0.9353 | 0.9270 | 0.9743   |
| 0.028         | 5.0   | 3125 | 0.1230          | 0.9143    | 0.9387 | 0.9264 | 0.9744   |
| 0.0256        | 6.0   | 3750 | 0.1204          | 0.9246    | 0.9423 | 0.9334 | 0.9765   |
| 0.018         | 7.0   | 4375 | 0.1332          | 0.9292    | 0.9416 | 0.9353 | 0.9770   |
| 0.0107        | 8.0   | 5000 | 0.1339          | 0.9280    | 0.9427 | 0.9353 | 0.9769   |
| 0.0079        | 9.0   | 5625 | 0.1368          | 0.9326    | 0.9442 | 0.9383 | 0.9785   |
| 0.0065        | 10.0  | 6250 | 0.1490          | 0.9284    | 0.9445 | 0.9364 | 0.9772   |
| 0.0061        | 11.0  | 6875 | 0.1566          | 0.9328    | 0.9433 | 0.9380 | 0.9778   |
| 0.0031        | 12.0  | 7500 | 0.1555          | 0.9339    | 0.9473 | 0.9406 | 0.9787   |
| 0.0024        | 13.0  | 8125 | 0.1548          | 0.9349    | 0.9462 | 0.9405 | 0.9787   |
| 0.0015        | 14.0  | 8750 | 0.1562          | 0.9330    | 0.9469 | 0.9399 | 0.9788   |
| 0.0013        | 15.0  | 9375 | 0.1600          | 0.9327    | 0.9470 | 0.9398 | 0.9785   |


### Framework versions


- Transformers 4.13.0.dev0
- Pytorch 1.10.0+cu113
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: slovakbert-ner


model author: crabz


task: token-classification


tags: pytorch, roberta, token-classification, sk, dataset:wikiann, transformers, license:mit, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


license: cc-by-nc-sa-4.0
tags:


- generated_from_trainer
datasets:
- wikiann
metrics:
- precision
- recall
- f1
- accuracy
language:
- sk
inference: false
model-index:
- name: fernet-sk-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: wikiann sk
type: wikiann
args: sk
metrics:
    - name: Precision
type: precision
value: 0.9359821760118826
    - name: Recall
type: recall
value: 0.9472378804960541
    - name: F1
type: f1
value: 0.9415763914830033
    - name: Accuracy
type: accuracy
value: 0.9789063466534702


# Named Entity Recognition based on FERNET-CC_sk


This model is a fine-tuned version of [fav-kky/FERNET-CC_sk](https://huggingface.co/fav-kky/FERNET-CC_sk) on the Slovak wikiann dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1763
- Precision: 0.9360
- Recall: 0.9472
- F1: 0.9416
- Accuracy: 0.9789


## Intended uses & limitation


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 24
- eval_batch_size: 24
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.1259        | 1.0   | 834  | 0.1095          | 0.8963    | 0.9182 | 0.9071 | 0.9697   |
| 0.071         | 2.0   | 1668 | 0.0974          | 0.9270    | 0.9357 | 0.9313 | 0.9762   |
| 0.0323        | 3.0   | 2502 | 0.1259          | 0.9257    | 0.9330 | 0.9293 | 0.9745   |
| 0.0175        | 4.0   | 3336 | 0.1347          | 0.9241    | 0.9360 | 0.9300 | 0.9756   |
| 0.0156        | 5.0   | 4170 | 0.1407          | 0.9337    | 0.9404 | 0.9370 | 0.9780   |
| 0.0062        | 6.0   | 5004 | 0.1522          | 0.9267    | 0.9410 | 0.9338 | 0.9774   |
| 0.0055        | 7.0   | 5838 | 0.1559          | 0.9322    | 0.9429 | 0.9375 | 0.9780   |
| 0.0024        | 8.0   | 6672 | 0.1733          | 0.9321    | 0.9438 | 0.9379 | 0.9779   |
| 0.0009        | 9.0   | 7506 | 0.1765          | 0.9347    | 0.9468 | 0.9407 | 0.9784   |
| 0.0002        | 10.0  | 8340 | 0.1763          | 0.9360    | 0.9472 | 0.9416 | 0.9789   |


### Framework versions


- Transformers 4.14.0.dev0
- Pytorch 1.10.0
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: FERNET-CC_sk-ner


model author: crabz


task: token-classification


tags: pytorch, bert, token-classification, sk, dataset:wikiann, transformers, license:cc-by-nc-sa-4.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


tags:


- generated_from_trainer
datasets:
- wikiann
metrics:
- precision
- recall
- f1
- accuracy
language:
- sk
model-index:
- name: bertoslav-limited-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: wikiann sk
type: wikiann
args: sk
metrics:
    - name: Precision
type: precision
value: 0.8985571260306242
    - name: Recall
type: recall
value: 0.9173994738819993
    - name: F1
type: f1
value: 0.9078805459481573
    - name: Accuracy
type: accuracy
value: 0.9700235061239639


# Named Entity Recognition based on bertoslav-limited


This model is a fine-tuned version of [crabz/bertoslav-limited](https://huggingface.co/crabz/bertoslav-limited) on the Slovak wikiann dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2119
- Precision: 0.8986
- Recall: 0.9174
- F1: 0.9079
- Accuracy: 0.9700


The following hyperparameters were used during training:


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.2953        | 1.0   | 834  | 0.1516          | 0.8413    | 0.8647 | 0.8529 | 0.9549   |
| 0.0975        | 2.0   | 1668 | 0.1304          | 0.8787    | 0.9056 | 0.8920 | 0.9658   |
| 0.0487        | 3.0   | 2502 | 0.1405          | 0.8916    | 0.8958 | 0.8937 | 0.9660   |
| 0.025         | 4.0   | 3336 | 0.1658          | 0.8850    | 0.9116 | 0.8981 | 0.9669   |
| 0.0161        | 5.0   | 4170 | 0.1739          | 0.8974    | 0.9127 | 0.9050 | 0.9693   |
| 0.0074        | 6.0   | 5004 | 0.1888          | 0.8900    | 0.9144 | 0.9020 | 0.9687   |
| 0.0051        | 7.0   | 5838 | 0.1996          | 0.8946    | 0.9145 | 0.9044 | 0.9693   |
| 0.0039        | 8.0   | 6672 | 0.2052          | 0.8993    | 0.9158 | 0.9075 | 0.9697   |
| 0.0024        | 9.0   | 7506 | 0.2112          | 0.8946    | 0.9171 | 0.9057 | 0.9696   |
| 0.0018        | 10.0  | 8340 | 0.2119          | 0.8986    | 0.9174 | 0.9079 | 0.9700   |


model name: bertoslav-limited-ner


tags: pytorch, distilbert, token-classification, sk, dataset:wikiann, transformers, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


language:


- en
- it
- scn
- nap


- exbert
- gpt2


license: apache-2.0


# GPTalian


This is a GPT2 model of Italian regional languages trained on [collections of Italian "dialect poetry"](http://dialectpoetry.com) by Luigi Bonaffini.


This is a multilingual model.  Italians use the word "dialect" to describe their regional languages, but they are separate languages.  And there's a lot of English in this dataset too.


The challenge of this project is to train a model to write the languages of Italy.


For those who do not know Italian, here's some (lowercase) text that you can type into the API box:


- oggi si parla il dialetto
- la sua poesia viene di
- ma non sempre trova


model name: GPTalian


model author: ErykWdowiak


task: text-generation


tags: pytorch, jax, gpt2, text-generation, en, it, scn, nap, transformers, exbert, license:apache-2.0


tags:


- conversational


#enlightened GPT model


model name: BrightBot-small


model author: Connorvr


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


license: mit
tags:


- generated_from_trainer
model-index:
- name: model
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# model


This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.6.0
- Datasets 1.18.3
- Tokenizers 0.10.1


model name: TeachingGen


model author: Connorvr


task: text-generation


tags: pytorch, gpt2, text-generation, transformers, license:mit, generated_from_trainer, model-index


tags:


- conversational


# Okabe Rintaro DialoGPT Model


model name: small-okaberintaro


model author: ModzabazeR


task: conversational


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- spearmanr
model-index:
- name: bert-base-cased-finetuned-stsb
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE STSB
type: glue
args: stsb
metrics:
    - name: Spearmanr
type: spearmanr
value: 0.8897907271421561


# bert-base-cased-finetuned-stsb


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the GLUE STSB dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4861
- Pearson: 0.8926
- Spearmanr: 0.8898
- Combined Score: 0.8912


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path bert-base-cased \\n  --task_name stsb \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir bert-base-cased-finetuned-stsb \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Combined Score | Validation Loss | Pearson | Spearmanr |
|:-------------:|:-----:|:----:|:--------------:|:---------------:|:-------:|:---------:|
| 1.1174        | 1.0   | 360  | 0.8816         | 0.5000          | 0.8832  | 0.8800    |
| 0.3835        | 2.0   | 720  | 0.8901         | 0.4672          | 0.8915  | 0.8888    |
| 0.2388        | 3.0   | 1080 | 0.8912         | 0.4861          | 0.8926  | 0.8898    |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: bert-base-cased-finetuned-stsb


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index, infinity_compatible


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-base-finetuned-qnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE QNLI
type: glue
args: qnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.8438586857038257


# fnet-base-finetuned-qnli


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE QNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4746
- Accuracy: 0.8439


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name qnli \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-qnli \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.4597        | 1.0   | 6547  | 0.3713          | 0.8411   |
| 0.3252        | 2.0   | 13094 | 0.3781          | 0.8420   |
| 0.2243        | 3.0   | 19641 | 0.4746          | 0.8439   |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-qnli


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: bert-large-cased-finetuned-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE COLA
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.5957317644481708


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-large-cased-finetuned-cola


This model is a fine-tuned version of [bert-large-cased](https://huggingface.co/bert-large-cased) on the GLUE COLA dataset.
It achieves the following results on the evaluation set:


- Loss: 0.8385
- Matthews Correlation: 0.5957


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5533        | 1.0   | 2138 | 0.7943          | 0.4439               |
| 0.5004        | 2.0   | 4276 | 0.7272          | 0.5678               |
| 0.2865        | 3.0   | 6414 | 0.8385          | 0.5957               |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: bert-large-cased-finetuned-cola


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: fnet-base-finetuned-mrpc
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE MRPC
type: glue
args: mrpc
metrics:
    - name: Accuracy
type: accuracy
value: 0.7720588235294118
    - name: F1
type: f1
value: 0.8502415458937198


# fnet-base-finetuned-mrpc


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE MRPC dataset.
It achieves the following results on the evaluation set:


- Loss: 0.9653
- Accuracy: 0.7721
- F1: 0.8502
- Combined Score: 0.8112


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name mrpc \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 5 \\n  --output_dir fnet-base-finetuned-mrpc \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     | Combined Score |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:--------------:|
| 0.544         | 1.0   | 230  | 0.5272          | 0.7328   | 0.8300 | 0.7814         |
| 0.4034        | 2.0   | 460  | 0.6211          | 0.7255   | 0.8298 | 0.7776         |
| 0.2602        | 3.0   | 690  | 0.9110          | 0.7230   | 0.8306 | 0.7768         |
| 0.1688        | 4.0   | 920  | 0.8640          | 0.7696   | 0.8489 | 0.8092         |
| 0.0913        | 5.0   | 1150 | 0.9653          | 0.7721   | 0.8502 | 0.8112         |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-mrpc


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: bert-base-cased-finetuned-rte
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE RTE
type: glue
args: rte
metrics:
    - name: Accuracy
type: accuracy
value: 0.6714801444043321


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-base-cased-finetuned-rte


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the GLUE RTE dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7260
- Accuracy: 0.6715


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path bert-base-cased \\n  --task_name rte \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir bert-base-cased-finetuned-rte \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.6915        | 1.0   | 156  | 0.6491          | 0.6606   |
| 0.55          | 2.0   | 312  | 0.6737          | 0.6570   |
| 0.3955        | 3.0   | 468  | 0.7260          | 0.6715   |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: bert-base-cased-finetuned-rte


tags: pytorch, tensorboard, bert, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index, infinity_compatible


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: fnet-large-finetuned-cola-copy2
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE COLA
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.0


# fnet-large-finetuned-cola-copy2


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE COLA dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6173
- Matthews Correlation: 0.0


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.6192        | 1.0   | 2138 | 0.6443          | 0.0                  |
| 0.6177        | 2.0   | 4276 | 0.6296          | 0.0                  |
| 0.6128        | 3.0   | 6414 | 0.6173          | 0.0                  |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-cola-copy2


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-large-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE WNLI
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.38028169014084506


# fnet-large-finetuned-wnli


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE WNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6953
- Accuracy: 0.3803


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.7217        | 1.0   | 159  | 0.6864          | 0.5634   |
| 0.7056        | 2.0   | 318  | 0.6869          | 0.5634   |
| 0.706         | 3.0   | 477  | 0.6875          | 0.5634   |
| 0.7032        | 4.0   | 636  | 0.6931          | 0.5634   |
| 0.7025        | 5.0   | 795  | 0.6953          | 0.3803   |


### Framework versions


model name: fnet-large-finetuned-wnli


model author: gchhablani


task: text-classification


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: fnet-large-finetuned-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE COLA
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.0


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fnet-large-finetuned-cola


- Loss: 0.6243
- Matthews Correlation: 0.0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.6195        | 1.0   | 2138 | 0.6527          | 0.0                  |
| 0.6168        | 2.0   | 4276 | 0.6259          | 0.0                  |
| 0.616         | 3.0   | 6414 | 0.6243          | 0.0                  |


### Framework versions


model name: fnet-large-finetuned-cola


model author: gchhablani


task: text-classification


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: fnet-large-finetuned-mrpc
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE MRPC
type: glue
args: mrpc
metrics:
    - name: Accuracy
type: accuracy
value: 0.8259803921568627
    - name: F1
type: f1
value: 0.8798646362098139


# fnet-large-finetuned-mrpc


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE MRPC dataset.
It achieves the following results on the evaluation set:


- Loss: 1.0872
- Accuracy: 0.8260
- F1: 0.8799
- Combined Score: 0.8529


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     | Combined Score |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:--------------:|
| 0.5656        | 1.0   | 917  | 0.6999          | 0.7843   | 0.8581 | 0.8212         |
| 0.3874        | 2.0   | 1834 | 0.7280          | 0.8088   | 0.8691 | 0.8390         |
| 0.1627        | 3.0   | 2751 | 1.1274          | 0.8162   | 0.8780 | 0.8471         |
| 0.0751        | 4.0   | 3668 | 1.0289          | 0.8333   | 0.8870 | 0.8602         |
| 0.0339        | 5.0   | 4585 | 1.0872          | 0.8260   | 0.8799 | 0.8529         |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-mrpc


model author: gchhablani


task: text-classification


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- spearmanr
model-index:
- name: fnet-large-finetuned-stsb
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE STSB
type: glue
args: stsb
metrics:
    - name: Spearmanr
type: spearmanr
value: 0.8532669137129205


# fnet-large-finetuned-stsb


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE STSB dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6250
- Pearson: 0.8554
- Spearmanr: 0.8533
- Combined Score: 0.8543


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Pearson | Spearmanr | Combined Score |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:---------:|:--------------:|
| 1.0727        | 1.0   | 1438 | 0.7718          | 0.8187  | 0.8240    | 0.8214         |
| 0.4619        | 2.0   | 2876 | 0.7704          | 0.8472  | 0.8500    | 0.8486         |
| 0.2401        | 3.0   | 4314 | 0.6250          | 0.8554  | 0.8533    | 0.8543         |


### Framework versions


model name: fnet-large-finetuned-stsb


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-base-finetuned-mnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE MNLI
type: glue
args: mnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.7674938974776241


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fnet-base-finetuned-mnli


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE MNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6443
- Accuracy: 0.7675


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name mnli \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-mnli \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.7143        | 1.0   | 24544 | 0.6169          | 0.7504   |
| 0.5407        | 2.0   | 49088 | 0.6218          | 0.7627   |
| 0.4178        | 3.0   | 73632 | 0.6564          | 0.7658   |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-mnli


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: fnet-large-finetuned-cola-copy4
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE COLA
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.0


# fnet-large-finetuned-cola-copy4


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE COLA dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6500
- Matthews Correlation: 0.0


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 4e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: polynomial
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.6345        | 1.0   | 2138 | 0.6611          | 0.0                  |
| 0.6359        | 2.0   | 4276 | 0.6840          | 0.0                  |
| 0.6331        | 3.0   | 6414 | 0.6500          | 0.0                  |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-cola-copy4


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- spearmanr
model-index:
- name: fnet-base-finetuned-stsb
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE STSB
type: glue
args: stsb
metrics:
    - name: Spearmanr
type: spearmanr
value: 0.8219397497728022


# fnet-base-finetuned-stsb


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE STSB dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7894
- Pearson: 0.8256
- Spearmanr: 0.8219
- Combined Score: 0.8238


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name stsb \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-stsb \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Combined Score | Validation Loss | Pearson | Spearmanr |
|:-------------:|:-----:|:----:|:--------------:|:---------------:|:-------:|:---------:|
| 1.5473        | 1.0   | 360  | 0.8120         | 0.7751          | 0.8115  | 0.8125    |
| 0.6954        | 2.0   | 720  | 0.8145         | 0.8717          | 0.8160  | 0.8130    |
| 0.4828        | 3.0   | 1080 | 0.8238         | 0.7894          | 0.8256  | 0.8219    |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-stsb


model author: gchhablani


task: text-classification


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: bert-base-cased-finetuned-qqp
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE QQP
type: glue
args: qqp
metrics:
    - name: Accuracy
type: accuracy
value: 0.9083848627256987
    - name: F1
type: f1
value: 0.8767633750332712


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-base-cased-finetuned-qqp


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the GLUE QQP dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3752
- Accuracy: 0.9084
- F1: 0.8768
- Combined Score: 0.8926


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path bert-base-cased \\n  --task_name qqp \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir bert-base-cased-finetuned-qqp \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     | Combined Score |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:--------------:|
| 0.308         | 1.0   | 22741 | 0.2548          | 0.8925   | 0.8556 | 0.8740         |
| 0.201         | 2.0   | 45482 | 0.2881          | 0.9032   | 0.8698 | 0.8865         |
| 0.1416        | 3.0   | 68223 | 0.3752          | 0.9084   | 0.8768 | 0.8926         |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: bert-base-cased-finetuned-qqp


tags: pytorch, tensorboard, bert, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index, infinity_compatible


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-large-finetuned-rte
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE RTE
type: glue
args: rte
metrics:
    - name: Accuracy
type: accuracy
value: 0.6425992779783394


# fnet-large-finetuned-rte


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE RTE dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7528
- Accuracy: 0.6426


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.7105        | 1.0   | 623  | 0.6887          | 0.5740   |
| 0.6714        | 2.0   | 1246 | 0.6742          | 0.6209   |
| 0.509         | 3.0   | 1869 | 0.7528          | 0.6426   |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-rte


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index


language: it
datasets:


- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Wav2Vec2 Large 53 Italian by Gunjan Chhablani
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice it
type: common_voice
args: it
metrics:
    - name: Test WER
type: wer
value: 11.49


# Wav2Vec2-Large-XLSR-53-Italian


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Italian using the [Common Voice](https://huggingface.co/datasets/common_voice) dataset.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "it", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained('gchhablani/wav2vec2-large-xlsr-it')
model = Wav2Vec2ForCTC.from_pretrained('gchhablani/wav2vec2-large-xlsr-it')

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Portuguese test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

import unicodedata
import jiwer

def chunked_wer(targets, predictions, chunk_size=None):
    if chunk_size is None: return jiwer.wer(targets, predictions)
    start = 0
    end = chunk_size
    H, S, D, I = 0, 0, 0, 0
    while start < len(targets):
        chunk_metrics = jiwer.compute_measures(targets[start:end], predictions[start:end])
        H = H + chunk_metrics["hits"]
        S = S + chunk_metrics["substitutions"]
        D = D + chunk_metrics["deletions"]
        I = I + chunk_metrics["insertions"]
        start += chunk_size
        end += chunk_size
    return float(S + D + I) / float(H + S + D)

allowed_characters = [
    " ",
    "'",
    'a',
    'b',
    'c',
    'd',
    'e',
    'f',
    'g',
    'h',
    'i',
    'j',
    'k',
    'l',
    'm',
    'n',
    'o',
    'p',
    'q',
    'r',
    's',
    't',
    'u',
    'v',
    'w',
    'x',
    'y',
    'z',
    'à',
    'á',
    'è',
    'é',
    'ì',
    'í',
    'ò',
    'ó',
    'ù',
    'ú',
    ]

def remove_accents(input_str):
    if input_str in allowed_characters:
        return input_str
    if input_str == 'ø':
        return 'o'
    elif input_str=='ß' or input_str =='ß':
        return 'b'
    elif input_str=='ё':
        return 'e'
    elif input_str=='đ':
        return 'd'
    nfkd_form = unicodedata.normalize('NFKD', input_str)
    only_ascii = nfkd_form.encode('ASCII', 'ignore').decode()

    if only_ascii is None or only_ascii=='':
        return input_str
    else:
        return only_ascii
def fix_accents(sentence):
    new_sentence=''
    for char in sentence:
        new_sentence+=remove_accents(char)
    return new_sentence

test_dataset = load_dataset("common_voice", "it", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained('gchhablani/wav2vec2-large-xlsr-it')
model = Wav2Vec2ForCTC.from_pretrained('gchhablani/wav2vec2-large-xlsr-it')
model.to("cuda")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

chars_to_remove= [",", "?", ".", "!", "-", ";", ":", '""', "%", '"', "�",'ʿ','“','”','(','=','`','_','+','«','<','>','~','…','«','»','–','\[','\]','°','̇','´','ʾ','„','̇','̇','̇','¡'] # All extra characters

chars_to_remove_regex = f'[{"".join(chars_to_remove)}]'

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_remove_regex, '', batch["sentence"]).lower().replace('‘',"'").replace('ʻ',"'").replace('ʼ',"'").replace('’',"'").replace('ʹ',"''").replace('̇','')
    batch["sentence"] = fix_accents(batch["sentence"])
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

        pred_ids = torch.argmax(logits, dim=-1)
        batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * chunked_wer(predictions=result["pred_strings"], targets=result["sentence"],chunk_size=5000)))
```


**Test Result**: 11.49 %


## Training


The Common Voice `train` and `validation` datasets were used for training. The code can be found [here](https://github.com/gchhablani/wav2vec2-week/blob/main/fine-tune-xlsr-wav2vec2-on-italian-asr-with-transformers_final.ipynb).


model name: wav2vec2-large-xlsr-it


model author: gchhablani


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, it, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: fnet-large-finetuned-qqp
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE QQP
type: glue
args: qqp
metrics:
    - name: Accuracy
type: accuracy
value: 0.8943111550828593
    - name: F1
type: f1
value: 0.8556565212985171


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fnet-large-finetuned-qqp


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE QQP dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5515
- Accuracy: 0.8943
- F1: 0.8557
- Combined Score: 0.8750


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step   | Validation Loss | Accuracy | F1     | Combined Score |
|:-------------:|:-----:|:------:|:---------------:|:--------:|:------:|:--------------:|
| 0.4574        | 1.0   | 90962  | 0.4946          | 0.8694   | 0.8297 | 0.8496         |
| 0.3387        | 2.0   | 181924 | 0.4745          | 0.8874   | 0.8437 | 0.8655         |
| 0.2029        | 3.0   | 272886 | 0.5515          | 0.8943   | 0.8557 | 0.8750         |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-qqp


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index


language: eo
datasets:


- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Wav2Vec2 Large 53 Esperanto by Gunjan Chhablani
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice eo
type: common_voice
args: eo
metrics:
    - name: Test WER
type: wer
value: 10.13


# Wav2Vec2-Large-XLSR-53-Esperanto


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Esperanto using the [Common Voice](https://huggingface.co/datasets/common_voice) dataset.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "eo", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained('gchhablani/wav2vec2-large-xlsr-eo')
model = Wav2Vec2ForCTC.from_pretrained('gchhablani/wav2vec2-large-xlsr-eo')

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Portuguese test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

import jiwer

def chunked_wer(targets, predictions, chunk_size=None):
    if chunk_size is None: return jiwer.wer(targets, predictions)
    start = 0
    end = chunk_size
    H, S, D, I = 0, 0, 0, 0
    while start < len(targets):
        chunk_metrics = jiwer.compute_measures(targets[start:end], predictions[start:end])
        H = H + chunk_metrics["hits"]
        S = S + chunk_metrics["substitutions"]
        D = D + chunk_metrics["deletions"]
        I = I + chunk_metrics["insertions"]
        start += chunk_size
        end += chunk_size
    return float(S + D + I) / float(H + S + D)

test_dataset = load_dataset("common_voice", "eo", split="test") #TODO: replace {lang_id} in your language code here. Make sure the code is one of the *ISO codes* of [this](https://huggingface.co/languages) site.
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained('gchhablani/wav2vec2-large-xlsr-eo')
model = Wav2Vec2ForCTC.from_pretrained('gchhablani/wav2vec2-large-xlsr-eo')
model.to("cuda")

chars_to_ignore_regex = """[\\\\\\\\,\\\\\\\\?\\\\\\\\.\\\\\\\\!\\\\\\\\-\\\\\\\\;\\\\\\\\:\\\\\\\\"\\\\\\\\“\\\\\\\\%\\\\\\\\‘\\\\\\\\”\\\\\\\\�\\\\\\\\„\\\\\\\\«\\\\\\\\(\\\\\\\\»\\\\\\\\)\\\\\\\\’\\\\\\\\']"""
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace('—',' ').replace('–',' ')
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

        pred_ids = torch.argmax(logits, dim=-1)
        batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * chunked_wer(predictions=result["pred_strings"], targets=result["sentence"],chunk_size=5000)))
```


**Test Result**: 10.13 %


## Training


The Common Voice `train` and `validation` datasets were used for training. The code can be found [here](https://github.com/gchhablani/wav2vec2-week/blob/main/fine-tune-xlsr-wav2vec2-on-esperanto-asr-with-transformers-final.ipynb).


model name: wav2vec2-large-xlsr-eo


model author: gchhablani


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, eo, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: mr
datasets:


- openslr
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Large 53 Marathi by Gunjan Chhablani
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: OpenSLR mr
type: openslr
metrics:
    - name: Test WER
type: wer
value: 14.53


# Wav2Vec2-Large-XLSR-53-Marathi


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Marathi using the [OpenSLR SLR64](http://openslr.org/64/) dataset. Note that this data contains only female voices. Please keep this in mind before using the model for your task, although it works very well for male voice too. When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows, assuming you have a dataset with Marathi `sentence` and `path` fields:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

# test_dataset = #TODO: WRITE YOUR CODE TO LOAD THE TEST DATASET. For sample see the Colab link in Training Section.

processor = Wav2Vec2Processor.from_pretrained("gchhablani/wav2vec2-large-xlsr-mr")
model = Wav2Vec2ForCTC.from_pretrained("gchhablani/wav2vec2-large-xlsr-mr")

resampler = torchaudio.transforms.Resample(48_000, 16_000) # The original data was with 48,000 sampling rate. You can change it according to your input.

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on 10% of the Marathi data on OpenSLR.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

# test_dataset = #TODO: WRITE YOUR CODE TO LOAD THE TEST DATASET. For sample see the Colab link in Training Section.

wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("gchhablani/wav2vec2-large-xlsr-mr")
model = Wav2Vec2ForCTC.from_pretrained("gchhablani/wav2vec2-large-xlsr-mr")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�\–\…]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"),
        attention_mask=inputs.attention_mask.to("cuda")).logits
        pred_ids = torch.argmax(logits, dim=-1)
        batch["pred_strings"] = processor.batch_decode(pred_ids)
        return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 14.53 %


## Training


90% of the OpenSLR Marathi dataset was used for training.
The colab notebook used for training can be found [here](https://colab.research.google.com/drive/1_BbLyLqDUsXG3RpSULfLRjC6UY3RjwME?usp=sharing).


model name: wav2vec2-large-xlsr-mr


model author: gchhablani


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, mr, dataset:openslr, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
- f1
model-index:
- name: fnet-base-finetuned-qqp
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE QQP
type: glue
args: qqp
metrics:
    - name: Accuracy
type: accuracy
value: 0.8847390551570616
    - name: F1
type: f1
value: 0.8466197090382463


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fnet-base-finetuned-qqp


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE QQP dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3686
- Accuracy: 0.8847
- F1: 0.8466
- Combined Score: 0.8657


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name qqp \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-qqp \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     | Combined Score |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:--------------:|
| 0.3484        | 1.0   | 22741 | 0.3014          | 0.8676   | 0.8297 | 0.8487         |
| 0.2387        | 2.0   | 45482 | 0.3011          | 0.8801   | 0.8429 | 0.8615         |
| 0.1739        | 3.0   | 68223 | 0.3686          | 0.8847   | 0.8466 | 0.8657         |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-qqp


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: bert-base-cased-finetuned-qnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE QNLI
type: glue
args: qnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.9099395936298736


# bert-base-cased-finetuned-qnli


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the GLUE QNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3986
- Accuracy: 0.9099


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path bert-base-cased \\n  --task_name qnli \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir bert-base-cased-finetuned-qnli \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Accuracy | Validation Loss |
|:-------------:|:-----:|:-----:|:--------:|:---------------:|
| 0.337         | 1.0   | 6547  | 0.9013   | 0.2448          |
| 0.1971        | 2.0   | 13094 | 0.9143   | 0.2839          |
| 0.1175        | 3.0   | 19641 | 0.9099   | 0.3986          |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: bert-base-cased-finetuned-qnli


tags: pytorch, tensorboard, bert, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index, infinity_compatible


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: fnet-base-finetuned-cola
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE COLA
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.35940659235571387


# fnet-base-finetuned-cola


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE COLA dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5929
- Matthews Correlation: 0.3594


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name cola \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-cola \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5895        | 1.0   | 535  | 0.6146          | 0.1699               |
| 0.4656        | 2.0   | 1070 | 0.5667          | 0.3047               |
| 0.3329        | 3.0   | 1605 | 0.5929          | 0.3594               |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-cola


model author: gchhablani


task: text-classification


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: bert-base-cased-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE WNLI
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.4647887323943662


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-base-cased-finetuned-wnli


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the GLUE WNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6996
- Accuracy: 0.4648


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path bert-base-cased \\n  --task_name wnli \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 5 \\n  --output_dir bert-base-cased-finetuned-wnli \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.7299        | 1.0   | 40   | 0.6923          | 0.5634   |
| 0.6982        | 2.0   | 80   | 0.7027          | 0.3803   |
| 0.6972        | 3.0   | 120  | 0.7005          | 0.4507   |
| 0.6992        | 4.0   | 160  | 0.6977          | 0.5352   |
| 0.699         | 5.0   | 200  | 0.6996          | 0.4648   |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: bert-base-cased-finetuned-wnli


language: hu
datasets:


- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Wav2Vec2 Large 53 Hungarian by Gunjan Chhablani
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice hu
type: common_voice
args: hu
metrics:
    - name: Test WER
type: wer
value: 46.75


# Wav2Vec2-Large-XLSR-53-Hungarian


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Hungarian using the [Common Voice](https://huggingface.co/datasets/common_voice) dataset.
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "hu", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("gchhablani/wav2vec2-large-xlsr-hu")
model = Wav2Vec2ForCTC.from_pretrained("gchhablani/wav2vec2-large-xlsr-hu")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


## Evaluation


The model can be evaluated as follows on the Portuguese test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "hu", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("gchhablani/wav2vec2-large-xlsr-hu")
model = Wav2Vec2ForCTC.from_pretrained("gchhablani/wav2vec2-large-xlsr-hu")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�\–\…]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

        pred_ids = torch.argmax(logits, dim=-1)
        batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 46.75 %


## Training


The Common Voice `train` and `validation` datasets were used for training. The code can be found [here](https://github.com/gchhablani/wav2vec2-week/blob/main/fine-tune-xlsr-wav2vec2-on-hungarian-asr.ipynb). The notebook containing the code used for evaluation can be found [here](https://colab.research.google.com/drive/1esYvWS6IkTQFfRqi_b6lAJEycuecInHE?usp=sharing).


model name: wav2vec2-large-xlsr-hu


model author: gchhablani


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, hu, dataset:common_voice, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-large-finetuned-sst2
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE SST2
type: glue
args: sst2
metrics:
    - name: Accuracy
type: accuracy
value: 0.9048165137614679


# fnet-large-finetuned-sst2


This model is a fine-tuned version of [google/fnet-large](https://huggingface.co/google/fnet-large) on the GLUE SST2 dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5240
- Accuracy: 0.9048


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.394         | 1.0   | 16838 | 0.3896          | 0.8968   |
| 0.2076        | 2.0   | 33676 | 0.5100          | 0.8956   |
| 0.1148        | 3.0   | 50514 | 0.5240          | 0.9048   |


### Framework versions


- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3


model name: fnet-large-finetuned-sst2


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-base-finetuned-rte
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE RTE
type: glue
args: rte
metrics:
    - name: Accuracy
type: accuracy
value: 0.628158844765343


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# fnet-base-finetuned-rte


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE RTE dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6978
- Accuracy: 0.6282


The model was fine-tuned to compare [google/fnet-base](https://huggingface.co/google/fnet-base) as introduced in [this paper](https://arxiv.org/abs/2105.03824) against [bert-base-cased](https://huggingface.co/bert-base-cased).


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


This model is trained using the [run_glue](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py) script. The following command was used:


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name rte \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-rte \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.6829        | 1.0   | 156  | 0.6657          | 0.5704   |
| 0.6174        | 2.0   | 312  | 0.6784          | 0.6101   |
| 0.5141        | 3.0   | 468  | 0.6978          | 0.6282   |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-rte


model author: gchhablani


task: text-classification


tags: pytorch, tensorboard, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


language:


- en
license: apache-2.0
tags:
- generated_from_trainer
- fnet-bert-base-comparison
datasets:
- glue
metrics:
- accuracy
model-index:
- name: fnet-base-finetuned-sst2
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE SST2
type: glue
args: sst2
metrics:
    - name: Accuracy
type: accuracy
value: 0.8944954128440367


# fnet-base-finetuned-sst2


This model is a fine-tuned version of [google/fnet-base](https://huggingface.co/google/fnet-base) on the GLUE SST2 dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4674
- Accuracy: 0.8945


```bash
#!/usr/bin/bash

python ../run_glue.py \\n  --model_name_or_path google/fnet-base \\n  --task_name sst2 \\n  --do_train \\n  --do_eval \\n  --max_seq_length 512 \\n  --per_device_train_batch_size 16 \\n  --learning_rate 2e-5 \\n  --num_train_epochs 3 \\n  --output_dir fnet-base-finetuned-sst2 \\n  --push_to_hub \\n  --hub_strategy all_checkpoints \\n  --logging_strategy epoch \\n  --save_strategy epoch \\n  --evaluation_strategy epoch \\n```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Accuracy | Validation Loss |
|:-------------:|:-----:|:-----:|:--------:|:---------------:|
| 0.2956        | 1.0   | 4210  | 0.8819   | 0.3128          |
| 0.1746        | 2.0   | 8420  | 0.8979   | 0.3850          |
| 0.1204        | 3.0   | 12630 | 0.8945   | 0.4674          |

### Framework versions

- Transformers 4.11.0.dev0
- Pytorch 1.9.0
- Datasets 1.12.1
- Tokenizers 0.10.3
```


model name: fnet-base-finetuned-sst2


tags: pytorch, tensorboard, rust, fnet, text-classification, en, dataset:glue, arxiv:2105.03824, transformers, license:apache-2.0, generated_from_trainer, fnet-bert-base-comparison, model-index


- en
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: bert-large-cased-finetuned-wnli
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: GLUE WNLI
type: glue
args: wnli
metrics:
    - name: Accuracy
type: accuracy
value: 0.352112676056338


# bert-large-cased-finetuned-wnli


This model is a fine-tuned version of [bert-large-cased](https://huggingface.co/bert-large-cased) on the GLUE WNLI dataset.
It achieves the following results on the evaluation set:


- Loss: 0.7087
- Accuracy: 0.3521


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training: