---
license: apache-2.0
tags:
- automatic-speech-recognition
- kresnik/zeroth_korean
- generated_from_trainer
datasets:
- zeroth_korean_asr
model-index:
- name: ''
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

#

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on the KRESNIK/ZEROTH_KOREAN - CLEAN dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0639
- Wer: 0.0449

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 7.5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2000
- num_epochs: 50.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 4.603         | 0.72  | 500   | 4.6572          | 0.9985 |
| 2.6314        | 1.44  | 1000  | 2.0424          | 0.9256 |
| 2.2708        | 2.16  | 1500  | 0.9889          | 0.6989 |
| 2.1769        | 2.88  | 2000  | 0.8366          | 0.6312 |
| 2.1142        | 3.6   | 2500  | 0.7555          | 0.5998 |
| 2.0084        | 4.32  | 3000  | 0.7144          | 0.6003 |
| 1.9272        | 5.04  | 3500  | 0.6311          | 0.5461 |
| 1.8687        | 5.75  | 4000  | 0.6252          | 0.5430 |
| 1.8186        | 6.47  | 4500  | 0.5491          | 0.4988 |
| 1.7364        | 7.19  | 5000  | 0.5463          | 0.4959 |
| 1.6809        | 7.91  | 5500  | 0.4724          | 0.4484 |
| 1.641         | 8.63  | 6000  | 0.4679          | 0.4461 |
| 1.572         | 9.35  | 6500  | 0.4387          | 0.4236 |
| 1.5256        | 10.07 | 7000  | 0.3970          | 0.4003 |
| 1.5044        | 10.79 | 7500  | 0.3690          | 0.3893 |
| 1.4563        | 11.51 | 8000  | 0.3752          | 0.3875 |
| 1.394         | 12.23 | 8500  | 0.3386          | 0.3567 |
| 1.3641        | 12.95 | 9000  | 0.3290          | 0.3467 |
| 1.2878        | 13.67 | 9500  | 0.2893          | 0.3135 |
| 1.2602        | 14.39 | 10000 | 0.2723          | 0.3029 |
| 1.2302        | 15.11 | 10500 | 0.2603          | 0.2989 |
| 1.1865        | 15.83 | 11000 | 0.2440          | 0.2794 |
| 1.1491        | 16.55 | 11500 | 0.2500          | 0.2788 |
| 1.093         | 17.27 | 12000 | 0.2279          | 0.2629 |
| 1.0367        | 17.98 | 12500 | 0.2076          | 0.2443 |
| 0.9954        | 18.7  | 13000 | 0.1844          | 0.2259 |
| 0.99          | 19.42 | 13500 | 0.1794          | 0.2179 |
| 0.9385        | 20.14 | 14000 | 0.1765          | 0.2122 |
| 0.8952        | 20.86 | 14500 | 0.1706          | 0.1974 |
| 0.8841        | 21.58 | 15000 | 0.1791          | 0.1969 |
| 0.847         | 22.3  | 15500 | 0.1780          | 0.2060 |
| 0.8669        | 23.02 | 16000 | 0.1608          | 0.1862 |
| 0.8066        | 23.74 | 16500 | 0.1447          | 0.1626 |
| 0.7908        | 24.46 | 17000 | 0.1457          | 0.1655 |
| 0.7459        | 25.18 | 17500 | 0.1350          | 0.1445 |
| 0.7218        | 25.9  | 18000 | 0.1276          | 0.1421 |
| 0.703         | 26.62 | 18500 | 0.1177          | 0.1302 |
| 0.685         | 27.34 | 19000 | 0.1147          | 0.1305 |
| 0.6811        | 28.06 | 19500 | 0.1128          | 0.1244 |
| 0.6444        | 28.78 | 20000 | 0.1120          | 0.1213 |
| 0.6323        | 29.5  | 20500 | 0.1137          | 0.1166 |
| 0.5998        | 30.22 | 21000 | 0.1051          | 0.1107 |
| 0.5706        | 30.93 | 21500 | 0.1035          | 0.1037 |
| 0.5555        | 31.65 | 22000 | 0.1031          | 0.0927 |
| 0.5389        | 32.37 | 22500 | 0.0997          | 0.0900 |
| 0.5201        | 33.09 | 23000 | 0.0920          | 0.0912 |
| 0.5146        | 33.81 | 23500 | 0.0929          | 0.0947 |
| 0.515         | 34.53 | 24000 | 0.1000          | 0.0953 |
| 0.4743        | 35.25 | 24500 | 0.0922          | 0.0892 |
| 0.4707        | 35.97 | 25000 | 0.0852          | 0.0808 |
| 0.4456        | 36.69 | 25500 | 0.0855          | 0.0779 |
| 0.443         | 37.41 | 26000 | 0.0843          | 0.0738 |
| 0.4388        | 38.13 | 26500 | 0.0816          | 0.0699 |
| 0.4162        | 38.85 | 27000 | 0.0752          | 0.0645 |
| 0.3979        | 39.57 | 27500 | 0.0761          | 0.0621 |
| 0.3889        | 40.29 | 28000 | 0.0771          | 0.0625 |
| 0.3923        | 41.01 | 28500 | 0.0755          | 0.0598 |
| 0.3693        | 41.73 | 29000 | 0.0730          | 0.0578 |
| 0.3642        | 42.45 | 29500 | 0.0739          | 0.0598 |
| 0.3532        | 43.17 | 30000 | 0.0712          | 0.0553 |
| 0.3513        | 43.88 | 30500 | 0.0762          | 0.0516 |
| 0.3349        | 44.6  | 31000 | 0.0731          | 0.0504 |
| 0.3305        | 45.32 | 31500 | 0.0725          | 0.0507 |
| 0.3285        | 46.04 | 32000 | 0.0709          | 0.0489 |
| 0.3179        | 46.76 | 32500 | 0.0667          | 0.0467 |
| 0.3158        | 47.48 | 33000 | 0.0653          | 0.0494 |
| 0.3033        | 48.2  | 33500 | 0.0638          | 0.0456 |
| 0.3023        | 48.92 | 34000 | 0.0644          | 0.0464 |
| 0.2975        | 49.64 | 34500 | 0.0643          | 0.0455 |

### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.3.dev0
- Tokenizers 0.11.0


---
tags:
- conversational
---

# My Awesome Model


KINO » The Many Saints of Newark (2021) ganzer film deutsch Online Anschauen , The Many Saints of Newark Stream Deutsch , Ganzer Film "The Many Saints of Newark" Stream Deutsch komplett, The Many Saints of Newark (2021) ganzer film deutsch Online Anschauen.

📀 📺 Schau jetzt ▶▶ https://world.zmovies25.com/de/524369/the-many-saints-of-newark.html

📀 📥 Jetzt Downloaden ▶▶ https://v.ht/eza_Ganteng

VIDEOS: TRAILERS, TEASERS, FEATURETTES

Veröffentlicht: 2021-09-23

Genre: Drama, Krimi,

Sterne: Michael Gandolfini, Alessandro Nivola, Jon Bernthal, Vera Farmiga, Ray Liotta, Corey Stoll, Billy Magnussen, John Magaro, Leslie Odom Jr., Michela De Rossi, Joey Diaz, William Ludwig, Matteo Russo, Spenser Granese, Phyllis Pastore, Danny Schoch, Rob Coletti, Chase Vacnin, Alex Morf, Nick Vallelonga, Lesli Margherita,

Direktor: James Wan, James Wan, Christian Wagner, Anne McCarthy, Carey Hayes

Tags: The Many Saints of Newark deutsche filme online gucken kostenlos,The Many Saints of Newark filme streamen movie2k,The Many Saints of Newark kostenlose filme online gratis kinofilme,The Many Saints of Newark filme online schauen movie2k, The Many Saints of Newark deutsche filme auf youtube in voller länge,The Many Saints of Newark film komplett deutsch youtube,The Many Saints of Newark youtube ganze filme deutsch liebesfilme,The Many Saints of Newark filme in voller länge kostenlos ansehen youtube ,The Many Saints of Newark youtube filme anschauen kostenlos deutsch,The Many Saints of Newark gute filme auf youtube in voller länge,The Many Saints of Newark youtube filme anschauen legal,The Many Saints of Newark komplette filme auf youtube ,The Many Saints of Newark youtube ganze filme deutsch

Familienvater Hutch (Bob Odenkirk) ist ein Niemand. Der Vorstadt-Familienvater wird von den meisten Leuten einfach ignoriert und zieht auch niemals die Aufmerksamkeit auf sich. Als zwei Verbrecher eines Nachts in sein Haus einsteigen, sieht er sich außerstande, sich oder seine Familie zu verteidigen. Schließlich will er Gewalt um jeden Preis verhindern. Sein Sohn Brady (Gage Munroe) wendet sich daraufhin enttäuscht von seinem Vater ab. Seine Frau Becca (Connie Nielsen) tut es ihm gleich und zieht sich noch mehr zurück. Doch der Einbruch löst bei Hutch etwas aus. Er sieht plötzlich rot und enthüllt eine dunkle Seite, die ihn auf einen Pfad der Gewalt schickt: Einst arbeitete der vermeintliche Versager nämlich als Killer für die Mafia. Und von nun an stellt er mit allen Mitteln sicher, dass seiner Familie nie wieder etwas geschehen und er als Niemand so schnell nicht mehr unterschätzt wird...

„The Many Saints of Newark“ - Hintergründe, Besetzung und Kinostart

Fünf Jahre sind vergangen, seit Disney mit „Vaiana“ den letzten Animationsfilm mit einem Original-Drehbuch in die Kinos gebracht hat. „The Many Saints of Newark“ tritt nun in die Fußstapfen und erzählt nach den Fortsetzungen im Jahr 2019 („Die Eiskönigin 2“ und „Ralph Reichts 2“) eine eigene Geschichte. Für das Drehbuch war die „Crazy Rich“-Autorin Adele Lim sowie Qui Nguyen verantwortlich.

Die beiden Hauptrollen von Raya und Wasserdrache Sisu werden von Kelly Marie Tran („Star Wars“-Reihe) und GThe Many Saints of Newarken-Globe-Gewinnerin Awkwafina („The Farewell“) eingesprochen. „The Many Saints of Newark“ ist der 62. abendfüllende Disney-Film und der erste in der neuen Dekade der 2020er. Das Regieduo Don Hall und Carlos López Estrada hat ganz unterschiedliche Erfahrungen in der Filmwelt. Don Hall führte Regie bei den Disney-Filmen „Baymax“ und „Vaiana“, Estrada ist für Indie-Filme „Summertime“ und „Blindspotting“ verantwortlich.

Der deutsche Kinostart von „The Many Saints of Newark“ ist mit etwas Verspätung am 11. März 2021.

Etikett

The Many Saints of Newark 2021 Ganzer Film Deutsch

The Many Saints of Newark 2021 kostenlos sehen

The Many Saints of Newark Anschauen Film

The Many Saints of Newark komplette Film Deutsch

The Many Saints of Newark komplett online sehen

The Many Saints of Newark Online Anschauen

The Many Saints of Newark Online Kostenlos

The Many Saints of Newark 2021 komplette Stream Deutsch

The Many Saints of Newark(2021) Deutsch HD Stream Online anschauen

The Many Saints of Newark stream Deutsch kinox

The Many Saints of Newark kinox Deutsch HD

The Many Saints of Newark ganzer film Deutsch

The Many Saints of Newark online stream

The Many Saints of Newark stream kostenlos

The Many Saints of Newark hd film germany

The Many Saints of Newark stream kkiste

The Many Saints of Newark kinostart.de

The Many Saints of Newark Deutsch Film

The Many Saints of Newark Ganzer Film Deutsch

The Many Saints of Newark Deutsch kinox

The Many Saints of Newark Deutsch Film HD

The Many Saints of Newark Deutsch Film online

The Many Saints of Newark Deutsch Film voll kostenlos

Uhr The Many Saints of Newark (2021) Röhrenuhr

The Many Saints of Newark (2021) HDQ Voller

The Many Saints of Newark Austria

Uhr The Many Saints of Newark (2021) herunterladen Untertitel

📺 TELEVISION SHOW AND HISTORY

A television show (often simply TV show) is any content produced for broadcast via over-the-air, satellite, cable, or internet and typically viewed on a television set, excluding breaking news, advertisements, or trailers that are typically placed TNTween shows. Television shows are most often scheduled well ahead of time and appear on electronic guides or other TV listings. A television show might also be called a television program (British English: programme), especially if it lacks a narrative structure. A television series is usually released in episodes that follow a narrative, and are usually divided into seasons (US and Canada) or series (UK) — yearly or semiannual sets of new episodes. A show with a limited number of episodes may be called a miniseries, serial, or limited series. A one-time show may be called a “special”. A television film (“made-for-TV movie” or “television movie”) is a film that is initially broadcast on television rather than released in theaters or direct-to-video. Television shows can be viewed as they are broadcast in real time (live), be recorded on home video or a digital video recorder for later viewing, or be viewed on demand via a set-top box or streamed over the internet.

📺 FORMATS AND GENRES

Television shows are more varied than most other forms of media due to the wide variety of formats and genres that can be presented. A show may be fictional (as in comedies and dramas), or non-fictional (as in documentary, news, and reality television). It may be topical (as in the case of a local newscast and some made-for-television films), or historical (as in the case of many documentaries and fictional series). They could be primarily instructional or educational, or entertaining as is the case in situation comedy and game shows. A drama program usually features a set of actors playing characters in a historical or contemporary setting. The program follows their lives and adventures. Before the 1980s, shows (except for soap opera-type serials) typically remained static without story arcs, and the main characters and premise changed little. If some change happened to the characters’ lives during the episode, it was usually undone by the end. Because of this, the episodes could be broadcast in any order. Since the 1980s, many series feature progressive change in the plot, the characters, or both. For instance, Hill Street Blues and St. Elsewhere were two of the first American prime time drama television series to have this kind of dramatic structure,while the later series Babylon 5 further exemplifies such structure in that it had a predetermined story running over its intended five-season run. In 2012, it was reported that television was growing into a larger component of major media companies’ revenues than film. Some also noted the increase in quality of some television programs. In 2012, Academy-Award-winning film director Steven Soderbergh, commenting on ambiguity and complexity of character and narrative, stated: “I think those qualities are now being seen on television and that people who want to see stories that have those kinds of qualities are watching television.

📺 CREDITS

Find all the movies that you can stream online, including those that were screened this week. If you are wondering what you can watch on this website, then you should know that it covers genres that include crime, Science, Fi-Fi, action, romance, thriller, Comedy, drama, Anime Movie, etc. Thank you very much. We tell everyone who is happy to receive us as news or information about this year’s film schedule and how you watch your favorite films. Hopefully we can become the best partner for you in finding recommendations for your favorite movies. That’s all from us, greetings! Thanks for watching Videos Today. I hope you enjoy the videos that I share. Give a thumbs up, like, or share if you enjoy what we’ve shared so that we more excited. Sprinkle cheerful smile so that the world back in a variety of colors, Stay safe and Stay home.


---
language:
- en
tags:
- text reranking
license: apache-2.0
datasets:
- MS MARCO document ranking
---

# BERT Reranker for MS-MARCO Document Ranking

## Model description

A text reranker trained for HDCT retriever on MS MARCO document dataset.

## Intended uses & limitations
It is possible to work with other retrievers like BM25 but using aligned HDCT works the best.

#### How to use
See our [project repo page](https://github.com/luyug/Reranker).

## Eval results
MRR @10: 0.434 on Dev.
MRR @10: 0.382 on Eval.

### BibTeX entry and citation info

```bibtex
@inproceedings{gao2021lce,
               title={Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline},
               author={Luyu Gao and Zhuyun Dai and Jamie Callan},
               year={2021},
               booktitle={The 43rd European Conference On Information Retrieval (ECIR)},

}
```


---
language:
- en
tags:
- text reranking
license: apache-2.0
datasets:
- MS MARCO document ranking
---

# BERT Reranker for MS-MARCO Document Ranking

## Model description

A text reranker trained for BM25 retriever on MS MARCO document dataset.

## Intended uses & limitations
It is possible to work with other retrievers like but using aligned BM25 works the best.

We used anserini toolkit's BM25 implementation and indexed with tuned parameters (k1=3.8, b=0.87) following [this instruction](https://github.com/castorini/anserini/blob/master/docs/experiments-msmarco-doc.md).

#### How to use
See our [project repo page](https://github.com/luyug/Reranker).

## Eval results
MRR @10: 0.423 on Dev.

### BibTeX entry and citation info

```bibtex
@inproceedings{gao2021lce,
               title={Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline},
               author={Luyu Gao and Zhuyun Dai and Jamie Callan},
               year={2021},
               booktitle={The 43rd European Conference On Information Retrieval (ECIR)},

}
```


---
tags:
- conversational
---

#Konosuba DialoGPT Model


---
license: mit
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
model-index:
- name: deberta-base-CoLA
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# deberta-base-CoLA

This model is a fine-tuned version of [microsoft/deberta-base](https://huggingface.co/microsoft/deberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1655
- Accuracy: 0.8482
- F1: 0.8961
- Roc Auc: 0.8987
- Mcc: 0.6288

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     | Roc Auc | Mcc    |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:-------:|:------:|
| 0.5266        | 1.0   | 535  | 0.4138          | 0.8159   | 0.8698 | 0.8627  | 0.5576 |
| 0.3523        | 2.0   | 1070 | 0.3852          | 0.8387   | 0.8880 | 0.9041  | 0.6070 |
| 0.2479        | 3.0   | 1605 | 0.3981          | 0.8482   | 0.8901 | 0.9120  | 0.6447 |
| 0.1712        | 4.0   | 2140 | 0.4732          | 0.8558   | 0.9008 | 0.9160  | 0.6486 |
| 0.1354        | 5.0   | 2675 | 0.7181          | 0.8463   | 0.8938 | 0.9024  | 0.6250 |
| 0.0876        | 6.0   | 3210 | 0.8453          | 0.8520   | 0.8992 | 0.9123  | 0.6385 |
| 0.0682        | 7.0   | 3745 | 1.0282          | 0.8444   | 0.8938 | 0.9061  | 0.6189 |
| 0.0431        | 8.0   | 4280 | 1.1114          | 0.8463   | 0.8960 | 0.9010  | 0.6239 |
| 0.0323        | 9.0   | 4815 | 1.1663          | 0.8501   | 0.8970 | 0.8967  | 0.6340 |
| 0.0163        | 10.0  | 5350 | 1.1655          | 0.8482   | 0.8961 | 0.8987  | 0.6288 |

### Framework versions

- Transformers 4.11.0
- Pytorch 1.9.0+cu102
- Datasets 1.12.1
- Tokenizers 0.10.3


---
language: tr
datasets:
- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Turkish by Mehmet Berk Souksu
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice tr
      type: common_voice
      args: tr
    metrics:
       - name: Test WER
         type: wer
         value: 29.80
---

# Wav2Vec2-Large-XLSR-53-Turkish

Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Turkish using the [Common Voice](https://huggingface.co/datasets/common_voice).
When using this model, make sure that your speech input is sampled at 16kHz.

## Usage

The model can be used directly (without a language model) as follows:

```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "tr", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("mbsouksu/wav2vec2-large-xlsr-turkish-large")
model = Wav2Vec2ForCTC.from_pretrained("mbsouksu/wav2vec2-large-xlsr-turkish-large")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```

## Evaluation

The model can be evaluated as follows on the Turkish test data of Common Voice.

```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "tr", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("mbsouksu/wav2vec2-large-xlsr-turkish-large")
model = Wav2Vec2ForCTC.from_pretrained("mbsouksu/wav2vec2-large-xlsr-turkish-large")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�\'\`\…\\]\\[\\&\\’\»«]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
	batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
	inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

	with torch.no_grad():
		logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

	pred_ids = torch.argmax(logits, dim=-1)
	batch["pred_strings"] = processor.batch_decode(pred_ids)
	return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```

**Test Result**: 29.80 %

## Training

The Common Voice `train`, `validation` datasets were used for training.

The script used for training can be found [here](https://github.com/mbsouksu/wav2vec2-turkish)


---
language: "id"
license: "mit"
datasets:
- Squad
- XQuad
- Tydiqa
widget:
- text: "I love you"
---

## Prefix use
Use prefix "question: {question} context: {context}" before input to generate the question answering

e.g

"question: siapa nama saya ? context: nama saya andi. saya tinggal di jakarta. istri saya bernama raisa"

for generate question prefix

generate questions: nama saya andi. saya tinggal di jakarta. istri saya bernama raisa

## Training data

Squad
XQuad
Tydiqa


---
tags:
- translation
language: "id"
license: "mit"
datasets:
- OPUS
- CC-aligned
widget:
- text: "I love you"
---

## MT5-Large-Translate-en-id

## Prefix use
Use prefix "translate:" before input to generate the translation
e.g
"translate: i love you"

## Training data

Opus (Open Subtittle and Wikimatrix)

CCaligned (en-id sentence pair)


---
language: id
datasets:
- common_voice
tags:
- speech
- audio
- automatic-speech-recognition
- xlsr-fine-tuning-week
license: apache-2.0
---
## Evaluation on Common Voice ID Test
```python
import torchaudio
from datasets import load_dataset, load_metric
from transformers import (
    Wav2Vec2ForCTC,
    Wav2Vec2Processor,
)
import torch
import re
import sys
model_name = "munggok/xlsr_indonesia"
device = "cuda"
chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"]'  # noqa: W605
model = Wav2Vec2ForCTC.from_pretrained(model_name).to(device)
processor = Wav2Vec2Processor.from_pretrained(model_name)
ds = load_dataset("common_voice", "id", split="test", data_dir="./cv-corpus-6.1-2020-12-11")
resampler = torchaudio.transforms.Resample(orig_freq=48_000, new_freq=16_000)
def map_to_array(batch):
    speech, _ = torchaudio.load(batch["path"])
    batch["speech"] = resampler.forward(speech.squeeze(0)).numpy()
    batch["sampling_rate"] = resampler.new_freq
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower().replace("’", "'")
    return batch
ds = ds.map(map_to_array)
def map_to_pred(batch):
    features = processor(batch["speech"], sampling_rate=batch["sampling_rate"][0], padding=True, return_tensors="pt")
    input_values = features.input_values.to(device)
    attention_mask = features.attention_mask.to(device)
    with torch.no_grad():
        logits = model(input_values, attention_mask=attention_mask).logits
    pred_ids = torch.argmax(logits, dim=-1)
    batch["predicted"] = processor.batch_decode(pred_ids)
    batch["target"] = batch["sentence"]
    return batch
result = ds.map(map_to_pred, batched=True, batch_size=16, remove_columns=list(ds.features.keys()))
wer = load_metric("wer")
print(wer.compute(predictions=result["predicted"], references=result["target"]))
```
**Result**: 25.7 %


---
tags:
- conversational
---

# Jett DialoGPT Model


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model_index:
- name: distilbert-base-uncased-finetuned-ner
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: conll2003
      type: conll2003
      args: conll2003
    metric:
      name: Accuracy
      type: accuracy
      value: 0.9836370279759162
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-ner

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the conll2003 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0604
- Precision: 0.9271
- Recall: 0.9381
- F1: 0.9326
- Accuracy: 0.9836

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.2324        | 1.0   | 878  | 0.0688          | 0.9146    | 0.9264 | 0.9205 | 0.9816   |
| 0.0517        | 2.0   | 1756 | 0.0620          | 0.9207    | 0.9329 | 0.9268 | 0.9829   |
| 0.0301        | 3.0   | 2634 | 0.0604          | 0.9271    | 0.9381 | 0.9326 | 0.9836   |

### Framework versions

- Transformers 4.9.1
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


TinyBERT: Distilling BERT for Natural Language Understanding
========

**This model is a copy of [this model repository](https://huggingface.co/huawei-noah/TinyBERT_General_4L_312D) from Huawei Noah at the specific commit `34707a33cd59a94ecde241ac209bf35103691b43`.**

TinyBERT is 7.5x smaller and 9.4x faster on inference than BERT-base and achieves competitive performances in the tasks of natural language understanding. It performs a novel transformer distillation at both the pre-training and task-specific learning stages. In general distillation, we use the original BERT-base without fine-tuning as the teacher and a large-scale text corpus as the learning data. By performing the Transformer distillation on the text from general domain, we obtain a general TinyBERT which provides a good initialization for the task-specific distillation. We here provide the general TinyBERT for your tasks at hand.

For more details about the techniques of TinyBERT, refer to our paper:
[TinyBERT: Distilling BERT for Natural Language Understanding](https://arxiv.org/abs/1909.10351)

Citation
========
If you find TinyBERT useful in your research, please cite the following paper:
```
@article{jiao2019tinybert,
  title={Tinybert: Distilling bert for natural language understanding},
  author={Jiao, Xiaoqi and Yin, Yichun and Shang, Lifeng and Jiang, Xin and Chen, Xiao and Li, Linlin and Wang, Fang and Liu, Qun},
  journal={arXiv preprint arXiv:1909.10351},
  year={2019}
}
```


---
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# DataikuNLP/distiluse-base-multilingual-cased-v1

**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased-v1) from sentence-transformers at the specific commit `3a706e4d65c04f868c4684adfd4da74141be8732`.**

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 512 dimensional dense vector space and can be used for tasks like clustering or semantic search.

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/distiluse-base-multilingual-cased-v1')
embeddings = model.encode(sentences)
print(embeddings)
```

## Evaluation Results

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/distiluse-base-multilingual-cased-v1)

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
  (2): Dense({'in_features': 768, 'out_features': 512, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)
```

## Citing & Authors

This model was trained by [sentence-transformers](https://www.sbert.net/).

If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


---
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# DataikuNLP/paraphrase-albert-small-v2

**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-albert-small-v2/) from sentence-transformers at the specific commit `1eb1996223dd90a4c25be2fc52f6f336419a0d52`.**

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-albert-small-v2')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-albert-small-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-albert-small-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-albert-small-v2)

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 100, 'do_lower_case': False}) with Transformer model: AlbertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

This model was trained by [sentence-transformers](https://www.sbert.net/).

If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


---
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
---

# average_word_embeddings_glove.6B.300d

**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/average_word_embeddings_glove.6B.300d) from sentence-transformers at the specific commit `5d2b7d1c127036ae98b9d487eca4d48744edc709`.**

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 300 dimensional dense vector space and can be used for tasks like clustering or semantic search.

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/average_word_embeddings_glove.6B.300d')
embeddings = model.encode(sentences)
print(embeddings)
```

## Evaluation Results

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/average_word_embeddings_glove.6B.300d)

## Full Model Architecture
```
SentenceTransformer(
  (0): WordEmbeddings(
    (emb_layer): Embedding(400001, 300)
  )
  (1): Pooling({'word_embedding_dimension': 300, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

This model was trained by [sentence-transformers](https://www.sbert.net/).

If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


---
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# DataikuNLP/paraphrase-multilingual-MiniLM-L12-v2

**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) from sentence-transformers at the specific commit `d66eff4d8a8598f264f166af8db67f7797164651`.**

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

This model was trained by [sentence-transformers](https://www.sbert.net/).

If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


---
language: fr
license: mit
datasets:
- oscar
---

# CamemBERT: a Tasty French Language Model

**This model is a copy of [this model repository](https://huggingface.co/camembert-base) at the specific commit `482393b6198924f9da270b1aaf37d238aafca99b`.**

## Introduction

[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model.

It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains.

For further information or requests, please go to [Camembert Website](https://camembert-model.fr/)

## Pre-trained models

| Model                          | #params                        | Arch. | Training data                     |
|--------------------------------|--------------------------------|-------|-----------------------------------|
| `camembert-base` | 110M   | Base  | OSCAR (138 GB of text)            |
| `camembert/camembert-large`              | 335M    | Large | CCNet (135 GB of text)            |
| `camembert/camembert-base-ccnet`         | 110M    | Base  | CCNet (135 GB of text)            |
| `camembert/camembert-base-wikipedia-4gb` | 110M    | Base  | Wikipedia (4 GB of text)          |
| `camembert/camembert-base-oscar-4gb`     | 110M    | Base  | Subsample of OSCAR (4 GB of text) |
| `camembert/camembert-base-ccnet-4gb`     | 110M    | Base  | Subsample of CCNet (4 GB of text) |

## How to use CamemBERT with HuggingFace

##### Load CamemBERT and its sub-word tokenizer :
```python
from transformers import CamembertModel, CamembertTokenizer

# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large".
tokenizer = CamembertTokenizer.from_pretrained("camembert-base")
camembert = CamembertModel.from_pretrained("camembert-base")

camembert.eval()  # disable dropout (or leave in train mode to finetune)

```

##### Filling masks using pipeline
```python
from transformers import pipeline

camembert_fill_mask  = pipeline("fill-mask", model="camembert-base", tokenizer="camembert-base")
results = camembert_fill_mask("Le camembert est <mask> :)")
# results
#[{'sequence': '<s> Le camembert est délicieux :)</s>', 'score': 0.4909103214740753, 'token': 7200},
# {'sequence': '<s> Le camembert est excellent :)</s>', 'score': 0.10556930303573608, 'token': 2183},
# {'sequence': '<s> Le camembert est succulent :)</s>', 'score': 0.03453315049409866, 'token': 26202},
# {'sequence': '<s> Le camembert est meilleur :)</s>', 'score': 0.03303130343556404, 'token': 528},
# {'sequence': '<s> Le camembert est parfait :)</s>', 'score': 0.030076518654823303, 'token': 1654}]

```

##### Extract contextual embedding features from Camembert output
```python
import torch
# Tokenize in sub-words with SentencePiece
tokenized_sentence = tokenizer.tokenize("J'aime le camembert !")
# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!']

# 1-hot encode and add special starting and end tokens
encoded_sentence = tokenizer.encode(tokenized_sentence)
# [5, 121, 11, 660, 16, 730, 25543, 110, 83, 6]
# NB: Can be done in one step : tokenize.encode("J'aime le camembert !")

# Feed tokens to Camembert as a torch tensor (batch dim 1)
encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0)
embeddings, _ = camembert(encoded_sentence)
# embeddings.detach()
# embeddings.size torch.Size([1, 10, 768])
# tensor([[[-0.0254,  0.0235,  0.1027,  ..., -0.1459, -0.0205, -0.0116],
#         [ 0.0606, -0.1811, -0.0418,  ..., -0.1815,  0.0880, -0.0766],
#         [-0.1561, -0.1127,  0.2687,  ..., -0.0648,  0.0249,  0.0446],
#         ...,
```

##### Extract contextual embedding features from all Camembert layers
```python
from transformers import CamembertConfig
# (Need to reload the model with new config)
config = CamembertConfig.from_pretrained("camembert-base", output_hidden_states=True)
camembert = CamembertModel.from_pretrained("camembert-base", config=config)

embeddings, _, all_layer_embeddings = camembert(encoded_sentence)
#  all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers)
all_layer_embeddings[5]
# layer 5 contextual embedding : size torch.Size([1, 10, 768])
#tensor([[[-0.0032,  0.0075,  0.0040,  ..., -0.0025, -0.0178, -0.0210],
#         [-0.0996, -0.1474,  0.1057,  ..., -0.0278,  0.1690, -0.2982],
#         [ 0.0557, -0.0588,  0.0547,  ..., -0.0726, -0.0867,  0.0699],
#         ...,
```

## Authors

CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.

## Citation
If you use our work, please cite:

```bibtex
@inproceedings{martin2020camembert,
  title={CamemBERT: a Tasty French Language Model},
  author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year={2020}
}
```


---
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# DataikuNLP/paraphrase-MiniLM-L6-v2

**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2/) from sentence-transformers at the specific commit `c4dfcde8a3e3e17e85cd4f0ec1925a266187f48e`.**

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-MiniLM-L6-v2')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-MiniLM-L6-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-MiniLM-L6-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-MiniLM-L6-v2)

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

This model was trained by [sentence-transformers](https://www.sbert.net/).

If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: wav2vec2-base-csa-10-rev3
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-base-csa-10-rev3

This model is a fine-tuned version of [facebook/wav2vec2-base](https://huggingface.co/facebook/wav2vec2-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.5869
- Wer: 1.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer |
|:-------------:|:-----:|:----:|:---------------:|:---:|
| 18.7934       | 25.0  | 200  | 3.5869          | 1.0 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.13.3
- Tokenizers 0.10.3


https://huggingface.co/blog/fine-tune-wav2vec2-english

Use the processor from https://huggingface.co/facebook/wav2vec2-base


---
language: it
tags:
- sentiment
- Italian
license: mit
widget:
- text: Giuseppe Rossi è un ottimo politico
---

# 🤗 + polibert_SA - POLItic BERT based Sentiment Analysis

## Model description

This model performs sentiment analysis on Italian political twitter sentences. It was trained starting from an instance of "bert-base-italian-uncased-xxl" and fine-tuned on an Italian dataset of tweets. You can try it out at https://www.unideeplearning.com/twitter_sa/ (in italian!)

#### Hands-on

```python
import torch
from torch import nn
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("unideeplearning/polibert_sa")
model = AutoModelForSequenceClassification.from_pretrained("unideeplearning/polibert_sa")

text = "Giuseppe Rossi è un pessimo politico"
input_ids = tokenizer.encode(text, add_special_tokens=True, return_tensors= 'pt')

logits, = model(input_ids)
logits = logits.squeeze(0)

prob = nn.functional.softmax(logits, dim=0)

# 0 Negative, 1 Neutral, 2 Positive
print(prob.argmax().tolist())
```

#### Hyperparameters

- Optimizer: **AdamW** with learning rate of **2e-5**, epsilon of **1e-8**
- Max epochs: **2**
- Batch size: **16**

## Acknowledgments

Thanks to the support from:
the [Hugging Face](https://huggingface.co/), https://www.unioneprofessionisti.com

https://www.unideeplearning.com/


---
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-demo-colab
  results: []

---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xlsr-53-demo-colab

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3966
- Wer: 0.4834

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.1516        | 4.21  | 400  | 2.7673          | 1.0    |
| 0.9134        | 8.42  | 800  | 0.4618          | 0.6418 |
| 0.3273        | 12.63 | 1200 | 0.4188          | 0.5535 |
| 0.2252        | 16.84 | 1600 | 0.4144          | 0.5232 |
| 0.1692        | 21.05 | 2000 | 0.3995          | 0.5030 |
| 0.1355        | 25.26 | 2400 | 0.4073          | 0.4920 |
| 0.1172        | 29.47 | 2800 | 0.3966          | 0.4834 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
license: apache-2.0
language: sah
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-sah-CV8
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice sah
      type: common_voice
      args: sah
    metrics:
       - name: Test WER
         type: wer
         value: 56.06
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xlsr-53-sah-CV8

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5089
- Wer: 0.5606

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.6849        | 16.67 | 500  | 1.1135          | 0.9344 |
| 0.8223        | 33.33 | 1000 | 0.5148          | 0.5686 |
| 0.5477        | 50.0  | 1500 | 0.5089          | 0.5606 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-med
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-Turkish-Tr-med

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4727
- Wer: 0.4677

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 60
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.8093        | 4.21  | 400  | 2.7831          | 1.0    |
| 0.9881        | 8.42  | 800  | 0.5088          | 0.6681 |
| 0.3519        | 12.63 | 1200 | 0.4496          | 0.6007 |
| 0.2436        | 16.84 | 1600 | 0.4993          | 0.5654 |
| 0.1874        | 21.05 | 2000 | 0.4793          | 0.5530 |
| 0.1561        | 25.26 | 2400 | 0.5187          | 0.5589 |
| 0.1336        | 29.47 | 2800 | 0.5135          | 0.5311 |
| 0.1163        | 33.68 | 3200 | 0.4960          | 0.5143 |
| 0.1056        | 37.89 | 3600 | 0.4795          | 0.5045 |
| 0.0959        | 42.11 | 4000 | 0.4883          | 0.4987 |
| 0.0819        | 46.32 | 4400 | 0.4799          | 0.4903 |
| 0.0756        | 50.53 | 4800 | 0.4822          | 0.4831 |
| 0.0692        | 54.74 | 5200 | 0.4621          | 0.4762 |
| 0.062         | 58.95 | 5600 | 0.4727          | 0.4677 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
license: apache-2.0
language: bas
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
- bas
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: wav2vec2-xls-r-300m-bas-CV8-v2
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 8
      type: mozilla-foundation/common_voice_8_0
      args: bas
    metrics:
      - name: Test WER
        type: wer
        value: 56.97
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-bas-CV8-v2

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6121
- Wer: 0.5697

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 90
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 6.5211        | 16.13 | 500  | 1.2661          | 0.9153 |
| 0.7026        | 32.25 | 1000 | 0.6245          | 0.6516 |
| 0.3752        | 48.38 | 1500 | 0.6039          | 0.6148 |
| 0.2752        | 64.51 | 2000 | 0.6080          | 0.5808 |
| 0.2155        | 80.63 | 2500 | 0.6121          | 0.5697 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8-Tr-med-CommonVoice8
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-Tr-med-CommonVoice8-Tr-med-CommonVoice8

This model is a fine-tuned version of [emre/wav2vec2-xls-r-300m-Tr-med-CommonVoice8](https://huggingface.co/emre/wav2vec2-xls-r-300m-Tr-med-CommonVoice8) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2708
- Wer: 0.5010

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 5
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 2.0402        | 0.67  | 500  | 0.3354          | 0.5681 |
| 0.7265        | 1.33  | 1000 | 0.3181          | 0.5444 |
| 0.6858        | 2.0   | 1500 | 0.3044          | 0.5322 |
| 0.6537        | 2.66  | 2000 | 0.2911          | 0.5217 |
| 0.6337        | 3.33  | 2500 | 0.2874          | 0.5164 |
| 0.6111        | 3.99  | 3000 | 0.2758          | 0.5059 |
| 0.5815        | 4.66  | 3500 | 0.2708          | 0.5010 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-small-CommonVoice8
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-Turkish-Tr-small-CommonVoice8

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4813
- Wer: 0.7207

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 5
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.2           | 0.53  | 400  | 3.1949          | 0.9964 |
| 2.9387        | 1.07  | 800  | 2.5015          | 1.0337 |
| 1.5975        | 1.6   | 1200 | 1.0928          | 0.9945 |
| 1.0688        | 2.13  | 1600 | 0.8388          | 0.9390 |
| 0.8977        | 2.66  | 2000 | 0.7106          | 0.8889 |
| 0.789         | 3.2   | 2400 | 0.6051          | 0.8273 |
| 0.7116        | 3.73  | 2800 | 0.5580          | 0.7855 |
| 0.6576        | 4.26  | 3200 | 0.5033          | 0.7433 |
| 0.6002        | 4.79  | 3600 | 0.4813          | 0.7207 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-hy-AM-CV8-v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-hy-AM-CV8-v1

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9145
- Wer: 0.9598

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 170
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 5.7132        | 83.31  | 500  | 1.9274          | 1.0523 |
| 1.017         | 166.62 | 1000 | 0.9145          | 0.9598 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


---
license: apache-2.0
language: gl
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-gl-CV8
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice gl
      type: common_voice
      args: gl
    metrics:
       - name: Test WER
         type: wer
         value: 0.2080
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-gl-CV8

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2151
- Wer: 0.2080

---

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.9427        | 4.9   | 500  | 2.8801          | 1.0    |
| 2.1594        | 9.8   | 1000 | 0.4092          | 0.4001 |
| 0.7332        | 14.71 | 1500 | 0.2151          | 0.2080 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


---
license: apache-2.0
language: as
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-as-CV8-v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-as-CV8-v1

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


---
language: tr
datasets:
- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech

license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Turkish by Davut Emre TASAR
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice tr
      type: common_voice
      args: tr
    metrics:
       - name: Test WER
         type: wer

---

# wav2vec-tr-lite-AG

## Usage

The model can be used directly (without a language model) as follows:

```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "tr", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("emre/wav2vec-tr-lite-AG")
model = Wav2Vec2ForCTC.from_pretrained("emre/wav2vec-tr-lite-AG")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.00005
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30.0
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.4388        | 3.7   | 400  | 1.366          | 0.9701 |
| 0.3766        | 7.4   | 800  | 0.4914          | 0.5374 |
| 0.2295        | 11.11 | 1200 | 0.3934          | 0.4125 |
| 0.1121        | 14.81 | 1600 | 0.3264          | 0.2904 |
| 0.1473        | 18.51 | 2000 | 0.3103          | 0.2671 |
| 0.1013        | 22.22 | 2400 | 0.2589          | 0.2324 |
| 0.0704        | 25.92 | 2800 | 0.2826          | 0.2339 |
| 0.0537        | 29.63 | 3200 | 0.2704          | 0.2309 |
### Framework versions
- Transformers 4.12.0.dev0
- Pytorch 1.8.1
- Datasets 1.14.1.dev0
- Tokenizers 0.10.3


---
license: apache-2.0
language: sah
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-W2V2-XLSR-300M-YAKUT-SMALL
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice sah
      type: common_voice
      args: sah
    metrics:
       - name: Test WER
         type: wer
         value: 79.00
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-W2V2-XLSR-300M-YAKUT-SMALL

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9068
- Wer: 0.7900

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.6926        | 19.05 | 400  | 2.7538          | 1.0    |
| 0.7031        | 38.1  | 800  | 0.9068          | 0.7900 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
language: tr

license: mit
---

# jurisprudence-textgen-gpt-2

Pretrained model on Turkish language using a causal language modeling (CLM) objective.

## Model description of Original GPT-2
GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.

More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to i but not the future tokens.

This way, the model learns an inner representation of the English language that can then be used to extract features useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a prompt.

## Model description of jurisprudence-textgen-gpt-2
Jurisprudence-textgen-gpt-2 is a transformers model for tensorflow pretrained with 18950 Turkish court Jurisprudence text data which has been obtained from [Bilirkisi GITHUB REPO TRAIN DATA] (https://github.com/Bilirkisi/Bilirkisi/tree/main/train) with 5 epochs.
Model Training results are:

Epoch 1/5
4986/4986 - 2770s 552ms/step - loss: 4.0122 - output_1_loss: 4.0122 - output_1_accuracy: 0.4544

Epoch 2/5
4986/4986 - 2753s 552ms/step - loss: 2.7074 - output_1_loss: 2.7074 - output_1_accuracy: 0.5843

Epoch 3/5
4986/4986 - 2754s 552ms/step - loss: 2.3411 - output_1_loss: 2.3411 - output_1_accuracy: 0.6214

Epoch 4/5
4986/4986 - 2754s 552ms/step - loss: 2.1241 - output_1_loss: 2.1241 - output_1_accuracy: 0.6431

Epoch 5/5
4986/4986 - 2754s 552ms/step - loss: 1.9647 - output_1_loss: 1.9647 - output_1_accuracy: 0.6597

## Intended uses & limitations
You can use the raw model for text generation or fine-tune it to a turkish law included downstream task. See the
[model hub](https://huggingface.co/models?filter=gpt2) to look for fine-tuned versions on a task that interests you.

### How to use
You can use this model directly with a pipeline for text generation.
Here is how to use this model to get the features of a given text in Tensorflow:
```python
>>> from transformers import GPT2Tokenizer , TFGPT2LMHeadModel
>>> tokenizer = GPT2Tokenizer.from_pretrained('emre/jurisprudence-textgen-gpt-2')
>>> model = TFGPT2LMHeadModel.from_pretrained('emre/jurisprudence-textgen-gpt-2')
>>> text = "Tarafların karşılıklı iddia ve savunmalarına," #Translation: "Mutual claims and defenses of the parties,"
>>> # encoding the input text
>>> input_ids = tokenizer.encode(text, return_tensors='tf')
>>> # getting out output

>>> beam_output = model.generate(
>>>   input_ids,
>>>   max_length = 250,
>>>   num_beams = 5,
>>>   temperature = 0.7,
>>>   no_repeat_ngram_size=2,
>>>   num_return_sequences=5
>>> )
>>> for i in range(5):
>>>   print(tokenizer.decode(beam_output[i]))
[{'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 25.12.2007 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz itirazlarının reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 15,60 TL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 30/12/2009 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 15,60 TL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 30/12/2009 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 25/12/2007 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 27/12/2007 gününde oybirliğiyle karar verildi."}]

```
### BibTeX entry and citation info
soon will be defined..


---
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-small
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-Turkish-Tr-small

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4375
- Wer: 0.5050

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.8735        | 4.21  | 400  | 2.8173          | 1.0002 |
| 1.0073        | 8.42  | 800  | 0.4981          | 0.6717 |
| 0.3395        | 12.63 | 1200 | 0.4470          | 0.5866 |
| 0.2254        | 16.84 | 1600 | 0.4349          | 0.5491 |
| 0.1648        | 21.05 | 2000 | 0.4454          | 0.5284 |
| 0.1325        | 25.26 | 2400 | 0.4552          | 0.5131 |
| 0.1102        | 29.47 | 2800 | 0.4375          | 0.5050 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
license: apache-2.0
language: tt
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
- tt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-W2V2-TATAR-SMALL
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 8
      type: mozilla-foundation/common_voice_8_0
      args: tt
    metrics:
      - name: Test WER
        type: wer
        value: 53.16
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xlsr-53-W2V2-TATAR-SMALL

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4714
- Wer: 0.5316

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 6.2446        | 1.17  | 400   | 3.2621          | 1.0    |
| 1.739         | 2.35  | 800   | 0.5832          | 0.7688 |
| 0.4718        | 3.52  | 1200  | 0.4785          | 0.6824 |
| 0.3574        | 4.69  | 1600  | 0.4814          | 0.6792 |
| 0.2946        | 5.86  | 2000  | 0.4484          | 0.6506 |
| 0.2674        | 7.04  | 2400  | 0.4612          | 0.6225 |
| 0.2349        | 8.21  | 2800  | 0.4600          | 0.6050 |
| 0.2206        | 9.38  | 3200  | 0.4772          | 0.6048 |
| 0.2072        | 10.56 | 3600  | 0.4676          | 0.6106 |
| 0.1984        | 11.73 | 4000  | 0.4816          | 0.6079 |
| 0.1793        | 12.9  | 4400  | 0.4616          | 0.5836 |
| 0.172         | 14.08 | 4800  | 0.4808          | 0.5860 |
| 0.1624        | 15.25 | 5200  | 0.4854          | 0.5820 |
| 0.156         | 16.42 | 5600  | 0.4609          | 0.5656 |
| 0.1448        | 17.59 | 6000  | 0.4926          | 0.5817 |
| 0.1406        | 18.77 | 6400  | 0.4638          | 0.5654 |
| 0.1337        | 19.94 | 6800  | 0.4731          | 0.5652 |
| 0.1317        | 21.11 | 7200  | 0.4861          | 0.5639 |
| 0.1179        | 22.29 | 7600  | 0.4766          | 0.5521 |
| 0.1197        | 23.46 | 8000  | 0.4824          | 0.5584 |
| 0.1096        | 24.63 | 8400  | 0.5006          | 0.5559 |
| 0.1038        | 25.81 | 8800  | 0.4994          | 0.5440 |
| 0.0992        | 26.98 | 9200  | 0.4867          | 0.5405 |
| 0.0984        | 28.15 | 9600  | 0.4798          | 0.5361 |
| 0.0943        | 29.33 | 10000 | 0.4714          | 0.5316 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
- summarization
metrics:
- rouge
model-index:
- name: arxiv27k-t5-abst-title-gen/
  results: []
---

# arxiv27k-t5-abst-title-gen/

This model is a fine-tuned version of mt5-small on the arxiv-abstract-title dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6002
- Rouge1: 32.8
- Rouge2: 21.9
- Rougel: 34.8
-

## Model description

Model has been trained with a colab-pro notebook in 4 hours.

## Intended uses & limitations

Can be used for generating journal titles from given abstracts

### Training args
model_args = T5Args()
model_args.max_seq_length = 256
model_args.train_batch_size = 8
model_args.eval_batch_size = 8
model_args.num_train_epochs = 6
model_args.evaluate_during_training = False
model_args.use_multiprocessing = False
model_args.fp16 = False
model_args.save_steps = 40000
model_args.save_eval_checkpoints = False
model_args.save_model_every_epoch = True
model_args.output_dir = OUTPUT_DIR
model_args.no_cache = True
model_args.reprocess_input_data = True
model_args.overwrite_output_dir = True
model_args.num_return_sequences = 1

### Framework versions

- Transformers 4.12.5
- Pytorch 1.10.0+cu111
- Datasets 1.15.1
- Tokenizers 0.10.3

### Contact
detasar@gmail.com
Davut Emre Taşar


---
license: apache-2.0
language: tr
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice tr
      type: common_voice
      args: tr
    metrics:
       - name: Test WER
         type: wer
         value: 49.14

---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-Tr-med-CommonVoice8

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2556
- Wer: 0.4914

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 1.4876        | 6.66  | 5000  | 0.3252          | 0.5784 |
| 0.6919        | 13.32 | 10000 | 0.2720          | 0.5172 |
| 0.5919        | 19.97 | 15000 | 0.2556          | 0.4914 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


---
license: apache-2.0
language: br
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Br-small
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice br
      type: common_voice
      args: br
    metrics:
       - name: Test WER
         type: wer
         value: 66.75
---

# wav2vec2-xls-r-300m-Br-small

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0573
- Wer: 0.6675

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.7464        | 2.79  | 400  | 1.7474          | 1.1018 |
| 1.1117        | 5.59  | 800  | 0.9434          | 0.8697 |
| 0.6481        | 8.39  | 1200 | 0.9251          | 0.7910 |
| 0.4754        | 11.19 | 1600 | 0.9208          | 0.7412 |
| 0.3602        | 13.98 | 2000 | 0.9284          | 0.7232 |
| 0.2873        | 16.78 | 2400 | 0.9299          | 0.6940 |
| 0.2386        | 19.58 | 2800 | 1.0182          | 0.6927 |
| 0.1971        | 22.38 | 3200 | 1.0456          | 0.6898 |
| 0.1749        | 25.17 | 3600 | 1.0208          | 0.6769 |
| 0.1487        | 27.97 | 4000 | 1.0573          | 0.6675 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


---
license: apache-2.0
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-W2V2-TR-MED
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xlsr-53-W2V2-TR-MED

This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4467
- Wer: 0.4598

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 60
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.1343        | 4.21  | 400  | 2.3674          | 1.0372 |
| 0.8075        | 8.42  | 800  | 0.4583          | 0.6308 |
| 0.3209        | 12.63 | 1200 | 0.4291          | 0.5531 |
| 0.2273        | 16.84 | 1600 | 0.4348          | 0.5378 |
| 0.1764        | 21.05 | 2000 | 0.4550          | 0.5326 |
| 0.148         | 25.26 | 2400 | 0.4839          | 0.5319 |
| 0.1268        | 29.47 | 2800 | 0.4515          | 0.5070 |
| 0.1113        | 33.68 | 3200 | 0.4590          | 0.4930 |
| 0.1025        | 37.89 | 3600 | 0.4546          | 0.4888 |
| 0.0922        | 42.11 | 4000 | 0.4782          | 0.4852 |
| 0.082         | 46.32 | 4400 | 0.4605          | 0.4752 |
| 0.0751        | 50.53 | 4800 | 0.4358          | 0.4689 |
| 0.0699        | 54.74 | 5200 | 0.4359          | 0.4629 |
| 0.0633        | 58.95 | 5600 | 0.4467          | 0.4598 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3