```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/distiluse-base-multilingual-cased-v1')
embeddings = model.encode(sentences)
print(embeddings)
```


## Evaluation Results


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/distiluse-base-multilingual-cased-v1)


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
  (2): Dense({'in_features': 768, 'out_features': 512, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)
```


## Citing & Authors


This model was trained by [sentence-transformers](https://www.sbert.net/).


If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):


```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


model name: distiluse-base-multilingual-cased-v1


model author: DataikuNLP


task: sentence-similarity


tags: pytorch, distilbert, feature-extraction, arxiv:1908.10084, sentence-transformers, license:apache-2.0, sentence-similarity, transformers, infinity_compatible


pipeline_tag: sentence-similarity
license: apache-2.0
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# DataikuNLP/paraphrase-albert-small-v2


**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-albert-small-v2/) from sentence-transformers at the specific commit `1eb1996223dd90a4c25be2fc52f6f336419a0d52`.**


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-albert-small-v2')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-albert-small-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-albert-small-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-albert-small-v2)


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 100, 'do_lower_case': False}) with Transformer model: AlbertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


model name: paraphrase-albert-small-v2


model author: DataikuNLP


task: sentence-similarity


tags: pytorch, albert, feature-extraction, arxiv:1908.10084, sentence-transformers, license:apache-2.0, sentence-similarity, transformers, infinity_compatible


- sentence-transformers
- feature-extraction
- sentence-similarity


# average_word_embeddings_glove.6B.300d


**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/average_word_embeddings_glove.6B.300d) from sentence-transformers at the specific commit `5d2b7d1c127036ae98b9d487eca4d48744edc709`.**


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 300 dimensional dense vector space and can be used for tasks like clustering or semantic search.


## Usage (Sentence-Transformers)


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/average_word_embeddings_glove.6B.300d')
embeddings = model.encode(sentences)
print(embeddings)
```


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/average_word_embeddings_glove.6B.300d)


```
SentenceTransformer(
  (0): WordEmbeddings(
    (emb_layer): Embedding(400001, 300)
  )
  (1): Pooling({'word_embedding_dimension': 300, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


This model was trained by [sentence-transformers](https://www.sbert.net/).


If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):


```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


model name: average_word_embeddings_glove.6B.300d


tags: arxiv:1908.10084, sentence-transformers, license:apache-2.0, feature-extraction, sentence-similarity


pipeline_tag: sentence-similarity
license: apache-2.0
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# DataikuNLP/paraphrase-multilingual-MiniLM-L12-v2


**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) from sentence-transformers at the specific commit `d66eff4d8a8598f264f166af8db67f7797164651`.**


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


This model was trained by [sentence-transformers](https://www.sbert.net/).


If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):


```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


model name: paraphrase-multilingual-MiniLM-L12-v2


model author: DataikuNLP


task: sentence-similarity


tags: pytorch, bert, feature-extraction, arxiv:1908.10084, sentence-transformers, license:apache-2.0, sentence-similarity, transformers, infinity_compatible


language: fr
license: mit
datasets:


- oscar


# CamemBERT: a Tasty French Language Model


**This model is a copy of [this model repository](https://huggingface.co/camembert-base) at the specific commit `482393b6198924f9da270b1aaf37d238aafca99b`.**


## Introduction


[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model.


It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains.


For further information or requests, please go to [Camembert Website](https://camembert-model.fr/)


## Pre-trained models


| Model                          | #params                        | Arch. | Training data                     |
|--------------------------------|--------------------------------|-------|-----------------------------------|
| `camembert-base` | 110M   | Base  | OSCAR (138 GB of text)            |
| `camembert/camembert-large`              | 335M    | Large | CCNet (135 GB of text)            |
| `camembert/camembert-base-ccnet`         | 110M    | Base  | CCNet (135 GB of text)            |
| `camembert/camembert-base-wikipedia-4gb` | 110M    | Base  | Wikipedia (4 GB of text)          |
| `camembert/camembert-base-oscar-4gb`     | 110M    | Base  | Subsample of OSCAR (4 GB of text) |
| `camembert/camembert-base-ccnet-4gb`     | 110M    | Base  | Subsample of CCNet (4 GB of text) |


## How to use CamemBERT with HuggingFace


##### Load CamemBERT and its sub-word tokenizer :


```python
from transformers import CamembertModel, CamembertTokenizer

# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large".
tokenizer = CamembertTokenizer.from_pretrained("camembert-base")
camembert = CamembertModel.from_pretrained("camembert-base")

camembert.eval()  # disable dropout (or leave in train mode to finetune)

```


##### Filling masks using pipeline


```python
from transformers import pipeline

camembert_fill_mask  = pipeline("fill-mask", model="camembert-base", tokenizer="camembert-base")
results = camembert_fill_mask("Le camembert est <mask> :)")
# results
#[{'sequence': '<s> Le camembert est délicieux :)</s>', 'score': 0.4909103214740753, 'token': 7200},
# {'sequence': '<s> Le camembert est excellent :)</s>', 'score': 0.10556930303573608, 'token': 2183},
# {'sequence': '<s> Le camembert est succulent :)</s>', 'score': 0.03453315049409866, 'token': 26202},
# {'sequence': '<s> Le camembert est meilleur :)</s>', 'score': 0.03303130343556404, 'token': 528},
# {'sequence': '<s> Le camembert est parfait :)</s>', 'score': 0.030076518654823303, 'token': 1654}]

```


##### Extract contextual embedding features from Camembert output


```python
import torch
# Tokenize in sub-words with SentencePiece
tokenized_sentence = tokenizer.tokenize("J'aime le camembert !")
# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!']

# 1-hot encode and add special starting and end tokens
encoded_sentence = tokenizer.encode(tokenized_sentence)
# [5, 121, 11, 660, 16, 730, 25543, 110, 83, 6]
# NB: Can be done in one step : tokenize.encode("J'aime le camembert !")

# Feed tokens to Camembert as a torch tensor (batch dim 1)
encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0)
embeddings, _ = camembert(encoded_sentence)
# embeddings.detach()
# embeddings.size torch.Size([1, 10, 768])
# tensor([[[-0.0254,  0.0235,  0.1027,  ..., -0.1459, -0.0205, -0.0116],
#         [ 0.0606, -0.1811, -0.0418,  ..., -0.1815,  0.0880, -0.0766],
#         [-0.1561, -0.1127,  0.2687,  ..., -0.0648,  0.0249,  0.0446],
#         ...,
```


##### Extract contextual embedding features from all Camembert layers


```python
from transformers import CamembertConfig
# (Need to reload the model with new config)
config = CamembertConfig.from_pretrained("camembert-base", output_hidden_states=True)
camembert = CamembertModel.from_pretrained("camembert-base", config=config)

embeddings, _, all_layer_embeddings = camembert(encoded_sentence)
#  all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers)
all_layer_embeddings[5]
# layer 5 contextual embedding : size torch.Size([1, 10, 768])
#tensor([[[-0.0032,  0.0075,  0.0040,  ..., -0.0025, -0.0178, -0.0210],
#         [-0.0996, -0.1474,  0.1057,  ..., -0.0278,  0.1690, -0.2982],
#         [ 0.0557, -0.0588,  0.0547,  ..., -0.0726, -0.0867,  0.0699],
#         ...,
```


## Authors


CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot.


## Citation


If you use our work, please cite:


```bibtex
@inproceedings{martin2020camembert,
  title={CamemBERT: a Tasty French Language Model},
  author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year={2020}
}
```


model name: camembert-base


model author: DataikuNLP


task: fill-mask


tags: pytorch, tf, camembert, fill-mask, fr, dataset:oscar, arxiv:1911.03894, transformers, license:mit, autonlp_compatible


pipeline_tag: sentence-similarity
license: apache-2.0
tags:


- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers


# DataikuNLP/paraphrase-MiniLM-L6-v2


**This model is a copy of [this model repository](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2/) from sentence-transformers at the specific commit `c4dfcde8a3e3e17e85cd4f0ec1925a266187f48e`.**


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('sentence-transformers/paraphrase-MiniLM-L6-v2')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-MiniLM-L6-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-MiniLM-L6-v2')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/paraphrase-MiniLM-L6-v2)


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


This model was trained by [sentence-transformers](https://www.sbert.net/).


If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):


```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "http://arxiv.org/abs/1908.10084",
}
```


model name: paraphrase-MiniLM-L6-v2


model author: DataikuNLP


task: sentence-similarity


tags: pytorch, bert, feature-extraction, arxiv:1908.10084, sentence-transformers, license:apache-2.0, sentence-similarity, transformers, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: wav2vec2-base-csa-10-rev3
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-base-csa-10-rev3


This model is a fine-tuned version of [facebook/wav2vec2-base](https://huggingface.co/facebook/wav2vec2-base) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 3.5869
- Wer: 1.0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer |
|:-------------:|:-----:|:----:|:---------------:|:---:|
| 18.7934       | 25.0  | 200  | 3.5869          | 1.0 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: wav2vec2-base-csa-10-rev3


model author: Jainil30


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, transformers, license:apache-2.0, generated_from_trainer, model-index


Use the processor from https://huggingface.co/facebook/wav2vec2-base


model name: wav2vec2-base-timit-demo


model author: kwang1993


tags: pytorch, wav2vec2, automatic-speech-recognition, transformers


language: it
tags:


- sentiment
- Italian
license: mit
widget:
- text: Giuseppe Rossi è un ottimo politico


# 🤗 + polibert_SA - POLItic BERT based Sentiment Analysis


## Model description


This model performs sentiment analysis on Italian political twitter sentences. It was trained starting from an instance of "bert-base-italian-uncased-xxl" and fine-tuned on an Italian dataset of tweets. You can try it out at https://www.unideeplearning.com/twitter_sa/ (in italian!)


#### Hands-on


```python
import torch
from torch import nn
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("unideeplearning/polibert_sa")
model = AutoModelForSequenceClassification.from_pretrained("unideeplearning/polibert_sa")

text = "Giuseppe Rossi è un pessimo politico"
input_ids = tokenizer.encode(text, add_special_tokens=True, return_tensors= 'pt')

logits, = model(input_ids)
logits = logits.squeeze(0)

prob = nn.functional.softmax(logits, dim=0)

# 0 Negative, 1 Neutral, 2 Positive
print(prob.argmax().tolist())
```


#### Hyperparameters


- Optimizer: **AdamW** with learning rate of **2e-5**, epsilon of **1e-8**
- Max epochs: **2**
- Batch size: **16**


## Acknowledgments


Thanks to the support from:
the [Hugging Face](https://huggingface.co/), https://www.unioneprofessionisti.com


model name: polibert_sa


model author: unideeplearning


task: text-classification


tags: pytorch, tf, jax, bert, text-classification, it, transformers, sentiment, Italian, license:mit, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-demo-colab
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-large-xlsr-53-demo-colab


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3966
- Wer: 0.4834


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.1516        | 4.21  | 400  | 2.7673          | 1.0    |
| 0.9134        | 8.42  | 800  | 0.4618          | 0.6418 |
| 0.3273        | 12.63 | 1200 | 0.4188          | 0.5535 |
| 0.2252        | 16.84 | 1600 | 0.4144          | 0.5232 |
| 0.1692        | 21.05 | 2000 | 0.3995          | 0.5030 |
| 0.1355        | 25.26 | 2400 | 0.4073          | 0.4920 |
| 0.1172        | 29.47 | 2800 | 0.3966          | 0.4834 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: wav2vec2-large-xlsr-53-demo-colab


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: sah
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-sah-CV8
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice sah
type: common_voice
args: sah
metrics:
    - name: Test WER
type: wer
value: 56.06


# wav2vec2-large-xlsr-53-sah-CV8


- Loss: 0.5089
- Wer: 0.5606


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 50
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.6849        | 16.67 | 500  | 1.1135          | 0.9344 |
| 0.8223        | 33.33 | 1000 | 0.5148          | 0.5686 |
| 0.5477        | 50.0  | 1500 | 0.5089          | 0.5606 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: wav2vec2-large-xlsr-53-sah-CV8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, sah, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-med
results: []


# wav2vec2-xls-r-300m-Turkish-Tr-med


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4727
- Wer: 0.4677


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 60
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.8093        | 4.21  | 400  | 2.7831          | 1.0    |
| 0.9881        | 8.42  | 800  | 0.5088          | 0.6681 |
| 0.3519        | 12.63 | 1200 | 0.4496          | 0.6007 |
| 0.2436        | 16.84 | 1600 | 0.4993          | 0.5654 |
| 0.1874        | 21.05 | 2000 | 0.4793          | 0.5530 |
| 0.1561        | 25.26 | 2400 | 0.5187          | 0.5589 |
| 0.1336        | 29.47 | 2800 | 0.5135          | 0.5311 |
| 0.1163        | 33.68 | 3200 | 0.4960          | 0.5143 |
| 0.1056        | 37.89 | 3600 | 0.4795          | 0.5045 |
| 0.0959        | 42.11 | 4000 | 0.4883          | 0.4987 |
| 0.0819        | 46.32 | 4400 | 0.4799          | 0.4903 |
| 0.0756        | 50.53 | 4800 | 0.4822          | 0.4831 |
| 0.0692        | 54.74 | 5200 | 0.4621          | 0.4762 |
| 0.062         | 58.95 | 5600 | 0.4727          | 0.4677 |


### Framework versions


model name: wav2vec2-xls-r-300m-Turkish-Tr-med


model author: emre


task: automatic-speech-recognition


license: apache-2.0
language: bas
tags:


- automatic-speech-recognition
- common_voice
- generated_from_trainer
- bas
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
model-index:
- name: wav2vec2-xls-r-300m-bas-CV8-v2
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 8
type: mozilla-foundation/common_voice_8_0
args: bas
metrics:
    - name: Test WER
type: wer
value: 56.97


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-300m-bas-CV8-v2


- Loss: 0.6121
- Wer: 0.5697


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 90
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 6.5211        | 16.13 | 500  | 1.2661          | 0.9153 |
| 0.7026        | 32.25 | 1000 | 0.6245          | 0.6516 |
| 0.3752        | 48.38 | 1500 | 0.6039          | 0.6148 |
| 0.2752        | 64.51 | 2000 | 0.6080          | 0.5808 |
| 0.2155        | 80.63 | 2500 | 0.6121          | 0.5697 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-bas-CV8-v2


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, bas, dataset:mozilla-foundation/common_voice_8_0, transformers, license:apache-2.0, common_voice, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8-Tr-med-CommonVoice8
results: []


# wav2vec2-xls-r-300m-Tr-med-CommonVoice8-Tr-med-CommonVoice8


This model is a fine-tuned version of [emre/wav2vec2-xls-r-300m-Tr-med-CommonVoice8](https://huggingface.co/emre/wav2vec2-xls-r-300m-Tr-med-CommonVoice8) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2708
- Wer: 0.5010


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 5
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 2.0402        | 0.67  | 500  | 0.3354          | 0.5681 |
| 0.7265        | 1.33  | 1000 | 0.3181          | 0.5444 |
| 0.6858        | 2.0   | 1500 | 0.3044          | 0.5322 |
| 0.6537        | 2.66  | 2000 | 0.2911          | 0.5217 |
| 0.6337        | 3.33  | 2500 | 0.2874          | 0.5164 |
| 0.6111        | 3.99  | 3000 | 0.2758          | 0.5059 |
| 0.5815        | 4.66  | 3500 | 0.2708          | 0.5010 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8-Tr-med-CommonVoice8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, model-index


license: apache-2.0
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-small-CommonVoice8
results: []


# wav2vec2-xls-r-300m-Turkish-Tr-small-CommonVoice8


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4813
- Wer: 0.7207


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 5
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.2           | 0.53  | 400  | 3.1949          | 0.9964 |
| 2.9387        | 1.07  | 800  | 2.5015          | 1.0337 |
| 1.5975        | 1.6   | 1200 | 1.0928          | 0.9945 |
| 1.0688        | 2.13  | 1600 | 0.8388          | 0.9390 |
| 0.8977        | 2.66  | 2000 | 0.7106          | 0.8889 |
| 0.789         | 3.2   | 2400 | 0.6051          | 0.8273 |
| 0.7116        | 3.73  | 2800 | 0.5580          | 0.7855 |
| 0.6576        | 4.26  | 3200 | 0.5033          | 0.7433 |
| 0.6002        | 4.79  | 3600 | 0.4813          | 0.7207 |


### Framework versions


model name: wav2vec2-xls-r-300m-Turkish-Tr-small-CommonVoice8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-hy-AM-CV8-v1
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-300m-hy-AM-CV8-v1


- Loss: 0.9145
- Wer: 0.9598


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 170
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 5.7132        | 83.31  | 500  | 1.9274          | 1.0523 |
| 1.017         | 166.62 | 1000 | 0.9145          | 0.9598 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-hy-AM-CV8-v1


model author: emre


task: automatic-speech-recognition


license: apache-2.0
language: gl
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-gl-CV8
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice gl
type: common_voice
args: gl
metrics:
    - name: Test WER
type: wer
value: 0.2080


# wav2vec2-xls-r-300m-gl-CV8


- Loss: 0.2151
- Wer: 0.2080


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 15
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.9427        | 4.9   | 500  | 2.8801          | 1.0    |
| 2.1594        | 9.8   | 1000 | 0.4092          | 0.4001 |
| 0.7332        | 14.71 | 1500 | 0.2151          | 0.2080 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-gl-CV8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, gl, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: as
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-as-CV8-v1
results: []


# wav2vec2-xls-r-300m-as-CV8-v1


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


### Framework versions


model name: wav2vec2-xls-r-300m-as-CV8-v1


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, as, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language: tr
datasets:


- common_voice
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech


license: apache-2.0
model-index:


- name: XLSR Wav2Vec2 Turkish by Davut Emre TASAR
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice tr
type: common_voice
args: tr
metrics:
    - name: Test WER
type: wer


# wav2vec-tr-lite-AG


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "tr", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("emre/wav2vec-tr-lite-AG")
model = Wav2Vec2ForCTC.from_pretrained("emre/wav2vec-tr-lite-AG")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.00005
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30.0
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.4388        | 3.7   | 400  | 1.366          | 0.9701 |
| 0.3766        | 7.4   | 800  | 0.4914          | 0.5374 |
| 0.2295        | 11.11 | 1200 | 0.3934          | 0.4125 |
| 0.1121        | 14.81 | 1600 | 0.3264          | 0.2904 |
| 0.1473        | 18.51 | 2000 | 0.3103          | 0.2671 |
| 0.1013        | 22.22 | 2400 | 0.2589          | 0.2324 |
| 0.0704        | 25.92 | 2800 | 0.2826          | 0.2339 |
| 0.0537        | 29.63 | 3200 | 0.2704          | 0.2309 |
### Framework versions
- Transformers 4.12.0.dev0
- Pytorch 1.8.1
- Datasets 1.14.1.dev0
- Tokenizers 0.10.3

```


model name: wav2vec-tr-lite-AG


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, tr, dataset:common_voice, transformers, audio, speech, license:apache-2.0, model-index


license: apache-2.0
language: sah
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-W2V2-XLSR-300M-YAKUT-SMALL
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice sah
type: common_voice
args: sah
metrics:
    - name: Test WER
type: wer
value: 79.00


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-300m-W2V2-XLSR-300M-YAKUT-SMALL


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.9068
- Wer: 0.7900


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 50
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.6926        | 19.05 | 400  | 2.7538          | 1.0    |
| 0.7031        | 38.1  | 800  | 0.9068          | 0.7900 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-W2V2-XLSR-300M-YAKUT-SMALL


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, sah, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language: tr


## license: mit


# jurisprudence-textgen-gpt-2


Pretrained model on Turkish language using a causal language modeling (CLM) objective.


## Model description of Original GPT-2


GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.


More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to i but not the future tokens.


This way, the model learns an inner representation of the English language that can then be used to extract features useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a prompt.


## Model description of jurisprudence-textgen-gpt-2


Jurisprudence-textgen-gpt-2 is a transformers model for tensorflow pretrained with 18950 Turkish court Jurisprudence text data which has been obtained from [Bilirkisi GITHUB REPO TRAIN DATA] (https://github.com/Bilirkisi/Bilirkisi/tree/main/train) with 5 epochs.
Model Training results are:


Epoch 1/5
4986/4986 - 2770s 552ms/step - loss: 4.0122 - output_1_loss: 4.0122 - output_1_accuracy: 0.4544


Epoch 2/5
4986/4986 - 2753s 552ms/step - loss: 2.7074 - output_1_loss: 2.7074 - output_1_accuracy: 0.5843


Epoch 3/5
4986/4986 - 2754s 552ms/step - loss: 2.3411 - output_1_loss: 2.3411 - output_1_accuracy: 0.6214


Epoch 4/5
4986/4986 - 2754s 552ms/step - loss: 2.1241 - output_1_loss: 2.1241 - output_1_accuracy: 0.6431


Epoch 5/5
4986/4986 - 2754s 552ms/step - loss: 1.9647 - output_1_loss: 1.9647 - output_1_accuracy: 0.6597


## Intended uses & limitations


You can use the raw model for text generation or fine-tune it to a turkish law included downstream task. See the
[model hub](https://huggingface.co/models?filter=gpt2) to look for fine-tuned versions on a task that interests you.


### How to use


You can use this model directly with a pipeline for text generation.
Here is how to use this model to get the features of a given text in Tensorflow:


```python
>>> from transformers import GPT2Tokenizer , TFGPT2LMHeadModel
>>> tokenizer = GPT2Tokenizer.from_pretrained('emre/jurisprudence-textgen-gpt-2')
>>> model = TFGPT2LMHeadModel.from_pretrained('emre/jurisprudence-textgen-gpt-2')
>>> text = "Tarafların karşılıklı iddia ve savunmalarına," #Translation: "Mutual claims and defenses of the parties,"
>>> # encoding the input text
>>> input_ids = tokenizer.encode(text, return_tensors='tf')
>>> # getting out output

>>> beam_output = model.generate(
>>>   input_ids,
>>>   max_length = 250,
>>>   num_beams = 5,
>>>   temperature = 0.7,
>>>   no_repeat_ngram_size=2,
>>>   num_return_sequences=5
>>> )
>>> for i in range(5):
>>>   print(tokenizer.decode(beam_output[i]))
[{'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 25.12.2007 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz itirazlarının reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 15,60 TL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 30/12/2009 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 15,60 TL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 30/12/2009 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 25/12/2007 gününde oybirliğiyle karar verildi."},
 {'generated_text': "Tarafların karşılıklı iddia ve savunmalarına, dayandıkları belgelere, temyiz olunan kararda yazılı gerekçelere göre yerinde bulunmayan temyiz sebeplerinin reddiyle usul ve kanuna uygun mahkeme kararının İİK. 366. ve HUMK. 438. maddeleri uyarınca (ONANMASINA), 13.10 YTL onama harcı temyiz edenden alındığından başkaca harç alınmasına mahal olmadığına, 27/12/2007 gününde oybirliğiyle karar verildi."}]

```


### BibTeX entry and citation info


soon will be defined..


model name: jurisprudence-textgen-gpt-2


model author: emre


tags: tf, gpt2, tr, transformers, license:mit


license: apache-2.0
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Turkish-Tr-small
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-300m-Turkish-Tr-small


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4375
- Wer: 0.5050


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 4.8735        | 4.21  | 400  | 2.8173          | 1.0002 |
| 1.0073        | 8.42  | 800  | 0.4981          | 0.6717 |
| 0.3395        | 12.63 | 1200 | 0.4470          | 0.5866 |
| 0.2254        | 16.84 | 1600 | 0.4349          | 0.5491 |
| 0.1648        | 21.05 | 2000 | 0.4454          | 0.5284 |
| 0.1325        | 25.26 | 2400 | 0.4552          | 0.5131 |
| 0.1102        | 29.47 | 2800 | 0.4375          | 0.5050 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-Turkish-Tr-small


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: tt
tags:


- automatic-speech-recognition
- common_voice
- generated_from_trainer
- tt
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-W2V2-TATAR-SMALL
results:
  - task:
name: Automatic Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 8
type: mozilla-foundation/common_voice_8_0
args: tt
metrics:
    - name: Test WER
type: wer
value: 53.16


# wav2vec2-large-xlsr-53-W2V2-TATAR-SMALL


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4714
- Wer: 0.5316


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 6.2446        | 1.17  | 400   | 3.2621          | 1.0    |
| 1.739         | 2.35  | 800   | 0.5832          | 0.7688 |
| 0.4718        | 3.52  | 1200  | 0.4785          | 0.6824 |
| 0.3574        | 4.69  | 1600  | 0.4814          | 0.6792 |
| 0.2946        | 5.86  | 2000  | 0.4484          | 0.6506 |
| 0.2674        | 7.04  | 2400  | 0.4612          | 0.6225 |
| 0.2349        | 8.21  | 2800  | 0.4600          | 0.6050 |
| 0.2206        | 9.38  | 3200  | 0.4772          | 0.6048 |
| 0.2072        | 10.56 | 3600  | 0.4676          | 0.6106 |
| 0.1984        | 11.73 | 4000  | 0.4816          | 0.6079 |
| 0.1793        | 12.9  | 4400  | 0.4616          | 0.5836 |
| 0.172         | 14.08 | 4800  | 0.4808          | 0.5860 |
| 0.1624        | 15.25 | 5200  | 0.4854          | 0.5820 |
| 0.156         | 16.42 | 5600  | 0.4609          | 0.5656 |
| 0.1448        | 17.59 | 6000  | 0.4926          | 0.5817 |
| 0.1406        | 18.77 | 6400  | 0.4638          | 0.5654 |
| 0.1337        | 19.94 | 6800  | 0.4731          | 0.5652 |
| 0.1317        | 21.11 | 7200  | 0.4861          | 0.5639 |
| 0.1179        | 22.29 | 7600  | 0.4766          | 0.5521 |
| 0.1197        | 23.46 | 8000  | 0.4824          | 0.5584 |
| 0.1096        | 24.63 | 8400  | 0.5006          | 0.5559 |
| 0.1038        | 25.81 | 8800  | 0.4994          | 0.5440 |
| 0.0992        | 26.98 | 9200  | 0.4867          | 0.5405 |
| 0.0984        | 28.15 | 9600  | 0.4798          | 0.5361 |
| 0.0943        | 29.33 | 10000 | 0.4714          | 0.5316 |


### Framework versions


model name: wav2vec2-large-xlsr-53-W2V2-TATAR-SMALL


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, tt, dataset:common_voice, transformers, license:apache-2.0, common_voice, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
tags:


- generated_from_trainer
- summarization
metrics:
- rouge
model-index:
- name: arxiv27k-t5-abst-title-gen/
results: []


# arxiv27k-t5-abst-title-gen/


This model is a fine-tuned version of mt5-small on the arxiv-abstract-title dataset.
It achieves the following results on the evaluation set:


- Loss: 1.6002
- Rouge1: 32.8
- Rouge2: 21.9
- Rougel: 34.8


## Model description


Model has been trained with a colab-pro notebook in 4 hours.


## Intended uses & limitations


Can be used for generating journal titles from given abstracts


### Training args


model_args = T5Args()
model_args.max_seq_length = 256
model_args.train_batch_size = 8
model_args.eval_batch_size = 8
model_args.num_train_epochs = 6
model_args.evaluate_during_training = False
model_args.use_multiprocessing = False
model_args.fp16 = False
model_args.save_steps = 40000
model_args.save_eval_checkpoints = False
model_args.save_model_every_epoch = True
model_args.output_dir = OUTPUT_DIR
model_args.no_cache = True
model_args.reprocess_input_data = True
model_args.overwrite_output_dir = True
model_args.num_return_sequences = 1


- Transformers 4.12.5
- Pytorch 1.10.0+cu111
- Datasets 1.15.1
- Tokenizers 0.10.3


### Contact


detasar@gmail.com
Davut Emre Taşar


model name: arxiv27k-t5-abst-title-gen


model author: emre


task: summarization


tags: pytorch, mt5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, summarization, model-index, autonlp_compatible


license: apache-2.0
language: tr
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice tr
type: common_voice
args: tr
metrics:
    - name: Test WER
type: wer
value: 49.14


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-xls-r-300m-Tr-med-CommonVoice8


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2556
- Wer: 0.4914


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 20
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 1.4876        | 6.66  | 5000  | 0.3252          | 0.5784 |
| 0.6919        | 13.32 | 10000 | 0.2720          | 0.5172 |
| 0.5919        | 19.97 | 15000 | 0.2556          | 0.4914 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-Tr-med-CommonVoice8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, tr, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: br
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Br-small
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice br
type: common_voice
args: br
metrics:
    - name: Test WER
type: wer
value: 66.75


# wav2vec2-xls-r-300m-Br-small


- Loss: 1.0573
- Wer: 0.6675


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.7464        | 2.79  | 400  | 1.7474          | 1.1018 |
| 1.1117        | 5.59  | 800  | 0.9434          | 0.8697 |
| 0.6481        | 8.39  | 1200 | 0.9251          | 0.7910 |
| 0.4754        | 11.19 | 1600 | 0.9208          | 0.7412 |
| 0.3602        | 13.98 | 2000 | 0.9284          | 0.7232 |
| 0.2873        | 16.78 | 2400 | 0.9299          | 0.6940 |
| 0.2386        | 19.58 | 2800 | 1.0182          | 0.6927 |
| 0.1971        | 22.38 | 3200 | 1.0456          | 0.6898 |
| 0.1749        | 25.17 | 3600 | 1.0208          | 0.6769 |
| 0.1487        | 27.97 | 4000 | 1.0573          | 0.6675 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-Br-small


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, br, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-53-W2V2-TR-MED
results: []


# wav2vec2-large-xlsr-53-W2V2-TR-MED


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4467
- Wer: 0.4598


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 60
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.1343        | 4.21  | 400  | 2.3674          | 1.0372 |
| 0.8075        | 8.42  | 800  | 0.4583          | 0.6308 |
| 0.3209        | 12.63 | 1200 | 0.4291          | 0.5531 |
| 0.2273        | 16.84 | 1600 | 0.4348          | 0.5378 |
| 0.1764        | 21.05 | 2000 | 0.4550          | 0.5326 |
| 0.148         | 25.26 | 2400 | 0.4839          | 0.5319 |
| 0.1268        | 29.47 | 2800 | 0.4515          | 0.5070 |
| 0.1113        | 33.68 | 3200 | 0.4590          | 0.4930 |
| 0.1025        | 37.89 | 3600 | 0.4546          | 0.4888 |
| 0.0922        | 42.11 | 4000 | 0.4782          | 0.4852 |
| 0.082         | 46.32 | 4400 | 0.4605          | 0.4752 |
| 0.0751        | 50.53 | 4800 | 0.4358          | 0.4689 |
| 0.0699        | 54.74 | 5200 | 0.4359          | 0.4629 |
| 0.0633        | 58.95 | 5600 | 0.4467          | 0.4598 |


### Framework versions


model name: wav2vec2-large-xlsr-53-W2V2-TR-MED


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: tr
tags:


- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-tr
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice tr
type: common_voice_8_0
args: tr
metrics:
    - name: Test WER
type: wer
value: 28.69


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# wav2vec2-large-xls-r-300m-tr


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - TR dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2224
- Wer: 0.2869


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 100.0
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 6.8222        | 0.64  | 500   | 3.5026          | 1.0    |
| 3.2136        | 1.28  | 1000  | 3.0593          | 1.0000 |
| 2.8882        | 1.91  | 1500  | 2.4670          | 0.9939 |
| 2.3743        | 2.55  | 2000  | 1.1844          | 0.8657 |
| 1.9456        | 3.19  | 2500  | 0.8228          | 0.7397 |
| 1.7781        | 3.83  | 3000  | 0.6826          | 0.6753 |
| 1.6848        | 4.46  | 3500  | 0.5885          | 0.6140 |
| 1.6228        | 5.1   | 4000  | 0.5274          | 0.5789 |
| 1.5768        | 5.74  | 4500  | 0.4900          | 0.5519 |
| 1.5431        | 6.38  | 5000  | 0.4508          | 0.5238 |
| 1.5019        | 7.02  | 5500  | 0.4248          | 0.5021 |
| 1.4684        | 7.65  | 6000  | 0.4009          | 0.4827 |
| 1.4635        | 8.29  | 6500  | 0.3830          | 0.4700 |
| 1.4291        | 8.93  | 7000  | 0.3707          | 0.4595 |
| 1.4271        | 9.57  | 7500  | 0.3570          | 0.4514 |
| 1.3938        | 10.2  | 8000  | 0.3479          | 0.4378 |
| 1.3914        | 10.84 | 8500  | 0.3396          | 0.4368 |
| 1.3767        | 11.48 | 9000  | 0.3253          | 0.4262 |
| 1.3641        | 12.12 | 9500  | 0.3251          | 0.4178 |
| 1.355         | 12.76 | 10000 | 0.3138          | 0.4136 |
| 1.336         | 13.39 | 10500 | 0.3121          | 0.4069 |
| 1.3292        | 14.03 | 11000 | 0.3041          | 0.4014 |
| 1.3249        | 14.67 | 11500 | 0.3014          | 0.3931 |
| 1.3156        | 15.31 | 12000 | 0.3014          | 0.3929 |
| 1.313         | 15.94 | 12500 | 0.2969          | 0.3968 |
| 1.3068        | 16.58 | 13000 | 0.2965          | 0.3966 |
| 1.2785        | 17.22 | 13500 | 0.2943          | 0.3850 |
| 1.2867        | 17.86 | 14000 | 0.2912          | 0.3782 |
| 1.2714        | 18.49 | 14500 | 0.2819          | 0.3747 |
| 1.2844        | 19.13 | 15000 | 0.2840          | 0.3740 |
| 1.2684        | 19.77 | 15500 | 0.2913          | 0.3828 |
| 1.26          | 20.41 | 16000 | 0.2739          | 0.3674 |
| 1.2543        | 21.05 | 16500 | 0.2740          | 0.3691 |
| 1.2532        | 21.68 | 17000 | 0.2709          | 0.3756 |
| 1.2409        | 22.32 | 17500 | 0.2669          | 0.3593 |
| 1.2404        | 22.96 | 18000 | 0.2673          | 0.3576 |
| 1.2347        | 23.6  | 18500 | 0.2678          | 0.3643 |
| 1.2351        | 24.23 | 19000 | 0.2715          | 0.3650 |
| 1.2409        | 24.87 | 19500 | 0.2637          | 0.3571 |
| 1.2152        | 25.51 | 20000 | 0.2785          | 0.3609 |
| 1.2046        | 26.15 | 20500 | 0.2610          | 0.3508 |
| 1.2082        | 26.79 | 21000 | 0.2619          | 0.3461 |
| 1.2109        | 27.42 | 21500 | 0.2597          | 0.3502 |
| 1.2014        | 28.06 | 22000 | 0.2608          | 0.3468 |
| 1.1948        | 28.7  | 22500 | 0.2573          | 0.3457 |
| 1.205         | 29.34 | 23000 | 0.2619          | 0.3464 |
| 1.2019        | 29.97 | 23500 | 0.2559          | 0.3474 |
| 1.1917        | 30.61 | 24000 | 0.2601          | 0.3462 |
| 1.1939        | 31.25 | 24500 | 0.2575          | 0.3387 |
| 1.1882        | 31.89 | 25000 | 0.2535          | 0.3368 |
| 1.191         | 32.53 | 25500 | 0.2489          | 0.3365 |
| 1.1767        | 33.16 | 26000 | 0.2501          | 0.3347 |
| 1.167         | 33.8  | 26500 | 0.2504          | 0.3347 |
| 1.1678        | 34.44 | 27000 | 0.2480          | 0.3378 |
| 1.1803        | 35.08 | 27500 | 0.2487          | 0.3345 |
| 1.167         | 35.71 | 28000 | 0.2442          | 0.3319 |
| 1.1661        | 36.35 | 28500 | 0.2495          | 0.3334 |
| 1.164         | 36.99 | 29000 | 0.2472          | 0.3292 |
| 1.1578        | 37.63 | 29500 | 0.2442          | 0.3242 |
| 1.1584        | 38.27 | 30000 | 0.2431          | 0.3314 |
| 1.1526        | 38.9  | 30500 | 0.2441          | 0.3347 |
| 1.1542        | 39.54 | 31000 | 0.2437          | 0.3330 |
| 1.1508        | 40.18 | 31500 | 0.2433          | 0.3294 |
| 1.1406        | 40.82 | 32000 | 0.2434          | 0.3271 |
| 1.1514        | 41.45 | 32500 | 0.2426          | 0.3255 |
| 1.1418        | 42.09 | 33000 | 0.2432          | 0.3233 |
| 1.1365        | 42.73 | 33500 | 0.2436          | 0.3240 |
| 1.1348        | 43.37 | 34000 | 0.2483          | 0.3257 |
| 1.1301        | 44.01 | 34500 | 0.2420          | 0.3271 |
| 1.1268        | 44.64 | 35000 | 0.2472          | 0.3225 |
| 1.1224        | 45.28 | 35500 | 0.2382          | 0.3205 |
| 1.1224        | 45.92 | 36000 | 0.2388          | 0.3184 |
| 1.1198        | 46.56 | 36500 | 0.2382          | 0.3202 |
| 1.1274        | 47.19 | 37000 | 0.2404          | 0.3172 |
| 1.1147        | 47.83 | 37500 | 0.2394          | 0.3164 |
| 1.121         | 48.47 | 38000 | 0.2406          | 0.3202 |
| 1.1109        | 49.11 | 38500 | 0.2384          | 0.3154 |
| 1.1164        | 49.74 | 39000 | 0.2375          | 0.3169 |
| 1.1105        | 50.38 | 39500 | 0.2387          | 0.3173 |
| 1.1054        | 51.02 | 40000 | 0.2362          | 0.3120 |
| 1.0893        | 51.66 | 40500 | 0.2399          | 0.3130 |
| 1.0913        | 52.3  | 41000 | 0.2357          | 0.3088 |
| 1.1017        | 52.93 | 41500 | 0.2345          | 0.3084 |
| 1.0937        | 53.57 | 42000 | 0.2330          | 0.3140 |
| 1.0945        | 54.21 | 42500 | 0.2399          | 0.3107 |
| 1.0933        | 54.85 | 43000 | 0.2383          | 0.3134 |
| 1.0912        | 55.48 | 43500 | 0.2372          | 0.3077 |
| 1.0898        | 56.12 | 44000 | 0.2339          | 0.3083 |
| 1.0903        | 56.76 | 44500 | 0.2367          | 0.3065 |
| 1.0947        | 57.4  | 45000 | 0.2352          | 0.3104 |
| 1.0751        | 58.04 | 45500 | 0.2334          | 0.3084 |
| 1.09          | 58.67 | 46000 | 0.2328          | 0.3100 |
| 1.0876        | 59.31 | 46500 | 0.2276          | 0.3050 |
| 1.076         | 59.95 | 47000 | 0.2309          | 0.3047 |
| 1.086         | 60.59 | 47500 | 0.2293          | 0.3047 |
| 1.082         | 61.22 | 48000 | 0.2328          | 0.3027 |
| 1.0714        | 61.86 | 48500 | 0.2290          | 0.3020 |
| 1.0746        | 62.5  | 49000 | 0.2313          | 0.3059 |
| 1.076         | 63.14 | 49500 | 0.2342          | 0.3050 |
| 1.0648        | 63.78 | 50000 | 0.2286          | 0.3025 |
| 1.0586        | 64.41 | 50500 | 0.2338          | 0.3044 |
| 1.0753        | 65.05 | 51000 | 0.2308          | 0.3045 |
| 1.0664        | 65.69 | 51500 | 0.2273          | 0.3009 |
| 1.0739        | 66.33 | 52000 | 0.2298          | 0.3027 |
| 1.0695        | 66.96 | 52500 | 0.2247          | 0.2996 |
| 1.06          | 67.6  | 53000 | 0.2276          | 0.3015 |
| 1.0742        | 68.24 | 53500 | 0.2280          | 0.2974 |
| 1.0618        | 68.88 | 54000 | 0.2291          | 0.2989 |
| 1.062         | 69.52 | 54500 | 0.2302          | 0.2971 |
| 1.0572        | 70.15 | 55000 | 0.2280          | 0.2990 |
| 1.055         | 70.79 | 55500 | 0.2278          | 0.2983 |
| 1.0553        | 71.43 | 56000 | 0.2282          | 0.2991 |
| 1.0509        | 72.07 | 56500 | 0.2261          | 0.2959 |
| 1.0469        | 72.7  | 57000 | 0.2216          | 0.2919 |
| 1.0476        | 73.34 | 57500 | 0.2267          | 0.2989 |
| 1.0494        | 73.98 | 58000 | 0.2260          | 0.2960 |
| 1.0517        | 74.62 | 58500 | 0.2297          | 0.2989 |
| 1.0458        | 75.26 | 59000 | 0.2246          | 0.2923 |
| 1.0382        | 75.89 | 59500 | 0.2255          | 0.2922 |
| 1.0462        | 76.53 | 60000 | 0.2258          | 0.2954 |
| 1.0375        | 77.17 | 60500 | 0.2251          | 0.2929 |
| 1.0332        | 77.81 | 61000 | 0.2277          | 0.2940 |
| 1.0423        | 78.44 | 61500 | 0.2243          | 0.2896 |
| 1.0379        | 79.08 | 62000 | 0.2274          | 0.2928 |
| 1.0398        | 79.72 | 62500 | 0.2237          | 0.2928 |
| 1.0395        | 80.36 | 63000 | 0.2265          | 0.2956 |
| 1.0397        | 80.99 | 63500 | 0.2240          | 0.2920 |
| 1.0262        | 81.63 | 64000 | 0.2244          | 0.2934 |
| 1.0335        | 82.27 | 64500 | 0.2265          | 0.2936 |
| 1.0385        | 82.91 | 65000 | 0.2238          | 0.2928 |
| 1.0289        | 83.55 | 65500 | 0.2219          | 0.2912 |
| 1.0372        | 84.18 | 66000 | 0.2236          | 0.2898 |
| 1.0279        | 84.82 | 66500 | 0.2219          | 0.2902 |
| 1.0325        | 85.46 | 67000 | 0.2240          | 0.2908 |
| 1.0202        | 86.1  | 67500 | 0.2206          | 0.2886 |
| 1.0166        | 86.73 | 68000 | 0.2219          | 0.2886 |
| 1.0259        | 87.37 | 68500 | 0.2235          | 0.2897 |
| 1.0337        | 88.01 | 69000 | 0.2210          | 0.2873 |
| 1.0264        | 88.65 | 69500 | 0.2216          | 0.2882 |
| 1.0231        | 89.29 | 70000 | 0.2223          | 0.2899 |
| 1.0281        | 89.92 | 70500 | 0.2214          | 0.2872 |
| 1.0135        | 90.56 | 71000 | 0.2218          | 0.2868 |
| 1.0291        | 91.2  | 71500 | 0.2209          | 0.2863 |
| 1.0321        | 91.84 | 72000 | 0.2199          | 0.2876 |
| 1.028         | 92.47 | 72500 | 0.2214          | 0.2858 |
| 1.0213        | 93.11 | 73000 | 0.2219          | 0.2875 |
| 1.0261        | 93.75 | 73500 | 0.2232          | 0.2869 |
| 1.0197        | 94.39 | 74000 | 0.2227          | 0.2866 |
| 1.0298        | 95.03 | 74500 | 0.2228          | 0.2868 |
| 1.0192        | 95.66 | 75000 | 0.2230          | 0.2865 |
| 1.0156        | 96.3  | 75500 | 0.2220          | 0.2869 |
| 1.0075        | 96.94 | 76000 | 0.2223          | 0.2866 |
| 1.0201        | 97.58 | 76500 | 0.2219          | 0.2866 |
| 1.0159        | 98.21 | 77000 | 0.2219          | 0.2876 |
| 1.0087        | 98.85 | 77500 | 0.2219          | 0.2873 |
| 1.0159        | 99.49 | 78000 | 0.2223          | 0.2867 |


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu102
- Datasets 1.18.2.dev0
- Tokenizers 0.11.0


model name: wav2vec2-large-xls-r-300m-tr


model author: emre


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, tr, dataset:common_voice, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language:


- ru
tags:
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-Russian-small
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ru
type: common_voice
args: ru
metrics:
    - name: Test WER
type: wer
value: 48.38


# wav2vec2-xls-r-300m-Russian-small


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3514
- Wer: 0.4838


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 5.512         | 1.32  | 400  | 3.2207          | 1.0    |
| 3.1562        | 2.65  | 800  | 3.0166          | 1.0    |
| 1.5211        | 3.97  | 1200 | 0.7134          | 0.8275 |
| 0.6724        | 5.3   | 1600 | 0.4713          | 0.6402 |
| 0.4693        | 6.62  | 2000 | 0.3904          | 0.5668 |
| 0.3693        | 7.95  | 2400 | 0.3609          | 0.5121 |
| 0.3004        | 9.27  | 2800 | 0.3514          | 0.4838 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-Russian-small


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, ru, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


license: apache-2.0
language: ab
tags:


- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-xls-r-300m-ab-CV8
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ab
type: common_voice
args: ab
metrics:
    - name: Test WER
type: wer
value: 54.74


# wav2vec2-xls-r-300m-ab-CV8


- Loss: 0.2105
- Wer: 0.5474


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 15
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 4.7729        | 0.63  | 500   | 3.0624          | 1.0021 |
| 2.7348        | 1.26  | 1000  | 1.0460          | 0.9815 |
| 1.2756        | 1.9   | 1500  | 0.4618          | 0.8309 |
| 1.0419        | 2.53  | 2000  | 0.3725          | 0.7449 |
| 0.9491        | 3.16  | 2500  | 0.3368          | 0.7345 |
| 0.9006        | 3.79  | 3000  | 0.3014          | 0.6936 |
| 0.8519        | 4.42  | 3500  | 0.2852          | 0.6767 |
| 0.8243        | 5.06  | 4000  | 0.2701          | 0.6504 |
| 0.7902        | 5.69  | 4500  | 0.2641          | 0.6221 |
| 0.7767        | 6.32  | 5000  | 0.2549          | 0.6192 |
| 0.7516        | 6.95  | 5500  | 0.2515          | 0.6179 |
| 0.737         | 7.59  | 6000  | 0.2408          | 0.5963 |
| 0.7217        | 8.22  | 6500  | 0.2429          | 0.6261 |
| 0.7101        | 8.85  | 7000  | 0.2366          | 0.5687 |
| 0.6922        | 9.48  | 7500  | 0.2277          | 0.5680 |
| 0.6866        | 10.11 | 8000  | 0.2242          | 0.5847 |
| 0.6703        | 10.75 | 8500  | 0.2222          | 0.5803 |
| 0.6649        | 11.38 | 9000  | 0.2247          | 0.5765 |
| 0.6513        | 12.01 | 9500  | 0.2182          | 0.5644 |
| 0.6369        | 12.64 | 10000 | 0.2128          | 0.5508 |
| 0.6425        | 13.27 | 10500 | 0.2132          | 0.5514 |
| 0.6399        | 13.91 | 11000 | 0.2116          | 0.5495 |
| 0.6208        | 14.54 | 11500 | 0.2105          | 0.5474 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: wav2vec2-xls-r-300m-ab-CV8


model author: emre


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, ab, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, robust-speech-event, model-index


language: Guj
datasets:


- google
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Guj by Jaimin
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Google
type: voice
args: guj
metrics:
    - name: Test WER
type: wer
value: 28.92


# wav2vec2-base-gujarati-demo


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Guj
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

common_voice_train,common_voice_test = load_dataset('csv', data_files={'train': 'train.csv','test': 'test.csv'},error_bad_lines=False,encoding='utf-8',split=['train', 'test']).

processor = Wav2Vec2Processor.from_pretrained("jaimin/wav2vec2-base-gujarati-demo")
model = Wav2Vec2ForCTC.from_pretrained("jaimin/wav2vec2-base-gujarati-demo")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = common_voice_test.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
     logits = model(inputs.input_values).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][0].lower())
```


## Evaluation


The model can be evaluated as follows on the {language} test data of Common Voice.


```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

common_voice_validation = load_dataset('csv', data_files={'test': 'validation.csv'},error_bad_lines=False,encoding='utf-8',split='test')
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("jaimin/wav2vec2-base-gujarati-demo")
model = Wav2Vec2ForCTC.from_pretrained("Amrrs/wav2vec2-base-gujarati-demo")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = common_voice_validation.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = common_voice_validation.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 28.92 %


## Training


The Google datasets were used for training.


The script used for training can be found [here](https://colab.research.google.com/drive/1-Klkgr4f-C9SanHfVC5RhP0ELUH6TYlN?usp=sharing)


model name: wav2vec2-base-gujarati-demo


model author: jaimin


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, Guj, dataset:google, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


tokenizer = AutoTokenizer.from_pretrained("jaimin/Gujarati-Model")
model = AutoModel.from_pretrained("jaimin/Gujarati-Model")


model name: Gujarati-Model


task: feature-extraction


tags: pytorch, roberta, feature-extraction, transformers, infinity_compatible


"hello"


model name: plagiarism_checker


task: text-classification


tags: pytorch, longformer, text-classification, transformers


license: mit
tags:


- generated_from_trainer
datasets:
- xtreme
metrics:
- f1
model-index:
- name: xlm-roberta-base-finetuned-panx-de
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: xtreme
type: xtreme
args: PAN-X.de
metrics:
    - name: F1
type: f1
value: 0.8657141230716685


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# xlm-roberta-base-finetuned-panx-de


This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the xtreme dataset.
It achieves the following results on the evaluation set:


- Loss: 0.1604
- F1: 0.8657


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | F1     |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.2648        | 1.0   | 1573 | 0.1923          | 0.8244 |
| 0.1398        | 2.0   | 3146 | 0.1573          | 0.8543 |
| 0.0851        | 3.0   | 4719 | 0.1604          | 0.8657 |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.1
- Datasets 1.18.1
- Tokenizers 0.10.3


model name: xlm-roberta-base-finetuned-panx-de


model author: robkayinto


task: token-classification


tags: pytorch, tensorboard, xlm-roberta, token-classification, dataset:xtreme, transformers, license:mit, generated_from_trainer, model-index, autonlp_compatible


tags:


- ner
- token-classification
- japanese
- bert


language:


- ja


license: mit


## bert-japanese-ner


## How to use


このモデルはTokenizerに上述の京都大学BERT日本語PretrainedモデルのTokenizerを利用します。\
当リポジトリにTokenizerは含まれていません。\
利用する際は別途ダウンロードしてご用意ください。


また、Tokenizerとは別に[Juman++](https://nlp.ist.i.kyoto-u.ac.jp/?JUMAN%2B%2B)と[pyknp](https://nlp.ist.i.kyoto-u.ac.jp/?PyKNP)を利用します。\
予めインストールしておいてください。


```py
from transformers import (
    BertForTokenClassification, BertTokenizer
)

from pyknp import Juman

jumanpp = Juman()
tokenizer = BertTokenizer.from_pretrained("ダウンロードした京都大学のTokenizerのファイルパス")

model = BertForTokenClassification.from_pretrained("ken11/bert-japanese-ner")

text = "なにか文章"
juman_result = jumanpp.analysis(text)
tokenized_text = [mrph.midasi for mrph in juman_result.mrph_list()]
inputs = tokenizer(tokenized_text, return_tensors="pt", padding='max_length', truncation=True, max_length=64, is_split_into_words=True)
pred = model(**inputs).logits[0]
pred = np.argmax(pred.detach().numpy(), axis=-1)
labels = []
for i, label in enumerate(pred):
    if i + 1 > len(tokenized_text):
        continue
    labels.append(model.config.id2label[label])
    print(f"{tokenized_text[i]}: {model.config.id2label[label]}")
print(tokenized_text)
print(labels)
```


## Training Data


学習には[ストックマーク株式会社が公開しているner-wikipedia-dataset](https://github.com/stockmarkteam/ner-wikipedia-dataset)を利用しました。\
便利なデータセットを公開していただきありがとうございます。


## Note


固有表現抽出のラベルは学習データセットのものをBILUO形式に変換して使用しています。\
ラベルの詳細については[ner-wikipedia-datasetの概要](https://github.com/stockmarkteam/ner-wikipedia-dataset#%E6%A6%82%E8%A6%81)をご確認ください。


## Licenese


[The MIT license](https://opensource.org/licenses/MIT)


model name: bert-japanese-ner


model author: ken11


task: token-classification


tags: pytorch, bert, token-classification, ja, transformers, ner, japanese, license:mit, autonlp_compatible, infinity_compatible


tags:


- translation
- japanese


language:


- ja
- en


license: mit


widget:


- text: "今日もご安全に"


## mbart-ja-en


このモデルは[facebook/mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25)をベースに[JESC dataset](https://nlp.stanford.edu/projects/jesc/index_ja.html)でファインチューニングしたものです。\
This model is based on [facebook/mbart-large-cc25](https://huggingface.co/facebook/mbart-large-cc25) and fine-tuned with [JESC dataset](https://nlp.stanford.edu/projects/jesc/index_ja.html).


## How to use


```py
from transformers import (
    MBartForConditionalGeneration, MBartTokenizer
)

tokenizer = MBartTokenizer.from_pretrained("ken11/mbart-ja-en")
model = MBartForConditionalGeneration.from_pretrained("ken11/mbart-ja-en")

inputs = tokenizer("こんにちは", return_tensors="pt")
translated_tokens = model.generate(**inputs, decoder_start_token_id=tokenizer.lang_code_to_id["en_XX"], early_stopping=True, max_length=48)
pred = tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]
print(pred)
```


## Training Data


I used the [JESC dataset](https://nlp.stanford.edu/projects/jesc/index_ja.html) for training.\
Thank you for publishing such a large dataset.


## Tokenizer


The tokenizer uses the [sentencepiece](https://github.com/google/sentencepiece) trained on the JESC dataset.


## Note


The result of evaluating the sacrebleu score for [JEC Basic Sentence Data of Kyoto University](https://nlp.ist.i.kyoto-u.ac.jp/EN/?JEC+Basic+Sentence+Data#i0163896) was `18.18` .


## Licenese


model name: mbart-ja-en


model author: ken11


task: translation


tags: pytorch, mbart, text2text-generation, ja, en, transformers, translation, japanese, license:mit, autonlp_compatible


tags:


- fill-mask
- japanese
- albert


language:


- ja


license: mit


widget:


- text: "2022年の[MASK]概要"


## albert-base-japanese-v1


日本語事前学習済みALBERTモデルです


## How to use


### ファインチューニング


このモデルはPreTrainedモデルです\
基本的には各種タスク用にファインチューニングして使用されることを想定しています


### Fill-Mask


このモデルではTokenizerにSentencepieceを利用しています\
そのままでは`[MASK]`トークンのあとに[余計なトークンが混入する問題](https://ken11.jp/blog/sentencepiece-tokenizer-bug)があるので、利用する際には以下のようにする必要があります


#### for PyTorch


```py
from transformers import (
    AlbertForMaskedLM, AlbertTokenizerFast
)
import torch

tokenizer = AlbertTokenizerFast.from_pretrained("ken11/albert-base-japanese-v1")
model = AlbertForMaskedLM.from_pretrained("ken11/albert-base-japanese-v1")

text = "大学で[MASK]の研究をしています"
tokenized_text = tokenizer.tokenize(text)
del tokenized_text[tokenized_text.index(tokenizer.mask_token) + 1]

input_ids = [tokenizer.cls_token_id]
input_ids.extend(tokenizer.convert_tokens_to_ids(tokenized_text))
input_ids.append(tokenizer.sep_token_id)

inputs = {"input_ids": [input_ids], "token_type_ids": [[0]*len(input_ids)], "attention_mask": [[1]*len(input_ids)]}
batch = {k: torch.tensor(v, dtype=torch.int64) for k, v in inputs.items()}
output = model(**batch)[0]
_, result = output[0, input_ids.index(tokenizer.mask_token_id)].topk(5)

print(tokenizer.convert_ids_to_tokens(result.tolist()))
# ['英語', '心理学', '数学', '医学', '日本語']
```


#### for TensorFlow


```py
from transformers import (
    TFAlbertForMaskedLM, AlbertTokenizerFast
)
import tensorflow as tf

tokenizer = AlbertTokenizerFast.from_pretrained("ken11/albert-base-japanese-v1")
model = TFAlbertForMaskedLM.from_pretrained("ken11/albert-base-japanese-v1")

text = "大学で[MASK]の研究をしています"
tokenized_text = tokenizer.tokenize(text)
del tokenized_text[tokenized_text.index(tokenizer.mask_token) + 1]

input_ids = [tokenizer.cls_token_id]
input_ids.extend(tokenizer.convert_tokens_to_ids(tokenized_text))
input_ids.append(tokenizer.sep_token_id)

inputs = {"input_ids": [input_ids], "token_type_ids": [[0]*len(input_ids)], "attention_mask": [[1]*len(input_ids)]}
batch = {k: tf.convert_to_tensor(v, dtype=tf.int32) for k, v in inputs.items()}
output = model(**batch)[0]
result = tf.math.top_k(output[0, input_ids.index(tokenizer.mask_token_id)], k=5)

print(tokenizer.convert_ids_to_tokens(result.indices.numpy()))
# ['英語', '心理学', '数学', '医学', '日本語']
```


## Training Data


学習には


を利用しています


## Tokenizer


トークナイザーは[Sentencepiece](https://github.com/google/sentencepiece)を利用しています\
こちらも学習データは同様です


## Licenese


model name: albert-base-japanese-v1


model author: ken11


task: fill-mask


tags: pytorch, tf, albert, fill-mask, ja, transformers, japanese, license:mit, autonlp_compatible, infinity_compatible


tags:


- generated_from_trainer
datasets:
- ade_corpus_v2
model-index:
- name: sagemaker-BioclinicalBERT-ADR
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# sagemaker-BioclinicalBERT-ADR


This model is a fine-tuned version of [emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) on the ade_corpus_v2 dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 1
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 171  | 0.9441          |


### Framework versions


- Transformers 4.12.3
- Pytorch 1.9.1
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: sagemaker-BioclinicalBERT-ADR


model author: anindabitm


task: question-answering


tags: pytorch, bert, question-answering, dataset:ade_corpus_v2, transformers, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- emotion
metrics:
- accuracy
model-index:
- name: sagemaker-distilbert-emotion
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: emotion
type: emotion
args: default
metrics:
    - name: Accuracy
type: accuracy
value: 0.9165


# sagemaker-distilbert-emotion


This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the emotion dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2434
- Accuracy: 0.9165


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.9423        | 1.0   | 500  | 0.2434          | 0.9165   |


### Framework versions


model name: sagemaker-distilbert-emotion


model author: anindabitm


task: text-classification


tags: pytorch, distilbert, text-classification, dataset:emotion, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: bert-base-uncased-8-10-2e-05
results:
  - task:
name: Text Classification
type: text-classification
dataset:
name: glue
type: glue
args: cola
metrics:
    - name: Matthews Correlation
type: matthews_correlation
value: 0.8411910241920598


# bert-base-uncased-8-10-2e-05


This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3003
- Matthews Correlation: 0.8412


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10


### Training results


| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| No log        | 1.0   | 100  | 0.3470          | 0.6290               |
| No log        | 2.0   | 200  | 0.3379          | 0.7349               |
| No log        | 3.0   | 300  | 0.2487          | 0.8069               |
| No log        | 4.0   | 400  | 0.2720          | 0.8069               |
| 0.1403        | 5.0   | 500  | 0.3003          | 0.8412               |
| 0.1403        | 6.0   | 600  | 0.3365          | 0.8069               |
| 0.1403        | 7.0   | 700  | 0.3521          | 0.8069               |
| 0.1403        | 8.0   | 800  | 0.3473          | 0.8069               |
| 0.1403        | 9.0   | 900  | 0.3561          | 0.8069               |
| 0.0064        | 10.0  | 1000 | 0.3538          | 0.8069               |


### Framework versions


- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


model name: bert-base-uncased-8-10-2e-05


model author: daisyxie21


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, dataset:glue, transformers, license:apache-2.0, generated_from_trainer, model-index, infinity_compatible


# A Multi-task learning model with two prediction heads


* One prediction head classifies between keyword sentences vs statements/questions
* Other prediction head corresponds to classifier for statements vs questions


## Scores


##### Spaadia SQuaD Test acc: **0.9891**


##### Quora Keyword Pairs Test acc: **0.98048**


## Datasets:


Quora Keyword Pairs: https://www.kaggle.com/stefanondisponibile/quora-question-keyword-pairs
Spaadia SQuaD pairs: https://www.kaggle.com/shahrukhkhan/questions-vs-statementsclassificationdataset


## Article


[Medium article](https://medium.com/@shahrukhx01/multi-task-learning-with-transformers-part-1-multi-prediction-heads-b7001cf014bf)


## Demo Notebook


[Colab Notebook Multi-task Query classifiers](https://colab.research.google.com/drive/1R7WcLHxDsVvZXPhr5HBgIWa3BlSZKY6p?usp=sharing)


## Clone the model repo


```bash
git clone https://huggingface.co/shahrukhx01/bert-multitask-query-classifiers
```


```python
%cd bert-multitask-query-classifiers/
```


## Load model


```python
from multitask_model import BertForSequenceClassification
from transformers import AutoTokenizer
import torch
model = BertForSequenceClassification.from_pretrained(
        "shahrukhx01/bert-multitask-query-classifiers",
        task_labels_map={"quora_keyword_pairs": 2, "spaadia_squad_pairs": 2},
    )
tokenizer = AutoTokenizer.from_pretrained("shahrukhx01/bert-multitask-query-classifiers")
```


## Run inference on both Tasks