| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.7338        | 0.03  | 1000  | 0.6722          | 0.7236   |
| 0.603         | 0.07  | 2000  | 0.6465          | 0.7399   |
| 0.5605        | 0.1   | 3000  | 0.5801          | 0.7728   |
| 0.55          | 0.14  | 4000  | 0.5994          | 0.7626   |
| 0.529         | 0.17  | 5000  | 0.5720          | 0.7697   |
| 0.5196        | 0.2   | 6000  | 0.5692          | 0.7769   |
| 0.5117        | 0.24  | 7000  | 0.5725          | 0.7785   |
| 0.5044        | 0.27  | 8000  | 0.5532          | 0.7787   |
| 0.5016        | 0.31  | 9000  | 0.5546          | 0.7812   |
| 0.5031        | 0.34  | 10000 | 0.5461          | 0.7870   |
| 0.4949        | 0.37  | 11000 | 0.5725          | 0.7826   |
| 0.4894        | 0.41  | 12000 | 0.5419          | 0.7933   |
| 0.4796        | 0.44  | 13000 | 0.5278          | 0.7914   |
| 0.4795        | 0.48  | 14000 | 0.5193          | 0.7953   |
| 0.4713        | 0.51  | 15000 | 0.5534          | 0.7771   |
| 0.4738        | 0.54  | 16000 | 0.5098          | 0.8039   |
| 0.481         | 0.58  | 17000 | 0.5244          | 0.7958   |
| 0.4634        | 0.61  | 18000 | 0.5215          | 0.7972   |
| 0.465         | 0.65  | 19000 | 0.5129          | 0.7985   |
| 0.4624        | 0.68  | 20000 | 0.5062          | 0.8047   |
| 0.4597        | 0.71  | 21000 | 0.5114          | 0.8029   |
| 0.4571        | 0.75  | 22000 | 0.5070          | 0.8073   |
| 0.4602        | 0.78  | 23000 | 0.5115          | 0.7993   |
| 0.4552        | 0.82  | 24000 | 0.5085          | 0.8052   |
| 0.4538        | 0.85  | 25000 | 0.5118          | 0.7974   |
| 0.4517        | 0.88  | 26000 | 0.5036          | 0.8044   |
| 0.4517        | 0.92  | 27000 | 0.4930          | 0.8062   |
| 0.4413        | 0.95  | 28000 | 0.5307          | 0.7964   |
| 0.4483        | 0.99  | 29000 | 0.5195          | 0.7938   |
| 0.4036        | 1.02  | 30000 | 0.5238          | 0.8029   |
| 0.3724        | 1.05  | 31000 | 0.5125          | 0.8082   |
| 0.3777        | 1.09  | 32000 | 0.5099          | 0.8075   |
| 0.3753        | 1.12  | 33000 | 0.5172          | 0.8053   |
| 0.367         | 1.15  | 34000 | 0.5188          | 0.8053   |
| 0.3819        | 1.19  | 35000 | 0.5218          | 0.8046   |
| 0.363         | 1.22  | 36000 | 0.5202          | 0.7993   |
| 0.3794        | 1.26  | 37000 | 0.5240          | 0.8048   |
| 0.3749        | 1.29  | 38000 | 0.5026          | 0.8054   |
| 0.367         | 1.32  | 39000 | 0.5198          | 0.8075   |
| 0.3759        | 1.36  | 40000 | 0.5298          | 0.7993   |
| 0.3701        | 1.39  | 41000 | 0.5072          | 0.8091   |
| 0.3742        | 1.43  | 42000 | 0.5071          | 0.8098   |
| 0.3706        | 1.46  | 43000 | 0.5317          | 0.8037   |
| 0.3716        | 1.49  | 44000 | 0.5034          | 0.8052   |
| 0.3717        | 1.53  | 45000 | 0.5258          | 0.8012   |
| 0.3714        | 1.56  | 46000 | 0.5195          | 0.8050   |
| 0.3781        | 1.6   | 47000 | 0.5004          | 0.8104   |
| 0.3725        | 1.63  | 48000 | 0.5124          | 0.8113   |
| 0.3624        | 1.66  | 49000 | 0.5040          | 0.8094   |
| 0.3657        | 1.7   | 50000 | 0.4979          | 0.8111   |
| 0.3669        | 1.73  | 51000 | 0.4968          | 0.8100   |
| 0.3636        | 1.77  | 52000 | 0.5075          | 0.8079   |
| 0.36          | 1.8   | 53000 | 0.4985          | 0.8110   |
| 0.3624        | 1.83  | 54000 | 0.5125          | 0.8070   |
| 0.366         | 1.87  | 55000 | 0.4918          | 0.8117   |
| 0.3655        | 1.9   | 56000 | 0.5051          | 0.8109   |
| 0.3609        | 1.94  | 57000 | 0.5083          | 0.8105   |
| 0.3672        | 1.97  | 58000 | 0.5129          | 0.8085   |
| 0.3545        | 2.0   | 59000 | 0.5467          | 0.8109   |
| 0.2938        | 2.04  | 60000 | 0.5635          | 0.8049   |
| 0.29          | 2.07  | 61000 | 0.5781          | 0.8041   |
| 0.2992        | 2.11  | 62000 | 0.5470          | 0.8077   |
| 0.2957        | 2.14  | 63000 | 0.5765          | 0.8073   |
| 0.292         | 2.17  | 64000 | 0.5472          | 0.8106   |
| 0.2893        | 2.21  | 65000 | 0.5590          | 0.8085   |
| 0.2883        | 2.24  | 66000 | 0.5535          | 0.8064   |
| 0.2923        | 2.28  | 67000 | 0.5508          | 0.8095   |
| 0.2868        | 2.31  | 68000 | 0.5679          | 0.8098   |
| 0.2892        | 2.34  | 69000 | 0.5660          | 0.8057   |
| 0.292         | 2.38  | 70000 | 0.5494          | 0.8088   |
| 0.286         | 2.41  | 71000 | 0.5653          | 0.8085   |
| 0.2939        | 2.45  | 72000 | 0.5673          | 0.8070   |
| 0.286         | 2.48  | 73000 | 0.5600          | 0.8092   |
| 0.2844        | 2.51  | 74000 | 0.5508          | 0.8095   |
| 0.2913        | 2.55  | 75000 | 0.5645          | 0.8088   |
| 0.2859        | 2.58  | 76000 | 0.5677          | 0.8095   |
| 0.2892        | 2.62  | 77000 | 0.5598          | 0.8113   |
| 0.2898        | 2.65  | 78000 | 0.5618          | 0.8096   |
| 0.2814        | 2.68  | 79000 | 0.5664          | 0.8103   |
| 0.2917        | 2.72  | 80000 | 0.5484          | 0.8122   |
| 0.2907        | 2.75  | 81000 | 0.5522          | 0.8116   |
| 0.2896        | 2.79  | 82000 | 0.5540          | 0.8093   |
| 0.2907        | 2.82  | 83000 | 0.5469          | 0.8104   |
| 0.2882        | 2.85  | 84000 | 0.5471          | 0.8122   |
| 0.2878        | 2.89  | 85000 | 0.5532          | 0.8108   |
| 0.2858        | 2.92  | 86000 | 0.5511          | 0.8115   |
| 0.288         | 2.96  | 87000 | 0.5491          | 0.8111   |
| 0.2834        | 2.99  | 88000 | 0.5541          | 0.8111   |


### Framework versions


- Transformers 4.12.3
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: convbert-base-turkish-mc4-cased-allnli_tr


model author: emrecan


task: zero-shot-classification


tags: pytorch, convbert, text-classification, tr, dataset:nli_tr, transformers, zero-shot-classification, nli, license:apache-2.0


language:


- tr
tags:
- zero-shot-classification
- nli
- pytorch
pipeline_tag: zero-shot-classification
license: apache-2.0
datasets:
- nli_tr
widget:
- text: "Dolar yükselmeye devam ediyor."
candidate_labels: "ekonomi, siyaset, spor"
- text: "Senaryo çok saçmaydı, beğendim diyemem."
candidate_labels: "olumlu, olumsuz"


model name: convbert-base-turkish-mc4-cased-multinli_tr


model name: bert-base-turkish-cased-snli_tr


tags: pytorch, bert, text-classification, tr, dataset:nli_tr, transformers, zero-shot-classification, nli, license:apache-2.0, infinity_compatible


- tr
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
datasets:
- nli_tr
- emrecan/stsb-mt-turkish
widget:
source_sentence: "Bu çok mutlu bir kişi"
sentences:
  - "Bu mutlu bir köpek"
  - "Bu sevincinden havalara uçan bir insan"
  - "Çok kar yağıyor"


# emrecan/bert-base-turkish-cased-mean-nli-stsb-tr


This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. The model was trained on Turkish machine translated versions of [NLI](https://huggingface.co/datasets/nli_tr) and [STS-b](https://huggingface.co/datasets/emrecan/stsb-mt-turkish) datasets, using example [training scripts](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training) from sentence-transformers GitHub repository.


## Usage (Sentence-Transformers)


Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:


```
pip install -U sentence-transformers
```


Then you can use the model like this:


```python
from sentence_transformers import SentenceTransformer
sentences = ["Bu örnek bir cümle", "Her cümle vektöre çevriliyor"]

model = SentenceTransformer('emrecan/bert-base-turkish-cased-mean-nli-stsb-tr')
embeddings = model.encode(sentences)
print(embeddings)
```


## Usage (HuggingFace Transformers)


Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.


```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ["Bu örnek bir cümle", "Her cümle vektöre çevriliyor"]

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('emrecan/bert-base-turkish-cased-mean-nli-stsb-tr')
model = AutoModel.from_pretrained('emrecan/bert-base-turkish-cased-mean-nli-stsb-tr')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```


## Evaluation Results


Evaluation results on test and development sets are given below:


| Split      | Epoch | cosine_pearson | cosine_spearman | euclidean_pearson | euclidean_spearman | manhattan_pearson | manhattan_spearman | dot_pearson | dot_spearman |
|------------|-------|----------------|-----------------|-------------------|--------------------|-------------------|--------------------|-------------|--------------|
| test       |   -   | 0.834          | 0.830           | 0.820             | 0.819              | 0.819             | 0.818              | 0.799       | 0.789        |
| validation | 1     | 0.850          | 0.848           | 0.831             | 0.835              | 0.83              | 0.83               | 0.80        | 0.806        |
| validation | 2     | 0.857          | 0.857           | 0.844             | 0.848              | 0.844             | 0.848              | 0.813       | 0.810        |
| validation | 3     | 0.860          | 0.859           | 0.846             | 0.851              | 0.846             | 0.850              | 0.825       | 0.822        |
| validation | 4     | 0.859          | 0.860           | 0.846             | 0.851              | 0.846             | 0.851              | 0.825       | 0.823        |


## Training


Training scripts [`training_nli_v2.py`](https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/nli/training_nli_v2.py) and [`training_stsbenchmark_continue_training.py`](https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/sts/training_stsbenchmark_continue_training.py) were used to train the model.


The model was trained with the parameters:


**DataLoader**:


`torch.utils.data.dataloader.DataLoader` of length 360 with parameters:


```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```


**Loss**:


`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`


Parameters of the fit()-Method:


```
{
    "epochs": 4,
    "evaluation_steps": 200,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 144,
    "weight_decay": 0.01
}
```


## Full Model Architecture


```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 75, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```


## Citing & Authors


<!--- Describe where people can find more information -->


model name: bert-base-turkish-cased-mean-nli-stsb-tr


model author: emrecan


task: sentence-similarity


tags: pytorch, bert, feature-extraction, tr, dataset:nli_tr, dataset:emrecan/stsb-mt-turkish, sentence-transformers, license:apache-2.0, sentence-similarity, transformers, infinity_compatible


language:


- tr
tags:
- zero-shot-classification
- nli
- pytorch
pipeline_tag: zero-shot-classification
license: apache-2.0
datasets:
- nli_tr
widget:
- text: "Dolar yükselmeye devam ediyor."
candidate_labels: "ekonomi, siyaset, spor"
- text: "Senaryo çok saçmaydı, beğendim diyemem."
candidate_labels: "olumlu, olumsuz"


model name: bert-base-multilingual-cased-snli_tr


task: zero-shot-classification


tags: pytorch, bert, text-classification, tr, dataset:nli_tr, transformers, zero-shot-classification, nli, license:apache-2.0, infinity_compatible


model name: distilbert-base-turkish-cased-multinli_tr


tags: pytorch, distilbert, text-classification, tr, dataset:nli_tr, transformers, zero-shot-classification, nli, license:apache-2.0, infinity_compatible


thumbnail: https://huggingface.co/front/thumbnails/dialogpt.png


tags:


- conversational


license: mit


# DialoGPT Trained on the Speech of a Game Character


This is an instance of [microsoft/DialoGPT-small](https://huggingface.co/microsoft/DialoGPT-small) trained on a game character, Neku Sakuraba from [The World Ends With You](https://en.wikipedia.org/wiki/The_World_Ends_with_You). The data comes from [a Kaggle game script dataset](https://www.kaggle.com/ruolinzheng/twewy-game-script).


Chat with the model:


```python
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("r3dhummingbird/DialoGPT-small-neku")

model = AutoModelWithLMHead.from_pretrained("r3dhummingbird/DialoGPT-small-neku")

# Let's chat for 4 lines
for step in range(4):
    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
    # print(new_user_input_ids)

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens,
    chat_history_ids = model.generate(
        bot_input_ids, max_length=200,
        pad_token_id=tokenizer.eos_token_id,
        no_repeat_ngram_size=3,
        do_sample=True,
        top_k=100,
        top_p=0.7,
        temperature=0.8
    )

    # pretty print last ouput tokens from bot
    print("NekuBot: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
```


model name: Ricky3


model author: Zane


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational, license:mit


```python
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("Zane/Ricky")

model = AutoModelWithLMHead.from_pretrained("Zane/Ricky")

# Let's chat for 4 lines
for step in range(4):
    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
    # print(new_user_input_ids)

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens,
    chat_history_ids = model.generate(
        bot_input_ids, max_length=200,
        pad_token_id=tokenizer.eos_token_id,
        no_repeat_ngram_size=3,
        do_sample=True,
        top_k=100,
        top_p=0.7,
        temperature=0.8
    )

    # pretty print last ouput tokens from bot
    print("NekuBot: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))
```


model name: Ricky


- fastai


# TODO: Fill this model card with more information about your fastai model!


model name: demo3


model author: espejelomar


tags: fastai


tags:


model name: pruebita23456


model name: demo1


Congratulations on hosting your fastai model on the 🤗Hub!


# Some next steps


1. Fill out this model card with more information ([documentation here](https://huggingface.co/docs/hub/model-repos))!
1. Create a demo in Gradio or Streamlit using the 🤗Spaces ([documentation here](https://huggingface.co/docs/hub/spaces)).
1. Join our fastai community on the Hugging Face Discord!


Greetings fellow fastlearner 🤝!


model name: demo5


- image-classification
- fastai
library_name: fastai
datasets:
- Oxford-IIIT Pet Dataset
- ImageNet


## Pet breeds classification model


Finetuned model on The Oxford-IIIT Pet Dataset. It was introduced in
[this paper](https://www.robots.ox.ac.uk/~vgg/publications/2012/parkhi12a/) and first released in
[this webpage](https://www.robots.ox.ac.uk/~vgg/data/pets/).


The pretrained model was trained on the ImageNet dataset, a dataset that has 100,000+ images across 200 different classes. It was introduced in [this paper](https://image-net.org/static_files/papers/imagenet_cvpr09.pdf) and available [in this webpage](https://image-net.org/download.php)


Disclaimer: The model was fine-tuned after [Chapter 5](https://github.com/fastai/fastbook/blob/master/05_pet_breeds.ipynb) of [Deep Learning for Coders with Fastai and Pytorch: AI Applications Without a PhD (2020)](https://github.com/fastai/fastbook) written by Jeremy Howard and Sylvain Gugger.


## Model description


The model was finetuned using the `cnn_learner` method of the fastai library suing a Resnet 34 backbone pretrained on the ImageNet dataset. The fastai library uses PyTorch for the undelying operations. `cnn_learner` automatically gets a pretrained model from a given architecture with a custom head that is suitable for the target data.


Resnet34 is a 34 layer convolutional neural network. It takes residuals from each layer and uses them in the subsequent connected layers. Advantages of a resnet arquitecture ([Neurohive, 2019](https://neurohive.io/en/popular-networks/resnet/)):


- Are easy to optimize, but the “plain” networks (that simply stack layers) shows higher training error when the depth increases.
- Can easily gain accuracy from greatly increased depth, producing results which are better than previous networks.


Please refer to the original paper '[Deep Residual Learning for Image Recognition](https://arxiv.org/pdf/1512.03385.pdf)' written by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun.


Specifically, the model was obtained:


```
learn = cnn_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(2)
```


## How to use


Download the model this way:


```python
from huggingface_hub import hf_hub_download
from fastai.learner import load_learner

model = load_learner(
    hf_hub_download('espejelomar/fastai-pet-breeds-classification', filename="model.pkl")
    )
```


Then you can use your downloaded fastai model in any way you want. For example, if the input is a PIL Image, with the following code you can obtain the resulting outputs for each class:


```python
_, _, preds = self.model.predict(np.array(inputs))
```


## Training data


The Resnet34 model was pretrained on [ImageNet](https://image-net.org/static_files/papers/imagenet_cvpr09.pdf), a dataset that has 100,000+ images across 200 different classes, and fine-tuned on [The Oxford-IIIT Pet Dataset](https://www.robots.ox.ac.uk/~vgg/data/pets/).


## Preprocessing


For more detailed information on the preprocessing procedure, refer to the [Chapter 5](https://github.com/fastai/fastbook/blob/master/05_pet_breeds.ipynb) of [Deep Learning for Coders with Fastai and Pytorch: AI Applications Without a PhD (2020)](https://github.com/fastai/fastbook).


Two main strategies are followed to presizing the images:


- Resize images to relatively "large" dimensions—that is, dimensions significantly larger than the target training dimensions.
- Compose all of the common augmentation operations (including a resize to the final target size) into one, and perform the combined operation on the GPU only once at the end of processing, rather than performing the operations individually and interpolating multiple times.


"The first step, the resize, creates images large enough that they have spare margin to allow further augmentation transforms on their inner regions without creating empty zones. This transformation works by resizing to a square, using a large crop size. On the training set, the crop area is chosen randomly, and the size of the crop is selected to cover the entire width or height of the image, whichever is smaller.


In the second step, the GPU is used for all data augmentation, and all of the potentially destructive operations are done together, with a single interpolation at the end." ([Howard and Gugger, 2020](https://github.com/fastai/fastbook))


Specifically, the following code is used for preprocessing:


```python
#hide_input
#id interpolations
#caption A comparison of fastai's data augmentation strategy (left) and the traditional approach (right).
dblock1 = DataBlock(blocks=(ImageBlock(), CategoryBlock()),
                   get_y=parent_label,
                   item_tfms=Resize(460))
# Place an image in the 'images/grizzly.jpg' subfolder where this notebook is located before running this
dls1 = dblock1.dataloaders([(Path.cwd()/'images'/'grizzly.jpg')]*100, bs=8)
dls1.train.get_idxs = lambda: Inf.ones
x,y = dls1.valid.one_batch()
_,axs = subplots(1, 2)

x1 = TensorImage(x.clone())
x1 = x1.affine_coord(sz=224)
x1 = x1.rotate(draw=30, p=1.)
x1 = x1.zoom(draw=1.2, p=1.)
x1 = x1.warp(draw_x=-0.2, draw_y=0.2, p=1.)

tfms = setup_aug_tfms([Rotate(draw=30, p=1, size=224), Zoom(draw=1.2, p=1., size=224),
                       Warp(draw_x=-0.2, draw_y=0.2, p=1., size=224)])
x = Pipeline(tfms)(x)
#x.affine_coord(coord_tfm=coord_tfm, sz=size, mode=mode, pad_mode=pad_mode)
TensorImage(x[0]).show(ctx=axs[0])
TensorImage(x1[0]).show(ctx=axs[1]);
```


### BibTeX entry and citation info


```bibtex
@book{howard2020deep,
  author    = {Howard, J. and Gugger, S.},
  title     = {Deep Learning for Coders with Fastai and Pytorch: AI Applications Without a PhD},
  isbn      = {9781492045526},
  year      = {2020},
  url       = {https://books.google.no/books?id=xd6LxgEACAAJ},
  publisher = {O'Reilly Media, Incorporated},
}
```


model name: fastai-pet-breeds-classification


model author: espejelomar


task: image-classification


tags: dataset:Oxford-IIIT Pet Dataset, dataset:ImageNet, arxiv:1512.03385, fastai, image-classification


RoBERTa model trained on 1M SMILES from PubChem 77M set in MoleculeNet. Uses Smiles-Tokenizer


model name: SmilesTokenizer_PubChem_1M


model author: DeepChem


task: feature-extraction


tags: pytorch, roberta, feature-extraction, transformers, infinity_compatible


tags:


- conversational


# Cartman DialoGPT Model


model name: CartmenBot


model author: CodeDanCode


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


# SouthPark Kyle Bot


model name: SP-KyleBot


## language: de


## xlm-roberta-large-finetuned-conll03-german


model name: xlm-roberta-large-finetuned-conll03-german


task: token-classification


tags: pytorch, rust, xlm-roberta, token-classification, de, transformers, autonlp_compatible


- generated_from_trainer
datasets:
- wikiann
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: indic-transformers-te-distilbert
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: wikiann
type: wikiann
args: te
metrics:
    - name: Precision
type: precision
value: 0.5657225853304285
    - name: Recall
type: recall
value: 0.6486261448792673
    - name: F1
type: f1
value: 0.604344453064391
    - name: Accuracy
type: accuracy
value: 0.9049186160277506


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# indic-transformers-te-distilbert


This model was trained from scratch on the wikiann dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2940
- Precision: 0.5657
- Recall: 0.6486
- F1: 0.6043
- Accuracy: 0.9049


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log        | 1.0   | 125  | 0.3629          | 0.4855    | 0.5287 | 0.5062 | 0.8826   |
| No log        | 2.0   | 250  | 0.3032          | 0.5446    | 0.6303 | 0.5843 | 0.9002   |
| No log        | 3.0   | 375  | 0.2940          | 0.5657    | 0.6486 | 0.6043 | 0.9049   |


### Framework versions


- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: indic-transformers-te-distilbert


model author: durgaamma2005


task: token-classification


tags: pytorch, tensorboard, distilbert, token-classification, dataset:wikiann, transformers, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


language: en
thumbnail:
license: mit
tags:


- question-answering
- mobilebert
datasets:
- squad
metrics:
- squad
widget:
- text: "Which name is also used to describe the Amazon rainforest in English?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
- text: "How many square kilometers of rainforest is covered in the basin?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."


## MobileBERT fine-tuned on SQuAD v1


[MobileBERT](https://arxiv.org/abs/2004.02984) is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance
between self-attentions and feed-forward networks.


This model was fine-tuned from the HuggingFace checkpoint `google/mobilebert-uncased` on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer).


## Details


| Dataset  | Split | # samples |
| -------- | ----- | --------- |
| SQuAD1.1 | train | 90.6K      |
| SQuAD1.1 | eval  | 11.1k     |


### Fine-tuning


- Python: `3.7.5`
- Machine specs:

  `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz`

  `Memory: 32 GiB`

  `GPUs: 2 GeForce GTX 1070, each with 8GiB memory`

  `GPU driver: 418.87.01, CUDA: 10.1`
- script:

  ```shell
  # after install https://github.com/huggingface/transformers

  cd examples/question-answering
  mkdir -p data

  wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json

  wget -O data/dev-v1.1.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json

  export SQUAD_DIR=`pwd`/data

  python run_squad.py \
    --model_type mobilebert \
    --model_name_or_path google/mobilebert-uncased \
    --do_train \
    --do_eval \
    --do_lower_case \
    --train_file $SQUAD_DIR/train-v1.1.json \
    --predict_file $SQUAD_DIR/dev-v1.1.json \
    --per_gpu_train_batch_size 16 \
    --per_gpu_eval_batch_size 16 \
    --learning_rate 4e-5 \
    --num_train_epochs 5.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --warmup_steps 1400 \
    --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v1 2>&1 | tee train-mobilebert-warmup-squad_v1.log
  ```


It took about 3 hours to finish.


### Results


**Model size**: `95M`


| Metric | # Value   | # Original ([Table 5](https://arxiv.org/pdf/2004.02984.pdf))|
| ------ | --------- | --------- |
| **EM** | **82.6** | **82.9** |
| **F1** | **90.0** | **90.0** |


Note that the above results didn't involve any hyperparameter search.


## Example Usage


```python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/mobilebert-uncased-squad-v1",
    tokenizer="csarron/mobilebert-uncased-squad-v1"
)

predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})

print(predictions)
# output:
# {'score': 0.7754058241844177, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}
```


> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp)


> Made with ❤ in New York.


model name: mobilebert-uncased-squad-v1


model author: csarron


task: question-answering


tags: pytorch, mobilebert, question-answering, en, dataset:squad, arxiv:2004.02984, transformers, license:mit, autonlp_compatible


- question-answering
- mobilebert
datasets:
- squad_v2
metrics:
- squad_v2
widget:
- text: "Which name is also used to describe the Amazon rainforest in English?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
- text: "How many square kilometers of rainforest is covered in the basin?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."


## MobileBERT fine-tuned on SQuAD v2


This model was fine-tuned from the HuggingFace checkpoint `google/mobilebert-uncased` on [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer).


## Details


| Dataset  | Split | # samples |
| -------- | ----- | --------- |
| SQuAD2.0 | train | 130k      |
| SQuAD2.0 | eval  | 12.3k     |


### Fine-tuning


- Python: `3.7.5`
- Machine specs:

  `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz`

  `Memory: 32 GiB`

  `GPUs: 2 GeForce GTX 1070, each with 8GiB memory`

  `GPU driver: 418.87.01, CUDA: 10.1`
- script:

  ```shell
  # after install https://github.com/huggingface/transformers

  cd examples/question-answering
  mkdir -p data

  wget -O data/train-v2.0.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json

  wget -O data/dev-v2.0.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json

  export SQUAD_DIR=`pwd`/data

  python run_squad.py \
    --model_type mobilebert \
    --model_name_or_path google/mobilebert-uncased \
    --do_train \
    --do_eval \
    --do_lower_case \
    --version_2_with_negative \
    --train_file $SQUAD_DIR/train-v2.0.json \
    --predict_file $SQUAD_DIR/dev-v2.0.json \
    --per_gpu_train_batch_size 16 \
    --per_gpu_eval_batch_size 16 \
    --learning_rate 4e-5 \
    --num_train_epochs 5.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --warmup_steps 1400 \
    --save_steps 2000 \
    --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v2 2>&1 | tee train-mobilebert-warmup-squad_v2.log
  ```


It took about 3.5 hours to finish.


### Results


**Model size**: `95M`


| Metric | # Value   | # Original ([Table 5](https://arxiv.org/pdf/2004.02984.pdf))|
| ------ | --------- | --------- |
| **EM** | **75.2** | **76.2** |
| **F1** | **78.8** | **79.2** |


## Example Usage


```python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/mobilebert-uncased-squad-v2",
    tokenizer="csarron/mobilebert-uncased-squad-v2"
)

predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})

print(predictions)
# output:
# {'score': 0.71434086561203, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}
```


> Made with ❤ in New York.


model name: mobilebert-uncased-squad-v2


model author: csarron


task: question-answering


tags: pytorch, mobilebert, question-answering, en, dataset:squad_v2, arxiv:2004.02984, transformers, license:mit, autonlp_compatible


language: en
thumbnail:
license: mit
tags:


- question-answering
- bert
- bert-base
datasets:
- squad
metrics:
- squad
widget:
- text: "Which name is also used to describe the Amazon rainforest in English?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
- text: "How many square kilometers of rainforest is covered in the basin?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."


## BERT-base uncased model fine-tuned on SQuAD v1


This model was fine-tuned from the HuggingFace [BERT](https://www.aclweb.org/anthology/N19-1423/) base uncased checkpoint on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer).
This model is case-insensitive: it does not make a difference between english and English.


## Details


| Dataset  | Split | # samples |
| -------- | ----- | --------- |
| SQuAD1.1 | train | 90.6K      |
| SQuAD1.1 | eval  | 11.1k     |


### Fine-tuning


- Python: `3.7.5`
- Machine specs:

  `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz`

  `Memory: 32 GiB`

  `GPUs: 2 GeForce GTX 1070, each with 8GiB memory`

  `GPU driver: 418.87.01, CUDA: 10.1`
- script:

  ```shell
  # after install https://github.com/huggingface/transformers

  cd examples/question-answering
  mkdir -p data

  wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json

  wget -O data/dev-v1.1.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json

  python run_squad.py \
    --model_type bert \
    --model_name_or_path bert-base-uncased \
    --do_train \
    --do_eval \
    --do_lower_case \
    --train_file train-v1.1.json \
    --predict_file dev-v1.1.json \
    --per_gpu_train_batch_size 12 \
    --per_gpu_eval_batch_size=16 \
    --learning_rate 3e-5 \
    --num_train_epochs 2.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --data_dir data \
    --output_dir data/bert-base-uncased-squad-v1 2>&1 | tee train-energy-bert-base-squad-v1.log
  ```


It took about 2 hours to finish.


### Results


**Model size**: `418M`


| Metric | # Value   | # Original ([Table 2](https://www.aclweb.org/anthology/N19-1423.pdf))|
| ------ | --------- | --------- |
| **EM** | **80.9** | **80.8** |
| **F1** | **88.2** | **88.5** |


Note that the above results didn't involve any hyperparameter search.


## Example Usage


```python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/bert-base-uncased-squad-v1",
    tokenizer="csarron/bert-base-uncased-squad-v1"
)

predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})

print(predictions)
# output:
# {'score': 0.8730505704879761, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}
```


> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp)


> Made with ❤ in New York.


model name: bert-base-uncased-squad-v1


model author: csarron


task: question-answering


tags: pytorch, jax, bert, question-answering, en, dataset:squad, transformers, license:mit, bert-base, autonlp_compatible, infinity_compatible


- question-answering
- roberta
- roberta-base
datasets:
- squad
metrics:
- squad
widget:
- text: "Which name is also used to describe the Amazon rainforest in English?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."
- text: "How many square kilometers of rainforest is covered in the basin?"
context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."


## RoBERTa-base fine-tuned on SQuAD v1


This model was fine-tuned from the HuggingFace [RoBERTa](https://arxiv.org/abs/1907.11692) base checkpoint on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer).
This model is case-sensitive: it makes a difference between english and English.


## Details


| Dataset  | Split | # samples |
| -------- | ----- | --------- |
| SQuAD1.1 | train | 96.8K      |
| SQuAD1.1 | eval  | 11.8k     |


### Fine-tuning


- Python: `3.7.5`
- Machine specs:

  `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz`

  `Memory: 32 GiB`

  `GPUs: 2 GeForce GTX 1070, each with 8GiB memory`

  `GPU driver: 418.87.01, CUDA: 10.1`
- script:

  ```shell
  # after install https://github.com/huggingface/transformers

  cd examples/question-answering
  mkdir -p data

  wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json

  wget -O data/dev-v1.1.json  https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json

  python run_energy_squad.py \
    --model_type roberta \
    --model_name_or_path roberta-base \
    --do_train \
    --do_eval \
    --train_file train-v1.1.json \
    --predict_file dev-v1.1.json \
    --per_gpu_train_batch_size 12 \
    --per_gpu_eval_batch_size 16 \
    --learning_rate 3e-5 \
    --num_train_epochs 2.0 \
    --max_seq_length 320 \
    --doc_stride 128 \
    --data_dir data \
    --output_dir data/roberta-base-squad-v1 2>&1 | tee train-roberta-base-squad-v1.log
  ```


It took about 2 hours to finish.


### Results


**Model size**: `477M`


| Metric | # Value   |
| ------ | --------- |
| **EM** | **83.0** |
| **F1** | **90.4** |


## Example Usage


```python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="csarron/roberta-base-squad-v1",
    tokenizer="csarron/roberta-base-squad-v1"
)

predictions = qa_pipeline({
    'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.",
    'question': "What day was the game played on?"
})

print(predictions)
# output:
# {'score': 0.8625259399414062, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'}
```


> Made with ❤ in New York.


model name: roberta-base-squad-v1


model author: csarron


task: question-answering


tags: pytorch, jax, roberta, question-answering, en, dataset:squad, arxiv:1907.11692, transformers, license:mit, roberta-base, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: bert-finetuned-ner
results:
  - task:
name: Token Classification
type: token-classification
dataset:
name: conll2003
type: conll2003
args: conll2003
metrics:
    - name: Precision
type: precision
value: 0.9328604420983174
    - name: Recall
type: recall
value: 0.9516997643890945
    - name: F1
type: f1
value: 0.9421859380206598
    - name: Accuracy
type: accuracy
value: 0.986342497203744


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# bert-finetuned-ner


This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the conll2003 dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0612
- Precision: 0.9329
- Recall: 0.9517
- F1: 0.9422
- Accuracy: 0.9863


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0904        | 1.0   | 1756 | 0.0686          | 0.9227    | 0.9355 | 0.9291 | 0.9820   |
| 0.0385        | 2.0   | 3512 | 0.0586          | 0.9381    | 0.9490 | 0.9435 | 0.9862   |
| 0.0215        | 3.0   | 5268 | 0.0612          | 0.9329    | 0.9517 | 0.9422 | 0.9863   |


### Framework versions


- Transformers 4.13.0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: bert-finetuned-ner


model author: Buntan


task: token-classification


tags: pytorch, tensorboard, bert, token-classification, dataset:conll2003, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


tags:


- flair
- token-classification
- sequence-tagger-model
language: fr
widget:
- text: "George Washington est allé à Washington"


# POET: A French Extended Part-of-Speech Tagger


- Corpora: [ANTILLES](https://github.com/qanastek/ANTILLES)
- Embeddings: [FastText](https://fasttext.cc/)
- Sequence Labelling: [Bi-LSTM-CRF](https://arxiv.org/abs/1011.4088)
- Number of Epochs: 115


**People Involved**


* [LABRAK Yanis](https://www.linkedin.com/in/yanis-labrak-8a7412145/) (1)
* [DUFOUR Richard](https://cv.archives-ouvertes.fr/richard-dufour) (2)


**Affiliations**


1. [LIA, NLP team](https://lia.univ-avignon.fr/), Avignon University, Avignon, France.
1. [LS2N, TALN team](https://www.ls2n.fr/equipe/taln/), Nantes University, Nantes, France.


## Demo: How to use in Flair


Requires [Flair](https://pypi.org/project/flair/): `pip install flair`


```python
from flair.data import Sentence
from flair.models import SequenceTagger

# Load the model
model = SequenceTagger.load("qanastek/pos-french")

sentence = Sentence("George Washington est allé à Washington")

# predict tags
model.predict(sentence)

# print predicted pos tags
print(sentence.to_tagged_string())
```


Output:


![Preview Output](preview.PNG)


## Training data


`ANTILLES` is a part-of-speech tagging corpora based on [UD_French-GSD](https://universaldependencies.org/treebanks/fr_gsd/index.html) which was originally created in 2015 and is based on the [universal dependency treebank v2.0](https://github.com/ryanmcd/uni-dep-tb).


Originally, the corpora consists of 400,399 words (16,341 sentences) and had 17 different classes. Now, after applying our tags augmentation we obtain 60 different classes which add linguistic and semantic information such as the gender, number, mood, person, tense or verb form given in the different CoNLL-03 fields from the original corpora.


We based our tags on the level of details given by the [LIA_TAGG](http://pageperso.lif.univ-mrs.fr/frederic.bechet/download.html) statistical POS tagger written by [Frédéric Béchet](http://pageperso.lif.univ-mrs.fr/frederic.bechet/index-english.html) in 2001.


The corpora used for this model is available on [Github](https://github.com/qanastek/ANTILLES) at the [CoNLL-U format](https://universaldependencies.org/format.html).


Training data are fed to the model as free language and doesn't pass a normalization phase. Thus, it's made the model case and punctuation sensitive.


## Original Tags


```plain
PRON VERB SCONJ ADP CCONJ DET NOUN ADJ AUX ADV PUNCT PROPN NUM SYM PART X INTJ
```


## New additional POS tags


| Abbreviation | Description | Examples |
|:--------:|:--------:|:--------:|
| PREP | Preposition | de |
| AUX | Auxiliary Verb | est |
| ADV | Adverb | toujours |
| COSUB | Subordinating conjunction | que |
| COCO | Coordinating Conjunction | et |
| PART | Demonstrative particle | -t |
| PRON | Pronoun | qui ce quoi |
| PDEMMS | Demonstrative Pronoun - Singular Masculine | ce |
| PDEMMP | Demonstrative Pronoun - Plural Masculine | ceux |
| PDEMFS | Demonstrative Pronoun - Singular Feminine | cette |
| PDEMFP | Demonstrative Pronoun - Plural Feminine | celles |
| PINDMS | Indefinite Pronoun - Singular Masculine | tout |
| PINDMP | Indefinite Pronoun - Plural Masculine | autres |
| PINDFS | Indefinite Pronoun - Singular Feminine | chacune |
| PINDFP | Indefinite Pronoun - Plural Feminine | certaines |
| PROPN | Proper noun | Houston |
| XFAMIL | Last name | Levy |
| NUM | Numerical Adjective | trentaine vingtaine |
| DINTMS | Masculine Numerical Adjective | un |
| DINTFS | Feminine Numerical Adjective | une |
| PPOBJMS | Pronoun complements of objects - Singular Masculine | le lui |
| PPOBJMP | Pronoun complements of objects - Plural Masculine | eux y |
| PPOBJFS | Pronoun complements of objects - Singular Feminine | moi la |
| PPOBJFP | Pronoun complements of objects - Plural Feminine | en y |
| PPER1S | Personal Pronoun First-Person - Singular | je |
| PPER2S | Personal Pronoun Second-Person - Singular | tu |
| PPER3MS | Personal Pronoun Third-Person - Singular Masculine | il |
| PPER3MP | Personal Pronoun Third-Person - Plural Masculine | ils |
| PPER3FS | Personal Pronoun Third-Person - Singular Feminine | elle |
| PPER3FP | Personal Pronoun Third-Person - Plural Feminine | elles |
| PREFS | Reflexive Pronoun First-Person - Singular | me m' |
| PREF | Reflexive Pronoun Third-Person - Singular | se s' |
| PREFP | Reflexive Pronoun First / Second-Person - Plural | nous vous |
| VERB | Verb | obtient |
| VPPMS | Past Participle - Singular Masculine | formulé |
| VPPMP | Past Participle - Plural Masculine | classés |
| VPPFS | Past Participle - Singular Feminine | appelée |
| VPPFP | Past Participle - Plural Feminine | sanctionnées |
| DET | Determinant | les l' |
| DETMS | Determinant - Singular Masculine | les |
| DETFS | Determinant - Singular Feminine | la |
| ADJ | Adjective | capable sérieux |
| ADJMS | Adjective - Singular Masculine | grand important |
| ADJMP | Adjective - Plural Masculine | grands petits |
| ADJFS | Adjective - Singular Feminine | française petite |
| ADJFP | Adjective - Plural Feminine | légères petites |
| NOUN | Noun | temps |
| NMS | Noun - Singular Masculine | drapeau |
| NMP | Noun - Plural Masculine | journalistes |
| NFS | Noun - Singular Feminine | tête |
| NFP | Noun - Plural Feminine | ondes |
| PREL | Relative Pronoun | qui dont |
| PRELMS | Relative Pronoun - Singular Masculine | lequel |
| PRELMP | Relative Pronoun - Plural Masculine | lesquels |
| PRELFS | Relative Pronoun - Singular Feminine | laquelle |
| PRELFP | Relative Pronoun - Plural Feminine | lesquelles |
| INTJ | Interjection | merci bref |
| CHIF | Numbers | 1979 10 |
| SYM | Symbol | € % |
| YPFOR | Endpoint | . |
| PUNCT | Ponctuation | : , |
| MOTINC | Unknown words | Technology Lady |
| X | Typos & others | sfeir 3D statu |


## Evaluation results


The test corpora used for this evaluation is available on [Github](https://github.com/qanastek/ANTILLES/blob/main/ANTILLES/test.conllu).


```plain
Results:
- F-score (micro): 0.952
- F-score (macro): 0.8644
- Accuracy (incl. no class): 0.952

By class:
              precision    recall  f1-score   support
      PPER1S     0.9767    1.0000    0.9882        42
        VERB     0.9823    0.9537    0.9678       583
       COSUB     0.9344    0.8906    0.9120       128
       PUNCT     0.9878    0.9688    0.9782       833
        PREP     0.9767    0.9879    0.9822      1483
      PDEMMS     0.9583    0.9200    0.9388        75
        COCO     0.9839    1.0000    0.9919       245
         DET     0.9679    0.9814    0.9746       645
         NMP     0.9521    0.9115    0.9313       305
       ADJMP     0.8352    0.9268    0.8786        82
        PREL     0.9324    0.9857    0.9583        70
       PREFP     0.9767    0.9545    0.9655        44
         AUX     0.9537    0.9859    0.9695       355
         ADV     0.9440    0.9365    0.9402       504
       VPPMP     0.8667    1.0000    0.9286        26
      DINTMS     0.9919    1.0000    0.9959       122
       ADJMS     0.9020    0.9057    0.9039       244
         NMS     0.9226    0.9336    0.9281       753
         NFS     0.9347    0.9714    0.9527       560
       YPFOR     0.9806    1.0000    0.9902       353
      PINDMS     1.0000    0.9091    0.9524        44
        NOUN     0.8400    0.5385    0.6562        39
       PROPN     0.8605    0.8278    0.8439       395
       DETMS     0.9972    0.9972    0.9972       362
     PPER3MS     0.9341    0.9770    0.9551        87
       VPPMS     0.8994    0.9682    0.9325       157
       DETFS     1.0000    1.0000    1.0000       240
       ADJFS     0.9266    0.9011    0.9136       182
       ADJFP     0.9726    0.9342    0.9530        76
         NFP     0.9463    0.9749    0.9604       199
       VPPFS     0.8000    0.9000    0.8471        40
        CHIF     0.9543    0.9414    0.9478       222
      XFAMIL     0.9346    0.8696    0.9009       115
     PPER3MP     0.9474    0.9000    0.9231        20
     PPOBJMS     0.8800    0.9362    0.9072        47
        PREF     0.8889    0.9231    0.9057        52
     PPOBJMP     1.0000    0.6000    0.7500        10
         SYM     0.9706    0.8684    0.9167        38
      DINTFS     0.9683    1.0000    0.9839        61
      PDEMFS     1.0000    0.8966    0.9455        29
     PPER3FS     1.0000    0.9444    0.9714        18
       VPPFP     0.9500    1.0000    0.9744        19
        PRON     0.9200    0.7419    0.8214        31
     PPOBJFS     0.8333    0.8333    0.8333         6
        PART     0.8000    1.0000    0.8889         4
     PPER3FP     1.0000    1.0000    1.0000         2
      MOTINC     0.3571    0.3333    0.3448        15
      PDEMMP     1.0000    0.6667    0.8000         3
        INTJ     0.4000    0.6667    0.5000         6
       PREFS     1.0000    0.5000    0.6667        10
         ADJ     0.7917    0.8636    0.8261        22
      PINDMP     0.0000    0.0000    0.0000         1
      PINDFS     1.0000    1.0000    1.0000         1
         NUM     1.0000    0.3333    0.5000         3
      PPER2S     1.0000    1.0000    1.0000         2
     PPOBJFP     1.0000    0.5000    0.6667         2
      PDEMFP     1.0000    0.6667    0.8000         3
           X     0.0000    0.0000    0.0000         1
      PRELMS     1.0000    1.0000    1.0000         2
      PINDFP     1.0000    1.0000    1.0000         1

    accuracy                         0.9520     10019
   macro avg     0.8956    0.8521    0.8644     10019
weighted avg     0.9524    0.9520    0.9515     10019
```


## BibTeX Citations


Please cite the following paper when using this model.


UD_French-GSD corpora:


```latex
@misc{
    universaldependencies,
    title={UniversalDependencies/UD_French-GSD},
    url={https://github.com/UniversalDependencies/UD_French-GSD}, journal={GitHub},
    author={UniversalDependencies}
}
```


LIA TAGG:


```latex
@techreport{LIA_TAGG,
  author = {Frédéric Béchet},
  title = {LIA_TAGG: a statistical POS tagger + syntactic bracketer},
  institution = {Aix-Marseille University & CNRS},
  year = {2001}
}
```


Flair Embeddings:


```latex
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```


## Acknowledgment


This work was financially supported by [Zenidoc](https://zenidoc.fr/)


model name: pos-french


model author: qanastek


task: token-classification


tags: pytorch, fr, flair, token-classification, sequence-tagger-model


tags:


- flair
- token-classification
- sequence-tagger-model
language: fr
widget:
- text: "George Washington est allé à Washington"


# POET: A French Extended Part-of-Speech Tagger


- Corpora: [ANTILLES](https://github.com/qanastek/ANTILLES)
- Embeddings: [FastText](https://fasttext.cc/)
- Sequence Labelling: [Bi-LSTM-CRF](https://arxiv.org/abs/1011.4088)
- Number of Epochs: 115


**People Involved**


* [LABRAK Yanis](https://www.linkedin.com/in/yanis-labrak-8a7412145/) (1)
* [DUFOUR Richard](https://cv.archives-ouvertes.fr/richard-dufour) (2)


**Affiliations**


1. [LIA, NLP team](https://lia.univ-avignon.fr/), Avignon University, Avignon, France.
1. [LS2N, TALN team](https://www.ls2n.fr/equipe/taln/), Nantes University, Nantes, France.


## Demo: How to use in Flair


Requires [Flair](https://pypi.org/project/flair/): `pip install flair`


```python
from flair.data import Sentence
from flair.models import SequenceTagger

# Load the model
model = SequenceTagger.load("qanastek/pos-french")

sentence = Sentence("George Washington est allé à Washington")

# predict tags
model.predict(sentence)

# print predicted pos tags
print(sentence.to_tagged_string())
```


Output:


![Preview Output](preview.PNG)


## Training data


`ANTILLES` is a part-of-speech tagging corpora based on [UD_French-GSD](https://universaldependencies.org/treebanks/fr_gsd/index.html) which was originally created in 2015 and is based on the [universal dependency treebank v2.0](https://github.com/ryanmcd/uni-dep-tb).


Originally, the corpora consists of 400,399 words (16,341 sentences) and had 17 different classes. Now, after applying our tags augmentation we obtain 60 different classes which add linguistic and semantic information such as the gender, number, mood, person, tense or verb form given in the different CoNLL-03 fields from the original corpora.


We based our tags on the level of details given by the [LIA_TAGG](http://pageperso.lif.univ-mrs.fr/frederic.bechet/download.html) statistical POS tagger written by [Frédéric Béchet](http://pageperso.lif.univ-mrs.fr/frederic.bechet/index-english.html) in 2001.


The corpora used for this model is available on [Github](https://github.com/qanastek/ANTILLES) at the [CoNLL-U format](https://universaldependencies.org/format.html).


Training data are fed to the model as free language and doesn't pass a normalization phase. Thus, it's made the model case and punctuation sensitive.


## Original Tags


```plain
PRON VERB SCONJ ADP CCONJ DET NOUN ADJ AUX ADV PUNCT PROPN NUM SYM PART X INTJ
```


## New additional POS tags


| Abbreviation | Description | Examples |
|:--------:|:--------:|:--------:|
| PREP | Preposition | de |
| AUX | Auxiliary Verb | est |
| ADV | Adverb | toujours |
| COSUB | Subordinating conjunction | que |
| COCO | Coordinating Conjunction | et |
| PART | Demonstrative particle | -t |
| PRON | Pronoun | qui ce quoi |
| PDEMMS | Demonstrative Pronoun - Singular Masculine | ce |
| PDEMMP | Demonstrative Pronoun - Plural Masculine | ceux |
| PDEMFS | Demonstrative Pronoun - Singular Feminine | cette |
| PDEMFP | Demonstrative Pronoun - Plural Feminine | celles |
| PINDMS | Indefinite Pronoun - Singular Masculine | tout |
| PINDMP | Indefinite Pronoun - Plural Masculine | autres |
| PINDFS | Indefinite Pronoun - Singular Feminine | chacune |
| PINDFP | Indefinite Pronoun - Plural Feminine | certaines |
| PROPN | Proper noun | Houston |
| XFAMIL | Last name | Levy |
| NUM | Numerical Adjective | trentaine vingtaine |
| DINTMS | Masculine Numerical Adjective | un |
| DINTFS | Feminine Numerical Adjective | une |
| PPOBJMS | Pronoun complements of objects - Singular Masculine | le lui |
| PPOBJMP | Pronoun complements of objects - Plural Masculine | eux y |
| PPOBJFS | Pronoun complements of objects - Singular Feminine | moi la |
| PPOBJFP | Pronoun complements of objects - Plural Feminine | en y |
| PPER1S | Personal Pronoun First-Person - Singular | je |
| PPER2S | Personal Pronoun Second-Person - Singular | tu |
| PPER3MS | Personal Pronoun Third-Person - Singular Masculine | il |
| PPER3MP | Personal Pronoun Third-Person - Plural Masculine | ils |
| PPER3FS | Personal Pronoun Third-Person - Singular Feminine | elle |
| PPER3FP | Personal Pronoun Third-Person - Plural Feminine | elles |
| PREFS | Reflexive Pronoun First-Person - Singular | me m' |
| PREF | Reflexive Pronoun Third-Person - Singular | se s' |
| PREFP | Reflexive Pronoun First / Second-Person - Plural | nous vous |
| VERB | Verb | obtient |
| VPPMS | Past Participle - Singular Masculine | formulé |
| VPPMP | Past Participle - Plural Masculine | classés |
| VPPFS | Past Participle - Singular Feminine | appelée |
| VPPFP | Past Participle - Plural Feminine | sanctionnées |
| DET | Determinant | les l' |
| DETMS | Determinant - Singular Masculine | les |
| DETFS | Determinant - Singular Feminine | la |
| ADJ | Adjective | capable sérieux |
| ADJMS | Adjective - Singular Masculine | grand important |
| ADJMP | Adjective - Plural Masculine | grands petits |
| ADJFS | Adjective - Singular Feminine | française petite |
| ADJFP | Adjective - Plural Feminine | légères petites |
| NOUN | Noun | temps |
| NMS | Noun - Singular Masculine | drapeau |
| NMP | Noun - Plural Masculine | journalistes |
| NFS | Noun - Singular Feminine | tête |
| NFP | Noun - Plural Feminine | ondes |
| PREL | Relative Pronoun | qui dont |
| PRELMS | Relative Pronoun - Singular Masculine | lequel |
| PRELMP | Relative Pronoun - Plural Masculine | lesquels |
| PRELFS | Relative Pronoun - Singular Feminine | laquelle |
| PRELFP | Relative Pronoun - Plural Feminine | lesquelles |
| INTJ | Interjection | merci bref |
| CHIF | Numbers | 1979 10 |
| SYM | Symbol | € % |
| YPFOR | Endpoint | . |
| PUNCT | Ponctuation | : , |
| MOTINC | Unknown words | Technology Lady |
| X | Typos & others | sfeir 3D statu |


## Evaluation results


The test corpora used for this evaluation is available on [Github](https://github.com/qanastek/ANTILLES/blob/main/ANTILLES/test.conllu).


```plain
Results:
- F-score (micro): 0.952
- F-score (macro): 0.8644
- Accuracy (incl. no class): 0.952

By class:
              precision    recall  f1-score   support
      PPER1S     0.9767    1.0000    0.9882        42
        VERB     0.9823    0.9537    0.9678       583
       COSUB     0.9344    0.8906    0.9120       128
       PUNCT     0.9878    0.9688    0.9782       833
        PREP     0.9767    0.9879    0.9822      1483
      PDEMMS     0.9583    0.9200    0.9388        75
        COCO     0.9839    1.0000    0.9919       245
         DET     0.9679    0.9814    0.9746       645
         NMP     0.9521    0.9115    0.9313       305
       ADJMP     0.8352    0.9268    0.8786        82
        PREL     0.9324    0.9857    0.9583        70
       PREFP     0.9767    0.9545    0.9655        44
         AUX     0.9537    0.9859    0.9695       355
         ADV     0.9440    0.9365    0.9402       504
       VPPMP     0.8667    1.0000    0.9286        26
      DINTMS     0.9919    1.0000    0.9959       122
       ADJMS     0.9020    0.9057    0.9039       244
         NMS     0.9226    0.9336    0.9281       753
         NFS     0.9347    0.9714    0.9527       560
       YPFOR     0.9806    1.0000    0.9902       353
      PINDMS     1.0000    0.9091    0.9524        44
        NOUN     0.8400    0.5385    0.6562        39
       PROPN     0.8605    0.8278    0.8439       395
       DETMS     0.9972    0.9972    0.9972       362
     PPER3MS     0.9341    0.9770    0.9551        87
       VPPMS     0.8994    0.9682    0.9325       157
       DETFS     1.0000    1.0000    1.0000       240
       ADJFS     0.9266    0.9011    0.9136       182
       ADJFP     0.9726    0.9342    0.9530        76
         NFP     0.9463    0.9749    0.9604       199
       VPPFS     0.8000    0.9000    0.8471        40
        CHIF     0.9543    0.9414    0.9478       222
      XFAMIL     0.9346    0.8696    0.9009       115
     PPER3MP     0.9474    0.9000    0.9231        20
     PPOBJMS     0.8800    0.9362    0.9072        47
        PREF     0.8889    0.9231    0.9057        52
     PPOBJMP     1.0000    0.6000    0.7500        10
         SYM     0.9706    0.8684    0.9167        38
      DINTFS     0.9683    1.0000    0.9839        61
      PDEMFS     1.0000    0.8966    0.9455        29
     PPER3FS     1.0000    0.9444    0.9714        18
       VPPFP     0.9500    1.0000    0.9744        19
        PRON     0.9200    0.7419    0.8214        31
     PPOBJFS     0.8333    0.8333    0.8333         6
        PART     0.8000    1.0000    0.8889         4
     PPER3FP     1.0000    1.0000    1.0000         2
      MOTINC     0.3571    0.3333    0.3448        15
      PDEMMP     1.0000    0.6667    0.8000         3
        INTJ     0.4000    0.6667    0.5000         6
       PREFS     1.0000    0.5000    0.6667        10
         ADJ     0.7917    0.8636    0.8261        22
      PINDMP     0.0000    0.0000    0.0000         1
      PINDFS     1.0000    1.0000    1.0000         1
         NUM     1.0000    0.3333    0.5000         3
      PPER2S     1.0000    1.0000    1.0000         2
     PPOBJFP     1.0000    0.5000    0.6667         2
      PDEMFP     1.0000    0.6667    0.8000         3
           X     0.0000    0.0000    0.0000         1
      PRELMS     1.0000    1.0000    1.0000         2
      PINDFP     1.0000    1.0000    1.0000         1

    accuracy                         0.9520     10019
   macro avg     0.8956    0.8521    0.8644     10019
weighted avg     0.9524    0.9520    0.9515     10019
```


## BibTeX Citations


Please cite the following paper when using this model.


UD_French-GSD corpora:


```latex
@misc{
    universaldependencies,
    title={UniversalDependencies/UD_French-GSD},
    url={https://github.com/UniversalDependencies/UD_French-GSD}, journal={GitHub},
    author={UniversalDependencies}
}
```


LIA TAGG:


```latex
@techreport{LIA_TAGG,
  author = {Frédéric Béchet},
  title = {LIA_TAGG: a statistical POS tagger + syntactic bracketer},
  institution = {Aix-Marseille University & CNRS},
  year = {2001}
}
```


Flair Embeddings:


```latex
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```


## Acknowledgment


This work was financially supported by [Zenidoc](https://zenidoc.fr/)


model name: pos-french-camembert-flair


model author: qanastek


task: token-classification


tags: pytorch, fr, flair, token-classification, sequence-tagger-model


tags:


- Transformers
- token-classification
- sequence-tagger-model
language: fr
widget:
- text: "George Washington est allé à Washington"


# POET: A French Extended Part-of-Speech Tagger


- Corpora: [ANTILLES](https://github.com/qanastek/ANTILLES)
- Embeddings & Sequence Labelling: [CamemBERT](https://arxiv.org/abs/1911.03894)
- Number of Epochs: 115


**People Involved**


* [LABRAK Yanis](https://www.linkedin.com/in/yanis-labrak-8a7412145/) (1)
* [DUFOUR Richard](https://cv.archives-ouvertes.fr/richard-dufour) (2)


**Affiliations**


1. [LIA, NLP team](https://lia.univ-avignon.fr/), Avignon University, Avignon, France.
1. [LS2N, TALN team](https://www.ls2n.fr/equipe/taln/), Nantes University, Nantes, France.


## Demo: How to use in HuggingFace Transformers


Requires [transformers](https://pypi.org/project/transformers/): `pip install transformers`


```python
from transformers import CamembertTokenizer, CamembertForTokenClassification, TokenClassificationPipeline

tokenizer = CamembertTokenizer.from_pretrained('qanastek/pos-french-camembert')
model = CamembertForTokenClassification.from_pretrained('qanastek/pos-french-camembert')
pos = TokenClassificationPipeline(model=model, tokenizer=tokenizer)

def make_prediction(sentence):
    labels = [l['entity'] for l in pos(sentence)]
    return list(zip(sentence.split(" "), labels))

res = make_prediction("George Washington est allé à Washington")
```


Output:


![Preview Output](preview.PNG)


## Training data


`ANTILLES` is a part-of-speech tagging corpora based on [UD_French-GSD](https://universaldependencies.org/treebanks/fr_gsd/index.html) which was originally created in 2015 and is based on the [universal dependency treebank v2.0](https://github.com/ryanmcd/uni-dep-tb).


Originally, the corpora consists of 400,399 words (16,341 sentences) and had 17 different classes. Now, after applying our tags augmentation we obtain 60 different classes which add linguistic and semantic information such as the gender, number, mood, person, tense or verb form given in the different CoNLL-03 fields from the original corpora.


We based our tags on the level of details given by the [LIA_TAGG](http://pageperso.lif.univ-mrs.fr/frederic.bechet/download.html) statistical POS tagger written by [Frédéric Béchet](http://pageperso.lif.univ-mrs.fr/frederic.bechet/index-english.html) in 2001.


The corpora used for this model is available on [Github](https://github.com/qanastek/ANTILLES) at the [CoNLL-U format](https://universaldependencies.org/format.html).


Training data are fed to the model as free language and doesn't pass a normalization phase. Thus, it's made the model case and punctuation sensitive.


## Original Tags


```plain
PRON VERB SCONJ ADP CCONJ DET NOUN ADJ AUX ADV PUNCT PROPN NUM SYM PART X INTJ
```


## New additional POS tags


| Abbreviation | Description | Examples |
|:--------:|:--------:|:--------:|
| PREP | Preposition | de |
| AUX | Auxiliary Verb | est |
| ADV | Adverb | toujours |
| COSUB | Subordinating conjunction | que |
| COCO | Coordinating Conjunction | et |
| PART | Demonstrative particle | -t |
| PRON | Pronoun | qui ce quoi |
| PDEMMS | Demonstrative Pronoun - Singular Masculine | ce |
| PDEMMP | Demonstrative Pronoun - Plural Masculine | ceux |
| PDEMFS | Demonstrative Pronoun - Singular Feminine | cette |
| PDEMFP | Demonstrative Pronoun - Plural Feminine | celles |
| PINDMS | Indefinite Pronoun - Singular Masculine | tout |
| PINDMP | Indefinite Pronoun - Plural Masculine | autres |
| PINDFS | Indefinite Pronoun - Singular Feminine | chacune |
| PINDFP | Indefinite Pronoun - Plural Feminine | certaines |
| PROPN | Proper noun | Houston |
| XFAMIL | Last name | Levy |
| NUM | Numerical Adjective | trentaine vingtaine |
| DINTMS | Masculine Numerical Adjective | un |
| DINTFS | Feminine Numerical Adjective | une |
| PPOBJMS | Pronoun complements of objects - Singular Masculine | le lui |
| PPOBJMP | Pronoun complements of objects - Plural Masculine | eux y |
| PPOBJFS | Pronoun complements of objects - Singular Feminine | moi la |
| PPOBJFP | Pronoun complements of objects - Plural Feminine | en y |
| PPER1S | Personal Pronoun First-Person - Singular | je |
| PPER2S | Personal Pronoun Second-Person - Singular | tu |
| PPER3MS | Personal Pronoun Third-Person - Singular Masculine | il |
| PPER3MP | Personal Pronoun Third-Person - Plural Masculine | ils |
| PPER3FS | Personal Pronoun Third-Person - Singular Feminine | elle |
| PPER3FP | Personal Pronoun Third-Person - Plural Feminine | elles |
| PREFS | Reflexive Pronoun First-Person - Singular | me m' |
| PREF | Reflexive Pronoun Third-Person - Singular | se s' |
| PREFP | Reflexive Pronoun First / Second-Person - Plural | nous vous |
| VERB | Verb | obtient |
| VPPMS | Past Participle - Singular Masculine | formulé |
| VPPMP | Past Participle - Plural Masculine | classés |
| VPPFS | Past Participle - Singular Feminine | appelée |
| VPPFP | Past Participle - Plural Feminine | sanctionnées |
| DET | Determinant | les l' |
| DETMS | Determinant - Singular Masculine | les |
| DETFS | Determinant - Singular Feminine | la |
| ADJ | Adjective | capable sérieux |
| ADJMS | Adjective - Singular Masculine | grand important |
| ADJMP | Adjective - Plural Masculine | grands petits |
| ADJFS | Adjective - Singular Feminine | française petite |
| ADJFP | Adjective - Plural Feminine | légères petites |
| NOUN | Noun | temps |
| NMS | Noun - Singular Masculine | drapeau |
| NMP | Noun - Plural Masculine | journalistes |
| NFS | Noun - Singular Feminine | tête |
| NFP | Noun - Plural Feminine | ondes |
| PREL | Relative Pronoun | qui dont |
| PRELMS | Relative Pronoun - Singular Masculine | lequel |
| PRELMP | Relative Pronoun - Plural Masculine | lesquels |
| PRELFS | Relative Pronoun - Singular Feminine | laquelle |
| PRELFP | Relative Pronoun - Plural Feminine | lesquelles |
| INTJ | Interjection | merci bref |
| CHIF | Numbers | 1979 10 |
| SYM | Symbol | € % |
| YPFOR | Endpoint | . |
| PUNCT | Ponctuation | : , |
| MOTINC | Unknown words | Technology Lady |
| X | Typos & others | sfeir 3D statu |


## Evaluation results


The test corpora used for this evaluation is available on [Github](https://github.com/qanastek/ANTILLES/blob/main/ANTILLES/test.conllu).


```plain
              precision    recall  f1-score   support

         ADJ     0.9040    0.8828    0.8933       128
       ADJFP     0.9811    0.9585    0.9697       434
       ADJFS     0.9606    0.9826    0.9715       918
       ADJMP     0.9613    0.9357    0.9483       451
       ADJMS     0.9561    0.9611    0.9586       952
         ADV     0.9870    0.9948    0.9908      1524
         AUX     0.9956    0.9964    0.9960      1124
        CHIF     0.9798    0.9774    0.9786      1239
        COCO     1.0000    0.9989    0.9994       884
       COSUB     0.9939    0.9939    0.9939       328
         DET     0.9972    0.9972    0.9972      2897
       DETFS     0.9990    1.0000    0.9995      1007
       DETMS     1.0000    0.9993    0.9996      1426
      DINTFS     0.9967    0.9902    0.9934       306
      DINTMS     0.9923    0.9948    0.9935       387
        INTJ     0.8000    0.8000    0.8000         5
      MOTINC     0.5049    0.5827    0.5410       266
         NFP     0.9807    0.9675    0.9740       892
         NFS     0.9778    0.9699    0.9738      2588
         NMP     0.9687    0.9495    0.9590      1367
         NMS     0.9759    0.9560    0.9659      3181
        NOUN     0.6164    0.8673    0.7206       113
         NUM     0.6250    0.8333    0.7143         6
        PART     1.0000    0.9375    0.9677        16
      PDEMFP     1.0000    1.0000    1.0000         3
      PDEMFS     1.0000    1.0000    1.0000        89
      PDEMMP     1.0000    1.0000    1.0000        20
      PDEMMS     1.0000    1.0000    1.0000       222
      PINDFP     1.0000    1.0000    1.0000         3
      PINDFS     0.8571    1.0000    0.9231        12
      PINDMP     0.9000    1.0000    0.9474         9
      PINDMS     0.9286    0.9701    0.9489        67
      PINTFS     0.0000    0.0000    0.0000         2
      PPER1S     1.0000    1.0000    1.0000        62
      PPER2S     0.7500    1.0000    0.8571         3
     PPER3FP     1.0000    1.0000    1.0000         9
     PPER3FS     1.0000    1.0000    1.0000        96
     PPER3MP     1.0000    1.0000    1.0000        31
     PPER3MS     1.0000    1.0000    1.0000       377
     PPOBJFP     1.0000    0.7500    0.8571         4
     PPOBJFS     0.9167    0.8919    0.9041        37
     PPOBJMP     0.7500    0.7500    0.7500        12
     PPOBJMS     0.9371    0.9640    0.9504       139
        PREF     1.0000    1.0000    1.0000       332
       PREFP     1.0000    1.0000    1.0000        64
       PREFS     1.0000    1.0000    1.0000        13
        PREL     0.9964    0.9964    0.9964       277
      PRELFP     1.0000    1.0000    1.0000         5
      PRELFS     0.8000    1.0000    0.8889         4
      PRELMP     1.0000    1.0000    1.0000         3
      PRELMS     1.0000    1.0000    1.0000        11
        PREP     0.9971    0.9977    0.9974      6161
        PRON     0.9836    0.9836    0.9836        61
       PROPN     0.9468    0.9503    0.9486      4310
       PUNCT     1.0000    1.0000    1.0000      4019
         SYM     0.9394    0.8158    0.8732        76
        VERB     0.9956    0.9921    0.9938      2273
       VPPFP     0.9145    0.9469    0.9304       113
       VPPFS     0.9562    0.9597    0.9580       273
       VPPMP     0.8827    0.9728    0.9256       147
       VPPMS     0.9778    0.9794    0.9786       630
       VPPRE     0.0000    0.0000    0.0000         1
           X     0.9604    0.9935    0.9766      1073
      XFAMIL     0.9386    0.9113    0.9248      1342
       YPFOR     1.0000    1.0000    1.0000      2750

    accuracy                         0.9778     47574
   macro avg     0.9151    0.9285    0.9202     47574
weighted avg     0.9785    0.9778    0.9780     47574
```


## BibTeX Citations


Please cite the following paper when using this model.


UD_French-GSD corpora:


```latex
@misc{
    universaldependencies,
    title={UniversalDependencies/UD_French-GSD},
    url={https://github.com/UniversalDependencies/UD_French-GSD}, journal={GitHub},
    author={UniversalDependencies}
}
```


LIA TAGG:


```latex
@techreport{LIA_TAGG,
  author = {Frédéric Béchet},
  title = {LIA_TAGG: a statistical POS tagger + syntactic bracketer},
  institution = {Aix-Marseille University & CNRS},
  year = {2001}
}
```


Flair Embeddings:


```latex
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```


## Acknowledgment


This work was financially supported by [Zenidoc](https://zenidoc.fr/)


model name: pos-french-camembert


model author: qanastek


task: token-classification


tags: pytorch, camembert, token-classification, fr, arxiv:1911.03894, transformers, Transformers, sequence-tagger-model, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: t5-base-ja-web-real-estate-qa-002.1
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# t5-base-ja-web-real-estate-qa-002.1


This model is a fine-tuned version of [megagonlabs/t5-base-japanese-web](https://huggingface.co/megagonlabs/t5-base-japanese-web) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2925
- Exact Match: 0.6684


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50


### Training results


| Training Loss | Epoch | Step | Validation Loss | Exact Match |
|:-------------:|:-----:|:----:|:---------------:|:-----------:|
| 55.7729       | 0.95  | 10   | 44.1808         | 0.0         |
| 43.4632       | 1.95  | 20   | 26.5466         | 0.0         |
| 16.528        | 2.95  | 30   | 5.7612          | 0.0         |
| 5.3828        | 3.95  | 40   | 3.3035          | 0.0         |
| 2.0384        | 4.95  | 50   | 0.9221          | 0.3316      |
| 1.0314        | 5.95  | 60   | 0.8274          | 0.3316      |
| 0.7438        | 6.95  | 70   | 0.5133          | 0.2263      |
| 0.4653        | 7.95  | 80   | 0.3065          | 0.3053      |
| 0.3025        | 8.95  | 90   | 0.2782          | 0.3158      |
| 0.2289        | 9.95  | 100  | 0.3112          | 0.3579      |
| 0.2161        | 10.95 | 110  | 0.2701          | 0.3737      |
| 0.1619        | 11.95 | 120  | 0.2359          | 0.5053      |
| 0.12          | 12.95 | 130  | 0.2429          | 0.5316      |
| 0.1042        | 13.95 | 140  | 0.2284          | 0.5474      |
| 0.0871        | 14.95 | 150  | 0.2320          | 0.5842      |
| 0.071         | 15.95 | 160  | 0.2244          | 0.5474      |
| 0.0628        | 16.95 | 170  | 0.2306          | 0.5895      |
| 0.0569        | 17.95 | 180  | 0.2290          | 0.6158      |
| 0.0582        | 18.95 | 190  | 0.2215          | 0.6421      |
| 0.0565        | 19.95 | 200  | 0.2213          | 0.6211      |
| 0.047         | 20.95 | 210  | 0.2434          | 0.6368      |
| 0.056         | 21.95 | 220  | 0.2391          | 0.6263      |
| 0.0376        | 22.95 | 230  | 0.2418          | 0.6211      |
| 0.0318        | 23.95 | 240  | 0.2509          | 0.6053      |
| 0.0369        | 24.95 | 250  | 0.2486          | 0.6421      |
| 0.0268        | 25.95 | 260  | 0.2440          | 0.6368      |
| 0.0227        | 26.95 | 270  | 0.2551          | 0.6789      |
| 0.0269        | 27.95 | 280  | 0.2494          | 0.6421      |
| 0.0204        | 28.95 | 290  | 0.2570          | 0.6632      |
| 0.0192        | 29.95 | 300  | 0.2547          | 0.6789      |
| 0.0172        | 30.95 | 310  | 0.2558          | 0.6789      |
| 0.0143        | 31.95 | 320  | 0.2638          | 0.6632      |
| 0.0178        | 32.95 | 330  | 0.2704          | 0.6895      |
| 0.0131        | 33.95 | 340  | 0.2767          | 0.6632      |
| 0.0148        | 34.95 | 350  | 0.2823          | 0.6684      |
| 0.0132        | 35.95 | 360  | 0.2803          | 0.6579      |
| 0.013         | 36.95 | 370  | 0.2810          | 0.6684      |
| 0.0119        | 37.95 | 380  | 0.2788          | 0.6632      |
| 0.0107        | 38.95 | 390  | 0.2819          | 0.6737      |
| 0.012         | 39.95 | 400  | 0.2890          | 0.6842      |
| 0.0109        | 40.95 | 410  | 0.2821          | 0.6684      |
| 0.0103        | 41.95 | 420  | 0.2826          | 0.6632      |
| 0.01          | 42.95 | 430  | 0.2883          | 0.6737      |
| 0.0099        | 43.95 | 440  | 0.2893          | 0.6684      |
| 0.0099        | 44.95 | 450  | 0.2901          | 0.6526      |
| 0.009         | 45.95 | 460  | 0.2912          | 0.6579      |
| 0.0071        | 46.95 | 470  | 0.2930          | 0.6684      |
| 0.0095        | 47.95 | 480  | 0.2926          | 0.6684      |
| 0.0083        | 48.95 | 490  | 0.2924          | 0.6684      |
| 0.0096        | 49.95 | 500  | 0.2925          | 0.6684      |


### Framework versions


- Transformers 4.9.0
- Pytorch 1.8.1
- Datasets 1.11.0
- Tokenizers 0.10.3


model name: t5-base-ja-web-real-estate-qa-002.1


model author: fnakamura


task: text2text-generation


tags: pytorch, tensorboard, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model-index:
- name: t5-base-ja-web-real-estate-qa-fn
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation


# t5-base-ja-web-real-estate-qa-fn


- Loss: 5.1723
- Exact Match: 0.0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10


### Training results


| Training Loss | Epoch | Step | Validation Loss | Exact Match |
|:-------------:|:-----:|:----:|:---------------:|:-----------:|
| 56.8452       | 0.99  | 53   | 45.3518         | 0.0         |
| 47.8834       | 1.99  | 106  | 39.6543         | 0.0         |
| 39.8258       | 2.99  | 159  | 33.6746         | 0.0         |
| 30.5708       | 3.99  | 212  | 25.0430         | 0.0         |
| 20.2812       | 4.99  | 265  | 14.8542         | 0.0         |
| 11.1975       | 5.99  | 318  | 6.2401          | 0.0         |
| 7.2441        | 6.99  | 371  | 5.6463          | 0.0         |
| 6.5766        | 7.99  | 424  | 5.3585          | 0.0         |
| 6.2686        | 8.99  | 477  | 5.2164          | 0.0         |
| 6.0645        | 9.99  | 530  | 5.1723          | 0.0         |


### Framework versions


- Transformers 4.10.0
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


model name: t5-base-ja-web-real-estate-qa-fn


model author: fnakamura


task: text2text-generation


tags: pytorch, tensorboard, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: t5-base-ja-web-real-estate-qa-002.2
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation


# t5-base-ja-web-real-estate-qa-002.2


- Loss: 0.2389
- Exact Match: 0.6947


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50


### Training results


| Training Loss | Epoch | Step | Validation Loss | Exact Match |
|:-------------:|:-----:|:----:|:---------------:|:-----------:|
| 38.2599       | 1.0   | 42   | 25.7663         | 0.0         |
| 3.9819        | 2.0   | 84   | 1.6016          | 0.0         |
| 1.2762        | 3.0   | 126  | 0.5876          | 0.3526      |
| 0.3335        | 4.0   | 168  | 0.2774          | 0.3316      |
| 0.2647        | 5.0   | 210  | 0.2152          | 0.4211      |
| 0.1191        | 6.0   | 252  | 0.2103          | 0.4895      |
| 0.0918        | 7.0   | 294  | 0.1941          | 0.5579      |
| 0.0916        | 8.0   | 336  | 0.2107          | 0.5895      |
| 0.0534        | 9.0   | 378  | 0.1958          | 0.6526      |
| 0.055         | 10.0  | 420  | 0.2063          | 0.6         |
| 0.0363        | 11.0  | 462  | 0.2106          | 0.6368      |
| 0.0357        | 12.0  | 504  | 0.2285          | 0.6789      |
| 0.0281        | 13.0  | 546  | 0.2132          | 0.6368      |
| 0.0178        | 14.0  | 588  | 0.2293          | 0.6684      |
| 0.0112        | 15.0  | 630  | 0.2484          | 0.6632      |
| 0.0144        | 16.0  | 672  | 0.2389          | 0.6895      |
| 0.0211        | 17.0  | 714  | 0.2424          | 0.6684      |
| 0.0108        | 18.0  | 756  | 0.2507          | 0.6579      |
| 0.0189        | 19.0  | 798  | 0.2546          | 0.6632      |
| 0.009         | 20.0  | 840  | 0.2474          | 0.6579      |
| 0.0058        | 21.0  | 882  | 0.2561          | 0.6632      |
| 0.0076        | 22.0  | 924  | 0.2680          | 0.6684      |
| 0.0035        | 23.0  | 966  | 0.2808          | 0.6632      |
| 0.0032        | 24.0  | 1008 | 0.2791          | 0.6737      |
| 0.0027        | 25.0  | 1050 | 0.2866          | 0.6579      |
| 0.0024        | 26.0  | 1092 | 0.2900          | 0.6632      |
| 0.0019        | 27.0  | 1134 | 0.2968          | 0.6842      |
| 0.0055        | 28.0  | 1176 | 0.2594          | 0.6526      |
| 0.0019        | 29.0  | 1218 | 0.2970          | 0.6368      |
| 0.0103        | 30.0  | 1260 | 0.2737          | 0.6684      |
| 0.0022        | 31.0  | 1302 | 0.2883          | 0.6684      |
| 0.0021        | 32.0  | 1344 | 0.3025          | 0.6789      |
| 0.0017        | 33.0  | 1386 | 0.3165          | 0.6632      |
| 0.0035        | 34.0  | 1428 | 0.3162          | 0.6526      |
| 0.0031        | 35.0  | 1470 | 0.3072          | 0.6579      |
| 0.0008        | 36.0  | 1512 | 0.2975          | 0.6737      |
| 0.0041        | 37.0  | 1554 | 0.3152          | 0.6789      |
| 0.0009        | 38.0  | 1596 | 0.3153          | 0.6579      |
| 0.0013        | 39.0  | 1638 | 0.3147          | 0.6684      |
| 0.0022        | 40.0  | 1680 | 0.3147          | 0.6842      |
| 0.0009        | 41.0  | 1722 | 0.3109          | 0.6789      |
| 0.0008        | 42.0  | 1764 | 0.3168          | 0.6789      |
| 0.0005        | 43.0  | 1806 | 0.3216          | 0.6789      |
| 0.0004        | 44.0  | 1848 | 0.3269          | 0.6789      |
| 0.0011        | 45.0  | 1890 | 0.3248          | 0.6789      |
| 0.0008        | 46.0  | 1932 | 0.3266          | 0.6842      |
| 0.0003        | 47.0  | 1974 | 0.3290          | 0.6842      |
| 0.0009        | 48.0  | 2016 | 0.3310          | 0.6842      |
| 0.0003        | 49.0  | 2058 | 0.3312          | 0.6842      |
| 0.0002        | 50.0  | 2100 | 0.3318          | 0.6842      |


### Framework versions


model name: t5-base-ja-web-real-estate-qa-002.2


model author: fnakamura


task: text2text-generation


tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: t5-base-ja-web-real-estate-qa-002.2.0
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# t5-base-ja-web-real-estate-qa-002.2.0


This model was trained from scratch on the None dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 0


### Framework versions


- Transformers 4.9.0
- Pytorch 1.8.1
- Datasets 1.11.0
- Tokenizers 0.10.3


model name: t5-base-ja-web-real-estate-qa-002.2.0


model author: fnakamura


task: text2text-generation


tags: pytorch, tensorboard, t5, text2text-generation, transformers, generated_from_trainer, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: t5-base-ja-web-real-estate-qa-002.0
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation


# t5-base-ja-web-real-estate-qa-002.0


This model is a fine-tuned version of [megagonlabs/t5-base-japanese-web](https://huggingface.co/megagonlabs/t5-base-japanese-web) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 31.2699
- Exact Match: 0.0


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1


### Training results


| Training Loss | Epoch | Step | Validation Loss | Exact Match |
|:-------------:|:-----:|:----:|:---------------:|:-----------:|
| 43.4949       | 0.95  | 10   | 31.2699         | 0.0         |


### Framework versions


model name: t5-base-ja-web-real-estate-qa-002.0


model author: fnakamura


task: text2text-generation


tags: pytorch, tensorboard, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, autonlp_compatible


tags:


- conversational


# Jack Sparrow GPT


model name: Jack_Sparrow_GPT


model author: Harshal6927


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


# Tony Stark GPT


My first AI model still learning, used small dataset so don't expect much


model name: Tony_Stark_GPT


korean Mental Health BERT


huggingface에 공개된 KR-Medium BERT를 아래의 dataset으로 MLM fine-tuining한 Bert Model입니다. 정신건강 문제 해결에 도움이 될만한 데이터셋이라고 판단하여 domain-adaptation하였고, 향후 정신건강 관련 감정 및 상태 classification 및 그에 따른 chatbot 구현에 사용할 수 있습니다. 이후 공개될 예정인 더 큰 규모의 데이터셋까지 Dapt할 예정입니다.


datasets from AIhub


웰니스 대화 스크립트 데이터셋1 & 2 (중복 제거 약 2만9천개)


model name: koMHBERT-krbert-based-v1


model author: eunjin


task: feature-extraction


tags: pytorch, jax, bert, feature-extraction, transformers, infinity_compatible


* skt/kogpt2-base-v2에 wellness 및 일상챗봇 데이터를 fine-tuning한 모델입니다.
* 유사한 정신건강 상담 도메인에서 바로 사용 가능합니다.
* 깃허브 사이트를 참조해주세요! https://github.com/eunjiinkim/WellnessChatbot


model name: kogpt2-finetuned-wellness


task: text-generation


tags: pytorch, gpt2, text-generation, transformers


kcBERT를 아래의 dataset으로 MLM fine-tuining한 Bert Model입니다. 정신건강 문제 해결에 도움이 될만한 데이터셋이라고 판단하여 domain-adaptation하였고, 향후 정신건강 관련 감정 및 상태 classification 및 그에 따른 chatbot 구현에 사용할 수 있습니다.
이후 공개될 예정인 더 큰 규모의 데이터셋까지 Dapt할 예정입니다.


@inproceedings{lee2020kcbert, title={KcBERT: Korean Comments BERT}, author={Lee, Junbum}, booktitle={Proceedings of the 32nd Annual Conference on Human and Cognitive Language Technology}, pages={437--440}, year={2020} }


model name: koMHBERT-kcbert-based-v1


korean Mental Health BERT -v2


huggingface에 공개된 KR-Medium BERT를 정신건강의학신문을 크롤링한 dataset으로 MLM fine-tuining한 Bert Model입니다.
정신건강 발화 관련 데이터를 모을 수 없는 상황에서 이를 대체할만한 데이터로 제시합니다.
향후 정신건강 관련 감정 및 상태 classification 및 그에 따른 chatbot 구현에 사용할 수 있습니다.


정신건강의학신문: http://www.psychiatricnews.net


model name: koMHBERT-krbert-based-v2


tags: pytorch, bert, feature-extraction, transformers, infinity_compatible


huggingface에 공개된 kcbert-base BERT를 정신건강의학신문을 크롤링한 dataset으로 MLM fine-tuining한 Bert Model입니다. 정신건강 발화 관련 데이터를 모을 수 없는 상황에서 이를 대체할만한 데이터로 제시합니다. 향후 정신건강 관련 감정 및 상태 classification 및 그에 따른 chatbot 구현에 사용할 수 있습니다.


model name: koMHBERT-kcbert-based-v2


model author: eunjin


task: feature-extraction


tags:


- conversational


# Rick Morty DialoGPT Model


model name: DialoGPT-medium-RickandMortyBot


model author: MAUtastic


task: conversational


RoBERTa trained on 8.8 Million Telugu Sentences


model name: ltrc-roberta


model author: ltrctelugu


task: fill-mask


tags: pytorch, roberta, fill-mask, transformers, autonlp_compatible, infinity_compatible


hello


model name: ltrc-albert


tags: pytorch, albert, fill-mask, transformers, autonlp_compatible, infinity_compatible


model name: ltrc-distilbert


tags: pytorch, distilbert, fill-mask, transformers, autonlp_compatible, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: distilroberta-base-finetuned-yulusa-hehealth
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilroberta-base-finetuned-yulusa-hehealth


This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 2.0466


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.3248        | 1.0   | 2781 | 2.1415          |
| 2.2214        | 2.0   | 5562 | 2.0863          |
| 2.1666        | 3.0   | 8343 | 2.0466          |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.9.0+cu111
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: distilroberta-base-finetuned-yulusa-hehealth


model author: saugat


task: fill-mask