```
git clone https://huggingface.co/templates/feature-extraction
cd feature-extraction
git remote set-url origin https://huggingface.co/$YOUR_USER/$YOUR_REPO_NAME
git push --force
```


model name: feature-extraction


model author: templates


task: feature-extraction


tags: generic, feature-extraction


tags:


- conversational


# Thor DialogGPT Model


model name: DialoGPT-small-thor


model author: J-Chiang


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


# Rikka DialoGPT Model


model name: DialoGPT-small-Rikka


model author: AriakimTaiyo


# Cultured Kumiko DialoGPT Model


model name: DialoGPT-cultured-Kumiko


a


model name: kumiko


# Revised Kumiko DialoGPT Model


model name: DialoGPT-revised-Kumiko


# Kumiko DialoGPT Model


model name: DialoGPT-small-Kumiko


tags:


- conversational


# Medium Kumiko DialoGPT Model


model name: DialoGPT-medium-Kumiko


task: conversational


tags: conversational


language: en
tags:


- image-classification
- image-captioning


# Poster2Plot


An image captioning model to generate movie/t.v show plot from poster. It generates decent plots but is no way perfect. We are still working on improving the model.


## Live demo on Hugging Face Spaces: https://huggingface.co/spaces/deepklarity/poster2plot


# Model Details


The base model uses a Vision Transformer (ViT) model as an image encoder and GPT-2 as a decoder.


We used the following models:


* Encoder: [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k)
* Decoder: [gpt2](https://huggingface.co/gpt2)


# Datasets


Publicly available IMDb datasets were used to train the model.


# How to use


## In PyTorch


```python
import torch
import re
import requests
from PIL import Image
from transformers import AutoTokenizer, AutoFeatureExtractor, VisionEncoderDecoderModel

# Pattern to ignore all the text after 2 or more full stops
regex_pattern = "[.]{2,}"

def post_process(text):
    try:
        text = text.strip()
        text = re.split(regex_pattern, text)[0]
    except Exception as e:
        print(e)
        pass
    return text

def predict(image, max_length=64, num_beams=4):
    pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
    pixel_values = pixel_values.to(device)

    with torch.no_grad():
        output_ids = model.generate(
            pixel_values,
            max_length=max_length,
            num_beams=num_beams,
            return_dict_in_generate=True,
        ).sequences

    preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
    pred = post_process(preds[0])

    return pred

model_name_or_path = "deepklarity/poster2plot"
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# Load model.

model = VisionEncoderDecoderModel.from_pretrained(model_name_or_path)
model.to(device)
print("Loaded model")

feature_extractor = AutoFeatureExtractor.from_pretrained(model.encoder.name_or_path)
print("Loaded feature_extractor")

tokenizer = AutoTokenizer.from_pretrained(model.decoder.name_or_path, use_fast=True)
if model.decoder.name_or_path == "gpt2":
    tokenizer.pad_token = tokenizer.eos_token

print("Loaded tokenizer")

url = "https://upload.wikimedia.org/wikipedia/en/2/26/Moana_Teaser_Poster.jpg"
with Image.open(requests.get(url, stream=True).raw) as image:
    pred = predict(image)

print(pred)

```


model name: poster2plot


model author: deepklarity


task: image-classification


tags: pytorch, vision-encoder-decoder, en, transformers, image-classification, image-captioning


Roberta-base training attempt on hindi datasets.


model name: roberta-base-hindi


tags:


- generated_from_trainer
model-index:
- name: gpt-4-est-base
results: []


# gpt-4-est-base


This is GPT for Estonian. Not GPT-4 :-) This is the base-size [GPT2](https://huggingface.co/docs/transformers/model_doc/gpt2) model, trained from scratch on 2.2 billion words (Estonian National Corpus + News Crawl + Common Crawl) for 3 epochs.


[Colab demo](https://colab.research.google.com/drive/1Bp7mGEQ1vmyqXPyXHV1yj68cRZEi2mq4?usp=sharing)


### Format


For training data was prepended with a text domain tag, and it should be added as prefix when using the model: >general<, >web<, >news<, >doaj< and >wiki< (standing for general texts, web crawled texts, news, article abstracts and wikipedia texts). Use the prefixes like this, e.g: ">web< Kas tead, et".


### Model details


- num. of layers: 12
- num. of heads: 12
- embedding size: 768
- context size: 1024
- total size: 118.68M params


Further details to be added soon.


### Framework versions


- Transformers 4.13.0.dev0
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: gpt-4-est-base


model author: tartuNLP


task: text-generation


tags: pytorch, gpt2, text-generation, transformers, generated_from_trainer, model-index


- generated_from_trainer
model-index:
- name: gpt-4-est-large
results: []


# gpt-4-est-large


This is GPT for Estonian. Not GPT-4 :-) This is the large-size [GPT2](https://huggingface.co/docs/transformers/model_doc/gpt2) model, trained from scratch on 2.2 billion words (Estonian National Corpus + News Crawl + Common Crawl).


- num. of layers: 24
- num. of heads: 24
- embedding size: 1536
- context size: 1024
- total size: 723.58M params


model name: gpt-4-est-large


# EstBERT_NER


## Model description


EstBERT_NER is a fine-tuned EstBERT model that can be used for Named Entity Recognition. This model was trained on the Estonian NER dataset created by [Tkachenko et al](https://www.aclweb.org/anthology/W13-2412.pdf). It can recognize three types of entities: locations (LOC), organizations (ORG) and persons (PER).


## How to use


You can use this model with Transformers pipeline for NER. Post-processing of results may be necessary as the model occasionally tags subword tokens as entities.


```
from transformers import BertTokenizer, BertForTokenClassification
from transformers import pipeline

tokenizer = BertTokenizer.from_pretrained('tartuNLP/EstBERT_NER')
bertner = BertForTokenClassification.from_pretrained('tartuNLP/EstBERT_NER')

nlp = pipeline("ner", model=bertner, tokenizer=tokenizer)
sentence = 'Eesti Ekspressi teada on Eesti Pank uurinud Hansapanga tehinguid , mis toimusid kaks aastat tagasi suvel ja mille käigus voolas panka ligi miljardi krooni ulatuses kahtlast raha .'

ner_results = nlp(sentence)
print(ner_results)
```


```
[{'word': 'Eesti', 'score': 0.9964128136634827, 'entity': 'B-ORG', 'index': 1}, {'word': 'Ekspressi', 'score': 0.9978809356689453, 'entity': 'I-ORG', 'index': 2}, {'word': 'Eesti', 'score': 0.9988121390342712, 'entity': 'B-ORG', 'index': 5}, {'word': 'Pank', 'score': 0.9985784292221069, 'entity': 'I-ORG', 'index': 6}, {'word': 'Hansapanga', 'score': 0.9979034662246704, 'entity': 'B-ORG', 'index': 8}]

```


## BibTeX entry and citation info


```
@misc{tanvir2020estbert,
      title={EstBERT: A Pretrained Language-Specific BERT for Estonian},
      author={Hasan Tanvir and Claudia Kittask and Kairit Sirts},
      year={2020},
      eprint={2011.04784},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


model name: EstBERT_NER


model author: tartuNLP


task: token-classification


tags: pytorch, jax, bert, token-classification, arxiv:2011.04784, transformers, autonlp_compatible, infinity_compatible


## language: et


# EstBERT


### What's this?


The EstBERT model is a pretrained BERTBase model exclusively trained on Estonian cased corpus on both 128 and 512 sequence length of data.


### How to use?


You can use the model transformer library both in tensorflow and pytorch version.


```
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("tartuNLP/EstBERT")
model = AutoModelForMaskedLM.from_pretrained("tartuNLP/EstBERT")
```


You can also download the pretrained model from here, [EstBERT_128]() [EstBERT_512]()


#### Dataset used to train the model


The EstBERT model is trained both on 128 and 512 sequence length of data. For training the EstBERT we used the [Estonian National Corpus 2017](https://metashare.ut.ee/repository/browse/estonian-national-corpus-2017/b616ceda30ce11e8a6e4005056b40024880158b577154c01bd3d3fcfc9b762b3/), which was the largest Estonian language corpus available at the time. It consists of four sub-corpora: Estonian Reference Corpus 1990-2008, Estonian Web Corpus 2013, Estonian Web Corpus 2017 and Estonian Wikipedia Corpus 2017.


### Why would I use?


Overall EstBERT performs better in parts of speech (POS), name entity recognition (NER), rubric, and sentiment classification tasks compared to mBERT and XLM-RoBERTa. The comparative results can be found below;


|Model   |UPOS                  |XPOS   |Morph  |bf UPOS   |bf XPOS                  |Morph                 |
|--------------|----------------------------|-------------|-------------|-------------|----------------------------|----------------------------|
| EstBERT      | ***97.89*** | **98.40** | **96.93** | **97.84** | ***98.43*** | ***96.80*** |
| mBERT        | 97.42                     | 98.06      | 96.24      | 97.43      | 98.13                     | 96.13                     |
| XLM-RoBERTa | 97.78                     | 98.36      | 96.53      | 97.80      | 98.40                     | 96.69                     |


|Model|Rubric128        |Sentiment128  | Rubric128   |Sentiment512         |
|-------------------|----------------------------|--------------------|-----------------------------------------------|----------------------------|
| EstBERT           | ***81.70*** | 74.36             | **80.96**                                   | 74.50                     |
| mBERT             | 75.67                     | 70.23             | 74.94                                        | 69.52                     |
| XLM\-RoBERTa      | 80.34                     | **74.50**        | 78.62                                        | ***76.07***|


|Model   |Precicion128   |Recall128                  |F1-Score128               |Precision512               |Recall512   |F1-Score512   |
|--------------|----------------|----------------------------|----------------------------|----------------------------|-------------|----------------|
| EstBERT      | **88.42**    | 90.38                     |***89.39*** | 88.35                     | 89.74      | 89.04         |
| mBERT        | 85.88         | 87.09                     | 86.51                     |***88.47*** | 88.28      | 88.37         |
| XLM\-RoBERTa | 87.55         |***91.19*** | 89.34                     | 87.50                     | **90.76** | **89.10**    |


model name: EstBERT


model author: tartuNLP


task: fill-mask


tags: pytorch, jax, bert, fill-mask, et, transformers, autonlp_compatible, infinity_compatible


Algebraic Notation model of sequences of moves of complete chess games.


model name: GameANchess


model author: raruidol


task: text-generation


tags: pytorch, gpt2, text-generation, transformers


# Argument Relation Mining


Best performing model trained in the "Transformer-Based Models for Automatic Detection of Argument Relations: A Cross-Domain Evaluation" paper.


Code available in https://github.com/raruidol/ArgumentRelationMining


Cite:


```
@article{ruiz2021transformer,
  title={Transformer-based models for automatic identification of argument relations: A cross-domain evaluation},
  author={Ruiz-Dolz, Ramon and Alemany, Jose and Heras, Stella and Garcia-Fornes, Ana},
  journal={IEEE Intelligent Systems},
  year={2021},
  publisher={IEEE}
}
```


model name: ArgumentRelation


task: text-classification


tags: pytorch, roberta, text-classification, transformers, infinity_compatible


Algebraic Notation model of sequences of moves done by a unique player in a chess game.


model name: PlayerANchess


Hello, I am Saba Rajput and I am a 22-year-old young and charming Delhi Call girls and I provide Delhi Escorts Service Safe & Secure High-Class Services Affordable Rate 100% Satisfaction, Unlimited Enjoyment. I am from Delhi and I am a working girl in an Escorts company based in Delhi


model name: Delhi-Escorts


model author: sabarajputseo2


tags:


- generated_from_trainer
datasets:
- wikitext
model-index:
- name: msft-regular-model
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# msft-regular-model


This model is a fine-tuned version of [](https://huggingface.co/) on the wikitext dataset.
It achieves the following results on the evaluation set:


- Loss: 5.3420


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20


### Training results


| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 9.1224        | 0.17  | 200   | 8.0736          |
| 7.5229        | 0.34  | 400   | 7.1536          |
| 7.0122        | 0.51  | 600   | 6.9072          |
| 6.8296        | 0.69  | 800   | 6.7582          |
| 6.709         | 0.86  | 1000  | 6.6436          |
| 6.5882        | 1.03  | 1200  | 6.5563          |
| 6.4807        | 1.2   | 1400  | 6.4784          |
| 6.4172        | 1.37  | 1600  | 6.4165          |
| 6.3403        | 1.54  | 1800  | 6.3555          |
| 6.2969        | 1.71  | 2000  | 6.3107          |
| 6.2346        | 1.89  | 2200  | 6.2691          |
| 6.1767        | 2.06  | 2400  | 6.2299          |
| 6.1326        | 2.23  | 2600  | 6.1937          |
| 6.1035        | 2.4   | 2800  | 6.1602          |
| 6.0624        | 2.57  | 3000  | 6.1241          |
| 6.0393        | 2.74  | 3200  | 6.0971          |
| 5.9982        | 2.91  | 3400  | 6.0656          |
| 5.9526        | 3.08  | 3600  | 6.0397          |
| 5.9086        | 3.26  | 3800  | 6.0104          |
| 5.8922        | 3.43  | 4000  | 5.9888          |
| 5.8631        | 3.6   | 4200  | 5.9661          |
| 5.8396        | 3.77  | 4400  | 5.9407          |
| 5.8055        | 3.94  | 4600  | 5.9177          |
| 5.7763        | 4.11  | 4800  | 5.9007          |
| 5.7314        | 4.28  | 5000  | 5.8834          |
| 5.7302        | 4.46  | 5200  | 5.8620          |
| 5.6987        | 4.63  | 5400  | 5.8451          |
| 5.6754        | 4.8   | 5600  | 5.8242          |
| 5.6571        | 4.97  | 5800  | 5.8059          |
| 5.615         | 5.14  | 6000  | 5.7871          |
| 5.596         | 5.31  | 6200  | 5.7817          |
| 5.5738        | 5.48  | 6400  | 5.7570          |
| 5.5641        | 5.66  | 6600  | 5.7431          |
| 5.5503        | 5.83  | 6800  | 5.7271          |
| 5.5214        | 6.0   | 7000  | 5.7108          |
| 5.4712        | 6.17  | 7200  | 5.7018          |
| 5.48          | 6.34  | 7400  | 5.6936          |
| 5.4527        | 6.51  | 7600  | 5.6812          |
| 5.4514        | 6.68  | 7800  | 5.6669          |
| 5.4454        | 6.86  | 8000  | 5.6509          |
| 5.399         | 7.03  | 8200  | 5.6408          |
| 5.3747        | 7.2   | 8400  | 5.6327          |
| 5.3667        | 7.37  | 8600  | 5.6197          |
| 5.3652        | 7.54  | 8800  | 5.6084          |
| 5.3394        | 7.71  | 9000  | 5.5968          |
| 5.3349        | 7.88  | 9200  | 5.5870          |
| 5.2994        | 8.05  | 9400  | 5.5826          |
| 5.2793        | 8.23  | 9600  | 5.5710          |
| 5.2716        | 8.4   | 9800  | 5.5623          |
| 5.275         | 8.57  | 10000 | 5.5492          |
| 5.264         | 8.74  | 10200 | 5.5449          |
| 5.241         | 8.91  | 10400 | 5.5322          |
| 5.2285        | 9.08  | 10600 | 5.5267          |
| 5.2021        | 9.25  | 10800 | 5.5187          |
| 5.1934        | 9.43  | 11000 | 5.5158          |
| 5.1737        | 9.6   | 11200 | 5.5044          |
| 5.1774        | 9.77  | 11400 | 5.5008          |
| 5.1841        | 9.94  | 11600 | 5.4960          |
| 5.1414        | 10.11 | 11800 | 5.4895          |
| 5.1491        | 10.28 | 12000 | 5.4849          |
| 5.1184        | 10.45 | 12200 | 5.4738          |
| 5.1136        | 10.63 | 12400 | 5.4690          |
| 5.1199        | 10.8  | 12600 | 5.4598          |
| 5.1056        | 10.97 | 12800 | 5.4536          |
| 5.0648        | 11.14 | 13000 | 5.4496          |
| 5.0598        | 11.31 | 13200 | 5.4449          |
| 5.0656        | 11.48 | 13400 | 5.4422          |
| 5.0664        | 11.65 | 13600 | 5.4367          |
| 5.0675        | 11.83 | 13800 | 5.4286          |
| 5.0459        | 12.0  | 14000 | 5.4249          |
| 5.0073        | 12.17 | 14200 | 5.4260          |
| 5.0229        | 12.34 | 14400 | 5.4175          |
| 5.0079        | 12.51 | 14600 | 5.4119          |
| 5.0           | 12.68 | 14800 | 5.4194          |
| 5.0094        | 12.85 | 15000 | 5.4068          |
| 4.9967        | 13.02 | 15200 | 5.3995          |
| 4.9541        | 13.2  | 15400 | 5.4002          |
| 4.9753        | 13.37 | 15600 | 5.3965          |
| 4.9732        | 13.54 | 15800 | 5.3925          |
| 4.9624        | 13.71 | 16000 | 5.3888          |
| 4.9559        | 13.88 | 16200 | 5.3824          |
| 4.9559        | 14.05 | 16400 | 5.3851          |
| 4.9109        | 14.22 | 16600 | 5.3815          |
| 4.9211        | 14.4  | 16800 | 5.3784          |
| 4.9342        | 14.57 | 17000 | 5.3735          |
| 4.9271        | 14.74 | 17200 | 5.3711          |
| 4.9328        | 14.91 | 17400 | 5.3646          |
| 4.8994        | 15.08 | 17600 | 5.3664          |
| 4.8932        | 15.25 | 17800 | 5.3642          |
| 4.8886        | 15.42 | 18000 | 5.3620          |
| 4.8997        | 15.6  | 18200 | 5.3584          |
| 4.8846        | 15.77 | 18400 | 5.3551          |
| 4.8993        | 15.94 | 18600 | 5.3516          |
| 4.8648        | 16.11 | 18800 | 5.3552          |
| 4.8838        | 16.28 | 19000 | 5.3512          |
| 4.8575        | 16.45 | 19200 | 5.3478          |
| 4.8623        | 16.62 | 19400 | 5.3480          |
| 4.8631        | 16.8  | 19600 | 5.3439          |
| 4.8576        | 16.97 | 19800 | 5.3428          |
| 4.8265        | 17.14 | 20000 | 5.3420          |
| 4.8523        | 17.31 | 20200 | 5.3410          |
| 4.8477        | 17.48 | 20400 | 5.3396          |
| 4.8507        | 17.65 | 20600 | 5.3380          |
| 4.8498        | 17.82 | 20800 | 5.3333          |
| 4.8261        | 17.99 | 21000 | 5.3342          |
| 4.8201        | 18.17 | 21200 | 5.3324          |
| 4.8214        | 18.34 | 21400 | 5.3341          |
| 4.8195        | 18.51 | 21600 | 5.3315          |
| 4.8216        | 18.68 | 21800 | 5.3335          |
| 4.8243        | 18.85 | 22000 | 5.3291          |
| 4.832         | 19.02 | 22200 | 5.3295          |
| 4.8085        | 19.19 | 22400 | 5.3309          |
| 4.8094        | 19.37 | 22600 | 5.3283          |
| 4.815         | 19.54 | 22800 | 5.3280          |
| 4.8219        | 19.71 | 23000 | 5.3270          |
| 4.8117        | 19.88 | 23200 | 5.3280          |


### Framework versions


- Transformers 4.13.0.dev0
- Pytorch 1.10.0
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: msft-regular-model


model author: mikaelsouza


task: text-generation


tags: pytorch, gpt2, text-generation, dataset:wikitext, transformers, generated_from_trainer, model-index


tags:


- conversational


# Anakin Skywalker DialoGPT Model


model name: DialoGPT-medium-Anakin


model author: DARKVIP3R


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language: "english"
license: "mit"
datasets:


- race
metrics:
- accuracy


# MCQ with Distilbert


model name: distilbert-qa


model author: russab0


tags: pytorch, distilbert, multiple-choice, english, dataset:race, transformers, license:mit, infinity_compatible


#Loki DialoGPT Model


model name: DialoGPT-small-loki


model author: Erikaka


# ChineseBERT-base


This repository contains code, model, dataset for **ChineseBERT** at ACL2021.


paper:\
**[ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://arxiv.org/abs/2106.16038)**\
*Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu and Jiwei Li*


code:\
[ChineseBERT github link](https://github.com/ShannonAI/ChineseBert)


## Model description


We propose ChineseBERT, which incorporates both the glyph and pinyin information of Chinese
characters into language model pretraining.


First, for each Chinese character, we get three kind of embedding.


- **Char Embedding:** the same as origin BERT token embedding.
- **Glyph Embedding:** capture visual features based on different fonts of a Chinese character.
- **Pinyin Embedding:** capture phonetic feature from the pinyin sequence ot a Chinese Character.


Then, char embedding, glyph embedding and pinyin embedding
are first concatenated, and mapped to a D-dimensional embedding through a fully
connected layer to form the fusion embedding.\
Finally, the fusion embedding is added with the position embedding, which is fed as input to the BERT model.\
The following image shows an overview architecture of ChineseBERT model.


ChineseBERT leverages the glyph and pinyin information of Chinese
characters to enhance the model's ability of capturing
context semantics from surface character forms and
disambiguating polyphonic characters in Chinese.


model name: ChineseBERT-base


model author: ShannonAI


tags: pytorch, arxiv:2106.16038


# ChineseBERT-large


model name: ChineseBERT-large


git lfs install
git clone https://huggingface.co/EleutherAI/gpt-j-6B


model name: Initialcallin


model author: Joejoe


tags:


KcELECTRA([https://github.com/Beomi/KcELECTRA](https://github.com/Beomi/KcELECTRA))의 Tokenizer에서 [UNK]로 대체되는 토큰들을 추가했습니다.


model name: bad-korean-tokenizer


model author: jiho0304


tags: electra, pretraining, transformers, infinity_compatible


ElectraBERT tuned with korean-bad-speeches


model name: curseELECTRA


task: text-classification


tags: pytorch, electra, text-classification, transformers, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: distilgpt2-finetuned-distilgpt2-med_articles
results:
  - task:
name: Causal Language Modeling
type: text-generation


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# distilgpt2-finetuned-distilgpt2-med_articles


This model is a fine-tuned version of [vishnun/distilgpt2-finetuned-distilgpt2-med_articles](https://huggingface.co/vishnun/distilgpt2-finetuned-distilgpt2-med_articles) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 3.3171


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 65   | 3.3417          |
| No log        | 2.0   | 130  | 3.3300          |
| No log        | 3.0   | 195  | 3.3231          |
| No log        | 4.0   | 260  | 3.3172          |
| No log        | 5.0   | 325  | 3.3171          |


### Framework versions


- Transformers 4.9.2
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


model name: distilgpt2-finetuned-distilgpt2-med_articles


model author: vishnun


task: text-generation


tags: pytorch, tensorboard, gpt2, text-generation, transformers, license:apache-2.0, generated_from_trainer


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: distilgpt2-finetuned-tamil-gpt
results:
  - task:
name: Causal Language Modeling
type: text-generation


# distilgpt2-finetuned-tamil-gpt


This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 4.4097


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 228  | 4.4097          |
| No log        | 2.0   | 456  | 4.4097          |
| 4.3169        | 3.0   | 684  | 4.4097          |
| 4.3169        | 4.0   | 912  | 4.4097          |
| 4.3116        | 5.0   | 1140 | 4.4097          |


### Framework versions


model name: distilgpt2-finetuned-tamil-gpt


model author: vishnun


task: text-generation


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- null
model_index:
- name: distilgpt2-finetuned-tamilmixsentiment
results:
  - task:
name: Causal Language Modeling
type: text-generation


# distilgpt2-finetuned-tamilmixsentiment


- Loss: 4.4572


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 5.6438        | 1.0   | 907  | 4.8026          |
| 4.774         | 2.0   | 1814 | 4.5953          |
| 4.5745        | 3.0   | 2721 | 4.5070          |
| 4.4677        | 4.0   | 3628 | 4.4688          |
| 4.4294        | 5.0   | 4535 | 4.4572          |


model name: distilgpt2-finetuned-tamilmixsentiment


# Tamil Mix Sentiment analysis


Model is trained on tamil-mix-sentiment dataset and finetuned with backend as bert-base-cased model


## Inference usage


On the hosted Inference type in the text for which you want to classify.
Eg: Super a iruku bro intha work, vera level mass


model name: bert-base-cased-tamil-mix-sentiment


task: text-classification


tags: pytorch, bert, text-classification, transformers, infinity_compatible


tags:


- generated_from_trainer
metrics:
- f1
model-index:
- name: rubert-base-srl
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# rubert-base-srl


This model is a fine-tuned version of [./ruBert-base/](https://huggingface.co/./ruBert-base/) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2429
- F1: 0.9563


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.06
- num_epochs: 10.0


### Training results


| Training Loss | Epoch | Step | Validation Loss | F1     |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 0.5816        | 1.0   | 57   | 0.3865          | 0.8371 |
| 0.3685        | 2.0   | 114  | 0.1707          | 0.9325 |
| 0.1057        | 3.0   | 171  | 0.0972          | 0.9563 |
| 0.0964        | 4.0   | 228  | 0.1429          | 0.9775 |
| 0.1789        | 5.0   | 285  | 0.2493          | 0.9457 |
| 0.0016        | 6.0   | 342  | 0.1900          | 0.6349 |
| 0.0013        | 7.0   | 399  | 0.2060          | 0.9563 |
| 0.0008        | 8.0   | 456  | 0.2321          | 0.9563 |
| 0.0006        | 9.0   | 513  | 0.2412          | 0.9563 |
| 0.0006        | 10.0  | 570  | 0.2429          | 0.9563 |


### Framework versions


- Transformers 4.13.0.dev0
- Pytorch 1.10.0+cu102
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: rubert-base-srl


model author: Rexhaif


task: text-classification


tags: pytorch, tensorboard, bert, text-classification, transformers, generated_from_trainer, model-index, infinity_compatible


tags:


- generated_from_trainer
model-index:
- name: rubert-base-srl-seqlabeling
results: []


# rubert-base-srl-seqlabeling


- Loss: 0.1537
- Causator Precision: 0.7677
- Causator Recall: 0.8352
- Causator F1: 0.8
- Causator Number: 91
- Expiriencer Precision: 0.9048
- Expiriencer Recall: 0.9694
- Expiriencer F1: 0.9360
- Expiriencer Number: 98
- Instrument Precision: 0.6
- Instrument Recall: 1.0
- Instrument F1: 0.7500
- Instrument Number: 6
- Other Precision: 0.0
- Other Recall: 0.0
- Other F1: 0.0
- Other Number: 1
- Predicate Precision: 0.9137
- Predicate Recall: 0.9845
- Predicate F1: 0.9478
- Predicate Number: 129
- Overall Precision: 0.8612
- Overall Recall: 0.9354
- Overall F1: 0.8968
- Overall Accuracy: 0.9661


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch | Step | Validation Loss | Causator Precision | Causator Recall | Causator F1 | Causator Number | Expiriencer Precision | Expiriencer Recall | Expiriencer F1 | Expiriencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Other Precision | Other Recall | Other F1 | Other Number | Predicate Precision | Predicate Recall | Predicate F1 | Predicate Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:---------------:|:------------:|:--------:|:------------:|:-------------------:|:----------------:|:------------:|:----------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.3043        | 1.0   | 56   | 0.3538          | 0.75               | 0.6264          | 0.6826      | 91              | 0.7981                | 0.8469             | 0.8218         | 98                 | 0.0                  | 0.0               | 0.0           | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.8741              | 0.9690           | 0.9191       | 129              | 0.8204            | 0.8154         | 0.8179     | 0.9142           |
| 0.2664        | 2.0   | 112  | 0.1961          | 0.8784             | 0.7143          | 0.7879      | 91              | 0.9175                | 0.9082             | 0.9128         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9398              | 0.9690           | 0.9542       | 129              | 0.9076            | 0.8769         | 0.8920     | 0.9399           |
| 0.0373        | 3.0   | 168  | 0.1275          | 0.8706             | 0.8132          | 0.8409      | 91              | 0.9223                | 0.9694             | 0.9453         | 98                 | 0.625                | 0.8333            | 0.7143        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9338              | 0.9845           | 0.9585       | 129              | 0.9066            | 0.9262         | 0.9163     | 0.9641           |
| 0.0496        | 4.0   | 224  | 0.1683          | 0.8                | 0.8352          | 0.8172      | 91              | 0.9143                | 0.9796             | 0.9458         | 98                 | 0.6667               | 1.0               | 0.8           | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9270              | 0.9845           | 0.9549       | 129              | 0.8815            | 0.9385         | 0.9091     | 0.9608           |
| 0.0529        | 5.0   | 280  | 0.1526          | 0.7917             | 0.8352          | 0.8128      | 91              | 0.8991                | 1.0                | 0.9469         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9203              | 0.9845           | 0.9513       | 129              | 0.8697            | 0.9446         | 0.9056     | 0.9627           |
| 0.0419        | 6.0   | 336  | 0.1402          | 0.7755             | 0.8352          | 0.8042      | 91              | 0.8962                | 0.9694             | 0.9314         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9203              | 0.9845           | 0.9513       | 129              | 0.8636            | 0.9354         | 0.8981     | 0.9651           |
| 0.0156        | 7.0   | 392  | 0.1498          | 0.8105             | 0.8462          | 0.8280      | 91              | 0.9048                | 0.9694             | 0.9360         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9137              | 0.9845           | 0.9478       | 129              | 0.8739            | 0.9385         | 0.9050     | 0.9661           |
| 0.0066        | 8.0   | 448  | 0.1509          | 0.7835             | 0.8352          | 0.8085      | 91              | 0.9057                | 0.9796             | 0.9412         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9137              | 0.9845           | 0.9478       | 129              | 0.8665            | 0.9385         | 0.9010     | 0.9680           |
| 0.0084        | 9.0   | 504  | 0.1548          | 0.7755             | 0.8352          | 0.8042      | 91              | 0.9048                | 0.9694             | 0.9360         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9137              | 0.9845           | 0.9478       | 129              | 0.8636            | 0.9354         | 0.8981     | 0.9656           |
| 0.0083        | 10.0  | 560  | 0.1537          | 0.7677             | 0.8352          | 0.8         | 91              | 0.9048                | 0.9694             | 0.9360         | 98                 | 0.6                  | 1.0               | 0.7500        | 6                 | 0.0             | 0.0          | 0.0      | 1            | 0.9137              | 0.9845           | 0.9478       | 129              | 0.8612            | 0.9354         | 0.8968     | 0.9661           |


### Framework versions


model name: rubert-base-srl-seqlabeling


model author: Rexhaif


task: token-classification


tags: pytorch, bert, token-classification, transformers, generated_from_trainer, model-index, autonlp_compatible, infinity_compatible


## About the model


The model has been trained on a collection of 500k articles with headings. Its purpose is to create a one-line heading suitable for the given article.


Sample code with a WikiNews article:


```python
import torch
from transformers import T5ForConditionalGeneration,T5Tokenizer

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = T5ForConditionalGeneration.from_pretrained("Michau/t5-base-en-generate-headline")
tokenizer = T5Tokenizer.from_pretrained("Michau/t5-base-en-generate-headline")
model = model.to(device)

article = '''
Very early yesterday morning, the United States President Donald Trump reported he and his wife First Lady Melania Trump tested positive for COVID-19. Officials said the Trumps' 14-year-old son Barron tested negative as did First Family and Senior Advisors Jared Kushner and Ivanka Trump.
Trump took to social media, posting at 12:54 am local time (0454 UTC) on Twitter, "Tonight, [Melania] and I tested positive for COVID-19. We will begin our quarantine and recovery process immediately. We will get through this TOGETHER!" Yesterday afternoon Marine One landed on the White House's South Lawn flying Trump to Walter Reed National Military Medical Center (WRNMMC) in Bethesda, Maryland.
Reports said both were showing "mild symptoms". Senior administration officials were tested as people were informed of the positive test. Senior advisor Hope Hicks had tested positive on Thursday.
Presidential physician Sean Conley issued a statement saying Trump has been given zinc, vitamin D, Pepcid and a daily Aspirin. Conley also gave a single dose of the experimental polyclonal antibodies drug from Regeneron Pharmaceuticals.
According to official statements, Trump, now operating from the WRNMMC, is to continue performing his duties as president during a 14-day quarantine. In the event of Trump becoming incapacitated, Vice President Mike Pence could take over the duties of president via the 25th Amendment of the US Constitution. The Pence family all tested negative as of yesterday and there were no changes regarding Pence's campaign events.
'''

text =  "headline: " + article

max_len = 256

encoding = tokenizer.encode_plus(text, return_tensors = "pt")
input_ids = encoding["input_ids"].to(device)
attention_masks = encoding["attention_mask"].to(device)

beam_outputs = model.generate(
    input_ids = input_ids,
    attention_mask = attention_masks,
    max_length = 64,
    num_beams = 3,
    early_stopping = True,
)

result = tokenizer.decode(beam_outputs[0])
print(result)
```


Result:


`Trump and First Lady Melania Test Positive for COVID-19`


model name: t5-base-en-generate-headline


model author: Michau


task: text2text-generation


tags: pytorch, tf, jax, t5, text2text-generation, transformers, autonlp_compatible


language: en
tags:


- splinter
- SplinterModel
license: apache-2.0


# Splinter base model


Splinter-base is the pretrained model discussed in the paper [Few-Shot Question Answering by Pretraining Span Selection](https://aclanthology.org/2021.acl-long.239/) (at ACL 2021). Its original repository can be found [here](https://github.com/oriram/splinter). The model is case-sensitive.


Note: This model **doesn't** contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model **with** those weights, see [tau/splinter-base-qass](https://huggingface.co/tau/splinter-base-qass).


## Model description


Splinter is a model that is pretrained in a self-supervised fashion for few-shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.


More precisely, it was pretrained with the Recurring Span Selection (RSS) objective, which emulates the span selection process involved in extractive question answering. Given a text, clusters of recurring spans (n-grams that appear more than once in the text) are first identified. For each such cluster, all of its instances but one are replaced with a special `[QUESTION]` token, and the model should select the correct (i.e., unmasked) span for each masked one. The model also defines the Question-Aware Span selection (QASS) layer, which selects spans conditioned on a specific question (in order to perform multiple predictions).


## Intended uses & limitations


The prime use for this model is few-shot extractive QA.


## Pretraining


The model was pretrained on a v3-8 TPU for 2.4M steps. The training data is based on **Wikipedia** and **BookCorpus**. See the paper for more details.


### BibTeX entry and citation info


```bibtex
@inproceedings{ram-etal-2021-shot,
    title = "Few-Shot Question Answering by Pretraining Span Selection",
    author = "Ram, Ori  and
      Kirstain, Yuval  and
      Berant, Jonathan  and
      Globerson, Amir  and
      Levy, Omer",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.239",
    doi = "10.18653/v1/2021.acl-long.239",
    pages = "3066--3079",
}
```


model name: splinter-base


model author: tau


task: question-answering


tags: pytorch, splinter, question-answering, en, transformers, SplinterModel, license:apache-2.0, autonlp_compatible


language: en
tags:


# Splinter large model


Splinter-large is the pretrained model discussed in the paper [Few-Shot Question Answering by Pretraining Span Selection](https://aclanthology.org/2021.acl-long.239/) (at ACL 2021). Its original repository can be found [here](https://github.com/oriram/splinter). The model is case-sensitive.


Note (1): This model **doesn't** contain the pretrained weights for the QASS layer (see paper for details), and therefore the QASS layer is randomly initialized upon loading it. For the model **with** those weights, see [tau/splinter-large-qass](https://huggingface.co/tau/splinter-large-qass).


Note (2): Splinter-large was trained after the paper was released, so the results are not reported. However, this model outperforms the base model by large margins. For example, on SQuAD, the model is able to reach 80% F1 given only 128 examples, whereas the base model obtains only ~73%). See the results for Splinter-large in the Appendix of [this paper](https://arxiv.org/pdf/2108.05857.pdf).


## Model description


The model was pretrained on a v3-32 TPU for 2.4M steps. The training data is based on **Wikipedia** and **BookCorpus**. See the paper for more details.


model name: splinter-large


tags: pytorch, splinter, question-answering, en, arxiv:2108.05857, transformers, SplinterModel, license:apache-2.0, autonlp_compatible


# Splinter base model (with pretrained QASS-layer weights)


Note: This model **does** contain the pretrained weights for the QASS layer (see paper for details). For the model **without** those weights, see [tau/splinter-base](https://huggingface.co/tau/splinter-base).


## Intended uses & limitations


## Pretraining


### BibTeX entry and citation info


```bibtex
@inproceedings{ram-etal-2021-shot,
    title = "Few-Shot Question Answering by Pretraining Span Selection",
    author = "Ram, Ori  and
      Kirstain, Yuval  and
      Berant, Jonathan  and
      Globerson, Amir  and
      Levy, Omer",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.239",
    doi = "10.18653/v1/2021.acl-long.239",
    pages = "3066--3079",
}

```


model name: splinter-base-qass


model author: tau


task: question-answering


We will soon upload Spider, proposed in the paper "Learning to Retrieve Passages without Supervision".


model name: spider


tags:


language: en
datasets:


- c4
- wikipedia
metrics:
- f1


# T5-V1.1-large-rss


This model is [T5-v1.1-large](https://huggingface.co/google/t5-v1_1-large) finetuned on RSS dataset. The model was finetuned as part of
["How Optimal is Greedy Decoding for Extractive Question Answering?"](https://arxiv.org/abs/2108.05857), while the RSS pretraining method was introduced in [this paper](https://arxiv.org/pdf/2101.00438.pdf).


## Model description


The original [T5-v1.1-large](https://huggingface.co/google/t5-v1_1-large) was only pre-trained on C4 excluding any supervised training. Our version is further trained on Rucurrent Span Selection scheme (RSS), using a sample from the dataset used to pretrain [Splinter](tau/splinter-large):


* contexts with a span occurring more than once are detected
* a single instance of the recurring span is maked
* the model is trained (teacher forcing) to predict the masked span
This training scheme naturally matches the extractive question answering task.


During training time, the masked span is replaced with `<extra_id_0>` and the labels are formatted as `<extra_id_0>span<extra_id_0>`. Unlike [Splinter](tau/splinter-large), only one span is mask at a time.


## Intended uses & limitations


This model naturally fits tasks where a span from a context is intended to be copied, like extractive question answering.
This checkpoint is primarily aimed to be used in zero-shot setting - further fine-tuning it on an annotated dataset gives equal results to those of the original T5-v1.1-large.


### How to use


You can use this model directly but it is recommended to format the input to be aligned with that of the training scheme and as a text-question context:


```python
from transformers import  AutoModelForSeq2SeqLM, AutoTokenizer
model = AutoModelForSeq2SeqLM.from_pretrained('tau/t5-v1_1-large-rss')
tokenizer = AutoTokenizer.from_pretrained('tau/t5-v1_1-large-rss')

passage = 'Barack Hussein Obama II is an American politician and attorney who served as the 44th president of the United States from 2009 to 2017. '
question = 'When was Obama inaugurated?'
text = f'Text: {passage}.\nQuestion: {question}\nAnswer:{tokenizer.additional_special_tokens[0]}.'
encoded_input = tokenizer(text, return_tensors='pt')
output_ids = model.generate(input_ids=encoded_input.input_ids, attention_mask=encoded_input.attention_mask,
               eos_token_id=tokenizer.additional_special_tokens_ids[1], num_beams=1, max_length=512, min_length=3)
tokenizer.decode(output_ids[0])
```


The generated answer is then `"<pad><extra_id_0> 2009<extra_id_1>"`, while the one generated by the original [T5-v1.1-large](https://huggingface.co/google/t5-v1_1-large) is `"<pad><extra_id_0> On January 20, 2009<extra_id_1>"` - a correct yet non-extractive answer.


### Limitations and bias


Although using the model with greedy decoding tends toward extracted outputs, is may sometimes produce non-extracted ones - may it be different casing or a whole different string (or substring) that may bear another semantic meaning.


### Pretraining


The model was finetuned with 100,000 rss-examples for 3 epochs using Adafactor optimizer with constant learning rate of 5e-5.


## Evaluation results


Evaluated over few-shot QA in a zero-shot setting (no finetuning on annotated examples):


|Model \ Dataset| SQuAD |TriviaQA | NaturalQs | NewsQA | SearchQA | HotpotQA | BioASQ | TextbookQA|
|:-------------:|:-----:|:-------:|:---------:|:------:|:--------:|:--------:|:------:|:---------:|
|T5             | 50.4  | 61.7    | 42.1      | 19.2   | 24.0     | 43.3     | 55.5   | 17.8      |
|T5-rss         | 71.4  | 69.3    | 57.2      | 43.2   | 29.7     | 59.0     | 65.5   | 39.0      |


The gap between the two models diminishes as more training examples are introduced, for additional result see the [paper]((https://arxiv.org/abs/2108.05857).


### BibTeX entry and citation info


```bibtex
@inproceedings{ram-etal-2021-shot,
    title = "Few-Shot Question Answering by Pretraining Span Selection",
    author = "Ram, Ori  and
      Kirstain, Yuval  and
      Berant, Jonathan  and
      Globerson, Amir  and
      Levy, Omer",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.239",
    doi = "10.18653/v1/2021.acl-long.239",
    pages = "3066--3079",
},
@misc{castel2021optimal,
      title={How Optimal is Greedy Decoding for Extractive Question Answering?},
      author={Or Castel and Ori Ram and Avia Efrat and Omer Levy},
      year={2021},
      eprint={2108.05857},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

```


model name: t5-v1_1-large-rss


model author: tau


task: text2text-generation


tags: pytorch, t5, text2text-generation, en, dataset:c4, dataset:wikipedia, arxiv:2108.05857, arxiv:2101.00438, transformers, autonlp_compatible


language: en
tags:


- splinter
- SplinterModel
license: apache-2.0


# Splinter large model, (with pretrained QASS-layer weights)


Splinter-large is the pretrained model discussed in the paper [Few-Shot Question Answering by Pretraining Span Selection](https://aclanthology.org/2021.acl-long.239/) (at ACL 2021). Its original repository can be found [here](https://github.com/oriram/splinter). The model is case-sensitive.


Note (1): This model **does** contain the pretrained weights for the QASS layer (see paper for details). For the model **without** those weights, see [tau/splinter-large](https://huggingface.co/tau/splinter-large).


Note (2): Splinter-large was trained after the paper was released, so the results are not reported. However, this model outperforms the base model by large margins. For example, on SQuAD, the model is able to reach 80% F1 given only 128 examples, whereas the base model obtains only ~73%). See the results for Splinter-large in the Appendix of [this paper](https://arxiv.org/pdf/2108.05857.pdf).


## Model description


Splinter is a model that is pretrained in a self-supervised fashion for few-shot question answering. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.


More precisely, it was pretrained with the Recurring Span Selection (RSS) objective, which emulates the span selection process involved in extractive question answering. Given a text, clusters of recurring spans (n-grams that appear more than once in the text) are first identified. For each such cluster, all of its instances but one are replaced with a special `[QUESTION]` token, and the model should select the correct (i.e., unmasked) span for each masked one. The model also defines the Question-Aware Span selection (QASS) layer, which selects spans conditioned on a specific question (in order to perform multiple predictions).


## Intended uses & limitations


The prime use for this model is few-shot extractive QA.


## Pretraining


The model was pretrained on a v3-32 TPU for 2.4M steps. The training data is based on **Wikipedia** and **BookCorpus**. See the paper for more details.


### BibTeX entry and citation info


```bibtex
@inproceedings{ram-etal-2021-shot,
    title = "Few-Shot Question Answering by Pretraining Span Selection",
    author = "Ram, Ori  and
      Kirstain, Yuval  and
      Berant, Jonathan  and
      Globerson, Amir  and
      Levy, Omer",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.239",
    doi = "10.18653/v1/2021.acl-long.239",
    pages = "3066--3079",
}
```


model name: splinter-large-qass


model author: tau


task: question-answering


tags: pytorch, splinter, question-answering, en, arxiv:2108.05857, transformers, SplinterModel, license:apache-2.0, autonlp_compatible


language: he
tags:


- roberta
- language model
datasets:
- oscar


# TavBERT base model


A Hebrew BERT-style masked language model operating over characters, pre-trained by masking spans of characters, similarly to SpanBERT (Joshi et al., 2020).


### How to use


```python
import numpy as np
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained("tau/tavbert-he")
tokenizer = AutoTokenizer.from_pretrained("tau/tavbert-he")

def mask_sentence(sent, span_len=5):
    start_pos = np.random.randint(0, len(sent) - span_len)
    masked_sent = sent[:start_pos] + '[MASK]' * span_len + sent[start_pos + span_len:]
    print("Masked sentence:", masked_sent)
    output = model(**tokenizer.encode_plus(masked_sent,
                                           return_tensors='pt'))['logits'][0][1:-1]
    preds = [int(x) for x in torch.argmax(torch.softmax(output, axis=1), axis=1)[start_pos:start_pos + span_len]]
    pred_sent = sent[:start_pos] + ''.join(tokenizer.convert_ids_to_tokens(preds)) + sent[start_pos + span_len:]
    print("Model's prediction:", pred_sent)
```


## Training data


OSCAR (Ortiz, 2019) Hebrew section (10 GB text, 20 million sentences).


model name: tavbert-he


task: fill-mask


tags: pytorch, roberta, fill-mask, he, dataset:oscar, transformers, language model, autonlp_compatible, infinity_compatible


language: ar
datasets:


- common_voice
- arabic_speech_corpus
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Mohammed XLSR Wav2Vec2 Large 53
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ar
type: common_voice
args: ar
metrics:
    - name: Test WER
type: wer
value: 36.69
    - name: Validation WER
type: wer
value: 36.69


# Wav2Vec2-Large-XLSR-53-Arabic


Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53)
on Arabic using the `train` splits of [Common Voice](https://huggingface.co/datasets/common_voice)
and [Arabic Speech Corpus](https://huggingface.co/datasets/arabic_speech_corpus).
When using this model, make sure that your speech input is sampled at 16kHz.


## Usage


The model can be used directly (without a language model) as follows:


```python
%%capture
!pip install datasets
!pip install transformers==4.4.0
!pip install torchaudio
!pip install jiwer
!pip install tnkeeh

import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ar", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("mohammed/ar")
model = Wav2Vec2ForCTC.from_pretrained("mohammed/ar")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("The predicted sentence is: ", processor.batch_decode(predicted_ids))
print("The original sentence is:", test_dataset["sentence"][:2])
```


The output is:


```
The predicted sentence is : ['ألديك قلم', 'ليست نارك مكسافة على هذه الأرض أبعد من يوم أمس']
The original sentence is: ['ألديك قلم ؟', 'ليست هناك مسافة على هذه الأرض أبعد من يوم أمس.']
```


## Evaluation


The model can be evaluated as follows on the Arabic test data of Common Voice:


```python

import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re
# creating a dictionary with all diacritics
dict = {
'ِ': '',
'ُ': '',
'ٓ': '',
'ٰ': '',
'ْ': '',
'ٌ': '',
'ٍ': '',
'ً': '',
'ّ': '',
'َ': '',
'~': '',
',': '',
'ـ': '',
'—': '',
'.': '',
'!': '',
'-': '',
';': '',
':': '',
'\'': '',
'"': '',
'☭': '',
'«': '',
'»': '',
'؛': '',
'ـ': '',
'_': '',
'،': '',
'“': '',
'%': '',
'‘': '',
'”': '',
'�': '',
'_': '',
',': '',
'?': '',
'#': '',
'‘': '',
'.': '',
'؛': '',
'get': '',
'؟': '',
'  ': ' ',
'\'ۖ ': '',
'\'': '',
 '\'ۚ' : '',
 ' \'': '',
 '31': '',
 '24': '',
 '39': ''
}

# replacing multiple diacritics using dictionary (stackoverflow is amazing)
def remove_special_characters(batch):
  # Create a regular expression  from the dictionary keys
  regex = re.compile("(%s)" % "|".join(map(re.escape, dict.keys())))
  # For each match, look-up corresponding value in dictionary
  batch["sentence"] = regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], batch["sentence"])
  return batch

test_dataset = load_dataset("common_voice", "ar", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("mohammed/ar")
model = Wav2Vec2ForCTC.from_pretrained("mohammed/ar")
model.to("cuda")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
test_dataset = test_dataset.map(remove_special_characters)
# Preprocessing the datasets.
# We need to read the audio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 36.69%


## Future Work


One can use *data augmentation*, *transliteration*, or *attention_mask* to increase the accuracy.


model name: ar


model author: mohammed


task: automatic-speech-recognition


tags: pytorch, jax, wav2vec2, automatic-speech-recognition, ar, dataset:common_voice, dataset:arabic_speech_corpus, transformers, audio, speech, xlsr-fine-tuning-week, license:apache-2.0, model-index


language: ar
datasets:


- common_voice
- arabic_speech_corpus
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Mohammed XLSR Wav2Vec2 Large 53
results:
  - task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice ar
type: common_voice
args: ar
metrics:
    - name: Test WER
type: wer
value: 36.699
    - name: Validation WER
type: wer
value: 36.699


# Wav2Vec2-Large-XLSR-53-Arabic


## Usage


```python
%%capture
!pip install datasets
!pip install transformers==4.4.0
!pip install torchaudio
!pip install jiwer
!pip install tnkeeh

import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ar", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("mohammed/wav2vec2-large-xlsr-arabic")
model = Wav2Vec2ForCTC.from_pretrained("mohammed/wav2vec2-large-xlsr-arabic")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("The predicted sentence is: ", processor.batch_decode(predicted_ids))
print("The original sentence is:", test_dataset["sentence"][:2])
```


The output is:


```python

import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re
# creating a dictionary with all diacritics
dict = {
'ِ': '',
'ُ': '',
'ٓ': '',
'ٰ': '',
'ْ': '',
'ٌ': '',
'ٍ': '',
'ً': '',
'ّ': '',
'َ': '',
'~': '',
',': '',
'ـ': '',
'—': '',
'.': '',
'!': '',
'-': '',
';': '',
':': '',
'\'': '',
'"': '',
'☭': '',
'«': '',
'»': '',
'؛': '',
'ـ': '',
'_': '',
'،': '',
'“': '',
'%': '',
'‘': '',
'”': '',
'�': '',
'_': '',
',': '',
'?': '',
'#': '',
'‘': '',
'.': '',
'؛': '',
'get': '',
'؟': '',
'  ': ' ',
'\'ۖ ': '',
'\'': '',
 '\'ۚ' : '',
 ' \'': '',
 '31': '',
 '24': '',
 '39': ''
}

# replacing multiple diacritics using dictionary (stackoverflow is amazing)
def remove_special_characters(batch):
  # Create a regular expression  from the dictionary keys
  regex = re.compile("(%s)" % "|".join(map(re.escape, dict.keys())))
  # For each match, look-up corresponding value in dictionary
  batch["sentence"] = regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], batch["sentence"])
  return batch

test_dataset = load_dataset("common_voice", "ar", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("mohammed/wav2vec2-large-xlsr-arabic")
model = Wav2Vec2ForCTC.from_pretrained("mohammed/wav2vec2-large-xlsr-arabic")
model.to("cuda")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
test_dataset = test_dataset.map(remove_special_characters)
# Preprocessing the datasets.
# We need to read the audio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)
    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```


**Test Result**: 36.699%


model name: wav2vec2-large-xlsr-arabic


tags:


- flair
- token-classification
widget:
- text: "does this work"


## Test model README


Some test README description


model name: flairmodel


model author: menciusds


task: token-classification


tags: pytorch, flair, token-classification


This model is for anyone using using Flux.jl and looking for a test model to make sue of the Hugging Face hub. You can see the source code to generate this model below:


```Julia
julia> using Flux

julia> model = Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)

julia> using BSON: @save

julia> @save "mymodel.bson" model
```


you can then load the model in Julia as follows:


```Julia
julia> using Flux

julia> using BSON: @load

julia> @load "mymodel.bson" model

julia> model
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
```


See here: https://fluxml.ai/Flux.jl/stable/saving/#Saving-and-Loading-Models for more details!


model name: BasicFluxjlModel


model author: LoganKilpatrick


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- wmt16
model-index:
- name: opus-mt-en-de-finetuned-en-to-de-wd01-fp16false
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# opus-mt-en-de-finetuned-en-to-de-wd01-fp16false


This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-de](https://huggingface.co/Helsinki-NLP/opus-mt-en-de) on the wmt16 dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: opus-mt-en-de-finetuned-en-to-de-wd01-fp16false


model author: fabiogr


task: text2text-generation


tags: pytorch, marian, text2text-generation, dataset:wmt16, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


language: pt
tags:


- electra
- pretraining
- pytorch
datasets:
- brwac


model name: electra-base-portuguese-uncased-brwac


model author: dlb


tags: pytorch, pt, dataset:brwac, transformers, electra, pretraining


Ner dataset sourced from https://huggingface.co/datasets/id_nergrit_corpus


This model is built by fine-tuning BERT Transformers with an accuracy gain of 92.61% with an F1-Score value of 74.80%


model name: bert-id-ner


model author: liandarizkia


task: fill-mask


tags: pytorch, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


# GFPGAN (CVPR 2021)


[**Paper**](https://arxiv.org/abs/2101.04061) **|** [**Project Page**](https://xinntao.github.io/projects/gfpgan)    [English](README.md) **|** [简体中文](README_CN.md)


GFPGAN is a blind face restoration algorithm towards real-world face images.


[Colab Demo](https://colab.research.google.com/drive/1sVsoBd9AjckIXThgtZhGrHRfFI6UUYOo)


### :book: GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior


> [[Paper](https://arxiv.org/abs/2101.04061)]   [[Project Page](https://xinntao.github.io/projects/gfpgan)]   [Demo]

[Xintao Wang](https://xinntao.github.io/), [Yu Li](https://yu-li.github.io/), [Honglun Zhang](https://scholar.google.com/citations?hl=en&user=KjQLROoAAAAJ), [Ying Shan](https://scholar.google.com/citations?user=4oXBp9UAAAAJ&hl=en)

Applied Research Center (ARC), Tencent PCG


#### Abstract


Blind face restoration usually relies on facial priors, such as facial geometry prior or reference prior, to restore realistic and faithful details. However, very low-quality inputs cannot offer accurate geometric prior while high-quality references are inaccessible, limiting the applicability in real-world scenarios. In this work, we propose GFP-GAN that leverages **rich and diverse priors encapsulated in a pretrained face GAN** for blind face restoration. This Generative Facial Prior (GFP) is incorporated into the face restoration process via novel channel-split spatial feature transform layers, which allow our method to achieve a good balance of realness and fidelity. Thanks to the powerful generative facial prior and delicate designs, our GFP-GAN could jointly restore facial details and enhance colors with just a single forward pass, while GAN inversion methods require expensive image-specific optimization at inference. Extensive experiments show that our method achieves superior performance to prior art on both synthetic and real-world datasets.


#### BibTeX


    @InProceedings{wang2021gfpgan,
        author = {Xintao Wang and Yu Li and Honglun Zhang and Ying Shan},
        title = {Towards Real-World Blind Face Restoration with Generative Facial Prior},
        booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
        year = {2021}
    }


## :wrench: Dependencies and Installation


- Python >= 3.7 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))
- [PyTorch >= 1.7](https://pytorch.org/)
- NVIDIA GPU + [CUDA](https://developer.nvidia.com/cuda-downloads)


### Installation


1. Clone repo

   ```bash
   git clone https://github.com/xinntao/GFPGAN.git
   cd GFPGAN
   ```
1. Install dependent packages

   ```bash
   # Install basicsr - https://github.com/xinntao/BasicSR
   # We use BasicSR for both training and inference
   # Set BASICSR_EXT=True to compile the cuda extensions in the BasicSR - It may take several minutes to compile, please be patient
   BASICSR_EXT=True pip install basicsr

   # Install facexlib - https://github.com/xinntao/facexlib
   # We use face detection and face restoration helper in the facexlib package
   pip install facexlib

   pip install -r requirements.txt
   ```


## :zap: Quick Inference


Download pre-trained models: [GFPGANv1.pth](https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/GFPGANv1.pth)


```bash
wget https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/GFPGANv1.pth -P experiments/pretrained_models
```


```bash
python inference_gfpgan_full.py --model_path experiments/pretrained_models/GFPGANv1.pth --test_path inputs/whole_imgs

# for aligned images
python inference_gfpgan_full.py --model_path experiments/pretrained_models/GFPGANv1.pth --test_path inputs/cropped_faces --aligned
```


## :computer: Training


We provide complete training codes for GFPGAN. <br>
You could improve it according to your own needs.


1. Dataset preparation: [FFHQ](https://github.com/NVlabs/ffhq-dataset)
1. Download pre-trained models and other data. Put them in the `experiments/pretrained_models` folder.
   1. [Pretrained StyleGAN2 model: StyleGAN2_512_Cmul1_FFHQ_B12G4_scratch_800k.pth](https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/StyleGAN2_512_Cmul1_FFHQ_B12G4_scratch_800k.pth)
   1. [Component locations of FFHQ: FFHQ_eye_mouth_landmarks_512.pth](https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/FFHQ_eye_mouth_landmarks_512.pth)
   1. [A simple ArcFace model: arcface_resnet18.pth](https://github.com/TencentARC/GFPGAN/releases/download/v0.1.0/arcface_resnet18.pth)
1. Modify the configuration file `train_gfpgan_v1.yml` accordingly.
1. Training


> python -m torch.distributed.launch --nproc_per_node=4 --master_port=22021 train.py -opt train_gfpgan_v1.yml --launcher pytorch


## :scroll: License and Acknowledgement


GFPGAN is realeased under Apache License Version 2.0.


## :e-mail: Contact


If you have any question, please email `xintao.wang@outlook.com` or `xintaowang@tencent.com`.


model name: GFPGANv1


model author: TencentARC


tags: arxiv:2101.04061


widget:


- text: "मुझे उनसे बात करना <mask> अच्छा लगा"
- text: "हम आपके सुखद <mask> की कामना करते हैं"
- text: "सभी अच्छी चीजों का एक <mask> होता है"


# RoBERTa base model for Hindi language


Pretrained model on Hindi language using a masked language modeling (MLM) objective. [A more interactive & comparison demo is available here](https://huggingface.co/spaces/flax-community/roberta-hindi).


> This is part of the
[Flax/Jax Community Week](https://discuss.huggingface.co/t/pretrain-roberta-from-scratch-in-hindi/7091), organized by [Hugging Face](https://huggingface.co/) and TPU usage sponsored by Google.


## Model description


RoBERTa Hindi is a transformers model pretrained on a large corpus of Hindi data(a combination of **mc4, oscar and indic-nlp** datasets)


### How to use


You can use this model directly with a pipeline for masked language modeling:


```python
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='flax-community/roberta-hindi')
>>> unmasker("हम आपके सुखद <mask> की कामना करते हैं")
[{'score': 0.3310680091381073,
  'sequence': 'हम आपके सुखद सफर की कामना करते हैं',
  'token': 1349,
  'token_str': ' सफर'},
 {'score': 0.15317578613758087,
  'sequence': 'हम आपके सुखद पल की कामना करते हैं',
  'token': 848,
  'token_str': ' पल'},
 {'score': 0.07826550304889679,
  'sequence': 'हम आपके सुखद समय की कामना करते हैं',
  'token': 453,
  'token_str': ' समय'},
 {'score': 0.06304813921451569,
  'sequence': 'हम आपके सुखद पहल की कामना करते हैं',
  'token': 404,
  'token_str': ' पहल'},
 {'score': 0.058322224766016006,
  'sequence': 'हम आपके सुखद अवसर की कामना करते हैं',
  'token': 857,
  'token_str': ' अवसर'}]
```


## Training data


The RoBERTa Hindi model was pretrained on the reunion of the following datasets:


- [OSCAR](https://huggingface.co/datasets/oscar) is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.
- [mC4](https://huggingface.co/datasets/mc4) is a multilingual colossal, cleaned version of Common Crawl's web crawl corpus.
- [IndicGLUE](https://indicnlp.ai4bharat.org/indic-glue/) is a natural language understanding benchmark.
- [Samanantar](https://indicnlp.ai4bharat.org/samanantar/) is a parallel corpora collection for Indic language.
- [Hindi Text Short and Large Summarization Corpus](https://www.kaggle.com/disisbig/hindi-text-short-and-large-summarization-corpus) is a collection of ~180k articles with their headlines and summary collected from Hindi News Websites.
- [Hindi Text Short Summarization Corpus](https://www.kaggle.com/disisbig/hindi-text-short-summarization-corpus) is a collection of ~330k articles with their headlines collected from Hindi News Websites.
- [Old Newspapers Hindi](https://www.kaggle.com/crazydiv/oldnewspapershindi) is a cleaned subset of HC Corpora newspapers.


## Training procedure


### Preprocessing


The texts are tokenized using a byte version of Byte-Pair Encoding (BPE) and a vocabulary size of 50265. The inputs of
the model take pieces of 512 contiguous token that may span over documents. The beginning of a new document is marked
with `<s>` and the end of one by `</s>`.


- We had to perform cleanup of **mC4** and **oscar** datasets by removing all non hindi (non Devanagari) characters from the datasets.
- We tried to filter out evaluation set of WikiNER of [IndicGlue](https://indicnlp.ai4bharat.org/indic-glue/) benchmark by [manual labelling](https://github.com/amankhandelia/roberta_hindi/blob/master/wikiner_incorrect_eval_set.csv)  where the actual labels were not correct and modifying the [downstream evaluation dataset](https://github.com/amankhandelia/roberta_hindi/blob/master/utils.py).


The details of the masking procedure for each sentence are the following:


- 15% of the tokens are masked.
- In 80% of the cases, the masked tokens are replaced by `<mask>`.
- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace.
- In the 10% remaining cases, the masked tokens are left as is.
Contrary to BERT, the masking is done dynamically during pretraining (e.g., it changes at each epoch and is not fixed).


### Pretraining


The model was trained on Google Cloud Engine TPUv3-8 machine (with 335 GB of RAM, 1000 GB of hard drive, 96 CPU cores).A randomized shuffle of combined dataset of **mC4, oscar** and other datasets listed above was used to train the model. Training logs are present in [wandb](https://wandb.ai/wandb/hf-flax-roberta-hindi).


## Evaluation Results


RoBERTa Hindi is evaluated on various downstream tasks. The results are summarized below.


| Task                    | Task Type            | IndicBERT | HindiBERTa | Indic Transformers Hindi BERT | RoBERTa Hindi Guj San | RoBERTa Hindi |
|-------------------------|----------------------|-----------|------------|-------------------------------|-----------------------|---------------|
| BBC News Classification | Genre Classification | **76.44**     | 66.86      | **77.6**                          | 64.9                  | 73.67         |
| WikiNER                 | Token Classification | -         | 90.68      | **95.09**                         | 89.61                 | **92.76**         |
| IITP Product Reviews    | Sentiment Analysis   | **78.01**     | 73.23      | **78.39**                         | 66.16                 | 75.53         |
| IITP Movie Reviews      | Sentiment Analysis   | 60.97     | 52.26      | **70.65**                         | 49.35                 | **61.29**         |


## Team Members


- Aman K ([amankhandelia](https://huggingface.co/amankhandelia))
- Haswanth Aekula ([hassiahk](https://huggingface.co/hassiahk))
- Kartik Godawat ([dk-crazydiv](https://huggingface.co/dk-crazydiv))
- Prateek Agrawal ([prateekagrawal](https://huggingface.co/prateekagrawal))
- Rahul Dev ([mlkorra](https://huggingface.co/mlkorra))


## Credits


Huge thanks to Hugging Face 🤗 & Google Jax/Flax team for such a wonderful community week, especially for providing such massive computing resources. Big thanks to [Suraj Patil](https://huggingface.co/valhalla) & [Patrick von Platen](https://huggingface.co/patrickvonplaten) for mentoring during the whole week.


<img src=https://pbs.twimg.com/media/E443fPjX0AY1BsR.jpg:medium>


model name: panini


model author: amankhandelia


task: fill-mask


tags: roberta, fill-mask, transformers, autonlp_compatible, infinity_compatible


license: mit
tags:


- generated_from_trainer
datasets:
- amazon_reviews_multi
model-index:
- name: xlm-roberta-base-finetuned-marc
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# xlm-roberta-base-finetuned-marc


This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the amazon_reviews_multi dataset.
It achieves the following results on the evaluation set:


- Loss: 0.9611
- Mae: 0.4749


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2


### Training results


| Training Loss | Epoch | Step | Validation Loss | Mae    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 1.0431        | 1.0   | 860  | 0.9819          | 0.4985 |
| 0.9079        | 2.0   | 1720 | 0.9611          | 0.4749 |


### Framework versions


- Transformers 4.11.3
- Pytorch 1.9.0+cu111
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: xlm-roberta-base-finetuned-marc


model author: claudio75


task: text-classification


tags: pytorch, tensorboard, xlm-roberta, text-classification, dataset:amazon_reviews_multi, transformers, license:mit, generated_from_trainer, model-index


metrics:


- accuracy
widget:
- text: "In war resolution, in defeat defiance, in victory magnanimity"
- text: "en la guerra resolución en la derrota desafío en la victoria magnanimidad"


[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1dqeUwS_DZ-urrmYzB29nTCBUltwJxhbh?usp=sharing)


# 22 Language Identifier - BERT


This model is trained to identify the following 22 different languages.


- Arabic
- Chinese
- Dutch
- English
- Estonian
- French
- Hindi
- Indonesian
- Japanese
- Korean
- Latin
- Persian
- Portugese
- Pushto
- Romanian
- Russian
- Spanish
- Swedish
- Tamil
- Thai
- Turkish
- Urdu


## Loading the model


```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("SharanSMenon/22-languages-bert-base-cased")

model = AutoModelForSequenceClassification.from_pretrained("SharanSMenon/22-languages-bert-base-cased")
```


## Inference


```python
def predict(sentence):
  tokenized = tokenizer(sentence, return_tensors="pt")
  outputs = model(**tokenized)
  return model.config.id2label[outputs.logits.argmax(dim=1).item()]
```


### Examples


```python
sentence1 = "in war resolution, in defeat defiance, in victory magnanimity"
predict(sentence1) # English

sentence2 = "en la guerra resolución en la derrota desafío en la victoria magnanimidad"
predict(sentence2) # Spanish

sentence3 = "هذا هو أعظم إله على الإطلاق"
predict(sentence3) # Arabic
```


model name: 22-languages-bert-base-cased


model author: SharanSMenon


tags: pytorch, bert, text-classification, transformers, infinity_compatible


language: en


widget:


- text: "USER USER USER USER لاحول ولاقوه الا بالله 💔 💔 💔 💔 HASH TAG متي يصدر قرار العشرين ! ! ! ! ! !"


Sub 1


model name: sa_sub1


model author: researchaccount


task: text-classification


tags: pytorch, jax, bert, text-classification, en, transformers, infinity_compatible


Sub 4


model name: sa_sub4


Sub 2


model name: sa_sub2


Sub 3


model name: sa_sub3


language: ro


# ALBert


The ALR-Bert , **cased** model for Romanian, trained on a 15GB corpus!
ALR-BERT is a multi-layer bidirectional Transformer encoder that shares ALBERT's factorized embedding parameterization and cross-layer sharing. ALR-BERT-base inherits ALBERT-base and features 12 parameter-sharing layers, a 128-dimension embedding size, 768 hidden units, 12 heads, and GELU non-linearities. Masked language modeling (MLM) and sentence order prediction (SOP) losses are the two objectives that ALBERT is pre-trained on. For ALR-BERT, we preserve both these objectives.
The model was trained using 40 batches per GPU (for 128 sequence length) and then 20 batches per GPU (for 512 sequence length). Layer-wise Adaptive Moments optimizer for Batch (LAMB) training was utilized, with a warm-up over the first 1\% of steps up to a learning rate of 1e4, then a decay. Eight NVIDIA Tesla V100 SXM3 with 32GB memory were used, and the pre-training process took around 2 weeks per model.


Training methodology follows closely work previous done in Romanian Bert (https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1)


### How to use


```python

from transformers import AutoTokenizer, AutoModel

import torch

# load tokenizer and model

tokenizer = AutoTokenizer.from_pretrained("dragosnicolae555/ALR_BERT")

model = AutoModel.from_pretrained("dragosnicolae555/ALR_BERT")

#Here add your magic

```


Remember to always sanitize your text! Replace `s` and `t` cedilla-letters to comma-letters with :


```
text = text.replace("ţ", "ț").replace("ş", "ș").replace("Ţ", "Ț").replace("Ş", "Ș")
```


because the model was **NOT** trained on cedilla `s` and `t`s. If you don't, you will have decreased performance due to <UNK>s and increased number of tokens per word.


### Evaluation


Here, we evaluate ALR-BERT on Simple Universal Dependencies task. One model for each task, evaluating labeling performance on the UPOS (Universal Part-of-Speech) and the XPOS (Extended Part-of-Speech) (eXtended Part-of-Speech). We compare our proposed ALR-BERT with Romanian BERT and multiligual BERT, using the cased version. To counteract the random seed effect, we repeat each experiment five times and simply provide the mean score.


| Model                          |  UPOS |  XPOS  |  MLAS  |  AllTags  |
|--------------------------------|:-----:|:------:|:-----:|:-----:|
| M-BERT (cased)  | 93.87 | 89.89 | 90.01  | 87.04|
| Romanian BERT (cased)    |  95.56 |  95.35 |  92.78 |  93.22 |
| ALR-BERT (cased)   |  **87.38** |  **84.05** |  **79.82** |  **78.82**|


### Corpus


The model is trained on the following corpora (stats in the table below are after cleaning):


| Corpus    	| Lines(M) 	| Words(M) 	| Chars(B) 	| Size(GB) 	|
|-----------	|:--------:	|:--------:	|:--------:	|:--------:	|
| OPUS      	|   55.05  	|  635.04  	|   4.045  	|    3.8   	|
| OSCAR     	|   33.56  	|  1725.82 	|  11.411  	|    11    	|
| Wikipedia 	|   1.54   	|   60.47  	|   0.411  	|    0.4   	|
| **Total**     	|   **90.15**  	|  **2421.33** 	|  **15.867**  	|   **15.2**   	|


model name: ALR_BERT


model author: dragosnicolae555


task: fill-mask


tags: pytorch, albert, fill-mask, ro, transformers, autonlp_compatible, infinity_compatible


#### Table of contents


1. [Introduction](#introduction)
1. [Pretrain model](#models)
1. [Using SimeCSE_Vietnamese with `sentences-transformers`](#sentences-transformers)
   - [Installation](#install1)
   - [Example usage](#usage1)
1. [Using SimeCSE_Vietnamese with `transformers`](#transformers)
   - [Installation](#install2)
   - [Example usage](#usage2)


#  SimeCSE_Vietnamese: Simple Contrastive Learning of Sentence Embeddings with Vietnamese


Pre-trained SimeCSE_Vietnamese models are the state-of-the-art of Sentence Embeddings with Vietnamese :


- SimeCSE_Vietnamese pre-training approach is based on [SimCSE](https://arxiv.org/abs/2104.08821) which optimizes the SimeCSE_Vietnamese pre-training procedure for more robust performance.
- SimeCSE_Vietnamese encode input sentences using a pre-trained language model such as  [PhoBert](https://www.aclweb.org/anthology/2020.findings-emnlp.92/)
- SimeCSE_Vietnamese works with both unlabeled and labeled data.


## Pre-trained models


Model | #params | Arch.
---|---|---
[`VoVanPhuc/sup-SimCSE-VietNamese-phobert-base`](https://huggingface.co/VoVanPhuc/sup-SimCSE-VietNamese-phobert-base) | 135M | base
[`VoVanPhuc/unsup-SimCSE-VietNamese-phobert-base`](https://huggingface.co/VoVanPhuc/unsup-SimCSE-VietNamese-phobert-base) | 135M | base


##  Using SimeCSE_Vietnamese with `sentences-transformers`


### Installation


- Install `sentence-transformers`:

  - `pip install -U sentence-transformers`
- Install `pyvi` to word segment:

  - `pip install pyvi`


### Example usage


```python
from sentence_transformers import SentenceTransformer
from pyvi.ViTokenizer import tokenize

model = SentenceTransformer('VoVanPhuc/sup-SimCSE-VietNamese-phobert-base')

sentences = ['Kẻ đánh bom đinh tồi tệ nhất nước Anh.',
          'Nghệ sĩ làm thiện nguyện - minh bạch là việc cấp thiết.',
          'Bắc Giang tăng khả năng điều trị và xét nghiệm.',
          'HLV futsal Việt Nam tiết lộ lý do hạ Lebanon.',
          'việc quan trọng khi kêu gọi quyên góp từ thiện là phải minh bạch, giải ngân kịp thời.',
          '20% bệnh nhân Covid-19 có thể nhanh chóng trở nặng.',
          'Thái Lan thua giao hữu trước vòng loại World Cup.',
          'Cựu tuyển thủ Nguyễn Bảo Quân: May mắn ủng hộ futsal Việt Nam',
          'Chủ ki-ốt bị đâm chết trong chợ đầu mối lớn nhất Thanh Hoá.',
          'Bắn chết người trong cuộc rượt đuổi trên sông.'
          ]

sentences = [tokenize(sentence) for sentence in sentences]
embeddings = model.encode(sentences)
```


##  Using SimeCSE_Vietnamese with `transformers`


- Install `transformers`:

  - `pip install -U transformers`
- Install `pyvi` to word segment:

  - `pip install pyvi`


```python
import torch
from transformers import AutoModel, AutoTokenizer
from pyvi.ViTokenizer import tokenize

PhobertTokenizer = AutoTokenizer.from_pretrained("VoVanPhuc/sup-SimCSE-VietNamese-phobert-base")
model = AutoModel.from_pretrained("VoVanPhuc/sup-SimCSE-VietNamese-phobert-base")

sentences = ['Kẻ đánh bom đinh tồi tệ nhất nước Anh.',
          'Nghệ sĩ làm thiện nguyện - minh bạch là việc cấp thiết.',
          'Bắc Giang tăng khả năng điều trị và xét nghiệm.',
          'HLV futsal Việt Nam tiết lộ lý do hạ Lebanon.',
          'việc quan trọng khi kêu gọi quyên góp từ thiện là phải minh bạch, giải ngân kịp thời.',
          '20% bệnh nhân Covid-19 có thể nhanh chóng trở nặng.',
          'Thái Lan thua giao hữu trước vòng loại World Cup.',
          'Cựu tuyển thủ Nguyễn Bảo Quân: May mắn ủng hộ futsal Việt Nam',
          'Chủ ki-ốt bị đâm chết trong chợ đầu mối lớn nhất Thanh Hoá.',
          'Bắn chết người trong cuộc rượt đuổi trên sông.'
          ]

sentences = [tokenize(sentence) for sentence in sentences]

inputs = PhobertTokenizer(sentences, padding=True, truncation=True, return_tensors="pt")

with torch.no_grad():
    embeddings = model(**inputs, output_hidden_states=True, return_dict=True).pooler_output
```


## Quick Start


[Open In Colab](https://colab.research.google.com/drive/12__EXJoQYHe9nhi4aXLTf9idtXT8yr7H?usp=sharing)


## Citation


    @article{gao2021simcse,
       title={{SimCSE}: Simple Contrastive Learning of Sentence Embeddings},
       author={Gao, Tianyu and Yao, Xingcheng and Chen, Danqi},
       journal={arXiv preprint arXiv:2104.08821},
       year={2021}
    }

    @inproceedings{phobert,
    title     = {{PhoBERT: Pre-trained language models for Vietnamese}},
    author    = {Dat Quoc Nguyen and Anh Tuan Nguyen},
    booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
    year      = {2020},
    pages     = {1037--1042}
    }


model name: unsup-SimCSE-VietNamese-phobert-base


model author: VoVanPhuc


tags: pytorch, roberta, arxiv:2104.08821, transformers, infinity_compatible


#### Table of contents


## Pre-trained models


model name: sup-SimCSE-VietNamese-phobert-base


language: en
thumbnail:
tags:


- pytorch
- text-classification
datasets:
- MNLI


# bert-base-uncased finetuned on MNLI


## Model Details and Training Data


We used the pretrained model from [bert-base-uncased](https://huggingface.co/bert-base-uncased) and finetuned it on [MultiNLI](https://cims.nyu.edu/~sbowman/multinli/) dataset.


The training parameters were kept the same as [Devlin et al., 2019](https://arxiv.org/abs/1810.04805) (learning rate = 2e-5, training epochs = 3, max_sequence_len = 128 and batch_size = 32).


## Evaluation Results


The evaluation results are mentioned in the table below.


| Test Corpus | Accuracy |
|:---:|:---------:|
| Matched | 0.8456 |
| Mismatched | 0.8484 |


model name: bert-base-uncased-mnli


model author: ishan


task: text-classification


tags: pytorch, jax, bert, text-classification, en, dataset:MNLI, arxiv:1810.04805, transformers, infinity_compatible


# distilbert-base-uncased finetuned on MNLI


We used the pretrained model from [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) and finetuned it on [MultiNLI](https://cims.nyu.edu/~sbowman/multinli/) dataset.


| Test Corpus | Accuracy |
|:---:|:---------:|
| Matched | 0.8223 |
| Mismatched | 0.8216 |


model name: distilbert-base-uncased-mnli


tags: pytorch, distilbert, text-classification, en, dataset:MNLI, arxiv:1810.04805, transformers, infinity_compatible


tags:


- conversational


# Harry Potter DialoGPT Model


model name: DialoGPT-small-harrypotter


model author: Kai0857


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


- spacy
- token-classification
language:
- en
model-index:
- name: en_pipeline
results:
  - task:
name: NER
type: token-classification
metrics:
    - name: NER Precision
type: precision
value: 0.9946896173
    - name: NER Recall
type: recall
value: 0.9916932907
    - name: NER F Score
type: f_score
value: 0.9931891941
  - task:
name: POS
type: token-classification
metrics:
    - name: POS Accuracy
type: accuracy
value: 0.0
  - task:
name: SENTER
type: token-classification
metrics:
    - name: SENTER Precision
type: precision
value: 1.0
    - name: SENTER Recall
type: recall
value: 1.0
    - name: SENTER F Score
type: f_score
value: 1.0
  - task:
name: UNLABELED_DEPENDENCIES
type: token-classification
metrics:
    - name: Unlabeled Dependencies Accuracy
type: accuracy
value: 0.0
  - task:
name: LABELED_DEPENDENCIES
type: token-classification
metrics:
    - name: Labeled Dependencies Accuracy
type: accuracy
value: 0.0


| Feature | Description |
| --- | --- |
| **Name** | `en_pipeline` |
| **Version** | `0.0.0` |
| **spaCy** | `>=3.1.0,<3.2.0` |
| **Default Pipeline** | `tok2vec`, `tagger`, `parser`, `ner`, `attribute_ruler`, `lemmatizer` |
| **Components** | `tok2vec`, `tagger`, `parser`, `ner`, `attribute_ruler`, `lemmatizer` |
| **Vectors** | 0 keys, 0 unique vectors (0 dimensions) |
| **Sources** | n/a |
| **License** | n/a |
| **Author** | [n/a]() |


### Label Scheme


<details>


<summary>View label scheme (114 labels for 3 components)</summary>


| Component | Labels |
| --- | --- |
| **`tagger`** | `$`, `''`, `,`, `-LRB-`, `-RRB-`, `.`, `:`, `ADD`, `AFX`, `CC`, `CD`, `DT`, `EX`, `FW`, `HYPH`, `IN`, `JJ`, `JJR`, `JJS`, `LS`, `MD`, `NFP`, `NN`, `NNP`, `NNPS`, `NNS`, `PDT`, `POS`, `PRP`, `PRP$`, `RB`, `RBR`, `RBS`, `RP`, `SYM`, `TO`, `UH`, `VB`, `VBD`, `VBG`, `VBN`, `VBP`, `VBZ`, `WDT`, `WP`, `WP$`, `WRB`, `XX`, ```` |
| **`parser`** | `ROOT`, `acl`, `acomp`, `advcl`, `advmod`, `agent`, `amod`, `appos`, `attr`, `aux`, `auxpass`, `case`, `cc`, `ccomp`, `compound`, `conj`, `csubj`, `csubjpass`, `dative`, `dep`, `det`, `dobj`, `expl`, `intj`, `mark`, `meta`, `neg`, `nmod`, `npadvmod`, `nsubj`, `nsubjpass`, `nummod`, `oprd`, `parataxis`, `pcomp`, `pobj`, `poss`, `preconj`, `predet`, `prep`, `prt`, `punct`, `quantmod`, `relcl`, `xcomp` |
| **`ner`** | `ARC`, `AST`, `BOOK`, `CAUSAL`, `COMPARISON`, `DATE`, `HEM`, `HOUR`, `HYPO`, `INSTRUMENT`, `JUDGEMENT`, `LAWS`, `MODEL`, `NAME`, `Observation`, `PAR`, `PLACE`, `QUANTITY`, `REASON`, `ZOD` |


</details>


### Accuracy


| Type | Score |
| --- | --- |
| `TAG_ACC` | 0.00 |
| `DEP_UAS` | 0.00 |
| `DEP_LAS` | 0.00 |
| `DEP_LAS_PER_TYPE` | 0.00 |
| `SENTS_P` | 100.00 |
| `SENTS_R` | 100.00 |
| `SENTS_F` | 100.00 |
| `ENTS_F` | 99.32 |
| `ENTS_P` | 99.47 |
| `ENTS_R` | 99.17 |
| `LEMMA_ACC` | 0.00 |
| `NER_LOSS` | 7790.09 |


model name: en_pipeline


model author: MohaAM


task: token-classification


tags: en, spacy, token-classification, model-index


### How to use


You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we
set a seed for reproducibility:


```python
>>> from transformers import pipeline, set_seed
>>> generator = pipeline('text-generation', model='e-tony/gpt2-rnm')
>>> set_seed(42)
>>> generator("Rick: I turned myself into a pickle, Morty!\nMorty: ", max_length=50, num_return_sequences=5)

[{'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty:  I didn't want to have children. It was my fate! I'll pay my mom and dad.\nSnuffles:  Well, at least we"},
 {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty:  you know what happened?\n(Steven begins dragging people down the toilet with his hand. As Steven falls) The whole thing starts.\nA man approaches Steven"},
 {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty:  Oh wait! And do you remember what I did to you?\nJerry:  Uh, it didn't hurt. It should have hurt a lot since I"},
 {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty:  Rick!\nKraven:  Wait! [wary gasp] What the hell are you doing this time?!\nJerry:  Hey, are you"},
 {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty:  Uh.\nJerry:  You don't have to put your finger on me today, do you?\nRick:  It's just, what do you"}]
```


### Training data


We used the original `gpt2` model and fine-tuned it on [Rick and Morty transcripts](https://rickandmorty.fandom.com/wiki/Category:Transcripts).


model name: gpt2-rnm


model author: e-tony


task: text-generation


tags: pytorch, jax, gpt2, text-generation, transformers


language: en
tags:


- sagemaker
- bart
- summarization
license: apache-2.0
datasets:
- samsum
model-index:
- name: bart-large-cnn-samsum
results:
  - task:
name: Abstractive Text Summarization
type: abstractive-text-summarization
dataset:
name: "SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization"
type: samsum
metrics:
    - name: Validation ROGUE-1
type: rogue-1
value: 43.2111
    - name: Validation ROGUE-2
type: rogue-2
value: 22.3519
    - name: Validation ROGUE-L
type: rogue-l
value: 33.315
    - name: Test ROGUE-1
type: rogue-1
value: 41.8283
    - name: Test ROGUE-2
type: rogue-2
value: 20.9857
    - name: Test ROGUE-L
type: rogue-l
value: 32.3602
widget:
- text: |
Sugi: I am tired of everything in my life.
Tommy: What? How happy you life is! I do envy you.
Sugi: You don't know that I have been over-protected by my mother these years. I am really about to leave the family and spread my wings.
Tommy: Maybe you are right.


## `bart-large-cnn-samsum`


This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
For more information look at:


- [🤗 Transformers Documentation: Amazon SageMaker](https://huggingface.co/transformers/sagemaker.html)
- [Example Notebooks](https://github.com/huggingface/notebooks/tree/master/sagemaker)
- [Amazon SageMaker documentation for Hugging Face](https://docs.aws.amazon.com/sagemaker/latest/dg/hugging-face.html)
- [Python SDK SageMaker documentation for Hugging Face](https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/index.html)
- [Deep Learning Container](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers)


## Hyperparameters


    {
    "dataset_name": "samsum",
    "do_eval": true,
    "do_predict": true,
    "do_train": true,
    "fp16": true,
    "learning_rate": 5e-05,
    "model_name_or_path": "facebook/bart-large-cnn",
    "num_train_epochs": 3,
    "output_dir": "/opt/ml/model",
    "per_device_eval_batch_size": 4,
    "per_device_train_batch_size": 4,
    "predict_with_generate": true,
    "seed": 7


## Usage


    from transformers import pipeline
    summarizer = pipeline("summarization", model="slauw87/bart-large-cnn-samsum")
    conversation = '''Sugi: I am tired of everything in my life.
    Tommy: What? How happy you life is! I do envy you.
    Sugi: You don't know that I have been over-protected by my mother these years. I am really about to leave the family and spread my wings.
    Tommy: Maybe you are right.
    '''
    nlp(conversation)


## Results


| key | value |
| --- | ----- |
| eval_rouge1 | 43.2111 |
| eval_rouge2 | 22.3519 |
| eval_rougeL | 33.3153 |
| eval_rougeLsum | 40.0527 |
| predict_rouge1 | 41.8283 |
| predict_rouge2 | 20.9857 |
| predict_rougeL | 32.3602 |
| predict_rougeLsum | 38.7316 |


model name: bart_summarisation


model author: slauw87


task: summarization


tags: pytorch, bart, text2text-generation, en, dataset:samsum, transformers, sagemaker, summarization, license:apache-2.0, model-index, autonlp_compatible


language: id
tags:


- pipeline:summarization
- summarization
- t5
datasets:
- indosum


# Indonesian T5 Summarization Base Model


Finetuned T5 base summarization model for Indonesian.


## Finetuning Corpus


`t5-base-indonesian-summarization-cased` model is based on `t5-base-bahasa-summarization-cased` by [huseinzol05](https://huggingface.co/huseinzol05), finetuned using [indosum](https://github.com/kata-ai/indosum) dataset.


## Load Finetuned Model


```python
from transformers import T5Tokenizer, T5Model, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("panggi/t5-base-indonesian-summarization-cased")
model = T5ForConditionalGeneration.from_pretrained("panggi/t5-base-indonesian-summarization-cased")
```


## Code Sample