| Component | Labels |
| --- | --- |
| **`morphologizer`** | `AdpType=Prep\|POS=ADP`, `Definite=Ind\|Gender=Com\|Number=Sing\|POS=NOUN`, `Mood=Ind\|POS=AUX\|Tense=Pres\|VerbForm=Fin\|Voice=Act`, `POS=PROPN`, `Definite=Ind\|Number=Sing\|POS=VERB\|Tense=Past\|VerbForm=Part`, `Definite=Def\|Gender=Neut\|Number=Sing\|POS=NOUN`, `POS=SCONJ`, `Definite=Def\|Gender=Com\|Number=Sing\|POS=NOUN`, `Mood=Ind\|POS=VERB\|Tense=Pres\|VerbForm=Fin\|Voice=Act`, `POS=ADV`, `Number=Plur\|POS=DET\|PronType=Dem`, `Degree=Pos\|Number=Plur\|POS=ADJ`, `Definite=Ind\|Gender=Com\|Number=Plur\|POS=NOUN`, `POS=PUNCT`, `POS=CCONJ`, `Definite=Ind\|Degree=Cmp\|Number=Sing\|POS=ADJ`, `Degree=Cmp\|POS=ADJ`, `POS=PRON\|PartType=Inf`, `Gender=Com\|Number=Sing\|POS=DET\|PronType=Ind`, `Definite=Ind\|Degree=Pos\|Number=Sing\|POS=ADJ`, `Case=Acc\|Gender=Neut\|Number=Sing\|POS=PRON\|Person=3\|PronType=Prs`, `Definite=Ind\|Gender=Neut\|Number=Plur\|POS=NOUN`, `Definite=Def\|Degree=Pos\|Number=Sing\|POS=ADJ`, `Gender=Neut\|Number=Sing\|POS=DET\|PronType=Dem`, `Degree=Pos\|POS=ADV`, `Definite=Def\|Number=Sing\|POS=VERB\|Tense=Past\|VerbForm=Part`, `Definite=Ind\|Gender=Neut\|Number=Sing\|POS=NOUN`, `POS=PRON\|PronType=Dem`, `NumType=Card\|POS=NUM`, `Definite=Ind\|Degree=Pos\|Gender=Neut\|Number=Sing\|POS=ADJ`, `Case=Acc\|Gender=Com\|Number=Sing\|POS=PRON\|Person=3\|PronType=Prs`, `Degree=Pos\|Gender=Com\|Number=Sing\|POS=ADJ`, `Case=Nom\|Gender=Com\|Number=Sing\|POS=PRON\|Person=3\|PronType=Prs`, `NumType=Ord\|POS=ADJ`, `Gender=Com\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `Mood=Ind\|POS=AUX\|Tense=Past\|VerbForm=Fin\|Voice=Act`, `POS=VERB\|VerbForm=Inf\|Voice=Act`, `Mood=Ind\|POS=VERB\|Tense=Past\|VerbForm=Fin\|Voice=Act`, `POS=NOUN`, `Mood=Ind\|POS=VERB\|Tense=Pres\|VerbForm=Fin\|Voice=Pass`, `POS=ADP\|PartType=Inf`, `Degree=Pos\|POS=ADJ`, `Definite=Def\|Gender=Com\|Number=Plur\|POS=NOUN`, `Number[psor]=Sing\|POS=DET\|Person=3\|Poss=Yes\|PronType=Prs`, `Case=Gen\|Definite=Def\|Gender=Com\|Number=Sing\|POS=NOUN`, `POS=AUX\|VerbForm=Inf\|Voice=Act`, `Definite=Ind\|Degree=Pos\|Gender=Com\|Number=Sing\|POS=ADJ`, `Gender=Com\|Number=Sing\|POS=DET\|PronType=Dem`, `Number=Plur\|POS=DET\|PronType=Ind`, `Gender=Com\|Number=Sing\|POS=PRON\|PronType=Ind`, `Case=Acc\|POS=PRON\|Person=3\|PronType=Prs\|Reflex=Yes`, `POS=PART\|PartType=Inf`, `Gender=Neut\|Number=Sing\|POS=DET\|PronType=Ind`, `Case=Acc\|Number=Plur\|POS=PRON\|Person=3\|PronType=Prs`, `Case=Gen\|Definite=Def\|Gender=Neut\|Number=Sing\|POS=NOUN`, `Case=Nom\|Number=Plur\|POS=PRON\|Person=3\|PronType=Prs`, `Case=Nom\|Gender=Com\|Number=Sing\|POS=PRON\|Person=1\|PronType=Prs`, `Case=Nom\|Gender=Com\|POS=PRON\|PronType=Ind`, `Gender=Neut\|Number=Sing\|POS=PRON\|PronType=Ind`, `Mood=Imp\|POS=VERB`, `Gender=Com\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs`, `Definite=Ind\|Number=Sing\|POS=AUX\|Tense=Past\|VerbForm=Part`, `POS=X`, `Case=Nom\|Gender=Com\|Number=Plur\|POS=PRON\|Person=1\|PronType=Prs`, `Case=Gen\|Definite=Def\|Gender=Com\|Number=Plur\|POS=NOUN`, `POS=VERB\|Tense=Pres\|VerbForm=Part`, `Number=Plur\|POS=PRON\|PronType=Int,Rel`, `POS=VERB\|VerbForm=Inf\|Voice=Pass`, `Case=Gen\|Definite=Ind\|Gender=Com\|Number=Sing\|POS=NOUN`, `Degree=Cmp\|POS=ADV`, `POS=ADV\|PartType=Inf`, `Degree=Sup\|POS=ADV`, `Number=Plur\|POS=PRON\|PronType=Dem`, `Number=Plur\|POS=PRON\|PronType=Ind`, `Definite=Def\|Gender=Neut\|Number=Plur\|POS=NOUN`, `Case=Acc\|Gender=Com\|Number=Sing\|POS=PRON\|Person=1\|PronType=Prs`, `Case=Gen\|POS=PROPN`, `POS=ADP`, `Degree=Cmp\|Number=Plur\|POS=ADJ`, `Definite=Def\|Degree=Sup\|POS=ADJ`, `Gender=Neut\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs`, `Degree=Pos\|Number=Sing\|POS=ADJ`, `Number=Plur\|Number[psor]=Sing\|POS=DET\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `Gender=Com\|Number=Sing\|Number[psor]=Plur\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs\|Style=Form`, `Number=Plur\|POS=PRON\|PronType=Rcp`, `Case=Gen\|Degree=Cmp\|POS=ADJ`, `Case=Gen\|Definite=Def\|Gender=Neut\|Number=Plur\|POS=NOUN`, `Number[psor]=Plur\|POS=DET\|Person=3\|Poss=Yes\|PronType=Prs`, `POS=INTJ`, `Number=Plur\|Number[psor]=Sing\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs`, `Degree=Pos\|Gender=Neut\|Number=Sing\|POS=ADJ`, `Gender=Neut\|Number=Sing\|Number[psor]=Plur\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs\|Style=Form`, `Case=Acc\|Gender=Com\|Number=Sing\|POS=PRON\|Person=2\|PronType=Prs`, `Gender=Com\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=2\|Poss=Yes\|PronType=Prs`, `Case=Gen\|Definite=Ind\|Gender=Neut\|Number=Plur\|POS=NOUN`, `Number=Sing\|POS=PRON\|PronType=Int,Rel`, `Number=Plur\|Number[psor]=Plur\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs\|Style=Form`, `Gender=Neut\|Number=Sing\|POS=PRON\|PronType=Int,Rel`, `Definite=Def\|Degree=Sup\|Number=Plur\|POS=ADJ`, `Case=Nom\|Gender=Com\|Number=Sing\|POS=PRON\|Person=2\|PronType=Prs`, `Gender=Neut\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `Definite=Ind\|Number=Sing\|POS=NOUN`, `Number=Plur\|POS=VERB\|Tense=Past\|VerbForm=Part`, `Number=Plur\|Number[psor]=Sing\|POS=PRON\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `POS=SYM`, `Case=Nom\|Gender=Com\|POS=PRON\|Person=2\|Polite=Form\|PronType=Prs`, `Degree=Sup\|POS=ADJ`, `Number=Plur\|POS=DET\|PronType=Ind\|Style=Arch`, `Case=Gen\|Gender=Com\|Number=Sing\|POS=DET\|PronType=Dem`, `Foreign=Yes\|POS=X`, `POS=DET\|Person=2\|Polite=Form\|Poss=Yes\|PronType=Prs`, `Gender=Neut\|Number=Sing\|POS=PRON\|PronType=Dem`, `Case=Acc\|Gender=Com\|Number=Plur\|POS=PRON\|Person=1\|PronType=Prs`, `Case=Gen\|Definite=Ind\|Gender=Neut\|Number=Sing\|POS=NOUN`, `Case=Gen\|POS=PRON\|PronType=Int,Rel`, `Gender=Com\|Number=Sing\|POS=PRON\|PronType=Dem`, `Abbr=Yes\|POS=X`, `Case=Gen\|Definite=Ind\|Gender=Com\|Number=Plur\|POS=NOUN`, `Definite=Def\|Degree=Abs\|POS=ADJ`, `Definite=Ind\|Degree=Sup\|Number=Sing\|POS=ADJ`, `Definite=Ind\|POS=NOUN`, `Gender=Com\|Number=Plur\|POS=NOUN`, `Number[psor]=Plur\|POS=DET\|Person=1\|Poss=Yes\|PronType=Prs`, `Gender=Com\|POS=PRON\|PronType=Int,Rel`, `Case=Nom\|Gender=Com\|Number=Plur\|POS=PRON\|Person=2\|PronType=Prs`, `Degree=Abs\|POS=ADV`, `POS=VERB\|VerbForm=Ger`, `POS=VERB\|Tense=Past\|VerbForm=Part`, `Definite=Def\|Degree=Sup\|Number=Sing\|POS=ADJ`, `Number=Plur\|Number[psor]=Plur\|POS=PRON\|Person=1\|Poss=Yes\|PronType=Prs\|Style=Form`, `Case=Gen\|Definite=Def\|Degree=Pos\|Number=Sing\|POS=ADJ`, `Case=Gen\|Degree=Pos\|Number=Plur\|POS=ADJ`, `Case=Acc\|Gender=Com\|POS=PRON\|Person=2\|Polite=Form\|PronType=Prs`, `Gender=Com\|Number=Sing\|POS=PRON\|PronType=Int,Rel`, `POS=VERB\|Tense=Pres`, `Case=Gen\|Number=Plur\|POS=DET\|PronType=Ind`, `Number[psor]=Plur\|POS=DET\|Person=2\|Poss=Yes\|PronType=Prs`, `POS=PRON\|Person=2\|Polite=Form\|Poss=Yes\|PronType=Prs`, `Gender=Neut\|Number=Sing\|Number[psor]=Sing\|POS=DET\|Person=2\|Poss=Yes\|PronType=Prs`, `POS=AUX\|Tense=Pres\|VerbForm=Part`, `Mood=Ind\|POS=VERB\|Tense=Past\|VerbForm=Fin\|Voice=Pass`, `Gender=Com\|Number=Sing\|Number[psor]=Sing\|POS=PRON\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `Degree=Sup\|Number=Plur\|POS=ADJ`, `Case=Acc\|Gender=Com\|Number=Plur\|POS=PRON\|Person=2\|PronType=Prs`, `Gender=Neut\|Number=Sing\|Number[psor]=Sing\|POS=PRON\|Person=3\|Poss=Yes\|PronType=Prs\|Reflex=Yes`, `Definite=Ind\|Number=Plur\|POS=NOUN`, `Case=Gen\|Number=Plur\|POS=VERB\|Tense=Past\|VerbForm=Part`, `Mood=Imp\|POS=AUX`, `Gender=Com\|Number=Sing\|Number[psor]=Sing\|POS=PRON\|Person=1\|Poss=Yes\|PronType=Prs`, `Number[psor]=Sing\|POS=PRON\|Person=3\|Poss=Yes\|PronType=Prs`, `Definite=Def\|Gender=Com\|Number=Sing\|POS=VERB\|Tense=Past\|VerbForm=Part`, `Number=Plur\|Number[psor]=Sing\|POS=DET\|Person=2\|Poss=Yes\|PronType=Prs`, `Case=Gen\|Gender=Com\|Number=Sing\|POS=DET\|PronType=Ind`, `Case=Gen\|POS=NOUN`, `Number[psor]=Plur\|POS=PRON\|Person=3\|Poss=Yes\|PronType=Prs`, `POS=DET\|PronType=Dem`, `Definite=Def\|Number=Plur\|POS=NOUN` |
| **`parser`** | `ROOT`, `acl:relcl`, `advcl`, `advmod`, `amod`, `appos`, `aux`, `case`, `cc`, `ccomp`, `compound:prt`, `conj`, `cop`, `dep`, `det`, `expl`, `fixed`, `flat`, `iobj`, `list`, `mark`, `nmod`, `nmod:poss`, `nsubj`, `nummod`, `obj`, `obl`, `obl:loc`, `obl:tmod`, `punct`, `xcomp` |
| **`ner`** | `LOC`, `MISC`, `ORG`, `PER` |


</details>


### Accuracy


| Type | Score |
| --- | --- |
| `POS_ACC` | 95.83 |
| `MORPH_ACC` | 95.70 |
| `DEP_UAS` | 84.92 |
| `DEP_LAS` | 81.76 |
| `SENTS_P` | 86.04 |
| `SENTS_R` | 87.41 |
| `SENTS_F` | 86.72 |
| `LEMMA_ACC` | 84.91 |
| `ENTS_F` | 82.32 |
| `ENTS_P` | 81.72 |
| `ENTS_R` | 82.92 |
| `TRANSFORMER_LOSS` | 41746686.63 |
| `MORPHOLOGIZER_LOSS` | 3458966.49 |
| `PARSER_LOSS` | 15104898.38 |
| `NER_LOSS` | 546098.45 |


## Bias and Robustness


Besides the validation done by SpaCy on the DaNE testset, DaCy also provides a series of augmentations to the DaNE test set to see how well the models deal with these types of augmentations.
The can be seen as behavioural probes akinn to the NLP checklist.


### Deterministic Augmentations


Deterministic augmentations are augmentation which always yield the same result.


| Augmentation | Part-of-speech tagging (Accuracy) | Morphological tagging (Accuracy) | Dependency Parsing (UAS) | Dependency Parsing (LAS) | Sentence segmentation (F1) | Lemmatization (Accuracy) | Named entity recognition (F1) |
| --- | --- | --- | --- |  --- | --- | --- |  --- |
| No augmentation | 0.98 | 0.974 | 0.868 |  0.836 | 0.936 | 0.844 | 0.765 |
| Æøå Augmentation | 0.955 | 0.948 | 0.823 |  0.783 | 0.922 | 0.754 | 0.718 |
| Lowercase | 0.974 | 0.97 | 0.862 |  0.828 | 0.905 | 0.848 | 0.681 |
| No Spacing | 0.229 | 0.229 | 0.004 |  0.003 | 0.824 | 0.225 | 0.048 |
| Abbreviated first names | 0.979 | 0.973 | 0.864 |  0.832 | 0.94 | 0.845 | 0.699 |
| Input size augmentation 5 sentences | 0.956 | 0.956 | 0.851 |  0.818 | 0.883 | 0.844 | 0.743 |
| Input size augmentation 10 sentences | 0.959 | 0.958 | 0.853 |  0.821 | 0.897 | 0.844 | 0.755 |


### Stochastic Augmentations


Stochastic augmentations are augmentation which are repeated mulitple times to estimate the effect of the augmentation.


| Augmentation | Part-of-speech tagging (Accuracy) | Morphological tagging (Accuracy) | Dependency Parsing (UAS) | Dependency Parsing (LAS) | Sentence segmentation (F1) | Lemmatization (Accuracy) | Named entity recognition (F1) |
| --- | --- | --- | --- |  --- | --- | --- |  --- |
| Keystroke errors 2% | 0.931 (0.003) | 0.929 (0.003) | 0.797 (0.003) |  0.753 (0.003) | 0.884 (0.003) | 0.772 (0.003) | 0.657 (0.003) |
| Keystroke errors 5% | 0.859 (0.003) | 0.863 (0.003) | 0.699 (0.003) |  0.641 (0.003) | 0.824 (0.003) | 0.681 (0.003) | 0.53 (0.003) |
| Keystroke errors 15% | 0.633 (0.006) | 0.662 (0.006) | 0.439 (0.006) |  0.358 (0.006) | 0.688 (0.006) | 0.459 (0.006) | 0.293 (0.006) |
| Danish names | 0.979 (0.0) | 0.974 (0.0) | 0.867 (0.0) |  0.835 (0.0) | 0.943 (0.0) | 0.847 (0.0) | 0.748 (0.0) |
| Muslim names | 0.979 (0.0) | 0.974 (0.0) | 0.865 (0.0) |  0.833 (0.0) | 0.94 (0.0) | 0.847 (0.0) | 0.732 (0.0) |
| Female names | 0.979 (0.0) | 0.974 (0.0) | 0.867 (0.0) |  0.835 (0.0) | 0.946 (0.0) | 0.847 (0.0) | 0.754 (0.0) |
| Male names | 0.979 (0.0) | 0.974 (0.0) | 0.867 (0.0) |  0.835 (0.0) | 0.943 (0.0) | 0.847 (0.0) | 0.748 (0.0) |
| Spacing Augmention 5% | 0.941 (0.002) | 0.936 (0.002) | 0.755 (0.002) |  0.725 (0.002) | 0.907 (0.002) | 0.811 (0.002) | 0.699 (0.002) |


<details>


<summary> Description of Augmenters </summary>


**No augmentation:**
Applies no augmentation to the DaNE test set.


**Æøå Augmentation:**
This augmentation replace the æ,ø, and å with their spelling variations ae, oe and aa respectively.


**Lowercase:**
This augmentation lowercases all text.


**No Spacing:**
This augmentation removed all spacing from the text.


**Abbreviated first names:**
This agmentation abbreviates the first names of entities. For instance 'Kenneth Enevoldsen' would turn to 'K. Enevoldsen'.


**Keystroke errors 2%:**
This agmentation simulate keystroke errors by replacing 2% of keys with a neighbouring key on a Danish QWERTY keyboard. As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Keystroke errors 5%:**
This agmentation simulate keystroke errors by replacing 5% of keys with a neighbouring key on a Danish QWERTY keyboard. As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Keystroke errors 15%:**
This agmentation simulate keystroke errors by replacing 15% of keys with a neighbouring key on a Danish QWERTY keyboard. As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Danish names:**
This agmentation replace all names with Danish names derived from Danmarks Statistik (2021). As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Muslim names:**
This agmentation replace all names with Muslim names derived from  Meldgaard (2005). As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Female names:**
This agmentation replace all names with Danish female names derived from Danmarks Statistik (2021). As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Male names:**
This agmentation replace all names with Danish male names derived from Danmarks Statistik (2021). As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


**Spacing Augmention 5%:**
This agmentation replace all names with Danish male names derived from Danmarks Statistik (2021). As this agmentation is stochastic it is repeated 20 times to obtain a consistent estimate and the mean is provided with its standard deviation in parenthesis.


 </details>
 <br />


### Hardware


This was run an trained on a Quadro RTX 8000 GPU.


model name: da_dacy_small_trf


model author: chcaa


task: token-classification


tags: da, spacy, token-classification, license:apache-2.0, model-index


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: urdu-colab
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# urdu-colab


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: urdu-colab


model author: js-rockstar


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, transformers, license:apache-2.0, generated_from_trainer, model-index


# KoGPT-Conditional-2


### Condition format


```python
# create condition sentence
random_main_logit = np.random.normal(
    loc=3.368,
    scale=1.015,
    size=1
    )[0].round(1)
random_sub_logit = np.random.normal(
    loc=1.333,
    scale=0.790,
    size=1
    )[0].round(1)
condition_sentence = f"{random_main_logit}만큼 행복감정인 문장이다. {random_sub_logit}만큼 놀람감정인 문장이다. "

```


### Input Format


```python
# make input sentence
input_sentence = "수상한 밤들이 계속되던 날, 언젠가부터 나는"
condition_plus_input = condition_sentence + input_sentence
print(condition_plus_input)
```


```
3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는
```


### How to infer


```
inferred_sentence = infer_sentence(condition_plus_input, k=10, output_token_length=max_token_length)
inferred_sentence
```


```
['3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 서서히 제정신을 차리고 일어날 수 있었다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 달 보는 걸 좋아하게 되었다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 수상한 사람들의 입을 들여다 볼 수 있었다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 이상한 나라의 앨리스가 되어 있었다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 기이한 경험을 했다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 이상하게도 평화가 찾아온다는 사실을 깨달았다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 어둠을 뚫는 무언가가 있다는 걸 알았다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 달빛의 의미를 이해하기 시작했다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 안방에서 잘 때 내 손을 꼭 잡았다',
 '3.9만큼 행복감정인 문장이다. 1.2만큼 놀람감정인 문장이다. 수상한 밤들이 계속되던 날, 언젠가부터 나는 이상한 나라의 앨리스처럼 눈을 반짝이며 주위를 탐구하기 시작했다']

```


model name: kogpt-conditional-2


model author: snoop2head


task: text-generation


tags: pytorch, gpt2, text-generation, transformers


# KoGPT-Joong-2


[Github Source](https://github.com/snoop2head/KoGPT-Joong-2)


### KoGPT-Joong-2's Acrostic Poem Generation Examples (N행시 예시)


```
[연세대(1)]
연민이라는 것은 양날의 검과 같다
세기의 악연일수도..
대가는 혹독할것이다 연기의 끝은 상처다

[연세대(2)]
연약한 마음으로 강한 척하지 말고 강한 마음을 먹자
세 마디 말보다 한마디 말이 더 진정성 있어 보인다.
대시 하지 마라.
```


```
[자탄풍]
자그마하게
탄식의 강을 건너고
풍경의 나무를 넘어가네
```


### KoGPT-Joong-2's Phrase Generation Examples


```
[너는 나의]
- 너는 나의 거짓말. 나는 너의 참말. 너를 잊었다는 나와 나를 잊었다는 너의 차이.
- 너는 나의 옷자락이고 머릿결이고 꿈결이고 나를 헤집던 사정없는 풍속이었다
```


```
[그대 왜 내 꿈에]
- 그대 왜 내 꿈에 나오지 않는 걸까요, 내 꿈 속에서도 그대 사라지면 어쩌나요
- 그대 왜 내 꿈에 불시착했는가.
```


### Dataset finetuned on


- [가사 데이터셋](_clones/char-rnn-tensorflow/data/lyricskor/input.txt)
- [글스타그램 데이터셋](https://github.com/Keracorn/geulstagram)


### Dependencies Installation


```bash
pip install -r requirements.txt
```


### References


- [KoGPT2-Transformers huggingface 활용 예시](https://github.com/taeminlee/KoGPT2-Transformers)
- [SKT-AI의 KoGPT2와 pytorch를 이용해 소설을 생성하는 GPT-2 모델.](https://github.com/shbictai/narrativeKoGPT2)
- [인공지능 수필 작가 블로그 글](https://jeinalog.tistory.com/entry/AI-x-Bookathon-%EC%9D%B8%EA%B3%B5%EC%A7%80%EB%8A%A5%EC%9D%84-%EC%88%98%ED%95%84-%EC%9E%91%EA%B0%80%EB%A1%9C-%ED%95%99%EC%8A%B5%EC%8B%9C%EC%BC%9C%EB%B3%B4%EC%9E%90) | [인공지능 수필 작가 코드](https://github.dev/jeina7/GPT2-essay-writer)


model name: KoGPT-Joong-2


model author: snoop2head


task: text-generation


tags:


- conversational


#Bill cipher chat bot


model name: BillCipherBot


model author: Sunnydx


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language:


- mr
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- openslr
- generated_from_trainer
- robust-speech-event
datasets:
- mozilla-foundation/common_voice_8_0
- openslr
- shivam/marathi_samanantar_processed
- shivam/marathi_pib_processed
- opus100
- tatoeba
- tapaco
model-index:
- name: wav2vec2-large-xls-r-300m-mr
results:
  - task:
type: automatic-speech-recognition
name: Speech Recognition
dataset:
type: mozilla-foundation/common_voice_8_0
name: Common Voice 8
args: mr
metrics:
    - type: wer    # Required. Example: wer
value: 31.05  # Required. Example: 20.90
name: Test WER    # Optional. Example: Test WER
    - name: Test CER
type: cer
value: 6.82


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - MR and OPENSLR - SLR64 - MR datasets.
It achieves the following results on the evaluation set:


- Loss: 0.494580
- Wer: 0.401524


### Eval results on Common Voice 8 "test" (WER):


| Without LM | With LM |
|---|---|
| 40.513437625350984 | 31.04693140794224 |


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 200.0
- mixed_precision_training: Native AMP


### Training results


| Step  | Training Loss  | Validation Loss  | Wer  |
|---|---|---|---|
| 400  | 3.794000  | 3.532227  | 1.000000  |
| 800  | 3.362400  | 3.359044  | 1.000000  |
| 1200  | 2.293900  | 1.011279  | 0.829924  |
| 1600  | 1.233000  | 0.502743  | 0.593662  |
| 2000  | 0.962600  | 0.412519  | 0.496992  |
| 2400  | 0.831800  | 0.402903  | 0.493783  |
| 2800  | 0.737000  | 0.389773  | 0.469314  |
| 3200  | 0.677100  | 0.373987  | 0.436021  |
| 3600  | 0.634400  | 0.383823  | 0.432010  |
| 4000  | 0.586000  | 0.375610  | 0.419575  |
| 4400  | 0.561000  | 0.387891  | 0.418371  |
| 4800  | 0.518500  | 0.386357  | 0.417569  |
| 5200  | 0.515300  | 0.415069  | 0.430004  |
| 5600  | 0.478100  | 0.399211  | 0.408744  |
| 6000  | 0.468100  | 0.424542  | 0.402327  |
| 6400  | 0.439400  | 0.430979  | 0.410750  |
| 6800  | 0.429600  | 0.427700  | 0.409146  |
| 7200  | 0.400300  | 0.451111  | 0.419976  |
| 7600  | 0.395100  | 0.463446  | 0.405134  |
| 8000  | 0.381800  | 0.454752  | 0.407942  |
| 8400  | 0.371500  | 0.461547  | 0.404733  |
| 8800  | 0.362500  | 0.461543  | 0.411151  |
| 9200  | 0.338200  | 0.468299  | 0.417168  |
| 9600  | 0.338800  | 0.480989  | 0.412355  |
| 10000  | 0.317600  | 0.475700  | 0.410750  |
| 10400  | 0.315100  | 0.478920  | 0.403530  |
| 10800  | 0.296200  | 0.480600  | 0.398315  |
| 11200  | 0.299000  | 0.477083  | 0.393502  |
| 11600  | 0.290000  | 0.465646  | 0.393903  |
| 12000  | 0.290900  | 0.490041  | 0.405937  |
| 12400  | 0.275600  | 0.489354  | 0.399519  |
| 12800  | 0.272600  | 0.494580  | 0.395909  |
| 13200 | 	0.265900 | 	0.497918 | 	0.397112 |
| 13600 | 	0.266300 | 	0.498627 | 	0.397513 |
| 14000 | 	0.259600 | 	0.504610 | 	0.401524 |


#### Evaluation Commands


To evaluate on `mozilla-foundation/common_voice_8_0` with split `test`


```bash
python eval.py --model_id smangrul/xls-r-mr-model --dataset mozilla-foundation/common_voice_8_0 --config mr --split test
```


### Framework versions


- Transformers 4.17.0.dev0
- Pytorch 1.10.2+cu113
- Datasets 1.18.3.dev0
- Tokenizers 0.11.0


model name: xls-r-mr-model


model author: smangrul


task: automatic-speech-recognition


tags: pytorch, mr, dataset:mozilla-foundation/common_voice_8_0, dataset:openslr, license:apache-2.0, automatic-speech-recognition, mozilla-foundation/common_voice_8_0, openslr, generated_from_trainer, robust-speech-event, model-index


language:


- mr
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- robust-speech-event
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-mr
results:
  - task:
type: automatic-speech-recognition
name: Speech Recognition
dataset:
type: mozilla-foundation/common_voice_8_0
name: Common Voice 8
args: mr
metrics:
    - type: wer    # Required. Example: wer
value: 49.70  # Required. Example: 20.90
name: Test WER    # Optional. Example: Test WER
    - name: Test CER
type: cer
value: 11.11


This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - MR dataset.
It achieves the following results on the evaluation set:


- Loss: 0.5319
- Wer: 0.5973


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 3.3987        | 22.73  | 500  | 3.3586          | 1.0    |
| 2.0563        | 45.45  | 1000 | 1.0375          | 0.8428 |
| 1.283         | 68.18  | 1500 | 0.5563          | 0.6996 |
| 1.0308        | 90.91  | 2000 | 0.4922          | 0.6398 |
| 0.8803        | 113.64 | 2500 | 0.4949          | 0.6153 |
| 0.7581        | 136.36 | 3000 | 0.4932          | 0.5965 |
| 0.6681        | 159.09 | 3500 | 0.5133          | 0.5921 |
| 0.6191        | 181.82 | 4000 | 0.5281          | 0.5909 |


### Framework versions


model name: xls-r-mr


model author: smangrul


task: automatic-speech-recognition


tags: pytorch, wav2vec2, automatic-speech-recognition, mr, dataset:common_voice, transformers, license:apache-2.0, mozilla-foundation/common_voice_8_0, generated_from_trainer, robust-speech-event, model-index


language:


- pt-br
tags:
- question-answering
license: apache-2.0
pipeline_tag: question-answering
metrics:
- em
- f1


# BraQuAD BERT


This is a question-answering model trained in BraQuAD 2.0, a version of SQuAD 2.0 translated to PT-BR using Google Cloud Translation API.


### Context


Edith Ranzini (São Paulo,[1] 1946) é uma engenheira brasileira formada pela USP, professora doutora da Pontifícia Universidade Católica de São Paulo[2] e professora sênior da Escola Politécnica da Universidade de São Paulo (Poli).[3] Ela compôs a equipe responsável pela  criação do primeiro computador brasileiro, o Patinho Feio,[1] em 1972, e participou do grupo de instituidores  da Fundação para o Desenvolvimento Tecnológico da Engenharia, sendo a única mulher do mesmo.[4][2] Atua nas áreas de inteligência artificial, engenharia de computação, redes neurais e sistemas gráficos.


Na sua época de prestar o vestibular, inscreveu-se para física na USP e para engenharia na Poli-USP,[3] sendo aprovada nesta última em 1965, ingressando como uma das 12 mulheres do total de 360 calouros.


Em 1969, formou-se como engenheira de eletricidade, permanecendo na universidade para fazer sua pós-graduação. Nessa época entrou para o Laboratório de Sistemas Digitais (LSD),atual Departamento de Engenharia de Computação e Sistemas Digitais, criado pelo professor Antônio Hélio Guerra Vieira.[3] Em 1970, deu início ao seu mestrado em Engenharia de Sistemas pela USP, concluindo o mesmo em 1975.[2] Nesse período, permaneceu no LSD e  fez parte do grupo responsável pelo desenvolvimento do primeiro computador brasileiro, o Patinho Feio (1971-1972) e do  G10 (1973-1975), primeiro computador brasileiro de médio porte,  feito para o Grupo de trabalho Especial (GTE), posteriormente Digibras.


### Examples:


1-Alem do Patinho feio qual outro projeto edith trabalhou? Answer: G10


2-Quantas mulheres entraram na Poli em 1965? Answer: 12


3-Qual grande projeto edith trabalhou? Answer: do primeiro computador brasileiro


4-Qual o primeiro computador brasileiro? Answer: Patinho Feio


## Expected results


As for an example, let's show a context and some questions you can ask, as well as the expected responses. This QnA pairs were not part of the training dataset.


#### How to use


```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer
import torch

mname = "piEsposito/braquad-bert-qna"
model = AutoModelForQuestionAnswering.from_pretrained(mname)
tokenizer = AutoTokenizer.from_pretrained(mname)

context = """Edith Ranzini (São Paulo,[1] 1946) é uma engenheira brasileira formada pela USP, professora doutora da Pontifícia Universidade Católica de São Paulo[2] e professora sênior da Escola Politécnica da Universidade de São Paulo (Poli).[3] Ela compôs a equipe responsável pela  criação do primeiro computador brasileiro, o Patinho Feio,[1] em 1972, e participou do grupo de instituidores  da Fundação para o Desenvolvimento Tecnológico da Engenharia, sendo a única mulher do mesmo.[4][2] Atua nas áreas de inteligência artificial, engenharia de computação, redes neurais e sistemas gráficos.

Na sua época de prestar o vestibular, inscreveu-se para física na USP e para engenharia na Poli-USP,[3] sendo aprovada nesta última em 1965, ingressando como uma das 12 mulheres do total de 360 calouros.[5]

Em 1969, formou-se como engenheira de eletricidade,[2][3] permanecendo na universidade para fazer sua pós-graduação. Nessa época entrou para o Laboratório de Sistemas Digitais (LSD),atual Departamento de Engenharia de Computação e Sistemas Digitais, criado pelo professor Antônio Hélio Guerra Vieira.[3] Em 1970, deu início ao seu mestrado em Engenharia de Sistemas pela USP, concluindo o mesmo em 1975.[2] Nesse período, permaneceu no LSD e  fez parte do grupo responsável pelo desenvolvimento do primeiro computador brasileiro, o Patinho Feio (1971-1972) e do  G10 (1973-1975), primeiro computador brasileiro de médio porte,  feito para o Grupo de trabalho Especial (GTE), posteriormente Digibras."""
# you can try this for all the examples above.

question = 'Qual grande projeto edith trabalhou?'

string = f"[CLS] {question} [SEP] {context} [SEP]"

as_tensor = torch.Tensor(tokenizer.encode(string)).unsqueeze(0)
starts, ends = model(as_tensor.long())
s, e = torch.argmax(starts[0]), torch.argmax(ends[0])

print(tokenizer.decode(tokenizer.encode(string)[s:e+1])) # 'do primeiro computador brasileiro'
```


#### Limitations and bias


- The model is trained on a dataset translated using Google Cloud API. Due to that, there are some issues with the labels, in some cases, not being identic to the answers. Due to that, the performance cannot reach the level it does with english, handly curated models. Anyway, it is a good progresso towards QnA in PT-BR.


## Training data


[BraQuAD dataset](https://github.com/piEsposito/br-quad-2.0).


## Training procedure


## Eval results


EM   | F1
-------|---------
0.62  | 0.69


### BibTeX entry and citation info


```bibtex
@inproceedings{...,
  year={2020},
  title={BraQuAD - Dataset para Question Answering em PT-BR},
  author={Esposito, Wladimir and Esposito, Piero and Tamais, Ana},
}
```


model name: braquad-bert-qna


model author: piEsposito


task: question-answering


tags: pytorch, jax, bert, question-answering, pt-br, transformers, license:apache-2.0, autonlp_compatible, infinity_compatible


tags:


- conversational


# Harry Potter DialoGPT Model


model name: DialoGPT-small-hermoine


model author: Galaxy


task: conversational


tags: pytorch, gpt2, text-generation, transformers, conversational


language:


- en
tags:
- bluebert
license: cc0-1.0
datasets:
- pubmed


# BlueBert-Base, Uncased, PubMed


## Model description


A BERT model pre-trained on PubMed abstracts


## Intended uses & limitations


#### How to use


Please see https://github.com/ncbi-nlp/bluebert


## Training data


We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models.
The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/).


Pre-trained model: https://huggingface.co/bert-base-uncased


## Training procedure


* lowercasing the text
* removing speical chars `\x00`-`\x7F`
* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html)


Below is a code snippet for more details.


```python
value = value.lower()
value = re.sub(r'[\r\n]+', ' ', value)
value = re.sub(r'[^\x00-\x7F]+', ' ', value)

tokenized = TreebankWordTokenizer().tokenize(value)
sentence = ' '.join(tokenized)
sentence = re.sub(r"\s's\b", "'s", sentence)
```


### BibTeX entry and citation info


```bibtex
@InProceedings{peng2019transfer,
  author    = {Yifan Peng and Shankai Yan and Zhiyong Lu},
  title     = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets},
  booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)},
  year      = {2019},
  pages     = {58--65},
}
```


model name: bluebert_pubmed_uncased_L-12_H-768_A-12


model author: bionlp


tags: pytorch, en, dataset:pubmed, transformers, bluebert, license:cc0-1.0


language:


- en
tags:
- bert
- bluebert
license: cc0-1.0
datasets:
- PubMed
- MIMIC-III


# BlueBert-Base, Uncased, PubMed and MIMIC-III


## Model description


A BERT model pre-trained on PubMed abstracts and clinical notes ([MIMIC-III](https://mimic.physionet.org/)).


## Intended uses & limitations


#### How to use


## Training data


Pre-trained model: https://huggingface.co/bert-large-uncased


## Training procedure


### Acknowledgments


This work was supported by the Intramural Research Programs of the National Institutes of Health, National Library of
Medicine and Clinical Center. This work was supported by the National Library of Medicine of the National Institutes of Health under award number 4R00LM013001-01.


We are also grateful to the authors of BERT and ELMo to make the data and codes publicly available.


We would like to thank Dr Sun Kim for processing the PubMed texts.


### Disclaimer


This tool shows the results of research conducted in the Computational Biology Branch, NCBI. The information produced
on this website is not intended for direct diagnostic use or medical decision-making without review and oversight
by a clinical professional. Individuals should not change their health behavior solely on the basis of information
produced on this website. NIH does not independently verify the validity or utility of the information produced
by this tool. If you have questions about the information produced on this website, please see a health care
professional. More information about NCBI's disclaimer policy is available.


model name: bluebert_pubmed_mimic_uncased_L-24_H-1024_A-16


model author: bionlp


tags: pytorch, jax, en, dataset:PubMed, dataset:MIMIC-III, transformers, bert, bluebert, license:cc0-1.0


language:


- en
tags:
- bert
- bluebert
license: cc0-1.0
datasets:
- PubMed


# BlueBert-Base, Uncased, PubMed


## Model description


A BERT model pre-trained on PubMed abstracts.


## Intended uses & limitations


#### How to use


Please see https://github.com/ncbi-nlp/bluebert


## Training data


We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models.
The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/).


## Training procedure


* lowercasing the text
* removing speical chars `\x00`-`\x7F`
* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html)


Below is a code snippet for more details.


```python
value = value.lower()
value = re.sub(r'[\r\n]+', ' ', value)
value = re.sub(r'[^\x00-\x7F]+', ' ', value)

tokenized = TreebankWordTokenizer().tokenize(value)
sentence = ' '.join(tokenized)
sentence = re.sub(r"\s's\b", "'s", sentence)
```


### BibTeX entry and citation info


```bibtex
@InProceedings{peng2019transfer,
  author    = {Yifan Peng and Shankai Yan and Zhiyong Lu},
  title     = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets},
  booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)},
  year      = {2019},
  pages     = {58--65},
}
```


### Acknowledgments


### Disclaimer


model name: bluebert_pubmed_uncased_L-24_H-1024_A-16


model author: bionlp


tags: pytorch, jax, en, dataset:PubMed, transformers, bert, bluebert, license:cc0-1.0


language:


- en
tags:
- bert
- bluebert
license: cc0-1.0
datasets:
- PubMed
- MIMIC-III


# BlueBert-Base, Uncased, PubMed and MIMIC-III


## Model description


A BERT model pre-trained on PubMed abstracts and clinical notes ([MIMIC-III](https://mimic.physionet.org/)).


## Intended uses & limitations


#### How to use


## Training data


Pre-trained model: https://huggingface.co/bert-base-uncased


## Training procedure


### BibTeX entry and citation info


This work was supported by the Intramural Research Programs of the National Institutes of Health, National Library of
Medicine and Clinical Center. This work was supported by the National Library of Medicine of the National Institutes of Health under award number 4R00LM013001-01.


We are also grateful to the authors of BERT and ELMo to make the data and codes publicly available.


We would like to thank Dr Sun Kim for processing the PubMed texts.


### Disclaimer


This tool shows the results of research conducted in the Computational Biology Branch, NCBI. The information produced
on this website is not intended for direct diagnostic use or medical decision-making without review and oversight
by a clinical professional. Individuals should not change their health behavior solely on the basis of information
produced on this website. NIH does not independently verify the validity or utility of the information produced
by this tool. If you have questions about the information produced on this website, please see a health care
professional. More information about NCBI's disclaimer policy is available.


model name: bluebert_pubmed_mimic_uncased_L-12_H-768_A-12


model author: bionlp


tags: pytorch, jax, en, dataset:PubMed, dataset:MIMIC-III, transformers, bert, bluebert, license:cc0-1.0


Model trained for PhoNER_COVID19 dataset


model name: Covid19_phoNER


model author: huyenbui117


task: token-classification


tags: pytorch, roberta, token-classification, transformers, autonlp_compatible, infinity_compatible


tags:


- generated_from_trainer
model-index:
- name: model_pretrained
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# model_pretrained


This model is a fine-tuned version of [Jean-Baptiste/camembert-ner](https://huggingface.co/Jean-Baptiste/camembert-ner) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 1.5619


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0


### Training results


| Training Loss | Epoch | Step   | Validation Loss |
|:-------------:|:-----:|:------:|:---------------:|
| 1.9603        | 1.0   | 100346 | 1.8005          |
| 1.7032        | 2.0   | 200692 | 1.6460          |
| 1.5879        | 3.0   | 301038 | 1.5570          |


### Framework versions


- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.0
- Tokenizers 0.10.3


model name: das22-10-camembert_pretrained


model author: HueyNemud


task: fill-mask


tags: pytorch, camembert, fill-mask, transformers, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xlsr-hindhi-demo-colab
results: []


# wav2vec2-large-xlsr-hindhi-demo-colab


This model is a fine-tuned version of [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on the common_voice dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.11.3
- Pytorch 1.10.0+cu102
- Datasets 1.13.3
- Tokenizers 0.10.3


model name: wav2vec2-large-xlsr-hindhi-demo-colab


model author: jawaharreddy247


task: automatic-speech-recognition


tags: pytorch, tensorboard, wav2vec2, automatic-speech-recognition, dataset:common_voice, transformers, license:apache-2.0, generated_from_trainer, model-index


language:


- en
- ar
tags:
- translation
- Arabic Abjad Characters
- Arabic
license: apache-2.0
datasets:
- marefa-mt


# Marefa-Mt-En-Ar


# نموذج المعرفة للترجمة الآلية من الإنجليزية للعربية


## Model description


This is a model for translating English to Arabic. The special about this model that is take into considration the
using of additional Arabic characters like `پ` or `گ`.


## عن النموذج


هذا النموذج للترجمة الآلية من اللغة الإنجليزية إلى اللغة العربية, هو أول نماذج الترجمة الآلية التي تصدر تحت رعاية
[موسوعة المعرفة](https://www.marefa.org)


يتميز هذا النموذج عن غيره من النماذج بدعمه لحروف الأبجدية العربية الإضافية لتمييز الصوتيات الخاصة في اللغة الإنجليزية مثل `پ` , `گ`.


يمكنك زيارة
[هذه الصفحة](https://www.marefa.org/%D8%A7%D9%84%D9%85%D8%B9%D8%B1%D9%81%D8%A9:%D8%AF%D9%84%D9%8A%D9%84_%D8%A7%D9%84%D8%A3%D8%B3%D9%84%D9%88%D8%A8#.D8.AD.D8.B1.D9.88.D9.81_.D8.A5.D8.B6.D8.A7.D9.81.D9.8A.D8.A9_.D9.84.D9.84.D9.86.D8.B7.D9.82_.D8.A7.D9.84.D8.B3.D9.84.D9.8A.D9.85)
لمعرفة أكثر عن أسلوب إستخدام هذه الحروف الأبجدية العربية


### How to use كيفية الإستخدام


Install transformers and sentencepiece (python >= 3.6)


`$ pip3 install transformers==4.3.0 sentencepiece==0.1.95 nltk==3.5 protobuf==3.15.3 torch==1.7.1`


> If you are using `Google Colab`, please restart your runtime after installing the packages.


```python
from transformers import MarianTokenizer, MarianMTModel
mname = "marefa-nlp/marefa-mt-en-ar"
tokenizer = MarianTokenizer.from_pretrained(mname)
model = MarianMTModel.from_pretrained(mname)

# English Sample Text
input = "President Putin went to the presidential palace in the capital, Kiev"

translated_tokens = model.generate(**tokenizer.prepare_seq2seq_batch([input], return_tensors="pt"))
translated_text = [tokenizer.decode(t, skip_special_tokens=True) for t in translated_tokens]

# translated Arabic Text
print(translated_text)
# ذهب الرئيس پوتن إلى القصر الرئاسي في العاصمة كييڤ
```


model name: marefa-mt-en-ar


model author: marefa-nlp


task: translation


tags: pytorch, marian, text2text-generation, en, ar, dataset:marefa-mt, transformers, translation, Arabic Abjad Characters, Arabic, license:apache-2.0, autonlp_compatible


## Arabic and English News Summarization NLP Model


### About


This model is for summarizing news stories in short highlights for both Arabic and English tasks.


نموذج معرفي متخصص في تلخيص الأخبار العربية و الإنجليزية الى مجموعة من أهم النقاط


### Fine-Tuning


The model was finetuned using the [Arabic T5 Model](https://huggingface.co/bakrianoo/t5-arabic-large) which developed by [Abu Bakr Soliman](http://github.com/bakrianoo).


The primary summarization model also developed by the same developer.


### How to Use


- You can use this [Colab Notebook](https://colab.research.google.com/drive/1DWND1CAfCXD4OxrfmLBEaKeXhjGmYkod?usp=sharing) to test the model


1. Install [PyTorch](https://pytorch.org/)
1. Install the following Python packages


`$ pip3 install transformers==4.7.0 nltk==3.5 protobuf==3.15.3 sentencepiece==0.1.96`


3. Run this code


```python

from transformers import AutoTokenizer, AutoModelWithLMHead
import torch
import nltk
nltk.download('punkt')
from nltk.tokenize import sent_tokenize

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
m_name = "marefa-nlp/summarization-arabic-english-news"

tokenizer = AutoTokenizer.from_pretrained(m_name)
model = AutoModelWithLMHead.from_pretrained(m_name).to(device)

def get_summary(text, tokenizer, model, device="cpu", num_beams=2):
    if len(text.strip()) < 50:
        return ["Please provide more longer text"]

    text = "summarize: <paragraph> " + " <paragraph> ".join([ s.strip() for s in sent_tokenize(text) if s.strip() != ""]) + " </s>"
    text = text.strip().replace("\n","")

    tokenized_text = tokenizer.encode(text, return_tensors="pt").to(device)

    summary_ids = model.generate(
            tokenized_text,
            max_length=512,
            num_beams=num_beams,
            repetition_penalty=1.5,
            length_penalty=1.0,
            early_stopping=True
    )

    output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
    return [ s.strip() for s in output.split("<hl>") if s.strip() != "" ]

## Prepare Samples

samples = [
    """
        قال المدافع الإيطالي ليوناردو بونوتشي إن منتخب بلاده ليس خائفا من مواجهة نظيره الانجليزي على أرضه في المباراة النهائية في بطولة يورو 2020 لكرة القدم، في حين وصف المدافع الانجليزي جون ستونز المباراة المرتقبة بأنها ستكون "أكثر تميزا".
        وسوف تقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
        وتسعى إيطاليا لإحراز اللقب الأوروبي للمرة الثانية بعد فوزها به أول مرة عام 1968.
        ولم يفز الفريق الانجليزي بهذا اللقب القاري من قبل. والبطولة الرئيسية الوحيدة التي فازت بها انجلترا هي كأس العالم عام 1966 الذي أقيمت مباراته النهائية في استاد ويمبلي.
    """,

    """
        On a night fraught with tension, Italy clinched its first major title for 15 years with a penalty shootout win over England in the Euro 2020 final.
        Luke Shaw's goal inside the opening two minutes gave England a lead it looked like it would hold onto all night, before a goalmouth scramble midway through the second half allowed Leonardo Bonucci to poke home an equalizer for Italy.
        For the remainder of the match, it felt as though extra-time and penalties were inevitable, as neither side seemed willing or brave enough to commit enough men forward to really trouble the opposing defenders.
        England had suffered innumerable heartbreaks on penalties over the years and this time it was Italy's turn to inflict yet more pain on beleaguered English fans as Marcus Rashford, Jadon Sancho and Bukayo Saka all missed from the spot.
    """,
]

## Get summaries

print("Original Article:", samples[0])
print("\n===========\nSummary: \n")
hls = get_summary(samples[0], tokenizer, model, device)
for hl in hls:
    print("\t-", hl)

print("Original Article:", samples[1])
print("\n=========== \nSummary: \n")
hls = get_summary(samples[1], tokenizer, model, device)
for hl in hls:
    print("\t-", hl)
```


Results


```
Original Article:

        قال المدافع الإيطالي ليوناردو بونوتشي إن منتخب بلاده ليس خائفا من مواجهة نظيره الانجليزي على أرضه في المباراة النهائية في بطولة يورو 2020 لكرة القدم، في حين وصف المدافع الانجليزي جون ستونز المباراة المرتقبة بأنها ستكون "أكثر تميزا".
        وسوف تقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
        وتسعى إيطاليا لإحراز اللقب الأوروبي للمرة الثانية بعد فوزها به أول مرة عام 1968.
        ولم يفز الفريق الانجليزي بهذا اللقب القاري من قبل. والبطولة الرئيسية الوحيدة التي فازت بها انجلترا هي كأس العالم عام 1966 الذي أقيمت مباراته النهائية في استاد ويمبلي.

===========
Summary:

	- وسوف تواجه إيطاليا إنجلترا في بطولة يورو 2020 لكرة القدم يوم الأحد.
	- ستقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
	- ولم يفز الفريق الانجليزي بهذا اللقب القاري قبل.

```


```
Original Article:

        On a night fraught with tension, Italy clinched its first major title for 15 years with a penalty shootout win over England in the Euro 2020 final.
        Luke Shaw's goal inside the opening two minutes gave England a lead it looked like it would hold onto all night, before a goalmouth scramble midway through the second half allowed Leonardo Bonucci to poke home an equalizer for Italy.
        For the remainder of the match it felt as though extra-time and penalties were inevitable, as neither side seemed willing or brave enough to commit enough men forward to really trouble the opposing defenders.
        England had suffered innumerable heartbreaks on penalties over the years and this time it was Italy's turn to inflict yet more pain on beleaguered English fans as Marcus Rashford, Jadon Sancho and Bukayo Saka all missed from the spot.

===========
Summary:

	- Luke Shaw's goal gave England a lead it looked like it would hold onto all night.
	- Leonardo Bonucci scored the equalizer for Italy.
	- Marcus Rashford, Jadon Sancho and Bukayo Saka all missed.
```


model name: summarization-arabic-english-news


model author: marefa-nlp


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, autonlp_compatible


language: ar
datasets:


- Marefa-NER
widget:
- text: "في استاد القاهرة، بدأ حفل افتتاح بطولة كأس الأمم الأفريقية بحضور رئيس الجمهورية و رئيس الاتحاد الدولي لكرة القدم"


# Tebyan تبيـان


## Marefa Arabic Named Entity Recognition Model


## نموذج المعرفة لتصنيف أجزاء النص


</p?


**Version**: 1.3


**Last Update:** 3-12-2021


## Model description


**Marefa-NER** is a Large Arabic Named Entity Recognition (NER) model built on a completely new dataset and targets to extract up to 9 different types of entities


```
Person, Location, Organization, Nationality, Job, Product, Event, Time, Art-Work
```


نموذج المعرفة لتصنيف أجزاء النص. نموذج جديد كليا من حيث البيانات المستخدمة في تدريب النموذج.
كذلك يستهدف النموذج تصنيف حتى 9 أنواع مختلفة من أجزاء النص


```
شخص - مكان - منظمة - جنسية - وظيفة - منتج - حدث - توقيت - عمل إبداعي
```


## How to use كيف تستخدم النموذج


*You can test the model quickly by checking this [Colab notebook](https://colab.research.google.com/drive/1OGp9Wgm-oBM5BBhTLx6Qow4dNRSJZ-F5?usp=sharing)*


Install the following Python packages


`$ pip3 install transformers==4.8.0 nltk==3.5 protobuf==3.15.3 torch==1.9.0 `


> If you are using `Google Colab`, please restart your runtime after installing the packages.


```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

import numpy as np
import nltk
nltk.download('punkt')
from nltk.tokenize import word_tokenize

custom_labels = ["O", "B-job", "I-job", "B-nationality", "B-person", "I-person", "B-location","B-time", "I-time", "B-event", "I-event", "B-organization", "I-organization", "I-location", "I-nationality", "B-product", "I-product", "B-artwork", "I-artwork"]

def _extract_ner(text: str, model: AutoModelForTokenClassification,
                 tokenizer: AutoTokenizer, start_token: str="▁"):
    tokenized_sentence = tokenizer([text], padding=True, truncation=True, return_tensors="pt")
    tokenized_sentences = tokenized_sentence['input_ids'].numpy()

    with torch.no_grad():
        output = model(**tokenized_sentence)

    last_hidden_states = output[0].numpy()
    label_indices = np.argmax(last_hidden_states[0], axis=1)
    tokens = tokenizer.convert_ids_to_tokens(tokenized_sentences[0])
    special_tags = set(tokenizer.special_tokens_map.values())

    grouped_tokens = []
    for token, label_idx in zip(tokens, label_indices):
        if token not in special_tags:
            if not token.startswith(start_token) and len(token.replace(start_token,"").strip()) > 0:
                grouped_tokens[-1]["token"] += token
            else:
                grouped_tokens.append({"token": token, "label": custom_labels[label_idx]})

    # extract entities
    ents = []
    prev_label = "O"
    for token in grouped_tokens:
        label = token["label"].replace("I-","").replace("B-","")
        if token["label"] != "O":

            if label != prev_label:
                ents.append({"token": [token["token"]], "label": label})
            else:
                ents[-1]["token"].append(token["token"])

        prev_label = label

    # group tokens
    ents = [{"token": "".join(rec["token"]).replace(start_token," ").strip(), "label": rec["label"]}  for rec in ents ]

    return ents

model_cp = "marefa-nlp/marefa-ner"

tokenizer = AutoTokenizer.from_pretrained(model_cp)
model = AutoModelForTokenClassification.from_pretrained(model_cp, num_labels=len(custom_labels))

samples = [
    "تلقى تعليمه في الكتاب ثم انضم الى الأزهر عام 1873م. تعلم على يد السيد جمال الدين الأفغاني والشيخ محمد عبده",
    "بعد عودته إلى القاهرة، التحق نجيب الريحاني فرقة جورج أبيض، الذي كان قد ضمَّ - قُبيل ذلك - فرقته إلى فرقة سلامة حجازي . و منها ذاع صيته",
    "في استاد القاهرة، قام حفل افتتاح بطولة كأس الأمم الأفريقية بحضور رئيس الجمهورية و رئيس الاتحاد الدولي لكرة القدم",
    "من فضلك أرسل هذا البريد الى صديقي جلال الدين في تمام الساعة الخامسة صباحا في يوم الثلاثاء القادم",
    "امبارح اتفرجت على مباراة مانشستر يونايتد مع ريال مدريد في غياب الدون كرستيانو رونالدو",
    "لا تنسى تصحيني الساعة سبعة, و ضيف في الجدول اني احضر مباراة نادي النصر غدا",
]

# [optional]
samples = [ " ".join(word_tokenize(sample.strip())) for sample in samples if sample.strip() != "" ]

for sample in samples:
    ents = _extract_ner(text=sample, model=model, tokenizer=tokenizer, start_token="▁")

    print(sample)
    for ent in ents:
        print("\t",ent["token"],"==>",ent["label"])
    print("========\n")

```


Output


```
تلقى تعليمه في الكتاب ثم انضم الى الأزهر عام 1873م . تعلم على يد السيد جمال الدين الأفغاني والشيخ محمد عبده
	 الأزهر ==> organization
	 عام 1873م ==> time
	 السيد جمال الدين الأفغاني ==> person
	 محمد عبده ==> person
========

بعد عودته إلى القاهرة، التحق نجيب الريحاني فرقة جورج أبيض، الذي كان قد ضمَّ - قُبيل ذلك - فرقته إلى فرقة سلامة حجازي . و منها ذاع صيته
	 القاهرة، ==> location
	 نجيب الريحاني ==> person
	 فرقة جورج أبيض، ==> organization
	 فرقة سلامة حجازي ==> organization
========

في استاد القاهرة، قام حفل افتتاح بطولة كأس الأمم الأفريقية بحضور رئيس الجمهورية و رئيس الاتحاد الدولي لكرة القدم
	 استاد القاهرة، ==> location
	 بطولة كأس الأمم الأفريقية ==> event
	 رئيس الجمهورية ==> job
	 رئيس ==> job
	 الاتحاد الدولي لكرة القدم ==> organization
========

من فضلك أرسل هذا البريد الى صديقي جلال الدين في تمام الساعة الخامسة صباحا في يوم الثلاثاء القادم
	 جلال الدين ==> person
	 الساعة الخامسة صباحا ==> time
	 يوم الثلاثاء القادم ==> time
========

امبارح اتفرجت على مباراة مانشستر يونايتد مع ريال مدريد في غياب الدون كرستيانو رونالدو
	 مانشستر يونايتد ==> organization
	 ريال مدريد ==> organization
	 كرستيانو رونالدو ==> person
========

لا تنسى تصحيني الساعة سبعة , و ضيف في الجدول اني احضر مباراة نادي النصر غدا
	 الساعة سبعة ==> time
	 نادي النصر ==> organization
	 غدا ==> time
========
```


## Fine-Tuning


Check this [notebook](https://colab.research.google.com/drive/1WUYrnmDFFEItqGMvbyjqZEJJqwU7xQR-?usp=sharing) to fine-tune the NER model


## Evaluation


We tested the model agains a test set of 1959 sentences. The results is in the follwing table


| type         |   f1-score |   precision |   recall |   support |
|:-------------|-----------:|------------:|---------:|----------:|
| person       |   0.93298  |    0.931479 | 0.934487 |      4335 |
| location     |   0.891537 |    0.896926 | 0.886212 |      4939 |
| time         |   0.873003 |    0.876087 | 0.869941 |      1853 |
| nationality  |   0.871246 |    0.843153 | 0.901277 |      2350 |
| job          |   0.837656 |    0.79912  | 0.880097 |      2477 |
| organization |   0.781317 |    0.773328 | 0.789474 |      2299 |
| event        |   0.686695 |    0.733945 | 0.645161 |       744 |
| artwork      |   0.653552 |    0.678005 | 0.630802 |       474 |
| product      |   0.625483 |    0.553531 | 0.718935 |       338 |
| **weighted avg** |   0.859008 |    0.852365 | 0.86703  |     19809 |
| **micro avg**    |   0.858771 |    0.850669 | 0.86703  |     19809 |
| **macro avg**   |   0.79483  |    0.787286 | 0.806265 |     19809 |


## Acknowledgment شكر و تقدير


قام بإعداد البيانات التي تم تدريب النموذج عليها, مجموعة من المتطوعين الذين قضوا ساعات يقومون بتنقيح البيانات و مراجعتها


- على سيد عبد الحفيظ - إشراف
- نرمين محمد عطيه
- صلاح خيرالله
- احمد علي عبدربه
- عمر بن عبد العزيز سليمان
- محمد ابراهيم الجمال
- عبدالرحمن سلامه خلف
- إبراهيم كمال محمد سليمان
- حسن مصطفى حسن
- أحمد فتحي سيد
- عثمان مندو
- عارف الشريف
- أميرة محمد محمود
- حسن سعيد حسن
- عبد العزيز علي البغدادي
- واثق عبدالملك الشويطر
- عمرو رمضان عقل الحفناوي
- حسام الدين أحمد على
- أسامه أحمد محمد محمد
- حاتم محمد المفتي
- عبد الله دردير
- أدهم البغدادي
- أحمد صبري
- عبدالوهاب محمد محمد
- أحمد محمد عوض


model name: marefa-ner


model author: marefa-nlp


task: token-classification


tags: pytorch, xlm-roberta, token-classification, ar, dataset:Marefa-NER, transformers, autonlp_compatible


{'C0': 'audit_rights',
'C1': 'licensee_indemnity',
'C2': 'licensor_indemnity',
'C3': 'license_grant',
'C4': 'eula_others',
'C5': 'licensee_infringement_indemnity',
'C6': 'licensor_exemption_liability',
'C7': 'licensor_limit_liabilty',
'C8': 'software_warranty'}


model name: KML_Software_License_v1


model author: kloon99


task: text-classification


tags: pytorch, distilbert, text-classification, transformers, infinity_compatible


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: trained_model2
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# trained_model2


This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15.0


### Training results


### Framework versions


- Transformers 4.13.0.dev0
- Pytorch 1.9.1
- Datasets 1.14.0
- Tokenizers 0.10.3


model name: KML_Eula_generate_v2


model author: kloon99


task: text-generation


tags: pytorch, gpt2, text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index


license: apache-2.0
tags:


# trained_model2


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


model name: KML_Eula_generate_v1


- generated_from_trainer
datasets:
- wmt16
metrics:
- bleu
model-index:
- name: opus-mt-en-de-finetuned-en-to-de-model-2
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: wmt16
type: wmt16
args: de-en
metrics:
    - name: Bleu
type: bleu
value: 27.7086


# opus-mt-en-de-finetuned-en-to-de-model-2


This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-de](https://huggingface.co/Helsinki-NLP/opus-mt-en-de) on the wmt16 dataset.
It achieves the following results on the evaluation set:


- Loss: 1.5389
- Bleu: 27.7086
- Gen Len: 24.7238


- learning_rate: 0.0002
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step   | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|
| 1.9613        | 1.0   | 284306 | 1.5389          | 27.7086 | 24.7238 |


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0+cu102
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: opus-mt-en-de-finetuned-en-to-de-model-2


model author: wandemberg-eld


task: text2text-generation


tags: pytorch, marian, text2text-generation, dataset:wmt16, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
datasets:
- wmt16
metrics:
- bleu
model-index:
- name: opus-mt-en-de-finetuned-en-to-de
results:
  - task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: wmt16
type: wmt16
args: de-en
metrics:
    - name: Bleu
type: bleu
value: 29.4312


# opus-mt-en-de-finetuned-en-to-de


- Loss: 1.4083
- Bleu: 29.4312
- Gen Len: 24.746


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step   | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:------:|:---------------:|:-------:|:-------:|
| 1.978         | 1.0   | 568611 | 1.4083          | 29.4312 | 24.746  |


### Framework versions


model name: opus-mt-en-de-finetuned-en-to-de


model author: wandemberg-eld


task: text2text-generation


Testing a new model


model name: SampleModel


model author: WayScriptDerrick


tags:


language: en


license: mit


Import it using pipeline


    from transformers import pipeline
    text_generation = pipeline('text-generation' , model='sparki/kinkyfurs-gpt2')


Then use it


    prefix_text = input()
    text_generation(prefix_text, max_length=50, num_beams=5,no_repeat_ngram_size=2,early_stopping=True)


model name: kinkyfurs-gpt2


model author: sparki


task: text-generation


tags: pytorch, gpt2, text-generation, en, transformers, license:mit


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: run_6
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# run_6


This model is a fine-tuned version of [t5-large](https://huggingface.co/t5-large) on the None dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30


### Training results


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: run_6


model author: adityay1221


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-base_Run_X2
results: []


# t5-base_Run_X2


This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.6185
- Bleu: 45.2719


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.33
- num_epochs: 50
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Bleu    |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| No log        | 10.0  | 50   | 3.2932          | 2.2530  |
| No log        | 20.0  | 100  | 1.3323          | 15.1192 |
| No log        | 30.0  | 150  | 0.7797          | 43.9378 |
| No log        | 40.0  | 200  | 0.6185          | 46.7000 |
| No log        | 50.0  | 250  | 0.5643          | 46.0571 |


### Framework versions


- Transformers 4.14.1
- Pytorch 1.10.1+cu102
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: t5-base_Run_X2


model author: adityay1221


task: text2text-generation


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-base_Run_X4
results: []


# t5-base_Run_X4


- Loss: 0.0634
- Bleu: 94.2179


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Bleu    |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| No log        | 2.78  | 50   | 1.1874          | 17.8807 |
| No log        | 5.56  | 100  | 0.3570          | 54.7073 |
| No log        | 8.33  | 150  | 0.1895          | 57.8991 |
| No log        | 11.11 | 200  | 0.1108          | 63.2364 |
| No log        | 13.89 | 250  | 0.0887          | 63.1200 |
| No log        | 16.67 | 300  | 0.0773          | 65.8133 |
| No log        | 19.44 | 350  | 0.0734          | 66.8279 |
| No log        | 22.22 | 400  | 0.0676          | 67.5501 |
| No log        | 25.0  | 450  | 0.0653          | 66.8709 |
| 0.484         | 27.78 | 500  | 0.0634          | 65.7688 |


### Framework versions


model name: t5-base_Run_X4


model author: adityay1221


task: text2text-generation


license: apache-2.0
tags:


- generated_from_trainer
model-index:
- name: Test_run_1
results: []


# Test_run_1


This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 5
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10


### Training results


- Transformers 4.13.0
- Pytorch 1.10.0+cu111
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: Test_run_1


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-small_Run_X
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# t5-small_Run_X


This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.2410
- Bleu: 76.8898


The following hyperparameters were used during training:


- learning_rate: 7.5e-05
- train_batch_size: 5
- eval_batch_size: 1
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10


| Training Loss | Epoch | Step | Validation Loss | Bleu    |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| No log        | 1.0   | 32   | 2.1699          | 1.2538  |
| No log        | 2.0   | 64   | 0.9729          | 6.1092  |
| No log        | 3.0   | 96   | 0.5267          | 11.1118 |
| No log        | 4.0   | 128  | 0.4019          | 12.6861 |
| No log        | 5.0   | 160  | 0.3371          | 12.4613 |
| No log        | 6.0   | 192  | 0.2971          | 12.9702 |
| No log        | 7.0   | 224  | 0.2718          | 12.8378 |
| No log        | 8.0   | 256  | 0.2524          | 12.9908 |
| No log        | 9.0   | 288  | 0.2446          | 13.0304 |
| No log        | 10.0  | 320  | 0.2410          | 13.0611 |


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: t5-small_Run_X


model author: adityay1221


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: Run_8
results: []


# Run_8


This model is a fine-tuned version of [google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: nan
- Bleu: 0.0843


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 7.5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP


### Training results


### Framework versions


model name: Run_8


model author: adityay1221


task: text2text-generation


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-base_Run_X5
results: []


# t5-base_Run_X5


This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.0011
- Bleu: 99.8146


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 30
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Bleu    |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| No log        | 0.42  | 50   | 3.3251          | 0.8615  |
| No log        | 0.83  | 100  | 1.5917          | 4.7904  |
| No log        | 1.25  | 150  | 0.7086          | 41.1927 |
| No log        | 1.67  | 200  | 0.3970          | 49.1976 |
| No log        | 2.08  | 250  | 0.2014          | 54.0266 |
| No log        | 2.5   | 300  | 0.1058          | 56.0118 |
| No log        | 2.92  | 350  | 0.0686          | 58.9581 |
| No log        | 3.33  | 400  | 0.0441          | 60.0015 |
| No log        | 3.75  | 450  | 0.0239          | 61.0581 |
| 1.0284        | 4.17  | 500  | 0.0191          | 61.4971 |
| 1.0284        | 4.58  | 550  | 0.0129          | 62.0168 |
| 1.0284        | 5.0   | 600  | 0.0128          | 61.9730 |
| 1.0284        | 5.42  | 650  | 0.0082          | 61.8373 |
| 1.0284        | 5.83  | 700  | 0.0054          | 62.7842 |
| 1.0284        | 6.25  | 750  | 0.0054          | 62.9966 |
| 1.0284        | 6.67  | 800  | 0.0045          | 63.0655 |
| 1.0284        | 7.08  | 850  | 0.0031          | 63.2890 |
| 1.0284        | 7.5   | 900  | 0.0028          | 63.6213 |
| 1.0284        | 7.92  | 950  | 0.0024          | 63.4271 |
| 0.0254        | 8.33  | 1000 | 0.0026          | 63.3434 |
| 0.0254        | 8.75  | 1050 | 0.0023          | 63.4567 |
| 0.0254        | 9.17  | 1100 | 0.0019          | 63.4567 |
| 0.0254        | 9.58  | 1150 | 0.0018          | 63.4567 |
| 0.0254        | 10.0  | 1200 | 0.0019          | 63.6718 |
| 0.0254        | 10.42 | 1250 | 0.0014          | 63.5642 |
| 0.0254        | 10.83 | 1300 | 0.0017          | 63.4567 |
| 0.0254        | 11.25 | 1350 | 0.0024          | 63.3006 |
| 0.0254        | 11.67 | 1400 | 0.0015          | 63.4567 |
| 0.0254        | 12.08 | 1450 | 0.0015          | 63.3786 |
| 0.0082        | 12.5  | 1500 | 0.0012          | 63.5642 |
| 0.0082        | 12.92 | 1550 | 0.0012          | 63.4082 |
| 0.0082        | 13.33 | 1600 | 0.0011          | 63.6718 |
| 0.0082        | 13.75 | 1650 | 0.0009          | 63.6718 |
| 0.0082        | 14.17 | 1700 | 0.0013          | 63.4567 |
| 0.0082        | 14.58 | 1750 | 0.0014          | 63.5642 |
| 0.0082        | 15.0  | 1800 | 0.0011          | 63.5642 |
| 0.0082        | 15.42 | 1850 | 0.0010          | 63.5158 |
| 0.0082        | 15.83 | 1900 | 0.0011          | 63.3301 |
| 0.0082        | 16.25 | 1950 | 0.0015          | 63.4567 |
| 0.0052        | 16.67 | 2000 | 0.0015          | 63.2225 |


### Framework versions


- Transformers 4.14.1
- Pytorch 1.10.1+cu102
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: t5-base_Run_X5


model author: adityay1221


task: text2text-generation


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-base_Run_X6
results: []


# t5-base_Run_X6


- Loss: 0.0011
- Bleu: 99.8146


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


### Training results


model name: t5-base_Run_X6


- generated_from_trainer
model-index:
- name: run_5
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# run_5


This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the None dataset.


The following hyperparameters were used during training:


- learning_rate: 0.001
- train_batch_size: 1
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10


### Framework versions


model name: run_5


model author: adityay1221


task: text2text-generation


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: t5-large_Run_X3
results: []


# t5-large_Run_X3


This model is a fine-tuned version of [t5-large](https://huggingface.co/t5-large) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: nan
- Bleu: 0.4892


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.33
- training_steps: 500
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Bleu   |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| No log        | 2.5   | 50   | nan             | 0.5945 |
| No log        | 5.0   | 100  | nan             | 0.5945 |
| No log        | 7.5   | 150  | nan             | 0.5945 |
| No log        | 10.0  | 200  | nan             | 0.5945 |
| No log        | 12.5  | 250  | nan             | 0.5945 |


### Framework versions


model name: t5-large_Run_X3


model author: adityay1221


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- bleu
model-index:
- name: Run_7
results: []


# Run_7


- Loss: nan
- Bleu: 0.1076


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 7.5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 121
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP


### Training results


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0
- Datasets 1.15.1
- Tokenizers 0.10.3


model name: Run_7


model author: adityay1221


task: text2text-generation


license: apache-2.0
datasets:


- null
model_index:
- name: dear-jarvis-v5
results:
  - task:
name: Text Classification
type: text-classification


# dear-jarvis-v5


This model is a fine-tuned version of [distilbert-base-cased](https://huggingface.co/distilbert-base-cased) on the None dataset.
It achieves the following results on the evaluation set:


- Loss: 0.3148


## Model description


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3


### Training results


| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 470  | 0.3106          |
| 0.3452        | 2.0   | 940  | 0.3064          |
| 0.2692        | 3.0   | 1410 | 0.3148          |


### Framework versions


- Transformers 4.7.0
- Pytorch 1.9.0+cu102
- Datasets 1.8.0
- Tokenizers 0.10.3


model name: dear-jarvis-v5


model author: JuliusAlphonso


task: text-classification


tags: pytorch, distilbert, text-classification, transformers, license:apache-2.0, infinity_compatible


Labels are based on Plutchik's model of emotions and may be combined:
![image](https://user-images.githubusercontent.com/12978899/122398897-f60d2500-cf97-11eb-8991-61e68f4ea1fc.png)


model name: distilbert-plutchik


tags: pytorch, distilbert, text-classification, transformers, infinity_compatible


## Model description


This model was trained on the XED dataset and achieved\
validation loss: 0.5995\
validation acc: 84.28% (ROC-AUC)


- Transformers 4.6.1
- Pytorch 1.8.1+cu101
- Datasets 1.8.0
- Tokenizers 0.10.3


model name: dear-jarvis-monolith-xed-en


language: en
license: mit
datasets:


- bookcorpus
- wikipedia


# XLNet (base-sized model)


XLNet model pre-trained on English language. It was introduced in the paper [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://arxiv.org/abs/1906.08237) by Yang et al. and first released in [this repository](https://github.com/zihangdai/xlnet/).


Disclaimer: The team releasing XLNet did not write a model card for this model so this model card has been written by the Hugging Face team.


XLNet is a new unsupervised language representation learning method based on a novel generalized permutation language modeling objective. Additionally, XLNet employs Transformer-XL as the backbone model, exhibiting excellent performance for language tasks involving long context. Overall, XLNet achieves state-of-the-art (SOTA) results on various downstream language tasks including question answering, natural language inference, sentiment analysis, and document ranking.


## Intended uses & limitations


The model is mostly intended to be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?search=xlnet) to look for fine-tuned versions on a task that interests you.


Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at models like GPT2.


## Usage


Here is how to use this model to get the features of a given text in PyTorch:


```python
from transformers import XLNetTokenizer, XLNetModel

tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
model = XLNetModel.from_pretrained('xlnet-base-cased')

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state
```


### BibTeX entry and citation info


```bibtex
@article{DBLP:journals/corr/abs-1906-08237,
  author    = {Zhilin Yang and
               Zihang Dai and
               Yiming Yang and
               Jaime G. Carbonell and
               Ruslan Salakhutdinov and
               Quoc V. Le},
  title     = {XLNet: Generalized Autoregressive Pretraining for Language Understanding},
  journal   = {CoRR},
  volume    = {abs/1906.08237},
  year      = {2019},
  url       = {http://arxiv.org/abs/1906.08237},
  eprinttype = {arXiv},
  eprint    = {1906.08237},
  timestamp = {Mon, 24 Jun 2019 17:28:45 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1906-08237.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
```


model name: xlnet-base-cased


task: text-generation


tags: pytorch, tf, rust, xlnet, text-generation, en, dataset:bookcorpus, dataset:wikipedia, arxiv:1906.08237, transformers, license:mit


language: "en"
tags:


- paraphrase-generation
- text-generation
- Conditional Generation
inference: false


# Paraphrase-Generation


## Model description


T5 Model for generating paraphrases of english sentences. Trained on the [Google PAWS](https://github.com/google-research-datasets/paws) dataset.


## How to use


PyTorch and TF models available


```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Vamsi/T5_Paraphrase_Paws")
model = AutoModelForSeq2SeqLM.from_pretrained("Vamsi/T5_Paraphrase_Paws")

sentence = "This is something which i cannot understand at all"

text =  "paraphrase: " + sentence + " </s>"

encoding = tokenizer.encode_plus(text,pad_to_max_length=True, return_tensors="pt")
input_ids, attention_masks = encoding["input_ids"].to("cuda"), encoding["attention_mask"].to("cuda")

outputs = model.generate(
    input_ids=input_ids, attention_mask=attention_masks,
    max_length=256,
    do_sample=True,
    top_k=120,
    top_p=0.95,
    early_stopping=True,
    num_return_sequences=5
)

for output in outputs:
    line = tokenizer.decode(output, skip_special_tokens=True,clean_up_tokenization_spaces=True)
    print(line)

```


For more reference on training your own T5 model or using this model, do check out [Paraphrase Generation](https://github.com/Vamsi995/Paraphrase-Generator).


model name: T5_Paraphrase_Paws


model author: Vamsi


tags: pytorch, tf, jax, t5, text2text-generation, en, transformers, paraphrase-generation, text-generation, Conditional Generation, autonlp_compatible


license: apache-2.0
tags:


- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-small-herblabels
results: []


<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# t5-small-herblabels


This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on an unknown dataset.
It achieves the following results on the evaluation set:


- Loss: 0.4823
- Rouge1: 3.0759
- Rouge2: 1.0495
- Rougel: 3.0758
- Rougelsum: 3.0431
- Gen Len: 18.9716


More information needed


## Intended uses & limitations


## Training and evaluation data


## Training procedure


### Training hyperparameters


The following hyperparameters were used during training:


- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP


### Training results


| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| No log        | 1.0   | 264  | 1.6010          | 2.4276 | 0.5658 | 2.3546 | 2.3099    | 18.9091 |
| 2.5052        | 2.0   | 528  | 1.0237          | 2.9016 | 0.3395 | 2.8221 | 2.783     | 18.9673 |
| 2.5052        | 3.0   | 792  | 0.7793          | 2.962  | 0.3091 | 2.9375 | 2.8786    | 18.9588 |
| 1.1552        | 4.0   | 1056 | 0.6530          | 2.98   | 0.4375 | 2.9584 | 2.8711    | 18.9588 |
| 1.1552        | 5.0   | 1320 | 0.5863          | 3.0023 | 0.5882 | 2.987  | 2.9155    | 18.9588 |
| 0.8659        | 6.0   | 1584 | 0.5428          | 3.0576 | 0.8019 | 3.0494 | 2.9989    | 18.9716 |
| 0.8659        | 7.0   | 1848 | 0.5145          | 3.0808 | 0.9476 | 3.0719 | 3.0237    | 18.9716 |
| 0.747         | 8.0   | 2112 | 0.4962          | 3.0748 | 1.0032 | 3.0683 | 3.0359    | 18.9716 |
| 0.747         | 9.0   | 2376 | 0.4856          | 3.0702 | 1.0196 | 3.0665 | 3.0328    | 18.9716 |
| 0.6987        | 10.0  | 2640 | 0.4823          | 3.0759 | 1.0495 | 3.0758 | 3.0431    | 18.9716 |


### Framework versions


- Transformers 4.12.5
- Pytorch 1.10.0
- Datasets 1.16.1
- Tokenizers 0.10.3


model name: t5-small-herblabels


model author: ggosline


task: text2text-generation


tags: pytorch, t5, text2text-generation, transformers, license:apache-2.0, generated_from_trainer, model-index, autonlp_compatible


language:


- en
thumbnail: https://cogcomp.seas.upenn.edu/images/logo.png
tags:
- text-classification
- bart
- xsum
license: cc-by-sa-4.0
datasets:
- xsum
widget:
- text: "<s> Ban Ki-moon was elected for a second term in 2007. </s></s> Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."
- text: "<s> Ban Ki-moon was elected for a second term in 2011. </s></s> Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."


# bart-faithful-summary-detector


## Model description


A BART (base) model trained to classify whether a summary is *faithful* to the original article. See our [paper in NAACL'21](https://www.seas.upenn.edu/~sihaoc/static/pdf/CZSR21.pdf) for details.


## Usage


Concatenate a summary and a source document as input (note that the summary needs to be the **first** sentence).


Here's an example usage (with PyTorch)


```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("CogComp/bart-faithful-summary-detector")
model = AutoModelForSequenceClassification.from_pretrained("CogComp/bart-faithful-summary-detector")

article = "Ban Ki-Moon was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011."

bad_summary = "Ban Ki-moon was elected for a second term in 2007."
good_summary = "Ban Ki-moon was elected for a second term in 2011."

bad_pair = tokenizer(text=bad_summary, text_pair=article, return_tensors='pt')
good_pair = tokenizer(text=good_summary, text_pair=article, return_tensors='pt')

bad_score = model(**bad_pair)
good_score = model(**good_pair)

print(good_score[0][:, 1] > bad_score[0][:, 1]) # True, label mapping: "0" -> "Hallucinated" "1" -> "Faithful"
```


### BibTeX entry and citation info


```bibtex
@inproceedings{CZSR21,
    author = {Sihao Chen and Fan Zhang and Kazoo Sone and Dan Roth},
    title = {{Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection}},
    booktitle = {NAACL},
    year = {2021}
}
```


model name: bart-faithful-summary-detector


model author: CogComp


task: text-classification


tags: pytorch, jax, bart, text-classification, en, dataset:xsum, transformers, xsum, license:cc-by-sa-4.0


Don’t worry we will make available extremely beautiful and Sensuous Mahipalpur escorts to make your stay in this area exciting and thrilling. The feminine charm of curvy Site: - Mahipalpur Escorts


Site: - https://www.sarakaur.com/mahipalpur-escorts.php
Site: - https://www.missdelhiescort.com/mahipalpur-escorts.html


https://cutt.ly/yUZ81up     https://cutt.ly/uUZ4rZa   https://cutt.ly/NOlaLek


model name: MahipalpurEscortsModel


model author: ankitatop10


tags:


language:


- en
thumbnail: https://raw.githubusercontent.com/JetRunner/BERT-of-Theseus/master/bert-of-theseus.png
tags:
- sequence
- classification
license: apache-2.0
datasets:
- imdb
metrics:
- accuracy


model name: bert-base-cased-imdb-sequence-classification


model author: ykacer


tags: pytorch, jax, bert, text-classification, en, dataset:imdb, transformers, sequence, classification, license:apache-2.0, infinity_compatible


This is the BERT-Medium model from Google: https://github.com/google-research/bert#bert. A BERT model with 8 layers, 512 hidden unit size, and 8 attention heads.


model name: BERT-Medium_L-8_H-512_A-8


model author: nreimers


task: feature-extraction


tags: pytorch, jax, bert, feature-extraction, transformers, infinity_compatible


# albert-small-v2


This is a 6 layer version of [albert-base-v2](https://huggingface.co/albert-base-v2).


model name: albert-small-v2


tags: pytorch, albert, feature-extraction, transformers, infinity_compatible


# MiniLMv2


This is a MiniLMv2 model from: [https://github.com/microsoft/unilm](https://github.com/microsoft/unilm/tree/master/minilm)


model name: MiniLMv2-L6-H384-distilled-from-BERT-Base


task: fill-mask


tags: pytorch, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


## license: mit


## MiniLM: 3 Layer Version


This is a 3 layer version of [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased/) by keeping only the layer [3, 7, 11].


model name: MiniLM-L3-H384-uncased


tags: pytorch, bert, feature-extraction, transformers, license:mit, infinity_compatible


# Multilingual MiniLMv2


model name: mMiniLMv2-L12-H384-distilled-from-XLMR-Large


model author: nreimers


tags: pytorch, xlm-roberta, fill-mask, transformers, autonlp_compatible


# BERT-Small-L-4_H-512_A-8


This is a port of the [BERT-Small model](https://github.com/google-research/bert) to Pytorch. It uses 4 layers, a hidden size of 512 and 8 attention heads.


model name: BERT-Small-L-4_H-512_A-8


task: feature-extraction


# MiniLMv2


model name: MiniLMv2-L6-H384-distilled-from-RoBERTa-Large


task: fill-mask


tags: pytorch, roberta, fill-mask, transformers, autonlp_compatible, infinity_compatible


model name: MiniLMv2-L6-H768-distilled-from-BERT-Base


model name: MiniLMv2-L12-H384-distilled-from-RoBERTa-Large


This is the [General_TinyBERT_v2(6layer-768dim)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT) ported to Huggingface transformers.


model name: TinyBERT_L-6_H-768_v2


This is the BERT-Medium model from Google: https://github.com/google-research/bert#bert. A BERT model with 4 layers, 256 hidden unit size, and 4 attention heads.


model name: BERT-Mini_L-4_H-256_A-4


tags: pytorch, jax, bert, feature-extraction, transformers, infinity_compatible


This is the [General_TinyBERT_v2(4layer-312dim)](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT) ported to Huggingface transformers.


model name: TinyBERT_L-4_H-312_v2


model author: nreimers


model name: MiniLMv2-L6-H768-distilled-from-RoBERTa-Large


This is a MiniLMv2 model from: [https://github.com/microsoft/unilm](https://github.com/microsoft/unilm/tree/master/minilm)


model name: MiniLMv2-L6-H768-distilled-from-BERT-Large


tags: pytorch, bert, fill-mask, transformers, autonlp_compatible, infinity_compatible


# MiniLMv2


model name: mMiniLMv2-L6-H384-distilled-from-XLMR-Large


task: fill-mask


## license: mit


## MiniLM: 6 Layer Version


This is a 6 layer version of [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased/) by keeping only every second layer.


model name: MiniLM-L6-H384-uncased


task: feature-extraction


tags: pytorch, jax, bert, feature-extraction, transformers, license:mit, infinity_compatible


This is the BERT-Medium model from Google: https://github.com/google-research/bert#bert. A BERT model with 2 layers, 128 hidden unit size, and 2 attention heads.


model name: BERT-Tiny_L-2_H-128_A-2


model name: MiniLMv2-L6-H384-distilled-from-BERT-Large


model author: nreimers


This is the zero-shot baseline model in the paper ["GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"](https://arxiv.org/abs/2112.07577)


The training setup:


1. Start from `distilbert-base-uncased`;
1. Mine 50 hard negatives for each query on MS MARCO with `sentence-transformers/msmarco-distilbert-base-v3` and `sentence-transformers/msmarco-MiniLM-L-6-v3`;
1. Do Margin-MSE training on the tuples (including queries, gold relevant, and hard negatives) with the teacher model `cross-encoder/ms-marco-MiniLM-L-6-v2` for 70K steps with batch size 75, max. sequence-length 350.


model name: msmarco-distilbert-margin-mse


model author: GPL


tags: pytorch, distilbert, feature-extraction, arxiv:2112.07577, transformers, infinity_compatible


Naming pattern:


1. `GPL/${dataset}-msmarco-distilbert-gpl`: Model with training order of (1) MarginMSE on MSMARCO -> (2) GPL on ${dataset};
1. `GPL/${dataset}-tsdae-msmarco-distilbert-gpl`: Model with training order of (1) TSDAE on ${dataset} -> (2) MarginMSE on MSMARCO -> (3) GPL on ${dataset};
1. `GPL/msmarco-distilbert-margin-mse`: Model trained on MSMARCO with MarginMSE;
1. `GPL/${dataset}-tsdae-msmarco-distilbert-margin-mse`: Model with training order of (1) TSDAE on ${dataset} -> (2) MarginMSE on MSMARCO;


Actually, models in 1. and 2. are built on top of 3. and 4., respectively.


model name: README


tags:


language: multilingual
tags:


- LID
- spoken language recognition
license: apache-2.0
datasets:
- VoxLingua107
metrics:
- ER
inference: false


# Spoken Language Identification Model


## Model description


The model can classify a speech utterance according to the language spoken.
It covers following different languages (
English,
Indonesian,
Japanese,
Korean,
Thai,
Vietnamese,
Mandarin Chinese).


model name: LanguageIdentification


model author: AkshaySg


tags: multilingual, dataset:VoxLingua107, LID, spoken language recognition, license:apache-2.0


language: multilingual
thumbnail:
tags:


- audio-classification
- speechbrain
- embeddings
- Language
- Identification
- pytorch
- ECAPA-TDNN
- TDNN
- VoxLingua107
license: "apache-2.0"
datasets:
- VoxLingua107
metrics:
- Accuracy
widget:
- example_title: English Sample
src: https://cdn-media.huggingface.co/speech_samples/LibriSpeech_61-70968-0000.flac


# VoxLingua107 ECAPA-TDNN Spoken Language Identification Model


This is a spoken language recognition model trained on the VoxLingua107 dataset using SpeechBrain.
The model uses the ECAPA-TDNN architecture that has previously been used for speaker recognition.


The model can classify a speech utterance according to the language spoken.
It covers 107 different languages (
Abkhazian,
Afrikaans,
Amharic,
Arabic,
Assamese,
Azerbaijani,
Bashkir,
Belarusian,
Bulgarian,
Bengali,
Tibetan,
Breton,
Bosnian,
Catalan,
Cebuano,
Czech,
Welsh,
Danish,
German,
Greek,
English,
Esperanto,
Spanish,
Estonian,
Basque,
Persian,
Finnish,
Faroese,
French,
Galician,
Guarani,
Gujarati,
Manx,
Hausa,
Hawaiian,
Hindi,
Croatian,
Haitian,
Hungarian,
Armenian,
Interlingua,
Indonesian,
Icelandic,
Italian,
Hebrew,
Japanese,
Javanese,
Georgian,
Kazakh,
Central Khmer,
Kannada,
Korean,
Latin,
Luxembourgish,
Lingala,
Lao,
Lithuanian,
Latvian,
Malagasy,
Maori,
Macedonian,
Malayalam,
Mongolian,
Marathi,
Malay,
Maltese,
Burmese,
Nepali,
Dutch,
Norwegian Nynorsk,
Norwegian,
Occitan,
Panjabi,
Polish,
Pushto,
Portuguese,
Romanian,
Russian,
Sanskrit,
Scots,
Sindhi,
Sinhala,
Slovak,
Slovenian,
Shona,
Somali,
Albanian,
Serbian,
Sundanese,
Swedish,
Swahili,
Tamil,
Telugu,
Tajik,
Thai,
Turkmen,
Tagalog,
Turkish,
Tatar,
Ukrainian,
Urdu,
Uzbek,
Vietnamese,
Waray,
Yiddish,
Yoruba,
Mandarin Chinese).


## Intended uses & limitations


The model has two uses:


- use 'as is' for spoken language recognition
- use as an utterance-level feature (embedding) extractor, for creating a dedicated language ID model on your own data


The model is trained on automatically collected YouTube data. For more
information about the dataset, see [here](http://bark.phon.ioc.ee/voxlingua107/).


#### How to use


```python
import torchaudio
from speechbrain.pretrained import EncoderClassifier
language_id = EncoderClassifier.from_hparams(source="TalTechNLP/voxlingua107-epaca-tdnn", savedir="tmp")
# Download Thai language sample from Omniglot and cvert to suitable form
signal = language_id.load_audio("https://omniglot.com/soundfiles/udhr/udhr_th.mp3")
prediction =  language_id.classify_batch(signal)
print(prediction)
  (tensor([[0.3210, 0.3751, 0.3680, 0.3939, 0.4026, 0.3644, 0.3689, 0.3597, 0.3508,
           0.3666, 0.3895, 0.3978, 0.3848, 0.3957, 0.3949, 0.3586, 0.4360, 0.3997,
           0.4106, 0.3886, 0.4177, 0.3870, 0.3764, 0.3763, 0.3672, 0.4000, 0.4256,
           0.4091, 0.3563, 0.3695, 0.3320, 0.3838, 0.3850, 0.3867, 0.3878, 0.3944,
           0.3924, 0.4063, 0.3803, 0.3830, 0.2996, 0.4187, 0.3976, 0.3651, 0.3950,
           0.3744, 0.4295, 0.3807, 0.3613, 0.4710, 0.3530, 0.4156, 0.3651, 0.3777,
           0.3813, 0.6063, 0.3708, 0.3886, 0.3766, 0.4023, 0.3785, 0.3612, 0.4193,
           0.3720, 0.4406, 0.3243, 0.3866, 0.3866, 0.4104, 0.4294, 0.4175, 0.3364,
           0.3595, 0.3443, 0.3565, 0.3776, 0.3985, 0.3778, 0.2382, 0.4115, 0.4017,
           0.4070, 0.3266, 0.3648, 0.3888, 0.3907, 0.3755, 0.3631, 0.4460, 0.3464,
           0.3898, 0.3661, 0.3883, 0.3772, 0.9289, 0.3687, 0.4298, 0.4211, 0.3838,
           0.3521, 0.3515, 0.3465, 0.4772, 0.4043, 0.3844, 0.3973, 0.4343]]), tensor([0.9289]), tensor([94]), ['th'])
# The scores in the prediction[0] tensor can be interpreted as cosine scores between
# the languages and the given utterance (i.e., the larger the better)
# The identified language ISO code is given in prediction[3]
print(prediction[3])
  ['th']

# Alternatively, use the utterance embedding extractor:
emb =  language_id.encode_batch(signal)
print(emb.shape)
  torch.Size([1, 1, 256])
```


#### Limitations and bias


Since the model is trained on VoxLingua107, it has many limitations and biases, some of which are:


- Probably it's accuracy on smaller languages  is quite limited
- Probably it works worse on female speech than male speech (because YouTube data includes much more male speech)
- Based on subjective experiments, it doesn't work well on speech with a foreign accent
- Probably it doesn't work well on children's speech and on persons with speech disorders


## Training data


The model is trained on [VoxLingua107](http://bark.phon.ioc.ee/voxlingua107/).


VoxLingua107 is a speech dataset for training spoken language identification models.
The dataset consists of short speech segments automatically extracted from YouTube videos and labeled according the language of the video title and description, with some post-processing steps to filter out false positives.


VoxLingua107 contains data for 107 languages. The total amount of speech in the training set is 6628 hours.
The average amount of data per language is 62 hours. However, the real amount per language varies a lot. There is also a seperate development set containing 1609 speech segments from 33 languages, validated by at least two volunteers to really contain the given language.


## Training procedure


We used [SpeechBrain](https://github.com/speechbrain/speechbrain) to train the model.
Training recipe will be published soon.


## Evaluation results


Error rate: 7% on the development dataset


### BibTeX entry and citation info