---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# aditeyabaral/sentencetransformer-roberta-hinglish-big

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

<!--- Describe your model here -->

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-roberta-hinglish-big')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-big')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-roberta-hinglish-big')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

<!--- Describe how your model was evaluated -->

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-roberta-hinglish-big)

## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 4617 with parameters:
```
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`

Parameters of the fit()-Method:
```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: RobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

<!--- Describe where people can find more information -->


---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
---

# aditeyabaral/sentencetransformer-xlm-roberta-base

This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

<!--- Describe your model here -->

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('aditeyabaral/sentencetransformer-xlm-roberta-base')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('aditeyabaral/sentencetransformer-xlm-roberta-base')
model = AutoModel.from_pretrained('aditeyabaral/sentencetransformer-xlm-roberta-base')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

<!--- Describe how your model was evaluated -->

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=aditeyabaral/sentencetransformer-xlm-roberta-base)

## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 9234 with parameters:
```
{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`

Parameters of the fit()-Method:
```
{
    "epochs": 10,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 100,
    "weight_decay": 0.01
}
```

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

<!--- Describe where people can find more information -->


Technique Classification for https://propaganda.qcri.org/ptc/index.html


---
language: Bengali
datasets:
- OpenSLR
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: cc-by-sa-4.0
model-index:
- name: XLSR Wav2Vec2 Bengali by Tanmoy Sarkar
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: OpenSLR
      type: OpenSLR
      args: ben
    metrics:
    - name: Test WER
      type: wer
      value: 88.58
---
# Wav2Vec2-Large-XLSR-Bengali
Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) Bengali using the [Bengali ASR training data set containing ~196K utterances](https://www.openslr.org/53/).
When using this model, make sure that your speech input is sampled at 16kHz.
## Usage
Dataset must be downloaded from [this website](https://www.openslr.org/53/) and preprocessed accordingly. For example 1250 test samples has been chosen.
```python
import pandas as pd
test_dataset = pd.read_csv('utt_spk_text.tsv', sep='\\t', header=None)[60000:61250]
test_dataset.columns = ["audio_path", "__", "label"]
test_dataset = test_data.drop("__", axis=1)
def add_file_path(text):
  path = "data/" + text[:2] + "/" + text + '.flac'
  return path
test_dataset['audio_path'] = test_dataset['audio_path'].map(lambda x: add_file_path(x))
```
The model can be used directly (without a language model) as follows:
```python
import torch
import torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
processor = Wav2Vec2Processor.from_pretrained("tanmoyio/wav2vec2-large-xlsr-bengali")
model = Wav2Vec2ForCTC.from_pretrained("tanmoyio/wav2vec2-large-xlsr-bengali")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
  speech_array, sampling_rate = torchaudio.load(batch["audio_path"])
  batch["speech"] = resampler(speech_array).squeeze().numpy()
  return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
with torch.no_grad():
  logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["label"][:2])
```
## Evaluation
The model can be evaluated as follows on the Bengali test data of OpenSLR.
```python
import torch
import torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re
wer = load_metric("wer")
processor = Wav2Vec2Processor.from_pretrained("tanmoyio/wav2vec2-large-xlsr-bengali")
model = Wav2Vec2ForCTC.from_pretrained("tanmoyio/wav2vec2-large-xlsr-bengali")
model.to("cuda")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
  batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["label"]).lower()
  speech_array, sampling_rate = torchaudio.load(batch["path"])
  batch["speech"] = resampler(speech_array).squeeze().numpy()
  return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
  inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)
  with torch.no_grad():
      logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits
      pred_ids = torch.argmax(logits, dim=-1)
      batch["pred_strings"] = processor.batch_decode(pred_ids)
      return batch
result = test_dataset.map(evaluate, batched=True, batch_size=8)
print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
```
**Test Result**: 88.58 %
## Training
The script used for training can be found [Bengali ASR Fine Tuning Wav2Vec2](https://colab.research.google.com/drive/1Bkc5C_cJV9BeS0FD0MuHyayl8hqcbdRZ?usp=sharing)


# Transferring Monolingual Model to Low-Resource Language: The Case Of Tigrinya:

## Proposed Method:

The proposed method transfers a mono-lingual Transformer model into new target language at lexical level by learning new token embeddings. All implementation in this repo uses XLNet as a source Transformer model, however, other Transformer models can also be used similarly.

## Main files:
All files are IPython Notebook files which can be excuted simply in Google Colab.

 - train.ipynb : Fine-tunes XLNet (mono-lingual transformer) on new target language (Tigrinya) sentiment analysis dataset.  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1bSSrKE-TSphUyrNB2UWhFI-Bkoz0a5l0?usp=sharing)

 - test.ipynb : Evaluates the fine-tuned model on test data.  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/17R1lvRjxILVNk971vzZT79o2OodwaNIX?usp=sharing)

 - token_embeddings.ipynb : Trains a word2vec token embeddings for Tigrinya language.  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1hCtetAllAjBw28EVQkJFpiKdFtXmuxV7?usp=sharing)

 - process_Tigrinya_comments.ipynb : Extracts Tigrinya comments from mixed language contents.  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1-ndLlBV-iLZNBW3Z8OfKAqUUCjvGbdZU?usp=sharing)

 - extract_YouTube_comments.ipynb : Downloads available comments from a YouTube channel ID.  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1b7G85wHKe18y45JIDtvDJdO5dOkRmDdp?usp=sharing)

 - auto_labelling.ipynb : Automatically labels Tigrinya comments in to positive or negative sentiments based on [Emoji's sentiment](http://kt.ijs.si/data/Emoji_sentiment_ranking/).    [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1wnZf7CBBCIr966vRUITlxKCrANsMPpV7?usp=sharing)

## Tigrinya Tokenizer:

A [sentencepiece](https://github.com/google/sentencepiece) based tokenizer for Tigrinya has been released to the public and can be accessed as in the following:

     from transformers import AutoTokenizer
     tokenizer = AutoTokenizer.from_pretrained("abryee/TigXLNet")
     tokenizer.tokenize("ዋዋዋው እዛ ፍሊም ካብተን ዘድንቀን ሓንቲ ኢያ ሞ ብጣዕሚ ኢና ነመስግን ሓንቲ ክብላ ደልየ ዘሎኹ ሓደራኣኹም ኣብ ጊዜኹም ተረክቡ")

 ## TigXLNet:
 A new general purpose transformer model for low-resource language Tigrinya is also released to the public and be accessed as in the following:

    from transformers import AutoConfig, AutoModel
    config = AutoConfig.from_pretrained("abryee/TigXLNet")
    config.d_head = 64
    model = AutoModel.from_pretrained("abryee/TigXLNet", config=config)

 ## Evaluation:

 The proposed method is evaluated using two datasets:
  - A newly created sentiment analysis dataset for low-resource language (Tigriyna).

            Models
            Configuration
            F1-Score

            BERT
            +Frozen BERT weights
            54.91

            +Random embeddings
            74.26

            +Frozen token embeddings
            76.35

            mBERT
            +Frozen mBERT weights
            57.32

            +Random embeddings
            76.01

            +Frozen token embeddings
            77.51

            XLNet
            +Frozen XLNet weights
            68.14

            +Random embeddings
            77.83

            +Frozen token embeddings
            81.62

  - Cross-lingual Sentiment dataset ([CLS](https://zenodo.org/record/3251672#.Xs65VzozbIU)).

            Models
            English
            German
            French
            Japanese
            Average

            Books
            DVD
            Music
            Books
            DVD
            Music
            Books
            DVD
            Music
            Books
            DVD
            Music

            XLNet
            92.90
            93.31
            92.02
            85.23
            83.30
            83.89
            73.05
            69.80
            70.12
            83.20
            86.07
            85.24
            83.08

            mBERT
            92.78
            90.30
            91.88
            88.65
            85.85
            90.38
            91.09
            88.57
            93.67
            84.35
            81.77
            87.53
            88.90

## Dataset used for this paper:
We have constructed new sentiment analysis dataset for Tigrinya language and it can be found in the zip file (Tigrinya Sentiment Analysis Dataset)

## Citing our paper:

 Our paper can be accessed from ArXiv [link](https://arxiv.org/pdf/2006.07698.pdf), and please consider citing our work.

     @misc{tela2020transferring,
          title={Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya},
          author={Abrhalei Tela and Abraham Woubie and Ville Hautamaki},
          year={2020},
          eprint={2006.07698},
          archivePrefix={arXiv},
          primaryClass={cs.CL}
      }
## Any questions, comments, feedback is appreciated! And can be forwarded to the following email: abrhalei.tela@gmail.com


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model_index:
- name: distilbert-base-uncased-finetuned-cola
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: glue
      type: glue
      args: cola
    metric:
      name: Matthews Correlation
      type: matthews_correlation
      value: 0.5481326292844919
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-cola

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8301
- Matthews Correlation: 0.5481

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5252        | 1.0   | 535  | 0.5094          | 0.4268               |
| 0.3515        | 2.0   | 1070 | 0.5040          | 0.4948               |
| 0.2403        | 3.0   | 1605 | 0.5869          | 0.5449               |
| 0.1731        | 4.0   | 2140 | 0.7338          | 0.5474               |
| 0.1219        | 5.0   | 2675 | 0.8301          | 0.5481               |

### Framework versions

- Transformers 4.9.2
- Pytorch 1.9.0
- Datasets 1.11.0
- Tokenizers 0.10.3


---
language:
- fa
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_7_0
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: common7
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# common7

This model is a fine-tuned version of [common7/checkpoint-18500](https://huggingface.co/common7/checkpoint-18500) on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - FA dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3448
- Wer: 0.3478

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 6e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 150.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| 2.957         | 3.29   | 500   | 2.9503          | 1.0    |
| 1.7225        | 6.58   | 1000  | 0.8860          | 0.7703 |
| 1.4907        | 9.86   | 1500  | 0.6555          | 0.6673 |
| 1.4177        | 13.16  | 2000  | 0.5784          | 0.6076 |
| 1.3425        | 16.45  | 2500  | 0.5379          | 0.5718 |
| 1.33          | 19.73  | 3000  | 0.4962          | 0.5245 |
| 1.4378        | 23.03  | 3500  | 0.4699          | 0.5098 |
| 1.1894        | 26.31  | 4000  | 0.4527          | 0.4848 |
| 1.1844        | 29.6   | 4500  | 0.4309          | 0.4651 |
| 1.1795        | 32.89  | 5000  | 0.4131          | 0.4524 |
| 1.1471        | 36.18  | 5500  | 0.4052          | 0.4435 |
| 1.1337        | 39.47  | 6000  | 0.3927          | 0.4363 |
| 1.1896        | 42.76  | 6500  | 0.3811          | 0.4254 |
| 1.1847        | 46.05  | 7000  | 0.3855          | 0.4129 |
| 0.9954        | 49.34  | 7500  | 0.3729          | 0.3981 |
| 1.0293        | 52.63  | 8000  | 0.3637          | 0.4014 |
| 1.0224        | 55.92  | 8500  | 0.3578          | 0.3885 |
| 1.012         | 59.21  | 9000  | 0.3629          | 0.3930 |
| 1.0772        | 62.5   | 9500  | 0.3635          | 0.3906 |
| 1.0344        | 65.79  | 10000 | 0.3469          | 0.3771 |
| 0.9457        | 69.08  | 10500 | 0.3435          | 0.3735 |
| 0.9307        | 72.37  | 11000 | 0.3519          | 0.3762 |
| 0.9523        | 75.65  | 11500 | 0.3443          | 0.3666 |
| 0.9523        | 78.94  | 12000 | 0.3502          | 0.3757 |
| 0.9475        | 82.24  | 12500 | 0.3509          | 0.3643 |
| 0.9971        | 85.52  | 13000 | 0.3502          | 0.3626 |
| 0.9058        | 88.81  | 13500 | 0.3472          | 0.3605 |
| 0.8922        | 92.1   | 14000 | 0.3530          | 0.3618 |
| 0.9           | 95.39  | 14500 | 0.3500          | 0.3574 |
| 0.9051        | 98.68  | 15000 | 0.3456          | 0.3535 |
| 0.9304        | 101.97 | 15500 | 0.3438          | 0.3578 |
| 0.9433        | 105.26 | 16000 | 0.3396          | 0.3530 |
| 0.8988        | 108.55 | 16500 | 0.3436          | 0.3539 |
| 0.8789        | 111.84 | 17000 | 0.3426          | 0.3516 |
| 0.8667        | 115.13 | 17500 | 0.3438          | 0.3506 |
| 0.8895        | 118.42 | 18000 | 0.3434          | 0.3503 |
| 0.8888        | 121.71 | 18500 | 0.3425          | 0.3494 |
| 0.9453        | 125.0  | 19000 | 0.3415          | 0.3480 |
| 0.9267        | 128.29 | 19500 | 0.3477          | 0.3503 |
| 0.8315        | 131.58 | 20000 | 0.3476          | 0.3505 |
| 0.8542        | 134.86 | 20500 | 0.3475          | 0.3506 |
| 0.8478        | 138.16 | 21000 | 0.3430          | 0.3481 |
| 0.8643        | 141.45 | 21500 | 0.3451          | 0.3485 |
| 0.8705        | 144.73 | 22000 | 0.3444          | 0.3474 |
| 0.9869        | 148.03 | 22500 | 0.3441          | 0.3493 |

### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2
- Datasets 1.18.3.dev0
- Tokenizers 0.10.3


---
language:
- fa
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: common6
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# common6

This model is a fine-tuned version of [common6/checkpoint-3500](https://huggingface.co/common6/checkpoint-3500) on the COMMON_VOICE - FA dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3706
- Wer: 0.3421

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 6e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 200.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 1.0344        | 10.0  | 500   | 0.4043          | 0.4511 |
| 0.9651        | 20.0  | 1000  | 0.3793          | 0.4159 |
| 0.9125        | 30.0  | 1500  | 0.3756          | 0.4046 |
| 0.8831        | 40.0  | 2000  | 0.3650          | 0.3876 |
| 0.8399        | 50.0  | 2500  | 0.3605          | 0.3772 |
| 0.819         | 60.0  | 3000  | 0.3622          | 0.3714 |
| 0.8029        | 70.0  | 3500  | 0.3561          | 0.3664 |
| 0.8104        | 80.0  | 4000  | 0.3595          | 0.3660 |
| 0.8118        | 90.0  | 4500  | 0.3460          | 0.3592 |
| 0.7831        | 100.0 | 5000  | 0.3566          | 0.3593 |
| 0.744         | 110.0 | 5500  | 0.3578          | 0.3535 |
| 0.7388        | 120.0 | 6000  | 0.3538          | 0.3520 |
| 0.714         | 130.0 | 6500  | 0.3682          | 0.3506 |
| 0.7291        | 140.0 | 7000  | 0.3625          | 0.3505 |
| 0.697         | 150.0 | 7500  | 0.3619          | 0.3479 |
| 0.6811        | 160.0 | 8000  | 0.3631          | 0.3440 |
| 0.6841        | 170.0 | 8500  | 0.3672          | 0.3460 |
| 0.6616        | 180.0 | 9000  | 0.3677          | 0.3410 |
| 0.6471        | 190.0 | 9500  | 0.3707          | 0.3420 |
| 0.6759        | 200.0 | 10000 | 0.3706          | 0.3421 |

### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2
- Datasets 1.18.3.dev0
- Tokenizers 0.10.3


---
language:
- fa
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: common8
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# common8

This model is a fine-tuned version of [wghts/checkpoint-20000](https://huggingface.co/wghts/checkpoint-20000) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - FA dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3174
- Wer: 0.3022

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 6
- total_train_batch_size: 192
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 250.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
|:-------------:|:------:|:-----:|:---------------:|:------:|
| 3.5847        | 1.93   | 500   | 3.5104          | 1.0    |
| 2.7858        | 3.86   | 1000  | 2.9601          | 1.0001 |
| 1.6827        | 5.79   | 1500  | 0.7853          | 0.7030 |
| 1.4656        | 7.72   | 2000  | 0.6076          | 0.6014 |
| 1.3693        | 9.65   | 2500  | 0.5114          | 0.5307 |
| 1.379         | 11.58  | 3000  | 0.4666          | 0.4940 |
| 1.2832        | 13.51  | 3500  | 0.4257          | 0.4593 |
| 1.1931        | 15.44  | 4000  | 0.4039          | 0.4427 |
| 1.2911        | 17.37  | 4500  | 0.3956          | 0.4295 |
| 1.1577        | 19.3   | 5000  | 0.3705          | 0.4114 |
| 1.1135        | 21.24  | 5500  | 0.3740          | 0.4010 |
| 1.19          | 23.17  | 6000  | 0.3611          | 0.3935 |
| 1.1008        | 25.1   | 6500  | 0.3503          | 0.3880 |
| 1.0805        | 27.03  | 7000  | 0.3427          | 0.3781 |
| 1.1556        | 28.96  | 7500  | 0.3442          | 0.3727 |
| 1.0596        | 30.89  | 8000  | 0.3398          | 0.3646 |
| 1.0219        | 32.82  | 8500  | 0.3312          | 0.3660 |
| 1.1042        | 34.75  | 9000  | 0.3287          | 0.3612 |
| 1.0273        | 36.68  | 9500  | 0.3236          | 0.3556 |
| 1.0383        | 38.61  | 10000 | 0.3217          | 0.3558 |
| 1.0498        | 40.54  | 10500 | 0.3205          | 0.3520 |
| 0.9969        | 42.47  | 11000 | 0.3125          | 0.3504 |
| 1.0658        | 44.4   | 11500 | 0.3120          | 0.3493 |
| 0.992         | 46.33  | 12000 | 0.3137          | 0.3476 |
| 0.9737        | 48.26  | 12500 | 0.3085          | 0.3413 |
| 1.0817        | 50.19  | 13000 | 0.3091          | 0.3418 |
| 0.9414        | 52.12  | 13500 | 0.3072          | 0.3344 |
| 0.9295        | 54.05  | 14000 | 0.3039          | 0.3322 |
| 1.0248        | 55.98  | 14500 | 0.2991          | 0.3325 |
| 0.9474        | 57.91  | 15000 | 0.3032          | 0.3348 |
| 0.928         | 59.85  | 15500 | 0.2999          | 0.3285 |
| 1.0321        | 61.78  | 16000 | 0.2982          | 0.3253 |
| 0.9255        | 63.71  | 16500 | 0.2970          | 0.3231 |
| 0.8928        | 65.64  | 17000 | 0.2993          | 0.3250 |
| 1.008         | 67.57  | 17500 | 0.2985          | 0.3222 |
| 0.9371        | 69.5   | 18000 | 0.2968          | 0.3216 |
| 0.9077        | 71.43  | 18500 | 0.3011          | 0.3299 |
| 1.0044        | 73.36  | 19000 | 0.3053          | 0.3306 |
| 0.9625        | 75.29  | 19500 | 0.3159          | 0.3295 |
| 0.9816        | 77.22  | 20000 | 0.3080          | 0.3304 |
| 0.9587        | 119.19 | 20500 | 0.3088          | 0.3284 |
| 0.9178        | 122.09 | 21000 | 0.3132          | 0.3320 |
| 1.0282        | 125.0  | 21500 | 0.3099          | 0.3266 |
| 0.9337        | 127.9  | 22000 | 0.3110          | 0.3317 |
| 0.8822        | 130.81 | 22500 | 0.3037          | 0.3247 |
| 0.9644        | 133.72 | 23000 | 0.3037          | 0.3238 |
| 0.9214        | 136.62 | 23500 | 0.3040          | 0.3234 |
| 0.9167        | 139.53 | 24000 | 0.3079          | 0.3203 |
| 0.9047        | 142.44 | 24500 | 0.3018          | 0.3177 |
| 0.8909        | 145.35 | 25000 | 0.3053          | 0.3181 |
| 0.9646        | 148.25 | 25500 | 0.3095          | 0.3229 |
| 0.8802        | 151.16 | 26000 | 0.3111          | 0.3192 |
| 0.8411        | 154.07 | 26500 | 0.3068          | 0.3123 |
| 0.9235        | 156.97 | 27000 | 0.3090          | 0.3177 |
| 0.8943        | 159.88 | 27500 | 0.3115          | 0.3179 |
| 0.8854        | 162.79 | 28000 | 0.3052          | 0.3157 |
| 0.8734        | 165.69 | 28500 | 0.3077          | 0.3124 |
| 0.8515        | 168.6  | 29000 | 0.3117          | 0.3128 |
| 0.912         | 171.51 | 29500 | 0.3039          | 0.3121 |
| 0.8669        | 174.42 | 30000 | 0.3120          | 0.3123 |
| 0.823         | 177.32 | 30500 | 0.3148          | 0.3118 |
| 0.9129        | 180.23 | 31000 | 0.3179          | 0.3101 |
| 0.8255        | 183.14 | 31500 | 0.3164          | 0.3114 |
| 0.8948        | 186.05 | 32000 | 0.3128          | 0.3101 |
| 0.8397        | 188.95 | 32500 | 0.3143          | 0.3068 |
| 0.8341        | 191.86 | 33000 | 0.3127          | 0.3136 |
| 0.873         | 194.76 | 33500 | 0.3149          | 0.3124 |
| 0.8232        | 197.67 | 34000 | 0.3166          | 0.3086 |
| 0.8002        | 200.58 | 34500 | 0.3149          | 0.3061 |
| 0.8621        | 203.49 | 35000 | 0.3160          | 0.3093 |
| 0.8123        | 206.39 | 35500 | 0.3141          | 0.3063 |
| 0.7995        | 209.3  | 36000 | 0.3174          | 0.3075 |
| 0.8271        | 212.21 | 36500 | 0.3173          | 0.3043 |
| 0.8059        | 215.12 | 37000 | 0.3176          | 0.3079 |
| 0.8835        | 218.02 | 37500 | 0.3169          | 0.3062 |
| 0.8027        | 220.93 | 38000 | 0.3203          | 0.3098 |
| 0.775         | 223.83 | 38500 | 0.3159          | 0.3068 |
| 0.8487        | 226.74 | 39000 | 0.3161          | 0.3072 |
| 0.7929        | 229.65 | 39500 | 0.3143          | 0.3037 |
| 0.7653        | 232.56 | 40000 | 0.3160          | 0.3048 |
| 0.8211        | 235.46 | 40500 | 0.3173          | 0.3031 |
| 0.7761        | 238.37 | 41000 | 0.3176          | 0.3025 |
| 0.7761        | 241.28 | 41500 | 0.3179          | 0.3027 |
| 0.7903        | 244.19 | 42000 | 0.3181          | 0.3016 |
| 0.7807        | 247.09 | 42500 | 0.3170          | 0.3027 |
| 0.8406        | 250.0  | 43000 | 0.3174          | 0.3022 |

### Framework versions

- Transformers 4.17.0.dev0
- Pytorch 1.10.2
- Datasets 1.18.3.dev0
- Tokenizers 0.10.3


---
language: vi
tags:
- gpt
widget:
- text: "Hôm qua những nhà khoa học Mỹ đã phát hiện ra loài cá lợn"
---

### GPT 2 News
**Update 02 Jan 2022**: Fixed mismatch tokenizer and model.wte size

### BibTex
```
@article{thanh21gpt2news,
  author  = {Thanh V. Le},
  title   = {Pretrained GPT-2 on Vietnamese news},
  journal = {https://huggingface.co/imthanhlv/gpt2news},
  year    = {2021},
}
```


---
tags:
- conversational
---

#Harry Potter DialoGPT Model


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- conll2003
metrics:
- precision
- recall
- f1
- accuracy
model_index:
- name: distilbert-base-uncased-finetuned-ner
  results:
  - task:
      name: Token Classification
      type: token-classification
    dataset:
      name: conll2003
      type: conll2003
      args: conll2003
    metric:
      name: Accuracy
      type: accuracy
      value: 0.984018301110458
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-ner

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the conll2003 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0614
- Precision: 0.9288
- Recall: 0.9374
- F1: 0.9331
- Accuracy: 0.9840

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.2399        | 1.0   | 878  | 0.0694          | 0.9126    | 0.9179 | 0.9152 | 0.9807   |
| 0.0522        | 2.0   | 1756 | 0.0604          | 0.9207    | 0.9342 | 0.9274 | 0.9833   |
| 0.0308        | 3.0   | 2634 | 0.0614          | 0.9288    | 0.9374 | 0.9331 | 0.9840   |

### Framework versions

- Transformers 4.9.1
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


---
tags:
- conversational
---

# Johnny Silverhand DialoGPT model


---
tags:
- conversational
---

# Shrek DialoGPT model


---
tags:
 - conversational
---

# Tohru DialoGPT model


---
language: multilingual
tags:
- pytorch
license: apache-2.0
datasets:
- multi_nli
- xnli
metrics:
- xnli

---

# mt5-large-finetuned-mnli-xtreme-xnli

## Model Description

This model takes a pretrained large [multilingual-t5](https://github.com/google-research/multilingual-t5) (also available from [models](https://huggingface.co/google/mt5-large)) and fine-tunes it on English MNLI and the [xtreme_xnli](https://www.tensorflow.org/datasets/catalog/xtreme_xnli) training set. It is intended to be used for zero-shot text classification, inspired by [xlm-roberta-large-xnli](https://huggingface.co/joeddav/xlm-roberta-large-xnli).

## Intended Use

This model is intended to be used for zero-shot text classification, especially in languages other than English. It is fine-tuned on English MNLI and the [xtreme_xnli](https://www.tensorflow.org/datasets/catalog/xtreme_xnli) training set, a multilingual NLI dataset. The model can therefore be used with any of the languages in the XNLI corpus:

- Arabic
- Bulgarian
- Chinese
- English
- French
- German
- Greek
- Hindi
- Russian
- Spanish
- Swahili
- Thai
- Turkish
- Urdu
- Vietnamese

As per recommendations in [xlm-roberta-large-xnli](https://huggingface.co/joeddav/xlm-roberta-large-xnli), for English-only classification, you might want to check out:
- [bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli)
- [a distilled bart MNLI model](https://huggingface.co/models?filter=pipeline_tag%3Azero-shot-classification&search=valhalla).

### Zero-shot example:

The model retains its text-to-text characteristic after fine-tuning. This means that our expected outputs will be text. During fine-tuning, the model learns to respond to the NLI task with a series of single token responses that map to entailment, neutral, or contradiction. The NLI task is indicated with a fixed prefix, "xnli:".

Below is an example, using PyTorch, of the model's use in a similar fashion to the `zero-shot-classification` pipeline. We use the logits from the LM output at the first token to represent confidence.

```python
from torch.nn.functional import softmax
from transformers import MT5ForConditionalGeneration, MT5Tokenizer

model_name = "alan-turing-institute/mt5-large-finetuned-mnli-xtreme-xnli"

tokenizer = MT5Tokenizer.from_pretrained(model_name)
model = MT5ForConditionalGeneration.from_pretrained(model_name)
model.eval()

sequence_to_classify = "¿A quién vas a votar en 2020?"
candidate_labels = ["Europa", "salud pública", "política"]
hypothesis_template = "Este ejemplo es {}."

ENTAILS_LABEL = "▁0"
NEUTRAL_LABEL = "▁1"
CONTRADICTS_LABEL = "▁2"

label_inds = tokenizer.convert_tokens_to_ids(
    [ENTAILS_LABEL, NEUTRAL_LABEL, CONTRADICTS_LABEL])

def process_nli(premise: str, hypothesis: str):
    """ process to required xnli format with task prefix """
    return "".join(['xnli: premise: ', premise, ' hypothesis: ', hypothesis])

# construct sequence of premise, hypothesis pairs
pairs = [(sequence_to_classify, hypothesis_template.format(label)) for label in
        candidate_labels]
# format for mt5 xnli task
seqs = [process_nli(premise=premise, hypothesis=hypothesis) for
        premise, hypothesis in pairs]
print(seqs)
# ['xnli: premise: ¿A quién vas a votar en 2020? hypothesis: Este ejemplo es Europa.',
# 'xnli: premise: ¿A quién vas a votar en 2020? hypothesis: Este ejemplo es salud pública.',
# 'xnli: premise: ¿A quién vas a votar en 2020? hypothesis: Este ejemplo es política.']

inputs = tokenizer.batch_encode_plus(seqs, return_tensors="pt", padding=True)

out = model.generate(**inputs, output_scores=True, return_dict_in_generate=True,
                     num_beams=1)

# sanity check that our sequences are expected length (1 + start token + end token = 3)
for i, seq in enumerate(out.sequences):
    assert len(
        seq) == 3, f"generated sequence {i} not of expected length, 3." \\\\
                   f" Actual length: {len(seq)}"

# get the scores for our only token of interest
# we'll now treat these like the output logits of a `*ForSequenceClassification` model
scores = out.scores[0]

# scores has a size of the model's vocab.
# However, for this task we have a fixed set of labels
# sanity check that these labels are always the top 3 scoring
for i, sequence_scores in enumerate(scores):
    top_scores = sequence_scores.argsort()[-3:]
    assert set(top_scores.tolist()) == set(label_inds), \\\\
        f"top scoring tokens are not expected for this task." \\\\
        f" Expected: {label_inds}. Got: {top_scores.tolist()}."

# cut down scores to our task labels
scores = scores[:, label_inds]
print(scores)
# tensor([[-2.5697,  1.0618,  0.2088],
#         [-5.4492, -2.1805, -0.1473],
#         [ 2.2973,  3.7595, -0.1769]])

# new indices of entailment and contradiction in scores
entailment_ind = 0
contradiction_ind = 2

# we can show, per item, the entailment vs contradiction probas
entail_vs_contra_scores = scores[:, [entailment_ind, contradiction_ind]]
entail_vs_contra_probas = softmax(entail_vs_contra_scores, dim=1)
print(entail_vs_contra_probas)
# tensor([[0.0585, 0.9415],
#         [0.0050, 0.9950],
#         [0.9223, 0.0777]])

# or we can show probas similar to `ZeroShotClassificationPipeline`
# this gives a zero-shot classification style output across labels
entail_scores = scores[:, entailment_ind]
entail_probas = softmax(entail_scores, dim=0)
print(entail_probas)
# tensor([7.6341e-03, 4.2873e-04, 9.9194e-01])

print(dict(zip(candidate_labels, entail_probas.tolist())))
# {'Europa': 0.007634134963154793,
# 'salud pública': 0.0004287279152777046,
# 'política': 0.9919371604919434}

```

Unfortunately, the `generate` function for the TF equivalent model doesn't exactly mirror the PyTorch version so the above code won't directly transfer.

The model is currently not compatible with the existing `zero-shot-classification` pipeline.

## Training

This model was pre-trained on a set of 101 languages in the mC4, as described in [the mt5 paper](https://arxiv.org/abs/2010.11934). It was then fine-tuned on the [mt5_xnli_translate_train](https://github.com/google-research/multilingual-t5/blob/78d102c830d76bd68f27596a97617e2db2bfc887/multilingual_t5/tasks.py#L190) task for 8k steps in a similar manner to that described in the [offical repo](https://github.com/google-research/multilingual-t5#fine-tuning), with guidance from [Stephen Mayhew's notebook](https://github.com/mayhewsw/multilingual-t5/blob/master/notebooks/mt5-xnli.ipynb). The resulting model was then converted to :hugging_face: format.

## Eval results

Accuracy over XNLI test set:

| ar   | bg   | de   | el   | en   | es   | fr   | hi   | ru   | sw   | th   | tr   | ur   | vi   | zh   | average  |
|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|
| 81.0 | 85.0 | 84.3 | 84.3 | 88.8 | 85.3 | 83.9 | 79.9 | 82.6 | 78.0 | 81.0 | 81.6 | 76.4 | 81.7 | 82.3 | 82.4 |


---
language:

  - en

tags:
- text-classification
- sentiment-analysis
- sentiment-classification
- targeted-sentiment-classification
- target-depentent-sentiment-classification

license: "apache-2.0"

datasets: "fhamborg/news_sentiment_newsmtsc"

---

# NewsSentiment: easy-to-use, high-quality target-dependent sentiment classification for news articles

## Important: [use our PyPI package](https://pypi.org/project/NewsSentiment/) instead of this model on the Hub
The Huggingface Hub architecture currently [does not support](https://github.com/huggingface/transformers/issues/14785) target-dependent sentiment classification since you cannot provide the required inputs, i.e., sentence and target. Thus, we recommend that you use our easy-to-use [PyPI package NewsSentiment](https://pypi.org/project/NewsSentiment/).

## Description

This model is the currently [best performing](https://aclanthology.org/2021.eacl-main.142.pdf)
targeted sentiment classifier for news articles. In contrast to regular sentiment
classification, targeted sentiment classification allows you to provide a target in a sentence.
Only for this target, the sentiment is then predicted. This is more reliable in many
cases, as demonstrated by the following simplistic example: "I like Bert, but I hate Robert."

This model is also available as an easy-to-use PyPI package named [`NewsSentiment`](https://pypi.org/project/NewsSentiment/) and
in its original GitHub repository named [`NewsMTSC`](https://github.com/fhamborg/NewsMTSC), where you will find the dataset the model was trained on, other models for sentiment classification, and a training and testing framework. More information on the model and the dataset (consisting of more than 10k sentences sampled from news articles, each
labeled and agreed upon by at least 5 annotators) can be found in our [EACL paper](https://aclanthology.org/2021.eacl-main.142.pdf). The
dataset, the model, and its source code can be viewed in our [GitHub repository](https://github.com/fhamborg/NewsMTSC).

We recommend to use our [PyPI package](https://pypi.org/project/NewsSentiment/) for sentiment classification since the Huggingface Hub platform seems to [not support](https://github.com/huggingface/transformers/issues/14785) target-dependent sentiment classification.

# How to cite
If you use the dataset or model, please cite our [paper](https://www.aclweb.org/anthology/2021.eacl-main.142/) ([PDF](https://www.aclweb.org/anthology/2021.eacl-main.142.pdf)):

```
@InProceedings{Hamborg2021b,
  author    = {Hamborg, Felix and Donnay, Karsten},
  title     = {NewsMTSC: (Multi-)Target-dependent Sentiment Classification in News Articles},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2021)},
  year      = {2021},
  month     = {Apr.},
  location  = {Virtual Event},
}
```


---

language:

- en

tags:

- text-classification

- zero-shot-classification

metrics:

- accuracy

widget:

- text: "The Movie have been criticized for the story. However, I think it is a great movie. [SEP] I liked the movie."

---

# DeBERTa-v3-large-mnli

## Model description

This model was trained on the Multi-Genre Natural Language Inference ( MultiNLI ) dataset, which consists of 433k sentence pairs textual entailment information.

The model used is [DeBERTa-v3-large from Microsoft](https://huggingface.co/microsoft/deberta-large). The v3 DeBERTa outperforms the result of Bert and RoBERTa in  majority of NLU benchmarks by using disentangled attention and enhanced mask decoder. More information about the orginal model is on [official repository](https://github.com/microsoft/DeBERTa) and the [paper](https://arxiv.org/abs/2006.03654)

## Intended uses & limitations

#### How to use the model

```python

premise = "The Movie have been criticized for the story. However, I think it is a great movie."

hypothesis = "I liked the movie."

input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")

output = model(input["input_ids"].to(device))  # device = "cuda:0" or "cpu"

prediction = torch.softmax(output["logits"][0], -1)

label_names = ["entailment", "neutral", "contradiction"]

print(label_names[prediction.argmax(0).tolist()])

```

### Training data

This model was trained on the MultiNLI dataset, which consists of 392K sentence textual entitlement.

### Training procedure

DeBERTa-v3-large-mnli was trained using the Hugging Face trainer with the following hyperparameters.

```

train_args = TrainingArguments(
    learning_rate=2e-5,

    per_device_train_batch_size=8,

    per_device_eval_batch_size=8,

    num_train_epochs=3,

    warmup_ratio=0.06,

    weight_decay=0.1,

    fp16=True,

    seed=42,
)

```

### BibTeX entry and citation info

Please cite the [DeBERTa paper](https://arxiv.org/abs/2006.03654) and [MultiNLI Dataset](https://cims.nyu.edu/~sbowman/multinli/paper.pdf) if you use this model and include this Huggingface hub.


---
language: da
tags:
- bert
- masked-lm
license: cc-by-4.0
datasets:
- common_crawl
- wikipedia
pipeline_tag: fill-mask
widget:
- text: "København er [MASK] i Danmark."
---

# Danish BERT (uncased) model

[BotXO.ai](https://www.botxo.ai/) developed this model. For data and training details see their [GitHub repository](https://github.com/botxo/nordic_bert).

The original model was trained in TensorFlow then I converted it to Pytorch using [transformers-cli](https://huggingface.co/transformers/converting_tensorflow_models.html?highlight=cli).

For TensorFlow version download here: https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1

## Architecture

```python
from transformers import AutoModelForPreTraining

model = AutoModelForPreTraining.from_pretrained("DJSammy/bert-base-danish-uncased_BotXO,ai")

params = list(model.named_parameters())
print('danish_bert_uncased_v2 has {:} different named parameters.\n'.format(len(params)))

print('==== Embedding Layer ====\n')
for p in params[0:5]:
    print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size()))))

print('\n==== First Transformer ====\n')
for p in params[5:21]:
    print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size()))))

print('\n==== Last Transformer ====\n')
for p in params[181:197]:
    print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size()))))

print('\n==== Output Layer ====\n')
for p in params[197:]:
    print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size()))))

# danish_bert_uncased_v2 has 206 different named parameters.

# ==== Embedding Layer ====

# bert.embeddings.word_embeddings.weight                  (32000, 768)
# bert.embeddings.position_embeddings.weight                (512, 768)
# bert.embeddings.token_type_embeddings.weight                (2, 768)
# bert.embeddings.LayerNorm.weight                              (768,)
# bert.embeddings.LayerNorm.bias                                (768,)

# ==== First Transformer ====

# bert.encoder.layer.0.attention.self.query.weight          (768, 768)
# bert.encoder.layer.0.attention.self.query.bias                (768,)
# bert.encoder.layer.0.attention.self.key.weight            (768, 768)
# bert.encoder.layer.0.attention.self.key.bias                  (768,)
# bert.encoder.layer.0.attention.self.value.weight          (768, 768)
# bert.encoder.layer.0.attention.self.value.bias                (768,)
# bert.encoder.layer.0.attention.output.dense.weight        (768, 768)
# bert.encoder.layer.0.attention.output.dense.bias              (768,)
# bert.encoder.layer.0.attention.output.LayerNorm.weight        (768,)
# bert.encoder.layer.0.attention.output.LayerNorm.bias          (768,)
# bert.encoder.layer.0.intermediate.dense.weight           (3072, 768)
# bert.encoder.layer.0.intermediate.dense.bias                 (3072,)
# bert.encoder.layer.0.output.dense.weight                 (768, 3072)
# bert.encoder.layer.0.output.dense.bias                        (768,)
# bert.encoder.layer.0.output.LayerNorm.weight                  (768,)
# bert.encoder.layer.0.output.LayerNorm.bias                    (768,)

# ==== Last Transformer ====

# bert.encoder.layer.11.attention.self.query.weight         (768, 768)
# bert.encoder.layer.11.attention.self.query.bias               (768,)
# bert.encoder.layer.11.attention.self.key.weight           (768, 768)
# bert.encoder.layer.11.attention.self.key.bias                 (768,)
# bert.encoder.layer.11.attention.self.value.weight         (768, 768)
# bert.encoder.layer.11.attention.self.value.bias               (768,)
# bert.encoder.layer.11.attention.output.dense.weight       (768, 768)
# bert.encoder.layer.11.attention.output.dense.bias             (768,)
# bert.encoder.layer.11.attention.output.LayerNorm.weight       (768,)
# bert.encoder.layer.11.attention.output.LayerNorm.bias         (768,)
# bert.encoder.layer.11.intermediate.dense.weight          (3072, 768)
# bert.encoder.layer.11.intermediate.dense.bias                (3072,)
# bert.encoder.layer.11.output.dense.weight                (768, 3072)
# bert.encoder.layer.11.output.dense.bias                       (768,)
# bert.encoder.layer.11.output.LayerNorm.weight                 (768,)
# bert.encoder.layer.11.output.LayerNorm.bias                   (768,)

# ==== Output Layer ====

# bert.pooler.dense.weight                                  (768, 768)
# bert.pooler.dense.bias                                        (768,)
# cls.predictions.bias                                        (32000,)
# cls.predictions.transform.dense.weight                    (768, 768)
# cls.predictions.transform.dense.bias                          (768,)
# cls.predictions.transform.LayerNorm.weight                    (768,)
# cls.predictions.transform.LayerNorm.bias                      (768,)
# cls.seq_relationship.weight                                 (2, 768)
# cls.seq_relationship.bias                                       (2,)
```

## Example Pipeline

```python
from transformers import pipeline
unmasker = pipeline('fill-mask', model='DJSammy/bert-base-danish-uncased_BotXO,ai')

unmasker('København er [MASK] i Danmark.')

# Copenhagen is the [MASK] of Denmark.
# =>

# [{'score': 0.788068950176239,
#  'sequence': '[CLS] københavn er hovedstad i danmark. [SEP]',
#  'token': 12610,
#  'token_str': 'hovedstad'},
# {'score': 0.07606703042984009,
#  'sequence': '[CLS] københavn er hovedstaden i danmark. [SEP]',
#  'token': 8108,
#  'token_str': 'hovedstaden'},
# {'score': 0.04299738258123398,
#  'sequence': '[CLS] københavn er metropol i danmark. [SEP]',
#  'token': 23305,
#  'token_str': 'metropol'},
# {'score': 0.008163209073245525,
#  'sequence': '[CLS] københavn er ikke i danmark. [SEP]',
#  'token': 89,
#  'token_str': 'ikke'},
# {'score': 0.006238455418497324,
#  'sequence': '[CLS] københavn er ogsa i danmark. [SEP]',
#  'token': 25253,
#  'token_str': 'ogsa'}]
```


---
language:
- da
license: mit
model-index:
- name: contract-ner-model-da
  results: []
widget:
- "Medarbejderen starter arbejdet den 1. januar 2020 og afslutter arbejdet den 21. januar 2020. Den ugentlige arbejdstid er 37 timer, og medarbejderen bliver aflønnet med 23.000,00 kr. om måneden. Arbejdsstedet er Supervej 21, 2000 Frederiksberg."
inference:
  parameters:
    aggregation_strategy: "first"
---

# contract-ner-model-da

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on a custom contracts dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0026
- Micro F1: 0.9297

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 919
- num_epochs: 500

### Training results

| Training Loss | Epoch | Step | Validation Loss | Micro F1 |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.8971        | 0.24  | 200  | 0.0205          | 0.0      |
| 0.0173        | 0.48  | 400  | 0.0100          | 0.2921   |
| 0.0092        | 0.73  | 600  | 0.0065          | 0.7147   |
| 0.0063        | 0.97  | 800  | 0.0046          | 0.8332   |
| 0.0047        | 1.21  | 1000 | 0.0047          | 0.8459   |
| 0.0042        | 1.45  | 1200 | 0.0039          | 0.8694   |
| 0.0037        | 1.69  | 1400 | 0.0035          | 0.8888   |
| 0.0032        | 1.93  | 1600 | 0.0035          | 0.8840   |
| 0.0025        | 2.18  | 1800 | 0.0029          | 0.8943   |
| 0.0023        | 2.42  | 2000 | 0.0024          | 0.9104   |
| 0.0023        | 2.66  | 2200 | 0.0032          | 0.8808   |
| 0.0021        | 2.9   | 2400 | 0.0022          | 0.9338   |
| 0.0018        | 3.14  | 2600 | 0.0020          | 0.9315   |
| 0.0015        | 3.39  | 2800 | 0.0026          | 0.9297   |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model-index:
- name: xlmr-base-texas-squad-de
  results: []
widget:
- text: "Welche Ausbildung hatte Angela Merkel?"
  context: "Angela Dorothea Merkel (geb. Kasner; * 17. Juli 1954 in Hamburg) ist eine deutsche Politikerin (CDU). Sie war vom 22. November 2005 bis zum 8. Dezember 2021 Bundeskanzlerin der Bundesrepublik Deutschland. Sie ist die achte Person, zugleich erste Frau, erste Person aus Ostdeutschland und erste Person, die nach der Gründung der Bundesrepublik geboren ist, die in dieses Amt gewählt wurde. Von April 2000 bis Dezember 2018 war sie Bundesvorsitzende der CDU. Merkel wuchs in der DDR auf und war dort als Physikerin am Zentralinstitut für Physikalische Chemie tätig. Erstmals politisch aktiv wurde sie während der Wendezeit in der Partei Demokratischer Aufbruch, die sich 1990 der CDU anschloss. In der ersten und gleichzeitig letzten demokratisch gewählten Regierung der DDR übte sie das Amt der stellvertretenden Regierungssprecherin aus."
---

# TExAS-SQuAD-de

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the TExAS-SQuAD-de dataset.
It achieves the following results on the evaluation set:
- Exact match: 61.45%
- F1-score: 66.12%

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.8084        | 1.0   | 4233  | 1.5897          |
| 1.5696        | 2.0   | 8466  | 1.5478          |
| 1.4196        | 3.0   | 12699 | 1.5754          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
language: en
tags:
- generated_from_trainer
model-index:
- name: verdict-classifier-en
  results:
  - task:
      type: text-classification
      name: Verdict Classification
widget:
- "Even though it might look true, it has been taken out of context."
---

# English Verdict Classifier
This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on 2,500 deduplicated verdicts from [Google Fact Check Tools API](https://developers.google.com/fact-check/tools/api/reference/rest/v1alpha1/claims/search), translated into English with the [Google Cloud Translation API](https://cloud.google.com/translate/docs/reference/rest/).
It achieves the following results on the evaluation set, being 1,000 such verdicts translated into English, but here including duplicates to represent the true distribution:
- Loss: 0.1290
- F1 Macro: 0.9171
- F1 Misinformation: 0.9896
- F1 Factual: 0.9890
- F1 Other: 0.7727
- Precision Macro: 0.8940
- Precision Misinformation: 0.9954
- Precision Factual: 0.9783
- Precision Other: 0.7083

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2500
- num_epochs: 1000

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1 Macro | F1 Misinformation | F1 Factual | F1 Other | Precision Macro | Precision Misinformation | Precision Factual | Precision Other |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-----------------:|:----------:|:--------:|:----------:|:-------------------:|:------------:|:----------:|
| 1.1493        | 0.16  | 50   | 1.1040          | 0.0550   | 0.0               | 0.1650     | 0.0      | 0.0300     | 0.0                 | 0.0899       | 0.0        |
| 1.0899        | 0.32  | 100  | 1.0765          | 0.0619   | 0.0203            | 0.1654     | 0.0      | 0.2301     | 0.6                 | 0.0903       | 0.0        |
| 1.0136        | 0.48  | 150  | 1.0487          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.9868        | 0.64  | 200  | 1.0221          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.9599        | 0.8   | 250  | 0.9801          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.9554        | 0.96  | 300  | 0.9500          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.935         | 1.12  | 350  | 0.9071          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.948         | 1.28  | 400  | 0.8809          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.9344        | 1.44  | 450  | 0.8258          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.9182        | 1.6   | 500  | 0.7687          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.8942        | 1.76  | 550  | 0.5787          | 0.3102   | 0.9306            | 0.0        | 0.0      | 0.2900     | 0.8701              | 0.0          | 0.0        |
| 0.8932        | 1.92  | 600  | 0.4506          | 0.4043   | 0.9628            | 0.0        | 0.25     | 0.3777     | 0.9753              | 0.0          | 0.1579     |
| 0.7448        | 2.08  | 650  | 0.2884          | 0.5323   | 0.9650            | 0.3303     | 0.3017   | 0.7075     | 0.9810              | 0.9474       | 0.1942     |
| 0.6616        | 2.24  | 700  | 0.2162          | 0.8161   | 0.9710            | 0.9724     | 0.5051   | 0.7910     | 0.9824              | 0.9670       | 0.4237     |
| 0.575         | 2.4   | 750  | 0.1754          | 0.8305   | 0.9714            | 0.9780     | 0.5421   | 0.7961     | 0.9881              | 0.9674       | 0.4328     |
| 0.5246        | 2.56  | 800  | 0.1641          | 0.8102   | 0.9659            | 0.9175     | 0.5472   | 0.7614     | 0.9892              | 0.8558       | 0.4394     |
| 0.481         | 2.72  | 850  | 0.1399          | 0.8407   | 0.9756            | 0.9780     | 0.5686   | 0.8082     | 0.9894              | 0.9674       | 0.4677     |
| 0.4588        | 2.88  | 900  | 0.1212          | 0.8501   | 0.9786            | 0.9783     | 0.5934   | 0.8247     | 0.9871              | 0.9574       | 0.5294     |
| 0.4512        | 3.04  | 950  | 0.1388          | 0.8270   | 0.9702            | 0.9836     | 0.5273   | 0.7904     | 0.9893              | 0.9677       | 0.4143     |
| 0.3894        | 3.2   | 1000 | 0.1270          | 0.8411   | 0.9737            | 0.9836     | 0.5660   | 0.8043     | 0.9905              | 0.9677       | 0.4545     |
| 0.3772        | 3.36  | 1050 | 0.1267          | 0.8336   | 0.9732            | 0.9890     | 0.5385   | 0.8013     | 0.9882              | 0.9783       | 0.4375     |
| 0.3528        | 3.52  | 1100 | 0.1073          | 0.8546   | 0.9791            | 0.9890     | 0.5957   | 0.8284     | 0.9883              | 0.9783       | 0.5185     |
| 0.3694        | 3.68  | 1150 | 0.1120          | 0.8431   | 0.9786            | 0.9890     | 0.5618   | 0.8244     | 0.9849              | 0.9783       | 0.5102     |
| 0.3146        | 3.84  | 1200 | 0.1189          | 0.8325   | 0.9738            | 0.9836     | 0.54     | 0.8016     | 0.9870              | 0.9677       | 0.45       |
| 0.3038        | 4.01  | 1250 | 0.1041          | 0.8648   | 0.9815            | 0.9836     | 0.6292   | 0.8425     | 0.9884              | 0.9677       | 0.5714     |
| 0.2482        | 4.17  | 1300 | 0.1245          | 0.8588   | 0.9773            | 0.9836     | 0.6154   | 0.8202     | 0.9929              | 0.9677       | 0.5        |
| 0.2388        | 4.33  | 1350 | 0.1167          | 0.8701   | 0.9808            | 0.9836     | 0.6458   | 0.8377     | 0.9918              | 0.9677       | 0.5536     |
| 0.2593        | 4.49  | 1400 | 0.1215          | 0.8654   | 0.9790            | 0.9836     | 0.6337   | 0.8284     | 0.9929              | 0.9677       | 0.5246     |
| 0.239         | 4.65  | 1450 | 0.1057          | 0.8621   | 0.9803            | 0.9890     | 0.6170   | 0.8349     | 0.9895              | 0.9783       | 0.5370     |
| 0.2397        | 4.81  | 1500 | 0.1256          | 0.8544   | 0.9761            | 0.9890     | 0.5981   | 0.8162     | 0.9929              | 0.9783       | 0.4776     |
| 0.2238        | 4.97  | 1550 | 0.1189          | 0.8701   | 0.9802            | 0.9836     | 0.6465   | 0.8343     | 0.9929              | 0.9677       | 0.5424     |
| 0.1811        | 5.13  | 1600 | 0.1456          | 0.8438   | 0.9737            | 0.9836     | 0.5741   | 0.8051     | 0.9917              | 0.9677       | 0.4559     |
| 0.1615        | 5.29  | 1650 | 0.1076          | 0.8780   | 0.9838            | 0.9836     | 0.6667   | 0.8581     | 0.9895              | 0.9677       | 0.6170     |
| 0.1783        | 5.45  | 1700 | 0.1217          | 0.8869   | 0.9831            | 0.9836     | 0.6939   | 0.8497     | 0.9953              | 0.9677       | 0.5862     |
| 0.1615        | 5.61  | 1750 | 0.1305          | 0.8770   | 0.9808            | 0.9836     | 0.6667   | 0.8371     | 0.9953              | 0.9677       | 0.5484     |
| 0.155         | 5.77  | 1800 | 0.1218          | 0.8668   | 0.9821            | 0.9890     | 0.6292   | 0.8460     | 0.9884              | 0.9783       | 0.5714     |
| 0.167         | 5.93  | 1850 | 0.1091          | 0.8991   | 0.9873            | 0.9890     | 0.7209   | 0.8814     | 0.9919              | 0.9783       | 0.6739     |
| 0.1455        | 6.09  | 1900 | 0.1338          | 0.8535   | 0.9773            | 0.9890     | 0.5941   | 0.8202     | 0.9906              | 0.9783       | 0.4918     |
| 0.1301        | 6.25  | 1950 | 0.1321          | 0.8792   | 0.9820            | 0.9890     | 0.6667   | 0.8439     | 0.9941              | 0.9783       | 0.5593     |
| 0.1049        | 6.41  | 2000 | 0.1181          | 0.9031   | 0.9879            | 0.9834     | 0.7381   | 0.8911     | 0.9908              | 0.9780       | 0.7045     |
| 0.1403        | 6.57  | 2050 | 0.1432          | 0.8608   | 0.9779            | 0.9890     | 0.6154   | 0.8237     | 0.9929              | 0.9783       | 0.5        |
| 0.1178        | 6.73  | 2100 | 0.1443          | 0.8937   | 0.9844            | 0.9945     | 0.7021   | 0.8644     | 0.9930              | 0.9890       | 0.6111     |
| 0.1267        | 6.89  | 2150 | 0.1346          | 0.8494   | 0.9786            | 0.9890     | 0.5806   | 0.8249     | 0.9871              | 0.9783       | 0.5094     |
| 0.1043        | 7.05  | 2200 | 0.1494          | 0.8905   | 0.9832            | 0.9945     | 0.6939   | 0.8564     | 0.9941              | 0.9890       | 0.5862     |
| 0.0886        | 7.21  | 2250 | 0.1180          | 0.8946   | 0.9873            | 0.9890     | 0.7073   | 0.8861     | 0.9896              | 0.9783       | 0.6905     |
| 0.1183        | 7.37  | 2300 | 0.1777          | 0.8720   | 0.9790            | 0.9890     | 0.6481   | 0.8298     | 0.9964              | 0.9783       | 0.5147     |
| 0.0813        | 7.53  | 2350 | 0.1405          | 0.8912   | 0.9856            | 0.9836     | 0.7045   | 0.8685     | 0.9919              | 0.9677       | 0.6458     |
| 0.111         | 7.69  | 2400 | 0.1379          | 0.8874   | 0.9838            | 0.9836     | 0.6947   | 0.8540     | 0.9941              | 0.9677       | 0.6        |
| 0.1199        | 7.85  | 2450 | 0.1301          | 0.9080   | 0.9879            | 0.9890     | 0.7473   | 0.8801     | 0.9953              | 0.9783       | 0.6667     |
| 0.1054        | 8.01  | 2500 | 0.1478          | 0.8845   | 0.9838            | 0.9890     | 0.6809   | 0.8546     | 0.9930              | 0.9783       | 0.5926     |
| 0.105         | 8.17  | 2550 | 0.1333          | 0.9021   | 0.9879            | 0.9890     | 0.7294   | 0.8863     | 0.9919              | 0.9783       | 0.6889     |
| 0.09          | 8.33  | 2600 | 0.1555          | 0.8926   | 0.9855            | 0.9890     | 0.7033   | 0.8662     | 0.9930              | 0.9783       | 0.6275     |
| 0.0947        | 8.49  | 2650 | 0.1572          | 0.8831   | 0.9856            | 0.9890     | 0.6747   | 0.8726     | 0.9885              | 0.9783       | 0.6512     |
| 0.0784        | 8.65  | 2700 | 0.1477          | 0.8969   | 0.9873            | 0.9890     | 0.7143   | 0.8836     | 0.9908              | 0.9783       | 0.6818     |
| 0.0814        | 8.81  | 2750 | 0.1700          | 0.8932   | 0.9861            | 0.9890     | 0.7045   | 0.8720     | 0.9919              | 0.9783       | 0.6458     |
| 0.0962        | 8.97  | 2800 | 0.1290          | 0.9171   | 0.9896            | 0.9890     | 0.7727   | 0.8940     | 0.9954              | 0.9783       | 0.7083     |
| 0.0802        | 9.13  | 2850 | 0.1721          | 0.8796   | 0.9832            | 0.9890     | 0.6667   | 0.8517     | 0.9918              | 0.9783       | 0.5849     |
| 0.0844        | 9.29  | 2900 | 0.1516          | 0.9023   | 0.9867            | 0.9890     | 0.7312   | 0.8717     | 0.9953              | 0.9783       | 0.6415     |
| 0.0511        | 9.45  | 2950 | 0.1544          | 0.9062   | 0.9879            | 0.9890     | 0.7416   | 0.8820     | 0.9942              | 0.9783       | 0.6735     |
| 0.0751        | 9.61  | 3000 | 0.1748          | 0.8884   | 0.9832            | 0.9945     | 0.6875   | 0.8571     | 0.9930              | 0.9890       | 0.5893     |
| 0.0707        | 9.77  | 3050 | 0.1743          | 0.8721   | 0.9802            | 0.9890     | 0.6471   | 0.8349     | 0.9941              | 0.9783       | 0.5323     |
| 0.0951        | 9.93  | 3100 | 0.1660          | 0.8899   | 0.9850            | 0.9890     | 0.6957   | 0.8622     | 0.9930              | 0.9783       | 0.6154     |
| 0.0576        | 10.1  | 3150 | 0.2029          | 0.8613   | 0.9766            | 0.9890     | 0.6182   | 0.8197     | 0.9952              | 0.9783       | 0.4857     |
| 0.0727        | 10.26 | 3200 | 0.1709          | 0.8920   | 0.9849            | 0.9890     | 0.7021   | 0.8612     | 0.9942              | 0.9783       | 0.6111     |
| 0.0654        | 10.42 | 3250 | 0.1599          | 0.8999   | 0.9861            | 0.9945     | 0.7191   | 0.8780     | 0.9919              | 0.9890       | 0.6531     |
| 0.0553        | 10.58 | 3300 | 0.2091          | 0.8920   | 0.9849            | 0.9890     | 0.7021   | 0.8612     | 0.9942              | 0.9783       | 0.6111     |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.9.0+cu102
- Datasets 1.9.0
- Tokenizers 0.10.2


---
license: mit
language:
- am
- ar
- hy
- eu
- bn
- bs
- bg
- my
- hr
- ca
- cs
- da
- nl
- en
- et
- fi
- fr
- ka
- de
- el
- gu
- ht
- iw
- hi
- hu
- is
- in
- it
- ja
- kn
- km
- ko
- lo
- lv
- lt
- ml
- mr
- ne
- no
- or
- pa
- ps
- fa
- pl
- pt
- ro
- ru
- sr
- zh
- sd
- si
- sk
- sl
- es
- sv
- tl
- ta
- te
- th
- tr
- uk
- ur
- ug
- vi
- cy
tags:
- generated_from_trainer
model-index:
- name: verdict-classifier-en
  results:
  - task:
      type: text-classification
      name: Verdict Classification
widget:
- "本文已断章取义。"
---

# Multilingual Verdict Classifier

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on 2,500 deduplicated multilingual verdicts from [Google Fact Check Tools API](https://developers.google.com/fact-check/tools/api/reference/rest/v1alpha1/claims/search), translated into 65 languages with the [Google Cloud Translation API](https://cloud.google.com/translate/docs/reference/rest/).
It achieves the following results on the evaluation set, being 1,000 such verdicts, but here including duplicates to represent the true distribution:
- Loss: 0.2238
- F1 Macro: 0.8540
- F1 Misinformation: 0.9798
- F1 Factual: 0.9889
- F1 Other: 0.5934
- Prec Macro: 0.8348
- Prec Misinformation: 0.9860
- Prec Factual: 0.9889
- Prec Other: 0.5294

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 162525
- num_epochs: 1000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | F1 Macro | F1 Misinformation | F1 Factual | F1 Other | Prec Macro | Prec Misinformation | Prec Factual | Prec Other |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:-----------------:|:----------:|:--------:|:----------:|:-------------------:|:------------:|:----------:|
| 1.1109        | 0.1   | 2000  | 1.2166          | 0.0713   | 0.1497            | 0.0        | 0.0640   | 0.2451     | 0.7019              | 0.0          | 0.0334     |
| 0.9551        | 0.2   | 4000  | 0.7801          | 0.3611   | 0.8889            | 0.0        | 0.1943   | 0.3391     | 0.8915              | 0.0          | 0.1259     |
| 0.9275        | 0.3   | 6000  | 0.7712          | 0.3468   | 0.9123            | 0.0        | 0.1282   | 0.3304     | 0.9051              | 0.0          | 0.0862     |
| 0.8881        | 0.39  | 8000  | 0.5386          | 0.3940   | 0.9524            | 0.0        | 0.2297   | 0.3723     | 0.9748              | 0.0          | 0.1420     |
| 0.7851        | 0.49  | 10000 | 0.3298          | 0.6886   | 0.9626            | 0.7640     | 0.3393   | 0.6721     | 0.9798              | 0.7727       | 0.2639     |
| 0.639         | 0.59  | 12000 | 0.2156          | 0.7847   | 0.9633            | 0.9355     | 0.4554   | 0.7540     | 0.9787              | 0.9062       | 0.3770     |
| 0.5677        | 0.69  | 14000 | 0.1682          | 0.7877   | 0.9694            | 0.9667     | 0.4270   | 0.7763     | 0.9745              | 0.9667       | 0.3878     |
| 0.5218        | 0.79  | 16000 | 0.1475          | 0.8037   | 0.9692            | 0.9667     | 0.4752   | 0.7804     | 0.9812              | 0.9667       | 0.3934     |
| 0.4682        | 0.89  | 18000 | 0.1458          | 0.8097   | 0.9734            | 0.9667     | 0.4889   | 0.7953     | 0.9791              | 0.9667       | 0.44       |
| 0.4188        | 0.98  | 20000 | 0.1416          | 0.8370   | 0.9769            | 0.9724     | 0.5618   | 0.8199     | 0.9826              | 0.9670       | 0.5102     |
| 0.3735        | 1.08  | 22000 | 0.1624          | 0.8094   | 0.9698            | 0.9368     | 0.5217   | 0.7780     | 0.9823              | 0.89         | 0.4615     |
| 0.3242        | 1.18  | 24000 | 0.1648          | 0.8338   | 0.9769            | 0.9727     | 0.5517   | 0.8167     | 0.9826              | 0.9570       | 0.5106     |
| 0.2785        | 1.28  | 26000 | 0.1843          | 0.8261   | 0.9739            | 0.9780     | 0.5263   | 0.8018     | 0.9836              | 0.9674       | 0.4545     |
| 0.25          | 1.38  | 28000 | 0.1975          | 0.8344   | 0.9744            | 0.9834     | 0.5455   | 0.8072     | 0.9859              | 0.9780       | 0.4576     |
| 0.2176        | 1.48  | 30000 | 0.1849          | 0.8209   | 0.9691            | 0.9889     | 0.5047   | 0.7922     | 0.9846              | 0.9889       | 0.4030     |
| 0.1966        | 1.58  | 32000 | 0.2119          | 0.8194   | 0.9685            | 0.9944     | 0.4954   | 0.7920     | 0.9846              | 1.0          | 0.3913     |
| 0.1738        | 1.67  | 34000 | 0.2110          | 0.8352   | 0.9708            | 0.9944     | 0.5405   | 0.8035     | 0.9881              | 1.0          | 0.4225     |
| 0.1625        | 1.77  | 36000 | 0.2152          | 0.8165   | 0.9709            | 0.9834     | 0.4950   | 0.7905     | 0.9835              | 0.9780       | 0.4098     |
| 0.1522        | 1.87  | 38000 | 0.2300          | 0.8097   | 0.9697            | 0.9832     | 0.4762   | 0.7856     | 0.9835              | 0.9888       | 0.3846     |
| 0.145         | 1.97  | 40000 | 0.1955          | 0.8519   | 0.9774            | 0.9889     | 0.5895   | 0.8280     | 0.9860              | 0.9889       | 0.5091     |
| 0.1248        | 2.07  | 42000 | 0.2308          | 0.8149   | 0.9703            | 0.9889     | 0.4854   | 0.7897     | 0.9835              | 0.9889       | 0.3968     |
| 0.1186        | 2.17  | 44000 | 0.2368          | 0.8172   | 0.9733            | 0.9834     | 0.4948   | 0.7942     | 0.9836              | 0.9780       | 0.4211     |
| 0.1122        | 2.26  | 46000 | 0.2401          | 0.7968   | 0.9804            | 0.8957     | 0.5143   | 0.8001     | 0.9849              | 1.0          | 0.4154     |
| 0.1099        | 2.36  | 48000 | 0.2290          | 0.8119   | 0.9647            | 0.9834     | 0.4874   | 0.7777     | 0.9880              | 0.9780       | 0.3671     |
| 0.1093        | 2.46  | 50000 | 0.2256          | 0.8247   | 0.9745            | 0.9889     | 0.5106   | 0.8053     | 0.9825              | 0.9889       | 0.4444     |
| 0.1053        | 2.56  | 52000 | 0.2416          | 0.8456   | 0.9799            | 0.9889     | 0.5679   | 0.8434     | 0.9805              | 0.9889       | 0.5610     |
| 0.1049        | 2.66  | 54000 | 0.2850          | 0.7585   | 0.9740            | 0.8902     | 0.4112   | 0.7650     | 0.9802              | 0.9865       | 0.3284     |
| 0.098         | 2.76  | 56000 | 0.2828          | 0.8049   | 0.9642            | 0.9889     | 0.4615   | 0.7750     | 0.9856              | 0.9889       | 0.3506     |
| 0.0962        | 2.86  | 58000 | 0.2238          | 0.8540   | 0.9798            | 0.9889     | 0.5934   | 0.8348     | 0.9860              | 0.9889       | 0.5294     |
| 0.0975        | 2.95  | 60000 | 0.2494          | 0.8249   | 0.9715            | 0.9889     | 0.5143   | 0.7967     | 0.9858              | 0.9889       | 0.4154     |
| 0.0877        | 3.05  | 62000 | 0.2464          | 0.8274   | 0.9733            | 0.9889     | 0.5200   | 0.8023     | 0.9847              | 0.9889       | 0.4333     |
| 0.0848        | 3.15  | 64000 | 0.2338          | 0.8263   | 0.9740            | 0.9889     | 0.5161   | 0.8077     | 0.9814              | 0.9889       | 0.4528     |
| 0.0859        | 3.25  | 66000 | 0.2335          | 0.8365   | 0.9750            | 0.9889     | 0.5455   | 0.8108     | 0.9859              | 0.9889       | 0.4576     |
| 0.084         | 3.35  | 68000 | 0.2067          | 0.8343   | 0.9763            | 0.9889     | 0.5376   | 0.8148     | 0.9837              | 0.9889       | 0.4717     |
| 0.0837        | 3.45  | 70000 | 0.2516          | 0.8249   | 0.9746            | 0.9889     | 0.5111   | 0.8097     | 0.9803              | 0.9889       | 0.46       |
| 0.0809        | 3.54  | 72000 | 0.2948          | 0.8258   | 0.9728            | 0.9944     | 0.5102   | 0.8045     | 0.9824              | 1.0          | 0.4310     |
| 0.0833        | 3.64  | 74000 | 0.2457          | 0.8494   | 0.9744            | 0.9944     | 0.5794   | 0.8173     | 0.9893              | 1.0          | 0.4627     |
| 0.0796        | 3.74  | 76000 | 0.3188          | 0.8277   | 0.9733            | 0.9889     | 0.5208   | 0.8059     | 0.9825              | 0.9889       | 0.4464     |
| 0.0821        | 3.84  | 78000 | 0.2642          | 0.8343   | 0.9714            | 0.9944     | 0.5370   | 0.8045     | 0.9870              | 1.0          | 0.4265     |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.9.0+cu102
- Datasets 1.9.0
- Tokenizers 0.10.2


---
language:
- da
- no
- nb
- nn
- sv
- fo
- is
license: mit
datasets:
- dane
- norne
- wikiann
- suc3.0
model-index:
- name: nbailab-base-ner-scandi
  results: []
widget:
- "Hans er en professor på Københavns Universitetet i København, og han er en rigtig københavner. Hans kat, altså Hans' kat, Lisa, er supersød. Han fik købt en Mona Lisa på tilbud i Netto og gav den til sin kat, og nu er Mona Lisa'en Lisa's kæreste eje. Hans bror Peter og Hans besluttede, at Peterskirken skulle have fint besøg. Men nu har de begge Corona."
inference:
  parameters:
    aggregation_strategy: "first"
---

# ScandiNER - Named Entity Recognition model for Scandinavian Languages

This model is a fine-tuned version of [NbAiLab/nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base) for Named Entity Recognition for Danish, Norwegian (both Bokmål and Nynorsk), Swedish, Icelandic and Faroese. It has been fine-tuned on the concatenation of [DaNE](https://aclanthology.org/2020.lrec-1.565/), [NorNE](https://arxiv.org/abs/1911.12146), [SUC 3.0](https://spraakbanken.gu.se/en/resources/suc3) and the Icelandic and Faroese parts of the [WikiANN](https://aclanthology.org/P17-1178/) dataset. It also works reasonably well on English sentences, given the fact that the pretrained model is also trained on English data along with Scandinavian languages.

The model will predict the following four entities:

| **Tag** | **Name** | **Description** |
| :------ | :------- | :-------------- |
| `PER` | Person | The name of a person (e.g., *Birgitte* and *Mohammed*) |
| `LOC` | Location | The name of a location (e.g., *Tyskland* and *Djurgården*) |
| `ORG` | Organisation | The name of an organisation (e.g., *Bunnpris* and *Landsbankinn*) |
| `MISC` | Miscellaneous | A named entity of a different kind (e.g., *Ūjķnustu pund* and *Mona Lisa*) |

## Quick start

You can use this model in your scripts as follows:

```python
>>> from transformers import pipeline
>>> import pandas as pd
>>> ner = pipeline(task='ner',
...                model='saattrupdan/nbailab-base-ner-scandi',
...                aggregation_strategy='first')
>>> result = ner('Borghild kjøper seg inn i Bunnpris')
>>> pd.DataFrame.from_records(result)
  entity_group     score      word  start  end
0          PER  0.981257  Borghild      0    8
1          ORG  0.974099  Bunnpris     26   34
```

## Performance

The following is the Micro-F1 NER performance on Scandinavian NER test datasets, compared with the current state-of-the-art. The models have been evaluated on the test set along with 9 bootstrapped versions of it, with the mean and 95% confidence interval shown here:

| **Model ID** | **DaNE** | **NorNE-NB** | **NorNE-NN** | **SUC 3.0** | **WikiANN-IS** | **WikiANN-FO** | **Average** |
| :----------- | -------: | -----------: | -----------: | ----------: | -------------: | -------------: | ----------: |
| saattrupdan/nbailab-base-ner-scandi | **87.44 ± 0.81** | **91.06 ± 0.26** | **90.42 ± 0.61** | **88.37 ± 0.17** | **88.61 ± 0.41** | **90.22 ± 0.46** | **89.08 ± 0.46** |
| chcaa/da\_dacy\_large\_trf | 83.61 ± 1.18 | 78.90 ± 0.49 | 72.62 ± 0.58 | 53.35 ± 0.17 | 50.57 ± 0.46 | 51.72 ± 0.52 | 63.00 ± 0.57 |
| RecordedFuture/Swedish-NER | 64.09 ± 0.97 | 61.74 ± 0.50 | 56.67 ± 0.79 | 66.60 ± 0.27 | 34.54 ± 0.73 | 42.16 ± 0.83 | 53.32 ± 0.69 |
| Maltehb/danish-bert-botxo-ner-dane | 69.25 ± 1.17 | 60.57 ± 0.27 | 35.60 ± 1.19 | 38.37 ± 0.26 | 21.00 ± 0.57 | 27.88 ± 0.48 | 40.92 ± 0.64 |
| Maltehb/-l-ctra-danish-electra-small-uncased-ner-dane | 70.41 ± 1.19 | 48.76 ± 0.70 | 27.58 ± 0.61 | 35.39 ± 0.38 | 26.22 ± 0.52 | 28.30 ± 0.29 | 39.70 ± 0.61 |
| radbrt/nb\_nocy\_trf | 56.82 ± 1.63 | 68.20 ± 0.75 | 69.22 ± 1.04 | 31.63 ± 0.29 | 20.32 ± 0.45 | 12.91 ± 0.50 | 38.08 ± 0.75 |

Aside from its high accuracy, it's also substantially **smaller** and **faster** than the previous state-of-the-art:

| **Model ID** | **Samples/second** | **Model size** |
| :----------- | -----------------: | -------------: |
| saattrupdan/nbailab-base-ner-scandi | 4.16 ± 0.18 | 676 MB |
| chcaa/da\_dacy\_large\_trf | 0.65 ± 0.01 | 2,090 MB |

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 90135.90000000001
- num_epochs: 1000

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Micro F1 | Micro F1 No Misc |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:----------------:|
| 0.6682        | 1.0   | 2816  | 0.0872          | 0.6916   | 0.7306           |
| 0.0684        | 2.0   | 5632  | 0.0464          | 0.8167   | 0.8538           |
| 0.0444        | 3.0   | 8448  | 0.0367          | 0.8485   | 0.8783           |
| 0.0349        | 4.0   | 11264 | 0.0316          | 0.8684   | 0.8920           |
| 0.0282        | 5.0   | 14080 | 0.0290          | 0.8820   | 0.9033           |
| 0.0231        | 6.0   | 16896 | 0.0283          | 0.8854   | 0.9060           |
| 0.0189        | 7.0   | 19712 | 0.0253          | 0.8964   | 0.9156           |
| 0.0155        | 8.0   | 22528 | 0.0260          | 0.9016   | 0.9201           |
| 0.0123        | 9.0   | 25344 | 0.0266          | 0.9059   | 0.9233           |
| 0.0098        | 10.0  | 28160 | 0.0280          | 0.9091   | 0.9279           |
| 0.008         | 11.0  | 30976 | 0.0309          | 0.9093   | 0.9287           |
| 0.0065        | 12.0  | 33792 | 0.0313          | 0.9103   | 0.9284           |
| 0.0053        | 13.0  | 36608 | 0.0322          | 0.9078   | 0.9257           |
| 0.0046        | 14.0  | 39424 | 0.0343          | 0.9075   | 0.9256           |

### Framework versions

- Transformers 4.10.3
- Pytorch 1.9.0+cu102
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model-index:
- name: xlmr-base-texas-squad-es
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlmr-base-texas-squad-es

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5504

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.0645        | 0.24  | 1000  | 1.7915          |
| 1.8458        | 0.47  | 2000  | 1.7873          |
| 1.8208        | 0.71  | 3000  | 1.6628          |
| 1.7743        | 0.95  | 4000  | 1.5684          |
| 1.5636        | 1.18  | 5000  | 1.5686          |
| 1.6017        | 1.42  | 6000  | 1.5484          |
| 1.6271        | 1.66  | 7000  | 1.5173          |
| 1.5975        | 1.89  | 8000  | 1.5209          |
| 1.477         | 2.13  | 9000  | 1.5766          |
| 1.4389        | 2.37  | 10000 | 1.5392          |
| 1.3389        | 2.6   | 11000 | 1.5298          |
| 1.437         | 2.84  | 12000 | 1.5504          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model-index:
- name: xlmr-base-texas-squad-fr
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xlmr-base-texas-squad-fr

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6133

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.1478        | 0.23  | 1000  | 1.8543          |
| 1.9827        | 0.46  | 2000  | 1.7643          |
| 1.8427        | 0.69  | 3000  | 1.6789          |
| 1.8372        | 0.92  | 4000  | 1.6137          |
| 1.7318        | 1.15  | 5000  | 1.6093          |
| 1.6603        | 1.38  | 6000  | 1.7157          |
| 1.6334        | 1.61  | 7000  | 1.6302          |
| 1.6716        | 1.84  | 8000  | 1.5845          |
| 1.5192        | 2.06  | 9000  | 1.6690          |
| 1.5174        | 2.29  | 10000 | 1.6669          |
| 1.4611        | 2.52  | 11000 | 1.6301          |
| 1.4648        | 2.75  | 12000 | 1.6009          |
| 1.5052        | 2.98  | 13000 | 1.6133          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
language:
- da
license: mit
tags:
- generated_from_trainer
model-index:
- name: xlmr-base-texas-squad-da
  results: []
widget:
- text: "Hvem handler artiklen om?"
  context: "Forfatter og musiker Flemming Quist Møller er død i en alder af 79 år. Den folkekære kunstner faldt om ved morgenbordet med en blodprop i hjertet i mandags. Det kunne forfatterens søn, Carl Quist-Møller, bekræfte over for TV 2 Lorry.- Han faldt om i det hus i Taarbæk, hvor han er vokset op og også har boet de sidste år af sit liv. Han blev lagt i koma på Rigshospitalet. Her har vi siddet omkring ham i en uge, siger Carl Quist-Møller til mediet.MindeordI mange år var Flemming Quist Møller en del af bandet Bazaar sammen med Peter Bastian, Anders Koppel og Mehmet Ozan.Anders Koppel er tydeligt rørt over vennens død, da Ekstra Bladet rækker ud til ham mandag aften.- Det er en stor del af mit liv, der er forsvundet med Flemmings liv, det er klart. Vi har spillet sammen i 37 år, siger han og fortsætter:- Jeg vil mest huske ham for hans ukonventionelle tilgang til alting. Flemming havde et meget stærkt blik for det autentiske og ærlige. Han var ikke bundet af normer -tværtimod, hvis han så en norm, hvor noget skulle gøres på en bestemt måde, så flygtede han eller prøvede at springe det i stumper og stykker.Ifølge den danske musiker og komponist er netop følgende ord rammende for Flemming Quist Møller: Original, vidende, kompromisløs og humoristisk."
---

# TExAS-SQuAD-da

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the TExAS-SQuAD-da dataset.
It achieves the following results on the evaluation set:
- Exact match: 63.96%
- F1-score: 68.40%

In comparison, the `jacobshein/danish-bert-botxo-qa-squad` model achieves 30.37% EM and 37.15% F1.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 1.6438        | 1.0   | 4183  | 1.4711          |
| 1.4079        | 2.0   | 8366  | 1.4356          |
| 1.2532        | 3.0   | 12549 | 1.4509          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model-index:
- name: icebert-texas-squad-is
  results: []
widget:
- text: "Hvenær var Halldór Laxness í menntaskóla ?"
  context: "Halldór Laxness ( Halldór Kiljan ) fæddist í Reykjavík 23. apríl árið 1902 og átti í fyrstu heima við Laugaveg en árið 1905 settist fjölskyldan að í Laxnesi í Mosfellssveit . Þar ólst Halldór upp en sótti skóla í Reykjavík á unglingsárum . Ungur hélt hann síðan utan og var langdvölum erlendis um árabil – í ýmsum Evrópulöndum og síðar í Ameríku . Þegar hann var heima bjó hann í Reykjavík þar til hann og kona hans , Auður Sveinsdóttir , byggðu sér húsið Gljúfrastein í Mosfellssveit og fluttu þangað árið 1945 . Þar var heimili þeirra alla tíð síðan og þar er nú safn til minningar um þau . Halldór lést 8. febrúar 1998 . Skólaganga Halldórs varð ekki löng . Árið 1918 hóf hann nám við Menntaskólann í Reykjavík en hafði lítinn tíma til að læra , enda var hann að skrifa skáldsögu , Barn náttúrunnar , sem kom út haustið 1919 – þá þegar var höfundurinn ungi farinn af landi brott . Sagan vakti þó nokkra athygli og í Alþýðublaðinu sagði m.a. : „ Og hver veit nema að Halldór frá Laxnesi eigi eftir að verða óskabarn íslensku þjóðarinnar . “ Upp frá þessu sendi Halldór frá sér bók nánast á hverju ári , stundum fleiri en eina , í yfir sex áratugi . Afköst hans voru með eindæmum ; hann skrifaði fjölda skáldsagna , sumar í nokkrum hlutum , leikrit , kvæði , smásagnasöfn og endurminningabækur og gaf auk þess út mörg greinasöfn og ritgerðir . Bækurnar eru fjölbreyttar en eiga það sameiginlegt að vera skrifaðar af einstakri stílgáfu , djúpum mannskilningi og víðtækri þekkingu á sögu og samfélagi . Þar birtast oft afgerandi skoðanir á þjóðfélagsmálum og sögupersónur eru margar einkar eftirminnilegar ; tilsvör þeirra og lunderni hafa orðið samofin þjóðarsálinni . Þekktustu verk Halldórs eru eflaust skáldsögurnar stóru og rismiklu , s.s. Salka Valka , Sjálfstætt fólk , Heimsljós , Íslandsklukkan og Gerpla , og raunar mætti telja upp mun fleiri ; Kvæðabók hans er í uppáhaldi hjá mörgum sem og minningabækurnar sem hann skrifaði á efri árum um æskuár sín ; af þekktum greinasöfnum og ritgerðum má nefna Alþýðubókina og Skáldatíma . Mikið hefur verið skrifað um verk og ævi skáldsins , en hér skal aðeins bent á ítarlega frásögn og greiningu Halldórs Guðmundssonar í bókinni Halldór Laxness – ævisaga ."
---

# TExAS-SQuAD-is

This model is a fine-tuned version of [IceBERT](https://huggingface.co/vesteinn/IceBERT) on the TExAS-SQuAD-is dataset.
It achieves the following results on the evaluation set:
- Exact match: xx.xx%
- F1-score: xx.xx%

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.5353        | 0.12  | 500   | 2.2356          |
| 2.364         | 0.24  | 1000  | 2.0607          |
| 2.2243        | 0.36  | 1500  | 2.0617          |
| 2.1403        | 0.49  | 2000  | 1.9934          |
| 2.1491        | 0.61  | 2500  | 2.0515          |
| 2.0604        | 0.73  | 3000  | 1.9602          |
| 2.0232        | 0.85  | 3500  | 1.8954          |
| 2.0905        | 0.97  | 4000  | 1.9474          |
| 1.9229        | 1.09  | 4500  | 1.9814          |
| 1.9162        | 1.22  | 5000  | 1.9053          |
| 1.8937        | 1.34  | 5500  | 1.9501          |
| 1.9085        | 1.46  | 6000  | 1.8882          |
| 1.8671        | 1.58  | 6500  | 1.8996          |
| 1.8997        | 1.7   | 7000  | 1.8340          |
| 1.8546        | 1.82  | 7500  | 1.8883          |
| 1.8935        | 1.95  | 8000  | 1.8567          |
| 1.7031        | 2.07  | 8500  | 1.9206          |
| 1.7699        | 2.19  | 9000  | 1.8790          |
| 1.7016        | 2.31  | 9500  | 1.8670          |
| 1.7744        | 2.43  | 10000 | 1.8951          |
| 1.7518        | 2.55  | 10500 | 1.9550          |
| 1.7503        | 2.68  | 11000 | 1.9120          |
| 1.7818        | 2.8   | 11500 | 1.8820          |
| 1.6955        | 2.92  | 12000 | 1.8908          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model-index:
- name: xlmr-base-texas-squad-is
  results: []
widget:
- text: "Hvenær var Halldór Laxness í menntaskóla ?"
  context: "Halldór Laxness ( Halldór Kiljan ) fæddist í Reykjavík 23. apríl árið 1902 og átti í fyrstu heima við Laugaveg en árið 1905 settist fjölskyldan að í Laxnesi í Mosfellssveit . Þar ólst Halldór upp en sótti skóla í Reykjavík á unglingsárum . Ungur hélt hann síðan utan og var langdvölum erlendis um árabil – í ýmsum Evrópulöndum og síðar í Ameríku . Þegar hann var heima bjó hann í Reykjavík þar til hann og kona hans , Auður Sveinsdóttir , byggðu sér húsið Gljúfrastein í Mosfellssveit og fluttu þangað árið 1945 . Þar var heimili þeirra alla tíð síðan og þar er nú safn til minningar um þau . Halldór lést 8. febrúar 1998 . Skólaganga Halldórs varð ekki löng . Árið 1918 hóf hann nám við Menntaskólann í Reykjavík en hafði lítinn tíma til að læra , enda var hann að skrifa skáldsögu , Barn náttúrunnar , sem kom út haustið 1919 – þá þegar var höfundurinn ungi farinn af landi brott . Sagan vakti þó nokkra athygli og í Alþýðublaðinu sagði m.a. : „ Og hver veit nema að Halldór frá Laxnesi eigi eftir að verða óskabarn íslensku þjóðarinnar . “ Upp frá þessu sendi Halldór frá sér bók nánast á hverju ári , stundum fleiri en eina , í yfir sex áratugi . Afköst hans voru með eindæmum ; hann skrifaði fjölda skáldsagna , sumar í nokkrum hlutum , leikrit , kvæði , smásagnasöfn og endurminningabækur og gaf auk þess út mörg greinasöfn og ritgerðir . Bækurnar eru fjölbreyttar en eiga það sameiginlegt að vera skrifaðar af einstakri stílgáfu , djúpum mannskilningi og víðtækri þekkingu á sögu og samfélagi . Þar birtast oft afgerandi skoðanir á þjóðfélagsmálum og sögupersónur eru margar einkar eftirminnilegar ; tilsvör þeirra og lunderni hafa orðið samofin þjóðarsálinni . Þekktustu verk Halldórs eru eflaust skáldsögurnar stóru og rismiklu , s.s. Salka Valka , Sjálfstætt fólk , Heimsljós , Íslandsklukkan og Gerpla , og raunar mætti telja upp mun fleiri ; Kvæðabók hans er í uppáhaldi hjá mörgum sem og minningabækurnar sem hann skrifaði á efri árum um æskuár sín ; af þekktum greinasöfnum og ritgerðum má nefna Alþýðubókina og Skáldatíma . Mikið hefur verið skrifað um verk og ævi skáldsins , en hér skal aðeins bent á ítarlega frásögn og greiningu Halldórs Guðmundssonar í bókinni Halldór Laxness – ævisaga ."
---

# TExAS-SQuAD-is

This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on the TExAS-SQuAD-is dataset.
It achieves the following results on the evaluation set:
- Exact match: 56.91%
- F1-score: 59.93%

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.1458        | 1.0   | 4219  | 1.8892          |
| 1.9202        | 2.0   | 8438  | 1.8566          |
| 1.7377        | 3.0   | 12657 | 1.8688          |

### Framework versions

- Transformers 4.12.2
- Pytorch 1.8.1+cu101
- Datasets 1.12.1
- Tokenizers 0.10.3