---
language:
- myv
license: apache-2.0
tags:
- automatic-speech-recognition
- mozilla-foundation/common_voice_8_0
- generated_from_trainer
- myv
- robust-speech-event
- model_for_talk
datasets:
- common_voice
model-index:
- name: wav2vec2-large-xls-r-300m-myv-v1
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice 8
      type: mozilla-foundation/common_voice_8_0
      args: myv
    metrics:
       - name: Test WER
         type: wer
         value: 0.599548532731377
       - name: Test CER
         type: cer
         value: 0.12953851902597
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xls-r-300m-myv-v1

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - MYV dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8537
- Wer: 0.6160

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.000222
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 150
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Wer    |
|:-------------:|:------:|:----:|:---------------:|:------:|
| 19.453        | 1.92   | 50   | 16.4001         | 1.0    |
| 9.6875        | 3.85   | 100  | 5.4468          | 1.0    |
| 4.9988        | 5.77   | 150  | 4.3507          | 1.0    |
| 4.1148        | 7.69   | 200  | 3.6753          | 1.0    |
| 3.4922        | 9.62   | 250  | 3.3103          | 1.0    |
| 3.2443        | 11.54  | 300  | 3.1741          | 1.0    |
| 3.164         | 13.46  | 350  | 3.1346          | 1.0    |
| 3.0954        | 15.38  | 400  | 3.0428          | 1.0    |
| 3.0076        | 17.31  | 450  | 2.9137          | 1.0    |
| 2.6883        | 19.23  | 500  | 2.1476          | 0.9978 |
| 1.5124        | 21.15  | 550  | 0.8955          | 0.8225 |
| 0.8711        | 23.08  | 600  | 0.6948          | 0.7591 |
| 0.6695        | 25.0   | 650  | 0.6683          | 0.7636 |
| 0.5606        | 26.92  | 700  | 0.6821          | 0.7435 |
| 0.503         | 28.85  | 750  | 0.7220          | 0.7516 |
| 0.4528        | 30.77  | 800  | 0.6638          | 0.7324 |
| 0.4219        | 32.69  | 850  | 0.7120          | 0.7435 |
| 0.4109        | 34.62  | 900  | 0.7122          | 0.7511 |
| 0.3887        | 36.54  | 950  | 0.7179          | 0.7199 |
| 0.3895        | 38.46  | 1000 | 0.7322          | 0.7525 |
| 0.391         | 40.38  | 1050 | 0.6850          | 0.7364 |
| 0.3537        | 42.31  | 1100 | 0.7571          | 0.7279 |
| 0.3267        | 44.23  | 1150 | 0.7575          | 0.7257 |
| 0.3195        | 46.15  | 1200 | 0.7580          | 0.6998 |
| 0.2891        | 48.08  | 1250 | 0.7452          | 0.7101 |
| 0.294         | 50.0   | 1300 | 0.7316          | 0.6945 |
| 0.2854        | 51.92  | 1350 | 0.7241          | 0.6757 |
| 0.2801        | 53.85  | 1400 | 0.7532          | 0.6887 |
| 0.2502        | 55.77  | 1450 | 0.7587          | 0.6811 |
| 0.2427        | 57.69  | 1500 | 0.7231          | 0.6851 |
| 0.2311        | 59.62  | 1550 | 0.7288          | 0.6632 |
| 0.2176        | 61.54  | 1600 | 0.7711          | 0.6664 |
| 0.2117        | 63.46  | 1650 | 0.7914          | 0.6940 |
| 0.2114        | 65.38  | 1700 | 0.8065          | 0.6918 |
| 0.1913        | 67.31  | 1750 | 0.8372          | 0.6945 |
| 0.1897        | 69.23  | 1800 | 0.8051          | 0.6869 |
| 0.1865        | 71.15  | 1850 | 0.8076          | 0.6740 |
| 0.1844        | 73.08  | 1900 | 0.7935          | 0.6708 |
| 0.1757        | 75.0   | 1950 | 0.8015          | 0.6610 |
| 0.1636        | 76.92  | 2000 | 0.7614          | 0.6414 |
| 0.1637        | 78.85  | 2050 | 0.8123          | 0.6592 |
| 0.1599        | 80.77  | 2100 | 0.7907          | 0.6566 |
| 0.1498        | 82.69  | 2150 | 0.8641          | 0.6757 |
| 0.1545        | 84.62  | 2200 | 0.7438          | 0.6682 |
| 0.1433        | 86.54  | 2250 | 0.8014          | 0.6624 |
| 0.1427        | 88.46  | 2300 | 0.7758          | 0.6646 |
| 0.1423        | 90.38  | 2350 | 0.7741          | 0.6423 |
| 0.1298        | 92.31  | 2400 | 0.7938          | 0.6414 |
| 0.1111        | 94.23  | 2450 | 0.7976          | 0.6467 |
| 0.1243        | 96.15  | 2500 | 0.7916          | 0.6481 |
| 0.1215        | 98.08  | 2550 | 0.7594          | 0.6392 |
| 0.113         | 100.0  | 2600 | 0.8236          | 0.6392 |
| 0.1077        | 101.92 | 2650 | 0.7959          | 0.6347 |
| 0.0988        | 103.85 | 2700 | 0.8189          | 0.6392 |
| 0.0953        | 105.77 | 2750 | 0.8157          | 0.6414 |
| 0.0889        | 107.69 | 2800 | 0.7946          | 0.6369 |
| 0.0929        | 109.62 | 2850 | 0.8255          | 0.6360 |
| 0.0822        | 111.54 | 2900 | 0.8320          | 0.6334 |
| 0.086         | 113.46 | 2950 | 0.8539          | 0.6490 |
| 0.0825        | 115.38 | 3000 | 0.8438          | 0.6418 |
| 0.0727        | 117.31 | 3050 | 0.8568          | 0.6481 |
| 0.0717        | 119.23 | 3100 | 0.8447          | 0.6512 |
| 0.0815        | 121.15 | 3150 | 0.8470          | 0.6445 |
| 0.0689        | 123.08 | 3200 | 0.8264          | 0.6249 |
| 0.0726        | 125.0  | 3250 | 0.7981          | 0.6169 |
| 0.0648        | 126.92 | 3300 | 0.8237          | 0.6200 |
| 0.0632        | 128.85 | 3350 | 0.8416          | 0.6249 |
| 0.06          | 130.77 | 3400 | 0.8276          | 0.6173 |
| 0.0616        | 132.69 | 3450 | 0.8429          | 0.6209 |
| 0.0614        | 134.62 | 3500 | 0.8485          | 0.6271 |
| 0.0539        | 136.54 | 3550 | 0.8598          | 0.6218 |
| 0.0555        | 138.46 | 3600 | 0.8557          | 0.6169 |
| 0.0604        | 140.38 | 3650 | 0.8436          | 0.6186 |
| 0.0556        | 142.31 | 3700 | 0.8428          | 0.6178 |
| 0.051         | 144.23 | 3750 | 0.8440          | 0.6142 |
| 0.0526        | 146.15 | 3800 | 0.8566          | 0.6142 |
| 0.052         | 148.08 | 3850 | 0.8544          | 0.6178 |
| 0.0519        | 150.0  | 3900 | 0.8537          | 0.6160 |

### Framework versions

- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.2
- Tokenizers 0.11.0


---
tags:
- spacy
- token-classification
language:
- sv
license: mit
model-index:
- name: sv_pipeline
  results:
  - task:
      name: POS
      type: token-classification
    metrics:
    - name: POS Accuracy
      type: accuracy
      value: 0.9818079056
  - task:
      name: SENTER
      type: token-classification
    metrics:
    - name: SENTER Precision
      type: precision
      value: 0.9212548015
    - name: SENTER Recall
      type: recall
      value: 0.9368489583
    - name: SENTER F Score
      type: f_score
      value: 0.9289864429
  - task:
      name: UNLABELED_DEPENDENCIES
      type: token-classification
    metrics:
    - name: Unlabeled Dependencies Accuracy
      type: accuracy
      value: 0.9198832946
  - task:
      name: LABELED_DEPENDENCIES
      type: token-classification
    metrics:
    - name: Labeled Dependencies Accuracy
      type: accuracy
      value: 0.9198832946
---


---
language: sv
tags:
- audio
- automatic-speech-recognition
- voxrex
license: cc0-1.0
---

# Wav2vec 2.0 large VoxRex (C)

**Disclaimer:** This is a work in progress.
**Update 2022-01-08:** Updated to VoxRex-C version, use git to get the older (B) version

This model has been pretrained for 400,000 updates on the P4-10k corpus which contains 10 000 hours of swedish local public service radio as well as 1500 hours of audio books and other speech from KBs collections.

![Accuracy during training](accuracy.svg "Accuracy")


---
language: sv

widget:
    - text: "Jag har ätit en <mask>"
---

## KB-BART

A [BART](https://arxiv.org/abs/1910.13461) model trained on a Swedish corpus consisting of 15 billion tokens (about 80GB of text). The model was trained with [Fairseq](https://github.com/pytorch/fairseq), and converted to be compatible with Huggingface.

Training code can be found [here](https://github.com/kb-labb/kb_bart).

## Usage

```python
from transformers import BartForConditionalGeneration, PreTrainedTokenizerFast, AutoTokenizer

model = BartForConditionalGeneration.from_pretrained("KBLab/bart-base-swedish-cased")
tok = AutoTokenizer.from_pretrained("KBLab/bart-base-swedish-cased")

model.eval()

input_ids = tok.encode(
    "Jag har ätit en utsökt <mask> på restaurang vid <mask> .", return_tensors="pt"
)

# Simple greedy search
output_ids = model.generate(
    input_ids,
    min_length=15,
    max_length=25,
    num_beams=1,
    do_sample=False,
)
tok.decode(output_ids[0])
# '</s><s> Jag har ätit en utsökt middag på restaurang vid havet på restaurang vid havet på restaurang vid havet.</s>'

# Sampling
output_ids = model.generate(
    input_ids,
    min_length=15,
    max_length=20,
    num_beams=1,
    do_sample=True,
)
tok.decode(output_ids[0])
#'</s><s> Jag har ätit en utsökt god mat som de tagit in på restaurang vid avröjda</s>'

# Beam search
output_ids = model.generate(
    input_ids,
    min_length=15,
    max_length=25,
    no_repeat_ngram_size=3,
    num_beams=8,
    early_stopping=True,
    do_sample=True,
    num_return_sequences=6
)
tok.decode(output_ids[0])
# '</s><s> Jag har ätit en utsökt middag på restaurang vid havet. Jag har varit ute och gått en sväng.</s><pad><pad>'

# Diverse beam generation
output_ids = model.generate(
    input_ids,
    min_length=50,
    max_length=100,
    no_repeat_ngram_size=3,
    num_beams=8,
    early_stopping=True,
    do_sample=False,
    num_return_sequences=6,
    num_beam_groups=8,
    diversity_penalty=2.0,
)
tok.decode(output_ids[0])
# '</s><s> Jag har ätit en utsökt middag på restaurang vid havet på restaurang. Jag har varit på restaurang i två dagar... Jag..,..!!!.. Så.. Nu.. Hej.. Vi.. Här.</s>'

```


---
tags:
- automatic-speech-recognition
- generated_from_trainer
- asr_seq2seq
---

Test


---
language: sv
datasets:
- common_voice
- NST Swedish ASR Database
- P4
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
license: cc0-1.0
model-index:
- name: Wav2vec 2.0 large VoxRex Swedish
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice
      type: common_voice
      args: sv-SE
    metrics:
    - name: Test WER
      type: wer
      value: 9.914
---
# Wav2vec 2.0 large VoxRex Swedish (C)

**Disclaimer:** This is a work in progress. See [VoxRex](https://huggingface.co/KBLab/wav2vec2-large-voxrex) for more details.

**Update 2022-01-10:** Updated to VoxRex-C version.

Finetuned version of KBs [VoxRex large](https://huggingface.co/KBLab/wav2vec2-large-voxrex) model using Swedish radio broadcasts, NST and Common Voice data. Evalutation without a language model gives the following: WER for NST + Common Voice test set (2% of total sentences) is **2.5%**. WER for Common Voice test set is **8.49%** directly and **7.37%** with a 4-gram language model.

When using this model, make sure that your speech input is sampled at 16kHz.

# Performance\*

![Comparison](comparison.png "Comparison")
*Chart shows performance without the additional 20k steps of Common Voice fine-tuning

## Training
This model has been fine-tuned for 120000 updates on NST + CommonVoice and then for an additional 20000 updates on CommonVoice only. The additional fine-tuning on CommonVoice hurts performance on the NST+CommonVoice test set somewhat and, unsurprisingly, improves it on the CommonVoice test set. It seems to perform generally better though [citation needed].

![WER during training](chart_1.svg "WER")

## Usage
The model can be used directly (without a language model) as follows:
```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").
processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-large-voxrex-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-large-voxrex-swedish")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


---
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
widget:
- source_sentence: "Mannen åt mat."
  sentences:
    - "Han förtärde en närande och nyttig måltid."
    - "Det var ett sunkigt hak med ganska gott käk."
    - "Han inmundigade middagen tillsammans med ett glas rödvin."
    - "Potatischips är jättegoda."
    - "Tryck på knappen för att få tala med kundsupporten."
  example_title: "Mat"
- source_sentence: "Kan jag deklarera digitalt från utlandet?"
  sentences:
    - "Du som befinner dig i utlandet kan deklarera digitalt på flera olika sätt."
    - "Du som har kvarskatt att betala ska göra en inbetalning till ditt skattekonto."
    - "Efter att du har deklarerat går vi igenom uppgifterna i din deklaration och räknar ut din skatt."
    - "I din deklaration som du får från oss har vi räknat ut vad du ska betala eller få tillbaka."
    - "Tryck på knappen för att få tala med kundsupporten."
  example_title: "Skatteverket FAQ"
- source_sentence: "Hon kunde göra bakåtvolter."
  sentences:
    - "Hon var atletisk."
    - "Hon var bra på gymnastik."
    - "Hon var inte atletisk."
    - "Hon var oförmögen att flippa baklänges."
  example_title: "Gymnastik"
---

# KBLab/sentence-bert-swedish-cased

This is a [sentence-transformers](https://www.SBERT.net) model: It maps Swedish sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. This model is a bilingual Swedish-English model trained according to instructions in the paper [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://arxiv.org/pdf/2004.09813.pdf) and the [documentation](https://www.sbert.net/examples/training/multilingual/README.html) accompanying its companion python package. We have used the strongest available pretrained English Bi-Encoder ([paraphrase-mpnet-base-v2](https://www.sbert.net/docs/pretrained_models.html#sentence-embedding-models)) as a teacher model, and the pretrained Swedish [KB-BERT](https://huggingface.co/KB/bert-base-swedish-cased) as the student model.

A more detailed description of the model can be found in an article we published on the [KBLab blog](https://kb-labb.github.io/posts/2021-08-23-a-swedish-sentence-transformer/).

<!--- Describe your model here -->

## Usage (Sentence-Transformers)

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

```
pip install -U sentence-transformers
```

Then you can use the model like this:

```python
from sentence_transformers import SentenceTransformer
sentences = ["Det här är en exempelmening", "Varje exempel blir konverterad"]

model = SentenceTransformer('KBLab/sentence-bert-swedish-cased')
embeddings = model.encode(sentences)
print(embeddings)
```

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

```python
from transformers import AutoTokenizer, AutoModel
import torch

#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

# Sentences we want sentence embeddings for
sentences = ['Det här är en exempelmening', 'Varje exempel blir konverterad']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('KBLab/sentence-bert-swedish-cased')
model = AutoModel.from_pretrained('KBLab/sentence-bert-swedish-cased')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, max pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)
```

## Evaluation Results

<!--- Describe how your model was evaluated -->

The model was primarily evaluated on [SweParaphrase v1.0](https://spraakbanken.gu.se/en/resources/sweparaphrase). This test set is part of [SuperLim](https://spraakbanken.gu.se/en/resources/superlim) -- a Swedish evaluation suite for natural langage understanding tasks.  We calculated Pearson and Spearman correlation between predicted model similarity scores and the human similarity score labels. The model achieved a Pearson correlation coefficient of **0.918** and a Spearman's rank correlation coefficient of **0.911**.

The following code snippet can be used to reproduce the above results:

```python
from sentence_transformers import SentenceTransformer
import pandas as pd

df = pd.read_csv(
    "sweparaphrase-dev-165.csv",
    sep="\t",
    header=None,
    names=[
        "original_id",
        "source",
        "type",
        "sentence_swe1",
        "sentence_swe2",
        "score",
        "sentence1",
        "sentence2",
    ],
)

model = SentenceTransformer("KBLab/sentence-bert-swedish-cased")

sentences1 = df["sentence_swe1"].tolist()
sentences2 = df["sentence_swe2"].tolist()

# Compute embedding for both lists
embeddings1 = model.encode(sentences1, convert_to_tensor=True)
embeddings2 = model.encode(sentences2, convert_to_tensor=True)

# Compute cosine similarity after normalizing
embeddings1 /= embeddings1.norm(dim=-1, keepdim=True)
embeddings2 /= embeddings2.norm(dim=-1, keepdim=True)

cosine_scores = embeddings1 @ embeddings2.t()
sentence_pair_scores = cosine_scores.diag()

df["model_score"] = sentence_pair_scores.cpu().tolist()
print(df[["score", "model_score"]].corr(method="spearman"))
print(df[["score", "model_score"]].corr(method="pearson"))
```

Examples how to evaluate the model on other test sets of the SuperLim suites can be found on the following links: [evaluate_faq.py](https://github.com/kb-labb/swedish-sbert/blob/main/evaluate_faq.py) (Swedish FAQ), [evaluate_swesat.py](https://github.com/kb-labb/swedish-sbert/blob/main/evaluate_swesat.py) (SweSAT synonyms), [evaluate_supersim.py](https://github.com/kb-labb/swedish-sbert/blob/main/evaluate_supersim.py) (SuperSim).

## Training

An article with more details on data and the model can be found on the [KBLab blog](https://kb-labb.github.io/posts/2021-08-23-a-swedish-sentence-transformer/).

Around 14.6 million sentences from English-Swedish parallel corpuses were used to train the model. Data was sourced from the [Open Parallel Corpus](https://opus.nlpl.eu/) (OPUS) and downloaded via the python package [opustools](https://pypi.org/project/opustools/). Datasets used were: JW300, Europarl, EUbookshop, EMEA, TED2020, Tatoeba and OpenSubtitles.

The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 227832 with parameters:
```
{'batch_size': 64, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
```

**Loss**:

`sentence_transformers.losses.MSELoss.MSELoss`

Parameters of the fit()-Method:
```
{
    "callback": null,
    "epochs": 7,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.SequentialEvaluator.SequentialEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "correct_bias": false,
        "eps": 1e-06,
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 10000,
    "weight_decay": 0.01
}
```

## Full Model Architecture
```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
```

## Citing & Authors

<!--- Describe where people can find more information -->
This model was trained by KBLab, a data lab at the National Library of Sweden.

You can cite the article on our blog: https://kb-labb.github.io/posts/2021-08-23-a-swedish-sentence-transformer/ .

## Acknowledgements

We gratefully acknowledge the HPC RIVR consortium (www.hpc-rivr.si) and EuroHPC JU ([eurohpc-ju.europa.eu](eurohpc-ju.europa.eu)) for funding this research by providing computing resources of the HPC system Vega at the Institute of Information Science (www.izum.si).


---
language: sv-SE
datasets:
- common_voice
- NST Swedish ASR Database
metrics:
- wer
- cer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Swedish by KBLab
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice sv-SE
      type: common_voice
      args: sv-SE
    metrics:
       - name: Test WER
         type: wer
         value: 14.298610
       - name: Test CER
         type: cer
         value: 4.925294
---

# Wav2Vec2-Large-XLSR-53-Swedish

Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) in Swedish using the [NST Swedish Dictation](https://www.nb.no/sprakbanken/en/resource-catalogue/oai-nb-no-sbr-17/).
When using this model, make sure that your speech input is sampled at 16kHz.

## Usage

The model can be used directly (without a language model) as follows:

```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").

processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-large-xlsr-53-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-large-xlsr-53-swedish")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()

    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```

## Evaluation

The model can be evaluated as follows on the Swedish test data of Common Voice.

```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "sv-SE", split="test")
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-large-xlsr-53-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-large-xlsr-53-swedish")
model.to("cuda")

chars_to_ignore_regex = '[,?.!\\-;:"“]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()

    return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)

# Preprocessing the datasets.
# We need to read the aduio files as arrays
def evaluate(batch):
    inputs = processor(batch["speech"], sampling_rate=16_000, return_tensors="pt", padding=True)

    with torch.no_grad():
        logits = model(inputs.input_values.to("cuda"), attention_mask=inputs.attention_mask.to("cuda")).logits

    pred_ids = torch.argmax(logits, dim=-1)
    batch["pred_strings"] = processor.batch_decode(pred_ids)

    return batch

result = test_dataset.map(evaluate, batched=True, batch_size=8)

print("WER: {:2f}".format(100 * wer.compute(predictions=result["pred_strings"], references=result["sentence"])))
print("CER: {:2f}".format(100 * wer.compute(predictions=[" ".join(list(entry)) for entry in result["pred_strings"]], references=[" ".join(list(entry)) for entry in result["sentence"]])))
```

**WER**: 14.298610%
**CER**: 4.925294%

## Training

First the XLSR model was further pre-trained for 50 epochs with a corpus consisting of 1000 hours spoken Swedish from various radio stations. Secondly [NST Swedish Dictation](https://www.nb.no/sprakbanken/en/resource-catalogue/oai-nb-no-sbr-17/) was used for fine tuning as well as [Common Voice](https://commonvoice.mozilla.org/en/datasets). Lastly only Common Voice dataset was used for final finetuning. The [Fairseq](https://github.com/fairseq) scripts were used.


---
language: sv-SE
datasets:
- common_voice
- NST Swedish ASR Database
metrics:
- wer
- cer
tags:
- audio
- automatic-speech-recognition
- speech
- voxpopuli
license: cc-by-nc-4.0
model-index:
- name: Wav2vec 2.0 large VoxPopuli-sv swedish
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice
      type: common_voice
      args: sv-SE
    metrics:
       - name: Test WER
         type: wer
         value: 10.994764
       - name: Test CER
         type: cer
         value: 3.946846
---
# Wav2vec 2.0 large-voxpopuli-sv-swedish

**PLEASE NOTE that [this](https://huggingface.co/KBLab/wav2vec2-large-voxrex-swedish) model performs better and has a less restrictive license.**

Additionally pretrained and finetuned version of Facebooks [VoxPopuli-sv large](https://huggingface.co/facebook/wav2vec2-large-sv-voxpopuli) model using Swedish radio broadcasts, NST and Common Voice data. Evalutation without a language model gives the following: WER for NST + Common Voice test set (2% of total sentences) is **3.95%**. WER for Common Voice test set is **10.99%** directly and **7.82%** with a 4-gram language model.

When using this model, make sure that your speech input is sampled at 16kHz.

## Training
This model has additionally pretrained on 1000h of Swedish local radio broadcasts, fine-tuned for 120000 updates on NST + CommonVoice and then for an additional 20000 updates on CommonVoice only. The additional fine-tuning on CommonVoice hurts performance on the NST+CommonVoice test set somewhat and, unsurprisingly, improves it on the CommonVoice test set. It seems to perform generally better though [citation needed].

## Usage
The model can be used directly (without a language model) as follows:
```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").
processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-large-voxpopuli-sv-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-large-voxpopuli-sv-swedish")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


---
language: sv-SE
datasets:
- common_voice
- NST Swedish ASR Database
metrics:
- wer
#- cer
tags:
- audio
- automatic-speech-recognition
- speech
- voxpopuli
license: cc-by-nc-4.0
model-index:
- name: Wav2vec 2.0 base VoxPopuli-sv swedish
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: NST Swedish ASR Database
    metrics:
       - name: Test WER
         type: wer
         value: 5.619804368919309
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice
      type: common_voice
      args: sv-SE
    metrics:
       - name: Test WER
         type: wer
         value: 19.145252414798616
---
# Wav2vec 2.0 base-voxpopuli-sv-swedish
Finetuned version of Facebooks [VoxPopuli-sv base](https://huggingface.co/facebook/wav2vec2-base-sv-voxpopuli) model using NST and Common Voice data. Evalutation without a language model gives the following: WER for NST + Common Voice test set (2% of total sentences) is **5.62%**, WER for Common Voice test set is **19.15%**.

When using this model, make sure that your speech input is sampled at 16kHz.

## Usage
The model can be used directly (without a language model) as follows:
```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
test_dataset = load_dataset("common_voice", "sv-SE", split="test[:2%]").
processor = Wav2Vec2Processor.from_pretrained("KBLab/wav2vec2-base-voxpopuli-sv-swedish")
model = Wav2Vec2ForCTC.from_pretrained("KBLab/wav2vec2-base-voxpopuli-sv-swedish")
resampler = torchaudio.transforms.Resample(48_000, 16_000)
# Preprocessing the datasets.
# We need to read the aduio files as arrays
def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(batch["path"])
    batch["speech"] = resampler(speech_array).squeeze().numpy()
    return batch
test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits
predicted_ids = torch.argmax(logits, dim=-1)
print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


---
language: hi
datasets:
- Interspeech 2021
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: XLSR Wav2Vec2 Hindi by Nalini Kumari
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Common Voice hi
      type: common_voice
      args: hi
    metrics:
       - name: Test WER
         type: wer
         value: 72.73
---

# Hindi-Speech to Text Model </br>
The primary objective of this project is to develop a speech recognition system for the Hindi language. As there are very few systems available for speech to text in the Hindi language. Therefore it is an attempt to develop a system where a language model is developed using Machine learning libraries for speech-to-text conversion in the Hindi language.


---
tags:
- conversational
---

# Jarvis DialoGPT Model


---
language: da
license: cc-by-4.0
---

# Danish ELECTRA small (cased)

An [ELECTRA](https://arxiv.org/abs/2003.10555) model pretrained on a custom Danish corpus (~17.5gb).
For details regarding data sources and training procedure, along with benchmarks on downstream tasks, go to: https://github.com/sarnikowski/danish_transformers/tree/main/electra

## Usage

```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("sarnikowski/electra-small-discriminator-da-256-cased")
model = AutoModel.from_pretrained("sarnikowski/electra-small-discriminator-da-256-cased")
```

## Questions?

If you have any questions feel free to open an issue on the [danish_transformers](https://github.com/sarnikowski/danish_transformers) repository, or send an email to p.sarnikowski@gmail.com


---
language: da
license: cc-by-4.0
---

# Danish ConvBERT medium small (cased)

[ConvBERT](https://arxiv.org/abs/2008.02496) model pretrained on a custom Danish corpus (~17.5gb).
For details regarding data sources and training procedure, along with benchmarks on downstream tasks, go to: https://github.com/sarnikowski/danish_transformers

## Usage

```python
from transformers import ConvBertTokenizer, ConvBertModel

tokenizer = ConvBertTokenizer.from_pretrained("sarnikowski/convbert-medium-small-da-cased")
model = ConvBertModel.from_pretrained("sarnikowski/convbert-medium-small-da-cased")
```

## Questions?

If you have any questions feel free to open an issue on the [danish_transformers](https://github.com/sarnikowski/danish_transformers) repository, or send an email to p.sarnikowski@gmail.com


---
language: da
license: cc-by-4.0
---

# Danish ConvBERT small (cased)

[ConvBERT](https://arxiv.org/abs/2008.02496) model pretrained on a custom Danish corpus (~17.5gb).
For details regarding data sources and training procedure, along with benchmarks on downstream tasks, go to: https://github.com/sarnikowski/danish_transformers

## Usage

```python
from transformers import ConvBertTokenizer, ConvBertModel

tokenizer = ConvBertTokenizer.from_pretrained("sarnikowski/convbert-small-da-cased")
model = ConvBertModel.from_pretrained("sarnikowski/convbert-small-da-cased")
```

## Questions?

If you have any questions feel free to open an issue on the [danish_transformers](https://github.com/sarnikowski/danish_transformers) repository, or send an email to p.sarnikowski@gmail.com


---
language: da
license: cc-by-4.0
---

# Danish ELECTRA small (cased)

An [ELECTRA](https://arxiv.org/abs/2003.10555) model pretrained on a custom Danish corpus (~17.5gb).
For details regarding data sources and training procedure, along with benchmarks on downstream tasks, go to: https://github.com/sarnikowski/danish_transformers/tree/main/electra

## Usage

```python
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("sarnikowski/electra-small-generator-da-256-cased")
model = AutoModel.from_pretrained("sarnikowski/electra-small-generator-da-256-cased")
```

## Questions?

If you have any questions feel free to open an issue in the [danish_transformers](https://github.com/sarnikowski/danish_transformers) repository, or send an email to p.sarnikowski@gmail.com


# MASC
The model will be available after the paper is accepted.


# Steps to use this model
This model uses tokenizer 'rinna/japanese-roberta-base'. Therefore, below steps are critical to run the model correctly.

1. Create a local root directory on your system and new python environment.
2. Install below requirements

```
transformers==4.12.2
torch==1.10.0
numpy==1.21.3
pandas==1.3.4
sentencepiece==0.1.96
```
3. Go to link: "https://huggingface.co/spaces/shubh2014shiv/Japanese_NLP/tree/main" and download the fine tuned weights "reviewSentiments_jp.pt" in same local root directory.
4. Rename the downloaded weights as "reviewSentiments_jp.pt"
5. Use below code in the newly created environment.

```
from transformers import T5Tokenizer,BertForSequenceClassification
import torch
tokenizer = T5Tokenizer.from_pretrained('rinna/japanese-roberta-base')
japanese_review_text = "履きやすい。タイムセールで購入しました。見た目以上にカッコいいです。(^^)"
encoded_data = tokenizer.batch_encode_plus([japanese_review_text ],
                                                   add_special_tokens=True,
                                                   return_attention_mask=True,
                                                   padding=True,
                                                   max_length=200,
                                                   return_tensors='pt',
                                                   truncation=True)
input_ids = encoded_data['input_ids']
attention_masks = encoded_data['attention_mask']
model = BertForSequenceClassification.from_pretrained("shubh2014shiv/jp_review_sentiments_amzn",
                                                                      num_labels=2,
                                                                      output_attentions=False,
                                                                      output_hidden_states=False)
model.load_state_dict(torch.load('reviewSentiments_jp.pt',map_location=torch.device('cpu')))
inputs = { 'input_ids': input_ids,
              'attention_mask': attention_masks}
with torch.no_grad():
  outputs = model(**inputs)

logits = outputs.logits
logits = logits.detach().cpu().numpy()
scores = 1 / (1 + np.exp(-1 * logits))
result = {"TEXT (文章)": jp_review_text,'NEGATIVE (ネガティブ)': scores[0][0], 'POSITIVE (ポジティブ)': scores[0][1]}
```

Output:

{'TEXT (文章)': '履きやすい。タイムセールで購入しました。見た目以上にカッコいいです。(^^)', 'NEGATIVE (ネガティブ)': 0.023672901, 'POSITIVE (ポジティブ)': 0.96819043}


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model_index:
- name: distilbert-base-uncased-finetuned-cola
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: glue
      type: glue
      args: cola
    metric:
      name: Matthews Correlation
      type: matthews_correlation
      value: 0.5526838482765232
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-cola

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7691
- Matthews Correlation: 0.5527

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5247        | 1.0   | 535  | 0.5390          | 0.4315               |
| 0.353         | 2.0   | 1070 | 0.5273          | 0.4994               |
| 0.2386        | 3.0   | 1605 | 0.6391          | 0.5089               |
| 0.17          | 4.0   | 2140 | 0.7691          | 0.5527               |
| 0.1348        | 5.0   | 2675 | 0.8483          | 0.5472               |

### Framework versions

- Transformers 4.9.2
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


Image Preprocessing Model

    Image preprocessing in a convolutional model

    Read more about the model »

    View Code
    ·
    Report Bug
    ·
    Start a discussion

The main objective of this project is to apply preprocessing to an image dataset while the model is being trained.

The solution has been taken because we do not want to apply preprocessing to the data before training (i.e. create a copy of the data but already preprocessed) because we want to apply data augmentation while the model trains.

The use of `Lambda` layers has been discarded because they do not allow the use of external libraries that do not work with tensors, since we want to use the functions provided by *OpenCV* and *NumPy*.

## Preprocessing
In this example found in this repository we wanted to divide the images from HSV color masks, where it is divided into:
* **Warm zones**: red and white colors are obtained.
* **Warm zones**: The green color is obtained.
* **Cold zones**: The color blue is obtained.

Within the code you can find the declaration of these filters as:
```python
filters = {
	"original": lambda x: x,
	"red": lambda x: data.getImageTensor(x, (330, 0, 0), (360, 255, 255)) + data.getImageTensor(x, (0, 0, 0), (50, 255, 255)),
	"green": lambda x: data.getImageTensor(x, (60, 0, 0), (130, 255, 255)),
	"blue": lambda x: data.getImageTensor(x, (180, 0, 0), (270, 255, 255)),
}
```

On the other hand, the preprocessing functions are located inside `scripts/Data.py` file as follows:
```python
def detectColor(self, image, lower, upper):
	if tf.is_tensor(image):
		temp_image = image.numpy().copy() # Used for training
	else:
		temp_image = image.copy() # Used for displaying the image

	hsv_image = temp_image.copy()
	hsv_image = cv.cvtColor(hsv_image, cv.COLOR_RGB2HSV)
	mask = cv.inRange(hsv_image, lower, upper)

	result = temp_image.copy()
	result[np.where(mask == 0)] = 0

	return result

def getImageTensor(self, images, lower, upper):
	results = []

	for img in images:
		results.append(np.expand_dims(self.detectColor(img, lower, upper), axis=0))

	return np.concatenate(results, axis=0)
```

## Model
The model used to solve our problem was a *CNN* with a preprocessing layer:

![Model](./model.png "Model")

This model can be found in the `scripts/Model.py` file in the following function:
```python
def create_model():
	class FilterLayer(layers.Layer):
		def __init__(self, filter, **kwargs):
			self.filter = filter

			super(FilterLayer, self).__init__(name="filter_layer", **kwargs)

		def call(self, image):
			shape = image.shape
			[image, ] = tf.py_function(self.filter, [image], [tf.float32])
			image = backend.stop_gradient(image)
			image.set_shape(shape)

			return image

		def get_config(self):
			return super().get_config()

	model = models.Sequential()

	model.add(layers.Input(shape=(215, 538, 3)))
	model.add(FilterLayer(filter=self.filter))

	model.add(layers.Conv2D(32, (3, 3), activation="relu"))
	model.add(layers.MaxPooling2D(pool_size=(2, 2)))

	model.add(layers.Conv2D(32, (3, 3), activation="relu"))
	model.add(layers.GlobalAveragePooling2D())

	model.add(layers.Dropout(rate=0.4))
	model.add(layers.Dense(32, activation="relu"))
	model.add(layers.Dropout(rate=0.4))
	model.add(layers.Dense(2, activation="softmax"))

	return model
```

## Contributors
This work has been possible thanks to:
- [Fernando Pérez Lara](https://www.linkedin.com/in/fernandoperezlara/) ([**@FernandoPerezLara**](https://github.com/FernandoPerezLara)) for having developed the model to make this idea come true.

## License
Copyright (c) 2021 Fernando Pérez Lara.

Licensed and distributed under the [MIT](LICENSE.txt) license.


---
tags:
- generated_from_trainer
datasets:
- klue
metrics:
- accuracy
model-index:
- name: bert-base-finetuned-nli
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: klue
      type: klue
      args: nli
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.085
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-base-finetuned-nli

This model is a fine-tuned version of [klue/bert-base](https://huggingface.co/klue/bert-base) on the klue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6210
- Accuracy: 0.085

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.0   | 196  | 0.6210          | 0.085    |
| No log        | 2.0   | 392  | 0.5421          | 0.0643   |
| 0.5048        | 3.0   | 588  | 0.5523          | 0.062    |
| 0.5048        | 4.0   | 784  | 0.5769          | 0.0533   |
| 0.5048        | 5.0   | 980  | 0.5959          | 0.052    |

### Framework versions

- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


---
widget:
- text: "Ну ты и придурок!!"
---

NER Toxic models

Fine-tuning [cointegrated/rubert-tiny-toxicity](https://huggingface.co/cointegrated/rubert-tiny-toxicity) model on data from [toxic_dataset_ner](https://huggingface.co/datasets/tesemnikov-av/toxic_dataset_ner)

language: RU

```python
!pip install transformers > /dev/null

from transformers import (
    AutoModelForTokenClassification,
    AutoTokenizer,
    pipeline
)

model = AutoModelForTokenClassification.from_pretrained('tesemnikov-av/rubert-ner-toxicity')
tokenizer = AutoTokenizer.from_pretrained('tesemnikov-av/rubert-ner-toxicity')

pipe = pipeline(model=model, tokenizer=tokenizer, task='ner', aggregation_strategy='average')
text = "Они охриневшие там все придурки!!"

print(text)
print(pipe(text))
```


---
widget:
- text: "В город Сергиев Посад приехал Курт Кобейн."
---

Fine-tuning [cointegrated/rubert-tiny](https://huggingface.co/cointegrated/rubert-tiny) model on sentences from Wiki auto annotated with PER, LOC, ORG tags [corus/WiNER](https://pypi.org/project/corus/#reference)

language: RU

NER Class:
- PER
- LOC
- ORG

license: mit


---
tags:
- conversational
---

# IronStarkBot


---
tags:
- conversational
---

# Harry Potter DaibloGPT Model


---
license: mit
tags:
- generated_from_trainer
model_index:
- name: IFIS_ZORK_AI_FANTASY
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# IFIS_ZORK_AI_FANTASY

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unkown dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 3

### Training results

### Framework versions

- Transformers 4.8.2
- Pytorch 1.9.0+cu102
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model_index:
- name: Zork_AI_SciFi
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Zork_AI_SciFi

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unkown dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 3

### Training results

### Framework versions

- Transformers 4.8.2
- Pytorch 1.9.0+cu102
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model_index:
- name: IFIS_ZORK_AI_MODERN
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# IFIS_ZORK_AI_MODERN

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unkown dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 3

### Training results

### Framework versions

- Transformers 4.8.2
- Pytorch 1.9.0+cu102
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model_index:
- name: IFIS_ZORK_AI_SCIFI
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# IFIS_ZORK_AI_SCIFI

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unkown dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 3

### Training results

### Framework versions

- Transformers 4.8.2
- Pytorch 1.9.0+cu102
- Tokenizers 0.10.3


---
license: mit
tags:
- generated_from_trainer
model_index:
- name: IFIS_ZORK_AI_HORROR
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# IFIS_ZORK_AI_HORROR

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unkown dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 3

### Training results

### Framework versions

- Transformers 4.8.2
- Pytorch 1.9.0+cu102
- Tokenizers 0.10.3


   return im

def main():
    st.title("Lowlight Enhancement")
    st.write("This is a simple lowlight enhancement app with great performance and does not require paired images to train.")
    st.write("The model runs at 1000/11 FPS on single GPU/CPU on images with a size of 1200*900*3")
    uploaded_file = st.file_uploader("Lowlight Image")
    if uploaded_file:
        data_lowlight = Image.open(uploaded_file)
        col1, col2 = st.columns(2)
        col1.write("Original (Lowlight)")
        col1.image(data_lowlight, caption="Lowlight Image", use_column_width=True)


---
language:
- en
widget:
- text: theft 3
- text: forgery
- text: unlawful possession short-barreled shotgun
- text: criminal trespass 2nd degree
- text: eluding a police vehicle
- text: upcs synthetic narcotic
---

# ROTA
## Rapid Offense Text Autocoder

[![HuggingFace Models](https://img.shields.io/badge/%F0%9F%A4%97%20models-2021.05.18.15-blue)](https://huggingface.co/rti-international/rota)
[![GitHub Model Release](https://img.shields.io/github/v/release/RTIInternational/rota?logo=github)](https://github.com/RTIInternational/rota)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4770492.svg)](https://doi.org/10.5281/zenodo.4770492)

Criminal justice research often requires conversion of free-text offense descriptions into overall charge categories to aid analysis. For example, the free-text offense of "eluding a police vehicle" would be coded to a charge category of "Obstruction - Law Enforcement". Since free-text offense descriptions aren't standardized and often need to be categorized in large volumes, this can result in a manual and time intensive process for researchers. ROTA is a machine learning model for converting offense text into offense codes.

Currently ROTA predicts the *Charge Category* of a given offense text. A *charge category* is one of the headings for offense codes in the [2009 NCRP Codebook: Appendix F](https://www.icpsr.umich.edu/web/NACJD/studies/30799/datadocumentation#).

The model was trained on [publicly available data](https://web.archive.org/web/20201021001250/https://www.icpsr.umich.edu/web/pages/NACJD/guides/ncrp.html) from a crosswalk containing offenses from all 50 states combined with three additional hand-labeled offense text datasets.

    Charge Category Example

### Data Preprocessing

The input text is standardized through a series of preprocessing steps. The text is first passed through a sequence of 500+ case-insensitive regular expressions that identify common misspellings and abbreviations and expand the text to a more full, correct English text. Some data-specific prefixes and suffixes are then removed from the text -- e.g. some states included a statute as a part of the text. Finally, punctuation (excluding dollar signs) are removed from the input, multiple spaces between words are removed, and the text is lowercased.

## Cross-Validation Performance

This model was evaluated using 3-fold cross validation. Except where noted, numbers presented below are the mean value across the 3 folds.

The model in this repository is trained on all available data. Because of this, you can typically expect production performance to be (unknowably) better than the numbers presented below.

### Overall Metrics

| Metric   | Value |
| -------- | ----- |
| Accuracy | 0.934 |
| MCC      | 0.931 |

| Metric    | precision | recall | f1-score |
| --------- | --------- | ------ | -------- |
| macro avg | 0.811     | 0.786  | 0.794    |

*Note*: These are the average of the values *per fold*, so *macro avg* is the average of the macro average of all categories per fold.

### Per-Category Metrics

| Category                                               | precision | recall | f1-score | support |
| ------------------------------------------------------ | --------- | ------ | -------- | ------- |
| AGGRAVATED ASSAULT                                     | 0.954     | 0.954  | 0.954    | 4085    |
| ARMED ROBBERY                                          | 0.961     | 0.955  | 0.958    | 1021    |
| ARSON                                                  | 0.946     | 0.954  | 0.95     | 344     |
| ASSAULTING PUBLIC OFFICER                              | 0.914     | 0.905  | 0.909    | 588     |
| AUTO THEFT                                             | 0.962     | 0.962  | 0.962    | 1660    |
| BLACKMAIL/EXTORTION/INTIMIDATION                       | 0.872     | 0.871  | 0.872    | 627     |
| BRIBERY AND CONFLICT OF INTEREST                       | 0.784     | 0.796  | 0.79     | 216     |
| BURGLARY                                               | 0.979     | 0.981  | 0.98     | 2214    |
| CHILD ABUSE                                            | 0.805     | 0.78   | 0.792    | 139     |
| COCAINE OR CRACK VIOLATION OFFENSE UNSPECIFIED         | 0.827     | 0.815  | 0.821    | 47      |
| COMMERCIALIZED VICE                                    | 0.818     | 0.788  | 0.802    | 666     |
| CONTEMPT OF COURT                                      | 0.982     | 0.987  | 0.984    | 2952    |
| CONTRIBUTING TO DELINQUENCY OF A MINOR                 | 0.544     | 0.333  | 0.392    | 50      |
| CONTROLLED SUBSTANCE - OFFENSE UNSPECIFIED             | 0.864     | 0.791  | 0.826    | 280     |
| COUNTERFEITING (FEDERAL ONLY)                          | 0         | 0      | 0        | 2       |
| DESTRUCTION OF PROPERTY                                | 0.97      | 0.968  | 0.969    | 2560    |
| DRIVING UNDER INFLUENCE - DRUGS                        | 0.567     | 0.603  | 0.581    | 34      |
| DRIVING UNDER THE INFLUENCE                            | 0.951     | 0.946  | 0.949    | 2195    |
| DRIVING WHILE INTOXICATED                              | 0.986     | 0.981  | 0.984    | 2391    |
| DRUG OFFENSES - VIOLATION/DRUG UNSPECIFIED             | 0.903     | 0.911  | 0.907    | 3100    |
| DRUNKENNESS/VAGRANCY/DISORDERLY CONDUCT                | 0.856     | 0.861  | 0.858    | 380     |
| EMBEZZLEMENT                                           | 0.865     | 0.759  | 0.809    | 100     |
| EMBEZZLEMENT (FEDERAL ONLY)                            | 0         | 0      | 0        | 1       |
| ESCAPE FROM CUSTODY                                    | 0.988     | 0.991  | 0.989    | 4035    |
| FAMILY RELATED OFFENSES                                | 0.739     | 0.773  | 0.755    | 442     |
| FELONY - UNSPECIFIED                                   | 0.692     | 0.735  | 0.712    | 122     |
| FLIGHT TO AVOID PROSECUTION                            | 0.46      | 0.407  | 0.425    | 38      |
| FORCIBLE SODOMY                                        | 0.82      | 0.8    | 0.809    | 76      |
| FORGERY (FEDERAL ONLY)                                 | 0         | 0      | 0        | 2       |
| FORGERY/FRAUD                                          | 0.911     | 0.928  | 0.919    | 4687    |
| FRAUD (FEDERAL ONLY)                                   | 0         | 0      | 0        | 2       |
| GRAND LARCENY - THEFT OVER $200                        | 0.957     | 0.973  | 0.965    | 2412    |
| HABITUAL OFFENDER                                      | 0.742     | 0.627  | 0.679    | 53      |
| HEROIN VIOLATION - OFFENSE UNSPECIFIED                 | 0.879     | 0.811  | 0.843    | 24      |
| HIT AND RUN DRIVING                                    | 0.922     | 0.94   | 0.931    | 303     |
| HIT/RUN DRIVING - PROPERTY DAMAGE                      | 0.929     | 0.918  | 0.923    | 362     |
| IMMIGRATION VIOLATIONS                                 | 0.84      | 0.609  | 0.697    | 19      |
| INVASION OF PRIVACY                                    | 0.927     | 0.923  | 0.925    | 1235    |
| JUVENILE OFFENSES                                      | 0.928     | 0.866  | 0.895    | 144     |
| KIDNAPPING                                             | 0.937     | 0.93   | 0.933    | 553     |
| LARCENY/THEFT - VALUE UNKNOWN                          | 0.955     | 0.945  | 0.95     | 3175    |
| LEWD ACT WITH CHILDREN                                 | 0.775     | 0.85   | 0.811    | 596     |
| LIQUOR LAW VIOLATIONS                                  | 0.741     | 0.768  | 0.755    | 214     |
| MANSLAUGHTER - NON-VEHICULAR                           | 0.626     | 0.802  | 0.701    | 139     |
| MANSLAUGHTER - VEHICULAR                               | 0.79      | 0.853  | 0.819    | 117     |
| MARIJUANA/HASHISH VIOLATION - OFFENSE UNSPECIFIED      | 0.741     | 0.662  | 0.699    | 62      |
| MISDEMEANOR UNSPECIFIED                                | 0.63      | 0.243  | 0.347    | 57      |
| MORALS/DECENCY - OFFENSE                               | 0.774     | 0.764  | 0.769    | 412     |
| MURDER                                                 | 0.965     | 0.915  | 0.939    | 621     |
| OBSTRUCTION - LAW ENFORCEMENT                          | 0.939     | 0.947  | 0.943    | 4220    |
| OFFENSES AGAINST COURTS, LEGISLATURES, AND COMMISSIONS | 0.881     | 0.895  | 0.888    | 1965    |
| PAROLE VIOLATION                                       | 0.97      | 0.953  | 0.962    | 946     |
| PETTY LARCENY - THEFT UNDER $200                       | 0.965     | 0.761  | 0.85     | 139     |
| POSSESSION/USE - COCAINE OR CRACK                      | 0.893     | 0.928  | 0.908    | 68      |
| POSSESSION/USE - DRUG UNSPECIFIED                      | 0.624     | 0.535  | 0.572    | 189     |
| POSSESSION/USE - HEROIN                                | 0.884     | 0.852  | 0.866    | 25      |
| POSSESSION/USE - MARIJUANA/HASHISH                     | 0.977     | 0.97   | 0.973    | 556     |
| POSSESSION/USE - OTHER CONTROLLED SUBSTANCES           | 0.975     | 0.965  | 0.97     | 3271    |
| PROBATION VIOLATION                                    | 0.963     | 0.953  | 0.958    | 1158    |
| PROPERTY OFFENSES - OTHER                              | 0.901     | 0.87   | 0.885    | 446     |
| PUBLIC ORDER OFFENSES - OTHER                          | 0.7       | 0.721  | 0.71     | 1871    |
| RACKETEERING/EXTORTION (FEDERAL ONLY)                  | 0         | 0      | 0        | 2       |
| RAPE - FORCE                                           | 0.842     | 0.873  | 0.857    | 641     |
| RAPE - STATUTORY - NO FORCE                            | 0.707     | 0.55   | 0.611    | 140     |
| REGULATORY OFFENSES (FEDERAL ONLY)                     | 0.847     | 0.567  | 0.674    | 70      |
| RIOTING                                                | 0.784     | 0.605  | 0.68     | 119     |
| SEXUAL ASSAULT - OTHER                                 | 0.836     | 0.836  | 0.836    | 971     |
| SIMPLE ASSAULT                                         | 0.976     | 0.967  | 0.972    | 4577    |
| STOLEN PROPERTY - RECEIVING                            | 0.959     | 0.957  | 0.958    | 1193    |
| STOLEN PROPERTY - TRAFFICKING                          | 0.902     | 0.888  | 0.895    | 491     |
| TAX LAW (FEDERAL ONLY)                                 | 0.373     | 0.233  | 0.286    | 30      |
| TRAFFIC OFFENSES - MINOR                               | 0.974     | 0.977  | 0.976    | 8699    |
| TRAFFICKING - COCAINE OR CRACK                         | 0.896     | 0.951  | 0.922    | 185     |
| TRAFFICKING - DRUG UNSPECIFIED                         | 0.709     | 0.795  | 0.749    | 516     |
| TRAFFICKING - HEROIN                                   | 0.871     | 0.92   | 0.894    | 54      |
| TRAFFICKING - OTHER CONTROLLED SUBSTANCES              | 0.963     | 0.954  | 0.959    | 2832    |
| TRAFFICKING MARIJUANA/HASHISH                          | 0.921     | 0.943  | 0.932    | 255     |
| TRESPASSING                                            | 0.974     | 0.98   | 0.977    | 1916    |
| UNARMED ROBBERY                                        | 0.941     | 0.939  | 0.94     | 377     |
| UNAUTHORIZED USE OF VEHICLE                            | 0.94      | 0.908  | 0.924    | 304     |
| UNSPECIFIED HOMICIDE                                   | 0.61      | 0.554  | 0.577    | 60      |
| VIOLENT OFFENSES - OTHER                               | 0.827     | 0.817  | 0.822    | 606     |
| VOLUNTARY/NONNEGLIGENT MANSLAUGHTER                    | 0.619     | 0.513  | 0.542    | 54      |
| WEAPON OFFENSE                                         | 0.943     | 0.949  | 0.946    | 2466    |

*Note: `support` is the average number of observations predicted on per fold, so the total number of observations per class is roughly 3x `support`.*

### Using Confidence Scores

If we interpret the classification probability as a confidence score, we can use it to filter out predictions that the model isn't as confident about. We applied this process in 3-fold cross validation. The numbers presented below indicate how much of the prediction data is retained given a confidence score cutoff of `p`. We present the overall accuracy and MCC metrics as if the model was only evaluated on this subset of confident predictions.

|     | cutoff | percent retained | mcc   | acc   |
| --- | ------ | ---------------- | ----- | ----- |
| 0   | 0.85   | 0.952            | 0.96  | 0.961 |
| 1   | 0.9    | 0.943            | 0.964 | 0.965 |
| 2   | 0.95   | 0.928            | 0.97  | 0.971 |
| 3   | 0.975  | 0.912            | 0.975 | 0.976 |
| 4   | 0.99   | 0.886            | 0.982 | 0.983 |
| 5   | 0.999  | 0.733            | 0.995 | 0.996 |


---
language: ar
widget:
 - text: "للوقايه من عدم انتشار [MASK]"
---
# arabert_c19: An Arabert model pretrained on 1.5 million COVID-19 multi-dialect Arabic tweets
**ARABERT COVID-19** is a pretrained (fine-tuned) version of the AraBERT v2 model (https://huggingface.co/aubmindlab/bert-base-arabertv02). The pretraining was done using 1.5 million multi-dialect Arabic tweets regarding the COVID-19 pandemic from the “Large Arabic Twitter Dataset on COVID-19” (https://arxiv.org/abs/2004.04315).
The model can achieve better results for the tasks that deal with multi-dialect Arabic tweets in relation to the COVID-19 pandemic.

# Classification results for multiple tasks including fake-news and hate speech detection when using arabert_c19 and mbert_ar_c19:
For more details refer to the paper (link)

|                                    | arabert  | mbert    | distilbert multi | arabert Covid-19 | mbert Covid-19 |
|------------------------------------|----------|----------|------------------|------------------|----------------|
| Contains hate (Binary)             |   0.8346 |   0.6675 |   0.7145         |   `0.8649`         |   0.8492       |
| Talk about a cure (Binary)         |   0.8193 |   0.7406 |   0.7127         |   0.9055         |   `0.9176`       |
| News or opinion (Binary)           |   0.8987 |   0.8332 |   0.8099         |   `0.9163`         |   0.9116       |
| Contains fake information (Binary) |   0.6415 |   0.5428 |   0.4743         |   `0.7739`         |   0.7228       |

# Preprocessing

```python
from arabert.preprocess import ArabertPreprocessor
model_name="moha/arabert_c19"
arabert_prep = ArabertPreprocessor(model_name=model_name)
text = "للوقايه من عدم انتشار كورونا عليك اولا غسل اليدين بالماء والصابون وتكون عملية الغسل دقيقه تشمل راحة اليد الأصابع التركيز على الإبهام"
arabert_prep.preprocess(text)
```

# Contacts
**Hadj Ameur**: [Github](https://github.com/MohamedHadjAmeur) | <mohamedhadjameur@gmail.com> | <mhadjameur@cerist.dz>


---
language:
- zh
tags:
- bert
license: "apache-2.0"
---

# Please use 'Bert' related functions to load this model!
## Chinese BERT with Whole Word Masking Fix MLM Parameters

Init parameters by https://huggingface.co/hfl/chinese-roberta-wwm-ext-large

miss mlm parameters issue https://github.com/ymcui/Chinese-BERT-wwm/issues/98

Only train MLM parameters and freeze other parameters

More info in github https://github.com/genggui001/chinese_roberta_wwm_large_ext_fix_mlm


---
tags:
- conversational
---
#my awesome model


---
tags:
- conversational
---
# My Awesome Model


BreitBot

Timothy W. Dooley
___________________________________________________
GitHub
The GitHub for the project can be found [here](https://github.com/twdooley/election_news)
Model

This model was trained on about 16,000 headlines from Breitbart.com spannning March 2019- 11 November 2020. The purpose of this project was to better understand how strongly polarized news crafts a narrative through Natural Language Processing. The BreitBot model was specifically created to understand the 'clickbaity' nature of a Breitbart headline. Many of the results are 'reasonable' within the scope of Breitbart's production. I will leave it to the user to make further interpretation. The full project noted that over 70% of Breitbart's articles from month to month have a negative sentiment score. Subjectively, I believe this is shown through the headlines generated.

Training

BreitBot is a finetuned on GPT2 with about 16,000 headlines. The maximum length allowed in the tokenizer was the length of the longest headline (~50 tokens). A huge credit goes to Richard Bownes, PhD whose article ["Fine Tuning GPT-2 for Magic the Gathering Flavour Text Generation"](https://medium.com/swlh/fine-tuning-gpt-2-for-magic-the-gathering-flavour-text-generation-3bafd0f9bb93) provided incredible direction and help in training this model. It was trained using a GPU on Google Colab.


Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large


I am adding my first README in order to test the interface. How good is it really?


---
language:
- ab
tags:
- automatic-speech-recognition
- common_voice
- generated_from_trainer
datasets:
- common_voice
model-index:
- name: ''
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

#

This model is a fine-tuned version of [hf-test/xls-r-dummy](https://huggingface.co/hf-test/xls-r-dummy) on the COMMON_VOICE - AB dataset.
It achieves the following results on the evaluation set:
- Loss: 133.5167
- Wer: 18.9286

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 2.0
- mixed_precision_training: Native AMP

### Training results

### Framework versions

- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: wav2vec2-large-xls-r-300m-guarani-colab
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-large-xls-r-300m-guarani-colab

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.2392
- Wer: 1.0743

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 18.2131       | 49.94 | 400  | 3.2901          | 1.0    |
| 2.0496        | 99.94 | 800  | 3.2392          | 1.0743 |

### Framework versions

- Transformers 4.16.0.dev0
- Pytorch 1.10.1+cu102
- Datasets 1.17.1.dev0
- Tokenizers 0.11.0


---
language: en
datasets:
- cornell_movie_dialog
license: gpl-3.0
tags:
  - NLP
  - ChatBot
  - Game AI
metrics:
  - rouge
widget:
- text: "personality: Hinata was soft-spoken and polite, always addressing people with proper honorifics. She is kind, always thinking of others more than for herself, caring for their feelings and well-being. She doesn't like being confrontational for any reason. This led to her being meek or timid to others, as her overwhelming kindness can render her unable to respond or act for fear of offending somebody.</s> inquiry: What's your name?"
  example_title: "Talk to Hinata"
- text: "personality: Voldemort is a raging psychopath, devoid of the normal human responses to other people's suffering. He has no conscience, feels no remorse or empathy, and does not recognize the worth and humanity of anybody except himself.</s> inquiry: What's your name?"
  example_title: "Talk to Voldemort"
inference:
  parameters:
    num_beams: 6
    diversity_penalty: 2.5
    num_beam_groups: 2
---
# FreeIsland AI

With the advancement of the graphical processing power of computers and sophisticated algorithms like [Nanite](https://docs.unrealengine.com/5.0/en-US/RenderingFeatures/Nanite/), simulating lifelike sceneries in real-time is never being easier. About a month ago Epic Games [showoff](https://www.youtube.com/watch?v=WU0gvPcc3jQ) the newest capabilities of their newest game engine by simulating an entire city including population, traffic, weather, etc running on a Playstore 5. That made me think what are the things missing from that simulation and how can I use my skills to improve it.

One of the main missing components that separate our world and the simulated world is people. More importantly, the interactivity of people in simulated worlds. Last year a game called cyberpunk got released and it had an option to [talk to any person](https://www.youtube.com/watch?v=Z1OtYGzUoSo) in its city but the problem with that was all the responses from the Non-player Characters (NPCs) are hardcoded which greatly reduce the immersion of the game.

So the goal of this project is to experiment with how the advancement of Natural Language Processing makes NPCs from video games interactive and enhances immersion in video games.

# Usage
```py
from transformers import AutoModelForSeq2SeqLM

trained_model = AutoModelForSeq2SeqLM.from_pretrained(f"Supiri/t5-base-conversation")

prompt = "What's your name?"

context = "Hinata was soft-spoken and polite, always addressing people with proper honorifics. She is kind, always thinking of others more than for herself, caring for their feelings and well-being. She doesn't like being confrontational for any reason. This led to her being meek or timid to others, as her overwhelming kindness can render her unable to respond or act for fear of offending somebody."

input_ids = tokenizer(f"personality: {context}", f"inquiry: {prompt}", return_tensors='pt').input_ids
outputs = trained_model.generate(input_ids, num_beams=6, diversity_penalty=2.5, num_beam_groups=2)

print("Answer:\t", tokenizer.decode(outputs[0], skip_special_tokens=True))

# Answer: My name is Hinata
```

# Evaluation

## Test 1
For this test, I sampled input from the test dataset. For this question the actual response is

> "It works a little."

But models' response was

> "I don't want to flirt with you."

Which reflect its bio which was filled by GPT-3

> "He stands primarily to gain self-esteem, which he often receives through the submission of others"

In gist, Dr. Greenbaum tried to tease Sebastian about his seductive traits but this model's go-to response was to shut her down since the biography of Sebastian states he often tries to assert his dominance over others.

```py
prompt = dataset['test'][66]['request']
contexts = dataset['test'][66]['bio']

input_ids = tokenizer(f"personality: {contexts}", f"inquiry: {prompt}", return_tensors='pt').input_ids
outputs = trained_model.generate(input_ids, num_beams=6, diversity_penalty=5.0, num_beam_groups=2)

print("Input to the Model")
print("Bio:\t",contexts)
print("\nPrompt:\t", prompt)

print("\nGround truth response")
print("\t", dataset['test'][66]['response'])

print("\nModel's Prediction")
print("Answer:\t", tokenizer.decode(outputs[0], skip_special_tokens=True))

```

```txt
Input to the Model
Bio:	 Sebastian is a very extreme representation of the trope of the "Confidence Man", and acts it out to a degree that is sometimes comedic but mostly frightening. He stands primarily to gain self-esteem, which he often receives through the submission of others or solely through his own perceptions. An artful seducer, his incredible charisma is both his greatest weapon and most intoxicating weakness.

Prompt:	 You think you can come in here with that cute little smirk on your face and try and flirt with me. It doesn't work, Sebastian.

Ground truth response
	 It works a little.

Model's Prediction
Answer:	 I don't want to flirt with you.
```

### Test 2

Hinata is a kind-hearted girl from the anime series Naruto. I took her bio from [personality database](https://www.personality-database.com/profile/2790/hinata-hyga-naruto-shippden-mbti-personality-type) and ask a few questions about her.

Off the top, you can see the model understands the context since when I asked the model, "**What's your name?**" it responded with the name given with the context.

Also, notice when prompted the same question differently (**"Who are you?"**), it still manages to answer it well.

```py
prompts = ["What's your name?", "How are you feeling?", "Do you like Star Wars?", "Who are you?", "Coffee or tea?"]

contexts = "Hinata was soft-spoken and polite, always addressing people with proper honorifics. She is kind, always thinking of others more than for herself, caring for their feelings and well-being. She doesn't like being confrontational for any reason. This led to her being meek or timid to others, as her overwhelming kindness can render her unable to respond or act for fear of offending somebody."

print("Bio:\t",contexts, "\n")

for prompt in prompts:
    input_ids = tokenizer(f"personality: {contexts}", f"inquiry: {prompt}", return_tensors='pt').input_ids
    outputs = trained_model.generate(input_ids, num_beams=6, diversity_penalty=5.0, num_beam_groups=2)
    print("Prompt:\t", prompt)
    print("Answer:\t", tokenizer.decode(outputs[0], skip_special_tokens=True), "\n")
```

```txt
Bio:	 Hinata was soft-spoken and polite, always addressing people with proper honorifics. She is kind, always thinking of others more than for herself, caring for their feelings and well-being. She doesn't like being confrontational for any reason. This led to her being meek or timid to others, as her overwhelming kindness can render her unable to respond or act for fear of offending somebody.

Prompt:	 What's your name?
Answer:	 My name is Hinata

Prompt:	 How are you feeling?
Answer:	 I'm fine.

Prompt:	 Do you like Star Wars?
Answer:	 No, I don't.

Prompt:	 Who are you?
Answer:	 My name is Hinata

Prompt:	 Coffee or tea?
Answer:	 No, I don't drink much.
```

# Conclusion

After training the `t5-base` model for 5 epochs, the model started getting adapted to the dataset but there are a lot more improvements that can be done.

1. During the dataset creation part I had to limit the dataset size to 200 unique characters out of 9,035 that's present in the dataset due to the **budget constraints**. So If I manage to cover at least half of the dataset this model will have come up with far better responses.
2. Both input size and batch size were severely constrained due to the lack of access to GPU memory. Having the batch size of 64 is in contrast to 8 would have massive improvements in both training time and **generalization of model**.
3. Using a bigger model like `t5-large` or `t5-3b` will certainly improve the performance.
4. One of the main downsides to using this pre-trained model is this model was trained in German, French, and Romanian. Which consumed a chunk of the **vocabulary size and trainable parameters**. Retraining this model from scratch will help to reduce both needed parameter count and training loss when it comes to this specific task.


# Helsinki-NLP/opus-mt-en-vi
- This model is a fine-tune checkpoint of [Helsinki-NLP/opus-mt-en-vi](https://huggingface.co/Helsinki-NLP/opus-mt-en-vi).
- This model reaches BLEU score = 33.086 on the test set of IWSLT'15 English-Vietnamese data.

# Fine-tuning hyper-parameters
- learning_rate = 1e-4
- batch_size = 4
- num_train_epochs = 3.0


# HebEMO - Emotion Recognition Model for Modern Hebrew

HebEMO is a tool that detects polarity and extracts emotions from modern Hebrew User-Generated Content (UGC), which was trained on a unique Covid-19 related dataset that we collected and annotated.

HebEMO yielded a high performance of weighted average F1-score = 0.96 for polarity classification.
Emotion detection reached an F1-score of 0.78-0.97, with the exception of *surprise*, which the model failed to capture (F1 = 0.41). These results are better than the best-reported performance, even when compared to the English language.

## Emotion UGC Data Description
Our UGC data includes comments posted on news articles collected from 3 major Israeli news sites, between January 2020 to August 2020. The total size of the data is ~150 MB, including over 7 million words and 350K sentences.
~2000 sentences were annotated by crowd members (3-10 annotators per sentence) for overall sentiment (polarity) and [eight emotions](https://en.wikipedia.org/wiki/Robert_Plutchik#Plutchik's_wheel_of_emotions): anger, disgust, anticipation , fear, joy, sadness, surprise and trust.
The percentage of sentences in which each emotion appeared is found in the table below.

|       | anger | disgust | expectation | fear | happy | sadness | surprise | trust | sentiment |
|------:|------:|--------:|------------:|-----:|------:|--------:|---------:|------:|-----------|
| **ratio** |  0.78 |    0.83 |        0.58 | 0.45 |  0.12 |    0.59 |     0.17 |  0.11 | 0.25      |

## Performance
### Emotion Recognition
| emotion     | f1-score | precision | recall   |
|-------------|----------|-----------|----------|
|       anger | 0.96 |  0.99 | 0.93 |
|     disgust | 0.97 |  0.98 | 0.96 |
|anticipation | 0.82 |  0.80 | 0.87 |
|        fear | 0.79 |  0.88 | 0.72 |
|       joy   | 0.90 |  0.97 | 0.84 |
|     sadness | 0.90 |  0.86 | 0.94 |
|    surprise | 0.40 |  0.44 | 0.37 |
|       trust | 0.83 |  0.86 | 0.80 |

*The above metrics is for positive class (meaning, the emotion is reflected in the text).*

### Sentiment (Polarity) Analysis
|              | precision | recall | f1-score |
|--------------|-----------|--------|----------|
| neutral      | 0.83      | 0.56   | 0.67     |
| positive     | 0.96      | 0.92   | 0.94     |
| negative     | 0.97      | 0.99   | 0.98     |
| accuracy     |           |        | 0.97     |
| macro avg    | 0.92      | 0.82   | 0.86     |
| weighted avg | 0.96      | 0.97   | 0.96     |

*Sentiment (polarity) analysis model is also available on AWS! for more information visit [AWS' git](https://github.com/aws-samples/aws-lambda-docker-serverless-inference/tree/main/hebert-sentiment-analysis-inference-docker-lambda)*

## How to use

### Emotion Recognition Model
An online model can be found at [huggingface spaces](https://huggingface.co/spaces/avichr/HebEMO_demo) or as [colab notebook](https://colab.research.google.com/drive/1Jw3gOWjwVMcZslu-ttXoNeD17lms1-ff?usp=sharing)
```
# !pip install pyplutchik==0.0.7
# !pip install transformers==4.14.1

!git clone https://github.com/avichaychriqui/HeBERT.git
from HeBERT.src.HebEMO import *
HebEMO_model = HebEMO()

HebEMO_model.hebemo(input_path = 'data/text_example.txt')
# return analyzed pandas.DataFrame

hebEMO_df = HebEMO_model.hebemo(text='החיים יפים ומאושרים', plot=True)
```

### For sentiment classification model (polarity ONLY):
	from transformers import AutoTokenizer, AutoModel, pipeline

	tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
	model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")

	# how to use?
	sentiment_analysis = pipeline(
	    "sentiment-analysis",
	    model="avichr/heBERT_sentiment_analysis",
	    tokenizer="avichr/heBERT_sentiment_analysis",
	    return_all_scores = True
	)

	sentiment_analysis('אני מתלבט מה לאכול לארוחת צהריים')
	>>>  [[{'label': 'neutral', 'score': 0.9978172183036804},
	>>>  {'label': 'positive', 'score': 0.0014792329166084528},
	>>>  {'label': 'negative', 'score': 0.0007035882445052266}]]

	sentiment_analysis('קפה זה טעים')
	>>>  [[{'label': 'neutral', 'score': 0.00047328314394690096},
	>>>  {'label': 'possitive', 'score': 0.9994067549705505},
	>>>  {'label': 'negetive', 'score': 0.00011996887042187154}]]

	sentiment_analysis('אני לא אוהב את העולם')
	>>>  [[{'label': 'neutral', 'score': 9.214012970915064e-05},
	>>>  {'label': 'possitive', 'score': 8.876807987689972e-05},
	>>>  {'label': 'negetive', 'score': 0.9998190999031067}]]

## Contact us
[Avichay Chriqui](mailto:avichayc@mail.tau.ac.il)

[Inbal yahav](mailto:inbalyahav@tauex.tau.ac.il)

The Coller Semitic Languages AI Lab

Thank you, תודה, شكرا

## If you used this model please cite us as :
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
```
@article{chriqui2021hebert,
  title={HeBERT \& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
  author={Chriqui, Avihay and Yahav, Inbal},
  journal={arXiv preprint arXiv:2102.01909},
  year={2021}
}
```


# HeBERT: Pre-trained BERT for Polarity Analysis and Emotion Recognition

HeBERT is a Hebrew pretrained language model. It is based on [Google's BERT](https://arxiv.org/abs/1810.04805) architecture and it is BERT-Base config.

HeBert was trained on three dataset:
1. A Hebrew version of [OSCAR](https://oscar-corpus.com/): ~9.8 GB of data, including 1 billion words and over 20.8 millions sentences.
2. A Hebrew dump of [Wikipedia](https://dumps.wikimedia.org/): ~650 MB of data, including over 63 millions words and 3.8 millions sentences
3. Emotion User Generated Content (UGC) data that was collected for the purpose of this study (described below).

## Named-entity recognition (NER)
The ability of the model to classify named entities in text, such as persons' names, organizations, and locations; tested on a labeled dataset from [Ben Mordecai and M Elhadad (2005)](https://www.cs.bgu.ac.il/~elhadad/nlpproj/naama/), and evaluated with F1-score.

### How to use
```
	from transformers import pipeline

	# how to use?
	NER = pipeline(
	    "token-classification",
	    model="avichr/heBERT_NER",
	    tokenizer="avichr/heBERT_NER",
	)
	NER('דויד לומד באוניברסיטה העברית שבירושלים')
```

## Other tasks
[**Emotion Recognition Model**](https://huggingface.co/avichr/hebEMO_trust).
An online model can be found at [huggingface spaces](https://huggingface.co/spaces/avichr/HebEMO_demo) or as [colab notebook](https://colab.research.google.com/drive/1Jw3gOWjwVMcZslu-ttXoNeD17lms1-ff?usp=sharing)

[**Sentiment Analysis**](https://huggingface.co/avichr/heBERT_sentiment_analysis).

[**masked-LM model**](https://huggingface.co/avichr/heBERT) (can be fine-tunned to any down-stream task).

## Contact us
[Avichay Chriqui](mailto:avichayc@mail.tau.ac.il)

[Inbal yahav](mailto:inbalyahav@tauex.tau.ac.il)

The Coller Semitic Languages AI Lab

Thank you, תודה, شكرا

## If you used this model please cite us as :
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
```
@article{chriqui2021hebert,
  title={HeBERT \& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
  author={Chriqui, Avihay and Yahav, Inbal},
  journal={arXiv preprint arXiv:2102.01909},
  year={2021}
}
```
[git](https://github.com/avichaychriqui/HeBERT)


## HeBERT: Pre-trained BERT for Polarity Analysis and Emotion Recognition
HeBERT is a Hebrew pre-trained language model. It is based on Google's BERT architecture and it is BERT-Base config [(Devlin et al. 2018)](https://arxiv.org/abs/1810.04805).

HeBert was trained on three datasets:
1. A Hebrew version of OSCAR [(Ortiz, 2019)](https://oscar-corpus.com/): ~9.8 GB of data, including 1 billion words and over 20.8 million sentences.
2. A Hebrew dump of Wikipedia: ~650 MB of data, including over 63 million words and 3.8 million sentences
3. Emotion UGC data was collected for the purpose of this study. (described below)
We evaluated the model on emotion recognition and sentiment analysis, for downstream tasks.

### Emotion UGC Data Description
Our User-Generated Content (UGC) is comments written on articles collected from 3 major news sites, between January 2020 to August 2020, Total data size of ~150 MB of data, including over 7 million words and 350K sentences.
4000 sentences annotated by crowd members (3-10 annotators per sentence) for 8 emotions (anger, disgust, expectation, fear, happy, sadness, surprise, and trust) and overall sentiment/polarity

In order to validate the annotation, we search for an agreement between raters to emotion in each sentence using Krippendorff's alpha [(krippendorff, 1970)](https://journals.sagepub.com/doi/pdf/10.1177/001316447003000105). We left sentences that got alpha > 0.7. Note that while we found a general agreement between raters about emotions like happiness, trust, and disgust, there are few emotions with general disagreement about them, apparently given the complexity of finding them in the text (e.g. expectation and surprise).

### Performance
#### sentiment analysis

|              | precision | recall | f1-score |
|--------------|-----------|--------|----------|
| natural      | 0.83      | 0.56   | 0.67     |
| positive     | 0.96      | 0.92   | 0.94     |
| negative     | 0.97      | 0.99   | 0.98     |
| accuracy     |           |        | 0.97     |
| macro avg    | 0.92      | 0.82   | 0.86     |
| weighted avg | 0.96      | 0.97   | 0.96     |

## How to use
### For masked-LM model (can be fine-tunned to any down-stream task)
```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT")
model = AutoModel.from_pretrained("avichr/heBERT")

from transformers import pipeline
fill_mask = pipeline(
    "fill-mask",
    model="avichr/heBERT",
    tokenizer="avichr/heBERT"
)
fill_mask("הקורונה לקחה את [MASK] ולנו לא נשאר דבר.")
```

### For sentiment classification model (polarity ONLY):
```
from transformers import AutoTokenizer, AutoModel, pipeline
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")

# how to use?
sentiment_analysis = pipeline(
    "sentiment-analysis",
    model="avichr/heBERT_sentiment_analysis",
    tokenizer="avichr/heBERT_sentiment_analysis",
    return_all_scores = True
)

>>>  sentiment_analysis('אני מתלבט מה לאכול לארוחת צהריים')
[[{'label': 'natural', 'score': 0.9978172183036804},
{'label': 'positive', 'score': 0.0014792329166084528},
{'label': 'negative', 'score': 0.0007035882445052266}]]

>>>  sentiment_analysis('קפה זה טעים')
[[{'label': 'natural', 'score': 0.00047328314394690096},
{'label': 'possitive', 'score': 0.9994067549705505},
{'label': 'negetive', 'score': 0.00011996887042187154}]]

>>>  sentiment_analysis('אני לא אוהב את העולם')
[[{'label': 'natural', 'score': 9.214012970915064e-05},
{'label': 'possitive', 'score': 8.876807987689972e-05},
{'label': 'negetive', 'score': 0.9998190999031067}]]
```

Our model is also available on AWS! for more information visit [AWS' git](https://github.com/aws-samples/aws-lambda-docker-serverless-inference/tree/main/hebert-sentiment-analysis-inference-docker-lambda)

## Stay tuned!
We are still working on our model and will edit this page as we progress.

Note that we have released only sentiment analysis (polarity) at this point, emotion detection will be released later on.

our git: https://github.com/avichaychriqui/HeBERT

## If you used this model please cite us as :
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
```
@article{chriqui2021hebert,
  title={HeBERT \\\\\\\\\\\\\\\\& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
  author={Chriqui, Avihay and Yahav, Inbal},
  journal={arXiv preprint arXiv:2102.01909},
  year={2021}
}
```


## HeBERT: Pre-trained BERT for Polarity Analysis and Emotion Recognition
HeBERT is a Hebrew pretrained language model. It is based on Google's BERT architecture and it is BERT-Base config [(Devlin et al. 2018)](https://arxiv.org/abs/1810.04805).

### HeBert was trained on three dataset:
1. A Hebrew version of OSCAR [(Ortiz, 2019)](https://oscar-corpus.com/): ~9.8 GB of data, including 1 billion words and over 20.8 millions sentences.
2. A Hebrew dump of [Wikipedia](https://dumps.wikimedia.org/hewiki/latest/): ~650 MB of data, including over 63 millions words and 3.8 millions sentences
3. Emotion UGC data that was collected for the purpose of this study. (described below)
We evaluated the model on emotion recognition and sentiment analysis, for a downstream tasks.

### Emotion UGC Data Description
Our User Genrated Content (UGC) is comments written on articles collected from 3 major news sites, between January 2020 to August 2020,. Total data size ~150 MB of data, including over 7 millions words and 350K sentences.
4000 sentences annotated by crowd members (3-10 annotators per sentence) for 8 emotions (anger, disgust, expectation , fear, happy, sadness, surprise and trust) and overall sentiment / polarity

In order to valid the annotation, we search an agreement between raters to emotion in each sentence using krippendorff's alpha [(krippendorff, 1970)](https://journals.sagepub.com/doi/pdf/10.1177/001316447003000105). We left sentences that got alpha > 0.7. Note that while we found a general agreement between raters about emotion like happy, trust and disgust, there are few emotion with general disagreement about them, apparently given the complexity of finding them in the text (e.g. expectation and surprise).
## How to use
### For masked-LM model (can be fine-tunned to any down-stream task)
```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT")
model = AutoModel.from_pretrained("avichr/heBERT")

from transformers import pipeline
fill_mask = pipeline(
    "fill-mask",
    model="avichr/heBERT",
    tokenizer="avichr/heBERT"
)
fill_mask("הקורונה לקחה את [MASK] ולנו לא נשאר דבר.")
```

### For sentiment classification model (polarity ONLY):
```
from transformers import AutoTokenizer, AutoModel, pipeline
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")

# how to use?
sentiment_analysis = pipeline(
    "sentiment-analysis",
    model="avichr/heBERT_sentiment_analysis",
    tokenizer="avichr/heBERT_sentiment_analysis",
    return_all_scores = True
)

>>>  sentiment_analysis('אני מתלבט מה לאכול לארוחת צהריים')
[[{'label': 'natural', 'score': 0.9978172183036804},
{'label': 'positive', 'score': 0.0014792329166084528},
{'label': 'negative', 'score': 0.0007035882445052266}]]

>>>  sentiment_analysis('קפה זה טעים')
[[{'label': 'natural', 'score': 0.00047328314394690096},
{'label': 'possitive', 'score': 0.9994067549705505},
{'label': 'negetive', 'score': 0.00011996887042187154}]]

>>>  sentiment_analysis('אני לא אוהב את העולם')
[[{'label': 'natural', 'score': 9.214012970915064e-05},
{'label': 'possitive', 'score': 8.876807987689972e-05},
{'label': 'negetive', 'score': 0.9998190999031067}]]
```
Our model is also available on AWS! for more information visit [AWS' git](https://github.com/aws-samples/aws-lambda-docker-serverless-inference/tree/main/hebert-sentiment-analysis-inference-docker-lambda)

### For NER model:

```
	from transformers import pipeline

	# how to use?
	NER = pipeline(
	    "token-classification",
	    model="avichr/heBERT_NER",
	    tokenizer="avichr/heBERT_NER",
	)
	NER('דויד לומד באוניברסיטה העברית שבירושלים')
```

## Stay tuned!
We are still working on our model and will edit this page as we progress.

Note that we have released only sentiment analysis (polarity) at this point, emotion detection will be released later on.

our git: https://github.com/avichaychriqui/HeBERT

## If you use this model please cite us as :
Chriqui, A., & Yahav, I. (2021). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. arXiv preprint arXiv:2102.01909.
```
@article{chriqui2021hebert,
  title={HeBERT \\& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
  author={Chriqui, Avihay and Yahav, Inbal},
  journal={arXiv preprint arXiv:2102.01909},
  year={2021}
}
```