---
tags:
- flair
- token-classification
- sequence-tagger-model
language: fr
datasets:
- conll2003
widget:
- text: "George Washington est allé à Washington"
---

## French NER in Flair (default model)

This is the standard 4-class NER model for French that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **90,61** (WikiNER)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-french")

# make example sentence
sentence = Sentence("George Washington est allé à Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (0.7394)]
Span [6]: "Washington"   [− Labels: LOC (0.9161)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington est allé à Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import WIKINER_FRENCH
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = WIKINER_FRENCH()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # GloVe embeddings
    WordEmbeddings('fr'),

    # contextual string embeddings, forward
    FlairEmbeddings('fr-forward'),

    # contextual string embeddings, backward
    FlairEmbeddings('fr-backward'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-french',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: de
datasets:
- conll2003
widget:
- text: "George Washington ging nach Washington"
---

## German NER in Flair (default model)

This is the standard 4-class NER model for German that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **87,94** (CoNLL-03 German revised)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-german")

# make example sentence
sentence = Sentence("George Washington ging nach Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (0.9977)]
Span [5]: "Washington"   [− Labels: LOC (0.9895)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington ging nach Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import CONLL_03_GERMAN
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = CONLL_03_GERMAN()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # GloVe embeddings
    WordEmbeddings('de'),

    # contextual string embeddings, forward
    FlairEmbeddings('de-forward'),

    # contextual string embeddings, backward
    FlairEmbeddings('de-backward'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-german',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: de
datasets:
- conll2003
widget:
- text: "George Washington ging nach Washington"
---

## German NER in Flair (large model)

This is the large 4-class NER model for German that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **92,31** (CoNLL-03 German revised)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on document-level XLM-R embeddings and [FLERT](https://arxiv.org/pdf/2011.06993v1.pdf/).

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-german-large")

# make example sentence
sentence = Sentence("George Washington ging nach Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (1.0)]
Span [5]: "Washington"   [− Labels: LOC (1.0)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington ging nach Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
import torch

# 1. get the corpus
from flair.datasets import CONLL_03_GERMAN

corpus = CONLL_03_GERMAN()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize fine-tuneable transformer embeddings WITH document context
from flair.embeddings import TransformerWordEmbeddings

embeddings = TransformerWordEmbeddings(
    model='xlm-roberta-large',
    layers="-1",
    subtoken_pooling="first",
    fine_tune=True,
    use_context=True,
)

# 5. initialize bare-bones sequence tagger (no CRF, no RNN, no reprojection)
from flair.models import SequenceTagger

tagger = SequenceTagger(
    hidden_size=256,
    embeddings=embeddings,
    tag_dictionary=tag_dictionary,
    tag_type='ner',
    use_crf=False,
    use_rnn=False,
    reproject_embeddings=False,
)

# 6. initialize trainer with AdamW optimizer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus, optimizer=torch.optim.AdamW)

# 7. run training with XLM parameters (20 epochs, small LR)
from torch.optim.lr_scheduler import OneCycleLR

trainer.train('resources/taggers/ner-german-large',
              learning_rate=5.0e-6,
              mini_batch_size=4,
              mini_batch_chunk_size=1,
              max_epochs=20,
              scheduler=OneCycleLR,
              embeddings_storage_mode='none',
              weight_decay=0.,
              )

)
```

---

### Cite

Please cite the following paper when using this model.

```
@misc{schweter2020flert,
    title={FLERT: Document-Level Features for Named Entity Recognition},
    author={Stefan Schweter and Alan Akbik},
    year={2020},
    eprint={2011.06993},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: de
datasets:
- legal
widget:
- text: "Herr W. verstieß gegen § 36 Abs. 7 IfSG."
---

## NER for German Legal Text in Flair (default model)

This is the legal NER model for German that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **96,35** (LER German dataset)

Predicts 19 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| AN   |  Anwalt   |
| EUN   |  Europäische Norm   |
| GS   |  Gesetz   |
| GRT   | Gericht    |
| INN   |  Institution   |
| LD   |  Land   |
| LDS   | Landschaft    |
| LIT   | Literatur    |
| MRK   |  Marke   |
| ORG   |  Organisation   |
| PER   |  Person   |
| RR   |  Richter   |
| RS   |  Rechtssprechung   |
| ST   |  Stadt   |
| STR   |  Straße   |
| UN   |  Unternehmen   |
| VO   |  Verordnung   |
| VS   |  Vorschrift   |
| VT   | Vertrag    |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

More details on the Legal NER dataset [here](https://github.com/elenanereiss/Legal-Entity-Recognition)

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-german-legal")

# make example sentence (don't use tokenizer since Rechtstexte are badly handled)
sentence = Sentence("Herr W. verstieß gegen § 36 Abs. 7 IfSG.", use_tokenizer=False)

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [2]: "W."   [− Labels: PER (0.9911)]
Span [5,6,7,8,9]: "§ 36 Abs. 7 IfSG."   [− Labels: GS (0.5353)]

```

So, the entities "*W.*" (labeled as a **person**) and "*§ 36 Abs. 7 IfSG*" (labeled as a **Gesetz**) are found in the sentence "*Herr W. verstieß gegen § 36 Abs. 7 IfSG.*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import LER_GERMAN
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = LER_GERMAN()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # GloVe embeddings
    WordEmbeddings('de'),

    # contextual string embeddings, forward
    FlairEmbeddings('de-forward'),

    # contextual string embeddings, backward
    FlairEmbeddings('de-backward'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-german-legal',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following papers when using this model.

```
@inproceedings{leitner2019fine,
  author = {Elena Leitner and Georg Rehm and Julian Moreno-Schneider},
  title = {{Fine-grained Named Entity Recognition in Legal Documents}},
  booktitle = {Semantic Systems. The Power of AI and Knowledge
                  Graphs. Proceedings of the 15th International Conference
                  (SEMANTiCS 2019)},
  year = 2019,
  pages = {272--287},
  pdf = {https://link.springer.com/content/pdf/10.1007%2F978-3-030-33220-4_20.pdf}}
```

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language:
- en
- de
- fr
- it
- nl
- pl
- es
- sv
- da
- no
- fi
- cs
datasets:
- ontonotes
widget:
- text: "Ich liebe Berlin, as they say."
---

## Multilingual Universal Part-of-Speech Tagging in Flair (fast model)

This is the fast multilingual universal part-of-speech tagging model that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **92,88** (12 UD Treebanks covering English, German, French, Italian, Dutch, Polish, Spanish, Swedish, Danish, Norwegian, Finnish and Czech)

Predicts universal POS tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
|ADJ |  adjective |
 |   ADP |  adposition |
 |   ADV |  adverb |
 |   AUX |  auxiliary |
 |   CCONJ |  coordinating conjunction |
 |   DET |  determiner |
 |   INTJ |  interjection |
 |   NOUN |  noun |
 |   NUM |  numeral |
 |   PART |  particle |
 |   PRON |  pronoun |
 |   PROPN |  proper noun |
 |   PUNCT |  punctuation |
 |   SCONJ |  subordinating conjunction |
 |   SYM |  symbol |
 |   VERB |  verb |
 |   X |  other |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/upos-multi-fast")

# make example sentence
sentence = Sentence("Ich liebe Berlin, as they say. ")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('pos'):
    print(entity)
```

This yields the following output:
```
Span [1]: "Ich"   [− Labels: PRON (0.9999)]
Span [2]: "liebe"   [− Labels: VERB (0.9999)]
Span [3]: "Berlin"   [− Labels: PROPN (0.9997)]
Span [4]: ","   [− Labels: PUNCT (1.0)]
Span [5]: "as"   [− Labels: SCONJ (0.9991)]
Span [6]: "they"   [− Labels: PRON (0.9998)]
Span [7]: "say"   [− Labels: VERB (0.9998)]
Span [8]: "."   [− Labels: PUNCT (1.0)]
```

So, the words "*Ich*" and "*they*" are labeled as **pronouns** (PRON), while "*liebe*" and "*say*" are labeled as **verbs** (VERB) in the multilingual sentence "*Ich liebe Berlin, as they say*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import MultiCorpus
from flair.datasets import UD_ENGLISH, UD_GERMAN, UD_FRENCH, UD_ITALIAN, UD_POLISH, UD_DUTCH, UD_CZECH, \
    UD_DANISH, UD_SPANISH, UD_SWEDISH, UD_NORWEGIAN, UD_FINNISH
from flair.embeddings import StackedEmbeddings, FlairEmbeddings

# 1. make a multi corpus consisting of 12 UD treebanks (in_memory=False here because this corpus becomes large)
corpus = MultiCorpus([
    UD_ENGLISH(in_memory=False),
    UD_GERMAN(in_memory=False),
    UD_DUTCH(in_memory=False),
    UD_FRENCH(in_memory=False),
    UD_ITALIAN(in_memory=False),
    UD_SPANISH(in_memory=False),
    UD_POLISH(in_memory=False),
    UD_CZECH(in_memory=False),
    UD_DANISH(in_memory=False),
    UD_SWEDISH(in_memory=False),
    UD_NORWEGIAN(in_memory=False),
    UD_FINNISH(in_memory=False),
])

# 2. what tag do we want to predict?
tag_type = 'upos'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # contextual string embeddings, forward
    FlairEmbeddings('multi-forward-fast'),

    # contextual string embeddings, backward
    FlairEmbeddings('multi-backward-fast'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type,
                        use_crf=False)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/upos-multi-fast',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: en
datasets:
- conll2000
widget:
- text: "The happy man has been eating at the diner"
---

## English Chunking in Flair (fast model)

This is the fast phrase chunking model for English that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **96,22** (CoNLL-2000)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| ADJP     | adjectival |
| ADVP     | adverbial |
| CONJP     | conjunction |
| INTJ     | interjection |
| LST     | list marker |
| NP     | noun phrase |
| PP     | prepositional |
| PRT     | particle |
| SBAR      | subordinate clause |
| VP      | verb phrase |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/chunk-english-fast")

# make example sentence
sentence = Sentence("The happy man has been eating at the diner")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('np'):
    print(entity)

```

This yields the following output:
```
Span [1,2,3]: "The happy man"   [− Labels: NP (0.9958)]
Span [4,5,6]: "has been eating"   [− Labels: VP (0.8759)]
Span [7]: "at"   [− Labels: PP (1.0)]
Span [8,9]: "the diner"   [− Labels: NP (0.9991)]

```

So, the spans "*The happy man*" and "*the diner*" are labeled as **noun phrases** (NP) and "*has been eating*" is labeled as a **verb phrase** (VP) in the sentence "*The happy man has been eating at the diner*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import CONLL_2000
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = CONLL_2000()

# 2. what tag do we want to predict?
tag_type = 'np'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # contextual string embeddings, forward
    FlairEmbeddings('news-forward-fast'),

    # contextual string embeddings, backward
    FlairEmbeddings('news-backward-fast'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/chunk-english-fast',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: nl
datasets:
- conll2003
widget:
- text: "George Washington ging naar Washington"
---

## Dutch NER in Flair (large model)

This is the large 4-class NER model for Dutch that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **95,25** (CoNLL-03 Dutch)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on document-level XLM-R embeddings and [FLERT](https://arxiv.org/pdf/2011.06993v1.pdf/).

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-dutch-large")

# make example sentence
sentence = Sentence("George Washington ging naar Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (1.0)]
Span [5]: "Washington"   [− Labels: LOC (1.0)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington ging naar Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
import torch

# 1. get the corpus
from flair.datasets import CONLL_03_DUTCH

corpus = CONLL_03_DUTCH()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize fine-tuneable transformer embeddings WITH document context
from flair.embeddings import TransformerWordEmbeddings

embeddings = TransformerWordEmbeddings(
    model='xlm-roberta-large',
    layers="-1",
    subtoken_pooling="first",
    fine_tune=True,
    use_context=True,
)

# 5. initialize bare-bones sequence tagger (no CRF, no RNN, no reprojection)
from flair.models import SequenceTagger

tagger = SequenceTagger(
    hidden_size=256,
    embeddings=embeddings,
    tag_dictionary=tag_dictionary,
    tag_type='ner',
    use_crf=False,
    use_rnn=False,
    reproject_embeddings=False,
)

# 6. initialize trainer with AdamW optimizer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus, optimizer=torch.optim.AdamW)

# 7. run training with XLM parameters (20 epochs, small LR)
from torch.optim.lr_scheduler import OneCycleLR

trainer.train('resources/taggers/ner-dutch-large',
              learning_rate=5.0e-6,
              mini_batch_size=4,
              mini_batch_chunk_size=1,
              max_epochs=20,
              scheduler=OneCycleLR,
              embeddings_storage_mode='none',
              weight_decay=0.,
              )

)
```

---

### Cite

Please cite the following paper when using this model.

```
@misc{schweter2020flert,
    title={FLERT: Document-Level Features for Named Entity Recognition},
    author={Stefan Schweter and Alan Akbik},
    year={2020},
    eprint={2011.06993},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: en
datasets:
- ontonotes
widget:
- text: "I love Berlin."
---

## English Universal Part-of-Speech Tagging in Flair (fast model)

This is the fast universal part-of-speech tagging model for English that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **98,47** (Ontonotes)

Predicts universal POS tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
|ADJ |  adjective |
 |   ADP |  adposition |
 |   ADV |  adverb |
 |   AUX |  auxiliary |
 |   CCONJ |  coordinating conjunction |
 |   DET |  determiner |
 |   INTJ |  interjection |
 |   NOUN |  noun |
 |   NUM |  numeral |
 |   PART |  particle |
 |   PRON |  pronoun |
 |   PROPN |  proper noun |
 |   PUNCT |  punctuation |
 |   SCONJ |  subordinating conjunction |
 |   SYM |  symbol |
 |   VERB |  verb |
 |   X |  other |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/upos-english-fast")

# make example sentence
sentence = Sentence("I love Berlin.")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('pos'):
    print(entity)

```

This yields the following output:
```
Span [1]: "I"   [− Labels: PRON (0.9996)]
Span [2]: "love"   [− Labels: VERB (1.0)]
Span [3]: "Berlin"   [− Labels: PROPN (0.9986)]
Span [4]: "."   [− Labels: PUNCT (1.0)]
```

So, the word "*I*" is labeled as a **pronoun** (PRON),  "*love*" is labeled as a **verb** (VERB) and "*Berlin*" is labeled as a **proper noun** (PROPN) in the sentence "*I love Berlin*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import ColumnCorpus
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. load the corpus (Ontonotes does not ship with Flair, you need to download and reformat into a column format yourself)
corpus: Corpus = ColumnCorpus(
                "resources/tasks/onto-ner",
                column_format={0: "text", 1: "pos", 2: "upos", 3: "ner"},
                tag_to_bioes="ner",
            )

# 2. what tag do we want to predict?
tag_type = 'upos'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # contextual string embeddings, forward
    FlairEmbeddings('news-forward-fast'),

    # contextual string embeddings, backward
    FlairEmbeddings('news-backward-fast'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/upos-english-fast',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: en
datasets:
- ontonotes
widget:
- text: "On September 1st George Washington won 1 dollar."
---

## English NER in Flair (Ontonotes fast model)

This is the fast version of the 18-class NER model for English that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **89.3** (Ontonotes)

Predicts 18 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| CARDINAL    | cardinal value |
| DATE         | date value |
| EVENT         | event name |
| FAC         | building name |
| GPE         | geo-political entity |
| LANGUAGE         | language name |
| LAW         | law name |
| LOC         | location name |
| MONEY         | money name |
| NORP         | affiliation |
| ORDINAL         | ordinal value |
| ORG         | organization name |
| PERCENT         | percent value |
| PERSON         | person name |
| PRODUCT         | product name |
| QUANTITY         | quantity value |
| TIME         | time value |
| WORK_OF_ART         | name of work of art |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-english-ontonotes-fast")

# make example sentence
sentence = Sentence("On September 1st George Washington won 1 dollar.")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [2,3]: "September 1st"   [− Labels: DATE (0.9655)]
Span [4,5]: "George Washington"   [− Labels: PERSON (0.8243)]
Span [7,8]: "1 dollar"   [− Labels: MONEY (0.8022)]
```

So, the entities "*September 1st*" (labeled as a **date**), "*George Washington*" (labeled as a **person**) and "*1 dollar*" (labeled as a **money**) are found in the sentence "*On September 1st George Washington won 1 dollar*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import ColumnCorpus
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. load the corpus (Ontonotes does not ship with Flair, you need to download and reformat into a column format yourself)
corpus: Corpus = ColumnCorpus(
                "resources/tasks/onto-ner",
                column_format={0: "text", 1: "pos", 2: "upos", 3: "ner"},
                tag_to_bioes="ner",
            )

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # GloVe embeddings
    WordEmbeddings('en-crawl'),

    # contextual string embeddings, forward
    FlairEmbeddings('news-forward-fast'),

    # contextual string embeddings, backward
    FlairEmbeddings('news-backward-fast'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-english-ontonotes-fast',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: en
datasets:
- conll2003
widget:
- text: "George Washington went to Washington"
---

## English NER in Flair (fast model)

This is the fast 4-class NER model for English that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **92,92** (corrected CoNLL-03)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-english-fast")

# make example sentence
sentence = Sentence("George Washington went to Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (0.9515)]
Span [5]: "Washington"   [− Labels: LOC (0.992)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington went to Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import CONLL_03
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = CONLL_03()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # GloVe embeddings
    WordEmbeddings('glove'),

    # contextual string embeddings, forward
    FlairEmbeddings('news-forward-fast'),

    # contextual string embeddings, backward
    FlairEmbeddings('news-backward-fast'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-english',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language:
- en
- de
- fr
- it
- nl
- pl
- es
- sv
- da
- no
- fi
- cs
datasets:
- ontonotes
widget:
- text: "Ich liebe Berlin, as they say"
---

## Multilingual Universal Part-of-Speech Tagging in Flair (default model)

This is the default multilingual universal part-of-speech tagging model that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **98,47** (12 UD Treebanks covering English, German, French, Italian, Dutch, Polish, Spanish, Swedish, Danish, Norwegian, Finnish and Czech)

Predicts universal POS tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
|ADJ |  adjective |
 |   ADP |  adposition |
 |   ADV |  adverb |
 |   AUX |  auxiliary |
 |   CCONJ |  coordinating conjunction |
 |   DET |  determiner |
 |   INTJ |  interjection |
 |   NOUN |  noun |
 |   NUM |  numeral |
 |   PART |  particle |
 |   PRON |  pronoun |
 |   PROPN |  proper noun |
 |   PUNCT |  punctuation |
 |   SCONJ |  subordinating conjunction |
 |   SYM |  symbol |
 |   VERB |  verb |
 |   X |  other |

Based on [Flair embeddings](https://www.aclweb.org/anthology/C18-1139/) and LSTM-CRF.

---

### Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/upos-multi")

# make example sentence
sentence = Sentence("Ich liebe Berlin, as they say. ")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('pos'):
    print(entity)
```

This yields the following output:
```
Span [1]: "Ich"   [− Labels: PRON (0.9999)]
Span [2]: "liebe"   [− Labels: VERB (0.9999)]
Span [3]: "Berlin"   [− Labels: PROPN (0.9997)]
Span [4]: ","   [− Labels: PUNCT (1.0)]
Span [5]: "as"   [− Labels: SCONJ (0.9991)]
Span [6]: "they"   [− Labels: PRON (0.9998)]
Span [7]: "say"   [− Labels: VERB (0.9998)]
Span [8]: "."   [− Labels: PUNCT (1.0)]
```

So, the words "*Ich*" and "*they*" are labeled as **pronouns** (PRON), while "*liebe*" and "*say*" are labeled as **verbs** (VERB) in the multilingual sentence "*Ich liebe Berlin, as they say*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import MultiCorpus
from flair.datasets import UD_ENGLISH, UD_GERMAN, UD_FRENCH, UD_ITALIAN, UD_POLISH, UD_DUTCH, UD_CZECH, \
    UD_DANISH, UD_SPANISH, UD_SWEDISH, UD_NORWEGIAN, UD_FINNISH
from flair.embeddings import StackedEmbeddings, FlairEmbeddings

# 1. make a multi corpus consisting of 12 UD treebanks (in_memory=False here because this corpus becomes large)
corpus = MultiCorpus([
    UD_ENGLISH(in_memory=False),
    UD_GERMAN(in_memory=False),
    UD_DUTCH(in_memory=False),
    UD_FRENCH(in_memory=False),
    UD_ITALIAN(in_memory=False),
    UD_SPANISH(in_memory=False),
    UD_POLISH(in_memory=False),
    UD_CZECH(in_memory=False),
    UD_DANISH(in_memory=False),
    UD_SWEDISH(in_memory=False),
    UD_NORWEGIAN(in_memory=False),
    UD_FINNISH(in_memory=False),
])

# 2. what tag do we want to predict?
tag_type = 'upos'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize each embedding we use
embedding_types = [

    # contextual string embeddings, forward
    FlairEmbeddings('multi-forward'),

    # contextual string embeddings, backward
    FlairEmbeddings('multi-backward'),
]

# embedding stack consists of Flair and GloVe embeddings
embeddings = StackedEmbeddings(embeddings=embedding_types)

# 5. initialize sequence tagger
from flair.models import SequenceTagger

tagger = SequenceTagger(hidden_size=256,
                        embeddings=embeddings,
                        tag_dictionary=tag_dictionary,
                        tag_type=tag_type,
                        use_crf=False)

# 6. initialize trainer
from flair.trainers import ModelTrainer

trainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/upos-multi',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik2018coling,
  title={Contextual String Embeddings for Sequence Labeling},
  author={Akbik, Alan and Blythe, Duncan and Vollgraf, Roland},
  booktitle = {{COLING} 2018, 27th International Conference on Computational Linguistics},
  pages     = {1638--1649},
  year      = {2018}
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---
tags:
- flair
- token-classification
- sequence-tagger-model
language: nl
datasets:
- conll2003
widget:
- text: "George Washington ging naar Washington."
---

# Dutch NER in Flair (default model)

This is the standard 4-class NER model for Dutch that ships with [Flair](https://github.com/flairNLP/flair/).

F1-Score: **92,58** (CoNLL-03)

Predicts 4 tags:

| **tag**                        | **meaning** |
|---------------------------------|-----------|
| PER         | person name |
| LOC         | location name |
| ORG         | organization name |
| MISC         | other name |

Based on Transformer embeddings and LSTM-CRF.

---
# Demo: How to use in Flair

Requires: **[Flair](https://github.com/flairNLP/flair/)** (`pip install flair`)

```python
from flair.data import Sentence
from flair.models import SequenceTagger

# load tagger
tagger = SequenceTagger.load("flair/ner-dutch")

# make example sentence
sentence = Sentence("George Washington ging naar Washington")

# predict NER tags
tagger.predict(sentence)

# print sentence
print(sentence)

# print predicted NER spans
print('The following NER tags are found:')
# iterate over entities and print
for entity in sentence.get_spans('ner'):
    print(entity)

```

This yields the following output:
```
Span [1,2]: "George Washington"   [− Labels: PER (0.997)]
Span [5]: "Washington"   [− Labels: LOC (0.9996)]
```

So, the entities "*George Washington*" (labeled as a **person**) and "*Washington*" (labeled as a **location**) are found in the sentence "*George Washington ging naar Washington*".

---

### Training: Script to train this model

The following Flair script was used to train this model:

```python
from flair.data import Corpus
from flair.datasets import CONLL_03_DUTCH
from flair.embeddings import WordEmbeddings, StackedEmbeddings, FlairEmbeddings

# 1. get the corpus
corpus: Corpus = CONLL_03_DUTCH()

# 2. what tag do we want to predict?
tag_type = 'ner'

# 3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

# 4. initialize embeddings
embeddings = TransformerWordEmbeddings('wietsedv/bert-base-dutch-cased')

# 5. initialize sequence tagger
tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type)

# 6. initialize trainer
trainer: ModelTrainer = ModelTrainer(tagger, corpus)

# 7. run training
trainer.train('resources/taggers/ner-dutch',
              train_with_dev=True,
              max_epochs=150)
```

---

### Cite

Please cite the following paper when using this model.

```
@inproceedings{akbik-etal-2019-flair,
    title = "{FLAIR}: An Easy-to-Use Framework for State-of-the-Art {NLP}",
    author = "Akbik, Alan  and
      Bergmann, Tanja  and
      Blythe, Duncan  and
      Rasul, Kashif  and
      Schweter, Stefan  and
      Vollgraf, Roland",
    booktitle = "Proceedings of the 2019 Conference of the North {A}merican Chapter of the Association for Computational Linguistics (Demonstrations)",
    year = "2019",
    url = "https://www.aclweb.org/anthology/N19-4010",
    pages = "54--59",
}
```

---

### Issues?

The Flair issue tracker is available [here](https://github.com/flairNLP/flair/issues/).


---

language: ["ru", "en", "be", "bg", "uk", "ro", "kz", "tg", "tat", "sv", "sl", "sr", "uz", "es", "fi"]

tags:

- russian

- fill-mask

- pretraining

- embeddings

- masked-lm

license: mit

widget:

- text: "<mask> на склад"

---

!!!
At the moment, the model is distilled, a version from one of the first checkpoints is available for download.
We plan to post the full model in the next few days.
!!!

This is a distilled HRBert model for an mlm task.

Sentence embeddings can be produced as follows:

```python

# pip install transformers

from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model='RabotaRu/HRBert-mini',
    tokenizer='RabotaRu/HRBert-mini'
)

fill_mask('<mask> на склад')
```


---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: distilroberta-base-finetuned-toxic
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilroberta-base-finetuned-toxic

This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2768

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.5338        | 1.0   | 313  | 2.3127          |
| 2.4482        | 2.0   | 626  | 2.2985          |
| 2.4312        | 3.0   | 939  | 2.2411          |

### Framework versions

- Transformers 4.16.0
- Pytorch 1.10.0
- Datasets 1.18.1
- Tokenizers 0.10.3


---
tags:
- generated_from_trainer
datasets:
- squad
model-index:
- name: biobert_squad2_cased-finetuned-squad
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# biobert_squad2_cased-finetuned-squad

This model is a fine-tuned version of [clagator/biobert_squad2_cased](https://huggingface.co/clagator/biobert_squad2_cased) on the squad dataset.

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Framework versions

- Transformers 4.15.0
- Pytorch 1.10.0+cu111
- Datasets 1.17.0
- Tokenizers 0.10.3


## Usage
```python
from transformers import BertForSequenceClassification
from transformers import BertTokenizer
model = BertForSequenceClassification.from_pretrained("trituenhantaoio/bert-base-vietnamese-uncased")
tokenizer = BertTokenizer.from_pretrained("trituenhantaoio/bert-base-vietnamese-uncased")
```

### References

```
@article{ttnt2020bert,
  title={Vietnamese BERT: Pretrained on News and Wiki},
  author={trituenhantao.io},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/trituenhantaoio/vn-bert-base-uncased}},
}
```

[trituenhantao.io](https://trituenhantao.io)


## Usage
```python
from transformers import BertForSequenceClassification
from transformers import BertTokenizer
model = BertForSequenceClassification.from_pretrained("trituenhantaoio/bert-base-vietnamese-diacritics-uncased")
tokenizer = BertTokenizer.from_pretrained("trituenhantaoio/bert-base-vietnamese-diacritics-uncased")
```

### References

```
@article{ttnt2020bertdiacritics,
  title={Vietnamese BERT Diacritics: Pretrained on News and Wiki},
  author={trituenhantao.io},
  year = {2020},
  publisher = {Hugging Face},
  journal = {Hugging Face repository}
}
```

[trituenhantao.io](https://trituenhantao.io)


---
tags:
- image-classification
- pytorch
- huggingpics
metrics:
- accuracy

model-index:
- name: new-york-tokyo-london
  results:
  - task:
      name: Image Classification
      type: image-classification
    metrics:
      - name: Accuracy
        type: accuracy
        value: 0.9104477763175964
---

# new-york-tokyo-london

Autogenerated by HuggingPics🤗🖼

Create your own image classifier for **anything** by running [the demo on Google Colab](https://colab.research.google.com/github/nateraw/huggingpics/blob/main/HuggingPics.ipynb).

Report any issues with the demo at the [github repo](https://github.com/nateraw/huggingpics).

## Example Images

#### London

![London](images/London.jpg)

#### New York

![New York](images/New_York.jpg)

#### Tokyo

![Tokyo](images/Tokyo.jpg)


---
thumbnail: https://huggingface.co/front/thumbnails/dialogpt.png
language:
- en
tags:
- conversational
- transformers
datasets:
- GDC
metrics:
- perplexity
license: cc-by-4.0
widget:
- text: "Jag ska fika."
---

## DialoGPT_SV

This is a fine-tuned model of the DialoGPT (medium) on the Swedish Gothenburg Dialogue Corpus (GDC). It is intended to be used as a Swedish conversational system. The GDC dataset it's trained on is limited in scope, as it's from the transcription of dialogues of about 25 different social activities, including debates, academic seminars and situations. The dataset has about 108K turns.

The perplexity achieved on the test and validation sets (from 80:10:10 split) are 27.81 and 27.81, respectivley. More information about the original pre-trained model can be found [here](https://huggingface.co/microsoft/DialoGPT-medium)

* Generation example from an interactive environment:

|Role | Response |
|---------|------------|
|User | Jag ska fika |
|Bot | ja det är ju så att jag kanske inte kan få den att göra det |
|User | tack så mycket |
|Bot | tack tack ses vi om sex vecko igen |

Please, find the information about preprocessing, training and full details of the DialoGPT in the [original DialoGPT repository](https://github.com/microsoft/DialoGPT)

The paper for this work was accepted at the Northern Lights Deep Learning (NLDL) conference 2022. Arxiv paper: [https://arxiv.org/pdf/2110.06273.pdf](https://arxiv.org/pdf/2110.06273.pdf)

### How to use

Now we are ready to try out how the model works as a chatting partner!

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("tosin/dialogpt_sv")
model = AutoModelForCausalLM.from_pretrained("tosin/dialogpt_sv")
# Let's chat for 5 lines
for step in range(5):
	# encode the new user input, add the eos_token and return a tensor in Pytorch
	new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
	# append the new user input tokens to the chat history
	bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
	# generated a response while limiting the total chat history to 1000 tokens,
	chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)
	# pretty print last ouput tokens from bot
	print("Swedish_GDC_Bot: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))


---
thumbnail: https://huggingface.co/front/thumbnails/dialogpt.png
language:
- en
license: cc-by-4.0
tags:
- conversational
- transformers
datasets:
- multi_woz_v22
metrics:
- perplexity
widget:
- text: "I would like to have breakfast."
---

## DialoGPT_MWOZ

This is a fine-tuned model of DialoGPT (medium) on the MultiWOZ v2.2 dataset. It is intended to be used as a conversational system.
The dataset it's trained on is limited in scope, as it covers only certain domains such as restaurants, hotel, taxi, train, hospital and police.

The perplexity achieved on the test and validation sets are 10.25 and 10.71, respectively. More information about the original pre-trained model can be found [here](https://huggingface.co/microsoft/DialoGPT-medium)

* Generation example from an interactive environment:

|Role | Response |
|---------|------------|
|User | I would like to have breakfast. |
|Bot | There are several options for Chinese in the city centre. Are you looking for a particular price range? |
|User | I'm looking for free food |
|Bot | There are 4 Chinese restaurants that are in the cheap price range. 3 in centre and one in north. |
|User | Not good enough |
|Bot | How about Charlie Chan? It is a Chinese restaurant in the centre of town. |

Please find the information about preprocessing, training and full details of the DialoGPT in the [original DialoGPT repository](https://github.com/microsoft/DialoGPT)

The paper for this work was accepted at the Northern Lights Deep Learning (NLDL) conference 2022. Arxiv paper: [https://arxiv.org/pdf/2110.06273.pdf](https://arxiv.org/pdf/2110.06273.pdf)

### How to use

Now we are ready to try out how the model works as a chatting partner!

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("tosin/dialogpt_mwoz")
model = AutoModelForCausalLM.from_pretrained("tosin/dialogpt_mwoz")
# Let's chat for 5 lines
for step in range(5):
	# encode the new user input, add the eos_token and return a tensor in Pytorch
	new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')
	# append the new user input tokens to the chat history
	bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
	# generated a response while limiting the total chat history to 1000 tokens,
	chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)
	# pretty print last ouput tokens from bot
	print("DialoGPT_MWOZ_Bot: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))


---
language: ja
tags:
- mobilebert
license: cc-by-sa-3.0
datasets:
- wikipedia
---

# MobileBERT 日本語事前学習済みモデル爆誕！！
AI関係の仕事をしている櫻本です。
2020年に発表されたBERTの発展型モデルの一つである「MobileBERT」の、日本語事前学習済みモデルを構築しました。
このページを見つけた方はかなりラッキーですから、ぜひ一度使ってみてください！！
BERTの推論速度の遅さを嘆いている方にお薦めです。

# 利用方法
既にtransformersでBERTを利用されている方向けの説明です。
トークナイザは東北大学さんのモデル(cl-tohoku/bert-large-japanese)からお借りしましたのでご指定ください。
後は、**BertFor**なんちゃら～のクラスを**MobileBertFor**なんちゃら～に直して、このリポジトリを指定するだけです！
```from transformers import BertJapaneseTokenizer, MobileBertForSequenceClassification
tokenizer = BertJapaneseTokenizer.from_pretrained("cl-tohoku/bert-large-japanese")
model = MobileBertForSequenceClassification.from_pretrained("ysakuramoto/mobilebert-ja") # 文書分類の場合
```
(注意：文書分類などのタスクに利用するには、ファインチューニングが必要です)

# BERTとの性能比較
文書分類と固有表現抽出について、ファインチューニング・性能評価を行いました。
参考程度にご覧ください。(ファインチューニング後の性能を保証するものではありません)

  - 文書分類(MobileBertForSequenceClassification)
  |メトリック|BERT|MobileBERT(高速化前)|MobileBERT(高速化後)|
  |-----------|-----------| ------- | -------- |
  |学習時間(s)|585.0|399.7|-|
  |推論時間(s)|259.0|108.7|70.5|
  |精度|86.4%|85.5%|86.4%|
  |モデルファイルサイズ(MB)|440.2|-|41.8|

    - 条件
      - ライブドアニュースコーパスのタイトルとカテゴリで学習・推論。
      - 比較対象のBERTモデルは東北大学さんの"cl-tohoku/bert-base-japanese-whole-word-masking"。
      - 推論データ n=1,474。精度はAccuracy
      - 学習パラメータ: エポック数=10, lr=1e-4
      - 推論時の高速化として、枝刈り(-20%)・量子化・jitコンパイルを実施。
      - Google Colabにて、学習にGPU、推論にCPUを利用。バッチ処理でなく1件ずつ推論。
      - それぞれ、学習～推論を3回実施した平均値。

  - 固有表現抽出(MobileBertForTokenClassification)
  |メトリック|BERT|MobileBERT(高速化前)|MobileBERT(高速化後)|
  |-----------|-----------| ------- | -------- |
  |学習時間(s)|428.0|294.0|-|
  |推論時間(s)|163.5|78.4|40.9|
  |精度|86.4%|82.5%|83.3%|
  |モデルファイルサイズ(MB)|440.2|-|41.8|

    - 条件
      - ストックマーク社さんのwikipediaデータセットで学習・推論。(https://github.com/stockmarkteam/ner-wikipedia-dataset)
      - 比較対象のBERTモデルは東北大学さんの"cl-tohoku/bert-base-japanese-whole-word-masking"。
      - 推論データ n=2,140。精度は完全一致のf-measure
      - 学習パラメータ: エポック数=10, lr=1e-4
      - 推論時の高速化として、枝刈り(-20%)・量子化・jitコンパイルを実施。
      - Google Colabにて、学習にGPU、推論にCPUを利用。バッチ処理でなく1件ずつ推論。
      - それぞれ、学習～推論を3回実施した平均値。

# モデルの説明
- モデルの構造
  - 論文中の"MobileBERT"構造に従いました。(論文中にはMobileBERTTINYというバージョンもありますがそちらではないです)
    - 論文中のTable.1 をご確認ください。 https://arxiv.org/abs/2004.02984
- 学習に利用したデータ
  - 東北大学さんが公開されている方法で、2021年8月時点のwikipediaデータを加工・利用しました。
    - 東北大学さんのgithub https://github.com/cl-tohoku/bert-japanese
- トークナイザ
  - 東北大学さんのモデル"cl-tohoku/bert-large-japanese"からお借りしました。vocab sizeは32,768です。
- 学習方法
  - Google ColabからTPUを用いて学習しました。
    1. IB-BERTLARGEをlr=5e-4で1Mステップ学習しました。
    1. IB-BERTLARGEを240kステップ蒸留後、mobileBERTをlr=5e-4で2Mステップ学習しました。
  - トータルで2ヶ月半くらいかかりました。。エラー出まくってつらかったです。

# ライセンス
[CC-BY SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/deed.ja)
トークナイザについては東北大学さんのモデル"cl-tohoku/bert-large-japanese"からお借りしました。

# 免責
このモデルを利用・参照することで発生したあらゆる不都合や損害について、一切の責任を負いかねます。


# XLNet Fine-tuned on SQuAD / Quoref Dataset
[XLNet](https://arxiv.org/abs/1906.08237) jointly developed by Google and CMU and fine-tuned on [SQuAD / SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) and [Quoref](https://leaderboard.allenai.org/quoref) for question answering down-stream task.

## Evaluation Result on Quoref
```
{
    "exact_match": 73.65591397848462,
    "f1": 77.9981532789881
}
```

## Results Comparison on Quoref
| Metric | XLNet Base Line | Model FT on SQuAD    |
| ------ | --------- | --------- |
| **EM** | **61.88** | **73.66** (+11.78)      |
| **F1** | **70.51** | **78.00** (+7.49)|

## How to Use
```
from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast

model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref)
tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref')
```


# XLNet Fine-tuned on SQuAD 2.0 Dataset
[XLNet](https://arxiv.org/abs/1906.08237) jointly developed by Google and CMU and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for question answering down-stream task.

## Training Results (Metrics)
```
{
    "HasAns_exact": 74.7132253711201
    "HasAns_f1": 82.11971607032643
    "HasAns_total": 5928
    "NoAns_exact": 73.38940285954584
    "NoAns_f1": 73.38940285954584
    "NoAns_total": 5945
    "best_exact": 75.67590331003116
    "best_exact_thresh": -19.554906845092773
    "best_f1": 79.16215426779269
    "best_f1_thresh": -19.554906845092773
    "epoch": 4.0
    "exact": 74.05036637749515
    "f1": 77.74830934598614
    "total": 11873
}
```

## Results Comparison
| Metric | Paper     | Model     |
| ------ | --------- | --------- |
| **EM** | **78.46** | **75.68** (-2.78)      |
| **F1** | **81.33** | **79.16** (-2.17)|

Better fine-tuned models coming soon.

## How to Use
```
from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast

model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-squadv2)
tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-squadv2')
```


---
tags:
- conversational
---

# Elon Musk DialogGPT Model


---
language: fr
widget:
- text: "generate question: Barack Hussein Obama, né le 4 aout 1961, est un homme politique américain et avocat. Il a été élu <hl> en 2009 <hl> pour devenir le 44ème président des Etats-Unis d'Amérique. </s>"
- text: "question: Quand Barack Obama a t'il été élu président? context: Barack Hussein Obama, né le 4 aout 1961, est un homme politique américain et avocat. Il a été élu en 2009 pour devenir le 44ème président des Etats-Unis d'Amérique. </s>"
tags:
- pytorch
- t5
- question-generation
- seq2seq
datasets:
- fquad
- piaf
---

# T5 Question Generation and Question Answering

## Model description

This model is a T5 Transformers model (airklizz/t5-base-multi-fr-wiki-news) that was fine-tuned in french on 3 different tasks
* question generation

* question answering

* answer extraction

It obtains quite good results on FQuAD validation dataset.

## Intended uses & limitations

This model functions for the 3 tasks mentionned earlier and was not tested on other tasks.

```python
from transformers import T5ForConditionalGeneration, T5Tokenizer
model = T5ForConditionalGeneration.from_pretrained("JDBN/t5-base-fr-qg-fquad")
tokenizer = T5Tokenizer.from_pretrained("JDBN/t5-base-fr-qg-fquad")
```

## Training data

The initial model used was https://huggingface.co/airKlizz/t5-base-multi-fr-wiki-news. This model was finetuned on a dataset composed of FQuAD and PIAF on the 3 tasks mentioned previously.

The data were preprocessed like this
* question generation: "generate question: Barack Hussein Obama, né le 4 aout 1961, est un homme politique américain et avocat. Il a été élu <hl> en 2009 <hl> pour devenir le 44ème président des Etats-Unis d'Amérique."

* question answering: "question: Quand Barack Hussein Obamaa-t-il été élu président des Etats-Unis d’Amérique? context: Barack Hussein Obama, né le 4 aout 1961, est un homme politique américain et avocat. Il a été élu en 2009 pour devenir le 44ème président des Etats-Unis d’Amérique."

* answer extraction: "extract_answers: Barack Hussein Obama, né le 4 aout 1961, est un homme politique américain et avocat. <hl> Il a été élu en 2009 pour devenir le 44ème président des Etats-Unis d’Amérique <hl>."

The preprocessing we used was implemented in https://github.com/patil-suraj/question_generation

## Eval results

#### On FQuAD validation set

| BLEU_1 | BLEU_2 | BLEU_3 | BLEU_4 | METEOR | ROUGE_L | CIDEr |
|--------|--------|--------|--------|--------|---------|-------|
|  0.290 | 0.203  |  0.149 |  0.111 |  0.197 |  0.284  | 1.038 |

#### Question Answering metrics

For these metrics, the performance of this question answering model (https://huggingface.co/illuin/camembert-base-fquad) on FQuAD original question and on T5 generated questions are compared.

| Questions | Exact Match | F1 Score |
|------------------|--------|--------|
|Original FQuAD | 54.015 | 77.466 |
|Generated | 45.765 | 67.306 |

### BibTeX entry and citation info

```bibtex
@misc{githubPatil,
author = {Patil Suraj},
title = {question generation GitHub repository},
year = {2020},
howpublished={\url{https://github.com/patil-suraj/question_generation}}
}

@article{T5,
    title={Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
    author={Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
    year={2019},
    eprint={1910.10683},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

@misc{dhoffschmidt2020fquad,
      title={FQuAD: French Question Answering Dataset},
      author={Martin d'Hoffschmidt and Wacim Belblidia and Tom Brendlé and Quentin Heinrich and Maxime Vidal},
      year={2020},
      eprint={2002.06071},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


# Pretrained Language Model for Vietnamese Health News

## Data
- 300MB data related to health news (e.g: cancer, vaccine, eating, nutrition, .etc)

## Usage
```
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("demdecuong/viHNews-bert")

model = AutoModelForMaskedLM.from_pretrained("demdecuong/viHNews-bert")
```

## Result
- TBD


This is finetune version of [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://arxiv.org/abs/2104.08821)
, train unsupervised on 570K stroke sentences from : stroke books, quora medical, quora's stroke and human annotates.

### Extract sentence representation
```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("demdecuong/stroke_simcse")
model = AutoModel.from_pretrained("demdecuong/stroke_simcse")

text = "What are disease related to red stroke's causes?"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)[1]
```
### Build up embedding for database

```
database = [
    'What is the daily checklist for stroke returning home',
    'What are some tips for stroke adapt new life',
    'What  should I consider when using nursing-home care'
]

embedding = torch.zeros((len(database),768))

for i in range(len(database)):
  inputs = tokenizer(database[i], return_tensors="pt")
  outputs = model(**inputs)[1]
  embedding[i] = outputs

print(embedding.shape)
```

### Result
On our Poc testset , which contains pairs of matching question related to stroke from human-generated.

| Model  | Top-1 Accuracy |
| ------------- | ------------- |
| SimCSE (supervised)  | 75.83  |
| SimCSE (ours)  | 76.66  |


This is finetune version of [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://arxiv.org/abs/2104.08821)

- Train supervised on 100K triplet samples samples related to stroke domain from : stroke books, quora medical, quora's stroke, quora's general and human annotates.
- Positive sentences are generated by paraphrasing and back-translate.
- Negative sentences are randomly selected in general domain.

### Extract sentence representation
```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("demdecuong/stroke_sup_simcse")
model = AutoModel.from_pretrained("demdecuong/stroke_sup_simcse")

text = "What are disease related to red stroke's causes?"
inputs = tokenizer(text, return_tensors='pt')
outputs = model(**inputs)[1]
```
### Build up embedding for database

```
database = [
    'What is the daily checklist for stroke returning home',
    'What are some tips for stroke adapt new life',
    'What  should I consider when using nursing-home care'
]

embedding = torch.zeros((len(database),768))

for i in range(len(database)):
  inputs = tokenizer(database[i], return_tensors="pt")
  outputs = model(**inputs)[1]
  embedding[i] = outputs

print(embedding.shape)
```

### Result
On our company's PoC project, the testset contains positive/negative pairs of matching question related to stroke from human-generation.
- SimCSE supervised + 100k : Train on 100K triplet samples contains : medical, stroke and general domain
- SimCSE supervised + 42k : Train on 42K triplet samples contains : medical, stroke domain

| Model  | Top-1 Accuracy |
| ------------- | ------------- |
| SimCSE supervised (author)  | 75.83  |
| SimCSE unsupervised (ours)  | 76.66  |
| SimCSE supervised + 100k (ours)  | 73.33  |
| SimCSE supervised + 42k (ours)  | 75.83  |


#Pretrained Language Model for Vietnamese Health News

## Data
300MB data related to health news (e.g: cancer, vaccine, eating, .etc)

## Usage
```
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("demdecuong/viHNews-bert-v1.1")

model = AutoModelForMaskedLM.from_pretrained("demdecuong/viHNews-bert-v1.1")
```

## Result
TBD


version https://git-lfs.github.com/spec/v1
oid sha256:49c11380f626964b7f870051fe1c23de7638cdb31bad6e18fb8326d2fe2f8e3a
size 1107


---
tags:
- generated_from_trainer
datasets:
- null
model-index:
- name: bert-base-multilingual-cased-urgency
  results:
  - task:
      name: Masked Language Modeling
      type: fill-mask
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-base-multilingual-cased-urgency

This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/) on the mWACH NEO dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2797

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 4.1408        | 1.0   | 5659  | 3.6705          |
| 2.8777        | 2.0   | 11318 | 2.5536          |
| 2.561         | 3.0   | 16977 | 2.2740          |

### Framework versions

- Transformers 4.10.0
- Pytorch 1.9.0+cu102
- Datasets 1.11.0
- Tokenizers 0.10.3


---
tags:
- conversational
---

# A certain person's AI


---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- matthews_correlation
model-index:
- name: distilbert-base-uncased-finetuned-cola
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: glue
      type: glue
      args: cola
    metrics:
    - name: Matthews Correlation
      type: matthews_correlation
      value: 0.5286324175580216
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-finetuned-cola

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the glue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7809
- Matthews Correlation: 0.5286

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Matthews Correlation |
|:-------------:|:-----:|:----:|:---------------:|:--------------------:|
| 0.5299        | 1.0   | 535  | 0.5040          | 0.4383               |
| 0.3472        | 2.0   | 1070 | 0.5284          | 0.4911               |
| 0.2333        | 3.0   | 1605 | 0.6633          | 0.5091               |
| 0.1733        | 4.0   | 2140 | 0.7809          | 0.5286               |
| 0.1255        | 5.0   | 2675 | 0.8894          | 0.5282               |

### Framework versions

- Transformers 4.10.2
- Pytorch 1.9.0+cu102
- Datasets 1.12.1
- Tokenizers 0.10.3


# CovidBERT-MedNLI

This is the model **CovidBERT** trained by DeepSet on AllenAI's [CORD19 Dataset](https://pages.semanticscholar.org/coronavirus-research) of scientific articles about coronaviruses.

The model uses the original BERT wordpiece vocabulary and was subsequently fine-tuned on the [SNLI](https://nlp.stanford.edu/projects/snli/) and the [MultiNLI](https://www.nyu.edu/projects/bowman/multinli/) datasets using the [`sentence-transformers` library](https://github.com/UKPLab/sentence-transformers/) to produce universal sentence embeddings [1] using the **average pooling strategy** and a **softmax loss**.
It is further fine-tuned on both MedNLI datasets available at Physionet.

[ACL-BIONLP 2019](https://physionet.org/content/mednli-bionlp19/1.0.1/)

[MedNLI from MIMIC](https://physionet.org/content/mednli/1.0.0/)

Parameter details for the original training on CORD-19 are available on [DeepSet's MLFlow](https://public-mlflow.deepset.ai/#/experiments/2/runs/ba27d00c30044ef6a33b1d307b4a6cba)

**Base model**: `deepset/covid_bert_base` from HuggingFace's `AutoModel`.


Fine-tuned CovidBERT on Med-Marco Dataset for passage ranking

# CovidBERT-MedNLI

This is the model **CovidBERT** trained by DeepSet on AllenAI's [CORD19 Dataset](https://pages.semanticscholar.org/coronavirus-research) of scientific articles about coronaviruses.

The model uses the original BERT wordpiece vocabulary and was subsequently fine-tuned on the [SNLI](https://nlp.stanford.edu/projects/snli/) and the [MultiNLI](https://www.nyu.edu/projects/bowman/multinli/) datasets using the [`sentence-transformers` library](https://github.com/UKPLab/sentence-transformers/) to produce universal sentence embeddings [1] using the **average pooling strategy** and a **softmax loss**.

It is further fine-tuned Med-Marco Dataset. MacAvaney et.al in their [paper](https://arxiv.org/abs/2010.05987) titled “SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search” used MedSyn a lexicon of layperson and expert terminology for various medical conditions to filter for medical questions. One can also replace this by UMLs ontologies but the beauty of MedSyn is that the terms are more general human conversation lingo and not terms based on scientific literature.

Parameter details for the original training on CORD-19 are available on [DeepSet's MLFlow](https://public-mlflow.deepset.ai/#/experiments/2/runs/ba27d00c30044ef6a33b1d307b4a6cba)

**Base model**: `deepset/covid_bert_base` from HuggingFace's `AutoModel`.


---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: wav2vec2-base-timit-demo-colab
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-base-timit-demo-colab

This model is a fine-tuned version of [facebook/wav2vec2-base](https://huggingface.co/facebook/wav2vec2-base) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4688
- Wer: 0.3417

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer    |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 3.4156        | 4.0   | 500  | 1.2721          | 0.8882 |
| 0.6145        | 8.0   | 1000 | 0.4712          | 0.4510 |
| 0.229         | 12.0  | 1500 | 0.4459          | 0.3847 |
| 0.1312        | 16.0  | 2000 | 0.4739          | 0.3786 |
| 0.0897        | 20.0  | 2500 | 0.4483          | 0.3562 |
| 0.0608        | 24.0  | 3000 | 0.4450          | 0.3502 |
| 0.0456        | 28.0  | 3500 | 0.4688          | 0.3417 |

### Framework versions

- Transformers 4.11.3
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.10.3


---
tags:
- conversational
---

# Rick and Morty DialoGPT Model


# BigBird for Mortality Prediction

Starting with Google's base BigBird model, we fine-tuned on binary mortality prediction in MIMIC admission notes. This
model seeks to predict whether a certain patient will expire within a given ICU stay, based on the text available upon
admission. Data prepared for this task as described in [this project](https://github.com/bvanaken/clinical-outcome-prediction),
using the simulated admission notes (taken from discharge summaries). This model will be used in an upcoming submission for
IMLH at ICML 2021.

### References
* Van Aken, et al., 2021: [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://www.aclweb.org/anthology/2021.eacl-main.75/)
* Zaheer, et al., 2020: [Big Bird: Transformers for Longer Sequences](https://papers.nips.cc/paper/2020/hash/c8512d142a2d849725f31a9a7a361ab9-Abstract.html)


# BERT Base Fine-tuned on MTSamples
This model is [BERT-base](https://huggingface.co/bert-base-uncased) fine-tuned on the MTSamples dataset, with a classification task defined in [this repo](https://github.com/socd06/medical-nlp).


# BioClinical BERT Fine-tuned on MTSamples
This model is simply [Alsentzer's Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) fine-tuned on the MTSamples dataset, with a classification task defined in [this repo](https://github.com/socd06/medical-nlp).


# PsychBERT
This domain adapted language model is pretrained from the `bert-base-cased` checkpoint on masked language modeling, using a dataset of ~40,000 PubMed papers in the domain of psychology, psychiatry, mental health, and behavioral health; as well as a dastaset of roughly 200,000 social media conversations about mental health. This work is submitted as an entry for BIBM 2021.

**Note**: the token-prediction widget on this page does not work with Flax models. In order to use the model, please pull it into a Python session as follows:

```
from transformers import FlaxAutoModelForMaskedLM, AutoModelForMaskedLM

# load as a flax model
flax_lm = FlaxAutoModelForMaskedLM.from_pretrained('mnaylor/psychbert-cased')

# load as a pytorch model
# requires flax to be installed in your environment
pytorch_lm = AutoModelForMaskedLM.from_pretrained('mnaylor/psychbert-cased', from_flax=True)
```

Authors: Vedant Vajre, Mitch Naylor, Uday Kamath, Amarda Shehu


---
language: ca
datasets:
- common_voice
- parlament_parla
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- xlsr-fine-tuning-week
license: apache-2.0
model-index:
- name: Catalan XLSR Wav2Vec2 Large
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    datasets:
      - name: Common Voice ca
        type: common_voice
        args: ca
      - name: ParlamentParla
        url: https://www.openslr.org/59/
    metrics:
       - name: Test WER
         type: wer
         value: 6.92
       - name: Google Crowsourced Corpus WER
         type: wer
         value: 12.99
       - name: Audiobook “La llegenda de Sant Jordi” WER
         type: wer
         value: 13.23
---

# Wav2Vec2-Large-XLSR-Català

Fine-tuned [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) on Catalan language using the [Common Voice](https://huggingface.co/datasets/common_voice) and [ParlamentParla](https://www.openslr.org/59/) datasets.

**Attention:** The split train/dev/test used does not fully map with the CommonVoice 6.1 dataset. A custom split was used combining both the CommonVoice and ParlamentParla dataset and can be found [here](https://github.com/ccoreilly/wav2vec2-catala). Evaluating on the CV test dataset will produce a biased WER as 1144 audio files of that dataset were used in training/evaluation of this model.
WER was calculated using this [test.csv](https://github.com/ccoreilly/wav2vec2-catala/blob/master/test.csv) which was not seen by the model during training/evaluation.

You can find training and evaluation scripts in the github repository [ccoreilly/wav2vec2-catala](https://github.com/ccoreilly/wav2vec2-catala)

When using this model, make sure that your speech input is sampled at 16kHz.

## Results

Word error rate was evaluated on the following datasets unseen by the model:

| Dataset | WER |
| ------- | --- |
| [Test split CV+ParlamentParla]((https://github.com/ccoreilly/wav2vec2-catala/blob/master/test.csv)) | 6.92% |
| [Google Crowsourced Corpus](https://www.openslr.org/69/) | 12.99% |
| Audiobook “La llegenda de Sant Jordi” | 13.23% |

## Usage

The model can be used directly (without a language model) as follows:

```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ca", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("ccoreilly/wav2vec2-large-xlsr-catala")
model = Wav2Vec2ForCTC.from_pretrained("ccoreilly/wav2vec2-large-xlsr-catala")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


---
language: ca
datasets:
- common_voice
- parlament_parla
metrics:
- wer
tags:
- audio
- automatic-speech-recognition
- speech
- speech-to-text
license: apache-2.0
model-index:
- name: Catalan VoxPopuli Wav2Vec2 Large
  results:
  - task:
      name: Speech Recognition
      type: automatic-speech-recognition
    datasets:
      - name: Common Voice ca
        type: common_voice
        args: ca
      - name: ParlamentParla
        url: https://www.openslr.org/59/
    metrics:
       - name: Test WER
         type: wer
         value: 5.98
       - name: Google Crowsourced Corpus WER
         type: wer
         value: 12.14
       - name: Audiobook “La llegenda de Sant Jordi” WER
         type: wer
         value: 12.02
---

# Wav2Vec2-Large-100k-VoxPopuli-Català

**⚠NOTICE⚠: THIS MODEL HAS BEEN MOVED TO THE FOLLOWING URL:**
https://huggingface.co/softcatala/wav2vec2-large-100k-voxpopuli-catala

Fine-tuned [facebook/wav2vec2-large-100k-voxpopuli](https://huggingface.co/facebook/wav2vec2-large-100k-voxpopuli) on Catalan language using the [Common Voice](https://huggingface.co/datasets/common_voice) and [ParlamentParla](https://www.openslr.org/59/) datasets.

**Attention:** The split train/dev/test used does not fully map with the CommonVoice 6.1 dataset. A custom split was used combining both the CommonVoice and ParlamentParla dataset and can be found [here](https://github.com/ccoreilly/wav2vec2-catala). Evaluating on the CV test dataset will produce a biased WER as 1144 audio files of that dataset were used in training/evaluation of this model.
WER was calculated using this [test.csv](https://github.com/ccoreilly/wav2vec2-catala/blob/master/test-filtered.csv) which was not seen by the model during training/evaluation.

You can find training and evaluation scripts in the github repository [ccoreilly/wav2vec2-catala](https://github.com/ccoreilly/wav2vec2-catala)

When using this model, make sure that your speech input is sampled at 16kHz.

## Results

Word error rate was evaluated on the following datasets unseen by the model:

| Dataset | WER |
| ------- | --- |
| [Test split CV+ParlamentParla]((https://github.com/ccoreilly/wav2vec2-catala/blob/master/test-filtered.csv)) | 5.98% |
| [Google Crowsourced Corpus](https://www.openslr.org/69/) | 12.14% |
| Audiobook “La llegenda de Sant Jordi” | 12.02% |

## Usage

The model can be used directly (without a language model) as follows:

```python
import torch
import torchaudio
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

test_dataset = load_dataset("common_voice", "ca", split="test[:2%]")

processor = Wav2Vec2Processor.from_pretrained("ccoreilly/wav2vec2-large-100k-voxpopuli-catala")
model = Wav2Vec2ForCTC.from_pretrained("ccoreilly/wav2vec2-large-100k-voxpopuli-catala")

resampler = torchaudio.transforms.Resample(48_000, 16_000)

# Preprocessing the datasets.
# We need to read the audio files as arrays
def speech_file_to_array_fn(batch):
	speech_array, sampling_rate = torchaudio.load(batch["path"])
	batch["speech"] = resampler(speech_array).squeeze().numpy()
	return batch

test_dataset = test_dataset.map(speech_file_to_array_fn)
inputs = processor(test_dataset["speech"][:2], sampling_rate=16_000, return_tensors="pt", padding=True)

with torch.no_grad():
	logits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits

predicted_ids = torch.argmax(logits, dim=-1)

print("Prediction:", processor.batch_decode(predicted_ids))
print("Reference:", test_dataset["sentence"][:2])
```


---

tags:

- conversational

---

Cave Johnson in town!


---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-base-finetuned-scitldr
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-base-finetuned-scitldr

This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0232
- Rouge1: 35.2134
- Rouge2: 16.8919
- Rougel: 30.8442
- Rougelsum: 30.9316
- Gen Len: 18.7981

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| 2.0533        | 1.0   | 996  | 2.0285          | 34.9774 | 16.6163 | 30.6177 | 30.7038   | 18.7981 |
| 2.0994        | 2.0   | 1992 | 2.0232          | 35.2134 | 16.8919 | 30.8442 | 30.9316   | 18.7981 |

### Framework versions

- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


This model generate the time shift's text of Norbit Company also generate the same ending of the textes of any phrases like base gpt model.


---
tags:
- image-classification
- pytorch
- huggingpics
metrics:
- accuracy

model-index:
- name: dogs
  results:
  - task:
      name: Image Classification
      type: image-classification
    metrics:
      - name: Accuracy
        type: accuracy
        value: 1.0
---

# dogs

Autogenerated by HuggingPics🤗🖼

## Example Images

#### golden retriever

![golden retriever](images/golden_retriever.jpg)

#### great dane

![great dane](images/great_dane.jpg)

#### husky

![husky](images/husky.jpg)


### Model information

    Fine tuning data 1: https://www.kaggle.com/andradaolteanu/rickmorty-scripts
    Fine tuning data 2: https://www.kaggle.com/mikhailgaerlan/spongebob-squarepants-completed-transcripts
    Base model: e-tony/gpt2-rnm
    Epoch: 2
    Train runtime: 790.0612 secs
    Loss: 2.8569

API page: [Ainize](https://ainize.ai/fpem123/GPT2-Rick-N-Morty-with-SpongeBob?branch=master)

Demo page: [End-point](https://master-gpt2-rick-n-morty-with-sponge-bob-fpem123.endpoint.ainize.ai/)

### ===Teachable NLP=== ###

To train a GPT-2 model, write code and require GPU resources, but can easily fine-tune and get an API to use the model here for free.

Teachable NLP: [Teachable NLP](https://ainize.ai/teachable-nlp)

Tutorial: [Tutorial](https://forum.ainetwork.ai/t/teachable-nlp-how-to-use-teachable-nlp/65?utm_source=community&utm_medium=huggingface&utm_campaign=model&utm_content=teachable%20nlp)


### Model information

    Fine tuning data 1: https://www.kaggle.com/andradaolteanu/rickmorty-scripts
    Base model: e-tony/gpt2-rnm
    Epoch: 1
    Train runtime: 3.4982 secs
    Loss: 3.0894

Training notebook: [Colab](https://colab.research.google.com/drive/1RawVxulLETFicWMY0YANUdP-H-e7Eeyc)

### ===Teachable NLP=== ###

To train a GPT-2 model, write code and require GPU resources, but can easily fine-tune and get an API to use the model here for free.

Teachable NLP: [Teachable NLP](https://ainize.ai/teachable-nlp)

Tutorial: [Tutorial](https://forum.ainetwork.ai/t/teachable-nlp-how-to-use-teachable-nlp/65?utm_source=community&utm_medium=huggingface&utm_campaign=model&utm_content=teachable%20nlp)


### Model information

    Fine tuning data 1: https://www.kaggle.com/andradaolteanu/rickmorty-scripts
    Base model: e-tony/gpt2-rnm
    Epoch: 3
    Train runtime: 7.1779 secs
    Loss: 2.5694

Training notebook: [Colab](https://colab.research.google.com/drive/12NvO1SIZevF8ybJqfN9O21I3i9bU1dOO#scrollTo=KUsyn02WWmf5)

### ===Teachable NLP=== ###

To train a GPT-2 model, write code and require GPU resources, but can easily fine-tune and get an API to use the model here for free.

Teachable NLP: [Teachable NLP](https://ainize.ai/teachable-nlp)

Tutorial: [Tutorial](https://forum.ainetwork.ai/t/teachable-nlp-how-to-use-teachable-nlp/65?utm_source=community&utm_medium=huggingface&utm_campaign=model&utm_content=teachable%20nlp)


---
language: ko
tags:
- bert
- mrc
datasets:
- klue
license: cc-by-sa-4.0
---

# bert-base for QA

**Code:**  See [Ainize Workspace](https://link.ainize.ai/3FjvBVn)

**klue-bert-base-mrc DEMO**: [Ainize DEMO](https://main-klue-mrc-bert-scy6500.endpoint.ainize.ai/)

**klue-bert-base-mrc API**: [Ainize API](https://ainize.ai/scy6500/KLUE-MRC-BERT?branch=main)

## Overview
**Language model:** klue/bert-base
**Language:** Korean
**Downstream-task:** Extractive QA
**Training data:** KLUE-MRC
**Eval data:** KLUE-MRC

## Usage
### In Transformers
```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ainize/klue-bert-base-mrc")
model = AutoModelForQuestionAnswering.from_pretrained("ainize/klue-bert-base-mrc")

context = "your context"
question = "your question"

encodings = tokenizer(context, question, max_length=512, truncation=True,
                      padding="max_length", return_token_type_ids=False)
encodings = {key: torch.tensor([val]) for key, val in encodings.items()}

input_ids = encodings["input_ids"]
attention_mask = encodings["attention_mask"]

pred = model(input_ids, attention_mask=attention_mask)

start_logits, end_logits = pred.start_logits, pred.end_logits

token_start_index, token_end_index = start_logits.argmax(dim=-1), end_logits.argmax(dim=-1)

pred_ids = input_ids[0][token_start_index: token_end_index + 1]

prediction = tokenizer.decode(pred_ids)
```

## About us
[Teachable NLP](https://ainize.ai/teachable-nlp) - Train NLP models with your own text without writing any code
[Ainize](https://ainize.ai/) - Deploy ML project using free gpu


# bert-base for KLUE Relation Extraction task.
Fine-tuned klue/bert-base using KLUE RE dataset.
- KLUE Benchmark Official Webpage
- KLUE Official Github
- KLUE RE Github
- Run KLUE RE on free GPU : Ainize Workspace

# Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("ainize/klue-bert-base-re")
model = AutoModelForSequenceClassification.from_pretrained("ainize/klue-bert-base-re")

# Add "&ltsubj&gt", "&lt/subj&gt" to both ends of the subject object and "&ltobj&gt", "&lt/obj&gt" to both ends of the object object.
sentence = "&ltsubj&gt손흥민&lt/subj&gt은 &ltobj&gt대한민국&lt/obj&gt에서 태어났다."

encodings = tokenizer(sentence,
                      max_length=128,
                      truncation=True,
                      padding="max_length",
                      return_tensors="pt")

outputs = model(**encodings)

logits = outputs['logits']

preds = torch.argmax(logits, dim=1)

# About us
- Teachable NLP - Train NLP models with your own text without writing any code
- Ainize - Deploy ML project using free gpu


---
license: apache-2.0
---

Original repository : <https://huggingface.co/EleutherAI/gpt-j-6B>


### Model information

    Fine tuning data: https://www.kaggle.com/mikhailgaerlan/spongebob-squarepants-completed-transcripts
    License: CC-BY-SA
    Base model: gpt-2 large
    Epoch: 50
    Train runtime: 14723.0716 secs
    Loss: 0.0268

API page: [Ainize](https://ainize.ai/fpem123/GPT2-Spongebob?branch=master)

Demo page: [End-point](https://master-gpt2-spongebob-fpem123.endpoint.ainize.ai/)

### ===Teachable NLP=== ###

To train a GPT-2 model, write code and require GPU resources, but can easily fine-tune and get an API to use the model here for free.

Teachable NLP: [Teachable NLP](https://ainize.ai/teachable-nlp)

Tutorial: [Tutorial](https://forum.ainetwork.ai/t/teachable-nlp-how-to-use-teachable-nlp/65?utm_source=community&utm_medium=huggingface&utm_campaign=model&utm_content=teachable%20nlp)