Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Monolingual Dutch Models for Zero-Shot Text CLassification
|
2 |
+
|
3 |
+
This family of Dutch models were finetuned on combined data from the (translated) snli (cite) and SICK-NL datasets (cite). They are intended to be used in zero-shot classification for Dutch through Huggingface Pipelines.
|
4 |
+
|
5 |
+
## The Models
|
6 |
+
|
7 |
+
| Base Model | Huggingface id (fine-tuned) |
|
8 |
+
|-------------------|---------------------|
|
9 |
+
| [BERTje](https://huggingface.co/GroNLP/bert-base-dutch-cased) | this model |
|
10 |
+
| [RobBERT V2](http://github.com/iPieter/robbert) | robbert-v2-dutch-snli |
|
11 |
+
| [RobBERTje](https://github.com/iPieter/robbertje) | robbertje-dutch-nli |
|
12 |
+
|
13 |
+
|
14 |
+
|
15 |
+
## How to use
|
16 |
+
|
17 |
+
While this family of models can be used for evaluating (monolingual) NLI datasets, it's primary intended use is zero-shot text classification in Dutch. In this setting, classification tasks are recast as NLI problems. Consider the following sentence pairing that can be used to simulate a sentiment classification problem:
|
18 |
+
|
19 |
+
- Premise: The food in this place was horrendous
|
20 |
+
- Hypothesis: This is a negative review
|
21 |
+
|
22 |
+
For more information on using Natural Language Inference models for zero-shot text classification, we refer to this(link) paper.
|
23 |
+
|
24 |
+
By default, all our models are fully compatible with the Huggingface pipeline for zero-shot classification. They can be downloaded and accessed through the following code:
|
25 |
+
|
26 |
+
|
27 |
+
```python
|
28 |
+
from transformers import pipeline
|
29 |
+
|
30 |
+
classifier = pipeline(
|
31 |
+
task="zero-shot-classification",
|
32 |
+
model='robbert-v2-dutch-base-snli'
|
33 |
+
)
|
34 |
+
|
35 |
+
|
36 |
+
text_piece = "Het eten in dit restaurant is heel lekker."
|
37 |
+
labels = ["positief", "negatief", "neutraal"]
|
38 |
+
template = "Het sentiment van deze review is {}"
|
39 |
+
|
40 |
+
predictions = classifier(text_piece,
|
41 |
+
labels,
|
42 |
+
multi_class=False,
|
43 |
+
hypothesis_template=template
|
44 |
+
)
|
45 |
+
```
|
46 |
+
|
47 |
+
|
48 |
+
## Model Performance
|
49 |
+
|
50 |
+
|
51 |
+
### Performance on NLI task
|
52 |
+
|
53 |
+
| Model | Accuracy [%] | F1 [%] |
|
54 |
+
|-------------------|--------------------------|--------------|
|
55 |
+
| BERTje-nli | 92.157 | 90.898 |
|
56 |
+
| RobBERT-v2-nli | 93.096 | 91.279 |
|
57 |
+
| RobBERTje-nli | **97.816** | **97.514** |
|
58 |
+
|
59 |
+
|
60 |
+
|
61 |
+
## Credits and citation
|
62 |
+
|
63 |
+
TBD
|
64 |
+
|