|
--- |
|
datasets: |
|
- cjvt/si_nli |
|
- jacinthes/slovene_mnli_snli |
|
language: |
|
- sl |
|
license: cc-by-sa-4.0 |
|
--- |
|
|
|
# CrossEncoder for Slovene NLI |
|
The model was trained using the [SentenceTransformers](https://sbert.net/) [CrossEncoder](https://www.sbert.net/examples/applications/cross-encoder/README.html) class. <br /> |
|
It is based on [SloBerta](https://huggingface.co/EMBEDDIA/sloberta), a monolingual Slovene model. |
|
|
|
## Training |
|
This model was trained on the [SI-NLI](https://huggingface.co/datasets/cjvt/si_nli) and the [slovene_mnli_snli](https://huggingface.co/datasets/jacinthes/slovene_mnli_snli) datasets.<br /> |
|
More details and the training script are available here: [repo](https://github.com/jacinthes/slovene-nli-benchmark) |
|
|
|
## Performance |
|
The model achieves the following metrics: |
|
- Test accuracy: 77.15 |
|
- Dev accuracy: 77.51 |
|
|
|
## Usage |
|
The model can be used for inference using the below code: |
|
```python |
|
from sentence_transformers import CrossEncoder |
|
|
|
model = CrossEncoder('jacinthes/cross-encoder-sloberta-si-nli-snli-mnli') |
|
premise = 'Pojdi z menoj v toplice.' |
|
hypothesis = 'Bova lepa bova fit.' |
|
prediction = model.predict([premise, hypothesis]) |
|
int2label = {0: 'entailment', 1: 'neutral', 2:'contradiction'} |
|
print(int2label[prediction.argmax()]) |
|
``` |