|
--- |
|
datasets: |
|
- multi_nli |
|
- snli |
|
- scitail |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
- f1 |
|
pipeline_tag: zero-shot-classification |
|
--- |
|
# RoBERTa NLI (Natural Language Inference) |
|
This model is a fine-tuned model of [roberta-large](https://huggingface.co/roberta-large) after being trained on a **mixture of NLI datasets**. |
|
|
|
This model can classify a pair of sentence (a <u>premise</u> and a <u>claim</u>) into 3 classes: |
|
- 'entailment': the claim can logically be inferred from the premise |
|
- 'contradiction': the claim contradicts the premise |
|
- 'neutral': the premise is unrelated or do not provide sufficient information to validate the claim |
|
|
|
This model can also be used for **zero-shot classification tasks** ! |
|
Please take a look at this [repo](https://github.com/AntoineBlanot/zero-nlp) for more information on zero-shot classification tasks. |
|
|
|
# Usage |
|
This model has been trained in an efficient way and thus cannot be load directly from HuggingFace's hub. To use that model, please follow instructions on this [repo](https://github.com/AntoineBlanot/efficient-llm). |
|
|
|
For **zero-shot classification** tasks, please take a look at this [repo](https://github.com/AntoineBlanot/zero-nlp). |
|
|
|
# Data used for training |
|
- multi_nli |
|
- snli |
|
- scitail |
|
|
|
# Evaluation results |
|
|
|
| Data | Accuracy | |
|
|:---:|:---------:| |
|
| MNLI (val. m) | 0.894 | |
|
| MNLI (val. mm) | 0.895 | |
|
| SNLI (val.) | 0.920 | |
|
| SciTail (val.) | 0.934 | |
