Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,24 @@ It should be noted that hypotheses and premises are randomly chosen between Engl
|
|
| 25 |
|
| 26 |
### Performance
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
# Zero-shot Classification
|
| 29 |
The primary appeal of training such models lies in their zero-shot classification performance. This means the model is capable of classifying any text with any label
|
| 30 |
without specific training. What sets the Bloomz-560m-NLI LLMs apart in this realm is their ability to model and extract information from significantly more complex
|
|
@@ -38,6 +56,15 @@ is the sentence we aim to classify.
|
|
| 38 |
|
| 39 |
### Performance
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
# How to use Bloomz-560m-NLI
|
| 42 |
|
| 43 |
```python
|
|
|
|
| 25 |
|
| 26 |
### Performance
|
| 27 |
|
| 28 |
+
| **class** | **precision (%)** | **f1-score (%)** | **support** |
|
| 29 |
+
| :----------------: | :---------------: | :--------------: | :---------: |
|
| 30 |
+
| **global** | 69.20 | 68.35 | 5,010 |
|
| 31 |
+
| **contradiction** | 63.66 | 70.60 | 1,670 |
|
| 32 |
+
| **entailment** | 73.45 | 73.01 | 1,670 |
|
| 33 |
+
| **neutral** | 70.75 | 61.45 | 1,670 |
|
| 34 |
+
|
| 35 |
+
### Benchmark
|
| 36 |
+
|
| 37 |
+
| **model** | **accuracy (%)** | **MCC (x100)** |
|
| 38 |
+
| :--------------: | :-----------: | :--------------: | :------------: |
|
| 39 |
+
| [cmarkea/distilcamembert-base-nli](https://huggingface.co/cmarkea/distilcamembert-base-nli) | 77.45 | 66.24 |
|
| 40 |
+
| [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) | 81.72 | 72.67 |
|
| 41 |
+
| [MoritzLaurer/mDeBERTa-v3-base-mnli-xnli](https://huggingface.co/MoritzLaurer/mDeBERTa-v3-base-mnli-xnli) | 83.43 | 75.15 |
|
| 42 |
+
| [cmarkea/bloomz-560m-nli](https://huggingface.co/cmarkea/bloomz-560m-nli) | 68.70 | 53.57 |
|
| 43 |
+
| [cmarkea/bloomz-3b-nli](https://huggingface.co/cmarkea/bloomz-3b-nli) | 81.08 | 71.66 |
|
| 44 |
+
| [cmarkea/bloomz-7b1-mt-nli](https://huggingface.co/cmarkea/bloomz-7b1-mt-nli) | 83.13 | 74.89 |
|
| 45 |
+
|
| 46 |
# Zero-shot Classification
|
| 47 |
The primary appeal of training such models lies in their zero-shot classification performance. This means the model is capable of classifying any text with any label
|
| 48 |
without specific training. What sets the Bloomz-560m-NLI LLMs apart in this realm is their ability to model and extract information from significantly more complex
|
|
|
|
| 56 |
|
| 57 |
### Performance
|
| 58 |
|
| 59 |
+
| **model** | **accuracy (%)** | **MCC (x100)** |
|
| 60 |
+
| :--------------: | :-----------: | :--------------: | :------------: |
|
| 61 |
+
| [cmarkea/distilcamembert-base-nli](https://huggingface.co/cmarkea/distilcamembert-base-nli) | 80.59 | 63.71 |
|
| 62 |
+
| [BaptisteDoyen/camembert-base-xnli](https://huggingface.co/BaptisteDoyen/camembert-base-xnli) | 86.37 | 73.74 |
|
| 63 |
+
| [MoritzLaurer/mDeBERTa-v3-base-mnli-xnli](https://huggingface.co/MoritzLaurer/mDeBERTa-v3-base-mnli-xnli) | 84.97 | 70.05 |
|
| 64 |
+
| [cmarkea/bloomz-560m-nli](https://huggingface.co/cmarkea/bloomz-560m-nli) | 71.13 | 46.3 |
|
| 65 |
+
| [cmarkea/bloomz-3b-nli](https://huggingface.co/cmarkea/bloomz-3b-nli) | 89.06 | 78.10 |
|
| 66 |
+
| [cmarkea/bloomz-7b1-mt-nli](https://huggingface.co/cmarkea/bloomz-7b1-mt-nli) | 95.12 | 90.27 |
|
| 67 |
+
|
| 68 |
# How to use Bloomz-560m-NLI
|
| 69 |
|
| 70 |
```python
|