SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
Model Sources
Model Labels
| Label |
Examples |
| 4 |
- "Ringing does become a bit of an obsession ,
admits Stephanie Pattenden , master of the band at St. Mary Abbot and one of England 's best female ringers . "</li><li>' All this may not be obvious to the public , which is concerned about advances in treatment , but I am convinced this basic research will begin showing results there soon . `` ' - 'So the driver started to curse at both of them as if they had been in a plot together to ruin his safe-driving record . '
|
| 5 |
- "But is n't the desire for profit the driving force behind those who subscribe to , and advertise in , your paper ? "
- 'Finally , the committee of discipline of the UEFA has sanctioned with two games the trainer of the team , Jose Mourinho , and with different economic amounts the implied players . '
- "In their filing , they referred to the Supreme Court 's 1966 decision overturning Dr. Sheppard 's conviction because of the trial coverage and made it clear they would claim that the new media required new rules . "
|
| 1 |
- 'We see that the peak of the Spanish selection is the goalkeeper Iker Casillas , who last Saturday against England ( game lost with 0-1 in London ) equaled the record of the ex goalkeeper Andoni Zubizarreta , with 126 international matches . '
- 'The one exception to this recent trend was the defeat of 13 of the 52 freshman Republicans brought into office in 1980 by the Reagan revolution and running for re-election in 1982 . '
- 'When the light went their way , they went on across the street . '
|
| 2 |
- 'This is a Johnson-era , Great Society creation that mandates certain government contracts be awarded noncompetitively to minority businesses . '
- 'Today , we know that the accumulation of several of these altered genes can initiate a cancer and , then , propel it into a deadly state .
'</li><li>'Let us sum up what we do know about education and about those education reforms that do work and do not work : -- Parental involvement `` is a bad idea . '
|
| 3 |
- 'Far above in the belfry , the huge bronze bells , mounted on wheels , swing madly through a full 360 degrees , starting and ending , surprisingly , in the inverted , or mouth-up position . '
- 'There you have a list of the declared variables . '
- 'Medical scientists are starting to uncover a handful of genes which , if damaged , unleash the chaotic growth of cells that characterizes cancer . '
|
| 0 |
- 'They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia . '
- 'Executes the instructions in a file sequentially . '
- 'In quoting from our research you emphasized the high prevalance of mental illness and alcoholism . '
|
| 6 |
- 'It often is preceded by the development of polyps in the bowel , which in some cases become increasingly malignant in identifiable stages -- progressing from less severe to deadly -- as though a cascade of genetic damage might be occurring . '
- 'I always work it that way -- and always at a time when the customer has an alibi . '
- 'A person who is born with one defective copy of a suppressor gene , or in whom one copy is damaged early in life , is especially prone to cancer because he need only lose the other copy for a cancer to develop . '
|
Evaluation
Metrics
| Label |
Accuracy |
| all |
0.1547 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("HelgeKn/SemEval-multi-class-20")
preds = model("For follow-up treatment , the animal owner can administer the tablets to the dog . ")
Training Details
Training Set Metrics
| Training set |
Min |
Median |
Max |
| Word count |
4 |
26.5857 |
74 |
| Label |
Training Sample Count |
| 0 |
20 |
| 1 |
20 |
| 2 |
20 |
| 3 |
20 |
| 4 |
20 |
| 5 |
20 |
| 6 |
20 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (2, 2)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 20
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
| Epoch |
Step |
Training Loss |
Validation Loss |
| 0.0029 |
1 |
0.3312 |
- |
| 0.1429 |
50 |
0.264 |
- |
| 0.2857 |
100 |
0.2359 |
- |
| 0.4286 |
150 |
0.2107 |
- |
| 0.5714 |
200 |
0.2034 |
- |
| 0.7143 |
250 |
0.114 |
- |
| 0.8571 |
300 |
0.0381 |
- |
| 1.0 |
350 |
0.0395 |
- |
| 1.1429 |
400 |
0.013 |
- |
| 1.2857 |
450 |
0.0035 |
- |
| 1.4286 |
500 |
0.0028 |
- |
| 1.5714 |
550 |
0.0025 |
- |
| 1.7143 |
600 |
0.002 |
- |
| 1.8571 |
650 |
0.002 |
- |
| 2.0 |
700 |
0.0026 |
- |
Framework Versions
- Python: 3.9.13
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.36.0
- PyTorch: 2.1.1+cpu
- Datasets: 2.15.0
- Tokenizers: 0.15.0
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
