SetFit with FacebookAI/roberta-base
This is a SetFit model that can be used for Text Classification. This SetFit model uses FacebookAI/roberta-base as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
Model Sources
Model Labels
Label |
Examples |
true |
- 'How can we apply your findings to optimize our processes?'
- 'Your presence at the meeting was greatly appreciated.'
- 'Your journey is quite inspiring, can you share more about it?'
|
false |
- 'What book are you currently reading?'
- 'It’s important to acknowledge your feelings, what’s been going through your mind?'
- 'You’ve been working hard on your mental health; how are you finding the journey?'
|
Evaluation
Metrics
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("richie-ghost/setfit-FacebookAI-roberta-base-phatic")
preds = model("Take it easy!")
Training Details
Training Set Metrics
Training set |
Min |
Median |
Max |
Word count |
1 |
9.8722 |
108 |
Label |
Training Sample Count |
false |
191 |
true |
169 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (4, 4)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch |
Step |
Training Loss |
Validation Loss |
0.0002 |
1 |
0.4475 |
- |
0.0122 |
50 |
0.4363 |
- |
0.0245 |
100 |
0.3668 |
- |
0.0367 |
150 |
0.177 |
- |
0.0489 |
200 |
0.0999 |
- |
0.0612 |
250 |
0.1043 |
- |
0.0734 |
300 |
0.0191 |
- |
0.0856 |
350 |
0.009 |
- |
0.0978 |
400 |
0.0028 |
- |
0.1101 |
450 |
0.0046 |
- |
0.1223 |
500 |
0.0012 |
- |
0.1345 |
550 |
0.0016 |
- |
0.1468 |
600 |
0.0012 |
- |
0.1590 |
650 |
0.0012 |
- |
0.1712 |
700 |
0.0164 |
- |
0.1835 |
750 |
0.025 |
- |
0.1957 |
800 |
0.0007 |
- |
0.2079 |
850 |
0.0013 |
- |
0.2202 |
900 |
0.0008 |
- |
0.2324 |
950 |
0.0005 |
- |
0.2446 |
1000 |
0.0004 |
- |
0.2568 |
1050 |
0.0002 |
- |
0.2691 |
1100 |
0.0004 |
- |
0.2813 |
1150 |
0.0003 |
- |
0.2935 |
1200 |
0.0002 |
- |
0.3058 |
1250 |
0.0002 |
- |
0.3180 |
1300 |
0.0003 |
- |
0.3302 |
1350 |
0.0002 |
- |
0.3425 |
1400 |
0.0001 |
- |
0.3547 |
1450 |
0.003 |
- |
0.3669 |
1500 |
0.0003 |
- |
0.3792 |
1550 |
0.0003 |
- |
0.3914 |
1600 |
0.0001 |
- |
0.4036 |
1650 |
0.0001 |
- |
0.4159 |
1700 |
0.0001 |
- |
0.4281 |
1750 |
0.0001 |
- |
0.4403 |
1800 |
0.0001 |
- |
0.4525 |
1850 |
0.0001 |
- |
0.4648 |
1900 |
0.0001 |
- |
0.4770 |
1950 |
0.0001 |
- |
0.4892 |
2000 |
0.0001 |
- |
0.5015 |
2050 |
0.0001 |
- |
0.5137 |
2100 |
0.0001 |
- |
0.5259 |
2150 |
0.0001 |
- |
0.5382 |
2200 |
0.0 |
- |
0.5504 |
2250 |
0.0 |
- |
0.5626 |
2300 |
0.0 |
- |
0.5749 |
2350 |
0.0001 |
- |
0.5871 |
2400 |
0.0 |
- |
0.5993 |
2450 |
0.0 |
- |
0.6115 |
2500 |
0.0001 |
- |
0.6238 |
2550 |
0.0001 |
- |
0.6360 |
2600 |
0.0 |
- |
0.6482 |
2650 |
0.0 |
- |
0.6605 |
2700 |
0.0 |
- |
0.6727 |
2750 |
0.0 |
- |
0.6849 |
2800 |
0.0 |
- |
0.6972 |
2850 |
0.0 |
- |
0.7094 |
2900 |
0.0001 |
- |
0.7216 |
2950 |
0.0001 |
- |
0.7339 |
3000 |
0.0 |
- |
0.7461 |
3050 |
0.0 |
- |
0.7583 |
3100 |
0.0006 |
- |
0.7705 |
3150 |
0.0606 |
- |
0.7828 |
3200 |
0.0 |
- |
0.7950 |
3250 |
0.0002 |
- |
0.8072 |
3300 |
0.0 |
- |
0.8195 |
3350 |
0.0001 |
- |
0.8317 |
3400 |
0.0001 |
- |
0.8439 |
3450 |
0.0 |
- |
0.8562 |
3500 |
0.0001 |
- |
0.8684 |
3550 |
0.0 |
- |
0.8806 |
3600 |
0.0 |
- |
0.8929 |
3650 |
0.0 |
- |
0.9051 |
3700 |
0.0 |
- |
0.9173 |
3750 |
0.0 |
- |
0.9295 |
3800 |
0.0 |
- |
0.9418 |
3850 |
0.0 |
- |
0.9540 |
3900 |
0.0 |
- |
0.9662 |
3950 |
0.0 |
- |
0.9785 |
4000 |
0.0 |
- |
0.9907 |
4050 |
0.0 |
- |
1.0 |
4088 |
- |
0.1621 |
1.0029 |
4100 |
0.0 |
- |
1.0152 |
4150 |
0.0 |
- |
1.0274 |
4200 |
0.0 |
- |
1.0396 |
4250 |
0.0 |
- |
1.0519 |
4300 |
0.0 |
- |
1.0641 |
4350 |
0.0 |
- |
1.0763 |
4400 |
0.0 |
- |
1.0886 |
4450 |
0.0 |
- |
1.1008 |
4500 |
0.0 |
- |
1.1130 |
4550 |
0.0 |
- |
1.1252 |
4600 |
0.0 |
- |
1.1375 |
4650 |
0.0 |
- |
1.1497 |
4700 |
0.0 |
- |
1.1619 |
4750 |
0.0 |
- |
1.1742 |
4800 |
0.0 |
- |
1.1864 |
4850 |
0.0 |
- |
1.1986 |
4900 |
0.0 |
- |
1.2109 |
4950 |
0.0 |
- |
1.2231 |
5000 |
0.0 |
- |
1.2353 |
5050 |
0.0 |
- |
1.2476 |
5100 |
0.0 |
- |
1.2598 |
5150 |
0.0 |
- |
1.2720 |
5200 |
0.0 |
- |
1.2842 |
5250 |
0.0 |
- |
1.2965 |
5300 |
0.0 |
- |
1.3087 |
5350 |
0.0 |
- |
1.3209 |
5400 |
0.0 |
- |
1.3332 |
5450 |
0.0 |
- |
1.3454 |
5500 |
0.0 |
- |
1.3576 |
5550 |
0.0 |
- |
1.3699 |
5600 |
0.0 |
- |
1.3821 |
5650 |
0.0 |
- |
1.3943 |
5700 |
0.0 |
- |
1.4066 |
5750 |
0.0 |
- |
1.4188 |
5800 |
0.0 |
- |
1.4310 |
5850 |
0.0 |
- |
1.4432 |
5900 |
0.0 |
- |
1.4555 |
5950 |
0.0 |
- |
1.4677 |
6000 |
0.0 |
- |
1.4799 |
6050 |
0.0 |
- |
1.4922 |
6100 |
0.0 |
- |
1.5044 |
6150 |
0.0 |
- |
1.5166 |
6200 |
0.0 |
- |
1.5289 |
6250 |
0.0 |
- |
1.5411 |
6300 |
0.0 |
- |
1.5533 |
6350 |
0.0 |
- |
1.5656 |
6400 |
0.0 |
- |
1.5778 |
6450 |
0.0 |
- |
1.5900 |
6500 |
0.0 |
- |
1.6023 |
6550 |
0.0 |
- |
1.6145 |
6600 |
0.0 |
- |
1.6267 |
6650 |
0.0 |
- |
1.6389 |
6700 |
0.0 |
- |
1.6512 |
6750 |
0.0 |
- |
1.6634 |
6800 |
0.0 |
- |
1.6756 |
6850 |
0.0 |
- |
1.6879 |
6900 |
0.0 |
- |
1.7001 |
6950 |
0.0 |
- |
1.7123 |
7000 |
0.0 |
- |
1.7246 |
7050 |
0.0 |
- |
1.7368 |
7100 |
0.0 |
- |
1.7490 |
7150 |
0.0 |
- |
1.7613 |
7200 |
0.0 |
- |
1.7735 |
7250 |
0.0 |
- |
1.7857 |
7300 |
0.0 |
- |
1.7979 |
7350 |
0.0 |
- |
1.8102 |
7400 |
0.0 |
- |
1.8224 |
7450 |
0.0 |
- |
1.8346 |
7500 |
0.0 |
- |
1.8469 |
7550 |
0.0 |
- |
1.8591 |
7600 |
0.0 |
- |
1.8713 |
7650 |
0.0 |
- |
1.8836 |
7700 |
0.0 |
- |
1.8958 |
7750 |
0.0 |
- |
1.9080 |
7800 |
0.0 |
- |
1.9203 |
7850 |
0.0 |
- |
1.9325 |
7900 |
0.0 |
- |
1.9447 |
7950 |
0.0 |
- |
1.9569 |
8000 |
0.0 |
- |
1.9692 |
8050 |
0.0 |
- |
1.9814 |
8100 |
0.0 |
- |
1.9936 |
8150 |
0.0 |
- |
2.0 |
8176 |
- |
0.1131 |
2.0059 |
8200 |
0.0 |
- |
2.0181 |
8250 |
0.0 |
- |
2.0303 |
8300 |
0.0 |
- |
2.0426 |
8350 |
0.0 |
- |
2.0548 |
8400 |
0.0 |
- |
2.0670 |
8450 |
0.0 |
- |
2.0793 |
8500 |
0.0 |
- |
2.0915 |
8550 |
0.0 |
- |
2.1037 |
8600 |
0.0 |
- |
2.1159 |
8650 |
0.0 |
- |
2.1282 |
8700 |
0.0 |
- |
2.1404 |
8750 |
0.0 |
- |
2.1526 |
8800 |
0.0 |
- |
2.1649 |
8850 |
0.0 |
- |
2.1771 |
8900 |
0.0 |
- |
2.1893 |
8950 |
0.0 |
- |
2.2016 |
9000 |
0.0 |
- |
2.2138 |
9050 |
0.0 |
- |
2.2260 |
9100 |
0.0 |
- |
2.2383 |
9150 |
0.0 |
- |
2.2505 |
9200 |
0.0 |
- |
2.2627 |
9250 |
0.0 |
- |
2.2750 |
9300 |
0.0 |
- |
2.2872 |
9350 |
0.0 |
- |
2.2994 |
9400 |
0.0 |
- |
2.3116 |
9450 |
0.0 |
- |
2.3239 |
9500 |
0.0 |
- |
2.3361 |
9550 |
0.0 |
- |
2.3483 |
9600 |
0.0 |
- |
2.3606 |
9650 |
0.0 |
- |
2.3728 |
9700 |
0.0 |
- |
2.3850 |
9750 |
0.0 |
- |
2.3973 |
9800 |
0.0 |
- |
2.4095 |
9850 |
0.0 |
- |
2.4217 |
9900 |
0.0 |
- |
2.4340 |
9950 |
0.0 |
- |
2.4462 |
10000 |
0.0 |
- |
2.4584 |
10050 |
0.0 |
- |
2.4706 |
10100 |
0.0 |
- |
2.4829 |
10150 |
0.0 |
- |
2.4951 |
10200 |
0.0 |
- |
2.5073 |
10250 |
0.0 |
- |
2.5196 |
10300 |
0.0 |
- |
2.5318 |
10350 |
0.0 |
- |
2.5440 |
10400 |
0.0 |
- |
2.5563 |
10450 |
0.0 |
- |
2.5685 |
10500 |
0.0 |
- |
2.5807 |
10550 |
0.0 |
- |
2.5930 |
10600 |
0.0 |
- |
2.6052 |
10650 |
0.0 |
- |
2.6174 |
10700 |
0.0 |
- |
2.6296 |
10750 |
0.0 |
- |
2.6419 |
10800 |
0.0 |
- |
2.6541 |
10850 |
0.0 |
- |
2.6663 |
10900 |
0.0 |
- |
2.6786 |
10950 |
0.0 |
- |
2.6908 |
11000 |
0.0 |
- |
2.7030 |
11050 |
0.0 |
- |
2.7153 |
11100 |
0.0 |
- |
2.7275 |
11150 |
0.0 |
- |
2.7397 |
11200 |
0.0 |
- |
2.7520 |
11250 |
0.0 |
- |
2.7642 |
11300 |
0.0 |
- |
2.7764 |
11350 |
0.0 |
- |
2.7886 |
11400 |
0.0 |
- |
2.8009 |
11450 |
0.0 |
- |
2.8131 |
11500 |
0.0 |
- |
2.8253 |
11550 |
0.0 |
- |
2.8376 |
11600 |
0.0 |
- |
2.8498 |
11650 |
0.0 |
- |
2.8620 |
11700 |
0.0 |
- |
2.8743 |
11750 |
0.0 |
- |
2.8865 |
11800 |
0.0 |
- |
2.8987 |
11850 |
0.0 |
- |
2.9110 |
11900 |
0.0 |
- |
2.9232 |
11950 |
0.0 |
- |
2.9354 |
12000 |
0.0 |
- |
2.9477 |
12050 |
0.0 |
- |
2.9599 |
12100 |
0.0 |
- |
2.9721 |
12150 |
0.0 |
- |
2.9843 |
12200 |
0.0 |
- |
2.9966 |
12250 |
0.0 |
- |
3.0 |
12264 |
- |
0.1127 |
3.0088 |
12300 |
0.0 |
- |
3.0210 |
12350 |
0.0 |
- |
3.0333 |
12400 |
0.0 |
- |
3.0455 |
12450 |
0.0 |
- |
3.0577 |
12500 |
0.0 |
- |
3.0700 |
12550 |
0.0 |
- |
3.0822 |
12600 |
0.0 |
- |
3.0944 |
12650 |
0.0 |
- |
3.1067 |
12700 |
0.0 |
- |
3.1189 |
12750 |
0.0 |
- |
3.1311 |
12800 |
0.0 |
- |
3.1433 |
12850 |
0.0 |
- |
3.1556 |
12900 |
0.0 |
- |
3.1678 |
12950 |
0.0 |
- |
3.1800 |
13000 |
0.0 |
- |
3.1923 |
13050 |
0.0 |
- |
3.2045 |
13100 |
0.0 |
- |
3.2167 |
13150 |
0.0 |
- |
3.2290 |
13200 |
0.0 |
- |
3.2412 |
13250 |
0.0 |
- |
3.2534 |
13300 |
0.0 |
- |
3.2657 |
13350 |
0.0 |
- |
3.2779 |
13400 |
0.0 |
- |
3.2901 |
13450 |
0.0 |
- |
3.3023 |
13500 |
0.0 |
- |
3.3146 |
13550 |
0.0 |
- |
3.3268 |
13600 |
0.0 |
- |
3.3390 |
13650 |
0.0 |
- |
3.3513 |
13700 |
0.0 |
- |
3.3635 |
13750 |
0.0 |
- |
3.3757 |
13800 |
0.0 |
- |
3.3880 |
13850 |
0.0 |
- |
3.4002 |
13900 |
0.0 |
- |
3.4124 |
13950 |
0.0 |
- |
3.4247 |
14000 |
0.0 |
- |
3.4369 |
14050 |
0.0 |
- |
3.4491 |
14100 |
0.0 |
- |
3.4614 |
14150 |
0.0 |
- |
3.4736 |
14200 |
0.0 |
- |
3.4858 |
14250 |
0.0 |
- |
3.4980 |
14300 |
0.0 |
- |
3.5103 |
14350 |
0.0 |
- |
3.5225 |
14400 |
0.0 |
- |
3.5347 |
14450 |
0.0 |
- |
3.5470 |
14500 |
0.0 |
- |
3.5592 |
14550 |
0.0 |
- |
3.5714 |
14600 |
0.0 |
- |
3.5837 |
14650 |
0.0 |
- |
3.5959 |
14700 |
0.0 |
- |
3.6081 |
14750 |
0.0 |
- |
3.6204 |
14800 |
0.0 |
- |
3.6326 |
14850 |
0.0 |
- |
3.6448 |
14900 |
0.0 |
- |
3.6570 |
14950 |
0.0 |
- |
3.6693 |
15000 |
0.0 |
- |
3.6815 |
15050 |
0.0 |
- |
3.6937 |
15100 |
0.0 |
- |
3.7060 |
15150 |
0.0 |
- |
3.7182 |
15200 |
0.0 |
- |
3.7304 |
15250 |
0.0 |
- |
3.7427 |
15300 |
0.0 |
- |
3.7549 |
15350 |
0.0 |
- |
3.7671 |
15400 |
0.0 |
- |
3.7794 |
15450 |
0.0 |
- |
3.7916 |
15500 |
0.0 |
- |
3.8038 |
15550 |
0.0 |
- |
3.8160 |
15600 |
0.0 |
- |
3.8283 |
15650 |
0.0 |
- |
3.8405 |
15700 |
0.0 |
- |
3.8527 |
15750 |
0.0 |
- |
3.8650 |
15800 |
0.0 |
- |
3.8772 |
15850 |
0.0 |
- |
3.8894 |
15900 |
0.0 |
- |
3.9017 |
15950 |
0.0 |
- |
3.9139 |
16000 |
0.0 |
- |
3.9261 |
16050 |
0.0 |
- |
3.9384 |
16100 |
0.0 |
- |
3.9506 |
16150 |
0.0 |
- |
3.9628 |
16200 |
0.0 |
- |
3.9750 |
16250 |
0.0 |
- |
3.9873 |
16300 |
0.0 |
- |
3.9995 |
16350 |
0.0 |
- |
4.0 |
16352 |
- |
0.1019 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.0
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}