Hugging Face's logo

richie-ghost
/
setfit-FacebookAI-roberta-large-phatic

Text Classification
setfit
Safetensors
sentence-transformers
roberta
generated_from_setfit_trainer
Eval Results
Model card Files Community
Edit model card

SetFit with FacebookAI/roberta-large

This is a SetFit model that can be used for Text Classification. This SetFit model uses FacebookAI/roberta-large as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
true
  • 'See you soon!'
  • 'You look well!'
  • 'Your journey is quite inspiring, can you share more about it?'
false
  • 'What are the core components of your business model?'
  • 'How do you balance your personal and professional life?'
  • "There is a situation where a daughter of a narcissistic mother denigrated the father. When the mother complained to the daughter about the father and how poor he was a a husband and person and how badly he treated the wife. The mother's claims were inaccurate and overblown. The mother said I inappropriate things to the daughter such as he flirted with other women, or the mother could have done much better than marrying him. After such episodes, the daughter was dismissive and rude to the father. What are the signs of parental alienation and what are the impacts on a daughter growing up and as an adult?"

Evaluation

Metrics

Label Accuracy
all 0.96

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("richie-ghost/setfit-FacebookAI-roberta-large-phatic")
# Run inference
preds = model("Take it easy!")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 9.8722 108
Label Training Sample Count
false 191
true 169

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (4, 4)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0002 1 0.4745 -
0.0122 50 0.441 -
0.0245 100 0.4422 -
0.0367 150 0.2339 -
0.0489 200 0.1182 -
0.0612 250 0.0806 -
0.0734 300 0.1183 -
0.0856 350 0.0551 -
0.0978 400 0.0146 -
0.1101 450 0.0115 -
0.1223 500 0.0042 -
0.1345 550 0.0053 -
0.1468 600 0.0021 -
0.1590 650 0.0596 -
0.1712 700 0.0029 -
0.1835 750 0.0009 -
0.1957 800 0.0002 -
0.2079 850 0.0005 -
0.2202 900 0.0013 -
0.2324 950 0.0008 -
0.2446 1000 0.0004 -
0.2568 1050 0.0004 -
0.2691 1100 0.0004 -
0.2813 1150 0.0003 -
0.2935 1200 0.0003 -
0.3058 1250 0.0012 -
0.3180 1300 0.0001 -
0.3302 1350 0.0002 -
0.3425 1400 0.0003 -
0.3547 1450 0.0024 -
0.3669 1500 0.0008 -
0.3792 1550 0.0015 -
0.3914 1600 0.0002 -
0.4036 1650 0.0002 -
0.4159 1700 0.1842 -
0.4281 1750 0.0009 -
0.4403 1800 0.0001 -
0.4525 1850 0.0013 -
0.4648 1900 0.0637 -
0.4770 1950 0.0002 -
0.4892 2000 0.0007 -
0.5015 2050 0.0001 -
0.5137 2100 0.0 -
0.5259 2150 0.0 -
0.5382 2200 0.0 -
0.5504 2250 0.0 -
0.5626 2300 0.0001 -
0.5749 2350 0.0 -
0.5871 2400 0.0 -
0.5993 2450 0.0 -
0.6115 2500 0.0 -
0.6238 2550 0.0 -
0.6360 2600 0.0 -
0.6482 2650 0.0 -
0.6605 2700 0.0001 -
0.6727 2750 0.0 -
0.6849 2800 0.0 -
0.6972 2850 0.0 -
0.7094 2900 0.0 -
0.7216 2950 0.0 -
0.7339 3000 0.0 -
0.7461 3050 0.0 -
0.7583 3100 0.0001 -
0.7705 3150 0.0 -
0.7828 3200 0.0 -
0.7950 3250 0.0 -
0.8072 3300 0.0 -
0.8195 3350 0.0 -
0.8317 3400 0.0 -
0.8439 3450 0.0001 -
0.8562 3500 0.0 -
0.8684 3550 0.0 -
0.8806 3600 0.0 -
0.8929 3650 0.0 -
0.9051 3700 0.0 -
0.9173 3750 0.0 -
0.9295 3800 0.0 -
0.9418 3850 0.0 -
0.9540 3900 0.0 -
0.9662 3950 0.0 -
0.9785 4000 0.0 -
0.9907 4050 0.0 -
1.0 4088 - 0.0815
1.0029 4100 0.0 -
1.0152 4150 0.0 -
1.0274 4200 0.0 -
1.0396 4250 0.0 -
1.0519 4300 0.0 -
1.0641 4350 0.0 -
1.0763 4400 0.0 -
1.0886 4450 0.0 -
1.1008 4500 0.0 -
1.1130 4550 0.0 -
1.1252 4600 0.0 -
1.1375 4650 0.0 -
1.1497 4700 0.0 -
1.1619 4750 0.0 -
1.1742 4800 0.0 -
1.1864 4850 0.0 -
1.1986 4900 0.0 -
1.2109 4950 0.0 -
1.2231 5000 0.0 -
1.2353 5050 0.0 -
1.2476 5100 0.0 -
1.2598 5150 0.0 -
1.2720 5200 0.0 -
1.2842 5250 0.0 -
1.2965 5300 0.0 -
1.3087 5350 0.0 -
1.3209 5400 0.0 -
1.3332 5450 0.0 -
1.3454 5500 0.0 -
1.3576 5550 0.0 -
1.3699 5600 0.0 -
1.3821 5650 0.0 -
1.3943 5700 0.0 -
1.4066 5750 0.0 -
1.4188 5800 0.0 -
1.4310 5850 0.0 -
1.4432 5900 0.0 -
1.4555 5950 0.0 -
1.4677 6000 0.0 -
1.4799 6050 0.0 -
1.4922 6100 0.0 -
1.5044 6150 0.0112 -
1.5166 6200 0.4712 -
1.5289 6250 0.3977 -
1.5411 6300 0.2112 -
1.5533 6350 0.318 -
1.5656 6400 0.2523 -
1.5778 6450 0.2829 -
1.5900 6500 0.2736 -
1.6023 6550 0.2493 -
1.6145 6600 0.3112 -
1.6267 6650 0.2291 -
1.6389 6700 0.2855 -
1.6512 6750 0.2642 -
1.6634 6800 0.2376 -
1.6756 6850 0.2983 -
1.6879 6900 0.2853 -
1.7001 6950 0.3095 -
1.7123 7000 0.2497 -
1.7246 7050 0.2305 -
1.7368 7100 0.2433 -
1.7490 7150 0.2505 -
1.7613 7200 0.2292 -
1.7735 7250 0.3028 -
1.7857 7300 0.2394 -
1.7979 7350 0.2601 -
1.8102 7400 0.2417 -
1.8224 7450 0.2086 -
1.8346 7500 0.2573 -
1.8469 7550 0.2344 -
1.8591 7600 0.2381 -
1.8713 7650 0.2772 -
1.8836 7700 0.2614 -
1.8958 7750 0.2659 -
1.9080 7800 0.2536 -
1.9203 7850 0.2385 -
1.9325 7900 0.2695 -
1.9447 7950 0.2512 -
1.9569 8000 0.2216 -
1.9692 8050 0.2291 -
1.9814 8100 0.2443 -
1.9936 8150 0.2579 -
2.0 8176 - 0.5
2.0059 8200 0.2605 -
2.0181 8250 0.2528 -
2.0303 8300 0.2361 -
2.0426 8350 0.2891 -
2.0548 8400 0.2692 -
2.0670 8450 0.25 -
2.0793 8500 0.2362 -
2.0915 8550 0.2833 -
2.1037 8600 0.2698 -
2.1159 8650 0.2195 -
2.1282 8700 0.2621 -
2.1404 8750 0.2564 -
2.1526 8800 0.2657 -
2.1649 8850 0.2629 -
2.1771 8900 0.2503 -
2.1893 8950 0.2583 -
2.2016 9000 0.2694 -
2.2138 9050 0.2824 -
2.2260 9100 0.2675 -
2.2383 9150 0.2699 -
2.2505 9200 0.2515 -
2.2627 9250 0.2511 -
2.2750 9300 0.2518 -
2.2872 9350 0.2555 -
2.2994 9400 0.2512 -
2.3116 9450 0.2374 -
2.3239 9500 0.2546 -
2.3361 9550 0.2846 -
2.3483 9600 0.2617 -
2.3606 9650 0.2474 -
2.3728 9700 0.2454 -
2.3850 9750 0.2265 -
2.3973 9800 0.2272 -
2.4095 9850 0.2442 -
2.4217 9900 0.236 -
2.4340 9950 0.2382 -
2.4462 10000 0.2645 -
2.4584 10050 0.2707 -
2.4706 10100 0.2573 -
2.4829 10150 0.2435 -
2.4951 10200 0.2705 -
2.5073 10250 0.2808 -
2.5196 10300 0.2581 -
2.5318 10350 0.2544 -
2.5440 10400 0.2333 -
2.5563 10450 0.2544 -
2.5685 10500 0.2497 -
2.5807 10550 0.2575 -
2.5930 10600 0.2382 -
2.6052 10650 0.2451 -
2.6174 10700 0.2702 -
2.6296 10750 0.2569 -
2.6419 10800 0.249 -
2.6541 10850 0.2366 -
2.6663 10900 0.2278 -
2.6786 10950 0.2568 -
2.6908 11000 0.2721 -
2.7030 11050 0.2593 -
2.7153 11100 0.2439 -
2.7275 11150 0.2543 -
2.7397 11200 0.2478 -
2.7520 11250 0.2325 -
2.7642 11300 0.2538 -
2.7764 11350 0.2968 -
2.7886 11400 0.2505 -
2.8009 11450 0.2377 -
2.8131 11500 0.2547 -
2.8253 11550 0.2529 -
2.8376 11600 0.2502 -
2.8498 11650 0.2293 -
2.8620 11700 0.2676 -
2.8743 11750 0.2371 -
2.8865 11800 0.2495 -
2.8987 11850 0.2937 -
2.9110 11900 0.2355 -
2.9232 11950 0.2482 -
2.9354 12000 0.2336 -
2.9477 12050 0.2344 -
2.9599 12100 0.257 -
2.9721 12150 0.2557 -
2.9843 12200 0.2854 -
2.9966 12250 0.2455 -
3.0 12264 - 0.5
3.0088 12300 0.2323 -
3.0210 12350 0.2566 -
3.0333 12400 0.2319 -
3.0455 12450 0.2552 -
3.0577 12500 0.2796 -
3.0700 12550 0.2823 -
3.0822 12600 0.2303 -
3.0944 12650 0.2448 -
3.1067 12700 0.2502 -
3.1189 12750 0.2516 -
3.1311 12800 0.2537 -
3.1433 12850 0.251 -
3.1556 12900 0.2639 -
3.1678 12950 0.2321 -
3.1800 13000 0.282 -
3.1923 13050 0.2577 -
3.2045 13100 0.2448 -
3.2167 13150 0.2352 -
3.2290 13200 0.281 -
3.2412 13250 0.2337 -
3.2534 13300 0.268 -
3.2657 13350 0.261 -
3.2779 13400 0.2378 -
3.2901 13450 0.2588 -
3.3023 13500 0.266 -
3.3146 13550 0.2604 -
3.3268 13600 0.2202 -
3.3390 13650 0.2217 -
3.3513 13700 0.2464 -
3.3635 13750 0.2684 -
3.3757 13800 0.2279 -
3.3880 13850 0.2379 -
3.4002 13900 0.2741 -
3.4124 13950 0.2713 -
3.4247 14000 0.2581 -
3.4369 14050 0.2638 -
3.4491 14100 0.2125 -
3.4614 14150 0.2348 -
3.4736 14200 0.2253 -
3.4858 14250 0.2627 -
3.4980 14300 0.2463 -
3.5103 14350 0.2533 -
3.5225 14400 0.2422 -
3.5347 14450 0.2296 -
3.5470 14500 0.2532 -
3.5592 14550 0.2733 -
3.5714 14600 0.2258 -
3.5837 14650 0.2253 -
3.5959 14700 0.2388 -
3.6081 14750 0.2217 -
3.6204 14800 0.3033 -
3.6326 14850 0.2349 -
3.6448 14900 0.2596 -
3.6570 14950 0.2415 -
3.6693 15000 0.2494 -
3.6815 15050 0.2826 -
3.6937 15100 0.2633 -
3.7060 15150 0.2636 -
3.7182 15200 0.2351 -
3.7304 15250 0.264 -
3.7427 15300 0.2652 -
3.7549 15350 0.2724 -
3.7671 15400 0.2731 -
3.7794 15450 0.2825 -
3.7916 15500 0.2611 -
3.8038 15550 0.2574 -
3.8160 15600 0.261 -
3.8283 15650 0.219 -
3.8405 15700 0.2323 -
3.8527 15750 0.2442 -
3.8650 15800 0.2509 -
3.8772 15850 0.26 -
3.8894 15900 0.2475 -
3.9017 15950 0.2452 -
3.9139 16000 0.2598 -
3.9261 16050 0.2377 -
3.9384 16100 0.2445 -
3.9506 16150 0.2451 -
3.9628 16200 0.2714 -
3.9750 16250 0.2755 -
3.9873 16300 0.2579 -
3.9995 16350 0.2338 -
4.0 16352 - 0.5
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.7.0
  • Transformers: 4.40.0
  • PyTorch: 2.2.1+cu121
  • Datasets: 2.19.0
  • Tokenizers: 0.19.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
7
Safetensors
Model size
355M params
Tensor type
F32
·

Finetuned from

Evaluation results

View on Papers With Code