Hugging Face's logo

CrisisNarratives
/
setfit-13classes-single_label

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
Edit model card

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
5
  • 'please show us the evidence I asked for'
  • 'Please follow https://youtu.be/WpTCt-S-qLM ??'
  • 'Answer the question. The first is illegal in NY, the second is legal.'
7
  • 'This guy sounds like he needs to clear his throat'
  • 'very goiot'
  • 'Semen ???? what kind of name it is????'
9
  • 'Yeah I can do that.'
  • "I can totally accept that the government fund green energy, it's for the best."
  • 'Yes, he is right ! My Dr. did exactly what he is saying. He started antibiotics then 5 days started the steroids. Hopefully other dr will do the same.'
0
  • 'If only most senates in the US can see how climate change can not only effect our planets environment, it also our economys like theirs.'
  • '1st Comment!'
  • 'People moving interstate will reduce unemployment in SA.'
2
  • 'ok boomer'
  • 'I can see where ur coming from she did invite this cousin to live there which mind you is likely an act of kindness, and it is ur private space. But tbh you are also being extremely petty, this man is on the couch, he has early morning shifts, using ur bathroom would not disturb a single person, while using ur roommates means this man has to be very uncomfortable walk through a sleeping persons room every morning and sneak back out again. Likely waking up the cousin in the process. Thats why I believe its an Everyone Sucks here cause theyre both asshole like moves.'
  • 'suggesting the vaccine to women who are pregnant, when it can cause for many women earlier heavier longer periods, means it triggers a period, we know women in our personal lives that have experienced that along with their period coming twice that month, and we know to avoid foods that can trigger a period because that could potentially cause a miscarriage, so why suggest it?? this is how the whole world will lose confidence in science and medicine because they can down right lie to our face claiming they understand more then you meanwhile propagating the agenda of pharmaceutical companies. absaloutly disgusting and shame on you for betraying our trust.'
3
  • 'Nothing is hotter than Shawn, not even the sun mate??'
  • 'Incorrect. Geologists have not claimed that warming "historically has always been accompanied by a mass extinction event".'
  • 'No, it doesnt change anything in cats or in humans. Vaccines dont cause autism.'
6
  • 'So youre taking a government course?'
  • 'Is this journalist on work experience ?'
  • 'Who is here after the movie'
4
  • 'Forward looking required now, which the leaders are doing and doing their best, sleep deprived and a world of responsibility on them. Thanks to all'
  • "Oh, great, you could do that? That'd help me out really."
  • "Couldn't be more proud and happy that these heroes are finally taking a stand to the horrible ways the current government is pushing the country"
1
  • "Maybe STOP paying commanders who have NEVER even picked up a hose and give the power back to the brigades CAPTAIN'S. And things will start to get better, from someone who is actually doing something. Snowy mountains Australia."
  • "@Keith Bawden You have an education in science? Me too! Did you study in any field relevant to the topic? I am currently doing a PhD in biogeochemistry, and my research group is involved in climate science. I can tell you the VAST majority of scientists in fields directly related to or peripheral to climate change accept that it is indeed a real phenomenon, and it is caused by humans. The exceptions you can name are exactly that, exceptions. I'll grant you, Zarkhova may be right about a coming grand solar minimum, but even if so, all it would do is slightly slow temperature increase. There would be no mini ice age (Fuelner and Rahmstorf, 2010). The question is, in 2-3 years, when a 'mini ice age' does not occur, will you change your mind, or find some other reason to deny?"
  • "You should have gotten herd immunity in Changi, considering 95% efficacy of Pfizer and 80% or more are vaccinated.\r\nIt's either the efficacy is faked or the vaccine is useless against the indian variant."
10
  • 'No Im not trolling.'
  • "I'm not being hurtful. I'm being honest. You need to vaccinate your cat"
  • 'I said no such thing. The OP asked about equipment, not lifting. There are probably dozens, even hundreds, or submissions on here about lifting technique he can research.'
8
  • 'Maybe you could try drinking more tea instead of coffee.'
  • 'I think you should definitely take steroids. every single steroid you can find, take it every single day.'
  • 'The government should buy more masks for the public.'
11
  • '@Constantine Nikitin.I did it.'
  • "Ok, you're right, it was my fault, I shouldn't have said that."
  • 'Sure, I was wrong.'
12
  • "Things got out of hand, I'm sorry."
  • 'Oh no, I did not mean it that way, it was completely misunderstood what I was saying. Didnt mean to offend you, sorry!'
  • 'Sorry.'

Evaluation

Metrics

Label Metric
all 0.5465

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("CrisisNarratives/setfit-13classes-single_label")
# Run inference
preds = model("my dad had huge ones..so they may be real..")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 25.8891 1681
Label Training Sample Count
0 119
1 81
2 64
3 34
4 46
5 39
6 35
7 37
8 24
9 26
10 18
11 11
12 7

Training Hyperparameters

  • batch_size: (8, 8)
  • num_epochs: (4, 4)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 22
  • body_learning_rate: (1.698e-05, 1.698e-05)
  • head_learning_rate: 1.698e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 39
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0004 1 0.3701 -
0.0185 50 0.2605 -
0.0370 100 0.2727 -
0.0555 150 0.2389 -
0.0739 200 0.2466 -
0.0924 250 0.206 -
0.1109 300 0.2218 -
0.1294 350 0.1745 -
0.1479 400 0.2955 -
0.1664 450 0.1405 -
0.1848 500 0.202 -
0.2033 550 0.1614 -
0.2218 600 0.1953 -
0.2403 650 0.067 -
0.2588 700 0.0841 -
0.2773 750 0.0769 -
0.2957 800 0.0824 -
0.3142 850 0.0629 -
0.3327 900 0.0086 -
0.3512 950 0.0589 -
0.3697 1000 0.0469 -
0.3882 1050 0.0312 -
0.4067 1100 0.0597 -
0.4251 1150 0.0054 -
0.4436 1200 0.0029 -
0.4621 1250 0.0031 -
0.4806 1300 0.0638 -
0.4991 1350 0.0024 -
0.5176 1400 0.0023 -
0.5360 1450 0.0094 -
0.5545 1500 0.0017 -
0.5730 1550 0.0017 -
0.5915 1600 0.0371 -
0.6100 1650 0.0005 -
0.6285 1700 0.0014 -
0.6470 1750 0.0009 -
0.6654 1800 0.0103 -
0.6839 1850 0.0035 -
0.7024 1900 0.0007 -
0.7209 1950 0.0219 -
0.7394 2000 0.0014 -
0.7579 2050 0.0008 -
0.7763 2100 0.0007 -
0.7948 2150 0.0006 -
0.8133 2200 0.0054 -
0.8318 2250 0.0008 -
0.8503 2300 0.0008 -
0.8688 2350 0.0007 -
0.8872 2400 0.0007 -
0.9057 2450 0.001 -
0.9242 2500 0.0005 -
0.9427 2550 0.0005 -
0.9612 2600 0.0009 -
0.9797 2650 0.0003 -
0.9982 2700 0.0008 -
1.0166 2750 0.0006 -
1.0351 2800 0.0004 -
1.0536 2850 0.0002 -
1.0721 2900 0.001 -
1.0906 2950 0.0006 -
1.1091 3000 0.0007 -
1.1275 3050 0.001 -
1.1460 3100 0.0003 -
1.1645 3150 0.0004 -
1.1830 3200 0.0016 -
1.2015 3250 0.0016 -
1.2200 3300 0.0001 -
1.2384 3350 0.0001 -
1.2569 3400 0.0003 -
1.2754 3450 0.0002 -
1.2939 3500 0.0003 -
1.3124 3550 0.0003 -
1.3309 3600 0.0003 -
1.3494 3650 0.001 -
1.3678 3700 0.0002 -
1.3863 3750 0.0003 -
1.4048 3800 0.0002 -
1.4233 3850 0.0001 -
1.4418 3900 0.0003 -
1.4603 3950 0.0001 -
1.4787 4000 0.0002 -
1.4972 4050 0.0001 -
1.5157 4100 0.0001 -
1.5342 4150 0.0001 -
1.5527 4200 0.0003 -
1.5712 4250 0.0001 -
1.5896 4300 0.0003 -
1.6081 4350 0.0005 -
1.6266 4400 0.0002 -
1.6451 4450 0.0001 -
1.6636 4500 0.0001 -
1.6821 4550 0.0002 -
1.7006 4600 0.0001 -
1.7190 4650 0.0001 -
1.7375 4700 0.0002 -
1.7560 4750 0.0001 -
1.7745 4800 0.0 -
1.7930 4850 0.0002 -
1.8115 4900 0.0001 -
1.8299 4950 0.0001 -
1.8484 5000 0.0001 -
1.8669 5050 0.0001 -
1.8854 5100 0.0002 -
1.9039 5150 0.0001 -
1.9224 5200 0.0001 -
1.9409 5250 0.0 -
1.9593 5300 0.0001 -
1.9778 5350 0.0002 -
1.9963 5400 0.0001 -
2.0148 5450 0.0001 -
2.0333 5500 0.0002 -
2.0518 5550 0.0001 -
2.0702 5600 0.0003 -
2.0887 5650 0.0001 -
2.1072 5700 0.0002 -
2.1257 5750 0.0002 -
2.1442 5800 0.0001 -
2.1627 5850 0.0001 -
2.1811 5900 0.0001 -
2.1996 5950 0.0001 -
2.2181 6000 0.0001 -
2.2366 6050 0.0001 -
2.2551 6100 0.0001 -
2.2736 6150 0.0001 -
2.2921 6200 0.0001 -
2.3105 6250 0.0001 -
2.3290 6300 0.0002 -
2.3475 6350 0.0002 -
2.3660 6400 0.0002 -
2.3845 6450 0.0001 -
2.4030 6500 0.0001 -
2.4214 6550 0.0001 -
2.4399 6600 0.0001 -
2.4584 6650 0.0001 -
2.4769 6700 0.0001 -
2.4954 6750 0.0001 -
2.5139 6800 0.0001 -
2.5323 6850 0.0002 -
2.5508 6900 0.0001 -
2.5693 6950 0.0003 -
2.5878 7000 0.0001 -
2.6063 7050 0.0001 -
2.6248 7100 0.0001 -
2.6433 7150 0.0009 -
2.6617 7200 0.0004 -
2.6802 7250 0.0001 -
2.6987 7300 0.0 -
2.7172 7350 0.0002 -
2.7357 7400 0.0001 -
2.7542 7450 0.0001 -
2.7726 7500 0.0 -
2.7911 7550 0.0001 -
2.8096 7600 0.0001 -
2.8281 7650 0.0001 -
2.8466 7700 0.0001 -
2.8651 7750 0.0001 -
2.8835 7800 0.0001 -
2.9020 7850 0.0001 -
2.9205 7900 0.0002 -
2.9390 7950 0.0002 -
2.9575 8000 0.0001 -
2.9760 8050 0.0001 -
2.9945 8100 0.0001 -
0.0003 1 0.0002 -
0.0168 50 0.0001 -
0.0336 100 0.0002 -
0.0504 150 0.0001 -
0.0672 200 0.0001 -
0.0840 250 0.0 -
0.1008 300 0.0001 -
0.1176 350 0.0001 -
0.1344 400 0.0001 -
0.1512 450 0.0004 -
0.1680 500 0.0001 -
0.1848 550 0.0003 -
0.2016 600 0.0003 -
0.2184 650 0.0007 -
0.2352 700 0.0005 -
0.2520 750 0.0 -
0.2688 800 0.0002 -
0.2856 850 0.0002 -
0.3024 900 0.0002 -
0.3192 950 0.0001 -
0.3360 1000 0.0002 -
0.3528 1050 0.0007 -
0.3696 1100 0.0001 -
0.3864 1150 0.0004 -
0.4032 1200 0.0002 -
0.4200 1250 0.0004 -
0.4368 1300 0.0004 -
0.4536 1350 0.0037 -
0.4704 1400 0.0406 -
0.4872 1450 0.0003 -
0.5040 1500 0.0001 -
0.5208 1550 0.0003 -
0.5376 1600 0.0002 -
0.5544 1650 0.0001 -
0.5712 1700 0.0002 -
0.5880 1750 0.0002 -
0.6048 1800 0.0001 -
0.6216 1850 0.0 -
0.6384 1900 0.0001 -
0.6552 1950 0.0003 -
0.6720 2000 0.0 -
0.6888 2050 0.0001 -
0.7056 2100 0.0003 -
0.7224 2150 0.0 -
0.7392 2200 0.1019 -
0.7560 2250 0.0001 -
0.7728 2300 0.0001 -
0.7897 2350 0.0001 -
0.8065 2400 0.0 -
0.8233 2450 0.0 -
0.8401 2500 0.0002 -
0.8569 2550 0.0001 -
0.8737 2600 0.0001 -
0.8905 2650 0.0001 -
0.9073 2700 0.0001 -
0.9241 2750 0.0001 -
0.9409 2800 0.0002 -
0.9577 2850 0.0 -
0.9745 2900 0.0001 -
0.9913 2950 0.0001 -
1.0081 3000 0.0001 -
1.0249 3050 0.0 -
1.0417 3100 0.0001 -
1.0585 3150 0.0001 -
1.0753 3200 0.0001 -
1.0921 3250 0.0 -
1.1089 3300 0.0001 -
1.1257 3350 0.0001 -
1.1425 3400 0.0001 -
1.1593 3450 0.0001 -
1.1761 3500 0.0001 -
1.1929 3550 0.0 -
1.2097 3600 0.0001 -
1.2265 3650 0.0 -
1.2433 3700 0.0001 -
1.2601 3750 0.0001 -
1.2769 3800 0.0 -
1.2937 3850 0.0001 -
1.3105 3900 0.0 -
1.3273 3950 0.0001 -
1.3441 4000 0.0002 -
1.3609 4050 0.0001 -
1.3777 4100 0.0001 -
1.3945 4150 0.0001 -
1.4113 4200 0.0 -
1.4281 4250 0.0001 -
1.4449 4300 0.0 -
1.4617 4350 0.0001 -
1.4785 4400 0.0001 -
1.4953 4450 0.0001 -
1.5121 4500 0.0001 -
1.5289 4550 0.0001 -
1.5457 4600 0.0 -
1.5625 4650 0.0001 -
1.5793 4700 0.0001 -
1.5961 4750 0.0001 -
1.6129 4800 0.0002 -
1.6297 4850 0.0 -
1.6465 4900 0.0002 -
1.6633 4950 0.0 -
1.6801 5000 0.0 -
1.6969 5050 0.0001 -
1.7137 5100 0.0001 -
1.7305 5150 0.0 -
1.7473 5200 0.0 -
1.7641 5250 0.0001 -
1.7809 5300 0.0001 -
1.7977 5350 0.0 -
1.8145 5400 0.0003 -
1.8313 5450 0.0 -
1.8481 5500 0.0001 -
1.8649 5550 0.0001 -
1.8817 5600 0.0001 -
1.8985 5650 0.0001 -
1.9153 5700 0.158 -
1.9321 5750 0.0012 -
1.9489 5800 0.0424 -
1.9657 5850 0.0011 -
1.9825 5900 0.0002 -
1.9993 5950 0.1197 -
2.0161 6000 0.0001 -
2.0329 6050 0.2476 -
2.0497 6100 0.0029 -
2.0665 6150 0.0 -
2.0833 6200 0.0 -
2.1001 6250 0.0 -
2.1169 6300 0.0001 -
2.1337 6350 0.1151 -
2.1505 6400 0.0001 -
2.1673 6450 0.0001 -
2.1841 6500 0.0003 -
2.2009 6550 0.1204 -
2.2177 6600 0.0001 -
2.2345 6650 0.0 -
2.2513 6700 0.0016 -
2.2681 6750 0.0001 -
2.2849 6800 0.0008 -
2.3017 6850 0.0001 -
2.3185 6900 0.0 -
2.3353 6950 0.0 -
2.3522 7000 0.0 -
2.3690 7050 0.0003 -
2.3858 7100 0.0 -
2.4026 7150 0.0 -
2.4194 7200 0.0001 -
2.4362 7250 0.0 -
2.4530 7300 0.0001 -
2.4698 7350 0.0001 -
2.4866 7400 0.0001 -
2.5034 7450 0.0 -
2.5202 7500 0.0001 -
2.5370 7550 0.0001 -
2.5538 7600 0.0 -
2.5706 7650 0.0 -
2.5874 7700 0.0 -
2.6042 7750 0.0002 -
2.6210 7800 0.0001 -
2.6378 7850 0.0001 -
2.6546 7900 0.0 -
2.6714 7950 0.0001 -
2.6882 8000 0.0001 -
2.7050 8050 0.0 -
2.7218 8100 0.0 -
2.7386 8150 0.0001 -
2.7554 8200 0.0 -
2.7722 8250 0.0 -
2.7890 8300 0.0 -
2.8058 8350 0.0 -
2.8226 8400 0.0 -
2.8394 8450 0.0 -
2.8562 8500 0.0 -
2.8730 8550 0.0 -
2.8898 8600 0.0001 -
2.9066 8650 0.0001 -
2.9234 8700 0.0 -
2.9402 8750 0.0002 -
2.9570 8800 0.0 -
2.9738 8850 0.0001 -
2.9906 8900 0.0001 -
3.0074 8950 0.0001 -
3.0242 9000 0.0001 -
3.0410 9050 0.0 -
3.0578 9100 0.0 -
3.0746 9150 0.0001 -
3.0914 9200 0.0001 -
3.1082 9250 0.0001 -
3.125 9300 0.0 -
3.1418 9350 0.0 -
3.1586 9400 0.0001 -
3.1754 9450 0.0001 -
3.1922 9500 0.0 -
3.2090 9550 0.0 -
3.2258 9600 0.0 -
3.2426 9650 0.0 -
3.2594 9700 0.0 -
3.2762 9750 0.0002 -
3.2930 9800 0.0001 -
3.3098 9850 0.0 -
3.3266 9900 0.0 -
3.3434 9950 0.0 -
3.3602 10000 0.0 -
3.3770 10050 0.0001 -
3.3938 10100 0.0001 -
3.4106 10150 0.0 -
3.4274 10200 0.0 -
3.4442 10250 0.0001 -
3.4610 10300 0.0 -
3.4778 10350 0.1212 -
3.4946 10400 0.0001 -
3.5114 10450 0.0 -
3.5282 10500 0.1183 -
3.5450 10550 0.0 -
3.5618 10600 0.0002 -
3.5786 10650 0.0001 -
3.5954 10700 0.0 -
3.6122 10750 0.0 -
3.6290 10800 0.0001 -
3.6458 10850 0.0001 -
3.6626 10900 0.0 -
3.6794 10950 0.0 -
3.6962 11000 0.0 -
3.7130 11050 0.0 -
3.7298 11100 0.0 -
3.7466 11150 0.0 -
3.7634 11200 0.0 -
3.7802 11250 0.0 -
3.7970 11300 0.0 -
3.8138 11350 0.0 -
3.8306 11400 0.0 -
3.8474 11450 0.0 -
3.8642 11500 0.0001 -
3.8810 11550 0.0 -
3.8978 11600 0.0001 -
3.9147 11650 0.0 -
3.9315 11700 0.0001 -
3.9483 11750 0.0001 -
3.9651 11800 0.0001 -
3.9819 11850 0.0 -
3.9987 11900 0.0 -

Framework Versions

  • Python: 3.9.16
  • SetFit: 1.0.1
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.0
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.14.6
  • Tokenizers: 0.14.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
8
Safetensors
Model size
109M params
Tensor type
F32
·

Finetuned from

Evaluation results

View on Papers With Code