SetFit
This is a SetFit model that can be used for Text Classification. A LinearSVC instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Classification head: a LinearSVC instance
- Maximum Sequence Length: 512 tokens
- Number of Classes: 2 classes
Model Sources
Model Labels
Label |
Examples |
SUBJ |
- 'Now suppose that under stress of abnormal public revenue the structure of government is somewhat rationalized and that by such means as economy and efficiency the cost of government by measure is much reduced.'
- 'Modern Russia is a propaganda state, but not in the same way as the Soviet Union.'
- 'The spender of public money will never want followers.'
|
OBJ |
- 'But a top buying agent tells me that access to 13 can be gained if you know the right people.'
- '“Normally, the majority opinion would speak for itself.” The decision is “really about policy—our state has values of inclusion and diversity.” The ruling is based “on policy, which is the definition of judicial activism.'
- 'asked American Federation of Teachers President Randi Weingarten.'
|
Evaluation
Metrics
Label |
Accuracy |
all |
0.7737 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("SOUMYADEEPSAR/SetFit_SubjectivityDetection")
preds = model("What could possibly go wrong?")
Training Details
Training Set Metrics
Training set |
Min |
Median |
Max |
Word count |
3 |
22.085 |
77 |
Label |
Training Sample Count |
OBJ |
100 |
SUBJ |
100 |
Training Hyperparameters
- batch_size: (32, 32)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
Epoch |
Step |
Training Loss |
Validation Loss |
0.0016 |
1 |
0.2686 |
- |
0.0791 |
50 |
0.2494 |
- |
0.1582 |
100 |
0.2639 |
- |
0.2373 |
150 |
0.2258 |
- |
0.3165 |
200 |
0.0176 |
- |
0.3956 |
250 |
0.0027 |
- |
0.4747 |
300 |
0.0017 |
- |
0.5538 |
350 |
0.0013 |
- |
0.6329 |
400 |
0.0016 |
- |
0.7120 |
450 |
0.001 |
- |
0.7911 |
500 |
0.0009 |
- |
0.8703 |
550 |
0.001 |
- |
0.9494 |
600 |
0.001 |
- |
1.0285 |
650 |
0.0009 |
- |
1.1076 |
700 |
0.0008 |
- |
1.1867 |
750 |
0.0008 |
- |
1.2658 |
800 |
0.0006 |
- |
1.3449 |
850 |
0.0007 |
- |
1.4241 |
900 |
0.0006 |
- |
1.5032 |
950 |
0.0007 |
- |
1.5823 |
1000 |
0.0006 |
- |
1.6614 |
1050 |
0.0005 |
- |
1.7405 |
1100 |
0.0006 |
- |
1.8196 |
1150 |
0.0007 |
- |
1.8987 |
1200 |
0.0005 |
- |
1.9778 |
1250 |
0.0006 |
- |
2.0570 |
1300 |
0.0005 |
- |
2.1361 |
1350 |
0.0005 |
- |
2.2152 |
1400 |
0.0004 |
- |
2.2943 |
1450 |
0.0005 |
- |
2.3734 |
1500 |
0.0004 |
- |
2.4525 |
1550 |
0.0004 |
- |
2.5316 |
1600 |
0.0004 |
- |
2.6108 |
1650 |
0.0004 |
- |
2.6899 |
1700 |
0.0005 |
- |
2.7690 |
1750 |
0.0005 |
- |
2.8481 |
1800 |
0.0004 |
- |
2.9272 |
1850 |
0.0005 |
- |
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.4.0
- Transformers: 4.37.2
- PyTorch: 2.1.0+cu121
- Datasets: 2.17.1
- Tokenizers: 0.15.2
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}