SetFit

This is a SetFit model that can be used for Text Classification. A LinearSVC instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Classification head: a LinearSVC instance
Maximum Sequence Length: 512 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
SUBJ	'Now suppose that under stress of abnormal public revenue the structure of government is somewhat rationalized and that by such means as economy and efficiency the cost of government by measure is much reduced.' 'Modern Russia is a propaganda state, but not in the same way as the Soviet Union.' 'The spender of public money will never want followers.'
OBJ	'But a top buying agent tells me that access to 13 can be gained if you know the right people.' '“Normally, the majority opinion would speak for itself.” The decision is “really about policy—our state has values of inclusion and diversity.” The ruling is based “on policy, which is the definition of judicial activism.' 'asked American Federation of Teachers President Randi Weingarten.'

Evaluation

Metrics

Label	Accuracy
all	0.7737

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("SOUMYADEEPSAR/SetFit_SubjectivityDetection")
# Run inference
preds = model("What could possibly go wrong?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	22.085	77

Label	Training Sample Count
OBJ	100
SUBJ	100

Training Hyperparameters

batch_size: (32, 32)
num_epochs: (3, 3)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0016	1	0.2686	-
0.0791	50	0.2494	-
0.1582	100	0.2639	-
0.2373	150	0.2258	-
0.3165	200	0.0176	-
0.3956	250	0.0027	-
0.4747	300	0.0017	-
0.5538	350	0.0013	-
0.6329	400	0.0016	-
0.7120	450	0.001	-
0.7911	500	0.0009	-
0.8703	550	0.001	-
0.9494	600	0.001	-
1.0285	650	0.0009	-
1.1076	700	0.0008	-
1.1867	750	0.0008	-
1.2658	800	0.0006	-
1.3449	850	0.0007	-
1.4241	900	0.0006	-
1.5032	950	0.0007	-
1.5823	1000	0.0006	-
1.6614	1050	0.0005	-
1.7405	1100	0.0006	-
1.8196	1150	0.0007	-
1.8987	1200	0.0005	-
1.9778	1250	0.0006	-
2.0570	1300	0.0005	-
2.1361	1350	0.0005	-
2.2152	1400	0.0004	-
2.2943	1450	0.0005	-
2.3734	1500	0.0004	-
2.4525	1550	0.0004	-
2.5316	1600	0.0004	-
2.6108	1650	0.0004	-
2.6899	1700	0.0005	-
2.7690	1750	0.0005	-
2.8481	1800	0.0004	-
2.9272	1850	0.0005	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.3
Sentence Transformers: 2.4.0
Transformers: 4.37.2
PyTorch: 2.1.0+cu121
Datasets: 2.17.1
Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

SOUMYADEEPSAR
/

SetFit_SubjectivityDetection