lucienbaumgartner/lifestyle_vs_disease

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
lifestyle	'but i got tested for diabetes over and over again because of my “unhealthy diet” .' 'history: no history of stomach issues besides the occasional stomach ache and diarrhea after eating a greasy/unhealthy meal.' 'but i can not run 3 seconds without breathing for 10 minutes that should say how unhealthy i am.\n\n '
disease	'there were three other people in the house, 2 of who have lived there for decades, and they seem to all be healthy. \n\n\n' "md said that it's not actually true and that the germ can mess up with your intestine for month after the contagion, even if i am healthy." 'healthy, 27f, only past medical hx is 2nd degree type 1 heart block related to high vagal tone as per the cardiologist i saw at the time.'

Evaluation

Metrics

Label	Accuracy	Precision	Recall	F1
all	0.8421	0.8421	0.8421	0.8421

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("and she has been in the hospital constantly for an otherwise healthy wonderful loving lady. 

")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	12	29.9474	188

Label	Training Sample Count
disease	29
lifestyle	47

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (10, 10)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (2e-05, 2e-05)
• head_learning_rate: 2e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 3786
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0053	1	0.3721	-
0.2632	50	0.2312	-
0.5263	100	0.1102	-
0.7895	150	0.0036	-
1.0526	200	0.0006	-
1.3158	250	0.0003	-
1.5789	300	0.0002	-
1.8421	350	0.0002	-
2.1053	400	0.0001	-
2.3684	450	0.0001	-
2.6316	500	0.0001	-
2.8947	550	0.0001	-
3.1579	600	0.0001	-
3.4211	650	0.0001	-
3.6842	700	0.0001	-
3.9474	750	0.0001	-
4.2105	800	0.0001	-
4.4737	850	0.0001	-
4.7368	900	0.0	-
5.0	950	0.0	-
5.2632	1000	0.0	-
5.5263	1050	0.0	-
5.7895	1100	0.0	-
6.0526	1150	0.0	-
6.3158	1200	0.0	-
6.5789	1250	0.0	-
6.8421	1300	0.0	-
7.1053	1350	0.0	-
7.3684	1400	0.0	-
7.6316	1450	0.0	-
7.8947	1500	0.0	-
8.1579	1550	0.0	-
8.4211	1600	0.0	-
8.6842	1650	0.0	-
8.9474	1700	0.0	-
9.2105	1750	0.0	-
9.4737	1800	0.0	-
9.7368	1850	0.0	-
10.0	1900	0.0	-

Framework Versions

• Python: 3.11.9
• SetFit: 1.1.2
• Sentence Transformers: 4.1.0
• Transformers: 4.52.4
• PyTorch: 2.7.1
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}