SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
Model Sources
Model Labels
| Label |
Examples |
| 0 |
- 'I’m on Mounjaro 12.5, started 3 weeks ago. Before that I was on Ozempic (max dose), and Bydureon before that. I’ve been on GLP-1 drugs for probably 6 years. Had terrible gastro side effects for months after starting Bydureon, but no gastro side effects on anything since those resolved. I switched from Ozempic to Mounjaro because Ozempic was no longer keeping my a1c controlled. The first couple of weeks on Mounjaro 12.5 I had overwhelming fatigue, but that has improved a little. But I have noticed I am very, very angry on this dose.'
- 'What will obesity rates be like in 2100 - Trust me, Ozempic (Semaglitude) is NOT the "Miracle Drug" that you think it is. Take it from someone who is currently taking it. Sure, it helps some.'
- "Yup, I'm on CRF as well and have probably gained about 50 lbs over time. It sucks. I'm currently taking a smaller dose of mirtazapine and am also on ozempic for weight loss."
|
| 1 |
- "What's the cheapest way possible to get semaglutide? I'm currently taking 2000mg of Metformin with compounded semaglutide with no issues. I have PCOS and not Type 2, so I sadly don't qualify for Ozempic through insurance."
- 'I know 2 people that took Ozempic and quit because they were going downhill on it'
- 'New Ozempic and Wegovy side effects come to light - After I stopped taking it I developed Gallbladder disease and Pancreatitis'
|
Evaluation
Metrics
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("bhaskars113/ozempic-taking-medications-classifier")
preds = model("Has anyone tried ozempic with pcos? I experienced terrible exhaustion on Ozempic. I don't on Mounjaro. Overall, I've had way fewer side effects on Mounjaro.")
Training Details
Training Set Metrics
| Training set |
Min |
Median |
Max |
| Word count |
13 |
32.7931 |
94 |
| Label |
Training Sample Count |
| 0 |
15 |
| 1 |
14 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 20
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
| Epoch |
Step |
Training Loss |
Validation Loss |
| 0.0137 |
1 |
0.2348 |
- |
| 0.6849 |
50 |
0.0013 |
- |
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.0
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
