fti-sc/pra-sentiment-classifier

SetFit with sentence-transformers/all-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 384 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
Critical	'"That's why our state's local pharmacies are so essential. They provide people access to the care they need when they need it. But now, many pharmacies are under serious threatand our most vulnerable patients along with them. Over the past 14 years, the number of Oregon pharmacies has decreased more than 26%. Accessing medications or treatments should be simple, but unfortunately it's only becoming more difficult. Why is this happening? One reason involves middlemen insurers called pharmacy benefit managers (PBMs)."' '"Unfortunately, anti-patient policies practiced by health insurance companies and health care middlemen known as pharmacy benefit managers (PBMs) impose unnecessary access and affordability barriers for epilepsy patients ? things like fail first or step therapy requirement, prior authorization, and pocketing billions in discounts without passing savings onto patients. Many patients benefit from copay coupons and copay assistance, which often come in the form of discounts from drug manufacturers and charitable organizations to help patients afford their medicine."' '"But PBMs operate with little to no transparency within the drug pricing system, and they often take advantage of their opaque position at the expense of patients. Their work includes establishing formularies, contracting with pharmacies, and negotiating rebates and discounts with drug manufacturers. But instead of passing these savings on to consumers, PBMs retain these costs, and the patients do not benefit at the pharmacy counter. But it's actually worse than that. Just as a rising tide lifts all boats, PBMs' rebate manipulation inflates health care prices generally and that ultimately increases the cost of patients' medications."'
Supportive	'"Supporters of these bills claim they are about ?protecting patient choice,? but there?s not much of a choice when you can?t afford your medication to begin with. Patients don?t need laws that make it easier for Big Pharma to charge more. They need laws that encourage competition and lower prices. The average patient saves ? over $1,000 a year thanks to PBM negotiations. Take that away, and the only winner is the pharmaceutical industry. These bills don?t lower drug prices, they just shift the cost burden onto families, employers, and taxpayers. That?s not reform."' '"This legislation, meant to punish a Pharmacy Benefit Manager, is driving up the cost of drugs for hard-working Tennesseans who were receiving their drugs at little to no cost. Not only is this in-house pharmacy losing business, but the school system is also having to include additional funding into its health insurance plan to cover additional pharmacy costs costs which were completely imposed by government action and not the rising cost of insurance. Remarkably, this means that the state government's actions are now being paid for by a local government."' '"PBMs are third-party administrators of prescription medicine plans for insurance companies, businesses large and small, and government health plans. They administer the plan's drug formulary, process prescription claims and negotiate discounts with drug manufacturers. Basically, PBMs act as a check and balance like in our system of government on pharmaceutical companies, obtaining price discounts for the consumer in the form of rebates. Sanders' bill would gut their ability to negotiate, under the mistaken assumption that they are the \"bad guy,\" and it sailed through the Senate health committee by a terrifying 18-3 vote."'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("\"PBMs also compile lists of drugs, called formularies, that providers of health benefits agree to cover; establish pharmacy networks that patients can access; and run their own mail-order pharmacies. Although PBMs are supposed to help lower costs, some of their practices may well do the opposite. PBMs often keep a portion of the rebates they negotiate, which can incentivize them to favor more expensive drugs on their formularies. (A $1 million drug, for example, would fetch a bigger fee than a $100 one.\"")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	74	88.9474	100

Label	Training Sample Count
Supportive	8
Critical	11

Training Hyperparameters

• batch_size: (8, 8)
• num_epochs: (2, 2)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0385	1	0.201	-
1.9231	50	0.1192	-

Framework Versions

• Python: 3.10.6
• SetFit: 1.1.1
• Sentence Transformers: 3.4.1
• Transformers: 4.50.1
• PyTorch: 2.6.0
• Datasets: 3.4.1
• Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}