Edit model card

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
3
  • 'A Reinvented True Detective Plays It Cool'
  • "It's owl season in Massachusetts. Here's how to spot them"
  • 'Taylor Swift class at Harvard: Professor needs to hire more teaching assistants'
6
  • 'Springfield Mayor Domenic Sarno tests positive for COVID-19'
  • 'How to Take Care of Your Skin in the Fall and Winter'
  • 'Subbing plant-based milk for dairy options is a healthy decision'
2
  • 'Mattel Has a New Cherokee Barbie. Not Everyone Is Happy About It.'
  • 'Who Is Alan Garber, Harvards Interim President?'
  • 'Springfield Marine training in Japan near Mount Fuji (Photos)'
0
  • 'Heres which Northampton businesses might soon get all-alcohol liquor licenses'
  • 'People in Business: Jan. 15, 2024'
  • 'Come Home With Memories, Not a Shocking Phone Bill'
7
  • '3 Patriots vs. Chiefs predictions'
  • 'Tuskegee vs. Alabama State How to watch college football'
  • 'WMass Boys Basketball Season Stats Leaders: Who leads the region by class?'
8
  • 'Biting Cold Sweeping U.S. Hits the South With an Unfamiliar Freeze'
  • 'Some Sunday storms and sun - Boston News, Weather, Sports'
  • 'More snow on the way in Mass. on Tuesday with slippery evening commute'
4
  • 'title'
  • 'This sentence is label'
  • 'This sentence is label'
1
  • 'Two cars crash through former Boston Market in Saugus'
  • 'U.S. Naval Officer Who Helped China Is Sentenced to 2 Years in Prison'
  • 'American Airlines flight attendant arrested after allegedly filming teenage girl in bathroom on flight to Boston - Boston News, Weather, Sports'
5
  • 'Opinion

Evaluation

Metrics

Label Accuracy
all 0.7061

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Kevinger/setfit-hub-report")
# Run inference
preds = model("Opinion | When the World Feels Dark, Seek Out Delight")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 7.2993 21
Label Training Sample Count
0 16
1 16
2 16
3 16
4 9
5 16
6 16
7 16
8 16

Training Hyperparameters

  • batch_size: (16, 2)
  • num_epochs: (1, 16)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0010 1 0.3619 -
0.0481 50 0.097 -
0.0962 100 0.0327 -
0.1442 150 0.0044 -
0.1923 200 0.0013 -
0.2404 250 0.0011 -
0.2885 300 0.001 -
0.3365 350 0.0008 -
0.3846 400 0.001 -
0.4327 450 0.0006 -
0.4808 500 0.0008 -
0.5288 550 0.0005 -
0.5769 600 0.0012 -
0.625 650 0.0005 -
0.6731 700 0.0006 -
0.7212 750 0.0004 -
0.7692 800 0.0005 -
0.8173 850 0.0005 -
0.8654 900 0.0006 -
0.9135 950 0.0014 -
0.9615 1000 0.0006 -

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.16.1
  • Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
6
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for Kevinger/setfit-hub-report

Finetuned
(241)
this model
Finetunes
2 models

Evaluation results