Edit model card

SetFit with sentence-transformers/paraphrase-mpnet-base-v2 on sst2

This is a SetFit model trained on the sst2 dataset that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. For classification, it uses a LogisticRegression instance.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
negative
  • 'stale and uninspired . '
  • "the film 's considered approach to its subject matter is too calm and thoughtful for agitprop , and the thinness of its characterizations makes it a failure as straight drama . ' "
  • "that their charm does n't do a load of good "
positive
  • "broomfield is energized by volletta wallace 's maternal fury , her fearlessness "
  • 'flawless '
  • 'insightfully written , delicately performed '

Evaluation

Metrics

Label Accuracy
all 0.8588

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from 🤗 Hub
model = SetFitModel.from_pretrained("tomaarsen/setfit-paraphrase-mpnet-base-v2-sst2-8-shot")
# Run inference
preds = model("a fast , funny , highly enjoyable movie . ")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 2 11.4375 33
Label Training Sample Count
negative 8
positive 8

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (10, 10)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.1111 1 0.2126 -
1.1111 10 0.1604 -
2.2222 20 0.0224 0.1761
3.3333 30 0.0039 -
4.4444 40 0.0029 0.1935
5.5556 50 0.0026 -
6.6667 60 0.0008 0.1944
7.7778 70 0.0009 -
8.8889 80 0.0027 0.1941
10.0 90 0.0004 -
  • The bold row denotes the saved checkpoint.

Environmental Impact

Carbon emissions were measured using CodeCarbon.

  • Carbon Emitted: 0.003 kg of CO2
  • Hours Used: 0.027 hours

Training Hardware

  • On Cloud: No
  • GPU Model: 1 x NVIDIA GeForce RTX 3090
  • CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
  • RAM Size: 31.78 GB

Framework Versions

  • Python: 3.9.16
  • SetFit: 1.0.0.dev0
  • Sentence Transformers: 2.2.2
  • Transformers: 4.29.0
  • PyTorch: 1.13.1+cu117
  • Datasets: 2.15.0
  • Tokenizers: 0.13.3

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
12
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Finetuned from

Dataset used to train tomaarsen/setfit-paraphrase-mpnet-base-v2-sst2-8-shot

Collection including tomaarsen/setfit-paraphrase-mpnet-base-v2-sst2-8-shot

Evaluation results