SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 5 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
product discoverability	'Can you show me all the products for oily skin?' 'Do you have any makeup remover?' 'Can you show me all the products for dark spots?'
order tracking	'What is the estimated delivery time for orders within the same state?' 'I need to know the status of my recent order. Can you check if it has been dispatched?' 'I ordered the Cake Decorating Kit 4 days ago, can you provide the tracking information?'
product faq	'What are the different shades available in the Color Affair Nail Polish Pixie Dust Collection?' 'Is the Touch-N-Go Lip & Cheek Tint a vegan and cruelty-free product?' 'Is this product suitable for oily skin?'
general faq	'How often should I use exfoliants to reduce open pores?' 'What are the most effective ingredients for treating acne?' 'Are home remedies effective for severe acne?'
product policy	'Are your products suitable for sensitive skin?' 'How can I track my order on the Plum Goodness app?' 'What is the contact number for customer support?'

Evaluation

Metrics

Label	Accuracy
all	0.9167

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("What makeup products do you have for eyes?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	11.0	24

Label	Training Sample Count
general faq	20
order tracking	24
product discoverability	16
product faq	24
product policy	12

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (2, 2)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0022	1	0.2082	-
0.1101	50	0.1229	-
0.2203	100	0.0262	-
0.3304	150	0.0015	-
0.4405	200	0.001	-
0.5507	250	0.0008	-
0.6608	300	0.0005	-
0.7709	350	0.0004	-
0.8811	400	0.0003	-
0.9912	450	0.0003	-
1.1013	500	0.0002	-
1.2115	550	0.0002	-
1.3216	600	0.0004	-
1.4317	650	0.0002	-
1.5419	700	0.0003	-
1.6520	750	0.0002	-
1.7621	800	0.0002	-
1.8722	850	0.0002	-
1.9824	900	0.0003	-

Framework Versions

Python: 3.9.19
SetFit: 1.0.3
Sentence Transformers: 2.7.0
Transformers: 4.40.2
PyTorch: 2.2.2
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Shankhdhar
/

classifier_woog_plum