Hugging Face's logo

Shankhdhar
/
jewellery_query_setfit_classifier

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
jewellery_query_setfit_classifier / README.md
Shankhdhar's picture
Shankhdhar
Add SetFit model
6054aed verified
preview code
|
raw history blame contribute delete
No virus
12.3 kB
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      I recently purchased the Reevati Gold Pearl Necklace and upon receiving
      it, I noticed that the pearls are not properly aligned and some seem to be
      of different sizes. This is not what I expected based on the images on
      your site.
  - text: >-
      I recently ordered the Once in a Blue Moon Statement Ring but haven't
      received any shipping updates yet. Can you provide me with the current
      status of my order?
  - text: >-
      I recently bought the Golden Love Affair Pendant, but it seems to have
      tarnished very quickly. I'm not satisfied with the quality. What can you
      do about this?
  - text: >-
      I recently purchased the Three Crystal Proposal Ring, but I'm disappointed
      to find that one of the crystals is loose. Can you assist me with this
      issue?
  - text: >-
      I recently purchased the Bloomingdale Pendant, but I've noticed that the
      quality does not meet the standards promised on the website. The pendant
      looks tarnished and is different from the images shown.
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.8024691358024691
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
product faq
  • 'What are the different sizes available for the Love is in the Air Proposal Ring, and do they come at different price points?'
  • 'What is the material of the Open Pear Cut Ring and are there different sizes available?'
  • 'What is the material used for making the Golden Spin Hoop Earring, and does it come with any kind of warranty or guarantee?'
product discoveribility
  • 'What are the latest choker styles available for a wedding occasion?'
  • "I'm interested in sustainable jewelry; do you have any eco-friendly necklaces?"
  • 'Could you recommend some necklaces with a vintage vibe to them?'
order tracking
  • 'I recently purchased the Seher Pearl Choker Set and I would like to know the current status of my order delivery.'
  • "I placed an order for the Tiara Silver Ring, but I haven't received any shipping updates yet. Can you provide me with the current status of my order?"
  • 'I recently ordered the Toes Of Love Pendant but have not received any shipping confirmation. Could you please provide me with the tracking details?'
product policy
  • 'Are there any restocking fees for bracelet returns?'
  • "Can I exchange a ring if it doesn't fit properly?"
  • 'Are there any care instructions included with the purchase of a ring?'

Evaluation

Metrics

Label Accuracy
all 0.8025

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("I recently purchased the Three Crystal Proposal Ring, but I'm disappointed to find that one of the crystals is loose. Can you assist me with this issue?")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 6 16.4474 30
Label Training Sample Count
negative 0
positive 0

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (4, 4)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0016 1 0.1464 -
0.0822 50 0.0907 -
0.1645 100 0.0059 -
0.2467 150 0.0013 -
0.3289 200 0.0009 -
0.4112 250 0.0007 -
0.4934 300 0.0004 -
0.5757 350 0.0003 -
0.6579 400 0.0001 -
0.7401 450 0.0002 -
0.8224 500 0.0002 -
0.9046 550 0.0002 -
0.9868 600 0.0001 -
1.0 608 - 0.2272
1.0691 650 0.0001 -
1.1513 700 0.0001 -
1.2336 750 0.0001 -
1.3158 800 0.0001 -
1.3980 850 0.0001 -
1.4803 900 0.0001 -
1.5625 950 0.0001 -
1.6447 1000 0.0001 -
1.7270 1050 0.0001 -
1.8092 1100 0.0 -
1.8914 1150 0.0001 -
1.9737 1200 0.0001 -
2.0 1216 - 0.2807
2.0559 1250 0.0001 -
2.1382 1300 0.0001 -
2.2204 1350 0.0001 -
2.3026 1400 0.0 -
2.3849 1450 0.0001 -
2.4671 1500 0.0001 -
2.5493 1550 0.0 -
2.6316 1600 0.0001 -
2.7138 1650 0.0 -
2.7961 1700 0.0001 -
2.8783 1750 0.0 -
2.9605 1800 0.0 -
3.0 1824 - 0.3011
3.0428 1850 0.0 -
3.125 1900 0.0001 -
3.2072 1950 0.0001 -
3.2895 2000 0.0 -
3.3717 2050 0.0001 -
3.4539 2100 0.0001 -
3.5362 2150 0.0 -
3.6184 2200 0.0001 -
3.7007 2250 0.0001 -
3.7829 2300 0.0 -
3.8651 2350 0.0 -
3.9474 2400 0.0001 -
4.0 2432 - 0.311
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.9.16
  • SetFit: 1.0.3
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.2
  • PyTorch: 2.1.1
  • Datasets: 2.15.0
  • Tokenizers: 0.15.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}