metadata

library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      Quels sont les recours possibles en cas de conflit entre un employeur et
      un employé ?
  - text: Comment déclarer mes impôts et taxes ?
  - text: Quelles sont les règles de tenue de la comptabilité ?
  - text: Quels sont les frais associés à cette procédure ?
  - text: >-
      Quelles sont les procédures de recours possibles contre une décision
      administrative ?
pipeline_tag: text-classification
inference: true
base_model: intfloat/multilingual-e5-small
model-index:
  - name: SetFit with intfloat/multilingual-e5-small
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.9473684210526315
            name: Accuracy

SetFit with intfloat/multilingual-e5-small

This is a SetFit model that can be used for Text Classification. This SetFit model uses intfloat/multilingual-e5-small as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: intfloat/multilingual-e5-small
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
follow_up	'Quels sont les régimes matrimoniaux possibles ?' 'Quelles sont les conséquences économiques ou sociales de cette loi ?' "Est-ce que cette loi s'applique à mon cas particulier ?"
independent	'Quelles sont les règles en matière de temps de travail et de congés ?' "Quels sont les types de structures d'entreprise autorisés en Algérie ?" 'Quels sont les droits et obligations des travailleurs en Algérie ?'

Evaluation

Metrics

Label	Accuracy
all	0.9474

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("super-cinnamon/fewshot-followup-multi-e5")
# Run inference
preds = model("Comment déclarer mes impôts et taxes ?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	2	9.76	16

Label	Training Sample Count
independent	39
follow_up	36

Training Hyperparameters

batch_size: (8, 8)
num_epochs: (10, 10)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0028	1	0.3779	-
0.1381	50	0.3395	-
0.2762	100	0.1385	-
0.4144	150	0.1179	-
0.5525	200	0.0172	-
0.6906	250	0.0006	-
0.8287	300	0.0014	-
0.9669	350	0.0004	-
1.1050	400	0.0002	-
1.2431	450	0.0002	-
1.3812	500	0.0002	-
1.5193	550	0.0005	-
1.6575	600	0.0001	-
1.7956	650	0.0001	-
1.9337	700	0.0001	-
2.0718	750	0.0002	-
2.2099	800	0.0001	-
2.3481	850	0.0002	-
2.4862	900	0.0003	-
2.6243	950	0.0001	-
2.7624	1000	0.0001	-
2.9006	1050	0.0001	-
3.0387	1100	0.0	-
3.1768	1150	0.0001	-
3.3149	1200	0.0001	-
3.4530	1250	0.0001	-
3.5912	1300	0.0001	-
3.7293	1350	0.0	-
3.8674	1400	0.0001	-
4.0055	1450	0.0001	-
4.1436	1500	0.0001	-
4.2818	1550	0.0002	-
4.4199	1600	0.0001	-
4.5580	1650	0.0001	-
4.6961	1700	0.0002	-
4.8343	1750	0.0	-
4.9724	1800	0.0001	-
5.1105	1850	0.0	-
5.2486	1900	0.0001	-
5.3867	1950	0.0	-
5.5249	2000	0.0	-
5.6630	2050	0.0001	-
5.8011	2100	0.0	-
5.9392	2150	0.0	-
6.0773	2200	0.0001	-
6.2155	2250	0.0001	-
6.3536	2300	0.0001	-
6.4917	2350	0.0	-
6.6298	2400	0.0	-
6.7680	2450	0.0	-
6.9061	2500	0.0	-
7.0442	2550	0.0	-
7.1823	2600	0.0001	-
7.3204	2650	0.0	-
7.4586	2700	0.0	-
7.5967	2750	0.0001	-
7.7348	2800	0.0	-
7.8729	2850	0.0001	-
8.0110	2900	0.0	-
8.1492	2950	0.0	-
8.2873	3000	0.0	-
8.4254	3050	0.0	-
8.5635	3100	0.0001	-
8.7017	3150	0.0	-
8.8398	3200	0.0001	-
8.9779	3250	0.0	-
9.1160	3300	0.0	-
9.2541	3350	0.0	-
9.3923	3400	0.0	-
9.5304	3450	0.0	-
9.6685	3500	0.0	-
9.8066	3550	0.0	-
9.9448	3600	0.0	-

Framework Versions

Python: 3.10.12
SetFit: 1.0.1
Sentence Transformers: 2.2.2
Transformers: 4.35.2
PyTorch: 2.1.0+cu118
Datasets: 2.15.0
Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}