metadata
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
base_model: sentence-transformers/paraphrase-MiniLM-L3-v2
metrics:
- accuracy
widget:
- text: Tiong Bahru Plaza, DDC-L2-5, AHU-L2-03 trip alarm
- text: >-
Tiong Bahru Plaza, DDC L4-1, PAU-L4-03 supply air temperature (Units:
°C).2
- text: Tiong Bahru Plaza, DDC-L2-5, AHU-L2-03 VSD control
- text: 'Tiong Bahru Plaza, VAV 19-7, Discharge Air Flow (Units: m3/h)'
- text: Tiong Bahru Plaza, DDC-L1-4, PAU-L1-05 VSD control
pipeline_tag: text-classification
inference: true
model-index:
- name: SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8337264150943396
name: Accuracy
SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-MiniLM-L3-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/paraphrase-MiniLM-L3-v2
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 128 tokens
- Number of Classes: 47 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
28 |
|
6 |
|
15 |
|
43 |
|
4 |
|
0 |
|
10 |
|
40 |
|
26 |
|
5 |
|
2 |
|
32 |
|
34 |
|
24 |
|
39 |
|
13 |
|
9 |
|
17 |
|
14 |
|
44 |
|
27 |
|
21 |
|
7 |
|
29 |
|
1 |
|
46 |
|
11 |
|
33 |
|
8 |
|
16 |
|
35 |
|
30 |
|
3 |
|
20 |
|
23 |
|
45 |
|
41 |
|
25 |
|
36 |
|
12 |
|
42 |
|
37 |
|
31 |
|
38 |
|
18 |
|
19 |
|
22 |
|
Evaluation
Metrics
Label | Accuracy |
---|---|
all | 0.8337 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Varun1010/all-MiniLM-L6-v2-polaris-tb-new-v1")
# Run inference
preds = model("Tiong Bahru Plaza, DDC-L2-5, AHU-L2-03 trip alarm")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 5 | 8.4138 | 14 |
Label | Training Sample Count |
---|---|
0 | 10 |
1 | 10 |
2 | 10 |
3 | 10 |
4 | 10 |
5 | 10 |
6 | 10 |
7 | 10 |
8 | 10 |
9 | 10 |
10 | 10 |
11 | 10 |
12 | 10 |
13 | 10 |
14 | 10 |
15 | 10 |
16 | 10 |
17 | 10 |
18 | 3 |
19 | 3 |
20 | 10 |
21 | 10 |
22 | 1 |
23 | 1 |
24 | 10 |
25 | 4 |
26 | 10 |
27 | 8 |
28 | 4 |
29 | 3 |
30 | 3 |
31 | 4 |
32 | 4 |
33 | 9 |
34 | 5 |
35 | 4 |
36 | 3 |
37 | 2 |
38 | 3 |
39 | 9 |
40 | 10 |
41 | 4 |
42 | 2 |
43 | 3 |
44 | 8 |
45 | 8 |
46 | 2 |
Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 16)
- max_steps: 500
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0006 | 1 | 0.1516 | - |
0.0302 | 50 | 0.1292 | - |
0.0604 | 100 | 0.0796 | - |
0.0905 | 150 | 0.068 | - |
0.1207 | 200 | 0.0498 | - |
0.1509 | 250 | 0.06 | - |
0.1811 | 300 | 0.0415 | - |
0.2112 | 350 | 0.0422 | - |
0.2414 | 400 | 0.0327 | - |
0.2716 | 450 | 0.0247 | - |
0.3018 | 500 | 0.0253 | - |
0.3319 | 550 | 0.0192 | - |
0.3621 | 600 | 0.0347 | - |
0.3923 | 650 | 0.0166 | - |
0.4225 | 700 | 0.034 | - |
0.4526 | 750 | 0.0242 | - |
0.4828 | 800 | 0.031 | - |
0.5130 | 850 | 0.0102 | - |
0.5432 | 900 | 0.0145 | - |
0.5733 | 950 | 0.0096 | - |
0.6035 | 1000 | 0.0166 | - |
0.6337 | 1050 | 0.0098 | - |
0.6639 | 1100 | 0.0091 | - |
0.6940 | 1150 | 0.005 | - |
0.7242 | 1200 | 0.008 | - |
0.7544 | 1250 | 0.0085 | - |
0.7846 | 1300 | 0.0242 | - |
0.8147 | 1350 | 0.0049 | - |
0.8449 | 1400 | 0.0082 | - |
0.8751 | 1450 | 0.0053 | - |
0.9053 | 1500 | 0.0092 | - |
0.9354 | 1550 | 0.0086 | - |
0.9656 | 1600 | 0.0054 | - |
0.9958 | 1650 | 0.0052 | - |
1.0260 | 1700 | 0.0101 | - |
1.0561 | 1750 | 0.0184 | - |
1.0863 | 1800 | 0.004 | - |
1.1165 | 1850 | 0.0082 | - |
1.1467 | 1900 | 0.0188 | - |
1.1768 | 1950 | 0.0097 | - |
1.2070 | 2000 | 0.0067 | - |
1.2372 | 2050 | 0.004 | - |
1.2674 | 2100 | 0.0076 | - |
1.2975 | 2150 | 0.0076 | - |
1.3277 | 2200 | 0.0192 | - |
1.3579 | 2250 | 0.0088 | - |
1.3881 | 2300 | 0.0049 | - |
1.4182 | 2350 | 0.0034 | - |
1.4484 | 2400 | 0.0028 | - |
1.4786 | 2450 | 0.0031 | - |
1.5088 | 2500 | 0.0075 | - |
1.5389 | 2550 | 0.0093 | - |
1.5691 | 2600 | 0.0037 | - |
1.5993 | 2650 | 0.0151 | - |
1.6295 | 2700 | 0.0044 | - |
1.6596 | 2750 | 0.002 | - |
1.6898 | 2800 | 0.0027 | - |
1.7200 | 2850 | 0.0039 | - |
1.7502 | 2900 | 0.003 | - |
1.7803 | 2950 | 0.0101 | - |
1.8105 | 3000 | 0.0082 | - |
1.8407 | 3050 | 0.0025 | - |
1.8709 | 3100 | 0.004 | - |
1.9010 | 3150 | 0.0064 | - |
1.9312 | 3200 | 0.0025 | - |
1.9614 | 3250 | 0.0021 | - |
1.9916 | 3300 | 0.0061 | - |
2.0217 | 3350 | 0.0055 | - |
2.0519 | 3400 | 0.0021 | - |
2.0821 | 3450 | 0.0034 | - |
2.1123 | 3500 | 0.002 | - |
2.1424 | 3550 | 0.0034 | - |
2.1726 | 3600 | 0.0027 | - |
2.2028 | 3650 | 0.0021 | - |
2.2330 | 3700 | 0.0056 | - |
2.2631 | 3750 | 0.0017 | - |
2.2933 | 3800 | 0.0024 | - |
2.3235 | 3850 | 0.0021 | - |
2.3537 | 3900 | 0.0033 | - |
2.3838 | 3950 | 0.0024 | - |
2.4140 | 4000 | 0.0029 | - |
2.4442 | 4050 | 0.0022 | - |
2.4744 | 4100 | 0.0015 | - |
2.5045 | 4150 | 0.0016 | - |
2.5347 | 4200 | 0.0028 | - |
2.5649 | 4250 | 0.0024 | - |
2.5951 | 4300 | 0.0041 | - |
2.6252 | 4350 | 0.0025 | - |
2.6554 | 4400 | 0.0019 | - |
2.6856 | 4450 | 0.0014 | - |
2.7158 | 4500 | 0.0031 | - |
2.7459 | 4550 | 0.0064 | - |
2.7761 | 4600 | 0.0047 | - |
2.8063 | 4650 | 0.004 | - |
2.8365 | 4700 | 0.0032 | - |
2.8666 | 4750 | 0.0017 | - |
2.8968 | 4800 | 0.0017 | - |
2.9270 | 4850 | 0.0039 | - |
2.9572 | 4900 | 0.0018 | - |
2.9873 | 4950 | 0.0015 | - |
3.0175 | 5000 | 0.0015 | - |
3.0477 | 5050 | 0.002 | - |
3.0779 | 5100 | 0.0015 | - |
3.1080 | 5150 | 0.0034 | - |
3.1382 | 5200 | 0.0022 | - |
3.1684 | 5250 | 0.0013 | - |
3.1986 | 5300 | 0.0165 | - |
3.2287 | 5350 | 0.0011 | - |
3.2589 | 5400 | 0.0012 | - |
3.2891 | 5450 | 0.0015 | - |
3.3193 | 5500 | 0.0021 | - |
3.3494 | 5550 | 0.003 | - |
3.3796 | 5600 | 0.0052 | - |
3.4098 | 5650 | 0.0011 | - |
3.4400 | 5700 | 0.0012 | - |
3.4701 | 5750 | 0.0013 | - |
3.5003 | 5800 | 0.0007 | - |
3.5305 | 5850 | 0.0013 | - |
3.5607 | 5900 | 0.0058 | - |
3.5908 | 5950 | 0.003 | - |
3.6210 | 6000 | 0.0015 | - |
3.6512 | 6050 | 0.001 | - |
3.6814 | 6100 | 0.0022 | - |
3.7115 | 6150 | 0.0056 | - |
3.7417 | 6200 | 0.0029 | - |
3.7719 | 6250 | 0.0009 | - |
3.8021 | 6300 | 0.0021 | - |
3.8322 | 6350 | 0.0047 | - |
3.8624 | 6400 | 0.0026 | - |
3.8926 | 6450 | 0.001 | - |
3.9228 | 6500 | 0.0015 | - |
3.9529 | 6550 | 0.0012 | - |
3.9831 | 6600 | 0.0154 | - |
4.0133 | 6650 | 0.0012 | - |
4.0435 | 6700 | 0.0014 | - |
4.0736 | 6750 | 0.0016 | - |
4.1038 | 6800 | 0.0044 | - |
4.1340 | 6850 | 0.0013 | - |
4.1642 | 6900 | 0.003 | - |
4.1943 | 6950 | 0.0019 | - |
4.2245 | 7000 | 0.0013 | - |
4.2547 | 7050 | 0.0007 | - |
4.2849 | 7100 | 0.0019 | - |
4.3150 | 7150 | 0.0007 | - |
4.3452 | 7200 | 0.0012 | - |
4.3754 | 7250 | 0.0008 | - |
4.4056 | 7300 | 0.0009 | - |
4.4357 | 7350 | 0.0011 | - |
4.4659 | 7400 | 0.0157 | - |
4.4961 | 7450 | 0.0009 | - |
4.5263 | 7500 | 0.0009 | - |
4.5564 | 7550 | 0.0018 | - |
4.5866 | 7600 | 0.001 | - |
4.6168 | 7650 | 0.001 | - |
4.6470 | 7700 | 0.001 | - |
4.6771 | 7750 | 0.001 | - |
4.7073 | 7800 | 0.001 | - |
4.7375 | 7850 | 0.0018 | - |
4.7677 | 7900 | 0.001 | - |
4.7978 | 7950 | 0.0011 | - |
4.8280 | 8000 | 0.0011 | - |
4.8582 | 8050 | 0.001 | - |
4.8884 | 8100 | 0.0008 | - |
4.9185 | 8150 | 0.0009 | - |
4.9487 | 8200 | 0.0034 | - |
4.9789 | 8250 | 0.001 | - |
0.0020 | 1 | 0.8971 | - |
0.0998 | 50 | 0.3923 | - |
0.1996 | 100 | 0.0047 | - |
0.2994 | 150 | 0.0013 | - |
0.3992 | 200 | 0.0009 | - |
0.4990 | 250 | 0.0005 | - |
0.5988 | 300 | 0.0003 | - |
0.6986 | 350 | 0.0004 | - |
0.7984 | 400 | 0.0003 | - |
0.8982 | 450 | 0.0003 | - |
0.9980 | 500 | 0.0004 | - |
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.0
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}