SetFit
This is a SetFit model that can be used for Text Classification. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 256 tokens
- Number of Classes: 3 classes
Model Sources
Model Labels
Label |
Examples |
negative |
- 'Biasanya cuma janji saja tapi tak pernah buat pun'
- 'Aku faham kalau diorang buat macam ni sebab kalau aku jadi doktor pun aku penat la kimak. Bodoh punya kerajaan !'
- 'Jika parlimen tidak dipanggil dalam masa 14 hari maka UMNO akan keluar menjadi pembangkang iaitu lebih kurang 4 Jul https://t.co/AvJSf1F8ux'
|
positive |
- 'Kek telapuk kuda, kek lembut rasa premium! Tgh kumpoi order ni utk warga Sungai Petani. Tak terlioq ka tengok? Mai https://t.co/wgcEADvxQK'
- 'Kalau projek ini berjalan, peneroka Felda tak payah masuk kebun lagi, harga sawit baru RM500 1 tan, murah.'
- 'Justeru, kita berharap lebih banyak pusat rawatan seperti ini diwujudkan untuk membantu.'
|
neutral |
- '08.05 WIB #Jalan_Layang_MBZ Cikunir - Tambun - Cikarang - Karawang LANCAR. ; Karawang - Cikarang - Tambun - Cikunir LANCAR.'
- '5) Menilai Kualiti Kandungan. Selepas dah faham apa yang penulis cuba sampaikan, kita boleh nilai sama ada ia bena https://t.co/LnkjtBc3Nm'
- '@syafirazbd_ Siapp'
|
Evaluation
Metrics
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
model = SetFitModel.from_pretrained("babysharkdododo/setfit-all-minilm-l6-v2-malay_en_cn_sentiment_analysis")
preds = model("Kebanyakan orang Cina yang kamu temui di sini akan beritahu kamu, kebimbangan mereka tentang isu yang melanda negara.")
Training Details
Training Set Metrics
Training set |
Min |
Median |
Max |
Word count |
1 |
16.598 |
58 |
Label |
Training Sample Count |
positive |
235 |
neutral |
77 |
negative |
188 |
Training Hyperparameters
- batch_size: (128, 128)
- num_epochs: (10, 10)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch |
Step |
Training Loss |
Validation Loss |
0.0001 |
1 |
0.189 |
- |
0.0052 |
50 |
0.3053 |
- |
0.0104 |
100 |
0.2779 |
- |
0.0156 |
150 |
0.4526 |
- |
0.0208 |
200 |
0.3073 |
- |
0.0261 |
250 |
0.4156 |
- |
0.0313 |
300 |
0.3912 |
- |
0.0365 |
350 |
0.2259 |
- |
0.0417 |
400 |
0.2445 |
- |
0.0004 |
1 |
0.3311 |
- |
0.0208 |
50 |
0.3569 |
- |
0.0417 |
100 |
0.2946 |
- |
0.0625 |
150 |
0.3397 |
- |
0.0008 |
1 |
0.3177 |
- |
0.0417 |
50 |
0.2764 |
- |
0.0833 |
100 |
0.226 |
- |
0.125 |
150 |
0.2649 |
- |
0.1667 |
200 |
0.282 |
- |
0.2083 |
250 |
0.2438 |
- |
0.25 |
300 |
0.2425 |
- |
0.2917 |
350 |
0.2555 |
- |
0.3333 |
400 |
0.2266 |
- |
0.375 |
450 |
0.14 |
- |
0.4167 |
500 |
0.1446 |
- |
0.4583 |
550 |
0.152 |
- |
0.5 |
600 |
0.184 |
- |
0.5417 |
650 |
0.095 |
- |
0.5833 |
700 |
0.1358 |
- |
0.625 |
750 |
0.0859 |
- |
0.6667 |
800 |
0.0756 |
- |
0.7083 |
850 |
0.0622 |
- |
0.75 |
900 |
0.0719 |
- |
0.7917 |
950 |
0.0681 |
- |
0.8333 |
1000 |
0.0684 |
- |
0.875 |
1050 |
0.0356 |
- |
0.9167 |
1100 |
0.0233 |
- |
0.9583 |
1150 |
0.0126 |
- |
1.0 |
1200 |
0.0022 |
0.3748 |
1.0417 |
1250 |
0.0095 |
- |
1.0833 |
1300 |
0.0095 |
- |
1.125 |
1350 |
0.0376 |
- |
1.1667 |
1400 |
0.0075 |
- |
1.2083 |
1450 |
0.0075 |
- |
1.25 |
1500 |
0.0142 |
- |
1.2917 |
1550 |
0.0113 |
- |
1.3333 |
1600 |
0.0022 |
- |
1.375 |
1650 |
0.0006 |
- |
1.4167 |
1700 |
0.0005 |
- |
1.4583 |
1750 |
0.0005 |
- |
1.5 |
1800 |
0.0003 |
- |
1.5417 |
1850 |
0.0021 |
- |
1.5833 |
1900 |
0.0004 |
- |
1.625 |
1950 |
0.0006 |
- |
1.6667 |
2000 |
0.001 |
- |
1.7083 |
2050 |
0.0002 |
- |
1.75 |
2100 |
0.0002 |
- |
1.7917 |
2150 |
0.0002 |
- |
1.8333 |
2200 |
0.0002 |
- |
1.875 |
2250 |
0.0059 |
- |
1.9167 |
2300 |
0.0002 |
- |
1.9583 |
2350 |
0.0005 |
- |
2.0 |
2400 |
0.0001 |
0.3806 |
2.0417 |
2450 |
0.0001 |
- |
2.0833 |
2500 |
0.0012 |
- |
2.125 |
2550 |
0.0001 |
- |
2.1667 |
2600 |
0.0002 |
- |
2.2083 |
2650 |
0.0002 |
- |
2.25 |
2700 |
0.0001 |
- |
2.2917 |
2750 |
0.0011 |
- |
2.3333 |
2800 |
0.0002 |
- |
2.375 |
2850 |
0.0001 |
- |
2.4167 |
2900 |
0.0003 |
- |
2.4583 |
2950 |
0.0007 |
- |
2.5 |
3000 |
0.0001 |
- |
2.5417 |
3050 |
0.0001 |
- |
2.5833 |
3100 |
0.0001 |
- |
2.625 |
3150 |
0.0001 |
- |
2.6667 |
3200 |
0.0001 |
- |
2.7083 |
3250 |
0.0001 |
- |
2.75 |
3300 |
0.0001 |
- |
2.7917 |
3350 |
0.0002 |
- |
2.8333 |
3400 |
0.0001 |
- |
2.875 |
3450 |
0.0001 |
- |
2.9167 |
3500 |
0.0001 |
- |
2.9583 |
3550 |
0.0001 |
- |
3.0 |
3600 |
0.0001 |
0.4004 |
3.0417 |
3650 |
0.0001 |
- |
3.0833 |
3700 |
0.0001 |
- |
3.125 |
3750 |
0.0001 |
- |
3.1667 |
3800 |
0.0001 |
- |
3.2083 |
3850 |
0.0002 |
- |
3.25 |
3900 |
0.0001 |
- |
3.2917 |
3950 |
0.0001 |
- |
3.3333 |
4000 |
0.0005 |
- |
3.375 |
4050 |
0.0001 |
- |
3.4167 |
4100 |
0.0001 |
- |
3.4583 |
4150 |
0.0001 |
- |
3.5 |
4200 |
0.0004 |
- |
3.5417 |
4250 |
0.0 |
- |
3.5833 |
4300 |
0.0001 |
- |
3.625 |
4350 |
0.0001 |
- |
3.6667 |
4400 |
0.0001 |
- |
3.7083 |
4450 |
0.0001 |
- |
3.75 |
4500 |
0.0 |
- |
3.7917 |
4550 |
0.0 |
- |
3.8333 |
4600 |
0.0 |
- |
3.875 |
4650 |
0.0001 |
- |
3.9167 |
4700 |
0.0001 |
- |
3.9583 |
4750 |
0.0001 |
- |
4.0 |
4800 |
0.0 |
0.4004 |
4.0417 |
4850 |
0.0001 |
- |
4.0833 |
4900 |
0.0003 |
- |
4.125 |
4950 |
0.0 |
- |
4.1667 |
5000 |
0.0001 |
- |
4.2083 |
5050 |
0.0001 |
- |
4.25 |
5100 |
0.0 |
- |
4.2917 |
5150 |
0.0003 |
- |
4.3333 |
5200 |
0.0001 |
- |
4.375 |
5250 |
0.0 |
- |
4.4167 |
5300 |
0.0 |
- |
4.4583 |
5350 |
0.0002 |
- |
4.5 |
5400 |
0.0 |
- |
4.5417 |
5450 |
0.0001 |
- |
4.5833 |
5500 |
0.0001 |
- |
4.625 |
5550 |
0.0 |
- |
4.6667 |
5600 |
0.0006 |
- |
4.7083 |
5650 |
0.0 |
- |
4.75 |
5700 |
0.0 |
- |
4.7917 |
5750 |
0.0 |
- |
4.8333 |
5800 |
0.0 |
- |
4.875 |
5850 |
0.0 |
- |
4.9167 |
5900 |
0.0 |
- |
4.9583 |
5950 |
0.0 |
- |
5.0 |
6000 |
0.0001 |
0.391 |
5.0417 |
6050 |
0.0 |
- |
5.0833 |
6100 |
0.0001 |
- |
5.125 |
6150 |
0.0 |
- |
5.1667 |
6200 |
0.0 |
- |
5.2083 |
6250 |
0.0 |
- |
5.25 |
6300 |
0.0 |
- |
5.2917 |
6350 |
0.0 |
- |
5.3333 |
6400 |
0.0 |
- |
5.375 |
6450 |
0.0 |
- |
5.4167 |
6500 |
0.0 |
- |
5.4583 |
6550 |
0.0 |
- |
5.5 |
6600 |
0.0001 |
- |
5.5417 |
6650 |
0.0 |
- |
5.5833 |
6700 |
0.0 |
- |
5.625 |
6750 |
0.0 |
- |
5.6667 |
6800 |
0.0 |
- |
5.7083 |
6850 |
0.0 |
- |
5.75 |
6900 |
0.0001 |
- |
5.7917 |
6950 |
0.0 |
- |
5.8333 |
7000 |
0.0001 |
- |
5.875 |
7050 |
0.0 |
- |
5.9167 |
7100 |
0.0 |
- |
5.9583 |
7150 |
0.0 |
- |
6.0 |
7200 |
0.0001 |
0.4026 |
6.0417 |
7250 |
0.0 |
- |
6.0833 |
7300 |
0.0 |
- |
6.125 |
7350 |
0.0 |
- |
6.1667 |
7400 |
0.0 |
- |
6.2083 |
7450 |
0.0 |
- |
6.25 |
7500 |
0.0 |
- |
6.2917 |
7550 |
0.0 |
- |
6.3333 |
7600 |
0.0 |
- |
6.375 |
7650 |
0.0 |
- |
6.4167 |
7700 |
0.0 |
- |
6.4583 |
7750 |
0.0 |
- |
6.5 |
7800 |
0.0 |
- |
6.5417 |
7850 |
0.0 |
- |
6.5833 |
7900 |
0.0 |
- |
6.625 |
7950 |
0.0 |
- |
6.6667 |
8000 |
0.0001 |
- |
6.7083 |
8050 |
0.0005 |
- |
6.75 |
8100 |
0.0063 |
- |
6.7917 |
8150 |
0.0 |
- |
6.8333 |
8200 |
0.0 |
- |
6.875 |
8250 |
0.0 |
- |
6.9167 |
8300 |
0.0 |
- |
6.9583 |
8350 |
0.0 |
- |
7.0 |
8400 |
0.0 |
0.4018 |
7.0417 |
8450 |
0.0 |
- |
7.0833 |
8500 |
0.0 |
- |
7.125 |
8550 |
0.0 |
- |
7.1667 |
8600 |
0.0 |
- |
7.2083 |
8650 |
0.0 |
- |
7.25 |
8700 |
0.0 |
- |
7.2917 |
8750 |
0.0 |
- |
7.3333 |
8800 |
0.0 |
- |
7.375 |
8850 |
0.0 |
- |
7.4167 |
8900 |
0.0 |
- |
7.4583 |
8950 |
0.0 |
- |
7.5 |
9000 |
0.0 |
- |
7.5417 |
9050 |
0.0 |
- |
7.5833 |
9100 |
0.0 |
- |
7.625 |
9150 |
0.0 |
- |
7.6667 |
9200 |
0.0 |
- |
7.7083 |
9250 |
0.0 |
- |
7.75 |
9300 |
0.0 |
- |
7.7917 |
9350 |
0.0 |
- |
7.8333 |
9400 |
0.0 |
- |
7.875 |
9450 |
0.0 |
- |
7.9167 |
9500 |
0.0 |
- |
7.9583 |
9550 |
0.0 |
- |
8.0 |
9600 |
0.0 |
0.4001 |
8.0417 |
9650 |
0.0 |
- |
8.0833 |
9700 |
0.0 |
- |
8.125 |
9750 |
0.0 |
- |
Framework Versions
- Python: 3.10.14
- SetFit: 1.0.3
- Sentence Transformers: 3.0.1
- Transformers: 4.39.0
- PyTorch: 2.2.2
- Datasets: 2.20.0
- Tokenizers: 0.15.2
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}