---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- accuracy
widget:
- text: Ini adalah kisah tentang dua orang yang tidak selaras dan tidak memiliki kesempatan
sendirian, tetapi bersama-sama mereka luar biasa.
- text: ia tidak percaya pada dirinya sendiri, ia tidak memiliki rasa humor ... ia
hanya merasa bosan.
- text: Keberanian band dalam menghadapi represi resmi sangat menginspirasi, terutama
bagi para hippie yang telah menua (termasuk saya sendiri).
- text: film yang cepat, lucu, dan sangat menghibur.
- text: film ini mencapai dampak yang sama besar dengan menyimpan pemikiran-pemikiran
ini tersembunyi seperti halnya film "Quills" yang menunjukkannya.
pipeline_tag: text-classification
inference: true
base_model: firqaaa/indo-sentence-bert-base
model-index:
- name: SetFit with firqaaa/indo-sentence-bert-base
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8
name: Accuracy
---
# SetFit with firqaaa/indo-sentence-bert-base
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [firqaaa/indo-sentence-bert-base](https://huggingface.co/firqaaa/indo-sentence-bert-base) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [firqaaa/indo-sentence-bert-base](https://huggingface.co/firqaaa/indo-sentence-bert-base)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 2 classes
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| positif | - 'benar-benar lucu'
- 'gulungan dari sebuah tong tong yang tersesat'
- ', mereka menemukan rute-rute baru melalui lingkungan yang sudah dikenal'
|
| negatif | - 'tidak menarik atau berbau tidak sedap'
- "telah melakukan kesalahan nyaris fatal dengan menjadi apa yang orang Inggris sebut 'terlalu pintar setengah mati'."
- 'untuk roboh'
|
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.8 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("firqaaa/indo-setfit-bert-base-p1")
# Run inference
preds = model("film yang cepat, lucu, dan sangat menghibur.")
```
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:-------|:----|
| Word count | 1 | 9.4825 | 51 |
| Label | Training Sample Count |
|:--------|:----------------------|
| negatif | 200 |
| positif | 200 |
### Training Hyperparameters
- batch_size: (32, 32)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:-------:|:--------:|:-------------:|:---------------:|
| 0.0004 | 1 | 0.3079 | - |
| 0.0199 | 50 | 0.3644 | - |
| 0.0398 | 100 | 0.2816 | - |
| 0.0597 | 150 | 0.2254 | - |
| 0.0796 | 200 | 0.1798 | - |
| 0.0995 | 250 | 0.0478 | - |
| 0.1194 | 300 | 0.0049 | - |
| 0.1393 | 350 | 0.0016 | - |
| 0.1592 | 400 | 0.0011 | - |
| 0.1791 | 450 | 0.0005 | - |
| 0.1990 | 500 | 0.0003 | - |
| 0.2189 | 550 | 0.0004 | - |
| 0.2388 | 600 | 0.0003 | - |
| 0.2587 | 650 | 0.0003 | - |
| 0.2786 | 700 | 0.0001 | - |
| 0.2984 | 750 | 0.0002 | - |
| 0.3183 | 800 | 0.0001 | - |
| 0.3382 | 850 | 0.0001 | - |
| 0.3581 | 900 | 0.0001 | - |
| 0.3780 | 950 | 0.0001 | - |
| 0.3979 | 1000 | 0.0001 | - |
| 0.4178 | 1050 | 0.0001 | - |
| 0.4377 | 1100 | 0.0001 | - |
| 0.4576 | 1150 | 0.0001 | - |
| 0.4775 | 1200 | 0.0001 | - |
| 0.4974 | 1250 | 0.0001 | - |
| 0.5173 | 1300 | 0.0001 | - |
| 0.5372 | 1350 | 0.0001 | - |
| 0.5571 | 1400 | 0.0001 | - |
| 0.5770 | 1450 | 0.0001 | - |
| 0.5969 | 1500 | 0.0001 | - |
| 0.6168 | 1550 | 0.0001 | - |
| 0.6367 | 1600 | 0.0001 | - |
| 0.6566 | 1650 | 0.0001 | - |
| 0.6765 | 1700 | 0.0002 | - |
| 0.6964 | 1750 | 0.0001 | - |
| 0.7163 | 1800 | 0.0001 | - |
| 0.7362 | 1850 | 0.0001 | - |
| 0.7561 | 1900 | 0.0001 | - |
| 0.7760 | 1950 | 0.0001 | - |
| 0.7959 | 2000 | 0.0001 | - |
| 0.8158 | 2050 | 0.0001 | - |
| 0.8357 | 2100 | 0.0001 | - |
| 0.8556 | 2150 | 0.0001 | - |
| 0.8754 | 2200 | 0.0001 | - |
| 0.8953 | 2250 | 0.0 | - |
| 0.9152 | 2300 | 0.0001 | - |
| 0.9351 | 2350 | 0.0 | - |
| 0.9550 | 2400 | 0.0 | - |
| 0.9749 | 2450 | 0.0 | - |
| 0.9948 | 2500 | 0.0 | - |
| **1.0** | **2513** | **-** | **0.2622** |
| 1.0147 | 2550 | 0.0 | - |
| 1.0346 | 2600 | 0.0 | - |
| 1.0545 | 2650 | 0.0 | - |
| 1.0744 | 2700 | 0.0 | - |
| 1.0943 | 2750 | 0.0 | - |
| 1.1142 | 2800 | 0.0 | - |
| 1.1341 | 2850 | 0.0 | - |
| 1.1540 | 2900 | 0.0 | - |
| 1.1739 | 2950 | 0.0 | - |
| 1.1938 | 3000 | 0.0 | - |
| 1.2137 | 3050 | 0.0 | - |
| 1.2336 | 3100 | 0.0 | - |
| 1.2535 | 3150 | 0.0 | - |
| 1.2734 | 3200 | 0.0 | - |
| 1.2933 | 3250 | 0.0 | - |
| 1.3132 | 3300 | 0.0 | - |
| 1.3331 | 3350 | 0.0 | - |
| 1.3530 | 3400 | 0.0 | - |
| 1.3729 | 3450 | 0.0 | - |
| 1.3928 | 3500 | 0.0 | - |
| 1.4127 | 3550 | 0.0 | - |
| 1.4326 | 3600 | 0.0 | - |
| 1.4524 | 3650 | 0.0 | - |
| 1.4723 | 3700 | 0.0 | - |
| 1.4922 | 3750 | 0.0 | - |
| 1.5121 | 3800 | 0.0 | - |
| 1.5320 | 3850 | 0.0 | - |
| 1.5519 | 3900 | 0.0 | - |
| 1.5718 | 3950 | 0.0 | - |
| 1.5917 | 4000 | 0.0 | - |
| 1.6116 | 4050 | 0.0 | - |
| 1.6315 | 4100 | 0.0 | - |
| 1.6514 | 4150 | 0.0 | - |
| 1.6713 | 4200 | 0.0 | - |
| 1.6912 | 4250 | 0.0 | - |
| 1.7111 | 4300 | 0.0 | - |
| 1.7310 | 4350 | 0.0 | - |
| 1.7509 | 4400 | 0.0 | - |
| 1.7708 | 4450 | 0.0 | - |
| 1.7907 | 4500 | 0.0 | - |
| 1.8106 | 4550 | 0.0 | - |
| 1.8305 | 4600 | 0.0 | - |
| 1.8504 | 4650 | 0.0 | - |
| 1.8703 | 4700 | 0.0 | - |
| 1.8902 | 4750 | 0.0 | - |
| 1.9101 | 4800 | 0.0 | - |
| 1.9300 | 4850 | 0.0 | - |
| 1.9499 | 4900 | 0.0 | - |
| 1.9698 | 4950 | 0.0 | - |
| 1.9897 | 5000 | 0.0 | - |
| 2.0 | 5026 | - | 0.269 |
| 2.0096 | 5050 | 0.0 | - |
| 2.0294 | 5100 | 0.0 | - |
| 2.0493 | 5150 | 0.0 | - |
| 2.0692 | 5200 | 0.0 | - |
| 2.0891 | 5250 | 0.0 | - |
| 2.1090 | 5300 | 0.0 | - |
| 2.1289 | 5350 | 0.0 | - |
| 2.1488 | 5400 | 0.0 | - |
| 2.1687 | 5450 | 0.0 | - |
| 2.1886 | 5500 | 0.0 | - |
| 2.2085 | 5550 | 0.0 | - |
| 2.2284 | 5600 | 0.0 | - |
| 2.2483 | 5650 | 0.0 | - |
| 2.2682 | 5700 | 0.0 | - |
| 2.2881 | 5750 | 0.0 | - |
| 2.3080 | 5800 | 0.0 | - |
| 2.3279 | 5850 | 0.0 | - |
| 2.3478 | 5900 | 0.0 | - |
| 2.3677 | 5950 | 0.0 | - |
| 2.3876 | 6000 | 0.0 | - |
| 2.4075 | 6050 | 0.0 | - |
| 2.4274 | 6100 | 0.0 | - |
| 2.4473 | 6150 | 0.0 | - |
| 2.4672 | 6200 | 0.0 | - |
| 2.4871 | 6250 | 0.0 | - |
| 2.5070 | 6300 | 0.0 | - |
| 2.5269 | 6350 | 0.0 | - |
| 2.5468 | 6400 | 0.0 | - |
| 2.5667 | 6450 | 0.0 | - |
| 2.5865 | 6500 | 0.0 | - |
| 2.6064 | 6550 | 0.0 | - |
| 2.6263 | 6600 | 0.0 | - |
| 2.6462 | 6650 | 0.0 | - |
| 2.6661 | 6700 | 0.0 | - |
| 2.6860 | 6750 | 0.0 | - |
| 2.7059 | 6800 | 0.0 | - |
| 2.7258 | 6850 | 0.0 | - |
| 2.7457 | 6900 | 0.0 | - |
| 2.7656 | 6950 | 0.0 | - |
| 2.7855 | 7000 | 0.0 | - |
| 2.8054 | 7050 | 0.0 | - |
| 2.8253 | 7100 | 0.0 | - |
| 2.8452 | 7150 | 0.0 | - |
| 2.8651 | 7200 | 0.0 | - |
| 2.8850 | 7250 | 0.0 | - |
| 2.9049 | 7300 | 0.0 | - |
| 2.9248 | 7350 | 0.0 | - |
| 2.9447 | 7400 | 0.0 | - |
| 2.9646 | 7450 | 0.0 | - |
| 2.9845 | 7500 | 0.0 | - |
| 3.0 | 7539 | - | 0.2744 |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.10.13
- SetFit: 1.0.3
- Sentence Transformers: 2.3.1
- Transformers: 4.36.2
- PyTorch: 2.1.2+cu121
- Datasets: 2.16.1
- Tokenizers: 0.15.0
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```