metadata

library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: >-
      Ini adalah kisah tentang dua orang yang tidak selaras dan tidak memiliki
      kesempatan sendirian, tetapi bersama-sama mereka luar biasa.
  - text: >-
      ia tidak percaya pada dirinya sendiri, ia tidak memiliki rasa humor ... ia
      hanya merasa bosan.
  - text: >-
      Keberanian band dalam menghadapi represi resmi sangat menginspirasi,
      terutama bagi para hippie yang telah menua (termasuk saya sendiri).
  - text: film yang cepat, lucu, dan sangat menghibur.
  - text: >-
      film ini mencapai dampak yang sama besar dengan menyimpan
      pemikiran-pemikiran ini tersembunyi seperti halnya film "Quills" yang
      menunjukkannya.
pipeline_tag: text-classification
inference: true
base_model: firqaaa/indo-sentence-bert-base
model-index:
  - name: SetFit with firqaaa/indo-sentence-bert-base
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.82
            name: Accuracy

SetFit with firqaaa/indo-sentence-bert-base

This is a SetFit model that can be used for Text Classification. This SetFit model uses firqaaa/indo-sentence-bert-base as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: firqaaa/indo-sentence-bert-base
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
positif	'Layak untuk kunjungan lain.' 'gulungan dari sebuah tong tong yang tersesat' 'adalah film yang hebat .'
negatif	'Anda berada di rumah menonton film itu daripada di bioskop menonton yang ini.' 'dengan banyak warna biru gelap dan merah muda yang serius' 'hal buruk'

Evaluation

Metrics

Label	Accuracy
all	0.82

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("firqaaa/indo-setfit-bert-base-p1")
# Run inference
preds = model("film yang cepat, lucu, dan sangat menghibur.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	9.4029	51

Label	Training Sample Count
negatif	350
positif	350

Training Hyperparameters

batch_size: (32, 32)
num_epochs: (3, 3)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.3328	-
0.0065	50	0.4117	-
0.0130	100	0.2903	-
0.0195	150	0.3104	-
0.0260	200	0.2411	-
0.0326	250	0.2341	-
0.0391	300	0.2144	-
0.0456	350	0.1785	-
0.0521	400	0.1649	-
0.0586	450	0.037	-
0.0651	500	0.0447	-
0.0716	550	0.0472	-
0.0781	600	0.0361	-
0.0846	650	0.0016	-
0.0912	700	0.0013	-
0.0977	750	0.0011	-
0.1042	800	0.0006	-
0.1107	850	0.0009	-
0.1172	900	0.0006	-
0.1237	950	0.0004	-
0.1302	1000	0.0004	-
0.1367	1050	0.0005	-
0.1432	1100	0.0004	-
0.1498	1150	0.0002	-
0.1563	1200	0.0003	-
0.1628	1250	0.0004	-
0.1693	1300	0.0003	-
0.1758	1350	0.0001	-
0.1823	1400	0.0002	-
0.1888	1450	0.0003	-
0.1953	1500	0.0002	-
0.2018	1550	0.0002	-
0.2084	1600	0.0287	-
0.2149	1650	0.0003	-
0.2214	1700	0.0002	-
0.2279	1750	0.0002	-
0.2344	1800	0.0002	-
0.2409	1850	0.0004	-
0.2474	1900	0.0001	-
0.2539	1950	0.0001	-
0.2605	2000	0.0001	-
0.2670	2050	0.0001	-
0.2735	2100	0.0001	-
0.2800	2150	0.0001	-
0.2865	2200	0.0001	-
0.2930	2250	0.0003	-
0.2995	2300	0.0001	-
0.3060	2350	0.0002	-
0.3125	2400	0.0001	-
0.3191	2450	0.0	-
0.3256	2500	0.0001	-
0.3321	2550	0.0001	-
0.3386	2600	0.0001	-
0.3451	2650	0.0001	-
0.3516	2700	0.0003	-
0.3581	2750	0.0002	-
0.3646	2800	0.0003	-
0.3711	2850	0.0002	-
0.3777	2900	0.0002	-
0.3842	2950	0.0001	-
0.3907	3000	0.0001	-
0.3972	3050	0.0001	-
0.4037	3100	0.0001	-
0.4102	3150	0.0	-
0.4167	3200	0.0001	-
0.4232	3250	0.0	-
0.4297	3300	0.0001	-
0.4363	3350	0.0001	-
0.4428	3400	0.0001	-
0.4493	3450	0.0001	-
0.4558	3500	0.0001	-
0.4623	3550	0.0	-
0.4688	3600	0.0001	-
0.4753	3650	0.0001	-
0.4818	3700	0.0001	-
0.4883	3750	0.0	-
0.4949	3800	0.0001	-
0.5014	3850	0.0	-
0.5079	3900	0.0	-
0.5144	3950	0.0	-
0.5209	4000	0.0	-
0.5274	4050	0.0	-
0.5339	4100	0.0	-
0.5404	4150	0.0	-
0.5469	4200	0.0	-
0.5535	4250	0.0	-
0.5600	4300	0.0	-
0.5665	4350	0.0001	-
0.5730	4400	0.0	-
0.5795	4450	0.0	-
0.5860	4500	0.0	-
0.5925	4550	0.0	-
0.5990	4600	0.0	-
0.6055	4650	0.0	-
0.6121	4700	0.0	-
0.6186	4750	0.0	-
0.6251	4800	0.0	-
0.6316	4850	0.0001	-
0.6381	4900	0.0001	-
0.6446	4950	0.0086	-
0.6511	5000	0.0	-
0.6576	5050	0.0	-
0.6641	5100	0.0	-
0.6707	5150	0.0	-
0.6772	5200	0.0	-
0.6837	5250	0.0007	-
0.6902	5300	0.0	-
0.6967	5350	0.0001	-
0.7032	5400	0.0	-
0.7097	5450	0.0001	-
0.7162	5500	0.0	-
0.7228	5550	0.0	-
0.7293	5600	0.0	-
0.7358	5650	0.0	-
0.7423	5700	0.0003	-
0.7488	5750	0.0001	-
0.7553	5800	0.0	-
0.7618	5850	0.0	-
0.7683	5900	0.0	-
0.7748	5950	0.0	-
0.7814	6000	0.0	-
0.7879	6050	0.0	-
0.7944	6100	0.0	-
0.8009	6150	0.0	-
0.8074	6200	0.0	-
0.8139	6250	0.0	-
0.8204	6300	0.0	-
0.8269	6350	0.0	-
0.8334	6400	0.0	-
0.8400	6450	0.0	-
0.8465	6500	0.0	-
0.8530	6550	0.0	-
0.8595	6600	0.0	-
0.8660	6650	0.0	-
0.8725	6700	0.0	-
0.8790	6750	0.0	-
0.8855	6800	0.0	-
0.8920	6850	0.0	-
0.8986	6900	0.0	-
0.9051	6950	0.0	-
0.9116	7000	0.0	-
0.9181	7050	0.0	-
0.9246	7100	0.0	-
0.9311	7150	0.0	-
0.9376	7200	0.0	-
0.9441	7250	0.0	-
0.9506	7300	0.0	-
0.9572	7350	0.0	-
0.9637	7400	0.0	-
0.9702	7450	0.0	-
0.9767	7500	0.0	-
0.9832	7550	0.0	-
0.9897	7600	0.0	-
0.9962	7650	0.0	-
1.0	7679	-	0.2894
1.0027	7700	0.0	-
1.0092	7750	0.0	-
1.0158	7800	0.0	-
1.0223	7850	0.0	-
1.0288	7900	0.0	-
1.0353	7950	0.0	-
1.0418	8000	0.0	-
1.0483	8050	0.0	-
1.0548	8100	0.0	-
1.0613	8150	0.0	-
1.0678	8200	0.0	-
1.0744	8250	0.0	-
1.0809	8300	0.0	-
1.0874	8350	0.0	-
1.0939	8400	0.0	-
1.1004	8450	0.0	-
1.1069	8500	0.0	-
1.1134	8550	0.0	-
1.1199	8600	0.0	-
1.1264	8650	0.0	-
1.1330	8700	0.0	-
1.1395	8750	0.0	-
1.1460	8800	0.0	-
1.1525	8850	0.0	-
1.1590	8900	0.0	-
1.1655	8950	0.0	-
1.1720	9000	0.0	-
1.1785	9050	0.0	-
1.1851	9100	0.0	-
1.1916	9150	0.0	-
1.1981	9200	0.0	-
1.2046	9250	0.0	-
1.2111	9300	0.0	-
1.2176	9350	0.0	-
1.2241	9400	0.0	-
1.2306	9450	0.0	-
1.2371	9500	0.0	-
1.2437	9550	0.0	-
1.2502	9600	0.0	-
1.2567	9650	0.0	-
1.2632	9700	0.0	-
1.2697	9750	0.0	-
1.2762	9800	0.0	-
1.2827	9850	0.0	-
1.2892	9900	0.0	-
1.2957	9950	0.0	-
1.3023	10000	0.0	-
1.3088	10050	0.0	-
1.3153	10100	0.0	-
1.3218	10150	0.0	-
1.3283	10200	0.0	-
1.3348	10250	0.0	-
1.3413	10300	0.0	-
1.3478	10350	0.0	-
1.3543	10400	0.0	-
1.3609	10450	0.0	-
1.3674	10500	0.0	-
1.3739	10550	0.0	-
1.3804	10600	0.0	-
1.3869	10650	0.0	-
1.3934	10700	0.0	-
1.3999	10750	0.0	-
1.4064	10800	0.0	-
1.4129	10850	0.0	-
1.4195	10900	0.0	-
1.4260	10950	0.0	-
1.4325	11000	0.0	-
1.4390	11050	0.0	-
1.4455	11100	0.0	-
1.4520	11150	0.0	-
1.4585	11200	0.0	-
1.4650	11250	0.0	-
1.4715	11300	0.0	-
1.4781	11350	0.0	-
1.4846	11400	0.0	-
1.4911	11450	0.0	-
1.4976	11500	0.0	-
1.5041	11550	0.0	-
1.5106	11600	0.0	-
1.5171	11650	0.0	-
1.5236	11700	0.0	-
1.5301	11750	0.0	-
1.5367	11800	0.0	-
1.5432	11850	0.0	-
1.5497	11900	0.0	-
1.5562	11950	0.0	-
1.5627	12000	0.0	-
1.5692	12050	0.0	-
1.5757	12100	0.0	-
1.5822	12150	0.0	-
1.5887	12200	0.0	-
1.5953	12250	0.0	-
1.6018	12300	0.0	-
1.6083	12350	0.0	-
1.6148	12400	0.0	-
1.6213	12450	0.0	-
1.6278	12500	0.0	-
1.6343	12550	0.0	-
1.6408	12600	0.0	-
1.6473	12650	0.0	-
1.6539	12700	0.0	-
1.6604	12750	0.0	-
1.6669	12800	0.0	-
1.6734	12850	0.0	-
1.6799	12900	0.0	-
1.6864	12950	0.0	-
1.6929	13000	0.0	-
1.6994	13050	0.0	-
1.7060	13100	0.0	-
1.7125	13150	0.0	-
1.7190	13200	0.0	-
1.7255	13250	0.0	-
1.7320	13300	0.0	-
1.7385	13350	0.0	-
1.7450	13400	0.0	-
1.7515	13450	0.0	-
1.7580	13500	0.0	-
1.7646	13550	0.0	-
1.7711	13600	0.0	-
1.7776	13650	0.0	-
1.7841	13700	0.0	-
1.7906	13750	0.0	-
1.7971	13800	0.0	-
1.8036	13850	0.0	-
1.8101	13900	0.0	-
1.8166	13950	0.0	-
1.8232	14000	0.0	-
1.8297	14050	0.0	-
1.8362	14100	0.0	-
1.8427	14150	0.0	-
1.8492	14200	0.0	-
1.8557	14250	0.0	-
1.8622	14300	0.0	-
1.8687	14350	0.0	-
1.8752	14400	0.0	-
1.8818	14450	0.0	-
1.8883	14500	0.0	-
1.8948	14550	0.0	-
1.9013	14600	0.0	-
1.9078	14650	0.0	-
1.9143	14700	0.0	-
1.9208	14750	0.0	-
1.9273	14800	0.0	-
1.9338	14850	0.0	-
1.9404	14900	0.0	-
1.9469	14950	0.0	-
1.9534	15000	0.0	-
1.9599	15050	0.0	-
1.9664	15100	0.0	-
1.9729	15150	0.0	-
1.9794	15200	0.0	-
1.9859	15250	0.0	-
1.9924	15300	0.0	-
1.9990	15350	0.0	-
2.0	15358	-	0.2831
2.0055	15400	0.0	-
2.0120	15450	0.0	-
2.0185	15500	0.0	-
2.0250	15550	0.0	-
2.0315	15600	0.0	-
2.0380	15650	0.0	-
2.0445	15700	0.0	-
2.0510	15750	0.0	-
2.0576	15800	0.0	-
2.0641	15850	0.0	-
2.0706	15900	0.0	-
2.0771	15950	0.0	-
2.0836	16000	0.0	-
2.0901	16050	0.0	-
2.0966	16100	0.0	-
2.1031	16150	0.0	-
2.1096	16200	0.0	-
2.1162	16250	0.0	-
2.1227	16300	0.0	-
2.1292	16350	0.0	-
2.1357	16400	0.0	-
2.1422	16450	0.0	-
2.1487	16500	0.0	-
2.1552	16550	0.0	-
2.1617	16600	0.0	-
2.1683	16650	0.0	-
2.1748	16700	0.0	-
2.1813	16750	0.0	-
2.1878	16800	0.0	-
2.1943	16850	0.0	-
2.2008	16900	0.0	-
2.2073	16950	0.0	-
2.2138	17000	0.0	-
2.2203	17050	0.0	-
2.2269	17100	0.0	-
2.2334	17150	0.0	-
2.2399	17200	0.0	-
2.2464	17250	0.0	-
2.2529	17300	0.0	-
2.2594	17350	0.0	-
2.2659	17400	0.0	-
2.2724	17450	0.0	-
2.2789	17500	0.0	-
2.2855	17550	0.0	-
2.2920	17600	0.0	-
2.2985	17650	0.0	-
2.3050	17700	0.0	-
2.3115	17750	0.0	-
2.3180	17800	0.0	-
2.3245	17850	0.0	-
2.3310	17900	0.0	-
2.3375	17950	0.0	-
2.3441	18000	0.0	-
2.3506	18050	0.0	-
2.3571	18100	0.0	-
2.3636	18150	0.0	-
2.3701	18200	0.0	-
2.3766	18250	0.0	-
2.3831	18300	0.0	-
2.3896	18350	0.0	-
2.3961	18400	0.0	-
2.4027	18450	0.0	-
2.4092	18500	0.0	-
2.4157	18550	0.0	-
2.4222	18600	0.0	-
2.4287	18650	0.0	-
2.4352	18700	0.0	-
2.4417	18750	0.0	-
2.4482	18800	0.0	-
2.4547	18850	0.0	-
2.4613	18900	0.0	-
2.4678	18950	0.0	-
2.4743	19000	0.0	-
2.4808	19050	0.0	-
2.4873	19100	0.0	-
2.4938	19150	0.0	-
2.5003	19200	0.0	-
2.5068	19250	0.0	-
2.5133	19300	0.0	-
2.5199	19350	0.0	-
2.5264	19400	0.0	-
2.5329	19450	0.0	-
2.5394	19500	0.0	-
2.5459	19550	0.0	-
2.5524	19600	0.0	-
2.5589	19650	0.0	-
2.5654	19700	0.0	-
2.5719	19750	0.0	-
2.5785	19800	0.0	-
2.5850	19850	0.0	-
2.5915	19900	0.0	-
2.5980	19950	0.0	-
2.6045	20000	0.0	-
2.6110	20050	0.0	-
2.6175	20100	0.0	-
2.6240	20150	0.0	-
2.6306	20200	0.0	-
2.6371	20250	0.0	-
2.6436	20300	0.0	-
2.6501	20350	0.0	-
2.6566	20400	0.0	-
2.6631	20450	0.0	-
2.6696	20500	0.0	-
2.6761	20550	0.0	-
2.6826	20600	0.0	-
2.6892	20650	0.0	-
2.6957	20700	0.0	-
2.7022	20750	0.0	-
2.7087	20800	0.0	-
2.7152	20850	0.0	-
2.7217	20900	0.0	-
2.7282	20950	0.0	-
2.7347	21000	0.0	-
2.7412	21050	0.0	-
2.7478	21100	0.0	-
2.7543	21150	0.0	-
2.7608	21200	0.0	-
2.7673	21250	0.0	-
2.7738	21300	0.0	-
2.7803	21350	0.0	-
2.7868	21400	0.0	-
2.7933	21450	0.0	-
2.7998	21500	0.0	-
2.8064	21550	0.0	-
2.8129	21600	0.0	-
2.8194	21650	0.0	-
2.8259	21700	0.0	-
2.8324	21750	0.0	-
2.8389	21800	0.0	-
2.8454	21850	0.0	-
2.8519	21900	0.0	-
2.8584	21950	0.0	-
2.8650	22000	0.0	-
2.8715	22050	0.0	-
2.8780	22100	0.0	-
2.8845	22150	0.0	-
2.8910	22200	0.0	-
2.8975	22250	0.0	-
2.9040	22300	0.0	-
2.9105	22350	0.0	-
2.9170	22400	0.0	-
2.9236	22450	0.0	-
2.9301	22500	0.0	-
2.9366	22550	0.0	-
2.9431	22600	0.0	-
2.9496	22650	0.0	-
2.9561	22700	0.0	-
2.9626	22750	0.0	-
2.9691	22800	0.0	-
2.9756	22850	0.0	-
2.9822	22900	0.0	-
2.9887	22950	0.0	-
2.9952	23000	0.0	-
3.0	23037	-	0.2771

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.13
SetFit: 1.0.3
Sentence Transformers: 2.2.2
Transformers: 4.36.2
PyTorch: 2.1.2+cu121
Datasets: 2.16.1
Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}