twright8/setfit_lobbying_classifier

SetFit with BAAI/bge-base-en-v1.5

This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-base-en-v1.5 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

undefined = Health 1 = Housing 2 = Defence 3 = Climate

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: BAAI/bge-base-en-v1.5
• Classification head: a SetFitHead instance
• Maximum Sequence Length: 512 tokens

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Evaluation

Metrics

Label	F1	Accuracy
all	0.9667	0.9421

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("twright8/setfit_lobbying_classifier")
# Run inference
preds = model("Growth")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	39.4538	282

Training Hyperparameters

• batch_size: (16, 2)
• num_epochs: (4, 9)
• max_steps: -1
• sampling_strategy: undersampling
• body_learning_rate: (1.0797496673911536e-05, 3.457046714445997e-05)
• head_learning_rate: 0.0004470582121407239
• loss: CoSENTLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: True
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0002	1	2.097	-
0.0077	50	8.5514	-
0.0155	100	3.5635	-
0.0232	150	2.9266	-
0.0310	200	2.1173	-
0.0387	250	3.1002	-
0.0465	300	3.6942	-
0.0542	350	3.4905	-
0.0620	400	4.0804	-
0.0697	450	1.6071	-
0.0774	500	2.3018	-
0.0852	550	2.3876	-
0.0929	600	0.2511	-
0.1007	650	0.2435	-
0.1084	700	2.2596	-
0.1162	750	1.121	-
0.1239	800	0.0907	-
0.1317	850	0.2172	-
0.1394	900	3.06	-
0.1471	950	0.0074	-
0.1549	1000	0.457	-
0.1626	1050	0.0575	-
0.1704	1100	0.0002	-
0.1781	1150	0.0003	-
0.1859	1200	0.0047	-
0.1936	1250	0.0004	-
0.2014	1300	0.0006	-
0.2091	1350	0.0027	-
0.2169	1400	0.0004	-
0.2246	1450	0.0009	-
0.2323	1500	0.0006	-
0.2401	1550	0.0003	-
0.2478	1600	0.0077	-
0.2556	1650	0.0004	-
0.2633	1700	0.0003	-
0.2711	1750	0.0005	-
0.2788	1800	0.0004	-
0.2866	1850	0.0007	-
0.2943	1900	0.0009	-
0.3020	1950	0.0062	-
0.3098	2000	0.0003	-
0.3175	2050	0.0001	-
0.3253	2100	0.0685	-
0.3330	2150	0.0008	-
0.3408	2200	0.0	-
0.3485	2250	0.0004	-
0.3563	2300	0.0004	-
0.3640	2350	0.0002	-
0.3717	2400	0.0001	-
0.3795	2450	0.0004	-
0.3872	2500	0.0004	-
0.3950	2550	0.0001	-
0.4027	2600	0.0001	-
0.4105	2650	0.0001	-
0.4182	2700	0.0005	-
0.4260	2750	0.0002	-
0.4337	2800	0.0001	-
0.4414	2850	0.0003	-
0.4492	2900	0.0005	-
0.4569	2950	0.0014	-
0.4647	3000	0.0001	-
0.4724	3050	0.0001	-
0.4802	3100	0.0002	-
0.4879	3150	0.0	-
0.4957	3200	0.0006	-
0.5034	3250	0.0	-
0.5112	3300	0.0	-
0.5189	3350	0.0002	-
0.5266	3400	0.0001	-
0.5344	3450	0.0006	-
0.5421	3500	0.0002	-
0.5499	3550	0.0001	-
0.5576	3600	0.0001	-
0.5654	3650	0.0001	-
0.5731	3700	0.0	-
0.5809	3750	0.0002	-
0.5886	3800	0.0	-
0.5963	3850	0.0044	-
0.6041	3900	0.0002	-
0.6118	3950	0.0001	-
0.6196	4000	0.0003	-
0.6273	4050	0.0005	-
0.6351	4100	0.0002	-
0.6428	4150	0.0	-
0.6506	4200	0.0003	-
0.6583	4250	0.0	-
0.6660	4300	0.0001	-
0.6738	4350	0.0	-
0.6815	4400	0.0008	-
0.6893	4450	0.0	-
0.6970	4500	0.0004	-
0.7048	4550	0.0001	-
0.7125	4600	0.0	-
0.7203	4650	0.0	-
0.7280	4700	0.0	-
0.7357	4750	0.0001	-
0.7435	4800	0.0001	-
0.7512	4850	0.001	-
0.7590	4900	0.0001	-
0.7667	4950	0.0	-
0.7745	5000	0.0001	-
0.7822	5050	0.0	-
0.7900	5100	0.0018	-
0.7977	5150	0.0001	-
0.8055	5200	0.0	-
0.8132	5250	0.0003	-
0.8209	5300	0.0003	-
0.8287	5350	0.0003	-
0.8364	5400	0.0001	-
0.8442	5450	0.0001	-
0.8519	5500	0.0001	-
0.8597	5550	0.0001	-
0.8674	5600	0.0001	-
0.8752	5650	0.0	-
0.8829	5700	0.0003	-
0.8906	5750	0.0003	-
0.8984	5800	0.0001	-
0.9061	5850	0.0001	-
0.9139	5900	0.0002	-
0.9216	5950	0.0	-
0.9294	6000	0.0001	-
0.9371	6050	0.0	-
0.9449	6100	0.0	-
0.9526	6150	0.0001	-
0.9603	6200	0.0	-
0.9681	6250	0.0001	-
0.9758	6300	0.0002	-
0.9836	6350	0.0	-
0.9913	6400	0.0	-
0.9991	6450	0.0002	-
1.0	6456	-	1.3837
1.0068	6500	0.0001	-
1.0146	6550	0.0001	-
1.0223	6600	0.0002	-
1.0300	6650	0.0001	-
1.0378	6700	0.0005	-
1.0455	6750	0.0001	-
1.0533	6800	0.0001	-
1.0610	6850	0.0	-
1.0688	6900	0.0	-
1.0765	6950	0.0009	-
1.0843	7000	0.0	-
1.0920	7050	0.0032	-
1.0998	7100	0.0001	-
1.1075	7150	0.0001	-
1.1152	7200	0.0001	-
1.1230	7250	0.0	-
1.1307	7300	0.0001	-
1.1385	7350	0.0	-
1.1462	7400	0.0	-
1.1540	7450	0.0002	-
1.1617	7500	0.0	-
1.1695	7550	0.0427	-
1.1772	7600	0.0	-
1.1849	7650	0.0	-
1.1927	7700	0.0	-
1.2004	7750	0.0002	-
1.2082	7800	0.0	-
1.2159	7850	0.0	-
1.2237	7900	0.0	-
1.2314	7950	0.0	-
1.2392	8000	0.0001	-
1.2469	8050	0.0	-
1.2546	8100	0.0001	-
1.2624	8150	0.0	-
1.2701	8200	0.0	-
1.2779	8250	0.0	-
1.2856	8300	0.0	-
1.2934	8350	0.0	-
1.3011	8400	0.0	-
1.3089	8450	0.0	-
1.3166	8500	0.0	-
1.3243	8550	0.0001	-
1.3321	8600	0.0	-
1.3398	8650	0.0002	-
1.3476	8700	0.0	-
1.3553	8750	0.0006	-
1.3631	8800	0.0	-
1.3708	8850	0.0	-
1.3786	8900	0.0001	-
1.3863	8950	0.0	-
1.3941	9000	0.0001	-
1.4018	9050	0.0	-
1.4095	9100	0.0002	-
1.4173	9150	0.0	-
1.4250	9200	0.0	-
1.4328	9250	0.0	-
1.4405	9300	0.0	-
1.4483	9350	0.0	-
1.4560	9400	0.0	-
1.4638	9450	0.0	-
1.4715	9500	0.0	-
1.4792	9550	0.0	-
1.4870	9600	0.0	-
1.4947	9650	0.0005	-
1.5025	9700	0.0	-
1.5102	9750	0.0001	-
1.5180	9800	0.0001	-
1.5257	9850	0.0001	-
1.5335	9900	0.0	-
1.5412	9950	0.0	-
1.5489	10000	0.0	-
1.5567	10050	0.0	-
1.5644	10100	0.0001	-
1.5722	10150	0.0	-
1.5799	10200	0.0002	-
1.5877	10250	0.0001	-
1.5954	10300	0.0005	-
1.6032	10350	0.0	-
1.6109	10400	0.0	-
1.6186	10450	0.0003	-
1.6264	10500	0.0002	-
1.6341	10550	0.0	-
1.6419	10600	0.0	-
1.6496	10650	0.0001	-
1.6574	10700	0.0002	-
1.6651	10750	0.0002	-
1.6729	10800	0.0054	-
1.6806	10850	0.0005	-
1.6884	10900	0.0001	-
1.6961	10950	0.0	-
1.7038	11000	0.0	-
1.7116	11050	0.0001	-
1.7193	11100	0.0001	-
1.7271	11150	0.0	-
1.7348	11200	0.0001	-
1.7426	11250	0.0	-
1.7503	11300	0.0001	-
1.7581	11350	0.0004	-
1.7658	11400	0.0	-
1.7735	11450	0.0001	-
1.7813	11500	0.0	-
1.7890	11550	0.0	-
1.7968	11600	0.0	-
1.8045	11650	0.0	-
1.8123	11700	0.0001	-
1.8200	11750	0.0002	-
1.8278	11800	0.0	-
1.8355	11850	0.0001	-
1.8432	11900	0.0	-
1.8510	11950	0.0001	-
1.8587	12000	0.0	-
1.8665	12050	0.0	-
1.8742	12100	0.0	-
1.8820	12150	0.0001	-
1.8897	12200	0.0	-
1.8975	12250	0.0	-
1.9052	12300	0.0	-
1.9129	12350	0.0	-
1.9207	12400	0.0	-
1.9284	12450	0.0	-
1.9362	12500	0.0	-
1.9439	12550	0.0003	-
1.9517	12600	0.0001	-
1.9594	12650	0.0	-
1.9672	12700	0.0001	-
1.9749	12750	0.0	-
1.9827	12800	0.0	-
1.9904	12850	0.0	-
1.9981	12900	0.0001	-
2.0	12912	-	2.611
2.0059	12950	0.0	-
2.0136	13000	0.0001	-
2.0214	13050	0.0001	-
2.0291	13100	0.0	-
2.0369	13150	0.0	-
2.0446	13200	0.0001	-
2.0524	13250	0.0	-
2.0601	13300	0.0002	-
2.0678	13350	0.0	-
2.0756	13400	0.0	-
2.0833	13450	0.0001	-
2.0911	13500	0.0001	-
2.0988	13550	0.0003	-
2.1066	13600	0.0	-
2.1143	13650	0.0001	-
2.1221	13700	0.0001	-
2.1298	13750	0.0001	-
2.1375	13800	0.0001	-
2.1453	13850	0.0	-
2.1530	13900	0.0	-
2.1608	13950	0.0	-
2.1685	14000	0.0	-
2.1763	14050	0.0	-
2.1840	14100	0.0001	-
2.1918	14150	0.0	-
2.1995	14200	0.0	-
2.2072	14250	0.0001	-
2.2150	14300	0.0	-
2.2227	14350	0.0	-
2.2305	14400	0.0004	-
2.2382	14450	0.0001	-
2.2460	14500	0.0	-
2.2537	14550	0.0003	-
2.2615	14600	0.0	-
2.2692	14650	0.0001	-
2.2770	14700	0.0001	-
2.2847	14750	0.0	-
2.2924	14800	0.0	-
2.3002	14850	0.0005	-
2.3079	14900	0.0	-
2.3157	14950	0.0002	-
2.3234	15000	0.0	-
2.3312	15050	0.0	-
2.3389	15100	0.0001	-
2.3467	15150	0.0001	-
2.3544	15200	0.0002	-
2.3621	15250	0.0001	-
2.3699	15300	0.0	-
2.3776	15350	0.0	-
2.3854	15400	0.0002	-
2.3931	15450	0.0003	-
2.4009	15500	0.0	-
2.4086	15550	0.0	-
2.4164	15600	0.0	-
2.4241	15650	0.0001	-
2.4318	15700	0.0	-
2.4396	15750	0.0	-
2.4473	15800	0.0002	-
2.4551	15850	0.0	-
2.4628	15900	0.0	-
2.4706	15950	0.0	-
2.4783	16000	0.0	-
2.4861	16050	0.0001	-
2.4938	16100	0.0	-
2.5015	16150	0.0	-
2.5093	16200	0.0	-
2.5170	16250	0.0	-
2.5248	16300	0.0	-
2.5325	16350	0.0	-
2.5403	16400	0.0	-
2.5480	16450	0.0	-
2.5558	16500	0.0	-
2.5635	16550	0.0001	-
2.5713	16600	0.0	-
2.5790	16650	0.0	-
2.5867	16700	0.0	-
2.5945	16750	0.0	-
2.6022	16800	0.0009	-
2.6100	16850	0.0001	-
2.6177	16900	0.0	-
2.6255	16950	0.0001	-
2.6332	17000	0.0	-
2.6410	17050	0.0	-
2.6487	17100	0.0001	-
2.6564	17150	0.0	-
2.6642	17200	0.0	-
2.6719	17250	0.0	-
2.6797	17300	0.0	-
2.6874	17350	0.0004	-
2.6952	17400	0.0	-
2.7029	17450	0.0	-
2.7107	17500	0.0	-
2.7184	17550	0.0	-
2.7261	17600	0.0	-
2.7339	17650	0.0	-
2.7416	17700	0.0001	-
2.7494	17750	0.0	-
2.7571	17800	0.0	-
2.7649	17850	0.0001	-
2.7726	17900	0.0	-
2.7804	17950	0.0001	-
2.7881	18000	0.0001	-
2.7958	18050	0.0	-
2.8036	18100	0.0	-
2.8113	18150	0.0	-
2.8191	18200	0.0	-
2.8268	18250	0.0	-
2.8346	18300	0.0001	-
2.8423	18350	0.0	-
2.8501	18400	0.0	-
2.8578	18450	0.0	-
2.8656	18500	0.0	-
2.8733	18550	0.0	-
2.8810	18600	0.0	-
2.8888	18650	0.0	-
2.8965	18700	0.0	-
2.9043	18750	0.0	-
2.9120	18800	0.0001	-
2.9198	18850	0.0	-
2.9275	18900	0.0	-
2.9353	18950	0.0	-
2.9430	19000	0.0	-
2.9507	19050	0.0	-
2.9585	19100	0.0	-
2.9662	19150	0.0	-
2.9740	19200	0.0	-
2.9817	19250	0.0003	-
2.9895	19300	0.0001	-
2.9972	19350	0.0	-
3.0	19368	-	2.0845
3.0050	19400	0.0	-
3.0127	19450	0.0001	-
3.0204	19500	0.0	-
3.0282	19550	0.0	-
3.0359	19600	0.0	-
3.0437	19650	0.0	-
3.0514	19700	0.0	-
3.0592	19750	0.0	-
3.0669	19800	0.0001	-
3.0747	19850	0.0	-
3.0824	19900	0.0	-
3.0901	19950	0.0001	-
3.0979	20000	0.0	-
3.1056	20050	0.0	-
3.1134	20100	0.0	-
3.1211	20150	0.0001	-
3.1289	20200	0.0	-
3.1366	20250	0.0	-
3.1444	20300	0.0	-
3.1521	20350	0.0	-
3.1599	20400	0.0	-
3.1676	20450	0.0001	-
3.1753	20500	0.0	-
3.1831	20550	0.0001	-
3.1908	20600	0.0	-
3.1986	20650	0.0	-
3.2063	20700	0.0	-
3.2141	20750	0.0	-
3.2218	20800	0.0	-
3.2296	20850	0.0003	-
3.2373	20900	0.0	-
3.2450	20950	0.0	-
3.2528	21000	0.0	-
3.2605	21050	0.0	-
3.2683	21100	0.0001	-
3.2760	21150	0.0001	-
3.2838	21200	0.0	-
3.2915	21250	0.0	-
3.2993	21300	0.0	-
3.3070	21350	0.0	-
3.3147	21400	0.0	-
3.3225	21450	0.0001	-
3.3302	21500	0.0	-
3.3380	21550	0.0	-
3.3457	21600	0.0	-
3.3535	21650	0.0	-
3.3612	21700	0.0	-
3.3690	21750	0.0	-
3.3767	21800	0.0	-
3.3844	21850	0.0	-
3.3922	21900	0.0001	-
3.3999	21950	0.0	-
3.4077	22000	0.0	-
3.4154	22050	0.0001	-
3.4232	22100	0.0	-
3.4309	22150	0.0001	-
3.4387	22200	0.0	-
3.4464	22250	0.0	-
3.4542	22300	0.0	-
3.4619	22350	0.0001	-
3.4696	22400	0.0	-
3.4774	22450	0.0	-
3.4851	22500	0.0	-
3.4929	22550	0.0001	-
3.5006	22600	0.0002	-
3.5084	22650	0.0001	-
3.5161	22700	0.0	-
3.5239	22750	0.0001	-
3.5316	22800	0.0	-
3.5393	22850	0.0	-
3.5471	22900	0.0001	-
3.5548	22950	0.0	-
3.5626	23000	0.0	-
3.5703	23050	0.0	-
3.5781	23100	0.0	-
3.5858	23150	0.0001	-
3.5936	23200	0.0	-
3.6013	23250	0.0001	-
3.6090	23300	0.0001	-
3.6168	23350	0.0	-
3.6245	23400	0.0003	-
3.6323	23450	0.0	-
3.6400	23500	0.0	-
3.6478	23550	0.0001	-
3.6555	23600	0.0	-
3.6633	23650	0.0	-
3.6710	23700	0.0	-
3.6787	23750	0.0001	-
3.6865	23800	0.0	-
3.6942	23850	0.0001	-
3.7020	23900	0.0002	-
3.7097	23950	0.0	-
3.7175	24000	0.0	-
3.7252	24050	0.0	-
3.7330	24100	0.0	-
3.7407	24150	0.0001	-
3.7485	24200	0.0	-
3.7562	24250	0.0	-
3.7639	24300	0.0	-
3.7717	24350	0.0	-
3.7794	24400	0.0	-
3.7872	24450	0.0	-
3.7949	24500	0.0001	-
3.8027	24550	0.0001	-
3.8104	24600	0.0	-
3.8182	24650	0.0	-
3.8259	24700	0.0	-
3.8336	24750	0.0	-
3.8414	24800	0.0001	-
3.8491	24850	0.0	-
3.8569	24900	0.0	-
3.8646	24950	0.0	-
3.8724	25000	0.0	-
3.8801	25050	0.0	-
3.8879	25100	0.0	-
3.8956	25150	0.0001	-
3.9033	25200	0.0	-
3.9111	25250	0.0002	-
3.9188	25300	0.0001	-
3.9266	25350	0.0	-
3.9343	25400	0.0	-
3.9421	25450	0.0	-
3.9498	25500	0.0001	-
3.9576	25550	0.0	-
3.9653	25600	0.0	-
3.9730	25650	0.0001	-
3.9808	25700	0.0	-
3.9885	25750	0.0	-
3.9963	25800	0.0	-
4.0	25824	-	2.3576

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• SetFit: 1.0.3
• Sentence Transformers: 3.0.1
• Transformers: 4.39.0
• PyTorch: 2.3.0+cu121
• Datasets: 2.20.0
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}