camembert_ccnet_classification_tools_NEFTune_fr

This model is a fine-tuned version of camembert/camembert-base-ccnet on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2866
• Accuracy: 0.95
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 192
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rate
2.038	1.0	7	1.7370	0.65	0.0001
1.6169	2.0	14	1.2630	0.825	0.0001
1.1861	3.0	21	0.8659	0.95	0.0001
0.8284	4.0	28	0.6075	0.95	0.0001
0.6032	5.0	35	0.4207	0.975	0.0001
0.3928	6.0	42	0.3817	0.95	9e-05
0.2458	7.0	49	0.3378	0.95	0.0001
0.1683	8.0	56	0.4320	0.9	0.0001
0.127	9.0	63	0.3592	0.95	0.0001
0.0909	10.0	70	0.3695	0.925	0.0001
0.0719	11.0	77	0.3377	0.925	0.0001
0.0679	12.0	84	0.2450	0.95	8e-05
0.0865	13.0	91	0.2783	0.9	0.0001
0.0519	14.0	98	0.2265	0.975	0.0001
0.0497	15.0	105	0.2801	0.95	0.0001
0.0993	16.0	112	0.3733	0.925	0.0001
0.0358	17.0	119	0.4012	0.9	0.0001
0.0356	18.0	126	0.2591	0.95	7e-05
0.0279	19.0	133	0.2687	0.95	0.0001
0.0303	20.0	140	0.2650	0.95	0.0001
0.0246	21.0	147	0.2337	0.95	0.0001
0.0257	22.0	154	0.2274	0.95	0.0001
0.0448	23.0	161	0.2223	0.975	0.0001
0.0567	24.0	168	0.2157	0.975	6e-05
0.0182	25.0	175	0.2096	0.975	0.0001
0.0282	26.0	182	0.2118	0.975	0.0001
0.0232	27.0	189	0.2146	0.975	0.0001
0.0212	28.0	196	0.2162	0.975	0.0001
0.0197	29.0	203	0.2185	0.975	0.0001
0.0203	30.0	210	0.2215	0.975	5e-05
0.0172	31.0	217	0.2263	0.975	0.0000
0.0174	32.0	224	0.2347	0.975	0.0000
0.0152	33.0	231	0.2426	0.95	0.0000
0.0164	34.0	238	0.2443	0.95	0.0000
0.018	35.0	245	0.2557	0.95	0.0000
0.0328	36.0	252	0.2624	0.95	4e-05
0.0152	37.0	259	0.2602	0.95	0.0000
0.0147	38.0	266	0.2615	0.95	0.0000
0.0152	39.0	273	0.2634	0.95	0.0000
0.015	40.0	280	0.2699	0.95	0.0000
0.0147	41.0	287	0.2726	0.95	0.0000
0.0148	42.0	294	0.2783	0.95	3e-05
0.033	43.0	301	0.2793	0.95	0.0000
0.0143	44.0	308	0.2742	0.95	0.0000
0.0143	45.0	315	0.2681	0.95	0.0000
0.0139	46.0	322	0.2683	0.95	0.0000
0.0141	47.0	329	0.2706	0.95	0.0000
0.0132	48.0	336	0.2715	0.95	2e-05
0.0157	49.0	343	0.2785	0.95	0.0000
0.0142	50.0	350	0.2809	0.95	0.0000
0.0138	51.0	357	0.2818	0.95	0.0000
0.0141	52.0	364	0.2852	0.95	0.0000
0.015	53.0	371	0.2868	0.95	0.0000
0.0145	54.0	378	0.2876	0.95	1e-05
0.0135	55.0	385	0.2854	0.95	0.0000
0.0146	56.0	392	0.2862	0.95	0.0000
0.0136	57.0	399	0.2857	0.95	5e-06
0.014	58.0	406	0.2853	0.95	0.0000
0.0133	59.0	413	0.2862	0.95	0.0000
0.0125	60.0	420	0.2866	0.95	0.0

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.1