darrow_legallens_ner_results_pissa

This model is a fine-tuned version of CohereForAI/aya-101 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6217
• Law Precision: 0.7630
• Law Recall: 0.8655
• Law F1: 0.8110
• Law Number: 119
• Violated by Precision: 0.7241
• Violated by Recall: 0.7326
• Violated by F1: 0.7283
• Violated by Number: 86
• Violated on Precision: 0.4711
• Violated on Recall: 0.5876
• Violated on F1: 0.5229
• Violated on Number: 97
• Violation Precision: 0.6277
• Violation Recall: 0.7024
• Violation F1: 0.6630
• Violation Number: 1270
• Overall Precision: 0.6321
• Overall Recall: 0.7093
• Overall F1: 0.6685
• Overall Accuracy: 0.9517

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Law Precision	Law Recall	Law F1	Law Number	Violated by Precision	Violated by Recall	Violated by F1	Violated by Number	Violated on Precision	Violated on Recall	Violated on F1	Violated on Number	Violation Precision	Violation Recall	Violation F1	Violation Number	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.0233	1.0	178	0.7936	0.0015	0.0168	0.0027	119	0.0	0.0	0.0	86	0.0	0.0	0.0	97	0.0142	0.0520	0.0223	1270	0.0111	0.0433	0.0176	0.8174
0.3451	2.0	356	0.2801	0.0391	0.2353	0.0670	119	0.0	0.0	0.0	86	0.0	0.0	0.0	97	0.1576	0.1551	0.1563	1270	0.1131	0.1431	0.1263	0.9087
0.2021	3.0	534	0.1972	0.1731	0.6387	0.2724	119	0.4925	0.3837	0.4314	86	0.125	0.1031	0.1130	97	0.4111	0.4276	0.4191	1270	0.3471	0.4211	0.3806	0.9378
0.0888	4.0	712	0.2134	0.7290	0.6555	0.6903	119	0.52	0.4535	0.4845	86	0.2179	0.1753	0.1943	97	0.4578	0.4874	0.4722	1270	0.4671	0.4790	0.4730	0.9447
0.1455	5.0	890	0.2108	0.6225	0.7899	0.6963	119	0.5714	0.6977	0.6283	86	0.1742	0.2784	0.2143	97	0.5224	0.6236	0.5686	1270	0.5049	0.6190	0.5562	0.9472
0.0467	6.0	1068	0.2176	0.8017	0.8151	0.8083	119	0.6962	0.6395	0.6667	86	0.3175	0.4124	0.3587	97	0.5483	0.6031	0.5744	1270	0.5560	0.6094	0.5815	0.9471
0.0433	7.0	1246	0.2967	0.7109	0.7647	0.7368	119	0.6818	0.6977	0.6897	86	0.3650	0.5155	0.4274	97	0.5631	0.6291	0.5943	1270	0.5643	0.6361	0.5981	0.9482
0.0399	8.0	1424	0.2804	0.7559	0.8067	0.7805	119	0.7143	0.6977	0.7059	86	0.3968	0.5155	0.4484	97	0.6018	0.6354	0.6182	1270	0.6037	0.6444	0.6234	0.9498
0.0144	9.0	1602	0.3334	0.7071	0.8319	0.7645	119	0.6506	0.6279	0.6391	86	0.4309	0.5464	0.4818	97	0.5785	0.6819	0.6259	1270	0.5817	0.6819	0.6278	0.9501
0.0096	10.0	1780	0.4068	0.7338	0.8571	0.7907	119	0.6436	0.7558	0.6952	86	0.3711	0.6082	0.4609	97	0.6266	0.6780	0.6513	1270	0.6131	0.6915	0.6499	0.9515
0.0095	11.0	1958	0.3343	0.6923	0.8319	0.7557	119	0.6346	0.7674	0.6947	86	0.4203	0.5979	0.4936	97	0.6036	0.6811	0.6400	1270	0.5985	0.6921	0.6419	0.9523
0.0065	12.0	2136	0.4131	0.7071	0.8319	0.7645	119	0.6667	0.7442	0.7033	86	0.4667	0.6495	0.5431	97	0.6380	0.7063	0.6704	1270	0.6320	0.7144	0.6706	0.9537
0.0061	13.0	2314	0.4760	0.6783	0.8151	0.7405	119	0.6939	0.7907	0.7391	86	0.4959	0.6186	0.5505	97	0.6281	0.6835	0.6546	1270	0.6267	0.6953	0.6592	0.9524
0.0063	14.0	2492	0.4601	0.7021	0.8319	0.7615	119	0.7381	0.7209	0.7294	86	0.472	0.6082	0.5315	97	0.6090	0.6843	0.6444	1270	0.6128	0.6927	0.6503	0.9499
0.0017	15.0	2670	0.5064	0.7206	0.8235	0.7686	119	0.7529	0.7442	0.7485	86	0.4833	0.5979	0.5346	97	0.6173	0.6858	0.6498	1270	0.6227	0.6940	0.6564	0.9517
0.0007	16.0	2848	0.5764	0.7246	0.8403	0.7782	119	0.6667	0.7209	0.6927	86	0.4961	0.6495	0.5625	97	0.6113	0.6787	0.6433	1270	0.6148	0.6915	0.6509	0.9505
0.0039	17.0	3026	0.6285	0.7574	0.8655	0.8078	119	0.7	0.7326	0.7159	86	0.4914	0.5876	0.5352	97	0.6296	0.6906	0.6587	1270	0.6340	0.6997	0.6653	0.9518
0.001	18.0	3204	0.6129	0.7143	0.8403	0.7722	119	0.7111	0.7442	0.7273	86	0.4459	0.6804	0.5388	97	0.6166	0.6953	0.6536	1270	0.6149	0.7080	0.6582	0.9505
0.0004	19.0	3382	0.6282	0.7630	0.8655	0.8110	119	0.7294	0.7209	0.7251	86	0.4957	0.5979	0.5421	97	0.6302	0.6992	0.6629	1270	0.6363	0.7067	0.6697	0.9516
0.0031	20.0	3560	0.6217	0.7630	0.8655	0.8110	119	0.7241	0.7326	0.7283	86	0.4711	0.5876	0.5229	97	0.6277	0.7024	0.6630	1270	0.6321	0.7093	0.6685	0.9517

Framework versions

• PEFT 0.10.1.dev0
• Transformers 4.39.3
• Pytorch 2.2.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2