tst_fine-tuning-lilt2

This model is a fine-tuned version of doc2txt/lilt_roberta_like_en on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.4439
• Answer: {'precision': 0.14533622559652928, 'recall': 0.2460220318237454, 'f1': 0.18272727272727274, 'number': 817}
• Header: {'precision': 0.033482142857142856, 'recall': 0.12605042016806722, 'f1': 0.05291005291005291, 'number': 119}
• Question: {'precision': 0.27887323943661974, 'recall': 0.4596100278551532, 'f1': 0.34712482468443195, 'number': 1077}
• Overall Precision: 0.1972
• Overall Recall: 0.3532
• Overall F1: 0.2531
• Overall Accuracy: 0.4392

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.3672	10.53	200	1.6951	{'precision': 0.11161290322580646, 'recall': 0.211750305997552, 'f1': 0.14617659484579637, 'number': 817}	{'precision': 0.0498220640569395, 'recall': 0.11764705882352941, 'f1': 0.06999999999999999, 'number': 119}	{'precision': 0.26964933494558646, 'recall': 0.4141132776230269, 'f1': 0.32662028560966677, 'number': 1077}	0.1816	0.3145	0.2303	0.4331
0.4121	21.05	400	2.9692	{'precision': 0.12375621890547264, 'recall': 0.24357405140758873, 'f1': 0.16412371134020617, 'number': 817}	{'precision': 0.023622047244094488, 'recall': 0.07563025210084033, 'f1': 0.036, 'number': 119}	{'precision': 0.26241534988713316, 'recall': 0.43175487465181056, 'f1': 0.3264303264303264, 'number': 1077}	0.1789	0.3343	0.2331	0.4366
0.0862	31.58	600	4.0881	{'precision': 0.1258692628650904, 'recall': 0.2215422276621787, 'f1': 0.1605321507760532, 'number': 817}	{'precision': 0.020833333333333332, 'recall': 0.07563025210084033, 'f1': 0.032667876588021776, 'number': 119}	{'precision': 0.26891220320265047, 'recall': 0.4521819870009285, 'f1': 0.3372576177285318, 'number': 1077}	0.1839	0.3363	0.2378	0.4278
0.0334	42.11	800	4.5480	{'precision': 0.11213626685592619, 'recall': 0.193390452876377, 'f1': 0.14195867026055706, 'number': 817}	{'precision': 0.021691973969631236, 'recall': 0.08403361344537816, 'f1': 0.034482758620689655, 'number': 119}	{'precision': 0.2576306259699948, 'recall': 0.4623955431754875, 'f1': 0.33089700996677734, 'number': 1077}	0.1751	0.3308	0.2290	0.4257
0.0171	52.63	1000	4.8940	{'precision': 0.13886671987230648, 'recall': 0.21297429620563035, 'f1': 0.1681159420289855, 'number': 817}	{'precision': 0.026217228464419477, 'recall': 0.11764705882352941, 'f1': 0.042879019908116385, 'number': 119}	{'precision': 0.2647058823529412, 'recall': 0.4596100278551532, 'f1': 0.335934848998982, 'number': 1077}	0.1868	0.3393	0.2409	0.4297
0.0101	63.16	1200	5.0531	{'precision': 0.13835701050030882, 'recall': 0.2741738066095471, 'f1': 0.18390804597701146, 'number': 817}	{'precision': 0.024886877828054297, 'recall': 0.09243697478991597, 'f1': 0.0392156862745098, 'number': 119}	{'precision': 0.2734418865805727, 'recall': 0.4521819870009285, 'f1': 0.34079776067179846, 'number': 1077}	0.1879	0.3587	0.2466	0.4270
0.0058	73.68	1400	5.2725	{'precision': 0.13805185704274703, 'recall': 0.24112607099143207, 'f1': 0.17557932263814616, 'number': 817}	{'precision': 0.033707865168539325, 'recall': 0.12605042016806722, 'f1': 0.05319148936170213, 'number': 119}	{'precision': 0.2645301466594242, 'recall': 0.4521819870009285, 'f1': 0.3337902673063742, 'number': 1077}	0.1883	0.3472	0.2441	0.4293
0.0029	84.21	1600	5.2656	{'precision': 0.13925822253324002, 'recall': 0.24357405140758873, 'f1': 0.17720391807658056, 'number': 817}	{'precision': 0.043726235741444866, 'recall': 0.19327731092436976, 'f1': 0.07131782945736433, 'number': 119}	{'precision': 0.2663316582914573, 'recall': 0.4428969359331476, 'f1': 0.33263598326359833, 'number': 1077}	0.1866	0.3472	0.2428	0.4289
0.0013	94.74	1800	5.4292	{'precision': 0.1434659090909091, 'recall': 0.24724602203182375, 'f1': 0.18157303370786518, 'number': 817}	{'precision': 0.03333333333333333, 'recall': 0.13445378151260504, 'f1': 0.05342237061769616, 'number': 119}	{'precision': 0.26543878656554715, 'recall': 0.45496750232126276, 'f1': 0.3352719808416011, 'number': 1077}	0.1896	0.3517	0.2464	0.4291
0.0006	105.26	2000	5.4360	{'precision': 0.14817572598659717, 'recall': 0.24357405140758873, 'f1': 0.18425925925925926, 'number': 817}	{'precision': 0.03456221198156682, 'recall': 0.12605042016806722, 'f1': 0.054249547920433995, 'number': 119}	{'precision': 0.2717271727172717, 'recall': 0.4586815227483751, 'f1': 0.3412780656303972, 'number': 1077}	0.1969	0.3517	0.2525	0.4365
0.0005	115.79	2200	5.4540	{'precision': 0.14905933429811866, 'recall': 0.2521419828641371, 'f1': 0.18735788994997726, 'number': 817}	{'precision': 0.038817005545286505, 'recall': 0.17647058823529413, 'f1': 0.06363636363636363, 'number': 119}	{'precision': 0.28058823529411764, 'recall': 0.4428969359331476, 'f1': 0.34353619013323733, 'number': 1077}	0.1943	0.3497	0.2498	0.4337
0.0003	126.32	2400	5.4439	{'precision': 0.14533622559652928, 'recall': 0.2460220318237454, 'f1': 0.18272727272727274, 'number': 817}	{'precision': 0.033482142857142856, 'recall': 0.12605042016806722, 'f1': 0.05291005291005291, 'number': 119}	{'precision': 0.27887323943661974, 'recall': 0.4596100278551532, 'f1': 0.34712482468443195, 'number': 1077}	0.1972	0.3532	0.2531	0.4392

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1