A.X-patent-maxlen8192-train

This model is a fine-tuned version of skt/A.X-Encoder-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0010
• Micro F1: 0.8700
• Macro F1: 0.8675
• Sample F1: 0.8849
• Empty Rate: 0.0125
• Anchor Weighted F1: 0.8234

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 12

Training results

Training Loss	Epoch	Step	Anchor Weighted F1	Empty Rate	Validation Loss	Macro F1	Micro F1	Sample F1
0.0026	0.5000	12618	0.6625	0.2317	0.0009	0.6280	0.6587	0.6010
0.0020	1.0000	25236	0.7220	0.1113	0.0007	0.7139	0.7312	0.7135
0.0016	1.4999	37854	0.7542	0.1349	0.0006	0.7408	0.7586	0.7313
0.0010	1.9999	50472	0.7775	0.0558	0.0005	0.7931	0.8011	0.8018
0.0009	2.4999	63090	0.7882	0.0501	0.0004	0.8105	0.8179	0.8195
0.0010	2.9999	75708	0.7967	0.0360	0.0004	0.8243	0.8285	0.8356
0.0006	3.4999	88326	0.8056	0.0318	0.0004	0.8288	0.8363	0.8455
0.0006	3.9998	100944	0.8086	0.0268	0.0004	0.8384	0.8438	0.8535
0.0004	4.4998	113562	0.8098	0.0312	0.0004	0.8387	0.8435	0.8517
0.0003	4.9998	126180	0.8133	0.0180	0.0004	0.8491	0.8518	0.8657
0.0003	5.4998	138798	0.8162	0.0189	0.0005	0.8488	0.8525	0.8650
0.0004	5.9998	151416	0.8165	0.0245	0.0004	0.8518	0.8552	0.8645
0.0002	6.4997	164034	0.8155	0.0170	0.0005	0.8513	0.8549	0.8690
0.0002	6.9997	176652	0.8173	0.0199	0.0005	0.8537	0.8580	0.8708
0.0001	7.4997	189270	0.8198	0.0154	0.0006	0.8570	0.8602	0.8749
0.0001	7.9997	201888	0.8153	0.0176	0.0006	0.8567	0.8599	0.8735
0.0001	8.4997	214506	0.8184	0.0138	0.0007	0.8610	0.8638	0.8785
0.0001	8.9996	227124	0.0007	0.8624	0.8598	0.8790	0.0114	0.8175
0.0001	9.4996	239742	0.0007	0.8659	0.8633	0.8810	0.0135	0.8212
0.0001	9.9996	252360	0.0007	0.8673	0.8643	0.8810	0.0148	0.8235
0.0000	10.4996	264978	0.0009	0.8675	0.8651	0.8834	0.0121	0.8227
0.0000	10.9996	277596	0.0009	0.8669	0.8642	0.8831	0.0109	0.8230
0.0000	11.4995	290214	0.0010	0.8689	0.8664	0.8842	0.0125	0.8234
0.0000	11.9995	302832	0.0010	0.8700	0.8675	0.8849	0.0125	0.8236
0.0000	12.0	302844	0.0010	0.8700	0.8675	0.8849	0.0125	0.8234

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2