indo-ner-indobert-large-cpt

This model is a fine-tuned version of indobenchmark/indobert-large-p1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.7538

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
28.9612	0.0243	500	6.7911
24.6225	0.0486	1000	6.0564
22.9936	0.0730	1500	5.7198
22.9773	0.0973	2000	5.4587
22.7848	0.1216	2500	5.2777
21.4019	0.1459	3000	5.1477
20.9692	0.1702	3500	4.7566
16.5949	0.1946	4000	3.7330
14.8576	0.2189	4500	3.4046
13.8743	0.2432	5000	3.3494
13.6011	0.2675	5500	3.2540
13.6552	0.2918	6000	3.1844
13.0754	0.3162	6500	3.1372
13.4083	0.3405	7000	3.1149
13.1592	0.3648	7500	3.0550
12.4606	0.3891	8000	3.0529
12.4061	0.4134	8500	3.0304
12.6295	0.4378	9000	2.9921
12.3568	0.4621	9500	2.9569
12.3713	0.4864	10000	2.9387
12.4043	0.5107	10500	2.9214
12.2125	0.5351	11000	2.9173
12.3826	0.5594	11500	2.9215
11.8485	0.5837	12000	2.8445
12.2704	0.6080	12500	2.8672
12.1440	0.6323	13000	2.8039
12.1876	0.6567	13500	2.8502
11.6199	0.6810	14000	2.8381
11.8377	0.7053	14500	2.8255
11.9495	0.7296	15000	2.8050
11.1944	0.7539	15500	2.7935
11.5834	0.7783	16000	2.7605
11.7730	0.8026	16500	2.8011
11.8273	0.8269	17000	2.7532
11.5610	0.8512	17500	2.7990
11.6905	0.8755	18000	2.7663
11.4057	0.8999	18500	2.7724
11.4191	0.9242	19000	2.7364
11.5424	0.9485	19500	2.7517
11.3731	0.9728	20000	2.7482
11.3491	0.9971	20500	2.7538

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.3
• Tokenizers 0.22.2