radar-modernbert-large-encoder-unfreeze

This model is a fine-tuned version of yusr9/radar-modernbert-large-encoder-freeze on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1627
• Roc-auc: 0.994
• Brier: 0.971
• C@1: 0.969
• F1: 0.969
• F05u: 0.976
• Mean: 0.976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 0.03
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Roc-auc	Brier	C@1	F1	F05u	Mean
0.1905	0.4153	500	0.2420	0.964	0.911	0.876	0.882	0.891	0.905
0.1327	0.8306	1000	0.1539	0.985	0.941	0.923	0.923	0.962	0.947
0.0533	1.2458	1500	0.1980	0.987	0.94	0.925	0.932	0.917	0.94
0.0354	1.6611	2000	0.0843	0.994	0.973	0.966	0.968	0.974	0.975
0.0414	2.0764	2500	0.0928	0.993	0.973	0.968	0.97	0.972	0.975
0.0624	2.4917	3000	0.0881	0.994	0.974	0.967	0.969	0.97	0.975
0.0075	2.9070	3500	0.1101	0.994	0.971	0.966	0.968	0.97	0.974
0.0132	3.3223	4000	0.1248	0.991	0.968	0.964	0.966	0.973	0.972
0.0259	3.7375	4500	0.1162	0.994	0.973	0.97	0.971	0.982	0.978
0.0090	4.1528	5000	0.1001	0.995	0.975	0.971	0.973	0.981	0.979
0.0005	4.5681	5500	0.1389	0.992	0.969	0.967	0.968	0.983	0.976
0.0016	4.9834	6000	0.1241	0.994	0.974	0.972	0.974	0.983	0.979
0.0001	5.3987	6500	0.1174	0.995	0.975	0.973	0.975	0.981	0.98
0.0001	5.8140	7000	0.1315	0.994	0.975	0.973	0.974	0.978	0.979
0.0001	6.2292	7500	0.1335	0.994	0.974	0.973	0.974	0.984	0.98
0.0001	6.6445	8000	0.1348	0.994	0.975	0.973	0.974	0.983	0.98
0.0001	7.0598	8500	0.1329	0.995	0.976	0.974	0.976	0.983	0.981
0.0001	7.4751	9000	0.1368	0.995	0.975	0.974	0.975	0.983	0.981
0.0001	7.8904	9500	0.1383	0.995	0.975	0.974	0.976	0.984	0.981
0.0000	8.3056	10000	0.1373	0.995	0.976	0.974	0.976	0.982	0.981
0.0001	8.7209	10500	0.1399	0.995	0.975	0.974	0.976	0.984	0.981
0.0000	9.1362	11000	0.1413	0.995	0.975	0.974	0.975	0.984	0.981
0.0000	9.5515	11500	0.1416	0.995	0.975	0.974	0.975	0.984	0.981
0.0000	9.9668	12000	0.1408	0.995	0.975	0.974	0.975	0.984	0.981

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2