deberta-v3-large-survey-related_passage_old_facts-rater-half-gpt4

This model is a fine-tuned version of microsoft/deberta-v3-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5850
• Krippendorff: 0.9463
• Spearman: 0.9528
• Absolute Agreement: 0.8173
• Agreement Within One: 0.9399

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Krippendorff	Spearman	Absolute Agreement	Agreement Within One
No log	1.0	52	1.9330	-0.0890	nan	0.0556	0.8056
No log	2.0	104	1.9327	-0.0890	nan	0.0556	0.8056
No log	3.0	156	1.9302	-0.0890	nan	0.0556	0.8056
No log	4.0	208	1.9298	-0.0890	nan	0.0556	0.8056
No log	5.0	260	1.9269	-0.6469	nan	0.1111	1.0
No log	6.0	312	1.9607	-0.6469	nan	0.1111	1.0
No log	7.0	364	2.1026	-0.6469	nan	0.1111	1.0
No log	8.0	416	2.0244	-0.1249	-0.1055	0.1667	0.7222
No log	9.0	468	2.0627	-0.0719	-0.0558	0.1667	0.6389
1.768	10.0	520	2.0573	0.1385	0.0985	0.1944	0.6944
1.768	11.0	572	2.0929	0.0704	0.0962	0.1944	0.6667
1.768	12.0	624	2.4193	0.0499	0.0924	0.1944	0.6944
1.768	13.0	676	2.2569	0.2089	0.1890	0.1667	0.7222
1.768	14.0	728	2.2547	0.1851	0.2376	0.25	0.7222
1.768	15.0	780	2.4887	0.2271	0.2263	0.1667	0.7222
1.768	16.0	832	2.4946	0.2272	0.2488	0.2222	0.7222
1.768	17.0	884	2.8018	0.2519	0.2900	0.2222	0.7222
1.768	18.0	936	2.8760	0.1849	0.1906	0.1667	0.7222
1.768	19.0	988	3.0296	0.2389	0.2558	0.1389	0.7222
0.9194	20.0	1040	3.0490	0.2238	0.2630	0.25	0.7222

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.12.1