GQA_RoBERTa_German_legal_SQuAD_part_augmented_17

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2674

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 128
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 17

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	4	3.7900
No log	2.0	8	3.1774
No log	3.0	12	2.8018
No log	4.0	16	2.5378
No log	5.0	20	2.2274
No log	6.0	24	2.0786
No log	7.0	28	1.9560
No log	8.0	32	1.7749
No log	9.0	36	1.6928
No log	10.0	40	1.5814
No log	11.0	44	1.4800
No log	12.0	48	1.4188
No log	13.0	52	1.3761
No log	14.0	56	1.3139
No log	15.0	60	1.2892
No log	16.0	64	1.2752
No log	17.0	68	1.2674

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu121
• Datasets 2.14.7
• Tokenizers 0.15.0