rinna-roberta22-qa-ar22

This model is a fine-tuned version of xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 10.5084

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 70

Training results

Training Loss	Epoch	Step	Validation Loss
3.1604	6.8182	150	5.4713
1.7441	13.6364	300	7.0904
0.9026	20.4545	450	8.4058
0.3606	27.2727	600	10.7776
0.182	34.0909	750	10.4600
0.1023	40.9091	900	11.1625
0.0632	47.7273	1050	9.6665
0.0278	54.5455	1200	10.8137
0.0098	61.3636	1350	9.7877
0.0058	68.1818	1500	10.5084

Framework versions

• Transformers 4.41.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1