token-classification-llmlingua2-xlm-roberta-1k7_kept_yte_10_epoch_paper_v2

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4344

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 96
• eval_batch_size: 96
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	15	0.4691
No log	2.0	30	0.4634
No log	3.0	45	0.4570
No log	4.0	60	0.4537
No log	5.0	75	0.4499
No log	6.0	90	0.4450
No log	7.0	105	0.4407
No log	8.0	120	0.4371
No log	9.0	135	0.4350
No log	10.0	150	0.4344

Framework versions

• Transformers 4.39.0.dev0
• Pytorch 2.2.1+cu118
• Datasets 2.18.0
• Tokenizers 0.15.2