Edit model card

xlm-roberta-base-azsci-topics

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on azsci_topics dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5075
  • Precision: 0.8624
  • Recall: 0.8707
  • F1: 0.8645
  • Accuracy: 0.8707

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 64
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 1.0 288 0.8674 0.6762 0.7422 0.6848 0.7422
1.3706 2.0 576 0.6110 0.8177 0.8264 0.8104 0.8264
1.3706 3.0 864 0.5391 0.8407 0.8490 0.8415 0.8490
0.5184 4.0 1152 0.5059 0.8505 0.8559 0.8493 0.8559
0.5184 5.0 1440 0.5075 0.8624 0.8707 0.8645 0.8707

Evaluation results

Topic Precision Recall F1 Support
Aqrar elmlər 0.703704 0.703704 0.703704 27
Astronomiya 0 0 0 2
Biologiya elmləri 0.886598 0.819048 0.851485 105
Coğrafiya 0.75 0.705882 0.727273 17
Filologiya elmləri 0.91954 0.914286 0.916905 175
Fizika 0.710526 0.794118 0.75 34
Fəlsəfə 0.7 0.5 0.583333 14
Hüquq elmləri 1 1 1 29
Kimya 0.75 0.934426 0.832117 61
Memarlıq 1 0.4 0.571429 5
Mexanika 0 0 0 4
Pedaqogika 0.854545 1 0.921569 47
Psixologiya 0.823529 0.777778 0.8 18
Riyaziyyat 0.891892 0.846154 0.868421 39
Siyasi elmlər 0.785714 0.88 0.830189 25
Sosiologiya 0 0 0 4
Sənətşünaslıq 0.843137 0.914894 0.877551 47
Tarix 0.857143 0.846154 0.851613 78
Texnika elmləri 0.803922 0.788462 0.796117 104
Tibb elmləri 0.929936 0.986486 0.957377 148
Yer elmləri 0.692308 0.692308 0.692308 13
İqtisad elmləri 0.972603 0.934211 0.95302 152
Əczaçılıq elmləri 0 0 0 4
macro avg 0.690222 0.671213 0.673235 1152
weighted avg 0.862363 0.87066 0.864467 1152

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2
Downloads last month
6
Safetensors
Model size
278M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for hajili/xlm-roberta-base-azsci-topics

Finetuned
(2462)
this model

Dataset used to train hajili/xlm-roberta-base-azsci-topics