LiLT-RE-ZH / README.md
kavg's picture
LiLT-RE-ZH
a3a6a03 verified
metadata
license: mit
base_model: nielsr/lilt-xlm-roberta-base
tags:
  - generated_from_trainer
datasets:
  - xfun
metrics:
  - precision
  - recall
  - f1
model-index:
  - name: checkpoints
    results: []

checkpoints

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on the xfun dataset. It achieves the following results on the evaluation set:

  • Precision: 0.3911
  • Recall: 0.6703
  • F1: 0.4940
  • Loss: 0.1352

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 6
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • training_steps: 10000

Training results

Training Loss Epoch Step F1 Validation Loss Precision Recall
0.1469 20.83 500 0 0.1467 0 0
0.0896 41.67 1000 0.0837 0.1454 0.2946 0.0487
0.1027 62.5 1500 0.1225 0.1353 0.3333 0.0750
0.0485 83.33 2000 0.3536 0.1571 0.3364 0.3727
0.0597 104.17 2500 0.4448 0.1546 0.3535 0.5997
0.0367 125.0 3000 0.4940 0.1352 0.3911 0.6703
0.033 145.83 3500 0.4977 0.1749 0.3902 0.6870
0.0176 166.67 4000 0.5087 0.2262 0.4034 0.6883
0.0123 187.5 4500 0.5050 0.2358 0.3978 0.6915
0.0194 208.33 5000 0.5173 0.2976 0.4090 0.7037
0.0118 171.88 5500 0.4159 0.6863 0.5179 0.2836
0.0054 187.5 6000 0.4356 0.6703 0.5280 0.3100
0.01 203.12 6500 0.4229 0.6979 0.5266 0.3430
0.0062 218.75 7000 0.4272 0.7062 0.5324 0.3652
0.0051 234.38 7500 0.4306 0.6947 0.5317 0.3496
0.0048 250.0 8000 0.4400 0.6940 0.5386 0.3943
0.0087 265.62 8500 0.4290 0.6992 0.5317 0.3782
0.0077 281.25 9000 0.4394 0.7049 0.5414 0.3855
0.0014 296.88 9500 0.4363 0.7004 0.5377 0.3933
0.0035 312.5 10000 0.4350 0.6992 0.5363 0.4045

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.1