Edit model card

lilt-roberta-en-base-finetuned-funsd

This model is a fine-tuned version of nielsr/lilt-roberta-en-base on the funsd-layoutlmv3 dataset. It achieves the following results on the evaluation set:

  • Loss: 1.6552
  • Precision: 0.8762
  • Recall: 0.8857
  • F1: 0.8809
  • Accuracy: 0.8068

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 0.1
  • training_steps: 2000

Training results

Training Loss Epoch Step Validation Loss Precision Recall F1 Accuracy
No log 5.26 100 1.1789 0.8506 0.8485 0.8495 0.7869
No log 10.53 200 1.2382 0.8360 0.8788 0.8569 0.7970
No log 15.79 300 1.3766 0.8557 0.8897 0.8724 0.7909
No log 21.05 400 1.5590 0.8368 0.8763 0.8561 0.7792
0.04 26.32 500 1.4379 0.8562 0.8813 0.8685 0.7992
0.04 31.58 600 1.5397 0.8593 0.8947 0.8766 0.8054
0.04 36.84 700 1.6132 0.8621 0.8723 0.8672 0.7933
0.04 42.11 800 1.6483 0.8566 0.8872 0.8716 0.7777
0.04 47.37 900 1.6593 0.8641 0.8813 0.8726 0.7895
0.0044 52.63 1000 1.6704 0.8595 0.8718 0.8656 0.7925
0.0044 57.89 1100 1.6795 0.8495 0.8803 0.8646 0.7748
0.0044 63.16 1200 1.5515 0.8604 0.8912 0.8755 0.7991
0.0044 68.42 1300 1.6665 0.8573 0.8867 0.8718 0.7821
0.0044 73.68 1400 1.5893 0.8604 0.8877 0.8738 0.7895
0.0008 78.95 1500 1.5613 0.8603 0.8872 0.8736 0.8123
0.0008 84.21 1600 1.5853 0.8521 0.8872 0.8693 0.8040
0.0008 89.47 1700 1.6539 0.8707 0.8833 0.8769 0.8077
0.0008 94.74 1800 1.6634 0.8787 0.8813 0.8800 0.8079
0.0008 100.0 1900 1.6534 0.8810 0.8862 0.8836 0.8073
0.0004 105.26 2000 1.6552 0.8762 0.8857 0.8809 0.8068

Framework versions

  • Transformers 4.23.0.dev0
  • Pytorch 1.12.1+cu113
  • Datasets 2.5.1
  • Tokenizers 0.13.0
Downloads last month
269
Inference Examples
Inference API (serverless) has been turned off for this model.

Finetuned from

Evaluation results