thai-squad

This model is a fine-tuned version of deepset/xlm-roberta-base-squad2 on Thai dataset from iApp Technology Co., Ltd..

Intended uses & limitations

This model intends to use with Thai question and answering task

Training and evaluation data

Trained and evaluated by iApp Technology Co., Ltd. dataset.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Performance

Evaluated on the SQuAD 1.0 test dataset

"exact": 62.51728907330567
"f1": 73.62388955749958
"total": 723

Framework versions

• Transformers 4.11.3
• Pytorch 1.9.0+cu111
• Datasets 1.14.0
• Tokenizers 0.10.3