README.md

metadata

tags:
  - generated_from_trainer
datasets:
  - custom
model-index:
  - name: xlm_r-joint_nlu-custom_ds
    results: []

xlm_r-joint_nlu-custom_ds

This model was trained from scratch on the custom dataset. It achieves the following results on the evaluation set:

• Loss: 0.0312
• Intent Accuracy: 1.0
• Intent F1 Macro: 1.0
• Slot F1: 0.9506
• Semantic Accuracy: 0.9474

Evaluation on the test set:

• Intent Accuracy: 1.0
• Slot F1: 0.9506294471811714
• Semantic Accuracy: 0.9473684210526315

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Intent Accuracy	Intent F1 Macro	Slot F1	Semantic Accuracy
No log	1.0	47	2.1385	0.6809	0.4650	0.1429	0.1809
No log	2.0	94	1.0050	0.9043	0.8890	0.2806	0.2128
No log	3.0	141	0.4169	0.9787	0.9582	0.3632	0.2660
No log	4.0	188	0.2661	0.9894	0.9798	0.6908	0.5745
No log	5.0	235	0.2036	0.9894	0.9798	0.7454	0.5532
No log	6.0	282	0.1547	0.9894	0.9881	0.7699	0.6489
No log	7.0	329	0.1094	1.0	1.0	0.8216	0.6596
No log	8.0	376	0.1061	1.0	1.0	0.9080	0.7128
No log	9.0	423	0.0639	1.0	1.0	0.9575	0.8511
No log	10.0	470	0.0571	1.0	1.0	0.9597	0.8511
0.7099	11.0	517	0.0527	1.0	1.0	0.9763	0.8723
0.7099	12.0	564	0.0408	1.0	1.0	0.9708	0.8723
0.7099	13.0	611	0.0415	1.0	1.0	0.9899	0.9043
0.7099	14.0	658	0.0347	1.0	1.0	0.9661	0.9149
0.7099	15.0	705	0.0388	1.0	1.0	0.9899	0.9149
0.7099	16.0	752	0.0333	1.0	1.0	0.9983	0.9255
0.7099	17.0	799	0.0533	1.0	1.0	0.9899	0.8936
0.7099	18.0	846	0.0404	1.0	1.0	0.9899	0.9043
0.7099	19.0	893	0.0408	1.0	1.0	0.9805	0.9043
0.7099	20.0	940	0.0387	1.0	1.0	0.9899	0.9255

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.14.7
• Tokenizers 0.15.0