Redex-t5-small-finetuned-en-to-regex

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0813
• Semantic-accuracy: 0.916
• Gen Len: 10.476

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Semantic-accuracy	Gen Len
0.2376	1.0	563	0.0826	0.918	10.478
0.0817	2.0	1126	0.0691	0.92	10.506
0.0693	3.0	1689	0.0703	0.916	10.442
0.0625	4.0	2252	0.0720	0.916	10.506
0.0577	5.0	2815	0.0697	0.922	10.486
0.0544	6.0	3378	0.0711	0.908	10.458
0.0514	7.0	3941	0.0745	0.924	10.5
0.0481	8.0	4504	0.0765	0.91	10.462
0.0467	9.0	5067	0.0758	0.916	10.456
0.0451	10.0	5630	0.0741	0.924	10.48
0.0428	11.0	6193	0.0769	0.914	10.496
0.0432	12.0	6756	0.0779	0.912	10.446
0.0423	13.0	7319	0.0799	0.918	10.502
0.042	14.0	7882	0.0801	0.914	10.494
0.0405	15.0	8445	0.0769	0.914	10.476
0.0387	16.0	9008	0.0793	0.916	10.476
0.0383	17.0	9571	0.0818	0.916	10.474
0.0376	17.76	10000	0.0813	0.916	10.476

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2