codet5-small-ft-v8-cpatd

This model is a fine-tuned version of Salesforce/codet5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4831
• Rouge1: 63.6456
• Rouge2: 52.7919
• Rougel: 63.1978
• Rougelsum: 63.2113
• Gen Len: 17.0514

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.6958	1.0	3063	0.5815	61.3112	50.8406	60.8334	60.8509	17.2215
0.6116	2.0	6126	0.5190	62.7088	51.7838	62.2533	62.2661	17.0108
0.5564	3.0	9189	0.4942	63.4158	52.592	62.9418	62.9443	17.1236
0.5447	4.0	12252	0.4831	63.6456	52.7919	63.1978	63.2113	17.0514

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0