codet5-small-ft-v5 / README.md
ayeshgk's picture
End of training
d3b795c verified
metadata
license: apache-2.0
base_model: Salesforce/codet5-small
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: codet5-small-ft-v5
    results: []

codet5-small-ft-v5

This model is a fine-tuned version of Salesforce/codet5-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.2463
  • Rouge1: 65.8333
  • Rouge2: 56.2963
  • Rougel: 65.6481
  • Rougelsum: 65.5556
  • Gen Len: 12.3333

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 4
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 8 3.3878 36.9391 29.7138 36.367 36.2452 16.5278
No log 2.0 16 2.6133 61.7593 52.7981 61.8981 61.3889 12.1111
No log 3.0 24 2.3167 61.0185 51.7328 61.0185 60.5556 11.8611
No log 4.0 32 2.2463 65.8333 56.2963 65.6481 65.5556 12.3333

Framework versions

  • Transformers 4.38.0.dev0
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0