---
license: llama2
library_name: peft
tags:
- generated_from_trainer
base_model: codellama/CodeLlama-13b-Instruct-hf
metrics:
- accuracy
- bleu
- sacrebleu
- rouge
model-index:
- name: CodeLlama-13b-Instruct-hf_Fi__CMP_TR_size_304_epochs_10_2024-06-22_21-11-23_3558624
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# CodeLlama-13b-Instruct-hf_Fi__CMP_TR_size_304_epochs_10_2024-06-22_21-11-23_3558624

This model is a fine-tuned version of [codellama/CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.9516
- Accuracy: 0.486
- Chrf: 0.064
- Bleu: 0.0
- Sacrebleu: 0.0
- Rouge1: 0.113
- Rouge2: 0.0
- Rougel: 0.101
- Rougelsum: 0.113
- Meteor: 0.125

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 1
- eval_batch_size: 1
- seed: 3407
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-06
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 304
- training_steps: 3040

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Chrf  | Bleu | Sacrebleu | Rouge1 | Rouge2 | Rougel | Rougelsum | Meteor |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-----:|:----:|:---------:|:------:|:------:|:------:|:---------:|:------:|
| 0.8913        | 1.0   | 304  | 3.5216          | 0.458    | 0.071 | 0.0  | 0.0       | 0.062  | 0.0    | 0.061  | 0.062     | 0.186  |
| 0.0623        | 2.0   | 608  | 4.0963          | 0.456    | 0.01  | 0.0  | 0.0       | 0.003  | 0.0    | 0.003  | 0.003     | 0.004  |
| 0.0664        | 3.0   | 912  | 3.4598          | 0.496    | 0.013 | 0.0  | 0.0       | 0.02   | 0.001  | 0.02   | 0.02      | 0.027  |
| 0.0251        | 4.0   | 1216 | 3.3636          | 0.459    | 0.046 | 0.0  | 0.0       | 0.012  | 0.0    | 0.012  | 0.012     | 0.165  |
| 1.117         | 5.0   | 1520 | 3.4675          | 0.455    | 0.014 | 0.0  | 0.0       | 0.004  | 0.0    | 0.004  | 0.004     | 0.084  |
| 0.0704        | 6.0   | 1824 | 3.7849          | 0.459    | 0.016 | 0.0  | 0.0       | 0.024  | 0.0    | 0.024  | 0.024     | 0.068  |
| 0.0489        | 7.0   | 2128 | 3.3296          | 0.486    | 0.041 | 0.0  | 0.0       | 0.066  | 0.0    | 0.066  | 0.066     | 0.125  |
| 0.0588        | 8.0   | 2432 | 3.0692          | 0.486    | 0.06  | 0.0  | 0.0       | 0.059  | 0.0    | 0.056  | 0.057     | 0.145  |
| 0.0243        | 9.0   | 2736 | 2.9771          | 0.464    | 0.064 | 0.0  | 0.0       | 0.091  | 0.0    | 0.089  | 0.091     | 0.124  |
| 0.0546        | 10.0  | 3040 | 2.9516          | 0.486    | 0.064 | 0.0  | 0.0       | 0.113  | 0.0    | 0.101  | 0.113     | 0.125  |


### Framework versions

- PEFT 0.7.1
- Transformers 4.37.0
- Pytorch 2.2.1+cu121
- Datasets 2.20.0
- Tokenizers 0.15.2