---
license: apache-2.0
tags:
- generated_from_trainer
base_model: codeparrot/codeparrot-small
model-index:
- name: codeparrot-small-custom-functions-dataset-python
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# codeparrot-small-custom-functions-dataset-python

This model is a fine-tuned version of [codeparrot/codeparrot-small](https://huggingface.co/codeparrot/codeparrot-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4238

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 1.216         | 0.12  | 1    | 1.0747          |
| 1.051         | 0.25  | 2    | 1.0005          |
| 0.9855        | 0.38  | 3    | 0.9462          |
| 0.9259        | 0.5   | 4    | 0.9042          |
| 0.9236        | 0.62  | 5    | 0.8675          |
| 0.8644        | 0.75  | 6    | 0.8331          |
| 0.8148        | 0.88  | 7    | 0.8030          |
| 0.7554        | 1.0   | 8    | 0.7800          |
| 0.7815        | 1.12  | 9    | 0.7600          |
| 0.784         | 1.25  | 10   | 0.7440          |
| 0.635         | 1.38  | 11   | 0.7309          |
| 0.6666        | 1.5   | 12   | 0.7170          |
| 0.7676        | 1.62  | 13   | 0.6993          |
| 0.6608        | 1.75  | 14   | 0.6835          |
| 0.6885        | 1.88  | 15   | 0.6696          |
| 0.69          | 2.0   | 16   | 0.6582          |
| 0.6343        | 2.12  | 17   | 0.6463          |
| 0.709         | 2.25  | 18   | 0.6324          |
| 0.5446        | 2.38  | 19   | 0.6206          |
| 0.5298        | 2.5   | 20   | 0.6102          |
| 0.6478        | 2.62  | 21   | 0.6016          |
| 0.546         | 2.75  | 22   | 0.5941          |
| 0.6297        | 2.88  | 23   | 0.5871          |
| 0.4518        | 3.0   | 24   | 0.5814          |
| 0.566         | 3.12  | 25   | 0.5769          |
| 0.6285        | 3.25  | 26   | 0.5702          |
| 0.5938        | 3.38  | 27   | 0.5631          |
| 0.514         | 3.5   | 28   | 0.5568          |
| 0.5113        | 3.62  | 29   | 0.5504          |
| 0.512         | 3.75  | 30   | 0.5451          |
| 0.4392        | 3.88  | 31   | 0.5407          |
| 0.5097        | 4.0   | 32   | 0.5370          |
| 0.4866        | 4.12  | 33   | 0.5326          |
| 0.5028        | 4.25  | 34   | 0.5285          |
| 0.5438        | 4.38  | 35   | 0.5228          |
| 0.5424        | 4.5   | 36   | 0.5166          |
| 0.5156        | 4.62  | 37   | 0.5108          |
| 0.4335        | 4.75  | 38   | 0.5056          |
| 0.4298        | 4.88  | 39   | 0.5013          |
| 0.5268        | 5.0   | 40   | 0.4978          |
| 0.4714        | 5.12  | 41   | 0.4938          |
| 0.4659        | 5.25  | 42   | 0.4907          |
| 0.4573        | 5.38  | 43   | 0.4874          |
| 0.4689        | 5.5   | 44   | 0.4847          |
| 0.4346        | 5.62  | 45   | 0.4824          |
| 0.4563        | 5.75  | 46   | 0.4794          |
| 0.4505        | 5.88  | 47   | 0.4761          |
| 0.7359        | 6.0   | 48   | 0.4732          |
| 0.4704        | 6.12  | 49   | 0.4706          |
| 0.4223        | 6.25  | 50   | 0.4685          |
| 0.4789        | 6.38  | 51   | 0.4651          |
| 0.4402        | 6.5   | 52   | 0.4624          |
| 0.4454        | 6.62  | 53   | 0.4597          |
| 0.4496        | 6.75  | 54   | 0.4566          |
| 0.3942        | 6.88  | 55   | 0.4539          |
| 0.2915        | 7.0   | 56   | 0.4515          |
| 0.3926        | 7.12  | 57   | 0.4496          |
| 0.4102        | 7.25  | 58   | 0.4474          |
| 0.4235        | 7.38  | 59   | 0.4456          |
| 0.4841        | 7.5   | 60   | 0.4441          |
| 0.3914        | 7.62  | 61   | 0.4423          |
| 0.4417        | 7.75  | 62   | 0.4404          |
| 0.4212        | 7.88  | 63   | 0.4384          |
| 0.4343        | 8.0   | 64   | 0.4369          |
| 0.4159        | 8.12  | 65   | 0.4355          |
| 0.4193        | 8.25  | 66   | 0.4343          |
| 0.4393        | 8.38  | 67   | 0.4333          |
| 0.4507        | 8.5   | 68   | 0.4319          |
| 0.3855        | 8.62  | 69   | 0.4305          |
| 0.4064        | 8.75  | 70   | 0.4293          |
| 0.4044        | 8.88  | 71   | 0.4283          |
| 0.2957        | 9.0   | 72   | 0.4275          |
| 0.4442        | 9.12  | 73   | 0.4266          |
| 0.4142        | 9.25  | 74   | 0.4260          |
| 0.4022        | 9.38  | 75   | 0.4253          |
| 0.4161        | 9.5   | 76   | 0.4248          |
| 0.3828        | 9.62  | 77   | 0.4244          |
| 0.384         | 9.75  | 78   | 0.4241          |
| 0.3985        | 9.88  | 79   | 0.4239          |
| 0.4912        | 10.0  | 80   | 0.4238          |


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3