---
tags:
- generated_from_trainer
model-index:
- name: DS-Chatbox-gpt2-vietnamese-V3
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# DS-Chatbox-gpt2-vietnamese-V3

This model is a fine-tuned version of [NlpHUST/gpt2-vietnamese](https://huggingface.co/NlpHUST/gpt2-vietnamese) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.2578

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0015
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 2

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 3.1989        | 0.06  | 1000  | 3.0855          |
| 2.9934        | 0.12  | 2000  | 2.9327          |
| 2.8951        | 0.18  | 3000  | 2.8807          |
| 2.8375        | 0.23  | 4000  | 2.8522          |
| 2.8234        | 0.29  | 5000  | 2.8395          |
| 2.8097        | 0.35  | 6000  | 2.8198          |
| 2.804         | 0.41  | 7000  | 2.8119          |
| 2.8038        | 0.47  | 8000  | 2.8194          |
| 2.8005        | 0.53  | 9000  | 2.8135          |
| 2.7875        | 0.59  | 10000 | 2.8138          |
| 2.7649        | 0.64  | 11000 | 2.8101          |
| 2.7662        | 0.7   | 12000 | 2.7934          |
| 2.7563        | 0.76  | 13000 | 2.7802          |
| 2.7366        | 0.82  | 14000 | 2.7552          |
| 2.707         | 0.88  | 15000 | 2.7383          |
| 2.6932        | 0.94  | 16000 | 2.7115          |
| 2.6671        | 1.0   | 17000 | 2.6839          |
| 2.5019        | 1.05  | 18000 | 2.6664          |
| 2.4742        | 1.11  | 19000 | 2.6425          |
| 2.4658        | 1.17  | 20000 | 2.5986          |
| 2.4333        | 1.23  | 21000 | 2.5585          |
| 2.4084        | 1.29  | 22000 | 2.5246          |
| 2.3733        | 1.35  | 23000 | 2.4904          |
| 2.3384        | 1.41  | 24000 | 2.4525          |
| 2.2983        | 1.46  | 25000 | 2.4152          |
| 2.2626        | 1.52  | 26000 | 2.3866          |
| 2.2241        | 1.58  | 27000 | 2.3538          |
| 2.2054        | 1.64  | 28000 | 2.3278          |
| 2.1699        | 1.7   | 29000 | 2.3026          |
| 2.1467        | 1.76  | 30000 | 2.2826          |
| 2.1362        | 1.82  | 31000 | 2.2706          |
| 2.1312        | 1.87  | 32000 | 2.2625          |
| 2.1158        | 1.93  | 33000 | 2.2587          |
| 2.1291        | 1.99  | 34000 | 2.2578          |


### Framework versions

- Transformers 4.30.1
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3