---
base_model: vinai/phobert-base-v2
tags:
- generated_from_trainer
model-index:
- name: CS505-GPT_T3vsLabel2T5_CSI-PhoBERT
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# CS505-GPT_T3vsLabel2T5_CSI-PhoBERT

This model is a fine-tuned version of [vinai/phobert-base-v2](https://huggingface.co/vinai/phobert-base-v2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0002

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 41
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 1.0   | 332  | 0.1938          |
| 0.302         | 1.99  | 664  | 0.1052          |
| 0.302         | 2.99  | 996  | 0.0698          |
| 0.158         | 3.99  | 1328 | 0.0487          |
| 0.0811        | 4.98  | 1660 | 0.0182          |
| 0.0811        | 5.98  | 1992 | 0.0239          |
| 0.053         | 6.98  | 2324 | 0.0264          |
| 0.025         | 7.98  | 2656 | 0.0086          |
| 0.025         | 8.97  | 2988 | 0.0048          |
| 0.0135        | 9.97  | 3320 | 0.0035          |
| 0.0102        | 10.97 | 3652 | 0.0075          |
| 0.0102        | 11.96 | 3984 | 0.0027          |
| 0.0094        | 12.96 | 4316 | 0.0013          |
| 0.0057        | 13.96 | 4648 | 0.0005          |
| 0.0057        | 14.95 | 4980 | 0.0002          |


### Framework versions

- Transformers 4.39.1
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2