---
base_model: Qwen/Qwen-14B
tags:
- generated_from_trainer
model-index:
- name: nampdn-ai_tiny-textbooks
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# nampdn-ai_tiny-textbooks

This model is a fine-tuned version of [Qwen/Qwen-14B](https://huggingface.co/Qwen/Qwen-14B) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.3572

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 0.01
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step  | Validation Loss |
|:-------------:|:-----:|:-----:|:---------------:|
| 2.4651        | 0.02  | 200   | 2.3996          |
| 2.4335        | 0.04  | 400   | 2.3799          |
| 2.3848        | 0.06  | 600   | 2.3746          |
| 2.4037        | 0.08  | 800   | 2.3714          |
| 2.3985        | 0.1   | 1000  | 2.3693          |
| 2.4072        | 0.12  | 1200  | 2.3673          |
| 2.4028        | 0.14  | 1400  | 2.3665          |
| 2.3748        | 0.16  | 1600  | 2.3643          |
| 2.4119        | 0.18  | 1800  | 2.3635          |
| 2.4002        | 0.2   | 2000  | 2.3640          |
| 2.3865        | 0.22  | 2200  | 2.3635          |
| 2.4           | 0.24  | 2400  | 2.3628          |
| 2.4096        | 0.26  | 2600  | 2.3625          |
| 2.3976        | 0.28  | 2800  | 2.3614          |
| 2.3767        | 0.3   | 3000  | 2.3618          |
| 2.4151        | 0.32  | 3200  | 2.3616          |
| 2.3835        | 0.34  | 3400  | 2.3605          |
| 2.3995        | 0.36  | 3600  | 2.3608          |
| 2.4121        | 0.38  | 3800  | 2.3602          |
| 2.4262        | 0.4   | 4000  | 2.3591          |
| 2.3604        | 0.42  | 4200  | 2.3594          |
| 2.3954        | 0.44  | 4400  | 2.3594          |
| 2.3743        | 0.46  | 4600  | 2.3587          |
| 2.4069        | 0.48  | 4800  | 2.3591          |
| 2.4103        | 0.5   | 5000  | 2.3585          |
| 2.4133        | 0.52  | 5200  | 2.3585          |
| 2.4229        | 0.54  | 5400  | 2.3578          |
| 2.4397        | 0.56  | 5600  | 2.3581          |
| 2.4237        | 0.58  | 5800  | 2.3581          |
| 2.4109        | 0.6   | 6000  | 2.3577          |
| 2.43          | 0.62  | 6200  | 2.3575          |
| 2.3999        | 0.64  | 6400  | 2.3572          |
| 2.3771        | 0.66  | 6600  | 2.3577          |
| 2.4119        | 0.68  | 6800  | 2.3576          |
| 2.3877        | 0.7   | 7000  | 2.3576          |
| 2.411         | 0.72  | 7200  | 2.3569          |
| 2.3808        | 0.74  | 7400  | 2.3570          |
| 2.3989        | 0.76  | 7600  | 2.3571          |
| 2.422         | 0.78  | 7800  | 2.3569          |
| 2.3768        | 0.8   | 8000  | 2.3569          |
| 2.3988        | 0.82  | 8200  | 2.3572          |
| 2.3927        | 0.84  | 8400  | 2.3572          |
| 2.3961        | 0.86  | 8600  | 2.3573          |
| 2.4021        | 0.88  | 8800  | 2.3570          |
| 2.3889        | 0.9   | 9000  | 2.3570          |
| 2.404         | 0.92  | 9200  | 2.3570          |
| 2.3982        | 0.94  | 9400  | 2.3572          |
| 2.4018        | 0.96  | 9600  | 2.3573          |
| 2.3717        | 0.98  | 9800  | 2.3572          |
| 2.4076        | 1.0   | 10000 | 2.3572          |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.1.0
- Datasets 2.14.5
- Tokenizers 0.14.1