llama_7b_qlora_pds-eval
This model is a fine-tuned version of DevaMalla/llama7b on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.9304
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 10
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
1.2419 | 0.03 | 50 | 1.0315 |
0.9965 | 0.05 | 100 | 0.9918 |
0.9747 | 0.08 | 150 | 0.9787 |
0.9771 | 0.11 | 200 | 0.9705 |
0.9653 | 0.14 | 250 | 0.9645 |
0.9507 | 0.16 | 300 | 0.9599 |
0.9596 | 0.19 | 350 | 0.9563 |
0.9497 | 0.22 | 400 | 0.9542 |
0.9481 | 0.24 | 450 | 0.9517 |
0.9475 | 0.27 | 500 | 0.9495 |
0.9376 | 0.3 | 550 | 0.9476 |
0.943 | 0.33 | 600 | 0.9463 |
0.952 | 0.35 | 650 | 0.9447 |
0.9489 | 0.38 | 700 | 0.9433 |
0.9347 | 0.41 | 750 | 0.9421 |
0.9307 | 0.44 | 800 | 0.9414 |
0.9366 | 0.46 | 850 | 0.9406 |
0.9246 | 0.49 | 900 | 0.9393 |
0.9267 | 0.52 | 950 | 0.9383 |
0.9358 | 0.54 | 1000 | 0.9373 |
0.9405 | 0.57 | 1050 | 0.9365 |
0.9276 | 0.6 | 1100 | 0.9359 |
0.9403 | 0.63 | 1150 | 0.9353 |
0.9218 | 0.65 | 1200 | 0.9347 |
0.9406 | 0.68 | 1250 | 0.9343 |
0.9257 | 0.71 | 1300 | 0.9337 |
0.92 | 0.73 | 1350 | 0.9334 |
0.9356 | 0.76 | 1400 | 0.9328 |
0.9279 | 0.79 | 1450 | 0.9322 |
0.9214 | 0.82 | 1500 | 0.9320 |
0.9214 | 0.84 | 1550 | 0.9316 |
0.9258 | 0.87 | 1600 | 0.9312 |
0.9308 | 0.9 | 1650 | 0.9310 |
0.9251 | 0.93 | 1700 | 0.9307 |
0.9207 | 0.95 | 1750 | 0.9306 |
0.926 | 0.98 | 1800 | 0.9304 |
Framework versions
- Transformers 4.32.0.dev0
- Pytorch 2.0.0
- Datasets 2.12.0
- Tokenizers 0.13.3
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.
Model tree for DevaMalla/llama_7b_qlora_pds-eval
Base model
DevaMalla/llama7b