Update README.md

README.md

@@ -1,3 +1,70 @@
----
-license: mit
----
+---
+license: mit
+---
+# 0506_9_8
+
+This model is a fine-tuned version of [../../models/Qwen1.5-7B-sft-0502](https://huggingface.co/../../models/Qwen1.5-7B-sft-0502) on the alpaca_formatted_review_new_data_0505_greater_8 dataset.
+It achieves the following results on the evaluation set:
+
+- Loss: 0.5497
+
+## Model description
+
+Qwen1.5 is the beta version of Qwen2, a transformer-based decoder-only language model pretrained on a large amount of data. In comparison with the previous released Qwen, the improvements include:
+
+* 8 model sizes, including 0.5B, 1.8B, 4B, 7B, 14B, 32B and 72B dense models, and an MoE model of 14B with 2.7B activated;
+* Significant performance improvement in Chat models;
+* Multilingual support of both base and chat models;
+* Stable support of 32K context length for models of all sizes
+* No need of `trust_remote_code`.
+
+For more details, please refer to the [blog post](https://qwenlm.github.io/blog/qwen1.5/) and [GitHub repo](https://github.com/QwenLM/Qwen1.5).
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+
+- learning_rate: 7e-05
+- train_batch_size: 2
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 2
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 16
+- total_eval_batch_size: 2
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 13
+- num_epochs: 5.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+| :-----------: | :----: | :--: | :-------------: |
+|    0.6358     | 0.7619 |  20  |     0.5865      |
+|    0.6379     | 1.5238 |  40  |     0.5621      |
+|    0.6067     | 2.2857 |  60  |     0.5561      |
+|    0.5339     | 3.0476 |  80  |     0.5515      |
+|    0.6749     | 3.8095 | 100  |     0.5500      |
+|    0.6351     | 4.5714 | 120  |     0.5497      |
+
+
+### Framework versions
+
+- PEFT 0.10.0
+- Transformers 4.40.0
+- Pytorch 2.1.0+cu121
+- Datasets 2.14.5
+- Tokenizers 0.19.1