PEFT
code
instruct
zephyr
souvik0306 commited on
Commit
8d6f37f
1 Parent(s): f0dd1b8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -8
README.md CHANGED
@@ -14,6 +14,7 @@ license: apache-2.0
14
  ### Finetuning Overview:
15
 
16
  **Model Used:** HuggingFaceH4/zephyr-7b-alpha
 
17
  **Dataset:** HuggingFaceH4/no_robots
18
 
19
  #### Dataset Insights:
@@ -35,21 +36,17 @@ With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](ht
35
  - **Total Finetuning Cost:** $1.212
36
  - **Model Path:** HuggingFaceH4/zephyr-7b-alpha
37
  - **Learning Rate:** 0.0002
38
- - **Data Split:** 99% train 1% validation
39
  - **Gradient Accumulation Steps:** 4
40
  - **lora r:** 32
41
  - **lora alpha:** 64
42
 
43
  ---
44
- Prompt Structure
45
  ```
46
- ### INSTRUCTION:
47
- [instruction]
48
-
49
- ### RESPONSE:
50
- [text]
51
  ```
52
- Train loss :
53
 
54
  ![training loss](https://cdn-uploads.huggingface.co/production/uploads/63ba46aa0a9866b28cb19a14/7sEdsTGZhlAYTJqOKzbVo.png)
55
 
 
14
  ### Finetuning Overview:
15
 
16
  **Model Used:** HuggingFaceH4/zephyr-7b-alpha
17
+
18
  **Dataset:** HuggingFaceH4/no_robots
19
 
20
  #### Dataset Insights:
 
36
  - **Total Finetuning Cost:** $1.212
37
  - **Model Path:** HuggingFaceH4/zephyr-7b-alpha
38
  - **Learning Rate:** 0.0002
39
+ - **Data Split:** 100% train
40
  - **Gradient Accumulation Steps:** 4
41
  - **lora r:** 32
42
  - **lora alpha:** 64
43
 
44
  ---
45
+ #### Prompt Structure
46
  ```
47
+ <|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>
 
 
 
 
48
  ```
49
+ #### Train loss :
50
 
51
  ![training loss](https://cdn-uploads.huggingface.co/production/uploads/63ba46aa0a9866b28cb19a14/7sEdsTGZhlAYTJqOKzbVo.png)
52