PEFT
code
instruct
gpt2
souvik0306 commited on
Commit
0a6a2da
1 Parent(s): 3a6b7dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -10
README.md CHANGED
@@ -13,6 +13,7 @@ license: apache-2.0
13
  ### Finetuning Overview:
14
 
15
  **Model Used:** gpt2
 
16
  **Dataset:** HuggingFaceH4/no_robots
17
 
18
  #### Dataset Insights:
@@ -34,22 +35,17 @@ With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](ht
34
  - **Total Finetuning Cost:** $0.101
35
  - **Model Path:** gpt2
36
  - **Learning Rate:** 0.0002
37
- - **Data Split:** 99% train 1% validation
38
  - **Gradient Accumulation Steps:** 4
39
  - **lora r:** 32
40
  - **lora alpha:** 64
41
 
42
- ---
43
- Prompt Structure
44
  ```
45
- ### INSTRUCTION:
46
- [instruction]
47
-
48
- ### RESPONSE:
49
- [output]
50
  ```
51
- Training loss :
52
- ![training loss](https://cdn-uploads.huggingface.co/production/uploads/63ba46aa0a9866b28cb19a14/1iJWZwrORvuXmqRTq90qv.png)
53
 
 
54
 
55
  license: apache-2.0
 
13
  ### Finetuning Overview:
14
 
15
  **Model Used:** gpt2
16
+
17
  **Dataset:** HuggingFaceH4/no_robots
18
 
19
  #### Dataset Insights:
 
35
  - **Total Finetuning Cost:** $0.101
36
  - **Model Path:** gpt2
37
  - **Learning Rate:** 0.0002
38
+ - **Data Split:** 100% train
39
  - **Gradient Accumulation Steps:** 4
40
  - **lora r:** 32
41
  - **lora alpha:** 64
42
 
43
+ #### Prompt Structure
 
44
  ```
45
+ <|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>
 
 
 
 
46
  ```
47
+ #### Training loss :
 
48
 
49
+ ![training loss](https://cdn-uploads.huggingface.co/production/uploads/63ba46aa0a9866b28cb19a14/9bgb518kFwtDsFtrHzmTu.png)
50
 
51
  license: apache-2.0