PEFT
code
instruct
zephyr
souvik0306 commited on
Commit
f0dd1b8
1 Parent(s): 9a6c51f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -6,8 +6,7 @@ tags:
6
  - zephyr
7
 
8
  datasets:
9
- - Zangs3011/no_robots_FalconChatFormated
10
-
11
  base_model: HuggingFaceH4/zephyr-7b-alpha
12
  license: apache-2.0
13
  ---
@@ -15,11 +14,11 @@ license: apache-2.0
15
  ### Finetuning Overview:
16
 
17
  **Model Used:** HuggingFaceH4/zephyr-7b-alpha
18
- **Dataset:** Zangs3011/no_robots_FalconChatFormated
19
 
20
  #### Dataset Insights:
21
 
22
- No Robots is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better.
23
 
24
  #### Finetuning Details:
25
 
@@ -38,6 +37,8 @@ With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](ht
38
  - **Learning Rate:** 0.0002
39
  - **Data Split:** 99% train 1% validation
40
  - **Gradient Accumulation Steps:** 4
 
 
41
 
42
  ---
43
  Prompt Structure
 
6
  - zephyr
7
 
8
  datasets:
9
+ - HuggingFaceH4/no_robots
 
10
  base_model: HuggingFaceH4/zephyr-7b-alpha
11
  license: apache-2.0
12
  ---
 
14
  ### Finetuning Overview:
15
 
16
  **Model Used:** HuggingFaceH4/zephyr-7b-alpha
17
+ **Dataset:** HuggingFaceH4/no_robots
18
 
19
  #### Dataset Insights:
20
 
21
+ [No Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better.
22
 
23
  #### Finetuning Details:
24
 
 
37
  - **Learning Rate:** 0.0002
38
  - **Data Split:** 99% train 1% validation
39
  - **Gradient Accumulation Steps:** 4
40
+ - **lora r:** 32
41
+ - **lora alpha:** 64
42
 
43
  ---
44
  Prompt Structure