Update README.md
Browse files
README.md
CHANGED
@@ -36,19 +36,19 @@ print(result[0]['generated_text'])
|
|
36 |
```
|
37 |
|
38 |
## Training Details
|
39 |
-
lora_r: 16
|
40 |
-
lora_alpha: 8
|
41 |
lora_dropout: 0.05
|
42 |
-
target_modules: q, k, v, o, gate_proj, down_proj, up_proj, lm_head
|
43 |
-
weight_decay: 0
|
44 |
-
optmizer: paged_adamw_32bit
|
45 |
-
lr: 1e-4
|
46 |
-
lr_scheduler: cosine
|
47 |
-
max_seq_len: 4096
|
48 |
-
batch_size: 4
|
49 |
-
max_grad_norm: 0.5
|
50 |
-
warmup_ratio: 0.05
|
51 |
-
num_epochs: 1
|
52 |
|
53 |
Training was performed on the python subset of the ds-coder-instruct dataset.
|
54 |
|
|
|
36 |
```
|
37 |
|
38 |
## Training Details
|
39 |
+
lora_r: 16
|
40 |
+
lora_alpha: 8
|
41 |
lora_dropout: 0.05
|
42 |
+
target_modules: q, k, v, o, gate_proj, down_proj, up_proj, lm_head
|
43 |
+
weight_decay: 0
|
44 |
+
optmizer: paged_adamw_32bit
|
45 |
+
lr: 1e-4
|
46 |
+
lr_scheduler: cosine
|
47 |
+
max_seq_len: 4096
|
48 |
+
batch_size: 4
|
49 |
+
max_grad_norm: 0.5
|
50 |
+
warmup_ratio: 0.05
|
51 |
+
num_epochs: 1
|
52 |
|
53 |
Training was performed on the python subset of the ds-coder-instruct dataset.
|
54 |
|