Commit
•
3537a63
1
Parent(s):
df95f8b
Update README.md
Browse files
README.md
CHANGED
@@ -44,9 +44,8 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
44 |
## Training Details
|
45 |
|
46 |
- Trained for one epoch on SlimOrca-Dedup
|
47 |
-
- Learning rate:
|
48 |
- Cosine learning rate decay to 0
|
49 |
- Optimizer: AdamW
|
50 |
-
-
|
51 |
-
- Gradient accumulation steps (mini batch size): 64 (4)
|
52 |
- Trained with FP32
|
|
|
44 |
## Training Details
|
45 |
|
46 |
- Trained for one epoch on SlimOrca-Dedup
|
47 |
+
- Learning rate: 1e-5
|
48 |
- Cosine learning rate decay to 0
|
49 |
- Optimizer: AdamW
|
50 |
+
- Batch size: 256
|
|
|
51 |
- Trained with FP32
|