Update README.md
Browse files
README.md
CHANGED
|
@@ -120,11 +120,11 @@ Please refer to [togethercomputer/RedPajama-Data-1T](https://huggingface.co/data
|
|
| 120 |
|
| 121 |
**Training Procedure**
|
| 122 |
|
| 123 |
-
- **Hardware:**
|
| 124 |
-
- **Optimizer:**
|
| 125 |
-
- **Gradient Accumulations**:
|
| 126 |
-
- **Num of Tokens:**
|
| 127 |
-
- **Learning rate:**
|
| 128 |
|
| 129 |
## Community
|
| 130 |
|
|
|
|
| 120 |
|
| 121 |
**Training Procedure**
|
| 122 |
|
| 123 |
+
- **Hardware:** 8 A100
|
| 124 |
+
- **Optimizer:** Adam
|
| 125 |
+
- **Gradient Accumulations**: 1
|
| 126 |
+
- **Num of Tokens:** 131M tokens
|
| 127 |
+
- **Learning rate:** 1e-5
|
| 128 |
|
| 129 |
## Community
|
| 130 |
|