Commit
•
9644c4c
1
Parent(s):
8e7ca09
Update README.md
Browse files
README.md
CHANGED
|
@@ -52,5 +52,6 @@ The following hyperparameters were used during pre-training:
|
|
| 52 |
- num_devices: 4
|
| 53 |
- batch_size: 512
|
| 54 |
- training_steps: 250,000
|
| 55 |
-
- encoder
|
| 56 |
-
-
|
|
|
|
|
|
| 52 |
- num_devices: 4
|
| 53 |
- batch_size: 512
|
| 54 |
- training_steps: 250,000
|
| 55 |
+
- encoder layers: 12
|
| 56 |
+
- decoder layers: 12
|
| 57 |
+
- hidden size: 1024
|