qfournier commited on
Commit
747e707
·
verified ·
1 Parent(s): 8632702

Fix typo in the learning rate of Stage 2

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -47,7 +47,7 @@ AMPLIFY is an efficient, state-of-the-art protein language model pre-trained usi
47
  | `max-steps` | 1000000 | 25000 (120M) or 50000 (350M) |
48
  | `max-length` | 512 | 2048 |
49
  | `optimizer` | adamw | adamw |
50
- | `lr` | 0.001 | 0.001 |
51
  | `betas` | (0.9, 0.95) | (0.9, 0.95) |
52
  | `eps` | 1.0e-08 | 1.0e-08 |
53
  | `weight-decay` | 0.01 | 0.01 |
 
47
  | `max-steps` | 1000000 | 25000 (120M) or 50000 (350M) |
48
  | `max-length` | 512 | 2048 |
49
  | `optimizer` | adamw | adamw |
50
+ | `lr` | 0.001 | 0.0001 |
51
  | `betas` | (0.9, 0.95) | (0.9, 0.95) |
52
  | `eps` | 1.0e-08 | 1.0e-08 |
53
  | `weight-decay` | 0.01 | 0.01 |