sileod commited on
Commit
0c75e5b
1 Parent(s): 8ab1bd9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -323,7 +323,7 @@ Training code: https://colab.research.google.com/drive/1iB4Oxl9_B5W3ZDzXoWJN-olU
323
 
324
 
325
  This is the shared model with the MNLI classifier on top. Each task had a specific CLS embedding, which is dropped 10% of the time to facilitate model use without it. All multiple-choice model used the same classification layers. For classification tasks, models shared weights if their labels matched.
326
- The number of examples per task was capped to 64k. The model was trained for 120k steps with a batch size of 384, and a peak learning rate of 2e-5. Training took 10 days on Nvidia A30 24GB gpu.
327
 
328
  # Citation
329
 
 
323
 
324
 
325
  This is the shared model with the MNLI classifier on top. Each task had a specific CLS embedding, which is dropped 10% of the time to facilitate model use without it. All multiple-choice model used the same classification layers. For classification tasks, models shared weights if their labels matched.
326
+ The number of examples per task was capped to 64k. The model was trained for 200k steps with a batch size of 384, and a peak learning rate of 2e-5. Training took 12 days on Nvidia A30 24GB gpu.
327
 
328
  # Citation
329