Scaling Data-Constrained Language Models
Paper
•
2305.16264
•
Published
•
16
Papers - Training - Epoch - 4 Epochs by Default
See Page 7 Figure 5 on right: Repeating for 4 epochs is almost as good as new data
See Page 7 Figure 5 on right: Repeating for 4 epochs is almost as good as new data