Update README.md

README.md

@@ -56,6 +56,7 @@ The following table shows the hyper-paramters we used in our training process.
 | Peak Learning Rate    | 5e-5        |
 | Batch Size            | 4M          |
 | Weight Decay          | 0.1         |
+| Context Length        | 2k          |
 
 **Second phase**: We further adjusted the training corpus ratio, incorporating more domain-specific datasets (e.g., Math, Coding), and continued training for 50B tokens.
 
@@ -66,6 +67,7 @@ The following table shows the hyper-paramters we used in our training process.
 | Peak Learning Rate    | 5e-6        |
 | Batch Size            | 4M          |
 | Weight Decay          | 0.01        |
+| Context Length        | 4k          |
 
 ## Performance Evaluation Results