Update README.md

README.md

@@ -26,11 +26,11 @@ ctx_len = 1024
 n_layer = 32
 n_embd = 4096
 
+(there are ctx_len 2048 and 4096 models though they might be slightly weaker at generating short contents)
+
 Final checkpoint: RWKV-4-Pile-7B-20221115-8047.pth : Trained on the Pile for 332B tokens.
 * Pile loss 1.8415
 * LAMBADA ppl 4.38, acc 67.18%
 * PIQA acc 76.06%
 * SC2016 acc 73.44%
 * Hellaswag acc_norm 65.51%
-
-I am fine-tuning it to ctx_len 4096.