Update README.md
Browse files
README.md
CHANGED
|
@@ -18,6 +18,9 @@ datasets:
|
|
| 18 |
|
| 19 |
RWKV-4 14B is a L40-D5120 causal language model trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
|
| 20 |
|
|
|
|
|
|
|
|
|
|
| 21 |
Use https://github.com/BlinkDL/ChatRWKV to run it.
|
| 22 |
|
| 23 |
RWKV-4-Pile-14B-2023xxxx-ctx4096-testxxx.pth : Fine-tuned to ctx_len 4096.
|
|
|
|
| 18 |
|
| 19 |
RWKV-4 14B is a L40-D5120 causal language model trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
|
| 20 |
|
| 21 |
+
args.n_layer = 40
|
| 22 |
+
args.n_embd = 5120
|
| 23 |
+
|
| 24 |
Use https://github.com/BlinkDL/ChatRWKV to run it.
|
| 25 |
|
| 26 |
RWKV-4-Pile-14B-2023xxxx-ctx4096-testxxx.pth : Fine-tuned to ctx_len 4096.
|