| language: | |
| - en | |
| tags: | |
| - spiking neural netwroks | |
| - text-generation | |
| - rwkv | |
| license: bsd-2-clause | |
| datasets: | |
| - bookcorpus | |
| --- | |
| --- | |
| # RWKV-4 7B | |
| ## Model Description | |
| SpikeGPT-BookCorpus is a L18-D768 SpikeGPT model trained on BookCorpus. See https://github.com/ridgerchu/SpikeGPT for details | |
| ctx_len = 1024 | |
| n_layer = 18 | |
| n_embd = 768 | |