Can we add https://huggingface.co/BlinkDL/rwkv-4-pile-14b/blob/main/RWKV-4-Pile-14B-20230313-ctx8192-test1050.pth
#6
by
BlinkDL
- opened
You can use https://pypi.org/project/rwkv/ :)
It's already a strong chatbot without finetuning and supports INT8 too: https://github.com/BlinkDL/ChatRWKV
Chat prompts: https://github.com/BlinkDL/ChatRWKV/tree/main/v2/prompt/default
Yes that's something I want to work on.
- What is the status of the PR integrating it to transformers?
- What is the VRAM budget of the largest version of the model?
The best would be to open an issue here: https://github.com/huggingface/text-generation-inference/issues
HF Integration: Takes time as the community is working on it (slowly).
VRAM budget: 16G is enough for 14B ctx8192 INT8 :)
I am training a new version on Pile v2 (my version: 1.73T tokens, 30% code).
BlinkDL
changed discussion status to
closed
BlinkDL
changed discussion status to
open