GGML models for CPU inference

#1
by Vetehinen - opened

Thank you for the amazing work TurkuNLP!

For curious people with less compute power at their disposal, I uploaded bloomz.cpp GGML 4 bit quantized versions of 3B, 8B and 13B models.
No GPU required, only ability to compile and run bloomz.cpp and some RAM to load model.

https://huggingface.co/Vetehinen/gpt3-finnish-3B-GGML
https://huggingface.co/Vetehinen/gpt3-finnish-8B-GGML
https://huggingface.co/Vetehinen/gpt3-finnish-13B-GGML

Sign up or log in to comment