is there the GGUF version of the model?
#11
by
MatrixIA
- opened
I tried converting it myself but colab keeps crashing due to memory use.
@MatrixIA - I've tried using this one:
https://huggingface.co/bartowski/llama-3-sqlcoder-8b-GGUF
However, I'm getting gibberish - not sure what's going on. All the llama2 models I've been working with work fine with llama.cpp (python), but I can't seem to get any llama3 models to work. I'm currently only using GGUF models.
I would love to know if you can get things working on your end!
I will try using this one : https://huggingface.co/bartowski/llama-3-sqlcoder-8b-GGUF
I have also converted it myself to the gguf version I just need to upload the files to the hub.
i will let u know about the results as soon as possible.