LocalAI Model Loading

#2
by FIWisher - opened

Hello, is there a version available that we can load this into LocalAI with?
See: https://localai.io/model-compatibility/
Or does this require you to install pytorch and python locally and wrap a web service around it if you want it running locally?

Together org

Hi @FIWisher , I think the ggml version from TheBloke might work: https://huggingface.co/TheBloke/Llama-2-7B-32K-Instruct-GGML

Sign up or log in to comment