How to install this locally and use offline? #1

by - opened

How to install this locally and use offline? Any references or videos would be great

this is actually the GGML version (my bad) - you should be able to use it with LLama.cpp

@TheStamp with llama.cpp, as in I need both this model and llama.cpp locally? I’m trying to run it in code using langchain instead of openai. Sorry for the newbiness

Then you want llama-cpp-python, which are llama.cpp bindings for Python. Allows you to load GGML files exactly the same as llama.cpp does, but easily accessible from code.

It can then be used either direct from your own Python code, or via an OpenAI-compatible API which you can put LangChain at, or any other client.

Thanks everyone, goal accomplished

Sign up or log in to comment