File size: 792 Bytes
beaba41 2e7367a beaba41 be7ffa4 beaba41 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
language:
- en
---
Converted version of [gpt4all](https://github.com/nomic-ai/gpt4all) weights with ggjt magic for use in [llama.cpp](https://github.com/ggerganov/llama.cpp) or [pyllamacpp](https://github.com/nomic-ai/pyllamacpp).
Full credit goes to the [GPT4All project](https://github.com/nomic-ai/gpt4all).
## Usage via pyllamacpp
Installation: `pip install pyllamacpp`
Download and inference:
```python
from huggingface_hub import hf_hub_download
from pyllamacpp.model import Model
#Download the model
hf_hub_download(repo_id="LLukas22/gpt4all-lora-quantized-ggjt", filename="ggjt-model.bin", local_dir=".")
#Load the model
model = Model(ggml_model="ggjt-model.bin", n_ctx=2000)
#Generate
prompt="User: How are you doing?\nBot:"
result=model.generate(prompt,n_predict=50)
``` |