|
--- |
|
datasets: |
|
- tatsu-lab/alpaca |
|
language: |
|
- en |
|
--- |
|
Converted version of [Sosaka/Alpaca-native-4bit-ggml](https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml) with ggjt magic for use in [llama.cpp](https://github.com/ggerganov/llama.cpp) or [pyllamacpp](https://github.com/nomic-ai/pyllamacpp). |
|
Full credit goes to [Sosaka](https://huggingface.co/Sosaka) |
|
|
|
## Usage via pyllamacpp |
|
|
|
Installation: `pip install pyllamacpp` |
|
|
|
Download and inference: |
|
|
|
```python |
|
from huggingface_hub import hf_hub_download |
|
from pyllamacpp.model import Model |
|
|
|
#Download the model |
|
hf_hub_download(repo_id="LLukas22/alpaca-native-7B-4bit-ggjt", filename="ggjt-model.bin", local_dir=".") |
|
|
|
#Load the model |
|
model = Model(ggml_model="ggjt-model.bin", n_ctx=2000) |
|
|
|
#Generate |
|
prompt="The meaning of life is" |
|
|
|
result=model.generate(prompt,n_predict=50) |
|
``` |
