Create README.md

README.md

@@ -0,0 +1,30 @@
+---
+datasets:
+- tatsu-lab/alpaca
+language:
+- en
+---
+Converted version of [Sosaka/Alpaca-native-4bit-ggml](https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml) with ggjt magic for use in [llama.cpp] or [pyllamacpp](https://github.com/nomic-ai/pyllamacpp).
+Full creadit goes to [Sosaka](https://huggingface.co/Sosaka)
+
+## Usage via pyllamacpp
+
+Installation: `pip install pyllamacpp`
+
+Download and inference:
+
+```python
+from huggingface_hub import hf_hub_download
+from pyllamacpp.model import Model
+
+#Download the model
+hf_hub_download(repo_id="LLukas22/alpaca-native-7B-4bit-ggjt", filename="ggjt-model.bin", local_dir=".")
+
+#Load the model
+model = Model(ggml_model="ggjt-model.bin", n_ctx=2000)
+
+#Generate
+prompt="The meaning of life is"
+
+result=model.generate(prompt,n_predict=50)
+```