Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,76 @@
+---
+base_model: 01-ai/Yi-6B-200K
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- llama-cpp
+- gguf-my-repo
+widget:
+- example_title: Yi-34B-Chat
+  text: hi
+  output:
+    text: ' Hello! How can I assist you today?'
+- example_title: Yi-34B
+  text: There's a place where time stands still. A place of breath taking wonder,
+    but also
+  output:
+    text: ' an eerie sense that something is just not right…
+
+      Between the two worlds lies The Forgotten Kingdom - home to creatures long since
+      thought extinct and ancient magic so strong it defies belief! Only here can
+      you find what has been lost for centuries: An Elixir Of Life which will restore
+      youth and vitality if only those who seek its power are brave enough to face
+      up against all manner of dangers lurking in this mysterious land! But beware;
+      some say there may even exist powerful entities beyond our comprehension whose
+      intentions towards humanity remain unclear at best ---- they might want nothing
+      more than destruction itself rather then anything else from their quest after
+      immortality (and maybe someone should tell them about modern medicine)? In any
+      event though – one thing remains true regardless : whether or not success comes
+      easy depends entirely upon how much effort we put into conquering whatever challenges
+      lie ahead along with having faith deep down inside ourselves too ;) So let’s
+      get started now shall We?'
+---
+
+# Angel367/Yi-6B-200K-Q4_K_M-GGUF
+This model was converted to GGUF format from [`01-ai/Yi-6B-200K`](https://huggingface.co/01-ai/Yi-6B-200K) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
+Refer to the [original model card](https://huggingface.co/01-ai/Yi-6B-200K) for more details on the model.
+
+## Use with llama.cpp
+Install llama.cpp through brew (works on Mac and Linux)
+
+```bash
+brew install llama.cpp
+
+```
+Invoke the llama.cpp server or the CLI.
+
+### CLI:
+```bash
+llama-cli --hf-repo Angel367/Yi-6B-200K-Q4_K_M-GGUF --hf-file yi-6b-200k-q4_k_m.gguf -p "The meaning to life and the universe is"
+```
+
+### Server:
+```bash
+llama-server --hf-repo Angel367/Yi-6B-200K-Q4_K_M-GGUF --hf-file yi-6b-200k-q4_k_m.gguf -c 2048
+```
+
+Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
+
+Step 1: Clone llama.cpp from GitHub.
+```
+git clone https://github.com/ggerganov/llama.cpp
+```
+
+Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
+```
+cd llama.cpp && LLAMA_CURL=1 make
+```
+
+Step 3: Run inference through the main binary.
+```
+./llama-cli --hf-repo Angel367/Yi-6B-200K-Q4_K_M-GGUF --hf-file yi-6b-200k-q4_k_m.gguf -p "The meaning to life and the universe is"
+```
+or 
+```
+./llama-server --hf-repo Angel367/Yi-6B-200K-Q4_K_M-GGUF --hf-file yi-6b-200k-q4_k_m.gguf -c 2048
+```