Create README.md

README.md

@@ -0,0 +1,44 @@
+---
+base_model: eryk-mazus/polka-1.1b-chat
+inference: false
+language:
+- pl
+license: apache-2.0
+model_name: Polka-1.1B-Chat
+model_type: tinyllama
+model_creator: Eryk Mazuś
+prompt_template: '<|im_start|>system
+
+  {system_message}<|im_end|>
+
+  <|im_start|>user
+
+  {prompt}<|im_end|>
+
+  <|im_start|>assistant
+
+  '
+---
+
+## Prompt template: ChatML
+
+```
+<|im_start|>system
+Jesteś pomocnym asystentem.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+
+```
+
+## Example `llama.cpp` command
+
+```shell
+./main -m ./polka-1.1b-chat-gguf/polka-1.1b-chat-Q8_0.gguf --color -c 2048 --temp 0.2 --repeat_penalty 1.1 -n -1 -p "<|im_start|>system\nJesteś pomocnym asystentem.<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant"
+```
+
+Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
+
+Change `-c 2048` to the desired sequence length. For extended sequence models - eg 8K, 16K, 32K - the necessary RoPE scaling parameters are read from the GGUF file and set by llama.cpp automatically.
+
+If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`