Update README.md

README.md

@@ -1,6 +1,8 @@
 ---
 inference: false
 license: other
+datasets:
+- jondurbin/airoboros-gpt4-1.2
 ---
 
 <!-- header start -->
@@ -34,6 +36,14 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
 * [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/airoboros-33B-gpt4-1.2-GGML)
 * [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/jondurbin/airoboros-33b-gpt4-1.2)
 
+## Prompt template
+
+```
+A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input.
+USER: prompt
+ASSISTANT: 
+```
+
 <!-- compatibility_ggml start -->
 ## Compatibility
 
@@ -80,7 +90,6 @@ Refer to the Provided Files table below to see what files use which methods, and
 | airoboros-33b-gpt4-1.2.ggmlv3.q6_K.bin | q6_K | 6 | 26.69 GB | 29.19 GB | New k-quant method. Uses GGML_TYPE_Q8_K - 6-bit quantization - for all tensors |
 | airoboros-33b-gpt4-1.2.ggmlv3.q8_0.bin | q8_0 | 8 | 34.56 GB | 37.06 GB | Original llama.cpp quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |
 
-
 **Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
 
 ## How to run in `llama.cpp`
@@ -88,7 +97,7 @@ Refer to the Provided Files table below to see what files use which methods, and
 I use the following command line; adjust for your tastes and needs:
 
 ```
-./main -t 10 -ngl 32 -m airoboros-33b-gpt4-1.2.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
+./main -t 10 -ngl 32 -m airoboros-33b-gpt4-1.2.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "USER: Write a story about llamas\nASSISTANT:"
 ```
 Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
 
@@ -130,3 +139,71 @@ Thank you to all my generous patrons and donaters!
 
 # Original model card: John Durbin's Airoboros 33B GPT4 1.2
 
+### Overview
+
+This is a qlora fine-tuned 33b parameter LlaMa model, using completely synthetic training data created gpt4 via https://github.com/jondurbin/airoboros
+
+This is mostly an extension of [1.1](https://huggingface.co/jondurbin/airoboros-33b-gpt4-1.1) with thousands of new training data and an update to allow "PLAINFORMAT" at the end of coding prompts to just print the code without backticks or explanations/usage/etc.
+
+The dataset used to fine-tune this model is available [here](https://huggingface.co/datasets/jondurbin/airoboros-gpt4-1.2), with a specific focus on:
+- coding
+- math/reasoning (using orca style ELI5 instruction/response pairs)
+- trivia
+- role playing
+- multiple choice and fill-in-the-blank
+- context-obedient question answering
+- theory of mind
+- misc/general
+
+This model was fine-tuned with a fork of [qlora](https://github.com/jondurbin/qlora), which among other things was updated to use a slightly modified vicuna template to be compatible with the 7b/13b versions:
+
+```
+A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input. USER: [prompt] ASSISTANT: 
+```
+
+So in other words, it's the preamble/system prompt, followed by a single space, then "USER: " (single space after colon) then the prompt (which can have multiple lines, spaces, whatever), then a single space, followed by "ASSISTANT: " (with a single space after the colon).
+
+### Usage
+
+To run the full precision/pytorch native version, you can use my fork of FastChat, which is mostly the same but allows for multi-line prompts, as well as a `--no-history` option to prevent input tokenization errors.
+```
+pip install git+https://github.com/jondurbin/FastChat
+```
+
+Be sure you are pulling the latest branch!
+
+Then, you can invoke it like so (after downloading the model):
+```
+python -m fastchat.serve.cli \
+  --model-path airoboros-33b-gpt4-1.2 \
+  --temperature 0.5 \
+  --max-new-tokens 2048 \
+  --no-history
+```
+
+Alternatively, please check out TheBloke's quantized versions:
+
+- https://huggingface.co/TheBloke/airoboros-33B-gpt4-1.2-GPTQ
+- https://huggingface.co/TheBloke/airoboros-33B-gpt4-1.2-GGML
+
+### Coding updates from gpt4/1.1:
+
+I added a few hundred instruction/response pairs to the training data with "PLAINFORMAT" as a single, all caps term at the end of the normal instructions, which produce plain text output instead of markdown/backtick code formatting.
+
+It's not guaranteed to work all the time, but mostly it does seem to work as expected.
+
+So for example, instead of:
+```
+Implement the Snake game in python.
+```
+
+You would use:
+```
+Implement the Snake game in python.  PLAINFORMAT
+```
+
+### Other updates from gpt4/1.1:
+
+- Several hundred role-playing data.
+- A few thousand ORCA style reasoning/math questions with ELI5 prompts to generate the responses (should not be needed in your prompts to this model however, just ask the question).
+- Many more coding examples in various languages, including some that use specific libraries (pandas, numpy, tensorflow, etc.)