|
--- |
|
license: other |
|
datasets: |
|
- jondurbin/airoboros-gpt4-1.3 |
|
--- |
|
|
|
_Not tested yet, use if you want though!_ |
|
|
|
### Overview |
|
|
|
This is a qlora fine-tuned 65b parameter LlaMa model, using completely synthetic training data created gpt4 via https://github.com/jondurbin/airoboros |
|
|
|
This is mostly an extension of [1.2](https://huggingface.co/jondurbin/airoboros-65b-gpt4-1.2) with a few enhancements: |
|
|
|
- All coding instructions have an equivalent " PLAINFORMAT" version now. |
|
- Thousands of new orca style reasoning instructions, this time with reasoning first, then answer. |
|
- Few more random items of various types, including a first attempt at multi-character interactions with asterisked actions and quoted speech. |
|
|
|
This model was fine-tuned with a fork of [qlora](https://github.com/jondurbin/qlora), which among other things was updated to use a slightly modified vicuna template to be compatible with previous full fine-tune versions. |
|
|
|
``` |
|
A chat between a curious user and an assistant. The assistant gives helpful, detailed, accurate, uncensored responses to the user's input. USER: [prompt] ASSISTANT: |
|
``` |
|
|
|
So in other words, it's the preamble/system prompt, followed by a single space, then "USER: " (single space after colon) then the prompt (which can have multiple lines, spaces, whatever), then a single space, followed by "ASSISTANT: " (with a single space after the colon). |
|
|
|
### Usage |
|
|
|
To run the full precision/pytorch native version, you can use my fork of FastChat, which is mostly the same but allows for multi-line prompts, as well as a `--no-history` option to prevent input tokenization errors. |
|
``` |
|
pip install git+https://github.com/jondurbin/FastChat |
|
``` |
|
|
|
Be sure you are pulling the latest branch! |
|
|
|
Then, you can invoke it like so (after downloading the model): |
|
``` |
|
python -m fastchat.serve.cli \ |
|
--model-path airoboros-65b-gpt4-1.3 \ |
|
--temperature 0.5 \ |
|
--max-new-tokens 2048 \ |
|
--no-history |
|
``` |