--- license: llama2 language: - en pipeline_tag: text-generation inference: false tags: - storywriting - finetuned - not-for-all-audiences - gguf base_model: SnakyMcSnekFace/Psyfighter2-13B-vore model_type: llama prompt_template: > ### Instruction: Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Input: {prompt} ### Response: --- # Model Card for Psyfighter2-13B-vore-GGUF This is a quantized version of [SnakyMcSnekFace/Psyfighter2-13B-vore](https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore) model. You can find the `F16` precision model weights and details of how the model was trained [in that repository](https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore). This model is a version of [KoboldAI/LLaMA2-13B-Psyfighter2](https://huggingface.co/KoboldAI/LLaMA2-13B-Psyfighter2) finetuned to better understand vore context. The primary purpose of this model is to be a storywriting assistant, a conversational model in a chat, and an interactive choose-your-own-adventure text game. The model has preliminary support for Adventure Mode. It is still work in progress, and the model will be further trained to be better at storytelling and to not require complicated configuration. **(See instructions below for configuring Adventure Mode for optimal performance.)** ## Model Details The model behaves similarly to `KoboldAI/LLaMA2-13B-Psyfighter2`, which it was derived from. Please [see the README.md here](https://huggingface.co/KoboldAI/LLaMA2-13B-Psyfighter2/blob/main/README.md) to learn more. This model was fine-tuned on ~55 MiB of free-form text, containing stories focused around the vore theme. As a result, it has a strong vorny bias. ## How to Get Started with the Model The model can be used with any AI chatbots and front-ends designed to work with `.gguf` models. The model fits fully into 8GB VRAM, but can also run with degraded performance on smaller graphics cards. Similarly to the base model, the less prompt the model receives, the more creative is the output. For example, the writing assistant will generate an entire story when prompted with only 2-3 words. In the chat mode, if the conversation is not going where you would like it to go, edit the model's output and let it continue generation. The model will also match the style of the conversation. There are two versions of the model: `Q4_K_M` (smaller and faster) and `Q8_0` (slower, but better prose quality). ### Koboldcpp Colab Notebook The easiest way to try out the model is [Koboldcpp Colab Notebook](https://colab.research.google.com/github/lostruins/koboldcpp/blob/concedo/colab.ipynb). This method doesn't require you to have a powerful graphics card. - Open the notebook - Paste the model URL into the `Model` field: `https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore-GGUF/resolve/main/Psyfighter2-13B-vore.Q4_K_M.gguf` - Start the notebook, wait for the URL to CloudFlare tunnel to appear at the bottom and click it - Select "Settings" and configure them as follows: - In "Basic" tab: - Temperature = 0.8 - Amount to Gen. = 512 - Top p Sampling = 0.9 - Repetition Pentalty = 1.1 - In "Advanced" tab: - Min-P = 0.1 - EOS Token Ban = Unban - Placeholder Tags = Checked - Select "Scenarios" -> "New Story" to use the model as a writing assistant #### To run `Q8_0` model in Colab notebook - Paste the model URL into the `Model` field: `https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore-GGUF/resolve/main/Psyfighter2-13B-vore.Q8_0.gguf` - Set `Layers` field to `30` (reduce this number if the model fails to start) #### Adventure Mode The model has **preliminary** support for the adventure mode. Select "Scenarios" -> "New Adventure" to start an adventure. **IMPORTANT - The context must be configured exactly as described below or the model will not perform well.** - Select "Context" - Paste this text into "Memory" verbatim: ``` ### Instruction: Text transcript of a never-ending adventure story, written by the AI assistant. AI assistant uses vivid and evocative language to create a well-written novel. Characters are proactive and take initiative. Think about what goals the characters of the story have and write what they do to achieve those goals. ### Input: ``` - Ensure that "Newline After Memory" is checked - Paste this text into "Author's note" verbatim **(IMPORTANT: the Author's Note must end with two empty lines)**: ``` Write a few paragraphs that advance the plot of the story. ### Response: ``` - Set "A/N Strength" to "Immediate" - Set "Author's Note Template" to `<|>` #### Adventure - creating premise and taking turns Select "Story" in the bottom left corner to generate premise of the story, and "Action" to take actions with your character. In the adventure mode, the model expects all player actions to be written in second person. For example: ``` As you venture deeper into the damp cave, you come across a lone goblin. The vile creature mumbles something to itself as it stares at the glowing text on a cave wall. It doesn't notice your approach. > You sneak behind the goblin and hit it with the sword. ``` Check "Allow Editing" to make edits to the story to overwrite and re-generate parts of the model's response. This is useful if the model makes a mistake or the story doesn't go in the direction that you like. ### Backyard AI Another convenient way to use the model is [Backyard AI](https://backyard.ai/) application, which allows you to run the model locally on your computer. You'll need a graphics card with at least 8GB VRAM to use the model comfortably. If you don't have a powerful GPU, Backyard AI provides an option of running the model on their servers, but it costs money. #### Download directly from HuggingFace (beta) In the left panel, click `Manage Models`, then select `Hugging face models`. Paste `https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore-GGUF` into the text field and press `Fetch Models`. Click `Download` button to the next to the model format. Once the model is downloaded, you can select it in your character card or set it as a default model. #### Download manually Download [Psyfighter2-13B-vore.Q4_K_M.gguf](https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore-GGUF/resolve/main/Psyfighter2-13B-vore.Q4_K_M.gguf) or [Psyfighter2-13B-vore.Q8_0.gguf](https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore-GGUF/resolve/main/Psyfighter2-13B-vore.Q8_0.gguf) file into `%appdata%\faraday\models` folder on your computer. The model should appear in `Manage Models` menu under `Downloaded Models`. You can then select it in your character card or set it as a default model. ## Model updates - 09/02/2024 - fine-tuned the model to follow Kobold AI Adventure Mode format - 06/02/2024 - fixed errors in training and merging, significantly improving the overall prose quality - 05/25/2024 - updated training process, making the model more coherent and improving the writing quality - 04/13/2024 - uploaded the first version of the model