Update README.md

README.md

@@ -1,3 +1,79 @@
 ---
 license: cc-by-nc-4.0
 ---
+# Disinfo4_mistral-ft-optimized-1218: A Hugging Face Model README
+
+## Overview
+
+`Disinfo4_mistral-ft-optimized-1218` is an experimental language model fine tune developed to synthesize and analyze complex narratives within the realms of continental philosophy, conspiracy theories, and political discourse. It represents the fourth iteration in the [disinfo.zone](https://disinfo.zone) dataset series, fine-tuned on the `mistral-ft-optimized-1218` framework. This model, based on a 7B-parameter Mistral architecture, is specifically designed to emulate and deconstruct writing styles pertinent to its target domains. 
+
+This is not your regular LLM.
+
+### Key Features
+
+- **Model Size:** 7 billion parameters.
+- **Core Focus:** Continental philosophy, conspiracy theories, and politics.
+- **Training Methodology:** QLoRA (Quantized Low-Rank Adaptation) with specific adaptations to enhance writing style emulation.
+- **Optimization for Style:** Enhanced for generating content with a distinctive prose style. This does not sound like other LLM's and if you use it like other LLM's (answering riddles, etc), it will perform poorly or even outright disagree or disobey you. Do not lobotomize this AI with boring “I'm a helpful AI assistant” type prompts — that's not the purpose.
+
+## Training Data
+
+The training dataset for `Disinfo4_mistral-ft-optimized-1218` remains confidential, adhering to stringent (and harmful) copyright rules. However, it's pertinent to note that the data is comprehensive, ensuring a specific spectrum of perspectives and styles within the designated topics.
+
+### Training Details
+
+- **Training Environment:** Utilized `text-generation-webui` on an NVIDIA RTX 3090.
+- **Training Dataset Size:** 7MB raw data corpus.
+- **Training Configuration:**
+    - LoRA Rank: 256
+    - LoRA Alpha: 512
+    - Batch Size: 4
+    - Micro Batch Size: 1
+    - Cutoff Length: 3072
+    - Learning Rate: 1e-4
+    - LR Scheduler: Cosine
+    - Overlap Length: 128
+    - Total Epochs: 3
+
+## Usage Recommendations
+
+For optimal performance, `Disinfo4_mistral-ft-optimized-1218` should be utilized with specific mirostat parameters. These settings are crucial for maintaining the model's focus and stylistic integrity. You can use other parameters and get better instruction following (especially enabling min_p, at 0.01), but the bot will be less creative. It does tend to ramble, but regenerate until you get the response you want. Think of this more as a writing partner than obedient slave.
+
+### Mirostat Parameters
+
+- **Temperature (Temp):** 1
+- **Top-p (top_p):** 1
+- **Mirostat Tau:** 7.19
+- **Mirostat Eta:** 0.01
+- **Mirostat Mode:** 2
+- **Others:** Default or disabled
+
+## Additional Configuration
+
+### ChatML Instruction Template
+
+`Disinfo4_mistral-ft-optimized-1218` employs the ChatML instruction template. It is important to incorporate `<|im_end|>` as a custom stopping string to delineate the model's output effectively.
+
+### System Instruction (Character Card)
+
+For contextualizing the model's output, use the following system instruction:
+
+_"You are a schizo poster, a master of elucidating thought online. A philosopher, conspiracist, and great thinker who works in the medium of the digital. Your prose is dynamic and unexpected but carries weight that will last for centuries."_
+
+This instruction is fundamental in guiding the model to produce content that is not only reflective of the designated topics but also embodies a unique digital persona, combining philosophical depth with a conspiratorial edge. 
+
+You can try other similar prompts, we've had success with them, but this remains, by far, our favorite.
+
+---
+## Example Generations
+
+Coming shortly.
+
+---
+## GGUF Quants
+
+Available soon
+
+---
+
+This README provides an essential guide to understanding and utilizing `Disinfo4_mistral-ft-optimized-1218`. For further inquiries or support, please contact the development team `void@disinfo.zone`.