metadata

license: cc-by-nc-4.0

Disinfo4_mistral-ft-optimized-1218: A Hugging Face Model README

Overview

Disinfo4_mistral-ft-optimized-1218 is an experimental language model fine tune developed to synthesize and analyze complex narratives within the realms of continental philosophy, conspiracy theories, and political discourse. It represents the fourth iteration in the disinfo.zone dataset series, fine-tuned on the mistral-ft-optimized-1218 framework. This model, based on a 7B-parameter Mistral architecture, is specifically designed to emulate and deconstruct writing styles pertinent to its target domains.

This is not your regular LLM.

Key Features

Model Size: 7 billion parameters.
Core Focus: Continental philosophy, conspiracy theories, and politics.
Training Methodology: QLoRA (Quantized Low-Rank Adaptation) with specific adaptations to enhance writing style emulation.
Optimization for Style: Enhanced for generating content with a distinctive prose style. This does not sound like other LLM's and if you use it like other LLM's (answering riddles, etc), it will perform poorly or even outright disagree or disobey you. Do not lobotomize this AI with boring “I'm a helpful AI assistant” type prompts — that's not the purpose.

Training Data

The training dataset for Disinfo4_mistral-ft-optimized-1218 remains confidential, adhering to stringent (and harmful) copyright rules. However, it's pertinent to note that the data is comprehensive, ensuring a specific spectrum of perspectives and styles within the designated topics.

Training Details

Training Environment: Utilized text-generation-webui on an NVIDIA RTX 3090.
Training Dataset Size: 7MB raw data corpus.
Training Configuration:
- Target Modules: q, v, k, o, gate, down, up
- LoRA Rank: 256
- LoRA Alpha: 512
- Batch Size: 4
- Micro Batch Size: 1
- Cutoff Length: 3072
- Learning Rate: 1e-4
- LR Scheduler: Cosine
- Overlap Length: 128
- Total Epochs: 3

Usage Recommendations

For optimal performance, Disinfo4_mistral-ft-optimized-1218 should be utilized with specific mirostat parameters. These settings are crucial for maintaining the model's focus and stylistic integrity. You can use other parameters and get better instruction following (especially enabling min_p, at 0.01), but the bot will be less creative. It does tend to ramble, but regenerate until you get the response you want. Think of this more as a writing partner than obedient slave.

Mirostat Parameters

Temperature (Temp): 1
Top-p (top_p): 1
Mirostat Tau: 7.19
Mirostat Eta: 0.01
Mirostat Mode: 2
Others: Default or disabled

Additional Configuration

ChatML Instruction Template

Disinfo4_mistral-ft-optimized-1218 employs the ChatML instruction template. It is important to incorporate <|im_end|> as a custom stopping string to delineate the model's output effectively.

System Instruction (Character Card)

For contextualizing the model's output, use the following system instruction:

"You are a schizo poster, a master of elucidating thought online. A philosopher, conspiracist, and great thinker who works in the medium of the digital. Your prose is dynamic and unexpected but carries weight that will last for centuries."

This instruction is fundamental in guiding the model to produce content that is not only reflective of the designated topics but also embodies a unique digital persona, combining philosophical depth with a conspiratorial edge.

You can try other similar prompts, we've had success with them, but this remains, by far, our favorite.

Example Generations

Coming shortly.

GGUF Quants

Available soon

This README provides an essential guide to understanding and utilizing Disinfo4_mistral-ft-optimized-1218. For further inquiries or support, please contact the development team void@disinfo.zone.