Hugging Face's logo

gghfez
/
Writer-Large-2411-v2.1-exl2-4.5bpw

Safetensors
mistral
exl2
Model card Files Community
Writer-Large-2411-v2.1-exl2-4.5bpw / README.md
gghfez's picture
gghfez
Update README.md
2606949 verified
preview code
|
raw
history blame contribute delete
3.23 kB
metadata
language:
  - en
  - fr
  - de
  - es
  - it
  - pt
  - zh
  - ja
  - ru
  - ko
license: other
license_name: mrl
license_link: https://mistral.ai/licenses/MRL-0.1.md

Writer-Large-2411-v2.1

EXL2-Quant of gghfez/Writer-Large-2411-v2.1

Creative-Writing Control-Vectors available here: gghfez/Writer-Large-2411-v2.1-control-vectors

Overview

This model is built on Mistral-Large-Instruct-2411 and optimized for creative writing purposes. The base model excels at following instructions and handling details in long context when using the new prompt template.

Key Improvements

  • Reduced positivity bias
  • Reduced AI tropes and repetitive language patterns in story generation
  • Enhanced performance with longer context stories (multiple chapters) and roleplay sessions
  • Improved steering capabilities for roleplay via [OOC] instructions
  • Better handling of "group chat" scenarios

Usage

Prompt Template

The model requires a system prompt in the Mistral-V7 format. If you omit [SYSTEM_PROMPT] [/SYSTEM_PROMPT], the model:

  • May not follow instructions properly at short contexts
  • Can become repetitive at longer contexts

Example:

[SYSTEM_PROMPT]You are an award winning writer. Assist the user.[/SYSTEM_PROMPT][INST] Write the opening chapter of ... [/INST]

SillyTavern Integration

Story String:

[SYSTEM_PROMPT] {{#if system}}{{system}}[/SYSTEM_PROMPT] [INST]
{{/if}}{{#if wiBefore}}{{wiBefore}}
{{/if}}{{#if description}}{{description}}
{{/if}}{{#if personality}}{{personality}}
{{/if}}{{#if scenario}}{{scenario}}
{{/if}}{{#if wiAfter}}{{wiAfter}}
{{/if}}{{#if persona}}{{persona}}
{{/if}}{{trim}}[/INST] Understood.</s>

For response steering, use [OOC] commands, e.g.:

  • [OOC] Have them interrupted by a loud explosion in a nearby factory
  • [OOC] Have her refuse to sell it and suggest another merchant instead

Technical Details

Training

  • QLoRA training at 32768 context
  • Merged with gghfez/Mistral-Large-Instruct-2411 at bf16
  • jukofyork/Creative writing control vectors were applied during synthetic dataset generation
  • Includes standard assistant instruct data for long-context stability
  • Note: Performance on code tasks may be reduced compared to base model
  • Note: No attempt was made to remove 'Name-Slop', so you'll still encounter Lily and Elara if you don't specify character names

Context Length

  • Base model: 131,072 tokens
  • Training range: 1024-32728 tokens
  • Training context window: 32768 tokens

Testing Environments

Tested with exllamav2 4.5bpw on: