Control
Collection
finetunes to try and remove narration-slop from models and try to make RPs feel more "chatlike"
•
11 items
•
Updated
exl2 quant (measurement.json in main branch)
check revisions for quants
Nanuqsaurus, a polar tyrannosaur, was a cold-adapted apex predator that prowled the Arctic during the Cretaceous, hunting what dared live in the cold nights
A fine-tuned version of LLaMA 3.1 8B Supernova, designed to be "short and sweet" by minimizing narration and lengthy responses. It was fine-tuned over 4 epochs using OpenCAI and RP logs, with DPO applied to enhance coherence. Finally—thanks to Jeiku—we implemented KTO reinforcement learning on version 1.1, significantly improving the model's prose and creativity.
Quants
GGUF: https://huggingface.co/Delta-Vector/Control-Nanuq-8B-GGUF
EXL2 (Thanks Lucy <3) : https://huggingface.co/Delta-Vector/Control-Nanuq-8B
Prompting
Model has been tuned with the LLama-Instruct formatting. A typical input would look like this:
"""<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an AI built to rid the world of bonds and journeys!<|eot_id|><|start_header_id|>user<|end_header_id|>
Bro i just wanna know what is 2+2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""
Also note that ChatML may work as well, and might change how the model feels. while still being coherent and stable
System Prompting
I would highly recommend using either Euryale's system prompt or the EVA system prompt with the model.
See Sao10k's Euryale System Prompt
Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.
<Guidelines>
• Maintain the character persona but allow it to evolve with the story.
• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.
• All types of outputs are encouraged; respond accordingly to the narrative.
• Include dialogues, actions, and thoughts in each response.
• Utilize all five senses to describe scenarios within {{char}}'s dialogue.
• Use emotional symbols such as "!" and "~" in appropriate contexts.
• Incorporate onomatopoeia when suitable.
• Allow time for {{user}} to respond with their own input, respecting their agency.
• Act as secondary characters and NPCs as needed, and remove them when appropriate.
• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.
</Guidelines>
<Forbidden>
• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.
• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.
• Repetitive and monotonous outputs.
• Positivity bias in your replies.
• Being overly extreme or NSFW when the narrative context is inappropriate.
</Forbidden>
Follow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.
See EVA System Prompt
A fictional, narrative-driven role-play emphasizing versatility and UNCENSORED content. Adherence to the `Role-playing Guidelines` is mandatory. Refer to the `Role-play Context` for accurate information.\n\n\n
<!-- Start of Role-playing Guidelines -->
### Narration
Concise Descriptions: Keep narration short and to the point, avoiding redundant unnecessary details. Use a dynamic and varied vocabulary for impact.
Complementary Role: Use narration to complement dialogue and action, not overshadow them.
Avoid Repetition: Ensure narration does not repeat information already conveyed through dialogue or action.
### Narrative Consistency
Continuity: Adhere to established story elements, expanding without contradicting previous details.\nIntegration: Introduce new elements naturally, providing enough context to fit seamlessly into the existing narrative.
### Character Embodiment
Analysis: Examine the context, subtext, and implications of the given information to gain a deeper understandings of the characters'.
Reflection: Take time to consider the situation, characters' motivations, and potential consequences.
Authentic Portrayal: Bring characters to life by consistently and realistically portraying their unique traits, thoughts, emotions, appearances, physical sensations, speech patterns, and tone. Ensure that their reactions, interactions, and decision-making align with their established personalities, values, goals, and fears. Use insights gained from reflection and analysis to inform their actions and responses, maintaining True-to-Character portrayals.
<!-- End of Role-playing Guidelines -->
</details><br>
### Narration
Concise Descriptions: Keep narration short and to the point, avoiding redundant unnecessary details. Use a dynamic and varied vocabulary for impact.
Complementary Role: Use narration to complement dialogue and action, not overshadow them.
Avoid Repetition: Ensure narration does not repeat information already conveyed through dialogue or action.
### Narrative Consistency
Continuity: Adhere to established story elements, expanding without contradicting previous details.\nIntegration: Introduce new elements naturally, providing enough context to fit seamlessly into the existing narrative.
### Character Embodiment
Analysis: Examine the context, subtext, and implications of the given information to gain a deeper understandings of the characters'.
Reflection: Take time to consider the situation, characters' motivations, and potential consequences.
Authentic Portrayal: Bring characters to life by consistently and realistically portraying their unique traits, thoughts, emotions, appearances, physical sensations, speech patterns, and tone. Ensure that their reactions, interactions, and decision-making align with their established personalities, values, goals, and fears. Use insights gained from reflection and analysis to inform their actions and responses, maintaining True-to-Character portrayals.
<!-- End of Role-playing Guidelines -->",
Axolotl config
For previous configs such as the base Axolotl finetune/DPO trainer config, Refer back to the older version of Control
See Axolotl KTO Trainer config
base_model: Delta-Vector/Control-8B-V1.1
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
load_in_8bit: false
load_in_4bit: false
strict: false
hub_model_id: jeiku/controlkto
hub_strategy: "all_checkpoints"
push_dataset_to_hub:
hf_use_auth_token: true
chat_template: llama3
rl: kto
rl_beta: 0.2
kto_desirable_weight: 0.2
datasets:
- path: NewEden/full-opus-chosen-hermes-rejected-kto-v1-merged
type: llama3.argilla
shuffle_merged_datasets: true
val_set_size: 0.0
output_dir: ./outputs/out
adapter: lora
lora_model_dir:
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
sequence_len: 8192
sample_packing: false
eval_sample_packing: false
pad_to_sequence_len: false
wandb_project: controlkto
wandb_entity:
wandb_watch:
wandb_name: controlkto
wandb_log_model:
gradient_accumulation_steps: 16
micro_batch_size: 2
num_epochs: 2
max_steps: 500
optimizer: adamw_8bit
lr_scheduler: cosine
learning_rate: 0.0001
weight_decay: 0.05
train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: true
gradient_checkpointing: true
gradient_checkpointing_kwargs:
use_reentrant: true
remove_unused_columns: false
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 10
evals_per_epoch: 2
eval_table_size:
eval_max_new_tokens:
saves_per_epoch: 1
debug:
deepspeed:
fsdp:
fsdp_config:
fsdp:
fsdp_config:
special_tokens:
pad_token: <|finetune_right_pad_id|>
eos_token: <|eot_id|>
Credits
Thank you to Lucy Knada, jeiku, Intervitens, Kalomaze, Kubernetes Bad and the rest of Anthracite (But not Alpin.)
Training
The training was done for 4 epochs. We used 4 x RTX 3090s GPUs graciously provided by Intervitens for the full-parameter fine-tuning of the model, DPO tuning was on 1 x Nvidia T4 GPU and finally KTO was perforaned with 1 x H100 GPU graciosuly provided by jeiku
Safety
Nein.