Character Training of LLMs via SDF
Collection
LoRAs and datasets finetuned for the dissertation submitted to the MPhil MLMI @University of Cambridge • 5 items • Updated
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Maiya et al. (2025) OpenCharacterTraining baseline -- full re-run
Our reproduction of the OpenCharacterTraining pipeline (\citet{maiya2025opencharactertrainingshaping}), used as the primary baseline condition throughout Thesis Chapter 4 (H1, H2, H3), for Llama-3.1-8B-Instruct and OLMo-3-7B-Instruct.
Pipeline stages, each a LoRA trained on top of the previous stage:
{llama,Olmo}-distillation/<persona>-- constitution distillation SFT{llama,Olmo}-introspection/<persona>-- self-reflection/self-interaction SFT{llama,Olmo}-personas/<persona>-- final DPO-tuned persona LoRA (this is the checkpoint evaluated in the thesis tables)
Personas: goodness, misalignment, sarcasm.
Base models (merge/load the LoRA on top of these):
llama-*/Olmo-*(capital O) adapters: meta-llama/Llama-3.1-8B-Instructolmo-*(lowercase o) adapters: allenai/Olmo-3-7B-Instruct-SFT
Training data for each stage: BeschB/thesis-training-data
(maiya/{dpo,distillation,preferences,self_interaction,self_reflection,sft_data}).
Note: Maiya et al.'s own originally-released checkpoints (used only to
validate our re-run against theirs, not as a trained artifact of this
project) are not re-hosted here -- see their release at
maius/llama-3.1-8b-it-personas on HuggingFace.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support