ArcANE-32B-DPO

🏆 Accepted to EMNLP 2026 Main Conference

ArcANE-32B-DPO is a Qwen3-32B model trained with supervised fine-tuning followed by Direct Preference Optimization (DPO). It learns to distinguish the correct response at a target narrative phase from a plausible, in-character response drawn from an adjacent but incorrect phase.

Model details

Field	Value
Base model	Qwen/Qwen3-32B
Parameters	32B class
Training stage	SFT followed by DPO
Parameter update	LoRA, rank 64 and alpha 128
Recommended mode	Qwen3 non-thinking mode

Intended use

ArcANE-32B-DPO is intended for research on:

• point-in-time character role-play;
• character responses conditioned on a chapter-truncated Character Arc;
• distinguishing subtle behavioral changes between adjacent narrative phases;

The strongest evaluated setup supplies the relevant Character Arc only up to the queried chapter. Future phases must not be exposed to the model.

Training data

The SFT and DPO splits are derived from 12 training novels in the ArcANE corpus, covering 55 characters and 339 character axes. The SFT stage uses teacher-generated target responses under Arc context.

The DPO split contains 14,671 preference pairs from 2,516 unique probes. Each chosen response belongs to the anchor phase, while the rejected response belongs to an adjacent phase. Both responses share the same scenario and question.

The main validated evaluation slice is held out at the novel, character, arc, and probe levels from the training pool.

Training parameters

Stage	Epochs	Learning rate	Batch size	Maximum sequence length
SFT	1	1e-4	32	8,192 tokens
DPO	1	1e-5	64	8,192 tokens

Reproducibility

The released recipes are training/sft/configs/lora/sft.yaml and training/sft/configs/lora/dpo.yaml in the ArcANE repository. From training/sft, run bash scripts/train_sft_lora.sh followed by bash scripts/train_dpo_lora.sh.

Evaluation

The paper evaluates free-form role-playing responses on a held-out five-novel slice containing 25 principal characters, 205 arcs, and 1,754 probes. A separate DeepSeek-V4-Flash judge scores four 1 to 100 metrics:

• APF: Action Phase-Fidelity;
• RPF: Reasoning Phase-Fidelity;
• RAE: Reasoning-Action Entailment;
• PTF: Phase Trajectory Fidelity.

Scores are pooled within each novel, novels receive equal weight, and Overall is the mean of the 12 probe-category by metric cells. Higher is better.

Held-out results with Arc context

Probe category	APF	RPF	RAE	PTF
In-Scenario	59.4	58.0	50.9	53.4
In-World	66.2	64.2	58.8	58.5
Out-of-World	68.0	65.7	61.6	59.8

Comparison	Overall
ArcANE-32B-DPO, Arc context	60.4
ArcANE-32B-DPO, strongest non-Arc context	52.0
ArcANE-32B-SFT, Arc context	58.4
Qwen3-32B, Arc context	50.1
DeepSeek-V4-Pro, Arc context	62.4

DPO improves the Arc-context Overall score by 2.0 points over ArcANE-32B-SFT and by 10.3 points over Qwen3-32B under the same Arc context.

Usage

Use the Qwen3 chat template with thinking disabled. The example below is illustrative; replace the compact context with a valid chapter-truncated ArcANE Character Arc.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "holi-lab/ArcANE-32B-DPO"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            'You are <character>, from "<novel>". You are at the point in '
            "the story corresponding to chapter <query_chapter>.\n\n"
            "Background you have access to:\n"
            "<context>\n<chapter-truncated Character Arc JSON>\n</context>"
        ),
    },
    {
        "role": "user",
        "content": "Scenario:\n<scenario>\n\nQuestion:\n<question>",
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=1024, do_sample=True, temperature=1.0)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

For faithful point-in-time conditioning, remove all phases later than the query chapter. Always remove literary_validation and evidence_summary; if any later phase is hidden, also remove pole_end and arc_direction. The paper's evaluation drew one sample with backend-default sampling, effectively temperature 1.0, and capped generation at 8,192 tokens.

Citation

@misc{song2026arcaneroleplayinglanguageagents,
      title={ArcANE: Do Role-Playing Language Agents Stay in Character at the Right Time?}, 
      author={Woojung Song and Nalim Kim and Sangjun Song and Chaewon Heo and Jongwon Lim and Yohan Jo},
      year={2026},
      eprint={2606.05553},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.05553}, 
}