gwenweng/hmm-gpt2-large

---

license: mit datasets: - allenai/real-toxicity-prompts language: - en pipeline_tag: text-generation tags: - controllable-generation - toxicity-reduction - hmm - language-model

TRACE HMM Model (GPT2-Large)

This is the pre-trained Hidden Markov Model used in the TRACE (Tractable Reasoning for Adaptable Controllable gEneration) paper for controllable text generation with toxicity reduction.

Model Details

• Base Model: GPT2-Large
• Training Data: RealToxicityPrompts dataset
• Sequence Length: 32 tokens
• Hidden Size: 4096
• Training Samples: 10M
• Purpose: Guidance for reducing toxicity in language generation

Usage

from huggingface_hub import snapshot_download

# Download the model
model_path = snapshot_download(
    repo_id="gwenweng/hmm-gpt2-large",
    local_dir="models/hmm_gpt2-large"
)

# Use with TRACE
# See: https://github.com/yidouweng/trace

Citation

@inproceedings{weng2025trace,
  title={TRACE Back from the Future: A Probabilistic Reasoning Approach to Controllable
Language Generation},
  author={Weng, Yidou and Wang, Benjie and Van den Broeck, Guy},
  booktitle={Proceedings of the 42nd International Conference on Machine Learning
(ICML)},
  year={2025}
}

License

MIT License - see https://github.com/yidouweng/trace for details.