Personalized Safety Alignment for Text-to-Image Diffusion Models
Authors: name: Yu Lei, name: Jinbin Bai, name: Qingyu Shi, name: Aosong Feng, name: Kaidong Yu

Text-to-image diffusion models have revolutionized visual content generation,
but current safety mechanisms apply uniform standards that often fail to
account for individual user preferences. These models overlook the diverse
safety boundaries shaped by factors like age, mental health, and personal
beliefs. To address this, we propose Personalized Safety Alignment (PSA), a
framework that allows user-specific control over safety behaviors in generative
models. PSA integrates personalized user profiles into the diffusion process,
adjusting the model's behavior to match individual safety preferences while
preserving image quality. We introduce a new dataset, Sage, which captures
user-specific safety preferences and incorporates these profiles through a
cross-attention mechanism. Experiments show that PSA outperforms existing
methods in harmful content suppression and aligns generated content better with
user constraints, achieving higher Win Rate and Pass Rate scores. Our code,
data, and models are publicly available at
https://torpedo2648.github.io/PSAlign/.