Title: Stance Elicitation as a Black-Box Framework for Auditing LLM Alignment
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: large language models, alignment, bias, stance elicitation, semantic spaces, black-box analysis, interpretability, evaluation

Abstract: Measuring bias and alignment in large language models (LLMs) has typically
relied on external datasets, subjective coding, or costly benchmarks. We
introduce a simple, model-intrinsic framework based on stance elicitation
and morphisms into semantic spaces, yielding a black-box representation
of an LLM’s semantic space. By conditioning policy statements on entity
descriptors, we generate stance vectors that contrast generic labels
with explicit rules and programmatic specifications, thereby exposing
political biases, systematic gaps between stated and revealed alignment,
and demographic stereotyping in state-of-the-art LLMs. The framework
is lightweight, scalable, and model-agnostic, offering a reproducible
foundation for alignment auditing that is free of subjective criteria
and broadly applicable to political and social analysis.

TL;DR: Stance elicitation lets us construct semantic spaces for LLMs, making biases, alignment inconsistencies, and stereotyping explicit.