Title: The Secret Agenda: LLMs Strategically Lie and Our Current Safety Tools Are Blind
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Mechanical interpretability, Explainable AI, SuperAlignment, Large Language Models

Abstract: We investigate strategic deception in large language models using two complementary testbeds: Secret Agenda (across 38 models) and Insider Trading compliance (via SAE architectures). Secret Agenda reliably induced lying when deception advantaged goal achievement across all model families. Analysis revealed that autolabeled SAE features for "deception" rarely activated during strategic dishonesty, and feature steering experiments across 100+ deception-related features failed to prevent lying. Conversely, insider trading analysis using unlabeled SAE activations separated deceptive versus compliant responses through discriminative patterns in heatmaps and t-SNE visualizations. These findings suggest autolabel-driven interpretability approaches fail to detect or control behavioral deception, while aggregate unlabeled activations provide population-level structure for risk assessment. Results span Llama 8B/70B SAE implementations and GemmaScope under resource constraints, representing preliminary findings that motivate larger studies on feature discovery, labeling methodology, and causal interventions in realistic deception contexts.

TL;DR: Current autolabeled SAE interpretability tools fail to detect strategic deception in competitive AI scenarios despite successfully identifying ethical compliance patterns, revealing critical blind spots in AI safety mechanisms.