Title: GTAlign: Game-Theoretic Alignment of LLM Assistants for Mutual Welfare
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Human-Centered AI, Game Theory, Large Language Model, Reinforcement Learning

Abstract: Large Language Models (LLMs) have achieved remarkable progress in reasoning, yet often behave irrationally in tasks such as writing, information seeking, or providing practical guidance. Conventional alignment practices typically assume that maximizing model reward also maximizes user welfare, but this assumption frequently fails in practice: models may over-clarify or generate overly verbose reasoning when users prefer concise answers. Such behaviors resemble the prisoner’s dilemma, where individually rational choices lead to socially suboptimal outcomes. The fundamental challenge is that LLMs lack a principled mechanism for mutually beneficial decision making. We propose Game-Theoretic Alignment ($\textbf{GTAlign}$), an alignment framework that integrates game-theoretic decision making into both reasoning and training. During reasoning, the model explicitly treats user LLM interaction as a strategic game: it constructs payoff matrices within its reasoning chain to estimate welfare for both itself and the user, and then selects actions that are mutually beneficial. During training, we introduce a mutual welfare reward that reinforces cooperative responses, aligning model behavior with socially efficient outcomes. In addition, we introduce an inference technique that leverages game-theoretic reasoning to dynamically adapt LLM's response when payment modes of LLM service change. Extensive experiments demonstrate that GTAlign substantially improves reasoning efficiency, answer quality, and mutual welfare over baseline methods across diverse tasks.

TL;DR: We train the LLM assistant through reinforcement learning guided by game-theoretic principles.