Debiasing Large Language Models in T hai Political Stance Detection via Counterfactual Calibration (2025)

Abstract: Political stance detection in low-resource and culturally complex settings poses a critical challenge for large language models (LLMs). In the Thai political landscape—rich with indirect expressions, polarized figures, and sentiment-stance entanglement—LLMs often exhibit systematic biases, including sentiment leakage and entity favoritism. These biases not only compromise model fairness but also degrade predictive reliability in real-world applications. We introduce ThaiFACTUAL, a lightweight, model-agnostic calibration framework that mitigates political bias without fine-tuning LLMs. ThaiFACTUAL combines counterfactual data augmentation with rationale-based supervision to disentangle sentiment from stance and neutralize political preferences. We curate and release the first high-quality Thai political stance dataset with stance, sentiment, rationale, and bias markers across diverse political entities and events. Our results show that ThaiFACTUAL substantially reduces spurious correlations, improves zero-shot generalization, and enhances fairness across multiple LLMs. This work underscores the need for culturally grounded bias mitigation and offers a scalable blueprint for debiasing LLMs in politically sensitive, underrepresented languages.

## introduction stance detection, the task of identifying an author's attitude toward a given topic or target, has gained increasing attention in computational social science and political nlp @xcite . in southeast asia, and thailand in particular, political discourse is often coded, indirect, or emotionally charged, making the task especially challenging. as user-generated content surges on platforms like twitter, facebook, and pantip, stance detection becomes a valuable tool for understanding public opinion on contested issues, such as constitutional reform, monarchyrelated debates, or election campaigns @xcite . with the rise of llms-e.g., chatgpt foot_0 , gemini foot_1 , and llama @xcite -stance detection capabilities have advanced, yet their deployment in politically sensitive domains remains problematic. these models are trained on largescale internet corpora, which often encode cultural, regional, or ideological biases. in the case of thai political content, this leads to unreliable predictions, particularly when sentiment is used as a proxy for stance, or when certain figures are consistently associated with positive or negative views. our study identifies two dominant forms of bias in llms applied to thai political stance detection: we further demonstrate a significant inverse correlation between the level of bias and model accuracy, showing that reducing bias improves performance. previous work in bias mitigation has focused on training data balancing or re-weighting @xcite , or adversarial debiasing, but such methods either require access to model parameters or risk degrading generalization (a) sentiment leakage. same sentiment results in same stance across entities. to overcome these limitations, we propose factual-thai-a plug-and-play debiasing method using a counterfactual augmented calibration module. instead of altering the base llm, we construct auxiliary calibration models that learn to adjust the output stance label using context-aware rationales and counterfactual variants of the input. by introducing counterfactual perturbations to both causal (topic-related) and non-causal (sentiment or named entities) dimensions, we enable the calibration model to better disentangle spurious from reliable cues. unlike prior work that primarily focuses on english or high-resource settings, we situate our study in thai political discourse, where cultural nuances, code-switching, and sociopolitical sensitivities amplify the challenges of bias mitigation and demand methods that generalize under resource scarcity. however, @xcite demonstrated that llms exhibit positional bias in multiple-choice settings, which cannot be addressed by traditional retraining strategies. in our work, we extend this analysis to thai political stance detection, a domain marked by sharp polarization and sentiment-driven discourse. ## limitations while our proposed thaifactual framework significantly improves fairness and robustness in thai political stance detection, several limitations remain: our study faces several limitations: counterfactual augmentation is currently restricted to entity substitutions and does not yet capture broader political events or abstract ideologies, with automated generation still an open challenge; thaifactual operates in a post-hoc black-box setting, limiting deeper integration of counterfactual signals; subtle cultural priors (e.g., historical associations between political figures) may still leak into model behavior; the dataset, though carefully curated, remains small and limited to three entities, reducing generalizability as political discourse evolves; and finally, our evaluation centers on sentiment-stance disentanglement and target fairness, leaving other bias dimensions such as dialect, user ideology, and media framing for future exploration. finally, while our study focuses on fairness improvements at the stance level, we do not explicitly measure downstream impacts on tasks such as political event forecasting, misinformation detection, or ideological clustering. future research should examine how debiased stance predictions propagate into these broader applications. to evaluate and calibrate llms for thai political stance detection, we constructed a novel dataset of thai-language tweets covering high-profile political figures, curated with attention to topic balance, linguistic diversity, and sentiment/stance disambiguation.