ObfusQAte: A Proposed Framework to Evaluate LLM Robustness on Obfuscated
  Factual Question Answering
Authors: name: Shubhra Ghosh; name: Abhilekh Borah; name: Aditya Kumar Guru; name: Kripabandhu Ghosh
Published: 2025-08-10 12:27:52

Abstract: The rapid proliferation of Large Language Models (LLMs) has significantly
contributed to the development of equitable AI systems capable of factual
question-answering (QA). However, no known study tests the LLMs' robustness
when presented with obfuscated versions of questions. To systematically
evaluate these limitations, we propose a novel technique, ObfusQAte and,
leveraging the same, introduce ObfusQA, a comprehensive, first of its kind,
framework with multi-tiered obfuscation levels designed to examine LLM
capabilities across three distinct dimensions: (i) Named-Entity Indirection,
(ii) Distractor Indirection, and (iii) Contextual Overload. By capturing these
fine-grained distinctions in language, ObfusQA provides a comprehensive
benchmark for evaluating LLM robustness and adaptability. Our study observes
that LLMs exhibit a tendency to fail or generate hallucinated responses when
confronted with these increasingly nuanced variations. To foster research in
this direction, we make ObfusQAte publicly available.