TABARD : A Novel Benchmark for Tabular Anomaly Analysis, Reasoning and Detection (2025)

Abstract: We study the capabilities of large language models (LLMs) in detecting fine-grained anomalies in tabular data. Specifically, we examine: (1) how well LLMs can identify diverse anomaly types including factual, logical, temporal, and value-based errors; (2) the impact of prompt design and prompting strategies; and (3) the effect of table structure and anomaly type on detection accuracy. To this end, we introduce TABARD, a new benchmark constructed by perturbing tables from WikiTQ, FeTaQA, Spider, and BEAVER. The dataset spans multiple domains and eight anomaly categories, including paired clean and corrupted tables. We evaluate LLMs using direct, indirect, and Chain-of-Thought (CoT) prompting. Our results reveal notable limitations in standard prompting, especially for complex reasoning tasks and longer tables. To overcome these issues, we propose a unified framework combining multi-step prompting, self-verification, and constraint-based rule execution. Our approach significantly improves precision and recall, offering a promising direction for robust and interpretable anomaly detection in tables.

## introduction tables are a core data format across domains such as finance, healthcare, scientific research, and government reporting. they have gained increasing attention in machine learning (ml), natural language processing (nlp), and broader ai research, supporting tasks like question answering, table-to-text generation, schema understanding, and data integration. however, real-world tables is often noisy, incomplete, or inconsistent (figure 2 ), issues that can severely impact model performance as llms and analytics systems rely on such data. even subtle anomalies can cascade into downstream errors, affecting outputs and decisions. tables are especially prone to a variety of anomalies, including * * these authors contributed equally to this work. incorrect values, logical inconsistencies, temporal misalignments, arithmetic errors, and security flaws, all of which compromise data reliability. as llms increasingly move to reasoning over tabular data (e.g., tableqa, fact verification), their reliability depends on the ability to detect subtle inconsistencies. however, anomaly detection in tables remains underexplored compared to text and vision, with limited benchmarks, tools, and systematic evaluation. prior work has largely focused on unstructured or semi-structured data, with minimal attention to relational tables @xcite . the absence of standardized benchmarks has contributed to the limited exploration of tabular anomaly detection, emphasizing the need for a systematic framework to evaluate model performance across diverse anomaly types. to bridge this gap, we introduce tabard, a comprehensive benchmark for evaluating anomaly detection in tabular data. tabard is constructed by aggregating and perturbing tables from widelyused sources such as wikitq @xcite , fetaqa @xcite , spider @xcite , and beaver @xcite . tabard features tables with systematically injected anomalies, including value, factual, logical, temporal, arithmetic, security, normalization, and consistency violations, while preserving clean counterparts to support supervised evaluation. we evaluate large language models (llms) using a four-tier prompting framework-ranging from zero-shot to few-shot settings, with and without chain-of-thought (cot) reasoning. this comprehensive analysis exposes key limitations in the models' ability to detect and explain anomalies in tabular data. to address these challenges, we propose three novel methods: (1) a multi-reasoning self-verification strategy that uses cot to iteratively refine outputs; (2) a cot-based approach enhanced with recursive attention prompting to improve contextual coherence and consistency; and (3) a neuro-figure 1 : a comprehensive table containing instances of all anomaly types. each arrow highlights an anomalous cell, annotated with the anomaly category and a brief explanation of its cause. the dataset, along with associated scripts, and other information are available at https:// coral-lab-asu.github.io/tabard/ . ## table anomaly detection given a table @xmath0,j } with m rows and n columns, where each cell t i,j can take values from a mixed-type domain d (e.g., text, number, date), the goal is to learn a binary function f (t i,j ) ∈ {0, 1} indicating whether the cell is anomalous. for each cell t i,j , the model predicts a binary label y i,@xmath1, 1}, where: this can be framed as a binary classification task over each cell t i,j , where the model learns a function: ## our tabard dataset we introduce tabard, a benchmark for anomaly detection, analysis, and reasoning in tabular data, specifically designed to evaluate the capabilities of large language models (llms). tabard is constructed by selecting and cleaning tables from four widely-used datasets: wikitq @xcite , fetaqa @xcite , spider @xcite , and beaver @xcite . the benchmark covers eight distinct types of anomalies: as illustrated in figure 2 , the benchmark includes diverse examples: a duplicate order id in the second row reflects a data consistency anomaly; the year 1600 in the date (order/ship) column is temporally implausible; an order date occurring after the ship date indicates a logical anomaly. a negative quantity in the qty column represents a value anomaly, while a miscalculated total in the first row exemplifies a calculation-based anomaly. exposure of unencrypted card details denotes a security anomaly, and assigning "atlantis" as the locale illustrates a factual anomaly. tabard generation tabard is constructed by perturbing original tables from multiple sources using large language models (llms) to introduce diverse types of anomalies as defined earlier. we use numeric and mixed-type tables from public datasets such as wikitq, spider, beaver, and fetaqa as our table sources. tables are filtered based on column-level metadata-such as data type histograms, keyword matches, and composite-key heuristics-to guide downstream anomaly generation. we retain only those tables that contain at least a few numeric columns and have no missing cells. each selected table, along with its original schema, is passed to an llm using specialized prompts designed to induce different anomaly types. we employ eight distinct perturbation prompts, each corresponding to a specific anomaly category. to generate ground truth labels, we track all cells modified by the llm and label them as anomalous. additionally, we store metadata including the anomaly type (i.e., the prompt used) and the llm's explanation for the perturbation, which aids in subsequent data verification. since tabard preserves the original schema of the tables, all sql and analytic queries valid on the clean version remain executable on its perturbed counterpart. this ensures structural robustness, with anomalies that survive downstream processes such as type casting and serialization. tabard verification. we perform human verification to assess the validity of anomalies generated by the llm (gpt-4o). a random 15% subset of the perturbed tables is sampled for manual review. ## modeling approaches we introduce three categories of methods as described in figure 2 (problem modeling approaches) with increasing complexity. the first category consists of direct and indirect prompts, each organized into four levels, with and without chainof-thought reasoning. then we have a multireasoning self-verification method that leverages cot, a self-verifying cot method augmented with recursive attention prompting technique, and a neuro symbolic constraint based method. ## experimental evaluation through our experiments, we aim to investigate the following research questions: (a) how challenging is our benchmark dataset, tabard, for current state-of-the-art models? (b) to what extent do various prompting strategies-such as few-shot learning, chain-of-thought (cot), and self verification-improve anomaly detection performance? (c) how do different language models perform on the specific task of fine-grained anomaly detection in tables? evaluation: our primary task is binary prediction, so we evaluate model performance using precision (p) and recall (r). f1 scores are also computed and reported in the appendix for completeness. all evaluations assume the current month and year to be may 2025. models. we evaluate four advanced language models: chatgpt-4o, gemini 1.5 pro, llama 3.1 70b instruct, and deepseek-v3. our constraintbased neuro-symbolic method is executed on chatgpt-4o and gemini 1.5 pro. due to limitations in external tool access, we were unable to deploy the constraint-based method on llama and deepseek. ## related works to the best of our knowledge, this work is the first to investigate fine-grained, cell-level anomaly detection in tabular data using llms, extending prior anomaly detection efforts beyond unstructured and coarse-grained settings. in the visual domain, vision-language models have improved anomaly detection in images and videos @xcite @xcite . in multimodal settings, these models have also been employed to detect fake news by integrating textual and visual information @xcite . in the domain of log data, llms have been fine-tuned to identify anomalies in system logs, effectively handling both structured and unstructured formats @xcite @xcite . within tabular data, @xcite demonstrated the potential of llms as zero-shot batch-level anomaly detectors and proposed synthetic data generation and fine-tuning strategies to enhance performance, achieving results on par with stateof-the-art methods. anollm @xcite introduced a framework for unsupervised tabular anomaly detection that operates directly on raw textual features and showed competitive performance on several datasets. despite recent progress, fine-grained cell-level anomaly detection in tables remains underexplored. to address this, we introduce tabard, a benchmark spanning eight anomaly types, and evaluate llms across diverse prompting strategies. we also propose three detection methods-including two prompting-based and one constraint-based approach-that outperform baselines and improve explainability in tabular anomaly detection. ## conclusion and future work this work introduces tabard, a comprehensive benchmark and evaluation framework for fine-grained anomaly detection in tabular data using large language models (llms). by per-turbing real-world tables with diverse anomaly types and evaluating multiple prompting strategies-including multi-reasoning, self-verification, and a novel neuro-symbolic constraint-based method-the study reveals fundamental strengths and limitations of current llms. while traditional prompting with cot enhances recall, especially on simpler datasets, our advanced methods like mu-seve, sevcot, and nscm achieve substantial improvements in precision, recall, and explainability across more complex settings. tabard establishes a principled foundation for future research on robust, interpretable, and scalable anomaly detection systems in structured data. we identify several promising directions for future research. we aim to detect mixed and taskdriven anomalies, address incomplete or missing table data, and extend anomaly detection to multimodal tables with visual elements. additionally, we plan to strengthen our constraint-based framework by incorporating feedback-guided refinement and meta-level modeling for more reliable constraint generation. we provide a detailed discussion of these directions in the appendix section-b.