TableBench: A Comprehensive and Complex Benchmark for Table Question
  Answering
Authors: name: Xianjie Wu; name: Jian Yang; name: Linzheng Chai; name: Ge Zhang; name: Jiaheng Liu; name: Xinrun Du; name: Di Liang; name: Daixin Shu; name: Xianfu Cheng; name: Tianzhen Sun; name: Guanglin Niu; name: Tongliang Li; name: Zhoujun Li
Published: 2024-08-17 11:40:10

Abstract: Recent advancements in Large Language Models (LLMs) have markedly enhanced
the interpretation and processing of tabular data, introducing previously
unimaginable capabilities. Despite these achievements, LLMs still encounter
significant challenges when applied in industrial scenarios, particularly due
to the increased complexity of reasoning required with real-world tabular data,
underscoring a notable disparity between academic benchmarks and practical
applications. To address this discrepancy, we conduct a detailed investigation
into the application of tabular data in industrial scenarios and propose a
comprehensive and complex benchmark TableBench, including 18 fields within four
major categories of table question answering (TableQA) capabilities.
Furthermore, we introduce TableLLM, trained on our meticulously constructed
training set TableInstruct, achieving comparable performance with GPT-3.5.
Massive experiments conducted on TableBench indicate that both open-source and
proprietary LLMs still have significant room for improvement to meet real-world
demands, where the most advanced model, GPT-4, achieves only a modest score
compared to humans.