Table-GPT: Table-tuned GPT for Diverse Table Tasks
Authors: name: Peng Li; name: Yeye He; name: Dror Yashar; name: Weiwei Cui; name: Song Ge; name: Haidong Zhang; name: Danielle Rifinski Fainman; name: Dongmei Zhang; name: Surajit Chaudhuri
Published: 2023-10-13 17:20:56

Abstract: Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable
abilities to follow diverse human instructions and perform a wide range of
tasks. However, when probing language models using a range of basic
table-understanding tasks, we observe that today's language models are still
sub-optimal in many table-related tasks, likely because they are pre-trained
predominantly on one-dimensional natural-language texts, whereas
relational tables are two-dimensional objects.
  In this work, we propose a new "table-tuning" paradigm, where we
continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using
diverse table-tasks synthesized from real tables as training data, with the
goal of enhancing language models' ability to understand tables and perform
table tasks. We show that our resulting Table-GPT models demonstrate (1) better
table-understanding capabilities, by consistently outperforming the
vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout
unseen tasks, and (2) strong generalizability, in its ability to respond
to diverse human instructions to perform new table-tasks, in a manner similar
to GPT-3.5 and ChatGPT.