Generating Tables from the Parametric Knowledge of Language Models (2025)

Abstract: We explore generating factual tables from the parametric knowledge of large language models (LLMs). While LLMs have demonstrated impressive capabilities in recreating knowledge bases and generating free-form text, their ability to generate structured tabular data has received little attention. To address this gap, we explore the table generation abilities of eight state-of-the-art LLMs, including GPT-4o and Llama3.1-405B, using three prompting methods: full-table, row-by-row, and cell-by-cell. To facilitate evaluation we introduce WikiTabGen, a new benchmark consisting of 119 manually curated Wikipedia tables and their description. Our findings show that table generation remains challenging, with the best performing model (LLaMA3.1-405B) reaching only 25.4% accuracy. We further analyze how properties like table size, popularity, and numerical content impact performance. This study highlights the unique challenges of LLM-based table generation and offers a foundation for future research in this area. All code, data, and prompts are publicly available.

## introduction automated table generation has broad applications in fields such as healthcare, finance, scientific research and education @xcite @xcite where converting unstructured factual data into structured tables can significantly enhance decision-making, streamline workflows, and improve data accessibility enabling knowledge extraction and facilitating further analysis through statistical and visualization tools @xcite . large language models (llms) @xcite @xcite have demonstrated remarkable performance on various natural language processing tasks, including free-form text generation, knowledge retrieval, and 1 https://github.com/analysis-bots/wikitabgen table description "create a table with the detailed information about the achievements of susen tiedtke from 1987 to 2000" columns year, competition, venue, position (a) full table (b) row-by-row llm table generation methods (c) cell-by-cell venue birmingham tokyo … position 3rd 5th … year 1987 1991 … competition european junior world championship … output table figure 1: an example llm-based table generation task along with three alternative prompting methods. llms are pre-trained on vast amounts of text, which includes factual information presented in both plain text and structured formats, such as tables @xcite . through this training, llms encode a wealth of factual information in their parameters. while previous studies have shown that llms can retrieve factual information for tasks like recreating knowledge bases (kbs) @xcite @xcite or generating wikipedia-like articles @xcite , little attention has been given to their ability to generate structured tables from their parametric knowledge. unlike question answering over tables or text-to-sql translation @xcite , generating tables requires models to retrieve and organize specific factual data into structured formats, posing unique challenges. the lack of dedicated methods for table generation and appropriate evaluation benchmarks highlights a particular gap in current research. to address this gap, we introduce wikitabgen, a benchmark designed to evaluate llms' ability to generate tables from their parametric knowledge. it consists of 119 manually curated wikipedia tables, each paired with a textual description and a set of target columns. with an average of 1,457 tokens per table, wikitabgen features significantly larger tables compared to previous tabular generation tasks @xcite . this benchmark facilitates a systematic evaluation of how factors such as table size, numerical content, and popularity @xcite affect table generation. we also introduce and evaluate three prompting methods: full-table generation, row-by-row generation and cell-by-cell generation. our key contributions are: (1) formulating the problem of generating structured tables from llms' parametric knowledge. (2) introducing wikitabgen, a benchmark consisting of diverse tables that vary in size, structure, and content, to evaluate table generation capabilities. (3) implementing and evaluating three prompting methods across eight state-of-the-art llms, including gpt-4 and llama3.1-405b. (4) providing a comprehensive analysis of the factors that impact table generation performance. our experiments reveal that generating tables from llms remains a challenging task, with the highest f1 score reaching only 25.4%. we observe that factors such as table size and numerical content significantly affect performance. these findings highlight the need for further research to improve llm-based table generation. we hope that our benchmark and analysis will inspire future research on generating structured data from llms. ## problem definition given a short user description, our task is to generate a factually accurate table. following @xcite , a relational table @xmath0, c) is a set of rows @xmath1, r 2 , . . . } and a set of columns @xmath2, c 2 . . . }. a table cell, denoted r[c], contains the value of column c in row r. key columns are a subset c k ⊂ c that uniquely define each entry (row) in t and the corre-sponding cells do not contain null or empty values. for example, the table in fig. 1 has the columns year and competition as its keys. each table entry such as venue, corresponds to a unique year, competition pair. given a table description d and a list of desired table columns c, our task is to generate a corresponding table t ( r, c), where the generated rows r contain factually accurate information. an example problem is shown in fig. 1 , where the table description is "achievements of susen tiedke from 1987 to 2000" and the target columns are: year, competition, venue, and position. each of our proposed prompting methods ( §3) can then be used for the llm to generate table t ( r, c), as shown in the bottom of the figure. ## prompting llms to generate tables given a table description and list of target columns c, we evaluate llm performance on generating the corresponding table t ( r, c). our focus is on extracting the knowledge stored in the llm, with retrieval-augmented methods @xcite being orthogonal to our study. we implement three prompting methods to generate tables, shown in fig. 1 . first, the full table method prompts the llm to generate the table all at once. however, the output table may be quite large, with evaluation tables have 1.5k tokens on average ( §4). therefore, we also experiment with a modular prompting approach @xcite , where one llm instance generates the table keys, and another generates either complete rows or individual cells. we refer to these two modular prompting methods as row-by-row and cell-by-cell respectively. an in-depth example of our prompting methods is provided in fig. 2 . note that all prompts in the figure are appended with the table description and columns (prompt 1 in fig. 2 ). next, we describe each of the prompts used in our three methods. all of our prompts are listed in §a and in our public code repository. ## wikitabgen benchmark to evaluate our methods ( §3), we introduce a new table generation benchmark called wikitabgen. each instance of wikitabgen consists of a short manually written description d, a list of target columns c and a corresponding table @xmath3, c). ## experimental setting we describe our experimental setting for evaluating the table generation capabilities of llms. all models were evaluated on the wikitabgen benchmark. next, we list the llms, prompts and evaluation methods used for table generation. last, we detail our different experimental scenarios. ## results and analysis following, we summarize results obtained by the 8 llms, 3 prompting methods and two evaluation metrics. we then analyze the generation cost and accuracy trade-offs of the prompting methods. next, we discuss the table generation performance in our additional scenarios: example row and oracle keys, and finally, examine the effect of table properties on the llm generation performance. ## related work machine reasoning on table using pre-trained llms has largely been explored in the context of data augmentation @xcite to improve the performance on downstream tasks. the focus has largely been on tasks where a table is provided as input to the model namely: qa over tables @xcite @xcite ), text-to-sql translation @xcite , table editing @xcite and table-to-text generation @xcite . conversely, our approach receives only a user query and schema as input, and is tasked with generating an entire table. closest to ours are the recent table generation datasets by @xcite ; @xcite . in these works the llm is provided with a user query (in text or sql) and is tasked with generating a table, as the query answer. @xcite evaluate on tables from the spider dataset @xcite , which contains domainspecific information that is less likely to be stored in the parametric knowledge of llms. in @xcite the authors evaluate table generation from long-form text describing nba games, taken from the rotowire dataset @xcite . in their setting the generated table content is already present as part of the user query, where the llm challenge is to re-structure the user input as a table. by contrast, our setting requires the llm to generate information that does not explicitly appear in the user input query (fig. 1 ). similar to us, akhtar et al. (2024) rely on wikipedia however, they automatically construct new tables which are relatively small (average of 6.7 rows, 4 columns). by comparison our evaluation is on larger tables with the median number of rows being 48 (average of 77.5 rows, 6.9 columns). this emphasizes our focus on extracting long-form tabular data from llms, thereby extending past attempts on kbs and text @xcite @xcite . our key generation phase in §3 is an instance of a list question answering problem. the challenge of list qa in llms has been explored in recent works @xcite . however, we further expand this challenge by focusing on generating the entire table. ## conclusion this paper explores the capability of state-of-theart llms to generate entire tables, by relying exclusively on their parametric knowledge. we introduced three prompt-based table generation methods and evaluated them on our newly constructed benchmark, wikitabgen. our results underscore the challenge table generation poses to llms. we hope that wikitabgen and our comprehensive analysis will provide a concrete framework for future research on table generation using llms.