N ame G uess: Column Name Expansion for Tabular Data (2023)

Abstract: Recent advances in large language models have revolutionized many sectors, including the database industry. One common challenge when dealing with large volumes of tabular data is the pervasive use of abbreviated column names, which can negatively impact performance on various data search, access, and understanding tasks. To address this issue, we introduce a new task, called NameGuess, to expand column names (used in database schema) as a natural language generation problem. We create a training dataset of 384K abbreviated-expanded column pairs using a new data fabrication method and a human-annotated evaluation benchmark that includes 9.2K examples from real-world tables. To tackle the complexities associated with polysemy and ambiguity in NameGuess, we enhance auto-regressive language models by conditioning on table content and column header names – yielding a fine-tuned model (with 2.7B parameters) that matches human performance. Furthermore, we conduct a comprehensive analysis (on multiple LLMs) to validate the effectiveness of table content in NameGuess and identify promising future opportunities. Code has been made available at https://github.com/amazon-science/nameguess.

## introduction tabular data is widely used for storing and organizing information in web @xcite and enterprise applications @xcite . one common practice when creating tables in databases is to use abbreviations for column headers due to character length limits in many standard database systems. for example, the maximum length for column names in an sql database is 256 bytes, leading to the use of abbreviations such as "d_id" for "department id" and "e_name" for "employee figure 1 : an example of the column name expansion task. the input are query column names with table context and the output are expanded logical names. name" as in figure 1 . while abbreviations can be convenient for representation and use in code, they can cause confusion, especially for those unfamiliar with the particular tables or subject matter. column headers are essential for many table-related tasks @xcite , and using abbreviations makes it challenging for end users to search and retrieve relevant data for their tasks. abbreviated column names can negatively impact the usefulness of the underlying data. for example, in the text2sql semantic parsing task, which converts natural language into formal programs or queries for retrieval, abbreviations can lead to a mismatch with the terms used in the natural language queries. in fact, in the human-labeled text2sql spider dataset @xcite , 6.6% of column names are abbreviations. figure 2 shows an example containing abbreviated column names like "c_name" and "acc_bal" in tables, which mismatch the terms "the name of all customers" and "account balance". simple changes in using abbreviated column names in the spider dataset result in a performance degradation of over ten percentage points, with the exact match score of 66.63% @xcite dropping to 56.09% on the t5-large model @xcite . the effect of the abbreviated column names on table question answering (qa) @xcite , and column relation disthe terms "the name of all customers" and "account balance" do not match the abbreviated column names c_name and acc_bal. instead, they match with the column names customer_id and account_type. expanding column names and generating descriptive headers also has other beneficial aspects. first, using expanded column names can increase the readability of tables, especially when complex or technical data is present. the expansion also enables data integration by allowing users to easily distinguish between tables with similar column names but different meanings and helping identify relationships between tables with different abbreviated column names. finally, expanded column names can also improve the efficacy of keywordbased searches for discovering related tables. this work addresses the task of expanding abbreviated column names in tabular data. to the best of our knowledge, this is the first work to introduce and tackle this problem. unlike previous textual abbreviation expansion works that formulated the task as a classification problem with a predefined set of candidate expansions @xcite , we formulate nameguess as a natural language generation problem. acquiring extensive candidate expansions can be laborious, as pairs of abbreviated and expanded column names are seldom present in the same table. conversely, abbreviation-expansion pairs can be gleaned from textual data through cooccurrence signals, such as parenthetical expressions. moreover, abbreviated headers may exhibit ambiguity and polysemy arising from developerspecific naming conventions and domain-related variations in expansions. to tackle nameguess, we first built a large dataset consisting of 163,474 tables with 384,333 column pairs and a human-annotated benchmark with 9,218 column pairs on 895 tables. we then proposed a method to produce training data by selectively abbreviating well-curated column names from web tables using abbreviation look-ups and probabilistic rules. next, we enhanced autoregressive language models with supervised finetuning, conditioned on table content and column headers, and conducted extensive experiments to evaluate state-of-the-art llms. the overall model performance is shown in figure 3 . while gpt-4 exhibited promising performance on nameguess, the deployment of such llms comes with much larger memory and computation overheads. our findings indicate that supervised fine-tuning of smaller 2.7b parameter models achieves close to human performance, and including table contents consistently improved performance. however, all models found the task challenging, with the best only achieving 54.7% accuracy on the extra-hard examples, indicating room for improvement in expanding abbreviated column names. our main contributions are: ## problem formulation we formulate the nameguess task as a natural language generation problem: given a query name x from table t, generate a logical name y that describes the column. a table contains table content and various schema data, like a table name, column headers, and data types. let f θ be a generator with parameters θ, the formulation becomes @xmath0. note that the query column names within a table may take different forms, including abbreviations and full names. the output logical names are expanded column names and should be easily understandable without additional knowledge of the table content. see figure 1 for example inputs and outputs in the "employee_salary_2022" table, where "sal" stands for "salary" and "comm" stands for "commission". ## dataset creation we created a training dataset comprising 384,333 columns spanning 163,474 tables and a humanlabeled evaluation benchmark containing 9,218 examples across 895 tables for nameguess. the main challenge in obtaining abbreviated-expanded column name pairs is the rare co-occurrence of these two names within the same table or database. therefore, we employed the strategies of converting well-curated column names to abbreviated names that align with the naming convention of database developers and annotating the abbreviated column names based on the information in the input table. figure 4 illustrates the main steps for creating the training and evaluation datasets. the details of the source tables are discussed in section 3.1. the training and evaluation datasets are constructed in section 3.2 and section 3.3. ## methods recent advances in pre-trained lms @xcite have shown a strong ability to generate fluent text and the "emergent" performance boost when scaling up lms @xcite . therefore, we evaluated the performance of both small and large lms for nameguess. we adopted two learning paradigms. first, we employed prompt-based learning with llms without tuning model parameters. second, we fine-tuned small lms. in particular, we utilized the supervised fine-tuning @xcite paradigm with task-specific prompts. training. we fine-tuned pre-trained lms by contextualizing column query names with table content, incorporating sampled cell values and table schema data. to limit the sequence length of a linearized table, we selected n cell values in the corresponding column (after removing duplicates) for each query name. we truncated cell values when they had more than 20 characters. moreover, instead of predicting individual query name separately, we jointly predicted k query names. the columns are stacked together to form the table context data t ′ . this structured input is then serialized and combined with a task prompt q. specifically, here k represents the number of query names, and n is the number of sampled rows in a table. {x i |@xmath1, ..., k} refers to the abbreviated column names from the same table. {y i |@xmath2, ..., k} represents the ground-truth logical name. and {c j i |@xmath3, ..., k; @xmath4, ...n } denotes the sampled cell values of each query name. we set k and n to 10 based on the ablation study results after testing different k and n values in appendix a.6 prediction. given k column names x 1 , ..., x k in a table t, we used lm to predict the corresponding logical names y 1 , ..., y k . to generate predictions, we concatenated the linearized table context t ′ and the task prompt q as the input sequence. we used the same query prompt as in training during inference, except that we removed the ground truth. the modified query prompt becomes "as abbreviations of column names from a table, {x 1 }|...|{x k } stand for". for the non-finetuned llms, we provided a single demonstration before the query prompt to ensure that the model can generate answers in a desired format, i.e., "as abbreviations of column names from a table, c_name | pcd | dt stand for customer name | product code | date." we extracted the answer in the predicted sequence when @xmath5. ## experiments we performed a comprehensive set of experiments to answer (1) can nameguess be solved as a natural language generation task? (2) how does fine-tuning and scaling the number of parameters help the model handle nameguess? (3) can table contents aid disambiguation? ## conclusion and future works we introduced a new task related to expanding the commonly-used abbreviated column names in tabular data, developed and created two benchmark datasets to facilitate the study of this task, and analyzed the performance of many language modeling methods to solve this task. one future direction is to utilize similar examples that provide contextual information to solve this task, preferably feeding these examples through in-context learning.