Automatic Table Union Search with Tabular Representation Learning (2023)

Abstract: Given a data lake of tabular data as well as a query table, how can we retrieve all the tables in the data lake that can be unioned with the query table? Table union search constitutes an essential task in data discovery and preparation as it enables data scientists to navigate massive open data repositories. Existing methods identify uniability based on column representations (word surface forms or token embeddings) and column relation represented by column representation similarity. However, the semantic similarity obtained between column representations is often insufficient to reveal latent relational features to describe the column relation between pair of columns and not robust to the table noise. To address these issues, in this paper, we propose a multi-stage self-supervised table union search framework called AutoTUS, which represents column relation as a vector– column relational representation and learn column relational representation in a multi-stage manner that can better describe column relation for unionability prediction. In particular, the large language model powered contextualized column relation encoder is updated by adaptive clustering and pseudo label classification iteratively so that the better column relational representation can be learned. Moreover, to improve the robustness of the model against table noises, we propose table noise generator to add table noise to the training table data. Experiments on real-world datasets as well as synthetic test set augmented with table noise show that AutoTUS achieves 5.2% performance gain over the SOTA baseline.

## introduction the growing availability of tabular data from academia and industry brings new opportunities for economic growth and social benefit, which has also attracted considerable attentions in the natural language processing (nlp) community. a number of work has been done, such as table qa @xcite , table fact verification @xcite , table summarization @xcite , table understanding @xcite @xcite , etc. more recently, table union search @xcite @xcite is proposed to facilitate tabular dataset discovery applications @xcite @xcite , which can benefit the tabular data integration and analysis. the table union search aims to find all tables in a data lake that have the columns from the same domain as the query table. with the help of the table union search, the understanding of the tabular data is largely improved and it can potentially benefit other tabular-focused nlp downstream tasks. therefore, table union search is a non-trival nlp research problem. recent literature shows that column representation methods are useful for table union search. the basic idea is to map the column to a latent vector space (e.g. column representations @xcite or token embeddings @xcite @xcite ) and then compute the table unionability score based on the column relation obtained from the similarity scores of col-umn pairs between query table and target tables. however, these methods only consider the column representation and their similarity as the column relation to compute the table unionability score, which is not enough to describe the relationship between two columns and may cause incorrect prediction. figure 1 shows an example of the table union search with a query table and a unionable table. by looking at the second columns (in red) of two tables, the computed column similarity score (a scalar) from column embedding is low due to the dissimilar token embeddings. in this case, the column relation is not intuitive, taking only the similarity between column embeddings to describe the column relation is not enough to predict the table unionability accurately. however, if we combine each column representation and model them as an embedding (a vector), the new representation (column relational representation) is more informative and can better describe the unintuitive column relation to make the table unionability prediction more accurately. therefore, a better approach is needed, which can make the table unionability prediction via the column relational representation. on the other hand, real-world table noises widely exist @xcite . taking column "clg" in the query table and column "pos" in the data lake table as examples, both columns have abbreviated column names, tokens are repeated in column values (e.g., "ucla" becomes "ucucla") and proximal characters in keyboard are replaced (e.g., "shooting guard" becomes 'shootung guard"). computing column representations or column relational representations based on token embeddings with tabular noise introduces unavoidable wrong predictions. therefore, it requires the table union search model to be robust to the table noise during optimization. to address above issues, we propose a multistage self-supervised table union search framework called autotus, which represents column relation as a vector-column relational representation and learn column relational representation in a multi-stage manner that can better describe column relation for unionability prediction. in particular, the large language model powered contextualized column relation encoder is updated by adaptive clustering and pseudo label classification iteratively so that the better column relational representation can be learned. moreover, to improve the robustness of the model against table noises, we pro-pose table noise generator to add table noise to the training table data. extensive experiments are conducted on three real-world datasets and show significant performance gains compared to existing state-of-the-art methods and demonstrates the robustness of autotus against the table noise. our contributions are four-folded: 1) we propose a novel framework autotus that can leverage column relational representation for table union search instead of pairwise column similarity. 2) we propose a multi-stage self-supervised method that can update the large language model powered contextualized column relation encoder in a step by step and iterative way to learn better column relational representation. 3) we propose a table noise generator and add table noise to the training tabular data to improve the robustness of the model against the table noise. 4) we conduct extensive experiment on real-world datasets as well as synthetic test set augmented with table noise and demonstrate significant performance gain of autotus over the strong baselines and shows the robustness of autotus against the table noise. ## problem definition table union search discovers all tables in a set of data lake tables that are unionable with the query table. specifically, we focus on the automatically case where no labeled training data is available. we define a set of data lake tables as t . for each table @xmath0, t 2 , • • • , t m }. there is a column relation e @xmath1, t j that describes their relationship. we employ a contextualized column relational encoder to obtain the column relational representation z e @xmath2. the table unionability score u (t i ,t j ) ∈ u is obtained by the table scorer that goes over all column relational representations between two tables t i , t j . similar to previous studies @xcite @xcite , given a query table q, table union search retrieves the top-y unionable tables q ⊆ t , where |q| = y and ∀@xmath3, we have u (q,t ) ≥ u (q,t ′ ) . ## proposed method in this section, we introduce the proposed au-totus. the framework (as shown in figure 2 ) consists of following components: multi-stage self-supervised column relational representation ## experimental evaluation we conduct extensive experiments on real-world public datasets as well as a synthetic test set augmented with table noise to show the effectiveness of our autotus on table union search task, and give a detailed analysis. ## conclusions in this paper, we propose a multi-stage selfsupervised table union search framework, which represents column relation as column relational representation and learn this representation in a multi-stage manner that can better describe column relation for unionability prediction. in additional, a table noise generator is proposed to improved the robustness of our approach against the table noise. the conducted extensive experiments demonstrate the effectiveness of the proposed approach.