Joint Learning of Representations for Web-tables, Entities and Types using Graph Convolutional Network (2021)

Abstract: Existing approaches for table annotation with entities and types either capture the structure of table using graphical models, or learn embeddings of table entries without accounting for the complete syntactic structure. We propose TabGCN, that uses Graph Convolutional Networks to capture the complete structure of tables, knowledge graph and the training annotations, and jointly learns embeddings for table elements as well as the entities and types. To account for knowledge incompleteness, TabGCN’s embeddings can be used to discover new entities and types. Using experiments on 5 benchmark datasets, we show that TabGCN significantly outperforms multiple state-of-the-art baselines for table annotation, while showing promising performance on downstream table-related applications.

## introduction table data abounds in webpages and organizational documents. annotation of table entries, such as columns, cells and rows, using available background knowledge (e.g. yago, dbpedia, freebase, etc.), such as knowledge of entities and their types, helps in better understanding and semantic interpretation of such tabular data. the challenge, however, is that such web tables do not adhere to any standard format, schema or convention @xcite . additionally, knowledge graphs are typically incomplete -entities and types mentioned in tables may not always exist in the knowledge graph. therefore, it becomes necessary to expand the knowledge graph with new entities @xcite and types for annotating tables. initial research on table annotation @xcite @xcite used probabilistic graphical models to capture the complete row-column structure of tables and also the knowledge graph for collective annotation. more recent approaches using embeddings @xcite @xcite @xcite only partly capture the syntactic structure of tables, and also ignore the structure of the knowledge graph. the problem of incompleteness of the knowledge representation @xcite is mostly not addressed. in this work, we propose the tabgcn model that uses a graph convolutional network (gcn) @xcite to unify the complete syntactic structure of tables (rows, columns and cells) and that of the knowledge graph (entities and types) via available annotations. the embeddings of the table elements as well as knowledge graph entities and types are trained jointly and end-to-end. while gcns have been used for learning embeddings for many nlp tasks using the syntactic and semantic structure of natural language sentences @xcite , encoding tabular structure using gcns has not been addressed before. the model and embeddings thus trained are used to annotate new tables with known entities and types, while discovering hitherto unseen entities and types. additionally, we use the trained embeddings for tables and rows for downstream table-related tasks -identifying similar tables, and identifying the appropriate table for any row. we demonstrate these capabilities of tabgcn using experiments on 5 benchmark web table datasets comparing against 5 existing models. we show that webgcn significantly improves performance for entity and type annotation. for the other tasks, we show that the same embeddings show impressive performance. no existing model can perform all of these tasks. our contributions are as follows @xcite we propose a model called tabgcn based on the gcn architecture that captures the complete syntactic structure of tables as well as the knowledge representation, and learns embeddings of tables, rows, columns and cells, as well as entities and types jointly and in an end-to-end fashion. (b) tabgcn addresses incompleteness in the knowledge representation by discovering new entities and types. (c) tabgcn significantly outperforms 5 existing approaches in 5 different benchmark datasets for the task of table annotation. (d) the trained embeddings show impressive performance in downstream tasks such as identifying similar tables and assignment of rows to appropriate tables. ## related work existing literature on table annotation considers two different types of tables. in general, web tables @xcite @xcite contain mentions of entities under every column, which need to be annotated. in contrast, relational tables @xcite @xcite refer to a single entity in each row, with a single core or anchor column and multiple attribute columns. here, the entire row is annotated with a single entity. while both tasks are important, our focus is on the first category. in terms of approaches, one category of work considers graphical models to capture table structure and performs joint inference for entity and type classification @xcite @xcite . @xcite uses a markov random field that captures the structure of tables, and creates dependencies between entity and type annotations to jointly classify entities for cells and types for columns. the markov random field (mrf) potentials capture domain knowledge about similarities between cells, between cells and entity lemmas and between entities using the type hierarchy. it cannot handle entities and types not used as labels during training. meimei @xcite extends this mrf framework to handle numeric columns and focuses on multi-label classifiers for entities with multiple types. additionally, it constructs embeddings of entities and types using knowledge graph structure to avoid expensive graph traversal for computing potential function features. however, it still requires manual construction of these features based on domain knowledge. our gcn architecture is motivated by the structure of the graphical model in these papers. both these models also jointly label pairs of columns with known relationships, which we do not address in our work. the second category focuses on embeddings for tables @xcite @xcite . all of these models transform table data to word sequences and then make use of neural language models. zhang and balog (2018) make use of rdf2vec for embedding tables, which internally uses neural language models after transforming graphs to sequences. colnet @xcite considers columns as cell sequences and uses a cnn to learn the representations of individual cells, which are used to predict entity types for columns. tabert @xcite focuses on retrieving table rows for natural language utterances and learns a joint representation for utterances and cell sequences in table rows using bert. in summary, these fail to capture the complete rowcolumn structure of tables in the embeddings. in contrast, our gcn architecture captures the structure of all tables, the entities, the types and the training annotations. for the problem of extending the knowledge graph from tables, @xcite consider discovery of new entities, but only in the context of relational tables, with a single core column. their approach make use of pre-trained neural embeddings (word2vec) and cosine similarity in addition to lexical similarity. our focus is on web tables, where we need to discover new entities and types for all columns in a table. our approach makes use of gcn embeddings trained using entity and type labels to detect new entities and types. ## problem: table annotation in this section, we first define tables, the background knowledge of entities and types, annotation of tables with entities and types, and, finally, the problem of semi-supervised table annotation with incomplete knowledge. tables: we are given a set of tables s. fig. 1 shows an example at the top. the k th table s @xmath0, consists of m k rows and n k columns of individual cells. the individual cell in the i th row and j th column of the k th table is denoted as x k ij . we consider textual tables only, so that each x k ij takes a string as value. also, we denote the i th row as r k i and the j th column as c k j . entities and types: we assume background knowledge of entities and entity types (or simply, table with 2 columns and 2 rows. the column annotations with types and the cell annotations with entities are shown within brackets. (b) gcn graph for example table with table nodes as boxes and entities and types nodes as ovals. table edges are shown using fine solid lines, entity-type edges with thick solid lines and annotation edges using dashed lines. lexical similarity edges are typically between cells in different tables and are not shown. ## joint table and knowledge embedding we first present at a high level the network architecture of our model, which we call tabgcn. the core components tabgcn are (i) a graph convolutional network (gcn), which captures the various syntactic relationships between table and knowledge elements, and then jointly learns embeddings for these via the convolution operation. the gcn embeddings of table elements contain information about both types and entities. these are fed into two parallel components: (ii) the type classification component, and (iii) the entity classification component. the type classification component first projects the gcn embedding of table columns to a type space using a type-projection matrix, and then uses a soft-max layer to classify this type embedding according to observed types. similarly, the entity classification component first projects the gcn embeddings of table cells to an entity space using an entity-projection matrix, and then uses a soft-max layer to classify this entity embedding according to observed entities. fig. 2 shows the high level architecture of our model for a subgraph of our example table graph in fig. 1 . the parameters of all three components are trained jointly in an endto-end fashion using training entity annotations for cells and type annotations for columns via backpropagation. we next describe these components in greater detail. graph convolutional network: graph convolutional networks (gcn) @xcite extend the notion of convolution to graphs. let @xmath1, r) be a directed graph where v is the set of n nodes and r the set of directed edges. an edge between nodes u, @xmath2, v, l uv ) ∈ r. the edge set includes an inverse edge for each edge and a self-loop for each node. an input feature matrix @xmath3 luv are label specific model parameters at k th layer and h (1) @xmath4. for classification, a linear classification layer is added on top of final gcn layer. function f () is a non-linear activation for which we used relu. ## annotating tables using embeddings training the network in sec.4 generates estimates of the model parameters as well as embeddings for all table and knowledge elements. in this section, we describe the use of these parameters and embeddings for the tasks defined in sec.3. we use h(v) for the gcn embedding of a node v instead of h (k) v for brevity. ## experiments in this section, we first present experimental results for table annotation, and then for down-stream table-related tasks. we compare our proposed model tabgcn with appropriate state-of-the-art baselines. we have uploaded our source code as supplementary material for reproducibility. web manual @xcite contains tables fetched by web-crawling using wiki manual tables as queries. these two tables are manually annotated with entities and types from yago. the tables and annotations are noisier than wiki manual. limaye @xcite corrects many incorrect annotation in wiki manual using entities and types from dbpedia 2015. t2dv2 @xcite ) contains tables from common web crawl. wikipedia is a publicly available subset of the data used by @xcite . this has html tables from wikipedia 2013 with class attribute "wikitable". the last three datasets are annotated using dbpedia. but t2dv2 and wikipedia contain only type annotations, and entity annotations are not available. for all datasets, we first set aside 30% of the unique entities and types, and effectively all their annotations as unseen during training. of the remaining annotations, again 30% was removed during training. baselines: we compare tabgcn against three existing table annotation approaches. pgm (limaye et al., 2010) uses a probabilistic graphical model and hand-crafted features. since our entitytype set t is flat instead of being a hierarchical graph, we use logistic regression instead of structural svm to estimate model parameters. meimei @xcite also uses a markov random field but embeds the entities and types for fast computation of clique potentials. for both models, we ignore the relation labels for column-pairs and associated potential functions, since we do not address relation detection. table2vec @xcite learns various word2vec-based embeddings for tables for entity annotation. it does not address type annotation. it focuses on relational tables and associates a single entity with a row for its core column. we adapted this model for web tables which associate an entity for each cell. we use their table2vece version, which models a row as a sequences of all entities that appear within cells of that row. after training the word2vec model, instead of considering the embedding for the entire row as in table2vece, we create cell-specific embeddings from only the tokens for that cell. col-net @xcite models each column as a sequence of words from the cells under the column and uses a cnn to predict the column type. we also use an adaptation of tabert @xcite which trains a joint language model for retrieval from tables given utterances. we adapt their approach to independently linearize each row and column of a table as a sequence of cells under that row and column, and get column and row embeddings using the mean of corresponding cell embeddings. for cells, we use pre-trained bert embeddings. hyper-parameters: we used one-hot encodings as inputs for cells and entities with lemmas. we used relu as the nonlinear gcn activation, 2 gcn layers with 512 and 256 dimensions. as a result, the 8 gcn weight matrices were vx512 in the first layer, where v is the vocabulary size, detection results: we first present results for entity and type detection, addressed by most of the baselines. for this task, all models predict entity labels for all cells and type labels for all columns that are unannotated. however, evaluation is only for those cells and columns whose true entity and type labels are contained in the observed entities e o and observed types t o respectively. for evaluation, as in earlier table annotation papers @xcite , we use micro-averaged f1, which takes the weighted average of the f1 scores over the entities or types. this takes class imbalance into account and is therefore more appropriate than accuracy. we note that meimei addresses the multi-label setting with multiple possible type labels, and therefore uses ranking evaluation measures (e.g. ndcg). this is not meaningful in our single-labeled setting. tab. 1 shows detailed results for all models across datasets. we can see that tabgcn significantly outperforms all baselines on all datasets for both detection tasks. the only exception is for wiki manual. the graphical model based approaches with handcrafted potential functions outperform the representation learning approaches, possibly on account of the smallness of the dataset. among the embedding based approaches, tabgcn performs the best. novelty classification results: in our second experiment, we consider novelty classification. this is an unsupervised task, where a model makes a binary decision for each unannotated column (novel type classification) and for each unannotated cell (novel entity classification). since the decision depends on the thresholds for type ( t ) and entity ( e ), we plot f1 score on the y-axis against the corresponding threshold on the x-axis. of the baselines, only pgm can address this task, but outputting a none label for the type or entity. fig. 3 shows novelty classification performance for web manual. results for wiki manual and limaye are very similar. tabgcn reaches f1 around 0.8 for both tasks for appropriate thresholds. across thresholds, tabgcn significantly out-models wiki m web m limaye type ent. type ent. type ent. -k 0.29 0.23 0.84 0.82 0.83 0.78 -e 0.22 -0.82 -0.81 --t -0.22 -0.63 -0.63 -l 0.33 0.23 0.80 0.75 0.81 0.74 tabgcn 0.33 0.24 0.84 0.83 0.84 0.79 type and entity discovery results: the final annotation tasks are type discovery, where all unannotated columns that do not correspond to known types in t o need to be clustered into distinct new types, and entity discovery, where all unannotated cells that do not correspond to known entities in e 0 need to be clustered into distinct new entities. we used normalized mutual information (nmi) n m @xmath5,y ) h(c)+h(y ) between the assigned cluster labels (y) and the true entity or type labels (c), where i(, ) denotes mutual information and h() denotes entropy. in tab. 2, we see that tabgcn performs consistently above 80% for entity and type discovery across datasets, significantly outperforming colnet and tabert. ablation study: next, we analyze the performance of tabgcn, using multiple ablations of the full model. -k leaves out the knowledge nodes and their incident edges from the gcn graph during training. -e focuses only on types by removing all entity nodes and entity-related edges (type-entity and cell-entity annotation) from the gcn graph. it is trained only using type loss. note that it cannot perform tasks associated with entities, specifically entity detection, novel entity classification and novel entity discovery. similarly, -t focuses only on entities by removing all type nodes and type-related edges (type-entity and column-type annotation) from the gcn graph. it is trained only using entity loss, and cannot perform tasks associated with types. finally, -l removes the lexical similarity edges from the gcn graph. the results are recorded in table . 5. we can see that all the components of the architecture contribute to performance improvements. the improvement is statistically significant (using the wilson interval with α = 0.05) for all ablations other than -k. while -k performs comparably here, its performance drops downstream table related tasks: finally, we include some results for the table and row related inference tasks defined at the end of sec.5. this is to demonstrate how the learnt embeddings can benefit potential down-stream tasks. note that tabgcn directly outputs table embeddings. of the baselines, tab2vec and tabert output row embeddings. for these models, we create table embeddings by averaging the corresponding row embeddings. in tab. 6, we record performance for table clustering. tabgcn again significantly outperforms both baselines for both datasets. we finally consider row-to-table assignment. in this task, one randomly selected row is removed from every table during training. these rows then need to be classified according to their parent table. since the models output a ranking of tables for each row, we evaluate using two ranking related measures. hits@1 measures the fraction of rows with the correct table at rank 1. mean reciprocal rank (mrr) is the mean of the reciprocal rank of the correct table over all rows, and its perfect value is 1.0. in tab. 3, we again see that tabgcn performs the best across datasets. in summary, we have demonstrated the usefulness of learning embeddings of table elements and knowledge elements jointly using both entity and type losses in an end-to-end fashion for type and entity annotation on 5 benchmark datasets. in addition, we have demonstrated how the learned embeddings can be useful for downstream table-related tasks. in all cases, tabgcn has significantly outperformed multiple state-of-the-art baselines using probabilistic graphical models as well as other neural approaches. ## conclusion we have proposed a model for that jointly learns representations of tables, rows, columns and cell, as well as entities and types by capturing the complete syntactic structure of all tables, the relevant entities and types and the available annotations using the graph convolutional network. as a result, tabgcn unifies the benefits of probabilistic graphical model based approaches and embedding based approaches for table annotation. using these embeddings, tabgcn significantly outperforms existing approaches for table annotation, as well as entity and type discovery.