R o CEL : Advancing Table Entity Linking through Distinctive Row and Column Contexts (2024)

Abstract: Table entity linking (TEL) aims to map entity mentions in the table to their corresponding entities in a knowledge base (KB). The core of this task is to leverage structured contexts, specifically row and column contexts, to enhance the semantics of mentions in entity disambiguation. Most entity linking (EL) methods primarily focus on understanding sequential text contexts, making it difficult to adapt to the row and column structure of tables. Additionally, existing methods for TEL indiscriminately mix row and column contexts together, overlooking their semantic differences. In this paper, we explicitly distinguish the modeling of row and column contexts, and propose a method called RoCEL to capture their distinct semantics. Specifically, for row contexts in tables, we take the attention mechanism to learn the implicit relational dependencies between each cell and the mention. For column contexts in tables, we employ a set-wise encoder to learn the categorical information about the group of mentions. At last, we merge both contexts to obtain the final mention embedding for link prediction. Experiments on four benchmarks show that our approach outperforms the state-of-the-art (SOTA) baseline by about 1.5% on the in-domain dataset, and by 3.7% on average across three out-of-domain datasets.

## introduction tel is an important task in natural language processing (nlp), which serves as a building block of table understanding. it benefits many downstream table-oriented tasks. for instance, in table question answering @xcite , it helps locate the answer for entity-centric questions. in knowledge base population @xcite , it aids in mining relationships between entities from the tables. the task of tel is to map the mentions in a table to their referent entities in a given kb. best-selling albums of the 21st century, 13-19 million copies: figure 1 shows an example, where the mention "fallen" is linked to the entity "fallen (q155612)" in wikidata. the major challenge of tel is how to make use of the structured contexts in tables to help link mentions to their corresponding entities. most of the existing works have focused on text el, which aims to model the sequential context of the mention @xcite @xcite . however, tables often exhibit different structures and relationships between mentions and contexts compared to text. firstly, the context in tables is presented in a two-dimensional structure, i.e., row and column, whereas in text, it is displayed in a one-dimensional sequential form. secondly, the row context typically includes descriptional information about the mention, and the column context contains similar entities to the mention. however, the sequential context in texts is often natural language sentences with syntactic and grammatical constraints. in this way, the tel entails a more complex structured context, rendering it more challenging than the text. recently, some works have tried to model the structured context in tables. for example, @xcite proposes a pretrained language model tapas for table question answering, and @xcite adopts markup languages to leverage large language model (llm) for table-to-text generation. the most relevant works to ours are turl @xcite and npel @xcite , where turl employs a transformer with a masked selfattention mechanism, and npel uses gcn to encode row and column contexts. however, neither method can well distinguish the semantic differences between row and column contexts. this results in a coarse representation of table information, and limits the accuracy of tel. in fact, the row and column in the table usually provide contexts of different aspects about the mention @xcite , which calls for distinct modeling strategies. in the row context, each cell is either the properties or related objects to the mention. for example in figure 1 , the row context of the mention "fallen" contains the release year (i.e., "2003"), the performer (i.e., "evanescence"), the nationality (i.e., "united states") and the sales (i.e., "17.0"). in the column context, each cell typically represents an entity similar to the mention, all belonging to the same category specified by the column header. as is shown in figure 1 , the column context of the mention "fallen" includes other albums such as "the fame" and "let go". in this study, we propose row-column differentiated table entity linking (rocel) to learn the distinct context in the row and column of the mention. specifically, (1) to learn the descriptional information of mentions, we serialize row contexts with implicit dependencies, and encode them with the self-attention mechanism. (2) to capture the categorical information of mentions, we treat a column of independent mentions as an unordered set, and learn the category through a set-wise encoder. to evaluate the effectiveness of our method, we conduct experiments on four benchmark datasets. the results demonstrate that our method outperforms all sota baselines, including cell-based, text-based, and table-based methods. further experiments show that our method can effectively utilize row and column contexts on both in-domain and out-of-domain data. additionally, we evaluate the open-sourced llm llama-3 (ai@meta, 2024b) on this task, and find it challenging for the model to leverage the structured contexts for tel. the contribution of this work is as follows: ## preliminary table contexts. in this paper, the term "table" refers to "relational table" @xcite , which is similar to a table in a database. it contains multiple rows and columns, where each row represents an individual record and each column represents a field of the records. given a table t with i rows (excluding headers) and j columns, as illustrated in figure 2 , we denote: (1) t θ as the metadata of t , such as its title, caption, or surrounding text; the caption of this table: ## method in this section, we will give a detailed introduction to our model, namely rocel. figure 3 shows an overview of the rocel, which comprises of three major components: (1) row context encoding, aiming to represent the descriptional information from row contexts; (2) column context encoding, aiming to represent the categorical information from column contexts; and (3) semantics fusion & entity linking, which fuses the embeddings of row and column contexts into a final mention embedding, and links it to the target entity. the fame lady ... usa 2003 fallen evan ... usa 2002 let go avril ... canada ⋯ ⋯ ⋯ ⋯ year album artist nation fallen the fame let go ⋯ fallen (q155612) dot argmax e ij year album artist nation 2003 fallen evan ... usa ## experimental settings in this section, we will introduce our experimental datasets, baselines, and implementation details. ## experimental results in this section, we will conduct experiments to answer the following research questions: ## conclusion in this work, we propose a table entity linking method rocel, in which the row and column contexts are encoded in two manners due to their semantic differences. our approach outperforms all sota baselines on four benchmark datasets, demonstrating its better utilization for table contexts. in contrast, further evaluations of text-based methods including llama-3 indicate their difficulty in leveraging table contexts. for future work, we plan to achieve a global entity linking method by imposing different constraints on the predictions within rows and columns. additionally, the warming up of the row encoder remains to be explored, since we only warm up the column encoder.