Table-based Fact Verification with Self-labeled Keypoint Alignment (2022)

Abstract: Table-based fact verification aims to verify whether a statement sentence is trusted or fake. Most existing methods rely on graph feature or data augmentation but fail to investigate evidence correlation between the statement and table effectively. In this paper, we propose a self-Labeled Keypoint Alignment model, named LKA, to explore the correlation between the two. Specifically, a dual-view alignment module based on the statement and table views is designed to discriminate the salient words through multiple interactions, where one regular and one adversarial alignment network cooperatively character the alignment discrepancy. Considering the interaction characteristic inherent in the alignment module, we introduce a novel mixture-of experts block to elaborately integrate the interacted information for supporting the alignment and final classification. Furthermore, a contrastive learning loss is utilized to learn the precise representation of the structure-involved words, encouraging the words closer to words with the same table attribute and farther from the words with the unrelated attribute. Experimental results on three widely-studied datasets show that our model can outperform the state-of-the-art baselines and capture interpretable evidence words.

## introduction table-based fact verification aims to uncover the factuality attribute of the sentence relying on the available structured @xcite @xcite textual evidence. current methods can be divided into two groups. the first one exploits the logical form of the statement with graph neural networks @xcite @xcite . the other focuses on extending table-aware pre-trained language models (plms) @xcite . in plm-oriented fact verification, the majority of methods treat the statement-table pair as plain text and then further capture latent essential information relying on multiple transformer @xcite blocks. it is intuitive that only partial table cells are associated with the statement while other cells are redundant @xcite . due to the lack of explicit guidance signals in the statement-table pair, the capability of checking various statements is hindered for the plms, deteriorating model performance and interpretability. taking figure 1 as an example, the clues are derived from the statement and some scattered table cells. if the model pays attention to the unrelated words (e.g., "sliver", "bronze", "total"), the prediction would not be able to convincingly correct. in other words, failing to align the latent salient words, which are denoted as keypoints, may lead to some misleading information being focused on as evidence. despite impressive process, we empirically find that few methods are committed to the keypoint alignment across the statement words and table cells. the main reasons are: 1) there are alignment discrepancies in the alignment space, where one statement is associated with one table. however, one table may be involved with several statements; 2) the essence of alignment is to perceive salient evidence for the final classifi-cation, which requires a well-designed interaction network to aggregate the statement words and table cells; 3) flexible table structures hinder the representation of words since the significant cost of designing a general structure-aware plm for tablebased fact verification task. in summary, exploring keypoint alignment feature in the statement-table pair is a major challenge. to tackle the above deficiencies, we propose a model called self-labeled keypoint alignment (lka) for table-based fact verification, focusing on aligning salient evidence and aggregating essential information between the statement and table. specifically, we design a dual-view alignment module (da) for dealing with the discrepancy of the alignment characteristics. an interaction network is first applied for aggregating the interacted statement and the table representation in multiple steps. the da then employs a regular alignment network to learn keypoint correlation from the enhanced statement view and force another adversarial alignment network to perceive the correspondingly reverse correlation (i.e., unrelated words) from the table view. for providing aggregated information in the interaction network, we design an adaptive aggregation experts (aae) block. the aae employs a mixture-of-experts (moe) network @xcite that incorporates multiple operating units to sufficiently aggregate the statement and table information. besides, inspired by the contrastive learning theory @xcite , we adopt a structure-aware contrastive learning loss to obtain precise representation for the structure-involved words. the amended representation can force the statement and table closer to its local sub-structure zone (e.g., statement, row, column, etc.) and farther away from others. our contributions are summarized in three folds: tab-facts. experimental results demonstrate that our model bring performance gains by 0.67%/3.63%/3.07% compared with several state-of-the-art models, and the captured salient words can be interpreted. ## related work unlike fever @xcite utilizing textual evidence or feverous (rami aly and mittal, 2021) using textual-table mixed evidence, tablebased fact verification @xcite concentrates on structured or semi-structured evidence text. the currently popular methods employ a tree-style neural network @xcite or graph network @xcite to encode the logical form of statements. however, labeling massive accurate logical forms is labor-intensive. meanwhile, structure-aware models @xcite @xcite have been investigated to deal with the table-based fact verification task. among these approaches, tapas++ @xcite projects structural information of tables to a pre-trained language model by importing row, column, numeric features into the embedding layer. some researchers also design novel data augmentation strategies to enhance tapas++, such as decomposing complex statements @xcite , replacing non-salient tokens @xcite , or generating massive question-answer pairs @xcite . however, upgrading vanilla plms for precise representation requires a considerable expense of pre-training or data augmenting. in addition to the rational structured table data, there are entity tables @xcite and matrix pdf tables @xcite in table-based fact verification scenarios. mainstream approaches @xcite employ tapas++ and tabfact data to check matrix pdf tables while various scanning @xcite and filtering @xcite methods are proposed to deal with entity tables. different from the aforementioned works, our model exploits a novel alignment of salient words for the statementtable pair and a structure-oriented loss for precise representation. ## conclusion this paper takes full advantage of alignment signals to facilitate a self-labeled learning procedure from the statement and table views. more importantly, an moe-level aggregation module is designed to explore the valuable information. besides, a contrastive learning loss is introduced to promote the awareness of table structure. future research could be extended as follows: 1) exploring alignment mechanism in the table-based question-answer tasks; 2) developing fact verification approaches in the multi-evidence table setting.