Adaptive Feature Discrimination and Denoising for Asymmetric Text Matching (2022)

Abstract: Asymmetric text matching has becoming increasingly indispensable for many downstream tasks (e.g., IR and NLP). Here, asymmetry means that the documents involved for matching hold different amounts of information, e.g., a short query against a relatively longer document. The existing solutions mainly focus on modeling the feature interactions between asymmetric texts, but rarely go one step further to recognize discriminative features and perform feature denoising to enhance relevance learning. In this paper, we propose a novel adaptive feature discrimination and denoising model for asymmetric text matching, called ADDAX. For each asymmetric text pair, ADDAX is devised to explicitly distinguish discriminative features and filter out irrelevant features in a context-aware fashion. Concretely, a matching-adapted gating siamese cell (MAGS) is firstly devised to identify discriminative features and produce the corresponding hybrid representations for a text pair. Afterwards, we introduce a locality-constrained hashing denoiser to perform feature-level denoising by learning a discriminative low-dimensional binary codes for redundantly longer text. Extensive experiments on four real-world datasets from different downstream tasks demostrate that the proposed ADDAX obtains substantial performance gain over 36 up-to-date state-of-the-art alternatives.

## introduction given a pair of documents, text matching aims to precisely predict the semantic relations between them. an efficient and effective matching algorithm is now an indispensable asset in many information retrieval, question answering and dialogue systems. in these application scenarios, a text pair in matching (e.g., query-document and question-answer pair) ususlly has a large disparity in the quantity of information, a.k.a. asymmetric text matching. for example, a matching pair have 7.15 and 95.54 words respectively on average in insuranceqa dataset @xcite (i.e., the difference being about an order of magnitude). this asymmetry between a short query and a long document renders it as a nontrivial task. the existing solutions can be grouped into two categories, namely representation-based and interaction-based models @xcite . the former category mainly utilizes convolutional neural networks (cnn) and recurrent neural networks (rnn) to learn the latent representation of a document independently, including dssm @xcite , snrm @xcite . on contrast, the latter category focuses on leveraging fine-grained interaction signals between them. it is widely recognized that exploiting interaction signals would largely improve the relevance learning capacity. examples include drmm @xcite , knrm @xcite . recently, with the prominence of deep pre-trained language models (lms) like bert @xcite , uptodate lms-based deep relevance models significantly push the frontier of the state-of-theart further @xcite @xcite . though significant performance gain is achieved by these efforts, they mainly overlook further feature discrimination and denoising between asymmetric texts, which can be potentially useful to enhance matching performance. to explain this point, we give an illustrative example in figure 1 . here, polysemous word like "power station" in the query side hinders the precise matching. on the other hand, the semantic association among "who", "pop rock concert group" and "singers" assists the relevance learning process. also, word pair like "who" and "factory" describe two distinct things, which can be reflected via the feature-level interactions. hence, recognizing the discriminative features and filtering out noisy features certainly enhance the relevance learning process. to this end, in this paper, we propose an adaptive feature discrimination and denoising model for asymmetric text matching, named ad-dax. specifically, addax consists of a bertbased context encoder, a matching-adapted gating siamese cell (called mags), a locality-constrained hashing denoiser, and a maxsim @xcite ) based relevance predictor. for each document, we firstly derive the word-level contextual representations through a bert-based context encoder. afterwards, mags utilizes an crossattention mechanism to represent a document with relevant information from its counterpart in the matching pair, which produces the word-level reference representations for the former. then, the resultant word-level attention information is leveraged to discriminate the importance of these reference representations and their divergence against the orginal representations. we then utilize a highway network to adaptively composite these two kinds of semantic signals as the context-aware hybrid representations. after this feature-level discrimination, we utilize a locality-constrained hashing denoiser to project the long document into low-dimensional binary space. the hashing denoiser is formulated as an autoencoder over the hybrid representations. that is, the semantics relevant to the text pair will be preserved by the denoising process, which further facilitates the relevance learning. finally, a maxsim operator @xcite ) is employed to calculate final relevance score. overall, the key contributions are summarized as below: ## the proposed method in this section, we first describe the task formulation in section 3.1. afterwards, we present each component of addax in detail. ## conclusion in this paper, we introduce an adaptive feature discrimination and denoising model for asymmetric text matching. specifically, we first design a matching-adapted gating siamese cell in addax to perform feature discrimination and generate the hybrid representations together for the asymmetric text pair. we then present a locality-constrained hashing denoiser for filtering semantic noise for redundant long documents. extensive experimental results on four benchmarks have demonstrated the effectiveness and superiority of our proposed addax. as future work, we plan to investigate the possibility of feature discrimination and denoising in other asymmetric scenarios like document abstractive summarization, caption generation and more.