TITA: A Two-stage Interaction and Topic-Aware Text Matching Model

In this paper, we focus on the problem of keyword and document matching by considering different relevance levels.In our recommendation system, different people follow different hot keywords with interest.We need to attach documents to each keyword and then distribute the documents to people who follow these keywords.The ideal documents should have the same topic with the keyword, which we call topic-aware relevance.In other words, topic-aware relevance documents are better than partially-relevance ones in this application.However, previous tasks never define topic-aware relevance clearly.To tackle this problem, we define a three-level relevance in keyword-document matching task: topicaware relevance, partially-relevance and irrelevance.To capture the relevance between the short keyword and the document at abovementioned three levels, we should not only combine the latent topic of the document with its deep neural representation, but also model complex interactions between the keyword and the document.To this end, we propose a Two-stage Interaction and Topic-Aware text matching model (TITA).In terms of "topicaware", we introduce neural topic model to analyze the topic of the document and then use it to further encode the document.In terms of "two-stage interaction", we propose two successive stages to model complex interactions between the keyword and the document.Extensive experiments reveal that TITA outperforms other well-designed baselines and shows excellent performance in our recommendation system.query-document matching is an important feature 045 in the ranking models.(2) as for the task of ques-046 tion answering (Yang et al., 2016), query-document 047 matching method can be used to find document can-048 didates or to help predict the answer span.(3) it 049 is also widely applied to recommendation systems 050 (Jiang et al., 2019).051 In many scenarios, we need to distinguish dif-052 ferent keyword-document (query-document) rele-053 vance levels.For instance, in our recommendation 054 system, we need to attach documents to some hot 055 keywords and then distribute the documents to the 056 people who follow the keywords.In this circum-057 stance, the document and the keyword should better 058 have the same topic, which we call topic-aware rel-059 evance.As shown in Table 1, for the hot keyword 060 "cherry blossoms", the document (labeled 2) should 061 be the ideal document which should be attached be-062 cause it has the same topic with the keyword while 063 the document (labeled 1) should be a secondary 064 choice, because only several words or phrases in 065 this document match the keyword but the topics of 066 the document mismatch the keyword.067 To tackle this problem, we define a three-068 level relevance: topic-aware relevance, partially-069 relevance and irrelevance.The topic-aware rele-070 vance means the keyword and the document have 071 the same topic while the partially-relevance means 072 only part of the document matches with the key-073 word.Our task is more challenging than previous 074 query-document matching tasks.To capture the rel-075 evance between the keyword and the document at 076 above-mentioned three levels, we should not only 077 combine the latent topic of the document with its 078 deep representation, but also model complex inter-079 actions between the keyword and the document.080 Previous neural query-document matching mod-081 els (similar as keyword-document matching) can 082 be divided into two categories according to their 083 model architectures (Guo et al., 2016).One is the