TITA : A Two-stage Interaction and Topic-Aware Text Matching Model (2021)

Abstract: In this paper, we focus on the problem of keyword and document matching by considering different relevance levels. In our recommendation system, different people follow different hot keywords with interest. We need to attach documents to each keyword and then distribute the documents to people who follow these keywords. The ideal documents should have the same topic with the keyword, which we call topic-aware relevance. In other words, topic-aware relevance documents are better than partially-relevance ones in this application. However, previous tasks never define topic-aware relevance clearly. To tackle this problem, we define a three-level relevance in keyword-document matching task: topic-aware relevance, partially-relevance and irrelevance. To capture the relevance between the short keyword and the document at above-mentioned three levels, we should not only combine the latent topic of the document with its deep neural representation, but also model complex interactions between the keyword and the document. To this end, we propose a Two-stage Interaction and Topic-Aware text matching model (TITA). In terms of “topic-aware”, we introduce neural topic model to analyze the topic of the document and then use it to further encode the document. In terms of “two-stage interaction”, we propose two successive stages to model complex interactions between the keyword and the document. Extensive experiments reveal that TITA outperforms other well-designed baselines and shows excellent performance in our recommendation system.

## introduction the keyword-document matching is mostly like the query-document matching task. the querydocument matching task, aiming to calculate relevance score between a query and a document, has been extensively studied over the past few years. it is widely applicable in many real scenarios: (1) in the information retrieval systems @xcite , 5432 keyword: cherry blossoms original keyword: 樱花 label: 0 irrelevance case translated document: there was a flower shop which has opened for a few months. i bought some flowers to decorate my house. the shop had common flowers such as lilies and carnations, but there were not many colors to be chosen... original document: 这家花店开了有几个月了。我买了一些花来装饰我的房子。店里有百合 花、康乃馨等普通花卉，但可供选择的颜色不多... label: 1 partially-relevance case translated document: the food in this restaurant is very delicious. i tried some dishes, such as foie gras, steak, squid, noodles, desserts, etc. all the dishes are really yummy, especially the filet mignon... by the way, there is a cherry blossoms exhibition near this restaurant. original document: 这家餐馆的菜都很好吃，我试吃了一些菜，如鹅肝、牛排、鱿鱼、面条、 甜点等。所有的菜都很好吃，尤其是菲力牛排...顺便说一下，这家餐厅附近有樱花展。 label: 2 topic-aware relevance case translated document: yuyuantan park is the best to enjoy cherry blossoms. the cherry blossoms in the park are available in a variety of colors and varieties. their flowering period is short... original document: 玉渊潭公园是赏樱花的最佳去处。公园里的樱花有各种颜色和品种。它们 的花期很短... our research contributions can be summarized as follows. ## online deployment and online gains because the model is heavy and the total numbers of keywords are limited (8901 in total), we generate data in offline, as shown in figure 2 . in offline data processor, we first use bm25 to retrieve and rank billions of document candidates and keep the top-10000 candidates for tita model to further conduct query-document relation prediction. after that we can get a ranked list of topic matching documents and partially relevance documents for all keywords, which will be stored in a kv database.