RTSM : Knowledge Distillation with Diverse Signals for Efficient Real-Time Semantic Matching in E -Commerce (2025)

Abstract: Semantic matching plays a pivotal role in e-commerce by facilitating better product discovery and driving sales within online stores. Transformer models have proven exceptionally effective in mapping queries to an embedding space, positioning semantically related entities (queries or products) in close proximity. Despite their effectiveness, the high computational demands of large transformer models pose challenges for their deployment in real-time scenarios. This paper presents RTSM, an advanced knowledge distillation framework designed for Real-Time Semantic Matching. Our approach develops accurate, low-latency student models by leveraging both soft labels from a teacher model and ground truth generated from pairwise query-product and query-query signals. These signals are sourced from direct audits, synthetic examples created by LLMs, user interaction data, and taxonomy-based datasets, with custom loss functions enhancing learning efficiency. Experimental evaluations on internal and external e-commerce datasets demonstrate a 2-2.5% increase in ROC-AUC compared to directly trained student models, outperforming both the teacher model and state-of-the-art knowledge distillation benchmarks.

## introduction precise real-time semantic matching, which involves the identification of semantic similar entities (e.g., queries or products) for a user query, has become increasingly crucial for e-commerce product search. in order to bridge the semantic gap between the user query and the semantic similar entities, this matching process typically performed in two ways, as depicted in figure 1: (1) semantic query reformulation (sqr), where a user's poorly constructed query (e.g., containing code-mixed language or misspellings) is mapped to semantically similar, well-structured queries that produce a broader range of products. (2) semantic real-time semantic matching model real-time user query (q) query pool product pool n-dimensional representation (q) semantic representation (query pool) semantic representation (product pool) q1 q2 q3 ... semantically similar queries p1 p2 p3 ... ## problem statement our main objective is to enhance the performance of the student model in both semantic query reformulation (sqr) and semantic product retrieval (spr) tasks by creating effective representations of queries and products within a shared semantic space, all while substantially reducing inference time. achieving this versatility would enable us to minimize the expenses associated with maintenance and production. here, c i denotes the total number of purchases of product p i after firing query q i . while this data may be too noisy for direct modeling of query-product matches, it can be utilized to identify highly similar queries based on the overlap in associated product purchases. specifically, we define the distribution over query pairs as the gram matrix corresponding to the normalized query-product purchase counts and identify query pairs d qq+ = {(q i , q ′ i )} i that exhibit significantly higher occurrence relative to random chance using normalized pointwise mutual information (npmi)-based criteria (refer to section 4.1.3). (iv) product browse taxonomy: given a set of queries and classifiers capable of mapping queries to a product browse taxonomy, one can determine the taxonomy labels for all queries and construct pairs d qq-= {(q i , q ′ i )} i with nonmatching labels, which can be regarded as hardnegatives (refer to section 4.1.4). with these signals at hand, the aim is to train an effective model m so that for any user query q, product p, and another query q ′ , the similarity of their corresponding embeddings m (q), m (p), m (q ′ ) closely aligns with the relationships conveyed in the input signals. ## proposed method our solution strategy involves two primary phases. initially, we develop a teacher model considering the diverse signals outlined in section 4.1. subsequently, we train an effective student model, which not only replicates the soft labels of the teacher model but also integrates the original ground truth (section 4.2). in sec. 4.3, we elaborate on practical adjustments aimed at enhancing model efficacy. ## experiments and results we report our findings on the benefit of our proposed method for real-time semantic matching tasks. we start by presenting the dataset details. datasets: 1. e-commerce datasets from regions in india for evaluating query-product relevance. all datasets used in our analysis are anonymized, aggregated, and do not represent production distribution. 2. the publicly available esci dataset from amazon for the us (english) market. more details on the generation and construction of these datasets can be found in appendix a. reproducibility and hyperparameters: for details regarding the reproducibility of our experiments and the hyperparameter configurations, please refer to appendix b. ## conclusion in this paper, we introduce a kd approach for realtime semantic matching, where siamese student models acquire nuanced semantic representations by emulating both (i) the soft relevance labels from the siamese teacher model and (ii) the hard relevance labels annotated by humans. to address the needs of query reformulation and product retrieval tasks, we integrate a variety of similarity and dissimilarity signals, along with synthetic data generated from llms, and employ tailored loss functions to efficiently capture relevance and similarity intricacies. by leveraging both internal and public datasets, we demonstrate the superior effectiveness of our proposed method compared to existing sota kd benchmarks. junho yim, donggyu joo, jihoon bae, and junmo kim. 2017. a gift from knowledge distillation: fast optimization, network minimization and transfer learning. in proceedings of the ieee conference on computer vision and pattern recognition, pages 4133-4141. a dataset generation 1. proprietary amazon dataset: we gathered customer behavior data, denoted as d purchase qp , from historical logs of the in marketplace spanning from january 2024 to june 2024. to ensure data quality, pairs with fewer than 15 purchases were filtered out. for data generation based on taxonomy, we utilized an internal service to acquire browse node associations for 200k randomly chosen queries from the d purchase qp dataset. subsequently, d qq-was generated using browse node mappings to maintain the separation of irrelevant query-query pairs within the embedding space. for d llm s qq , we compiled a dataset comprising 200k search queries. we used an instruct llm to generate the top 10 positive and negative reformulations for each user query. additionally, we utilized a relevance model (see section b.4 on relevance model details) and browse node associations to refine d llm s qq further. regarding the d qp dataset, we collected a sample of 5.6 million human-@xmath0, product @xmath1. since our experiments focus on the indian marketplace, we constructed validation and test datasets by randomly selecting 50k query-ad pairs each from the in marketplace, removing these 100k pairs from training. in our performance evaluation, strict and standard relevance are treated as positive classes, while irrelevance is considered a negative class.