KD -Boost: Boosting Real-Time Semantic Matching in E -commerce with Knowledge Distillation (2023)

Abstract: Real-time semantic matching is vital to web and product search. Transformer-based models have shown to be highly effective at encoding queries into an embedding space where semantically similar entities (queries or results) are in close proximity. However, the computational complexity of large transformer models limits their utilization for real-time matching. In this paper, we propose KD-Boost, a novel knowledge distillation algorithm designed for real-time semantic matching. KD-Boost trains low latency accurate student models by leveraging soft labels from a teacher model as well as ground truth via pairwise query-product and query-query signal derived from direct audits, user behavior, and taxonomy-based data using custom loss functions. Experiments on internal and external e-commerce datasets demonstrate an improvement of 2-3% ROC-AUC compared to training student models directly, outperforming teacher and SOTA knowledge distillation benchmarks. Simulated online A/B tests using KD-Boost for automated Query Reformulation (QR) indicate a 6.31% increase in query-to-query matching, 2.76% increase in product coverage, and a 2.19% improvement in relevance.

## introduction accurate real-time semantic matching, i.e., identifying matching entities for a query, has become increasingly essential for web search and e-commerce product search. to address the semantic gap between the pool of queries and results(e.g., products in product search), this matching is often performed in two steps shown in figure 1: (1) automated query reformulation (qr), which maps a poorly formed (e.g., including code-mixed language, typos) user query to semantically similar well-formulated queries with wider coverage of results. (2) result/product sourcing (ps) which retrieves matching results for a given user query. in input query query pool product pool semantic matching model q1 q2 q3 ... p1 p2 p3 ... query reformulation product sourcing search results p1' p2' p3' ... search index mapping queries mapping products this work, we focus on enhancing real-time representation models for both queries and results aiming to improve both qr and ps tasks. since product search is our primary application, we refer to results as products though the matching problem has broader applicability. existing sota techniques for semantic matching are mostly based on siamese networks @xcite that comprise of two identical sub-networks that generate semantic representations for the pair of entities (query-query or query-product) to be compared. of these, transformer-based models such as bert and distilbert @xcite have been shown to yield highly accurate matching performance. however, these models fail to satisfy the strict latency requirements of large scale product search in b2c e-commerce. on the other hand, smaller encoder models such as minilm @xcite with low latency often result in poor matches. a common approach for addressing this performance gap is via knowledge distillation (kd) @xcite methods that transfer information from a larger teacher model to a smaller student model via soft labels from the teacher. the resulting models are often superior to those obtained via direct training of student models but inferior to the teacher model. further, this approach does not permit the student models to correct for errors in the teacher model itself. contributions. in this work, we consider the problem of semantic matching of queries with other queries and products. we propose an efficient kd techniques that learns via imitation of soft relevance labels from a larger teacher model as well as the original ground truth. this approach allows the student model to capture the nuanced knowledge from the teacher model and also incorporate explicit guidance from the ground truth. below we summarize our key contributions: 1. we propose kd-boost, a novel kd algorithm designed for real-time semantic matching, which leverages soft labels from one or more teacher models as well as ground truth to learn a highly accurate and compute-efficient student model. 2. to address the dual needs of query reformulation and product sourcing, we utilize multiple sources of similarity and dissimilarity signals (e.g., queryproduct pairs with editorial ordinal relevance labels, user-behavioural data such as clicks and purchases, product taxonomy) with tailored loss functions to ensure that the model learns representations that can effectively capture the nuances of relevance and similarity. 3. experimental evaluation of the proposed approach on both internal and external e-commerce datasets @xcite results in a significant boost of 2-3% roc-auc on the query-product relevance task compared to training student models directly and even surpasses teacher and sota kd benchmarks. 4. lastly, online a/b testing of the proposed approach in real-time product search resulted in an increase of query-to-query automated query reformulation rate by 6.31% which in turn yielded in improved product coverage (+ 2.76%) and relevance (+ 2.19%.) note that our kd approach has wider applicability to other real-time semantic matching scenarios beyond product search. it can also be used with any choice of student and teacher encoder models. ## problem statement our primary goal is to improve the performance of the student model on both query reformulation and product sourcing tasks via effective representations of queries and products in a shared semantic space while significantly reducing inference time. building such a versatile model would allow us to minimize the costs associated with maintenance and production. we now define the problem formally in terms of the three available input signals. (i) expert labels on product-query pairs: let d label p @xmath0, p i , y i )} n @xmath1, q i and p i represent the query and product entities respectively, and y i represents the ground truth label belonging to one of the three classes: (i) high relevant, (ii) low relevant, or (iii) irrelevant. (ii) user behavioral data: let d purchase p @xmath2, p i , c i )} m @xmath3. while this data is too noisy for direct modeling of product-query match, it can be used to arrive at highly similar queries based on the overlap in the associated product purchases. specifically, we define the distribution over query pairs as the gram matrix corresponding to the normalized product-query purchase counts and identify query pairs d qq+ = {(q i , q ′ i )} i that show much higher occurrence relative to random chance by employing normalized pointwise mutual information (npmi) based criteria (section 4.1.2). (iii) product browse taxonomy: further, given a set of queries and classifiers that can map the queries to a product browse taxonomy, one can identify the taxonomy labels for all the queries and construct pairs d qq-= {(q i , q given all these signals, the objective is to learn an efficient model m such that for a given query q, product p and another query q ′ , the proximity of corresponding embeddings m (q), m (p), m (q ′ ) is close to that expressed in the input signals. ## proposed kd-boost our solution approach comprises two key stages. first, we build a teacher model that accounts for the various signals mentioned in section 4.1. then, we learn an efficient student model that not only mimics the soft labels of the teacher model but also uses the original ground truth (section 4.2). in section 4.3, we discuss practical modifications that further improve model performance. ## experiments and results we start by presenting the datasets and the setup of the experiments. note that further details on dataset generation and experimental setup are presented in appendix a. ## conclusion in this paper, we introduce kd-boost, a novel knowledge distillation technique specifically designed for real-time semantic matching. kd-boost effectively trains low latency student models by leveraging soft labels from a teacher model, along with ground truth information obtained from pairwise query-product and query-query signals sourced from diverse channels. through the utilization of both internal and public datasets, we showcase the superior efficiency of our proposed method compared to existing sota kd benchmarks.