M ini ELM : A Lightweight and Adaptive Query Rewriting Framework for E -Commerce Search Optimization (2025)

Abstract: Query rewriting (QR) is a critical technique in e-commerce search, addressing the lexical gap between user queries and product descriptions to enhance search performance. Existing QR approaches typically fall into two categories: discriminative models and generative methods leveraging large language models (LLMs). Discriminative models often struggle with natural language understanding and offer limited flexibility in rewriting, while generative LLMs, despite producing high-quality rewrites, face high inference latency and cost in online settings. These limitations force offline deployment, making them vulnerable to issues like information staleness and semantic drift. To overcome these challenges, we propose a novel hybrid pipeline for QR that balances efficiency and effectiveness. Our approach combines **offline knowledge distillation** to create a lightweight but efficient student model with **online reinforcement learning (RL)** to refine query rewriting dynamically using real-time feedback. A key innovation is the use of LLMs as **simulated human feedback**, enabling scalable reward signals and cost-effective evaluation without manual annotations. Experimental results on Amazon ESCI dataset demonstrate significant improvements in query relevance, diversity, and adaptability, as well as positive feedback from the LLM simulation. This work contributes to advancing LLM capabilities for domain-specific applications, offering a robust solution for dynamic and complex e-commerce search environments.

## introduction context. product search is a central component of e-commerce platforms like amazon or ebay, enabling users to discover relevant items from vast catalogs. in these platforms, users often face challenges when formulating queries, leading to suboptimal search experiences. these challenges are magnified in scenarios where users may not use precise or correct terminology, employ synonyms, or mix languages in their search phrases due to ineptitude of language proficiency. additionally, the search terms might be misspelled or overly general, making it difficult for traditional search systems to retrieve relevant products. for example, a user may search for "dress", which is too broad, while others might input "summer dress", "boho maxi dress", or "red evening gown", each reflecting different intents but lacking clarity without additional context. as ecommerce platforms continue to grow in both scale and diversity, ensuring accurate and relevant product retrieval becomes increasingly difficult, necessitating the need for advanced query rewriting techniques. query rewriting (qr) refers to the process of transforming an input query into one or more alternative queries that are semantically similar but may be phrased differently, thereby improving the likelihood of retrieving more relevant products. in the context of e-commerce platforms, effective qr is crucial for bridging the gap between user intent and the diverse ways products can be described in the catalog (figure 1 ). previous literature. query rewriting (qr) methods can be broadly categorized into discriminative and generative approaches. further details about existing work are provided in appendix 2. discriminative methods @xcite @xcite @xcite focus on reformulating queries by identifying similar terms from a pre-defined query rewriting set, leveraging sparse retrieval techniques to find relevant products. for example, using traditional information retrieval (ir) techniques, a query like "laptop under 500" might be rewritten as "budget laptop" or "cheap laptops" by detecting semantically similar phrases. while computationally efficient, these methods face critical limitations. they often struggle with long-tail queries, where reformulation sets lack appropriate alternatives, leading to inadequate or irrelevant rewrites. furthermore, their reliance on static, predefined mappings limits flexibility, particularly for queries with complex or ambiguous user intent. addressing these challenges requires a more dynamic and adaptable approach capable of handling diverse user inputs. in response to these limitations, generative methods @xcite @xcite , such as those using large language models (llms), have gained popularity due to their superior language understanding and contextual flexibility. by training on extensive corpora of query-reformulation pairs, generative models can produce diverse, contextually relevant rewrites. for instance, an llm might reformulate the query "best wireless headphones" into alternatives like "top-rated wireless earphones" or "best bluetooth headphones", potentially enhancing the coverage and relevance of search results. these methods represent a significant leap forward, offering the ability to dynamically generate novel query reformulations without relying on predefined sets. however, generative methods also have their drawbacks, particularly in real-world e-commerce applications. the large-scale nature of llms results in high inference latency and computational costs, making real-time deployment impractical. to mitigate this, llms are often deployed in an "offline" manner, precomputing query rewrites for popular searches and storing them in cache memory @xcite . while this reduces latency, it introduces issues like information staleness, as the models are not continuously updated to reflect new products, trends, or user behavior. this is especially problematic in e-commerce, where product catalogs and user preferences evolve rapidly, leading to outdated or irrelevant rewrites. these challenges highlight the need for a solution that combines the language ability of llms with a compact, efficient, and real-time adaptable framework. the online deployment of an efficient and effective query rewriting module in e-commerce search sys-tems remains a significant challenge for existing approaches. ideally, such a module should retain the strong language capabilities of an llm while being compact, resource-efficient, and practical for real-time deployment. contribution. in this paper, we propose a novel adaptive query rewriting pipeline that effectively balances efficiency and performance, addressing the limitations of current approaches. our solution employs a dual-phase training framework for a large language model (llm), integrating offline and online training. in the offline phase, we leverage knowledge distillation to create a compact and efficient student model, termed the mini e-commerce language model (minielm), distilled from a large foundation teacher model while preserving semantic fidelity. in the online phase, minielm is fine-tuned using reinforcement learning with dynamic reward signals derived from simulated user feedback. this approach not only reduces inference costs but also ensures that the model aligns with and captures relevance, diversity, and user preferences in product retrievals. a key innovation of our method is the use of simulated human feedback via llms, replacing resource-intensive manual annotations. this mechanism effectively mimics real-world deployment scenarios while enabling scalable evaluation and continuous model refinement. additionally, we introduce reward models that assess query rewrites on relevance, diversity, and coverage of user intent, ensuring comprehensive performance metrics. experimental results on the amazon esci dataset @xcite ) validate minielm's effectiveness across both offline and online stages, demonstrating its superiority over baseline methods. in summary, our contributions are as follows: ## problem statement let @xmath2, where here, t j i denotes the j th token in the i th user query. the objective of the query rewriting (qr) task is to produce a corresponding set of rewritten queries, minielm (student 𝑆) users input query (𝑄) rewritten query (𝑄 @xmath0! 𝑡 # normal deployment rl update (b) simulation of online deployment. minielm (student 𝑆) normal search engine (elastic search) input query (𝑄) rewritten query (𝑄 @xmath1 (𝑃 ! " ) llm-driven human (simulation) reward calculation module online training (c) online training. since there is no definitive ground truth for an ideal rewritten query, nor should there be-this would restrict the flexibility of potential rewrites-we instead define a set of novel metrics to evaluate the quality of a rewritten query q relative to the original query q. these metrics are computed by comparing the lists of products retrieved by the ecommerce search engine for the original query q and the rewritten query q, denoted as p q and p q, respectively. the key metrics are as follows: details on the calculation of these metrics, which serve both as reward signals and evaluation criteria, are provided in section 4.2. ## method our approach for qr employs a dual-phased pipeline that integrates offline and online training methodologies (figure 2 ). this pipeline leverages the natural language understanding of large language models (llms) while addressing efficiency and adaptability challenges through knowledge distillation and simulated user feedback. in the offline phase, we create minielm, a compact yet powerful model optimized for query rewriting, using supervised fine-tuning (sft) on a custom q2q dataset and knowledge distillation (kd) to retain semantic fidelity while reducing computational overhead. this ensures minielm inherits the capabilities of a larger teacher model while aligning with domainspecific objectives in query rewriting. the online phase then dynamically adapts minielm to prioritize relevance and diversity while evolving to reflect simulated user preferences and updates in the product catalog. together, these phases form a cohesive framework: the offline phase establishes a robust and efficient foundation, and the online phase continuously refines and personalizes the model for real-world deployment. ## experiments and results the primary goal of our experiments is to evaluate our proposed approach using the esci dataset. we begin by measuring performance across three offline metrics, followed by five online signals. the experiments demonstrate how knowledge distillation (kd) enhances query rewriting capabilities in the offline phase, while reinforcement learning (rl) improves performance across the five online signal scores. finally, we qualitatively analyze specific query rewriting tasks to highlight how the online phase further refines and improves the model. ## conclusion this paper introduces minielm, a hybrid query rewriting pipeline for e-commerce that optimizes latency, cost, and adaptability. it balances performance and efficiency through offline knowledge distillation and online reinforcement learning. experiments show improved query relevance, diversity, and user engagement. by leveraging llmsimulated interactions, minielm adapts to evolving user behavior and catalogs without costly annotations, offering a scalable, cost-effective solution for dynamic e-commerce.