Improving Retrieval in Sponsored Search by Leveraging Query Context Signals (2024)

Abstract: Accurately retrieving relevant bid keywords for user queries is critical in Sponsored Search but remains challenging, particularly for short, ambiguous queries. Existing dense and generative retrieval models often fail to capture the nuanced user intent in these cases. To address this, we propose an approach to enhance query understanding by augmenting queries with rich contextual signals derived from web search results and large language models, stored in an online cache. Specifically, we use web search titles and snippets to ground queries in real-world information, and utilize GPT-4 to generate query rewrites and explanations that clarify user intent. These signals are efficiently integrated through a Fusion-in-Decoder based Unity architecture, enabling both dense and generative retrieval with serving costs on par with traditional context-free models. To address scenarios where context is unavailable in the cache, we introduce context glancing, a curriculum learning strategy that improves model robustness and performance even without contextual signals during inference. Extensive offline experiments demonstrate that our context-aware approach substantially outperforms context-free models. Furthermore, online A/B testing on a prominent search engine across 160+ countries shows significant improvements in user engagement and revenue.

## introduction sponsored search is a primary revenue model for many search engines, where advertisements are displayed alongside organic search results. in this model, advertisers bid on specific keywords to target user intents relevant to their business objectives. they can select various match types to determine how closely their keywords align with user queries. for instance, the exact match type restricts keyword matching to queries that precisely share the same intended meaning. in query: ad623armz reel web results: title 1: ad623armz-reel7 analog devices inc. | integrated circuits (ics) | digikey snippet 1: ad623armz-reel7 -instrumentation amplifier 1 circuit rail-to-rail 8-msop from analog devices inc . . . query profile: rewrites: ad623armz analog amplifier reel, analog devices reel of ad623 armz, ad623armz tape and reel, ... intent: the user is looking for a specific integrated circuit (ic) chip, the ad623armz, that is sold in a reel package. a reel package is a type of bulk packaging that contains many ic chips ... retrieved keywords: unity (context-free): 1. fishing rods reels and gear 2. fishing reels with rod 3. fishing rods fishing reels augmented unity (with context): 1. ad623armz reel analog devices 2. ad623armz reel7 analog devices 3. ad623armz microchip prior work and challenges: traditionally, keyword retrieval has been approached as a standard information retrieval task. some methods utilize dual encoders to map queries and keywords into a shared semantic space, with optional use of a one-vs-all classifier to rerank the shortlisted keywords @xcite @xcite . another line of research considers this problem as a constrained natural language generation (nlg) task, where language models transform queries into relevant keywords @xcite @xcite . a recent approach, unity @xcite , integrates dense retrieval (dr) and nlg methodologies into a unified model, harnessing the strengths of both while requiring only a single model. despite these advancements, existing methods struggle with short and ambiguous queries. as shown in our contribution: to address these limitations, we introduce augmented unity, a framework for context-aware retrieval in sponsored search. our approach utilizes a large dynamic cache to enrich queries with contextual signals. this cache includes organic search results, such as titles and snippets from the top-k web documents for each query. additionally, we create a query profile for each query, containing multiple rewrites and a description of the user's potential intent, generated using gpt-4, and store them in the cache. if a query is absent from the cache, an offline pipeline is triggered to generate this context, ensuring its availability for future instances of the same query within a short timeframe. our augmented unity model employs a fusion-in-decoder architecture, which enables efficient processing of diverse contexts. we train this model using a curriculum learning strategy termed context glancing, which progressively introduces more challenging scenarios with varying levels of context availability. our evaluations show that augmented unity significantly outperforms the context-free unity model by 19.9% in exact match precision at 100, while maintaining a comparable online gpu serving cost (within 7-9%). moreover, augmented unity, trained with context glancing, demonstrates robust performance even when context signals are absent, matching the performance of the unity model. through extensive online a/b testing, we show that augmented unity achieves a 1% and 1.4% increase in ad revenue for english and non-english queries, respectively, without any statistical change in ad defects. ## proposed method: augmented unity figure 1 provides an overview of our augmented unity workflow. we use a context cache to store and retrieve query context signals. for incoming queries, we first check the cache. in case of a cache miss, an asynchronous offline pipeline is triggered to generate the context signals. these signals are then used to update the cache, ensuring their availability for future occurrences of the same query. this section is structured as follows: section 2.1 details the various context signals employed. section 2.2 outlines our efficient model architecture for integrating these signals. section 2.3 introduces context glancing, our curriculum learning strategy designed to enhance model performance and robustness in scenarios with missing context signals. ## results & discussion in this section, we begin by outlining the offline experimental setup, including the datasets used and evaluation metrics in section 3.1. sections 3.2 and 3.3 discuss the core offline results, showcasing the performance gains achieved by augmented unity. we further dissect the impact of different components within our approach through detailed ablation studies in section 3.4. finally, in section 3.5, we discuss our online experiments and results. ## conclusion in this paper, we introduced augmented unity, a novel approach that leverages rich query context to enhance sponsored search retrieval. by integrating web search results and gpt-4 generated query profiles, augmented unity effectively disambiguates user intent and retrieves more relevant bid keywords. furthermore, our proposed context glancing strategy ensures robust performance even when contextual information is unavailable. through extensive offline experiments and rigorous online a/b testing on a commercial search engine, we showed substantial improvements in key metrics such as retrieval accuracy, user engagement (ad clicks), and ad revenue. these findings underscore the significant potential of incorporating contextual information for achieving more effective and efficient sponsored search retrieval. ravisri valluri, akash kumar mohankumar, kushal dave, amit singh, jian jiao, manik varma, and gaurav sinha. 2024. scaling the vocabulary of non-autoregressive models for efficient generative retrieval. arxiv, abs/2406.06739.