Sponsored search is at the center of a multibillion dollar market established by search technology . Accurate ad click prediction is a key component for this market to function since the pricing mechanism heavily relies on the estimation of click probabilities . Lexical features derived from the text of both the query and ads play a significant role , complementing features based on historical click information . The purpose of this paper is to explore the use of word embedding techniques to generate effective text features that can capture not only lexical similarity between query and ads but also the latent user intents . We identify several potential weaknesses of the plain application of conventional word embedding methodologies for ad click prediction . These observations motivated us to propose a set of novel joint word embedding methods by leveraging implicit click feedback . We verify the effectiveness of these new word embedding models by adding features derived from the new models to the click prediction system of a commercial search engine . Our evaluation results clearly demonstrate the effectiveness of the proposed methods . To the best of our knowledge this work is the first successful application of word embedding techniques for the task of click prediction in sponsored search . Sponsored search is a multibillion dollar market ( Easley and Kleinberg , 2010 ) that makes most search engine revenue and is one of the most successful ways for advertisers to reach their intended audiences . When search engines deliver results to a user , sponsored advertisement impressions ( ads ) are shown alongside the organic search results ( Figure 1 ) . Typically the advertiser pays the search engine based on the pay-per-click model . In this model the advertiser pays only if the impression that accompanies the search results is clicked . The price is usually set by a generalized second-price ( GSP ) auction ( Edelman et al . , 2005 ) that encourages advertisers to bid truthfully . An advertiser wins if the expected revenue for this advertiser , which is the bid Figure 1 : Sponsored ads when " pizza " was searched at Yahoo ! ( www.yahoo.com ) . price times the expected click probability ( also know as click through rate , or CTR ) , is ranked the highest . The price the advertiser pays , known as cost-per-click ( CPC ) , is the bid price for the second ranked advertiser times the ratio of the expected CTR between the second and first ranked advertisers . From this discussion it should be clear that CTR plays a key role in deciding both the ranking and the pricing of the ads . Therefore it is very important to predict CTR accurately . The state of the art search engine typically uses a machine learning model to predict CTR by exploiting various features that have been found useful in practice . These include historical click performance features such as historical click probability for the query , the ad , the user , and a combination of these ; contextual features such as temporal and geographical information ; and text-based features such as query keywords or ad title and description . Among these , historical click performance features often have the most predictive power for queries , ads and users that have registered many impressions . For queries , ads and users that have not registered many impressions , however , historical CTR may have too high a variance to be useful . Hillard et al . ( 2011 ) observed that the number of impressions and clicks recorded on query-ad pairs have a very long tail : only 61 % of queries has greater than three clicks . They also reported a drastic drop in the accuracy of the click prediction model when fewer historical observations are available . Furthermore , fine-grained historical CTR information takes a huge amount of space , which makes it costly to maintain . On the other hand , text features are always readily available , and thus are particularly useful for those cases for which there is insufficient historical information . Multiple researchers , for example ( Richardson , 2007 ; Cheng and Cantú-Paz , 2010 ) , reported the usage of text features including simple lexical similarity scores between the query and ads , word or phrase overlaps and the number of overlapping words and characters . Such features rely on the assumption that query-ad overlap is correlated with perceived relevance . While this is true to a certain extent , the use of simple lexical similarity can not capture semantic information such as synonyms , entities of the same type and strong relationships between entities ( e.g. CEO-company , brandmodel , part-of ) . Recently a host of studies on word embedding have been conducted ; all map words into a vector space such that semantically relevant words are placed near each other in the space ( Mikolov et al . , 2013a ; Pennington et al . , 2014 ; Baroni et al . , 2014 ) . The use of continuous word vectors has been shown to be helpful for a wide range of NLP tasks by better capturing both syntactic and semantic information than simple lexical features ( Socher et al . , 2012a ) . No previous research on sponsored search has successfully used word embeddings to generate text features . In this paper , we explore the use of word embeddings for click prediction . However , it is clear that conventional word embeddings ( which solely rely on word co-occurrence in a context window ) can only offer limited discriminative power because queries and ad text are typically very short . In addition , conventional word embeddings can not capture user intents , preferences and desires . Wang et al . ( 2013 ) showed that specific frequently occurring lexical patterns , e.g. , x% off , guaranteed return in x days and official site , are effective in triggering users desires , and thus lead to significant differences in CTR . Conventional word embeddings can not capture these phenomena since they do not incorporate the implicit feedback users provide through clicks and non-clicks . These observations naturally lead us to leverage click feedback to infuse users ' intentions and desires into the vector space . The simplest way to harness click feedback is to train conventional word embedding models on a corpus that only includes clicked impressions , where each " sentence " is constructed by mixing the query and ad text . Having trained a word embedding model , we simply take the average of word vectors of the query and ads respectively to obtain sentence ( or paragraph ) vectors , which in turn are used to compute the similarity scores between the query and ads . Our experiments show that this method does improve click prediction performance . However , this method has several potential weaknesses . First , the use of only clicked impressions ignores the large amount of negative signals contained in the non-clicked ad impressions . Second , the use of indirect signals ( word co-occurrences ) can be noisy or even harmful to our ultimate goal ( accurate click prediction ) when it is combined with direct signals ( impressions with click feedback ) . Third , without explicit consideration about the averaging step in the training process of word embedding models , a simple averaging scheme across word vectors may be a suboptimal . We therefore propose several joint word embedding models ; all of these aim to put query vectors close to relevant ad vectors by explicitly utilizing both positive and negative click feedback . We evaluate all these models against a large sponsored search data set from a commercial search engine , and demonstrate that our proposed models significantly improve click prediction performance . The rest of this paper is organized as follows . In Section 2 we present a brief summary of related work . In Section 3 we give some background information on ad click prediction in sponsored search . In Section 4 we describe our methods . In Section 5 we discuss our experiments . We finish with some conclusions and future directions in Section 6 . In this paper we explored the use of word embedding techniques to overcome the shortcomings of traditional lexical features for ad click prediction in sponsored search . We identified several potential weaknesses of the plain application of conventional word embedding methodologies : the lack of the right machinery to harness both positive and negative click feedback , the limited utility of pure word co-occurrence signals , and no consideration of vector composition in the word embedding training process . We proposed a set of new implicit feedback-based joint word embedding methods to address those issues . We evaluated the new word embedding methods in the context of a very good baseline click prediction system , on a large scale data set collected from Yahoo ! search engine logs . Our experimental results clearly demonstrate the effectiveness of the proposed methods . We also presented several examples for qualitative analysis to advance our understanding on how each algorithm really contributes to the improved performance . To the best of our knowledge this work is the first successful application of word embedding techniques for the sponsored search task . There are multiple interesting research directions for future work . One of these directions is to extend the vocabulary by identifying significant phrases ( as well as words ) before training word vectors . Hillard et al . ( 2011 ) employed Conditional Random Fields to divide queries with multiple words into segments and collected historical CTR on the segment level . We also like to investigate more structured embedding methods such as RNNs ( probably for ad descriptions ) . In case the computational cost of such methods are too high to be practical for sponsored search , we can employ them only for a small fraction of ads filtered by faster methods . It may be possible to deal with the implicit negative feedback of unclicked ad impressions in a more principled way by adopting ranking-based loss functions . However , this is only possible with the extra cost of identifying and aggregating related ads into a single transaction . Though not directly related to NLP , yet another promising direction is to jointly embed not only text data but also a variety of user activities ( e.g. , organic search results , mobile app usages , other daily activities ) all together in the same vector space . Since many of the different sources contain their own unique information , we might be able to obtain a much better understanding about the user state and intent through this rich joint embedding space . Joint embedding with rich information can also help us to perform automatic clustering of users , eventually leading to powerful smoothing methods for personalized historical CTR statistics .

Challenge: Conventional word embeddings with a simple integration of click feedback information and averaging to obtain sentence representations do not work well for ad click prediction.
Approach: They propose several joint word embedding methods to leverage positive and negative click feedback which put query vectors close to relevant ad vectors.
Outcome: The use of features obtained from the new models improves on a large sponsored search data of commercial Yahoo! search engine.