Leveraging Bidding Graphs for Advertiser-Aware Relevance Modeling in Sponsored Search (2021)

Abstract: Recently, sponsored search has become one of the most lucrative channels for marketing. As the fundamental basis of sponsored search, relevance modeling has attracted increasing attention due to the tremendous practical value. Most existing methods solely rely on the query-keyword pairs. However, keywords are usually short texts with scarce semantic information, which may not precisely reflect the underlying advertising intents. In this paper, we investigate the novel problem of advertiser-aware relevance modeling, which leverages the advertisers’ information to bridge the gap between the search intents and advertising purposes. Our motivation lies in incorporating the unsupervised bidding behaviors as the complementary graphs to learn desirable advertiser representations. We further propose a Bidding-Graph augmented Triple-based Relevance model BGTR with three towers to deeply fuse the bidding graphs and semantic textual data. Empirically, we evaluate the BGTR model over a large industry dataset, and the experimental results consistently demonstrate its superiority.

## introduction large commercial search engines typically provide organic web results in response to user queries and then supplement with sponsored ads. advertisers can bid on keywords so that their ads show up when people are looking for exactly the kind of things they sell. as the fundamental component of sponsored search systems, relevance model measures the semantic closeness between an input query and a candidate keyword, which is capable of improving the user experience and driving revenue for the advertisers @xcite . existing relevance models can be roughly categorized into two sets: one-tower and two-tower struc-apple pie merchant apple pie target.com good apple pie merchant apple pie delish.com bad salt water fishing salt water fish amazon.com good salt water fishing salt water fish petco.com bad different from traditional relevance models which solely rely on the query-keyword pairs, in this paper we investigate the novel problem of triple-based (i.e., query-keyword-advertiser) relevance modeling. two critical challenges need to be addressed. firstly, existing approaches usually learn the keyword representations by encoding the semantic text, which obtain identical embeddings for the same keywords. however, as discussed above, same keywords may have different meanings given different advertisers (e.g., keyword "apple pie" bid by "delish.com" is more similar to "apple pie recipe"). hence, the keyword representations should be advertiser-aware. secondly, how to learn the desirable representations for the advertisers is not straight-forward. information from the domain url is too obscure to indicate the intrinsic features of the advertisers (e.g., it is troublesome to interpret the url "indeed.com " as a job seeking website literally). the homepages are full of various html elements and commodities, and extracting useful information from such noisy data is non-trivial. external knowledge graph (e.g., freebase) may also not be a good solution as many small businesses are not included, leading to the comparatively lower coverage rate. in this paper, we propose to leverage the bidding behaviors of advertisers to learn the quality representations beyond the semantic texts. as shown in the left part of figure 1 , orders are placed by advertisers to the search engine, which contain a set of keywords belonging to the same category. three components (advertisers, orders and key-words) can naturally form two types of bidding graphs: the co-order graph and the ad-keyword graph. for each advertiser, we construct a homogeneous co-order graph, in which nodes are the keywords bid by this advertiser and the edges denote the co-order relationships. these co-order graphs facilitate the learning of advertiser-aware keyword representations. for example, as shown in figure 1 , with the co-order keywords "apple pie menu" and "pie recipe", we can understand the keyword "apple pie" bid by "delish.com" refers to recipes. the ad-keyword graph is a bipartite graph contains two types of nodes: advertisers and keywords, in which nodes are connected by the bidding behaviors. our insight lies in the phenomenon of homophily as advertisers with similar bid keywords are also tend to be similar, which can be leveraged to learn quality advertiser representation with high converge rate. based on these observations, we further propose a bidding-graph augmented triple-based relevance model bgtr, which includes three towers: the query encoder, the keyword encoder and the advertiser encoder. bgtr model is capable of deeply fusing the semantic textual information and the bidding graphs. experimental results on the large industry dataset demonstrate that our proposal can effectively improve the performance of relevance modeling. we summarize the main contributions of this paper as follows. ## problem definition in this section, we will formally define the studied problem. different from traditional querykeyword based methods, here we introduce the definition of "advertiser" to form up the triple: @xmath0, k i , a @xmath1, in which q i denotes the input query, k i denotes the keyword and a i is an advertiser who bid the keyword k i . for each advertiser a i , its corresponding co-order graph is defined as ## experiments in this section, we extensively evaluate the proposed bgtr model over an industry dataset. in section 4.1, we present the statistics of the dataset and training details. then we go through several sota baseline models in section 4.2. section 4.3 exhibits the overall performance of bgtr and baseline models. section 4.4 conducts two ablation studies to investigate the effectiveness of different gnn aggregation strategies and the disentangled advertiser encoder part. finally, we study the performance sensitivities of bgtr on the neighbor sampling strategies and the neighbor numbers. ## related work in this section, we will briefly summarize the related works of relevance modeling in sponsored search. traditional methods like lsa @xcite , lda @xcite and bi-lingual topic models @xcite seek to mapping sentences to low-dimensional continuous vectors using shallow language representation models. then the similarity can be calculated on this latent space. in recent years, with the success of deep learning in nlp area, deep semantic models, especially the siamese structure models is adopted in a range of works @xcite @xcite @xcite . @xcite presents a deep semantic similarity model for recommending target documents to be of interest to a user based on a source document that she is reading with special convolutional-pooling structure. some interactionbased structure @xcite @xcite are also proven to be useful in relevance modeling. yang et al. @xcite propose an attention-based neural matching model with value-shared weighting scheme for combining different matching signals. @xcite employs a joint deep architecture at the query term level for relevance matching to bridge the gap between semantic matching and relevance matching. mitra et al. @xcite propose a document ranking model composed of two separate deep networks that that matches the query and the document on separate representations. bai et al. @xcite propose query n-gram embedding to improve the modeling of query-ad relevance. grbovic et al. @xcite propose a real-time personalization in search ranking and similar listing recommendations using listing and user embedding techniques. huang et al. @xcite design a unified embedding framework to model semantic embeddings for personalized search with various tricks including ann parameter tuning and full-stack optimization. ## conclusion in this paper, we thoroughly study the novel problem of advertiser-aware relevance modeling . the bidding behaviors of advertiser are incorporated to provide complementary information beyond the semantic texts. we propose a triple-tower based model bgtr to deeply fuse the bidding graphs and the semantic information. our proposal is extensively evaluated over an industry dataset, and the results demonstrate the superiority of the bgtr model.