D eep G en: Diverse Search Ad Generation and Real-Time Customization (2022)

Abstract: Demo: https://youtu.be/WQLL93TPB-cAbstract:We present DeepGen, a system deployed at web scale for automatically creating sponsored search advertisements (ads) for BingAds customers. We leverage state-of-the-art natural language generation (NLG) models to generate fluent ads from advertiser’s web pages in an abstractive fashion and solve practical issues such as factuality and inference speed. In addition, our system creates a customized ad in real-time in response to the user’s search query, therefore highlighting different aspects of the same product based on what the user is looking for. To achieve this, our system generates a diverse choice of smaller pieces of the ad ahead of time and, at query time, selects the most relevant ones to be stitched into a complete ad. We improve generation diversity by training a controllable NLG model to generate multiple ads for the same web page highlighting different selling points. Our system design further improves diversity horizontally by first running an ensemble of generation models trained with different objectives and then using a diversity sampling algorithm to pick a diverse subset of generation results for online selection. Experimental results show the effectiveness of our proposed system design. Our system is currently deployed in production, serving ~4% of global ads served in Bing.

## introduction search advertising is the largest segment of digital advertising for its projected @xmath0515b market share worldwide in 2022 @xcite . traditionally, advertisers manually create ads for their web pages to start an advertising campaign. there is a growing need to automate this process, either to lessen the burden for small and medium businesses, or to create millions of ads for large businesses that have lots of products. a classical automated ad generation system relies on extraction rules as described in section 2.1, figure 1 : an example of an ad copy (grey box) comprised of ad assets. red box is used for ad title assets, and green box is used for ad description assets. this ad could be shown for search query "surface 8". for example, extracting key phrases from advertiser's web pages as ad titles. however, per our experience, extraction-based methods are not very successful in generating the much longer ad description. refer to figure 1 for the example ad title and description assets. therefore, we aim to generate ads in an abstractive fashion. in this work, we focus on improving ad performance from two aspects: factuality and customization. to achieve the optimal ad performance, our current system creates a customized ad in real-time in response to a user's search query. as shown in figure 2 , different ads are displayed for different queries, although they are advertising the same web page. we dynamically customize ad copies by stitching the generated ad assets together given the user's search context, approximating the ultimate goal of real-time customized generation. our work makes the following contributions: url: https://www.surveymonkey.com/mp/online-polls ad asset preparation all generated assets semantically diverse assets ad copy serving customized ads created based on user query extracted landing page text bing document understanding generation model generation model semantic de-duplication real-time ad stitching create poll online survey template how to create an online poll -surveymonkey audience https:/ / www.surveymonkey.com/ polls/ surveys ad create simple to sophisticated surveys online. measure & act on feedback.. give your teams the data they need to make smarter decisions create a poll for free -customizable survey templates https:/ / www.surveymonkey.com/ polls/ surveys ad are you looking for a fast way to get feedback? get instant feedback with online polls search engine user search engine user ## ads generation system our system for ad content generation and stitching is automated end-to-end as shown in figure 2 . advertisers only need to supply us with their domain names, landing page targeting rules, and a bid for each rule (e.g., bid $0.5 for urls containing "shoes"). our search indexing infrastructure crawls all landing pages under advertiser domain names that match targeting rules and runs the document understanding (du) pipeline to extract textual information as per section 2.1. after that, we run multiple nlg models concurrently. this parallel design enables us to scale modeling horizontally: we can add or remove generation models at will. the models can either generate an ad asset or a full ad copy. for a full ad copy we simply split it into assets. at the end of generation stage, we have many title and description assets generated for each advertiser url. ## a/b testing deepgen is deployed globally to serve dynamic search ads (dsa), which accounts for ∼ 4% of all bing ads displayed globally. in a/b testing, we split the production user traffic randomly between the treatment experiment that enables the proposed experimental techniques and the control experiment that uses existing production techniques. we use 10% of production traffic for the control experiment. we use the difference in business metrics between two experiments to decide if treatment is effective. two key business metrics are revenue per mille (rpm) -revenue per every thousand search result page views (srpv) and quick back rate (qbr) -the rate of users clicking the back button after clicking on an ad, which is a proxy for user dissatisfaction (lower qbr is better). rpm is driven by impression yield (iy, number of ads shown divided by number of search result page views) and click-through rate (ctr, number of clicks divided by metric exp. 1 exp. 2 explanation days 5 10 number of days for the experiment. traffic% 5.0 10.0 percentage of the bing user traffic allocated for the experiment. ∆rpm% +24.87 +10.65 revenue (usd) from 1000 search results page views (srpvs). ∆iy% +11.87 +14.43 average number of ads shown per page. ∆ctr% +13.28 -0.19 proportion of ads clicked from ads shown. ∆qbr% +5.27 +1.82 proportion of ad clicks that resulted in a back-click within 20 sec. in exp. 1, we compare deepgen (treatment) against the extraction system (control). as shown in we use exp. 2 as an ablation for real-time customization. deepgen is used in both treatment and control, but we replace real-time stitching with precomputed stitching in control. for this experiment, we build a separate model to stitch assets into multiple ad copies offline, and only rank the pre-stitched ad texts during query time (online). there is significant rpm (revnue) gain, though it is mainly driven by iy but not ctr. this may suggest that online stitching has a higher chance of winning the auction as it covers much larger permutation space than the offline stitching. but for those ad copies that did win an auction, they have similar attractiveness to the user whether stitched online or offline. this experiment shows online stitching to be an integral part of our system. thus, deepgen increases revenue by generating high-quality ads customized to the user while being mindful of user satisfaction by ensuring the ads to be faithful to the landing page. ## related work the early automated content generation approaches focused on template-based ad text generation @xcite @xcite . these approaches have potential to suffer from ad fatigue @xcite . more recently, deep reinforcement learning (rl) was shown effective for ad text generation @xcite @xcite , using a general attractiveness model as a reward policy and yielding up to 7.01% observed ctr gain per @xcite . ctr is an important metrics, as reflects the relevance of an ad from user's perspective @xcite . product headline generation is a closely related direction of work, where a single headline is generated to advertise a line of related products, based on each product's advertiser-written title. @xcite use bert-large @xcite encoder finetuned for generation with unilm-like masked attention, as per @xcite , optimized using a self-critical rl objective, as per @xcite . @xcite further produce sc-cobart by finetuning a bart model, using control codes, as per @xcite , for bucketized ctr and length of a headline, optimized with a mixture of mle and selfcricial rl objectives. sc-cobart improves estimated ctr by 5.82% over their previous work @xcite . in another line of work, product descriptions are generated either with templates @xcite , pointer-generator encoders @xcite , commonsense knowledge-base guidance @xcite , or cvaes @xcite , yielding up to 13.17% ctr gain in a/b test per @xcite . ## conclusion in this work, we present an automated end-to-end search advertisement text generation solution. we employ deep nlg modeling for ad content generation and diverse selection. we leverage real-time lr rankers for content stitching. the generation techniques provide us a rich source of high-quality ad content, which performs strongly against human and extraction baselines. we further apply diverse selection via semantic embedding, which allows us to surpass human content diversity, while ensuring the system's scalability. finally, we use real-time ranking to stitch not just attractive, but a truly customized ad for each user based on query and search intent. the system combines several nlp approaches to provide a cutting edge solution to automated ad generation and showcases an significant ctr gain over an extraction baseline.