Automated Tuning of Ad Auctions

We present an application of counterfactual reasoning for completely automated tuning of online ad auctions. We discuss practical considerations for applying the approach on a major search engine and present results showing parity between a reliable simulation engine and our approach that is orders of magnitude more efficient computationally. 1 Online Ad Auctions Selection, pricing, and allocation of ads on search result pages is determined via an auction that runs in real time whenever a query is submitted to a search engine. In each instance, relevant ads are selected to participate and allocation and pricing rules are subsequently executed to determine the placement and click-contingent price for each ad. Varian [3] popularized an abstraction of this auction known as Generalized Second Price, whereby ads are allocated in descending order of rank score and priced such that the payment per click is equal to the minimum bid required to maintain its current position according to the rank score function. Varian describes the rank score of ad i with bid bi as Si(bi) = qibi, where qi is some advertiser dependent quality score. In this paper, we consider a more generalized form of the rank score function, parameterized by auxiliary parameters {λj}1 , that influence pricing and allocation via advertiser, user and query dependent quality scores {qj}1 , namely Si(bi|λ,q). The auxiliary parameters control the trade-off between different key performance indicators (KPIs) such as revenue (RPM), click through rate (CTR), quick back rate (QBR), and ad coverage. The parameter values can be adjusted to achieve a particular operating point. Changes in the models that feed into the auction as well as changes in the marketplace make it necessary for the parameters to be readjusted, a process we refer to as tuning. Typically, tens of experiments need to be regularly tuned per market per device per month with many markets worldwide. The number of tunings required per month and the dynamic nature of marketplace requirements makes this a daunting task. This work builds upon [1] and focuses on the practical aspects of utilizing counterfactual reasoning in an end-to-end automated ad auction tuning pipeline for a major search engine. The idea is to use a distribution over the parameters at serve time, which enables asking “what-if” questions and predicting what the expected KPIs would be given alternative marketplace parameter settings. Here, we focus on the practical aspects of this approach and refer the reader to [1] for details on the basic technical details. 2 Quantifying and Controlling Randomization Cost We make use of randomized auction parameters which are then used in conjunction with importance sampling in order to perform counterfactual estimation. For simplicity, we restrict our attention to the case with proposal N (μ,Σ). Choice of the proposal distribution p for which to sample the 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain. Figure 1: Quantifying randomization cost using Monte Carlo simulations. The choice of the proposal distribution should be guided by the desired amount of exploration and the amount of deviation from the non-randomized experiment’s KPIs that can be afforded. Randomization cost can be reduced using parameter correlations. auction parameters from affects the exploration power of the method. Proposals with larger variance will allow a larger range of counterfactuals to be considered. However, randomizing the parameters associated with a real-time auction has an impact on the overall KPIs as well, so there is a trade-off that must be considered. A larger variance will result in parameter samples further away from the current operating point. If the KPIs are not linear functions of the parameters, then the KPIs will be impacted in some way with the effect of the impact (typically) increasing with the size of the variance. The correlation structure of the parameters will impact the KPIs as well. Specifically, it is natural to correlate parameters that impact KPIs similarly so that more samples are exploring interesting regions of the counterfactual space. The corresponding joint covariance can have a different impact on the KPIs than that of an independent covariance with the same marginals as the joint. We utilized an existing auction replay and user simulation platform to obtain simulated counterfactuals in order to compute the correlation structure of the marketplace parameters, estimate the impact of randomization on the KPIs and finally decide on a covariance for online experiments. For computing the correlation structure, we simulated counterfactual KPIs for a large grid of marketplace parameters. We applied rather lose constraints on the counterfactual KPIs to eliminate points that lead to very poor operating points. We treated the parameter settings for the remaining points as possibly viable for a tuning scenario, and computed the correlation on these points. Next, we tested the impact of randomization using independent and correlated distributions with increasing variance. Let f be our KPI of interest and x our auction parameters. If the marketplace parameter distribution is p(x) and the distribution of the KPI given a parameter setting is p(f |x), then we are interested in the quantity Ep(f)[f ] = Ep(f,x)[f ] = ∫ fp(x)p(f |x)dxdf = ∫ dxp(x) ∫ fp(f |x)df = Ep(x)[Ep(f |x)[f |x]]. (1) The simulation platform performs its estimations given user queries by training user click models and applying these to simulated auctions and clicks with computationally intensive simulation models, providing a reliable estimate of the KPI: f̂(x) ≈ Ep(f |x)[f |x]. Given this approximation, we can construct a Monte Carlo estimate of the KPI with Ep(f)[f ] ≈ Ep(x)[f̂(x)] ≈ ∑