Optimizing adaptive marketing experiments with the multi-armed bandit

Sequential decision making is central to a range of marketing problems. Both firms and consumers aim to maximize their objectives over time, yet they remain uncertain about the best course of action. So they allocate resources to both explore to reduce uncertainty (learning) and exploit their current information for immediate reward (earning). This explore/exploit tradeoff is best captured by the multi-armed bandit, the conceptual and methodological backbone of this dissertation. We focus on this class of marketing problems and aim to make the following substantive and methodological contributions. Our substantive contribution is that we solve an important and practical marketing problem with challenges that exceed those handled by existing multi-armed bandit methods: sequentially allocating resources for online advertising to acquire customers. Online advertisers serve millions of ad impressions to learn which ads work best on which websites. However, recognizing that ad effectiveness differs by website in unobserved ways creates a methodological challenge. Our methodological contribution is that we propose a novel bandit policy that simultaneously handles attributes of ads and how their importance differs across websites (heterogeneity) to generate recommended allocations of ad impressions. We not only test this in simulation, but we also run a live field experiment with a large retail bank to improve customer acquisition rates, lowering the firm's cost per acquisition. Serving ads across websites is just one of a broader class of problems. Broadening our scope to that class, we aim to contribute a body of empirical results to better understand how key managerial issues in marketing experiments affect the performance of bandit methods. As firms become more sophisticated in their ability to test and adapt quickly, managers and researchers should understand the empirical realities of the problems and policies, such as, under what conditions do certain methods perform better than other methods. We run a numerical experiment motivated by common managerial issues to learn about these contingencies of bandit methods. Finally, while the literature spans disparate fields of research, we hope to organize the various streams of work to better guide future research using multi-armed bandit methods to solve marketing problems. Degree Type Dissertation Degree Name Doctor of Philosophy (PhD) Graduate Group Marketing First Advisor Eric T. Bradlow Second Advisor Peter S. Fader This dissertation is available at ScholarlyCommons: http://repository.upenn. edu/edissertations/694 Subject Categories Advertising and Promotion Management | Marketing This dissertation is available at ScholarlyCommons: http://repository.upenn. edu/edissertations/694 OPTIMIZING ADAPTIVE MARKETING EXPERIMENTS WITH THE MULTI-ARMED BANDIT Eric M. Schwartz A DISSERTATION in Marketing For the Graduate Group in Managerial Science and Applied Economics Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2013 Supervisor of Dissertation Co-Supervisor of Dissertation Eric T. Bradlow Peter S. Fader Professor of Marketing, Statistics and Education Professor of Marketing Graduate Group Chairperson Eric T. Bradlow Professor of Marketing, Statistics and Education Dissertation Committee Raghuram Iyengar, Associate Professor of Marketing Christophe Van den Bulte, Professor of Marketing OPTIMIZING ADAPTIVE MARKETING EXPERIMENTS WITH THE MULTI-ARMED BANDIT c © COPYRIGHT 2013 Eric M. Schwartz This work is licensed under the Creative Commons Attribution NonCommercial-ShareAlike 3.0 License To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/