# Ad Creative Discontinuation Prediction with Multi-Modal Multi-Task Neural Survival Networks
## 1 Introduction  
With the increasing importance of digital advertising, the significance of ad operations has also increased. Ad operations consist of various tasks, such as preparing ad creatives, determining the bid price and the target audience, and discontinuing inefficient ad creatives. These operations are essential for business revenue; however, operators' abilities are dependent on their experience. Supporting ad operations is an important topic of study in the field of machine learning (ML) [1, 2, 3, 4, 5]. Discontinuing ad creatives at the appropriate time is a crucial operation and can have a significant impact on sales. However, to the best of our knowledge, no studies have been conducted so far that support these operations. To support ad operations, we address the discontinuation of ineffective ad creatives based on ML methods. Ad creatives are generally discontinued manually by ad operators when they become less effective. Considering that ad-serving algorithms (e.g., bid price optimization [3, 4]) are required to serve a large number of effective ad creatives,multiple studies have been conducted to develop algorithms that can provide accurate predictions [6; 7]. However, we posit that predicting ad performance for operational efficiency requires other mechanisms because the utilization of the results is different from efficient ad serving. In other words, in this study, we focus on predicting the effectiveness of the ad creatives so as to properly discontinue ineffective ones. Pre-analyzing 1,000,000 real-world ad creatives, we found two types/patterns of discontinuation: short-term and long-term. These discontinuations were empirically identified as falling into two categories: (a) _cut-out_ (i.e., for short-term discontinuation) and (b) _wear-out_ (i.e., for long-term discontinuation). In cut-out discontinuation, ad creatives are discontinued after a short period of time due to incompatible serving effects, whereas in wear-out discontinuation, ad creatives are discontinued when ad creatives' performance deteriorates after a long time. Ad operators need to take different actions for each type of discontinuation. We clarified these empirical differences by analyzing real-world data (described in detail in Section 3). To address the discontinuation of the ad creatives that have an impact on business revenue, we propose a discontinuation prediction framework based on the real-world data; this is the first piece of work focusing on ineffective ad creatives, rather than effective ones. Our framework is a multi-modal deep neural network (DNN) with multiple ad creative features (e.g., text, categorical, image, numerical features) as inputs and incorporates a hazard function, which is frequently used in survival prediction, into a loss function (It is controversial to solve ad discontinuation prediction as a survival prediction. In this paper, we do not elaborate on this topic but only discuss the idea of the hazard function. We intend to tackle this issue in our future studies). To improve the prediction performance for the two different types of discontinuations and for the ad creatives that contribute to sales, we introduce two new techniques: (1) a two-term estimation technique with multi-task learning (MTL) [8] and (2) a click-through rate (CTR)-weighting technique for the loss function (described in detail in Section 4). Using the large-scale ad creative dataset, we practically evaluated the proposed framework and techniques with various metrics such as concordance index (CI) [9], the F1 score, and operational performance measures (described in detail in Sections 5 and 6). The contributions of our study are summarized as follows:  
* **Analyzing large-scale ad creative dataset**: We analyzed 1,000,000 real-world ad creatives, including 10 billion scale impressions, and identified two important aspects. (a) There are two reasons for the discontinuation of ad creatives: cut-out (for the short term) and wear-out (for the long term).