# A New Click-Through Rates Prediction Model Based on Deep&Cross Network
## 1 Introduction  
Online advertising has become the new mode of businesses in recent years, which can make full use of customers' digital traces to achieve more precise outputs. Compared with traditional advertising, online advertising has the advantages of lower production cost, faster update and maintenance, larger information capacity, wider audience, more precise delivery, more efficient retrieval, and faster communication speed [1]. The underlying driving technology for online advertising is Click-Through Rates (CTR) estimation, in which the task is to predict the click probability of the browsers for some commodities in certain scenarios [2]. Accurate prediction of CTR will not only benefit advertisers' promotion of products but also ensure users' good experiences and interests [3]. However, the purpose of browsers for a web page is not to click the ads; thus, the overall click rates are low, with an average of about 0.25%. Therefore, the collected click rate data is sparse, which leads to the cold start problem in recommendation systems [4]. At present, the research of CTR can be generally divided into two categories: CTR based on machine learning and CTR based on Deep Neural Networks (DNN). In the early stage, the task of CTR prediction mainly depends on the statistical characteristics of data, such as the logistic regression (LR) model that has been proposed in Reference [5, 6]. And LR model is suitable for dealing with large-scale sparse features, using \(L1\) regularization for training, and can accomplish most of the click rate prediction tasks at that time. But, LR model is a linear model, and it is very dependent on manual feature engineering. So, it can not deal with complex feature combinations, and it is difficult for nonlinear tasks, too. The factorization machines (FM) model is then presented in Reference [7, 8] topredict the click rate that combines the factorization and Support Vector Machine (SVM) and can realize feature combinations by implicit vector multiplications. It has more powerful functions than LR model and has been widely used in the click rate prediction and recommendation systems. Then, a Field-aware Factorization Machines (FFM) model is introduced [9]. The concept of field is defined and integrated into the FFM model to classify features of similar properties into the same field, which can refine the representation of hidden vectors. However, FFM is still unable to achieve high-order feature combinations. And Facebook has proposed the GBDT+LR fusion model for click rate prediction, in Reference [10], which incorporates the advantages of gradient boosting tree algorithm and logic regression model. GBDT can automatically combine high-order features, and logistic regression can deal with large-scale sparse features, so as to ensure little loss of information. Then, Google proposed Follow The Regularized Leader (FTRL), in Reference [11], for more advanced regularization and generalization. The FTRL model combined the power of forward backward splitting algorithm (FOBOS) [12] and regularized dual averaging algorithm (RDA) [13] and has been successfully be used for online optimization of logistic regression model. In recent years, CTR based on Deep Neural Networks (DNN) has attracted a lot of attention from both academia and industry. For instance, a factorization neural network (FNN) is proposed in Reference [14] for CTR. The FNN can reduce the dimension of the class features, and the neural network can realize the non-linearity of the model. Google presents a Wide&Deep model in Reference [15], in which the core idea of the model is to combine the storage ability of linear models and the generalization ability of deep learning. The Product-based Neural Network (PNN) model is put forward in Reference [16] to predict the click through rate. The model first uses DNN to combine data features, and then uses product layer to further refine the cross operation of data features. The inner and outer product operations in this layer significantly improve the prediction performance of the PNN model.