## 4 Validation and Implementation
It is used in statistical software to understand the relationship between the dependent variable and one or more independent variables by estimating probabilities using a logistic regression equation [33]. * LightGBM--Light gradient boosting machine, is a distributed gradient boosting framework for machine learning originally developed by Microsoft. It is based on decision tree algorithms and used for ranking, classification and other machine learning tasks [34]
* Deep learning state-of-the-art such as:
* ONN--Operational neural networks (ONNs), can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. The operation-aware embedding method learns different representations for each feature when performing different operations [35]. * xDeepfm--xCreme Deep Factorization Machine (xDeepFM). A compressed interaction network-CIN (which aims to generate feature interactions in an explicit fashion and at the vector-wise level) is combined with a classical Deep NN into one unified model. The xDeepFM is able to learn certain bounded-degree feature interactions explicitly and arbitrary low- and high-order feature interactions implicitly [36]. * IFM--Input-aware factorization machine (IFM) learns a unique input-aware factor for the same feature in different instances via a neural network [37]. * DCN V2--An improved version of deep and cross network (DCN), which automatically and efficiently learns bounded-degree predictive feature interaction. DCN keeps the benefits of a DNN model, and beyond that, it introduces a novel cross network that is more efficient in learning certain bounded-degree feature interactions. In particular, DCN explicitly applies feature crossing at each layer, requires no manual feature engineering, and adds negligible extra complexity to the DNN model [38]. FiBiNet--FiBiNET is an abbreviation for 'feature importance and bilinear feature interaction network'. Proposes to dynamically learn the feature importance and fine-grained feature interactions. On the one hand, the FiBiNET can dynamically learn the importance of features via the squeeze-excitation network (SENET) mechanism; on the other hand, it is able to effectively learn the feature interactions via bilinear function [39]. \(\circ\) FLEN--Field-leveraged embedding network (FLEN) devises a field-wise bi-interaction pooling technique. By suitably exploiting field information, the field-wise bi-interaction pooling captures both inter-field and intra-field feature conjunctions with a small number of model parameters and an acceptable time complexity [28]. \(\circ\) Ensemble--Ensembling various models and combining the predictions of different models is a technique that is used by many researchers for improving results [5; 35]. In our case, an average-based ensembler has been built. In more detail, the six state-of-the-art deep learning models that are described above are used [40]. After training the models, the average of predictions from all the models is used to make the final prediction (the average probability that a user will accept the advertisement). Experiments have been conducted in both datasets. Regarding the first dataset (Avazu), the first 1 M rows were used. Data were shuffled randomly and 80% used for training and 20% for testing (the sample contained nearly 84% '0-No' and 16% '1-Yes' regarding the dependent/target variable). A technique that is followed by many researchers in this dataset when they are testing the models (also other researchers use the data without shuffling-sequentially, and the last day for testing) [28]. This approach was followed since the aim was to evaluate the models in a random dataset to test the validity of the approach. Regarding the parameters and the tuning of the models (learning rate, activation function, etc. ), the default values/fine tuning that are proposed were used. Area under the curve (AUC) and log-loss were calculated for each model [41].