# Enhancing Logistic Regression Using Neural Networks for Classification in Actuarial Learning
## 1 Introduction
* In addition to the one-layer perceptron NN model extensions of the logistic regression, we construct alternative feedforward neural network-based models with many hidden layers for addressing the same classification problem. Each layer can be thought of as a possibly nonlinear function whose output is the output to the last layer. The last layer computes the prediction of the model, which, in the setup we adopt herein, is the probability that is associated with the desirable objective for a certain classification task. In particular, we consider the following deep neural network-based models: 1. The multilayer perceptron NN model. This is the feed-forward neural network with multiple hidden layers consisting of several neurons. 2. The multilayer perceptron NN model with embeddings. The embedding layers which embed categorical features into low-dimensional Euclidean spaces have weights which are learned during the representation learning phase, i.e., the calibration of the model, thus enabling us to investigate the proximity of such features to the classification task. 3. The combined multilayer perceptron NN-logistic regression model. Following the CANN approach of [5; 6], the logistic regression can be interpreted as a skip connection which is added to the neural network directly connecting the input with the output layer. The logistic regression predicts the probability of an outcome related to a specific classification task and the neural network learns nonlinear transformations and interactions between variables from the data that the logistic regression cannot capture. * Furthermore, this study investigates how the predictive performance of the proposed models can be improved for addressing the complexity of all sorts of binary classification problems in insurance and special analysis tools are introduced in order to establish their interpretability:
* Neural network architecture. The multilayer perceptron NN-based models, as it can be clearly understood, are more difficult to train and more prone to overfitting to the training data than the one-layer perceptron NN model extensions of the logistic regression. * The choice of activation functions. Regarding the one-layer perceptron NN model extensions of the logistic regression, the chosen activation function has to be
nonlinear in order to capture the stylized characteristics of the data. There are several options for nonlinear activation functions that we discuss later in the paper and the appropriate choice of suitable activation functions at each layer are crucial steps when designing a neural network architecture. Furthermore, we outline the procedure that must be followed for selecting a sufficiently large number of neurons for the one-layer perceptron NN model extensions of the logistic regression for receiving the required accuracy for a particular classification task. - Loss function. Configuring the loss or cost function for the one-layer perceptron NN model extensions of the logistic regression and the multilayer perceptron NN-based models is an important step for ensuring that the models will perform well in the intended manner since it is a measure of error between what outcome or value the models predict and what the outcome actually is. - Regularization procedures to prevent overfitting due to the complexity of the multilayer perceptron NN-based models. We consider the following regularization methods: 1. Dropout regularization which goes over all the layers in a neural network and randomly setting the probability of keeping or removing some nodes together with all of their incoming and outgoing connections, except the input and output layers which are kept the same. 2. Early stopping which is a cross-validation approach. We divide the learning data into training and validation datasets. The models are trained on the learning data and evaluated on the validation data after each training epoch.