# TD-DNN: A Time Decay-Based Deep Neural Network for Recommendation System
## 5 Experimental Evaluation
### Selection of Hyper Parameters  
Before beginning the experiment, we must define certain parameters that will be utilized to train our model. For example, in the proposed TD-DNN model, an activation function is used to determine whether a neuron should be activated or not. It is performed by computing a weighted sum and then adding bias to it. The activation function is included in an Artificial Neural Network to aid in the learning of complex patterns that exist in the data. Some of the existing activation functions [79] are Linear, Tanh, Sigmoid, ReLu, Leaky ReLU, SoftMax, etc. In the proposed model, the ReLU activation function is used, since it does not activate all the neurons simultaneously; therefore, it is superior to the other activation functions. A regularizer [12] is utilized to penalize the layer parameters or layer activity while optimizing. A network optimizes the loss function by adding all of these penalties together. The L1 and L2 are the two regularizers, where L1 penalizes the sum of absolute values and produces multiple solutions, whereas L2 penalizes the sum of square weights and produces a single solution. Out of the two, L2 Regularizer is chosen because it can learn complex data patterns and provide more accurate predictions [11,12]. In addition, it is useful when we have features that are collinear or codependent. Apart from these, some parameters' best values have been discovered during experimentation. For example, a loss function measures the difference between the actual output and model's predicted output. It accepts two inputs: our model's output value and the anticipated output. The output of the loss function tells how well our model predicted the outcome. A higher loss value indicates that our model performed badly, and a lower value of loss indicates that our model worked well. In the experiment, the performance of Huber loss function is compared with the Absolute and Squared loss functions. Figure 3 shows the experimental findings, which indicate that the MAE and RMSE values of Huber loss are lower than the Squared loss and Absolute loss; hence, it is utilized in the proposed model. It is also deduced that the Absolute loss function performs better than the Squared loss function. Another crucial parameter is the optimizer [69] that modifies the weights of a neural network. As a consequence, it aids in the reduction in total loss while also increasing accuracy. In the proposed work, the Adam optimizer [11] is used because it is simple to  
Figure 3: Comparison between loss functions. construct, computationally efficient, takes minimal memory, and can easily handle the large data sets and parameters. Its performance is compared with the Adamax and Adagrade optimizers. The experimental findings shown in Figure 4 depict that the Adam optimizer's MAEs and RMSEs are lower than the other two optimizers; hence, it is better. In addition, determining the effective number of hidden layers is also an important parameter. A neural network [11] is made up of neurons that are organized in layers, i.e., an input layer, hidden layer, and output layer. The number of input and output layers in the network is only one, but the hidden layers can vary according to the problem. The hidden layers collect data from one set of neurons (input layer) and send it to a different set of neurons (output layer); thus, they are hidden. The advantage of adding more hidden layers is that we can allow the model to fit more complex functions. We employed our model with three, four, and five MLP layers, where the number of neurons is present as \(<\)256, 64, 8\(>\), \(<\)512, 256, 64, 8\(>\). and \(<\)512, 256, 128, 64, 8\(>\), respectively. The MAE and RMSE results of the experiment shown in Figure 5 indicate that our proposed approach achieves the lower values when MLP is set to 4. Another parameter is a learning rate [11], which depicts how well our model converges to a local minima. It determines how rapidly a network's parameters, such as weights and biases, are updated.