## 4 Methodology
### Machine Learning Modelling
#### 4.2.1 Proposed Deep Neural Network Model  
A machine learning model's performance is heavily dependent on its hyper-parameters and in general, the hyper-parameters are tunable, and finding an optimised value for these parameters can directly influence the performance of the model [23]. It is essential to understand that this study focuses on optimising a shallow ANN model's hyper-parameters to obtain a more accurate and useful deep learning model. On the other hand, hyper-parameters are external parameters that are set by the operator of the network [24]. For example, there are two types of hyper-parameters: Hyper-parameters related to neural network structure (number of hidden layers, dropout, activation function, etc.) and \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Index** & **Season** & **Number of Occupants** & **Hour** & **Energy Consumption (Wh)** \\ \hline
3 & 0 & 1 & 2 & 230 \\
4 & 0 & 1 & 3 & 230 \\
5 & 0 & 1 & 4 & 230 \\ : & : & : & : & : \\
99,996 & 0 & 2 & 2 & 303.92 \\
99,997 & 1 & 1 & 22 & 272.93 \\
99,998 & 0 & 3 & 18 & 784.27 \\
99,999 & 1 & 1 & 3 & 177.36 \\
100,000 & 1 & 4 & 7 & 518.85 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: The Final version of the load profile. hyper-parameters related to training algorithm (learning rate, epoch, iterations, batch size, etc.) [24]. In this study, an iterative process of fine-tuning the shallow ANN model's hyper-parameters is performed by optimising the number of hidden layers, activation function, and dropout layers to result in the proposed deep ANN model. A deep neural network, also known as a multi-layer neural network, has more hidden layers than a shallow one. Which enables the deep neural network models to learn more abstractions relationships within the input data and how the features interact with each other on a non-linear level [25]. Hidden layers are the layers of neurons in between the input layer and the output layer. Increasing the number of hidden layers increases the network model's ability. However, there is a limit to the number of hidden layers added before its effectiveness declines. Optimising this value is a challenging task in creating deep neural network models, and in this proposed model, the optimised number of hidden layers was roughly around six. Besides that, two dropout layers were also added. Dropout layers are the layers that randomly \"kill\" a certain percentage of neurons in every training iteration to ensure some information learned is randomly removed, reducing the risk of over-fitting the data during the training phase [24]. Having the right combination of hidden and dropout layers in ANN makes it a useful prediction model, and in this case, we are calling this developed model a deep ANN. Additionally, the activation function is a set of rule that determines if a neuron should \"fire\" or not [23]. There are many types of activation functions, and each one is suited for distinctive situations. For example, a sigmoid function returns an output of \"1\" when the neuron's input is one or higher. Similarly, it produces a negative one when the input is below the negative one and returns the same value to the input when the input is between \"\(-\)1\" to \"1\". Rectified Linear is another activation function. This function outputs 0 when input is negative, while the output matches the input when input is positive. In this study, the sigmoid function was used for the network's hidden layers, while rectified linear function was used for the output layer. Apart from the modification mentioned above, the number of neurons was also varied and tested to find an optimum neuron number for the effective deep ANN model. Ultimately, the study is aimed to introduce a new optimised and finely tuned deep ANN model. Figure 6 below depicts the improvements being made on the neural network to transform it from a shallow ANN model to a deep ANN model, while Figure 7 demonstrates the architecture of the optimised deep neural network. Figure 5: Data-flow of machine learning with the input and output choices.