## 4 Network Architecture Design
### Training the Neural Network  
The task of training a deep neural network is the biggest setback of all the processes therein. This setback worsens as the depth of the network increases. [PERSON] et al. [39] discussed the difficulty of training deep neural networks and suggested the work around procedures to be taken during training. In this section, different methods are explored to optimize the network output. The general network training flowchart is shown in Figure 9.  
Network training is all about determining the optimal values of network parameters for which the network performs best. Neural networks are a common category of parametric nonlinear functions of the form that transforms a vector \(\mathbf{u}\) of input variables to a vector \(\mathbf{v}\) of output variables. A simple way to obtain network parameters is to make the similarity of curve fitting, and thus minimize the sum of squares error function. Given a training set containing a set of input vectors \(\mathbf{u}_{n}\), where \(n=1,\ldots,N\), together with a corresponding set of target vectors \(\top_{n}\), we calculate the minimum of the error function. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer (Type)** & **Output Shape** & **Parameter** \\ \hline Rescaling & (None, 180, 180, 3) & 0 \\ conv1 (Conv2D) & (None, 180, 180, 32) & 896 \\ max-pooling1 (MaxPooling2D) & (None, 90, 90, 32) & 0 \\ conv2 (Conv2D) & (None, 90, 90, 64) & 18,496 \\ max-pooling2 (MaxPooling2D) & (None, 45, 45, 64) & 0 \\ conv3 (Conv2D) & (None, 45, 45, 128) & 73,856 \\ max-pooling3 (MaxPooling2D) & (None, 22, 22, 128) & 0 \\ Flatten & (None, 61,952) & 0 \\ dense (Dense) & (None, 256) & 15,859,968 \\ dense-1 (Dense) & (None, 5) & 1285 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Some layers of the neural network used for object classification training. Figure 9: Flowchart of neural network training, analysis and application process. \[\xi(\mathbf{\Omega})=\frac{1}{2}\sum_{n=1}^{N}\|\mathbf{v}(\mathbf{u}_{n},\mathbf{\Omega})- \top_{n}\|^{2}. \tag{1}\]  
Considering a single target variable \(k\) that can take any real value and that \(k\) has a Gaussian distribution with an \(x\)-dependent mean, which is given by the output of the neural network, so that  
\[p(k|\mathbf{u},\mathbf{\Omega})=\mathbb{N}(\top|v(\mathbf{u},\mathbf{\Omega}),\alpha^{ -1}), \tag{2}\]  
where \(\alpha\) is the precision of Gaussian noise. For \(N\) independent data \(\mathbf{\mathcal{O}}=\{\mathbf{u}_{1},...,\mathbf{u}_{N}\}\) with corresponding target values \(\top=\{\top_{1},...,\top_{N}\}\), we can build the consistent probability function. \[p(\top|\mathbf{\mathcal{O}},\mathbf{\Omega},\alpha)=\prod_{n=1}^{N}p(\top_{n}| \mathbf{u}_{n},\mathbf{\Omega},\alpha). \tag{3}\]  
We can evaluate the error function by calculating the negative logarithm of the probability function. \[\frac{\alpha}{2}\sum_{n=1}^{N}\{v(\mathbf{u}_{n},\mathbf{\Omega})-\top_{n}\}^{2} -\frac{N}{2}\ln\alpha+\frac{N}{2}\ln(2\pi). \tag{4}\]  
Equation (4) is used to learn parameters \(\mathbf{\Omega}\) and \(\alpha\). In neural networks design literature, it is customary to take into account the minimization of the error function more than the maximization of the probability function. We are following the same approach here. First we obtain \(\mathbf{\Omega}\) by maximizing the probability function (or minimizing the sum-of-squares error function). \[\xi(\mathbf{\Omega})=\frac{1}{2}\sum_{n=1}^{N}\{v(\mathbf{u}_{n},\mathbf{\Omega})- \top_{n}\}^{2}, \tag{5}\]  
where operational constants are rejected. The value of \(\mathbf{\Omega}\) obtained by minimizing \(\xi(\mathbf{\Omega})\) represented by \(\mathbf{\Omega}_{mxl}\) corresponds to the maximum probability solution.