# Classification of Architectural Heritage Images Using Deep Learning Techniques
## 3 Results
### Residual Networks (ResNet and Inception-ResNet-v2)
#### 3.2.1 Full Training of a Residual Network (ResNet)  
We also decided to use the original residual network developed by [PERSON] et al., of Microsoft [15], which has led to a growing adoption of this specific type of network due to its good results. The depth of the networks has a decisive influence on their learning, but adjusting this parameter optimally is a very difficult task. In theory, when the number of layers in a network increases, its performance should also improve. However, in practice, this is not true for two main reasons: the vanishing gradient (many neurons become ineffective/useless during the training of such deep networks); and the optimization of parameters is highly complex (by increasing the number of layers, it increases the number of parameters to adjust, which makes training these networks very difficult, leading to higher errors than in the case of shallower networks).  
The residual networks seek to increase the network's depth without such problems affecting the results. The central idea of residual networks is based on the introduction of an identity function between layers. In conventional networks, there is a nonlinear function \(y=H\) (\(x\)) between layers (underlying mapping), as shown on the left of Figure 5. In residual networks, we have a new nonlinear function \(y=F\left(x\right)+id\left(x\right)=F\left(x\right)+x\), here \(F\left(x\right)\) is the residual (on the right of Figure 5). This modification (called shortcut connections) allows important information to be carried from the previous layer to the next layers. Doing this avoids the problem of the vanishing gradient.  
The advantages of the residual networks are that they manage to increase the depth of the network without increasing the number of parameters to optimize, thus accelerating the training speed of very deep networks. They also reduce the effects of the disappearance gradient problem, thus improving the accuracies obtained.  
The hyperparameters used in this case are those shown in Table 4.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Momentum**} & **Initial Learning** & **Learning Rate Decay** & **Number of** & **Weight** & **End Learning** & **Batch** \\  & **Rate** & **Factor** & **Epochs Per Decay** & **Decay Rate** & **Rate** & **Size** \\ \hline
0.9 & 0.1 & 1/10 every 15,000 iter & 2 & 0.0002 & 0.0001 & 128 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Hyperparameters used in training the ResNet network.  
Figure 5: Normal Convolutional Neural Network (**left**); and shortcut connections of ResNet architecture (**right**) [15].  
Figure 6 shows the evolution of the accuracy achieved by the ResNet network using the validation images of our dataset (the number of iterations is represented in the abscissa the value of the accuracy in the ordinates).  
The accuracy values reached were 0.896 using 32 \(\times\) 32 pixel images in the training (graph shown in Figure 6) and an accuracy value of 0.930 using 64 \(\times\) 64 pixel images. Logically, the necessary training time in the case of using images of 64 \(\times\) 64 pixels was much higher (3-4 times higher than training using 32 \(\times\) 32 pixel images).