# Classification of Architectural Heritage Images Using Deep Learning Techniques
## 3 Results
### Convolutional Neural Networks (AlexNet, Inception V3)
#### 3.1.1 Full Training of AlexNet Network  
For the first of the tests carried out, the AlexNet network was chosen, which is a well-known network and widely used in this type of task. It is recognized as the one that led to the resurgence of these techniques. This network was developed by [PERSON] et al. (2016), is a deep CNN architecture and was the winning model in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC-2012) ([PERSON] et al., 2016). In this challenge, the models try to classify the images into 1000 different categories (generic such as \"volcano\", \"obelisk\" or \"lemur\"). In contrast to earlier CNN models, AlexNet consists of five convolutional layers, of which the first, second, and fifth are followed by pooling layers, and three fully connected layers (a total of approximately 60 million parameters). The success of AlexNet is attributed to certain practical solutions, such as Rectified Linear Units (ReLU), data augmentation and dropout. The ReLU, which is simply a half-wave rectifier function such that \(f(x)=max\ (x;0)\), can significantly accelerate the training phase; Data augmentation is an effective way to reduce over-fitting when training a large CNN, generating more training images by trimming small patches and horizontally flipping those patches; while the dropout technique, which reduces the co-adaptations of neurons by randomly establishing the zero value at the exit of some hidden neurons, is used in fully connected layers to reduce overfitting. In short, the success of AlexNet popularized the application of large CNNs in the tasks of visual recognition, so it has become a classic architecture within the CNNs.  
Figure 1 shows schematically the network architecture used in this article (which is a slight variation of the original AlexNet network, concerning mainly the size of the input images and the number of outputs).  
Figure 1: Scheme of the AlexNet network used.  
We have evaluated several combinations of the parameters of the neural network to be tuned. The hyperparameters finally used for offering the best results are presented in Table 2.  
Figure 2 shows the evolution of the accuracy achieved during the training phase of the AlexNet network using the validation images of our dataset (with training images, accuracy not surprisingly reached 1). In abscissa, the number of iterations is represented and, in ordinates, the value of the accuracy. In the best case, this network obtained an accuracy value of 0.823 using 32 \(\times\) 32 pixel images in the training (graph shown) and an accuracy value of 0.857 using 64 \(\times\) 64 pixel images.  
The Figure 3 shows how the associated cost was reduced during training.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Momentum**} & **Initial** & **Learning Rate** & **Moving** & **Number of** & **Weight** & **Batch** \\  & **Learning Rate** & **Decay Factor** & **Average Decay** & **Epochs Per Decay** & **Decay** & **Size** \\ \hline
0.9 & 0.1 & 0.1 & 0.9999 & 350 & 0.0005 & 128 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyperparameters used in the implemented AlexNet network.  
Figure 3: Reduction of associated cost during training of the AlexNet network.  
Figure 2: Accuracy results of the AlexNet network using validation images.  
Figure 4 shows the weights of the 64 filters of the first convolutional layer of the network at the beginning (left image) and end (right image) of the training. It is usual to only visualize the first layer because this is the most easily interpretable to the naked eye. The visualization of these filters also serves as an idea of the operation of the network (the basic characteristics detected are intuited) to make a simple diagnosis of the training process and evaluate the contributions of each filter.