# A Crowdsourcing Recommendation Model for Image Annotations in Cultural Heritage Platforms
## 5 Experimental Results and Discussion
### Neural Network Classifier  
In the case of an NN classifier, the number of hidden layers is also a relevant factor that decisively conditions the final model and its results. In this work, the model of the NN classifier has been created with only one hidden layer because of two reasons: first, it is the most frequent value in the literature for binary classifiers in terms of time complexity [72; 73; 74], and second, because the size and complexity of the dataset were small enough to overcome any overfitting problem with only one hidden layer. Following the same strategy we adopted in the case of the RF classifier, a curve was plotted to compare the accuracy depending on the number of neurons used in the hidden layer to be able to select the number of neurons to obtain the best results with our dataset. A significant number of tests were carried out using the MLP classifier on an underlying neural network, which is based on optimizing the log-loss function using the stochastic gradient descend (SGD) algorithm. Regarding the activation functions of the hidden layer, the following four were tested: the _logistic sigmoid_ function, the _identity_ (or linear) function, the _rectified linear unit (relu)_ function, and the _hyperbolic tan_ (_tanh_) function. In addition, two solvers were used in order to optimize the weights of the NN: _adam_, which is an SGD optimizer, and _lbfgs_, which is significantly popular for parameter estimation when trying to minimize the activation function [75]. This way, the accuracy of different combinations of solvers and activation functions were plotted for several selections of the number of neurons for the NN hidden layer, as shown in Figures 7 and 8, where all the tests were performed with the training-testing distribution corresponding to the 70-30 split. Firstly, in Figure 7 the _logistic_ and _identity_ activation functions' curves are depicted. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Training–** & **Accuracy \%** & **Average** & **F-Score** & **F-Score** & **F-Score** \\
**Testing** & **Accuracy \%** & **Precision** & **(Macro)** & **(Micro)** & **(Weighted)** \\ \hline
60–40 & 65.8 & 0.58 & 0.630 & 0.658 & 0.630 \\
70–30 & 66.5 & 0.59 & 0.650 & 0.665 & 0.650 \\
80–20 & 66.4 & 0.59 & 0.640 & 0.664 & 0.630 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Tests results for random forest classifier. Figure 7: Testing accuracy vs. number of neurons for logistic and identity functions. As it can be seen in the plot of the _logistic_ function with the _adam_ solver, the accuracy decreases as the number of neurons increases, so a hidden layer with only one neuron was selected for the NN classifier with this configuration. When changing to the _lbfgs_ solver, the curve of testing accuracy fluctuates and decreases after eight neurons, so five neurons were considered as the selected value. Next, repeating the same process for the _identity_ activation function, we concluded that five neurons were an optimal value for the _adam_ solver, while the best accuracy was achieved for three neurons in the case of the _lbfgs_ solver. Second, a similar analysis is carried out for the results of the _relu_ and _tanh_ activation functions, and the corresponding plots are presented in Figure 8.  
The _relu_ function with the _adam_ solver plot shows that starting with three neurons, the accuracy increases, but there was no further improvement with increasing the number of neurons, so three neurons were used in the hidden layer for training the NN. Regarding the _lbfgs_ solver used with the _relu_ function, the curve shows a rise starting at two neurons, with a peak at four neurons, and no further improvement in testing accuracy is observed if the number of neurons is increased above four, so four neurons were selected as the value for this configuration. The _tanh_ function with _adam_ solver curve started at the peak at two neurons and then began to decline. In addition, with the _lbfgs_ solver, two neurons were also sufficient for training the NN.