## 5 Conclusions  
This paper presents an ensemble of convolutional neural networks for classification of cloud image patches on small datasets. We designed a CNN model with a suitable number of convolutional layers and fully connected layers for small datasets. The central problem of small datasets is overfitting, for which we applied two regularization techniques--L2 weight regularization and data augmentation--to avoid overfitting and to increase generalization of the model. To further improve the classification performance, we applied a model average ensemble, which separately trained several different models with the same architecture and then combined their predictions for testing. The reason why the model average ensemble works so well is that different CNN models will usually not make all the same  
Figure 11: Accuracy of the ensemble model vs. single model. The red circles show accuracy of the single models and the blue stars and curve show accuracy of the ensemble model: (**a**) Fold 1; (**b**) Fold 2; (**c**) Fold 3; (**d**) Fold 4; (**e**) Fold 5.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**No.** & **Method** & **Accuracy** & **F1 Score** & **Cohen’s Kappa** \\ \hline
1. & [PERSON] et al. [20] & 0.911 & 0.897 & 0.888 \\
2. & [PERSON] et al. [16] & 0.951 & 0.953 & 0.939 \\
3. & [PERSON] et al. [19] & 0.983 & 0.984 & 0.979 \\
4. & [PERSON] and [PERSON] [26] & 0.986 & 0.982 & 0.982 \\
5. & [PERSON] et al. [27] & 0.986 & 0.987 & 0.983 \\
6. & [PERSON] et al. [24] & 0.987 & - & - \\
7. & **Proposed method** & **0.995** & **0.993** & **0.993** \\ \hline \end{tabular}
\end{table}
Table 5: Comparison with previous publications.  
errors on the test set. The difference in errors will be more in the case of small datasets, so different models compensate for errors of the other ones. Therefore, a model average ensemble of CNN models performs better than any random selected model of its members. We experimented with SWIMCAT, which is a small dataset with only 784 images. The small dataset is also sensitive to the specifics of the training data and random initialization. To ensure that the proposed model is robust, we applied k-fold cross-validation in the experiments, and employed F1 score and [PERSON]'s kappa coefficient for performance evaluation. The results of all the experiments showed very high classification accuracy. With a minimum accuracy of 99.4% and maximum accuracy of 100%, the overall average accuracy of our model was 99.5%. Both F1 score and [PERSON]'s kappa coefficient are 0.993. The results prove that the proposed model not only achieves a high accuracy but is also robust. Compared with all other previous methods, the proposed method achieved the best results.  
Conceptualization, V.H.P.; Methodology, V.H.P.; Project administration, E.J.R.; Resources, E.J.R.; Writing-original draft, V.H.P.; Writing-review & editing, V.H.P. and E.J.R.  
This research received no external funding.  
The authors declare no conflict of interest.  
The following abbreviations are used in this paper:  
Meaning Adam Adaptive moment estimation CNN Convolutional neural networks SVM Support Vector Machine SWIMCAT Singapore Whole-sky Imaging CATegories SPD Symmetric Positive Define WAHRSIS Wide Angle High-Resolution Sky Imaging System