# Deep Learning-Based Water Crystal Classification
## 5 Experiments and Results
### Experiment Results
#### 5.3.2 Classification Model  
We trained the classification models proposed in Section 4. Stochastic Gradient Descent (SGD) with Nesterov momentum optimizer was used to update parameters, with learning rate \(\alpha=10^{-3}\), momentum \(\Delta=0.9\) and regularization \(\gamma=10^{-4}\). To enrich the dataset, we used transform techniques such as flip image (vertical and horizontal) with probability \(p=0.5\), rotating the image with a random degree in the range from \(-90\) to \(90\) degree, random cropping. The classification model is first trained with \(100\) epochs.  
As in Section 4, we used 2 kinds of pre-trained models to build classifiers: RAE pre-trained model and ImageNet pre-trained models. RAE model was trained with an unlabeled dataset, as mentioned in the previous result. With ImageNet pre-trained models, we chose the most popular deep learning models, which had won the ILSVRC: AlexNet, VGG, SqueezeNet, DenseNet, and ResNet. All parameters were adjusted during the training progress. The 5K EPP dataset was then divided into a training and a test set with a ratio of 80:20.  
After visualizing and doing statistics on the prediction, we realized that the definition with 13 categories for water crystal was not too good and gave unclear instructions. The classification model sometimes misclassified between water crystal and its spatial form. Because we used a 2D image to classify the plates, it is hard to see the differences between a plate with and without space elements. As mentioned in [30], we should apply machine learning in a problem that humans can do well. We modified the definition to eliminate ambiguity in the labeling process. The major change was combining the space misclassified categories. Additionally, we also found a new category named double plates. Finally, we delivered 12 categories, which are defined in Section 3.  
Two fully-connected layers are added on the top of modified models, followed by a ReLU activation layer. In addition to regularization, we also used the traditional dropout method to prevent overfitting problems [31]: a dropout layer is put after each fully-connected layer, except the last one. We fine-tuned parameters and the applied data transformation techniques mentioned in Section 3 to enrich the dataset.  
With the new dataset, the model has kept training with the same configuration and parameters. The new definition obtained significant performance improvement. The model could overcome the overfitting problem and obtain high accuracy for both training and test progress.  
We used different pre-trained models as back-bone and trained the model with the same parameters and set up conditions. The standard accuracy and \(F_{1}\)-score were calculated and compare among models. The results are shown in Table 3. The model trained with RAE outperforms models that used AlexNet and SqueezeNet as back-bone, with 4% higher than AlexNet and 8% higher than SqueezeNet in \(F_{1}\)-score metric. Although other pre-trained models (such as VGG, DenseNet) have high accuracy, the \(F_{1}\)-scores of Alexnet and SqueezeNet are much lower. In addition, the loss values of VGG and DensetNet are two times bigger than the lowest one (e.g., RestNet). ResNet outperforms other models in both loss value and accuracy with 98.50% Top-1 accuracy and 97.25% \(F_{1}\)-score. We concluded  
Figure 4: Reconstruct image generated by RAE model train with BCE and Spherical metric separately. The SSIM index is calculated with each reconstructed image. Spherical one outperforms the BCE one.  
that the ResNet back-bone is the best solution for our problem. When evaluating the model with test set, the ResNet accuracy is approximately 93%.