# Vegetation Removal on 3D Point Cloud Reconstruction of Cut-Slopes Using U-Net
## 3 Methodology
### U-Nets  
The deep learning methods used for the land usage investigation showed a good performance [9; 10]. It proved that many deep learning models were suitable for vegetation recognition. In this study, the classification problem is not complex (vegetation or non-vegetation) and the aim is to find a deep learning model requiring a smaller dataset whilst keeping details of images. U-Net (Figure 13) was firstly used in medical image segmentation, the name comes from the encoder-decoder shape. It was invented for a small dataset and was well trained with only 30 images [30]. The skip connections between the encoder and decoder keep details of the input images. In U-Net, the encoder part is similar with some image classification models and, thus, U-Net can be built based on an image classification model. The pre-trained weight of an image classification model can be used for transfer-learning reducing time-consumption for model training. Two image classification models, VGG16 and MobileNetV2, were used to build U-Nets and the results were compared. VGG16 uses a smaller convolution kernel, which helps extract complex features [31], whilst MobileNetV2 was designed for mobile applications with a smaller model size, resulting in a reduced amount of computation [32]. Figure 11: Example of image segmentation strategy in the Cityscapes dataset. Figure 12: Examples of manually trimmed vegetation: (**a**) Original images, (**b**) Colour index mask, and (**c**) Manually trimmed mask  
For training and testing, a dataset of 70 images was used: 35 images taken from summer and the other 35 images from winter. The images from winter were selected to cover the situations in Figures 9 and 10 and were all manually trimmed. The images from summer were randomly selected and segmented successfully by \(mExG\) only. To make full use of the 70 segmented images, each \(4000\times 3000\) pixels image was cut into 25 images of \(800\times 600\) pixels. In addition, horizontal-flip data augmentation was used so that the 70 images were augmented as \(3500\) (\(=70\times 25\times 2\)) images. Among them, 70% (2450 images):30% (1050 images) were used as training and test datasets respectively. The augmented images were resized to \(512\times 512\) pixels to be fed to the U-net. Performance metrics used in training/testing were (1) the binary accuracy on how often predictions equal to labels, and (2) the binary cross-entropy loss function between true and predicted labels. The batch size was 16, and the initial learning rate was \(10^{-4}\) with the Adam optimiser. Figure 14 shows the training history with the two metrics. Figure 14 shows that the accuracy on the training dataset was close to the accuracy on the test dataset. This was the same for the binary cross-entropy loss index as well. Both the accuracy and binary cross-entropy loss plots showed converging trends. U-Nets based on VGG16 and MobileNet V2 reached approximately 95%  
Figure 14: Model training histories for binary accuracy (**top**) and binary cross entropy loss (**bottom**). Figure 13: Structure of the VGG16-based U-Net architecture. after 16 training epochs, respectively. The binary cross-entropy loss of VGG16 U-Net is much lower than MobileNetV2 U-Net. U-Net based on MobilenetV2 took approximately 1/3 training time of that based on VGG16, and the model size was less than half of it. Meanwhile, the binary accuracy difference between the two U-Nets was about 5%. Figure 15 shows the vegetation segmentation results of \(mExG\), and the U-Nets with VGG16 or MobileNetV2, both with 50% confidence level and above. The first row in Figure 15 shows the original images taken in different locations and seasons. The second row shows the mask obtained with the colour index method and the third row were the manually trimmed results. In the third row, the first two images were the high-quality results obtained by the colour index method only, whilst the last two images were manually trimmed based on the masks generated with the colour index method.