## 4 Case Study
### Supervised Classification--Data Processing  
In this step, the automatic identification of the created pathologies based on the collected images and through the application of image processing tools available by means of deep learning tools (DL) is implemented. This step adopts supervised classification, in which the DL model is trained considering the representative data to classify by the user. The machine learning model was produced in Jupyter Lab, where PyTorch, fast.ai, pandas, glob and matplotlib were some of the libraries used to proceed with the model. This DL model is placed online by means of an application web on Binder. This application web is then connected to Revit by means of a plug-in, which allows the subsequent automated classification of anomalies within Revit. The dataset collected during the data acquisition represents several building anomalies and it was used to proceed with a DL model to automate the detection of anomalies in facades. In the DL model, the automatic segmentation and classification are performed by the PyTorch pipeline, where the images are pre-processed. First, all images are labeled according to their structural defects, which, in this case, correspond to 14 classes (each class corresponds to an anomaly). Then, each image is resized to the dimensions of 224 \(\times\) 224 pixels to fit within the GPU memory. The full image dataset consists of 204 images, which were subject to a random split to obtain 80% of the images as a training dataset and 20% as a validation/test dataset. Since the dataset of images was not particularly large, to avoid overfitting of the models, the dataset was enlarged using data augmentation, by applying random transformations into the training dataset, as follows:  
* random rotation in an angle between [\(-\)45; 45 degrees] with a probability of 100%;
* random horizontal flip with a probability of 50%;
* random vertical flip with a probability of 50%;
* random resized crop of the images with a scale of 0.8, 1.0 and a ratio of 1. The transformation of all images to black-and-white (BW) format was also included, to ensure that the models could learn the structural defect features presented in the images. All images were then normalized, so their pixel color values varied within the range of \(\in\) R[\(-\)1.,1.]. Afterwards, the images were converted to tensor format to be used by the CNN model. As the main backbone model for the classification task, the Residual Network (ResNet) was selected as the base architecture. ResNets were first proposed in [16] and consist of a deep neural network (DNN) built as blocks of convolutional layers together with shortcut connections (or skip layers) that help the ResNet to avoid problems associated with DNNs --in particular, the well-known gradient vanishing/exploding problem [39]. In this analysis, the discriminant power of the ResNet with an increasing number of layers--ResNet-18, ResNet-34, ResNet-50 and ResNet-101--was tested. All models were trained using the Adam optimizer [40] with an initial learning rate of 3e-4; the models were trained for 10 epochs, and the learning rate was reduced by half of its value every time the loss did not improve. This dynamic change in value for the learning rate is referred to as a reduction in the plateau. As a loss function for the model, the negative log-likelihood (NLLLoss) was used, which was appropriate to this case due to the number of classes in the dataset. To evaluate the performance of each CNN model, the F1 score metric was used. The values of the scores for each ResNet architecture are displayed in Table 2.  
The F1 score is a metric that combines precision with recall, and its value indicates the general quality of the model. By means of the architectures used--ResNet--the model had obtained more than 50% accuracy, except for ResNet-101. It is possible to notice that 34-layer/residual nets are comparably more accurate. For the architectures used and presented in Table 2, it is possible to conclude that the precision is above 50% and less than 75%.