# Method for Constructing a Facade Dataset through Deep Learning-Based Automatic Image Labeling
## 2 Background and Related Work
### Facade Labeling Related Research  
1. [noitemsep,topsep=0 pt]
2. [PERSON], [PERSON], [PERSON], and [PERSON] [2] improved the facade image labeling performance by using shape grammar, random walk, map classification, and machine learning methods to improve urban understanding and facade image-based 3D modeling performance. In particular, it was possible to obtain a right-angled label image through shape grammar and the labeling performance was improved using machine learning [12]. 3. [PERSON] (2012) used three layers to improve building facade-labeling performance. Each layer was algorithmized using (1) an RNN, which is a machine learning technique; (2) enhanced labeling to enhance the recognition of the facade components; and (3) structural knowledge to make the label image structurally valid [13]. 4. [PERSON], [PERSON], [PERSON], [PERSON], [PERSON], [PERSON], and [PERSON] [3] improved facade labeling performance by combining low-level classification models with medium-level object recognition models to compensate for the complicated research of labeling building facades using existing shape grammar [14]. 5. [PERSON], [PERSON], and [PERSON] [4] calculated the accuracy of the average 49.04, overall 75.23, and IoU 39.57 based on a 2D image by dividing the facade through automatic object recognition known as auto-context to label the point cloud of the building facade image [15]. 6. [PERSON] (2017) divided the facade through auto-context for quickly and accurately dividing the 2D image and 3D point cloud of the facade compared to the research published in 2015. The overall 84.4, IoU 63.7, and runtime (min) 150 accuracy were calculated [16]. 7. Research has been conducted in the field of computer vision to automatically generate facade-based 3D modeling or to improve facade-labeling performance. It is difficult to find cases where data are collected for architectural design studies. As such, previous studies on architectural elements using computer vision technology were mainly conducted to construct spatial information indoors and outdoors. In the case of building facade automatic labeling research, the main area of study in the field of computer vision was the automatic generation of 3D modeling based on the facade or improvement of facade labeling performance, and it is difficult to find a data collection for \begin{table}
\begin{tabular}{c c c} \hline \hline
**No** & **Author** & **Title** \\ \hline \multirow{3}{*}{1} & \multirow{3}{*}{[PERSON], [PERSON]} & A Study on Types of Elevation Composition and Color \\  & & Characteristics of Buildings by Urban Streetscape-Focused on \\  & & the Building of Central Aesthetic Districts in Gwang-Ju-(2006) \\ \cline{1-1} \multirow{3}{*}{2} & \multirow{3}{*}{[PERSON]} & A Study on Establishing Color Ranges of Facade on Urban \\  & & Central Street-Focusing on Buildings of Central Aesthetic \\ \cline{1-1}  & & District in Gwangju -(2007) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Research related to facade database. architectural design research. It was confirmed that studies No. 5 and 6 in Table 2 used the CMP database as training data, and these two studies recorded the highest performance. This shows that the CMP database satisfies an accurate and large amount of training data, a prerequisite for improving the building facade automatic labeling performance. Image processing methods include image classification, localization/detection, and image segmentation, as shown in Table 3[17]. In calculating the facade information, the prediction of multiple labels and the width of the object are necessary. As shown in Figure 1, semantic segmentation is most suitable for the automatic label of the facade image that requires information such as the area and number of the same class. Semantic segmentation classifies each pixel according to a specified class and does not distinguish objects of the same class, whereas instance segmentation distinguishes each object when the labels are distinguished.