# A Semantically Aware Multi-View 3D Reconstruction Method for Urban Applications
## 2 Materials and Methods
### Image Semantic Segmentation  
We utilized the Cityscapes dataset as the training dataset for semantic training (Table 1). Cityscapes is an open-source dataset featuring street scenes from 50 different cities, with pixel-level and instance-level annotations. The dataset comprises 5000 finely annotated images and 20,000 coarsely annotated images [30]. It categorizes urban scenes into 8 groups and 30 categories. Owing to its detailed classification, accurate annotations, and ample data volume, this dataset was the preferred choice for our semantic training. The urban semantic recognition capabilities acquired through training with this dataset served as the foundation for our semantic 3D reconstruction of complex urban scenes. The Cityscapes dataset comprises 30 categories, but the majority of training applications are aimed at providing perception capabilities for autonomous driving. Consequently, in most semantic segmentation competitions, the categories are reclassified into 19 types, with the categories marked with a \"+\" considered invalid. Among these 19 categories, greater emphasis is placed on factors affecting driving. For instance, cars are divided into as many as 8 different types, and roads are categorized into road and sidewalk. These detailed classifications are critical for autonomous driving and road perception but are overly specific for 3D reconstruction. The three categories \"ground+ dynamic+ static+\" in the \"void\" group of the original dataset include all other targets that cannot be specifically classified. In previous semantic segmentation competitions and most applications, these three categories were often not considered. However, in the 3D reconstruction process, these three categories are important parts that cannot be ignored. Therefore, we reclassified the semantic segmentation categories. The specific classifications are shown in Table 2.  
Based on the classification criteria of the groups outlined in Table 1, we added static and dynamic categories which were renamed as dynamic-other and static-other. These categories were used to accommodate other objects in urban scenes that do not fit into the \begin{table}
\begin{tabular}{c c} \hline \hline
**Group** & **Classes** \\ \hline flat & road \(\cdot\) sidewalk \(\cdot\) parking+ \(\cdot\) rail track+ \\ human & person \(\cdot\) rider \\ vehicle & car \(\cdot\) truck \(\cdot\) bus \(\cdot\) on rails \(\cdot\) motorcycle \(\cdot\) bicycle \(\cdot\) caravan+ \(\cdot\) trailer+ \\ construction & building \(\cdot\) wall \(\cdot\) fence \(\cdot\) guard rail+ \(\cdot\) bridge+ \(\cdot\) tunnel+ \\ object & pole \(\cdot\) pole group+ \(\cdot\) traffic sign \(\cdot\) traffic light \\ nature & vegetation \(\cdot\) terrain \\ sky & sky \\ void & ground+ \(\cdot\) dynamic+ \(\cdot\) static+ \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Classification of the Cityscapes dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Old Classes** & **New Classes** \\ \hline road \(\cdot\) sidewalk \(\cdot\) parking+ \(\cdot\) rail track+ \(\cdot\) ground+ & flat \\ person \(\cdot\) rider & human \\ car \(\cdot\) truck \(\cdot\) bus \(\cdot\) on rails \(\cdot\) caravan+ \(\cdot\) trailer+ & vehicle \\ motorcycle \(\cdot\) bicycle & cycle \\ building \(\cdot\) wall \(\cdot\) fence \(\cdot\) guard rail+ \(\cdot\) bridge+ \(\cdot\) tunnel+ & construction \\ pole \(\cdot\) pole group+ \(\cdot\) traffic sign \(\cdot\) traffic light & object \\ vegetation \(\cdot\) terrain & nature \\ Sky & sky \\ dynamic & dynamic-other \\ static & static-other \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Redefined semantic segmentation classes. previously defined eight categories. Additionally, we included surfaces outside of roads into the flat category and made appropriate integrations to several other categories.