# Towards Robust Representations of Spatial Networks Using Graph Neural Networks
## 4 Methods
### Results  
We present the results of our experiments in Tables 4-7. We use the macro and micro F1-score to evaluate the model performance. The macro F1-score treats all classes equally irrespective of size, whereas the micro F1-score gives uniform importance to each sample making the macro F1-score better suited for showing the effects of class imbalance. The  
Figure 3: Figure showing the network schema for the homogeneous and heterogeneous graph representations used in our experiments. To the left, is the network schema of the homogeneous representations for streets and buildings. To the right, is the network schema of the heterogeneous graph representations. results shown are the average scores over 20 runs on the test sets, we also show the standard deviation alongside. The best results for each case are in bold and second-best results are underlined. The columns GCN, GAT, GraphSAGE are the results for the homogeneous models which also serve as the baselines. The HetSpatial column is the result for the heterogeneous model. The split column indicates the size of training data used for the particular model. We make the following observations from the results. * In Table 4, which refers to the macro F1-score for building type inference, the results are not significantly impacted by the training size. Particularly for the GraphSAGE and the heterogeneous models. This provides evidence that model performance is not significantly impacted by the size of the training data for building type inference. Further, the homogeneous models seem to perform best for the building type inference task, with the heterogeneous models following closely behind. * Table 5 refers to the macro F1-score for street type inference. The heterogeneous models perform best across all training sizes for all datasets. The homogeneous models show similar performance for all training sizes. This suggests that the homogeneous models may not benefit from an increase in training data for the task of street type inference. * We add Tables 6 and 7 for completeness. We observe that the models perform sufficiently well and stable in most cases. However, there is no clear benefit to the impact of train size on model performance and the homogeneous and heterogeneous models perform relatively similar with no clear winner. Our observations demonstrate that model performance can be improved based on the data representation used. Specifically, the heterogeneous representation improves model performance for street type inference while the homogeneous representation is better suited for building type inference. To express the differences succinctly, we compute the percentage improvement derived from either the heterogeneous models or the homogeneous models using the mean macro F1-score. For the heterogeneous models, we use the mean results over the splits in Tables 4 and 5, i.e., sum the values, divided by four. For the homogeneous models, we compute the mean results using the best performing model at each split, also from Tables 4 and 5. The percentage improvement for the street type inference problem using the _heterogeneous_ models is Dublin--26.0%, Frankfurt--21.0%, Toronto--3.5%, Manchester--37.0%, where the percentage improvement for the building type inference problem using the _homogeneous_ models is Dublin--13.75% Frankfurt--9.75%, Toronto--15.0%, Manchester--22.5%. Considering class imbalance and paying attention to Table 5 which is the macro F1-score, we see a clear benefit to the heterogeneous model as it outperforms the homogeneous models for the street type inference in all cases. However, this is not the case for building type inference as seen in Table 4. We suspect that this may be as a result of the smaller number of building nodes available to the model for training compared to the number of street nodes. Perhaps, for heterogeneous representations, the size of the node types used for training could impact the inference performance for that node type. This suggests that _type imbalance_ could be a limitation of heterogeneous graph representations.