# XGBoost-Enhanced Graph Neural Networks: A New Architecture for Heterogeneous Tabular Data
## 3 Models and Methods
### Why Use GNN for TDL? Although traditional machine learning methods perform well when dealing with tabular data, they may have limitations when it comes to nonlinear relationships, high-dimensional features, or the need to consider complex associations between features. GNNbased learning methods for tabular data have also achieved state-of-the-art status in various applications such as click-through rate prediction [18], cybersecurity [19], medical risk prediction for population health records [20], and missing data input [21]. We summarize why GNN can excel at tabular data learning in the following five areas:  
1. Modelling instance correlation. When dealing with downstream tasks, it is necessary to consider not only the features of each instance itself but also the correlation between instances. The key idea is to learn high-quality feature representations of instances by modelling correlations between instances. If two instances have similar downstream labels, they may be closer in the feature space because they may share certain features or represent similar attributes. On the contrary, if two instances have different downstream labels, then they may be farther away from each other in the feature space because they may be significantly different from each other. 2. Feature interactions. In table prediction tasks, individual features may not be sufficient to adequately describe the data because there may be complex interactions between features. Traditional methods learn feature interactions by manually enumerating possible feature combinations, but this approach is time-consuming and requires domain knowledge. Deep learning methods can automatically learn feature interactions, but they usually simply connect the learned feature representations and cannot model structured correlations between features. However, GNNs can perform better in prediction tasks by learning how features interact in graph structures, naturally picking up on complex structured correlations between features, and making their own embeddings to show how features interact. 3. Higher-order connectivity. Higher-order connectivity refers to modelling complex relationships between data elements through the interaction of multiple hopping neighbors. To better learn the feature representation of data elements and improve prediction performance, higher-order connectivity between instances, between features, and between instances and features needs to be considered. In GNNs, through message passing and aggregation mechanisms, data elements can receive embeddings from multi-hop neighbors in the graph to learn more complex relationships between data elements [22]. 4. Monitoring signals. In some real-world applications, such as fraud detection, health prediction, and personalized marketing, it is challenging to collect a sufficient amount of tagged-form data. This is due to the fact that obtaining labelled data can be time-consuming and resource-intensive, often resulting in restricted access. However, graph neural networks are capable of learning without explicit instructions. This means that they can use the connections between the nodes in the graph to make better feature representations of unlabeled data, which helps with the supervised sparsity problem. We co-design the self-supervised task by combining the features and the graph structure, leveraging the semi-supervised learning property of graph neural networks. This further improves the model's performance on the supervised sparsity problem and brings new breakthroughs in tabular data learning. 5. Generalization ability. GNNs can learn to generalize what they have learned from training data. This means that even if they see nodes or graph structures, they have not seen before, they can still figure out what the results should be based on the patterns they have learned from the training data. During the testing phase, they can incorporate additional features and perform feature extrapolation to learn how to represent tabular data.