# Simulation of Calibrated Complex Synthetic Population Data with XGBoost
## 2 Synthetic Data Generation
### Synthetic Data Validation  
Evaluating the quality of simulated synthetic data is a crucial step to ensure that the generated data are useful, reliable and fit for their intended purpose. The primary dimensions and methods used to assess synthetic data quality from an \"absolute\" perspective are the following:  
* **Statistical similarity** measures how well the synthetic data replicate the statistical properties of the real data. Key aspects include descriptive statistics, distributional properties and correlation structures.
* **Data utility** measures how well synthetic data can be used in place of real data for specific tasks. This includes the predictive performance by training the machine learning models on the synthetic data and evaluating their performance on real data. Moreover, models trained on synthetic data should generalize well to real data, indicating that the synthetic data capture the essential patterns and relationships.
* **Integrity and consistency checks** ensure that the synthetic data maintain logical and business rules inherent in the real data:
* While synthetic data should closely resemble real data, they should also introduce some **variability**, e.g., ensure the synthetic data contain new combinations of featuresthat are plausible within the context of the real data but were not present in the original data set.  
In the following, we concentrate on statistical similarity and utility, indicating that variability is guaranteed by the simPop framework and consistency is mainly modeled by design (see Section 2.2).  
The newly proposed XGBoost synthetic generation method was compared to the multinomial approach developed by [14]. Both methods generated 100 synthetic versions of the Austrian European Union Statistics on Income and Living Conditions (EU-SILC) survey from 2019. First, a single generated synthetic population is examined, and second, all 100 synthetic populations together are compared. The mosaic plots in Figure 3 show the comparison for a single synthetic population for the two categorical variables, marital status (P114000) and chronic disease (P103000). The red areas show a lower than expected group frequency, so under-representation of the respective group, whereas the blue areas indicate an over-representation. The XGBoost approach shows promising results, as the overall deviation from the sample is smaller than that of the multinomial approach. As this is only an excerpt of one combination of one generated population, Figure 4 shows a comparison across all 100 synthetic populations. The color scale ranges from white (small mean chi-square test statistic) to red (large mean chi-square test statistic) for the variable combinations. As seen for the single population comparison, the multi-run comparison confirmed the good performance of the XGBoost approach.  
Table 1 shows the performance across all generated variables, namely, the self-defined current economic status (P001000), citizenship (P111010 nu), marital status (P114000) and  
Figure 4: Comparison between the average of all 100 synthetic population and the original survey data by the residual colored mosaic plot of the marital status (P114000), and chronic illness (P103000). The label -2 regards to non-selected persons and \(*\) to missing values.  
Figure 3: Comparison between a single synthetic population and the original survey data by the residual colored mosaic plot of chronic illness (P103000). The label -2 regards to non-selected persons and \(*\) to missing values.  
highest level of education achieved (P137000) in all synthetic populations, and chronic illness (P103000). The XGBoost approach outperforms the multinomial for all three performance metrics: mean squared error (MSE), mean absolute error (MAE) and the \(\chi^{2}\) test statistics.