# Simulation of Calibrated Complex Synthetic Population Data with XGBoost
## 2 Synthetic Data Generation
### Synthetic Data Generation with XGBoost
#### Sequential Approach to Simulate Variables of a Data Set Using XGBoost  
After the simulation of the household structure of the population, other variables can be generated (cf. Figure 1, point 2). An additional variable is simulated using the (simplified) scheme below. In the XGBoost training step, one response variable of the original survey \(\mathbf{S}\) is predicted using variables \(\mathbf{x}_{,1}^{S},\ldots,\mathbf{x}_{,j}^{S}\) as a predictor matrix with \(n\) observations and \(j\) dimension. The predictors must already be simulated in the synthetic data set.  
\[\mathbf{S}=\ \overbrace{\left(\begin{array}{cccccc}\overset{\text{predictors}}{x _{1,1}^{S}}&\overset{\text{predictors}}{x_{1,2}^{S}}&\cdots&x_{1,j}^{S}& \overset{\text{response}}{x_{1,j+1}^{S}}&\overset{\text{additional}}{x_{1,j+2}^{S} }&\cdots\\ x_{,1}^{S}&x_{2,2}^{S}&\cdots&x_{,j}^{S}&\overset{\text{response}}{x_{2,j+1}^{S} }&\overset{\text{additional}}{x_{2,j+2}^{S}}&\cdots\\ \vdots&\vdots&\ddots&\vdots&\vdots&\vdots&\vdots\\ x_{n,1}^{S}&x_{n,2}^{S}&\cdots&x_{n,j}^{S}&\overset{\text{response}}{x_{n,j+1}^ {S}}&\overset{\text{additional}}{x_{n,j+2}^{S}}&\cdots\end{array}\right)}^{ \text{predictors}} \tag{3}\]  
In the synthesis step, the new variable in position \(j+1\) is generated for the synthesized population \(U\) with observations \(N\) using a prediction with the XGBoost parameters formerly trained.  
and the MSE is used to compare the discrepancies between the expected \(t_{i}^{S}\) and the realized \(t_{i}^{U^{\prime}}\) frequencies.  
\[\text{MSE}=\frac{1}{n}\sum_{i=1}^{n}(e_{i})^{2} \tag{7}\]  
The hyperparameters with the lowest average MSE are then used to generate the variable for the synthetic population \(U^{\prime}\).  
A grid/combination of various hyperparameters can be set for which the optimal combination is then selected. Such hyperparameters are, for example, the maximum depth of a tree, the learning rate and the evaluation metric. A complete list is provided here [[https://xgboost.readthedocs.io/en/latest/parameter.html](https://xgboost.readthedocs.io/en/latest/parameter.html)]([https://xgboost.readthedocs.io/en/latest/parameter.html](https://xgboost.readthedocs.io/en/latest/parameter.html)) (accessed on 6 May 2024).  
```
1:Use X the data to be synthesized
2:choose one of the following:
3:Option 1: set simPop \(\leftarrow\) result from Algorithm 1
4:Option 2: set simPop \(\leftarrow\) result from Algorithm 1 plus already simulated additional variables with Algorithm 2.
5:set availlevars \(\leftarrow\) variables of simPop
6:set newvar \(\leftarrow\) variable to be simulated in the synthetic population
7:Define model parameters for the XGBoost algorithm
8:\(\bullet\) set max.depth \(\in[0,\infty]\)\(\triangleright\) maximum depth of a tree. 0: no limit on depth.
9:set eta \(\in[0,1]\)\(\triangleright\) learning rate. The higher the more conservative boosting.
10:set gamma \(\in[0,\infty]\)\(\triangleright\) minimum loss reduction required
11:set \(\ldots\)\(\triangleright\) additional hyperparameters
12:set objective \(\leftarrow\) \"multi:softprob\" for categorical response und squared error fur continuous response as default (can be overwritten)
13:Initialize xgboost_params with max.depth, eta, gamma, \(\ldots\), objective
14:fitted_params \(\leftarrow\) train XGBoost (X, availlevars, newvar, model_params). Fit on the data to be synthetized using the same variables as contained in simPop and newvar as shown in Equation (3). \(\triangleright\) Hyperparameter training with cross validation
15:update simPop \(\leftarrow\) simulate variable (simPop, availlevars, newvar, xgboost_params, fitted_params) \(\triangleright\) Demonstrated in Equation (4)
```  
**Algorithm 2** Simulate an Additional Variable using XGBoost  
Appendix C.3 shows a selection of hyperparameters and the corresponding cross-validation procedure with the help of the R package simPop.