# Interpretation for Variational Autoencoder Used to Generate Financial Synthetic Tabular Data
## 3 Materials and Methods
### Experiment Preparation
#### 3.5.2 Application Examples  
To test the methodology outlined in Sections 3.3 and 3.4, three real-world datasets were downloaded from an open source Kaggle, which are publicly available. This study concentrates on tabular banking data, making the tabular data as the chosen data type for examining the effect of each input feature on the latent space when VAE is used to synthesize data. Dataset 1 comprises 10,000 rows representing various data samples (clients) and 13 columns representing different client information. This dataset is divided into two classes based on whether clients will retain their bank accounts. Dataset 2, from a marketing campaign of a Portuguese banking institution, includes approximately 41,000 data samples and 20 columns. This dataset is categorized into two classes based on whether clients will subscribe to a term deposit. Dataset 3 comprises approximately 164,000 rows and 14 columns. This dataset is grouped into 3 categories based on the customer's interest rates for loans, 1%, 2%, and 3%. The number of data samples in each category for the three datasets is illustrated in Figures 3-5. Figure 3: Distribution of dataset 1. Figure 2: Flow chart of applying sensitivity analysis to interpret VAE. During data preprocessing, columns, including \"row number\", \"customer ID number\", and \"name\", are removed from the three datasets as they do not provide useful customer information. Columns with a significant amount of missing data are also dropped, which results in the total number of input features for the three datasets being 10, 19, and 11, respectively. To make the categorical features recognizable by a VAE, target encoding is used to transform them into numerical values. Lastly, to ensure that the VAE does not give more weight to larger values regardless of their unit measurements, three banking datasets are rescaled through normalization using the scikit-learn object MinMaxScaler. Since this research focuses on using interpretability techniques to explain decisions made by an ML model rather than solely achieving optimal performance, model hyperparameters are adjusted to ensure the interpretations generated are both accurate and stable. To prevent overfitting while maintaining good learning capacity, the dimensions of input features for all three datasets were compressed to 75% of their original sizes and stored in the latent space of VAEs. Through experimentation, it was found that learning rates of 0.01 for dataset 1 and dataset 3 and 0.05 for dataset 2 work well for optimizing the VAE. The same VAE structure, consisting of a 4-layer encoder and 4-layer decoder, was used to synthesize data for three datasets. For each dataset, VAE was trained for 300 epochs. After fine-tuning the hyperparameters, the final loss values, calculated as the sum of similarity loss and KL Divergence, decreased from 71.43 to 13.48 for dataset 1, from 34.97 to 12.65 for dataset 2, and from 53.12 to 9.46 for dataset 3. Figure 4: Distribution of dataset 2. Figure 5: Distribution of dataset 3.  
by multiplying feature importance values with a normalization factor, as described in Section 3. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Variable** & **Feature Name** & **Variable** & **Feature Name** \\ \hline X1 & requested loan amount & X7 & debt to income ratio \\ \hline X2 & length of employment & X8 &
\begin{tabular}{c} number of inquities in last \\ 6 months \\ \end{tabular} \\ \hline X3 & ownership of a house & X9 & number of open accounts \\ \hline X4 & annual income & X10 & number of total accounts \\ \hline X5 & if income is verified & X11 & gender \\ \hline X6 & purpose of loan & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Variables on x-axis and their corresponding feature names for dataset 3.