## 6 Empirical Experiments
### Generative Task  
To test our first hypothesis, i.e., whether the imposed dependence-minimisation constraint, a large \(\alpha\), leads to varied patterns of digit generation, we trained the EVAE with the binarised MNIST dataset to generate MNIST-like digits. Under our hypothesis, a different \(\alpha\) should lead to differentiated impacts on the final loss function of the EVAE. We then chose 10 representative \(\alpha\) values, \(0.0001,0.001,0.01,1,5,10,15,20,25\), in the construction of ten versions of the unsupervised EVAE. For comparative purposes, we also included the case of \(\alpha=0\) to observe the generative performance of two unrelated vanilla VAEs. To obtain an objective quantitative evaluation to assess the performance of our proposed EVAE in a generative task is difficult. Some commonly applied metrics like the estimated negative log-likelihood of a set of samples and optimised lower bound were not applicable here, as such indices are varied across the choice of \(\alpha\) in an EVAE. In comparison with the fixed optimised lower bound of a conventional VAE, the optimised lower bound of an EVAE is largely varied and dependent upon the value of \(\alpha\). Since the prior-mentioned quantitative measures are less accountable, here we resorted to a qualitative evaluation to validate Hypothesis 1. The considered qualitative-evaluation metric relies on visual inspection of the generated pixel space (digits) based on the linear interpolation of the encoded latent space. This metric has widely been used in assessing the qualitative performance of a generative model [30]. For comparative purposes, we set the start and end of this linear interpolation as digits \"2\" and \"9\", respectively, to observe how the two decoders (from the base and scaffolding VAE, respectively) generate sequences of digit transformation from \"2\" to \"9\". Under Hypothesis 1, with the imposed dependence-minimisation constraint, that is, a large \(\alpha\), the generative processes of two component VAEs diverged, leading toward varied patterns of digit generation for this generative task. \begin{table}
\begin{tabular}{l l} \hline \hline
**Dataset** & **Model Architecture and Training Configurations** \\ \hline  & Input: 784 (flattened 28 \(\times\) 28 \(\times\) 1) \\  & Base encoder: FC 500, 300, ReLU activation \\  & Scaffolding encoder:FC 500, 200, 100, \\  & ReLU activation \\ MNIST digits & Decoders FC 200, 784, \\  & Sigmoid activation, Gaussian \\  & Batch size: 100 \\  & num of batch: 100 \\  & optimiser: rmsprop [28] \\ \hline  & Input: 32 \(\times\) 32 \(\times\) 3 \\  & Base encoder: Conv 32 \(\times\) 3 \(\times\) 3 (stride 2), \\  & Pooling(stride 2). \\  & Conv 64 \(\times\) 3 \(\times\) 3 (stride 2) FC 200. \\  & ReLU activation \\  & Scaffolding Encoder: Conv 32 \(\times\) 3 \(\times\) 3 (stride 2), \\  & Pooling(stride 2). \\  & Conv 32 \(\times\) 3 \(\times\) 3 (stride 2) FC 200. Categorical. \\  & ReLU activation \\  & Batch Size: 100 \\  & num of batch: 50 \\  & optimiser:[PERSON] [29] \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training configurations on the MNIST and CIFAR-10(4K) datasets. As shown in Figure 3, with small \(\alpha\) values (0.1 and 1), the generative processes of two-component VAEs (base and scaffolding VAEs) in EVAE were unfolded in an identical manner. Sequences of images on digit transformation from two-component VAEs did not differ in terms of visual inspection. Both generative processes underwent the transition from digit \"2\" to \"1\", then finally toward digit \"9\". Figure 3: \(\alpha\)-wise linear interpolation of latent spaces of base and scaffolding VAEs in EVAE on MNIST dataset. For comparative purposes, we fixed the start and end digits to be “2” and “9”, respectively. For each \(\alpha\), we linearly interpolate the latent spaces of the two-component VAEs in EVAE to observe their entire digit-transformation process.