## 6 Empirical Experiments
### Discriminative Task  
To validate our second hypothesis, i.e., whether the regularisation learning constraint, a lower \(\alpha\), leads to a better-performing semisupervised EVAE in solving a classification task, we constructed 11 versions of a semisupervised EVAE with the same model architecture but varied \(\alpha\), that is, \(0,0.0001,0.001,0.01,0.1,1,5,10,15,20,25\). To be consistent with unsupervised EVAE models in the above generative task, we adopted an identical \(\alpha\) to construct these 11 versions of semisupervised EVAE for this discriminative task. For comparative purposes, we pitted these 11 semisupervised EVAE models against existing state-of-the-art semisupervised learning approaches on the CIFAR-10(4k) dataset. Importantly, to avoid a conflating comparison, rather than reporting the classification performance of these state-of-the-art semisupervised learning approaches that were implemented in previous comparative studies [27; 31], we also implemented some of these learning approaches on our testbed. These state-of-the-art learning approaches went through an identical training curriculum as the derived semisupervised EVAE. These state-of-the-art models range from the ladder network [19], the spike-and-slab sparse-coding approach (S3C) [20], to the virtual adversarial training approach (VAT) with \(\epsilon=1.08\xi_{\zeta}^{\prime}=1e^{-4}\)[21]), the mean-teacher learning approach [32] and the Speed-as-a-Supervisor (SaaS) learning algorithm [31]. Lastly, to serve as the benchmark, we also included a purely supervised \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline the value of \(\alpha\) & 0 & 0.0001 & 0.001 & 0.01 & 0.1 & 1 & 5 & 10 & 15 & 20 & 25 \\ \hline average similarity rating & 2.5 & 2.6 & 2.9 & 2.6 & 1.2 & 1.1 & 3.7 & 2.9 & 3.2 & 2.9 & 3.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: \(\alpha\)-wise similarity ratings on linearly interpolated digits from two-component VAEs in EVAE. approach, i.e., a Bayesian neural network (BNN), which was trained on 4000 annotated CIFAR-10 training samples in this experiment. For more reliable results, we ran each candidate model five times to report its average performance (with standard deviation) on the CIFAR-10(4K) classification task in Table 3.  
From the reported test error rates in Table 3, it is clear that, under our implementation, except for the fine-tuned ladder networks [19] (test error rate at \(16.5\pm 0.3\)), imposing the regularisation learning constraint on a semisupervised EVAE, (\(\alpha=0.001\) (test error rate at \(18.2\pm 0.1\))) could produce a competitive classification result even in the face of state-of-the-art semisupervised learning algorithms. However, from the bottom part of Table 3, even the best-performing semisupervised EVAE (\(\alpha=0.001\)) was inferior to state-of-the-art semisupervised learning approaches under other implementations. This observed performance gap may be due to different adopted training curricula and the less-convoluted base model we used in our implementation. Diving deeper into our implemented 11 versions of a semisupervised EVAE, as demonstrated in Figure 4, the empirical results of these 11 semisupervised EVAE variants on the classification benchmark reveal an interesting relation between the \(\alpha\) value and the final test error rate. With the imposed regularisation constraint, that is, the small \(\alpha\) values, their corresponding semisupervised EVAE models performed better compared to large \(\alpha\) value alternatives in terms of classification test error rates (\(18.2\%(\alpha=0.001)\) vs. \(47.6\%(\alpha=25)\)). Surprisingly, with a further-minimised \(\alpha=0.0001\), no greater performance gain was observed in this experiment. In the extreme case of \(\alpha=0\), its corresponding semisupervised EVAE also lost its competitive edge with the cited state-of-the-art learning approaches.