# Amortized Bayesian Meta-Learning with Accelerated Gradient Descent Steps
## 4 Experiments
### Mini-ImageNet and Omniglot Dataset Performances  
Compared with other state-of-the-art meta-learning methods, we test the few-shot image classification performance of SIN on the Mini-ImageNet and Omniglot datasets in this subsection. Following [12], the feature network \(f\) is chosen as a four-layer convolution network with 64 filter channels (Conv4-4-64). Each layer in \(f\) consists of four sub-blocks: Convolution + BatchNormalized + ReLu + MaxPooling, and \(f\) is pre-trained as a standard classification task on the Mini-ImageNet and Omniglot datasets. The amortization network \(\phi\) is designed as an MLP network with two hidden layers. We trained our model using the Adam optimizer with a learning rate of 0.003 and batch size of 8 for 35,000 steps. By default, the size of the query set \(\widetilde{\mathcal{D}}_{Q}^{(i)}\) is set at 15\(k\), where \(k\) is the shot number. We predict the labels of the query set during training, and the evaluation metric is the average classification accuracy over tasks. Tables 1 and 2 provide a comparison between the state-of-the-art approaches and our method in classification accuracy. With fewer training epochs, SIN shows a clear improvement in obtaining better variational posterior results, as expected. Moreover, the training of the teacher-student network is effective and it can avoid the latent weight from falling into the local optimum. Moreover, when \(\theta_{0}^{t}\) is close to a local minimum, the \(\theta_{t}^{t}\) resample from SIN can converge to \(\theta^{*}\) in a few descent steps. However, when \(\theta_{0}^{t}\) is close to the global minimum, the resample from SIN may decrease the accuracy if the descent step is not enough.  
Since ABML-SIN introduces an amortized neural network (SIN) to predict specific parameters (\(\mathcal{I}\)) during the training process. The generated latent variable \(\theta_{t}^{t}\) is sampled from \(\mathcal{I}\) based on the Wishart distribution. After implementing the few-step gradient descent, the model is able to converge at an optimal point. Moreover, \(\theta_{t}^{t}\) is also denoted as a \"re-sampling\" process, whose weight follows the Wishart distribution, which indicates that the descent procedure starts from a good point. To provide a convincible example of this hypothesis, we add another experiment to the Mini-ImageNet dataset, which records the accuracy factor with the increasing number of training epochs (in the 1-shot and 5-shot  
Figure 8: Selected samples from the MNIST dataset with the generated posterior distribution \(p(z|\mathcal{D})\).  
meta-learning). As shown in Figure 10, we observe that the proposed method outperforms other state-of-the-art methods in terms of uncertainty estimations (accuracy factor) during the training process. It converges to the optimal point after 30 k epochs, while others do not converge.  
The impacts of different hyperparameters on the performance of ABML-SIN should be thoroughly investigated and discussed. One of the important hyperparameters of the proposed method is threshold \(\xi\) in Algorithm 1. This threshold indicates whether to accept SIN to generate \(\theta_{t}^{\prime}\) or use the teacher network to train through the SGD method. To provide a sensitivity analysis on robustness, we added a new experiment by changing the normalized value of \(\xi\) during the training process. We used the Mini-ImageNet dataset for unseen image classification tasks. Figure 11 displays the accuracy factor with different normalized values of \(\xi\), it shows the trend that the accuracy factor increases first and then decreases when the value of \(\xi\) grows. This performance evaluation also provides a sensitivity analysis to show the robustness of the proposed method.  
Figure 9: True posteriors \(p(\psi|\mathcal{D})\) and its approximate \(q_{\psi}(\psi|\mathcal{D})\) for test samples in the experiment.