SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In the case when the task distribution is not a mixture, our model would degenerate to and perform similarly to the general metric-based meta-learning methods, e.g., ProtoNet, which only considers a uni-component distribution. To confirm this, we added an experiment that compares our model with ProtoNet-Aug on Mini-ImageNet [47], which does not have the same explicit mixture distributions as in the Plain-Multi and Art-Multi datasets in Section 4. The results are summarized in Table 9. From the table, we observe our method performs comparably to ProtoNet, which validates the aforementioned guess. Meanwhile, together with the results in Table 1 and Table 2, the proposed method could be considered as a generalization of the metric-based methods to the mixture of task distributions.
D.7 Task Embedding Visualization.
We provide visualizations of the task embeddings learnt by HTGM. Fig 4 is the visualization of the task embeddings for Few-Shot Classification on Plain-Multi dataset (Sec. 4.1) via t-SNE [45]. Every yellow point corresponds to a task sampled from Aircraft. Every blue point corresponds to a task sampled from Texture. Every red point corresponds to a task sampled from Fungi. Every green point corresponds to a task sampled from Bird. As we can see, in general, different classes of tasks are well clustered in the task embedding space, indicating that HTGM learnt task embeddings that capture the difference of tasks from different groups.
Figure 4: Clusters of task embeddings on Plain-Multi.
D.8 Computational Cost
We report the cost of ProtoNet-Aug, NCA, FEATS and our model because they use the same encoder architecture. We evaluated and trained all of the models on RTX 6000 GPU with 24 GB memory.
Training Time: According to the training logs, the training of the ProtoNet-Aug took 10 hours, the training of NCA took 6.5 hours, the training of FEATS took 10.5 hours, and the training of the
proposed model HTGM took 13 hours. Please note that our algorithm and FEATS require pre-training the encoder with ProtoNet-Aug. The 10.5 and 13 hours include the 10 hours pre-training phase. The main cost of our model is not from the energy function, because we have reduced its partition function to a constant using Eq. (6) in Theorem 3.1, whose training cost is negligible.
The higher cost is because (1) our model needs to jointly learn a GMM model in every EM step and (2) jointly learning the generative model and classification model takes more learning steps to converge. Given the pre-trained ProtoNet-Aug encoder, the FEATS took about 3000 steps to converge, the proposed model took about 10000 steps to converge. In other words, the EM training algorithm takes more computational overhead. However, the advantage comes with the training cost is the better classification and novel task detection performance.
Test Time: Because we approximated the partition function of the energy function with a constant upper bound, we almost added zero computational cost to the model inference. The test time of NCA, FEATS, ProtoNet, and our model are all around 85-95 seconds for 1000 tasks.