Discrete InfoMax Codes for Meta-Learning
Authors: Yoonho Lee, Wonjae Kim, Seungjin Choi
Keywords: meta-learning, generalization, discrete representations
TL;DR: We derive a generalization bound for meta-learning, and propose a meta-learning model that generalizes well
Abstract: This paper analyzes how generalization works in meta-learning. Our core contribution is an information-theoretic generalization bound for meta-learning, which identifies the expressivity of the task-specific learner as the key factor that makes generalization to new datasets difficult. Taking inspiration from our bound, we present Discrete InfoMax Codes (DIMCO), a novel meta-learning model that trains a stochastic encoder to output discrete codes. Experiments show that DIMCO requires less memory and less time for similar performance to previous metric learning methods and that our method generalizes particularly well in a challenging small-data setting.