  Meta learning with multiple objectives can be formulated as a Multi-Objective
Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to
solve several possible conflicting targets for the meta learner. However,
existing studies either apply an inefficient evolutionary algorithm or linearly
combine multiple objectives as a single-objective problem with the need to tune
combination weights. In this paper, we propose a unified gradient-based
Multi-Objective Meta Learning (MOML) framework and devise the first
gradient-based optimization algorithm to solve the MOBLP by alternatively
solving the lower-level and upper-level subproblems via the gradient descent
method and the gradient-based multi-objective optimization method,
respectively. Theoretically, we prove the convergence properties of the
proposed gradient-based optimization algorithm. Empirically, we show the
effectiveness of the proposed MOML framework in several meta learning problems,
including few-shot learning, neural architecture search, domain adaptation, and
multi-task learning.
