Title: Variational Model Merging for Pareto Front Estimation in Multitask Finetuning
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: Variational Bayes, Bayesian methods, Pareto front, multitask finetuning, model merging

Abstract: We propose a new variational model merging method that can yield arbitrarily accurate Pareto fronts in multitask finetuning. The idea is to first compute posterior-approximations on each task separately and then quickly merge them to obtain a cheap estimate of Pareto front. The main theoretical result is to show that more flexible posteriors necessarily yield better estimates, for example, a Pareto front obtained by merging full Gaussian posteriors is expected to be better than those obtained with isotropic Gaussians. This is because the error incurred by a specific class of distributions can always be reduced by increasing the size of the class. We validate the theory through extensive empirical results on deep networks (Vision and Language Transformers) where better Gaussian families consistently yields better or comparable Pareto fronts. Our work is a rare instance where Bayesian ideas are used to improve Pareto analysis.

TL;DR: We propose a new variational model merging method that can yield arbitrary accurate Pareto fronts in multitask finetuning.