Vision Superalignment: Weak-to-Strong Generalization for Vision
  Foundation Models
Authors: name: Jianyuan Guo; name: Hanting Chen; name: Chengcheng Wang; name: Kai Han; name: Chang Xu; name: Yunhe Wang
Published: 2024-02-06 06:30:34

Abstract: Recent advancements in large language models have sparked interest in their
extraordinary and near-superhuman capabilities, leading researchers to explore
methods for evaluating and optimizing these abilities, which is called
superalignment. In this context, our paper delves into the realm of vision
foundation models, focusing on the concept of weak-to-strong generalization,
which involves using a weaker model to supervise a stronger one, aiming to
enhance the latter's capabilities beyond the former's limits. We introduce a
novel and adaptively adjustable loss function for weak-to-strong supervision.
Our comprehensive experiments span various scenarios, including few-shot
learning, transfer learning, noisy label learning, and common knowledge
distillation settings. The results are striking: our approach not only exceeds
the performance benchmarks set by strong-to-strong generalization but also
surpasses the outcomes of fine-tuning strong models with whole datasets. This
compelling evidence underscores the significant potential of weak-to-strong
generalization, showcasing its capability to substantially elevate the
performance of vision foundation models. The code is available at
https://github.com/ggjy/vision_weak_to_strong.