AnyUp: Universal Feature Upsampling
Authors: name: Thomas Wimmer, name: Prune Truong, name: Marie-Julie Rakotosaona, name: Michael Oechsle, name: Federico Tombari, name: Bernt Schiele, name: Jan Eric Lenssen
Organization: Max Planck Institute for Informatics

We introduce AnyUp, a method for feature upsampling that can be applied to
any vision feature at any resolution, without encoder-specific training.
Existing learning-based upsamplers for features like DINO or CLIP need to be
re-trained for every feature extractor and thus do not generalize to different
feature types at inference time. In this work, we propose an inference-time
feature-agnostic upsampling architecture to alleviate this limitation and
improve upsampling quality. In our experiments, AnyUp sets a new state of the
art for upsampled features, generalizes to different feature types, and
preserves feature semantics while being efficient and easy to apply to a wide
range of downstream tasks.