UP2You: Fast Reconstruction of Yourself from Unconstrained Photo
  Collections
Authors: name: Zeyu Cai, name: Ziyang Li, name: Xiaoben Li, name: Boqian Li, name: Zeyu Wang, name: Zhenyu Zhang, name: Yuliang Xiu

We present UP2You, the first tuning-free solution for reconstructing
high-fidelity 3D clothed portraits from extremely unconstrained in-the-wild 2D
photos. Unlike previous approaches that require "clean" inputs (e.g., full-body
images with minimal occlusions, or well-calibrated cross-view captures), UP2You
directly processes raw, unstructured photographs, which may vary significantly
in pose, viewpoint, cropping, and occlusion. Instead of compressing data into
tokens for slow online text-to-3D optimization, we introduce a data rectifier
paradigm that efficiently converts unconstrained inputs into clean, orthogonal
multi-view images in a single forward pass within seconds, simplifying the 3D
reconstruction. Central to UP2You is a pose-correlated feature aggregation
module (PCFA), that selectively fuses information from multiple reference
images w.r.t. target poses, enabling better identity preservation and nearly
constant memory footprint, with more observations. We also introduce a
perceiver-based multi-reference shape predictor, removing the need for
pre-captured body templates. Extensive experiments on 4D-Dress, PuzzleIOI, and
in-the-wild captures demonstrate that UP2You consistently surpasses previous
methods in both geometric accuracy (Chamfer-15%, P2S-18% on PuzzleIOI) and
texture fidelity (PSNR-21%, LPIPS-46% on 4D-Dress). UP2You is efficient (1.5
minutes per person), and versatile (supports arbitrary pose control, and
training-free multi-garment 3D virtual try-on), making it practical for
real-world scenarios where humans are casually captured. Both models and code
will be released to facilitate future research on this underexplored task.
Project Page: https://zcai0612.github.io/UP2You