Title: Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image
Primary area: generative models
Keywords: 3D generation, novel view synthesis, satellite to street-view generation, feed-forward image to 3D, outdoor scene generation

Abstract: Street‑level 3D generated from a single satellite image enables mapping, robotics, simulation, and media creation at scale, but existing methods trade off semantics and geometry: geometry‑colorization pipelines yield clean yet building‑only reconstructions, while proxy‑based renderers preserve satellite semantics but produce coarse, unstable geometry. We propose Sat3DGen, a feed‑forward satellite‑to‑3D framework that learns a differentiable 3D field under 2D supervision, with a focus on scene‑level geometric quality. The method introduces three components to address warped geometry, boundary errors, and rooftop ambiguity: a gravity‑based density variation loss that biases the volumetric field toward gravity‑aligned structures; a spatial token that expands the effective ground‑plane extent to stabilize the peripheral layout; and a monocular relative‑depth prior that constrains satellite‑view depth. Besides, we further strengthen the supervision, without extra data sources, by jointly training on high‑resolution panoramas and their projected perspective views to increase viewpoint coverage and photometric consistency. Although Sat3DGen includes no tailored image‑quality modules, it reduces FID on the unseen‑city split of VIGOR‑OOD from about 40 to 19 compared to state-of-the-art satellite-to-street-view video model Sat2Density++, with improvements attributable to more accurate, view‑consistent geometry. The resulting 3D assets support downstream applications, including semantic‑map‑to‑3D synthesis, surround‑view multi‑camera video generation from satellite imagery, large‑area mesh construction, and single‑image DSM estimation without depth ground truth.