CityDreamer: Compositional Generative Model of Unbounded 3D Cities
Authors: name: Haozhe Xie; name: Zhaoxi Chen; name: Fangzhou Hong; name: Ziwei Liu
Published: 2023-09-01 17:57:02

Abstract: In recent years, extensive research has focused on 3D natural scene
generation, but the domain of 3D city generation has not received as much
exploration. This is due to the greater challenges posed by 3D city generation,
mainly because humans are more sensitive to structural distortions in urban
environments. Additionally, generating 3D cities is more complex than 3D
natural scenes since buildings, as objects of the same class, exhibit a wider
range of appearances compared to the relatively consistent appearance of
objects like trees in natural scenes. To address these challenges, we propose
CityDreamer, a compositional generative model designed specifically for
unbounded 3D cities, which separates the generation of building instances from
other background objects, such as roads, green lands, and water areas, into
distinct modules. Furthermore, we construct two datasets, OSM and GoogleEarth,
containing a vast amount of real-world city imagery to enhance the realism of
the generated 3D cities both in their layouts and appearances. Through
extensive experiments, CityDreamer has proven its superiority over
state-of-the-art methods in generating a wide range of lifelike 3D cities.