Urban OpenGen: diffusion inpainter
A conditional diffusion model that rebuilds a missing piece of a real city so that the streets, blocks and buildings continue into the fabric around them. Give it a site with an area cut out and, optionally, six numbers the new area has to meet, and it fills the area with generated urban fabric. 23.2 million parameters, trained from scratch on sixty cities rendered from OpenStreetMap.
It carries no city label on purpose. The surrounding fabric is the style condition, which is what lets it work on cities it never saw.
- Code, viewer and full documentation: https://github.com/differential-studio/urban-opengen
- Training data: urban-opengen-tiles-v2
- Companion model: urban-opengen-generator, a GAN that draws whole tiles from nothing
What it produces
Not a picture. Every output is four channels of measurable data:
| channel | content |
|---|---|
| 0 | building footprint |
| 1 | building height, decodable to metres through the height LUT |
| 2 | street |
| 3 | planted ground |
So anything it makes can be measured with the same functions the real tiles are measured
with, traced into polygons with heights and street centrelines, and exported as GeoJSON
or a Rhino .3dm. That is the point of the project: a design instrument rather than an
image generator.
Architecture
Pixel-space conditional UNet, written against PyTorch with no diffusion library in between. Input is nine channels: the noisy fields, the same four context fields with the hole zeroed, and the mask. Cosine schedule, v-prediction, min-SNR-5 weighting, and a loss weighted 1 inside the hole and 0.1 outside so most of the gradient goes where the work is. Sampling is DDIM with RePaint, which re-injects the known surroundings at every step and is why the seams close.
Scale is part of the condition. Training crops whole-city rasters at a random physical scale, 0.75 to 2.0 times the 400 m tile, so 300 m to 800 m per window. One set of weights therefore produces individual plots at the small end and district structure at the large end. A canvas bigger than one window is filled by MultiDiffusion: overlapping windows denoised together and averaged where they overlap, so there is no tiling seam.
The condition is six numbers measured from the hole before it was emptied: coverage, green share, street share, mean height, floor area ratio and block size. Each arrives with a given flag, so any subset can be withheld, and the all-withheld case is the unconditional model that classifier-free guidance needs.
| Parameters | 23.2 M |
| Checkpoint | run02/latest.pt, step 46,000 |
| Window sizes | 300 to 800 m, any size at inference through MultiDiffusion |
| Resolution | 128 px per window, so 2.3 to 6.25 m per pixel |
| Trained on | one laptop RTX 4070 |
How to use it
The viewer is the easy path. From the repository:
git clone https://github.com/differential-studio/urban-opengen
cd urban-opengen
pip install -r requirements.txt
python phase4/serve.py --lut assets/height_lut.json --cities assets/cities.json \
--inpaint <path to>/latest.pt --osm-contact you@example.com
Open http://127.0.0.1:5000, type an address, draw a shape over part of it, generate. The
site loads live from OpenStreetMap, so put a real address in --osm-contact: their usage
policy asks for a way to reach whoever is making the requests.
No NVIDIA repository, no dataset and no training are needed for this. Headless sampling is
phase6/sample.py; commands are in phase6/README.md.
Limitations, and please read these
- Resolution. At 800 m across 128 px a pixel is 6.25 m, about one storey. Block shapes come out irregular and fine grain is lost. This is the main limitation and higher resolution is the main fix.
- Guidance extrapolates. Classifier-free guidance is a multiplier on the distance from
the unconditional prediction, so at
--cfg 2.0every achieved metric is roughly2 × asked − prior. A delivered value near 2 at cfg 2.0 is arithmetic, not a bug. Pushing further leaves the region the model was trained on and the fabric smears, and asking for a value below the corpus prior drives the result towards zero. There is also a metric guidance path that differentiates the measurement itself at each step, which works whether or not the network learned to use the condition. - Which run is released. A later run with stronger metric control produced visibly worse geometry for the reason above. The released checkpoint is the earlier, weaker-conditioned one, because convincing fabric was worth more to us than a dial that overshoots.
- Reading a training log. Reconstruction loss is dominated by the fabric around the
hole, so a model ignoring the condition entirely still scores well.
condition deliveredis the honest number: it moves a target and measures what comes out. - Height data. In cities where OpenStreetMap carries no height tags, most buildings sit at a 7 m default in the training data, so the model has learned those places as flatter than they are. Quantified per city in the dataset card.
- Not a planning tool. It produces plausible morphology at the scale of a few blocks. It knows nothing about regulation, ownership, topography, ground conditions or anybody who lives there.
Licence and attribution
Weights are CC BY 4.0. Commercial use is fine, credit is required:
Urban OpenGen by Differential (differential.studio),
github.com/differential-studio/urban-opengen
The training data is derived from OpenStreetMap and is ODbL, © OpenStreetMap contributors. The code in the repository is Apache-2.0. Note that the companion GAN is not CC BY: it was produced with NVIDIA's stylegan3 repository and is non-commercial research use only.
Citation
CITATION.cff is in the repository. Developed by Differential.