DeFLOCNet

Deep Image Editing via Flexible Low-Level Controls 路 CVPR 2021

Paper 路 Live demo 路 GitHub 路 Training

HDTF portrait and general-scene inference. Each row shows the masked input, the completion, and the original.

Selected outputs from both released checkpoints. White marks the hole. These examples use the mask alone. Image credits.

DeFLOCNet fills masked regions at 256 脳 256 RGB and accepts optional sketch and color guidance. Choose the checkpoint that fits your image.

Checkpoints

Checkpoint Intended images Training data Step
HDTF Portrait Faces and close portraits Dedicated HDTF face crops 10,000
General Scenes Landscapes, buildings, and everyday images Mixed images including Places2, Unsplash, T2I, and a small HDTF component 23,500

Each file is an FP32 generator-only inference bundle of about 323 MiB. See the local inference guide for input formats and commands.

Edit with sketch and color

In DeFLOCNet Studio, paint the edit region in white. Add a sketch for structure or sparse color marks inside the hole. Compare both uses the same image, mask, hints, and seed for the two checkpoints. The completion copies pixels outside the mask from the prepared image.

The same masked coffee image completed with the mask alone and with a hand-drawn arc and sparse color hints.

Both outputs use the same crop, mask, seed, and General Scenes checkpoint. The black arc and brown dots are the supplied sketch and color hints.

Architecture

The sd_color_v2 residual U-Net with multiscale skip connections, separate sketch and sparse-color pyramids, and a recurrent color state in the decoder.

Both checkpoints use sd_color_v2, a five-level residual U-Net that completes the image in one forward pass. Separate pyramids encode the sketch and the supplied color pixels. Sketch features gate the color state as it propagates from coarse to fine. See the architecture notes and the original CVPR 2021 code.

Checksums

File SHA-256
deflocnet-hdtf-step10000.pt 2de135908683ca9c3a882de794efebc103e34101a6e813b3348c662007ed1bb0
deflocnet-general-step23500.pt bab2bf7eedbf6f905895e40be3731ee499d3f485ff7546071ebb90041fc17806
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Space using KumaPower/DeFLOCNet 1