AbsoluteReality OpenPose Control-Inpaint — QNN (Snapdragon NPU)
Pose-guided inpainting running entirely on a Snapdragon NPU. A 16-input UNet takes the 9-channel inpainting latent and 13 ControlNet residuals, so the model can be told both where to paint and what pose to paint.
Measured on a Samsung S25 Ultra (SM8750, HTP v79), 512×512 / 20 steps:
| mode | time |
|---|---|
| pose-guided inpainting | ~6.0 s |
| plain inpainting (no hint) | ~5.1 s |
⭐ One model, two modes. Omit the control image and it is an ordinary 9-channel inpainting model; the ControlNet branch is skipped entirely rather than run and multiplied by zero.
Contents
| file | what |
|---|---|
unet.bin |
16-input, 9-channel control-UNet (latent 9ch + 12 down-block + 1 mid-block residual) |
controlnet.bin |
OpenPose ControlNet branch (4-channel latent + 512×512 hint → 13 residuals) |
vae_encoder.bin, vae_decoder.bin |
AbsoluteReality's own VAE — the encoder is required for inpainting |
clip_v2.mnn, pos_emb.bin, token_emb.bin, tokenizer.json |
text side (MNN, runs on CPU) |
Built with QAIRT 2.49, w8a16, per-channel weights, NHWC IO, dsp_arch v73 (_8gen2).
⚠ A 2.49-built context declares an fp16 requirement. Chips that lack it reject the model at load. This build therefore targets 8 Gen 2 and newer parts that report fp16; it is not the widest-compatibility build.
Inputs
Requires an image and a mask (white = repaint). The control image is optional; when supplied it must be an OpenPose skeleton rendering at 512×512, not a photo — no pose detection happens on device.
Provenance and licences
- Base weights: Lykon/absolute-reality-1.6525-inpainting — CreativeML OpenRAIL-M. The licence travels with these weights.
- Control branch: lllyasviel/control_v11p_sd15_openpose. Credit to lllyasviel; that model's licence applies to the control branch.
- Conversion used the pipeline from xororz/local-dream (CC BY-NC 4.0), whose C++ server also runs it.
These are conversions, not retrainings: no weights were fine-tuned, only exported and quantized.