litert-community/GFPGAN-v1.4-LiteRT

GFPGAN v1.4 — LiteRT (CompiledModel GPU)

On-device GFPGAN v1.4 blind face restoration: it reconstructs degraded / low-quality faces using a StyleGAN2 generative facial prior. Converted for LiteRT CompiledModel with the GPU (ML Drift) accelerator and verified running fully on the GPU of a Pixel 8a (551/551 nodes delegated to LITERT_CL, ~1.2 s per face).

Files

File	What	I/O
gfpgan_fp16.tflite	GFPGAN v1.4 restoration (431 MB, fp16)	[1,3,512,512] NCHW [-1,1] → [1,3,512,512] NCHW [-1,1]
yunet_fp16.tflite	YuNet face detector (0.3 MB) for alignment	[1,3,640,640] BGR 0-255 → 5 landmarks

Pipeline

1. Detect the face + 5 landmarks with YuNet.
2. Align: similarity-warp the face to the standard FFHQ 512 template (GFPGAN's StyleGAN prior mangles the mouth on off-template crops).
3. Restore: normalize the aligned face to [-1,1], run gfpgan_fp16.tflite, denormalize (x+1)*127.5.

Minimal usage

Android (Kotlin, CompiledModel GPU)

// 431 MB — stage into filesDir and load by path
val model = CompiledModel.create("${context.filesDir}/gfpgan_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(chw)              // [1,3,512,512] RGB in [-1,1], FFHQ-aligned face
model.run(inputs, outputs)
val restored = outputs[0].readFloat()  // [1,3,512,512] in [-1,1] -> (x+1)*127.5

Python (desktop verification)

import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

# input must be an FFHQ-aligned 512x512 face crop (YuNet 5-landmark warp; see Pipeline)
img = Image.open("aligned_face.png").convert("RGB").resize((512, 512))
x = (np.asarray(img, np.float32) / 127.5 - 1.0).transpose(2, 0, 1)[None]  # [1,3,512,512]

it = Interpreter(model_path="gfpgan_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
y = it.get_tensor(it.get_output_details()[0]["index"])[0]                 # [3,512,512], [-1,1]
Image.fromarray(((y.transpose(1, 2, 0) + 1) * 127.5).clip(0, 255).astype(np.uint8)).save("restored.png")

Conversion notes (GPU compatibility)

Converted with litert-torch (NCHW preserved). The only substantial re-authoring is the StyleGAN2 ModulatedConv2d, whose original form builds a 5D weight (b,c_out,c_in,k,k) at runtime from the style vector and convolves with that runtime filter — both GPU-incompatible (>4D tensor; a GPU CONV_2D needs a constant filter). It is rewritten to an exact 4D form:

• modulation — conv(x, W·style) == conv(x · style_per_in_channel, W_const) (conv is linear), so the style becomes an input channel-scale and the filter stays constant.
• demodulation — rsqrt(Σ (W·style)² + eps) == rsqrt((style²) @ Wsqᵀ + eps) where Wsq[o,i] = Σ_k W[o,i,k]² is a constant matrix — a small matmul + RSQRT.

fp16 note (Mali): the demod sum Σ style²·Wsq overflows fp16 — the style vectors reach |s|~1000, so the sum reaches ~2.3e6 ≫ 65504, giving rsqrt(inf)=0 and collapsing the decoder to a flat color (it still compiles and runs). Normalizing the style by its per-image max before squaring keeps every intermediate in fp16 range; the scale cancels exactly against the demod, so the on-device output is identical to the desktop fp32 result.

License

Apache-2.0, following the upstream TencentARC/GFPGAN.