jessolmstead/greycard-rorem

RORem-mixed, ONNX fp16, for greycard

The ONNX export greycard runs for its generative remove: RORem-mixed (Li et al., RORem: Training a Robust Object Remover with Human-in-the-Loop, CVPR 2025; code, weights at revision 22d02de), an SDXL inpainting UNet fine-tuned on removal pairs and trained at 512 and 1024. These files are modified from the original: converted to ONNX and fp16, as below.

How the files were made

1. UNet. RORem-mixed's UNet traced in fp32 with the legacy exporter at opset 17, static shapes, batch 1, no classifier-free guidance, at a latent of 64 and of 128; converted to fp16 with onnxruntime.transformers.float16, timestep kept fp32.
2. Resize fix. The converter makes Resize's scales input fp16, which ONNX forbids; it is put back to fp32.
3. conv_in pad. ONNX Runtime's WebGPU Conv answers wrong for 9 input channels (1.27 and 1.28); the 9-channel input is padded to 12 with zeros, and conv_in's weight with zero input channels, so the result is unchanged.
4. VAE. RORem's own VAE (the fp16-fix one) as an encoder, image in -1 to 1 to the unscaled latent mean, and a decoder, unscaled latent to image; fixed 512 and 1024, fp32 trace at opset 17, then fp16.
5. Embeddings. RORem takes the empty prompt at guidance 1.0, so SDXL's two text encoders were run once on "" and their outputs kept; no text encoder ships.

File	Bytes	SHA-256
unet-1024.onnx (latent 128)	2,521,003	d81aed3cc00059e60e7c476be8aa679a9c78e3872cc73848bdd01b02c4b5fdf2
unet-512.onnx (latent 64)	2,520,999	0cfff605ed573a5695ccee47b91fec586b03464f421c8ceb37834db2218e9796
unet.onnx_data (both graphs' weights)	5,134,931,840	3c883667b7e45d131d0d45d1339e28d38dfa3c210a3258f2afb637e58cfe50a5
vae-encoder-512.onnx	68,427,415	0d5368ce83a6a6a5cdcac5ffbb082f5102b9557441e7df8436949a0648290ca1
vae-encoder-1024.onnx	68,427,415	8175c93cb33d553325ce940be504a61adca76ec0fbe1ed41c8e5bc46402044e6
vae-decoder-512.onnx	99,084,062	2acf1e075627ec41e68a243f048a5fd0bcee57548dc3b41eef4e5d101b2d8e6a
vae-decoder-1024.onnx	99,084,064	25bcb399f90e732cac2fa8d28c15b61acf2872f095737c870df99160b34e1a7f
prompt_embeds.f16, [1, 77, 2048]	315,392	739efee6b5c6deb2c4ecf3d1bf8908242e9e1a5b27ae8896d16fc904873b2eb0
text_embeds.f16, [1, 1280]	2,560	c2e5c4dff874db93dc53e5cc78d9c3fb3789f1342738729eb755c9fe5a2b82fa
time_ids.f16, [1, 6]	12	4c84e226abda27b53afd64b62938667278fc0d8b0babc9bbb0c384b49cde554c

The .f16 files are raw little-endian fp16. time_ids is SDXL's size conditioning at 1024 (original height and width, crop top and left, target height and width); at 512 the sizes are 512. UNet inputs: sample [1, 9, L, L] fp16 (noisy latents, mask, masked-image latents), timestep [1] fp32, encoder_hidden_states, text_embeds, and time_ids fp16; output out_sample [1, 4, L, L]. Latents are scaled by 0.13025. 5,475,314,762 bytes in all.

License

The weights are a full fine-tune of SDXL inpainting, made from Stability AI's SDXL base 1.0, and carry its CreativeML Open RAIL++-M License, with its use-based restrictions (Attachment A), in LICENSE. RORem's card and repository say Apache-2.0, but neither ships a license file, and a fine-tune does not shed the base's restrictions. Anyone using these files must comply with Attachment A and pass the same restrictions on to their users.

Training data

The RORem paper names, among its training sets, MuLAn (CC BY-NC-SA 4.0) and DIV2K (academic research only). Whether their terms bind the weights is not settled; read them before use.

How greycard fetches it

greycard downloads these files on first use of the generative remove, after showing this license, into its model store (~/.cache/greycard/models/rorem-mixed-fp16/ on Linux), checks each against the size and SHA-256 above, and resumes a large file whose download was cut off. It runs them through ONNX Runtime on WebGPU only: diffusers' SDXL inpainting arithmetic at guidance 1.0, Euler discrete over 25 steps from strength 0.99 (24 UNet calls), in about 10 to 14 s at 1024 on a 16 GB card.