yyoraa/visual-latent-rae-decoders

Visual Latent RAE Decoders

This public model repository contains production and experimental decoder checkpoints used for visual-latent reconstruction and pixel-grounding analysis.

lvr_monet/rae_full_best.pt

• Used by LVR and Monet.
• Large RAE decoder (approximately 158M parameters), patch head with refinement.
• Maps 3584-dimensional Qwen2.5-VL visual/latent embeddings to image patches at 28 pixels per token.
• Trained on 417,453 ViSCoT bbox-region pairs with MSE + 0.3 SSIM.
• Checkpoint metadata: epoch 18, validation loss 0.4490462315, variable_grid=true.

mirage/mirage_rae_best.pt

• Used by Mirage only.
• Base RAE decoder over four average-pooled Mirage latent tokens arranged as a 2x2 grid, reconstructing a 56x56 helper image.
• Trained with MSE + 0.5 SSIM + 0.1 VGG perceptual loss.
• Checkpoint metadata: validation MSE 0.0135339280.

The Mirage checkpoint is intentionally separate because Mirage's four-token average-pooling representation differs from LVR's bbox-aligned latent geometry.

experimental/lvr/image_embed/p3_perceptual/best.pt

• Experimental decoder_new checkpoint for the image_embed condition.
• Maps a variable-length sequence of 3584-dimensional Qwen2.5-VL bbox visual embeddings through a Transformer adapter and image decoder.
• Stage P3 perceptual checkpoint: latent-space, pixel, and LPIPS objectives.
• This is the visual-encoder reference condition; the corresponding tf and si checkpoints decode teacher-forced and self-inference LLM latent states.

Integrity

See SHA256SUMS for checkpoint digests. Loading requires the corresponding RAE decoder implementation from the vlr_x code repository.