You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Whole-Patch Causal-MAP Gaze Selector

This private repository contains the gaze selector currently used to build reusable dynamic video payloads for NVILA evaluation.

Snapshot

Training run: wholepatch_causalmap_ntprecovery_frombestntp800_idlr1e3_map1e5_g8_b64_10ep_20260722_200736
Snapshot: checkpoint_step500/checkpoint_gaze
Files: model.safetensors, config.json, and preprocessor_config.json
Native input: up to 16 frames at 224 x 224
Native patch vocabulary: 265 patch IDs plus EOS
Weights: approximately 17 MB

The paired evaluation defaults are:

gaze_mode=spatial49
stopping_threshold=0.15
strict_max_actions=96
strict_map_hint_mode=previous
do_sample=false

Loading

Install the matching AutoGaze package, then:

import torch
from autogaze.models.autogaze import AutoGaze, AutoGazeImageProcessor

repo = "yinanzhou1/wholepatch-causalmap-gaze-selector"
model = AutoGaze.from_pretrained(repo).cuda().eval()
processor = AutoGazeImageProcessor.from_pretrained(repo)

For the 392-pixel NVILA tile pipeline, initialize the processor with size={"height": 392, "width": 392}. The runtime calls input_res_adapt, performs the native strict rollout, and then calls recover_output_from_res_adapt to obtain the 1060-ID 56+112+196+392 target patch space.

See Inference Contract for the exact rollout and output contract.

Scope

This is a patch selector, not a standalone video QA model. Its selected patch sequence must be combined with the NVILA sparse SigLIP and LLM runtime.

Downloads last month: 2

Safetensors

Model size

4.25M params

Tensor type

F32

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support