EgoGazeLite

Pretrained weights for EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input (Matteo Stoiber, Niels Buus Lassen).

Paper (arXiv:2608.15614) · Code (GitHub)

EgoGazeLite is a lightweight dual-stream gaze prediction model for egocentric video. It is meant to replace dedicated eye-tracking hardware, in the GazeLLM (Rekimoto, 2025) pipeline. The model has 15.7M parameters and needs 6.71 GFLOPs per frame. On an iPhone 15 Pro, the full gaze-and-crop pipeline runs end-to-end in 21.6 ms per frame, and the model's forward pass on the Neural Engine takes 8.9 ms of that.

Files

File	Training data	AUC	AAE (°)	Pixel dist.
egogazelite_option_a.pt	Ego-Exo4D Option A (~4h per domain, ~28h). Used in the paper.	0.9655	8.33	26.35
egogazelite_option_b.pt	Ego-Exo4D Option B (~10h per domain)	0.9644	8.25	26.44

Both checkpoints were evaluated following the evaluation protocol of Huang et al. (2018) on the Ego-Exo4D in-distribution validation set described in the repository. That set covers seven domains; Basketball is held out. AAE is reported as AAE_argmax.

Usage

Clone the GitHub repo and run the evaluation script:

python evaluate.py \
    --checkpoint egogazelite_option_a.pt \
    --data_root  /path/to/EgoExo4D \
    --option     a

Or load the model directly:

import torch
from src.models.gaze_lite import GazeLite

ckpt = torch.load("egogazelite_option_a.pt", map_location="cpu", weights_only=True)
model = GazeLite(pretrained_backbone=False, center_bias_sigma=0.4, fusion_mode="residual")
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

Training data and license

The model was trained on Ego-Exo4D (Grauman et al., CVPR 2024). The dataset itself is not distributed here; access requires signing the Ego-Exo4D license. The weights are released under the MIT license.

Citation

If you use these weights, please cite:

@article{stoiber2026egogazelite,
  title   = {EgoGazeLite: On-Device Egocentric Gaze Prediction for Token-Efficient Multimodal LLM Video Input},
  author  = {Stoiber, Matteo and Lassen, Niels Buus},
  journal = {arXiv preprint arXiv:2608.15614},
  year    = {2026}
}