thibautloiseau/gekko-vitl-500k

Gekko ViT-L (500k steps)

Self-supervised cross-view pre-training. Gekko learns co-visibility without any 3D annotation by predicting how much the cross-view reconstruction of a masked target improves over a purely monocular one.

This repository holds the final pre-trained backbone from the paper "Revisiting Cross-View Completion: Self-Supervised Pre-Training via Reconstruction Error Comparison": ViT-L, 500,000 steps at global batch 768, on a 12-source raw-video mix consumed with a stride curriculum and no structure-from-motion, depth, pose or overlap preprocessing.

• Code: https://github.com/thibautloiseau/gekko
• Project page: https://thibautloiseau.github.io/projects/gekko/
• Paper: https://arxiv.org/abs/2609.01530

The same network is run three times on a pair of video frames. It reconstructs a 90%-masked target view with the reference view, reconstructs the same masked target without it, and then regresses the per-pixel gap between the two reconstruction errors from the unmasked pair, through one extra output channel. That gap is large where a pixel is co-visible and close to zero where it is not.

The idea in one paragraph

Cross-view completion (CroCo) masks patches of a target image and reconstructs them using both the visible target patches and a reference view. Where the two views overlap this forces the network to find correspondences. Where they do not overlap the reference contributes nothing, and the training signal silently degenerates to monocular inpainting.

Gekko turns that failure into supervision. Run the same network twice on the same mask, once with the reference (error ℓ_CroCo) and once without it (error ℓ_MAE), and the relative improvement

C(p) = (ℓ_MAE(p) − ℓ_CroCo(p)) / ℓ_MAE(p)

is large exactly where the pixel is co-visible and near zero where it is not. Gekko predicts C in a single extra output channel Ĉ, trained with

ℓ_RI(p) = ( sg[ℓ_MAE(p) − ℓ_CroCo(p)] − sg[ℓ_MAE(p)] · Ĉ(p) )²

which down-weights pixels whose MAE error is already small, precisely where the ratio is unreliable (uniform or self-similar regions). The architecture is identical to CroCo up to that one channel.

Results

Frozen backbone, identical head and schedule per column. Higher is better for pose, lower for pointmap and flow.

Frozen ViT-L backbone	ScanNet-1500 pose @0.25m ↑	Pointmap Chamfer ↓ (ScanNet / ETH3D / DL3DV)	Sintel AEPE clean ↓
Gekko-L (this model)	33.2	0.077 / 0.317 / 1.122	6.17
CroCo v2-L	19.7	0.109 / 0.394 / 1.359	13.08
MuM-L	19.9	0.106 / 0.377 / 1.321	22.44
VGGT (fully supervised, ~1B)	92.3	0.030 / 0.240 / 1.065	n/a

VGGT is a fully supervised reference rather than a baseline: it trains with explicit camera and depth supervision, and its corpus contains ScanNet and DL3DV.

The predicted channel is a genuine co-visibility detector, thresholded on a single unmasked forward pass and measured on scenes never seen in pre-training. These are the paper's numbers for the ViT-B model trained 200k steps on the same 12-source mix, not for the ViT-L released here:

Score	ScanNet-1500 AP ↑	ROC-AUC ↑	7-Scenes AP ↑	ROC-AUC ↑
ℓ_CroCo (CroCo-B baseline)	0.576	0.577	0.799	0.603
Ĉ (Gekko-B, 200k)	0.763	0.737	0.859	0.702

Chance AP is 0.502 and 0.726 respectively. Ĉ even beats the pseudo-label it was trained on (0.555 AP), because a per-mask estimate is noisier than the quantity the network learns to regress from it.

Usage

No checkout required -- Torch Hub fetches the code from GitHub and the weights from this repository:

import torch

model = torch.hub.load("thibautloiseau/gekko", "from_pretrained").eval()

From a local clone of the code repository, the same model:

from hubconf import Gekko          # or `from models import Gekko`

model = Gekko.from_pretrained("thibautloiseau/gekko-vitl-500k").eval()

To read the co-visibility channel itself, run the pair unmasked and slice the extra channel off the reconstruction head: that is the Ĉ scored above. The head packs RGB first and Ĉ last, so use get_confidence rather than indexing by hand, and unpatchify to fold it back onto the pixel grid:

feat_ref, pos_ref, _ = model._encode_image(reference)
out   = model(target, feat_ref, pos_ref, do_mask=False)
covis = model.unpatchify(model.get_confidence(out["croco_out"]),
                         target, channels=1)          # (B, 1, H, W), trained towards [0, 1]

scripts/visualize_covisibility.py in the code repository renders this as a figure, and eval/covisibility/ scores it against geometric ground truth. The repository also holds the downstream heads (pose, pointmap, depth, segmentation, flow) used in the paper.

Training details

Architecture	ViT-L encoder (1024 dim, 24 layers, 16 heads) + cross-attention decoder (768 dim, 12 layers, 12 heads)
Parameters	418.7 M
Steps	500,000
Global batch	768
Precision	fp16 mixed
Masking	90% of target patches, shared between the cross-view and MAE passes
Objective	ℓ_MAE + ℓ_CroCo + ℓ_RI, patch-normalised pixel targets
Data	ScanNet, ScanNet++, ARKitScenes, RealEstate10K, EPIC-KITCHENS, Hypersim, CO3Dv2, Objectron, DL3DV, KITTI-360, VirtualKITTI2, TartanAir, all as raw video

Weights are float32 and this file contains weights only; the optimizer state of the original training checkpoint has been stripped.

Citation

@article{loiseau2026gekko,
  title={Revisiting Cross-View Completion: Self-Supervised Pre-Training via Reconstruction Error Comparison},
  author={Loiseau, Thibaut and Bourmaud, Guillaume and Lepetit, Vincent},
  journal={arXiv preprint arXiv:2609.01530},
  year={2026}
}