geotessera/TESSERA-V-2.0-2B-Teacher

TESSERA v2 — 2B teacher (2.06 B)

The large-capacity pixel-wise encoder of the TESSERA v2 family. It maps one pixel's annual Sentinel-2 + Sentinel-1 observation history to a 1024-dimensional representation.

Part of the TESSERA v2 release described in TESSERA v2: Scaling Pixel-wise Earth Foundation Models.

This model is released mainly as a distillation target, not as a deployment model. Inference evaluates 2.06 billion parameters per pixel, which makes tile-scale — let alone global-scale — embedding generation impractical on ordinary hardware; a GPU is effectively required even for modest workloads. If you want TESSERA v2 embeddings for real work, use the distilled students below: they reach comparable downstream quality at a tiny fraction of the compute and storage.

Student	Parameters	Output
TESSERA-V-2.0-2B-N	1.07 M	128-d Matryoshka
TESSERA-V-2.0-2B-S	7.11 M	128-d Matryoshka
TESSERA-V-2.0-2B-M	21.03 M	128-d Matryoshka
TESSERA-V-2.0-2B-L	43.83 M	128-d Matryoshka

Architecture

2,064,266,242 parameters, fp32, 1024-d output.

Component	Configuration
S2 backbone	band embedding (10 → 4096) + DOY encoding, 4-layer pre-LN Transformer, d_model 4096, 4 heads, FFN 16384, QK-norm on, attention pooling
S1 backbone	identical, band embedding (2 → 4096); ascending + descending share one merged stream
Fusion	2 modality tokens + learnable modality embedding → 2-layer Transformer (d_model 4096, FFN 8192) → concat to 8192-d
dim_reducer	Linear 8192 → 16384, LayerNorm, ReLU, Dropout, Linear 16384 → 1024, non-affine LayerNorm

The closing non-affine LayerNorm locks the output scale: every representation is exactly mean 0 / std 1 across its 1024 dimensions.

Input contract

Two conventions here are easy to get wrong, and both fail silently — the model will happily return plausible-looking but meaningless vectors.

1. The Sentinel-2 channel order is not ascending wavelength. It is exactly:

B04  B02  B03  B08  B8A  B05  B06  B07  B11  B12

2. Sentinel-1 uses a single merged stream with pooled statistics. Concatenate ascending and descending along the time axis in raw units, then z-score the merged stream with the pooled S1 constants. Do not z-score ascending and descending separately.

• Sentinel-2: (T, 10) per pixel, raw L2A reflectance (unscaled DN), plus a (T,) day-of-year array and a (T,) validity mask (1 = clear, 0 = cloud).
• Sentinel-1: (T, 2) per pixel (VV, VH), raw RTC values, plus day-of-year. All-zero timesteps are treated as missing.
• Day-of-year is passed as a raw integer 1–365 (not normalized).

infer.py applies all of this internally when standardize=True (the default); the constants live in model.py. Per pixel the valid-observation count is bucketized to the nearest bin in {8, 16, 24, ..., 256} and the series padded or subsampled to that size, matching the training-time procedure.

Usage

from huggingface_hub import snapshot_download
import sys, torch

path = snapshot_download("geotessera/TESSERA-V-2.0-2B-Teacher")
sys.path.insert(0, path)

from model import load_model
from infer import encode_pixels, encode_tile

model = load_model(f"{path}/ckpt/tessera_v2_2B_teacher.pt", torch.device("cuda"))

# Per-pixel time series -> (N, 1024)
emb = encode_pixels(
    model, s2_bands, s2_doys, s2_masks=s2_masks,
    s1_asc_bands=s1_asc, s1_asc_doys=s1_asc_doys,
    s1_desc_bands=s1_desc, s1_desc_doys=s1_desc_doys,
    batch_pixels=512, device=torch.device("cuda"),
    autocast_dtype=torch.bfloat16,     # recommended: halves activation memory
)

# One tile -> (H, W, 1024).  Note the memory and time cost at this scale.
emb_map = encode_tile(model, s2_bands_tile, s2_doys, s2_masks=masks_tile, ...)

Loading the checkpoint needs roughly 8 GB for the weights alone; budget additional memory for activations, and lower batch_pixels if you hit limits. autocast_dtype=torch.bfloat16 is the cheapest lever.

Files

TESSERA-V-2.0-2B-Teacher/
├── model.py          TesseraTeacher2B + load_model()  (torch + numpy only)
├── infer.py          encode_pixels() / encode_tile()
├── requirements.txt  torch>=2.0, numpy>=1.21
└── ckpt/tessera_v2_2B_teacher.pt

The checkpoint is a plain torch.save payload with three keys: encoder_state_dict, arch, and repr_dim. No custom classes are pickled and nothing beyond torch and numpy is needed to load it. load_model uses strict=True — the shipped model.py matches the checkpoint exactly.

nn.RMSNorm requires PyTorch ≥ 2.4.

Intended use and limitations

Intended primarily as a distillation target — the model from which the compact TESSERA v2 students are derived — and as a research artifact for work on large pixel-wise Earth-observation representation models.

Limitations:

• Cost. 2.06B parameters per pixel. Not suitable for large-scale or production embedding generation; use the distilled students.
• Representations are annual, 10 m spectral-temporal features — not raw imagery, and not a real-time monitoring product.
• Quality degrades where a pixel has very few valid observations (persistent cloud, sparse revisit).
• Output is 1024-d and not Matryoshka-ordered; truncating it to a prefix is not meaningful. The students provide the nested {16, 32, 64, 128} structure.
• No benchmark scores are quoted here; see the preprint for the evaluation methodology.

License

Released under CC0 1.0 (public domain dedication), matching the rest of the TESSERA release. Use of TESSERA is additionally governed by the project's Acceptable Use Policy.

Citation

@article{tessera_v2_2026,
  title   = {TESSERA v2: Scaling Pixel-wise Earth Foundation Models},
  author  = {Feng, Zhengpeng and Jaffer, Sadiq and Shokar, Ira and
             Knezevic, Jovana and Elvers, Mark and Atzberger, Clement and
             Young, Robin and Naik, Aneesh and Robinson, Niall and
             Blake, Andrew and Coomes, David and Madhavapeddy, Anil and
             Keshav, Srinivasan},
  journal = {arXiv preprint arXiv:2607.03949},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.03949}
}

Links

• Code and releases: https://github.com/ucam-eo/tessera
• Preprint: https://arxiv.org/abs/2607.03949
• Project website: https://geotessera.org/