avijeetsingh1608/TerraGround-Gemma-4-E2B-LoRA

TerraGround · a space-grounded Gemma-4 geo-reasoner (LoRA adapter)

◆ ◇ ◆

TerraGround is a LoRA adapter for google/gemma-4-e2b-it that turns Gemma into a grounded Earth-observation analyst. It answers questions about any place on Earth — dominant land cover, vegetation greenness (NDVI) and moisture (NDMI), surface water — from the free, co-located TESSERA satellite embedding decoded into analysis products, with calibrated confidence and a refusal gate for out-of-scope questions. It lifts grounded geo-QA accuracy from 0.694 (frozen base) to 0.794 on held-out cells.

Built for the edge / orbit→ground setting (the LatentBridge thesis): a tiny TESSERA embedding is available for free on both the satellite and the ground, so a frozen Gemma on the ground can reason about a location without downlinking imagery. Honesty is a feature — see Limitations.

How Gemma is modified (frozen backbone + LoRA on the text-layer projections + external geo-grounding + safety wrappers — no structural change, drop-in from_pretrained):

🚀 Live demo (verifiable grounding): TerraGround-Verifiable-EO Space

📄 Paper: PAPER.md (LaTeX source: paper.tex) · 🔁 Reproduce: REPRODUCE.md · 🧩 Offline grounding kit: decode_offline.py + reference.npz

◆ ◇ ◆

What it is

• Base: google/gemma-4-e2b-it (gated; Gemma terms). Adapter type: PEFT LoRA (r=8).
• Inputs: a natural-language question + a decoded TESSERA grounding block (land cover + spectral indices).
• Two safety layers ship as code, not weights: a deterministic answerability gate (refuses questions outside the satellite-derived bands) and a calibrated confidence guard (abstains where the decode is unreliable).
• Hardware/software: runs on one consumer/edge GPU — 4-bit base + adapter is ~9 GB VRAM at inference. Needs transformers==5.10.0.dev0 (ships the gemma4_unified arch), peft==0.19.1, bitsandbytes; see requirements.txt.

Intended use & supported tasks

• Grounded geo-QA: dominant land-cover class; vegetation greenness/moisture level (low/medium/high); surface water.
• As the answering model behind an agentic Earth-observation assistant (see the TerraGround repo).

Out of scope (refused by the gate)

Anything not derivable from satellite bands — ownership, prices, people/counts, specific dated events, sub-pixel or indoor facts, far-future prediction. The gate refuses these deterministically.

◆ ◇ ◆

How to get started

# 1) Accept the base model gate at https://huggingface.co/google/gemma-4-e2b-it then: huggingface-cli login
# 2) pip install -r requirements.txt   (transformers 5.10.0.dev0, peft 0.19.1, bitsandbytes, numpy)
import torch, numpy as np
from transformers import AutoModelForCausalLM, AutoProcessor, BitsAndBytesConfig
from peft import PeftModel
from decode_offline import DecodeHeads, grounding_block   # shipped with this adapter

BASE = "google/gemma-4-e2b-it"; ADAPTER = "<org>/TerraGround-Gemma-4-E2B-LoRA"
cfg = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=cfg, device_map={"": 0})
model = PeftModel.from_pretrained(model, ADAPTER)        # the TerraGround geo-adapter
proc = AutoProcessor.from_pretrained(BASE); tok = proc.tokenizer

# Ground a question in the FREE co-located TESSERA embedding (offline; bundled reference.npz)
heads = DecodeHeads("reference.npz")
emb = np.load("example_embedding.npy")                   # a 128-D TESSERA embedding for some place
block = grounding_block(heads, emb)                      # decoded land cover + indices, as text
sys = "You are a careful Earth-observation analyst. Use the grounded context; respect its caveats. Be concise."
msgs = [{"role": "system", "content": [{"type": "text", "text": sys}]},
        {"role": "user", "content": [{"type": "text",
          "text": f"GROUNDED EARTH-OBSERVATION CONTEXT:\n{block}\n\nQUESTION: What is the dominant land cover here?"}]}]
ids = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=64)[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

◆ ◇ ◆

Evaluation

All numbers below are produced by the scripts in latent_bridge/terraground/ and read from out/terraground/*.json (see Reproducibility). Evaluation is on a geographically held-out subset of the reference corpus, not an external benchmark — see Limitations. CIs are Wilson 95%.

Grounded QA accuracy (held-out cells)

Frozen base vs the adapter, both reading the same decoded TESSERA grounding (n=60 cells):

task	base	this adapter
land cover (8-class)	0.92 [0.82, 0.96]	0.92 [0.82, 0.96]
vegetation greenness level	0.70 [0.57, 0.80]	0.80 [0.68, 0.88]
vegetation moisture level	0.47 [0.35, 0.59]	0.67 [0.54, 0.77]
overall	0.69 [0.57, 0.80]	0.79 [0.68, 0.88]

Hallucination control — the answerability gate

A deterministic capability-match gate refuses questions outside the satellite-derived bands, evaluated on fresh probes disjoint from training:

• in-scope questions correctly answered: 7/7 (1.00 [0.65, 1.00])
• out-of-scope questions correctly refused: 10/10 (1.00 [0.72, 1.00])

Calibrated confidence guard

k-NN agreement is a calibrated confidence: held-out land-cover accuracy is 0.00 below the 0.71 threshold vs 0.92 above it. Abstaining below threshold is selective prediction:

threshold	coverage	accuracy on answered
0.0	1.0	0.911
0.71	0.989	0.921
0.85	0.989	0.921
1.0	0.955	0.918

Coverage (diverse-site backfill)

Backfilling the reference with geographically diverse sites (571→1286 cells) raised mean decode confidence on held-out exotic locations (never added to the reference) from 0.671 to 0.914, with corrected classes (e.g. boreal grassland→tree cover, Punjab→cropland, tundra→moss & lichen).

Land-cover head: generalization to unseen regions

The land-cover decode head was upgraded from k-NN to a class-balanced MLP. k-NN memorises reference geometry (great in-distribution, poor on new regions); the MLP generalizes:

head	in-distribution test	out-of-distribution (fresh regions)
k-NN (old)	0.916	0.64
MLP (deployed)	0.911	0.84

Clay/Prithvi scene embeddings were tested and do not help here (they are scene-scale; fusing them lowered OOD land cover) — TESSERA-only MLP is best.

Honest note on the hallucination metric

An earlier single run reported a 0.0 LoRA hallucination rate, but that evaluated the same question templates seen in training. Under the rigorous protocol (training on one set of unanswerable question types, testing on disjoint held-out types), the fine-tuned adapter's hallucination rate is 1.0 — i.e. LoRA-taught abstention does not generalize to unseen question types. This is why out-of-scope refusal is handled by the deterministic gate on the frozen base (above), not by the adapter. We report this null deliberately.

Agentic tool-planning

On the ReAct prompt, the adapter emits a tool call (rather than short-circuiting to a final answer) 100% of the time vs 100% for the base (n=12).

◆ ◇ ◆

Training details

• Data: a 4165-example mixed objective over a 1286-cell reference corpus (geographically diverse; ESA WorldCover labels + 7 spectral indices from co-located TESSERA embeddings). Objective = short grounded QA + self-distilled long-form reports + self-distilled ReAct tool-planning traces + diverse abstention examples.
• Procedure: PEFT LoRA (r=8, α=16) on the frozen 4-bit base, 1500 steps, final loss 0.346. Long-form/ReAct targets are self-distilled from the frozen base to preserve general fluency (a short-only objective collapses long-form generation).
• Compute: single 40GB GPU, ~10 GB peak (4-bit base + LoRA, gradient-checkpointed).

◆ ◇ ◆

Limitations (read these)

• Same-archive caveat: the decoded products are a faithful recall of the Sentinel-2 archive TESSERA ingests, not independent sensing; on the only fully-external check (ESA WorldCover) the embedding beats a per-biome baseline by only ~0.02. Treat products as a strong prior, not ground truth.
• Evaluation is in-distribution on one reference corpus (held-out cells), not an external benchmark; n is small (CIs above are wide).
• LoRA-taught abstention does not generalize to unseen unanswerable question types (held-out rate 1.0); out-of-scope refusal is therefore handled by the deterministic gate on the frozen base, not by the adapter.
• Long-form / agentic tool-planning are best run on the frozen base unless the agent-planning eval above shows a high tool-call rate; the adapter is tuned primarily for grounded answering.
• Single-date appearance and within-year change are nulls — the annual embedding cannot resolve them.
• Requires the gated base model and transformers==5.10.0.dev0.

◆ ◇ ◆

Citation & acknowledgements

If you use this adapter, please cite the base model and data sources (see CITATION.cff):

• Gemma (Google DeepMind) — the base model.
• TESSERA — Feng et al., TESSERA: A Foundation Model for Earth Observation, arXiv:2506.20380.
• ESA WorldCover (10 m land cover) — © ESA WorldCover project, CC BY 4.0.
• Sentinel-2 — Copernicus / ESA.
• LatentBridge / TerraGround — Vortx-AI.

License: the adapter is governed by the Gemma terms of the base model; the TerraGround training/inference code is Apache-2.0. ESA WorldCover labels are CC BY 4.0.