GasVLM

Trained weights for GasVLM, a sub-billion-parameter grounded vision-language model for optical gas imaging (OGI). Given a thermal frame and a question, the model generates a text answer that may contain [SEG] tokens; each [SEG] drives a mask decoder to produce a binary segmentation. Gas concentrations and the CH4/CO2 ratio class appear inside the answer text.

Code: https://github.com/toqitahamid/GasVLM

Model

Frozen V-JEPA 2 ViT-L with a single low-rank LoRA adapter on the attention projections, a pixel-shuffle projector, and SmolLM2-360M-Instruct with q/k/v/o LoRA and trainable token embeddings. A multi-scale [SEG]-conditioned mask decoder reads the encoder features to produce one mask per [SEG].

Roughly 700M parameters, of which about 58M (8.3%) are trainable. Trained on GasGround, an OGI set covering CH4 and CO2 plume video across three thermal cameras.

Usage

git clone https://github.com/toqitahamid/GasVLM && cd GasVLM && pip install -e .
hf auth login
python tools/download_weights.py --repo-id toqi/GasVLM --filename gasvlm.pt

The checkpoint lands in checkpoints/. To reproduce the numbers below you also need the GasGround test split:

python tools/unpack_gasground.py --repo-id toqi/GasGround --out GasGround --splits test

Point the data: paths in configs/gasvlm_headline.yaml at the unpacked JSONL, then:

bash dist_eval.sh eval.py 1 --config configs/gasvlm_headline.yaml \
    --checkpoint checkpoints/gasvlm.pt --output-dir runs/eval --split test --levels 1

gasvlm.pt is a plain state_dict and loads into GasVLM with strict=True.

Results

GasGround test split.

Level	Metric	Value
RES	gIoU / cIoU	0.705 / 0.701
RES	gas IoU / source IoU	0.691 / 0.834
VQA	ratio-class accuracy / within-1	0.713 / 0.963
VQA	CH4 / CO2 MAE	6.45 / 6.45
GCG	mIoU / mask AP50	0.679 / 0.611

License

The weights are released under CC BY-NC 4.0. Commercial use is not permitted. The code at https://github.com/toqitahamid/GasVLM is under the MIT license.