SAMF (Swin-T + DSEM + MARM + NetVLAD) trained on Boson-nighttime

A retrieval model that matches a thermal UAV image to the satellite tile showing the same place. Both images are turned into one 8192-dimensional descriptor; the nearest satellite descriptor to the thermal descriptor is the predicted location.

This is a reproduction of the SAMF model (paper, code) trained with the authors' released recipe, not an official release. This repository holds only the weights (samf_final_best.pth). The model code comes from the SAMF code repository.

How to use it (weights only)

1. Get the model code and the packages. The weights have no code, so clone the SAMF repository next to your script (Python 3.10+):

git clone https://github.com/YuxinWang879/SAMF.git
pip install torch torchvision timm==1.0.22 numpy h5py einops faiss-cpu scikit-learn pillow huggingface_hub transformers

SAMF pins torch 2.9.1, torchvision 0.24.1 and timm 1.0.22; other versions may work but are untested.

2. Load the weights and embed images. Copy this into a script or notebook:

import sys
import torch
import torch.nn.functional as F
import torchvision.transforms as T
import torchvision.transforms.functional as TF
from PIL import Image
from huggingface_hub import hf_hub_download

SAMF_ROOT = "./SAMF"                       # the cloned code repository
device = "cuda" if torch.cuda.is_available() else "cpu"

# --- build the model with the paper settings (they are SAMF's defaults) ---
sys.path.insert(0, SAMF_ROOT)
import parser as samf_parser               # SAMF's own parser.py
from model import network

sys.argv = [sys.argv[0], "--datasets_folder", ".", "--device", device]   # the folder is not used for inference
args = samf_parser.parse_arguments()
args.pretrain = "none"                     # the weights come from the checkpoint, no ImageNet download
model = network.GeoLocalizationNet(args)

# --- load the weights (the file is a pickle: only load checkpoints from a source you trust) ---
path = hf_hub_download("ElectroPython/samf", "samf_final_best.pth")
ckpt = torch.load(path, map_location="cpu", weights_only=False)
state = {(k[7:] if k.startswith("module.") else k): v for k, v in ckpt["model_state_dict"].items()}
model.load_state_dict(state)
model = model.to(device).eval()

# --- preprocessing, exactly as SAMF's dataset code does it ---
norm = T.Compose([T.ToTensor(), T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

def prep(img, thermal):
    img = img.convert("RGB")
    if thermal:
        img = TF.adjust_contrast(img, 3)   # contrast factor 3 for thermal queries only
    return TF.resize(norm(img), [256, 256])

@torch.no_grad()
def embed(images, thermal, batch_size=16):
    """list of PIL images -> (N, 8192) L2-normalised descriptors. thermal=True for the query, False for satellite tiles."""
    out = []
    for i in range(0, len(images), batch_size):
        batch = images[i:i + batch_size]
        x = torch.stack([prep(im, thermal) for im in batch]).to(device)
        modality = torch.full((len(batch),), 1 if thermal else 0, dtype=torch.long, device=device)   # 1 = thermal, 0 = satellite
        out.append(F.normalize(model(x, modality=modality).float(), dim=1).cpu())
    return torch.cat(out)

3. Rank satellite tiles for a thermal image:

tiles = [Image.open(p) for p in tile_paths]            # your satellite tiles
q = embed([Image.open("thermal.png")], thermal=True)   # (1, 8192)
t = embed(tiles, thermal=False)                        # (N, 8192): compute once per map and keep it
scores, idx = (q @ t.T).topk(5, dim=1)                 # cosine similarity (descriptors are L2-normalised)
print([tile_paths[i] for i in idx[0]])

Things that matter: images are read as RGB (a grayscale thermal image is replicated to 3 channels); inputs are resized to 256 x 256; and the modality id must be 1 for thermal, 0 for satellite, since using the wrong one gives worse descriptors.

Results

Dataset: Boson-nighttime (Xiao et al., STGL), splits of 10,256 / 13,011 / 26,568 query-tile pairs (train / val / test). A prediction counts as correct if the retrieved tile is within 50 m of the query. Search is over the whole map of the split (no position prior).

Split	R@1	R@5	R@10	R@20
validation (epoch 28, the saved checkpoint)	96.0	98.3	98.8	99.3
test	89.3	94.8	96.5	97.8

The SAMF paper reports 93.6% test R@1. This reproduction is 4.3 points lower, and I have not found out why (seed, recipe details and the resumes during training are untested possibilities). The validation set is easier than the test set: its database is half the size.

Training

• Backbone Swin-T (timm) from ImageNet weights, fine-tuned end to end (SAMF's "freeze stages" code is a no-op, so nothing is frozen); DSEM and MARM modules; NetVLAD with 64 clusters.
• Triplet margin loss (margin 0.1), 10 hard negatives per query, Adam, learning rate 3e-5, 4 queries per batch, 256 x 256 input, 30 epochs of 10,000 queries.
• The checkpoint is the epoch with the best validation R@5 (epoch 28). Training ran on a single A100 and was resumed several times after runtime disconnects (model and optimizer state were restored each time; the data order after a resume differs from an uninterrupted run).

Intended use and limitations

• Intended for research on thermal-to-satellite geo-localization.
• Domain: thermal tiles at about 1 m/px (512 m footprint), mostly desert at night, matched to Bing satellite imagery. Different sensors, altitudes, seasons or satellite sources are untested and may degrade a lot.
• It is a retrieval model: the answer is the centre of the best tile, with a tile-spacing resolution. It does not give a confidence or a "no match" decision.
• One training run, one seed: differences of about a point between runs are not meaningful.

Licence and data terms

• The Boson-nighttime dataset card says it is "provided solely for non-commercial research and educational purposes" and is based on Bing satellite imagery (Bing Maps Print Rights); its metadata also lists "mit". Weights trained on it should be treated accordingly. TODO: confirm and set the licence above.
• Check the SAMF code repository's licence before redistributing anything derived from it. TODO.

Citation

• SAMF: https://arxiv.org/abs/2610.12835 (code: https://github.com/YuxinWang879/SAMF)
• Dataset and baseline: Xiao, Tortei, Roura, Loianno, "Long-range UAV Thermal Geo-localization with Satellite Imagery", https://arxiv.org/abs/2306.02994