SFR-Net

English | 简体中文

Learning Scale-Frustum Representations for Ultra-Wide Area
Remote Sensing Image Segmentation

Overview 🧭

SFR-Net is designed for semantic segmentation of ultra-wide area (UWA) remote sensing images, where both the pixel count and geographical coverage are extremely large. It constructs aligned local, short-range, and long-range observations around the same Projection Reference Point (PRP), resizes them to a unified input size, and distinguishes them with learnable scale embeddings. A Cascaded Cross-Scale Fusion (CCSF) module then injects contextual information into the local representation progressively, preserving fine details while improving long-range semantic continuity.

News 📰

• 2026-08-26: We updated the codebase, fixed known bugs, improved the inference, testing, and visualization scripts, and released trained weights for GID, FBPS, and Inria Aerial.
• 2026-07-11: We received the first-round review decision from IEEE Transactions on Geoscience and Remote Sensing (IEEE TGRS), and the manuscript was invited for major revision.
• 2026-05-25: Our paper, “SFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentation”, was released on arXiv.
• 2026-05-20: Our paper, “SFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentation,” was submitted to IEEE TGRS.
• 2026-05-11: We released the initial code version with training and testing scripts and pretrained weights.

Highlights ✨

• We formulate ultra-wide area remote sensing image segmentation as a task that jointly considers large pixel counts, extremely wide geographical coverage, significantly varying object scales, and long-range semantic continuity.
• Scale-Frustum Representations unify local, short-range, and long-range observations around the same PRP. The released GID/FBPS configs use distances [1, 3, 14], while the Inria Aerial config uses [1, 3, 10].
• Learnable scale embeddings explicitly identify resized observations from different spatial ranges.
• The CCSF module progressively introduces nearby and broader contextual cues into detailed local features.
• SFR-Net achieves state-of-the-art results on the UWA GID and FBPS benchmarks. The SFR representation can also improve the accuracy and convergence speed of generic segmentation networks.

Performance 📊

The following table is taken from the paper. SFR-Net reaches 74.67% mIoU on GID and 77.24% mIoU on FBPS in the paper setting.

Repository Layout 🗂️

SFR-Net/
├── configs/
│   ├── _base_/
│   │   ├── datasets/
│   │   ├── schedules/
│   │   └── default_runtime.py
│   ├── gid/sfrnet_swinl_320k_gid.py
│   ├── fbps/sfrnet_swinl_320k_fbps.py
│   └── inria_aerial/sfrnet_swinl_320k_inria_aerial.py
├── mmseg/
│   ├── datasets/transforms/sfr_loading.py
│   ├── datasets/uwa_dataset.py
│   ├── models/backbones/sfr_net.py
│   └── models/necks/ccsf_neck.py
├── tools/
│   ├── train.py
│   ├── test.py
│   ├── sfr_inference.py
│   ├── get_res_iou.py
│   └── visualizer.py
├── pics/
├── pretrain/
├── weights/
├── README.md
└── README_zh-CN.md

The release keeps the default SFR-Net pathway and the GID, FBPS, and Inria Aerial configurations. Multi-distance ablations and other experimental-only modules are intentionally excluded.

Weights 🔑

All pretrained backbones and released SFR-Net checkpoints are hosted in the SFR-Net Hugging Face repository.

Available files

Type	File	Expected location
ResNet-18 ImageNet pretraining	resnet18_v1c-b5776b93.pth	pretrain/resnet18_v1c-b5776b93.pth
Swin-Large ImageNet-22K pretraining	swin_large_patch4_window12_384_22k_20220412-6580f57d.pth	pretrain/swin_large_patch4_window12_384_22k_20220412-6580f57d.pth
GID checkpoint	iter_320000_gid.pth	weights/iter_320000_gid.pth
FBPS checkpoint	iter_320000_fbps.pth	weights/iter_320000_fbps.pth
Inria Aerial checkpoint	iter_320000_inria.pth	weights/iter_320000_inria.pth

You can download the files with the Hugging Face CLI:

pip install -U huggingface_hub
hf download shadowwalk/SFR-Net --local-dir downloads/SFR-Net
cp -r downloads/SFR-Net/pretrain/. pretrain/
cp -r downloads/SFR-Net/weights/. weights/

Released checkpoint results

Dataset	OA (%)	mIoU (%)	mF1 (%)	Checkpoint
GID	86.82	74.46	85.73	weights/iter_320000_gid.pth
FBPS	93.50	77.86	66.72	weights/iter_320000_fbps.pth
Inria Aerial	96.91	83.96*	91.28*	weights/iter_320000_inria.pth

* For Inria Aerial, IoU and F1 are reported for the building class only. The released checkpoints were trained with random seed 42; their results therefore differ slightly from the values reported in the paper.

The backbone paths are currently defined in mmseg/models/backbones/sfr_net.py. No code change is required if the two pretrained files are kept under pretrain/ and commands are executed from the repository root.

Installation 🛠️

Create an environment with a PyTorch/CUDA combination suitable for your GPU, then install SFR-Net from the repository root:

conda create -n sfrnet python=3.10 -y
conda activate sfrnet

# Install PyTorch first according to https://pytorch.org/get-started/locally/
pip install -U openmim
mim install mmengine "mmcv>=2.0.0"
pip install -r requirements.txt
pip install -v -e .
pip install mxnet

mxnet is used by tools/sfr_inference.py to read the original ultra-wide images.

Data Preparation 🗃️

Official dataset pages:

Dataset	Website
GID	Gaofen Image Dataset
FBPS	Five-Billion-Pixels
Inria Aerial	Inria Aerial Image Labeling Dataset

Organize the datasets as follows:

SFR-Net/
└── data/
    ├── GID/
    │   ├── Image_train/
    │   ├── Image_test/
    │   ├── annos_train_5l/
    │   ├── annos_test_5l/
    │   ├── annos_train_24l/
    │   └── annos_test_24l/
    └── inria_aerial/
        ├── images/
        │   ├── train/
        │   ├── val/
        │   └── test/
        └── Label/
            ├── train/
            ├── val/
            └── test/

GID and FBPS use the same GF-2 images but different label folders. GID uses the 5-category annotations and produces 6 class indices including background; FBPS uses the 24-category annotations and produces 25 class indices including background. Inria Aerial uses two class indices: background and building.

The released configs still contain the original local absolute paths. Before training or validation, update these three files:

# configs/_base_/datasets/gid.py
data_root = 'data/GID'

# configs/_base_/datasets/fbps.py
data_root = 'data/GID'

# configs/_base_/datasets/inria_aerial.py
data_root = 'data/inria_aerial'

Alternatively, keep the datasets elsewhere and set each data_root to the corresponding absolute path. The folder names below data_root must still match the structure shown above.

Training 🏋️

Before training:

1. Set data_root in the appropriate file under configs/_base_/datasets/ as described in Data Preparation.
2. Check batch_size and num_workers in the selected experiment config. The released configs use batch size 4 and override num_workers to 64; reduce them if your GPU memory or CPU resources are limited.
3. Keep the two backbone checkpoints under pretrain/, or update depth2ckpt in mmseg/models/backbones/sfr_net.py if you use different locations.

Train with random seed 42 (the default in configs/_base_/default_runtime.py and tools/train.py):

python tools/train.py configs/gid/sfrnet_swinl_320k_gid.py \
  --work-dir work_dirs/gid

python tools/train.py configs/fbps/sfrnet_swinl_320k_fbps.py \
  --work-dir work_dirs/fbps

python tools/train.py configs/inria_aerial/sfrnet_swinl_320k_inria_aerial.py \
  --work-dir work_dirs/inria_aerial

Add --amp to enable automatic mixed precision. Use --resume with the same --work-dir to continue from its latest checkpoint.

Inference 🛰️

tools/sfr_inference.py contains original-machine defaults in the DATASETS dictionary, including /mnt/dataset/zhongchuyu/.... Either replace the src entries with data/GID/Image_test and data/inria_aerial/images/test, or pass --src explicitly as shown below. Command-line values take precedence over those defaults.

python tools/sfr_inference.py \
  --dataset gid \
  --src data/GID/Image_test \
  --dst work_dirs/gid_predictions \
  --config configs/gid/sfrnet_swinl_320k_gid.py \
  --ckpt weights/iter_320000_gid.pth \
  --stride 128

python tools/sfr_inference.py \
  --dataset fbps \
  --src data/GID/Image_test \
  --dst work_dirs/fbps_predictions \
  --config configs/fbps/sfrnet_swinl_320k_fbps.py \
  --ckpt weights/iter_320000_fbps.pth \
  --stride 128

python tools/sfr_inference.py \
  --dataset inria_aerial \
  --src data/inria_aerial/images/test \
  --dst work_dirs/inria_aerial_predictions \
  --config configs/inria_aerial/sfrnet_swinl_320k_inria_aerial.py \
  --ckpt weights/iter_320000_inria.pth \
  --stride 128

The default --load-type random builds the complete scale-frustum representation. Predictions are saved as single-channel class-index PNG masks.

Metrics and Visualization 🎨

Metrics

tools/get_res_iou.py currently stores the original ground-truth paths in its DATASETS dictionary and does not provide a --gt argument. Update that dictionary before evaluation:

DATASETS = {
    'gid': ('data/GID/annos_test_5l', 6),
    'fbps': ('data/GID/annos_test_24l', 25),
    'inria_aerial': ('data/inria_aerial/Label/test', 2),
}

Then compute the metrics:

python tools/get_res_iou.py --dataset gid \
  --pred work_dirs/gid_predictions

python tools/get_res_iou.py --dataset fbps \
  --pred work_dirs/fbps_predictions

python tools/get_res_iou.py --dataset inria_aerial \
  --pred work_dirs/inria_aerial_predictions

Visualization

tools/visualizer.py has no fixed dataset path; provide the input and output directories on the command line. Its PALETTES dictionary contains the GID, FBPS, and Inria Aerial color maps and only needs modification if your class-index convention changes.

python tools/visualizer.py --dataset gid \
  --src work_dirs/gid_predictions \
  --dst work_dirs/gid_visualizations

python tools/visualizer.py --dataset fbps \
  --src work_dirs/fbps_predictions \
  --dst work_dirs/fbps_visualizations

python tools/visualizer.py --dataset inria_aerial \
  --src work_dirs/inria_aerial_predictions \
  --dst work_dirs/inria_aerial_visualizations

Contact ✉️

If you find this work useful, please cite our paper:

@article{zhong2026sfr,
  title={SFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentation},
  author={Zhong, Chuyu and Chen, Keyan and Yang, Qinzhe and Chen, Bowen and Zou, Zhengxia and Shi, Zhenwei},
  journal={arXiv preprint arXiv:2605.25737},
  year={2026}
}

Questions and bug reports are welcome at buaazcy@buaa.edu.cn.

If you find this repository helpful, please give it a star. Finally, here is Phoebe. You are not allowed to bully her.