ayzeksalimli/segformer-dw-b0

SegFormer-B0-DW

SegFormer-B0 with one depthwise 3x3 convolution added to the decoder after feature fusion.

Summary

The all-MLP decoder of SegFormer consists of 1x1 convolutions only, so its receptive field on the fused feature map is a single position. One depthwise 3x3 convolution with batch normalization and ReLU is inserted between feature fusion and the classifier. The MiT-B0 encoder and the rest of the decoder are unchanged. The block adds 2816 parameters (0.08 percent of the model) and no measurable latency. A paired ablation with identical training isolates its effect.

Results

Oxford-IIIT Pet, 3 classes (background, pet, border), 256x256 input, official test split (3669 images).

Decoder	Params	mIoU	Border IoU	Mask mAP50-95	FPS
1x1 only (paired ablation)	3,714,915	0.8102	0.6016	0.8205	163.8
1x1 + depthwise 3x3 (this model)	3,717,731	0.8161	0.6137	0.8270	161.9

FPS: batch 1, 256x256, Apple Silicon (MPS).

Training: ImageNet-pretrained MiT-B0, AdamW, lr 3e-4 (encoder lr x0.1), weight decay 0.01, polynomial schedule with 5 percent warmup, batch 16, 20 epochs, cross-entropy loss, seed 1337.

Usage

import torch
from huggingface_hub import hf_hub_download
import importlib.util

path = hf_hub_download("ayzeksalimli/segformer-dw-b0", "modeling_segformer_dw.py")
spec = importlib.util.spec_from_file_location("modeling_segformer_dw", path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)

model = mod.SegFormerDW.from_pretrained("ayzeksalimli/segformer-dw-b0").eval()
x = torch.rand(1, 3, 256, 256)                         # RGB in [0, 1]
mean = torch.tensor(mod.MEAN).view(1, 3, 1, 1)
std = torch.tensor(mod.STD).view(1, 3, 1, 1)
with torch.no_grad():
    pred = model((x - mean) / std).argmax(1)          # 0 background, 1 pet, 2 border

Requirements: torch, safetensors, huggingface_hub.

Publication

Salimli A. M. Improving the SegFormer decoder for semantic segmentation by adding a depthwise convolution block. Molodoy Uchenyy: XXIII International Research Competition. MCNS Nauka i Prosveshchenie, 2026. Pp. 52-58. https://naukaip.ru/wp-content/uploads/2026/10/MK-2794.pdf

---

by Salimli Ayzek: https://mathematiclove.github.io