Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary
  Detection
Authors: name: Haoxuan Wang; name: Qingdong He; name: Jinlong Peng; name: Hao Yang; name: Mingmin Chi; name: Yabiao Wang
Published: 2024-09-13 03:23:52

Abstract: Open-vocabulary detection (OVD) aims to detect objects beyond a predefined
set of categories. As a pioneering model incorporating the YOLO series into
OVD, YOLO-World is well-suited for scenarios prioritizing speed and
efficiency.However, its performance is hindered by its neck feature fusion
mechanism, which causes the quadratic complexity and the limited guided
receptive fields.To address these limitations, we present Mamba-YOLO-World, a
novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation
Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce
an innovative State Space Model-based feature fusion mechanism consisting of a
Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan
algorithm with linear complexity and globally guided receptive fields. It
leverages multi-modal input sequences and mamba hidden states to guide the
selective scanning process.Experiments demonstrate that our model outperforms
the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and
fine-tuning settings while maintaining comparable parameters and FLOPs.
Additionally, it surpasses existing state-of-the-art OVD methods with fewer
parameters and FLOPs.