# Panoptic Segmentation Method Based on Feature Fusion and Edge Guidance
## 1 Introduction  
Panoptic segmentation, as one of the core tasks in computer vision, has witnessed rapid advancements from methodological exploration to performance breakthroughs since [PERSON] et al. [1] formally introduced the concept of panoptic segmentation in 2018, gradually emerging as a prominent research direction in image understanding. Early research in panoptic segmentation often involved integrating separate semantic [2] and instance segmentation techniques. Instance segmentation itself evolved from classical computer vision methods (e.g., thresholding, region growing) towards deep learning approaches. Object detection techniques, progressing from models like Faster R-CNN [3] and YOLO [4] to instance segmentation models like Mask R-CNN [5] by adding mask prediction heads, formed the basis for many early panoptic methods. To unify the tasks, three mainstream panoptic approaches emerged: (1) Top-down methods, typically extending object detectors like Mask R-CNN, such as UPSNet [6], which perform detection then masking but can suffer from inter-task conflicts. (2) Bottom-up methods, like Panoptic-DeepLab [7],which group pixels into instances but can struggle with boundary accuracy in dense scenes. (3) Unified framework methods [8], often leveraging Transformers for end-to-end prediction, such as Mask2 Former [9], which treat 'thing' and'stuff' uniformly and have achieved State-of-the-Art results.  
Despite the success of unified methods like Mask2 Former [9], specific limitations remain that hinder performance in complex scenarios. These include: (1) reliance on static feature pyramid fusion strategies (e.g., simple addition), limiting adaptability to diverse object scales; (2) suboptimal segmentation of small objects and blurred boundaries for intricate structures, potentially due to insufficient high-frequency detail processing in the decoder; and (3) limited interaction pathways in standard decoders, which often process pixel and query information sequentially, hindering optimal pixel-to-instance assignment.  
To address these specific limitations observed in State-of-the-Art models like Mask2 Former, we propose PSM-FFEG, which introduces targeted innovations: (1) a dynamic multi-scale feature fusion mechanism incorporating attention to improve adaptive context modeling, moving beyond static fusion; (2) an explicit edge guidance module with dedicated supervision to enhance boundary definition, particularly for small objects and detailed structures; and (3) a dual-path Transformer decoder enabling richer, iterative interaction between pixel features and object queries to improve segmentation in complex layouts.  
The main contributions of this paper are as follows:  
1. A dynamic multi-scale feature fusion module is proposed, which adaptively captures contextual information through cascaded deformable convolutional kernels and dual-dimensional attention in both channel and spatial domains.
2. An explicit edge feature guidance module is introduced, establishing a bidirectional enhancement mechanism between pixel features and edge gradients.
3. A dual-path Transformer decoder architecture is designed, enabling collaborative optimization of global semantics and local features through cross-attention mechanisms.  
The remainder of this paper is organized as follows: Section 2 reviews related work in panoptic segmentation, feature fusion techniques, and edge-guided approaches. Section 3 details the proposed PSM-FFEG network architecture, including its dynamic fusion, edge guidance, and dual-path decoder modules. Section 4 presents the experimental setup, datasets, evaluation metrics, main results, ablation studies, qualitative analysis, and a discussion of computational cost. Finally, Section 5 concludes the paper, summarizing contributions and limitations, and outlining directions for future work.