# EGSDK-Net: Edge-Guided Stepwise Dual Kernel Update Network for Panoptic Segmentation
## 1 Introduction
Yet, we find that while the initialization process has been optimized, the kernel update still maintains the conventional \"progressive\" approach. When the kernel update moves to the next update step, the information from the previous step does not contribute to the process, which undoubtedly leads to a waste of information and ultimately results in suboptimal mask prediction outcomes. As shown in Figure 1, our method achieves the best PQ on the benchmark dataset Cityscapes, surpassing RT-K-Net with a particularly small increase in time cost, and significantly outperforming other real-time panoptic segmentation algorithms [10; 11; 12; 13]. The information from the earlier step cannot only prevent information from heading in the wrong direction but also contribute to stabilizing the inference and training processes. Making effective use of this information will be crucial for enhancing the performance of panoptic segmentation. Based on the considerations mentioned above, we propose a real-time edge guidance module and a stepwise dual kernel update module. The former is designed to better align with the requirements of the panoptic segmentation task. Panoptic segmentation classifies almost all target pixels, while the traditional concept of edges also refers to the boundaries between any objects. The two possess a fundamental similarity in their task properties. Distinguishing different objects through edges will enhance the segmentation accuracy in target boundary areas and promote stronger contrast in target regions. To achieve this, we employ a simple structure to extract edge cues. We begin by taking feature maps from the lower stage, followed by extracting edge features using convolutional structures. We apply supervision to these edge features to ensure ongoing optimization. Finally, we utilize both the max pooling layer and the average pooling layer along the channel dimension to improve attention to spatial edge positions and incorporate the edge information into the backbone's final output to achieve edge-enhanced feature maps. The stepwise dual kernel update module optimizes the kernel update process based on RT-K-Net [5]. Specifically, previous kernel update modules only had sequential connections between the structural stages, which did not fully utilize information from the earlier stage. To this end, we optimize the kernel generation process in the kernel update step. We first generate the kernels for the current and the next stages. Then, the kernel from the current stage is fused with the kernel from the previous stage (if available) and is convolved with the feature maps to generate the mask for the current stage. Meanwhile, the kernel for the next stage is preserved and will be used in the subsequent stage. The combination of the two with RT-K-Net forms our proposed edge-guided stepwise dual kernel update network (EGSDK-Net). Our contributions can be summarized as follows:  
1. We propose a novel architecture for the field of panoptic segmentation, called EGSDK-Net. It liberates panoptic segmentation from the constraints of semantic and instance segmentation for the first time and offers valuable structural contributions to the field. 2. A real-time edge guidance module (RTEGM) is designed. This module not only introduces a novel theory regarding the relationship between edge detection and panoptic segmentation but also enhances segmentation performance through a lightweight structure. 3. A stepwise dual kernel update module (SDKUM) is proposed. Considering the limitations of past methods in utilizing system information, SDKUM addresses this issue through a clever design. By using information more effectively, it successfully promotes advancements in model capabilities. Figure 1: Comparison between EGSDK-Net and state-of-the-art methods [2; 5; 10; 11; 12; 13] on the Cityscapes dataset [14]. Our method outperforms K-Net [2], Panoptic-DeepLab [11], Petrovai and Nedevschi [12], and [PERSON] et al. [13] in terms of Panoptic Quality and runtime.