# EGSDK-Net: Edge-Guided Stepwise Dual Kernel Update Network for Panoptic Segmentation
## 4 Experiments
### Ablation Study  
We first conduct ablation experiments on the core components of EGSDK-Net. To save on training costs, the maximum number of training iterations for the ablation experiments is set to 60 k. The experimental results and visual comparisons are shown in Table 2 and Figure 6. It is evident that inserting SDKUM into the baseline leads to significant improvements in all three metrics, particularly in the \"thing\" classes, which rise from 48.9 to 50.0. After inserting RTEGM, we obtain the final structure of the proposed EGSDK-Net. It is evident that the scores for PQ and \(PQ_{th}\) keep increasing, while \(PQ_{st}\) shows some decline. We believe this may be related to the Canny operator [26]. This operator extracts edges based on the intrinsic properties of the image, and the contours of the \"stuff\" classes are quite challenging to predict. When the Canny operator [26] inaccurately predicts these contours, it leads to errors in the model, which in turn results in a decline in the evaluation results for the object classes. Given the significant improvement in \(PQ_{th}\) after adding RTEGM, we believe that the benefits of this module outweigh its drawbacks, making it worthwhile to retain. Additionally, the results in Table 2 support our analysis of performance sources discussed in Section 4.3. Next, we investigate the internal structure of RTEGM. We set up a baseline by removing the max pooling and average pooling layers, and the results are shown in the first row of Table 3. Next, we conduct experiments that restore either max pooling or average pooling, as shown in the second and third rows of Table 3. We observe that the enhancement from the max pooling layer is more significant. This may be because the max pooling layer functions similarly to non-maximum suppression, allowing it to eliminate false positive points with relatively lower probability values, thus producing more accurate edge feature maps. In contrast, the improvement from the average pooling layer is less pronounced, and there is a marked decline in the \"stuff\" classes. We think this remains tied to the difficult-to-predict edges of the \"stuff\" classes, as the average pooling layer does not filter out unreliable points as effectively as the max pooling layer. On the contrary, the averaging process across the channel dimension leads to more confusion and complexity in predicting the contours of the \"stuff\" classes, ultimately causing a drop in results. When both pooling methods are applied simultaneously, the results for the three metrics show improvement compared to when they are used separately. This might be due to the combined use of max pooling and average pooling layers, which provides a more thorough evaluation of the edges from two perspectives, thereby leading to better performance in PQ, \(PQ_{th}\), and \(PQ_{st}\). In addition, we conduct experiments on RTEGM as shown in Table 4. We remove the existing structure of RTEGM and its corresponding supervision, instead relying entirely on the non-trainable Canny [26] for edge prediction. The results indicate that the current strategy significantly outperforms the direct use of Canny [26]. While the results of Canny [26] act as the ground truth for the edge module during training, why is the direct application of Canny [26] less effective? We attribute this to the backpropagation process, as the direct integration of Canny [26] remains separate from the entire system, which prevents timely feedback on the \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Backbone** & **PQ** & \(PQ_{th}\) & \(PQ_{st}\) & **GPU** \\ \hline Panoptic- & SWideRNet- & 58.4 & - & - & V100 \\ DeepLab [11] & (0.25,0.25,0.75) & 58.8 & 52.1 & 63.7 & V100 \\ [PERSON] et al. [13] & ResNet50-FPN & 58.8 & 52.1 & 63.7 & V100 \\ Panoptic & ResNet-18 & 55.9 & - & - & RTX3090 \\ YOSO. [6] & ResNet50 & 59.7 & 51.0 & 66.1 & V100 \\ RT-K-Net [5] & RTFormer & 59.3 & 48.9 & 66.7 & V100 \\ \hline Ours & RTFormer & **60.6** & 51.9 & **66.8** & V100 \\ \hline \hline \end{tabular}
\end{table}