# EGSDK-Net: Edge-Guided Stepwise Dual Kernel Update Network for Panoptic Segmentation
## 3 Method
### Stepwise Dual Kernel Update Module
The purpose of doubling the channel count during the use of fully connected layers is to output two updated kernels in the subsequent steps: one is the current stage kernel \(K_{i}\) and the other is an early prediction of the next stage kernel \(K_{i+1}\), which is only temporarily retained. (3) Kernel Interaction The kernel interaction process facilitates the sharing of contextual information among different groups of each kernel. Considering both convenience and effectiveness, it is common to use a combination of multi-head attention and feedforward neural network for kernel interaction, resulting in the updated kernel \(K_{i}\) at the current stage, which can be formulated as follows:  
\[K_{i},K_{i+1}=Split\left(FFN\left(MHSA\left(\widetilde{K}\right)\right) \right). \tag{7}\]Figure 4: Kernel and mask initialization utilized in K-Net [2] (**a**), RT-K-Net [5] (**b**) and EGSDK-Net (**c**). We initialize \(K_{1}\) for the first stage of kernel updates while initializing the \(K_{0}\). Figure 5: The structure of the kernel update in SDKUM. The entire update process comprises n iterative stages, with each stage receiving the kernel \(K_{i-1}\), mask \(M_{i-1}\), and pre-predicted kernel \(K_{i}\) from the preceding stage as inputs (the initial inputs for the first stage are the initialized kernel \(K_{0}\), mask \(M_{0}\), and kernel \(K_{1}\)). Upon completion of the update process, the predicted class \(Cls_{n}\) and mask \(M_{n}\) are output. Subsequently, we utilize different feedforward neural networks (FFNs) to generate predictions for both the mask and class probabilities. This can be formulated as follows:  
\[M_{i}=FFN_{M}(K_{i})*F; \tag{8}\]  
\[Cls_{i}=FFN_{C}(K_{i})*F. \tag{9}\]  
Before entering the next repeated update stage, we concatenate the additional predictions from the previous stage for the current stage kernel with \(K_{i}\), formulated as follows:  
\[K_{i}=FFN\Big{(}\Big{[}K_{i},K_{i}\Big{]}\Big{)}, \tag{10}\]  
where \([. ]\) represents the channel concatenation operation. At this point, the process of kernel updating is complete. Our SDKUM is obtained by combining the kernel and mask initialization proposed in RT-K-Net [5] with this kernel updating operation. Our innovation primarily lies in increasing the number of channels in the previous stage and additionally predicting the kernel for the next stage, which provides two advantages compared to conventional methods: (1) The information extracted from the previous stage is directly incorporated into the update step of the next stage, leading to better information utilization within the module. (2) Since the process of generating \(K_{i}\) and \(K_{i+1}\) involves extensive information exchange, this means that \(K_{i+1}\) is rich in content from the current stage. Additionally, with the effects of backpropagation in iterations, \(K_{i+1}\) also exhibits characteristics of the next stage kernel, thereby contributing to the reduction of semantic information gaps between stages. In conclusion, the stepwise dual kernel update module promotes the effective use of stage information with only a slight increase in model complexity, thereby improving the consistency between the stage kernels. Detailed experimental validation will be presented in Section 4.4.