# MLKD-Net: Lightweight Single Image Dehazing via Multi-Head Large Kernel Attention
## 3 Method
### Multi-Head Large Kernel Attention (MKLA) Module  
To improve spatial modeling within CNN-based architectures, this study introduces the Multi-Head Large Kernel Attention (MLKA) module, which extends the original LKA [12] by employing multiple attention heads.  
Specifically, the MLKA module introduces multiple attention heads to expand the diversity of receptive fields and enhance the model's representational capacity. By dividing different channel subsets into separate large kernel attention paths, the model can capture richer and more diverse spatial dependencies. In addition, a Channel Shuff  
Figure 1: This figure contains five panels: (**a**) overall architecture of the proposed MLKD-Net. (**b,c**) MLKD block structure. (**b**) Multi-Head Large Kernel Attention (MLKA) module. (**d**) Large Kernel (LK) block. (**e**) Channel Shuffle Block.  
file block is integrated to facilitate inter-head communication and improve the module's representational efficiency.  
This design maintains a balance between computational efficiency and expressive power, resulting in improved quantitative performance, as validated through the ablation studies shown in Section 5.1.  
\[[X_{0},X_{1}]=\text{Split}(\text{Conv}_{1\times 1}(\text{BN}(F))) \tag{2}\]  
\[\widetilde{X}=\text{Concat}(\text{LK}(X_{0}),\text{LK}(X_{1})) \tag{3}\]  
\[X=\sigma(\text{DWConv}(\text{CS}(\widetilde{X}))) \tag{4}\]  
As illustrated in Figure 1b, the MLKA module comprises three key stages: (1) channel-wise splitting, (2) spatial feature extraction, and (3) feature refinement. Initially, MLKA applies Batch Normalization (BN) followed by a 1 \(\times\) 1 convolution to the input feature map, after which the output is split into multiple attention heads (Equation (2)). Each head is independently processed using a Large Kernel (LK) block to capture wide spatial context, and the resulting outputs are concatenated (Equation (3)). The LK block, shown in Figure 1d, comprises two components: a standard depth-wise convolution (DWConv) and a dilated depth-wise convolution (DWConv). Instead of relying on a single large kernel, this design approximates a wide receptive field by combining these two operations, following the decomposition scheme proposed in [12]. Specifically, the effective kernel sizes of the two branches are defined as  
\[\text{DWConv}=(2d-1)\times(2d-1) \tag{5}\]  
\[\text{DWConv}=\left\lfloor\frac{K}{d}\right\rfloor\times\left\lfloor\frac{K}{d}\right\rfloor \tag{6}\]  
where \(K\) is the original kernel size, and \(d\) denotes the dilation rate. By combining these two operations, the LK block effectively approximates the receptive field of a much larger convolution (e.g., \(K\times K\)) while significantly reducing computational complexity.  
The concatenated features are subsequently refined using a Channel Shuffle block, as described in Equation (4). Within this block, the Channel Shuffle (CS) operation refers specifically to the process of reshaping and transposing the feature map to interleave the output channels from multiple attention heads. In the MLKA module, each head processes a distinct subset of channels independently to capture diverse spatial patterns. However, this independence may lead to isolated feature learning and hinder effective integration across heads. The CS operation addresses this by mixing channel information across heads, enabling the subsequent 7 \(\times\) 7 depth-wise convolution to access a more diverse and unified representation. This enhances inter-head communication and contributes to improved spatial modeling performance.  
Through this process, spatial features learned by different heads are effectively fused, allowing their outputs to influence one another. This inter-head communication not only mitigates the potential fragmentation of learned features but also enhances the overall expressiveness and spatial modeling capability of the MLKA module.