# Visual Clue Guidance and Consistency Matching Framework for Multimodal Named Entity Recognition
## 3 Methodology
### Visual Clue Guidance Module
#### 3.4.1 Cross-Scale Attention Module  
The detailed structure of the cross-scale attention (CSA) module is depicted in Figure 6. Given an input feature map \(\mathbf{F}\in\mathbb{R}^{\mathrm{C}\times H\times W}\), CSA infers a 3D attention map \(\mathbf{M}(\mathbf{F})\in\mathbb{R}^{\mathrm{C}\times H\times W}\). The refined feature map \(\mathbf{F}^{\prime}\) is computed as follows:  
\[\mathbf{F}^{\prime}=\mathbf{F}+\mathbf{F}\otimes\mathbf{M}(\mathbf{F}) \tag{1}\]  
where \(\otimes\) denotes element-wise multiplication. We employed a residual learning scheme combined with attention mechanisms to facilitate gradient flow. To design an efficient yet powerful module, we initially computed channel attention \(\mathbf{M_{c}}(\mathbf{F})\in\mathbb{R}^{\mathrm{C}}\) and spatial attention \(\mathbf{M_{s}}(\mathbf{F})\in\mathbb{R}^{H\times W}\) separately in two branches. The attention map \(\mathbf{M}(\mathbf{F})\) was then calculated as follows:  
\[\mathbf{M}(\mathbf{F})=\sigma(\mathbf{M_{c}}(\mathbf{F})+\mathbf{M_{s}}( \mathbf{F})) \tag{2}\]  
where \(\boldsymbol{\sigma}\) is the sigmoid function. The outputs from both branches were resized to \(\mathbb{R}^{\mathrm{C}\times H\times W}\) before being added. Channel attention branch. We leverage relationships between channels since each channel contains specific feature responses. To aggregate feature maps in each channel, we apply global average pooling to the feature map \(\mathbf{F}\), generating a channel vector \(\mathbf{F_{c}}\in\mathbb{R}^{\mathrm{C}\times 1\times 1}\). This vector softly encodes global information across all channels. To estimate cross-channel attention from the channel vector \(\mathbf{F_{c}}\), we employ a multilayer perceptron (MLP) with a single hidden layer. To save on parameter costs, the hidden activation size is set to \(\mathbb{R}^{\mathrm{C}/r\times 1\times 1}\), where \(r\) is the reduction ratio. Following the MLP, we add a batch normalization (BN) layer [24] to scale the output in proportion to the spatial branch. In summary, the channel attention computation is as follows:  
Figure 5: Pyramid feature maps. Figure 6: Structure diagram of the cross-scale attention module. \[\mathbf{M_{c}}(\mathbf{F}) = BN(MLP(AvgPool(\mathbf{F}))) \tag{3}\] \[= BN(\mathbf{W_{1}}(\mathbf{W_{0}}AvgPool(\mathbf{F})+\mathbf{b_{0}}) +\mathbf{b_{1}})\]  
**Spatial attention branch. ** A spatial attention map \(\mathbf{M_{s}}(\mathbf{F})\in\mathbf{R}^{H\times W}\) is generated to emphasize or suppress features at different spatial positions. It is crucial to utilize contextual information to determine which spatial locations should be attended to. Effectively utilizing contextual information requires a larger receptive field. We employ dilated convolution [25] to efficiently expand the receptive field, as dilated convolution is known to construct spatial mappings more effectively than standard convolution. Our spatial branch adopts the \"bottleneck structure\" recommended by ResNet [16], reducing the parameter count and computational cost. Specifically, the feature map \(\mathbf{F}\in\mathbb{R}^{C\times H\times W}\) is projected to the reduced dimension \(\mathbb{R}^{C/r\times H\times W}\) through a \(1\times 1\) convolution, integrating and compressing the feature map in the channel dimension. For simplicity, we use the same reduction ratio \(r\) as the channel branch. After reduction, two \(3\times 3\) dilated convolutions are applied to effectively leverage contextual information. Finally, the features are reduced to the spatial attention map \(\mathbb{R}^{1\times H\times W}\) again through a \(1\times 1\) convolution. To adjust to the scale, a batch normalization layer is applied at the end of the spatial branch.