# A Dual-Enhanced Hierarchical Alignment Framework for Multimodal Named Entity Recognition
## 3 Methodology
### Multimodal Feature Encoding Layer
#### 3.1.2 Three-Level Visual Pyramid Feature Encoding  
Social media images often contain a multitude of visual objects, yet in most cases only a subset of these objects is associated with the entities mentioned in the text. To address the complex distribution of visual objects and the sparse association with textual entities in social media images, we propose a three-level visual pyramid encoding structure that enhances the model's ability to capture complex visual semantics through multi-scale, fine-grained representations. The visual pyramid comprises three levels of features--grids of \(64\times 64\), \(32\times 32\), and \(16\times 16\), respectively--as illustrated in Figure 2. Visual features are extracted in three granularities: global context, region semantics, and local details. Denser grids cover smaller regions of the original image, yielding finer feature granularity that accommodates the multi-level semantic requirements of entity descriptions in text. Given the inherent differences in semantic demands and modality alignment objectives across varying visual granularities, a single architecture cannot simultaneously balance multi-scale feature extraction and noise suppression. Consequently, we employ distinct neural network architectures for each of the three hierarchical image encoding layers. The global context features (a \(16\times 16\) grid) capture the overall image layout and scene semantics at a macro granularity, requiring high-level semantic abstraction to represent the entire scene. ResNet-101, through its deep convolutional layers and spatial pooling, naturally generates a \(16\times 16\) low-resolution feature map. The final convolutional layer of this deep network excels at extracting macro-level semantics, such as scene categories, while filtering out irrelevant detail noise, thereby aligning with the global descriptions in text. Consequently, we adopt the ResNet-101 [20] backbone to extract global image features \(V_{g}\in\mathbb{R}^{d}\), where \(d=1024\) denotes the feature dimension. The region semantics features (a \(32\times 32\) grid) target meso-level granularity to localize primary objects directly associated with text entities, necessitating precise object-level detection and selection capabilities. Faster R-CNN [21] employs an object detection mechanism to accurately generate candidate regions, retaining the top \(k\) salient local visual objects based on classification confidence, which directly correspond to text entities such as person or brand names. This yields region-level features \(V_{r}=\{v_{r}^{1},\dots,v_{r}^{k}\}\in\mathbb{R}^{k\times d}\), where each region feature \(v_{r}^{i}\in\mathbb{R}^{d}\) encapsulates the semantic information within a detection box, effectively covering key entity-related objects while filtering out low-relevance backgrounds. The local detail features (a \(64\times 64\) grid) focus on micro-level granularity to capture auxiliary details, such as text or textures, requiring the preservation of high-resolution spatial information. We utilize a Dilated Convolutional Network [22] with its dilation property, setting a dilation rate (e.g., \(r=2\)) to expand the receptive field to \(7\times 7\), thereby capturing local context while maintaining a \(64\times 64\) high-resolution feature map. This approach avoids information loss from traditional pooling operations, preserving pixel-level features \(V_{l}\in\mathbb{R}^{h\times w\times d}\), where \(h\times w\) represents the spatial dimensions of the feature map. The large receptive field effectively models local contextual relationships. The three-level features, after dimensionality alignment, are concatenated to form a spatial pyramid feature, denoted \(V=[V_{g};V_{r};V_{l}]\in\mathbb{R}^{(1+khw)\times d}\).