MAM: A multipath attention mechanism for image recognition

IET Image ProcessingVolume 16, Issue 3 p. 691-702 ORIGINAL RESEARCH PAPEROpen Access MAM: A multipath attention mechanism for image recognition Hao Zhang, Hao Zhang orcid.org/0000-0002-0404-6941 School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorGuoqin Peng, Guoqin Peng School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorZhichao Wu, Zhichao Wu School of Artificial Intelligence, Beijing Normal University, Beijing, ChinaSearch for more papers by this authorJian Gong, Jian Gong School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorDan Xu, Corresponding Author Dan Xu danxu@ynu.edu.cn orcid.org/0000-0003-4602-3550 School of Information Science and Engineering, Yunnan University, Kunming, China Correspondence Dan Xu, School of Information Science and Engineering, Yunnan University, Kunming, China. Email: danxu@ynu.edu.cnSearch for more papers by this authorHongzhen Shi, Hongzhen Shi School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this author Hao Zhang, Hao Zhang orcid.org/0000-0002-0404-6941 School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorGuoqin Peng, Guoqin Peng School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorZhichao Wu, Zhichao Wu School of Artificial Intelligence, Beijing Normal University, Beijing, ChinaSearch for more papers by this authorJian Gong, Jian Gong School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this authorDan Xu, Corresponding Author Dan Xu danxu@ynu.edu.cn orcid.org/0000-0003-4602-3550 School of Information Science and Engineering, Yunnan University, Kunming, China Correspondence Dan Xu, School of Information Science and Engineering, Yunnan University, Kunming, China. Email: danxu@ynu.edu.cnSearch for more papers by this authorHongzhen Shi, Hongzhen Shi School of Information Science and Engineering, Yunnan University, Kunming, ChinaSearch for more papers by this author First published: 13 December 2021 https://doi.org/10.1049/ipr2.12370 The code for this paper is available at: https://github.com/haoyev5/A-Multipath-Attention-Mechanism-for-Image-Recognition- AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Attention mechanism has shown excellent performance in many computer vision tasks, while the previous literature may not adequately consider different types of attention mechanisms or is individual elaborate designed for a certain network. In this paper, a general yet effective multipath attention mechanism (MAM) to explore the effect of visual attention for image recognition is proposed. In contrast with other attentions that leverage global pooling, the main advantage is that the MAM considers both the correlation of featuremaps and different scale structural information into account. The backbone representations are enhanced by adding MAM laterally along independent and separate dimensions, channel and spatial. Due to only a simple and unified calculation block is generated, MAM can be flexibly integrated into various CNNs within few parameters and trained together end-to-end. Furthermore, the topology structures of attention path arrangement are investigated using different connection schemes. Experimental results on several image recognition datasets show that the model outperforms various existing models. Finally, performance improvement through visualisation is intuitively discussed. The source code for the proposed attention module is publicly available. 1 INTRODUCTION Imagine the way human vision observes. When observing an image, we are not looking at the entire picture. It is often done by quickly scanning the global image to obtain the attention areas and then combining them to form the final understanding. Research findings have shown that the visual attention mechanism, a unique brain signal processing mechanism for human vision, often plays a vital role in the human cognitive process [1-3]. For example, whether it is a red flower or a white car, they can attract our attention more in a grey background. With this strategy, we can easily identify subtle but dominant differences in similar objects in our visual system. Thanks to neural networks' powerful representation capabilities, it has been widely used in image recognition and has shown excellent performance [4-7]. Połap et al. [8] applied convolutional neural networks (CNNs) to the monitoring system on the river, using a cascade of artificial intelligence techniques including penalties and rewards. Similar approaches were based on a neural bag-of-words mechanism. Wu et al. [4] employed CNNs to construct a medical burn image recognition system. Given the low accuracy of traditional power equipment recognition, Zhou et al. [9] proposed a technique based on dual-channel CNNs and random forest. Diverse applications with novel technology have greatly enriched the development of image recognition. In addition, some researchers starting from the network structure, which is mainly dedicated to building a more profound and broader network model or finding more suitable optimisation targets for different visual tasks [10-13]. Furthermore, the Transformer architecture has become the de facto standard for natural language processing tasks (NLP). However, its application in computer vision (CV) is still limited. In vision tasks, attention is either applied in conjunction with CNNs, or adopted to replace specific components of CNNs while keeping their overall structure in place [14, 15]. In particular, Dosovitskiy et al. abandoned the dependence of attention on CNNs and applied pure transformers directly to the sequence of image patches [16]. Even though the transformer-based model can improve network performance, it generally contains more parameters and increases training costs. Therefore, we prefer to care about CNNs structure with attention module. By enhancing the visual representations of original low dimensional embedding to identify subtle but salient features, the deep model can achieve better performance [2, 3, 17, 18]. Previous attention literature has explored channel and spatial dependencies. SENet-like channel-wise attention [19-21] employ global average pooling to compress channel statistics and then adopt a multi-layer perceptron to predict a set of attention factors and apply them together with the original featuremaps. STN-like spatial-wise attention [22-24] take average or maximum statistics of low dimension embedding and then employ a sizable large convolution kernel operation. Finally, an attention mask is determined to emphasise or suppress the input featuremaps. Some studies combine channel attention and spatial attention [25, 26]. Simply speaking, attention mechanisms can guide networks to model the interdependencies of channels and locate interest regions to identify the differences of similar objects quickly. Similarly, to improve deep representations for image recognition, we prefer to build an attention module to capture the salient attributes of different visual features. In this paper, to tackle these issues, we present a simple multipath attention module, which combines the channel-wise attention strategy and multipath spatial attention mechanism with the backbone layout. Our attention module captures the cross-channel feature correlations and spatial response while preserving independent representations in the original structure. As shown in Figure 1, this module consists of a channel attention module (CAM) and a spatial attention module (SAM). The proposed MAM exploits both inter-and intra-channel relationships by using the proposed CAM and SAM, respectively. CAM performs a set of transformations on low-dimensional embeddings to capture the interdependence of featuremaps, and interest regions are obtained through SAM. We refer to such computation block as multipath attention module (MAM). Since only several convolutional layers and activations, few parameters are introduced. Besides, our proposed MAM can be integrated into various CNNs to form new networks, which refer to as MAMNet. Experimental results demonstrate our proposed multipath attention module performance via both thorough analysis and fair comparison with the state-of-the-art methods. FIGURE 1Open in figure viewerPowerPoint The overall architecture of the proposed multipath attention module. It consists of a channel attention module and a spatial attention module. Channel attention models the correlation between feature channels and improves performance by enhancing essential features. Spatial attention captures the multiscale information of interest regions in the featuremaps The main contributions of this paper are as follows: (1) Our MAM can be integrated into different backbone CNN, forming MAMNet for image recognition. Since only a simple calculation block is introduced, it can be flexibly integrated into various CNNs within few parameters and trained together end-to-end. (2) Following the topology of the attention path connection, we developed two alternative topologies using different connection schemes, i.e. cascaded and parallel topologies. In most cases, the performance of cascaded topology outperforms parallels. (3) The proposed attention mechanism can extract and fuse multipath features, thereby correcting intermediate representations without changing original structure. Experiments obtained from several datasets show the effectiveness of the proposed module. 2 RELATED WORKS Visual attention has attracted extensive attention from researchers in CNNs. Boltzmann constructed a top-down attention mechanism by reconstructing its structure in training [27, 28]. Another model, such as the RNN, LSTM and GRU, also introduced a visual attention mechanism [29-31]. The deep model can achieve complete information flow and better performance by recombining the features extracted from the encoder and providing additional information to the decoder. In image recognition, RANs [32] first explicitly applied the attention mechanism in image classification and obtained the most advanced performance at the time; the self-codec was used to calculate the attention in the spatial domain, but this mechanism was more complex. As a result, other models face many detailed challenges when they adopt this mechanism, thus, the result is unsatisfactory. Momenta, an autonomous driving company, proposed a novel image recognition structure, SENet [19], which generated attention based on the channel of the featuremaps, and then multiplied it by original featuremaps. In other words, the correlation is modelled between the feature channels, and the performance is improved by enhancing the essential features. However, SENet only focuses on the channel level where some layers have stronger feedback, and the spatial attention information cannot be well-reflected. Conversely, ref. [22] only considered spatial attention without channel attention information. A spatial domain attention module is proposed to form a spatial transformation network to address the effects of featuremaps due to translation rotation, scaling etc. Ref. [25] offered a new CBAM, which performs attention on channels and spatial axes. When calculating spatial attention, an additional global maximum pooling branch is introduced as a supplement to the global average pooling branch. However, global pooling will cause the information contained in the featuremaps to be lost. Some studies have adapted the attention mechanism to super-resolution. For example, the BAM [33] and RAM [34] obtain a 3D attention map through two parallel separated paths, i.e. channel and position attention, and then multiply them with the backbone network to modify the original featuremaps. Moreover, ref. [35] explores the internal correlation of the interlayer features and develops a nonlocally residual-enhanced group (NLRG) to capture the long-distance spatial content information, which optimises the attention mechanism for super-resolution tasks. Combine a series of training strategies, Zhang et al. [36] developed a deep model pipeline composed of stacking a series of split attention blocks. Unlike other plug-in-style attention networks, the split attention block divides the input representations into several Cardinals, which contain a set of splits. Therefore, split attention prefers to adjust input by partial correction, which is contrary to the overall adjustment in this work. The meticulous special design also makes the internal structure difficult to modify. Inspired by the outperformance of transformer in the NLP community, some researchers use it for image recognition. Dosovitskiy et al. divided the image into image blocks and applied the pure Transformer directly to the image patches sequence [16]. However, the input of image patches to the transformer still fails to get rid of linear engineering, and it can be regarded as a kind of convolution. Some studies use transformer either applied in conjunction with CNN or to replace specific components of CNN [14, 15]. The current CV studies usually directly apply the transformer structure from NLP and do not explicitly design for vision tasks. However, the amount of information of an image is much more significant than that of text, so the current computational overhead is still considerable. In general, the previous methods may not fully consider different types of attention mechanisms or have carried out a separate careful design for ResNet. Visual attention can selectively enhance or suppress features so that visual representations can be better captured and transfer. The previous methods may not fully consider different types of attention mechanisms or have carried out a separate design for ResNet. Some models are more complicated to build and bring a certain amount of parameters and calculations, making training time and hardware costs relatively high. When it is necessary to build a deeper level, this also puts forward higher requirements on the datasets of the training model, which leads to more significant difficulties in the practical application of many computer vision tasks. These factors make it limited to be promoted in specific practical applications. In this paper, we propose a hybrid attention mechanism module by combining channel and spatial attention modules (CAM and SAM) in a uniform framework, as shown in Figure 2. CAM models the correlation between channels, learns the dependence of each channel in the featuremap and adjusts accordingly. While in SAM, different scales interest regions are obtained by multiple paths, from local to global. Dilated convolutions of different rates are adopted to gain multi-scale information instead of violent global pooling with asynchronous length like CBAM [25]. On the one hand, global pooling may cause a large amount of information to be lost and weaken the extraction of features. The multipath construction enables our attention mechanism to extract sufficient features to help information transmission from detail to whole. On the other hand, the featuremaps are separated and combined into group convolutional layers to control additional parameters and computational cost. Note that we employ dilated convolution instead of convolution as the former has a larger receptive field to obtain more global information, with the same additionally parameters. Then the 1 × $\times$ 1 convolution is adopted as a supplement to the visual details. It is ensured that the proposed MAM can extract subtle but discriminative features and can be easily embedded into other CNN models. FIGURE 2Open in figure viewerPowerPoint CAM and SAM. The left panel is the CAM, and the right panel is the SAM. The channel attention module passes through two convolutional layer encodings after global average pooling, and finally encodes the input features after activation with the sigmoid. Spatial attention is grouped by the channel for multi feature extraction, and then, the output of the grouping is concatenated and fed back to the features 3 MULTIPATH ATTENTION MODULE 3.1 Network architecture The architecture of our proposed MAM is illustrated in Figure 1. The MAM consists of a CAM and a SAM, which can detect the attention information of the featuremaps from the channel and space. This architecture can be embedded between any layer of any CNN structure to enhance or suppress the representations of each layer. Therefore, the model of the image recognition task can be trained more accurately and effectively. Given a CNN with L layers, F l ∈ R H × W × C $\mathbf {F}_{l} \in \mathbb {R}^{H \times W \times C}$ is the featuremaps of the l $l$ th layer. Assuming that our proposed multipath attention mechanism structure is embedded between the l $l$ th and l + 1 $l+1$ th layers, F l ∈ R H × W × C $\mathbf {F}_{l} \in \mathbb {R}^{H \times W \times C}$ is the input feature of the MAM. First, F l $\mathbf {F}_{l}$ flows into the CAM, resulting in a 1D channel attention vector, denoted as C A l ( F l ) ∈ R H × W × C $ \mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})\in \mathbb {R}^{H \times W \times C}$ . Then F l $\mathbf {F}_{l}$ is multiplied by C A l ( F l ) $\mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})$ , and the resulting featuremaps F ′ $\mathbf {F}^{\prime }$ is adopted as the input of the SAM. The spatial attention featuremaps S A l ( F l ) ∈ R H × W × 1 $ \mathbf {S} \mathbf {A}_{l}(\mathbf {F}_{l})\in \mathbb {R}^{H \times W \times 1}$ is produced by the SAM. S A l ( F l ) $\mathbf {S} \mathbf {A}_{l}(\mathbf {F}_{l})$ acts as weight times F ′ $\mathbf {F}^{\prime }$ , resulting in F ′ ′ $\mathbf {F}^{\prime }{\prime }$ . The final featuremaps F l + 1 $\mathbf {F}_{l+1}$ is generated by cooperating with the original feature F l $\mathbf {F}_{l}$ . The formulation of our MAM is as follows: F ′ = C A l F l ⊙ F l F ′ ′ = S A l F ′ ⊙ F ′ F l + 1 = F ′ ′ + F l \begin{eqnarray} \begin{aligned} &\mathbf {F}^{\prime }=\mathbf {C} \mathbf {A}_{l}{\left(\mathbf {F}_{l}\right)} \odot \mathbf {F}_{l}\\ &\mathbf {F}^{\prime \prime }=\mathbf {S A}_{l}{\left(\mathbf {F}^{\prime }\right)} \odot \mathbf {F}^{\prime }\\ &\mathbf {F}_{l+1}=\mathbf {F}^{\prime \prime }+\mathbf {F}_{l} \end{aligned} \end{eqnarray} (1) where ⊙ $\odot$ denotes elementwise multiplication, namely, the Hadamard product. C A l ( · ) $\mathbf {C} \mathbf {A}_{l}(\cdot )$ and S A l ( · ) $\mathbf {S} \mathbf {A}_{l}(\cdot )$ are the operations of the CAM and SAM, respectively. Therefore, our proposed MAM learns two types of attention information, i.e. the channel attention vector and spatial attention mask. Both are weights of the input feature and have the same size input features. Thus, the two attention modules generate the mask of the input feature, and Equation (1) can be reduced to the following: F l + 1 = mask F l + 1 ⊙ F l \begin{eqnarray} \begin{aligned} \mathbf {F}_{l+1}={\left(\mathbf {mask}{\left(\bm{F}_{l}\right)}+1\right)} \odot \mathbf {F}_{l} \end{aligned} \end{eqnarray} (2) where mask ( · ) $\mathbf {mask} (\cdot )$ involves CA l ( · ) $\mathbf {CA}_{l}(\cdot )$ and SA l ( · ) $\mathbf {SA}_{l}(\cdot )$ . The two modules are shown in Figure 2. We will discuss them in detail later. Algorithm 1 gives a consistent description of the calculation process. ALGORITHM 1. Multipath attention module 3.2 Channel attention module It is well-known that featuremaps at different levels have multiple channels that represent different characteristics, such as colour and texture, which are based on different convolutions with different kernels. For different objects, different characteristics are vital to distinguish them. For example, colour may be a key feature of some objects, while texture may be a key feature of another object. Therefore, the CNN model should strengthen the key features of different objects, which is helpful to improve the object recognition ability of the model. This strengthening is what the visual attention of the channel achieves. For the channel visual attention mechanism, some previous works proposed the multilayer perceptron (MLP) [19, 25, 34]. However, MLP is not good at reducing or increasing the number of channels in the convolution process. Thus, two convolution layers with a 1 × $\times$ 1 kernel are adopted in the CAM. As a result, the proposed CAM with the nonlinear transformation becomes a more powerful representation without introducing additional parameters. Compared with the MLP, our CAM does not change the feature dimension frequently, which is more convenient and efficient. F l ∈ R H × W × C $\mathbf {F}_{l} \in \mathbb {R}^{H \times W \times C}$ are the featuremaps of the l $l$ th layer and a 1D channel feature vector C A l ( F l ) ∈ R H × W × C $ \mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})\in \mathbb {R}^{H \times W \times C}$ is learned from the CAM. C A l ( F l ) $ \mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})$ is represented as follows: C A l F l = σ Conv × 2 GAP F l \begin{eqnarray} \begin{aligned} \mathbf {C} \mathbf {A}_{l}{\left(\mathbf {F}_{l}\right)}=\sigma {\left({\text{Conv}}_{\times 2}{\left({\text{GAP}}{\left(\mathbf {F}_{l}\right)}\right)}\right)} \end{aligned} \end{eqnarray} (3) where GAP ( · ) ${\text{GAP}} (\cdot )$ and Conv ( · ) ${\text{Conv}} (\cdot )$ are global average pooling and convolution operations, respectively. The subscript × $\times$ 2 means that convolution is carried out twice. σ ( · ) $\mathbf {\sigma } (\cdot )$ is the sigmoid activation function. The dimension of C A l ( F l ) $\mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})$ corresponds to the number of channels of the lth layer. C A l ( F l ) $\mathbf {C} \mathbf {A}_{l}(\mathbf {F}_{l})$ is regarded as the weight of each channel, and times of each channel. Thus, the CAM can learn different weights for every channel. The channels with greater weights are more important for our recognition task. As shown in Figure 2, circled by a green dotted frame, ReLU $\text{ReLU}$ is executed after the first convolution in the network structure of the CAM. 3.3 Spatial attention module The CAM focuses on the characteristics of objects, while the SAM pays more attention to spatial positions that should be focused on featuremaps. Previously, the spatial attention mechanism located cues mainly through downsampling or global pooling. Since the attention branch needs to be merged into the backbone network, downsampled feature faces the problem of dimensional reconstruction, which will increase the amount of calculation. Global pooling may cause a certain degree of lack of feature information. For image recognition tasks, there may be some large-scale features, such as the contour features of the object, foreground, and background. Global pooling may not be able to detect them well and guide information pass. To this end, the MAM presents the multipath spatial attention module to encode the low-dimensional embedding of the backbone network to locate the spatial response of featuremaps and enhance the information representation. As shown in Figure 2, SA is circled by a red dashed frame. To ensure that the model can effectively detect the small-scale features of the object, 1 × $\times$ 1 and 3 × $\times$ 3 convolution kernels are employed. While for the large-scale objects, two sets of dilated convolutions with different dilate rates are used to ensure sufficient receptive fields and get more significant feature representations. Simultaneously, to reduce the computing cost improve feature utilisation, the above operations on the featuremaps are performed grouped by channels instead of on all channels. We prefer our attention module makes it easy to identify subtle but dominant differences of similar objects in the neural perception system. Due to the CAM and SAM cascades, the output of the CAM, F l ′ ∈ R H × W × C $\mathbf {F}^{\prime }_{l} \in \mathbb {R}^{H \times W \times C}$ , is the input of the SAM, as shown in Equation (1). The SAM can learn a 2D featuremaps, i.e. S A l ( F l ′ ) ∈ R H × W × 1 $ \mathbf {S} \mathbf {A}_{l}(\mathbf {F}^{\prime }_{l})\in \mathbb {R}^{H \times W \times 1}$ . First, F l ′ $\mathbf {F}^{\prime }_{l}$ is grouped by channel. F l , i ′ $\mathbf {F}^{\prime }_{l,i}$ is the i $i$ th group of the l $l$ th layer, where i ∈ { 1 , 2 , 3 , 4 } $i\in \lbrace 1,2,3,4\rbrace$ means the number of groups. The first two groups are convoluted with kernels of 1 × $\times$ 1 and 3 × $\times$ 3. The last two groups undergo dilated convolution with a kernel of 3 × $\times$ 3 and dilation rates of 2 and 3. The different dilation rates are employed to obtain different resolutions and reception fields of featuremaps to improve the feature representation. Then, the convolutional result of all groups is concatenated, resulting in a convolutional layer with a 1 × $\times$ 1 kernel, which makes the dimension of concatenated features the same as that of the initial input features fusion features. The proposed SAM is formulated as Equation (4): Λ = Conv F l , 1 ′ , Conv F l , 2 ′ , DC F l , 3 ′ , 2 , DC F l , 4 ′ , 3 SA F l ′ = σ Conv c \begin{eqnarray} && {\bm \Lambda } = {\left[{\it \text{Conv}}{\left(\mathbf {F}_{l, 1}^{\prime }\right)}, {\it \text{Conv}}{\left(\mathbf {F}_{l, 2}^{\prime }\right)}, {\it \text{DC}}{\left(\mathbf {F}_{l, 3}^{\prime }, 2\right)}, {\it \text{DC}}{\left(\mathbf {F}_{l, 4}^{\prime }, 3\right)}\right]}\nonumber\\ &&\quad \mathbf {SA}{\left(\mathbf {F}_{l}^{\prime }\right)} =\sigma {\left({\it \text{Conv}}{\left({\bf c}\right)}\right)} \end{eqnarray} (4)where DC ( · ) $\text{DC} (\cdot )$ means dilated convolution, k in DC ( · , k ) $\text{DC} (\cdot ,k)$ means dilation rate and c ${\bf c}$ is the concatenate operation results of group convolutional. The final spatial attention mask SA ( F l ′ ) $ \mathbf {S A}(\mathbf {F}_{l}^{\prime })$ indicates where the original featuremaps should be emphasised and suppressed in the spatial position. 3.4 Topological structure As mentioned above, the proposed MAM consists of a CAM and a SAM. The CAM focuses on the channels, while the SAM strengthens the spatial that should be considered. How to combine them is a question that must be concerned. We design two topological structures and desire to determine the optimal structure by comparing their performances. Two topological structures are shown in Figure 3. For the cascaded structure, the SAM is arranged behind the CAM and activated by correcting the features in a sequence. For the parallel structure, the SAM input featuremaps proceeds to the excitation process without going through the channel attention process. Therefore, the CAM and SAM are considered at the same time. FIGURE 3Open in figure viewerPowerPoint Topologies of our MAM. The left is a cascaded connection, while the right is a parallel connection. Compared with the cascaded structure, the SAM input featuremaps in the parallel structure proceeds to the excitation process without going through the channel attention process For the first topology, it is intuitive to determine "which" first and then "where". First, the weight of the channel is determined in the CAM, that is the discriminative characteristics are found. Then, these channels are reinforced by weighting. Second, the spatial positions should be determined. The weight of each position can be determined in the SAM. For the second topology, the CAM and SAM work on the input features in parallel. The two output features are concatenated with each other, and a 1 × 1 $1 \times 1$ convolutional layer is adopted to reduce the dimension. The analysis of these two topologies and their performance comparison will be carried out in Section 4. The experimental results demonstrate that the cascaded topology outperforms the parallel topology. 4 COMPARATIVE EXPERIMENTS In this section, several experiments were performed to analyse the MAM performance and intuitively expressed it through visualisation. Our proposed MAM is implanted into many mainstream CNN models such as ResNet [12] with different layers, DenseNet [13] and VGG [10]. Experiments demonstrate that our proposed module can continuously improve the performance and outperforms state-of-the-art methods, including SPANet [37], ResNeSt [36] and visual transformer [14] etc. For easy observation, the best results are marked in bold in tables. 4.1 Datasets and implementation details Due to the limitation of computing resources, we analyse the performance of our proposed modules on CIFAR-10 [38], CIFAR-100 [39] and downsampled ImageNet [40]. Both CIFAR-10 and CIFAR-100 hav