## 2 Proposed Network
### Feature Extractor  
Hourglass network was introduced for the human pose estimation task by [PERSON] et al. [28]. The network consists of bottom-up and top-down structured modules, where input channels are expanded and dimensions of the feature maps are down-sampled by a series of convolutional, stride and max-pooling operations. Subsequently, upsampling operations are performed to produce symmetric feature map blocks in hourglass style. Skip connections are added during upsampling to prevent the loss of information. Hourglass network was used in CornerNet [16] for object detection. After that, the same framework was used in CenterNet [14]. Our hourglass-like feature extractor network employs the same arrangement of convolution blocks as [16]. In the feature extractor network, first, input feature maps (\(3\times 128\times 128\)) are passed through a convolutional block which reduces the input dimension by half by using a \(7\times 7\) convolutional operations and a stride of size 2 with 128 channels. After that the feature maps are fed into a residual block with \(3\times 3\) convolutional operations and a stride of size 2. It produces a feature maps with 256 channels and spatial dimension of \(128\times 128\). Subsequently, feature maps are fed into stacked hourglass modules to produce feature maps with global spatial and semantic information. Hourglass module consists of bottom-up and top-down design with residual learning blocks. There are five stages in downsampling and upsampling operations. The processing modules at each stage, including the skip connection modules (there are skip connections between symmetric blocks of a hourglass module, referring to Figure 2), consists of two residual blocks. Each residual block includes two convolutional layers and one skip connection layer. The spatial dimension of the feature map is reduced by a stride of size 2 which is employed for the first convolutional operation in the residual block. The rest of the convolution operations (including the second residual block) use a stride of size 1 and keep the spatial dimension unchanged. The kernel size of 3 \(\times\) 3 is used in every convolutional operation. The skip connection layer in the residual blocks uses linear transformation (1 \(\times\) 1 convolution) and matches the spatial and channel dimension of the input feature maps with the output of the convolution layer. The spatial resolution of feature maps is reduced by 5 times and the number of channels increases as [256; 384; 384; 512] along the way. Upsampling of feature maps is performed using the nearest-neighbor algorithm, followed by two residual blocks at each stage. The final output feature map has 256 channels and a 128 \(\times\) 128 spatial dimension. The detailed structure of the hourglass module is described in Table 1.  
Figure 1: Overall network architecture of Dual-Attention LogoNet. Based on the original hourglass architecture, our proposed network densely aggregates convolutional layers into each residual block at different scales. Each residual block has two convolutional layers and a skip connection layer. Residual learning uses a skip connection to add with the output of the second convolutional layer. We propose to aggregate outputs of both convolutional layers with skip connection within each convolutional block inspired by [29]. In each residual block, both convolutional operations and the skip connection layer generate feature maps of the same spatial and channel dimensions so that these output feature maps can be directly added without increasing network overhead. \[X=X_{s}+X_{1}+X_{2} \tag{1}\]  
where the input feature map passes through convolutional operations, \(X_{1}\) and \(X_{2}\) are the output of the two convolutional operations.