# Triple-Attention Mixed-Link Network for Single-Image Super-Resolution
## 3 Proposed Method
### Mixed-Link Connections  
Figure 2d shows the inner structure of the mixed-link connections. Operator M in the figure denotes the mixed-link operation, which gives a fusion of residual and dense connections between the current layer and previous layer. The mixed-link operation could be calculated as Formulas (1)-(4). This could be divided into three parts. The first slices the input channels into two parts equally; \(S(.)\) means the slice in Formula (1). Suppose the feature map \(x_{i}\) has \(2N\) channels, the output feature maps \(x_{i}^{1}\) and \(x_{i}^{2}\) will contain \(N\) channels after the slicing operation.  
\[x_{i}^{1},x_{i}^{2}=S(x_{i}) \tag{1}\]  
Second, the output of one layer or unit should also be sliced into two equal parts in the channel dimension. In Figure 2, structure (c) and structure (d) could be calculated as Formulas (2) and (3).  
\[\hat{x}_{i}^{1},\hat{x}_{i}^{2}=S(\sigma(W_{1}(x_{i}))) \tag{2}\]  
\[\hat{x}_{i}^{1},\hat{x}_{i}^{2}=S(A_{c}(A_{k}(\sigma(W_{1}(x_{i}))+\sigma(W_{2} (x_{i}))))) \tag{3}\]  
where \(A_{c}\) and \(A_{k}\) denote the channel and kernel attention, \(\sigma\) means the PReLU [17] activation function. \(W_{1}\) and \(W_{2}\) means the weight of convolution layer \(ConvK_{1}\) with \(3\times 3\) kernel size and \(ConvK_{2}\)\(5\times 5\) kernel size in Figure 2d.  
\[x_{i+1}=C(x_{i}^{1},C((\hat{x}_{i}^{1}+x_{i}^{2}),\hat{x}_{i}^{2})) \tag{4}\]  
The last step is shown as Formula (4). \(C(.)\) denotes the concatenate operation, \(x_{i}^{1}\) and \(x_{i}^{2}\) means the sliced parts of the feature from the previous layer while \(\hat{x}_{i}^{1}\) and \(\hat{x}_{i}^{2}\) means the parts from the current layer. The addition between \(\hat{x}_{i}^{1}\) and \(x_{i}^{2}\) makes the topology residual and the concatenation among  
Figure 2: Comparison of different network structures. (**a**) residual block in ResNet, (**b**) dense block in DenseNet, (**c**) mixed-link block with a topology fusion of residual block and dense block, (**d**) attention-enhanced mixed-link block. Feat1 and Feat2 are features from current and previous convolution layers. Polyline denotes the residual (skip) connection and the curve denotes dense (concatenate) connection.  
\(\hat{x}_{i}^{1}+x_{i}^{2}\), \(\hat{x}_{i}^{2}\) and \(x_{i}^{1}\) makes the topology dense. The above formulas enable the network to become a partial residual network [18] and partial dense network [19].  
There is also a local feature fusion in a mixed-link block, which filters the information from proceeding state and used a \(1\times 1\) convolution to fuse the information from the current block. The above operations consist of the basic mixed-link unit (MLU) in our proposed network and each AE-MLB contains 4 MLUs in our designed topology. In the final step of AE-MLBs we add the following two parts as the final output for block-level feature selection and fusion. The process could be seen as Formula (5):  
\[F_{i+1}=W_{F}(x_{i})+A_{c}(F_{i}) \tag{5}\]  
where \(F_{i-1}\) denotes the feature from the proceeding attention-enhanced mixed-link block and \(A_{c}\) is a channel attention module for block-feature filtering. \(F_{i+1}\) is the output feature of the current attention-enhanced mixed-link block. \(x_{i}\) is the feature from the last mixed-link units in the current block, \(W_{F}\) means the weight of a \(1\times 1\) convolution for block-feature fusion. With the help of mixed-link mechanism, the network could synchronously do both residual and dense connections, which decreased half of the growth parameters and improved the network performance.