## 3 System Design
### An End-to-End Automatic Speech Recognition Algorithm Based on Conformer  
In this paper, a more efficient network architecture for solving the ASR problem is designed to achieve a reduction in the complexity of the Conformer while achieving a lower Character Error Rate (CER) for a given computational budget. A novel convolution-attention hybrid downsampling layer is designed, as shown in Figure 6. The novel sampling layer diverges from the Macaron structural design of the Conformer block and resembles the standard Transformer structure more closely. The convolution module is placed between the two feed-forward networks, and the MHSA module is positioned in front of the feed-forward networks. The original convolutional module is replaced by a convolutional downsampling module, shown in Figure 7, to reduce the computational cost of CNNs, and enable faster training and inference. Downsampling is performed to reduce the sampling rate of the input sequences to 40 ms, followed by sequential downsampling to reduce the sampling rate of each input sequence to 80 ms. During this process, the feature sequences are projected to wider feature dimensions to maintain the complexity of each encoder. This effectively reduces the redundancy in the feature embedding vectors learned by the Conformer block, thus further reducing the computational overhead without loss of accuracy. Figure 6: The Conformer architecture (**Left**). The Conformer consists of two macroaron-like feed-forward layers, with a MHSA and convolutional module in the middle layer. The feature dimensions are kept constant throughout the network; The SAformerNet architecture (**Right**). SAformerNet consist of multiple Conformer blocks with a hybrid convolution-attention structure. The coded sequences are subjected to multiple downsampling operations before being projected to wider feature dimensions. However, temporal downsampling results in unstable and divergent training, and downsampling the feature vectors reduces the amount of contextual information available to the decoder, thus hindering the successful decoding of the entire sequence. Inspired by Res-Net, a new network structure is introduced as a squeeze-and-excite (SE) block [44; 45] in the convolution downsampling module by residually connecting some of the conformer blocks, as shown in Figure 7. The SE block can recalibrate features to selectively emphasize informative features, thereby mitigating information deficits attributable to downsampling. For input \(x\), the output \(y\) of the SE block is denoted by:  
\[\overline{x}_{t}=\frac{1}{T}\sum_{t}x_{t},\ \ \theta(x)=Sigmoid(W_{2}(Act(W_{1} \overline{x}_{t}+b_{1}))+b_{2}) \tag{8}\] \[SE(x)=\theta(x)\circ x_{T}\]  
In Equation (8), \(Act(\cdot)\) denotes the activation function, \(\theta(x)\) is a global channel-wise weight, and \(Sigmoid(\cdot)\) refers to the sigmoid function. Where \(\circ\) represents element-wise multiplication, \(W_{1}\), \(W_{2}\) are weight matrices, and \(b_{1}\), \(b_{2}\) are bias vectors. The residual module contains a pointwise projection layer and a 1D SE-Net, which uses a convolutional layer for feature extraction. The resulting features are then pooled and compressed into a 1D vector before being applied to the convolutional output through pointwise multiplications. The Convolution Downsampling module incorporates a strided depth-wise convolution responsible for the downsampling function, with batch normalization and activation applied after each convolution. The advantages of SE block feature recalibration can accumulate across the network and can be adapted to the needs of the network, especially in deep Conformer Encode. This allows for further reduced redundancy without compromising accuracy. To further reduce the computational overhead in SAformerNet, the MHSA Module is replaced by the Grouped MHSA Module [23].