## 2 Proposed Method
### TDNN Blocks with Multiple Fusion Layers  
The Res2 Net [36] module gets multi-scale frame-level features by enhancing the middle layer of the bottleneck block. However, we notice that serious gridding effect [32] emerged and the neighboring information is not fully taken into account when introducing constant dilation rate in 1D-Res2 Net, as shown in Figure 3. Assume that the input feature \(X\in R^{C\times T}\) is split into \(s\) feature map subsets, and the \(s\)th feature map \(X_{s}\) can be expressed as: \(X_{s}=\left\{x[1],x[2]\ldots,x[T]\right\}\in R^{C/s\times T}\). For the sake of convenience, \(C/s\) is defined as 1, which means we assume \(X_{s}\) to be a single-channel. Consequently, the dilated convolution can be mathematically represented by the subsequent formula:  
\[y[i]=\sum_{l}x[i+r\cdot l]\cdot w^{l} \tag{2}\]  
where \(x[i]\) and \(y[i]\) denote the input and output signal, respectively. \(w\) is the filter with a filter length of 3, and \(l=\left\{-1,0,1\right\}\). \(r\) means the dilation rate. Dilated convolutions return to standard convolutions when \(r=1\). So, for group convolutions in cascade, like Figure 3 shows, we can obtain:  
\[y_{n}[i]=\sum_{l}(x_{n}+y_{n-1})[i+r\cdot l]\cdot w^{l}_{n},n\geqslant 3 \tag{3}\]  
Thus, taking \(y_{4}\) as an example, where \(w_{k}\) means the weight of TDNN_\(k\) in Figure 3a, we can intuitively deduce that:  
\[ \begin{split} y_{4}[i]=&\sum_{l}\sum_{l^{\prime}} \sum_{l^{\prime\prime}}x_{2}[i+r\cdot(l+l^{\prime}+l^{\prime\prime})]\cdot w^{ l}_{1}w^{l^{\prime}}_{2}w^{l^{\prime\prime}}_{3}\\ &+\sum_{l^{\prime}}\sum_{l^{\prime\prime}}x_{3}[i+r\cdot(l^{ \prime}+l^{\prime\prime})]\cdot w^{l^{\prime}}_{2}w^{l^{\prime\prime}}_{3}\\ &+\sum_{l}x_{4}[i+r\cdot l]\cdot w^{l}_{3}\end{split}  \tag{4}\]  
Although the receptive field of \(y_{4}\) is extended to \(2 mr+1\), the actual receptive field is from isolated frames \(\left\{i,i\pm r,i\pm 2r,i\pm 3r\right\}\). Frankly, there is no correlation for neighboring frames when \(r>1\), and the accumulation between different TDNN blocks may cause a larger grid effect, which can result in information loss. The CNN-TDNN framework can be attributed to the CNN's ability to better handle local feature information. However, the research above has shown that the dilated TDNN structure suffers from information loss due to the grid effect during computation. Moreover, the existing CNN-TDNN  
Figure 3: Illustrations of (**a**) Dilated 1D-Res2 Net and (**b**) Gridding effect. 1D CNN with a kernel of 1 for multi-channel information integration in time dimension. TDNN_i means dilated TDNN block with a set dilated rate. structure only reinforces local features at the input end of TDNNs, resulting in progressive information loss during the computation process of the TDNN backbone. To address the information loss problem, and inspired by HRNet [29] to obtain high-quality features, a combination of repeated multi-scale fusions and dilated TDNN is introduced to compensate the TDNN blocks as demonstrated in Figure 4a. A fusion operator can be formulated as:  
\[X_{fusion}=ReLU\{X+\sum_{i=1}^{3}f_{i}[BN_{i}(Conv1d|_{k=1}(X^{i}))]\} \tag{5}\]  
with \(f_{i}[\ \cdot\ ]\) denoting upsample function with a scale factor of \(2^{i-1}\), and we simply employ the simplest nearest neighbor interpolation for upsampling. 'BN' means batchnorm. Therefore, the minor branches are extended to the same shape as the main branch to implement the multi-scale fusion. This enables each TDNN block's computation to receive multi-scale information from the CNN encoder repeatedly, with information upsampled from different scales to enhance the relatedness of local neighbor features in time dimension. All of the features with strong correlation upsampled from minor branches will be fused with the main branch to fulfill the compensation of dilated TDNN blocks. To exploit more information from different TDNN blocks, multi-layer feature aggregation (MFA) and residual summation connections [19] are adopted.