# Replay Speech Detection Based on Dual-Input Hierarchical Fusion Network
## 2 Methods  
### Hierarchical Fusion Network  
The proposed network had two inputs, as shown in Figure 1a, namely, the original speech signal and the time-reversal speech signal. These two signals came from the same speech segment, with the time-reversed speech signal inverted on the time axis relative to the original speech signal. To integrate the original speech signals and the associated time-reversal signals, we used a dual-input hierarchical fusion network (HFN), as shown in Figure 1b. The top network architecture of HFN was for processing the original signals, and the bottom network architecture was for processing the time-reversed signals. The model architectures of the original and reversed layers were the same. For the backbone part of the model, we used ResNet-50 [30] in each layer. Features from the ends of the two layers were merged with the fully connected layer to produce a binary vector for classification. In addition, our model improved fusion performance by more thoroughly fusing information from different feature levels, as shown in Figure 1c, where features from different feature levels were progressively fused by the hierarchical fusion model (HFM) [27].  
Figure 1: Diagram of dual-input hierarchical fusion network. (**a**) Original speech and time-flipped speech signal input; (**b**) hierarchical fusion network (HFN); (**c**) hierarchical fusion model (HFM).  
In the HFN network, the \(F_{j}\) of the \(jth\) HFM module combines feature maps from the top-level raw stream \(z_{j}^{t}\), the reversed stream \(z_{j}^{b}\), and the fused result from the previous hierarchy \(z_{j-1}\). \(z_{j}\) can be expressed as:  
\[z_{j}=F_{j}\Big{(}z_{j}^{t},z_{j}^{b},z_{j-1}\Big{)}=\varphi\Big{(}z_{j}^{t} \Big{)}+\varphi\Big{(}z_{j}^{b}\Big{)}+\theta\big{[}\varphi(z_{j-1})\big{]},j=1, 2,3,z_{0}=0 \tag{1}\]  
where \(\varphi\) is a convolutional layer used to align the number of channels of feature maps from different hierarchies, and \(\theta\) is the closest interpolation function used to upsample \(z_{j-1}\) so that it has the same size as both \(\varphi\Big{(}z_{j}^{b}\Big{)}\) and \(\varphi\Big{(}z_{j}^{t}\Big{)}\). The information fusion from multiple feature levels was utilized to address the face anti-spoofing issue [27].  
In order to anticipate a pixel map for pixel-level supervision, a fused feature map corresponding to a binary label was constructed at the final fusion level [31; 32]. The fused feature map was refined during the decoding process using pixel-level supervision, which optimizes the HFN by combining binary classification with pixel-wise supervision. In our experiments, the binary cross entropy loss (\(L_{BCE}\)) was applied to the binary vector output from the fully connected layer, where the fake speech was marked as \"0\" and genuine speech was marked as \"1\". The output pixel map of the HFN was then subjected to mean squared error (MSE) loss (\(L_{MSE}\)). Each fake or genuine speech sample was given a target pixel map with all components set to \"0\" or \"1\", respectively, corresponding to the binary labels. Thus, the final loss function of the model was:  
\[L_{loss}=L_{BCE}+L_{MSE} \tag{2}\]