# Singing Voice Detection in Electronic Music with a Long-Term Recurrent Convolutional Network
## 2 Materials and Methods
### LRCN Components  
The LRCN is a deep model that learns time-series relationships and integrates information in space for deep feature extraction. The CNN layer spatially extracts the audio features for deep feature extraction, while the LSTM layer performs a deep feature encoding of the convolutional layer output to learn the temporal relationship and obtain a final singing or no-singing label.  
Figure 1: Overview of the proposed SVD system, including preprocessing, feature extraction, and LRCN feeding in the both training and test phases.  
The LRCN has an input layer with the size of the combined feature vector, an output layer with a sigmoid unit and three hidden layers, and outputs values for each frame block between 0 (no singing voice) and 1 (singing voice). By combining a feature extraction CNN and a LSTM, the model passes the fusion features of successive audio frames into a new vector and then synthesizes its temporal dynamics in a sequential order. The LRCN layers and main functions are shown in Figure 2.  
The LRCN layer has a stack of one-dimensional inputs and a set of one-dimensional kernels. The max-pooling layer subsamples the inputs by taking the maximum over groups of contiguous frames, and the input layer receives the fusion features of continuous frames in a fixed block size. The parameters for the LRCN layers are shown in Table 1.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Layer** & **Output Shape** & **Parameter** \\ \hline reshape-1 (Reshape) & (None,1,1,20,276) & 0 \\ conv\_lstm\_2d-1 (ConvLSTM2D) & (None,1,17,13) & 60,164 \\ dropout-1 (Dropout) & (None,1,17,13) & 0 \\ max\_pooling\_2d-1 (MaxPooling2D) & (None,1,8,13) & 0 \\ dropout-2 (Dropout) & (None,1,8,13) & 0 \\ flatten-1 (Flatten) & (None,104) & 0 \\ dense-1 (Dense) & (None,200) & 21,000 \\ dropout-3 (Dropout) & (None,200) & 0 \\ dense-2 (Dense) & (None,50) & 10,050 \\ dropout-4 (Dropout) & (None,50) & 0 \\ dense-3 (Dense) & (None,1) & 51 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Parameters of the LRCN layers.  
Figure 2: The LRCN has a CNN layer to fuse the audio features into a vector, while the LSTM layer is in order to learn its sequencing to output a sing/nosing label.  
The input data are concatenated on continuous frames. The input vector is mapped to one label based on the ground truth. In the LRCN layer, 256 convolution filters and a convolution kernel operate a one-dimensional convolution along with the features, and it determines the cell state in the input grid and past states of its neighbors through a convolution operator in state-to-state and input-to-state transitions. The structure of the LRCN is shown in Figure 3. The equations are shown in Equations (1)-(5). All inputs, gates, hidden states, and cell outputs of the LRCN are 3D tensors.  
\[i(x)=\sigma(W_{i}*[conv(X(x)),H(x-1),C(x-1)]+b_{i}) \tag{1}\]  
\[o(x)=\sigma(W_{o}*[conv(X(x)),H(x-1),C(x)]+b_{o}) \tag{2}\]  
\[f(x)=\sigma(W_{f}*[conv(X(x)),H(x-1),C(x-1)]+b_{f}) \tag{3}\]  
\[C(x)=f(x)*C(x-1)+i(x)*tanh(W_{c}*[conv(X(x)),H(x-1)]+b_{c}) \tag{4}\]  
\[H(x)=o(x)*tanh(C(x)) \tag{5}\]  
where:  
\(i(x)=\) Input gate;  
\(o(x)=\) Output gate;  
\(f(x)=\) Forget gate;  
\(C(x)=\) Input LRCN cell;  
\(H(x)=\) Output LRCN cell;  
\(W=\) Matrix of weights;  
\(b=\) Vector of bias;  
\(\sigma=\) Sigmoid function;  
\(*=\) Element-wise product;  
\(conv.=\) Convolution operator.