# Effective Monoaural Speech Separation through Convolutional Top-Down Multi-View Network
## 3 Presented Method: ESC-MASD-Net
### Multi-View Attention Block  
The Multi-view Attention (MA) block, which consists of channel, global, and local attention, is integrated into the separation network's core architecture to fully represent the signal information provided by the output of the U-Conv Block. Channel attention, first of all, highlights representations from compressed channels. Meanwhile, dual-path-based global and local attention effectively reflects long sequential features. Figure 4 shows the architecture of the MA block. Fundamentally, the MA block performs the following changes to the latent representation [14]. * Path Separation: The input travels through three routes in the MA block, each of which has a convolution layer that changes the channel size from \(C\) to \(\lfloor C/3\rfloor\). We use chunking with a 50% overlap ratio for both global and local attention paths, splitting \(x\in\mathbb{R}^{\lfloor C/3\rfloor\times L}\) into \(x\in\mathbb{R}^{\lfloor C/3\rfloor\times P\times S}\), where \(P\) and \(S\) stand for chunk size and number of chunks, respectively. This way, long sequential features are efficiently presented due to separating the global and local information. The three perspectives of attention are described as follows. Figure 4: Flowchart of multi-view attention block. Channel Attention: Given the input \(x_{c}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times L}\) for the channel attention path, the average and max pooling processes are used to aggregate the signal information for each channel \(i=1,2,...,\left\lfloor C/3\right\rfloor\). The pooling outputs, \(x_{c}^{avg}\) and \(x_{c}^{max}\), are then passed through a common densely connected network with one hidden layer having \(\left\lfloor C/6\right\rfloor\) nodes, followed by the sigmoid activation function \(\sigma\) to obtain the channel attention weight \(\alpha_{c}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times 1}\): \[\alpha_{c}=\sigma\left(W_{1}\left(W_{0}\left(x_{c}^{avg}\right)\right)+W_{1}(W_ {0}(x_{c}^{max}))\right),\] (6) where \(W_{1}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times\left\lfloor C/6\right\rfloor}\) and \(W_{0}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times\left\lfloor C/6\right\rfloor}\) denote the weights of the used densely connected network. The channel attention weight is then broadcasted along the time dimension and multiplied with \(x_{c}\) to produce the channel attention output \(x_{c}^{\prime}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times L}\). 2. Global Attention: The global attention is based on the self-attention of Transformer, in which the chunk-wise representation \(x_{g}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times P\times S}\) for the global attention input is taken into account. The corresponding output \(x_{g}^{\prime}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times P\times S}\) is determined by multi-head self-attention (MHA): \[\alpha_{g}=\text{softmax}\left(\frac{QK^{\top}}{\sqrt{d_{k}}}\right),\quad x_{ g}^{\prime}=W\left(\alpha_{g}V\right),\] (7) where the linear transformation with each weight, \(W_{q,k,v}\in\mathbb{R}^{1\times 5\times 5}\), and \(x_{g}\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times P\times S}\), represent \(Q\), \(K\), and \(V\in\mathbb{R}^{\left\lfloor C/3\right\rfloor\times P\times S}\). Ultimately, a linear layer, \(W\in\mathbb{R}^{1\times 5\times 5}\), is applied to \(\alpha_{g}V\) in order to extract the global attention output. 3. Local Attention: The local sequential features in each chunk are represented by local attention. On the chunked input \(x_{L}\in\mathbb{R}^{P\times\left\lfloor C/3\right\rfloor\times S}\), a depthwise convolution layer with a kernel size of \(\left\lfloor S/2\right\rfloor-1\) is applied. Following the depthwise convolution layer, we concatenate the channel-wise average and max pooling to estimate the local attention weight \(\alpha_{l}\in\mathbb{R}^{P\times 1\times S}\) as follows: \[\alpha_{l}=\sigma(F([x_{l}^{avg};x_{l}^{max}])),\] (8) where \(F\) is the convolution layer that halves the channel size.