## 2 Proposed Approach
### Model Architecture
#### 2.3.3 Overall Procedure
\begin{table}
\begin{tabular}{|c|c|c|c|} \hline  & & **DOAE Stream** & **SED Stream** \\ \hline \multirow{4}{*}{Input} & \multirow{4}{*}{Feature Extraction} & (4 \(\times\) 279) & \multirow{4}{*}{Conv2d (3 \(\times\) 3, 4 \(\rightarrow\) 64)} \\  & & - Log-mel (256-dim) & (4 \(\times\) 279) \\  & & - Add. Spectral info.(11-dim) & - Log-mel (256-dim) \\  & & - Chroma (12-dim) & - Add. Spectral info.(11-dim) \\  & & + & - Chroma (12-dim) \\  & & (3 \(\times\) 279) & \\  & & - IV (256-dim) & \\  & & - Zero vector (23-dim) & \\ \hline \multirow{4}{*}{DOAE: B \(\times\) 64 \(\times\) 80 \(\times\) 139} & Conv2d (3 \(\times\) 3, 4 \(\rightarrow\) 64) & Conv2d (3 \(\times\) 3, 4 \(\rightarrow\) 64) \\  & & BN (64) & BN (64) \\  & & ReLU & ReLU \\  & & - & Conv2d (3 \(\times\) 3, 64 \(\rightarrow\) 64) \\  & & BN (64) & BN (64) \\  & & ReLU & ReLU \\ \cline{2-4}  & & Average Pooling (2 \(\times\) 2) & Average Pooling (2 \(\times\) 2) \\ \cline{2-4}  & & Learnable weights \(\mathbf{\alpha}^{i}\)(64) & Learnable weights \(\mathbf{\gamma}^{i}\)(64) \\ \cline{2-4}  & & Learnable weights \(\mathbf{\beta}^{i}\)(64) & Learnable weights \(\mathbf{\delta}^{i}\)(64) \\ \hline \multirow{4}{*}{Conv. Block 2} & Conv2d (3 \(\times\) 3, 64 \(\rightarrow\) 128) & Conv2d (3 \(\times\) 3, 64 \(\rightarrow\) 128) \\  & & BN (128) & BN (128) \\  & & ReLU & ReLU \\  & & ReLU & ReLU \\  & & & BN (128) \\  & & ReLU & ReLU \\ \cline{2-4}  & & Average Pooling (2 \(\times\) 2) & Average Pooling (2 \(\times\) 2) \\ \cline{2-4}  & & Learnable weights \(\mathbf{\alpha}^{i}\)(128) & Learnable weights \(\mathbf{\gamma}^{i}\)(128) \\  & & Learnable weights \(\mathbf{\beta}^{i}\)(128) & Learnable weights \(\mathbf{\delta}^{i}\)(128) \\ \hline \multirow{4}{*}{Conv. Block 3} & Conv2d (3 \(\times\) 3, 128 \(\rightarrow\) 256) & Conv2d (3 \(\times\) 3, 128 \(\rightarrow\) 256) & \\  & & BN (256) & BN (256) \\  & & ReLU & ReLU \\  & & ReLU & ReLU \\ \cline{2-4}  & & & BN (256) \\  & & ReLU & ReLU \\ \hline \multirow{4}{*}{Exoder} & \multirow{4}{*}{Soft parameter sharing 3} & Learnable weights \(\mathbf{\alpha}^{i}\)(256) & Learnable weights \(\mathbf{\gamma}^{i}\)(256) \\  & & Learnable weights \(\mathbf{\beta}^{i}\)(256) & Learnable weights \(\mathbf{\delta}^{i}\)(256) \\ \cline{1-1} \cline{2-4}  & & Conv2d (3 \(\times\) 3, 256 \(\rightarrow\) 512) & Conv2d (3 \(\times\) 3, 256 \(\rightarrow\) 512) \\ \cline{1-1} \cline{2-4}  & & BN (512) & BN (512) \\ \cline{1-1} \cline{2-4}  & & ReLU & ReLU \\ \cline{1-1} \cline{2-4}  & & Conv2d (3 \(\times\) 3, 512 \(\rightarrow\) 512) & Conv2d (3 \(\times\) 3, 512 \(\rightarrow\) 512) \\ \cline{1-1} \cline{2-4}  & & BN (512) & BN (512) \\ \cline{1-1} \cline{2-4}  & & ReLU & ReLU \\ \hline \multirow{4}{*}{Average Pooling} & \multirow{4}{*}{Average Pooling} & \multirow{4}{*}{Average Pooling (1 \(\times\) 2)} \\  & & & \\ \cline{1-1} \cline{2-4}  & & & \\ \cline{1-1} \cline{2-4}  & & & \\ \cline{1-1} \cline{2-4}  & & & \\ \cline{1-1} \cline{2-4}  & & & \\ \hline \multirow{4}{*}{Decoder} & Transformer decoder ( \(\times\) 2 layers) & Masked MHSA (512, heads=8) & Masked MHSA (512, heads=8) \\  & & LayerNorm (512) & LayerNorm (512) \\  & & DCMA (512 \(\rightarrow\) 512, heads=8) & DCMA (512 \(\rightarrow\) 512, heads=8) \\  & & LayerNorm (512) & LayerNorm (512) \\  & & FC(512 \(\rightarrow\) 2048) & FC(512 \(\rightarrow\) 2048) \\  & & FC(2048 \(\rightarrow\) 512) & FC(2048 \(\rightarrow\) 512) \\  & & LayerNorm (512) & LayerNorm (512) \\ \hline \multirow{2}{*}{Prediction Layer} & \multirow{2}{*}{DOAE: B \(\times\) 3 \(\times\) 40} & FC1 (512 \(\rightarrow\) 3 ), & FC1 (512 \(\rightarrow\) 12 ), \\  & & SED: B \(\times\) 12 \(\times\) 40 & FC2 (512 \(\rightarrow\) 3 ), & FC2 (512 \(\rightarrow\) 12 ) \\ \hline \end{tabular}
\end{table} 
Table 2: Feature dimensions and hyper-parameters of DCMA-SELD. \(\mathbf{\alpha}^{i}\), \(\mathbf{\beta}^{i}\), \(\mathbf{\gamma}^{i}\), and \(\mathbf{\delta}^{i}\) denote the learnable weights for the soft parameter sharing on the input embeddings for the \(i\)-th convolutional block.