Paper: Spiking Hybrid Attentive Mechanism with Decoupled Layer Normalization for Joint Sound Localization and Classification (ICLR-2024)

Reviewer comment (Novelty):
Though achieving state-of-the-art performance on SLoClas dataset, the proposed architecture still lacks novelty. According to my understanding, the core part of the architecture is the SHAF block, however its components are just copies or slight modifications/extensions of spiking self attention mechanism proposed in [zhou et al., 2022]. The extension seems straight forward and may not be regarded as a genuine technical contribution.

Author response:
While the SCA module in SHAF block is inspired by SSA module proposed in [1], their input types and functionalities are completely different. The SSA, based on the spiking self-attention mechanism, is utilized to model long-range interdependencies within the same input. But the SCA applies the spiking cross-attention mechanism to align features between different input stream. To the best of our knowledge, this is the first work to explore the effectiveness of cross-attention mechanism in SNNs. We believe that this investigation not only promotes the application of SNN in the audio field, but also paves the way for further research on multimodal SNN models.

Moreover, the whole SHAF block employs a hybrid attention mechanism, instead of a single self-attention mechanism or a single cross-attention mechanism. The block has strong ability in learning intra-modal acoustic temporal dependencies and aligning inter-modal acoustic relationships. Each component of the block has a significant impact on the model performance. 

In addition, the DSLN method we proposed is also a core part of the architecture. Different from vanilla LayerNorm, DSLN only calculates the mean while keeping the variance fixed during inference. And we apply a re-parameterization technique to fold the trained DSLN into the weights so that no additional computation and energy consumption are generated.

Reviewer comment (Writing):
Some of the notations are not align with the others. For example, sometimes the embedding feature dimension is denoted as $D$, but sometimes not (fig 2).

Author response:
We will correct them in the revised version, thank you for your help!

Reviewer comment (Experiments):
Though performing best on SLoClas dataset, it is hard to conclude that the current work is better than previous works in terms of sound localization and classification. More experiments on various datasets is recommended.

Author response:
Since there are limited datasets and models for joint sound localization and classification, we construct a suitable dataset named DCASE 2019 Part. The dataset is the part of the DCASE 2019 dataset without overlapping sound sources. It has 11 sound sources and the azimuth angle interval is 10 degrees. 
Considering previous SNN methods have not made their code available, we resort to using traditional ANN methods for comparison purposes. Specifically, we utilize a modified version of EINV2 [2] which is originally designed for sound event detection and implement an ANN version of our model (SLC-Net). 
When evaluating models on DCASE 2019 Part dataset, the threshold for accuracy calculation is set at 5 degrees, and it is set to 2.5 degrees on SLoClas dataset. 
The results are shown in Tab. R4-1. As we can see, our method outperforms other methods on both sound event localization and classification tasks.

| Method | Dataset | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :---- | :----: | :----: | :----: |
| GCC-PHAT-CNN | SLoClas |4.88 | 84.50 | 78.90 |
| EINV2 | SLoClas |0.80 | 96.92 | 98.08 |
| SLC-Net-2-128 | SLoClas | 0.87 | 96.52 | 99.11 |
| SpikSLC-Net-3-128 (Ours) | SLoClas | **0.45** | **97.39** | **99.42** |
| SLC-Net-2-128 | DCASE2019 Part | 3.89 | 80.42 | 99.88 |
| EINV2 | DCASE2019 Part |7.27 | 55.29 | 99.77 |
| SpikSLC-Net-1-64 (Ours) | DCASE2019 Part | **2.42**| **86.08** | **1.00** |
**Table R4-1: Comparison of different methods.**

Reviewer comment (Evaluation):
Comprehensive analysis of the robustness to noisy environment is insufficient.

Author response:
Here, we evaluate the noise robustness of our model under various noisy conditions. We select the well-trained SpikSLC-Net-2-128 on SLoClas dataset as the testing model. The noise sounds are from SLoClas dataset, which contains the recordings of 6 types of noise at 4 DoAs. We randomly select the noise audio from one of the four directions, 0&deg;, 90&deg;, 180&deg;, 270&deg;.
We regard the noise as the addictive background noise. In order to simulate the background noise from all directions, we add the noise sound clip to microphone signal in the channel dimension. 
In Tab. R4-2, we show the results on noisy data of various Signal-to-Noise ratio (SNR). It is understandable that MAE decreases as SNR increases across all conditions. For high SNR test data, our model performs reasonably well.

| SNR (dB)| DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: | 
| -10| 48.62 | 35.27 | 56.46 |
| 0 | 31.75| 56.73 | 71.77 | 
| 5 | 24.20 | 66.62 | 79.57 |
| 10 | 16.53| 76.25 | 87.37 |
| 15 | 9.74 | 85.17 | 93.43 |
| 20 | 4.59 |92.17 | 97.25 |
**Table R4-2: Analysis of the robustness to noisy environment.**

Reviewer comment (Evaluation):
As in [zhou et al., 2022], comparison of computation complexity and estimated power consumption are highly recommended to be embraced in the experiments.

Author response:
The contributions of previous SNN methods focus on coding methods rather than models, so it is difficult to make a fair comparison of algorithm complexity. Furthermore, other methods cannot perform both sound source localization and recognition simultaneously. Thus, we only provide the computation complexity and estimated power consumption of our models and other ANN models. 
In SNNs, the synaptic operations (SOPs) are used to describe the computation complexity, which is calculated by 

$\operatorname{SOPs}(l)=fr \times T \times \operatorname{FLOPs}(l)$,

where $l$ is a layer in SpikSLC-Net and $fr$ represents the firing rate of the input spike train, and $T$ denotes the time step. $\operatorname{FLOPs}(l)$ refers to floating point operations of $l$, which is the number of multiply-and-accumulate (MAC) operations. And SOPs is the number of spike-based accumulate (AC) operations.
Refer to previous works [1][3], we assume that the operations are implemented on the 45nm hardware [4], where the theoretical energy consumption $EMAC = 4.6pJ$ and $EAC = 0.9pJ$.

The results are shown in Tab. R4-3. From the results, it can be seen that the OPs and the theoretical energy consumption of our models are much lower compared with ANN models that have good performance.


| Method | Param (M)|OPs (G) | Power (mJ) |
| :---- | :----: | :----: | :----: |
| GCC-PHAT-CNN | 4.17 | 0.048 | 0.143 |
|EINV2 | 85.45 | 8.96 | 20.562 |
|SLC-Net-2-128 | 4.15 | 0.067 | 0.152 |
|SpikSLC-Net-1-128 (Ours) | **3.61** | **0.017** | **0.031** |
| SpikSLC-Net-3-128 (Ours) | 4.68 | 0.056 | 0.066 |
**Table R4-3: Comparison of computation complexity and estimated power consumption.**	


We will add these promising comparison results to the revised version.

Reviewer comment (Novelty):
Unclear why this particular combination of problems and approaches.

Author response:
**R:** Our research primarily focuses on the domains of spiking neural networks and audio.
The motivation of our work has been illustrated in Introduction section of our manuscript and we will give more discussion about the reason that we use SNNs in joint sound localization and classification.

Firstly, as we know, SNN has the characteristics of bio-interpretability, lower power consumption, higher robustness, and better ability to extract spatio-temporal features. So it provides a more energy-efficient and bio-plausible way to perform such tasks in the audio field. 

Secondly, for humans, localizing and identifying sound sources simultaneously through binaural cues is a crucial and basic ability. Therefore, it’s very meaningful to investigate the information sharing ability of brain-inspired SNNs from the SELC task.

Thirdly, previous work about SNNs has largely focused on the computer vision and the performance of SNN in handling other modal data has not been fully explored. Different from images, audio data has a temporal dimension and is more sparse, posing new challenges for SNN architectures. For example, how to integrate different acoustic features, how to normalize features at the frame level, and how to design low-power networks for audio. 

Finally, if we want to employ SNN in sound localization and classification, we have to solve the above basic problems in the specific task first. LN is a widely used normalization method to process temporal data like audio in ANNs. However, the vanilla LN do not conform to the calculation characteristics of SNNs, since it needs to be calculated dynamically during runtime and involves a substantial number of floating-point operations. Therefore, it’s very essential to develop a suitable LN method for SNNs.

Reviewer comment (Experiments):
Claims not supported well by experiments and evidence.

Author response:
**R:** **Multi-task learning.** Due to the powerful performance of our model and the initial high accuracy of sound source classification, multitask learning has limited impact on further improving sound source classification accuracy. Instead, it focuses more on enhancing sound source localization accuracy.
Our results indicate that multitask learning is feasible in SNNs, which will also help promote the development of SNNs.

**SHAF mechanism.** The SHAF mechanism consists of spike-form cross-attention and self-attention, which is able to integrate acoustic features. In SNNs, the direct input of LIF neurons can be real-valued [1]. The float-point $Q,K,V$ refers to the results of $XW$, serving as the input of LIF neurons. So it can be deployed on neuromorphic hardware.

**DSLN method.** Our goal is to fold the LN calculation process into other layers like FC while retaining as many statistics as possible so that we can eliminate the time cost and computational energy consumption of DSLN. In practice, we choose to use a fixed variance and dynamically calculate the mean in the stage of inference. The re-parameterization procedure are derived in Eq. 15.

Here, we conduct more ablation experiments on SHAF and DSLN to demonstrate the effectiveness of our method.
For the SHAF, we apply SpikSLC-Net-2-128 and the model has two FC layers in each head. For the DSLN, to avoid the impact of head design on model performance, we apply SpikSLC-Net-3-128 with lightweight heads which are equipped with only one FC layer.
The results are listed in Tab. R3-1 and Tab. R3-2. As we can see, their improvement in model performance is mainly reflected in the DoA estimation task, since the sound localization is more challenging than classification.
The SCA and SSA modules in SHAF both can greatly improve the performance of the model. And the DSLN module can effectively alleviate overfitting in larger model.

| Setting | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O SCA & SSA | 1.10 | 95.90 | 99.29 |
| W/O SSA | 0.76 | 96.81 | 99.35 | 
| W/O SCA | 0.85 | 96.57 | 99.32 | 
| W/ SCA & SSA | **0.47** | **97.46** | 99.33 |
**Table R3-1: Ablation studies on SHAF.**	

| Setting | DoA MAE(&deg;)  | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O DSLN | 1.06 | 95.67 | 99.33 | 
| W/ DSLN | **0.53** | **97.32** | **99.45** |
**Table R3-2: Ablation studies on DSLN.**

Reviewer comment (Experiments):
Lack of strong baselines.

Author response:
**R:** Our framework is based on full SNNs, so it’s reasonable to compare our model with other SOTA SNN methods on the same datasets they used. The previous best ANN method for sound localization and classification on SLoClas dataset is GCC-PHAT-CNN. In order to demonstrate the superiority of our model, we utilize a modified version of EINV2 [2] which is originally designed for sound event detection and implement an ANN version of our model (SLC-Net) for additional experiments. The corresponding results are shown in Tab. R3-3. The advantage of our model in DoA estimation is very obvious. According to the characteristics of SNN, we think that it is very possible for SNN to outperform ANN in specific cases. Additionally, it is plausible that the performance of the current best ANN model is still insufficient.

| Method | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: | 
| GCC-PHAT-CNN |4.88 | 84.50 | 78.90 |
| EINV2 | 0.80 | 96.92 | 98.08 |
| SLC-Net-2-128 | 0.87 | 96.52 | 99.11 |
| MTPC-CSNN | 1.02 | 94.72 | - |
| Hybrid Coding | 0.60 | 95.61 | - |
| SpikSLC-Net-3-128 (Ours) | **0.45** | **97.39** | **99.42** |
**Table R3-3: Comparison of different methods on SLoClas dataset.**

Reviewer comment (Experiments):
Very narrow evaluation on only a single, quite specific dataset.

Author response:
**R:** Here, we conduct more experiments on different datasets and analyze the computation complexity and estimated power consumption to show the superiority of our SNN approach on two tasks. 
Since there are limited datasets and models for joint sound localization and classification, we construct a suitable dataset named DCASE 2019 Part. The dataset is the part of the DCASE 2019 dataset without overlapping sound sources. It has 11 sound sources and the azimuth angle interval is 10 degrees. When evaluating models on DCASE 2019 Part dataset, the threshold for accuracy calculation is set at 5 degrees, and it is set to 2.5 degrees on SLoClas dataset. 

In SNNs, the synaptic operations (SOPs) are used to describe the computation complexity, which is calculated by 

$\operatorname{SOPs}(l)=fr \times T \times \operatorname{FLOPs}(l)$,

Where $l$ is a layer in SpikSLC-Net and $fr$ represents the firing rate of the input spike train, and $T$ denotes the time step. $\operatorname{FLOPs}(l)$ refers to floating point operations of $l$, which is the number of multiply-and-accumulate (MAC) operations. And SOPs is the number of spike-based accumulate (AC) operations.
Refer to previous works [1][3], we assume that the operations are implemented on the 45nm hardware[4], where the theoretical energy consumption $EMAC = 4.6pJ$ and $EAC = 0.9pJ$.

The results are shown in Tab. R3-4 and Tab. R3-5. The number of operations (OPs) refers to SOPs in SNN and FLOPs in ANN. 

As we can see, our models achieve the best performance while maintaining the lowest energy consumption compared with ANN models.

| Method | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: | 
| SLC-Net-2-128| 3.89 | 80.42 | 99.88 |
| EINV2 |7.27 | 55.29 | 99.77 |
| SpikSLC-Net-1-64 (Ours) | **2.42**| **86.08** | **1.00** |
**Table R3-4: Comparison of different methods on DCASE 2019 Part dataset.**

| Method | Param (M)|OPs (G) | Power (mJ) |
| :---- | :----: | :----: | :----: |
| GCC-PHAT-CNN | 4.17 | 0.048 | 0.143 |
|EINV2 | 85.45 | 8.96 | 20.562 |
|SLC-Net-2-128 | 4.15 | 0.067 | 0.152 |
|SpikSLC-Net-1-128 (Ours) | **3.61** | **0.017** | **0.031** |
| SpikSLC-Net-3-128 (Ours) | 4.68 | 0.056 | 0.066 |
**Table R3-5: Comparison of the number of parameters, computation complexity and estimated power consumption.**

Reviewer comment (Reproducibility):
Method is not really understandable from the paper, with lots of details missing.

Author response:
**R:** Thanks for your carefully reading. We will align the symbols and add more detailed clarifications in the revised version.

In Fig. 2, $N$, $F$, $B_g$ and $B_m$ represent the the total microphone pairs number, the frames number, the feature dimension of each frame in GCC-PHAT and the feature dimension of each frame in log-Mel spectrogram, respectively. 
In Eq. 8, the symbols are general, where $N$ is equal to $F$ in Fig. 2 and $D$ is equal to $E$ in Fig. 2. 

In the dimensions $T \times N \times D$, $T$ denotes the time step, $N$ denotes the frames number, and $D$ denotes the embedding dimension. The attention operation indeed extends over time step. 
The dimensions of $W_Q$, $W_K$ and $W_V$ are $D \times D$. The product $X_{\alpha} W_Q$ means projecting each frame of data at each time step to embedding space. According to [1], the batch norm around this product can normalize the embeddings and prevent neurons from firing dense spikes.
$\operatorname{SN(.)}$ represents the LIF neuron layers and the dimensions of input and output are both $T \times N \times D$. 

In Eq. 15, $W$ represents the parameters of the previous Linear layer. $m$ and $n$ denote the input channel dimension and output channel dimension, respectively.  $A_i$ and $W'_i$ are the expressions of constants that result from previous calculations.

Reviewer comment (Novelty):
It is not clear what the goal of the paper is. If it's about sound localization and classification, why are spiking neural nets needed? If it is about spiking neural nets, why such a narrow focus on this particular combination of tasks? The motivation for the development of the LN layer for spiking nets is unclear. The approach appears very specific to a narrow problem without being motivated by the problem's requirements, making it unclear what we can learn from this paper.

Author response:
**R:** Our research primarily focuses on the domains of spiking neural networks and audio.
The motivation of our work has been illustrated in Introduction section of our manuscript and we will give more discussion about the reason that we use SNNs in joint sound localization and classification.

Firstly, as we know, SNN has the characteristics of bio-interpretability, lower power consumption, higher robustness, and better ability to extract spatio-temporal features. So it provides a more energy-efficient and bio-plausible way to perform such tasks in the audio field. 

Secondly, for humans, localizing and identifying sound sources simultaneously through binaural cues is a crucial and basic ability. Therefore, it’s very meaningful to investigate the information sharing ability of brain-inspired SNNs from the SELC task.

Thirdly, previous work about SNNs has largely focused on the computer vision and the performance of SNN in handling other modal data has not been fully explored. Different from images, audio data has a temporal dimension and is more sparse, posing new challenges for SNN architectures. For example, how to integrate different acoustic features, how to normalize features at the frame level, and how to design low-power networks for audio. 

Finally, if we want to employ SNN in sound localization and classification, we have to solve the above basic problems in the specific task first. LN is a widely used normalization method to process temporal data like audio in ANNs. However, the vanilla LN do not conform to the calculation characteristics of SNNs, since it needs to be calculated dynamically during runtime and involves a substantial number of floating-point operations. Therefore, it’s very essential to develop a suitable LN method for SNNs.

Reviewer comment (Novelty):
The claim that the paper is the first to use multi-task learning in SNNs for audio-related tasks is not substantiated as a meaningful contribution. There is little evidence that multi-task learning is necessary for sound localization and classification or that it improves performance. Table 3 indicates that training individually works almost as well, and the absence of error bars makes it difficult to judge any effect.

Author response:
**R:** **Multi-task learning.** Due to the powerful performance of our model and the initial high accuracy of sound source classification, multitask learning has limited impact on further improving sound source classification accuracy. Instead, it focuses more on enhancing sound source localization accuracy.
Our results indicate that multitask learning is feasible in SNNs, which will also help promote the development of SNNs.

**SHAF mechanism.** The SHAF mechanism consists of spike-form cross-attention and self-attention, which is able to integrate acoustic features. In SNNs, the direct input of LIF neurons can be real-valued [1]. The float-point $Q,K,V$ refers to the results of $XW$, serving as the input of LIF neurons. So it can be deployed on neuromorphic hardware.

**DSLN method.** Our goal is to fold the LN calculation process into other layers like FC while retaining as many statistics as possible so that we can eliminate the time cost and computational energy consumption of DSLN. In practice, we choose to use a fixed variance and dynamically calculate the mean in the stage of inference. The re-parameterization procedure are derived in Eq. 15.

Here, we conduct more ablation experiments on SHAF and DSLN to demonstrate the effectiveness of our method.
For the SHAF, we apply SpikSLC-Net-2-128 and the model has two FC layers in each head. For the DSLN, to avoid the impact of head design on model performance, we apply SpikSLC-Net-3-128 with lightweight heads which are equipped with only one FC layer.
The results are listed in Tab. R3-1 and Tab. R3-2. As we can see, their improvement in model performance is mainly reflected in the DoA estimation task, since the sound localization is more challenging than classification.
The SCA and SSA modules in SHAF both can greatly improve the performance of the model. And the DSLN module can effectively alleviate overfitting in larger model.

| Setting | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O SCA & SSA | 1.10 | 95.90 | 99.29 |
| W/O SSA | 0.76 | 96.81 | 99.35 | 
| W/O SCA | 0.85 | 96.57 | 99.32 | 
| W/ SCA & SSA | **0.47** | **97.46** | 99.33 |
**Table R3-1: Ablation studies on SHAF.**	

| Setting | DoA MAE(&deg;)  | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O DSLN | 1.06 | 95.67 | 99.33 | 
| W/ DSLN | **0.53** | **97.32** | **99.45** |
**Table R3-2: Ablation studies on DSLN.**

Reviewer comment (Theory):
The description of the Spiking Hybrid Attention Fusion (SHAF) mechanism is unclear due to too many undefined symbols. Additionally, it is questionable whether the mechanism could be implemented with spiking neurons on neuromorphic hardware, since Q, K, V are claimed to be real-valued.

Author response:
**R:** **SHAF mechanism.** The SHAF mechanism consists of spike-form cross-attention and self-attention, which is able to integrate acoustic features. In SNNs, the direct input of LIF neurons can be real-valued [1]. The float-point $Q,K,V$ refers to the results of $XW$, serving as the input of LIF neurons. So it can be deployed on neuromorphic hardware.

Reviewer comment (Experiments):
The evaluation is too narrow. If the aim is to contribute to spiking neural nets in general, broader evaluation on more datasets/problems is needed. If the aim is to solve localization and classification specifically, there may be better approaches than SNNs, and it is unclear why SNNs are prioritized.

Author response:
**R:** Here, we conduct more experiments on different datasets and analyze the computation complexity and estimated power consumption to show the superiority of our SNN approach on two tasks. 
Since there are limited datasets and models for joint sound localization and classification, we construct a suitable dataset named DCASE 2019 Part. The dataset is the part of the DCASE 2019 dataset without overlapping sound sources. It has 11 sound sources and the azimuth angle interval is 10 degrees. When evaluating models on DCASE 2019 Part dataset, the threshold for accuracy calculation is set at 5 degrees, and it is set to 2.5 degrees on SLoClas dataset. 

In SNNs, the synaptic operations (SOPs) are used to describe the computation complexity, which is calculated by 

$\operatorname{SOPs}(l)=fr \times T \times \operatorname{FLOPs}(l)$,

Where $l$ is a layer in SpikSLC-Net and $fr$ represents the firing rate of the input spike train, and $T$ denotes the time step. $\operatorname{FLOPs}(l)$ refers to floating point operations of $l$, which is the number of multiply-and-accumulate (MAC) operations. And SOPs is the number of spike-based accumulate (AC) operations.
Refer to previous works [1][3], we assume that the operations are implemented on the 45nm hardware[4], where the theoretical energy consumption $EMAC = 4.6pJ$ and $EAC = 0.9pJ$.

The results are shown in Tab. R3-4 and Tab. R3-5. The number of operations (OPs) refers to SOPs in SNN and FLOPs in ANN. 

As we can see, our models achieve the best performance while maintaining the lowest energy consumption compared with ANN models.

| Method | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: | 
| SLC-Net-2-128| 3.89 | 80.42 | 99.88 |
| EINV2 |7.27 | 55.29 | 99.77 |
| SpikSLC-Net-1-64 (Ours) | **2.42**| **86.08** | **1.00** |
**Table R3-4: Comparison of different methods on DCASE 2019 Part dataset.**

| Method | Param (M)|OPs (G) | Power (mJ) |
| :---- | :----: | :----: | :----: |
| GCC-PHAT-CNN | 4.17 | 0.048 | 0.143 |
|EINV2 | 85.45 | 8.96 | 20.562 |
|SLC-Net-2-128 | 4.15 | 0.067 | 0.152 |
|SpikSLC-Net-1-128 (Ours) | **3.61** | **0.017** | **0.031** |
| SpikSLC-Net-3-128 (Ours) | 4.68 | 0.056 | 0.066 |
**Table R3-5: Comparison of the number of parameters, computation complexity and estimated power consumption.**

Reviewer comment (Writing):
The description of the methods is extremely hard to follow. Motivations for design choices are not clearly stated, and many symbols, terms, and dimensions are undefined or unclear.

Author response:
**R:** Thanks for your carefully reading. We will align the symbols and add more detailed clarifications in the revised version.

In Fig. 2, $N$, $F$, $B_g$ and $B_m$ represent the the total microphone pairs number, the frames number, the feature dimension of each frame in GCC-PHAT and the feature dimension of each frame in log-Mel spectrogram, respectively. 
In Eq. 8, the symbols are general, where $N$ is equal to $F$ in Fig. 2 and $D$ is equal to $E$ in Fig. 2. 

In the dimensions $T \times N \times D$, $T$ denotes the time step, $N$ denotes the frames number, and $D$ denotes the embedding dimension. The attention operation indeed extends over time step. 
The dimensions of $W_Q$, $W_K$ and $W_V$ are $D \times D$. The product $X_{\alpha} W_Q$ means projecting each frame of data at each time step to embedding space. According to [1], the batch norm around this product can normalize the embeddings and prevent neurons from firing dense spikes.
$\operatorname{SN(.)}$ represents the LIF neuron layers and the dimensions of input and output are both $T \times N \times D$. 

In Eq. 15, $W$ represents the parameters of the previous Linear layer. $m$ and $n$ denote the input channel dimension and output channel dimension, respectively.  $A_i$ and $W'_i$ are the expressions of constants that result from previous calculations.

Reviewer comment (Presentation):
In Fig. 2, the meaning of N, F, B_g, and B_m is unclear and not defined in the caption. N appears to be used for multiple meanings in different places, possibly referring to microphone pairs in Fig. 2 but something else in Eq. 8. There is ambiguity in how dimensions in Fig. 2 correspond to those in Eq. 8.

Author response:
**R:** Thanks for your carefully reading. We will align the symbols and add more detailed clarifications in the revised version.

In Fig. 2, $N$, $F$, $B_g$ and $B_m$ represent the the total microphone pairs number, the frames number, the feature dimension of each frame in GCC-PHAT and the feature dimension of each frame in log-Mel spectrogram, respectively. 
In Eq. 8, the symbols are general, where $N$ is equal to $F$ in Fig. 2 and $D$ is equal to $E$ in Fig. 2.

Reviewer comment (Writing):
In Eq. 8, the meanings of dimensions T, N, and D are unclear. The meanings and dimensions of W_Q, W_K, and W_V are not given. It is unclear why there is a batch norm around the product X_alpha W_Q, which is unusual in Transformer attention. The meaning of SN(.) is undefined, and Fig. 3’s action potential symbols are unhelpful. Input/output dimensions of SN(.) are unclear.

Author response:
**R:** In the dimensions $T \times N \times D$, $T$ denotes the time step, $N$ denotes the frames number, and $D$ denotes the embedding dimension. The attention operation indeed extends over time step. 
The dimensions of $W_Q$, $W_K$ and $W_V$ are $D \times D$. The product $X_{\alpha} W_Q$ means projecting each frame of data at each time step to embedding space. According to [1], the batch norm around this product can normalize the embeddings and prevent neurons from firing dense spikes.
$\operatorname{SN(.)}$ represents the LIF neuron layers and the dimensions of input and output are both $T \times N \times D$.

Reviewer comment (Writing):
In Section 4.3 (especially Eq. 15), W and W'_*,* are defined only by dimensions, without clear explanation of what they are, how they are derived, or the meanings of m and n. Also, A_i is not introduced.

Author response:
**R:** In Eq. 15, $W$ represents the parameters of the previous Linear layer. $m$ and $n$ denote the input channel dimension and output channel dimension, respectively.  $A_i$ and $W'_i$ are the expressions of constants that result from previous calculations.

Reviewer comment (Evaluation):
Most previous SEC task reported F1 score to account for potential bias in sound classes. Would it be possible to show F1 score benchmarking with previous SEC task (potentially could leverage some DCASE datasets) instead? Such high accuracy was unclear to me if there were any biases, and unclear whether your algorithm has specific bias on precision vs. recall.

Author response:
**R:** In order to fully demonstrate the superiority of our model, we select the part of the DCASE 2019 dataset without overlapping sound sources for additional experiments (DCASE 2019 Part). It has 11 sound sources. What’s more, we implement an ANN version of our model (SLC-Net) for comparison on SLoClas dataset and DCASE 2019 Part dataset. The corresponding metrics are shown in Tab. R1-3. For DCASE 2019 Part, the threshold for accuracy calculation is set at 5 degrees, as the azimuth angle interval is 10 degrees. We can find that our method displays absence of potential bias in sound classes, ensuring unbiased precision and recall measurements.

| Method | Dataset | DoA MAE(&deg;)  | DoA Acc(\%) | SEC Acc(\%) | Precision | Recall | F1 |
| :---- | :---- | :----: | :----: | :----: | :----: | :----: | :----: |
| GCC-PHAT-CNN | SLoClas |4.88 | 84.50 | 78.90 | 0.9022 | 0.7890 | 0.7956 |
| SLC-Net-2-128 | SLoClas | 0.87 | 96.52 | 99.11 | 0.9912 | 0.9910 | 0.9911 |
| SpikSLC-Net-1-128 (Ours) | SLoClas | 0.51 | 97.53 | 99.30 | 0.9931 | 0.9930 |0.9930 | 
| SpikSLC-Net-2-128 (Ours)  | SLoClas | 0.47 | 97.46 | 99.33 | 0.9934 | 0.9933 | 0.9933 |
| SpikSLC-Net-3-128 (Ours)  | SLoClas | 0.45 | 97.39 | 99.42 | 0.9942 | 0.9943 | 0.9942 |
| SLC-Net-2-128 | DCASE2019 Part | 3.89 | 80.42 | 99.88 | 0.9988 | 0.9988 | 0.9988 |
| SpikSLC-Net-1-64 (Ours) | DCASE2019 Part | 2.42| 86.08 | 1.0 | 1.0 | 1.0 | 1.0 |
**Table R1-3: Comparison of different metrics.**

Reviewer comment (Experiments):
Additionally, ablation studies did not show how self-attention and cross-attention modules could play an essential role in decoding these tasks. The non-significant differences in numbers of SHAF blocks and embedding dimensions (Table 2) posed a confusing and open question on where the major performance benefits from. It would be more persuading by providing a stronger ablation study, showing how SHAF blocks contribute to the essential performance boosting here, while the rest of modules (multi-task) remained in the model.

Author response:
**R:** We perform more ablation studies on SHAF and DSLN modules to show their significance in our models. 
For the SHAF experiment, we apply SpikSLC-Net-2-128 and the model has two FC layers in each head. For the DSLN experiment, to avoid the impact of head design on model performance, we apply lightweight heads which are equipped with only one FC layer.
The results are listed in Tab. R1-1 and Tab. R1-2. As we can see, their improvement in model performance is mainly reflected in the DoA estimation task, since the sound localization is more challenging than classification.
The SCA and SSA modules in SHAF both can greatly improve the performance of the model. Besides, the DSLN module can effectively alleviate overfitting in larger model.

| Setting | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O SCA & SSA | 1.10 | 95.90 | 99.29 |
| W/O SSA | 0.76 | 96.81 | 99.35 | 
| W/O SCA | 0.85 | 96.57 | 99.32 | 
| W/ SCA & SSA | **0.47** | **97.46** | 99.33 |
**Table R1-1: Ablation studies on SHAF.**	

| Setting | Architecture | DoA MAE(&deg;)  | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----:  | :----: | :----: | :----: |
| W/O DSLN | SpikSLC-Net-1-128 | 0.83 | 97.18 | 99.28 |
| W DSLN | SpikSLC-Net-1-128 | **0.55** | **97.56** | 99.32 | 
| W/O DSLN | SpikSLC-Net-3-128 | 1.06 | 95.67 | 99.33 | 
| W/ DSLN | SpikSLC-Net-3-128 | **0.53** | **97.32** | **99.45** |
**Table R1-2: Ablation studies on DSLN.**

Reviewer comment (Experiments):
Following above in weaknesses, I think it would be helpful to perform ablation studies, showing the role of the proposed SHAF blocks and DSLN specifically in the superior performance. The fact that ablation studies did not show strong impact wrt different hyperparameters in SHAF and LN modules, pose a question on whether the major benefit comes from the multi-task heads, or different algorithmic complexity of this SNN vs. previous SNN.

Author response:
**R:** We perform more ablation studies on SHAF and DSLN modules to show their significance in our models. 
For the SHAF experiment, we apply SpikSLC-Net-2-128 and the model has two FC layers in each head. For the DSLN experiment, to avoid the impact of head design on model performance, we apply lightweight heads which are equipped with only one FC layer.
The results are listed in Tab. R1-1 and Tab. R1-2. As we can see, their improvement in model performance is mainly reflected in the DoA estimation task, since the sound localization is more challenging than classification.
The SCA and SSA modules in SHAF both can greatly improve the performance of the model. Besides, the DSLN module can effectively alleviate overfitting in larger model.

| Setting | DoA MAE(&deg;) | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----: | :----: | :----: |
| W/O SCA & SSA | 1.10 | 95.90 | 99.29 |
| W/O SSA | 0.76 | 96.81 | 99.35 | 
| W/O SCA | 0.85 | 96.57 | 99.32 | 
| W/ SCA & SSA | **0.47** | **97.46** | 99.33 |
**Table R1-1: Ablation studies on SHAF.**	

| Setting | Architecture | DoA MAE(&deg;)  | DoA Acc(\%) | SEC Acc(\%) |
| :---- | :----:  | :----: | :----: | :----: |
| W/O DSLN | SpikSLC-Net-1-128 | 0.83 | 97.18 | 99.28 |
| W DSLN | SpikSLC-Net-1-128 | **0.55** | **97.56** | 99.32 | 
| W/O DSLN | SpikSLC-Net-3-128 | 1.06 | 95.67 | 99.33 | 
| W/ DSLN | SpikSLC-Net-3-128 | **0.53** | **97.32** | **99.45** |
**Table R1-2: Ablation studies on DSLN.**

Reviewer comment (Theory):
Can authors provide an algorithmic complexity analysis of this SNN vs. previous SNN?

Author response:
**R:** Since previous SNN methods do not release their code and their contributions focus on coding methods, it is difficult to make a fair comparison of algorithm complexity. Moreover, it is worth noting that other methods are unable to perform both sound source localization and recognition simultaneously, as we do. Here, we present the algorithmic complexity analysis of our models and other ANN models on SLoClas dataset in Tab. R1-4. The number of operations (OPs) refers to synaptic operations (SOPs) in SNN and FLOPs in ANN. It can be seen that our method achieves high accuracy while requiring the lowest algorithmic complexity.

| Method | OPs (M) | 
| :---- | :----: |
| GCC-PHAT-CNN | 48.08 |
|SLC-Net-2-128 | 66.82 |
|SpikSLC-Net-1-128 (Ours) | **17.15** |
| SpikSLC-Net-3-128 (Ours) | 55.80 |
**Table R1-4: Comparison of algorithmic complexity.**