SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network
Authors: Weiyu Guo
Ying Sun
Yijie Xu
Ziyue Qiao
Yongkui Yang
Hui Xiong
Venue: NeurIPS 2024 poster (2024)
Keywords: Surface Electromyography
Spiking Neural Network
Spiking Jaccard Attention
Source-Free Domain Adaptation
Human-computer Interaction
Gesture Recognition
Primary area: neuroscience_and_cognitive_science

Abstract: Surface electromyography (sEMG) based gesture recognition offers a natural and intuitive interaction modality for wearable devices. Despite significant advancements in sEMG-based gesture recognition models, existing methods often suffer from high computational latency and increased energy consumption. Additionally, the inherent instability of sEMG signals, combined with their sensitivity to distribution shifts in real-world settings, compromises model robustness. 
To tackle these challenges, we propose a novel SpGesture framework based on Spiking Neural Networks, which possesses several unique merits compared with existing methods: (1) Robustness: By utilizing membrane potential as a memory list, we pioneer the introduction of Source-Free Domain Adaptation into SNN for the first time. This enables SpGesture to mitigate the accuracy degradation caused by distribution shifts. (2) High Accuracy: With a novel Spiking Jaccard Attention, SpGesture enhances the SNNs' ability to represent sEMG features, leading to a notable rise in system accuracy. To validate SpGesture's performance, we collected a new sEMG gesture dataset which has different forearm postures, where SpGesture achieved the highest accuracy among the baselines ($89.26\%$). Moreover, the actual deployment on the CPU demonstrated a latency below 100ms, well within real-time requirements. This impressive performance showcases SpGesture's potential to enhance the applicability of sEMG in real-world scenarios. The code is available at https://github.com/guoweiyu/SpGesture/.

SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard A Attentive Spiking Neural Network
Weiyu Guo!
Ying Sun Yijie Xu' Ziyue Qiao² Yongkui Yang"
Hui Xiong 1.4."
Thrust of Artificial Intelligence, HKUST (Guangzhou), China School of Computing and Information Technology, Great Bay University. China
$ Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China DDppartment Computer Science and Engineering, HKUST, Hong Kong SAR, China wguo3998 hkust-gz. edu, cn; yingsehkust gz. edu. en; yxu4098connece hkust-gz. weausstgz.edu.oo yk. yangdsiat cn; xionghui@ust. Brichui@@st.hae
edu. cn; zyqiao8gbu.edt
Abstract
Surface electromyography (sEMG) based gesture recognition offers a natural and intuitive interaction modality for wearable devices. Despite significant advance- ments in sEMG-based gesture recognition models, existing methods often suffer from high computational latency and increased energy consumption. Additionally, the inherent instability of sEMG signals, combined with their sensitivity to distri- bution shifts real-world settings, compromises model robustness. To tackle these challenges, we propose novel SpGesture framework based on Spiking Neural Networks, which possesses several unique merits compared with existing methods: (1) Robustness: By utilizing membrane potential as memory list, we pioneer the introduction of Source-Free Free Domain Adaptation into SNN for the first time. This enables SpGesture to mitigate the accuracy degradation caused by distribution shifts. (2) High Accuracy: With a novel Spiking Jaccard Attention, SpGesture enhances the SNNs' ability to represent sEMG features, leading to notable rise in system accuracy.
validate SpGesture's performance, we collected a new sEMG gesture dataset which has different forearm postures, where SpGesture achieved the highest accuracy among the baselines (89.26%) Moreover, the actual deployment on the CPU demonstrated latency below 100ms, well within real-time requirements. This impressive performance showcases SpGesture's 's potential enhance the applicability of sEMG in real-world scenarios. The code is available at httts:://itthut hwt.ggtthb..om..eeeeeeemmmee wtwpeeiys//peesturreee/
Introduction
Surface electromyography (sEMG) is a sensing modality that decodes motor intentions from muscle electrical signals preceding movement to enable natural and intuitive interactions. It has distinct advantages gesture recognition for real-time applications. Specifically, sEMG provides rich and comprehensive motion information, making it an excellent resource for accurate and efficient wearable gesture recognition [12]. Moreover, sEMG signals can emerge anywhere from 50 to 150 milliseconds prior to the actual motor activity, enabling the anticipation of movements.
provide unparalleled chance for developing more practical and efficient sEMG-based gesture recognition systems. SNNs emulate
In recent years, Spiking Neural Networks (SNNs) 41
20
"Corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

the spiking behavior of biological neurons with unique binary information communication proto- col [53]. This binary communication particularly amenable the architectural specifics of sparse neuromorphic hardware [50]. Besides, the primary computations in SNNs revolve around spike-hased accumulate (AC) operations [3]. The event-driven nature of these networks [5T
23 X2 enables
calculations to be made only when there is change or 'event" in the input, thereby circumventing
the need to process zero values. Therefore, compared to conventional Artificial Neural Networks muuliply-aad-accumulle
(ANNs) that typically rely on energy energy-demanding and are normally deployed on high-computing-poyye -oower hardware like GPUs, SNNs demonstrate sub- stantially lower power consumption [4). positioning them a promising candidate for developing energy-efficieee gesture recognition systems 2 52).
(MAC) operations [64]
Although SNNs are computationally efficient, they struggle to match the accuracy of ANN- models [5]. particular, the major problem is that the binary and sparse feature representations make it difficult to perform regular contiguous similarity computations. This limitation hinders expressive operations like attention mechanisms and advanced representation alignment algorithms, such as domain adaptation. For example, attention- -bseed structures like the Transformer have demonstrated remarkable performance in Natural Language Processing [66] 40
g 69], Computer
Vision 226 30 39
70 68 Time-Series Processing 776 77 775 and Decision- Making tasks (56 (5],
leading to a wave of attention-centric architecture designs, underscoring the importance and versatility of attention mechanisms in deep learning. However, with the proportion of 's typically less than 5%, the dot product in cosine similarity inherent to attention mechanisms tends to yield results close to zero [67] Existing work often first converts spike signals into continuous values for similarity calculations, but this can increase the inference latency and energy consumption of SNNs. There is still lack of methods for directly implementing advanced operations on spike features in SNN for realizing effective EEMG-based gesture recognition systems.
To tackle these challenges, propose SNN-based solution for a low-power yet accurate sEMG- based gesture recognition framework. Specifically, we first propose a novel Jaccard Attention Spiking Neural Network (JASNN) to enhance the representativeness of the network for sEMG features. In particular, different from existing studies that exploit attention to regulate membrane potentials and subsequently influence spiking activity [67], we propose Spiking Jaccard Attention that calculates attention directly on spike sequences, which enables more straightforward computationally effective attention calculation under SNN schema. Indeed, such a computation process predominantly involves 'comparison' operations, aligning well with the design principles of neuromorphic chips and preserving the low-power properties of SNNs. Moreover, to address the distribution shift problem, we propose novel Spiking Source-Free Domain Adaptation based on Membrane Potential Memory, Our method leverages the changing membrane potential curve memory list and uses it to generate pseudo-labels based on the k-nearest neighbors that are most similar to the current sample. In particular, we incorporate a random exploration mechanism to avoid overfitting during pseudo-label generation and bolster the model's 's generalizability. With our method, we achieve knowledge transfer without sharing the data, which enhances gesture recognition accuracy in an unlabeled environment under privacy reservation.
To better reflect real-world conditions, we collect sEMG-based gesture dataset that includes different forearm postures, acknowledging that variations in forearm posture can significantly influ- ence the distribution of sEMG data. Our experimental results demonstrate that our algorithm not only significantly outperforms other SNN-based algorithms in gesture recognition accuracy but also matches the performance of state-of-the-art methods in the Deep Neural Networks (DNNs) category. Furthermore, the Spiking Jaccard attention method we proposed substantially enhances the accuracy of SNN algorithms. Regarding inference speed, Spiking Jaccard attention 36.37x faster on a CPU than traditional attention mechanisms. Our innovatively designed SSFDA method, which does not require source data or labels, improved the gesture recognition accuracy by 4.5%. These results collectively underline the effectiveness and efficiency of our proposed approach in addressing the challenges in BEMG-based gesture recognition. Our contribution can be summarized as follows:
We propose a Jaccard similarity- attention mechanism specifically designed for SNNs. This innovative approach preserves the original computational characteristics of SNNs, boosts inference efficiency, and counteracts the accuracy degradation caused by sparse spiking sequences.
To the best of our knowledge, we are among the first to propose an SNN-oriented SFDA algorithm.
This enables users to capture gesture actions under one specific forearm posture and empowers the
2

---

model to unsupervised learning the features under other forearm postures, thereby bolstering its robustness during actual use.
We collect new sEMM-baased gesture dataset that features a variety of forearm postures. This dataset can provide valuable resources for researchers aiming to develop robust gesture recognition algorithms for different forearm postures.
The experimental results demonstrate performance improvements over state-of-the-art sEMG gesture recognition models, with particular benefits under varying forearm orientations. Our model also provides higher efficiency than existing attention schemes.
2 Related Works
Spiking Neural Networks (SNNs), the third generation of neural networks, mimic biological neurons through binary spiking signals and handle temporal information effectively [[9]. SNNs are energy- efficient, activating only a small portion of neurons during computation, unlike dense ANNs that rely on energy-intensive operations (2). Neuromorphic chips like Tianjic [14), TrueNorth [1, and Loihi 13 exemplify this efficiency. Despite their energy advantages, SNNs have lower accuracy than DNNs due to sparse feature representation and simplistic structures. Attention mechanisms, widely used in DNNs (60), are under-explored in SNNs, posing challenges like spike degradation and gradient vanishing. Addressing these issues crucial for improving SNN performance.
Domain Adaptation (DA) aims to leverage labeled source domain data to improve performance on unlabeled target domains, addressing domain shifts 62] Traditional DA requires access both source and target data, which is impractical in scenarios involving privacy or resource constraints [36]. Source-Free Domain Adaptation (SFDA) addresses this by adapting models without source data, crucial for categorized into data-centric and model-centric approaches 449 Data-centric methods extend UDA techniques by reconstructing virtual domains or translating target data into source-style data [37] Model-centric methods, like pseudo-labeling (35), entropy minimization [6) and contrastive learn- ing (79), fine-tune models using target data. However, applying SFDA to SNNs is challenging due to their lower stability and sparse outputs.
applications like sEMG gesture recognition [22]. SFDA methods are
3 Preliminaries
Data Collection: In human-computt interaction studies involving sEMG, diverse and represen- tative data sets are crucial. Traditional research often collects sEMG data from single forearm posture 4 (11] but variations in forearm posture significantly influence sEMG data distribution. Our methodology incorporates gestures performed in different forearm postures to better reflect real-world conditions. Participants were instructed replicate gestures and forearm postures shown screen. Our dataset includes ten gestures across three forearm postures: Pl (forearm horizontal on a surface), P2 (forearm elevated diagonally with elbow anchored), and P3 (forearm horizontal). Each gesture was held for five seconds with a five-second relaxation period, repeated six times per posture. This approach aims to provide robust sEMG data reflecting practical variability. For further details dataset collection, including information about the acquisition devices and specific measures taken, please refer appendix A.I
We used Root Mean Square (RMS) for initial feature extraction to enhance gesture recognition stability. RMS efficiently summarizes signal magnitude, indicating signal power. A 100ms time window with a 0.5ms step size captured transient sEMG characteristics, extracting features while maintaining high-resolution signal variations. RMS is further explained in appendix A.2
Data Processing:
Method
In the subsequent sections of this paper, we will present sEMG-based gesture recognition solution with SNNs capable of handling distribution shifts. This solution can be divided into Jaccard Attention Spiking Neural Network (JASNN) and Spiking Source-Free Domain Adaptation (SSFDA). Firstly, we will introduce the unique SNN backbone JASNN deployed in our study. Following this, we will delve into our innovative design, the novel implementation of SSFDA within an SNNs framework first in the field. For more details about SNN, please refer to appendix A.3
3

---

Feature Extractor
Spiking Jaccard Attention ML
LFC
V
Raw sEMG
Spike Train
2N channel
Conv
LIF
Jaccard K 1
LIF
LT
Spike Eacoding
LIF
LI
CoavLIF
ConvLIP
Cony L.I---
Conv
LIF
Conv LIF
channel
Membrane Potential Memory
Figure 1: The pipeline of Jaceard Attention Spike Neural Network: Raw sEMG Data is first encoded into Spike Signals using ConyLIR These signals pass through CInvLIF layers with N and 2N channels, The processed data then goes through the Spiking Jaccard Attention mechanism.
4.1 Jaccard Attentive Spiking Neural Network
4.1.1 Network Overview
Our proposed Jaccard Attentive Spiking Neural Network (JASNN) comprises four primary compo- nents: Convolutional Leaky integrate-and-Fin (ConvLIF) based spike encoder and feature extractor, a Spiking Jaccard attention mechanism, LIF-based Classifier, and membrane potential recording module. We detailed the first and third components in appendix A.4 A.6
The ConvLIF-based. spike encoding layer dynamically encodes sEMG signals into spike trains, capturing temporal dynamics effectively. The Multi-Channel ConvLIF extractor transforms these spikes into a digher-dimensionall space for better feature representation. The Spiking Jaccard attention mechanism focuses task-relevant features, enhancing meaningful information. The modified LIF layer translates spiking activity into classification results based on the highest membrane potential. Finally, the membrane potential recording module converts output spikes into membrane potentials for source-free domain adaptation.
4.1.2 Spiking Jaccard Attention
Attention mechanisms have enhanced DNNs in time-series analysis by focusing on important temporal aspects for better predictions. However, applying attention mechanisms to Spiking Neural Networks (SNNs) presents unique challenges. Firstly, SNNs' sparse neuron activation makes the dot product operation in attention mechanisms produce sparse spike trains, hindering learning due to reduced signal strength. Secondly, using the softmax function for attention scores increases computational complexity and energy consumption, which is unsuitable for SNNs' efficient processing requirements.
To address these concerns, we propose a novel Spiking Jaccard Attention (SJA) mechanism specif- ically designed for SNNs. As shown in Figure 2, unlike the method by Yao et al. [67], SJA can directly calculate the similarity on spike trains and retains the attention's 's query mechanism.
Given the binary nature of SNN layers outputs, the dot product approach attention will make the feature too sparse. We introduce the SJA mechanism based the Jaccard similarity, which is better suited for binary data. The Jaccard similarity between two sets A and can be defined as:
| n B
Jaccard (A, B) = A J B
Generally speaking, designing spiking chip for SNNs mainly involves a large number of addition circuits and comparison circuits. Therefore, in the practical implementation of our proposed SIA, we retain the computational characteristics of the spiking chip to compute the Jaccard similarity more efficiently. This is achieved by calculating the intersection and union of the vectors using element-wise minimum and maximum operations, respectively. For two vectors and y, it can be described as:
Σ, min (xs, Y:)
Jaccard y) Σ, max (x,, Yz) +

---

Weights W
Weights W
Attention Weights
Yao ol.
Spike Signals [0.1]
Continuous Values E R
Querying Mechanism!
Continuous Values Lack Efffciency
Attention Weights
Our Mothod
Jaccurd Similarity
Spike Signals [0.1]
Spike Values
Spike Spike Values Ensure Efficiency
Possess Query Mechanism
Figure Comparison of MA-SNN and Spiking Jaccard Attention Modules. MA-SNN 67 uses fully connected layers with pooling but lacks a querying mechanism, leading to continuous intermediate values and lower efficiency. Our Spiking Jaccard Attention uses spike values for intermediate representations, enhancing efficiency and accuracy.
This approach enables efficient computation of the Jaccard similarity by taking advantage of the sparsity the data SNNs. By computing the sums over the element-wise minimum and maximum operations instead of using matrix dot multiplication operations, our algorithm becomes more easily deployable on Ncuromorphic chips, thereby enhancing the computational efficiency of the attention mechanism within SNNs. We add tiny constant to the denominator to avoid division by zero when there are no spikes the spike train.
So, we can modify the traditional attention formula by incorporating Jaccard similarity into the attention mechanism. The resulting SJA mechanism can be expressed as:
E, min (ge, kz)
SJA K)
Σ, max (yir kj) + V,
where Q, K. and V represent the query, key, and value matrices, respectively, and and k; are the corresponding elements in the query and key matrices.
First, we consider the channel-wise uniform weighting method. This approach implies that the same weighting coefficient applied to elements along the channel dimension of V. In this case, the attention weight is computed as scalar, calculated by aggregating the elements within each channel: where is the index of the elements within the channel. The resulting sealar is then used as weighting factor applied to each channel of V:
V Veew c, Jaccard (Q, K) V[i,c,
where e represents the channel index. In this way, the values across channels are scaled by the same weighting factor, thereby maintaining consistency across different channels.
Second, we consider the element- wise weighting method. this case, the Jaccard Q, K) result is computed independently for each element position g, k. This means that the attention weight for each element is obtained by calculating the value for the corresponding elements Q and K at that position. These weights are then applied element-wise to V:
V View I., n = Jaccard Q, K)J: n] cl,.c,n)
(5)
where n represents the index along the sequence length. Under this element-wise weighting strategy, different positions within V scaled independently based their respective attention weights, which enables the model to capture finergrrannn features.
The results presented in this paper derived using the channel-wise weighting approach, as it more suitable for the characteristics of sEMG data, and we did not validate the element-wise weighting approach due these characteristics.
5

---

These two weighting strategies each have their respective applications: channel uniform weight- ing more appropriate for preserving feature consistency, while element- wiie weighting is better suited for capturing localized differences. Depending on the computational complexity and the task requirements, an appropriate weighting strategy can be selected to achieve a balance between efficiency and performance.
The SJA mechanism leverages the sparsity of SNN outputs significantly reduce computational complexity. Unlike traditional attention mechanisms with a complexity of O d), SJA focuses only on non-zero elements, resulting in a complexity of O(b), where is the number of non-zero elements. This approach enhances computational efficiency and reduces energy consumption, as addition operations dominate SJA compared the multiplication-heat traditional attention, making SJA particularly advantageous for SNNs. Further complexity analysis can be found in appendix A.5
4.2 Spiking Source-Free Domain Adaptation based on Membrane Potential Memory
The formal definition of the problem as follows: given a labeled source domain D, = {(xt, yf) i=m an unlabeled target domain D. = و- N. and a model fs trained on D,, the goal is to adapt or fine-tune the model f. such that its performance on the target domain D; is optimized. The primary challenge stems from the different data distributions of the source and target domains, i.e., (z y) & (2. y). where P, and Pj denote the data distributions of the source and target domains, respectively. In SFDA, the added complexity is that the source data D, is not available when adapting or fine-tuning the model, while only having the source model fa. Thus, the adaptation must rely on the properties and capabilities of the source model and unlabeled target data.
Membrane Potential Memory List
Top K most similar Membrane Potential
Randomly select pseudo-labels
Pearson correlation coefficient
pseudo-lahel
Use mode as pseudo-labels
JASNN
a) NLL + KL
Raw sEMG
Figure 3: Computation flow of Spiking Source-Free Domain Adaptation. The process starts with selecting the k-nearest samples from the membrane potential memory using the Pearson correlation coefficient. Probabilistic Label Generation then produces pseudo-labels based on these k samples. Gradients are computed with Smooth NLL and KL. divergence loss. The membrane potential memory list is updated at each epoch's 's end.
Most previous methods consider similarity based on instance discrimination among all features in their loss functions, which can lead to high computational costs. This requirement can generate a significant computational overhead. In line with the approach taken by [65], we generate pseudo-labels using the k-most similar samples to the target sample with consistency regularization. Furthermore, we introduce an exploration mechanism to mitigate overfitting. This strategy effectively maintains computational efficiency while enhancing the robustness and generalization of our SFDA approach.
Another challenge is that the intermediate layer features SNNs are represented by Spike Trains, and existing methods for finding neighbors cannot directly compute them. To identify the semantically closest neighbors to a target domain sample, we utilize the membrane potential from the Memory Layer to construct Membrane Potential Memory List. Note that we only use target source data to generate the Membrane Potential Memory List. The membrane potential encapsulates both spatial and temporal features, rendering it a more informative and efficient tool for our purpose. Membrane
Potential Memory M,, = { [m t= can be computed by:
n,t = St +8 N 1),
(6)

---

where [0, 1) represents Gaussian noise with mean 0 and standard deviation 1, and ó is scaling factor. Integrating Gaussian noise with scaling offers two key benefits: regularization helps prevent overfitting, allowing the model to generalize better to unseen data, and noise introduction reduces the dominance of zeros in spike data, leading to a more balanced data representation. Figure shows the SSFDA computation flow.
The core of the loss function the alignment of predictions between the current target feature and its k-nearest neighbors in the Membrane Potential List, identified based on Pearson similarity. To achieve this, we introduce the following loss function Smooth Negative Log Likelihood (SNLL) Loss that combines two crucial components:
K
C KL (Pe ge),
-
log (p(a;) argmax (Sx)) + a Σ
12
where
mode
argmax (IM))))
with probability (1 p);
S₁
(8)
random
arrgmax
((M))); with probability p.
[M)] {Fy topp (Pearson (f(x;): F,
(9)
1 Σ Pe (z,) 9e
1 T.
for c 1,2,... C.
(10)
The loss function, L, is composed of two main terms: Consistency Term: The first component,
-(1 - aa En Σk log (p(z;) argmax (Sx)), is designed to advocate consistent predictions between a target feature and its k-nearest neighbors. It strives to minimize the negative logarithm of the inner product of the prediction score for the target sample, denoted by p (2;), and the aggregated prediction scores represented by argmax (S&) of its k-nearest neighbors. Sş represents either the
mode of the argmax values from the subset {M} with probability or random selection from the
same subset with probability (1 11-p) p). By inducing similarity in predictions among closely related features, our model can discover latent structures and associations within the data; Regularization
Term: The subsequent component, & Σ KL (pe |l ge), uses the Kullback-Leible divergence to measure the discrepancy between the model's 's average predicted class distribution, Pc. and the ideal uniform distribution across classes, ge- Specifically, P. denotes the model's 's average prediction probability for class e over all data samples. By comparing Pe with Jc. the divergence quantifies the deviation of the model's predictions from a perfectly balanced class distribution. The aim is to reduce the model's inclination to favor certain classes overly, ensuring more balanced prediction landscape. In this configuration, The scalar in the loss function acts as balancing factor between predictive consistency and regularization.
4.3 Training Method
Deep Spiking Neural Networks (SNNs) are typically trained using ANN-to-SNN conversion or direct training. While ANN-to-SNN conversion faces latency challenges, direct training is more time-step efficient and suitable for temporal tasks. We use rate coding for its support of complex SNNs. In this paper, we the SuperSpike (73 surrogate gradient calculate gradients, with detailed explanations provided in appendix A.7
5
Experiment
5.1 Gesture Recognition based on sEMG
We compared our model's 's performance with existing sEMG-based gesture estimation models, pri- marily categorized into DNN and SNN architectures. A comparison summary is in Table
In terms of Top-1 Accuracy, our JASNN model, which integrates the SNN framework with the SJA mechanism, outperforms other DNN models, including CNN, TCN (5) Transformer [60], GRU [ Informer [80), and hybrid TCN with an Attention mechanism. This superior performance is due to: 1) The SNN structure's 's alignment with the biological basis of sEMG generation, providing a natural
7

---

Table 1: Comparison with previous works on sEMG-based gesture estimation.
Methods
Work
Model
Top-1 Acc.(%)
Std. Dev. (%)
Asif al. 2020
CNN
75,46
0.52
Tsinganos 2020
59
TCN
79.69
0.83
Rahimian et al. 2021
Transformer
84,23
0.37
Chen et al. 2021
9
GRU
82.19
0.28
DNN
7hou al. 2021 [80]
Informer
88.32
0.36
Rahimian et al. 2022 48
TCN+Attention
87.10
0.57
Zhang et al. 2023
Transformer
86.24
0.31
Bellec et al. 2018
LSNN
86.24
0.22
SNN
Zhang et al. 2022 774
SIB+SNN
77.84
0.62
Xu el al. 2023 63
SCNN
84.30
0.23
SOTA backbone
LSNN+SIA(OurS,
88.10
0.25
SNN-Ours
This Work
JASNN
89.26
0.31
modeling of the processes. 2) The SJA mechanism's enhancement of sparse spike train features focuses the key characteristics of sEMG signals. Compared to other SNN models like LSNN (7). SIB+SNN [78]. and SCNN, our model achieves higher accuracy. Models like SIB+SNN and SCNN feature enhancement design like SJA, which is crucial for capturing the temporal dynamics of sEMG signals. Incorporating SJA into Xu et al. 's LSNN network (63 significantly improved performance, demonstrating SJA's scalability recurrent SNNs.
perform lower, likely due to the absence
5.2 Ablation Study
To validate the effectiveness of each module have proposed, present the results of an ablation study. Here, we discuss the impact the backbone's 's attention mechanisms and loss functions on the experimental results and the influence of different pseudo-label generation methods within SSFDA. All experimental results in this section are based on the mean values across all fifteen subjects in the dataset. The same learning rate, batch size, and optimizer were used during training, ensuring each network converges (with training set accuracy showing less than 0.2% improvement over five consecutive epochs). This thorough examination allows to isolate the individual contributions of the different components and clarify their specific roles in the performance of our proposed system.
5.2.1 Attention Mechanisms
In our ablation study on attention mechanisms, we compared Raw Attention [60]. MA-SNN [67]. and our proposed Spiking Jaccard Attention (SJA) on SCNN, keeping all other parameters consistent. As shown in Table using Raw Attention directly on spikes resulted in an accuracy of only 1.3 31% % due the high sparsity of spike sequences. This sparsity often leads to information loss when multiplying matrices with sparse values. MA-SNN converts spike sequences into continuous values and uses fully connected layer for attention, which increased SCNN's accuracy from 84.12% to 85.67%. However, this approach reduces the usability of SNNs spiking chips. In contrast, our SJA computes attention weights directly on the spike sequence, preserving compatibility with spiking hardware and further boosting accuracy to 87.44%. This highlights SJA's superior ability to handle spike sequence sparsity while maintaining hardware compatibility.
5.2.2 Loss Functions
The study compared Negative Log Likelihood (NLL) loss and our improved Smooth NLL with Kullback-Leibler divergence loss (SNLL+KLL) for classification tasks. Using SNLL+KLL in JASNN increased accuracy from 87.44% 89.72% (see Table This enhancement due to: Kull- back-Leibler (KL) divergence: KL divergence quantifies the difference between two probability distributions, encouraging predicted probabilities to closely match actual class distributions. This reduces model biases towards certain categories. Smooth NLL (SNLL): SNLL ensures consistent predictions between a feature and its k-nearest neighbors the embedding space, enhancing model sensitivity detailed class clusters and underlying patterns. In summary, adding KL divergence and SNLL improves model strength and fairness, enhancing flcxibility across various datasets and tasks.
5.2.3 Pseudo-Label Generation Methods
We evaluated three pseudo-lahel (PI.) generation methods: Duan al. 17 (PI.), Huang el 22 (NPL), and proposed Probabilistic Label Generation (PLG) method. Both PL and NPL improved

---

Table Ablation study results
Attentions
Loss Functions
Label Selection
Results
SCNN
RA
MA-SNN
SIA (Ours)
NLL
SNLL+KLL (Ouns)
NPL PLG (Ours)
ACC
Improved ACC
84.12%
11.51%
Backbone
85.67%
89.72%
1887%
Source-
3.81%:
4.10%
accuracy an unsupervised setting on JASNN by 1.87% and 2.33%, respectively. Our PLG method achieved significant boost, enhancing accuracy by 4.1%. Additionally, PLG increased accuracy MA-SNN by 3.81%, 1,, demonstrating its scalability across different architectures. The effectiveness of PLG comes from selecting the mode of neighboring labels as the pscudo-label and introducing a probabilistic mechanism to explore other labels, preventing overly compact feature distribution and enhancing generalization. This makes PLG powerful tool for unsupervised adaptation in SEMG-based gesture recognition.
5.3 Variant Distributions of Hand Gestures with Three Different Forearm Postures
In this study, we evaluated datasets from three forearm postures (P1, P2, P3) to train models without SSFDA, revealing challenges with sEMG data. The model trained on P1 achieved high accuracy on Pl's test set but dropped to 30% accuracy on P2 and P3 due to variations in motor neuron firing patterns. Figure iilusttraees this performance disparity, highlighting the out-of- of-dissribuutin (OOD) issue. These findings underscore the need to address OOD phenomena sEMG data to enhance the reliability and user experience of sEMG-based systems.
5.4 Result of Spiking Source-Free Domain Adaptation
In our investigation, we used the same dataset to experiment with three different methodologies, namely Pseudo-Label 34 17 method, Neighborhood-guidee Pseudo-Labels (8 method, and proposed Probabilistic Label Generation method. In this experiment, the Pseudo-Labe method determines the pseudo-label by taking the mode of the l samples. Conversely, the neighborhood- guided Pseudo-labeles method involves choosing the nearest k samples from the memory list and then randomly selecting one from these & samples the pseudo-label. Details of our proposed method have been elaborated on in the previous sections of the paper.
Poeae P1-2
100
Partre PLPP
90
8
Ours
Daan Mathid
Hunng etal
Uurs
Daan et Medood
Honng ofat
(a)
(b)
Figure 4: Comparison of performance before and after applying SSFDA for various methodologies:
Figures 4a and 4b are Violin Plots demonstrating this disparity.
Figure 4a and 4b represent the performance variations when deploying the model trained on Posture to Posture 2 and 3, respectively, both with and without the use of our SSFDA. This is portrayed using a violin plot. can be observed that the use of SSFDA indeed shifts the distribution of accuracy across different subjects upward as whole. Particularly, after applying SSFDA compared to the other two methods. Furthermore, the standard deviation of performance across various subjects is minimal for method, demonstrating the robustness of our methodology when employing SSEDA. We detailed the differences by individuals in Figure 10a 10b in the appendix.
method exhibits superior performance

---

5.5 Efficiency Analysis of Spiking Jaccard Attention
Inference latency influences user experience, with delays leading to missed or inappropriate actions. Attention mechanisms are pivotal for efficient interactions. We compared SJA's computational superi- ority over Raw [60] and Efficient [54] Attention, conducting 100 inference tests using pseudo-data on twelve channels, common practice in sEMG data. Results averaged and shown in Figure(5 highlight SJA's clear advantages. Regardless of the computing platform or data type, SJA demonstrated superior efficiency in inference speed and RAM consumption, making it ideal for real-time and mobile devices, Additionally, SJA showed better scalability, with only a gentle increase in inference time and RAM usage as sEMG data length increased, compared to Raw Attention's 's exponential growth.
Comparison
typer verying row
Times
Tha Time
Momory
Flat- Memery
120
2014
Nechho
102
305
Necheo-f
8:00
$110
102
2.10
1277
8:10
X00
102
2.10
1377
6710
Figure 5: Inference speed and RAM usage comparison between spike and float data for Raw Attention [60. Efficient Attention [54, and our Spiking Jaceard Attention: The first column shows inference time for float data, and the second for spike data. The third and fourth columns show RAM usage for these data types. The z-axis represents different data row counts, and the y-axis is logarithmic to highlight performance differences. Each experiment was conducted 100 times, with averaged results.
5.6 Real World Deployment
SpGesture has been deployed in real-world application using an in-house developed sEMG acquisi- tion system, as illustrated in appendix A.
Limitation and Future Work
Domain Adaptation Various Network Structures: We verified the ability of SJA and SSFDA to enhance the accuracy of sEMG-based gesture recognition, along with their adaptability to distribution shift based the Spiking Convolutional Neural Network architecture. Moving forward, we intend to assess their robustness across a wider variety of SNNs and different tasks.
Performance Analysis on Neuromorphic Chips: Our current evaluations of inference speed and memory utilization are conducted on CPU and GPU platforms, where our system demonstrates clear advantages over existing algorithms. We believe that these advantages will be further amplified on neuromorphic chips. We are currently developing neuromorphic chips and will conduct practical tests on these chips to measure the system's 's energy consumption and inference efficiency.
7 Conclusion
We presented SpGesture, an innovative framework for sEMG-based gesture recognition built on SNN, and innovatively introduced Spiking Source-Fres Domain Adaptation with Spiking Jaccard Attention, which directly enhances spike features. These novel contributions improve the system's 's robustness and accuracy in real-world scenarios. Our experimental results include the highest accuracy among baselines and system latency below 100ms on a CPU, demonstrating its real-world applicability. Our proposed SJA processes spike sequences at 36.37 times the speed of conventional attention and can be extended to other SNNs, such as LSNN. SpGesture not only offers practical solution to current challenges in gesture recognition but also opens new possibilities for Human- Human-computer Interaction.
Acknowledgments and Disclosure of Funding
This work is partly supported by the National Key Research and Development Program of China (No. 2023YFF0725001,, the National Natural Science Foundation of China (No. 92370204, 62306255), the Natural Science Foundation of Guangdong Province (No. 2024A 0024A151500 151501 1839), the Guangzhou- HKUST(GZ)) Joint Funding Program (00.2023003000005, and the Education Bureau of Guangzhou Municipality. Yijie Xu acknowledges the support from the modern matter laboratory, HKUST(GZZ.
10

---

References
[1] Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-lcaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, al. Truenorth: Design and tool flow of 65 mw I million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems, 34(10): 15370))5377-557, 2015.
[2] Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, Jeff Kusnitz, Michael Debole, Steve Esser, Tobi Delbruck, Myron Flickner, and Dharmendra Modha. A low power, fully event-based gesture recognition system. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7388-7397 2017.
[3] Ali Raza Asif, Asim Waris, Syed Omer Gilani, Mohsin Jamil, Hassan Ashraf, Muhammad Shafique, and Imran Khan Niazi. Performance evaluation of convolutional neural network for hand gesture recognition using emg. Sensors, 20(6): 2020.
[4] Manfredo Atzori, Arjan Gijsberts, Claudio Castellini, Barbara Caputo. Hager, Simone Elsig, Giorgio Giatsidis, Franco Bassetto. and Henning Müller. Electromyo- graphy data for non-invasive naturally-controlled robotic hand prostheses. Scientific data, 1(1):1-13, 2014.
Mittaz
[5] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, arXiv preprint 1803.00 2018.
[6] Mathilde Bateson. Hoel Kervadec, Jose Dolz, Hervé Lombaert, and Ismail Ben Ayed. Source- free domain adaptation for image segmentation. Medical Image Analysis, 82:102617, 2022.
[7] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-lent in networks of spiking neurons. Advances in neural information processing systems, 31, 2018.
[8] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901 2020.
[9] Rui Chen, YuanZhi Chen, Weiyu Guo, Chao Chen, Zheng Wang, and Yongkui Yang. semg- based gesture recognition using gru with strong robustness against forearm posture. In 2021 IEEE International Conference on Real-time Computing and Robotics (RCAR), pages 275-280. IEEE, 2021.
[10] Yanqi Chen, Zhaofei Yu, Wei Fang, Zhengyu Ma, Tiejun Huang, and Yonghong Tian. State transition of dendritic spines improves learning of sparse spiking neural networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, edi- tors, Proceedings of the 39th International Conference on Machine Leurning, volume 162 of Proceedings of Machine Learning Reseurch, pages 3701-3715. PMLR, 17-23 Jul 2022.
[11] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder- decoder for statistical machine translation. arXiv preprint urXiv::40k 1078 2014.
[12] Andrea Cimolato, Josephus JM Driessen, Leonardo S Mattos, Elena De Momi, Matteo Laf- franchi, and Lorenzo De Michieli. Emg-driven control in lower limb prostheses: A topic-based systematic review. Journal of NeuroEngineering and Rehabilitation, 19(1):1-26, 2022.
[13] Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A Fonseca Guerra, Prasad Joshi, Philipp Plank, and Sumedh R Risbud. Advancing neuromorphic computing with loihi: survey of results and outlook. Proceedings of the IEFF, (09(5)91119944 2021.
[14] Lei Deng, Guanrui Wang, Guoqi Li, Shuangchen Li, Ling Liang, Maohua Zhu, Yujie Wu, Zheyu Yang, Zhe Zou, Jing Pei, et al. Tianjic: unified and scalable chip bridging spike-based and continuous neural computation. IEEE Journal of Solid-State Circuits, 55(8):222822 222222226,,099 2020.
11

---

[15] Lei Deng, Yujie Wu, Xing Hu, Ling Liang, Yufei Ding, Guoqi Li, Guangshe Zhao, Peng Li, and Yuan Xie. Rethinking the performance comparison between smns and anns. Neural networks, 3077, 2020.
[16] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4181-4186, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics.
[17] Di Duan, Huanqi Yang, Guohao Lan, Tianxing Li, Xiaohua Jia, and Weitao Xu. Emgsense: low-effort self-supervised domain adaptation framework for emg sensing. In 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 160-170. IEEE, 2023.
[18] Steve K Esser, Rathinakumar Appuswamy, Paul Merolla, John Arthur, and Dharmendra Modha. Backpropagation for eneryy-effecient neuromorphic computing. Advances in neural information processing systems, 28, 2015.
[19] Haowen Fang, Amar Shrestha, Ziyi Zhao, and Qinru Qiu. Exploiting neuron and synapse filter dynamics in spatial temporal learning of deep spiking neural network. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 2799-28990 2021.
[20] Samanwoy Shosh-Dastidde and Hojjat Adeli. Spiking neural networks. International journal of neural systems, 19(04):255-308 2009.
[21] Samanwoy Ghosh-Dastidar and Hojjat Adeli, Third generation neural networks: Spiking neural networks. In Advances in computational intelligence, pages 167-178. Springer, 2009.
[22] Tianhao Gu, Zhe Wang, Xinlei Xu, Dongdong Li, Hai Yang, and Wenli Du. Frame-level teacher-student learning with data privacy for eeg emotion recognition. IEEE transactions on neural networks and learning systems, 2022.
[23] Jesse Hagenaars, Federico Paredes- Vallés, and Guido De Croon. Self-supervised learning of event-based optical flow with spiking neural networks. Advances in Neural Information Processing Systems, 34:7167-7179 2021.
[24] Bing Han and Kaushik Roy. Deep spiking neural network: Energy efficiency through time based coding. In European Conference on Computer Vision, pages 388-404. Springer, 2020.
(25) Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13558-13567, 2020.
[26] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang. An Xiao, Chunjing Xu, Yixing Xu, et al. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45())887-110. 2022.
[27] Mark Horowitz. 1.1 computing's energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10-14, 2014.
[28] Jiabo Huang, Qi Dong, Shaogang Gong, and Xiatian Zhu. Unsupervised deep learning by neighbourhood discovery. In International Conference on Machine Learning, pages 2849-28555 PMLR, 2019.
[29] Néstor Jarque-Bou Manfredo Atzori, and Henning Müller, A large calibrated database of hand movements and grasps kinematics. Scientific data, 7(1):12, 2020.
[30] Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. ACM computing surveys (CSUR), 54(10s):1-41, 2022.
12

---

[31] Rami N Khushaba, Ali AI- Al-Timemy, Sarath Kodagoda, and Kianoush Nazarpour. Combined influence of forearm orientation and muscular contraction on emg pattern recognition. Expert Systems with Applications, 61:154-161, 2016.
[32] Agamemnon Krasoulis, Iris Kyranou, Mustapha Suphi Erden, Kianoush Nazarpour, and Sethu Vijaykkuarr Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. Journal of neuroengineering and rehabilitation, 14:1-14, 2017.
[33] Agamemnon Krasoulis, Sethu Vijayakumar, and Kianoush Nazarpour. Effect of user practice on prosthetic finger control with an intuitive myoelectric decoder. Frontiers in neuroscience, 13:891, 2019.
[34] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896, Atlanta, 2013.
[35] Chen Li. Wei Chen, Xin Luo, Yulin He, and Yusong Tan. Adaptive pscudo labeling for source-free domain adaptation in medical image segmentation. ICASSP 2022 2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 1091- 1095, 2022.
[36] Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsuper- vised domain adaptation without source data. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9641-9650, 2020.
[37] Jian Liang, Dapeng Hu, Yunbo Wang. Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions Puttern Analysis and Machine Intelligence, 44(1 1):8622-8611 2021.
[38] Qi Liu and Zhixuan Zhang. Ultralow power always-on intelligent and connected snn- snn-based system for multimedia iot-enabled applications. IEÉE Internet of Things Journal, (17):15570- 15577, 2022.
[39] Yang Liu, Yao Zhang, Yixin Wang, Feng Hou, Jin Yuan, Jiang Tian, Yang Zhang, Zhongchao Shi, Jianping Fan, and Zhiqiang He. A survey of visual transformers. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[40] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint 1907 2019.
[41] Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural networks, 1699)1155- 1997.
[42] Byunggook Na, Jisoo Mok, Seongsik Park, Dongjin Lee, Hyeokjun Choe, and Sungroh Yoon. AutoSNN: Towards energy-efficient spiking neural networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 16253-16269. PMLR, 17-23 Jul 2022.
[43] Francesca Palermo, Matteo Cognolato, Arjan Gijsberts, Henning Müller, Barbara Caputo, and Manfredo Atzori. Repeatability of grasp recognition for robotic hand prosthesis control based on semg data. In 2017 7 International Conference on Rehabilitation Robotics (ICORR), pages 1154-1159 1159. IEEE, 2017.
[44] Christian Pehle and Jens Egholm Pedersen. Norse A deep learning library for spiking neural networks. GütHub. Zenodo, January 2021. Documentation: https://norse.ai// ai/doc..
[45] Nicolas Perez-Nieves and Dan Goodman. Sparse spiking gradient descent. Advances in Neural Information Processing Systems, 34:1 14411195--1800 2021.
[46] Stefano Pizzolato, Luca Tagliapietra, Mattoo Cognolato, Monica Reggiani, Henning Müller, and Manfredo Atzori. Comparison of six electromyography acquisition setups on hand movement classification tasks. PloS one, 1211012 2017.
13

---

[47] Elahe Rahimian, Soheil Zabihi, Amir Asif, Dario Farina, S Farokh Atashzar, and Arash Mohammadi. Temgnet: Deep raasformer-basee decoding of upperlimb semg for hand gestures recognition. arXiv preprint arXiv:2109. 12379, 2021.
[48] Elahe Rahimian, Soheil Zabihi, Amir Asif, Dario Farina, S Farokh Atashzar, and Arash Mo- hammadi. Hand gesture recognition using temporal convolutions and attention mechanism. In ICASSP 2022-2022 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1996-1000 IEEE, 2022.
[49] Alan Ramponi and Barbara Plank. Neural unsupervised domain adaptation in nlp-a survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6838 (855, 2020.
[50] Nitin Rathi, Indranil Chakraborty, Adarsh Kosta, Abhronil Sengupta, Aayush Ankit, Priyadarshini Panda, and Kaushik Roy. Exploring neuromorphic computing based on spiking neural networks: Algorithms to hardware. ACM Computing Surveys, 55(12):1-49, 2023.
[51] Eduardo Ros, Richard Carrillo, Eva M Ortigosa, Boris Barbour, and Rodrigo Agís. Event-driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal dynamics. Neural computation, 18(12):299--2993 2006.
[52] Ali Safa, Federico Corradi, Lars Keuninckx, Ilja Ocket, André Bourdoux, Francky Catthoor, and Georges GE Gielen. Improving the accuracy of spiking neural networks for radar gesture recognition through preprocessing. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[53] William Severa, Craig M Vineyard, Ryan Dellana, Stephen Verzi, and James B Aimone. Training deep neural networks for binary communication with the whetstone method. Nature Machine Intelligence, 122):66-94 2019.
[54] Zhuoran Shen, Mingyuan Zhang, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3531-3539, 2021.
[55] Gopalakrishnan Srinivasan, Chankyu Lee, Abhronil Sengupta, Priyadarshini Panda, Syed Shakib Sarwar, and Kaushik Roy. Training deep spiking neural networks for energy-efficient neuromor- phic computing. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8549-8553. IEEE, 2020.
[56] Ying Sun, Hengshu Zhu, Chuan Qin, Fuzhen Zhuang, Qing He, and Hui Xiong. Discerning decision- making process of deep neural networks with hierarchical voting transformation. Advances in Neurul Information Processing Systems, 34:17221-17234, 2021.
[57] Ying Sun, Fuzhen Zhuang, Hengshu Zhu, Qi Zhang, Qing He, and Hui Xiong. Market-oriented job skill valuation with cooperative composition neural network. Nature communications, 12(1):1992, 2021.
[58] Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothée Masquelier, and Anthony Maida. Deep learning in spiking neural networks. Neural networks, 111:47-63, 2019.
[59] Panagiotis Tsinganos, Bruno Cornelis, Jan Cornelis, Bart Jansen, and Athanassios Skodras. Improved gesture recognition based on semg signals and ten. In ICASSP 2019-2019 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), pages 1 1169- 1173. IEEE, 2019.
[60] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[61] Yuchen Wang, Malu Zhang, Yi Chen, and Hong Qu Signed neuron with memory: Towards simple, accurate and high-efficient ann-snn conversion. In International Joint Conference on Artificial Intelligence, 2022.
14

---

[62] Garrett Wilson and Diane J Cook. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST), 11(5):1-46, 2020.
[63] Mengjuan Xu, Xiang Chen, Antong Sun, Xu Zhang, and Xun Chen. novel event-driven spiking convolutional neural network for eiectromyography pattern recognition. IEEE Transactions on Biomedical Engineering, 2023.
[64] Helin Yang. Kwok- Yan Lam, Liang Xiao, Zehui Xiong, Hao Hu, Dusit Niyato, and H Vin- cent Poor. Lead federated neuromorphic learning for wireless edge artificial intelligence. Nature communications, (:)6994 2022.
[65] Shiqi Yang. Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neighborhood structure for source-free domain adaptation. Advances in neural information processing systems,
2021.
[66] Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang, Yihan Lin, Zhaoxu Yang, and Guoqi Li. Temporal-wise attention spiking neural networks for event streams classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10221- 10230, 2021.
[67] Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE transactions on pattern analysis and machine intelligence, 2023.
[68] Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Hui Xiong. Cross-modality data augmentation for end-to-end sign language translation. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13558-13571, Singapore, December 2023. Association for Computational Linguistics.
[69] Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, and Hui Xiong. Improving gloss free sign language translation by reducing representation density, arXiv preprint 21X1v22405.14312 2024.
[70] Jinhui Ye, Jiaming Zhou, Hui Xiong, and Junwei Liang. Geodeformer: Geometric deformable transformer for action recognition. arXiv preprint urXiv:2311.179111 2023.
[71] Bojian Yin, Federico Corradi. and Sander M Bohté. Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. Nature Machine Intelligence. 5(10):905-11) 2021.
[72] Yi Zeng, Tielin Zhang, and Bo Xu. Improving multi-layer spiking neural networks by incorpo- rating brain-inspired rules. Sci. China Inf. Sci, 60(5): 2017.
[73] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 30(6):1514-154) 2018.
[74] Anguo Zhang, Yuzhen Niu, Yueming Gao, Junyi Wu, and Zhipeng Gao. Second-order infor- mation bottleneck based spiking neural networks for semg recognition. Information Sciences, 585:543-555 558, 2022.
[75] He Zhang, Ying Sun, Weiyu Guo, Yafei Liu, Haonan Lu, Xiaodong Lin, and Hui Xiong. Inter- active interior design recommendation via coarse-to-fine multimodal reinforcement learning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 6472-6480, 2023.
[76] Qi Zhang, Tong Xu, Hengshu Zhu, Lifu Zhang, and Qi Liu. Aftershock detection with multi- scale description based neural network. In 2019 IEEE International Conference on Data Mining (ICDM), 2019.
[77] Qi Zhang. Hengshu Zhu, Qi Liu, Enhong Chen, and Hui Xiong. Exploiting real-time search en- gine queries for earthquake detection: A summary of results. ACM Transactions on Information Systems (TOIS), 2021.
[78] Wenli Zhang. Tingsong Zhao, Jianyi Zhang, and Yufei Wang. Lst-emg-net: Long short-term transformer feature fusion network for semg gesture recognition. Frontiers Neurorobotics, 17:1 127338, 2023.
15

---

[79] Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li. Siyuan Li, Liang Lin, and Guanbin Li. Divide and contrast: Source-free domain adaptation via adaptive contrastive learning. Advances in Neural Information Processing Systems, 35:5137-5149, 2022.
[80] Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 11106-11115, 2021.
[81] Shibo Zhou, Xiaohua Li, Ying Chen, Sanjeev T Chandrasekaran, and Arindam Sanyal. Temporal- coded deep spiking neural network with easy training and robust performance. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11143-11151, 2021.
[82] Yaoyu Zhu, Zhaofei Yu, Wei Fang, Xiaodong Xie, Tiejun Huang, and Timothée Masquelier. Training spiking neural networks with event-driven backpropagation. Advances in Neural Information Processing Systems, 35:30528-300 10041, 2022.
[83] Romain Zimmer, Thomas Pellegrini, Srisht Fateh Singh, and Timothée Masquelier. Technical report: supervised training of convolutional spiking neural networks with pytorch. arXiv preprint arXiv; 1911 2019.
16

---

A Appendix supplemental material
A.1
Details of Data Collection
Forearm lying
REST
Forearm raising diagonally with elbom surface(P2)
REST
Forear
REST
on horizontal surface (P1)
rasing
horizontally(P3)
0
5
5
6
2
7
2
7
8
3
8
8
4
4
4
9
Figure 6: Overview of our dataset: the compilation contains sEMG data for ten distinct actions, each across three postures. Varied background colors represent distinct forearm postures, while the digits ranging from 0 9 correspond to specific gesture actions. The 'Rest' label the top denotes a static hand gesture when no action is being performed.
In dumann-coppin interaction studies focusing on surface electromyographh (sEMG), the acquisition of diverse and representative data sets is crucial. Current research predominantly collects sEMG
data from gestures made with a single forearm posture 4 31
46
43
32
33
29
However, is
evident that variations in forearm posture can significantly influence the distribution of the sEMG data, potentially causing discrepancies between laboratory results and real-world applications. To address this, our data collection methodology incorporates gestures performed in different forearm postures, aiming to reflect the conditions and variability encountered in practical scenarios more accurately.
The experiments were carried out using the DataLITE wireless LE230 and DataLITE PIONEER, commercial sEMG acquisition systems from Biometrics Ltd The device's sampling rate is 2000Hz, allowing for high-resolution data capture of the electrical activities in the muscles during the per- formance of gestures. Eight LE230 sEMG sensors were uniformly and equidistantly affixed to the surface of the participant's 's right forearm.
17

---

RVA Peikee
Teorr
Figure 7: Summary our collected data: The three postures on the left illustrate distinct preparatory arm actions. The central data graph represents the sEMG data captured from subject under the three postures, with unique colors assigned to different channels. The data graph on the right showcases the acquired data after Root Mean Square (RMS) processing.
A total of fifteen subjects participated in the dataset, comprising ten males and five females. A significant proportion our participant pool (twelve individuals) was right-handed, while three individuals were left-handed. None of the participants had any neurological ensuring the generalizability of our results to healthy population. This study has been approved by the relevant university ethics committee, and it's worth noting that none of the participants had prior experience using sEMG collection devices.
muscle disorders,
Before the experiment, each participant was thoroughly briefed about the procedures. They willingly participated in the study, aware that their data would be open-source. During the experiment, the subjects were instructed to replicate the gestures and forearm postures displayed screen using their right or left hand. This methodology allowed us to systematically record sEMG signals from the participants across range of forearm positions and gestures.
Most existing datasets for surface electronyography (sEMG) gesture recognition predominantly use single forearm posture, which does not align with real-world scenarios, where forcarm posture varies dynamically and significantly influences the muscle state. As shown in Figure [ndd Figure 9 changes in forearm posture lead to alterations in the distribution of sEMG signals, resulting in substantial decline in the prediction accuracy of gesture estimation models. Acknowledging this limitation in prevailing datasets, our study aims bridge this gap. In study, we incorporated three distinct forearm postures for our data collection, as depicted in Figure 6 In the study, three distinet forearm postures were identified: P1, where the forearm was placed horizontally surface; P2, where the forearm was elevated diagonally with the elbow anchored on a surface; and crucial
flat
P3, where the forearm maintained in horizontal orientation. These configurations for understanding ubiquitous computing interactions related to forearm ergonomics. Our gesture included ten difey-life actions: Thumb up. Extension of the index and middle fingers, Flexion of the others; Flexion of the ring and little finger, Extension of the others; Thumb opposing base of little finger; Abduction of all fingers; Fingers flexed together in fist; Pointing index; Adduction of extended fingers; Wrist flexion and Wrist extension. During the experiment, participants were instructed to maintain one of the specified gestures for five seconds, followed by five-second relaxation period. This process was repeated six times for each gesture under each forearm posture. This methodology, combining varying forearm postures and gestures, aims provide robust and diverse sEMG data. In the future, we will collect more distribution shift scenarios, like the Electrode movement.
A.2 Details of Data Preprocessing
In the data preprocessing phase, utilized the Root Mean Square (RMS) as the initial feature extraction method to improve the stability gesture recognition. RMS serves as an advantageous choice for feature extraction due to its ability to efficiently summarize the magnitude of the signal
18

---

variation, providing a stable indication of the signal power. To capture the transient characteristics of the sEMG signals in our study, we used a time window length of 100ms with a step size of 0.5ms for the RMS calculation. This approach allowed us to extract representative RMS features from the signal while maintaining high-resolution view of the signal variations. RMS can be mathematically represented as follows:
1 N
RMS (X)
(11)
where z; represents each value in the signal E X, and N is the total number of values or samples in the signal.
A.3 Spiking Neural Networks
Spiking Neural Networks (SNNs) represent the third generation of neural networks [4] 21 20 and advantageous in several key aspects when compared to traditional Deep Learning (DL) models.
For instance, they inherently handle temporal information [99 66 8T]. efficiently process event-basee data [82), and offer lower energy consumption for certain tasks (2 due their sparse [45] [0] and asynchronous nature [6]]. Emulating the precise mechanism of neuronal spike transmission in the human brain, SNNs stand at the frontier of biologically inspired artificial intelligence (72). offering potential advancements in neuromorphic computing [18] and beyond. SNN processes information and generates 'spikes' 'impulses' only when certain dynamic threshold of neuron activation is reached, leading to a more efficient representation and transmission of information. Fundamentally, sEMG signals are generated by neural impulses, making them naturally compatible with the processing mechanism of SNNs. The spikes in an SNN represent discrete events in time, which parallels the nature of sEMG signals containing valuable information in spatial (across different muscles) and temporal (over time) dimensions. This innate alignment between SNNs and sEMG data enables the efficient decoding of intricate patterns for gesture recognition. Furthermore, SNNs have lower power consumption compared to traditional DL models [18]. This feature aligns with the typical use case of sEMG in wearable technology, where power efficiency is a crucial consideration [38] Traditional DL models require substantial computational resources for training and inference, but SNNs operate an event-driven manner, only processing data when a spike occurs (55). This unique attribute makes SNNs feasible solution for real-time, low-power wearable applications (5 providing an effective solution for practical constraints in human- machane interaction systems.
While SNNs show promise for sEMG-based gesture recognition, they face challenges, such as lower accuracy [58] due to sparser feature representation and difficulties in convergence because of the lack of natural gradient [15). To mitigate the issue of sparse features, we propose integrating an attention mechanism into our SNN model, allowing it to focus on more relevant features within the sEMG data. This combined approach aims to enhance the accuracy and training effectiveness of SNNs for gesture recognition tasks.
A.4 Conv-based Leaky Integrate-and-Fing Neurons
To transform the input into a spike train and learn the spike representation of sEMG samples, we use a smaller quantity of basv- Leaky ontegrate-and-Fei
(LIF) neurons. As shown Figure Bal Conv-based LIF neurons can process spatial and temporal features concurrently. Since sEMG signals consist of pulses from multiple motor neurons at the skin's surface combined with noise, using LIF neurons is biologically plausible for decoding these signals into spikes. Conv-LIF neurons also constrain the decoding range within local time, mitigating the impact of noise on global decoding. This method aligns with the biological characteristics of sEMG signals and effectively manages noise interference.
Initially convolution is used to extract spatial features St. from the present spatial input X These extracted features are then combined with the temporal features 1,m of the previous moment to form the current membrane potential M', Following this, the Fire and Leak module, based on the current membrane potential, generates the temporal features T for the subsequent moment and the spatial features St.++ for the next layer of the network. A significant event, known spike, occurs the membrane potential M exceeds a threshold value, denoted as VnA- In such an event, M is reset to value Vieser and g the post-synaptic potential, is calculated the product of 5t.++1 and
19

---

Conv
52h
gta
Mca
7
Via
v2g
0m
Frence
/errordco fa g''nyrr
(ecoadd (er00/ssssss00000 1
Verrard (g
Attr
Attn
LIF
O7. ((OD)
Attx OIJE N(O.1)
AuLa 01,7 N(0.5
Stae1
VR
(a) Conv-based LIF Layer
(b) Spiking Jaccard Attention Layer
Figure 8: Illustrations of different layers in the network.
V On the contrary, if the membrane potential M is lesser than the threshold Van spike does not occur. In this case, T is determined by M and is computed as the product of the time constant e -dt/ and M
Given that sEMG is composed of the summation of pulses produced by multiple motor neurons at the skin's surface, combined with series noise, is biologically plausible use Leaky Integrate-and- Fire (LIF) neurons, similar to motor neurons, to decode sEMG into spikes. Concurrently, Conv-LIF can constrain the decoding range within local time, avoiding the impact of noise on global decoding. This approach aligns with the biological characteristics of the signals and provides robust method for managing noise interference.
A.5 Complexity Analysis
Next, we will analyze and compare the complexity of traditional attention and SJA. The traditional attention mechanism consists of several operations that contribute its computational complexity. Firstly, the dot product QKT is computed, which has a complexity of O(n² d) where n is the sequence length and is the dimensionality of queries, keys, and values. The next operation scaling the dot product by 1/ l/d,,, which requires O(n") operations. Following this, the softmax function is applied. Computationally, this operation also requires O(n²) operations because, for each element, we must sum over all other elements to normalize them. Finally, we multiply the resulting matrix with overall time the value matrix V. This operation has complexity of (nn² d) as well. Therefore, multiplication complexity of the attention mechanism is Onn² d) due to the complexity of the matrix steps. Moreover, the space complexity is (n²) to store the attention weights for each token pair in the sequence, which can be particularly costly for long sequences. This quadratic dependency on the sequence length n is one of the main computational challenges attention mechanisms.
On the other hand, the computational complexity of SJA depends on the number of non-zero elements in the vectors, owing to the sparse nature of SNN outputs. This sparsity allows us focus our computation only on the non-zero elements, thus reducing the complexity. Specifically, for both intersection and union calculations, we perform minimum and maximum operations. respectively, between each pair of corresponding elements in vectors x and y. Given the sparsity, the complexity for both these operations is O(b), where stands for the number of non-zero elements. Additionally, the summation operation in the numerator and the denominator of the Jaccard similarity formula also has a complexity of (b)) because we are adding up the number of non-zeru non-zero elements. Hence, the overall time complexity of the Spiking Jaccard Attention mechanism is (b)). This is significantly more efficient than the traditional attention mechanism, especially in the context of SNNs, where the output is predominantly sparse.
This reduced complexity, while maintaining effective attention functionality, highlights the advan- tage of our proposed SJA mechanism for SNNs, enhancing their computational efficiency without compromising on performance. Additionally, the energy consumption of the traditional attention dominated by multiplication could be several times higher than that of the SJA dominated by addition.
20

---

For example, the energy cost of a multiplication (3.7 p.) is 4.1x to an addition (0.9 p.)), in 45mm CMOS technology (27]
A.6 Details of LIF-based Classifier
The Leaky Integrate-and- Fire (LIF) model 83 provides a biologically plausible and computationally efficient approximation of neuronal spike generation. We employ modified Leaky Integrate-and-Finn (LIF) layer as the classifier and add a memory module to record the membrane potentials, which receive spikes from the preceding network and translate them into membrane potentials. The number of LIF neurons corresponds to the task's categories, and the neuron with the highest membrane potential determines classification. This approach provides an efficient method for translating spiking activity into classification results.
Mathematically, the LIF model is represented by the following differential equation:
dV = V(t) + RT(t).
t dt
(12)
Here, (tt) characteristically denotes the membrane potential of a neuron, R represents the inherent membrane resistance, T(t) the incoming current or the external input signal, and the vital time constant of the neuron, which is fundamentally the product of the membrane resistance and its capacitance.
When the incoming signals (I()) cause the membrane potential (V(t)) to exceed a predefined threshold (0), the neuron 'fires' spike, and then its membrane potential reset resting potential. The 'leaky' aspect comes from the - (t) term in the equation, which models the neuron' 's natural decay towards the resting potential in the absence of input, effectively avoiding an unbounded increase of membrane potential. Striking balance between computational simplicity and biological characteristics, the LIF model computationally less demanding, making it suitable for wearable, real-time applications such as sEMG-based gesture recognition. Moreover, its inherent ability to handle time-series data is critical in capturing the temporal dynamics of sEMG signals.
The following equations provide a straightforward iterative formulation for the LIF-SNN layer, facilitating easier inference and training processes:
M = T + X",
St = Hea (M thh).
(13)
T = St + (e-# M¹) + (1 St).
In these equations, R represents the membrane potential at the t-th time step. X' represents the spatial feature and - represents the temporal feature. @ stands for element- multiplication. When M' reaches or exceeds the threshold un, is reset to Vreset and a spike (S* = 1) is emitted. Otherwise, M evolves according the given dynamics, and no spike is emitted (S'n) 0).
ddtt/ 1 stands for the decay factor.
A.7 Details of the Training Method
Deep Spiking Neural Networks (SNNs) are typically trained using two methods: ANN-to-SNI conversion and direct training. ANN-to-SNN conversion approximates ANN activation values with SNN firing rates. However, it involves a trade-off between accuracy and latency, requiring sufficient time steps for accurate rate-coding. Despite its application in aarge-scale structures like VGG and ResNet, it faces challenges in latency, restricting its practicality. Direct training of SNNs, on the other hand, applies continuous relaxation of non-smooth spiking for backpropagation. It outperforms ANN-to-SNN conversion in time step efficiency and is suitable for temporal tasks. Though it can employ various coding schemes, we choose rate coding in this paper for its ability to support complex SNNs. The SuperSpike 773 surrogate gradient can described as:
o' (U.) (1 + IU, =1)-
(14)
The formula is designed in such way that the contribution to the gradient becomes significant when the neuron's 's membrane potential (U+) is close to the firing threshold (v). In the case when U2 is much
21

---

higher than v, implying the neuron is certain to fire, the contribution of the neuron to the gradient is lessened. This because the neuron's firing state is unlikely be affected by small changes. Hence, it is not crucial for the current learning step. Conversely, the gradient contribution remains low if U; much lower than e, indicating the neuron is less likely to fire. This is because it would take substantial adjustment to this neuron's is activity to make it fire, suggesting it currently has a minimal impact on the network's overall output.
Hence, the SuperSpike algorithm optimizes the learning process by focusing on the neurons that are firing status, i.e., when Ug close u. This ensures the resources are directed towards neurons that can be effectively adjusted by the current learning step, improving the overall learning efficiency.
on the verge of changing their
A.8
Implementation Details
Our model was developed using Py and Norse 44 Norse a library that extends Py Torch to support the development of Spiking Neural Networks (SNNs). We trained our model on a server with two AMD EPYC 7543 32-core Processors, one NVIDIA RTX 3090 GPU, and ITB RAM. To ensure model convergence,
iterated over 200 epochs. Model convergence was determined by criteria wherein the accuracy improvement was less than 0.2% within ten epochs, the model was deemed have reached a convergent state.
The model was trained using the Adam optimizer, a popular choice for deep learning tasks due to its ability handle sparse gradients large-scale datasets and adapt the learning rate based the computation of adaptive learning rates for different parameters. The learning rate was initially set to 0.001, and the batch size set to 32. However, to guarantee fair comparison across all tested models, we train these models using the same parameters as the original papers.
In terms of data preparation and division, we maintained the same training-test set split across all datasets. Specifically, 70% of the data was used for training, and the remaining 30% was used for testing. We ensured no intersection between the training and testing sets prevent data leakage.
The sEMG data was segmented using a window length of 100ms. To increase the number of training instances and capture the transitional states between actions, we applied a 50% overlap between adjacent samples.
Finally, to ensure the fairness of the comparison, all the models were trained based on the Root Mean Square (RMS) features. This was performed regardless of the feature extraction methods initially proposed in their papers. This approach allows a more accurate comparison of the performance of the different models.
A.9 Source-free Domain Adaptation Out-of-distributtin Data
Accuracy of Model Predictions
Accurscy
12
Subjects
Figure 9: Demonstration of our dataset's inherent out-of-distribution (OOD) nature: We used data from Posture for inference data from Postures 2, and 3 and subsequently calculated the accuracy. This highlights the OOD characteristics of data with different pre-existing postures.
Figures Toa 10b illustrate the performance variations across different subjects when model trained on Posture is deployed on Posture 2 and 3, respectively, after using our SSFDA. It is noticeable that our method consistently yields improved performance instead deterioration. In the majority of the subjects, the accuracy improvement of our SSFDA is better than the other two
22

---

Poste::
Pakkoe PIP
Mothou
Dhe
3xardd Aecrd
8
10
12
15
14 menn
10
11
13
mon
Saly act
Satyoc
(a)
(b)
Figure 10: Comparison of performance before and after applying SSFDA for various methodologies: Figures 10a and [0b] present bar charts depicting the difference performance before and after SSFDA for different subjects using three methods.
strategies. Additionally, our average accuracy improvement is significantly higher than that of PL and NL. This indicates that our proposed Probabilistic Label Generation method is more conducive learning distribution with generalization capabilities than PL and NL.
A.10 Real World Deployment
SpGesture has been deployed a real-world application using an in-house developed sEMG acqui- sition system, as illustrated in Figure IT] The system comprises analog front-end sensing circuits (AFE), a microcontroller (MCU), and wireless module. The system is equipped with eight AFE channels that can be attached to the surface of the forearm. Dry electrodes and instrumentation amplifiers are utilized to sense and amplify the sEMG signals, respectively. A 32-bit MCU with ARM M4 is employed to control the acquisition system, and the eight-channel amplified sEMG signals are sampled by a 12-bit analog-to-digite converter (ADC) inside the MCU at a sampling rate of 2000Hz. The converted digital signals wirelessly transmitted low-power Bluetooth (BLE) 5.2 module. The acquisition system is powered by a Li-ion battery.
computer host through
We have tested our JASNN and SSFDA algorithms this hardware device, and the results are encouraging. Even under the constraint of utilizing only the CPU, our inference latency was found to be less than 100ms. This swift response time falls within the real-time requirements for most application scenarios. Such low latency, coupled with the carefully designed acquisition system, demonstrates that approach is not only theoretically sound but also practically viable for real-world deployment various sEMG-based applications.
Dry clectrodes
Figure 11: In-house developed sEMG acquisition system: from left to right are the acquisition system, dry electrodes, AFE, MCU with BLE 5.2 module, respectively.
A.11 Instructions to participants of research with human subjects
In our study, participants will engage in an interactive hand movement activity, which is detailed as follows:
Participants will watch series of short videos laptop screen, each showing specific hand movement.
Viewing the Movements:
After viewing each video, participants will mimic the hand movement shown using their right hand. This includes various gestures and movements as outlined in hand movement taxonomies and robotics literature.
Repeating the Movements:
Equipment Setup: Participants will wear special glove (dataglove) and an accelerometer attached to their wrist. These devices will record the kinematic information of the hand movements.
23

---

Muscle Activity Recording: We will attach 8 to 12 wireless electrodes to each participant's 's forearm to measure muscle activity. These electrodes will be placed specific locations around the forearm and on key muscle areas, like the biceps and triceps, following an anatomically informed strategy for precise data collection.
The electrodes will be secured using standard adhesive bands and hypoallergenic, latex-free elastic band to ensure they stay place throughout the activity.
Ensuring Comfort and Stability:
Performing the Task: Participants will be asked to repeat each demonstrated movement for seconds, followed by 3-second rest, This sequence will repeated times for each of the 49 different movements.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims about the proposed SpGesture framework for accurate and efficient sEMG-based gesture recognition, including the JASNN model and SSFDA method, are clearly stated and match the paper's 's contributions and scope described in the abstract and introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper includes dedicated "Limitation and Future Work" section that discusses limitations, such as the need to extend SSFDA to handle other causes of distribution shift beyond forearm posture, evaluating the methods on a wider variety of SNN architectures, and conducting performance analysis neuromorphic chips.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattio, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
25

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results or proofs, as it focuses on proposing novel computational framework and empirically evaluating its performance.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed information about the dataset, model architectures, training procedures, and hyperparameters in the Method and Experiment sections, well as the appendix, to enable reproducibility of the main experimental results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: This paper releases the link for the dataset and codes for all experiments for reproductivity in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
public/ pegllic/guide//// CwweSubmissiooooooo.....o
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https wttp:/iccggiiee//eeeeeeeeeeseeet htte:/gguuudee/oeCSubbbiiiii www.oodeuuiiiisonnn....
/ //nips.
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
The paper specifies the key experimental details, such data splits, hyper- parameters, and optimizer, in the Method and Experiment sections. Additional details are provided in the appendix A.8
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports accuracy numbers along with standard deviations across multiple subjects in Table T to quantify the statistical significance and variability of the experimental results.
27

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative етгог rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper provided sufficient information on the computer resources, including CPU, GPU, and RAM in appendix A.8
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: To the best of our knowledge, the research conducted conforms with the NeurIPS Code of Ethics. The data collection involved informed consent from human subjects and was approved by the university ethics committee.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
28

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper discusses potential positive impacts of the proposed EEMG-based
gesture recognition framework, such as enabling natural human-machine interaction and assisting individuals with motor impairments. Potential negative impacts, like privacy risks from sEMG data or system failures leading to incorrect actions, are also briefly mentioned.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release pretrained models or large datasets that pose high risks for misuse. The proposed computational techniques do not require special safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
29

---

Justification: The paper uses new sEMG dataset collected by the authors and does not rely on existing external datasets or code assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pataaaesss.ppesssai..s paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces a new sEMG gesture dataset. Key properties of this dataset, like the number of subjects, gestures, and forearm postures, are documented in the Data Collection section of the appendix. The appendix also mentions that informed consent obtained from participants.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We have given the instructions given participants in research with human subjects in appendix A.II
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
30

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [Yes]
The data collection study was approved by the relevant university ethics committee, as stated in the Data Collection section of the appendix. The paper does not describe the specific risks to participants but mentions that they provided informed consent.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "naRSAYvtUW", "reviewer": "Reviewer_K5du", "summary": "The paper proposes a novel attention mechanism that utilizes the Jaccard similarity to replace the traditional dot-product approach. This allows Spiking Neural Networks (SNNs) to maintain their binary characteristics (0 and 1) during the forward pass，which is very important for hardware computation. Additionally, the paper introduces a source-free domain adaptation method for SNNs, leveraging a probabilistic pseudo-labeling technique. The approach is validated on several sEMG datasets, demonstrating its innovation and potential for broader application.", "questions": "1. How is the probability  P  determined when generating pseudo-labels?\n2. How would Jaccard attention be calculated for 3D data?\n3. Can you provide evidence or a demonstration of the domain shift between different postures?", "limitations": "1. The method has only been validated on sEMG time-series data and has not yet been extended to other modalities.\n2. There is no analysis of the spatial complexity provided in the paper.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Innovative Jaccard Attention: The introduction of Jaccard attention presents a computationally friendly algorithm for SNNs, which shows great potential for wide application in the field.\n2. First Source-Free Domain Adaptation in SNNs: The paper proposes the first source-free domain adaptation method in the SNN domain, which enhances the domain generalization performance of SNNs.\n3. Comprehensive Validation: The paper includes data collection and validation across multiple datasets, with robust results indicating reliability.\n4. Open-source Code: Making the code publicly available enhances reproducibility and assists other researchers in replicating and building upon the work.", "weaknesses": "1. Selection of Probability P for Pseudo-Labeling: The paper does not clearly explain how the probability  P  is chosen when generating pseudo-labels.\n2. Jaccard Attention for 3D Data: There is a lack of discussion on how Jaccard attention would be calculated for 3D data.\n3. Domain Shift Between Different Postures: The method lacks a clear demonstration or proof of domain shift between different postures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***To reviewer:*** Thank you for your insightful review of our paper. We are thrilled that you found the introduction of Jaccard Attention to be an innovative and computationally friendly algorithm for SNNs. Your recognition of our pioneering work on source-free domain adaptation in SNNs and its enhancement of domain generalization performance is highly appreciated. We are also glad that you valued our comprehensive validation across multiple datasets and the open-sourcing of our code for reproducibility. Your constructive feedback and suggestions will significantly guide our future research. We sincerely appreciate your valuable time and effort in reviewing our work.\n\n***About how the probability $P$ is determined when generating pseudo-labels.*** Thank you for your question. The probability $P$  is determined by selecting the mode (most frequent value) among the top k membrane potentials as the high-probability pseudo-label. The probability $1-P$  involves randomly selecting a pseudo-label from the top k membrane potentials. The value of $P$ ranges between $0$ and $1$. In our validation, we searched the space from $0.1$ to $0.9$ in increments of $0.1$ to determine the optimal value.\n\n***About extending Jaccard Attention to 3D data.*** Thank you for your insightful question regarding extending Jaccard Attention to 3D data. To address this, we have extended the original Jaccard Attention formula to handle 3-dimensional data. The original equation is:\n\n\\begin{equation}\n\\mathrm{SJA}\\left(\\mathbf{Q}, \\mathbf{K}\\right) = \\frac{\\sum_{ij}\\min\\left(q_{ij}, k_{ij}\\right)}{\\sum_{ij}\\max\\left(q_{ij}, k_{ij}\\right) + \\epsilon} \\ \\mathbf{V}\n\\end{equation}\n\nFor 3D data, we extend the indices $(i,j)$ to $(i,j,k)$ to account for the additional dimension. The extended equation is:\n\n\\begin{equation}\n\\mathrm{SJA}\\left(\\mathbf{Q}, \\mathbf{K}\\right) = \\frac{\\sum_{ijk}\\min\\left(q_{ijk}, k_{ijk}\\right)}{\\sum_{ijk}\\max\\left(q_{ijk}, k_{ijk}\\right) + \\epsilon} \\ \\mathbf{V}\n\\end{equation}\n\nHere, $q_{ijk}$ and $k_{ijk}$ represent the elements of the 3D tensors $\\mathbf{Q}$ and $\\mathbf{K}$ respectively. The summations now run over all three dimensions $i$, $j$, and $k$. This extension allows us to apply the Jaccard Attention mechanism to 3-dimensional data, maintaining its computational efficiency and enhancing feature representation in SNNs.\n\n***About demonstration of the domain shift between different postures.*** The characteristics of the sEMG distribution shifts corresponding to different forearm postures are demonstrated in Appendix Figure 9. We used data from Posture 1 for inference on data from Postures 1, 2, and 3 and subsequently calculated the accuracy. This highlights the dataset's out-of-distribution (OOD) nature, showing the signal differences caused by different postures. Figure 9 shows that the accuracy of the same gestures significantly decreases in Posture 2 and Posture 3, further demonstrating the OOD problem.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ASaymtLjth", "reviewer": "Reviewer_7xEq", "summary": "This paper proposes SpGesture, a surface electromyography (sEMG) based gesture recognition framework using Spiking Neural Networks (SNNs). The main contributions include: 1) A novel Jaccard Attention SNN (JASNN) model that enhances sparse spike sequence representations by directly applying Jaccard similarity computation in SNNs. This is the first time that attention mechanisms do not alter the high energy efficiency of SNNs, ensuring that SNNs only involve 0 and 1 computations ; 2) The first introduction of Source-Free Domain Adaptation (SSFDA) in SNNs, using membrane potential as a memory list to generate pseudo-labels, improving model generalization in unlabeled environments; 3) A collected sEMG gesture dataset with different forearm postures. Experimental results show that SpGesture achieves the highest recognition accuracy of 89.26% on this dataset, with inference latency below 100ms on CPU, meeting real-time requirements", "questions": "1. Does the design of JASNN consider other similarity measures? Have attempts been made to apply Jaccard attention to other types of SNNs?\n2. What are the characteristics of the sEMG distribution shifts corresponding to different forearm postures observed in the dataset? Is it possible to visualize the signal differences caused by different postures?", "limitations": "The paper comprehensively analyzes the limitations of the work in the Limitation section: 1) Extending SSFDA to more distribution shift scenarios; 2) Evaluating the applicability of the methods on more SNN structures; 3) Evaluating performance on neuromorphic chips. Future research directions are provided.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. Proposes Jaccard attention designed specifically for SNNs, enhancing feature representation while maintaining SNN computational efficiency. Ablation experiments validate its effectiveness.\n2. Innovatively introduces source-free domain adaptation to SNNs, leveraging the membrane potential of SNNs as a memory feature and designing a label generation method. Improves generalization performance even when target domain data is unlabeled.\n3. Systematically compares the performance of SpGesture with various state-of-the-art methods in terms of accuracy, inference speed, and memory consumption. The experimental design is reasonable and the results are credible.\n4. Collects and open-sources a multi-posture sEMG dataset, providing a new benchmark for research. The data collection process is described in detail.", "weaknesses": "1. The SSFDA method currently only addresses distribution shifts caused by forearm posture variations. Its applicability to other potential factors such as electrode displacement needs further validation. The future work section could discuss how to extend the method.\n2. Performance evaluation on real hardware such as neuromorphic chips remains to be supplemented. The authors plan to conduct tests on self-developed chips in the future.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***To reviewer:*** Thank you for your thorough and insightful review of our paper. We are delighted that you found our contributions noteworthy. We appreciate your recognition of the Jaccard Attention SNN model, which **enhances feature representation while maintaining computational efficiency and is validated through ablation experiments. We are also grateful for your acknowledgment of our innovative introduction of source-free domain adaptation in SNNs, leveraging membrane potential as a memory feature to improve generalization performance.** Your positive remarks on our systematic comparison of SpGesture’s performance and the collection and open-sourcing of the multi-posture sEMG dataset further encourage us. Thank you for your valuable feedback and suggestions, which will guide our future research. Below, we will specifically address your questions.\n\n***About the design of JASNN, its consideration of other similarity measures, and the scalability of Jaccard attention.*** Thank you for this insightful question. Indeed, we did consider other similarity measures during the development of JASNN. We experimented with cosine similarity and Pearson correlation coefficient but found that Jaccard similarity performed best for binary sparse data, which aligns well with the spiking nature of SNNs.  Regarding the application to other SNN types, we have successfully applied Jaccard attention to other SNN architectures, including LSNN (Long Short-Term Memory Spiking Neural Networks). We observed consistent performance improvements across different SNN types, indicating the generalizability of our approach. In the revised version, we will include these additional results to demonstrate the broader applicability of our method.\n\n***About the characteristics of the sEMG distribution shifts corresponding to different forearm postures.*** The characteristics of the sEMG distribution shifts corresponding to different forearm postures are demonstrated in Appendix Figure 9. We used data from Posture 1 for inference on data from Postures 1, 2, and 3 and subsequently calculated the accuracy. This highlights the dataset's out-of-distribution (OOD) nature, showing the signal differences caused by different postures. From Figure 9, it can be seen that the accuracy of the same gestures significantly decreases in Posture 2 and Posture 3, further demonstrating the OOD problem.\n\n***About future suggestions, the extension of SSFDA to other distribution shift scenarios.*** We appreciate this excellent suggestion for our future work. You are correct that our current SSFDA method primarily focuses on distribution shifts caused by forearm posture variations. We acknowledge that this is an important area for expansion. In the revised version, we will extend our discussion in the future work section to address this point. We plan to outline our strategy for adapting SSFDA to handle other sources of distribution shift, such as electrode displacement, changes in skin conditions, and variations in muscle fatigue. This expansion will involve modifying our membrane potential memory mechanism and pseudo-label generation process to account for these additional factors.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lInp9n6AcX", "reviewer": "Reviewer_4Y6C", "summary": "This work introduces an innovative framework for sEMG-based gesture recognition. It leverages Spiking Neural Networks (SNNs) and introduces a Jaccard Attention mechanism and Source-Free Domain Adaptation (SSFDA) to enhance model robustness and accuracy in real-world applications.\n\nThe framework achieves high accuracy (89.26%) on a newly collected sEMG gesture dataset with different forearm postures and maintains system latency below 100ms on a CPU, meeting real-time requirements.\n\nThe novel Jaccard Attention mechanism directly computes attention on spike sequences, preserving SNNs' low-power characteristics, while SSFDA enhances model adaptation without needing source data.\n\nThis framework significantly improves the performance and efficiency of sEMG-based gesture recognition systems, demonstrating practical applicability and offering a robust solution for human-computer interaction.", "questions": "I don't understand how your proposed Jaccard attention reflects 'attention' within tokens. According to the formula, tokens at the same position in Q and K are computed to get a scalar representing 'attention', which is then used to multiply the token in V at the corresponding position. I don't understand the meaning of this procedure or how it works. I admit that this kind of attention calculation is very efficient, and it would be better if the article had an intuitive explanation of why the algorithm is effective.", "limitations": "see Weaknesses and Questions.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* This work proposed a Jaccard-based attention mechanism and  an SNN-oriented SFDA algorithm for SNNs, which achieves high accuracy (89.26%) on a newly collected sEMG gesture dataset.\n* The author focuses on the actual application experience of the algorithm in the real world and makes deployments, which is very valuable.", "weaknesses": "* There is a typo in the text. In the footnote to Figure 5, the explanations for the first and second columns are reversed.\n* In section 5.5, you intend to compare the benefits of using SJA, but the algorithm time and memory usage are for the entire algorithm. I think the overall time and memory usage of the entire algorithm is very important, but if you can get the time of each part of the algorithm and only compare the benefits of the attention part, this will be more in line with what section 5.5 is about.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***To reviewer*** Thank you for your thorough review and insightful feedback on our work. We are delighted that you recognize the strengths of our proposed Jaccard-based attention mechanism, our SNN-oriented SFDA algorithm, and the high accuracy (89.26%) achieved on our newly collected sEMG gesture dataset. We are particularly pleased that you appreciate our focus on real-world application experiences and deployments, which we believe are crucial for advancing human-computer interaction.\nWe value your suggestions for improvement and will consider them carefully to enhance the soundness and presentation of our work. Your feedback is instrumental in guiding our future research efforts.\n\n***About Jaccard attention reflects 'attention' within tokens and intuitive explanation.*** Thank you for your insightful question. I appreciate the opportunity to provide a more detailed explanation of how our proposed Jaccard-based attention mechanism (SJA) reflects \"attention\" within tokens.\n\nIn traditional attention mechanisms, the relationships between the query (Q), key (K), and value (V) matrices are established through dot product operations. These dot products are then normalized and used to weigh the values in the value matrix (V). **The essence of this process is to compute the similarity between each query vector and all key vectors**, using these similarities to allocate attention weights and ultimately obtain a weighted sum of the values.\n\nOur SJA mechanism replaces the dot product similarity with the Jaccard similarity, which is more suited for the binary nature of Spiking Neural Network (SNN) outputs. This is particularly useful as SNN data is often sparse and binary.\n\nThe SJA mechanism is defined by the following formula:\n\\begin{equation}\n\\mathrm{SJA}\\left(\\mathbf{Q}, \\mathbf{K}\\right) = \\frac{\\sum_{ij}\\min\\left(q_{ij}, k_{ij}\\right)}{\\sum_{ij}\\max\\left(q_{ij}, k_{ij}\\right) + \\epsilon} \\ \\mathbf{V},\n\\end{equation} are the corresponding elements in the query and key matrices.\n\n***Intuitive Explanation of SJA's Effectiveness:*** Imagine each token's spike train as a binary pattern representing different features or characteristics. The Jaccard similarity helps identify how much overlap (or commonality) there is between the features of the two tokens. By focusing on the overlapping features, the model can better understand the importance of these features and adjust the values (from \\( V \\)) accordingly, enhancing the overall representation and prediction accuracy. In detail,  \n\n1. **Utilizing Sparsity**: SNN outputs are typically sparse, and SJA leverages this sparsity by focusing on non-zero elements. Traditional dot product methods can be inefficient for sparse data as they involve numerous zero-value multiplications. SJA, however, uses element-wise minimum and maximum operations, concentrating on the non-zero elements, thus reducing computation time and energy consumption. \n2. **Element-wise Similarity Measurement**: The Jaccard similarity measures the intersection ratio to the union of two sets. For binary vectors, this effectively captures the proportion of shared active elements. In our formula, $ \\min(q_{ij}, k_{ij}) $ and $\\max(q_{ij}, k_{ij}) $ represent the intersection and union of corresponding elements, respectively. This method directly reflects the similarity between the query and key vectors at the same positions, providing a meaningful measure of attention for sparse binary data. \n3. **Improved Computational Efficiency**: Using element-wise minimum and maximum operations instead of matrix dot products, SJA significantly reduces computational complexity. Traditional attention mechanisms have a complexity of $O(n^2 \\cdot d) $, whereas SJA has a complexity of $ O(b) $, where $ b $ is the number of non-zero elements. This reduction in complexity is particularly beneficial for handling large-scale sparse data, making SJA more efficient and suitable for deployment on spiking chips.\n\nIn summary, the SJA mechanism captures attention by measuring the Jaccard similarity between the query and key vectors, focusing on the significant non-zero elements, and leveraging the sparsity of SNN outputs. This approach not only reduces computational complexity but also enhances energy efficiency, making it an effective and efficient method for attention in SNNs.I hope this explanation clarifies how SJA works and why it is effective. Thank you for your attention and for providing the opportunity to elaborate on our work.\n\n***About algorithm time and memory usage for the SJA.*** Thank you for your valuable feedback. We appreciate your suggestion to clarify the comparison of benefits in Section 5.5. We apologize for any confusion caused by our description. In Section 5.5, specifically in Figure 5, we indeed present the comparison of inference speed and RAM usage between SJA, Efficient Attention, and RAW Attention. The data shown pertains solely to the attention part of the algorithm, not the entire algorithm. Thank you for highlighting this area for improvement. We will ensure that this is more clearly communicated in the revised version of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Area Chair,

We would like to express our gratitude to you for your dedicated efforts and contributions and to the reviewers for their constructive feedback on our submission. We are encouraged by the positive evaluation from all reviewers. All three reviewers acknowledged the innovative aspects of our work, particularly the introduction of the Jaccard Attention mechanism for Spiking Neural Networks (SNNs) and the novel Source-Free Domain Adaptation (SSFDA) method. The reviewers highlighted our approach's practical applicability and robustness, as evidenced by its validation on multiple sEMG datasets and its potential for real-world applications.

### Key Strengths Recognized:
1. ***Innovative Jaccard Attention***:
• Reviewers appreciated the computational efficiency and the preservation of the binary characteristics (0 and 1) in SNNs, which is crucial for hardware computation.
2. ***Source-Free Domain Adaptation***:
• The introduction of SSFDA in the SNN domain was noted as a significant contribution, enhancing the domain generalization performance without needing source data.
3. ***Comprehensive Validation***:
• Our method’s validation across multiple datasets and the robust results were commended, indicating the reliability and potential of our approach.
4. ***Open-Source Contribution***:
• The availability of our code for public use was highlighted as a positive step towards reproducibility and further research in this area.

### Areas for Improvement and Future Work:
1. ***Jaccard Attention for 3D Data***:
• We plan to extend our discussion on how Jaccard attention can be calculated and applied to 3D data in future iterations of our work.
2. ***Performance on Neuromorphic Chips***:
• We acknowledge the need for performance evaluation on real hardware, such as neuromorphic chips, and plan to conduct these tests in future research.
3. ***Additional Similarity Measures***:
• We will explore other similarity measures and their applicability to different types of SNNs, as suggested by the reviewers.
4. ***Visualization of sEMG Distribution Shifts***:
• We have included relevant visualizations of the signal differences caused by different postures in the appendix to understand the sEMG distribution shifts better.

We are pleased that the reviewers found our work to be technically solid and of significant impact. We are committed to addressing the identified weaknesses and incorporating the suggested improvements in our future research. We believe these enhancements will further strengthen our contributions and the overall quality of our work.

Thank you for considering our submission. We look forward to your decision and are excited about the potential impact of our research in the field of SNNs and gesture recognition.

Meta-review: The authors present an approach for gesture recognition, where the input is data provided by surface electromyography (sEMG). Their so-called SpGesture algorithm is based on spiking neural networks (SNNs). The authors show that SNNs can be used in this application to create a robust system capable of handling the typical degradation in performance caused by distribution shifts. They achieve this by introducing a source-free domain adaptation method for SNNs, leveraging a probabilistic pseudo-labeling technique.

The reviewers were all in agreement that the paper should be accepted. Among the paper's strengths they listed the introduction of a Jaccard-based attention mechanism for SNNs, the domain adaptation technique, good evaluation, and the introduction of an sEMG dataset.

Amongst potential drawbacks the reviewers indicate the lack of testing the system on a neuromorphic chip, and the fact that the distribution shift algorithm only considers shifts caused by forearm posture variations.

The authors should make sure that in their final version they include the extra visualizations of signal differences, to mention the weaknesses identified by the reviewers (not tested on neuromorphic chips, exploring other similarity measures) and indicate that these weaknesses will constitute topics for future research.

In summary, the reviewers have reached a consensus towards accepting the paper. After my own reading of the manuscript, I agree with this assessment and I am happy to recommend acceptance.
Decision: Accept (poster)