Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in Surface Electromyographic Signal Analysis

## Abstract

Abstract Gesture recognition based on surface electromyography (sEMG) has been gaining importance in many 3D Interactive Scenes. However, sEMG is easily influenced by various forms of noise in real-world environments, leading to challenges in providing long-term stable interactions through sEMG. Existing methods often struggle to enhance model noise resilience through various predefined data augmentation techniques.
In this work, we revisit the problem from a short term enhancement perspective to improve precision and robustness against various common noisy scenarios with learnable denoise using sEMG intrinsic pattern information and sliding-window attention. We propose a Short Term Enhancement Module(STEM) which can be easily integrated with various models. STEM offers several benefits: 1) Learnable denoise, enabling noise reduction without manual data augmentation; 2) Scalability, adaptable to various models; and 3) Cost-effectiveness, achieving short-term enhancement through minimal weight-sharing in an efficient attention mechanism.
In particular, we incorporate STEM into a transformer, creating the Short Term Enhanced Transformer (STET).
Compared with best-competing approaches, the impact of noise on STET is reduced by more than 20%. We also report promising results on both classification and regression datasets and demonstrate that STEM generalizes across different gesture recognition tasks.

## 1 Introduction

Surface Electromyographic (sEMG) is a non-invasive technique for motoring muscle neurons firing, which is an effective way to capture human motion intention and has shown great application potential in the field of human-computer interaction (HCI) Xiong et al. (); Liu et al. (, ). A schematic diagram of the EMG-based HCI System is shown in Figure 1. Compared to traditional HCI channels, sEMG has the advantages of being generated prior to actual motion (50-150 ms), containing rich motion intention information, and being easy to collect Sun et al. (). Therefore, there has been increasing interest in exploring EMG-based motion track Liu et al. () and pathological analysis.

Figure: Figure 1: Schematic Diagram of EMG-based Human-Computer Interaction System.

By treating sEMG as time series, deep sequential models Bi et al. (); Tsinganos et al. (); Becker et al. (); Li et al. (); Du et al. () have been applied to sEMG modeling. For example, Zhang et al. employs a multi-task encoder-decoder framework improve the robustness of sEMG-based Sign Language Translation (SLT) Zhang et al. (). Rahimian et al. employs Vision Transformer (ViT)-based architecture (TEMGNet) to improve the accuracy of sEMG-based myocontrol of prosthetics Rahimian et al. (). Although these methods demonstrate enhanced performance compared to traditional approaches, they process sEMG signals as generic time-series data, without specifically tailoring their design to address the unique characteristics of sEMG, such as their high variability and sensitive to external noise and interference. This oversight leads to issues such as difficulty in handling low signal-to-noise ratios due to changes in skin surface conditions and signal interference, and failing to capture subtle but important motion information in sEMG signals. As a result, the robustness and accuracy of existing models remain significantly challenged.

Processing sEMG signals is challenging due to the complex noise mixed in the skin’s surface and the presence of patterns across various time scales. Existing works, mainly focusing on long-term sequences, have used transformers to treat sEMG as a typical time series, aiming to enhance long-term dependencies. These approaches overlook the critical features which present in short-time scales. Short-time scale features are important in sEMG analysis, as they aid in distinguishing subtle movements and facilitate the removal of variable noise. For example, gestures like Index Finger Extension (IFE) and Middle Extension (ME), while similar in global sEMG patterns, can be differentiated through localized short-term signal variations.

To this end, in this paper, we present a lightweight but powerful Module called Short-Term Enhanced Module (STEM) which utilizes sliding window attention with weight sharing to capture short-term features. Building on STEM, we further propose the Short-Term Enhanced Transformer (STET). STET leverages STEM to capture local signal changes, enhancing noise resistance, and then combines STEM with long-term features, further improving accuracy for downstream predictions. Furthermore, to enhance model robustness with minimal annotation, we propose a self-supervised paradigm based on sEMG Signal Masking to leverage the inherent variability in sEMG signals.

Finally, we conducted extensive experiments on the largest public sEMG datasets. The experimental results show that STET surpasses existing methods by a significant margin in both gesture classification and joint angle regression tasks for single-finger, multi-finger, wrist, and rest gestures. Meanwhile, STET achieves strong robustness even when trained on pure data and tested on noisy data. Compared with best-competing approaches, the impact of noise on STET is reduced by more than $20\%$. Moreover, through visualizations, we show that the long-term and short-term features are complementary in sEMG-based gesture recognition tasks, and the fusion of the two features can make the classification boundary more obvious. This clearly demonstrates that short-term information is critical for sEMG-based gesture recognition and will provide a new design paradigm for future sEMG model design. In particular, we have deployed STET as an important functional component in our HCI system, which can offer a more intuitive and effective experience. Our real-world deployment is shown in the appendix.

To the best of our knowledge, we are the first to highlight the short-term features in sEMG-based gesture recognition. Our contributions can be summarized as follows:

- 1)
From the perspective of enhancing short-term features, we propose STEM, a learnable, scalable, and low-cost noise-resistant module. The integration of STEM into various neural networks has resulted in a marked improvement in their performance;
- 2)
we introduce sEMG Signal Masking to self-supervised sEMG Intrinsic Pattern Capture Module to leverage the inherent variability in sEMG;
- 3)
we conduct experiments on the largest wrist sEMG dataset, showing that our proposed method outperforms existing approaches in terms of accuracy and robustness. And short-term enhancement can be extended to other models like Informer.

## 2 RELATED WORK

The EMG-based Intention Prediction of Human Motion can be broadly divided into model-based and data-driven methods. Model-based methods typically combine disciplines such as kinesiology, biomechanics, and human dynamics to explicitly model the relationship between EMG and outputs (such as joint angles and forces). The model often includes specific parameters, such as joint positions and bone-on-bone friction, that need to be repeatedly experimented with and adjusted until the desired performance is achieved. In terms of parameter selection and determination, model-based methods can be further divided into kinematic models Borbély and Szolgay (), dynamic models Koike and Kawato (); Koirala et al. (); Liu et al. (), and muscle-bone models Wang and Buchanan (); Zhao et al. (); Yao et al. (). Clancy et al. used a nonlinear dynamics model to identify the relationship between constant posture electromyography and torque at the elbow joint Clancy et al. (). Hashemi et al. used the Parallel Cascade Identification method to establish a mapping between forearm muscles and wrist forces Hashemi et al. (). However, model-based methods have a large number of parameters that are difficult to measure directly. Currently, only simple motion estimation with a limited number of joints and degrees of freedom is possible. In contrast to model-based approaches, data-driven methods do not require the measurement of various parameters. Recently, some researchers have begun to use temporal deep learning models to extract motion information from sEMG Lin et al. (); Zhang et al. (); Guo et al. (). Lin et al. proposed a method based on the BERT structure to predict hand movement from the Root Mean Square (RMS) feature of the sEMG signal Lin et al. (). Rahimian et al. proposed a novel Vision Transformer (ViT)-based neural network architecture to classify and recognize upper-limb hand gestures from sEMG for use in myocontrol of prostheses Rahimian et al. (). However, these methods have neglected the modeling of short-term dependencies and have not considered the inherent variability in sEMG signals.

## 3 Preliminaries

### 3.1 Dataset

We conduct the experiments on Gesture Recognition and Biometrics ElectroMyogram (GRABMyo) Dataset Pradhan et al. (), which is the largest known open-source wrist EMG dataset with 43 subjects and has great potential for developing new generation human-machine interaction based on sEMG.

*Data processing.* The subjects performed 17 gestures of hand and wrist (including a rest period sEMG) according to the prompts on the computer screen. Each gesture was repeated 7 times, each lasting 5 seconds. In order to improve the convergence speed of the model, we use two methods (Max-Min normalization, $\mu$-law normalization) to normalize the data Rahimian et al. (); Recommendation ().
After normalization, we use a time-sliding window to split samples. In this paper, we set the window size as 200ms, and the overlap of adjacent windows is 10ms. $\mu$-law normalization can logarithmically amplify the outputs of sensors with small magnitudes, which results in better performance than linear normalization.

###### Definition 1 (sEMG Signal Sequence) .

An sEMG signal sequence is defined as a temporal signal sequence sampled by multiple sensors from a human wrist, which can be formulated as $\mathbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},..,\mathbf{x}_{t}]$, where $t$ is the time window. $\mathbf{x}_{i}=[x_{i,1},x_{i,2},...x_{i,c}]$ represents the signal vector of $c$ sensors, where $x_{i,j}$ is the signal value of the $j$-th sensor in the $i$-th time step.

###### Definition 1 (sEMG Signal Sequence) .

## 4 Technology Detail

### 4.1 Model Overview

Figure 2 illustrates the overview of our proposed framework for gesture recognition, which contains three components: (1) The sEMG Intrinsic Pattern Capture module encodes the sEMG signal sequence into the hidden sEMG representations. A pre-training model with a segment masking strategy and MSE reconstructing loss is proposed to learn inherent variability from the sEMG signals into the model’s parameters.
(2) The Long-term and Short-term Enhanced module uses two decoupling heads to extract the long-term and short-term context information separately, which improves the sEMG representations in preserving both the global sEMG structure and multiple local signal changes of the sEMG.
(3) The Asymmetric Optimization strategy addresses the problems of sample biases and imbalance in gesture recognition via an asymmetric classification loss, which can make the model focus on hard and positive samples to improve the recognition.

Figure: Figure 2: Overview of STET. The sEMG signal is encoded using the sEMG Intrinsic Pattern Capture module, which is first pre-trained via sEMG signal Masking. A long-term and short-term enhanced module improves sEMG representations. An asymmetric optimization strategy addresses biases and imbalances in gesture recognition through an asymmetric classification loss.

### 4.2 sEMG Intrinsic Pattern Capture Module

#### 4.2.1 sEMG Signal Encoding

Given the sEMG signal sequence $\mathbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},..,\mathbf{x}_{t}]$, we first project each signal $\mathbf{x}_{i}\in\mathbb{R}^{c}$ into a hidden embedding via a transformation matrix and add each signal embedding with an absolute position embedding.
Then, we feed the output sequence into a $L$-layer Transformer and obtain the output signal embeddings $\mathbf{X}^{(L)}=[\mathbf{x}^{(L)}_{1},\mathbf{x}^{(L)}_{2},..,\mathbf{x}^{(L)
}_{t}]$, which incorporate temporal context signal information for each position in the sequence.

#### 4.2.2 sEMG Signal Masking

After the sEMG signal-extracting module is constructed,
we aim to use pre-training to exploit the intrinsic pattern and temporal semantics disclosed by the unlabeled sEMG signals (labeling sEMG is time-consuming and labor-intensive) and give a good initialization for the model parameters, then avoid the model focusing on some noisy features in the supervised learning task so as to over-fitting on some local minimums. Thus, we propose a sEMG Intrinsic Pattern Capture based on a signal masking strategy.

Specifically, given a transformed signal embedding sequence $\mathbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},..,\mathbf{x}_{t}]$, instead of adding masks on the sequence in terms of time steps like BERT, we add sensor-wise masks for the signal sequence of each sensor similar with Zerveas et al. (), which can encourage the model to learn more fine-grained temporal context dependency on the signal sequence of multiple electrodes.
For the signal sequence of the $i$-th sensor, formulated as $[x_{1,i},x_{2,i},...,x_{t,i}]$, i.e, the $i$-th column of $\mathbf{X}$, we generate a binary mask vector $\mathbf{m}_{i}\in\mathbb{R}^{t}$, where average $r$ radio of elements in $\mathbf{m}_{i}$ should be 0.15.
Randomly generating $\mathbf{m}_{i}$ may cause a lot of isolated-masked signals, meaning one masked signal whose adjacent signals are unmasked. However, a single signal can be easily predicted by its immediately preceding or succeeding signals, making self-supervised learning easy to fitting on ineffective patterns and poor for learning temporal semantic information.
In consideration of this, we introduce a more complex masking strategy that aims to generate multiple masked segments on the sequence with an average length $l_{m}$, which means $m_{i}$ is composed of contiguous masked segments and unmasked segments. The length of masked segments follows a geometric distribution with mean $l_{m}$, and the length of unmasked segments follows a geometric distribution with mean $l_{u}$. Also, the $\frac{l_{m}}{l_{u}}=\frac{r}{1-r}$ so that the number of masked elements would follow the proportion $r$.
The pseudocode of the masking algorithm is presented in Algorithm 1.

Figure: Algorithm 1 The Algorithm of sEMG Signal Masking

Then, we can mask the input sEMG signal sequence $\mathbf{X}$ by $\widehat{\mathbf{X}}=\mathbf{X}\odot\mathbf{M}$, where $\odot$ is elementwise multiplication and $\widehat{\mathbf{X}}$ is the masked input.
With the proposed Transformer-based sEMG signal encoder, we can obtain the output $\widehat{\mathbf{X}}^{(L)}=[\widehat{\mathbf{x}}_{1}^{(L)},\widehat{\mathbf{x}
}_{2}^{(L)},...,\widehat{\mathbf{x}}_{t}^{(L)}]$.
For self-supervised learning, we add a linear layer on the top of masked output to reconstruct each sEMG signal $\widehat{\mathbf{x}}_{i}^{(L)}$ as $\widetilde{\mathbf{x}}_{i}\in\mathbb{R}^{c}$,
which is the reconstructed sEMG signal in the $i$-th time step generated from the masked input.
Then, we minimize the Mean Squared Error (MSE) of the reconstructed signals and original signals on the masked positions for each sample:

$$ $\min\frac{1}{|\mathbf{M}|}\sum_{i=0}^{t}\sum_{j=0}^{c}\mathds{1}(\mathbf{M}_{i ,j}=0)(\widetilde{x}_{i,j}-x_{i,j})^{2},$ (1) $$

where $\mathds{1}(\cdot)$ is the indicator function, $\widetilde{x}_{i,j}$ and $x_{i,j}$ are the reconstructed value and original value of $j$-th sensor in $\widetilde{\mathbf{x}}_{i}$ and $\mathbf{x}_{i}$ respectively, and $\mathbf{M}_{i,j}$ is the element in the $i$-th row and the $j$-th column of $\mathbf{M}$.
Thus, we can pre-train the sEMG Intrinsic Pattern Capture via the above strategy to obtain well-initialized model parameters for the downstream task.
In practice, we empirically set the masking proportion $r$ as $0.15$ and the average length of masked segments as $3$.
The illustration of the pre-training procedure is in Figure 2 (a).

### 4.3 Long-term and Short-term Decoding

Then, based on the pre-trained sEMG Intrinsic Pattern Capture, we develop two decoder heads to further extract the long-term and short-term dependency on the signal sequences, respectively.
Intuitively, both the long-term and short-term information on signal sequences are significant in the gesture recognition problem. Long-term information refers to the global context of an sEMG sequence, which provides the overall structure of a signal to help the interpretation of the gesture. Short-term information refers to the movement signal in a short time interval of the whole sequence, which can provide specific local characteristics for accurate recognition when the overall structures of sEMGs are ambiguous. For example, distinguishing between Index Finger Extension (IFE) and Middle Extension (ME) movements requires a closer examination of the local signal changes in sEMG, whereas differentiating gestures with large variations, such as hand gestures and wrist gestures, necessitates a focus on the global sEMG information.

#### 4.3.1 Preserving Long-term sEMG Signal

Given the hidden output $\mathbf{X}^{(L)}=[\mathbf{x}_{1}^{(L)},\mathbf{x}_{2}^{(L)},...,\mathbf{x}_{t}
^{(L)}]$ of a sEMG signal sequence, we first build a long-term decoder to extract the long-term dependency on the complete output. Specifically, the long-term decoder is defined as a multi-head self-attention layer:

$$ $MultiHead_L\left(\mathbf{X}^{(L)}\right)=Concat\left(h_{1},\ldots,h_{d}\right )\mathbf{W}^{O},$ (2) $$

$$ $\left\{h_{i}\right\}_{i=0}^{d}=\left\{Attention\left(\mathbf{Q}_{i},\mathbf{K} _{i},\mathbf{V}_{i}\right)\right\}_{i=0}^{d},$ (3) $$

$$ $Attention\left(\mathbf{Q}_{i},\mathbf{K}_{i},\mathbf{V}_{i}\right)=Softmax \left(\frac{\mathbf{Q}_{i}\mathbf{K}_{i}^{T}}{\sqrt{h}}\right)\mathbf{V}_{i},$ (4) $$

$$ $where\ \mathbf{Q}_{i}=\mathbf{X}^{(L)}\mathbf{W}_{i}^{Q},\ \mathbf{K}_{i}= \mathbf{X}^{(L)}\mathbf{W}_{i}^{K},\ \mathbf{V}_{i}=\mathbf{X}^{(L)}\mathbf{W} _{i}^{V},$ (5) $$

where $\{\mathbf{W}_{i}^{Q},\mathbf{W}_{i}^{K},\mathbf{W}_{i}^{V}\}_{i=0}^{d}\in
\mathbb{R}^{h\times h}$ are parameter matrices and $d$ is the number of attention heads. $Concat(\cdot)$ represents the concatenate operation. $\mathbf{W}^{O}\in\mathbb{R}^{dh\times h}$ is the output parameter matrix to transform the concatenated outputs of $d$ attention heads.
Then, the long-term sEMG embeddings $\mathbf{H}^{l}\in\mathbb{R}^{t\times h}$ is obtained by
$\mathbf{H}^{l}=MultiHead_L(\mathbf{X}^{(L)})$.
Through the self-attention layer, the global context signal information is collected to the embeddings of $t$ timesteps with different attention weights.

#### 4.3.2 Preserving Short-term sEMG Signal

To model the local context information within a short time interval, we introduce a slide-window self-attention layer to extract the short-term dependency on the signal outputs. Similarly, we stack multiple attention heads and calculate the attention of context signals to weighted sum them up into the final representations. The difference is that, for each time step, we only calculate the attention of its nearest $w$ context. Specifically,
we can rewrite the $\mathrm{Attention}(\cdot)$ in Eq.4 as:

$$ $\small Attention_S(\mathbf{Q},\mathbf{K},\mathbf{V})=\left[Softmax\left(\frac {\mathbf{Q}_{i}{\mathbf{K}_{i}^{w}}^{T}}{\sqrt{h}}\right){\mathbf{V}_{i}^{w}} \right]_{i=1}^{t},$ (6) $$

$$ $\small\left[\mathbf{K}_{i}^{w}\right]_{i=1}^{t}=Unfold(\mathbf{K},w),\ \left[ \mathbf{V}_{i}^{w}\right]_{i=1}^{t}=Unfold(\mathbf{V},w),$ (7) $$

where $w$ is the sliding windows size, $\mathbf{Q}_{i}\in\mathbb{R}^{h}$ as the $i$-th query is the $i$-th row of $\mathbf{Q}$, $\mathbf{K}_{i}^{w}\in\mathbb{R}^{w\times h}$ and $\mathbf{V}_{i}^{w}\in\mathbb{R}^{w\times h}$ are the keys and values in a window around the $i$-th query.
For the key and value matrix, we utilize the $Unfold(\cdot)$ operation to generate the slide windows for each timestep. Noted that to avoid confusion, we omit the index of attention head in the above equation.

Thus, by stacking multiple sets of parameters in $Attention_S(\cdot)$ to constitute different attention heads, we can obtain the short-term sEMG embeddings $\mathbf{H}^{s}\in\mathbb{R}^{t\times h}$ by
$\mathbf{H}^{s}=MultiHead_S(\mathbf{X}^{(L)})$.
Using slide windows, each row in $\mathbf{H}^{s}$ preserves the local context signal information of the corresponding timestep, representing the movement from the past $w/2$ timesteps to the next $w/2$ timesteps.

Unlike LST-EMG-Net Zhang et al. () and Focal Transformer Yang et al. (), which process Long Term Feature and Short Term Feature sequentially, we handle them in parallel. This ensures that features processed earlier in a sequential manner are not overlooked.

#### 4.3.3 Fusion

Obtained the long-term embeddings $\mathbf{H}^{l}$ and the short-term embeddings $\mathbf{H}^{s}$ of an sEMG signal sequence, we first concatenate them in terms of the hidden dimension, then introduce a 1-D convolution to summarize the $t$-step sEMG embedding sequence into the final sEMG representation, which is fed into a Feed Forward Layer with a Sigmoid Layer to obtain the final classification probability of which gesture the sEMG belonging to, which can be written as:
$\widehat{\mathbf{y}}=\sigma(FC(\mathbf{u}^{T}\cdot[\mathbf{H}^{l}:\mathbf{H}^{
s}])),$
where $FC(\cdot)$ is a two-layer fully connected
layer, $\sigma(\cdot)$ is the activation function,
and $\widehat{\mathbf{y}}\in\mathbb{R}^{C}$ is the output classification probability of the sEMG signals.

### 4.4 Asymmetric Optimization

As the common use in multi-label classification, we reduce the gesture recognition problem into a series of binary classification tasks.
However, we consider two tricky problems that exist in the above model optimization.
(1) As the sampled signals are usually unstable over time, making the samples of the sEMG signal sequence may be critically biased. Some samples with strong signals are easily predicted, while many samples with fuzzy signals are hard to predict.
(2) As we set 17 classes for the sEMG signals, and each class contains a comparable number of samples. Thus, each class contains, on average, many more negative samples than positive ones. This imbalance may make the model eliminate the gradients from the positive samples in the optimization process, resulting in poor accuracy.
Realizing this, we introduce the Asymmetric loss Ridnik et al. () for the gesture classification task. Asymmetric loss is a variant of Focal loss.
(1) It uses focusing parameters to reduce the contribution of easily predicted samples and make the model optimization focus on hard samples;
(2) It further introduces asymmetric focusing parameters and asymmetric probability shifting to down-weight the contribution from massive easy negatives and emphasizes the contribution of positive samples.
Thus, we define the loss function as follows:

$$ $\displaystyle\mathcal{L}_{STET{}}=-\sum_{i=1}^{N}\sum_{j=1}^{C}$ $\displaystyle\left({y}_{i,j}\left(1-\widehat{y}_{i,j}\right)^{\gamma^{+}}\log \left(\widehat{y}_{i,j}\right)+\right.$ (8) $\displaystyle\left.\left(1-{y}_{i,j}\right)\left(\widehat{y}^{m}_{i,j}\right)^ {\gamma^{-}}\log\left(1-\widehat{y}^{m}_{i,j}\right)\right),$ $$

$$ $\small\widehat{y}^{m}_{i,j}=max\left(\widehat{y}_{i,j}-m,0\right),$ (9) $$

where ${y}_{i,j}$ and $\widehat{y}_{i,j}$ is the ground-truth and probability of the $i$-th sEMG signal sequence belonging to the $j$-th gesture. $\left(1-\widehat{y}_{i,j}\right)^{\gamma^{+}}$ and $\left(\widehat{y}^{m}_{i,j}\right)^{\gamma^{-}}$ are two terms to make the weights of hard predicted samples bigger than those easily predicted samples, $\gamma^{+}$, and $\gamma^{-}$ are two focusing parameters and $\gamma^{+}>\gamma^{-}$ lead to asymmetric focusing that help the optimization pay more focus on positive samples of each class. $\widehat{y}^{m}_{i,j}$ is the shifted probability and $m$ is shifting margin. The probability shifting for negative samples encourages the optimizer to further reduces their contribution.

## 5 Experiment

### 5.1 Settings

##### Implementation Details

STET is implemented in PyTorch Paszke et al. () and is trained using one RTX 3090 GPU.
During training, we use the RAdam Liu et al. (), which is a theoretically sound variant of the Adam optimizer with a weight decay of 1e-3. We pre-train on GRABMyo for 20 epochs using a fixed learning rate of 1e-4 for the backbone. In the decoder, we use two layers of full attention in the long-term decoder and two layers of sliding window attention in the short-term decoder. The short-term decoder’s window size is 41 and 21, and the window’s move step is set to 1. In both the pre-training and fine-tuning periods, we set the batch size to 16. To avoid overfitting, we set drop out to 0.2.

**Table 1: The gesture classification performance on the Single-finger, Multi-finger, Wrist, Rest, and Overall categories.**
| Model | Single-finger | Multi-finger | Wrist | Rest | Overall |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ACC | STD | ACC | STD | ACC | STD | ACC | STD | ACC | STD |  |
| Asif et al. Asif et al. () | 83.44% | 0.015 | 83.58% | 0.013 | 89.40% | 0.009 | 90.86% | 0.012 | 85.34% | 0.014 |
| TCN Tsinganos et al. () | 78.78% | 0.017 | 79.10% | 0.018 | 87.27% | 0.011 | 88.57% | 0.017 | 81.50% | 0.016 |
| GRU Chen et al. () | 84.45% | 0.015 | 84.88% | 0.013 | 90.06% | 0.009 | 89.42% | 0.019 | 86.30% | 0.015 |
| TEMGNet Rahimian et al. () | 77.70% | 0.019 | 74.00% | 0.029 | 84.04% | 0.017 | 87.46% | 0.014 | 78.02% | 0.019 |
| Zerveas et al. Zerveas et al. () | 78.45% | 0.016 | 77.20% | 0.020 | 87.28% | 0.016 | 86.76% | 0.017 | 80.43% | 0.018 |
| Informer Zhou et al. () | 86.88% | 0.016 | 86.54% | 0.017 | 91.90% | 0.011 | 83.56% | 0.024 | 87.71% | 0.016 |
| LST-EMG-Net Zhang et al. () | 87.21% | 0.011 | 83.16% | 0.012 | 88.36% | 0.018 | 82.52% | 0.021 | 85.31% | 0.015 |
| TEMGNET+STEM(ours) | 84.57% | 0.017 | 81.23% | 0.022 | 88.12% | 0.017 | 88.74% | 0.013 | 84.07% | 0.017 |
| Informer+STEM(ours) | 87.42% | 0.015 | 88.39% | 0.018 | 92.07% | 0.013 | 90.33% | 0.011 | 89.14% | 0.015 |
| STET | 88.27% | 0.014 | 89.93% | 0.015 | 93.77% | 0.010 | 95.33% | 0.012 | 90.76% | 0.012 |

#### 5.1.1 Evaluation Metrics

Following the prior worksGuo et al. (); Wang et al. (); Chen et al. (); Rahimian et al. (), we choose the below metrics to evaluate the model’s performance. Pearson Correlation Coefficient (CC) is a widely used measure of the linear relationship between two variables. It ranges from -1 to 1, where a larger CC value indicates greater similarity between the predicted and estimated joint angles curve, indicating improved estimation.
Root Mean Square Error (RMSE) is a common metric for evaluating the deviation between predicted and observed values. As the range of fluctuations in the curves of different joint angles can vary significantly, it is difficult to evaluate the performance of models using RMSE alone fairly. Normalization of RMSE addresses this issue, resulting in the Normalized RMSE (NRMSE).
Average curvature ($\kappa$) of all points for each joint is used to measure the smoothness of an estimated curve. A smaller $\kappa$ indicates a smoother curve.

### 5.2 Comparison with Baselines

We compare the accuracy (ACC) and Standard deviation (STD) between our proposed STET and previous popular sEMG-based gesture recognition methods.
Specifically, we train the model on the GRABMyo dataset Pradhan et al. () (the detail of data processing shown in section 3.1 ) and separately report the classification results on the categories of Single-finger gestures, Multi-finger gestures, Wrist gestures, Rest, and the overall results.
The ratio of the training set to the test set for each gesture is 5 to 2.

From the experimental results, we can observe that STET consistently performs best on four categories of gestures and overall data.
In particular, STET and Zerveas et al. () both use Transformer-based encoders.
While in the decoder part, STET introduces both the short-term and long-term decoder rather than the fully connected layers used in Zerveas et al. ().
As a result, the overall accuracy of STET is improved from 80.43% to 90.76% compared to Zerveas et al. (). This is because the proposed long-term and short-term decoupling module can extract both the global and fine-grained dependency on the signal dependency and thus can learn better sEMG representations.

Among the transformer-based methods, Informer and STET performed best, with accuracy rates of 87.71% and 90.76%, respectively. Informer relies heavily on max pooling layers to aggregate features, leading to the relative weakness in extracting some short-term features. STET enhances accuracy and stability by strengthening the short-term feature extraction. The improvement is remarkable on the Rest gestures, where the accuracy improves from 83.56% to 95.33%. Furthermore, after incorporating our designed short-term encoder into Informer, its accuracy rate increased from 87.71% to 89.14%, and the classification accuracy for Rest gestures improved from 83.56% to 90.33%.

Note that the Rest category of gesture, as the resting state of devices such as interactive bracelets, is the most frequent gesture that appears in the signals.
Thus, the stability of its prediction plays a significant role in the problem.
STET achieves the highest and most stable results on the prediction of the Rest category compared with all baselines, indicating the robustness of STET.

### 5.3 Ablation Studies

To validate the effects of the unsupervised sEMG Intrinsic Pattern Capture (EIPC), Long-term decoder, Short-term decoder, Fuse strategy, and loss function.
We designed variants of STET and reported their results in Table 2.

First, we can observe that, with the unsupervised EIPC, the accuracy of the transformer and STET is improved by 0.60% and 1.12% compared with training from scratch. This suggests that unsupervised EIPC can aid in discerning additional data features, such as the inherent variability in sEMG, without the requirement for new samples or extra annotations. Significantly, this process circumvents the need for external data, thus preserving user privacy in the context of data acquisition and processing.
Replacing the fully connected layer of the transformer’s decoder with the long-term decoder or short-term decoder,
the performance is improved by 1.16% and 1.39%, respectively.
Furthermore, the performance is comparable when using the long-term decoder or short-term decoder alone, indicating that the two kinds of features may play different significant roles in the sEMG signal recognition, and the short-term cannot be ignored.
Most importantly, when we used our designed fuse module to combine long-term and short-term features, the accuracy further improved by 2.46%. This suggests that the two decoders are complementary and that enhancing short-term features is necessary on the basis of the long-term decoder. We employed Asymmetric Loss (ASL) to drive the model’s focus on difficult samples. Compared to simply using the Cross-Entropy Loss (CEL), the accuracy improved by 0.73%, indicating the effectiveness of ASL.

**Table 2: The results of ablation study. EIPC: sEMG Intrinsic Pattern Capture; LT: Long-Term decoder; ST: Short-Term decoder; CEL: Cross Entropy Loss; ASL: ASymmetric Loss.**
| Transformer | EIPC | LT | ST | Fusing | CEL | ASL | ACC |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ✓ |  |  |  |  | ✓ |  | 85.73% |
| ✓ | ✓ |  |  |  | ✓ |  | 86.33% |
| ✓ | ✓ | ✓ |  |  | ✓ |  | 88.02% |
| ✓ | ✓ |  | ✓ |  | ✓ |  | 87.72% |
| ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |  | 89.37% |
| ✓ |  | ✓ | ✓ | ✓ |  | ✓ | 89.42% |
| ✓ | ✓ | ✓ | ✓ | ✓ |  | ✓ | 90.54% |

Figure: Figure 3: Accuracy versus Noise Intensity Curve.

**Table 3: Drop rates of accuracy calculated by $drop_rate=\frac{ACC_{raw}-ACC_{noise}}{ACC_{raw}}$. AG: Additive Gaussian noise, MG: Multiplicative Gaussian noise**
| Backbone | In STET framework | AG noise | MG noise | Signal loss |
| --- | --- | --- | --- | --- |
| Transformer | No | 25% | 16% | 14% |
| Transformer | Yes | 10% | 10% | 8% |
| Informer | No | 11% | 9% | 26% |
| Informer | Yes | 9% | 8% | 17% |

### 5.4 Robustness Analysis

To verify the robustness of the model, we only used high-quality data collected in the lab to train the model and added different types of noise (Additive Gaussian noise, Multiplicative Gaussian noise, and signal loss) during validation to simulate complex scenarios that might be encountered in real situations.

Additive noise typically refers to thermal noise, which is added to the original signal. This type of noise exists regardless of the presence of the original signal and is often considered the background noise of the system in sEMG acquisition. Multiplicative noise is generally caused by channel instability, and it has a multiplicative relationship with the original signal. Also, we simulated signal loss during transmission by randomly setting a portion of the signals to zero.

Figure 3 illustrates the influence exerted by three distinctive noise categories, namely additive noise, multiplicative noise, and signal loss, on the accuracy of the proposed model. The model using only the short-term decoder is less affected by noise compared to the long-term version. This relative robustness of the short-term decoder is potentially attributable to its unique capability to mitigate the global impact of noise by virtue of a sliding window multiple-sampling scheme, which effectively confines the sphere of noise impact. The model that integrates both long-term and short-term characteristics persistently outperforms models that rely on only one. This highlights the significant effectiveness of the integrating process in dealing with noise-induced interference. As depicted in Table 3, it is evident that both Transformer and Informer models demonstrate a notable enhancement in noise resistance when their decoders are replaced with the design from STET.

Figure: Figure 4: Visualization of (a) the long-term sEMG embeddings, (b) the short-term sEMG embeddings, and (c) the fused sEMG embeddings for gesture recognition. Note that we color each sample by its classes.

### 5.5 Visualizations

To demonstrate the distinction, we first obtained STET’s long-term, short-term, and fuse embeddings. The embeddings with dimensions $(N,T,H)$ were then flattened to $(N,T*H)$ and separately projected in 2D by t-SNE, the result shown in Figure 4. We colored each node by category for the illustration. As shown in Figure 4, The classification boundary generated by the long-term feature and the short-term feature is a significant difference, indicating that the long-term and short-term features are capable of recognizing different types of gestures. This further suggests that the two features are complementary in data representation. For example, short-term embedding can distinguish TA gesture and TIFO gesture very well, but TA gesture and TIFO gesture will be confused in long-term embedding. Meanwhile, long-term embedding can distinguish LP gesture and TIFE gesture very well, but short-term embedding will confuse them. As shown in Figure 4(c), after the fusion of the two types of features, the classification interface is wider, and the confusion points are significantly reduced, which indicates that the fusion module can effectively complement the strengths of the two types of features.

**Table 4: Comparison of STET with Other models in Predicting Joint Angle for Fingers.**
| Model | PCC | NRMSE | $\kappa$ | Time Cost/epoch(s) |
| --- | --- | --- | --- | --- |
| LSTM | 0.779 | 0.096 | 0.581 | 26.36 |
| TCN | 0.833 | 0.088 | 1.533 | 3.62 |
| BERT | 0.867 | 0.077 | 1.571 | 4.95 |
| sBERT-OHME | 0.869 | 0.076 | 0.532 | 4.96 |
| STET | 0.877 | 0.073 | 0.522 | 6.83 |

### 5.6 Regression: Hand Joint Angles Prediction

STET can conveniently handle regression tasks by changing the loss function to mean squared error (MSE) loss. Continuous motion estimation extracts continuous motion information, such as joint angles and torques, from sEMG signals. Since continuous motion estimation requires outputting subtle variations of the movement at each time instant, the local signal variations are particularly important for this type of estimation. In this section, we have re-selected the most competitive models known for sEMG-based joint angle prediction as the baseline and tested the performance of STET on the regression task of predicting the main 10 joint angles for fingers using the Ninapro DB2 Atzori et al. () dataset. As shown in Table 4, STET achieved the best performance in PCC, NRMSE, and $\kappa$, indicating that the joint angle curve predicted by STET is more in line with the real curve and has less abnormal fluctuations, which will significantly improve the user’s interactive experience.
In terms of training time, due to the addition of the short-term decoder, its training speed is slightly slower than BERT but still within an acceptable range.

## 6 Conclusion

Current sEMG-based gesture recognition models usually fail to handle various noisy and distinguish similar gestures, especially in non-laboratory settings. In this paper, we found using short-term information and self-supervised EIPC mitigates this issue. Therefore, we proposed STEM for capturing local signal changes and enhancing noise resistance. The STEM is easily deployable and serves as a plug-in that can potentially be applied to most time series deep learning models. According to our experimental results, our method significantly improved performance for both classification and regression tasks in sEMG, and the model’s ability to resist signal loss, Gaussian additive noise, and Gaussian multiplicative noise was clearly improved. This will further drive the practical application of sEMG in VR, AR, and other human-computer interaction scenarios.

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 1
presentation: 2
questions: See comments in the 'Weakness' section above. The current version of the manuscript is not satisfactory, but I am willing to raise my score if the authors can address any of my major concerns.
soundness: 2
strengths: Gesture recognition using sEMG is a significant field with applications in prosthetic control and rehabilitation.  Extensive experiments have been conducted in this work. The inclusion of code is appreciated. However, clearer instructions are needed.
summary: The authors propose a Short Term Enhancement Module (STEM) to improve noise resilience in sEMG-based gesture recognition. The results show that the proposed method performs the best, both with and without additive noise, on the publicly available EMG dataset, GRABMyo.
weaknesses: Major concerns on the Claims and Contributions: 
1. 'Learnable denoise, enabling noise reduction without manual data augmentation' (line 022-023).
The authors claim to introduce a "learnable denoise" method, enabling noise reduction without manual data augmentation. To the best of my knowledge, 'learnable denoise' typically refers to a model trained on paired noisy and clean data in an unsupervised manner, as illustrated in Fig. 2 of Ref [1] (published in CVPR'2024). Another common approach of denoising is to employ an autoencoder-based structure. However, this manuscript does not incorporate such a strategy in the pretraining stage. In this work, pretraining stage appears limited to signal or segment masking, as suggested by Equation 1, Figure 2(a), and Algorithm 1 in the Appendix. This is also evident in the provided code: https://anonymous.4open.science/r/short_term_semg/model/generate_mask.py.

2. 'Scalability, adaptable to various models' (line 023).
The claim of "scalability" as described here appears to lack justification. Please correct me if I am wrong, 'scalability' typically refers to a model’s ability to handle increased data or computational demands, which is not evidently addressed here. "Adaptability" may be a more appropriate term in this context.

3. 'Cost-effectiveness, achieving short-term enhancement through minimal weightsharing in an efficient attention mechanism.'  (line 024).
The term "cost-effectiveness" seems ambiguous, with no evidence for computational efficiency. While the authors claims the reduced training time, other metrics such as parameter count, inference time, FLOPs, and GPU memory consumption, are not discussed. 

4. Limited novely, 'Intrinsic Variation Capture Module' repeated 14 times in this manuscript. 
It appears this module provides only slight modification, a short-term learning module based on self-attention, along with a transformer. The authors should clarify the full set of modifications made to the transformer structure to adapt it for gesture recognition tasks.

5. 'We pre-train on GRABMyo for 20 epochs using a fixed learning rate of 1e-4 for the backbone.' (line 307-308). 
The manuscript states that GRABMyo was used for pretraining. However, the code indicates that pretraining was conducted on Ninapro DB2. Please see evidence: (1) https://anonymous.4open.science/r/short_term_semg/pretrain.py,  code line 21:  train_loader, val_loader = get_dataloader_db2(dataCfg, "EMG_CSV/S1_E1_A1/"); (2) https://anonymous.4open.science/r/short_term_semg/cfg/db2.yaml; 
(3) https://anonymous.4open.science/r/short_term_semg/dataloaders/Ninapro.py, code line 67, function get_dataloader_db2(cfg, path_s,exercise), which returns data from Ninapro.
The descriptions in the manuscript and code appear contradictory and require clarification. Furthremore, the manuscript does not discuss which dataset was used for fine-tuning. Based on the code in  https://anonymous.4open.science/r/short_term_semg/finetuning.py, fine-tuning seems to employ two datasets, namely "hospital" and Ninapro DB2.

6. Unclear Details on Dataset and Evaluation Protocol.
Clearer descriptions and justifications are requied: 
(1) Whether this work follows the protocol proposed by the original paper that published the dataset. If not, please provide justification, as the dataset’s original paper [2] appears to use a different evaluation protocol;
(2) Whether results for comparison methods directly reported from recent works, or re-implemented by the authors (e.g. Table 1);
(3) For works used in comparison (e.g., [3]) that employed the same dataset, was the same evaluation protocol followed? If not, please provide justification;
(4) Specify whether the experiments were participant-dependent or participant-independent;
(5) Clearly state the sizes of training, validation, and test sets.


7. Fairness of Comparison. 
(1) Pretraining: Most methods used for comparison (except [2]) were not pretrained on EMG data, while the proposed method employs comprehensive pretraining and fine-tuning stages. This may lead to an unfair comparison.
(2) Ablation Study: The contributions of proposed innovations appear marginal based on Table 2 and Table 3. Using focal loss seems to lead significantly better performance than cross-entropy. For fair comparison, consistency across methods (focal loss vs. cross-entropy) is recommended.


8. Generalizability. 
The generalizability of the proposed model remains uncertain as only one dataset was used for evaluation. In [3], a comparison method used in this work, has been evaluated on multiple EMG datasets. Additional dataset(s) should be used to evaluate the generalizability of the proposed framework. 



Minor concerns : 
1. 'STEM generalizes across different gesture recognition tasks' (line 029-030). 
Since the experiments are conducted using only one dataset, it would be more appropriate to state that the model "generalizes across different gesture recognition tasks within the GRABMyo dataset." This would avoid any implication of broader cross-dataset generalization.

2. Use of Linear Projection. 
Figure 2(b) shows that linear projection is applied in the long-term enhanced module but not in the short-term enhanced module. The rationale behind this selective application of linear projection requires further clarification.

3. Naming of 'Short-Term Enhanced Transformer' for the Proposed Method.
The proposed network integrates both short-term and long-term enhancement modules, yet is named "Short-Term Enhanced." This naming could be misleading, suggesting the network is optimized solely for short-term enhancement.




[1] Kim, Changjin, Tae Hyun Kim, and Sungyong Baik. "LAN: Learning to Adapt Noise for Image Denoising." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[2] Pradhan, Ashirbad, Jiayuan He, and Ning Jiang. "Multi-day dataset of forearm and wrist electromyogram for hand gesture recognition and biometrics." Scientific data 9.1 (2022): 733.

[3] Zhang, Wenli, et al. "LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition." Frontiers in Neurorobotics 17 (2023): 1127338.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: 1) The authors claim that the proposed STEM module is low-cost; however, the paper lacks a detailed description or analysis of the module's computational efficiency. Could the authors clarify how they evaluated STEM's efficiency and provide comparative metrics or benchmarks to substantiate its low-cost nature? Additionally, how does STEM’s computational cost compare to other common noise resilience approaches?
soundness: 3
strengths: 1) to address the risk of self-supervised learning fitting to ineffective patterns and missing meaningful temporal semantics, the authors introduce a sophisticated masking strategy. This approach uses a geometric distribution to control masked and unmasked segments, promoting more robust temporal feature learning and improving the model's ability to capture essential sEMG patterns.
2) the authors propose a dual-decoder approach that extracts both long-term and short-term dependencies within the signal sequences. The long-term decoder captures the global context and overall signal structure, while the short-term decoder focuses on specific local characteristics critical for accurate gesture differentiation, enhancing both precision and robustness.
3) an asymmetric loss function is introduced to address class imbalance and improve model optimization by emphasizing difficult samples. This loss function enhances the model's ability to generalize across varied gesture categories, prioritizing accurate classification for challenging cases without overfitting on easy negatives.
4) the paper is well-structured and logically organized, with clear presentation of theorems, statements, and methodological components. The writing is precise and effectively communicates the technical contributions, making the model and findings accessible to readers.
summary: Surface Electromyography (sEMG) is highly susceptible to noise in real-world environments. To enhance noise resilience in sEMG-based models, this paper presents three key contributions:
1) introduction of the Short-Term Enhancement Module (STEM): A scalable, learnable, and low-cost module that enhances noise resistance. When integrated into neural networks, STEM improves performance by focusing on short-term signal features critical for distinguishing gestures amidst noise.
2) Self-Supervised Signal Masking: This technique leverages intrinsic variability in sEMG signals to enhance pre-training, allowing the model to learn robust representations without requiring extensive labeled data.
3) comprehensive Evaluation on the GRABMyo Dataset: Experiments on the largest available wrist sEMG dataset, GRABMyo, demonstrate that the Short-Term Enhanced Transformer (STET) achieves superior accuracy and robustness compared to existing methods, with reduced accuracy drop rates under noise and more precise gesture classification boundaries.
weaknesses: 1) The experimental evaluation presents classification results solely on the GRABMyo dataset, which may introduce dataset bias. To improve the robustness and generalizability of the findings, it would be beneficial to include experiments on additional publicly available datasets.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: - The data collection done for the robustness analysis is unclear. Is it the experiment described in Figure 6? Similarly, it is unclear if the methods in ‘dataset’ section describe data collected for this work or different work
- How were hyperparameters determined e.g. the short window length, for this dataset? what about the hyperparameters for the other baselines? what was the train/test/validation strategy? Was the data tested on held out users or held out gestures within a user? What fraction of data was used for the unsupervised pretraining? 
- How was the model run online in a streaming fashion? Is the architecture benchmarked offline the same as run online? 
- Is it the short term or long term e.g. for table 3 what is the breakdown by gesture.
soundness: 2
strengths: - The approach appears to be novel. While I am somewhat surprised that this approach would be better than a single transformer with a variety of temporal scales, it is reminiscent of other work in CV integrating different spatial context.
- The ablations are thorough, and there are numerous baselines.
- The pretraining appears to reduce the effects of additive and multiplicative noise.
- the application domain is timely and interesting
summary: This manuscript proposes a new framework for gesture recognition and regression from sEMG signals. They two existing datasets, GRABMyo and Ninapro that record sEMG alongside gesture identity and finger joint angles and develop a framework, STET, to process the signals. STET combines a pre-trained masked autoencoder with two transformers that are designed to process short term and long term information in parallel. These signals are concatenated, and in the case of gesture recognition, use a new asymmetric loss to upweight positive samples. The approach is thus a combination of novel components, although there is considerable past precedent for these approaches in the literature (eg LST-EMG-Net for separately modeling long and short term context and CutMix for time and channel wise masking of temporal signals). The experiments compare accuracy for gesture classification with a variety of baselines and perform ablations of different network components. They perform separate experiments simulating the addition of noise.
weaknesses: Cons
- There are a few novel components here, but I don't find the combination especially well developed or compelling. It is likely the case that their combination of masking, parallel coarse and fine transformers and asymmetric losses improves performance but the intuition advances is a little ad hoc. Unsupervised pretraining might help, but likely not by reducing gaussian noise in the signal. It is unclear whether the short of long term components are more important for affecting different types of gestures, it is not clear what window sizes are important. There are experiments oriented at some of these, but they don't quite get to the bottom of things. 

- It is unclear if the developed approach can be run online, and how it performs in the most important generalization domain for sEMG, across novel participants. So in practice I am not sure how useful this approach is. 

- Given the lack of documentation of hyperparameters for all models (see below) or statistics it is hard to evaluate the results critically. 

- The novelty of the long and short timescale approach is questionable. As they note, other manuscripts in the literature use this approach eg L262 claims that these long and short timescales are run in serial in LST-EMG-Net, but based on Figure 3 of Zhang et al 2023 they appear to be run in parallel. 

- The added gaussian and multiplicative noise experiments are not especially convincing to me as a realistic noise aggressor for sEMG, where the predominant failure mode is generalization across participants or issues like contact loss or power line interference. Given that the masking approach only improves accuracy by 0.6-1% in the GrabMYO data, it seems like these noise sources are not large. I find the noise discussion is a bit of a digression from the rest of the manuscript. 

- The related work section can be better organized, and the novelty and distinction between the architectures can be better presented in the related work, rather than throughout the text and appendix.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 2
questions: 1. On line 80, the authors state, *“we show that the long-term and short-term features are complementary in sEMG-based gesture recognition tasks.”* While the methodology section explains these concepts, it would improve reader clarity if a brief explanation of short-term and long-term features were included earlier in the paper.
2. Please clarify the rationale behind the choice of a sensor-wise masking strategy over other potential approaches, and how this decision impacts the model’s ability to generalize to various sEMG signal conditions.
3. Please provide more detail on why the specific masking ratio (0.15) and the average length of masked segments were chosen. Also, please add explanation of the sensitivity of the model’s performance to these parameters.
4. Could the authors explain why a multi-head self-attention layer was chosen for the long-term decoder and a sliding-window self-attention layer for the short-term decoder. 
5. For the sliding-window self-attention, how was the window size determined, and how sensitive is the model’s performance to variations in this parameter?
6. Please clarify how normalization was performed for the NRMSE (Normalized Root Mean Squared Error) metric.
soundness: 2
strengths: The paper is well-written and structured clearly, making it easy to follow.
The parallel use of both long-term and short-term decoders is an innovative approach that preserves context without losing important information from different stages in the sequence. This approach helps address limitations in existing sEMG models and presents a new way of combining local and global signal features, adding originality and significance to the field.
The self-supervised pre-training method effectively reduces the need for extensive labeled data and boosts the model's generalization, making it practical for real-world applications.
Additionally, the emphasis on noise robustness, a common challenge in sEMG signal processing, enhances the model's real-world applicability.
summary: This paper proposes an innovative solution for improving sEMG-based gesture recognition, tackling issues such as noise interference and distinguishing similar gestures, especially in non-laboratory settings. The authors introduce the Short-Term Enhancement Module (STEM), which captures local signal variations to maintain noise resistance, and a self-supervised sEMG Intrinsic Pattern Capture (EIPC) for pre-training and learning intrinsic signal patterns. STEM is designed to be easily integrated into various time-series deep learning models. Experiments show it significantly improves performance in classification and regression tasks, making it suitable for practical applications.
weaknesses: Insufficient Methodological Rationale: The paper would benefit from more detailed explanations for certain design choices, such as the selection of the window size in the short-term decoder and the masking ratio in the EIPC module. Clarifying how these parameters were optimized or chosen could strengthen the methodological foundation.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - L035-036: motoring -> monitoring
- L240-241: slide -> sliding
- What kind of information does LT vs ST store?
- Could the author clarify what kind of data pre-training was done on vs. fine-tuning?
soundness: 3
strengths: - Thorough empirical investigation of their proposed approach with various ablation studies
- Comparisons to related work show SotA performance
- Approach tested on both hand joint angle prediction as well as gesture classification
- Statistical significance test performed, which isn't common but valuable (appendix)
- Combination of their proposed approach with prior works architecture demonstrates generalizability of the improvements (e.g TEMGNet 78% vs TEMGNET+STEM 84%)
summary: This paper addresses the task of gesture classification and hand joint angle prediction from sEMG signal from a multi-electrode array. Specifically, the authors make three main contributions: 1)  Self-supervised pre-training based off of a signal masking objective 2) The introduction of short-term and long-term modules which are fused to create a final prediction 3) Use of asymmetric loss function to address asymmetries to address the imbalance between positive and negative samples.
The self-supervised pre-training step enables training on unlabeled EMG signals which is valuable given the sparse state of labeled EMG datasets. The self-supervised objective consists of reconstruction of masked signals, its loss denoted by MSE. The masks were constructed such that contiguous chunks of signal are masked at once, as otherwise neighboring signal timestamps make the objective too simple.
The short-term and long-term modules are both transformer-based models. The main difference between the two is that for the short-term module, the attention is limited within a certain window size. This forces the model to focus only on this short-term context to perform predictions.
The asymmetric loss follows Ridnik et al., and was introduce after making the observation that the positive and negative samples given a gesture is severely unbalanced in favor of the negatives.
Experimental results demonstrate improved performance over related work. Ablation studies show improved robustness to artificially added noise, as well as the improvement each proposed component gives.
weaknesses: - Although the long term module has been empirically validated to show improved performance I would've liked to see better explanation / investigation on what kind of information is being stored in long term vs short term module. EMG inherently encodes force and gestures are performed on a short time scale. Therefore it is not immediately clear to me why the long term module improves performance
- In similar manner, the ablation study demonstrates the performance of one module over the other and over their combined performance, where the combined performance yields the best results. However, one could also conclude from these results that having an extra module is what lead to the improved performance (i.e extra model parameters), and not the the type of module. To that end, how would the model perform with two LT or ST units and how does that compare to the fused variant?

Meta-review: The paper proposes a method for gesture recognition from sEMG. The work tackles an important area and shows strong results. The paper is easy to follow and understand. The shortcomings of the paper initially raised by the reviewers include the lack of novelty in individual components of the method, limited dataset scope, lack of diverse baselines and comparisons with state-of-the-art methods, missing references, and fairness in evaluation protocols.

Decision: Reject