EMBridge: Enhancing Gesture Generalization from EMG Signals Through Cross-modal Representation Learning
Authors: Wenhui Cui
Christopher Michael Sandino
Hadi Pouransari
Ran Liu
Juri Minxha
Ellen L. Zippi
Erdrin Azemi
Behrooz Mahasseni
Venue: ICLR 2026 Poster (2026)
Keywords: EMG
Zero-shot Gesture Classification
Cross-modal
Representation Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We proposed a cross-modal representation learning framework, EMBridge, to align EMG representations with more structured Pose representations. EMBridge enables zero-shot classification on unseen gestures and achieved superior generalization..

Abstract: Hand gesture classification using high-quality structured data such as videos, images, and hand skeletons is a well-explored problem in computer vision. Alternatively, leveraging low-power, cost-effective bio-signals, e.g. surface electromyography (sEMG), allows for continuous gesture prediction on wearable devices. In this work, we aim to enhance EMG representation quality by aligning it with embeddings obtained from structured, high-quality modalities that provide richer semantic guidance, ultimately enabling zero-shot gesture generalization. Specifically, we propose EMBridge, a cross-modal representation learning framework that bridges the modality gap between EMG and pose. EMBridge learns high-quality EMG representations by introducing a Querying Transformer (Q-Former), a masked pose reconstruction loss, and a community-aware soft contrastive learning objective that aligns the relative geometry of the embedding spaces. We evaluate EMBridge on both in-distribution and unseen gesture classification tasks and demonstrate consistent performance gains over all baselines. To the best of our knowledge, EMBridge is the first cross-modal representation learning framework to achieve zero-shot gesture classification from wearable EMG signals, showing potential toward real-world gesture recognition on wearable devices.

Published as a conference paper at ICLR 2026
EMBRIDGE: ENHANCING GESTURE GENERALIZA- TION FROM EMG SIGNALS THROUGH CROSS-MODAL REPRESENTATION LEARNING
Wenhui Cuil.2; Christopher M. Sandino', Hadi Pouransar', Ran Liu', Juri Minxha',
Ellen L.. Zippi', Erdrin Azemi', Behrooz Mahasseni*
1 Apple
2 University of Southern California
fwcui23, csandino, mpouransari, ran.liu3, jminxha, ezippi, erdrin, bmahasseni bnahasseni/eapple.ccom /eapple.com com
ABSTRACT
Hand gesture classification using high-quality structured data such as videos, im- ages, and hand skeletons is well-explored problem in computer vision. Alterna- tively, leveraging low-power, cost-effective bio-signals, c.g., surface electromyo- graphy (sEMG), allows for continuous gesture prediction wearable devices. In this work, we aim to enhance EMG representation quality by aligning it with embeddings obtained from structured, high-quality modalities that provide richer semantic guidance, ultimately enabling zero-shot gesture generalization. Specif- ically, we propose EMBriddee, cross-modal representation learning framework that bridges the modality gap between EMG and pose. EMBridge learns high- quality EMG representations by introducing a Querying Transformer (Q-Former). a masked pose reconstruction loss, and community-awann soft contrastive learn- ing objective that aligns the relative geometry of the embedding spaces. We eval- uate EMBridge on both in-distribution and unseen gesture classification tasks and demonstrate consistent performance gains over all baselines. To the best of our knowledge, EMBridge is the first rross-modal representation learning framework to achieve zero-shot gesture classification from wearable EMG signals, showing potential toward real-world gesture recognition on wearable devices.
INTRODUCTION
Hand gesture recognition wearable devices has recently attracted significant interest (Pyun al.) 2024 Moin et al. 2021) and demonstrated potential across diverse applications such as rehabil- itation (Marcos-Antee
et al. 2023). computer interaction (Jarque-Bou Bouu al. 2021), and
prosthetic control (Yu et al. 2023). With advances in deep learning and the availability of large- scale visual data, including videos and motion capture (Casile et al.) 2023), vision-based models have achieved remarkable success (Pavlakos et al., 2023 Qi et al. 2024p. However, cameras suffer from high power demands and privacy concerns, and potential occlusions can destabilize vision- based classification. This has motivated growing interest in low-power, easily integrable sensors (Tchantchane et al. 2023). such as surface electromyography (sEMG), for gesture recognition on wearable devices (Tchantchane et al.) [023) Wang et al., (202)). Deep learning approaches have been explored for EMG-based gesture classification, including convolutional neural networks (CNNs)
(Atzori et al. 2016), recurrent neural networks (RNNs) (Liu et al. 202T). and Transformers (Mon-
fazerin et al. 2023). However, predicting hand gestures from wearable EMG, especially generaliz-
ing unseen gestures without task-specific training, remains challenging (Laput & Harrison 2019).
This is mainly because of the high variability and fine dexterity of human hand movements, sensor
noise, and/or the limited scale of publicly available data (Lee al. 2024a Pereira al., 2024
Tam et al.] 2024). Due to the noisy and heterogeneous nature of EMG signals, learning from EMG alone (through self-supervised learning supervised end-to-end training) may not reliably yield generalizable and discriminative representations (as later demonstrated in our experiments).
Work done during internship Apple.

---

Published as a conference paper at ICLR 2026
An effective strategy to overcome the above limitations is to leverage another modality that offers richer semantic structure and higher signal quality as guidance during representation learning. This can be achieved through cross-mode
representation learning, which has proven highly effective in improving the quality of learned embeddings and shown remarkable success in vision-language
models (Xie al. 2025: Li et al., 2023), audio-visual language models (Gurram al.) 2022 Guo et al. 2025), and in biosignals such as IMU-video-text alignment (Moon et al. (022)) and EEG-text alignment Feng et al. 2023). However, aligning representations across modalities has not yet been fully explored for wearable EMG signals. Given paired EMG recordings and kinematic hand pose annotations collected simultaneously, we can study cross-modal alignment between EMG and pose. Unlike visual data, pose data directly captures the kinematics of hand movements, making it in- formative modality for guiding EMG representation learning. Therefore, we introduce a cross-modal framework for EMG representation learning that leverages a high-quality anchor modality, where pose data provides richer supervisory signals by capturing structural and semantic relationships. Our goal is to improve the quality of EMG embeddings, enabling generalization to users and unseen gestures at test time without the need for additional training or large-scale data collection. A potential practical application of our framework is wearable Human-Computer Computer Interaction. In scenarios like VR/AR and prosthetic control applications (Jarque BBu et al., [021]] Yu et al. 2023). wrist-worn device must continuously infer hand gestures from EMG to drive virtual avatar or robotic hand. A critical bottleneck is that users cannot be expected to record training data for every possible gesture they might perform. Our framework is designed enable zero-shot generalization, allowing the model to recognize novel gestures without requiring training samples from users.
We first introduce two unimodal encoders trained separately on EMG and pose data, and then align their output embeddings. Unlike classical approaches such as CLIP or BLIP (Radford et al., 2021 Li et al. 2023 2022), which symmetrically update both encoders toward shared latent space, our design adopts asymmetric setup, where the pose encoder is frozen as anchor and only the EMG encoder is optimized. On this basis, we propose EMBridge, a cross-modal representa- tion learning framework that bridges the modality gap between EMG and pose and enhances the representation quality learned from EMG signals through advanced alignment with pose representa- tions. EMBridge consists of three components: Querying Transformer (Q-former) (Li et al. 2023) that extracts pose-informative queries and aligns EMG and pose, a masked pose reconstruction loss (MPRL) that encourages queries carry structured pose information, and a community contrastive learning (CASCLe) objective that considers the neighborhood structures of poses and aligns the relative geometry in the latent space across modalities. Standard contrastive learning approaches (c.g., InfoNCE) treat all non non-matching samples as equally distant negatives. However, this assumption is suboptimal for our pose data, which is inherently continuous. Moreover, poses across different gesture categories can be semantically close. To capture these structural similarities and avoid confusing the model with hard negatives, we introduce CASCLe, which utilizes geomet- ric proximity in the pose space to generate soft targets. Together, these objectives guide the EMG encoder to capture pose-relevant semantics and produce discriminative, generalizable embeddings. Unlike cnneral-purpose multi-modal alignment, EMBridge is designed as a specialized solution for EMG-based gesture classification through cross-modal supervision.
soft
We utilize large-scal public EMG datasets (Salter et al. 2024 Atzori et al. 2014 which provide simultancous paired EMG and pose recordings to pre-train our EMBridge model and perform down- stream evaluations. We design gesture classification task to evaluate EMG representation quality. Following CLIP evaluation protocol (Radford et al. 2021). we validate the learned EMG represen- tations through zero-shot classification and linear probing, demonstrating superior performance on both in-distribution and unseen gestures compared to benchmark models. In summary, our contri- butions are two-fold: (i). We propose a cross cross-modal representation learning strategy to enhance the quality of EMG representations learned from noisy EMG signals by aligning them with high-qualit, pose representations. (ii). To the best of our knowledge, EMBridge is the first cross-ne cross-modal modal frame- work enabling zero-shot classification of unseen gestures for EMG signals from wearable devices.
2

---

Published as a conference paper at ICLR 2026
(a). Why Cross-Modal Learning
(b). EMBridge
Contastivv Learning
Naskod Pose
Impat
Reconstrution
etppeendee rout EVIO only
Difluse' Entergled
Encoder
Fees-forard--
Dooocing Layer
Align
EMG Engoder
K.V
Groee-Atemion
Fand-toraart-22
Paee Encodar
Learned Emnbecdinga
Pose-MM MAE Encoder
Structured
Set onttention
only
Learnaile Gueries
Massiedd Ploss Takane
Inbul Poes
Figure 1: (a) Motivation for cross-modal representation learning: pose embeddings are semantically structured and well-separated across gestures (colors), whereas EMG embeddings are not. This motivates leveraging pose as guidance to structure the EMG repre- sentation space. (b) Detailed architecture of EMBridge. Only one transformer block (self-attention, cross-attention, and feed-forward layers) shown for clarity, the model uses four such blocks.
using the same MAE pre-training,
METHODOLOGY
2.1 PRELIMINARIES
Definition I. EMG, pose data, and gesture classes. Let X' = {x+ E RCXTJNN denote multi- channel EMG sequences with C' channels and window length T, and let p = {p; E JJxT) denote the paired pose sequences of joint angles, where J is the number of joints in predefined hand skeleton (Salter et al.] (024). Here, N the total number of paired samples. We define a pose the instantaneous state of the hand skeleton (20 joint angles) a gesture is temporal sequence of poses over a time window. Let y = (1....,K K} denote the gesture classes, and let {yr) | be the labels with yi E V. Each pose p; has unique label yi although multiple samples may share the same label. The paired dataset is
single time point, whereas
D {(x;, Pt, v:)
which split into Dur Dval, and D Drest Let Via C y be the in-distribution gesture classes and Vunseen C. y be the unseen gesture classes, with Vin n Munseen Ó. For subset S E (tr, val, test].
Din (x),,,)) Ds: y E Vin),
D Dunseen {(x, p.)) E Ds J.nseen
Definition& 2. Unimodal Encoders. Let Eg and Ep map EMG and pose, respectively. into Rd. When the pose encoder is used frozen, we write Eg.
Unimodal Encoder Pre-training. We adopt Transformer encoder (Vaswani et al. 2023) with linear tokenizer to map raw signals into d-dimensional token embeddings. A patch length along time yields L = [T/S] non-overlapping tokens. We flatten channels within each patch and project a; d Following masked autoencoders (MAE) (He et al. 2021). randomly mask certain ratio of input tokens. The encoder processes only unmasked tokens and the transformer decoder reconstructs all tokens. The reconstruction loss is mean squared error loss only applied to the masked tokens. We pre-train the EMG and pose encoders independently, yielding & and E,. We use mask ratio of 0.5 and patch length of S 200. Unlike CLIP, which trained billions of image- text pairs, we align strong unimodal encoders to reduce the need for large-scale scale paired data.
text
2.2 EMBRIDGE
The proposed cross-modal representation learning framework EMBridge comprises three compo- nents: (i) Querying Transformer that acts an information bottleneck, extracting pose-informative features from the EMG encoder; (ii) a Masked Pose Reconstruction Loss that strengthens represen- tation learning; and (iii) Community-Awan Soft Contrastive Learning objective that aligns the relative geometry of the EMG and pose spaces by matching their community-levee similarity struc- tures, yielding structured EMG latent space. The framework is shown in Figure
3

---

Published as a conference paper at ICLR 2026
2.2.1 QUERYING TRANSFORMER (Q (Q-FORMER).
Inspired by BLIP-2 (Li et al. 2023), we use set of learnable queries extract pose-informative features from the EMG encoder. Let Q(o) E RMxd be M learnable queries. The Q-Former Fa stacks 4 self-attention blocks (each with a feed-forward layer) and 2 cross-attention layers in- serted every other block, whose keys/values are from Eg (x). The self-attention modules of the Q-Former are initialized from the pre-trained Pose-MAE encoder E. while cross-attention lay- ers are randomly initialized. All Q-Former parameters and the EMG encoder & are trainable, while the pose encoder & remains frozen. The Q-Former takes the learnable queries Q in- put and produces the same length of queries with learned representations Q' E RM We train the Q-Former using contrastive objective following Li al. (2022). Given the pose embed- ding V/ = = (P;), we select the query token that has the highest cosine similarity with U; as W; = O (m)) mi (i) arg (1... M) T 1P:
2' Te:
Given a mini-batch of size B, define
Σe exp(u] v//r) Let I {0, 10 be the one- hot indicator matrix with Iij 1 iff j i (matching pairs on the diagonal). The Information Noise-Contrastio Estimation (InfoNCE) loss (van den Oord et al. 2019) can then be written as
the softmax over EMG-pose similarities dú = explu. v,/r)
B
LInGONCE
Is log Pij
Unlike CLIP, where both encoders are trained jointly, in our setup the pose encoder is frozen and serves as a fixed anchor, since pose representations are higher-quality and more structured compared to EMG representations. Thus, we adopt the standard InfoNCE loss (EMG~>pose) rather than symmetric variant, since gradients only flow into the EMG encoder while the pose embeddings remain fixed. By encouraging the learnable queries to extract EMG features that are most consistent with pose representations, the Q-Former efficiently and effectively aligns EMG and pose.
2.2.2 MASKED POSE RECONSTRUCTION Loss (MPRL)
Beyond the contrastive loss, we add a masked reconstruction objective on input pose tokens to en- rich the representation learning process (Li et al. 2023). Specifically, in the first forward pass, the Q-Former produces query embeddings Q' E Raixa In the second forward pass, we concatenate masked pose tokens
pp.....,p. pr.] with Q' and apply the same self-attention modules with an attention mask: pose tokens may attend to all queries, while queries attend only to themselves. Cross-attentie from pose tokens to EMG is also disabled, pose cannot directly access EMG fea- tures. Thus, the information required to reconstruct masked pose tokens must be captured in Q', enforcing that queries extract insseinformative content from EMG. Let M be the set of masked toke-token indices and Hp Fx(Q. PI) (M++)) x)xd denote the outputs of Q-Former. The mask ratio is r. A decoding layer maps the outputs back to input pose token space. We minimize
LMPRI
Σ MEM
[[HHp|m))- - Pim]l1
where P[m] is the ground-truth pose token at index Jointly optimizing LMPRI with the con- trastive loss encourages the Q-Former to learn denoised, poss-infformative EMG representations and then yields an EMG latent space with richer pose semantics that can extrapolate to new gestures.
2.2.3 COMMUNITY-AWARE SOFT CONTRASTIVE LEARNING (CASCLE)
We propose CASCLe, which aligns EMG to pose by matching relative geometry of embedding spaces rather than only instance-level pairs. In standard contrastive learning, all non-matching poses are treated as negatives, even some are semantically very close to the true positive. This treatment is suboptimal because grouping similar poses as strict negatives confuses the model and leads to unstable gradients. CASCLe addresses this by assigning soft targets. Poses that are more similar to the ground- -oound-truth pose receive higher probabilities, while relatively dissimilar ones receive lower or zero probabilities. Since the pose encoder E is frozen, its embedding space defines a fixed relational graph. CASCLe builds soft targets from this graph and trains the Q-Former queries to learn similar
4

---

Published as a conference paper at ICLR 2026
(a). Canceptual Ilussration Community Soft Contrastive Learning
(b). Hard One-hot vs. Soft
.Pll
Pase
Push
Pose
4
EMG
1.0
Pose1
Pose
Pess
Contrastive Loss
Ci foyal
124
Pose-Cluste Aflinity Matrix Sn,
Community
15ch 10.7008,,00000000.....
Sim(Pase1, Pose2)
1sc 1
0.72
Community 2- 1S 100000000000000000000000000
Pose-Pose
Sindovity
Unlike conventional contrastive loss that relies on one-hot targets, (a). CASCLe constructs soft targets based community-level similarity. Each community is represented by affinities to cluster centroids, and pose-pose similarity is computed from affinity vectors, Soft targets used in CASCLe are shown in (b), computed from batch of 64 samples for clearer visualization.
Figure 2:
neighborhood structures for EMG embeddings, which strengthens overall semantic consistency with pose. An illustration of CASCLe is shown in Figure 2
Pose communities. We cluster pre-trained Pose-MAE embeddings (offline) using k-means (Likas 2003) to obtain Ne centroids C RNN (all fg-normalized), For mini-batch of pose embeddings P = [01: :.. uB| E Bxd we compute pose-cluster affinity matrix Sp.c = PC RBxN, where each row is sparsified by keeping only the top-ki closest clusters. This keeps the community size reasonable and excludes irrelevant clusters, reducing noise in soft targets.
Sppllis + f1Smolis TopK(SSpel,, (0,
(3)
otherwise.
Pose-pose similarity matrix. The community-aware pose-pose similarity matrix is then defined as Sp.p = Sp.c P.c E BxB To prevent self-matches from dominating the probability distribution after softmax when generating soft targets, we remove the diagonal: Sp.p = Sp. diag(S,p.))) Sp.p). A similar strategy has been adopted prior work (Gao et al. 2023) for soft target construction and proven effective. Soft targets are then defined as
exp(Sp.pii.iI/7) T.j Σxy, expppp.ppipppppppppp (Spppi,AA//T)
J i
(4)
with temperature Ts > Intuitively, Øij is the probability that pose By is semantically relevant neighbor of in the fixed pose relational graph. Using the same EMG-pose similarities (s defindd earlier for InfoNCE, CASCLe minimizes a Soft contrastive objective defined the cross-entropy between soft targets Vi and qıj, +
B
B
CCASCLe =
ΣΣ {=
Tis log qij.
(5)
This objective can be interpreted predicting the degree of similarity between EMG-pos: PoGe pairs, where this degree is measured according to the structural organization of the pose latent space. The total training objective of EMBridge combines instance-level and structural community-level supervision:
LIntaNCE + LOASCI LMPRI,
with weights X. > 0. InfoNCE enforces instance alignment, while CASCLe aligns relational structure of the latent space between two modalities. In this way, the EMG encoder is guided to align not only with its exact pair but also with poses that share similar semantics, providing more robust and informative supervisory signals.
5

---

Published as a conference paper at ICLR 2026
Table 1: Dataset splits with gesture and user counts. Four unseen gestures evaluated out of six total.
Gesture Counts
User Counts
Split (totals)
Subset
In-dist.
Unseen
In-dist.
Unseen
Der (23 gestures/ / 158 users)
D probe-tr or
4
0
158
0
D nuno
4
4
0
3
Deal gestures/ 15 users)
D Dprctesr
0
4
0
12
Dreat (29 gestures 158 users)
Deval
4
4
0
20
3 EXPERIMENTS AND RESULTS
3.1 EVALUATION PROTOCOLS.
We employ two evaluation protocols to examine the learned representation quality. Linear prob- ing (LP): with labeled EMG data {(T,, v:)), we freeze &g and train a vandomly-initialize linear classifier C on top of its embeddings, reporting accuracy fication (ZS) is performed as k-nearest-neighhor voting in the embedding space, following stan- dard practice in representation learning (Marks et al., [024] Radford et al.] [02I). For each EMG sample, we retrieve its top-k nearest poses in the embedding space, then vote the corre- sponding gesture labels determine the predicted gesture. Given test EMG sample 2y, let R, TopKpePP.n. (E E (p))) be the set of & pose samples with highest cosine similarity to Zj. We then predict Úü = modef v(p) E R5}: where y(p) is the gesture class of pose p.
held-out split. Zero-shot classi-
3.2 DATASETS
emg2pose dataset. We use emg2pose (Salter et al., 2024), a large-scale open-source EMG dataset containing 370 hours of sEMG and synchronized hand pose data across 193 consenting users, 29 different behavioral groups that include a diverse range of discrete and continuous hand motions such making fist or counting five. The hand pose labels are generated using high-resolution motion capture system. The full dataset contains over 80 million pose labels and is of similar scale to the largest computer vision equivalents. Each user completed four recording sessions per gesture category, each with a different EMG-band placement. Each session lasted 45-120 s, during which users repeatedly performed mix of 3-5 similar gestures or unconstrained freeform movements. We use non-overlapping 2-second windows as input sequences. EMG is noramazzed band-pas filtered (2-250 Hz), and notch-filtered at 60 Hz. For more details, please refer to Salter et al. (2024).
Data Split for emg2pose. We evaluate on two disjoint gesture sets drawn from the public emg2pose corpus. First, we select four representative single-hand motions covering various finger movements as our in-distribution gestures. Second, from the six held-out classes that are not seen during train- ing, we exclude the woo-handed gesture and the highly variable "finger freeform" class, yielding four unseen gestures. Details of gesture classes are in Appendix. For data splits, we follow the public train Dtr. val Doal, test D testt splits and define our data splits for downstream gesture classification tasks shown in Table [ The model is pre-trained Din probo-tr and report final accuracy on the evaluation set Dia Di For linear probing on unseen gestures (which appear only in the original val and test splits), we train on Drotee-ee and report accuracy on Dunseen Zero-shot classification is evaluated only on Deval- All users Deval are unseen, so both the I.P and ZS results also assess user-level generalization. A held-out dataset Dinne: strictly disjoint from all other sets, reserved for hyper-parameter tuning.
the full Drr- A linear head is trained on
We utilized two NinaPro EMG datasets for more comprehensive evaluation of EMBridge. Specifically, Ninapro DB2 (Atzori et al. 2014) is used for pre-training, which includes paired EMG-pose data from 40 subjects. It contains 49 hand gestures (including basic finger flex- ions, functional grasps, and combined movements) performed by 40 healthy subjects. EMG signals recorded from 12 electrodes placed the forearm at sampling rate of 2 kHz, alongside hand kinematics data captured by a data glove. For downstream gesture classification, we use NinaPro DB7 (Krasoulis et al. 2017) which contains data from 20 non-amputated subjects collected with
NinaPro dataset:

---

Published as a conference paper at ICLR 2026
the same EMG device and gesture set as DB2 (more details on NinaPro Website Data split. The entire DB2 dataset was used for pre-training, except Gestures L, 5, 10, and 15 from exercise B, which were excluded to serve unseen gestures in DB7. Gestures 1, 5, and 10 from exercise C were used as in-distribution gestures. Within each gesture, sessions were randomly divided into probe-training and evaluation sets, and zero-shot evaluation conducted only on the latter.
3.3 GESTURE CLASSIFICATION RESULTS
Comparing Schemes. We evaluate EMBridge against various baselines. Unimodal models trained solely on EMG include: a supervised encoder-decoder Transformer (PoseT) regressing poses from EMG; the supervised regression models from the emg2pose benchmark (Salter et al.) (024) (emg2pose, Vemg2pose, NeuroPose); and a self-supervised MAE model trained only on EMG (EMG-MAE). We also compare multi-modal models: CLIP-style Contrastive Pose-EMG Pre-
training framework (CPEP) (Cui et al. 2025), which applies LIMfONCE directly to [CLS] tokens from EMG and pose encoders via projection layer; and a plain Q-Former variant trained only with LIaEONEE. Unlike Q-Former, CPEP does not introduce a transformer but uses a projection layer. We further evaluate label-smoothed variants of both models. Label smoothing has been shown im- prove contrastive learning by mitigating overconfidence and handling noisy similarities (Wickstrom ct al. 2022, et al. (2022). We introduce CPEP-LS and Q-Former-LSS, where InfoNCE targets are softened with smoothing factor of 0.1. For a fair comparison under linear probing, each baseline's encoder is frozen and a softmax linear head is trained on top. As an additional reference, we report an upper bound from linear probing the pre-trained PAS. Since the pose encoder is the fixed alignment anchor, this represents the best achievable performance EMG features were perfectly aligned. For LP, we use publicly available emg2pose checkpoints with the same data splits.
We evaluate EMG representation quality on both in-dist. and unseen gestures using two protocols: zero-shot classification (ZS) and linear probing (LP). Supervised baselines do not support zero-shot (K-nearest neighbor) classification in the embedding space. We report balanced accuracy on both emg2pose and NinaPro dataset to account for class imbalance across gesture classes. As shown in Table 2 EMBridge consistently outperforms all baselines, with the largest gains in zero-shot classi- fication, where even surpasses the LP performance of all unimodal models. The most significant improvements appear on unseen gestures for emg2pose and ZS on in-dist. gestures for NinaPro, demonstrating the stronger generalization capacity of EMBridge and its practical value for wear- able gesture recognition. We note that CPEP achieves higher LP performance on unseen gestures than EMBridge for both datasets. This likely due to our use of query averaging in EMBridge instead of selecting the query with maximum similarity to the paired pose, which may be subopti- mal. Maximum-siminn similarity selection is avoided here during LP to prevent potential data leakage. We also evaluate EMBridge in the few-shot setting with LP on emg2pose. Even with only 50% of the probe-training data, EMBridge outperforms all unimodal baselines?
Per-Gesture Classification Performance Breakdown. We conduct detailed analysis the emg2pose dataset by computing per-gesture FI scores from zero-shot classification on unseen ges- tures assess performance gains achieved by EMBridge over other cross- modal frameworks (CPEP and Q-Former). Reporting Fl scores provides a more comprehensive and balanced view of per-class performance. Confusion matrices in Figure (3 offer clearer illustration of improvements. Com- pared to CPEP and Q Q-Former, EMBridge achieves consistently higher F1 scores across all ges- ture classes, with particularly notable gains on Class (0.513 vs. 0.43%/0.494) and Class (0.504 vs. 0.436/0.458). Class 3 is the gesture of counting up and down then finger wiggling, which is very challenging for vision-bass based gesture classification due to visual occlusion (Salter et al.) 2024). The improvements underscore EMBridge's stronger discriminative capability difficult and frequently confused gestures, demonstrating its practical value in real-world cases where occlusion common.
Per-User Performance Gains Breakdown. Similarly, on unseen gestures, we compute the zero- shot classification performance (FI score) within each user, where all 20 users are held out from training. Figure (b) illustrates the per-user performance gains of EMBridge compared to CPEP and Q-Former. For per-user analysis, EMBridge achieves an overall improvement of 14.2% over CPEP (0.522 vs. 0.457) and 10.2% over Q-Former (0.522 vs. 0.473). The average relative per-user im-
https: / / napro. hevs. ch/ Mooe details are provided in the Appendix.
7

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of gesture classification results unimodal and multi-modal models.
Results are reported on the emg2pose dataset and the NinaPro dataset.
emg2pose
NinaPro
Unimodal Models
In-dist. LP
Unseen LP
In-dist. LP
Unseen LP
Upper-bound
0.851
0.649
0.769
0.632
EMG-MAR
0.347
0.334
0.283
0.256
NeuroPose (Salter et 2024
0.692
0.248
/
emg2pose (Salter et al.
2024
0.734
0,405
Vemg2pose (Salter et
2024
0.650
0.312
PoseT
0.705
0,433
0.694
0,425
In-dist.
Unseen
In-dist.
Unseen
Multi-modal Models
LP
ZS
LP
ZS
LP
ZS
LP
ZS
CPEP
0.782
0.757
0.536
0.481
0.675
0.604
0.483
0.413
CPEP-LS
0.780
0.759
0.538
0.487
0.681
0.617
0.494
0.424
Q-Former
0.782
0.763
0.493
0.498
0.688
0.613
0.481
0.447
Q-Former-LS
0.777
0.760
0.495
0.498
0.692
0.618
0.486
0.439
EMBridge
0.785
0.777
0.505
0.528
0.703
0.692
0,492
0.447
provement is 16.0% compared to CPEP and 11.6% compared to Q-Former, which demonstrates that EMBridge yields consistent improvements across unseen users, even under inter-subject variability.
(8). Performance Breakdown
Contasion Mabte CPeE
urreer
Confusion Matie EMErdgge
Clo
241
841
611
241
382
1779
5:000
6496
Cle
189
291
1000
372
204
1550
23
197
0.807
0.077
6400
43
204
336
Peddse
Dan
(c) Efficiency
11.0
11.0
02
09
Esswoodd AccUFaC
111
01
012
013
113
6.1
8.5
Figure 3: (a) Confusion matrices from ZS gestures, with per-class F7 scores shown beside row labels. (b) Per-user 2S performance on unseen gestures. (c) Data efficiency analysis via ZS on in-dist. and unseen gestures. Dotted lines indicate LP performance of unimodal baselines.
ABLATION STUDY
Individual Contribution of Components in EMBridge. We analyze three ablated variants of EM- Bridge, each removing a component (Q-Former, MPRL, CASCLe) to assess its individual impact. Removing Q-Former reduces the model to CPEP + CASCLe, since without the Q-Former architec- ture and learnable queries, the masked pose reconstruction task cannot be performed. As shown in
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation of EMBridge: individual component impact and soft contrastive objectives.
Ablated Variants
LP in-dist.
LP unseen
ZS in-dist.
ZS unseen
EMBridge w/o Q-Former
0.793
0.538
0.763
0.494
EMBridge w/o MPRL
0.783
0.494
0.764
0.516
EMBridge w/o CASCLe
0.784
0.485
0.764
0.509
Soft Contrastive Objective Label Smoothing (Fini et al. 2023)
0.777
0,489
0.759
0.511
SoftCLIP (Gao et al. 2023
0.788
0.490
0.760
0,510
CASCLe
0.785
0.505
0.777
0.528
Table removing any component leads to drop in zero-shot performance. Interestingly, remov- ing Q-Former yields slightly better linear probing results, consistent with prior CPEP findings, and demonstrates the versatility of CASCLe, which can be effectively integrated into the CPEP archi- tecture to improve performance. Removing MPRL or CASCLe reduces generalization to unseen gestures, underscoring their importance in cross-moda alignment and representation learning.
Soft Contrastive Objectives. We further compare CASCLe with alternative soft contrastive learn- ing objectives. Label Smoothing, explored in Fini et al. (2023), applies soft targets in CLIP and has shown consistent gains. We also adapt SoftCLIP (Gao et al. 2023) to our EMG-pose setup by deriving soft targets from instance-level pairwise similarities between pose samples, providing a fair baseline against CASCLe, which models community-
structural similarity. We replace LInfoNCE + a with label smoothing and the adapted SoftCLIP objective,, respectively, while keeping the rest of EMBrigge unchanged. Further discussion of soft contrastive learning is provided in Section 5 In Table 3 CASCLe outperforms both alternatives in ZS, which highlights the advan- tage of modeling community structure over simple soft labels or instance-level similarities.
Data Efficiency. We investigate how the scale of paired pre-training data influences downstream performance. Since collecting paired EMG-pose data is costly and time-consuming, it is essential to evaluate the data efficiency EMBridge. We uniformly downsample sessions within each ges- ture class to simulate limited pre-training data. We vary the proportion of paired pre-training data from 0.2 1.0 (full dataset) and report ZS performance on both in-dist. and unseen gestures in Fig- ure (c.. Remarkably, even when trained with only 40% of the paired data, EMBridge's s zero-shot classification still surpasses the LP performance of unimodal baselines trained on the full dataset.
Sensitivity to Hypee-parraeters. We analyze how 7S and LP performance of EMBridge is in- fluenced by different
choices. Specifically, we vary one papramerrretter at a time while keeping all others fixed to their optimal settings. Results are summarized in Figure 4 The choice of т2 and the number of top-k, clusters (used CASCLe; see Eq. 3) jointly determines the quality of the resulting soft targets. A smaller produces a sharper probability distribution, and as To - O. CASCLe degenerates to the Supervised Contrastive Loss (Khosla et al. [022] by assigning hard label of multiple positives. In contrast, moderate т. assigns reasonable weights off- diagonal soft positives, balancing contributions without allowing them to dominate or amplifying small differences. Increasing the number of top-k clusters (k 10, 20) degrades both LP and ZS performance (Figure 4p), confirming that including distant neighbors introduces noise when con- structing soft targets. EMBridge is robust to variations in À and mask ratio r in the masked pose reconstruction task, well as loss weight a. We also find that using 16 queries yields the best results. queries may insufficient to capture the full range information, while 32 queries can lead to overfitting. We also evaluate alternative similarity metrics for CASCLe. By default, cosine similarity used to compute both the pose- cluster and pose pose similarity matrices. Using L1 and L2 distances yields weaker results. This aligns with prior contrastive learning literature, where sine similarity is commonly preferred for representation alignment (Radford et al. 2021 Chen et al.] 2020). In Figure 4 (f), observe that using more nearest neighbors improves 7S performance.
[Cen

---

Published as a conference paper at ICLR 2026
30. Inpperttii
(b) Tos custers value o...
16) Mosk ralo MPRL
Weight
6.33
Weiggti CASCII
0:. Nambbr quarkc
0% Sim unly mulr cin CASCLe
0%: resemst gritoons
Figure 4: Sensitivity to hyper-parametert Dashed lines indicate the values used in the best setup,
5 DISCUSSION AND CONCLUSION
Related Work Soft Contrastive Learning. Traditional contrastive learning methods (van den Oord et al. 2019; Chen et al., [2200] Radford et al.) 2021) use one-hot targets, where only the exact matching pair is treated as positive and all others as negatives. However, when multiple positives or highly similar instances exist, this introduces false negatives. Recent work addresses this by incor- porating soft targets. Fini al. (003) employs label smoothing to generate soft targets. SoftCLIP (Gao et al. 2023) derives them from intra intra-modal similarity using fine-grained image features, while X-CLR (Sobal et al. 2024) builds a sample similarity graph and replaces binary labels with simi- larity scores. SofiCLT (Lee al. 2024b) softens targets based on temporal proximity, and Huang jet al. (2024) aligns cross cross-mode modal and uni uni-modal representations using teacher-derived similarity sig- nals. CASCLe differs by constructing soft targets from community- structural similarities in the embedding space, rather than relying solely on instance-leve relations.
The motivation of the asymmetric setup is to pose representations fixed anchors guide the learning of EMG representations. Training both encoders simultaneously could lead to suboptimal representations, the pose representations tations. Additionally, this asymmetric setup enhances extendability. Future work can pre-train the pose encoder on large-scale and unpaired pose data using MAE. This allows EMG representations by aligning with frozen pose embeddings, even when paired data is limited. Leveraging abundant unpaired data reduces reliance on paired samples and improves data efficiency.
altered be closer to the noisier EMG represen-
effectively learn
By leveraging pose as a rich supervisory signal, EMBridge learns pose-informed EMG embeddings that capture structural and semantic relationships and are discriminative in the latent space. Across in-distribution and unseen gesture classification tasks, EMBridge demonstrated strong performance gains, particularly the zero-shot setting. Overall, EMBridge provides an effective approach to zero-shot gesture classification on wearable EMG signals, and can serve foundation for future exploration of cross cross-modal representation learning on EMG and other bio-signals.
LIMITATIONS AND FUTURE WORK
While EMBridge significantly enhances zero-shot gesture classification on wearable EMG signals, we acknowledge limitations and room for future exploration. Our framework currently relies on paired EMG-pose data for cross- modal alignment. Although emg2pose and NinaPro are large-scale datasets, high-quality paired datasets remain scarce in the broader bio-signal community. Training on a single dataset could potentially limit the pose encoder's capacity. A promising direction for fu- ture work to leverage large-scale, publicly available unpaired pose data for unimodal pre-training. This would yield more robust pose representations, providing stronger supervision for EMG repre- sentation learning while significantly reducing the reliance on paired samples. EMBridge can further demonstrate its effectiveness in cross-modal learning with limited paired data. Furthermore, the pro- posed framework can potentially extended to other modalities such as RGB-EMG or Video-EMG in future work, which can be done by incorporating a pre-trained vision encoder. Another interesting
10

---

Published as a conference paper at ICLR 2026
direction for future work is explore probabilistic modeling when constructing pose communities. Specifically, a Gaussian Mixture Model could be used to assign each pose embedding a soft proba- biliity distribution multiple clusters or communities. Structural similarity between poses could then be computed based on the similarity between these membership distributions (e.g., using KI. divergence), enabling smoother and more continuous modeling of pose neighborhood structure.
7 REPRODUCIBILITY STATEMENT
All datasets used in this work are publicly available. Details of the model architecture, training objectives, and implementation details are provided in Section 2 and Appendix A.66] including all key hyper-parameters and model configurations to ensure reproducibility.
REFERENCES
Manfredo Atzori, Arjan Gijsberts, Claudio Castellini, Barbara Caputo, Anne-Gabrielle Mittaz Hager, Simone Elsig, Giorgio Giatsidis, Franco Bassetto, and Henning Müller. Electromyog- raphy data for non-invasive naturally-comtrolled robotic hand prostheses. Scientific data, 1(1):
1-13, 2014.
Manfredo Atzori, Arjan Gijsberts, Claudio Castellini, Barbara Caputo, Anne-Gabrielle Hager, Stephan Elsig, Giorgio Giatsidis, Franco Bassetto, and Henning Müller. Deep learning with convolutional neural networks applied electromyography data: A resource for the classification of movements for prosthetic hands. Frontiers in Neurorobotics, 10:9, 2016.
Antonino Casile, Giulia Fregna, Vittorio Boarini, Chiara Paoluzzi, Fabio Manfredini, Nicola Lam- berti, Andrea Baroni, and Sofia Straudi. Quantitative comparison of hand kinematics measured with a markerless commercial head-mounted display and a marker-based motion capture system in stroke survivors. Sensors, 23(18)77906, 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework /bs/
for contrastive learning of visual representations, 2020. URL
20C2. 05709
Wenhui Cui, Christopher Sandino, Hadi Pouransari, Ran Liu, Juri Minxha, Ellen Zippi, Aman Verma, Anna Sedlackova, Erdrin Azemi, and Behrooz Mahasseni. Cpep: Contrastive pose-emg pre-training enhances gesture generalization on emg signals, 2025. URL https://arxi::
org/abs/2500 04699
Xiachong Feng, Xiaocheng Feng, Bing Qin, and Ting Liu. Aligning semantic in brain and lan- guage: A curriculum contrastive method for dlectoencephulgrrahy-tttoo generation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:3874-3883, 2023. doi:
N0.1109//NSREE2223333311
Enrico Fini, Pietro Astolfi, Adriana Nomero-Soriane Jakob Verbeck, and Michal Drozdzal. Im- proved baselines for vision-language pre-training, 2023. URL htt.hhttss///eii...ooo abbs/
2305. 2305.08675
Yuting Gao, Jinfeng Liu, Zihan Xu, Tong Wu Enwei Zhang, Wei Liu, Jie Yang, Ke Li, and Xing Sun. Softelip: Softer modal alignment makes clip stronger, 2023. URL https://arxiv:
oxc/abs/2303.172
Yuxin Guo, Shuailei Ma, Shijie Ma, Xiaoyi Bao, Chen- Wei Xie, Kecheng Zheng, Tingyu Weng, Siyang Sun, Yun Zheng, and Wei Zou. Aligned better, listen better for audio-visual large language models, 2025. URL htttts::/// arxiv. hbtgg/abs/2555
02061
Sumanth Gurram, Andy Fang, David Chan, and John Canny. Lava: Language audio vision align- ment for contrastive video pre-training, 2022. URL
08C24
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners, 2021. URL hhtpps: /ttp:i//////..////// www...oogg...//111111 06377
11

---

Published as a conference paper at ICLR 2026
Hailang Huang. Zhijie Nie, Ziqiao Wang, and Ziyu Shang. Cross-modal and uni-modal soft-label alignment for image-text retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):18298-18888 March 2024. ISSN 5399. doi: 10. 10.16099aaai.v38816.22999
URL
http://dx.. doi. wtw.odoi...o.ooooooooo 10. 1609/aaaaa v3811 6. 29789
Néstor Jarque-Bou, Joaquin L Sancho Bancho-Bru, and Margarita Vergara. A systematic review of emg applications for the characterization of forearm and hand musele activity during activities of daily living: Results, challenges, and open issues. Sensors, 21 (1):33033, 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning, 2021. URL https:
//arxiv.org/aatt wwtp://rggaaaa///oeeee /204. 11004.11362. 11362
Agamemnon Krasoulis, Iris Kyranou, Mustapha Suphi Erden, Kianoush Nazarpour, and Sethu Vi- jayakumar. Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. Journal of neuroengineering and rehabilitation, 14(1):71, 2017.
Gierad Laput and Chris Harrison. Sensing fine-grained hand activity with smartwatches. In Proceed- ings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI '19, pp. 1-13, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450359702. doi:
04.144//29000553300566 URL https: //w.i www.oo../.o.o..... 11450/11114322 90605. 3300568
Donghee Lee, Dayoung You, Gyoungryul Cho. Hoirim Lee, Eunsoo Shin, Taehwan Choi, Sung- han Kim, Sangmin Lee, and Woochul Nam. Emg-based hand gesture classifier robust to daily variation: Recursive domain adversarial neural network with data synthesis. Biomedical Signal Processing and Control, 88:105600 2024a.
Seunghan Lee, Taeyoung Park, and Kibok Lee. Soft contrastive learning for time series, 2024b. URL https::/aaxiv..tmm org. /ttp:/ab.//311.222333
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022. URL https:
//axxiv.or abs 2201.22111... 12086
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 2023.
Aristidis Likas, Nikos Vlassis, and Jakob J Verbeek. The global k-means clustering algorithm.
Pattern recognition, 36(2):451-46) 2003.
Yilin Liu, Shijia Zhang, and Mahanth Gowda. Neuropose: 3d hand pose tracking using emg wearables. In Proceedings of the Web Conference 2021, www '21, pp. 1471-1482, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978 9781450383127. doi: URL URL/ttpps///6...org/1..111.1 1145 3449890
10.1145/3442381..4442 34449890.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts, 2017. URL https://arxin: / abs / 608 03983
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https:
//arxiv.org wwtpab////..1100011
S Marcos-Antón et al. semg-controlled forearm bracelet and serious game- rehabilitation.
Journal of NewroEngineering and Rehabilitation,, 2023. URL httpaauaaaa//eeuuucogggeaaaaaa
com. /atpa//aaaaaatttaaaaaaoe////////
Markus Marks, Manuel Knott, Neehar Kondapaneni, Elijah Cole, Thijs Defraeye, Fernando Perez- Cruz, and Pietro Perona, A closer look at benchmarking self-super pre-training with image classification, 2024. URL httttttts://aartt htaas/aco..oooo////////////////n
wrg.
12210
Ali Moin, Andy Zhou, Abbas Rahimi, Alisha Menon, Simone Benatti, George Alexandrov, Senam Tamakloe, Jonathan Ting, Natasha Yamamoto, Yasser Khan, et al. A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nature Electronics, 4(1): 54 54-63, 2021.
12

---

Published as a conference paper at ICLR 2026
Mansooreh Montazerin, Elahe Rahimian, Farnoosh Naderkhani, S Farokh Atashzar, Svetlana Yanushkevich, and Arash Mohammadi. Transformer- based hand gesture recognition from in- stantaneous to fused neural decomposition of high-density emg signals. Scientific reports, 13(1): 11000, 2023.
Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Alireza Dirafzoon, Aparajita Saraf, Amy Bear- man, and Babak Damavandi. Imu2clip: Multimodal contrastive learning for imu motion sensors from egocentric videos and text, 2022. URL plctps://arxiv/
abs 2210 14395
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers, 2023. URL https://arxiv.org
abs 00ss/2312.52555 05251
Joao Pereira, Dimitrios Chalatsis, Balint Hodossy, and Dario Farina. Tackling electrode shift in gesture recognition with hd-emg electrode subsets, 2024. URL [KKlhttpps///axxi.orr/////
2401.02773
Kyung Rok Pyun, Kangkyu Kwon, Myung Jin Yoo, Kyun Kyu Kim, Dohyeon Gong, Woon-Hong Yeo. Seungyong Han, and Seung Hwan Ko. Machine-learned wearable sensors for real-time hand-motion recognition: toward practical applications. National science review, 11()nnwad,29,, 2024.
Jing Qi, Li Ma, Zhenchao Cui, and Yushu Yu. Computer vision-basee hand gesture recognition for human-robot interaction: review. Complex & Intelligent Systems, 10(1):1581-1606, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL
httpss:/aaxixs attpa////axxxoo.r/aaaaooo. wwwp/ogg//////oooooo///o///aaaa/ 2203a 00020
Sasha Salter, Richard Warren, Collin Schlager, Adrian Spurr, Shangchen Han, Rohin Bhasin, Yujun Cai, Peter Walkington, Anuoluwapo Bolarinwa, Robert Wang, Nathan Danielson, Josh Merel, Eftychios Pnevmatikakis, and Jesse Marshall. emg2pose: A large and diverse benchmark for surface electromyographic hand pose estimation, 2024. URL
abs/
2412. 02725
Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro As- tolfi, Kyunghyun Cho, and Yann LeCun. X-sample contrastive loss: Improving contrastive learn- ing with sample similarity graphs, 2024. URL httl:atttt:/aati wttp://...rr/aab//.....o. 18134
Simon Tam, Shriram Tallam Puranam Raghu, Étienne Buteau, Erik Scheme, Mounir Boukadoum, Alexandre Campeau-Lecours, and Benoit Gosselin. Towards robust and interpretable emg- hand gesture recognition using deep metric meta learning, 2024. Bttthtts//// https::/aaxii..... org/ abs/24054 15360
Rayane Tchantchane, Hao Zhou, Shen Zhang, and Gursel Alici. A review of hand gesture recog- nition systems based on noninvasive wearable sensors. Advanced intelligent systems, 5(10); 2300207, 2023.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, 2019. URL htttthtt//a/xiiiii http://wwwww.og/abbb///aaaes wwtp///////////////// 03748
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is you need, 2023. Uttpsstss///axx
org/abs/ abs/ 1706. 03762
Huihui Wang, Bo Ru, Xin Miao, Qin Gao, Masood Habib, Long Liu, and Sen Qiu. Mems devices- based hand gesture recognition via wearable computing. Micromachines, 14(5):977, 2023.
13

---

Published as a conference paper at ICLR 2026
Kristoffer Wickstrøm, Michael Kampffmeyer, Karl Øyvind Mikalsen, and Robert Jenssen. Mixing up contrastive learning: Self-super representation learning for time series. Patter Recogni- fion Letters, 155:54 61. March 2022. ISSN 0167-8655. doi: www.0016..patre:2222.....0000000 URL
http:/ //tp:://xxx.o// doi. org/10. 101 10016 6 pat wec.tre.cc.. 2022. 02.007
Shaoan Xie, Lingjing Kong, Yujia Zheng, Yu Yao, Zeyu Tang, Eric P. Xing, Guangyi Chen, and Kun Zhang. Smartclip: Modular vision language alignment with identification guarantees, 2025.
htttss/:/ www.vvaaivvvvvvaa/aiva org/ abs 2507 22264
Guangjie Yu, Ziting Deng. Zhenchen Bao, Yue Zhang, and Bingwei He. Gesture classification in electromyography signals for real-time prosthetic hand control using convolutional neural network-enhanced channel attention model. Bioengineering, 10(11))1324, 2023.
A APPENDIX
A.1 THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used LLMs to assist with language polishing, including grammar checking and word choice refinement. All research ideas, experiment design, analyses, and essential contributions were con- ducted by the authors.
A.2 DETAILS OF EMG2POSE DATASET
A.2.1 DATA COLLECTION PROTOCOL
Data collection was divided into four distinct sessions, involving multiple device don-doff cycles (avg. 3.9) to account for sensor placement shifts. In each session, participants performed two repeti- tions of prompted stages (gesture categories), with each stage lasting 45 to 60 seconds. Annotations are available at this recording level, which we utilized as the gesture class labels. Each stage con- tains either mix of discrete gestures freeform unprompted movements. To ensure broad range of postures, participants were explicitly instructed to move their hand from right-to-left and up-and un-and- down while performing specific gestures.
A.2.2 DETAILS OF GESTURES
In-distribution (4 classes). Din contains approximately 80 sessions per class from 20 held-out users.
class 0: Thumb swipes whole hand; class 1: Hand claw, grasp, and flicks. class 2: ThumbsUp-
Down, ThumbRotations; class 3: FingerPinches, SingleFinger, PinchesMultipl;;
Unseen (4 classes). Dunseen the same 20 held-out users, with approximately 80 sessions each for classes 0-2 and 40 sessions for class
class 0: HookEmHoms, OK, and Scissors; class 1: Shaka and Vulcan peace; class 2: Counting up/down face side away; class 3: Counting up/down with finger wiggling and spreading.
A.3 VISUALIZZATION OF REPRESENTATIONS.
We visualize EMG embeddings before and after applying EMBridge using t-SNE (Van der Maaten & Hinton| 2008). and compare them with anchor pose embeddings from the pre-trained Pose-MAE. Points are colored by gesture classes. Before EMBridge, EMG embeddings (pre-trained with MAE), show mixed distributions across classes. After EMBridge, the EMG embeddings become more structured and separable across classes. As we observe in the pose space, some overlap between classes remains, which reflects miero-gestures within 2-second windows that share semantic simi- larity across gesture categories. This highlights both the improved alignment achieved by EMBridge and the intrinsic difficulty of gesture classification this dataset.
14

---

Published as a conference paper at ICLR 2026
Figure 5: Example visualizations of gestures used in gesture classification tasks.
Belor EMilnngge
After EMIB rége
Archor Pose Emzedd
Figure 6: t-SNE visualization of embeddings from in-dist. gestures, colored by gesture class labels.
A.4 FEW-SHOT EVALUATION OF EMBRIDGE
We evaluate the few-shot performance of EMBridge by gradually increasing the number of training samples within each class (n-shot) during linear probing. For each number of shots, repeat random sampling five times to obtain a more reliable estimate of performance. We report the average balanced accuracy, with the standard deviation indicated 50% of the probe-training data (40 shots), EMBridge can achieve performance almost comparable that of the full set on unseen gestures and surpasses all baselines trained on the complete dataset.
shaded region in Figure X With only
Defon DuSelgee
APb C143--5e
Anchol Posa Erbwoding
Component
Figure 7: t-SNE visualization of embeddings from unseen gestures, colored by gesture class labels.
15

---

Published as a conference paper at ICLR 2026
0.8
Balanced Accuracy
In-dist
full Unseen (80-shot)
0.4
0.3
10
20
30
40
50
Number of shots (samples class)
Figure 8: Few-shot evaluation of EMBridge. X-axis is the number of training samples within each class (n-shot) during linear probing. For each number of shots, repeat random sampling five times to obtain more reliable estimate of performance. We report the average balanced accuracy, with the standard deviation indicated as shaded region.
A.5 VISUAALIZATION OF SOFT TARGETS CONSTRUCTED IN CASCLE
We visualize the soft targets examine how the temperature and the number of top-k clusters ki influence the resulting probability distribution. shown in Figure a smaller т, produces sharper distribution, where probability mass is concentrated the model places high confidence on small number of nearest pose neighbors, approaching a hard- label regime. In contrast, a larger leads to smoother distribution with more evenly distributed weights, reflecting greater uncertainty and incorporating information from a broader set of pose samples. For the number of clusters ke, larger values expand the set of contributing clusters and potentially diluting the impact the most similar samples.
few dominant samples. This indicates that
A.6 IMPLEMENTATION DETAILS
We use 2s windows sampled at kHZ for both pose and EMG. EMG instance-normalieed. band- pass filtered (2-250 Hz), and notch-filtered at 60 Hz. Following (Salter et al. 2024). we apply channcl-rotation augmentation to EMG. Our MAE is encoder-decoder Transformer model with 4 encoder layers and 2 decoder layers, and the embedding dimension is d-256. We optimize with AdamW (Loshchilov Hutter 2019) 4e-4, weight decay 1e-5) and cosine annealing with warm restarts (Loshchilov & Hutter 2017). Token length is S0200 for pose and 5=50 for EMG, producing non-overlapping tokens along time. Mask ratio 0.5. Each MAE trained for 100 epochs. PoseT is encoder-decoder transformer model that consists of 4 encoder layers and decoder layers, trained using the same losses adopted in /alter et al. (2024).
For CPEP, attach I-layer projection head (hidden size 256) to the EMG encoder and train the EMG encoder plus projection head while keeping the pose encoder frozen. More details about CPEP be found in Cui et al. (2025). The contrastive temperature
learnable (initialized to 0.02). All output embeddings are l- normomaal All model trainings are conducted on 4x NVIDIA V100 GPUs; end-to-end training of each model takes approximately 6 to 8 hours. CPEP, Q-Former, and
16

---

Published as a conference paper at ICLR 2026
5. 0.1, 4
2 Targets
10
0.8
22
20
13
30
20
10
30
20
30
30
10
30
- 0.I.
10
5 --055
5. (1.7.
Targets
Target
Sort Targers
1.0
1.0
08
36
50
sampoe
Figure 9: Soft targets from batch of 64 samples for clearer visualization. We vary the value of temperature т, and the number of top-k clusters ke.
EMBridge are all trained for 40 epochs. N. 138 clusters computed using K-means from the anchor pose embeddings. The output embedding size used in linear probing is 256. Batch size is 256 for all models. Increasing batch size (512, 1024) for CPEP and Q-Fommer will degrade the performance, but EMBridge's s performance remains robust as shown Table
Table 4: Impact of batch size on zero-shot classification performance of unseen gestures.
Method
Batch Size 256
Batch Size 512
Batch Size 1024
CPEP
0.481
0.439
0.433
Q-Former
0,498
0,464
0.436
EMBridge (Ours)
0.528
0.525
0.523
In zero-shot retrieval, precompute pose embeddings for the entire corpus and, for each EMG query, retrieve the top-k neighbors by cosine similarity with k=10. The predicted label the ma- jority vote of the retrieved labels. For linear probing emg2pose benchmark models, replace the final decoding layer with a randomly initialized linear layer for classification, we average the embeddings at each timepoint as input to the classification layer.
17


Reviews:
[{"review_id": "TBRwfYbZSV", "reviewer": "Reviewer_VnDF", "summary": "This paper focuses on gesture generalization from EMG signals and pose. The authors propose EMBridge, a cross-modal representation learning framework consisting of three components: (1) a querying transformer that extracts pose-informative features from EMG signals and aligns them with pose features using an asymmetric setup, (2) a masked pose reconstruction loss to enrich the pose representations, and (3) a community-aware soft contrastive learning objective to account for varying pose similarities that are not captured by standard contrastive learning. The authors conduct a detailed ablation study on different components of the framework, evaluate performance on both in-distribution and unseen gesture classification tasks, and demonstrate improvements over baseline methods.", "questions": "- What is the motivation behind using an asymmetric setup with the pose estimator and EMG encoder? Have the authors experimented with a setup where the pose estimator is also trained during the contrastive learning process (in the Query Transformer)?\n\n- Regarding the relationship between paired gestures in experiments, are they independent or combinatorial? If they are combinatorial (for example, open hand + down), is this relationship captured in the pose similarity matrix, or could it be included as prior knowledge?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written, with clear motivation and challenges. It addresses an important and interesting problem in EMG-based gesture generalization to achieve zero-shot gesture classification. The proposed framework is original, well thought out, and carefully designed to meet domain-specific needs, such as incorporating pose communities for community-aware soft contrastive learning, which is particularly interesting. The experimental section is detailed and includes ablation studies on different components of the framework, along with analyses on three EMG datasets.", "weaknesses": "I do not see any major weaknesses in this work. However, a more detailed discussion of the limitations and potential future directions would strengthen the paper. While reproducibility details are sufficient, sharing the code in the future would further benefit the community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WWCItfdyfU", "reviewer": "Reviewer_jAkZ", "summary": "This paper investigates the task of hand gesture recognition. The authors propose a cross-modal representation learning framework that bridges the semantic gap between EMG representations and poses to achieve zero-shot gesture generalization. Specifically, they introduce a Q-Former, a masked pose reconstruction loss, and a community-aware soft contrastive learning objective to enhance EMG representation learning. Extensive experiments are conducted to validate the effectiveness of the proposed framework.", "questions": "1. In Appendix A.6, the authors claim that batch size does not affect the proposed EMBridge, while it degrades the performance of CPEP and Q-Former. Do these conclusions come from experimental results?\n\n2. The evaluation is conducted on the emg2pose and NinaPro datasets. Only four gestures are used in the unseen setting, would this setup reduce the difficulty of the zero-shot evaluation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed cross-modal representation learning method for hand gesture recognition is a novel and interesting attempt.\n\n2. The proposed cross-modal representation learning strategy effectively improves the quality of EMG representations.\n\n3. The authors proposed a community-level structural similarity framework for soft contrastive learning.\n\n4. The authors performed zero-shot classification on both in-distribution and unseen gesture categories.", "weaknesses": "1. The proposed community-aware soft contrastive learning mainly stems from previous work on contrastive learning. However, the motivation for introducing community-level structural similarities is not well explained.\n\n2. The proposed method is only validated on the pose–EMG paired data. Could the proposed method also be applied to RGB–EMG data?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hnDwKefp4G", "reviewer": "Reviewer_fYtB", "summary": "This paper presents a cross-modal framework that aligns EMG signals with hand pose embeddings to enable zero-shot gesture recognition. The approach is technically solid and yields consistent improvements.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The proposed solution represents a well executed and effective integration of recent state-of-the-art multimodal representation learning approaches, such as BLIP-style Q-Formers, into the EMG–pose alignment setting. The framework adapts these techniques in a way that is coherent and practically meaningful for wearable gesture recognition.\n2.The experimental results are comprehensive and show consistent improvements across multiple datasets and evaluation protocols, including both in-distribution and zero-shot gesture classification, demonstrating the practical effectiveness of the proposed method.", "weaknesses": "1.While the proposed Community-Aware Soft Contrastive Learning objective introduces a soft alignment mechanism, the community structure itself is derived from a hard K-means clustering, which inherently imposes discrete partitions on the pose embedding space. This feels contradicting the goal of modeling continuous semantic similarity. Should a probabilistic model such as a Gaussian Mixture or kernel-based affinity estimation better capture soft neighborhood structures?\n2.The construction of communities purely from geometric proximity in the pose latent space overlooks available semantic priors about gesture types. Integrating gesture semantics (e.g., coarse action categories) or using supervised/semi-supervised clustering could produce more meaningful communities and stronger cross-modal supervision.\n3.While the paper is well structured and the experimental settings are clearly described, the background context for this specific application domain remains somewhat underdeveloped. In particular, the work would benefit from a clearer definition and motivation of the pose modality (e.g., what exactly constitutes a “pose” sample, how gestures are defined, and what practical application scenarios are targeted). For such a specialized domain, providing stronger domain background and contextual grounding would make the contribution easier to interpret and appreciate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)