Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation
Authors: Edward Fish
Richard Bowden
Venue: NeurIPS 2025 poster ()
Keywords: Sign Language Translation
Graph Neural Networks
Contrastive Learning
Primary area: applications
TL;DR: Hyperbolic contrastive regularisation improves skeletal based sign language translation above RGB baselines.

Abstract: Recent progress in Sign Language Translation has focussed primarily on improving the representational capacity of large language models to incorporate sign-language features. This work explores an alternative direction: enhancing the geometric properties of skeletal representations themselves. We propose Geo-Sign, a method that leverages the properties of hyperbolic geometry to model the hierarchical structure inherent in sign language kinematics. By projecting skeletal features derived from Spatio-Temporal Graph Convolutional Networks (ST-GCNs) into the Poincaré ball model, we aim to create more discriminative embeddings, particularly for fine-grained motions like finger articulations. We introduce a hyperbolic projection layer, a weighted Fréchet mean aggregation scheme, and a geometric contrastive loss operating directly in hyperbolic space. These components are integrated into an end-to-end translation framework as a regularisation function, to enhance the representations within the language model. This work demonstrates the potential of hyperbolic geometry to improve skeletal representations for Sign Language Translation, improving on SOTA RGB methods while preserving privacy and improving computational efficiency.

Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation
Edward Fish CVSSP, University of Surrey edward. fish@surrey. ac.uk
Richard Bowden CVSSP, University of Surrey r. bowden@surrey. ac. uk
Abstract
Recent progress in Sign Language Translation (SLT) has focussed primarily improving the representational capacity of large language models to incorporate Sign Language features. This work explores alternative direction: enhancing the geometric properties of skeletal representations themselves. We propose Geo- Sign. method that leverages the properties of hyperbolic geometry to model the hierarchical structure inherent in sign language kinematics. By projecting Graph Convolutional Networks
skeletal features derived from Spatio-Tempone (ST-GCNs) into the Poincaré ball model, we aim to create more discriminative embeddings, particularly for ane-grained motions like finger articulations. We introduce hyperbolic projection layer, a weighted Fréchet mean aggregation scheme, and a geometric contrastive loss operating directly in hyperbolic space. These components are integrated into an end-to-end translation framework as a regularisation function, to enhance the representations within the language model. This work demonstrates the potential of hyperbolic geometry to improve skeletal representations for Sign Language Translation, improving on SOTA RGB methods while preserving privacy and improving computational efficiency. Code available here: https://github.com/c
Introduction
Sign Languages are rich, multi-channel linguistic systems where meaning is conveyed through a com- position of movements involving the upper body, hands, face, and mouth. Automatic Sign Language Translation (SLT) is an established research area focused on developing methods to convert these visual expressions directly into text. While Sign Languages kinematics, a persistent challenge for SLT methods lies in creating feature representations that con- currently preserve fine- ine-grained, local details (e.g., subtle finger configurations) while embedding the global structure inherent in larger, overarching body motions. Effectively modelling these multi-scale and relational dynamics within suitable geometric embedding space remains central hurdle.
expressed via fluid multi-articulator
Graph Convolutional Networks (ST-GCNs) offer a natural way to encode these hierarchical relationships by treating the body's joints and bones nodes and edges in a graph [79]. However, when their learned representations are projected into standard Euclidean geometry for processing via Large Language Model (LLM), essential fine-graines relational distances and movements can become blurred. For instance, the sign for "water" in American Sign Language (ASL) is communicated by forming W shape with the fingers and tapping the chin twice (a -raineddd "leaf-level" articulation), immediately followed by a sweeping hand movement away from the body "branch-level" gesture). When these features are aggregated in Euclidean geometry, the large translation and rotation of the wrist could dominate the vector's norm, effectively "pulling" the embedding toward the global motion and compressing the subtle finger tap into vanishing tail.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

PooTed
Token
RN ST-GON
pis, (fr)) (ou)
softmax
d(p
Hyp
P. ST GON
JV & my) iRt
ST-GCN
exp map
Poo
Pool ed
Linear
Duston
8
2
Translate: Today 1s the..
MT5
Figure 1: Geo-Sign hyperbolic framework: (Left) Skeletal features from ST-GCN's for different body parts are projected into Poincaré ball whose curvature learned, while the original branch fuses the features for processing via the MT5 language model. (Pooled) The pose features aggregated via Frechet Mean in 1, while the text embeddings from the final layer of the MT5 model are pooled and projected to the hyperbolic manifold. Geodesic distance between the text embedding and the mean pose features are minimised for positive samples using the contrastive loss in 5q (Token) Alternatively, hyperbolic pose features are used attention queries against all text embeddings to generate a pose-contextual text embedding. Note the movement of the text features Cpé in grey towards the pose feature in blue. (Right) representation of the Poincaré disk demonstrating the difference between Token, and Pooled methods in the tangent space.
Consequently, two signs that differ only in the timing or precision of that tap. which may be critical lexical meaning, can become nearly indistinguishable once projected into flat Euclidean space.
Large vision-based models [22, 23, 26] appear to be able to implicitly learn these hierarchical structures through extensive video pre-training and visual inductive biases. However, they do so at significant computational cost and with privacy concerns, as they retain identifiable facial and background details that skeletal representations inherently discard.
This work introduces hyperbolic geometry as a means to fundamentally enhance skeletal representa- tions for SLT. Unlike Euclidean space, where volume grows polynomially with radius and flatten hicrarchical structures, hyperbolic manifolds exhibit exponential volume growth. This property is naturally suited to encoding the compositional, tree-like structures found in sign language kinematics.
As illustrated in Figure 1, in the Poincaré ball model [B (with curvature x = - < 0), distances between points near the boundary expand exponentially relative to their Euclidean separation. This provides ample "space" to distinguish nuanced motions (c.g., an open versus a closed fist), while re- gions near the origin behave more like Euclidean space, suitable for representing broader phrase-level semantics. A key aspect of our approach that we learn the curvature parameter e end-to-end via Riemannian optimization. This allows the manifold to dynamically adapt its "zoom level": more negative curvature и (larger c) amplifies the separation of fine- ane-grained motions, whereas smoother curvature helps preserve sentence-level coherence.
Geo-Sign leverages this geometric inductive bias through a novel regularisation framework for pre-trained mT5 model [78]. By projecting skeletal features into hyperbolic space and aligning them with text embeddings a geometric contrastive loss, we guide the mT5 model internalize the hierarchical nature of sign language kinematics. Our primary contributions include:

---

Hyperbolic Skeletal Representation: We map multi-part skeletal features, derived from ST-GCNs, into the Poincaré ball using curvature-aware hyperbolic projection layers.
Geometric Contrastive Regularisation: We introduce a contrastive learning objective that operates directly in hyperbolic space, minimizing the geodesic distance between semantically corresponding hyperbolic pose and text embeddings.
Hierarchical Aggregation and Alignment Strategies: We explore two main strategies for this
contrastive alignment:
1. A global semantic alignment method, which uses a weighted Fréchet mean aggregate part-specific hyperbolic embeddings into a single global pose representation, then aligns this with global text embedding.
2. A fine-grained part-text alignment method, which employs novel hyperbolic attention mech- anism. This allows individual pose part embeddings to attend to specific text tokens within the hyperbolic space, generating contextual text embeddings for more detailed contrastive learning.
This geometric regularisation offers several advantages. aims inform the mT5 model's is under- standing by providing representations that inherently respect kinematic hierarchy. The learnable curvature allows the model to adapt the representational space to dataset-specific characteristics. Furthermore, by relying solely on anonymized skeletal data, our approach inherently preserves signer privacy and offers greater computational efficiency compared to methods requiring extensive video processing.
Experiments on the CSL-Daily benchmark [88] demonstrate Geo-Sign's efficacy. Our skeletal-based approach not only achieves a +1.81 BLEU4 and +3.03 ROUGE score over state-of-the-art pose-based methods but also matches the performance of comparable vision-based networks. We demonstrate that our method extends to American Sign Language and Isolated Sign Language Recognition. Finally, we also present the first method surpass SOTA gloss based methods (with respect the ROUGE score) with a gloss-free approach, highlighting the potential of geometrically representations.
Related Work
Our work intersects with several research areas: Sign Language Translation (SLT), the use of skeletal data for sign and action recognition, and the application of hyperbolic geometry in machine learning.
2.1 Sign Language Translation (SLT)
Sign Language Translation aims to bridge the communication gap between Deaf and hearing com- munities by automatically converting sign language videos into spoken or written language text [4, 19, 22, 64, 74]. Distinct from Sign Language Recognition (SLR), which often focuses on isolated signs gloss transcription [6, 28, 59, 65, 84), SLT tackles the more complex task of translating continuous signing across modalities with potentially disparate grammatical structures.
Early SLT methods often involved a two-stage process: recognizing sign glosses (individual lexical units of sign language grammar) and then translating the gloss sequence into the target language [6, 25, 27, 49, 49, 50, 69-72, 89]. However, this intermediate representation can lead to information loss, while gloss transcriptions are limited in availability. Consequently, end-to-end sequence-to- sequence models have become the dominant paradigm [5]. Initial approaches utilized Recurrent Neural Networks (RNNs) like LSTMs or GRUs, often with attention mechanisms [21, 63]. More recently. Transformer architectures [6] have demonstrated superior performance in capturing long- range dependencies and context [32, 62, 91], enabling direct video-to-text translation [9, 15, 22, 68, 74]. Many recent state-of-the-an architectures leverage large pre-trained language models, such as T5 variants, fine-tuned for the task of SLT [11, 89]. These often rely on large pre- trained visual encoders, with incremental improvements seen by upgrading visual backbones from ResNet [87], to I3D [67], and more recently to ViT variants like DINO [74, 75]. However, as these backbones increase in size, they can limit the number of frames processed concurrently due to quadratically scaling resource demands.
Key challenges in SLT remain, including the scarcity of large-scale annotated datasets [1, 37]. handling signer variability, modelling linguistic divergence between sign and spoken languages [12, 73], capturing co- oo-articlation effects [92], and distinguishing visually similar signs [17].
3

---

2.2 Skeletal Representations for Sign Language and Action Recognition
Using skeletal keypoints, extracted via pose estimation algorithms like OpenPose [8], MediaPipe [46], or MMPose [13] (in this work we use RTMPose for skeletal features [33]), offers several advantages RGB video for sign language analysis. Skeletal data is computationally efficient, robust to background and lighting variations, directly encodes articulation kinematics, enhances privacy by design, and can potentially improve generalization across different signers and environments 31,57, 92].
Graph Convolutional Networks (GCNs) and particularly Spatio-Temporal GCNs (ST-GCNs) have shown great promise by explicitly modelling the spatial structure of the skeleton and its temporal dynamics [14, 58, 76, 77, 79]. However, the quality of skeletal data is heavily dependent the accuracy of the underlying pose estimation algorithms [30]. Furthermore, skeletal data might discard subtle visual cues present in RGB video that could be important for disambiguation, While multi-moda fusion (RGB + pose) has been explored to combine the strengths of both modalities [54, 64, 75, 90], it typically increases computational cost. Our work focuses enhancing the representational power skeletal data itself by embedding it in hyperbolic space, aiming to improve its discriminability for SLT without resorting RGB fusion.
2.3 Hyperbolic Geometry in Machine Learning
Hyperbolic geometry, characterized by its constant negative curvature, offers unique properties for representation learning [18, 61]. Its most notable feature is the exponential growth of volume with radius, which allows hyperbolic spaces to embed tree-like hierarchical structures with significantly lower distortion than Euclidean spaces. This makes them particularly suitable for data where such latent hierarchies are believed to exist. Common models of hyperbolic geometry used in machine learning include the Poincaré ball model [51] and the Lorentz (or hyperboloid) model [52].
2.4 Hyperbolic Representation Learning Applications
The advantagcous properties of hyperbolic spaces for modelling hierarchies have led to their success- ful application in various domains. Hyperbolic Graph Neural Networks (HGNNs) have extended GNN principles to hyperbolic space, demonstrating strong performance on graph-related tasks, es- pecially those involving scale-free or hierarchical graphs [44, 81]. In Natural Language Processing (NLP), Poincaré embeddings [52] effectively captured word hierarchies (e.g.. WordNet taxonomies), leading the development of hyperbolic RNNs and Transformers for improved modeling of se- quential and relational data [82]. Applications in computer vision include hyperbolic Convolutional Neural Networks (CNNs) [2] and vision-language models that leverage hyperbolic spaces to better align visual and textual concept hierarchies [29].
Our work contributes to this growing body of research by applying hyperbolic representation learning specifically to the domain of skeletal Sign Language Translation. While hyperbolic geometry has been explored for general action recognition from skeletons [16, 38, 40] and in broader NLP contexts [48], its systematic application to enhance the discriminability of multi calti-pprt skeletal features for end-to-end SLT, particularly through geometric contrastive loss operating in hyperbolic space to regularize a large language model, represents a novel direction. We aim to leverage the geometric properties of the Poincaré ball to refine skeletal representations they processed by the language model, thereby improving the translation quality, especially for signs involving fine-grained hierarchical motion.
3 Methodology
Geo-Sign regularises pre-trained mT5 model [78] by integrating hyperbolic geometry to capture the hierarchical nature of sign kinematics. We employ the diyp-dimensional Poincaré ball model,
B.a = {x E Rd ||||| < 1/ve). with learnable curvature magnitude > 0. This section first briefly introduces essential hyperbolic operations, then details our pose encoding, hyperbolic projection, and two distinct contrastive alignment strategies.

---

3.1 Hyperbolic Geometry Essentials
Hyperbolic spaces exhibit exponential volume growth (v) oc e(d-1)r for large radius r), making them adept embedding hierarchies with low distortion compared to Fuclidean spaces (Væ(r) r") [18, 51J. In the Poincaré ball, geometry near the origin ([|x]] R 0) is approximately Euclidean, while near the boundary (||x||2 1/ /C), distances magnified, providing capacity to distinguish fine details.
The geodesic distance dz. (u, v) between points I is:
ds. (u, v) artanh (vell(-u) t vll2)
(I)
Vc
This utilizes Möbius addition We. the hyperbolic analogue of vector addition:
(1 + u,, v)2 + cllv +
cl u
(2)
2 v)2 + c²
To map Euclidean vectors v from the tangent space at the origin the exponential map the origin expī(-):
To R into B we use
expo (v) tanh \Vdlvvl2
v
(v + 0).
L/ |v|l2
Its inverse is the logarithmic map at the origin, (.). General maps exp&() and log (.) facilitate
operations at arbitrary points x B
3.2 Skeletal Feature Extraction and Hyperbolic Projection
3.2.1 ST-GCN Backbone
We process 2D skeletal keypoints extracted using RTM-Pose [33], partitioned into four anatomical groups (body, left/right hands, face). Each group is processed by part-specific ST-GCN [79] which combines spatial graph convolutions with temporal convolutions to model both joint interdependencies and motion dynamics. Residual connections allow information flow from body joints to hand/face
representations. The ST-GCNs output part-specific feature maps Zp Txxxdpn.cm (T is sequence length). For direct input to the mT5 encoder, these are concatenated and linearly projected to dmrs yielding dynamic Euclidean pose embeddings Epose Txxds For the hyperbolic regularisation
branch, each Zp is temporally mean-pooled to static summary vector fp RR "", capturing the overall kinematics of part p.
3.2.2 Part-Specific Projection to Poincaré Ball
Each Euclidean summary vector f, is projected to a hyperbolic embedding hp B This projection involves a linear transformation of f, to dimension dmyp using a learnable matrix WP, followed by multiplication with learnable positive scalar sp. This scalar 8p adaptively scales the features in the tangent space, allowing the model to place features from parts with varying motion scales at appropriate "depths" in the hyperbolic space. The resulting tangent vector is then mapped onto the Poincaré ball using the exponential map at the origin (Eq. 3):
hp expÊ (sp W
The set of hyperbolic part embeddings {hp} forms the input for the subsequent alignment strategies.
3.3 Hyperbolic Contrastive Loss
We regularize the mT5 model by minimizing a Geometric Contrastive Loss, adapted from InfoNCE [53], between hyperbolic pose and text embeddings. This loss encourages semantic consistency by pulling corresponding pose-text pairs closer in hyperbolic space while pushing non-corresponding pairs apart. For batch of B pose embeddings (p,) and text embeddings (t,} in Bd the loss for positive pair (p,, t;) is:
exp( dB. (p.. t/)T)
Luyp (Pr, t;) llo
(5)
D exp(- da, (pr, t,)/T + T(i ( f))
5

---

Here, т > 0 is learnable temperature scaling the similarities (negative distances), and m 2 0 is learnable additive margin for negative pairs. The total regularisation term Lhyp_re the batch average of Lhyp.pair
3.4 Alignment Methods
We present two methods for selecting features for alignment which offer benefits and trade-offs. The first takes the geometric mean of the pose and the text embeddings which comes at greater computational efficiency but decreased accuracy. The second method uses poses as individual queries over the text embeddings hyperbolic space. We then compute the distance between each pose and modified token pair. This improves translation accuracy but incurs additional memory and inference costs (Full details provided in the appendix.)
3.4.1 Strategy 1: Global Semantic Alignment (Pooled Method)
This strategy aligns holistic pose and text semantics, promoting high-level understanding.
Pose Embedding (p): A global hyperbolic pose plpoee E IB dad is computed as the weighted Fréchet mean of the part embeddings {hp]. The Fréchet mean is geometrically sound average hyperbolic space. Weights wp x cxp(ds, (0, hp)). normalized via softmax, emphasize parts with more distinct hyperbolic embeddings. The mean is found iteratively (Algorithm 1) us- ing general logarithmic maps log (.) and exponential maps expo (-) for tangent space computations.
Text Embedding (t): A global hyperbolic text embedding hext E Bhe is obtained by mean- pooling Euclidean token embeddings (e.g., from mT5 decoder's final layer) and then projecting this single sentence vector to Re using a hyperbolic projection layer (structurally similar to Eq. 4).
The contrastive loss Laypureg (Eq. 5) then computed between the sets of these global pose embed- dings { and global text embeddings (hwat,
Algorithm 1 Iterative Weighted Fréchet Mean in !
Require: Hyperbolic embeddings {bp) normalized positive weights {wp) Imax Etol-
1: Initialize (0) h1 (or other suitable initialization).
2: for k = 0 to I
do 3: Vage + 0 E T B dur
Aggregated tangent vector at current mean
4: for p to N do
5:
Vagg - "gg + u log (hp). end for
b Sum weighted log-mapped vectors
6:
7:
(k+11
(1te (es (Vage).
Update mean via exponential map
8: Project u (k++) into Bd numerically necessary.
9: if da, (n (1()+)), pe(t)) < Gal then
Check convergence b> Exit loop on convergence
10: break
11: end
12:
end for
13:
Mpuse <- 1 (k+11
Assign final mean
Ensure: Estimated Fréchet mean Mpose
3.4.2 Strategy 2: Fine-Grained Part-Text Alignment (Token Method)
This strategy aligns individual pose parts (Queries h2)) with relevant text segments via hyperbolic attention. For each h, E TB (from Eq. 4), specific context vector Cp E Ba is computed.
Tokenization:
First, the sequence of T Fuclidean text token embeddings, (ex GF is projected
into the Poincaré ball. This yields a sequence of hyperbolic embeddings V = [n, I= which serve the values and hold the original semantic meaning of each token.
1=1

---

Key Transformation: The value sequence V is transformed by a learnable Möbius affine transfor- mation (using M and b) to create a sequence of keys K [ka)
k (M v. + b
(6)
Attention Scores: Scores Spt are computed as the negative geodesic distance between each pose query hp and each text key kg. A smaller distance signifies greater relevance.
Spt = d3. (hp. ks)
These scores normalized via softmax (with masking for padding) to produce attention weights Opt-- The final context vector Cp is the hyperbolic weighted midpoint (μ) of the original values V weighted by the pose-shifted attention embeddings:
Context Vector:
Cy = MB. (wr)f=I {
(8)
The final Lyypreg is the average K individual contrastive losses (Eq. 5), one for each (hp. Gp) alignment pair.
3.5 Training Objective and Optimization
The model trained end-to-end by minimizing the total loss Cisal a- + Chypree This combines the standard cross-entropy translation loss CcE (with label smoothing) with the hyperbolic regularisation term Ctyp_reg from one of the alignment strategies. The blending factor a E [0.1, 1.0] is dynamically adjusted during training via a learnable parameter and training progress, allowing an initial focus on Chyp. before increasing the influence of LeE.
Optimization employs A [35, 45] for Euclidean parameters (ST-GCNs, m'5,, linear layers), with learning rate 3 10-5 Hyperbolic parameters, including the learnable curvature e (optimized in parameters, use Riemannian Adam (RAdam) [3] with
log-space, e.g.. log c) and maaifold-constraiiig comparable learning rate. RAdam adapts updates the manifold's geometry by operating in tangent spaces, All hyperbolic computations utilize high-precision floating-point numbers (c.g., 'float32') for numerical stability. A key stabilization step before applying any exponential map exp_ (v) involves projecting the input tangent vector via v + v/ max(1, /cllv + ee for a small > 0 (e.g., 10-5), ensuring the argument well-behaved and the output point remains strictly within the Poincaré ball.
Experiments
We evaluate Geo-Sign Chinese Sign Language (CSL) and American Sign Language (ASL). For CSL we use the CSL-Daily dataset [88, 89], large-scale corpus for Chinese Sign Language to Chinese text translation, comprising over 20,000 videos. For American Sign Language we perform translation experiments on How2Sign [15] and isolated sign language recognition experiments on WLASL2000 [39].
Translation quality is assessed using BLEU [55] (B-1, B-4) and ROUGE-L [42] (R-L) scores where higher percentage represents a more accurate translation.
4.1 Experimental Setup
Our framework builds upon the Uni-Sign architecture [41], using its pre-trained ST-GCN weights (trained on skeletal features from the CSL-News dataset [41] ] for CSL and the YTASI [68] dataset for ASL) and mT5 model [78] as the language decoder. Following Uni- Uni-Sign's fine-tuning protocol, which involves 40 epochs of supervised finetuning on CSL-Daily or YTASL, with fused skeletal and RGB features, we remove the RGB encoder and instead apply our hyperbolic regularisation. This allows for fair comparison of the impact of our geometric regularisation. We investigate both the "Pooled Method" (Strategy 1) and the "Token Method" (Strategy 2) for hyperbolic alignment. To assess the specific contribution of hyperbolic geometry, we also compare against "Euclidean regularisation" baseline, where contrastive loss operates on Euclidean projections to the Poincaré ball where curvature is minimal (0.001) and approximately Euclidean. Key hyperparameters for the hyperbolic components (initial curvature e 1.5, dimension dhyp = 256, and 0.70) are minimally tuned on the development set (further details in the appendix).
7

---

Modality
Dev Set
Test Set
Method
Pose
RGB
B-1
B-4
R-L
B-1
B-4
R-L
Gloss Based Methods (Prior Art)
SLRT [6]
37.47
11.88
37.96
37.38
11.79
36.74
TS-SLT [11]
55.21
25.76
55,10
55.44
25.79
55.72
CV-SLT [85]
28.24
56.36
58.29
28.94
57.06
Gloss- Methods (Prior Art)
MSLU [90]
33.28
10.27
33.13 1.
33.97
1 11.42
33.80
SLRT [6] GGles--Free variant)
21.03
4.04
20.51
20.00
3.03
19.67
GASLT [83]
19.90
4.07
20.35
GFSLT-VLP [88]
39.20
11.07
36.70
39.37
11.00
36.44
Fl.a-LLM [10]
37.13
14.20
37.25
Sign2GPT [74]
41.75
15.40
42.36
Li [19]
42.45
12.23
39.18
39.55
15.75
39.91
CLRL [9]
49.32
21.61
48.21
Our Models and Baselines
Uni-Sign [41] (Pose) Uni-Sign [41] (Pose+RGB)
53.24
25.27
54.34
53.86
25.61
54.92
v
55.30
26.25
56.03
55.08
26.36
56.51
Geo-Sign (Euclidean Pooled)
53.53
25.78
55.38
53.06
25.72
55.57
Geo-Sign (Euclidean Token)
53.93
25.91
55.20
54.02
25.98
53.93
Geo-Sign (Hyperbolic Pooled)
55.19
26.90
56.93
55.80
27.17
57.75
Geo-Sign (Hyperbolic Token)
55.57
27.05
57.27
55.89
27.42
57.95
Sign Language Translation performance on the CSL-Daily dataset. BLEU scores (B-1, B-4) and ROUGE-L (R-L) are reported percentages (%). Higher is better. 'Pose' and 'RGB' indicate input modalities. Uni-Sign the base architecture sharing pre- geeetraining/fiee-nue taaiiinggnituuuinngggg setups but without regularisation. Euclidean regularisation applies contrastive loss in Euclidean space. Our Hyperbolic Token method surpasses all other pose-only methods and is competitive with top RGB/multinodal methods. Gloss-based methods that outperform our method are underlined.
Table I:
4.2 Results on Chinese Sign Language (CSL)
Section 4.2.1 presents our main results on the CSI.-Daily test set, comparing Geo-Sign with prior art and baselines. Our Geo-Sign (Hyperbolic Token) model, using only pose data, achieves test BLEU-4 27.42% and ROUGE-L of 57.95%. This represents a significant improvement of +1.81 BLEU-4 and +3.03 ROUGE-L over the strong Uni-Sign (Pose) baseline (25.61 15.61% % BLEU-4, 54.92% ROUGE-L). Notably, this performance surpasses all other reported gloss-free pose-only gloss-based methods,
methods and is competitive with, exceeds, several RGB-only and even underscoring the efficacy our geometric regularisation. The Geo-Sign (Hyperbolic Pooled) variant also outperforms the Euclidean regularisation methods and the Uni-Sign pose baseline, demonstrating the general benefit of hyperbolic geometry. The "Fuclidean Token" regularisation already shows improvement over the Uni-Sign baseline, suggesting the contrastive alignment itself is beneficial, but the further gains from hyperbolic geometry are substantial.
4.2.1 Results American Sign Language
In Table show results on Sign Language Translation for American Sign Language on the How2Sign [15] dataset. Our method shows increased performance over all pose based methods but performs marginally worse than the best RGB method [60] which benefits from a longer pre-training duration and scale. In Table 3 we also compare our approach on Isolated Sign Language Recognition (ISLR) with the WLASL2000 [39] dataset. For isolated recognition our method shows small improvement in Top-1 Accuracy for both instance (40.12) and class-level (+0.57).
4.2.2 Ablation Studies
Ablation studies on the CSI.-Daily test set for our best performing Geo-Sign (Hyperbolic Token) model are presented in Table 4. We investigate the impact of the initial hyperbolic curvature and

---

Table 2: Sign Language Translation (SLT) results on the How2Sign dataset. Metrics are BLEU (B-1, B-4) and ROUGE-L (R-L). Higher is better.
Table 3: Isolated Sign Language Recognition
(ISLR) results on WLASL2000. We report
Top-1 I Accuracy for Per-Instance (P-I) and Per-
Class (P-C). from [23].
Modality
Test Set
Method
Mod.
Test Ace. (%)
Pose
RGB
B-1
B-4
R-L
Method
P
RGB
P-1
P-C
Gloss-Free Methods (Prior Art)
GloFE-VN [43]
14.9
2.2
12.6
Prior Art
YouTuhe-ASL. [68]
37.8
12.4
ST-GCN' [80]
34,40
32.53
MSLU [90]
20.1
2.4
17.2
SignBERT [23]
39.40
36.74
SLT-IV [67]
34.0
8.0
HMA [24]
37.91
35.90
C²RL. [9]
29.1
9.4
27.0
BEST [86]
46.25
43.52
Fl.--I.M [10]
29.8
9.7
27.8
SignBERT+ [26]
V
48.85
46.37
SignMusketeers [20]
41.5
14.3
MSLU [90]
V
56.29
53.29
SSVP-SLT [60]
43.2
15.5
38.4
NLA-SLR [93]
61.05
58.05
Sign-Rep [75]
61.05
58.89
Our Models and Baselines
Uni-Sign (Pose)
40.4
14,5
34.3
Our Models
Uni-Sign (Posc+RGB)
V
40.2
14.9
36.0
Uni-Sign (Pose)
63.13
60.90
Geo-Sign (Token)
40.8
15.1
35.4
Uni-Sign (Pose+RGB)
63.52
61.32
Geo-Sign (Token)
63.64
61.89
Table 4: Ablation studies for Geo-Sign on the CSL.-Daily test set, examining (a) initial curvature (b) loss blending factor a, and (c) robustness of poses to Gaussian noise. We demonstrate that hyperbolic regularisation improves robustness to pertubation and poor pose estimation.
(a) Impact of Curvature c.
(b) Impact of c.
(c) Noise Robustness (B-4).
Curvature (c)
B-4
R-L
a
B-4
R-L
Noise
Geo-Sign
Uni-Sign
0.00 (Euclidean)
25.98
53.93
0.10
25.74
56.20
0.00
27.42
26.25
0.10
26.56
57.56
0.50
26.79
57.38
0,01
26.30 (-4%)
24.14 (-8%)
0.50
26.34
56.30
0.70
27.42
57.95
0.02
24.60 (-10%)
21.50 (-18%)
1.00
27.04
57.67
0.90
26.92
57.67
0.03
19.07 (-30%)
14.40 (-45%)
1.50
27.42
57.95
0.04
11.63 (-58%)
7.20 (-73%)
2.00
27.25
58.08
0.05
5.98 (-78%)
3.01 (-89%)
the loss blending factor a. For curvature c (with a = 0.7), setting c= 0.001 effectively makes the projection Euclidean (as tanh(x) x for small x, which means almost zero hyperbolic warping). observe that increasing curvature from this Euclidean-
baseline (c = 0.001, BLEU-4 25.91 1%) generally improves performance, with optimal BLEU-4 (27.42% achieved peaks at c = 2.0 (58.08%) though BLEU-4 slightly dips to 27 25%, suggesting a trade-off. This indicates that significant degree of negative curvature is beneficial for capturing sign language structure. For the loss blending factor a (with 1.5), a value of a = 0.7 (i.e., 30% weight to the hyperbolic loss) yields the best BLEU-4 (27.42% and ROUGE-L (57.95%). Lower higher a values result in decreased performance, indicating that the hyperbolic regularisation provides a substantial complementary signal the primary translation loss. but should not entirely dominate it during the 40 epochs of fine-tuning. Finally, we assess the robustness of our approach to pose perturbation and poor pose estimation. To do so, we add Gaussian noise the pose embeddings before embedding via the ST-GCN. Our method shows how hyperbolic regularisation improves robustness pose noise compared the Uni-Sign Euclidean baseline. We attribute this to the larger geodesic margins between pose embeddings in the hyperbolic space, making the model less susceptible to noisy perturbation.
1.5. ROUGE-L
4.3 Qualitative Analysis: Visualizing Embedding Spaces
To intuitively understand the effect of hyperbolic regularisation, we visualise the learned pose embeddings. Figure shows UMAP [47] projections of these embeddings into the 2D Poincaré disk (by log-mapping hyperbolic embeddings to the tangent space the origin, then applying UMAP). We compare embeddings from our Geo-Sign (Hyperbolic Token) model against those from the

---

Geo-Sign (Euclidean Token) model, which uses the same contrastive token-level alignment but without hyperbolic projection (curvature c = 0).
The Euclidean embeddings (Figure 2, Left) appear relatively clustered and undifferentiated. In contrast, the hyperbolic embeddings (Figure 2, Right) exhibit
structured distribution. Notably, embeddings corresponding to hand articulations (often carrying fine-grained lexical information) tend to occupy regions further from the origin, towards the periphery of the Poincaré disk. This is consistent with hyperbolic geometry's 's property of expanding space near the boundary, providing more capacity to distinguish subtle variations. Conversely, features representing larger body movements or overall posture (often conveying prosodic or grammatical information) tend to be located more centrally. This visualised structure suggests that the hyperbolic model indeed learns to place features in a manner that reflects the hierarchical nature of sign kinematics. with fine details pushed to high-curvature regions and global features remaining near the low-curvature origin.
Fostoney
0.9-
Neurge Lird
0.72
Priss qet foed
6.5
Demeaker Shikoot
0.09
0.0
455
4.75
4.50
4.20
2.30
6.00
2.90
-1.13
3.50
0.29 10.9
01.5
3.00
Oseesie (1ockeed)
Figure 2: UMAP projection of pose part summary embeddings (f, onto the 2D Poincaré disk). (Left) Embeddings from the Euclidean Token regularisation model 0.001). (Right) Embeddings from the Geo-Sign (Hyperbolic Token) model. The hyperbolic embeddings show more structured distribution, with hand features (representing finer details) often pushed towards the periphery indicative of a learned kinematic hierarchy.
Conclusion
This paper introduced Geo-Sign, novel framework that enhances Sign Language Translation by leveraging hyperbolic geometry to model the inherent hierarchical structure of sign language kinematics. By projecting skeletal features from ST-GCNs into the Poincaré ball and employing a geometric contrastive loss, Geo-Sign regularises a pre-trained mT5 model, guiding it to Iearn more discriminative and geometrically aware representations. We explored two alignment strategies: a global pooled method and a fine-grained token-based attention method operating directly in hyperbolic space. Our experimental results both Chinese Sign Language and American Sign Language demonstrate the significant benefits of this approach.
Acknowledgements
This work was supported by the SNSF project 'SMILE II (CRSH15 193686), the Innosuisse IICT Flagship (PFFS-21-47) EPSRC grant APP24554 SSJnGGPTEP//23337001) and through funding from Google.org via the AI for Global Goals scheme. This work reflects only the author's views and the funders are not responsible for any use that may be made of the information it contains.
Thank you Low Jian He for reviewing the Chinese text translations.
10

---

References
[1] Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, Andrew McParland, al. Bbe-oxford british sign language dataset. In arXiv preprint arXiv:2111.03635, 2021.
[2] Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully hyperbolic convolutional neural networks for computer vision. In arXiv preprint arXiv:2303. 2..::3993./599 2023.
[3] Gary Bécigneul and Octavian-Euun Eugen Ganea. Riemannian adaptive optimization methods. In International Conference on Learning Representations (ICLR 2019), 2023.
[4] Jan Bungeroth and Hermann Ney. Statistical sign language translation. In sign-lang@ LREC 2004, 2004.
[5] Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. Neural sign language translation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
[6] Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. Sign language transformers: Joint end-to-end sign language recognition and translation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020.
[7] Necati Cihan Camgöz, Ben Saunders, Guillaume Rochette, Marco Giovanelli, Giacomo Inches, Robin Nachtrab-Ribback, and Richard Bowden. Content4all open research sign language translation datasets. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 2021.
[8] Zhe Cao, Gines Hidalgo. Tomas Simon, Shih-En Wei, and Yaser Sheikh. Openpose: realtime multi-person 2d pose estimation using part affinity fields. In TPAMI, volume 43. IEEE, 2019.
[9] Zhigang Chen, Benjia Zhou, Yiqing Huang, Jun Wan, Yibo Hu, Hailin Shi, Yanyan Liang, Zhen Lei, and Du Zhang. r Content and context representation learning for gloss-free sign language translation and retrieval. In arxiv, 2024.
[10] Zhigang Chen, Benjia Zhou, Jun Li, Jun Wan, Zhen Lei, Ning Jiang, Quan Lu, and Guoqing Zhao. Factorized learning assisted with large language model for gloss-free sign language translation. In LREC-COLING, 2024.
[11] Zhigang Chen, Benjia Zhou, Yiqing Huang. Jun Wan, Yibo Hu, Hailin Shi, Yanyan Liang, Zhen Lei, and Du Zhang. C 2 rl: Content and context representation learning for gloss-free sign language translation and retrieval. In IEEE Transactions on Circuits and Systems for Video Technology, 2025.
[12] Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, and Wenqiang Zhang. CiCo: Domain- aware sign language retrieval via cross-lingual contrastive learning. In Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[13] MMPose Contributors. Openmmlab pose estimation toolbox and benchmark, 2020.
[14] Haodong Duan, Yue Zhao, Kai Chen, Dahua Lin, and Bo Dai. Revisiting skeleton-based action recognition. In Proceedings of the IEEE/CVF international conference on computer vision, 2022.
[15] Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giro-i Nieto. How2sign: a large-scale multimodal dataset for continuous american sign language. In Proceedings of the IEEE/CVF international conference on computer vision, 2021.
[16] Luca Franco, Paolo Mandica, Bharti Munjal, and Fabio Galasso. Hyperbolic self-paced learning for self-supervised skeleton-based action representations. In arXiv preprint 2303.0233 2023.
[17] Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Kang Xia, Lei Xie, and Sanglu Lu. Contrastive learning for sign language recognition and translation. In IJCAI, 2023.
11

---

[18] Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. In Advances in Neural Information Processing Systems, volume 31, 2018.
[19] Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, and Jun Liu. Lims are good sign language translators. In Proceedings of the IEEE/CVF international conference on computer vision, 2024.
[20] Shester Gueuwon, Xiaodan Du, Greg Shakhnarovich, and Karen Livescu. SignMusketeers: An efficient multi-stream approach for sign language translation at scale. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria, 2025. Association for Computational Linguistics.
[21] Dan Guo, Wen gang Zhou, Houqiang Li, and M. Wang. Hierarchical Istm for sign language translation. In AAAI Conference on Artificial Intelligence, 2018.
[22] Zhengsheng Guo, Zhiwei He, Wenxiang Jiao, Xing Wang, Rui Wang, Kehai Chen, Zhaopeng Tu, Yong Xu, and Min Zhang. Unsupervised sign language translation and generation. In Lun-Wet Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, Bangkok, Thailand, 2024. Association for Computational Linguisties.
[23] Hezhen Hu, Weichao Zhao, Wengang Zhou, Yuechen Wang, and Houqiang Li. SignBERT: pre-training of of the IEEE/CVF international conference on computer vision, 2021.
representation for sign language recognition. In Proceedings
[24] Hezhen Hu, Wengang Zhou, and Houqiang Li. Hand sand-model-awee aware sign language recognition. In AAAI, 2021.
[25] Hezhen Hu, Wengang Zhou, Junfu Pu, and Houqiang Li. Global-local enhancement network for nmf-aware sign language recognition. In ACM transactions on multimedia computing, communications, and applications (TOMM). 2021.
[26] Hezhen Hu, Weichao Zhao, Wengang Zhou, and Houqiang Li. Signbert+: Hand-model-aware self-supervised pre-training for sign language understanding. In IEEE TPAMI, 2023.
[27] Lianyu Hu, Liqing Gao, Zekang Liu, and Wei Feng. Temporal lift pooling for continuous sign language recognition. In ECCV, 2022.
[28] Lianyu Hu, Liqing Gao, Zekang Liu, and Wei Feng, Continuous sign language recognition with correlation network. Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[29] Sarah Ibrahimi, Mina Ghadimi Atigh, Nanne Van Noord, Pascal Mettes, and Marcel Worring. Intriguing properties of hyperbolic embeddings in vision-language models. In Transactions on Machine Learning Research, 2024.
[30] Maksym Ivashechkin, Oscar Mendez, and Richard Bowden. Improving 3d pose estimation for sign language. In 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023. doi: 10. 00.1009CAASPPP55220000000000 1019299999
[31] Maksym Ivashechkin, Oscar Mendez, and Richard Bowden. Improving 3d pose estimation for sign language. In IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops. IEEE, 2023.
[32] Youngjoon Jang, Haran Raajesh, Liliane Momeni, Gül Varol, and Andrew Zisserman. Lost in translation, found in context: Sign language translation with contextual cues. In arXiv preprint arXiv:2501.09754 2025.
[33] Tao Jiang, Peng Lu, Li Zhang, Ning Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen. RTMPose: Real-time multi-person pose estimation based mmpose. In arxiv, 2023.
[34] Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, and Ping Luo. Whole-body human pose estimation in the wild. In ECCV, 2020.
12

---

[35] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[36] Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in pytorch, 2020.
[37] Oscar Koller, Jens Forster, and Hermann Ney. Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers. In CVIU, volume 141, 2015.
[38] Zhiying Leng, Shun-Cheng Wu, Mahdi Saleh, Antonio Montanaro, Hao Yu, Yin Wang, Nassir Navab, Xiaohui Liang, and Federico Tombari. Dynamic hyperbolic attention network for fine hand-object reconstruction. In Proceedings of the IEEE/CVF international conference computer Vision, 2023.
[39] Dongxu Li, Cristian Rodriguez, Xin Yu, and Hongdong Li. Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison. In WACV, 2020.
[40] Yue Li, Haoxuan Qu, Mengyuan Liu, Jun Liu, and Yujun Cai. Hyliformer: Hyperbolic linear attention for skeleton- beseeton-based human action recognition. In arXiv preprint arXiv:2502.05869 2025.
[41] Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, and Houqiang Li. Uni- sign: Toward unified sign language understanding at scale. In arXiv preprint arXiv:2501.15187, 2025.
[42] Chin- Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 2004.
[43] Kezhou Lin, Xiaohan Wang, Linchao Zhu, Ke Sun, Bang Zhang, and Yi Yang. Glofe: Gloss-free end-to-end sign language translation. In ACL, 2023.
[44] Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks. In Advances in Neural Information Processing Systems, volume 32, 2019.
[45] Uya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 2017.
[46] Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Mediapipe: A framework for building perception pipelines. In arXiv preprint (rXXv:1906.08122 2019.
[47] Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. Umap: Uniform manifold approximation and projection. In booktitle of Open Source Software, volume 3. The Open booktitle, 2018.
[48] Marko Valentin Micic and Hugo Chu. Hyperbolic deep learning for chinese natural language understanding. In arXiv preprint 18Xi: 10408 2018
[49] Liliane Momeni, Gul Varol, Samuel Albanie, Triantafyllos Afouras, and Andrew Zisserman. Watch, read and lookup: learning to spot signs from multiple supervisors. ACCV, 2020.
[50] Liliane Momeni, Hannah Bull, KR Prajwal, Samuel Albanie, Gül Varol, and Andrew Zisserman. Automatic dense annotation of harge-oocabuaag sign language videos. In ECCV, 2022.
[51] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical represen- tations. In Advances in Neural Information Processing Systems, volume 30, 2017.
[52] Maximillian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In International conference machine learning. PMLR, 2018.
[53] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, In arXiv preprint wxXiv:1807.03111 2018.
13

---

[54] Katerina Papadimitriou and Gerasimos Potamianos. Multimodal Sign Language Recognition via Temporal Deformable Convolutional Sequence Learning. In Interspeech. 2020.
[55] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002.
[56] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-ppeformance deep learning library. In Advances in Neural Information Processing Systems, 2019.
[57] Junfu Pu, Wengang Zhou, Hezhen Hu, and Houqiang Li. Boosting continuous sign language recognition via cross modality augmentation. In ACMMM, 2020.
[58] Zhaofan Qiu, Ting Yao, and Tao Mei. Learning spatio-temporal representation with pseudo-3D residual networks. In Proceedings of the IEEE/CVF international conference on computer vision, 2017.
[59] Razich Rastgoo, Kourosh Kiani, and Sergio Escalera. Sign language recognition: deep survey. In ESA, 2021.
[60] Phillip Rust, Bowen Shi, Skyler Wang, Necati Cihan Camgoz, and Jean Maillard. Towards privacy-aware sign language translation at scale. In ACL, 2024.
[61] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. In arXiv preprint arXiv:2006.00 200::0000.82000 10, 2020.
[62] Ozge Mercanoglu Sincan, Necati Cihan Camgoz, and Richard Bowden. Is context all you need? scaling neural sign language translation large domains of discourse. In Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[63] Stephanie Stoll, Necati Cihan Camgöz, Simon Hadfield, and R. Bowden. Sign language production using neural machine translation and generative adversarial networks. In British Machine Vision Conference, 2018.
[64] Shengeng Tang, Dan Guo, Richang Hong, and Meng Wang. Graph- based multimodal sequential embedding for sign language translation. In IEEE TMM, 2021.
[65] Shengeng Tang, Richang Hong, Dan Guo, and Meng Wang. Gloss semantic-enhanced network with online back-translation for sign language production. In ACM International Conference on Multimedia, 2022.
[66] Garrett Tanzer and Biao Zhang. Youtube-sl-25: A large-scale, open-domain multilingual sign language parallel corpus. In arxiv, 2024.
[67] Laia Tarrés, Gerard I. Gállego, Amanda Duarte, Jordi Torres, and Xavier Giró Nieto. Sign language translation from instructional videos. In CVPRW, 2023.
[68] Dave Uthus, Garrett Tanzer, and Manfred Georg. Youtube-asl: asl: A large-scale, open-domain american sign language-english parallel corpus. In Advances in Neural Information Processing Systems, 2024.
[69] Gul Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, and Andrew Zisserman. Read and attend: Temporal localisation in sign language videos. In Proceedings of the IEEE/CVF international conference on computer vision, 2021.
[70] Gül Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, and Andrew Zisserman. Scaling up sign spotting through sign language dictionaries. In IJCV, volume 130, 2022.
[71] Harry Walsh, Ozge Mercanoglu Sincan, Ben Saunders, and Richard Bowden. Gloss alignment using word embeddings. In 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW). IEEE, 2023.
14

---

[72] Harry Walsh, Abolfazl Ravanshad, Mariam Rahmani, and Richard Bowden, A data-driven representation for sign language production. In Proceedings of the 18th International Conference on Automatic Face and Gesture Recognition, 2024.
[73] Fangyun Wei and Yutong Chen. Improving continuous sign language recognition with cross- lingual signs. Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[74] Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. Sign2GPT: Leveraging large language models for gloss-free sign language translation. In ICLR, 2024.
[75] Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. Signrep: Enhancing self-supervised sign representations. In Proceedings of the IEEE/CVF international conference computer vision, 2025.
[76] Qinkun Xiao, Minying Qin, and Yuting Yin. Skeleton- chinese sign language recognition and generation for bidirectional communication between deaf and hearing people. In Neural networks, 2020.
[77] Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy, Rethinking spa- tiotemporal feature learning: apeed-accurase trade-offs in video classification. In ECCV, 2018.
[78] Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami AI-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mT5: A massively multilingual pre-trained text-to-text transformer. In NAACL, 2021.
[79] Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, 2018.
[80] Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, 2018.
[81] Menglin Yang. Min Zhou, Zhihao Li, Jiahong Liu, Lujia Pan, Hui Xiong, and Irwin King. Hyperbolic graph neural networks: A review of methods and applications. In arXiv preprint WxXi:22002.13852 2022.
[82] Menglin Yang. Harshit Verma, Delvin Ce Zhang, Jiahong Liu, Irwin King, and Rex Ying. Hypformer: Exploring efficient transformer fully in hyperbolic space. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024.
[83] Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, and Zhou Zhao Gloss attention for gloss-free sign language translation. In Proceedings of the IEEF/CVF international conference on computer vision, 2023.
[84] Jan Zelinka and Jakub Kanis. Neural sign language synthesis: Words are our glosses. In WACV, 2020.
[85] Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, and Yidong Chen. Conditional variational autoencoder for sign language translation with cross-modal alignment. In AAAI, 2024.
[86] Weichao Zhao, Hezhen Hu, Wengang Zhou, Jiaxin Shi, and Houqiang Li. BEST: Bert pre- training for sign language recognition with coupling tokenization. In AAAI, 2023.
[87] Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[88] Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, and Du Zhang. Gloss-free sign language translation: Improving from visual-language pretraining. In Proceedings of the IEEF/CVF international conference on computer vision, 2023.
15

---

[89] Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. Improving sign language translation with monolingual data by sign bock-translation In Proceedings of the IEEE/CVF international conference computer vision, 2021.
[90] Wengang Zhou, Weichao Zhao, Hezhen Hu, Zecheng Li, and Houqiang Li. Scaling up multi- modal pre-training for sign language understanding. In arxiv, 2024.
[91] Ronglai Zuo and Brian Mak. Local context- self-attention for continuous sign language recognition. In Proc. Interspeech, 2022.
[92] Ronglai Zuo and Brian Mak. Improving continuous sign language recognition with consistency constraints and signer removal. In ACM TOMM, volume 20, 2024.
[93] Ronglai Zuo, Fangyun Wei, and Brian Mak. Natural language-assisted sign language recognition. In Proceedings of the IEEF/CVF international conference on computer vision, 2023.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Demonstrated in results and ablation experiments.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors? Answer: [Yes]
Limitations section in the main paper and discussion in the supplementary material.
Justification:
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: Full experiment details provided in the supplementary material and code samples provided. The dataset used is open source.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Dataset open source, code samples are provided and full code and data available online.
6. Experimental setting/details
Question: Does the paper specify the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details provided in the supplementary material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We use the same training setup and hyperparameter setup as the code in Uni-Sign for fair comparison including random seed. Due compute constraints were not able to perform multiple experiments over additional random seeds.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Full details provided in the supplementary material.
17

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: htttpa:/neurrps.... httpps/.cccpplleeeheeeeeeeeeeeee
Answer: [Yes]
Justification: No conflicts.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Brief discussion on importance Sign Language Translation on improving communication between hearing and Deaf communities.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
18

---

Appendix
A Introduction
20
Hyperbolic Geometry Preliminaries: A Brief Refresher
20
C Methodology Details
21
C.I Pose Extraction and ST-GCN Architecture
C.2 Hyperbolic Alignment Strategies
22
Mathematical Foundations
24
D.1 Fréchet Mean the Poincaré Ball
24
D.2 Gradient of the Hyperbolic Distance
25
Learnable Model Parameters
25
E.1 Discussion on Learnable Curvature
25
E.2 Discussion on Loss Blending Factor alpha
26
Experimental Setup, Analysis, and Qualitative Results
26
F.1
Computational Considerations and Profiler Analysis
26
F22
Further Technical Implementation Details
30
F3
Limitations and Future Work
30
F4
Qualitative Results
31
G Code Listings
36
19

---

A Introduction
In this appendix, we provide comprehensive supplementary details to accompany our main paper. The goal is offer an in-depth understanding of methodology, experimental setup, and the underlying geometric principles, thereby ensuring clarity and facilitating the reproducibility of our work.
This document claborates on:
The specifies of pose feature extraction and the Opatio-Tempone Graph Convolutional Network (ST-GCN) architecture employed (Appendix C.I).
Detailed explanations and implementations of our proposed hyperbolic alignment strategies, including the Pooled Method and the Token Method (Appendix C.2).
Further mathematical derivations and discussions pertinent to hyperbolic operations, such as Fréchet mean computation and contrastive loss gradients (Appendix D).
Elaboration on the learnable parameters within our model, particularly the manifold curva- ture and the loss blending factor (Appendix E).
A discussion of computational considerations, experimental setup, and qualitative results (Appendix F).
Key code snippets for essential components of Geo-Sign are provided in Appendix G aid in understanding and replication.
B Hyperbolic Geometry Preliminaries: A Brief Refresher
To ensure this supplementary material self-contained and accessible, this section briefly recaps key concepts from hyperbolic geometry, as introduced in Section 3.1 ("Hyperbolic Geometry Essentials") of the main paper.
We operate within the dimenninalia Poincaré ball model, denoted I = {x E Rhv : ||2[|| < 1/ve]. This space is characterised by constant negative curvature x = -c, where c > 0 is a learnable parameter representing the magnitude of the curvature.
The Poincaré ball model is chosen for conformal nature, where angles are preserved locally, and its intuitive representation of hyperbolic space within Euclidean unit ball (scaled by 1/vC). Key operations include:
Geodesic Distance d3, (u, v): This is the shortest path between two points v within the curved space of the Poincaré ball. is formally defined in Eq. (1) of the main paper. Unlike Euclidean distance, it expands significantly as points approach the boundary of the ball.
Möbius Addition t.e V: This operation is the hyperbolic analogue vector addition in Euclidean space, defined in Eq. (2) of the main paper (consistent with formulations in, e.g.. [18]). It is essential for defining translations and other transformations hyperbolic space while respecting its geometry.
Exponential Map exp& (v): This map takes tangent vector v residing in the tangent space TxB." at point on the manifold and maps it to another point on the manifold along geodesic. The map from the origin, cxp6(.) (-) (Eq. (3), main paper), particularly important as it projects Euclidean feature vectors (which can be considered as residing in To into the Poincaré ball.
Logarithmic Map log (y): This is the inverse of the exponential map. It takes two points x, y on the manifold and returns the tangent vector at x that points along the geodesic towards y.
These are isometries (distance-preserviig transformations) of hyperbolic space. In our work, we use learnable Möbius transformations, such as Möbius matrix-vector products (M &. v = eXp (v)(())) and Möbius bias additions, to implement affine-like transformations within our hyperbolic attention mechanism.
Möbius Transformations:
20

---

These tools allow us to define neural network operations directly within hyperbolic space. As with all hyperbolic operations the paper, we utilise the geoopt library [36] in Pytorch.
C Methodology Details
C.1 Pose Extraction and ST-GCN Architecture Details
Our Geo-Sign framework utilizes skeletal pose data as input. This section details the extraction process and the architecture of the Spatio-Temporal Graph Convolutional Networks (ST-GCNs) used encode this data.
C.1.1 Pose Data Source and Preprocessing
We use the 2D skeletal keypoints provided by the UniSign [41] framework, which were originally extracted using RTMPose-X [33] based the Booo--holeBody keypoint definition [34]. The keypoints are organised into four distinct anatomical groups for targeted processing:
Body: Includes 9 joints (COCO indices 1, 4-11).
Left Hand: Includes 21 joints (COCO indices 92-1 112).
Right Hand: Includes 21 joints (COCO indices 113-133).
Face: Includes 16 keypoints from the facial region (COCO indices 24, 26, 28, 30, 32, 34, 36, 38, 40, 54, 84-91).
For normalization, specific anchor joints used for hand and face parts: joint 92 (left wrist) for the left hand, joint 113 (right wrist) for the right hand, and joint 54 (a central face point) for the face. The body part features anchor-normalised in this scheme preserve global torso positioning.
C.1.2 ST-GCN Architecture
Each anatomical group is processed by dedicated ST-GCN stream, following the methodology of Yan al. [79]. The ST-GCN is adept learning representations from skeletal data by explicitly modeling spatial joint relationships and temporal motion dynamics.
The core of the ST-GCN involves:
1. Graph Definition:
The skeletal structure for each part is defined graph, where joints are nodes and natural bone connections are edges. The Graph class, detailed in Listing (Appendix G), handles the construction of these graphs and their corresponding adjacency matrices.
2. Initial Projection: Input keypoint coordinates are first linearly projected higher- dimensional feature space using linear layer (referred to as proj_linear in our codebase).
3. ST-GCN Blocks: sequence ST-GCN blocks processes these features. Each block (see STGCN block in Listing 2, Appendix G) consists of:
Spatial Graph Convolution (SGC) layer, which aggregates information from neighboring joints. The operation for node (joint) at layer (!) can be expressed generally as:
four(u;) )on(t;)"">> (o (o(Ax
where (1) E NxC. is the matrix of input features for nodes with Ci,, channels, XC... are learnable weight matrices for the k-th kernel transforming node features to Cout channels. A E N is the adjacency matrix for the k-th spatial kernel, defining the neighborhood aggregation based on chosen strategies (we use the spatial configuration partitioning as in the original ST-GCN paper [79]). is an activation function (ReLU in our case), and (-); denotes selection of the i-th row (features for node vi). The precise implementation involving tensor reshaping and einsum for efficient aggregation over multiple adjacency kernels is detailed in the GCN_ unit code in Listing 2.
w! (!)) E XC...
21

---

A Temporal Convolutional Network (TCN) layer, which applies ID convolutions across the time dimension to model motion patterns.
4. Residual Connections: To allow richer feature interaction, residual connections are intro- duced from the body stream's ST-GCN output to the hand and face streams before their final temporal fusion layers. This allows global body posture context to inform the interpretation of fine-gr anneggrinedd hand and face movements. Details are in Listing (Appendix G). This design choice treats body features as fixed contextual input for the parts during each forward pass, isolating the body feature extractor from direct updates via part-specific losses.
The output of each part-specific ST-GCN stream is a feature map Zp E T where T the sequence length and gou is the GCN output feature dimension. For the hyperbolic regularization branch, these Zp are temporally mean-pooled to produce static summary vectors f, E R which encapsulate the overall kinematics of part p for subsequent hyperbolic projection.
C.2 Hyperbolic Alignment Strategies: Detailed Implementation
This section provides a more detailed explanation of the two hyperbolic alignment strategies intro- duced in Section 3.3 of the main paper. These strategies are designed to regularize the mT5 model by aligning pose and text representations within the Poincaré ball.
C.2.1 Pooled Method (Global Semantic Alignment)
This strategy aims to align the holistic semantic content of the sign language video (represented by pose features) with the corresponding text translation.
1. Part-Specific Hyperbolic Embeddings:
The temporally mean-pooled Euclidean feature vectors
Ep for each anatomical part p (body, hands, face) are projected into the Poincaré ball B." This projection, yielding hyperbolic embeddings hp. is achieved using the HyperbolliProoectioo layer (Listing 4 in Appendix G), as defined Eq. (4) of the main paper:
hp = cxpo (PPF,)
(10)
Here, WP represents linear layer for part p. and 8p is a learnable scalar that adaptively scales the tangent space representation before the exponential map (xpo(-) projects it onto the manifold.
2. Weighted Fréchet Mean for Global Pose Representation: The set of part-specific hyperbolic embeddings {h₂} is aggregated into single global pose representation Mlose B This is achieved by computing their weighted Fréchet mean, which is the hyperbolic analogue of a weighted average. The Fréchet mean is defined the point that minimizes the sum of squared weighted geodesic distances to all input points:
Mpose arggin
wodh (1e, hp).
(11)
μ.B. Ahyp p=1
The weights wp are designed to give more importance to parts whose embeddings are further from the origin of the Poincaré ball (i.e., parts with more "hyperbolic energy" or distinctness), normalised via softmax:
exp(da. (0. h/m))
P Upp E/=1 exp(ds, h,
(12)
Here, AE is temperature parameter for the softmax (c.g., fixed 1.0 in our experiments) controlling the sharpness of the weight distribution. The computation is performed iteratively as detailed in Algorithm of the main paper and Listing (Appendix G).
3. Global Text Representation: Similarly, a global hyperbolic text embedding hext E dow is derived from the mT5 model's output. Euclidean token embeddings from the final layer of the mT5 decoder are first mean-poole mooled (respecting padding masks) to obtain a single sentence-level vector e This vector is then projected into Be using a dedicated hyperbolic projection layer (structurally identical to Eq. (10)):
hext expó (Stext W e
(13)
22

---

The implementation details are shown in Listing 6 (Appendix G).
4. Contrastive Alignment: Finally, the geometric contrastive loss (Eq. (5) in the main paper) is applied between batches these glohal pose embeddings {Mpose.) and global text embeddings ( This encourages semantically similar pose-text pairs to be closer in hyperbolic space.
C.2.2 Token Method (Fine-Grained Part-Text Alignment)
This strategy facilitates a more detailed alignment by relating individual pose part embeddings (hp) with contextually relevant text segment embeddings {cp)-
Hyperbolic Pose Part Embeddings {h These are obtained exactly as in the Pooled Method, using Eq. (10). Each hp represents specific anatomical part's overall kinematic signature.
Hyperbolic Text Token Embeddings: Instead of global text embedding, each Euclidean text token embedding Croken,j (from the mT5 decoder's ' final layer) is individually projected into the Poincaré ball b
hoken.j (epp(ssus W Wrixs Croken.j))
(14)
This results in sequence of hyperbolic token embeddings ( where Le is the text sequence length.
3. Hyperbolic Attention Mechanism: For each hyperbolic pose part embedding hp (acting query), a contextual text embedding cp is generated. This achieved using hyperbolic attention mechanism (see Listing 7 in Appendix G) that operates as follows:
The hyperbolic text token embeddings ( serve keys. The embeddings are first transformed using learnable Möbius transformations to enhance their representational capacity:
Key Transformation:
kj- (Mey &e booken.)) @e bkey
where May is learnable Möbius matrix and Buey is learnable Möbius bias vector.
Attention Scores: Attention scores are computed based on the negative geodesic distance between each pose query h, and each transformed text key ky:
score -da, (hp, kj)-
Attention Weights: These scores are normalised using a softmax function (after applying padding masks) obtain attention weights Apj:
(cCreep
Opj softmax
Tatin
where Tatm is a learnable temperature parameter for the attention mechanism, distinct from the temperature in the contrastive loss.
Contextual Text Embedding cp: The contextual text embedding cp corresponding to pose part hp is then computed as the hyperbolic weighted midpoint of the original hyperbolic text token embeddings (hoken.j }. using the attention weights {apj].
4. Contrastive Alignment: The geometric contrastive loss (Eq. (5), main paper) is then applied for each pair (hp... Cp.i) across the batch. The total regularization loss for this strategy the average of these individual contrastive losses over all parts P.
C.2.3 Intuition Behind the Token Method
While the Pooled Method aligns the overall semantics of sign sequence with its translation, may not capture how specific signing elements (e.g., a handshape, movement, or facial expression) correspond particular words or phrases. The Token Method aims to establish this more fine-grained understanding.
The core intuition is follows:
Compositional Language Understanding: Sign languages, like spoken/written languages, are compositional. Different articulators (hands, body, face) convey distinct lexical or
23

---

grammatical information. The Token Method attempts to map these compositional units from pose to corresponding textual tokens (words/sub-words))
2. Targeted Part-to-Segment Alignment: Instead of single global comparison, this method learns to connect individual pose part representations (e.g.. features for the dominant hand, to the most relevant segments of the textual translation.
3. Pose Parts as Queries, Text Tokens as Sources: Each hyperbolic pose part embedding hp acts as a "query", effectively asking: "Which text tokens are most semantically relevant to this pose feature?" The sequence of hyperbolic text token embeddings (haken.j) } serves as the "information source" for these queries.
4. Hyperbolic Attention for Geometric Relevance:
Relevance between a pose part query h2 and a (transformed) text token key ky is measured by their geodesic distance dg, (hp. kj) in the learned hyperbolic space. A smaller distance implies higher relevance. Using hyperbolic geometry allows these comparisons potentially leverage latent hierarchical relationships between concepts. Learnable Möbius transformations on text tokens (to get keys kj) enable the model to learn distinct tokens relevant to different pose parts (e.g., a verb token might be transformed to be closer body movement embedding).
The attention weights apj then quantify the contribution of each text token j to the meaning conveyed by pose part p.
5. Learning Textual Context for Each Pose Part: The contextual text embedding c is hyperbolic weighted midpoint of all text token embeddings, using the attention weights apj- Thus, Cp summary of the sentence, but specifically customised by the interaction of pose part p.
6. Refined Contrastive Learning:
The model is regularised make each pose part embedding hp close to its corresponding contextual text view cp in hyperbolic space, while pushing it away from non-corresponding pairs.
7. Overall Benefit: This detailed, part-specific alignment encourages the mT5 model to learn more precise mappings between kinematic features different articulators and semantic units within the text. For example, it can help distinguish visually similar signs based on subtle hand details (encoded in h that correlate with specific words, leading more accurate and nuanced translations.
D Mathematical Foundations
This section recalls two geometric components that Geo-Sign relies on:
the Weighted Fréchet Mean inside the Poincaré ball (used in Algorithm of the paper);
the Euclidean gradient of the hyperbolic distance that appears the contrastive loss.
D.1 Fréchet Mean in the Poincaré Ball
Given points Ti,..., EN in a metric space (M, d) with normalised weights w; > 0, E. we = the Fréchet mean minimises
N
F(m) = Σ
Wi
(p,
Jr = arg min F(u).
PEM
1=1
Why not simply average the embeddings in Euclidean space? Two issues appear inside the curved Poincaré ball:
(a) Manifold constraint. A Euclidean average of interior points can fall outside the ball, i.c. outside valid hyperbolic space, forcing an ad-hoc projection that distorts geometry.
(b) Metric distortion. Fuclidean distance underestimates separation near the boundary because the hyperbolic metric stretches space there. straight average therefore over-emphasises central points and washes out fine structure carried by peripheral ones.

---

nn Parameter The intrinsic Fréchet mean lives on the manifold and uses the true hyperbolic distance, so it respects curvature.
Why distance-based weights? Each pose part (body, face, left hand, right hand) yields hyperbolic embedding hip We set up x exp(dR, (0, hpp/Aw) so parts farther from the origin, regions of higher curvature and greater discriminative power, receive more influence. Without this weighting the mean would drift toward the centre, diluting information contributed by the hands and face.
Iterative update. On any Riemannian manifold the mean is found by Riemannian gradient descent; the update at iteration k is
# ((t+)) = exp,, TK Σ wi log,n (k) (zi)),
(15)
with step size 1u: > O.
Proposition D.1 (Convergence in By). The Poincaré ball B is Hadamard manifold, hence F is strictly convex and has unique minimiser μ². Let L be the Lipschitz constant of V on the geodesic convex hull of {z,}. If < L for all k, the iterates (15) converge to p². In practice we observe 2 so the simple choice mą- 1 usually sufficient and used our approach.
D.2 Gradient of the Hyperbolic Distance
For u, v Id let w (-u) te (the Möbius difference, i.c. the "vector" from transported the origin). The Poincaré distance is
ds (u, v) = Ve
artanh (Ve[[(/)(
Differentiating [18, 51] gives the Euclidean gradient required for autograd:
2 x 1wlp
1
V dz. (u, v)
(16)
c x0
2
with conformal factor X
The same formula (with sign reversed) holds for V..
1
The update rule (15) and the gradient (16) provide all the geometric tools needed by Geo-Sign's hyperbolic contrastive regulariser.
E
Learnable Model Parameters: and a
Our Geo-Sign model incorporates several learnable parameters beyond standard network weights. This section details two key ones: the manifold curvature and the loss blending factor a.
E.1 Discussion on Learnable Curvature
The curvature of the Poincaré ball, x = - (where e > 0), is crucial hyperparameter that dictates the "shape" of the hyperbolic space. Instead of fixing heuristically, we make it learnable parameter of our model (see Listing 9 in Appendix G).
Optimization Strategy: The curvature magnitude is initialised (e.g., via args. init_ as men- tioned in the main paper's experiments) and then updated via standard gradient descent as part of the end-to-end training process. The geoopt library facilitates this by defining within its PoincareBall manifold object when TearnableTTrue.
The main paper's ablation studies (Table 2a) show that initializing in the range of 1.0 - 2.0 (e.g.. optimal BLEU-4 at = 1.5) yields strong performance. Figure illustrates how e adapts during training from different initializations.
25

---

150
1.40
0375
1.4
0:10
29333 Thriin
12502
2034
13300
83223
Taaning
(a) c initiaalised at 1.50, decreases and stabilizes around 1.42.
(b) initialised 0.10, increases and stabilizes around 0.20.
Figure 3: Evolution of the learnable manifold curvature e during training for different initializations. (a) When initialised at 1.50, the curvature magnitude slightly decreases, suggesting an optimal value around 1.42 for this setup. (b) When initialised low 0.10, the curvature increases, indicating the model benefits from more "hyperbolic space" initially. It stabilizes around e = 0.20, potentially influenced by the dynamic a schedule that reduces regularization emphasis over time.
E.2 Discussion on Loss Blending Factor
The total training loss Lisil weighted combination of the primary cross-entropy translation loss fCe and our hyperbolic contrastive regularization term Lhyp.reg:
Lixal = a LCE + (1-a) Lhyprreg
The blending factor is not fixed but is dynamically adjusted during training. This dynamic scheduling allows the model to potentially benefit from different loss emphases at different training stages. The calculation of at each training step (see Listing 10 in Appendix G) is:
O O'inal = clamp ((amm + 0.1 progress) + ologitt,,) 0.2, 0.1, 1.0),
(17)
where:
Oini is the initial value for the blending factor, specified as a hyperparameter (e.g.. args. alpha 0.7 from the main paper's s ablations, Table 2b, which was found to be optimal).
training step progress is the current training progress, calculated Total traiiinngnggee stops rranging from 0 to 1. This component introduces a linear ramp, potentially increasing bascline by up to 0.1 over the course of training.
logit,, is an nn Parameter (a learnable scalar, referred to as self losssosss___. lassss_alha___aa _ogit in the code). of) is the sigmoid function, (logit,,) ) maps this learnable scalar the range (0, 1). This term provides learnable adjustment to a in the range of [0, 0.2].
clamp(', 0.1, 1.0) ensures that the final Otinal remains within the bounds [0.1, 1.0].
This dynamic allows for an initial phase where the hyperbolic regularization might have more relative influence (if mit is smaller), gradually shifting emphasis or allowing the model to fine-tune the balance via the learnable component. The ablation study the main paper (Table 2b) indicates that an initial Ommi 0.7 (i.e., 30% weight to Lnyp reg initially) provides the best results, highlighting the complementary role of the hyperbolic regularization.
F Experimental Setup, Analysis, and Qualitative Results
F1
Computational Profile
This section discusses the computational profile of Geo-Sign, comparing to a baseline Uni-Sign (Pose) model without hyperbolic regularization. The analysis is based on DeepSpeed profiler outputs for models run with a batch size of on the CSI.-Daily dataset for the Sign Language Translation (SLT) task.
26

---

0.8
Body
Face
Left
Right
0.5
Feature Norm
0.2
50000
100000
150000
200000
Training Step
Figure 4: Plot of the geodesic distances from the origin (0) the Poincaré disk the hyperbolic pose embeddings (hp) during training, averaged per part type. This shows how features for different parts utilize the hyperbolic space. For instance, right hand features (often conveying detailed lexical information) tend to move further from the origin, leveraging more of the hyperbolic curvature for discriminability. Body and face features, which might represent broader semantics or prosody, may remain closer to the Euclidean-like central region.
Experimental Context: Key experimental conditions for fine-tuning include:
Hardware: 4 NVIDIA RTX 3090 GPUs.
Training Time: Approximately 10 hours for 40 epochs of fine-tuning on CSL-Daily.
Precision: Mixed-precision precision training (bfloat is used for standard PyTorch layers, while float32 is maintained for Geoopt hyperbolic operations ensure numerical stability.
Batching Strategy: With an effective batch size of per GPU, the model occupies =20GB of memory. During training. we increase the total batch size 32 and accumulate gradients over 8 steps, achieving hypothetical batch size of 256. For the following profiler analysis, we report results for single GPU with batch size of provide clear per-device profile.
F1111 Profiler Summary and Comparative Analysis
Table summarizes key metries from the profiler. Parameter counts are consistent with the main paper's Table 1, while MACs (Multppy-Accumulay operations) and Latency are derived from DeepSpeed profiler outputs for a batch size of Table provides comparison of model parameters against other gloss-free methods.
Table 5: Computational profile comparison at Batch Size 8: Baseline Uni-Sign (Pose) vs. Geo- variants. Parameter counts from main paper's Table MACs and Latency from DeepSpeed profiler outputs. "Hyperbolic Proj. Layer MACs" reflects profiled contributions from the learnalle linear transformations within these layers.
Medel Veriant (Batch Slze S0
Total Pamase (l)
Added Parano (M) Totd Fed MLCa (GMACs)
Hoyec Proj, Layer MACs (MMAACS
Fwd Lateney (um)
Lateser lncrense (9)
(Paa)
Cec-S Proded) Cen-5 ge (Rtgaabalte Tickes)
566-01
6:46
1630
589 to
1.33
116-59
od 9c
Sc0l m $13.91
Parameter Overhead: The increase in parameters due to the hyperbolic components is marginal compared the overall model size, which dominated by the mT55 language model (es 582.4M parameters).
27

---

1.0
0.75
0.50
Pose Farts
Polnccrd Disk Boundory
loft Jofthandd
nghtt hand
-100
4.75
4.90
0.25
0.00
0:25
0.50
0.75 1.00
Ciirersion (Roducud)
Figure 5: PCA projection of 1000 hyperbolic pose part embeddings (log-mappel to the tangent space origin, then PCA-reduced to 2D) visualised within the Poincaré disk. Body features (blue) are tightly clustered near the origin, suggesting their discriminability is well-handled in a more Euclidean-like region. Hand features (left: red square, right: pink diamond) and face features (light blue triangle) are more dispersed, with hand features often pushed towards the periphery. This indicates these parts benefit from the increased representational capacity near the boundary of the Poincaré disk, where hyperbolic geometry provides more "space" to distinguish subtle variations crucial for sign language semantics.
Baseline Uni-Sign (Pose): As 587.75M parameters.
Geo-Sign (Pooled): Adds = 0.46M parameters, primarily from the five hyperbolic projec- tion layers (one for each of the four pose parts and one for the pooled text embedding).
Geo-Sign (Token): Adds x 1.35M parameters. This includes the R 0.46M for projection layers plus an additional 0.89M for the learnable parameters within the hyperbolic attention mechanism (Möbius matrices and biases).
In both Geo-Sign variants, the parameter overhead from hyperbolic components is less than 0.25% of the total model size. As shown in Table 6, our Geo-Sign models achieve competitive or superior performance recent RGB-based methods while maintaining a significantly smaller total parameter count. This highlights the efficiency of enhancing skeletal representations with geometric priors, challenging the trend that relies solely on scaling up visual encoders and language model decoders for performance gains in SLT.
MACs Analysis: The DeepSpeed profiler indicates that the total forward MACs are very similar across all configurations at this batch size:
Baseline Uni-Sign (Pose): s 116.59 GMACs.
Geo-Sign (Hyperbolic Pooled): 116.60 GMACs. The profiler attributes 3.67 MMACs to the linear transformations within its HyperbolidProjection layers.
Geo-Sign (Hyperbolic Token): 116.60 GMACs. Its HyperbolicProjectioo layers account for ~ 9.96 MMACs from their linear components.
28

---

Table 6: Sign Language Translation performance (Test Set: BLEU-4, ROUGE-L) and model parame- ters on CSL-Daily. Scores are percentages (%). Higher is better. 'Pose' and 'RGB' indicate input modalities. VE/LM/Total Params are in Millions (M). Approx. values indicated by me. Data from CSL-Daily (Train: 18,401 sentences/ 20.62 hours).
Method
VE Name
YE
LM Name
LM
Total
Modality
Test Set
Params
Params (M)
Params (M)
Pose
RGB
R-L
(M)
Floss-FFree Methods (Prior
[SLU
BdfNet
mTS Base
58244
587.7
11.42
33,80
SLRT [6] (G-Free)
EffNet
5,3
Transformer
e:30
R335,3
V
3.03
19.67
[83]
Transformer
20.35
GFSLT-VLP [88]
ResNer18
11.7
mBart
680
691.7
4
11.00
36.44
Pl.--T.M [10]
ResNer18
11.7
mRart
680
691.7
14.20
37.25
Sign2GPT [74]
DinoV/
21.0
XGLM
1732,9
1753,9
15.40
42.
L [19]
ResNer18
11.7
LLaMA-7B
6738.4
6750.1
15.75
39.91
CRRL [9]
ReaNet18
11.7
mBart
680
691.7
21.61
48.21
Our Models and Baselines
Uni-Sign [41] (Pose)
mT
582.4
587.7
/
25.61
54.92
Uni Uni-Sign [41] RGus
BANe++GCN
9.7
mTS. Base
582.4
592.1
/
4
26.36
56.51
Sign (Hypetbolic Puoled)
582.4
/
27.17
57.75
Geo-Sign (H) perbolic Token)
GCN+Geo+AUth
6.7
mTT5-Bas
582.4
$89.1
27.42
$7.95
The MACs from the learnable linear transformations within the hyperbolic projection layers constitute very small fraction (< 0.01%) of the total model MACs. The bulk of MACs originates from the mT5 model (profiled at => 66.29 GMACs) and the ST-GCN modules (profiled at =s 49.93 GMACs). We should note, however, that standard profilers (like DeepSpeed's MAC counter) primarily quantify MACs from common operations like convolutions and linear layers. The computational cost of specialised geometric functions within geoopt (e.g., manifold dist, expmap0, logmap0, Möbius arithmetic) is not explicitly broken out as distinct hyperbolic operation MACs. These functions often involve sequences of elementary operations that are not MAC-based (e.g., square roots, divisions. trigonometric functions like artanh or tanh). Thus, their computational load may be underestimated by MAC counters and is often better reflected in measured latency.
Latency Analysis: Latency figures clearly reveal the primary computational overhead introduced by the hyperbolic components during training:
Baseline Uni-Sign (Pose): As 416 ms forward latency per batch.
Geo-Sign (Hyperbolic Pooled): = 1630 ms (1.63 s), an increase of = 1214 ms z 292% over the baseline (approx. 3.9x slowdown).
Geo-Sign (Hyperbolic Token): ~ 2550 ms (2.55 s), an increase of x 2134 ms z 513% over the baseline (approx. 6.1 slowdown).
The substantial increase in training latency, despite modest increases in parameters and profiled MACs from learnable layers, underscores that the geometric operations themselves are the main performance consideration during the training phase. These operations (e.g., geodesic distance, exponential/logarihmiie maps, Möbius transformations) are inherently more complex than their Euclidean counterparts. The Token method is notably slower than the Pooled method during training due to its per-token hyperbolic attention.
Importantly, a key advantage of our regularization approach is that these geometric operations and the hyperbolic branch not utilised at inference time. Consequently, Geo-Sign models incur additional latency increase over the baseline Uni-Sign (Pose) model during inference, preserving efficiency for deployment.
F.1.2 Discussion on Data Efficiency
While not directly evaluated, it is hypothesised that skeletal data's abstraction from visual noise (lighting, background, clothing) can enhance robustness and generalization [75], especially when training data limited. Hyperbolic geometry further imposes a structural prior on the representation space. This inductive bias could potentially improve data efficiency by guiding the learning process, particularly scenarios with sparse data, although specific experiments to quantify this effect were not part of the current study. One trade-off of this approach is that cannot directly leverage
29

---

large pre-trained visual encoders as in the case of other RGB approaches, and so pre-training on sign-specific dataset like CSL-News (1,985 hours, used by Uni-Sign) is essential. However, this pre-training data size is comparable to that used by other SLT methods which use datasets such as How2Sign [15] (2000 hours) or YouTube-ASL. [66, 68] (6000 hours). We anticipate that our method would continue to scale well with larger pre-training datasets in other sign languages, though resource constraints prevented evaluation of this aspect.
F.2 Further Technical Implementation Details
This section provides additional details that are pertinent for a full understanding and potential reimplementation of Geo-Sign.
Core Libraries: Our implementation relies on Py ToTchh [56] as the primary deep learning framework. For Transformer models, we utilize the HuggingFace Transformers library. All hyperbolic geometry operations and Riemannian optimization are handled by the Geoopt library [36]. For distributed training and profiling. DeepSpeed is employed.
Hyperparameter Tuning Strategy: Key hyperparameters specific to the hyperbolic com- ponents, such as the initial curvature c, the initial loss blending factor Oriin (referred as args. alpha in code/main paper), and the hyperbolic embedding dimension dhyp, were tuned using grid search strategy on the CSL-Daily development set. Full hyperparameters are outlined in Table 7.
Numerical Stability Measures:
Operations within geoopt are performed using float32 precision to maintain numer- ical stability, while the rest of the model uses mixed precision.
-Small epsiton values (e.g.. 10-5) are added in denominators and inside logarithms/arc- tanh functions where appropriate to prevent division by zeros.
-
'Tngeett Vector Clipping: Before applying an exponential map (xp from a point x with tangent vector v, especially expo (v). it's crucial to ensure the resulting point remains strictly within the Poincaré ball and that the norm of v doesn't I cause numerical issues in tanh(-). We apply a clipping strategy mentioned in Section 3.4 of the main paper:
-
Velipped
max V Velvvl2 + Estip)
for a small Eetip > 0 (e.g., 10 10-5). This ensures that the argument to tanh exp6 does not become excessively large and that mapped points do not reach exceed the boundary of the Poincaré ball. The project=True flag in geoopt's 's expmap functions also helps enforce this by projecting points back onto the ball if they numerically fall outside.
Gradient Clipping: Standard norm- based gradient clipping is applied to all model parame- ters during training to stabilize the optimization process.
In Table we provide the full hyper-parametern for the best performing model. The full code will be released following the review process.
F3 Limitations and Future Work
While offering representational benefits, hyperbolic operations can add computational overhead compared purely Euclidean ones though this generally offset by avoiding raw video process- ing. The optimal choice of hyperbolic model parameters (e.g., curvature strategy) warrants further study. Generalizability a wider range of sign languages also needs investigation. Promising directions include exploring other hyperbolic models (e.g.. Lorentz), developing more sophisticated dynamic curvature adaptation, integrating Geo-Sign's 's hyperbolic skeletal features into multi-modal frameworks, and applying these geometric principles to other sign language processing tasks like recognition or generation. Further research into the interpretability of learned hyperbolic embeddings could also yield deeper insights into how sign language structure is captured.
30

---

Table 7: Hyperparameter summary for Geo-Sign experiments. Values are for the best reported model
configuration.
Category Hy Hyperparameter
Value
Description
General Training Confguration
Random Sced
Sced for reproducibility
Training Epochs
Number fine tuming epochs on CSL-Daily
Batch Size (per (PU)
8
Micmo-hatch size per GPP
Gradient Accumulation Steps
8
steps gum_ Mixed procision training data type
Effective batch size becomes
Training Precision (duype)
bf16
Data Handling
Max Pose Sequence Length
256
Maximum number frames for pose sequences Max new tokens for generation during ovaluation
Max Target Text Longth tgt.e.ggt_een)
100
Optimizer (Euchidea:: ST-GCN, miS,, Linear Layers)
Optimizer Type (opt)
Adam
[-5]
Learning Rate (1x)
3 10-3
For Euclidean parameters (Adam W)
AdamW 3s (ept-betas)
[0.9, 0.999]
Exponenial decay cates for moment estimates
AdamW (opt-eps)
10
numorical stability
Weight Decay (seight-decay)
0.01
L2 penalty for Euclidean parametens
LR Scheduler (sched)
Cosine Annealing
Warmup Epochs (sarmup-apacha)
Number epochs for L.R. wamn-up
Minimum LR (min-1r)
x 10
Lower bound for LR in scheduler
Gradient Clipping Norm
1.0
Max norm for gradients
Optimizer (Hyperbolic; Manifold Parameters, Projections) Optimizer Typc
RAdam 1 10
Riemannian Adam For hyperbolic parameters (RAdam)
I Larnnig Rate (hyp_1r)
Model Architecture
ST-GCN Ourput Dimension (gen_cut__de cot__ii__d
256 768
Ourput dimension of ST-GCN part streams
m15 Projection Dimension (hidden_ddin)
Target dimension for projecting GCN features to match mi15
Hyperbolic Regularizaion
Hyperholic Embedding Dimension (hipp hyp..in)
256
Dimension of embeddings in Poincaré ball Initial value for icarnable curvature e (for best model)
Initial Curvature (Gei, init..))
1.5
Blend Oga (a1pha)
Initial blending factor for Ces vs foss.s (for model)
Text Compasison Mode Okyp_ten bext__xp)
boksen
Stralegy for aligning pose with text lokens (Token Method)
Hyperbolic Contrastive Loss Expp me!
Temperature (+)
Learnable Learnable 0.2
Temperature scaling distances contrastive lass
Margin (m)
margin negative pairs contrastive lass
Label Smoothing (label_ sxroothing..yp
Label smoothing for hyperbolic contrastive loss (InfoNCH)
Loss Functions
CE Loss Smoothing sabel_snoothing)
0.2
smoothing for mT5 creesentropy loss
Distributed Training (DeepSpeed)
ZeRO Optimization Stage (zero_stage)
2
Spepdpeed ZeRO Stage for memory efficiency
Offioad CPU (ad)
False
Whether offlead ogtinizerpuramis CPU
F.4 Qualitative Results
Additional Figures: Figure 4 (similar to aspects shown in Figure 2 of the main paper, concerning learned embedding distributions) illustrates the dynamic utilization of the hyperbolic manifold by showing the average geodesic distance of different pose part embeddings from the origin during training. Notably, features corresponding to hand articulations, which often carry fine-grained lexical information, tend to migrate towards the periphery of the Poincaré disk. This suggests that the model leverages the increased representational capacity in high-curvature regions to distinguish subtle hand-based signs.
Furthermore, Figure (again, related Figure 2 of the main paper, specifically the UMAP projections) provides PCA-reduced visualization of the learned hyperbolic pose part embeddings projected onto the 2D Poincaré disk for 1000 poses. This plot reveals structured distribution where body features cluster near the origin (a more Euclidean-like region suitable for broader semantics), while hand and face features are more dispersed, with hand features populating regions further towards the boundary. This geometric organization, reflecting a learned kinematic hierarchy, likely contributes to the improved discriminability and, consequently, the enhanced translation quality demonstrated in the following examples. These visualizations support the hypothesis that the geometric biases induced by hyperbolic space aid in forming more effective representations for sign language translation.
In this section, we provide an overview of translation samples generated by Geo-Sign All predictions are from our best- pest-performing "Token" model. First, in Table 8, we show examples of prediction errors with analysis and a general measure of semantic similarity (introduced for readability, not a quantitative metric). English translations are automatically generated and then verified by a native Chinese speaker. We observe that translation quality with respect to semantics generally high, though our method, like many SLT systems, can sometimes miss pronouns or struggle with complex tenses. In Table 9, we showcase examples where our approach generates
Translation Results:
31

---

perfect or near-pan near-perfect translations. Finally, in Table 10, we select some examples to compare our model's output with that of the Uni-Sign (Pose) baseline. These comparisons illustrate improvements in semantic meaning and accuracy, consistent with the quantitative gains in ROUGE and BLEU-4 scores reported in the main paper.
Table 8: Examples of Prediction Errors and Analysis from Geo-Sign (Token Method)
Prediction
Ground Truth
Analysis of Error
Semantic Similarity
M 4 # 50 Ø
(She
tit 4 $
ln
(He
Pronoun error: tit (she) vs. fif (he). Number error: "5 0" (50) vs.
Partial (topic: age)
is 50 years old.)
years old.)
1y (four). The prediction gets the topic (age) but is wrong subject and specific age.
ラ (To-
$ x Fe Hi J (What day of week is today?)
Statement vs. Question: Prediction states a specific day. GT asks for the day. Character error: Hi (five) /L (how many/which).
Partial (topic: day of week)
day is Friday.)
L in Mi ii #1 ? (When did you AND Xiao Zhang meel?)
Prediction lacks "Hl"" (and) and the particle "#f"" This subtly changes the mcaning from a one-way way recognition to a mutual acquaintance.
Missing words:
High
(When did you meet Xiao Zhang?)
1 t
R f Σ mT (I want to go to supermarket to buy a chair, are you going?)
Missing clausc/question: Predic- tion omits the follow-up question "Ms せ ?" (are you going?).
High (core statement iden- tical)
(I want to go to the supermarket to buy chair.)
8 1T tit ft 2 ? (What you [plural] going do in the aftermoon?)
tee fN F 4 _ht f ft 4 (What they going to do in the afternoon?)
Pronoun error: f8 fi (you plural) fit fi1 (they).
High
4 t T A 2 ? (What do you [plural] need to do this afternoo?))
te f F
Pronoun error: f8 ff (you plural) vs. tE fiT (they). Word choice: 15 g (need) T (going to/want to)
High
(What are they going to do this afternoon?)
subtle semantic shift, GT more
natural for general plans.
\ Joj
Subject error: A 3 (everyone) vs. te f'] (they). Verb choice: tt 14 (feel/think) vs. AE (want/think).
High
x does everyone think should go buy chairs?)
(When
2 X 7 ?When
do they want go buy chairs?)
X (Is this watch yours?)
la T
c fi A1 m
Different intent: Prediction states loss. GT asks about ownership of present watch. Both are about watches but different scenarios.
Medium (topic: watch)
(My watch missing.)
$ $
X $ 1 T
Missing context/words: Prediction is bit abrupt. GT more com- plete with "iø is" (this) and "T D''" (bought for).
High
(How much is your watch?)
A ? (How much did you buy this watch for?)
E (Did you my cup?)
Completely different semantic in- tent and topic. Prediction about idol, GT is about missing cup.
Very Low
#
(1 discovered his
idol.)
Continued on next page
32

---

Table 8 continued from previous page
Prediction
Ground Truth
Analysis Error
Semantic Similarity
ς (It has become big in
A R ff] i 1 (The room on the left my parents', , their room
Garbled/incomplete prediction: The prediction is grammatically awkward and misses the entire context the GT.
Low
dad's room Dad's room has become bigger.)
very big.)
A : A 7
Pronoun error: füll (he) vs. kit (she). Logic error: Prediction asks why he does take taxi, GT asks why she doesn't.
Medium
f #
? (The company far
(The company is very far from home, why doesn't 't she take taxi?)
from home, why does he take taxi to the com-
pany?) ?)
2 H0, 8J X (What to say on a cloudy day? Weather something, have things do tomorrow.)
M T ft 2
X E
T E i
Nonsensical/Garbled prediction: Prediction very disjointed and doesn't 't make sense, while GT is a coherent conversation about weather and plans.
Low
#
A * (Cloudy day, TV says it's overcast, what's up? Got plans tomorrow?)
# #4, Ay
* #5 #: 48 ft 2 (There are many drinks on the table, what do you want drink?)
Slight phrasing difference: " F fi" (On the table there are) vs. " E At से. 18 8" (On the table are placed many). GT is slightly more natural. Prediction is still good.
High
f 2 (There are
drinks on the table, what do you want drink?)
R 1
R T TENTREE
Different speaker and intent: Pre- diction is a confused statement about searching. GT is directive and question someone else.
Low
T just looked for table home, not looked for.)
I: ? (Go look in the
room, just placed on the table?)
Λ A $
Vague and unnatural prediction: " 74 { n| AE" is awkward. GT is precise about "organs" and "# (cancerous change).
Medium
: (A
person's cancer will become very possible.)
(Many organs of the
human body can become cancerous.)
Nonsensical and irrelevant pre- diction: "Mil T * (and not walk cars) makes no sense. The GT describes specific scenario.
Very Low
B m
B
iti ボ
Bl T क
#
(When elderly people
AT
Hu
iy E
cross the crosswalk, they can use the crosswalk, and not walk cars.)
(An old man was slowly crossing the crosswalk, but the waiting driver impatiently honked the horn.)
Table 9: Examples of Correct Predictions by Geo-Sign (Token Method)
Reference (Ground Truth)
Our Model Prediction (Perfect Match)
'4 * # A.l (Today want to eat noodles.)
M &
'4> * 2 tel
Ui
MI $
(Today want to eat noodles.)
* R
fx उे f 12
f उे At m3 & (Did you buy the apples?)
f
(Did you buy the apples?)
Continued on next page
33

---

Table 9- continued from previous page
Reference (Ground Truth)
Our Model Prediction (Perfect Match)
*R t:: 7 ff di _ (I was a bit tired yesterday.)
' x ff Mi # (I was a bit tired yesterday.)
' R. 4 Ve xhm & DE म.. * R
'I S 4 Wi T E IE c.. * R
(Eat more fruit after lunch.)
(Eat more fruit after lunch.)
* A1
के F
5 E T
#
* Atj के F E E T
E2 J $ A B
(My wife has cold, will drive her to the hospi- tal.)
(My wife has a cold, will drive her to the hospi-
tal.)
' f[] i
(We will contact you via text message.)
(We will contact you via text message.)
R fiT A K Hi th E #l Jj
* i 12
* fll $ म H
(We will use random checks for inspection.)
(We will use random checks for inspection.)
*r
38 1C
l J a 0i Тラ [5]
' th I
E AF
A [o
(You need to grasp the direction of your own
(You need to grasp the direction of your life.)
life.)
7 de #
18 t 30
T 5 1 _ #
(Medical records are not allowed to he smeared or altered.)
(Medical records are not allowed to be smeared or altered.)
*ite
#
# (He ahandoned his family and fled with the fam-
(He abandoned his family and fled with the fam-
ily's belongings.)
ily's belongings.)
X
(This glass workshop has a hundred years of history.)
(This glass workshop has hundred years of history.)
Table 10: Comparative Analysis: Geo-Sign (Token) vs. Uni-Sign (Pose) Selected Examples
Reference (Ground Truth)
Geo-Sign (Token) Prediction
Uni-Sign (Pose) Prediction
*I क R [ t to A R
tt कJ R * 12
L a X * t3 It 1e R
(He is very tired every day when
(He comes very tired every day.)
(He has enough time [to be/and
he comes back.)
is] very tired every day.)
*/ Hh. H न AJ
11 26 D I2 &
*I t hB 1 xx 12 A m3 (Xiao Zhang, is that girl from your company?) ?)
B म T
mi ? tx की tu (Xiao Zhang, is that girl from your company? Do you know her?)
(That Xiao Zhang is the person in charge of this company, do you understand/knoww)
A 12 :
# 2
" 7 s8 X x 7
M x
(Cloudy day, TV says it's over- cast, what's 's up? Got plans tomor- row?)
(What to say on a cloudy day?
(Sarcastically talking about talent, something happened unknow- ingly.)
Weather something, have things to do tomorrow.)
[ ft T ;
fh Ht
t #t ft 2 t0
18, 1/: 2े2
< f/ M ftt ft 2
(But you haven't lost any weight, what exercise have you been doing?)
(But you didn't eat too full, what exercise are you going to do?)
(But you already can't stand it, what can you do?)
Continued on next page
34

---

Table 10 100-com continued from previous page
Reference (Ground Truth)
Geo-Sign (Token) Prediction
Uni-Sign (Pose) Prediction
T [ f.
* t T fe HR t #1 ff)
(The mobile phone cannot select seat near the window.)
(Seats near the window cannot be selected.)
(Sitting in seat near the window, you cannot take your mobile phone.)
A T (He has always had high stan- dards and strict requirements for himself.)
# M (He has strict standard require- ments for himself.)
(He put forward higher require- ments for his strictest standards.)
T Mi Hl
X
(The chef's work is very popular.)
xt
(The desserts made by this chef are all popular.)
(The chef is very popular when designing works.)
35

---

G
Code Listings
These code examples provide an overview key components in the architecture to help improve readability of the paper.
Defines the skeleton topology and a row-normalised adjacency tensor A.
def h distancc (num_nodes, edges, max max_hopp :ex_hop=1):
""" Shortest path Length (<= map_hop) for every pair of modes. ... adj mp. zeros ((num_oodes, num_nodes))
for i, j in edges:
adjli, ]] [dj[], i] =
hop full (adj, up ipfinf, dtype-float)
for d range (max hop 1):
hnp linalg matrii_porer (adj, d) 0] return hop
class Graph:
12
__init__ (self, layout='hand',
max_hop-1);
13
self .it.ttiiiti....ii _init_edges (layout)
14
distance (self num_nodes, self edges, max_hop) self.A self adjacency (strategy)
self.ho
16
17
edge lists def iiiit _dgesseeeeeeeeee (self, layout):
18
if layout in ('left', 'right'): joints)
hand (21
20
self num_nodes 21
21
fingers ([0,1,2,3,4], (,,,,,,,,0,,,,,,,) (,,,,,,,,,,,100000 10,9,10, 11, 12]
22
(1,,14,16,16)
188 (8,,9,201]
links = [(i, i) for in range (21)]
links [(f[i], f(i+1]) for f in fingers for i in range
len (f)
25
self.edges self.cence
links,
26
elif layout == bbody"
torso
27
self num num_nodes
28
torso [(0,1) for i range (1,
29
arms = ( (5,7). 6) (6,8)]
self.edges center = ((i,1) for i in range (9)] + arms. elif layout 'face_all';
31
32
self num_nodes 16
33
ring (1+1 X16) for i in range (16)
self edges, self center [(1,1) for i in range (16) + ring,
35
else:
36
raise ValueError (f
37
# adjacency
39
def _adjacency (self, strategy):
40
A (self hop 1) astype (float)
neighbours
I-hop
if strategy == 'uniform':
A (A. (1, kespdims=True) 1e-6)
43
elif strategy 'distance';
/ hop
distance A
44
+ 1e-6); A[A -- np inf]
A sum keepdims- 1e-6)
46
return torch. tensor (A, dtype-torch. float32). unsqueeze (0)
36

---

2. ST-GCN definition. GCNUnit applies K spatial kernels; STGCNBlock adds temporal cony and an optional residual path.
class GCNUnit t (nn Module))
def
-_init_- (self, Cin, Cout, A, stride-1, K-None, adaptive-True) super () -i..iint
self K = k or A shape [o]
$ #adjacency kernels
self.A A mn. Parameter (A. clonee)) ()) if adaptive else
self conv mn. Conv2d (Cin, Cout=self .,, (1,1))
self. bn mn nn. www.aachhoom2ddooo
self. act nn ReLU (inplace=Tru)))
10
def forvard x):
2: (N, Cin, V,
11
N, T, shape
12
x self conv view (N, self.K. -1, T, V) x torch einsum ('nkctv kva->nctw', self A) spatial agg
aelf (aelf (nllfbbxx))) (x))
15
16
class STeCNBlock (nn Module)
17
def -niitt (self, Cin, Cout, A, t_kernel-3, stride-1, residual- True):
18
super () init ()
19
self gen GCNUnit (Cin, Cout, A) pad (t_kernel-1) //2 self ten nn Sequential Conv2d Cout, (t_kernel 1), (stride, 1), (pad, 00), nn (atchlorr2d(Cout)) self.rew res (nn. Idn.iiyy if Cin--Cout and stride--1 else mn Conv2d (Cin, Cout, 1, (stride,1)1 self act nn n (implace=rr))
20
23
24
25
28
def forward (self, x):
29
return self.ace act (self. ten (self gen (x)) self res (x))
3. Body-to- to part residual: body features broadcast to hands / face.models.py residual context
body_ctx None
for part in ('body', ''ady', "rightt,'face__uul
self proj_linear [part] (sra_input [part]) permute (0,3,1,2) self. felffgen_speeee gen_spatial [part] (x)
part == 'body':
body_ctx detach
freeze content
else:
joint = body_ctx idx_map [part]]
select joint broadcast to V
joint. unsqueeze (-1)
10
out [part] self gen. temporal [part]
Project Euclidean vector the Poincaré ball.
class HyperbolicProoectioon (nn Module);
S-nit__(eelf d_in, d_out, manifold):
super ()._.ini___6 self manifold manifold self proj nn Linear d_out)
self log_e
Parameter (torch zeros (())
in (scale))
def forward x):
t - self proj (x) * self log scg.scale.. exp ()
tangent vec
return self. manifold. (t., project-
5. Weighted Frechet mean (Algorithm 1).
def frechet_ mean (pts, R, M, max iter=50, tol col-1e-5, eta-1.0): "aa sts: (H, (8,B) (8.) >> (B.D). D). ***
37

---

unsqueeze (-1) / (w. su.summe (0, keepdim=True) 1e-8) pts [0] clone
for range (max_iter):
v (w и. M.gomppp (mu unsqueeze (0), pts)) sum sum (0)
mu_next M. expmap (mu, eta*v, ProjectTTrue
if (M. dist (mu_next, mu) < tol). all( break
10
return mu
6. Sentence-level text embedding (pooled method).
mask txt mask unsqueeze (-1) float ()
(B T, 1) # mean pool
sent (emb mask) sum (1) / mask. sum (1) clamp_mimne min (1) h_text self hyp-proj-ss text (sent)
7. Hyperbolic attention (token method).
h tok self hyp-proj_text ( (tok_emb)
(B, D)
h_pose. unsqueeze (2)
# (B, (B,P,1, 1,
self www.iifoddmmbbus.eee
self www.mmoo....oo.u...oooo....
(i_key, h_tok, unsqueeze (1)),
b_key)
logits self mww.mool. k)
(B,P,I)
logits. wttpsmmsseedilll__lle (ttllllllllllloooooiiilllllllle ("tok_makk. mask (eteeeott.a.aeeeeeeeaaaaaaa (1), -1e9)
alpha = ssoftmax
tau_atta, taa -1)
weights
etx
self www.:mooll...iiiee..m.mi....eee. (h_tok (1squeeze alpha,
[2])
8. InfoNCE loss calculated in hyperbolic space.
(nn Module) class i_init__- (self, X, tau0=0.5, m0=0.1) super () .._..iintt initt__
def
self M self log_tau nn nn. Parameter (torch. logit (torch tensor (tau0 ) self.m
nn Parameter (torch, tensor (m0))
def forvard a, b):
(B,D) gairs
d self dist unsqueeze (1), b. unsqueeze (0)) s -d (torch. (self. - ("torchh eye (len (a), dtype=torch. bool, Device=a.device) self clamp_min (0)
10
+ 0.01)
12
target torch. arange (len (a), device=a device) return F, cross crooss_etrrpyCCC (s, target)
13
9. Manifold with leamable curvature initialisation.
self manifold = geoopt PoincareBall (c=efg init_c, learnable=True)
10. Dynamic alpha for loss blending.
progreas self global_atep max (1, self total_
alpha_base base efg. alpha_init 0. progress
0.9
alpha_learn 0.2 * torch. torch. sigmoid (self. alpha_lgit)
alpha
(alpha_base + learn) clamp (0.1, 0.99) alpha ce_loss + (1 alpha) hyp-loss
loss
38


Reviews:
[{"review_id": "gfxjsz31Hm", "reviewer": "Reviewer_b7X1", "summary": "Geo-Sign introduces a novel approach to Sign Language Translation (SLT) by enhancing the geometric representation of skeletal pose features. Instead of relying on RGB inputs or conventional Euclidean embeddings, this work employs hyperbolic geometry—specifically the Poincaré ball model—to capture the hierarchical structure of sign kinematics. Integrating hyperbolic projections, Fréchet mean aggregation, and a geometric contrastive loss into an mT5-based translation pipeline, Geo-Sign significantly improves translation quality, particularly for fine-grained hand gestures. Experimental results on CSL-Daily show state-of-the-art performance with pose-only data.", "questions": "1. How would the model generalise to non-Chinese sign languages with different kinematic properties (e.g. [1])?\n2. Could dynamic curvature be more explicitly leveraged across layers or time steps?\n3. Have the authors considered comparing Möbius-attention to simpler attention variants in hyperbolic space (e.g., distance-only or dot-product-based)?\n4.  In terms of the gradient of the hyperbolic distance in Appendix D.2, the exponential expansion of distances at the boundary could cause instability in backpropagation. How does the model prevent exploding gradients when points lie near the boundary of the Poincaré ball?\n\n\n[1] A. Duarte et al., How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language. CVPR, 2021", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The paper is well-written. It presents a well-motivated, thoroughly validated method for improving SLT using hyperbolic geometry, with strong experimental support and clear implementation details.\n-  Achieves SOTA performance in gloss-free SLT with significantly fewer parameters and privacy-respecting pose data. Demonstrates that geometric priors can substitute for expensive visual encoders to some extent.\n- The first one to combines hyperbolic embedding, contrastive loss, and part-wise attention in SLT task. The application of Riemannian optimization and learnable curvature is innovative.\n\n**Weaknesses:**\n- Only evaluation on CSL-Daily. It would be great if this method can also generalize to other languages. I appreciate the author to acknowledge the lack of significance testing due to computational and time limits, but still a weakness when interpreting modest improvements (e.g., +1.81 BLEU4).\n- As the author discussed in Line 108, full reliance on accurate pose extraction (RTMPose) could degrade under noisy conditions. It is very important to have visual features and more detailed facial expression from RGB videos. The in-the-wild application performance will be of concern because the quality of the pose data is highly unpredictable and could be jittery. Does this method also maintain good performance under noisy pose?\n- The author claimed that one of the reasons to drop the RGB information is to reduce the computation computational cost. However, increased training latency also presents due to hyperbolic computations, although mitigated by not affecting inference. There is no analysis on this (e.g. run-time, memory cost).\n- No statistical significance measures reported due to compute limitations.\n- How sensitive is the weighted Fréchet mean computation to initialization or high curvature values near the boundary of the ball?", "was_revised": true, "final_justification": "The authors addressed all my concerns and questions in the rebuttal with detailed experiments and clarification. Though in my view, the lack of RGB information is still a problem in terms of the application system, regardless of the claim of being privacy-aware. I still think this algorithm is valuable by improving existing SotA approaches. Therefore, I will support the acceptance of this work due to its novelty and contribution to the SLT field.", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback. We are happy to address the weaknesses and questions below.\n\n### Addressing Weaknesses\n\n**1. Generalization to Other Datasets and Tasks**\n\nAs requested, we provide additional results on **How2Sign (ASL Translation)** and **WLASL2000 (ASL Recognition)**.\n\n**How2Sign (SLT)** \n\nOn this translation task, our geometric regularization provides a clear improvement over the Uni-Sign baseline, demonstrating generalization to a new language (ASL) and data domain.\n\n| Method | BLEU-4 | ROUGE-L |\n| :--- | :--- | :--- |\n| Uni-Sign (Pose) | 14.5 | 34.3 |\n| **Geo-Sign (Ours)** | **15.1** | **35.4** |\n\n**WLASL2000 (ISLR)** \n\nOn the recognition task, our pose-only method not only improves upon the pose-only baseline but also surpasses the stronger multi-modal (Pose+RGB) variant of Uni-Sign demonstrating that the method is adaptable to recognition as well as translation tasks. \n\n| Method | Pose | RGB | WLASL2000 (P-I) | WLASL2000 (P-C) |\n| :--- | :-: | :-: | :-: | :-: |\n| Uni-Sign (Pose) | ✔ | | 63.13 | 60.90 |\n| Uni-Sign (Pose+RGB)| ✔ | ✔ | 63.52 | 61.32 |\n| **Geo-Sign (Ours)** | ✔ | | **63.64** | **61.89** |\n\nWe have only included the baseline (Uni-Sign) as reference here for brevity but can provide the full table on request. We will add these results and some discussion to the final paper.\n\n**2. Robustness to Pose Noise and Non-Manual Features**\n\nThe reviewer raises a great point regarding the importance of non-manual features (NMFs) that can only be captured accurately with RGB data and the challenges of pose-only methods. This work is explicitly framed to address privacy-aware SLT and so we don't consider RGB features, however we do include 16 keypoints for the face which should capture some of this information. To empirically address the concern of in-the-wild poses, we conducted a new ablation study, adding Gaussian noise to the pose representations. The results demonstrate that our hyperbolic method is substantially more robust to noise than the baseline, maintaining higher performance and degrading more gracefully. We will add these results as an additional ablation in the paper. \n\n| Noise Level ($\\\\sigma$) | Geo-Sign (BLEU-4) | Baseline (BLEU-4) | Geo-Sign Drop (%) | Baseline Drop (%) |\n| :--- | :--- | :--- | :--- | :--- |\n| **0.00 (Original)** | **27.42** | **26.25** | **-** | **-** |\n| 0.01 | 26.30 | 24.14 | **-4.1%** | -8.0% |\n| 0.02 | 24.60 | 21.50 | **-10.3%** | -18.1% |\n| 0.03 | 19.07 | 14.40 | **-30.4%** | -45.2% |\n| 0.04 | 11.63 | 7.20 | **-57.6%** | -72.5% |\n| 0.05 | 5.98 | 3.01 | **-78.2%** | -88.5% |\n\nWe hypothesize this increased robustness stems from the geometric properties of the Poincaré ball. The exponential volume growth forces a sparser representation, creating larger geodesic margins between distinct sign classes, making the model less susceptible to noisy perturbations.\n\n**3. Computational Cost**\n\nThe reviewer is correct that our hyperbolic branch introduces computational overhead during training, a point we address in detail in **Appendix F.1**. As discussed, this latency is partly due to the necessity of using **FP32** for hyperbolic operations to maintain numerical stability. However, this is a one-time training cost. Crucially, at inference time, the overhead is zero, as the regularization branch is removed. \n\n**4. Statistical Significance**\n\nWe can only agree with the reviewer, though we note it is a limitation shared by all prior work in this area due to the computational requirements for training these large SLT models. To ensure fairness and reproducibility, we used the same seed and hyperparameters as the baseline. We also include a detailed overview of all hyperparameters in the supplementary material. The code for all experiments and weights are also available for both our pretrained and fine-tuned models. \n\n**5. Sensitivity of the Weighted Fréchet Mean**\n\nAs discussed in **Appendix D.1 (Proposition D.1)**, the computation is highly stable. As we discuss in the supplementary, the Poincaré ball is a Hadamard manifold, where the squared geodesic distance is a strictly convex function. This guarantees that a unique Fréchet mean exists and that our iterative algorithm (**Algorithm 1**) is guaranteed to converge to it, irrespective of initialization. We are happy to move this into the main paper at the reviewers request.\n\n###  Questions\n\n**1. Generalization to other sign languages** \n\nAs demonstrated in our new experiments on How2Sign (ASL) and WLASL2000 (ASL), our method generalizes well. We hypothesize this is because the hierarchical nature of kinematics (torse $\\\\to$ arm $\\\\to$ hand) is a fundamental principle in sign languages. Our model, by using a geometry inherently suited to such hierarchies, learns this compositional structure, allowing it to adapt across different languages.\n\n**2. Dynamic Curvature** \n\nThis is an interesting idea! We did experiment with learning distinct curvature parameters for each body part but, as the reviewer might anticipate, we found it introduced instability into the Riemannian optimization. It is technically feasible to have adaptable curvature over the temporal dimension, perhaps adapting it based on kinematic properties, but again challenging to optimize. \n\n**3. Simpler Attention Variants** \n\nOur choice of Möbius transformations was principled, as they are the formal isometries of the Poincaré ball and thus the geometrically sound analogue to the affine transformations critical to standard Euclidean attention. This enables a dynamic, learned alignment, which we argue is crucial for the complex alignment task in SLT. However, we agree that a simpler distance based metric would be a nice ablation to the paper. Unfortunately we don't have the resources during the rebuttal period, but will perform an additional ablation for the final version of the paper.\n\n**4. Gradient Stability** \n\nThe reviewer correctly identifies this challenge, which we address in the paper. As detailed in **Section 3.4** and expanded in **Appendix G.2**, we employ two primary safeguards: 1) **tangent vector clipping** before the exponential map and 2) maintaining **FP32 precision** for all geometric operations. These steps have proven effective in ensuring stable training.\n\nThanks again for the nice review, we look forward to the discussion period."}, {"review_id": "7ZzQsoNsgG", "reviewer": "Reviewer_TKSE", "summary": "This paper presents Geo-Sign, a novel framework that introduces hyperbolic contrastive regularisation for Sign Language Translation (SLT) using skeletal data. The authors leverage the Poincaré ball model to enhance the hierarchical representation of sign kinematics and regularise a pre-trained mT5 language model via a geometric contrastive loss. The proposed method achieves state-of-the-art performance on the CSL-Daily benchmark in a gloss-free, pose-only setting.", "questions": "Plz see weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Innovative Geometric Perspective:\nThe paper introduces a principled and novel use of hyperbolic geometry to model the hierarchical and compositional nature of sign language motion, which is well-motivated both theoretically and empirically.\n\n2. Effective Performance in a Privacy-Preserving Setting:\nBy relying solely on skeletal data, the method maintains user privacy while achieving competitive or superior performance compared to RGB- and gloss-based approaches. This is particularly valuable in real-world applications with privacy concerns.\n\n3. Comprehensive Evaluation and Strong Empirical Results:\nThe paper provides thorough experiments, including ablation studies, visualization of embedding structures, and comparison with strong baselines. The performance improvements, especially in the gloss-free setup, demonstrate the effectiveness of the proposed regularisation strategy.\n\nWeaknesses:\n\n1. SLT May Not Be the Most Suitable Task for Validating Embedding Quality.\nThe core contribution of the paper is a more structured representation of sign language motion in embedding space. However, SLT (Sign Language Translation) involves complex language mapping from signs to natural language, where the alignment between sign motion and textual output (especially gloss-free translation) is often indirect and ambiguous. This issue is particularly pronounced in datasets like How2Sign, where the semantic gap between gloss and text is wide. CSL-Daily may obscure this limitation due to its high degree of alignment. Therefore, isolated or continuous sign language recognition would be a more appropriate task to validate the proposed embedding method directly.\n\n2. Lack of Explanation on Positive Sample Selection.\nThe paper does not clarify how positive pose-text pairs are sampled for the contrastive loss. CSL-Daily contains many repeated sentences, making positive sampling easier. However, this assumption does not hold in more diverse datasets, where repeated samples are rare. The method’s dependence on such dataset characteristics is a critical limitation and could hinder generalisation.\n\n3. Dependency on Pose Estimation Quality.\nThe method is highly dependent on the accuracy of the underlying pose estimation system (e.g., RTMPose). Inaccurate or noisy skeleton data can significantly affect downstream performance, and the paper does not attempt to mitigate or quantify this risk.\n\n4. Limited Dataset and Language Scope.\nAll experiments are conducted solely on the CSL-Daily dataset (Chinese Sign Language). No evaluation is provided on datasets with other sign languages or different linguistic and kinematic structures, raising concerns about the method’s cross-linguistic generalisability.\n\n5. Unclear Notation in Figure 1.\nFigure 1 contains symbols (e.g., d, h) that are undefined or ambiguously used in the diagram. The lack of annotation impairs readability and reproducibility, especially for readers unfamiliar with hyperbolic geometry conventions.", "was_revised": true, "final_justification": "The authors have addressed all of my concerns with detailed and thoughtful responses. While some limitations remain, the clarifications and additional analyses are sufficient to justify a borderline acceptance.", "rebuttal": "We thank the reviewer for their constructive and detailed review. We agree with the weaknesses presented and have addressed them below.\n\n### Addressing Weaknesses\n\n**1. On Task Suitability (SLT vs. SLR)**\n\nThe reviewer raises a very fair point about the suitability of SLT for evaluating embedding quality, and we agree that an evaluation on ISLR is missing from the paper. As discussed in our other rebuttals, since our initial submission, we have evaluated our method on the **WLASL2000 benchmark for ISLR**. We are pleased to report that our method outperforms not only the pose-only baseline but also the stronger multi-modal (Pose+RGB) variant of Uni-Sign.\n\n| Method | Pose | RGB | WLASL2000 (P-I) | WLASL2000 (P-C) |\n| :--- | :-: | :-: | :-: | :-: |\n| Uni-Sign (Pose) | ✔ | | 63.13 | 60.90 |\n| Uni-Sign (Pose+RGB)| ✔ | ✔ | 63.52 | 61.32 |\n| **Geo-Sign (Ours)** | ✔ | | **63.64** | **61.89** |\n\nWe will add the full table of results and additional details to the full paper or can post here during the discussion period if requested.\n\nTo address concerns about the limited dataset scope and potential dependence on CSL-Daily's characteristics, we have conducted new experiments on the **How2Sign dataset** for ASL-to-English translation.\n\n| Method | BLEU-4 | ROUGE-L |\n| :--- | :--- | :--- |\n| Uni-Sign (Pose) | 14.5 | 34.3 |\n| **Geo-Sign (Ours)** | **15.1** | **35.4** |\n\nAs the reviewer correctly notes, How2Sign has a wider semantic gap and fewer repeated sentences. The fact that our method still provides a significant improvement on this more challenging dataset provides strong evidence that our results are not an artifact of CSL-Daily's structure and generalizes well.\n\n**2 Sampling in InfoNCE**\n\nTo clarify, our work uses the standard in-batch sampling protocol of InfoNCE-style loss presented in **Equation (5)**. There is no separate or special sampling step.\n\nFor any given anchor pose embedding $\\mathbf{p}_i$ from a training batch of size $B$, its corresponding text embedding $\\mathbf{t}_i$ is treated as the sole positive sample. The other $B - 1$ text embeddings in that same batch, $\\{\\mathbf{t}_j\\}_{j \\neq i}$, automatically serve as the in-batch negatives.\n\nThis objective, a softmax cross-entropy over the geodesic distances, requires the model to correctly identify the positive pair $(\\mathbf{p}_i, \\mathbf{t}_i)$ from the set of all $B$ candidates within the batch. Minimizing this loss directly forces the model to decrease the distance between corresponding pairs while simultaneously increasing the distance to all other non-corresponding pairs in the batch.\n\n**3. On Dependency on Pose Estimation Quality**\n\nThe reviewer was not the only one to bring up the dependency on pose estimation quality. To quantify this risk, we conducted a new experiment where we added Gaussian noise to the pose representations. The results show that our hyperbolic method is significantly more robust and degrades more gracefully than the baseline.\n\n| Noise Level ($\\\\sigma$) | Geo-Sign (BLEU-4) | Baseline (BLEU-4) | Geo-Sign Drop (%) | Baseline Drop (%) |\n| :--- | :--- | :--- | :--- | :--- |\n| **0.00 (Original)** | **27.42** | **26.25** | **-** | **-** |\n| 0.01 | 26.30 | 24.14 | **-4.1%** | -8.0% |\n| 0.02 | 24.60 | 21.50 | **-10.3%** | -18.1% |\n| 0.03 | 19.07 | 14.40 | **-30.4%** | -45.2% |\n| 0.04 | 11.63 | 7.20 | **-57.6%** | -72.5% |\n| 0.05 | 5.98 | 3.01 | **-78.2%** | -88.5% |\n\nAs discussed with the other reviewers, we believe this is because the learned hyperbolic geometry creates larger decision margins between sign classes, making the representations more resilient to perturbations. We will add this full analysis to the appendix.\n\n**4. On Limited Dataset Scope (Addressed in Point 2)**\n\nAs shown above, our new results on How2Sign and WLASL provide examples of our method's cross-linguistic and cross-task generalisability.\n\n**5. On Unclear Notation in Figure 1**\n\nThank you for pointing this out; we agree it is unclear. We will revise the caption for Figure 1 in the final version to explicitly define all symbols. \n\nWe are confident that these new experiments address the weaknesses raised and look forward to the discussion period. Thanks again for taking the time to review the paper and giving some nice feedback and comments to improve the paper."}, {"review_id": "q0UEPYyReG", "reviewer": "Reviewer_aWyF", "summary": "The paper presents Geo-Sign, a novel approach for Sign Language Translation (SLT). \nGeo-Sign uses hyperbolic geometry to model the hierarchical structure of sign language kinematics.\nThey do so by projecting skeletal features derived from Spatio-Temporal Graph Convolutional Networks (ST-GCNs) into the Poincaré ball model.\nThey further present two hyperbolic contrastive alignment strategies and compare them.", "questions": "1. Did you try to apply your method over other common SLT datasets, such as How2Sign or PHOENIX14T?\n2. I believe more intuition and an overview of the method before diving into details would improve the clarity of the method and paper. For example, what is the connection between the two alignment strategies and why are we testing both? and in general, how are all the parts in the method combined? what are the steps?\n3. Please fix the inaccuracies mentioned above. Although CV-SLT are also gloss-free and achieve higher BLEU scores, I believe your method is still interesting and I don't think it is a reason for rejection.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The paper presents a novel approach for solving the important and interesting problem of SLT.\n* The paper explores two strategies for contrastive alignment in the hyperbolic representation space.\n\nWeaknesses:\n* The framing of the method and motivations are not very clear. In particular, there is no high-level overview of the method before diving into details, and there is no explanation before describing the two strategies explaining why are both of them explained at once, if they are related or when would we use each.\n* The method is only applied over one dataset, hence the results may not be immediately applicable to other languages or data.\n* There are some inaccuracies regarding prior art. For example, CV-SLT [65] is treated as a \"gloss-based method\", although their paper specifically states that \"In this work, no additional gloss supervision was introduced during the SLT training.\".", "was_revised": true, "final_justification": "The authors addressed most of my concerns by performing additional experiments on How2Sign and saying they will add clarifications to the method and method motivations.\nTherefore, I lean toward accepting the paper.", "rebuttal": "We thank the reviewer for their positive feedback on our work's novelty and for the constructive comments. \n\n### 1. On Clarity, Framing, and Motivation (Weakness 1, Question 2)\n\nWe agree that a clearer high-level overview would improve the paper's readability. In the revised version, we will add a new introductory paragraph to Section 3 (Methodology* that provides a step-by-step walkthrough of the Geo-Sign framework before delving into the geometric details.\n\nTo clarify the motivation for our two alignment strategies: we designed them as a deliberate comparative study. The \"Pooled\" method tests the baseline hypothesis of aligning global, sentence-level semantics. The \"Token\" method tests a more complex hypothesis, i.e that a fine-grained, attention-based alignment between individual pose parts and text tokens would yield superior results. We will explicitly state this motivation in Section 3.3. Furthermore in the supplementary material we provide a detailed analysis showing that the pooled method incurs less latency at a small cost in performance. We can incorporate this into the main paper to make the motivation for both approaches clearer and improve the flow of this section.\n\n### 2. On Generalization and Evaluation on Other Datasets (Weakness 2, Question 1)\n\nTo address your concern (and other reviewers) about generalisation, we have, since our submission, evaluated our method on two additional, widely-used ASL benchmarks: **How2Sign (for SLT)** and **WLASL2000 (for ISLR)**.\n\n**How2Sign (SLT):** Our method demonstrates an improvement over the Uni-Sign pose-only baseline, confirming its benefits generalize to American Sign Language.\n\n| Method | BLEU-4 | ROUGE-L |\n| :--- | :--- | :--- |\n| Uni-Sign (Pose) | 14.5 | 34.3 |\n| **Geo-Sign (Ours)** | **15.1** | **35.4** |\n\n**WLASL2000 (ISLR):** On the recognition task, our pose-only method not only improves upon the pose-only baseline but also surpasses the stronger multi-modal (Pose+RGB) version of Uni-Sign.\n\n| Method | Pose | RGB | WLASL2000 (P-I) | WLASL2000 (P-C) |\n| :--- | :-: | :-: | :-: | :-: |\n| Uni-Sign (Pose) | ✔ | | 63.13 | 60.90 |\n| Uni-Sign (Pose+RGB)| ✔ | ✔ | 63.52 | 61.32 |\n| **Geo-Sign (Ours)** | ✔ | | **63.64** | **61.89** |\n\nThese new results provide strong evidence that our approach is not limited to a single dataset and generalises effectively across different languages and tasks.\n\n### 3. On the Classification of Prior Art (CV-SLT) (Weakness 3, Question 3)\n\nThis is an interesting discussion point. We would like to note that in the official repository and methodology it can be seen that their code uses visual features extracted from a model pre-trained on a sign-to-gloss (s2g) task (specifically, the pre-trained embeddings from MMTLB, which are themselves from a gloss-prediction network).\n\nWhile the CV-SLT authors state they do not use \"additional gloss supervision during the SLT training,\" we could argue that their method is, by definition, gloss-based. The visual features are not learned end-to-end from video-text pairs; they use prior gloss-level supervision. This is a bit different from truly gloss-free methods like ours, which learn representations without any exposure to gloss annotations at any stage. We do not feel that strongly about moving it in the table (i.e putting it with gloss free) so we will let the reviewer decide what they prefer given this context. Furthermore the architecture uses RGB features which are not directly comparable with our pose-only approach.\n\nThanks again for the nice review and we look forward to the discussion period."}, {"review_id": "zVUA5bFe5y", "reviewer": "Reviewer_itHK", "summary": "The paper proposes a novel method, Geo-Sign, that incorporates the properties of hyperbolic geometry for sign language translation. They project the hands, pose, and face skeletal features from the ST-GCN model into a Poincaré ball and align them with text embeddings via a geometric contrastive loss. They evaluate their method on the CSL-Daily dataset and compare it with SOTA results.", "questions": "In Figure 2, the explanation accompanying the analysis may not be entirely accurate. While the figure on the right suggests that the hyperbolic token model focuses on the relevant parts of the input, I believe that for the sign corresponding to the character 面条, the model should attend to both the hands and the face. I am not a native Chinese Sign Language interpreter, so my interpretation may not be fully accurate. However, based on Table 1 in the paper, it seems that even with the Euclidean token model, the model was able to produce fairly good results, despite the absence of a structured embedding space as it is in the hyperbolic token case.\n\n\n| Character | Body Part Involved        | Notes                      |\n| --------- | ------------------------- | -------------------------- |\n| 今         | Right hand, face/chest    | One-hand motion near chest |\n| 天         | Right hand, forehead/head | Symbolizes sky/day         |\n| 我         | Right hand, chest (body)  | Points to self             |\n| 想         | Right hand, chest/face    | Pulling/holding motion     |\n| 吃         | Right hand, mouth (face)  | Eating gesture             |\n| 面条        | Both hands, chest/face    | Noodle-eating motion       |", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- Introducing the idea of hyperbolic geometry in the SLT domain is interesting.\n- Preserving privacy, although less motivated, is considered a positive point.\n- The paper is well motivated.\n\n**Weaknesses**\n- The results are only reported on CSL-Daily using the ST-GCN weights from Uni-Sign, which were pre-trained on CSL-News. This limited evaluation makes it difficult to assess the generalizability of the method to other datasets.\nTherefore, to strengthen the paper, I'd have considered/expected the following:\n\n(1) Experiment on at least another dataset, such as Phoenix-2014T, or How2Sign.\n\n(2) More ablation studies to get a better understanding of how the geometric properties are working", "was_revised": true, "final_justification": "**final score**: The paper introduces a new direction for sign language translation by incorporating the properties of hyperbolic geometry. There were two major concerns about the experiments: 1) experiments only in CSL-Daily, 2) lack of justification for the geometry properties. After the discussion, I am convinced that the method can generalize to other studies and even ISLR, another domain for sign language. They provided clear comparisons that outperformed the SOTA, Uni-Sign paper. Therefore, I increase my score to **4: Borderline accept**.", "rebuttal": "We thank the reviewer for their constructive comments. We have conducted several new experiments since our submission to directly address their concerns.\n\n### 1. New Experiments on Other Datasets (Weakness 1)\n\nTo address the primary concern about the limited evaluation, we are pleased to report new results on two widely-used ASL benchmarks: **How2Sign (for SLT)** and **WLASL2000 (for ISLR)**.\n\n**How2Sign (SLT):** \n\nOur method shows a clear improvement over the baseline, confirming its benefits generalise to a new language and a more challenging translation task.\n\n| Method | BLEU-4 | ROUGE-L |\n| :--- | :--- | :--- |\n| Uni-Sign (Pose) | 14.5 | 34.3 |\n| **Geo-Sign (Ours)** | **15.1** | **35.4** |\n\n**WLASL2000 (ISLR):** \n\nOn the recognition task, our pose-only method surpasses both the pose-only and the stronger multi-modal (Pose+RGB) baselines.\n\n| Method | Pose | RGB | WLASL2000 (P-I) | WLASL2000 (P-C) |\n| :--- | :-: | :-: | :-: | :-: |\n| Uni-Sign (Pose) | ✔ | | 63.13 | 60.90 |\n| Uni-Sign (Pose+RGB)| ✔ | ✔ | 63.52 | 61.32 |\n| **Geo-Sign (Ours)** | ✔ | | **63.64** | **61.89** |\n\n### 2. Deeper Ablation on Geometric Properties (Weakness 2)\n\nTo provide a better understanding of how the geometric properties are working, we conducted a new ablation study on **pose noise robustness**. We added Gaussian noise to the final pose representations and measured the impact on performance. The results show that our hyperbolic model is significantly more robust than the baseline, demonstrating a tangible benefit of the learned geometric structure.\n\n| Noise Level ($\\\\sigma$) | Geo-Sign (BLEU-4) | Baseline (BLEU-4) | Geo-Sign Drop (%) | Baseline Drop (%) |\n| :--- | :--- | :--- | :--- | :--- |\n| **0.00 (Original)** | **27.42** | **26.25** | **-** | **-** |\n| 0.01 | 26.30 | 24.14 | **-4.1%** | -8.0% |\n| 0.02 | 24.60 | 21.50 | **-10.3%** | -18.1% |\n| 0.03 | 19.07 | 14.40 | **-30.4%** | -45.2% |\n| 0.04 | 11.63 | 7.20 | **-57.6%** | -72.5% |\n| 0.05 | 5.98 | 3.01 | **-78.2%** | -88.5% |\n\nWe also direct the reviewer to the supplementary material where we include several key analyses on geometric structure. Specifically, **Figure 3 (Supp)** offers a direct visualization of the learned representation space, demonstrating a clear hierarchical structure where fine-grained hand/face features are pushed to the high-curvature periphery while coarse body features remain central. This visual structure is supported quantitatively by **Figure 2 (Supp)**, which plots the geodesic distances of embeddings from the origin, showing that the model consistently learns to place hand features further into the manifold to leverage its greater representational capacity. \n\n### 3. On the Interpretation of the Attention Heatmap and Euclidean Performance (Question 1)\n\nThe reviewer's intuition is correct: the sign for “吃面条” (chī miàntiáo, 'eating noodles') involves coordinated motion of the dominant hand (mimicking chopsticks) and the face/mouth area. The Euclidean model's heatmap, however, shows diffuse attention. We presume that it still provides an improvement over the baseline by learning a \"brute-force\" correlation rather than identifying the most salient features. This would also explain why the Euclidean Token version actually performs worse than the Pooled version on ROUGE, because the fine-grained alignment required by the Token method can be detrimental when the feature space is not sufficiently discriminative. The attention method however, also introduces a small number of additional trainable parameters (around 1M), and these may be helping to regularise the model improving on the other n-gram metrics. We will add a discussion of this result to the paper and include additional attention heatmaps to the supplementary for review.\n\nFinally, we will correct the \"sign-language\" typo in the abstract in the final version. \n\nWe hope these new results and clarifications address all the concerns and we are happy to answer any further questions or implement any suggestions the reviewer may have. Thanks again for the review and we look forward to the discussion period."}]

Author rebuttal:

Meta-review: This paper focuses on Sign Language Translation using skeletal data and proposes an approach that uses ideas from hyperbolic geometry to model the hierarchical structure of sign language motions. Initially, the paper received two Borderline Accept and two Borderline Reject ratings. The reviewers appreciated the novelty of the approach and the geometric intuition, the privacy preserving nature of the approach and the strong empirical results. However, there were also concerns, particularly related to the evaluation, which omitted some baselines and was only performed on a single dataset. The authors provided a rebuttal which addressed these main concerns from the reviewers. Eventually, all reviewers gave a Borderline Accept rating for the paper. Given the unanimous acceptance recommendation from four knowledgable reviewers, there is no basis to overturn reviews. The AC recommends acceptance. Authors should still consider any additional comments or feedback from the reviewers while preparing their final version and of course update the manuscript to include any additional promised changes, analysis, results and/or discussion they provided in their rebuttal, particularly the extended quantitative evaluation results.
Decision: Accept (poster)