The Natural Geometry of Code: Hyperbolic Representation Learning for Program Reasoning
Authors: Weilin Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Geometric Deep Learning
Hyperbolic Representation Learning
Source Code
Program Reasoning
Graph Neural Networks
Abstract Syntax Tree (AST)
Primary area: learning on graphs and other geometries & topologies
TL;DR: This work demonstrates that representing source code in its natural hyperbolic geometry, rather than Euclidean space, significantly enhances program reasoning models.

Abstract: State-of-the-art models for code representation, such as GraphCodeBERT, embed the hierarchical structure of source code into Euclidean space. This approach can lead to significant representation distortion, especially when embedding deep or highly branched hierarchies,limiting the models' ability to capture deep program semantics. We argue that the natural geometry for code is hyperbolic, as its exponential volume growth perfectly matches the tree-like structure of a code's Abstract Syntax Tree (AST), enabling low-distortion hierarchical embeddings. We introduce {HypeCodeNet}, a geometric deep learning framework that operates natively in hyperbolic space. Formulated in the numerically stable Lorentz model, its manifold-aware components include a hyperbolic embedding layer, a tangent space message-passing mechanism, and a geodesic-based attention module. On code clone detection, code completion, and link prediction, HypeCodeNet significantly outperforms existing Euclidean models, especially on tasks requiring deep structural understanding. Our work suggests that hyperbolic geometry offers a geometrically sound foundation for code representation, establishing hyperbolic geometry as a key to unlocking the structured semantics of code.

Published as a conference paper at ICLR 2026
THE NATURAL GEOMETRY OF CODE: HYPERBOLIC REPRESENTATION LEARNING FOR PROGRAM REA- SONING
Weilin Zhou
Nanjing Tech University
202211408@@rrttcchhhee edu. cn
ABSTRACT
State-of-the-art models for code representation, such GraphCodeBERR,, embed the hierarchical structure of source code into Euclidean space. This approach can lead to significant representation distortion, especially when embedding deep highly branched hierarchies. limitiin the models' ability to capture deep program semantics. We argue that the natural geometry for code is hyperbolic, as its ex- ponential volume growth perfectly matches the tree-like structure of code's Ab- stract Syntax Tree (AST), enabling low-distortion hierarchical embeddings. We introduce HypeCodeNet, a geometric deep learning framework that operates na- tively in hyperbolic space. Formulated in the numerically stable Lorentz model, its manifold-awe aware components include a hyperbolic embedding layer, a tangent space message-passing mechanism, and a geodesic-based attention module. On code clone detection, code completion, and link prediction, HypeCodeNet signif- icantly outperforms existing Euclidean models, especially on tasks requiring deep structural understanding. Our work suggests that hyperbolic geometry offers geometrically sound foundation for code representation, establishing hyperbolic geometry as key to unlocking the structured semantics of code.
INTRODUCTION
Modern code representation models, from CodeBERT to GraphCodeBERT, are built on limiting geometric assumption: that the rich, hierarchical structure of source code be faithfully repre- sented in Fuclidean space. This assumption creates critical bottleneck. Source code's Abstract Syntax Tree (AST) is fundamentally tree-like, and embedding such structures into Euclidean space inevitably causes high representation distortion. As established by foundational results like the Bourgain Theorem, this distortion collapses the tree's hierarchy, obscuring crucial semantic infor- mation like variable scope and logical nesting. Our Our core thesis is that hyperbolic geometry is the natural geometric language for code. a manifold with constant negative curvature, its volume expands exponentially with its radius, mirroring the exponential growth of nodes in a tree. This property allows for embeddings of hierarchical structures with minimal distortion. A more detailed discussion on the geometric correspondence between hyperbolic space and fundamental program- ming constructs is provided in Appendix Based on this principle, we introduce HypeCodeNet, the first end-to-end framework for learning universal code representations directly in hyperbolic space. To ensure stability and efficiency, our model is formulated in the Lorentz model and fea- tures manifold-natio
operations coupled with Riemannian optimization. Our contributions are: (1) We provide the first systematic argument and empirical evidence that hyperbolic space superior geometry for representing code's 's hierarchical structure. (2) We propose HypeCodeNet, novel, end-to-end hyperbolic GNN for code with manifold-aware components. (3) We achieve new state- of-the-art results on three diverse program reasoning tasks, validating our geometric approach.
2 RELATED WORK
2.1 EUCLIDEAN REPRESENTATION LEARNING FOR CODE
Code representation research has progressed from sequence-based models like CodeBERT (Feng
et al. 2020) and CodeT5 (Wang et al., 2021 | to graph-aware models like GraphCodeBERT (Guo
al. 2020) and UniXcoder (Guo et al. 2022) While incorporating structural information like

---

Published as a conference paper at ICLR 2026
data-flow graphs has yielded significant gains, these models all fundamentally limited by their Euclidean foundation. They cannot escape the high distortion incurred when embedding the natural hierarchy of code, which remains a key architectural bottleneck.
2.2 HYPERBOLIC DEEP LEARNING
Hyperbolic deep learning has proven highly effective for modeling hierarchical data in other do- mains. The pioneering work of Nickel & Kiela (017) and Nickel & Kiela (2018) established hy- perbolic space as a powerful medium for low-distortion tree embeddings. This paradigm has since 2023).
delivered strong results in fields like NLP (Chami al. 2019) and computer vision (Mettes et al.,
However, its application to source code analysis remains nascent and task-specific. Existing studies have explored its for narrow tasks like code retrieval (Tang et al.] (2023) and software evolution analysis (angg et al.) (023). A general, end-to-end framework for learning universal hyperbolic code representations has been notable missing piece. HypeCodeNet is the first work to fill this gap, proposing a unified hyperbolic framework and demonstrating its broad effectiveness across standard code intelligence benchmarks.
3 METHODOLOGY: HYPECODENET
Our work introduces HypeCodeNet, a novel Graph Neural Network designed to operate natively within hyperbolic space for learning source code representations. The core hypothesis is that the in- herent hierarchical structure of code's Abstract Syntax Tree (AST) is best captured in negatively curved geometry. This section details the geometric background, the architecture of HypeCodeNet, and the specific design choices made to ensure numerical stability and high model expressiveness. The geometric intuition is visualized in Figure la which illustrates the exponential growth property of hyperbolic geometry that mirrors the structure of code's AST. The structure originates from central AST Root and expands outwards with increasing hierarchy depth. Critically, the exponen- tial separation between nodes the periphery demonstrates how the space naturally accommodates deep hicrarchical structures with low distortion, directly addressing the representational bottleneck of Euclidean models.
3.1 GEOMETRIC PRELIMINARIES: THE LORENTZ MODEL
To mitigate the numerical instability associated with the Poincaré Ball model (Nickel & Kicla, 2017 we formulate HypeCodeNet within the Lorentz model (Nickel & Kiela| 2018). This model offers superior stability and computational efficiency for deep learning applications. To support techniques like curvature annealing (Ganea et al.) 2018), we define our manifold with variable negative cur- vature e < 0. The d-dimensional Lorentz model with curvature e is defined as the manifold:
c {x E Rd+1
(x,,)) 1/c, vn 0}
(1)
where ( is the Lorentz inner product: (I,, v) -ooo Σi Jayi Note that since 0, the squared norm 1/c is negative, consistent with the metric signature (-,t...., +). The geodesic distance between two points u, E dd is parameterized by the curvature:
de(u,)) v) 1
arcosh(c(u, v)2)
We observe that for points on the upper sheet of the hyperboloid, (u, v)c < 1/c (both negative). Thus, the term c(u, v)2 is always > 1, ensuring the arcosh function is well-defined.
The key operations for building neural networks on this manifold are the exponential and logarithmic move between the manifold and its local tangent spaces. For point p E Ld. the tangent space T TpCC is the set of vectors {r E gd+1 (p, v) = 0}. The logarithmic map, logg, Ld. + Cd maps point on the manifold vector in the tangent space at p:
maps, which allow
(p u)
log Vip, u)2 (1/c)² (u c(p, u)
(3)
2

---

Published as a conference paper at ICLR 2026
The norm of vector TpC' is defined as ||e||c v),,
The exponential map, exp; :
Tp - C is its inverse, mapping a tangent vector back the manifold:
(x) = cosh(o cosh(v-cllulle)+ sinh(v-c(lollc),
(4)
V-dluu|e
Note that when - 0, these operations smoothly approach their Euclidean counterparts. All our geometric operations are implemented using the Geoopt library in PyTorch (Kochurov et al. 2020).
3.2 HYPECODENET ARCHITECTURE
HypeCodeNet processes Abstract Syntax Tree (AST), represented as a graph G = (V, (AL- lamanis et al. 2017). by learning a low-distortion embedding function V L that maps each node E into the d-dimensional Lorentz manifold with curvature c. The model is com- posed of three main stages: (1) an input embedding layer that maps initial node features from a Euclidean space into the hyperbolic manifold, (2) series of stacked hyperbolic graph convolutional layers (Chami et al. 2019) that iteratively refine these representations, and (3) an output layer that maps the final hyperbolic representations to a task specifi prediction space. For comprehensive description of each component and its hyperparameters, please refer to Appendix B
1. Input and Embedding Layer. This layer is designed to bridge the standard Euclidean feature space of tokenizers with the curved geometry of our model. Each node v E is first initialized with Euclidean feature vector x. from pre-trained Byte-Pair Encoding (BPE) tokenizer (Sennrich
et al. 2015; Kanade et al.] 2020). To learn a more suitable initial representation, this vector is passed through multi-layer perceptron (MLP):
(5) The resulting vector = E Rdd must now be mapped onto our manifold LC We achieve this by lifting it to the tangent space at the manifold's origin, or "base point," o, = 0,, 0) The tangent space the origin, To, To.C%. is isomorphic Rd. We use an injection function to, to map ze into the ambient space Rd+1 such that it lies in To, fd.
= ML.(x..)
0,
" to= (20)
E (0, -0,1 5 E
This vector oy now satisfies (or, u c = Finally, we use the exponential map at the origin, expo. to transport this tangent vector onto the manifold itself. This yields the initial hyperbolic representation () for node v:
0 = exp (1
To ensure a stable initialization, we scale the output the MLP by a small factor y R 10-2 This re- sults in initial tangent vectors of having a small norm. Consequently, the initial node representations (.) expE, (o ) are mapped to region close to the manifold's origin. This hyperbolic-awan ini- tialization prevents points from being pushed to the periphery of the manifold the start of training, a common cause of numerical instability and vanishing gradients. The entire embedding process can be summarized as:
expo, (Y Lo, (MLP(a.
Hyperbolic Graph Convolutional Layer. The core of HypeCodeNet its message-passing layer, which updates the node representation 11(0) to n 111+1) Since there is no global vector space on curved manifold, all aggregation and update operations paradigm. The process for updating a central node
performed locally within tangent
spaces, following a robust "log-aggregate-evv" v involves three steps, as illustrated in Figure Ib] The manifold-aware message passing mechanism unfolds follows: 1) Log-map: Representations of neighbor nodes (hw,,) on the Lorentz manifold (CF) are projected into the local tangent space (Th, C5) of the central node (fx,), becoming Euclidean vectors (logn, (ha;)) )). 2) Aggregation: These vectors are aggregated in the tangent space using attention. 3) Exp-mmp: The final update is mapped back onto the manifold to produce the new representation (L+1) This "log-aggregatteexp"" paradigm allows for principled deep learning on curved manifolds.
Message Transtormation. For a central node v with representation (!) we gather information from its neighborhood (()).. Each neighbor E (u))ss representation (l) is first transported into
3

---

Published as a conference paper at ICLR 2026
Bonendd Soparation
hui
To, C
07 Foct
hv
og,, (h.s)
carthy Devni
Pus
(a) The geometric intuition for using hyperbolic space to represent code's AST structure.
(b) The "log- Mog-aggregate-exp paradigm manifold- aware message passing a HypeCodeNet layer.
Figure 1: Core geometric concepts HypeCodeNet. (a) illustrates the suitability of hyperbolic space for embedding ASTs with low distortion. passing mechanism.
(b) details the three-step manifold-aware message
the local frame of reference of its tangent space g Cg. This is achieved using the logarithmic map at (0:
m u -m = log (h!
The resulting vector m E T 2g is local Euclidean vector representing the geodesic path from This transformation allows process geometric relationships using linear algebra.
Attention-based Aggregation in Tangent Space. Once all neighbor messages (m M(v)) reside in the same tangent space, we can safely aggregate them. To incorporate the node's own information (a self-loop) (Xu et al. 2018), augment the neighborhood with the node itself. * (v) (v) {v). We employ a multi-head attention mechanism (Vaswani et al. 2017) to weigh messages by their importance.
For each head k, the attention score (A) between node and node E (v) is based on their geodesic distance, reflecting the inductive bias that structurally closer nodes are more relevant Veličković et al. 2017):
de (h 1
(10)
Tk
where 0x per-head learnable temperature parameter. These scores are normalized via softmax across the augmented neighborhood:
(A)) softmax
(11)
The messages in the tangent space, including the self self-message m log (h 0,
(1)
linearly transformed by weight matrix w(() The aggregated message for head & is the weighted
sum:
m
(l.k)
Σ
(.)
(w/") m
(12)
DEN+(()
The outputs of all attention heads are then concatenated, passed through a final linear layer w(O)) and dropout layer for regularization to produce the final aggregated message:
m ( Dropout (w!?) (CONCAT 1=1 (t.k))
(13)
4

---

Published as a conference paper at ICLR 2026
Update via Exponential Map. The aggregated message mh is vector in the tangent space T A 2% that represents the full update for node v. Before applying it, we apply Layer Normalization [Ba et al. 2016) to the vector to stabilize its magnitude:
(
m LayerNorm(m
(14)
Finally, we transport this normalized update vector from the tangent space back onto the manifold using the exponential map. This also serves as a residual connection, as the update is applied relative to the original position h.5):
hg (1++)) exp (mn!))
(15)
This formulation is geometrically sound, respects the manifold's structure, and integrates stan- dard GNN components like attention, self-loops, and normalization in a principled manner (Kipl
& WWeling 2016).
Complexity Analysis. The time complexity of single HypeCodeNet layer is dominated by the geometric operations. For an input graph with [V] nodes and [E] edges, and an embedding dimen- sion of the complexity is O(E)| + [V]d2). The term |Eld arises from the logarithmic mapping of all neighbor messages, while [Vdd stems from the linear transformations within the attention mechanism. While the constant factor for hyperbolic operations is higher than for their Fuclidean counterparts, all core computations. including the exponential and logarithmic maps, are highly par- allelizable modern GPUs. Our implementation leverages batched operations (loffe & Szegedy) 2013) to efficiently compute tangent space projections for all edges simultancously. ensuring that the model remains scalable to large-scale Abstract Syntax Trees.
3.3 TRAINING STABILITY AND OPTIMIZATION
Training deep neural networks in hyperbolic space presents unique challenges related to numerical precision and optimization dynamics (Shimizu et al.) 2021). To ensure stable and efficient training of HypeCodeNet, we incorporate several key techniques specifically designed for deep geometric learning.
Riemannian Optimization. Standard optimizers like Adam are designed for Euclidean space and fail to account for the manifold's curvature, leading to suboptimal updates. We therefore use the Riemannian Adam (RAdam) optimizer (Becigneul & Ganea 2019). RAdam adapts the standard Adam update rule by computing gradients within the appropriate tangent spaces. Crucially, it uses parallel transport (Cho & Lee 2017) to move momentum vectors between tangent spaces at each training step, ensuring that all updates are geometrically correct. This approach leads to significantly faster and more stable convergence compared to naively applying standard optimizers, Apppeeddssspppnxixiee.e C provides a further justification for this choice and an empirical comparison.
Curvature Annealing. In the early stages of training, strong negative curvature can cause rep- resentations to diverge. To mitigate this, we employ a curvature annealing strategy (Skopek et al.) 2019). The curvature e of our manifold C is treated as a trainable parameter. We initialize e to value very close zero (e.g.. 10 -10-), "), making the geometry nearly Euclidean. This allows the model to first learn basic patterns in a stable geometry. Over a predetermined number of training epochs, is gradually annealed towards its target value of 11. This curriculum allows the model to gently adapt to the hyperbolic space learns more complex hierarchical relationships, preventing carly-stage divergence. A key property of our formulation is that as the curvature c + 0-, the Lorentz model smoothly contracts to Euclidean space. Specifically, the geodesic distance do(u, v) degenerates to the standard Euclidean distance, and the exponential map expş(v) approaches the vector addition p v. This property ensures that during the initial stages of curvature annealing, our model's 's behavior closely approximates that of a standard Euclidean GNN. As training progresses and the curvature becomes more negative, the powerful hierarchical representation capabilities of hyperbolic geometry are gradually activated. This provides a stable and natural learning trajectory for the model.
Sixxed-Precisonn Training. Geometric computations in the Lorentz model, particularly the expo- nential and logarithmic maps, can be sensitive to floating-pont point precision errors. To prevent numer- ical underflow or overflow, which can silently corrupt gradients, we use FP64 (double precision) for all core geometric computations. Concurrenty, to maintain high computational efficiency. stan- dard FP32 (single precision) is used for less sensitive operations, such as the linear transformations
5

---

Published as a conference paper at ICLR 2026
(W()) within the attention mechanism. This mixed-precisson strategy provides robust balance between numerical stability and training speed.
3.4 OUTPUT LAYER FOR DOWNSTREAM TASKS
After the final hyperbolic graph convolutional layer, we obtain final node representation h! for each node in the AST. To perform predictions for downstream tasks, we need to map these hyperbolic representations back to standard Euclidean space where linear classifiers or other output heads can be applied.
For node-level tasks (e.g., predicting a masked token), we map the final representation 144h) to the tangent space at the origin using the logarithmic map:
Fy log (N.4))
(16)
For graph-level tasks (c.g., code clone detection). we must aggregate all node representations
(h(") Beev into single graph-level representation hgg Simple averaging is ill-defined in curved space. We therefore compute the Fréchet mean, the geometric generalization of the mean Rie- mannian manifolds, which is defined as the point that minimizes the sum of squared geodesic dis- tances all nodes. As this requires an iterative optimization process, we provide detailed descrip- tion of the algorithm and its justification in Appendix B.3 This robust, geometrically
pooling operation yields the final graph representation hig E L. which is then mapped the tangent space:
out "G
log... log.. (ha)
(17)
out The resulting vectors, out or #G now reside in a standard Euclidean space and can be fed into final MLP or linear layer to produce the desired logits for classification regression.
EXPERIMENTS
We rigorously evaluate HypeCodeNet across three diverse program reasoning tasks to validate central thesis: that hyperbolic geometry offers superior inductive bias for modeling the hierarchical structure of code. We benchmark against comprehensive suite of sequence-based, graph-based, and hybrid models demonstrate the advantages of our geometric approach.
4.1 EXPERIMENTAL SETUP
Tasks and Datasets. We select three tasks to cover different aspects of code understanding: 1) Code Clone Detection, structurally demanding task, evaluated on BigCloneBench (BCB) (Sva- jienko et al. 2014) and POJ-104 (Mou et al.. 2016); 2) Code Completion, which balances local and global context, evaluated on the Code line-level subtask for Python and Java (Lu et al.) 2021; ; and Link Prediction, pure structural reasoning task large-scale function call graph built from the GitHub Java Corpus (Allamanis Sutton 2013).TK The specific loss function formula- tions for each task are detailed in Appendix D
Baselines. We compare against a strong and diverse set of baselines. Sequence-based models in- clude CodeBERT (Feng et al.] 2020) and CodeT5 (Wang et al. [222]) Graph-based models include standard GNNs like GCN (Kipf Welling 2016) and GAT (Veličković el. 2017). alongside code- specific architectures like GGNN (Li et al. | 2015)) and CodeGNN, Hybrid models, which combine sequential and graph information, include GraphCodeBERT (Guo al. 2020). UniXcoder (Guo) jet al. 2022), and the ofate-of-the-art CodeFORMER (Liu et al. 2023).
Implementation Details. HypeCodeNet is implemented in PyTorch using the Geoopt library (Kochuovv et al. 2020). We use the Riemannian Adam (RAdam) optimizer (Becigneul & Ganea 2019) with learning rate of 5 x 10-5 and a linear warmup schedule. Our model's hidden dimen- sion is set to 768 for fair comparison with BERT-base architectures. To ensure all comparisons are equitable, our evaluation methodology is carefully aligned with prior work.
4.2 TASK 1: CODE CLONE DETECTION
This task directly tests model's ability to comprehend deep program semantics. As shown in Table П. HypeCodeNet establishes a new state-of-the-an on both benchmarks. The advantage is particularly pronounced BigCloneBenchh which features challenging semantic clones (Type-3/4).

---

Published as a conference paper at ICLR 2026
Even powerful hybrid models like CodeFORMER fundamentally limited by the representation distortion of Euclidean space. By embedding the AST's hierarchy low-distortion geometry. HypeCodeNet captures program logic more faithfully. Its superior performance over graph- nativenative models like CodeGNN further demonstrates that the choice of geometry critical the of graph structures themselves.
Table 1: Main results Code Clone Detection. HypeCodeNet consistently outperforms all base- lines across both datasets, demonstrating its superior ability to model complex code structures. Best scores are in bold.
BigCloneBench (Java)
POJ-104 (C/C++)
Model
Fi-score
MAP
Accuracy
Sequence-bar based Models
CodeBERT
0.849 0.855
0,851 0.853
0.891
CodeT5
0.902
Graph-based Models
GCN
0.868
0.864
0.915
GAT
0.871
0,867
0.921
GGNN
0.875
0,870
0,928
CodeGNN
0.890
0.885
0.935
Hybrid (Sequence + Graph) Models
GraphCodeBERT
0.912
0.910
0.962
UniXcoder
0.925
0.923
0.971
CodeFORMER
0.928
0.926
0.974
HypeCodeNet (Ours)
0.940
0.938
0.981
4.3 TASK 2: CODE COMPLETION (CLOZE TEST)
For line-level code completion, HypeCodeNet again outperforms all baselines for both Python and Java, as shown Table 2 While this task heavily influenced by local token co-occurrence domain where Transformer-based models excel-our our model's superior structural representation pro- vides discernible edge. By emhedding the code's AST in hyperholic space, HypeCodeNet better resolves long-range dependencies, such as variable scope or class inheritance, leading to more ac- curate predictions. The consistent gains across both dynamically and statically typed languages underscore the general applicability of our geometric inductive bias
Table 2: Performance on Code Completion (evaluated as Cloze Test/Masked Token Prediction) on the CodeXGLUE benchmark. HypeCodeNet demonstrates consistent improvements over strong Transformer based baselines both Python and Java. All metrics are reported as percentages (%). Best scores are in bold.
Python
Java
Model
Accuracy
Edit Sim.
Accuracy
Edit Sim.
Sequence-based Models CodeBERT CodeT5
32.4
45.1
29.8
43.2
41.2
52.3
38.9
50.9
Graph-based Models
GCN
23.1
33.6
20.5
30.1
GAT
23.8
34.2
21.2
30.9
GGNN
24.5
35.0
21.9
31.5
CodeGNN
25.8
36.4
23.0
32.8
Hybrid (Sequence Graph) Models
GraphCodeBERT
35.6
47.8
33.1
46.5
UniXcoder
43.5
54.6
40.8
53.1
CodeFORMER
44.0
55.2
41.4
53.7
HypeCodeNet (Ours)
45.0
56.2
42.2
54.6

---

Published as a conference paper at ICLR 2026
4.4 TASK 3: LINK PREDICTION IN FUNCTION CALL GRAPHS
To isolate and evaluate pure structural reasoning, we use link prediction task on large-scale function call graph. The results in Table 3 provide the most compelling evidence for our thesis. HypeCodeNet's commanding lead reveals fundamental limitation in existing models. Function call graphs are inherently hierarchical with strong community structures, which Euclidean models inevitably distort. In contrast, HypeCodeNet leverages the geometry of hyperbolic space to create high-fin high-fidelity representations that preserve these properties. This confirms that for structural reason- ing tasks, the correct geometric inductive bias is not just beneficial it is transformative.
Table 3: Performance on Link Prediction on the GitHub Java Corpus call graph. HypeCodeNet demonstrates commanding lead, underscoring the profound advantage of hyperbolic geometry for modeling network structures in code. Best scores are bold.
Model
AUC
Hits@ 10
Sequence-based Models
CodeBERT
0.815
0.613
CodeT5
0.828
0.631
Graph-based Models
GGNN
0.841
0.654
GCN
0.852
0.678
GAT
0.874
0.712
CodeGNN
0.880
0.725
Hybrid (Sequence + Graph) Models
GraphCodeBERT
0.905
0.756
UniXcoder
0.910
0.762
CodeFORMER
0.915
0.768
HypeCodeNet (Ours)
0.965
0.820
4.5 ABLATION STUDIES
We conduct ablation studies on BigCloneBench to deconstruct HypeCodeNet's performance gains, with results summarized in Table
Table 4: Ablation study results on the BigCloneBench dataset (Fl-score). The study confirms that hyperbolic geometry is the primary performance driver, with other components providing significant benefits.
Study
Model Variant
Fl-score
Full Model
0.940
1. Geometry
HypeCodeNNe-Enclddean
0.923
w/o Hyperbolic Attention
0.932
2. Components
w/o Curvature Annealing
0.935
HypeCodeNet (dim=32)
0.928
HypeCodeNet (dim=64)
0.934
3. Dimensionality
HypeCodeNet (dim=128)
0.938
HypeCodeNer (dim=256)
0.939
Geometry is Key. Replacing hyperbolic operations with their Euclidean counterparts ClspeCoodeNet-Facllieee causes significant performance drop to the level of top baselines. This confirms that the hyperbolic geometry itself, not just the architecture, the primary driver of our model's superior performance.
Manifold-Aware Components. Removing the geodesic-basee attention or the curvature anneal- ing strategy both degrade performance, validating their importance for stable and effective learning in curved space.
Embedding Capacity and Dimensionality. A comeestone theoretical ad advantage of hyperbolic ge- ometry is its ability to embed hierarchical data with significantly fewer dimensions than Fuclidean
8

---

Published as a conference paper at ICLR 2026
space. Our results, summarized in Table 4 provide strong empirical validation for this claim in the domain source code. HypeCodeNet achieves remarkable performance even at very low dimen- sions. With just 32 dimensions, our model achieves an Fl-score of 0.928, precisely matching the performance of the 768-dimensional state-of-the-art baseline CodeFORMER. At only 128 dimen- sions, it surpasses all competitors with an Fl-score 0.938, reaching near its peak performance of 0.940.
This phenomenon direct consequence of the geometrie inductive bias. The exponential volume growth of hyperbolic space naturally mirrors the exponential growth of nodes in an Abstract Syntax Tree, enabling low-distortion embeddings. In contrast, Euclidean models must number of dimensions to mitigate the inherent distortion of embedding a tree into polynomially growing space. Our findings confirm that hyperbolic space captures the hierarchical structure of code not only more accurately, but also far more compactly, establishing it highly efficient geometry for program representation.
vastly larger
4.6 QUALITATIVE ANALYSIS: ATTENTION MECHANISM
To provide intuitive understanding of HypeCodeNet's structural reasoning, we conduct a qualita- tive case study on a challenging code completion task. We analyze the model's 'ss attention mechanism to show how it leverages the code's hierarchical structure, based on realistic sub-word tokenization scheme, to make accurate predictions.
Attention from Query Token to Key Tokens
def calculate Losistt__ttaliieee (items):
total_sum "sum' is Token
Quary Token Index
Altandion Weight
18
for item items:
price item.price
0.4
total_sun price "sum" is
0..2
must predict correct variable for the return statement
return MMSS]] <-- [MASK] i Token
35
Listing (1) Code snippet with key sub-word token indices.
26
30
Attended-tto Taken Index (Key)
(a) Attention heatmap from the "MMASK] token. Figure 2: Qualitative analysis of HypeCodeNet'ss attention mechanism. Figure (a) shows the code snippet with key token indices based on a sub-word tokenizer, annotated directly in the code. The variable total sotal_sum sum split into sub-words;
highlight the index of its core component, 'sum'. Figure (b) visualizes the attention from each query token (y-axis) to key tokens (x-axis). We focus on the query from the "[MASK]* token (Token 35). The model correctly assigns the highest attention weights (darkest squares) to the 'sum' sub-word from its initial declaration (Token 12) and its most recent update (Token 30). This demonstrates a clear link between the visual attention and the code's ' semantic structure.
The analysis Figure providess concrete evidence for our claims. While a sequential model might struggle with the distance between the 'return' statement and the variable's declaration, HypeCo- deNet effectively navigates the Abstract Syntax Tree. The high attention paid to structurally relevant but sequentially distant sub-word tokens is a direct result of its low-distortion hierarchical embed- ding. This allows the model to perform robust, long-ran longerange structural reasoning that is essential for deep code cetpesttanninnnndiiiiiiin hyperparameter sensitivity, qualitative error analysis, and compelling visualizations of embedding distortion, presented in Appendix G
experimental results, including comprehensive ablation study,

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
This work challenges the prevailing Euclidean assumption in code representation, arguing that hy- perbolic geometry provides a superior inductive bias for code's s inherent hierarchy. We introduced HypeCodeNet, deep geometric framework that learns low-distortion AST embeddings and signifi- cantly outperforms strong Fuclidean baselines on diverse reasoning tasks. Our findings advocate for a foundational shift towards non Euclidean geometries to build more powerful and efficient models for program understanding and generation.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Miltiadis Allamanis and Charles Sutton. Mining source code repositories at massive scale using lan- guage modeling. In 2013 10th Working Conference on Mining Software Repositories (MSR), pp. 207-216, 2013. ht..ts.tss://eeet
sew.ntt cscho org httprooor//ooops:1113133330
Miltiadis Allamanis, Mare Brockschmidt, and Mahmoud Khademi. Learning to represent programs with graphs. arXiv preprint arXiv::711.007440 abs/1111.00744/ 2017. URL https://apil
semanti wwwcccccaarro...o wtssccolla..org/ooruussllll. 3495200
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. URL
https://arxia http://ww...//////1.....66.. 06450
Gary Becigneul and Octavian- Ganea. Riemannian adaptive optimization methods. In In- ternational Conference on Learning Representations, 2019. URL https:://pennevver.
net fort/Eorm?2e ddrrme1irrrrr1111
Ines Chami, Rex Ying, Christopher Re, and Jure Leskovec. Hyperbolic graph convolutional neural networks, Proceedings of the 33rd International Conference on Neural Information Processing Systems, NeurlPS' 19, 2019.
Minhyung Cho and Jaehyung Lee. Riemannian approach to batch normalization. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 5231-5244 2017.
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Ting Liu, Daxin Jiang. and Ming Zhou. Codebert: A pre-trained model for programming and natural 139.
languages. pp. 1536-1547 , 01 2020. doi: 16653//12020..fndiig
Outavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NeurlPS' 18, pp. 5350-5360, 2018.
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang. Shujie Liu, Long Zhou, Nan Duan, Jian Yin, Daxin Jiang, and M. Zhou. Graphcodebert: Pre-training code representations with data flow. ArXiv, 200.00860...66 2020. htt.::tts//eee.. www.nttsschholar.ccoooo
CorpusiD:221761144
Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. Unixcoder: Unified cross- modal pre-training for code representation. pp. 7212-7225, 01 2022. doi: 18653/ 1/2022. acl-long.499.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.031167 02 2015. doi: 10.48550/arXiv. 1502.03167.
Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. Learning and evaluating contextual embedding of source code. In Proceedings of the 37th International Conference Machine Learning, volume 119 ICML.'20, pp. 5110-5121, 2020.
Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional net- works. arXiv preprint arXiv: 1609.029700 09 2016. doi: 10.48550/arXiv. 1609.02907,
Max Kochurov, Rasul Karimov, and Sergei Kozlukov. Geoopt: Riemannian optimization in
pytorch. arXiv preprint 2rxiiv2005.022119 abs/2005.02819, 2020. URL https://api.
semanticscholat htt.c//l......oooooooraaaaaaeaaa
Yujia Li, Daniel Tarlow, Mare Brockschmidt, and Richard S. Zemel.
Gated graph se-
quence neural networks. arXiv preprint arXiv:1571.052 1,0493, 2015. URL https://api,
semant cscholar....... htt.rttttscccaa....ooo//oooopllp
8393918
Guanjun Liu, Xiaobing Zhou, Jianmin Pang, Feng Yue, Weijun Liu, and Jinyu Wang. Codeformer: A gen-nested transformer model for binary code similarity detection. Electronics, 12(7):1722, 2023. doi: 10.33900electrooiiss12000 2071722..
11

---

Published as a conference paper at ICLR 2026
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, and Shujie Liu. Codexglue: machine learning benchmark dataset for code understanding and generation. arXiv preprint ar arXiv:2202.046600 2021. doi: 00.48500/rxxiw210220666.
Pascal Mettes, Mina Atigh, Martin Keller-Ressel Jeffrey Gu, and Serena Yeung. Hyperbolic deep learning in computer vision: A survey. arXiv preprint arXiv:2305.066111 05 2023. doi: 10.48550/
MrXiv.2305.0611.
Lili Mou, Ge Li, Lu Zhang, Tao Wang, and Zhi Jin. Convolutional neural networks over tree struc- tures for programming language processing. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 1287-1293. AAAI Press, 2016.
Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representa- tions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 6341-6350, 2017.
Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hy- perbolic geometry, ArXiv, wws//800..oo.. 1806.03417 2018. URL htt.a/pp...eeaaaiccccc.coccoocoa
org/Corpus 4701 6889
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv preprint arXiv: 07999,999, 08 2015. doi: 11885500aXXi..1588.779999
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. In Interna- fional Conference on Learning Representations, 2021. URL. http:/s://pppeeeeiiiieeeee// net/
Eorum?1d=Bc85bot UwbA
Ondrej Skopek, Octavian-Eugen Ganea, and Gary Bécigneul. Mixed-curvature variational au- toencoders. arXiv preprint arXiv: 191108411 htt///911..81111 2019. URL httpss://ppi,
Jeffrey Svajlenko, Judith Islam, Iman Keivanloo, Chanchal Roy, and Mohammad Mia. Towards big data curated benchmark of inter-project code clones. In Proceedings 30th International Conference on Software Maintenance and Evolution, ICSME 2014, pp. 476-480, 09 2014. doi: 10. 10.1109//CSME.22114.77.
Xunzhu Tang, zhenghan Chen, Saad Ezzini, Haoye Tian, Yewei Song, Jacques Klein, and Tegawendé Bissyandé. Hyperbolic code retrieval: A novel approach for efficient code search using hyperbolic space embeddings. arXiv preprint arXiv:2308. /5234, 08 2023. doi: 10.48550/ arXiv.2308.1522 15234.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st In- ternational Conference on Neural Information Processing Systems, NIPS' 17, pp. 6000 0000-000, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv: 10903, abs/171001009) 2017.
URL httph::t/p/pppptt www.nteeaaennm..m icccholar orgg Corpus ID: 32 32002
Yue Wang, Weishi Wang. Shafiq Joty, and Steven Hoi. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. pp. 8696-8708, 01 2021. doi:
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv: 181000006. abs/1810.00826, 2018. URL https:
semanticscholar.co http:/oaaaa.rreeooooossssssssss
Menglin Yang, Min Zhou, Rex Ying, Yankai Chen, and Irwin King. Hyperbolic representation learn- ing: revisiting and advancing. In Proceedings of the 40th International Conference Machine Learning, ICMI'23, 2023.
12

---

Published as a conference paper at ICLR 2026
A A DEEPER DIVE INTO THE GEOMETRIC INTUITION FOR CODE STRUCTURES
While the main paper establishes that the overall tree-like structure of an Abstract Syntax Tree (AST) a natural fit for hyperbolic space, a deeper theoretical justification lies in how this geometry faith- fully models the semantics of fundamental programming constructs. The representational power of hyperbolic geometry extends beyond the haaro-hierarchy of AST to the micro-structure of control flow and scope. This section provides a detailed theoretical analysis of why specific code structures, such as variable scopes, conditional branches, loops, and function calls, are intrinsically suited for hyperbolic representation.
A.1 MODELING VARIABLE SCOPE AS GEOMETRIC PROXIMITY
The concept of scope is fundamental to program semantics, defining a hierarchy where variables in an inner scope can access variables in an outer scope, but not vice-versa. This creates a natural parent-child relationship that hyperbolic geometry models with high fidelity.
Consider a variable 'x' declared in a function's main body and another variable "y" declared inside nested 'if' block. Semantically, 'y' is "subordinate" to 'x'. In hyperbolic embedding. the node representing the scope of the 'if' block would be a descendant of the node representing the function's s scope. Consequently. the representation of "y" would be placed "further out" from the origin than 'x', naturally making it a child in the hierarchy. The geodesic distance (scopee scope would be small, reflecting their direct hierarchical link. Critically, two variables "y and 'y2 declared in separate, non-nested inner scopes would be embedded in different diverging branches of the hierarchy. The hyperbolic distance between them, (5 322), would be large, correctly capturing their semantic and operational independence, a property that is often distorted in Fuclidean space where they might be placed close together simply due to their textual proximity.
A.2 CONDITIONAL BRANCHES AS GEODESIC DIVERGENCE
Conditional statements (e.g., 'if-else') create divergent, mutually exclusive execution paths. These paths represent distinct, hypothetical "futures" for the program state, originating from a single deci- sion point. This structure is poorly represented in Euclidean space, which struggles accommodate such branching without representational "crowding."
Hyperbolic geometry, with its exponential volume growth, provides the ideal substrate for this. The 'if" condition node serves as branching point in the AST. The nodes constituting the 'if' block and the 'else' block can be embedded along two distinct geodesics diverging from this point. Due to the nature of hyperbolic space, these two paths extend indefinitely with deep nesting, yet the distance between nodes at the same depth in each respective branch grows exponentially. This large separation naturally models the semantic exclusivity of the two branches; operations within the iff block are semantically distant from operations in the 'else' block because they can never co-occur. A Euclidean embedding, in contrast, would be forced to place these two paths close together, creating a distorted representation where mutually exclusive operations appear geometrically similar.
A.3 LOOPS AS CONTRACTIVE TRANSFORMATION IN A HIERARCHY
Loops (e.g.. 'for', 'while") introduce a recursive or iterative structure that operates within a confined scope. The body of the loop can be seen as a subgraph that is repeatedly applied, transforming the program state. While the execution is iterative, the static representation in AST is hierarchical: the loop body is a child of the loop statement, which itself exists within a larger scope.
Hyperbolic geometry can capture this duality. The loop body, as a subtree of the AST, is embed- ded a hierarchical descendant of the loop initialization statement. The iterative nature of the loop-where variables within the loop are repeatedly updated-<on can be interpreted as a series of transformations localized to this specific region of the hyperbolic space. Furthermore, the loop creates a distinct, nested scope. Variables defined inside the loop are hierarchically subordinate to variables outside it. HypeCodeNet can learn to place the representations for these loop-local vari- ables in tight cluster "orbiting" the representation of the loop statement itself, effecttvvly capturing both the containment relationship (via hierarchy) and the tight semantic coupling of the operations within the loop body. This geometric containment is much harder to enforce in Euclidean space, where there is no natural concept of a "parent" region.
13

---

Published as a conference paper at ICLR 2026
A.4 FUNCTION CALLS AS LONG-RANGE, HIERARCHICAL SHORTCUTS
Function calls introduce a powerful abstraction mechanism, creating "shortcuts" in the execution flow that connect distant parts of the codebase. call from a high-level function 'A" low- level utility function 'B' creates a strong semantic link, but their definitions might be textually and structurally distant in the overall project AST.
Hyperbolic embeddings can model this relationship effectivelyy While the AST provides the primary tree structure, the function call itself acts as a non-tree edge. In our model, this is handled by the passage-passing mechanism. However, the *representation* of the function call'' 's meaning benefits from the geometry. The utility function context. The function 'A' resides in another branch. The act of 'A' calling 'B' implies that 'A"s learn to
can be embedded deep within its own hierarchical
logic is dependent 'B''s entire encapsulated hierarchy. In hyperbolic space, a model represent this dependency by ensuring that the geodesic path from 'A' to 'B' is relatively short, even if their "tree distance" in the AST is large. This ability to represent both the rigid local hierarchy of the AST and the abstract, long-range dependencies of the call graph is key strength. The negative curvature allows for the existence of these "wormholes" or shortcuts without disturbing the local geometric structure of each function's own implementation, a balance that is notoriously difficult to achieve in flat Euclidean space.
In summary, the theoretical connection between hyperbolic geometry and code representation is not merely analogy about trees. It is deeply-rootes correspondence where the geometric properties of negatively curved space-such as exponential growth, natural hierarchies, and the concept of boundary directly mirror the foundational semantic principles of scope, control flow, and abstrac- tion that define computer programs.
B DETAILED MODEL ARCHITECTURE
This section provides a comprehensive description of the architectural components and hyperparam- eters of HypeCodeNet, supplementing the methodology presented the main paper ensure full reproducibility.
B. INPUT AND EMBEDDING LAYER
The Input and Embedding Layer serves as the critical bridge between the standard Euclidean feature space of pre-trained tokenizers and the hyperbolic geometry of our model. Its design is crucial for learning high-quality initial representation that can be effectively propagated through the network. The process consists of three main steps: feature transformation, projection the tangent space, and a stable mapping onto the Lorentz manifold.
1. High-Expressiviig Feature Transformation. Each node v in the Abstract Syntax Tree (AST) initialized with a feature vector = RRdastee from the pre-trained BPE tokenizer's embedding lookup table. To transform these generic features into a representation better suited for geometric embedding, employ sophisticated Multi-Layer Perceptron (MLP). This is not simple linear projection but a feed-forward sub-network designed for high expressivity, structured as follows:
(i) A linear layer expands the feature dimension from dembed to an intermediate dimension du, which is typically four times larger than the model's hidden dimension d.
(ii) The GELU (Gaussian Error Linear Unit) activation function is applied, chosen for its smooth, non-saturating properties that benefit deep architectures.
(iii) A dropout layer is applied for regularization to prevent overfitting on the initial features.
(iv) A second linear layer projects the representation from du down to the model's main hidden dimension d.
(v) Finally, a Layer Normalization is applied to the output vector. This step vital as stabi- lizes the distribution of the Euclidean vectors before they are projected into the hyperbolic space, ensuring that their magnitudes are well-behaved.
The entire transformation can be summarized by the following equation:
(x
(x:
(18)
This design provides the model with sufficient capacity to learn powerful, tesk-relevant initial mapping from raw token embeddings.
14

---

Published as a conference paper at ICLR 2026
2. Projection to the Origin's Tangent Space. As detailed in the main paper, the resulting vector z E 12 is then projected into the tangent space at the origin of the Lorentz model, To, C. This direct and computationally efficient operation:
o proja, (zg² 20, 2.
F
R
(19)
This places the vector into Euclidean subspace where its first coordinate is zero, satisfying the condition (0e, v <
Scaled Mapping the Manifold. To complete the embedding process, we map the tangent vector on onto the Lorentz manifold using the exponential map. A crucial component of this step is the application of small, fixed scaling factor 7
1 = expo (7.
(20)
The role of y is to control the initial distribution of node embeddings on the manifold. By initializing all nodes in small region the origin (the "root" of the hyperbolic space), prevent numerical instability that can arise from large geodesic distances between points the start of training. This
controlled initialization ensures smoother and more stable optimization trajectory.
Hy pereraarameters for the Input and Embedding Layer. our experiments are listed below:
The specific hyperparameters used in
BPE Embedding Dimension (dembed): 768
MLP Feed-Forward Dimension (dn): 3072
Model Hidden Dimension (d): 768
Activation Function: GELU
Dropout Rate: 0.1
Initialization Scaling Factor (y): 0.01
B.2 HYPERBOLIC GRAPH CONVOLUTIONAL LAYER: COdE-AWARE INSTANTIATION
The core of HypeCodeNet's expressive power lies in its stack of hyperbolic graph convolutional layers. We adopt the principled "log-age toggaagregate-epp paradigm, but our key innovation sophis- ticated, **code-aware aggregation mechanism ** that moves beyond simple geometric priors. fuses both the geometric relationships the manifold and the rich semantic content of code nodes, enabling a more nuanced understanding of program structure.
1. Message Transformation via Logarithmic Map. The first step is to project information from central node v's neighbors, E NN(),, into its local tangent space T2 Cg. This is achieved via the logarithmic map, which converts the geometric relationship between and into Euclidean vector representing the geodesic path.
m u = log (h(2) E &
(21)
This provides set of local message vectors {mo->e) ready for our advanced aggregation step. To handle the node's self-information, we also define the self- melf-messaee m
(1)
Hybrid Aggregation: Fusing Geometric and Semantic Attention. A purely distance-based attention mechanism is too restrictive for code, as fails to account for situations where two struc- turally distant nodes are highly semantically related (c.g.. function's 's definition and its call site). To address this, we introduce hybrid attention mechanism that computes final score by blending two distinct components: geometric score and semantic score.
For each attention head k. first compute the two scores in parallel:
Geometrie Attention Score (ewe (geom)); This component, identical our previous design, captures the structural prior. It assumes that nodes closer on the manifold are more relevant. It is controlled by per-head learnable temperature Øk, allowing heads to specialize different structural scales.
de (hul (1))
(geom, e usu
(22)
0k
2. Semantic Attention Score (e (sem) analogous standard dot-product attention in Transformers. All operations occur within
): This component captures the content-based based relevance,
15

---

Published as a conference paper at ICLR 2026
the tangent space A Co- We generate "query" vector from the central node and "key" vector from each neighbor using per-head learnable weight matrices, W W
and
Q.
K.A"
(2) w (1) + 6
Q.
(23)
(R))
K, m a-t0
(24)
Since the self-message m is theoretically O, strictly linear transformations would yield zero query vector. To allow the central node to actively query its neighbors based on its learned position, we explicitly include learnable bias term ((0)
Note:
(go
(1) ku
(sem,k) A) u0
(25)
dx
Score Fusion. The final attention logit e is leamable fusion of the geometric and semantic scores. We introduce a per-head scalar gating parameter A E [0, 1J, which is passed through a sigmoid function to be learned during training. This gate allows each head to dynamically decide the importance of geometry versus semantics for specific task.
>
e (geom,k) + (1 As) e (semm.k)
(26)
This hybrid score is then normalized across the neighborhood using softmax: softmax (k)
(A)
The aggregated message for head k is the weighted sum of "value" vectors, where values linear transformations of the original messages (u(A) W(())
Vik
ma mu-so)
agg msg! (1,R)
Σ
(27)
WENT(() (v)
Finally, the outputs concatenated and projected produce the final update vector me This tire mechanism allows HypeCodeNet to learn far more expressive aggregation function, capturing the multifaceted nature of code by balancing structural priors with semantic content.
(I)
Update via Normalization and Exponential Map. The aggregated message vector m! is first stabilized using Layer Normalization, rig = LayerNorm(ms) and then mapped back onto the manifold via the exponential map, which acts hyperbolic residual connection:
((+1) exp A
(m)))
(28)
Stacking Layers and Component Summary. The full architecture stacks L such layers. The learnable parameters now include matrices for query, key, and value transformations, as well as the fusion gates, as detailed in Table
Table 5: Summary of learnable parameters for an enhanced Hybrid Attention Hyperbolic GNN Layer.
Parameter
Shape/
Description
(1) Q. K.k (
(d/KKxd
Query, Key, and Value transformation matrices for each head k.
ddxd
Output linear transformation matrix.
okk
Scalar
Learnable temperature for the geometric attention component of head k.
X
Scalar
Learnable gating parameter for fusing geometric and semantic scores for head k.
Integer Integer
Total number of GNN layers stacked in the model. Number of attention heads used in the aggregation step.
K
16

---

Published as a conference paper at ICLR 2026
B.3 OUTPUT LAYER FOR DOWNSTREAM TASKS
After the final hyperbolic graph convolutional layer, the model produces a set of rich, structurally- aware node representations {he JuEV, with each h residing on the Lorentz manifold C2 To utilize these representations for specific downstream tasks, we must map them back into a standard Euclidean space where linear classifiers, regression heads, and standard loss functions (like cross- entropy) can be applied. The design of this output layer differs based on the nature of the task.
1. Output Mapping for Node-Level Tasks. For tasks that require a prediction for each individual node, such as masked token prediction in the code completion task, the process is direct. Each final node representation ( is mapped from the manifold the tangent space at the origin, To C..
2 log (h(
(29)
The choice of the origin's tangent space as the target Fuclidean space is principled. serves as canonical "flat" view of the learned representations, effectively "un-warping" the geometry to produce set of standard vectors. These resulting vectors {= can then be processed by a final classification or projection head.
2. The Challenge of Graph-Level Representation. For graph-level tasks, such as code clone detection, we must first aggregate the entire set of node representations (A(") into a single, fixed- size vector lig that represents the semantics of the entire graph. This aggregation, or pooling, is non-trivial in curved space.
naive approach, such as averaging the (d++1) didmensiioo coordinates of the node representations, is geometrically invalid. The Lorentz model is manifold, not vector space, so linear operations like summation and averaging do not preserve its geometric structure and lead to meaningless results. Instead, we require a pooling operation that is native to the manifold's geometry. The goal is to find single point on the manifold that can be considered the geometric "center of mass" of all node representations.
The principled solution for this is the Fréchet mean, which generalizes the concept of the mean to Riemannian manifolds. The Fréchet mean of set of points is defined as the point the manifold that minimizes the sum of squared geodesic distances to all points in the set. This provides a robust, geometrically sound method for graph-level pooling, which we detail next.
3. Graph Pooling via Iterative Fréchet Mean. To compute the graph-level representation has employ the Fréchet mean. For a of node representations {h (b.(),.... 1/( } on the manifold ..... cd. their Fréchet mean hgg is the point that minimizes the sum of squared geodesic distances to all other points:
V
hg arg min
Σ
dp 1 12)z2
(30)
PECC
This optimization problem does not have a closed-form solution and must be solved numerically. We use an iterative algorithm that is effectively a form of Riemannian gradient descent. Our imple- mentation relies on the robust and numerically stable solver provided by the Geoopt library. The algorithm proceeds as follows:
(i) Initialization: A starting point for the mean, Po. is initialized. A robust initialization strategy is select random node representation from the set, (5) serve the initial estimate.
(ii) Iterative Refinement: At each step f, the algorithm computes the gradient of the objective function in the tangent space at the current estimate pe. This gradient is equivalent to the average of the logarithmic maps of all points relative to μ4:
Σ log (1)
(31)
V
The current estimate of the mean is then updated by moving it along this gradient direction using the exponential map. A small step size nj (learning rate) used to ensure stable convergence:
p++1 = expi p. (1 v1)
(32)
17

---

Published as a conference paper at ICLR 2026
(iii) Convergence:
The iteration continues until the norm of the gradient vector Up falls below predefined tolerance threshold, or a maximum number of iterations is reached. This ensures that stable geometric center has been found.
The specific hyperparameters for our Fréchet mean computation are:
Implementation: geoopt Frrchet_mear
Step Size (7)): 1.0 (as is common for this algorithm)
Maximum Iterations: 100
Convergence Tolerance: 1 x 10-5
4. Discussion on the Choice of Pooling Method. We considered simpler, computationally cheaper alternatives but found them be geometrically flawed. A common approximation is the "Einstein Midpoint," which involves mapping all points to the tangent space at the origin, averaging them, and mapping the result back: K/2 expo. (M E, log.. ())
While faster, this method produces biased estimator that is highly dependent the choice of the anchor point (in this case, the origin 02). The result is not a true geometric center and does not satisfy the desirable properties of mean. In contrast, the Fréchet mean is the only method that is invariant to isometries and provides a true measure of central tendency on the manifold. Given that graph-level tasks like code clone detection require highly accurate and robust representation of the entire program's semantics, the superior theoretical properties and geometric correctness of the Fréchet mean justify its higher computational cost.
5. Final Classification Head. Once final Euclidean representation has been abtained-eitte 20 for node-level task or 2G for graph-level task-it is fed into final sub-network to produce the task-specific output logits. This component is standard Multi-Layer Perceptron (MLP), but its design is tailored provide sufficient non-linear modeling capacity while preventing overfitting the downstream task.
The architecture of this classification head is identical for both node-level and graph-level tasks, ensuring a consistent approach. It consists of:
(i) A linear layer that projects the input vector nnt E IRd to an intermediate hidden represen- tation of the same dimension, d.
(ii) A non-linear activation function. We use the hyperbolic tangent (Tanh) function, which squashes the values into [-1, range. providing stable gradients for the final layers of the network.
(iii) A dropout layer for regularization, which is critical to prevent the final classifier from memorizing the training data.
(iv) A final linear layer that projects the intermediate representation to the desired output di- mension, Nelasses. which corresponds to the number of classes for the specific task (e.g.,
Nelasses 2 for binary clone detection, or Nelasses /ocabulary| for code completion).
The entire process can be summarized by the following equation, where n represents either 2 or =G
logits Woni (Dropout (Tanh (W... bmm))) + beut
(33)
where Win E gdxd and Wout E R igNNiton xd are learnable weight matrices. This two-layer MLP design provides a good balance, offering enough expressive power to learn a complex decision boundary from the rich geometric features without being overly complex. The hyperparameters for this component are consistent across main experiments, listed in Table
Table 6: Hyperparameters for the final MLP classification head.
Parameter
Value
Input Dimension
d 768
MLP Hidden Dimension
768
Activation Function
Tanh
Dropout Rate
0.1
Output Dimension (Netasses)
Task-specific (e.g.. 2 for BigCloneBench)
18

---

Published as a conference paper at ICLR 2026
C FURTHER DISCUSSION ON RIEMANNIAN OPTIMIZATION
The choice of optimizer is a critical, yet often overlooked, component in deep geometric learn- ing. While the main text states our use of Riemannian Adam (RAdam), this section provides a deeper justification for this decision. We first compare RAdam to its simpler counterpart, Rieman- nian SGI (RSGD), to clarify its advantages. We then present crucial comparative experiment against a "naive" application of the standard Euclidean Adam optimizer, empirically and theoreti- cally demonstrating the necessity of manifold-aware approach for stable and efficient training in hyperbolic space.
C.I RATIONALE FOR SELECTING RIEMANNIAN ADAM OVER RIEMANNIAN SGD
Both Riemannian SGD (RSGD) and Riemannian Adam (RAdam) are designed to perform gradient- based optimization on manifolds. However, they differ significantly in their mechanism and practical performance, especially in the context of complex, non-convex loss landscapes encountered when training deep models like HypeCodeNet.
Riemannian SGD (RSGD). RSGD is the most direct generalization of stochastic gradient descent to Riemannian manifold M. At each step, it performs two core operations:
(i) Gradient Projection: computes the Euclidean gradient VC((,) in the ambient space and projects onto the tangent space Te.M at the current parameter point 6, This yields the Riemannian gradient 9: projr, (VL()))), which represents the direction of steepest descent on the manifold.
(ii) Retraction: It updates the parameters by moving along the geodesic in the direction the negative gradient. This is achieved via the exponential map, 0z+1 exps, (-19)), where is the learning rate.
While principled, RSGD is first-order method that suffers from the same drawbacks as its Eu- clidean counterpart: slow convergence in ill-conditioned landscapes and high sensitivity to the choice of learning rate. It does not maintain any momentum or history of the gradients.
Riemannian Adam (RAdam). RAdam extends the adaptive moment estimation of the standard Adam optimizer to Riemannian manifolds. It maintains estimates of the first moment (momentum) and second moment (uncentered variance) of the gradients. The key challenge in curved space is that the momentum vector from previous step, m M resides a different tangent space than the current gradient, 9: E M. These vectors cannot be directly combined.
RAdam solves this fundamental problem using parallel transport, a geometrically sound operation that moves a tangent vector from one tangent space to another along the geodesic connecting the two points, without changing its length or its angle relative to the path. The RAdam update involves these crucial additions:
(i) Transported Momentum Update: Before combining the previous momentum with the current gradient, it first transports the momentum vector: m -o (m,-1). (m--1)) The
update then proceeds in the current tangent space: m, = 8 3mmp11 (1 31)g. similar process is applied to the second moment estimate.
(ii) Adaptive Update: The final update step in the tangent space is scaled adaptively based on the moments, similar to standard Adam: A, -/pt/(//, + e). (iii) Retraction: The final update applied via the exponential map: O.41 expo, (A.).
Conclusion.
The of parallel transport to maintain geometrically consistent momentum is the defining advantage of RAdam. This allows the optimizer to accumulate knowledge about the gra- dient history in way that respects the manifold's curvature, leading to significantly faster conver- gence, greater stability in high-curvature regions, and reduced sensitivity to learning rate selection. For a high-dimensional and deeply-structure model like HypeCodeNett. these properties are not just beneficial but essential for achieving robust and state-of-the-art performance. RSGD), lacking this adaptive momentum, would converge far more slowly and require extensive, impractical tuning.
C.2 EXPERIMENTAL ANALYSIS: THE INVALIDITY OF NAIVE ADAM IN HYPERBOLIC SPACE
To underscore the importance of using purpose- built Riemannian optimizer, we conducted an experiment comparing the performance of RAdam against standard Adam optimizer applied
19

---

Published as a conference paper at ICLR 2026
"naively" to the parameters in the ambient Euclidean space R The "naive" approach ignores the geometric constraints of the Lorentz manifold. Specifically, its update step, 0 = 0, - 7) A.. is a simple vector subtraction. This operation is geometrically invalid, as it does not guarantee that the updated parameter 0.+1 will remain on the manifold (i.e., satisfy (0.+1, 01.1) 1/c).
The empirical consequences of this geometric error are severe, as illustrated in Figure
Optimizer Convergence in Hyperbolic Space
Optimizer
2.0
Riemannian Adam (RAdam)
Naive Adam (Cuclidean)
Tmining Loss
1.5
1.0
0.5
200
400
600
800
Training Steps
Figure 3: Convergence of Riemannian Adam vs. Naive Adam in Hyperbolic Space. The training loss curves demonstrate the critical importance of manifold-aware optimizer. Riemannian Adam (RAdam) exhibits fast and stable convergence, efficiently navigating the curved loss landscape. In contrast, Naive Adam shows highly unstable and inefficient optimization trajectory, characterized by severe oscillations and failure converge to low-loss solution. This behavior is a direct result of its geometrically invalid updates, which fail to respect the structure of the hyperbolic manifold.
The results are unequivocal. RAdam demonstrates smooth, rapid, and stable convergence to low-loss region. In stark contrast, the Naive Adam optimizer exhibits highly erratic and inefficient trajectory. Its loss function oscillates dramatically and fails to converge consistently. This instability arises from two technical failures:
1. Deviation from the Manifold: The Euclidean update steps continuously push the model's parameters off the surface of the Lorentz hyperboloid. While one could add projection step force them back, this introduces bias and does not fix the underlying issue. This deviation leads to numerical errors subsequent geometric computations (e.g., log maps), causing training instability.
2. Incorrect Gradient Aggregation: Without parallel transport, the momentum updates in- correctly combine vectors from different, misaligned tangent spaces. This results in a corrupted estimate of the true gradient direction, leading the optimizer on suboptimal, oscillating path through the parameter space.
This experiment provides compelling evidence that naive application of a Euclidean optimizer is fundamentally unsuited for deep learning in hyperbolic space. The of geometrically principled optimizer like Adamm not merely minor improvement but an absolute prerequisite for successful and reliable training.
D Loss FUNCTION FORMULATIONS FOR DOWNSTREAM TASKS
To ensure full reproducibility and clarity our experimental methodology, this section provides the precise mathematical formulations of the loss functions used for each of the three downstream tasks evaluated in Section 4 The choice of loss function is tailored to the specific objective of each task.
D. CODE CLONE DETECTION: BINARY CROSS-ENTROPY LOss
Task Formulation. Code clone detection is framed as a binary classification problem. Given pair of code snippets, (C1, C2), the model must predict a binary label y E [0, where = indicates that the snippets semantic clones, and y indicates they not.
20

---

Published as a conference paper at ICLR 2026
Model Output. For each code snippet C₁, our model computes a graph-level representation ha, via Fréchet mean pooling, which is then mapped to Euclidean vector ²G, R (as described in Ap- pendix B..). To produce a single prediction for the pair, we combine these two vectors. standard and effective method for this is to compute both their element-win product (Hadamard product) and their absolute difference, concatenating them to capture both similarity and dissimilarity signals:
Upair CONCAT (=G, (= G |2 2G, a GG, E 1gd
2
(34)
This combined vector is then passed through the final MLP classification head to produce a single scalar logit, E R.
Loss Function. We use the standard Binary Cross-Entree with Logits Loss (BCEWithL logit- sLoss), which combines a Sigmoid activation layer and the Binary Cross-Entropy loss in one nu- merically stable operation. For single training pair with ground-truth label and predicted logit a, the loss is calculated
LACE o) = - ly log (0)(sigmoid(r)) + (1 - y) log(1 sigmoid(a))]
(35)
where sigmoid(a) 1/(1 + + is the predicted probability of the pair being a clone. The total loss for mini-batch is the average of this value over all pairs in the batch.
D.2 CODE COMPLETION: MASKED LANGUAGE MODELING
Loss
Task Formulation. Line-level code completion is formulated as a masked language modeling (MLM) task. Within a given code context, one more tokens corresponding the target line are masked. The model's 's objective is to predict the original token for each masked position from fixed vocabulary V. This is multi-class classification problem.
Model Output. For each masked token position i, HypeCodeNet produces a final node represen- tation ( which is mapped to Euclidean output vector 20t Rd This vector is then passed through the final MLP head, which projects to the dimension of the vocabulary, producing a logit vector a; E T MI, Each element øig represents the model's un-normalized score for the j-th token in the vocabulary being the correct one for position i.
Loss Function. We use the standard Cross-Entropy Loss, which is ideal for multi-class classifi- cation. It combines LogSoftmax and a Negative Log- Like-iiel loss. For single masked token at position i, let yi the ground- tround-truth token index. The loss is calculated as:
exp(Os..1)
CCE( σ.) - log
(36)
exp(or.)
This loss penalizes the model based on how much probability it assigns to the incorrect tokens. If multiple tokens are masked in the target line, the total loss for the code snippet the average of the cross-entropy losses over all masked positions.
D.3 LINK PREDICTION: BINARY CROSS-
CROSS-ENTROPY ENTROPY LOSS
Task Formulation. Link prediction in the function call graph is also formulated binary clas- sification task. Given a pair of nodes (functions), (u, v). the model must predict whether directed edge (u, v) (representing a function call from u to v) exists in the graph. For each true edge in the training set, we sample one or more negative edges (pairs of nodes with connecting edge) to create a balanced training objective.
Model Output. After training, the model produces a final Euclidean representation for each func- tion node, zu and zout, To predict the existence of an edge, we need a function that maps these two vectors to a single score. We employ a simple yet effective method: a learnable bilinear transfor- mation, which can model complex interactions between the source and target node representations. This produces& single logit owv
Oue (2 T 2innk2
(37)
where Wink E dxd is learnable weight matrix.
Loss Function. As with code clone detection, this is a binary prediction problem. We again use the Binary Cross-Entropy with Logits Loss. For given pair (u, v) with ground-truth label you
21

---

Published as a conference paper at ICLR 2026
{0, 1} and predicted logit duv. the loss is identical in form that of clone detection: (BC Fuv)
[yuv log(sigmoid(Au.)) + (1 Yuv) log(( (o
(38)
The total loss is averaged over all positive and negative samples in a mini-batch. This formulation encourages the model to leam embeddings where functions that likely to interact have high
score under the bilinear transformation.
E PSEUDOCODE FOR LORENTZ MODEL OPERATIONS
To ensure clarity and reproducibility, this section provides detailed pseudocode for the ge- ometric operations on the Lorentz manifold used throughout HypeCodeNet.. These operations form the computational foundation for the model's manifold-aware components. All operations are presented for a d-dimensional Lorentz model L with negative curvature c < 0. We as- sume access to fundamental function, 1 lorentz inner product (x, y), which computes (a, v) Toll + Σ Fays..
Logarithmic Map. The logarithmic map, (u),, projects a point u from the manifold into the tangent space at point p. The resulting vector represents the direction and magnitude of the shortest path (geodesic) from p to u.
Algorithm Logarithmic Map: (u)
Input: Points p, Cg curvature <
2: Output: Tangent vector Ty
3:
4: a -c- - low.entz_innerrprodieiie u)
Compute the Lorentz inner product term
5:
6: if'a S 1 then
Handle numerical precision issues at the boundary is small constant, e.g., 1e-8
7: 2-6-13 1
8:
end if'
9:
dpu + - arcosh(a:)
Calculate the geodesic distance
12:
D A stable way to compute the coefficient
dpu
(um)) -1/0)
coeff d V
15: F 6 coeff (u -u-e e- lorentz_jiner_ppoduucee. u) p)
16: return
Exponential Map. The exponential map, exp)(u), is the inverse of the logarithmic map. It takes a tangent vector from the tangent space at and maps it back onto the manifold by "traveling" along the geodesic defined by
22

---

Published as a conference paper at ICLR 2026
Algorithm 2 Exponential Map: exp;(u)
1: Input: Point p E C tangent vector TyL% curvature < O.
2: Output: Point C%
3:
4: norm < V /max(0,, lorentz inner prrduuctr, u))
Compute the norm in the tangent space
5:
6: if norm v < then
b Handle the zero-vector case
7: return p
8:
end if
9:
0 <- V norm
11: coeff, + (0)
12: coeff, sinh(0)
Use stable implementation for sinh(x)/2 near
13:
14: u t -cefffp p p+ coeff, return
Parallel Transport. Parallel transport is a fundamental operation for Riemannian optimization (e.g., in RAdam). It provides geometrically sound method for moving langent vector u from one langent space, TpCe, to another, T. along the unique geodesic connecting points p and The process ensures that the vector's length and its angle relative to the geodesic are preserved during transport.
The formula for parallel transport in the Lorentz model involves vectors that originate from different langent spaces (e.g., log,u. from Tz and logup from T. A naive addition of these vectors seems geometrically ill-defined. However, it is crucial to understand that this operation is a projec- tion performed within the ambient Euclidean space gd+1 in which the manifold is embedded. The term (logpu. + logup) represents the component of vector that is orthogonal to the target tangent space Tufe after being moved. By subtracting the correct projection of this component from the original vector v, the resulting vector is guaranteed to lie within the target tangent space T.cd (i.e., (v', u) 0), thus satisfying the geometric requirements of transported vector.
Algorithm Parallel Transport: (v)
1: Input: Points p, u E Ld, tangent vector u E T Cp curvature e < 0.
2: Output: Transported tangent vector E T.C.
3:
4: log log mpp(p, u, c)
Vector from p u, lies in T Vector from u to lies in Cg
5: logup f- log map(u, (ap(u,,) p,
6:
7: d2 pu
+ lorentz inner product(log,.. log,u)
Squared geodesic distance
8:
9: if 22 < then
If points identical, transport is needed
10: return v
11: end if
13:
b The following operations occur in the ambient space dR++1
14: inner sirer_prod- +- incentz.inner. produeerrpoout(r,,..mmm log.p)
15: coeff <-
Pm
16:
This projection correctly maps v E Tp v' E To
18: o <- coeff (logpu + logup)
19: return v
23

---

Published as a conference paper at ICLR 2026
F CONNECTION TO EUCLIDEAN GEOMETRY AS CURVATURE APPROACHES ZERO
key motivation for our curvature annealing strategy is the fact that the Lorentz model of hyperbolic geometry, with negative curvature smoothly degenerates to standard Euclidean geometry - 0-. This property allows our model to begin training nearly-flat, stable space and gradually adapt to the more expressive curved geometry. This section provides the mathematical derivations for this convergence for the three core geometric operations: the geodesic distance, the exponential map, and the logarithmic map.
1. Geodesic Distance.
The geodesic distance between two points u, v E Lg is given by:
de(u, V-c arcosh(-c((u)
(39)
To analyze the limit O-, consider the Euclidean vectors ug, Ug R that correspond to the points u, u on the manifold. For points close to the origin a space with very low curvature, the term -c, e)e can be well approximated by:
-c(u, v) ~
- |
(40)
2
Now, we use the Taylor series expansion for arcosh(1 + +r) for small x 2 0, which is arcosh (1+ R V2r. Let x 2 u ugll?. As e + O x also approaches O. Substituting this into the distance formula gives:
-C |lu wzll
(u, ~
arcosh
1
2.
| ug 112
2
| -c.
(v-c. llug
UEll)
+ |ug Vgll This shows that the curvature vanishes, the geodesic distance the manifold smoothly converges to the standard Euclidean L2-norm distance.
Exponential Map.
The exponential map at point p E C. for a tangent vector E Tppe is:
(x) cosh cosh(v-cllelll) sinh(v-clle
(41)
V-clju|l
Let = /-cllull As + 07, We use the Taylor series expansions for cosh(O) and sinh(O) around 0:
cosh(() = 1 8a 0000) +1
sinh(0)
+ 0 +
8²
6 + 0(8*) +1
Substituting these limits back into the exponential map formula, get:
lim c-0 (xp = -1) v==
(42)
This demonstrates that the exponential map, which represents movement along a geodesic on the manifold, becomes simple vector addition in the limit, which is the corresponding operation in Euclidean space.
3. Logarithmic Map. The logarithmic map is the inverse of the exponential map. Given the smooth convergence of the exponential map its Euclidean counterpart, the logarithmic map must also converge to the inverse operation of Euclidean vector addition, which is vector subtraction.
If u (x)) then log((u).
(43)
24

---

Published as a conference paper at ICLR 2026
Taking the limit as e - 0 on both sides:
lim c00
u lim (xp =
11 = p
(44)
c-+
Solving for v, we find
p. Therefore:
lim log((u) (u)
(45)
The logarithmic map, which finds the tangent vector representing the shortest path between two points on the manifold, smoothly becomes vector subtraction.
Justification for Curvature Annealing as a Learning Curriculum for Code. These mathemati-
cal derivations provide strong theoretical foundation for curvature annealing strategy, framing it as a form of **curriculum learning** that is particularly well-suited to the nature of source code. The effectiveness of this strategy goes beyond mere numerical stability; it guides the model through a structured learning process that mirrors how complex program semantics are built upon simpler syntactic foundations.
The curriculum unfolds in two distinct phases:
1. Phase 1: Learning Local Syntactic Patterns (Low Curvature). In the initial stages of training, the curvature e is very close to zero, making the geometry nearly Euclidean. In this regime, model's operations approximate those of standard Euclidean Graph Neural Network. The loss landscape is smoother and less complex, allowing the model to focus on learning the "easy" patterns first. For source code, this corresponds to local, syntactic relationships: token co-oceurrence (e.g., 'public static void'), simple expressions (a=b+c') and immediate parent-child connections in the AST. The model first learns the fundamental grammar and local structure of the code in stable, well-behaved geometric space.
2. Phase 2: Learning Global Hierarchical Semantics (Increasing Curvature). As training progresses, the curvature is gradually annealed towards its target value of -1. This process slowly "warps" the space, introducing the powerful hierarchical inductive bias of hyper- bolic geometry. The model, having already learned the basic syntactic building blocks. is now guided arrange these blocks into meaningful global structure. To continue mini- mizing the loss, it must learn to place representations in a way that respects the geometry's 's exponential growth. This forces it to capture deep, hierarchical semantics such as variable scope, control-flow nesting, class inheritance, and long-rang pong-range dependencies between func- tion calls. The nodes representing higher-level scopes are naturally positioned closer to the origin, while their subordinate components are pushed outwards into distinct branches.
Starting directly with a high-curvature space would be akin to asking the model to understand the global architecture of program without first learning its basic syntax. The strong hierarchical con- straints could overwhelm the optimization process from the outset, leading chaotic placement of embeddings and potential numerical instability. By contrast, curvature annealing provides a smooth and natural learning trajectory. It allows the model to first master the local "what" of code before progressively guiding it to understand the hierarchical "why," making a principled and highly effective strategy for learning the complex, multi-scale structure of software.
G ADDITIONAL EXPERIMENTAL RESULTS AND ANALYSES
To provide a more comprehensive understanding of HypeCodeNet's architectural strengths and the contributions of its core components, we present a series of extended experimental analyses. This section expands upon the results presented in the main paper, including more detailed ablation study, an analysis of the model's sensitivity to key hyperparameters, and qualitative error analysis.
G.1 FULL ABLATION STUDY RESULTS
The ablation study presented in the main paper (Section 4.5) focused on the F1-score to demon- strate the impact of key model components. Here, we provide a more complete picture by including Precision and Recall metrics on the BigCloneBench dataset. Furthermore, we introduce additional ablation variants to investigate the necessity of other fundamental design choices within our frame- work.
25

---

Published as a conference paper at ICLR 2026
The extended results summarized in Table 7 component of HypeCodeNet crucial for achieving its state-of-the-ard performance. The new vari- ants investigate:
The analysis reinforces our central claim: each
w/o Residual Connection: In this variant, remove the residual nature of the update step. Instead using the exponential map exp,, 1, (-), we first map the representation the
origin's tangent space, add the update vector, and map it back, effectively performing the
update global tangent space rather than locally.
Mean Aggregation: We replace the coometrically-awan attention mechanism in the tan- gent space with simple, unweighted mean of all neighbor message vectors.
Poincaré Ball Backend: We re- re-implement the entire model using the Poincaré Ball model of hyperbolic geometry instead of the Lorentz model, to empirically test the stability and performance trade-offs.
Table 7: Complete results of the ablation study on the BigCloneBench dataset. This table extends the analysis in the main paper by including Precision and Recall, and investigating additional architec- tural components. The consistent performance drop across all variants underscores the importance of each component in HypeCodeNet's design.
Model Variant
Precision
Recall
Fi-score
Full HypeCodeNet Model Full Model
0.941
0.939
0.940
Ablation of Core Geometric Components
0.925
0.921
0.923
w/o Hyperbolic Attention
0.933
0.931
0.932
w/o Curvature Annealing
0,936
0.934
0.935
Newly Added Ablations
w/o Residual Connection
0.928
0.924
0.926
Mean Aggregation
0.921
0.918
0.920
Poincaré Ball Backend
0.934
0.930
0.932
The results from the new variants provide further insights:
The significant performance drop in the w/o Residual Connection variant highlights the importance of the local, geometrically-sound update via the exponential map. This hyper- bolic residual connection is critical for stably training deep stack of GNN layers.
Replacing attention with Mean Aggregation leads to one of the largest performance drops, second only to removing the hyperbolic geometry itself. This confirms that simple aggre- gation of structural neighbors is insufficient; the model must learn to dynamically weight the importance of different neighbors based on their geometric relationships to capture complex program semantics.
The Poincaré Ball Backend variant also shows slight degradation in performance. While conceptually equivalent, we observed that this backend was more prone numerical insta- bility during training, requiring more careful hyperparameter tuning and gradient clipping, which aligns with findings in the broader hyperbolic deep learning literature and justifies our choice of the Lorentz model.
Collectively, this comprehensive analysis demonstrates that HypeCodeNet's performance is not at- tributable to single component, but rather to the synergistic interplay of its hyperbolic geometry, manifold-aware attention, stable residual updates, and principled optimization in the Lorentz model.
G.2 HYPERPARAMETER SENSITIVITY ANALYSIS
To demonstrate the robustness of HypeCodeNet and to provide empirical justification for our choice of key hyperparameters, we conduct a sensitivity analysis on the BigCloneBench dataset. In this analysis, we vary one hyperparameter at a time while keeping all other parameters fixed at their optimal values as used in our main experiments. The results, visualized in Figure 4 illustrate the model's 's performance response to changes in four critical hyperparameters: the embedding dimen- sion (d), the number of GNN layers (L), the number of attention heads (K), and the learning rate.
Our observations from this analysis are as follows:
26

---

Published as a conference paper at ICLR 2026
Hyperparameter Sensitivity Analysis BigClone BigCloneBench Bench
Impact of Embedding Dimension
Impact of GNN Layers
0.9400
0.940
0.9375
0.938
F1 6.arr
0.9225
0.934
0,9300
0.932
Selected Model (0.94)
Selected Model (0.94)
0.9250
0.928
100
400
500
800
Embodding Dimension
Number Layers
Impact of Attention Heads
Impact of Learning Rate
0.939
0.938
0.938
0.936
F1
0.934
0.932
Selected Model (0.94)
Selected Model (0.94)
0.935
12
16
0.4
0.6
0.8
1.0
Number Heads
Learning
16-4
Figure 4: Sensitivity analysis of HypeCodeNet with respect to four key hyperparametens: embed- ding dimension (d), number of GNN layers (L), number of attention heads (K), ), and learning rate, All experiments are evaluated based on the Fi-score on the BigCloneBench validation set. The red dashed line indicates the performance of our final model configuration, demonstrating that our cho- sen parameters lie at or near the performance peak for each setting, and the model remains robust to minor variations.
Embedding Dimension (d) The model's performance generally improves as the embedding di- mension increases, which is expected as higher dimension provides greater representational pacity. However, we observe diminishing returns, with the Fl-score plateauing after d 512. Our choice of d 768 aligns with standard BERT-base architectures for fair comparison and sits comfortably on this performance plateau, indicating that further increases would likely offer no significant benefit while substantially increasing computational cost.
Number of GNN Layers (L) The analysis reveals clear optimal range for the model's depth. Performance peaks at = layers. With fewer than 4 layers, the model likely lacks the capacity to propagate information across larger code structures (underfitting). Beyond 8 layers, we observe slight but consistent degradation in performance. This is well-known phenomenon in GNNs, of- ten attributed to smoothioo where node representations become increasingly indistinguishable with excessive message passing. Our choice of L 6 provides the best balance between expressive power and the risk of over-smoo
Number of Attention Heads (K) The number of attention heads also exhibits distinct sweet spot. The performance is optimal with K = 8 heads. Using fewer heads (e.g., K 4) may restrict the model's ability to attend to diverse structural patterns simultancously. Conversely, using too many heads (e.g.. K = 16) can make each head's subspace too small, potentially hindering its ability learn complex relationships. Our selection of K proves to be an effective choice.
As is common in deep learning models, performance is highly sensitive to the learning rate. A rate that is too low (e.g.. 1 x 10 $) slows convergence and may result in suboptimal performance, while rate that too high (e.g., x 10-4) leads to unstable training and sharp drop in performance. Our chosen learning rate of 5 x 10 is situated precisely at the peak of the performance curve, validating it as the optimal choice for this task.
Learning Rate
27

---

Published as a conference paper at ICLR 2026
In summary, this analysis confirms that our selected hyperparameters not arbitrary but pirically validated to at or near optimal. The model demonstrates reasonable robustness to small deviations from these values, reinforcing the stability and reliability of our experimental results.
G.3 QUALITATIVE ERROR ANALYSIS
To better understand the limitations and current boundaries of HypeCodeNet,, we performed qual- itative error analysis on challenging cases from the BigCloneBench validation set. By examining specific instances of False Positives (non-ciones incorrectly identified as clones) and False Negatives (clones that the model failed identify), we can gain valuable insights into the model's reasoning process and identify promising directions for future work.
G.3.1 ANALYSIS OF FALSE POSITIVES: THE CHALLENGE OF SUPERFICIAL SIMILARITY
False Positives typically occur when two code fragments share significant lexical or structural sim- ilarities, which can mislead the model into overlooking critical semantic difference. Figure presents canonical example of such a case.
private void resetState ()
if (a.val. (1)
private void setState () if (a.val a.state = true;
a.state true;
else { a.state false;
5
6
Listing (2) Code A: simple state setter based on a zero-check.
Listing (3) Code B: A conditional state resetter based parity check.
Figure 5: An example of False Positive. Despite structural similarities (e.g.. modifying 'a.state" inside an 'if' block), the core logic of these two functions is semantically distinct.
Analysis. At a glance, these two functions appear related. Both have similar names ('setState', "resetStatee,, access the same object 'a', and modify the same field "a.state" based on condition involving valval' A model relying heavily on token co-occurrence or shallow AST structure could easily mistake them for clones. The ASTs for both would contain similar node patterns, such as 'IfStatement', 'FieldAccess', and 'Assignment'.
However, the core semantic logic fundamentally different. Code performs specific value check ('a.val = 0*), while Code B performs a property check ('a.val.isEvved) This subtle distinction completely alters the function's behavior and intent. HypeCodeNet's failure in this scenario sug- gests that even with low-distortion hierarchical embeddings. the overwhelming structural similarity in very small code snippets can sometimes overshadow the semantic weight of a single, critical operation (e.g., the method call "iisEven()'). This highlights key challenge: learning to assign appropriate importance to specific API calls or operators within a larger, shared structure.
G.3.2 ANALYSIS OF FALSE NEGATIVES: THE CHALLENGE OF IMPLEMENTATION DIVERGENCE
False Negatives represent a more profound challenge, often occurring when two code fragments implement the same functionality (i.e., they semantic clones) but vastly different algorithms or syntactic structures. Figure shows a classic example: computing a factorial using recursion versus iteration.
Analysis. These two functions are perfect semantic clones; for the same input, they produce the same output. However, their implementations are structurally divergent. Code C uses recursion, which results in an AST characterized by chain of nested 'MethodInvocation'' nodes. In contrast, Code uses a FooSStatement, leading to a flatter, loop-based AST. The variable names and even parameter types also differ.
model's s failure to identify these as clones reveals the fundamental gap between representing struc- ture and understanding algorithms. Even HypeCodeNet, which excels at creating high-fidelity em- beddings of *given* structure, struggle to recognize that two topologically distinct AST's
28

---

Published as a conference paper at ICLR 2026
public static long factorial (long if (n <= 1) { return 1; else return factorial (n 1)
public static long factorial (int
nun) {
2 long fact 1;
for (int - 1 <- numy i++)
fact i;
6
return fact;
Listing (4) Code C: Factorial computation using a recursive implementation.
Listing (5) Code D: Factorial computation using an iterative implementation.
Figure 6: An example of False Negative. Both functions compute the factorial and are therefore semantic clones (Type-3/4), but their syntactic and structural representations are entirely different.
algorithmically equivalent. Mapping these disparate graph structures to nearby points in the hyper- bolic embedding space requires level of abstraction that goes beyond mere structural preservation. This type of error not justust a limitation of our model but represents frontier in the field of code rep- resentation. It underscores the need for future research into methods that can Iearn representations invariant to algorithmic implementation details, moving closer to true program comprehension.
G.4 VISUALIZATION OF EMBEDDING DISTORTION
A core theoretical advantage of hyperbolic geometry is its ability embed tree-like structures with significantly lower distortion than Euclidean space. To provide a clear, empirical demonstration of this property, we visualize the relationship between the ground-truth distance of nodes in an Abstract Syntax Tree (AST) and their corresponding distance in the learned embedding space. The "Graph Distance"' is the shortest path length between two nodes in the AST, while the "Embedding Distance" is the geodesic (or Euclidean) distance between their learned representations.
Ideally, an embedding should be isometric, meaning the embedding distance would be directly pro- portional to the graph distance, forming a straight line. Any deviation from this line represents representation distortion. Figure 7 plots this relationship for both HypeCodeNet and a strong Eu- clidean baseline.
As illustrated in Figure 7 the results are striking. The curve for HypeCodeNet maintains a near- linear relationship with the ideal line, demonstrating that it successfully preserves the structural distances from the AST. The tight confidence interval (shaded area) further suggests that this preser- vation is highly consistent. In stark contrast, the Euclidean baseline exhibits a classic case of rep- resentational collapse. For nodes with small graph distance, performs reasonably well, but as the distance increases, the curve flattens significantly. This shows that distant nodes in the AST are being "squashed" into crowded region of the embedding space, losing their crucial hierarchical information. This visualization provides direct, compelling evidence for our paper's s central thesis.
To provide more qualitative and intuitive understanding of this distortion, we visualize the em- bedded positions of all nodes from sample Abstract Syntax Tree. Figure 8 offers a side-by side-by-side comparison of the final topological structures learned by a Euclidean model versus our hyperbolic HypeCodeNet.
The visual evidence presented in Figure 8 is unequivocal. The Euclidean embedding (a) fails to preserve the global structure of the AST. While may maintain some local neighborhood informa- tion, the overall hierarchy is lost in a "collapsed" cluster of points, making it impossible to visually discern the program's 's structure. In complete contrast, the hyperbolic embedding (b) produced by HypeCodeNet provides a faithful, low-distortion representation of the AST. The root is placed at the origin, with its children and their descendants arranged in clear, radiating pattern that perfectly reflects the underlying hierarchy. This topological visualization serves as a powerful and intuitive confirmation of our central thesis: hyperbolic geometry is the natural geometric language for repre- senting the inherent structure of code.
29

---

Published as a conference paper at ICLR 2026
Embedding Distortion Analysis
HypeCodeNeel (Hyperboli))
Baseline (Eudidean))
ideal (No Distortion)
Embedding Space Distance
12
10
8
8
12 14
Graph Distance AST (Path Length)
Figure 7: Analysis of Representation Distortion. The plot shows the mean and standard deviation (shaded area) embedding space distances as a function of the true graph distance in the AST. (1) HypeCodeNet (Hyperbolic) closely tracks the ideal distortion frreioono-eeeeee line, indicating its ability faithfully preserve the hierarchical structure of the code. (2) The Euclidean Baseline suffers from significant distortion; as nodes become farther apart the AST, their representations begin to "collapse" in the embedding space, evidenced by the saturating curve and increasing variance. This visualization provides strong evidence for the geometric superiority of hyperbolic space for code representation.
Qualitative Comparison of Embedded AST Structures
Hyperbolic Embedding (Ours)
Euclidean Embedding
Depth is
"ookapadd
ZElyarrchy with not-alcevier stratore.
Figure 8: Topological Visualization of Sample AST Embedding. Both plots show the 2D pro- jected positions of all nodes from the same tree, colored by their depth (darker = closer to the root). (a) Euclidean Embedding: The hierarchical structure is lost nodes are "collapsed"" into dense, undifferentiated cluster. Distinguishing parent-child relationships or identifying subtrees is nearly impossible. (b) Hyperbolic Embedding (HypeCodeNet)): The tree's seess hierarchy is naturally preserved. The root node is located the center of the Poincaré disk, and its descendants radiate outwards towards the boundary as their depth increases. The clear separation between subtrees pro- vides powerful, direct visual evidence of hyperbolic geometry's 's superior capacity for representing hierarchical data like source code.
H DATASET DETAILS AND PREPROCESSING
This section provides a detailed description of the datasets used in our experiments, along with comprehensive overview of our data preprocessing pipeline. Our goal is to ensure full reproducibil-
30

---

Published as a conference paper at ICLR 2026
ity by documenting the statistical properties of the data and the exact steps taken to transform raw source code into a format suitable for HypeCodeNet.
H.1 DATASETS FOR EXPERIMENTAL EVALUATION
We selected four widely recognieed datasets across three distinct program reasoning tasks. The statistical properties of these datasets are summarized in Table
Table 8: Key statistics of the datasets used for evaluation. LOC denotes Lines of Code, and AST nodes are estimated based on standard parsing tools. For the GitHub Java Corpus, statistics refer to the constructed function call graph.
Dataset
Metric
Value (Approx.)
Task
Total Pairs
900,000
Split (Tt/V/T)
800k / 50k 50k
BigCloneBench
Avg. LOC
25-40
Avg. AST Nodes
150-250
Clone Detection
Total Samples
52,000 (from 104 problems)
Split (Tr/V/T)
32k 8k/ / 12k
POJ-104
Avg. LOC
40-60
Avg. AST Nodes
100-200
Total Samples
251,862
CodeXGLUE (Python)
Split (Tt/V/T
249.8k 1 / 1k
Avg. Context
5-10 Lines
Code Completion
Total Samples
182,000
CodeXGLUE (Java)
Split (Tr/V/T)
180k / Ik 1k
Avg. Context
6-12 Lines
Graph Nodes
5-10 Million (Methods)
Graph Edges
20-50 Million (Calls)
GitHub Java Corpus
Link Prediction
Edge Split (Tr/V/T)
80% / 10% / 10%
Avg. Degree
4-6
BigCloneBench (BCB). This is large-scale, real-world dataset for Java clone detection, sourced from over 25,000 open-source projects the IJaDataset repository. Its primary strength lies in its fine-grained categorization of clones, including significant portion of challenging semantic clones (Type-3 and 4ype-4), which constitute approximately 15-20% of the true clone pairs. For our experiments, we use a standard split of 800,000 pairs for training and 50,000 pairs each for validation and testing. Negative samples (non-clones) are generated by pairing code fragments from different functionalities, maintaining a balanced 1:1 ratio of true to false clones during training.
POJ-104. This dataset is designed for algorithmic code clone detection and consists of 52,000 C/C+ solutions to 104 distinct programming problems from an online judge system. A key char- acteristic is its high degree of implementation variance for the same problem, making it ideal for evaluating model's ability to capture algorithmic semantics over syntactic form. The dataset is partitioned by problem ID to prevent data leakage, with 64 problems for training, 16 for validation, and 24 for testing. This partitioning scheme ensures that the model must generalize entirely new algorithmic problems at test time.
CodeXGLUE (Line-Level Completion). We use the Python and Java subsets of the COLUX benchmark for the line-level code completion task. The Python data is derived from the PY150 dataset, and the Java data is from the GitHub Java Corpus. In this task, single line within code snippet is masked, and the model must predict the missing line. The context provided typically ranges from to 12 lines of preceding code, requiring the model to understand local variable scope, method context, and syntactic structure.
GitHub Java Corpus (for Link Prediction). To evaluate pure structural reasoning, structed a large-scale function call graph (FCG) from a curated snapshot of the GitHub Java Corpus, which contains approximately 100 GB of code from over 10,000 high-quality projects. The result- ing graph contains millions of nodes (methods) and tens of millions of edges (calls). Critically, the
31

---

Published as a conference paper at ICLR 2026
graph's 's degree distribution exhibits clear power-law characteristic (a: = 2.5), hallmark of scale- free networks with inherent hierarchical and community structures. This property makes a perfect testbed for our hypothesis, as such structures are precisely what hyperbolic geometry is theorized represent with low distortion. For the link prediction task, randomly hide 20% of the edges, using 10% for validation and 10% for testing, while ensuring the training graph remains connected.
H.2 PREPROCESSING PIPELINE
Our preprocessing pipeline is designed to transform raw source code into a structured graph rep- resentation (specifically, an Abstract Syntax Tree) that serves as the input to HypeCodeNet. The pipeline consists of four main stages:
1. Code Parsing and AST Generation. For each code snippet, we first generate its Abstract Syntax Tree (AST). We employ robust, industry-standie parsers to ensure fighffideitty tree con- struction:
Java and C/C++: We use the Cree-sitten parsing framework, which is known for its speed, robustness syntax errors, and consistent output across different languages.
Python: We utilize the built-in ast module, which provides a reliable and standardized AST representation.
During parsing, we discard code comments and string literals to focus the model on the structural and logical aspects of the code. The output of this stage is a raw tree structure where each node represents a syntactic construct (e.g.. 'IfStatement', "Methodnnoccation',,
Node Feature Initialization Each node in the AST must be initialized with numeric feature vector. We adopt sub-word tokenization strategy to handle the large and often out-of-vocabulary nature of identifiers in source code.
The code token associated with each AST node (e.g., the function name cal calculate-total for a Furcti ienieiinnaaaaai node) is segmented into sub-word units using pre-trained Byte-Pair Encoding (BPE) tokenizer with vocabulary size of 50.265.
Tokenization:
(ii) Embedding Lookup:
These sub-words are then mapped to dense vectors using the to- kenizer's is embedding lookup table, resulting in initial 768-dimensional feature vector x. for each node v. For nodes representing syntactic types without specific token (c.g., ForStatemert)). we use a special placeholder token (e.g., <for>) that has its own embed- ding.
This approach allows the model to capture semantic similarities between related identifiers (e.g., total-sum and sum-value) and gracefully handle unseen variable names.
Graph Construction. The AST naturally defines the graph structure. We construct graph G E) where V is the set of all nodes the AST and is the set of directed edges representing parent-child relationships. An edge (u, v) E E' exists if is the parent of v in the AST. This raw hierarchical structure the primary input for HypeCodeNet. For tasks that do not inherently rely on ASTs (like link prediction on the FCG), the graph is constructed directly from the dataset's specified relations (i.e., function calls).
4. Data Serialization. Finally, the constructed graphs, along with their node features, seri- alized into a binary format using Py PyTocc Geometric's data structures. This allows for efficient loading and batching during the training and evaluation phases. Each serialized sample contains the edge index list, the node feature matrix, and the corresponding label for the downstream task.
I BASELINE IMPLEMENTATION AND FAIR COMPARISON
To ensure rigorous and scientifically sound evaluation, we assert that all comparisons made in this work are grounded in a principle of strict fairness. The performance metrics reported for all baseline models in Section 4 are the result of our own re-implementation and re-training efforts within unified experimental framework, rather than being directly cited from their original publications. This approach is critical for climinating confounding variables arising from differences in data splits, preprocessing techniques, or hardware environments.
32

---

Published as a conference paper at ICLR 2026
Our methodology for ensuring fair comparison is as follows:
Unified Data Pipeline: All baseline models were trained, validated, and tested on the exact same data splits generated by the preprocessing pipeline described in Appendix H For graph-based baselines (c.g.. GCN, GAT, GraphCodeBERT), the input graphs (ASTs) were constructed using the identical parsing and feature initialization process as for Hy- peCodeNet. This guarantees that all models begin with the same structural and semantic information.
Implementation Sources:
Whenever possible, we utilized the official, publicly available source code provided by the authors of the original baseline models. For models where of- ficial code was not available, we meticulously re-implemented their architectures following the specifications and mathematical formulations detailed in their respective papers.
Hyperparameter Optimization: We conducted thorough hyperparameter search for each baseline model on each task. Key hyperparameters, including learning rate, batch size, dropout rate, and the number of layers, were tuned on the validation set to ensure that each baseline was evaluated its optimal performance level within our experimental setup. Our reported results reflect these optimized configurations. For fair comparison of model capacity, all models with Transformer-base backbones (c.g., CodeBERT, Graph- CodeBERT, CodEFORMER) were configured the "base" size, with hidden dimension of 768.
Consistent Evaluation Protocol: All models evaluated using the same metrics and evaluation scripts. This consistency ensures that the observed performance differences are attributable the models' architectural and geometric inductive biases, which is the central focus of our investigation.
By adhering to this stringent protocol, we are confident that our experimental results provide a fair and direct comparison of HypeCodeNet's capabilities against the existing state-of-the-art.
J TRAINING INFRASTRUCTURE AND COMPUTATIONAL COST
This section details the hardware and software environment used for training and evaluating Hy- peCodeNet, reports the empirical training durations across key tasks, and provides a comparative analysis of model parameters and inference speed against the GraphCodeBERT baseline.
J.1 TRAINING INFRASTRUCTURE
All experiments were conducted on a nigh-performance computing cluster. A typical training node was configured with **4 x NVIDIA A100 GPUs (80GB)**, interconnected via NVLink. Our soft- ware stack included PyTorch 2.5.0, CUDA 12.5, and Geoopt 1.0 for hyperbolic operations. Multi- GPU training was managed using PyTorch's Distributed Data Parallel (DDP) framework, which ensured efficient scaling. The mixed-precision training strategy, combining FP64 for geometric sta- bility and FP32 for computational efficiency, was natively supported by this environment.
J.2 TRAINING DURATION
We report the total training time required for HypeCodeNet to converge on our primary tasks, as well the average duration per training epoch. All timings were measured the 4-GPU A100 node described above.
Table 9: Training duration for HypeCodeNet on key datasets. Total time reflects the wall-clock time required for the model converge based our early stopping criteria the validation set.
Dataset
Total Training Time
Epochs to Converge
Avg. Time per Epoch
BegChoneBench (BCB)
"16 hours
5
"3.2 hours
CodeXGLUE (Python)
"10 hours
-..5 hours
POJ-104
-7 hours
78
550 minutes
The training times, summarized in Table reflect the dataset scale and task complexity. Training on BigCloneBench is the most computationally intensive. The increased cost per epoch for HypeCo-
deNet primarily due the higher computational complexity the manifold-aware operations. Our analysis indicates that these core geometric operations (c.g.. logarithmic and exponential maps) introduce overhead of approximately ** 1-5-2xx** ** compared to their Fuclidean counterparts (e.g..
33

---

Published as a conference paper at ICLR 2026
vector subtraction and addition). However, this localized overhead is substantially mitigated the model level by optimized, batched implementations in Geoopt and our mixed-precision strategy, resulting in the more modest overall slowdown observed during inference. This trade-off between computational cost and representational power is central aspect of our model's design.
J.3 MODEL PARAMETERS AND INFERENCE SPEED
To contextualize the efficiency of HypeCodeNet, we compare its parameter count and inference performance against the strong GraphCodeBBERT-aus baseline. All inference metrics were bench- marked on a single NVIDIA A100 GPU using FP32 precision. To maximize GPU utilization, we used optimized batch size of 64 for GraphCodeBERT and 32 for HypeCodeNet.
Parameter Count. A key design advantage of HypeCodeNet is its parameter efficiency, which stems from its GNN architecture being inherently more compact than full Transformer. Table T0 provides a direct comparison.
Table 10: Comparison of parameter counts for models with a hidden dimension of d = 768.
Model
Total Parameters
GraphCodeBERT-baar (12-layer Transformer) HypeCodeNet (6-layer GNN, Ours)
125 Million
"8 Million
HypeCodeNet achieves its state-of-the-an performance with approximately 32
Inference Speed. While more parameter-effiiient, the geometric computations in HypeCodeNet incur a higher computational cost per forward pass. We measured two key metrics: the average latency to encode single code snippet (AST) and the corresponding encoder throughput.
Table 11: Inference speed comparison on single A100 GPU. AST size averaged from the BCB test set ("200 nodes).
Model
Avg. Latency AST
Encoder Throughput (ASTs/ Sec)
GraphCodeeBRRT-bas HypeCodeNet (Ours)
1.5 ms
-667
1.8 ms
"556
The results in Table I1 show that GraphCodeBERTTs highly optimized Transformer architecture leads to faster inference. HypeCodeNet's forward pass is approximately 1.2x slower, direct conse- quence of the Eun-Eucideann geometric operations. This modest slowdown, despite the higher cost of core geometric functions, is attributable two factors: (1) the model's shallower architecture (6 GNN layers vs. 12 Transformer layers), and (2) the fact that a significant portion of the computation is still performed by highly optimized Euclidean operations (e.g., linear transformations) in the tan- gent spaces. This highlights a clear trade-off: HypeCodeNet's superior ability model hierarchical structure comes at the cost of a modest increase in inference latency. However, as demonstrated in ablation studies (Table 4). HypeCodeNet achieves competitive performance even at much lower dimensions, offering path bridging this efficiency gap in future optimizations.
K QUANTITATIVE GEOMETRIC VALIDATION AND SENSITIVITY ANALYSIS
To address the need for rigorous geometric validation the learned representations beyond visual inspection, and to clarify the model's scalability, we conducted a series of quantitative analyses on the BigCloneBench test These experiments were designed to empirically verify whether HypeCodeNet truly leverages the hyperbolic geometry to represent the hierarchical structure of Ab- stract Syntax Trees (ASTs) and to assess its efficiency.
K.1 VALIDATION OF HIERARCHICAL EMBEDDINGS
We computed statistical metrics to quantify the correspondence between the graph-theoretic proper- ties of the AST nodes and their geometric properties in the learned Lorentz manifold. Specifically, we evaluated:
Depth-Norm Correlation: The Spearman's rank correlation coefficient (p) between node's depth in the AST and its hyperbolic distance the origin (norm) in the embed-
34

---

Published as a conference paper at ICLR 2026
ding space. A high positive correlation indicates that the model learns to place deeper
nodes closer to the boundary.
The average hyperbolic norm of root nodes versus leaf nodes. Structural Separation: The average pairwise geodesic distance for connected parent-child pairs (dpe) compared to random node pairs at the same depth (dsame-depth). )
Root vs, Leaf Placement:
The results, summarized in Table 12 confirm our theoretical hypotheses. We observe a strong positive correlation (p 0.89) between tree depth and embedding norm. Furthermore, the model effectively utilizes the exponential volume of the space: while parent-child nodes remain geometri- cally close (0.84), semantically distinct branches at the same depth separated by large geodesic distance (1 (11.35) 1355,, avoiding the "crowding" problem inherent Fuclidean space.
Table 12: Quantitative validation of hierarchical structure preservation in HypeCodeNet embed- dings. Metrics are computed on 1,000 randomly sampled ASTs from the BigCloneBench test set.
Metric
Value
Std. Dev.
Interpretation
Spearman's p (Depth vs. Norm)
0.89
#0.02
Strong preservation of hierarchy order.
Avg. Norm (Root Nodes)
0.12
0.05
Roots mapped near the origin.
Avg. Norm (Leaf Nodes)
6.81
11.2
Leaves are pushed to the boundary.
Avg. Distance (Parent-Child)
0.84
+0.15
Connected nodes stay locally clustered.
Avg. Distance (Same-Depte Random)
11.35
22.10
Distinct branches are exponentially separated.
K.2 DISTORTION ANALYSIS BY AST COMPLEXITY
To determine if the benefits of hyperbolic geometry hold consistently across different levels of code complexity, we stratified the test set ASTs into three groups based on their maximum depth: Shallow (< 10), Medium (10 20), and Deep (> 20). We measured the average representation distortion Seeg = ddeb dgraphl/ htrappl/4grap /graph for HypeCodeNet compared to the Euclidean baseline.
As shown in Table 13 the performance gap is correlated with structural complexity. While HypeCo- deNet offers a modest improvement on shallow trees (where Euclidean space often sufficient), its advantage becomes decisive on deep. complex ASTs, reducing distortion by over 60%. This con- firms that geometric approach is particularly effective for modeling deep, highly-nested program logic.
Table 13: Comparison of representation distortion (gaeg) stratified by AST depth. Lower is better. HypeCodeNet's advantage scales with the complexity of the code structure.
AST Complexity (Depth) Euclidean Baseline
HypeCodeNet (Ours)
Relative Improvement
Shallow (< 10)
0.18
0.15
+16.7%
Medium (10- 20)
0.32
0.19
+40.6%
Deep (> 20)
0.58
0.22
+62.1% %
K.3 MEMORY EFFICIENCY AND SCALABILITY
A key concern in geometric deep learning is the scalability of memory consumption relative to Eu- clidean models. While hyperbolic operations involve more complex scalar computations, HypeCo- deNet exhibits superior memory efficiency when scaling to large ASTs due to its high embedding capacity.
Memory consumption for GNNs is dominated by the storage of node feature matrices (O(VV) -d)), where [V] is the number of nodes and d is the embedding dimension. As demonstrated our Ablation Study (Table 4. HypeCodeNet achieves state-of-the-ard performance with a significantly smaller dimension (d 128) compared to the standard dimension required by Euclidean baselines (d (d=77 (68).
Table 14 illustrates that HypeCodeNet can reduce memory usage for node representations by factor of while maintaining superior performance. This makes our approach particularly well-suited for processing very large ASTs or for deployment in memory memooy-constrained environments, as the dimen- sionality reduction far outweighs the negligible memory overhead of manifold-aware operations.
35

---

Published as a conference paper at ICLR 2026
Table 14: Memory consumption comparison for storing node embeddings a large AST batch (approx. 100k nodes total). HypeCodeNet achieves comparable or better accuracy with significantly lower memory footprint.
Model Configuration
Hidden Dim (d)
Memory Usage (MB)
Performance (F1)
Euclidean Baseline
768
300 MB
0.928
HypeCodeNet (Ours)
768
3300 MB
0,940
HypeCodeNet (Efficient)
128
550 MB
0.938
K.4 AST STATISTICS FOR GEOMETRIC EVALUATION
To ensure the representativeness our geometric analysis, we provide the detailed statistics of ASTs used in the evaluations above. The statistics in Tajae derived from the stratified random sample of 1, 1,000 ASTs from the BigCloneBench
set. The wide range in node count and depth confirms that our evaluation covers a diverse spectrum of code, from simple utility functions to complex algorithmic implementations.
Table 15: Detailed statistics of the Abstract Syntax Trees (ASTs) used for geometric validation.
Statistic
Min
Max
Median
Mean
Number of Nodes
18
1,243
156
184.5
Tree Depth
4
68
14
16.2
Branching Factor
1.0
12.0
2.4
3.1
L EXTENDED GEOMETRIC ANALYSIS AND MANIFOLD ABLATION
To rigorously address the theoretical inquiries regarding the optimality of pure hyperbolic geome- try for code representation, and to quantify the relationship between curvature and representation fidelity, performed extended series of experiments. These analyses specifically target the comparison with product manifolds and the formal evaluation of embedding distortion.
L.1 COMPARATIVE ANALYSIS OF MANIFOLD TOPOLOGIES
While our main results demonstrate the superiority of the Lorentz model over Euclidean baselines, a pertinent question whether a Product
both Euclidean and Hyperbolic ceateeetts-iihhhe better capture the multifaceted nature of code (c.g., linear sequential logic mixed with hierarchical nesting). To investigate this, we implemented variant of HypeCodeNet operating on a product manifold M Ed/2 x Ld/2, where node embeddings are the concatenation of Euclidean vector and a Lorentz vector.
We compared three topological configurations on the BigCloneBench task, keeping the total embed- ding dimension constant at d 768.
Table 16: Performance comparison across different manifold topologies on BigCloneBench. The Lorentz (Pure Hyperbolic) model achieves the best performance, supporting the hypothesis that the dominant structural characteristic of the AST is hierarchical, best served by a fully negatively curved space. The Product Manifold offers improvements over the Fuclidean baseline but does not. match the pure Lorentz model.
Manifold Topology
Geometry Signature
F1-score
MAP
Euclidean (E')
Zero Curvature (c = 0)
0.923
0.921
Product Manifold (E4/2 gd/2) Mixed Curvature (0, < 0)
0.933
0.930
Lorentz (Cd) [Ours]
Negative Curvature (c < 0)
0.940
0.938
Table 16 reveals that while the Product Manifold outperforms the pure Euclidean baseline (0.933 0.923), it falls short of the pure Lorentz model (0.940). This suggests that allocating dimensions to a flat Euclidean component reduces model's capacity to embed the exponentially growing hierarchy of the AST. Since the AST is the primary input structure, maximizing the "hyperbolic capacity" yields the optimal representation.
36

---

Published as a conference paper at ICLR 2026
L.2 QUANTITATIVE ANALYSIS OF CURVATURE AND DISTORTION
To theoretically substantiate claim that "hyperbolic geometry is natural for code," we quantified the Embedding Distortion as function of the manifold's curvature. We define the mean average distortion davg over the test graphs as:
dgraph (u, u)
davg
(46)
Fiestt
dgraph (1, 2))
(u.e)EPicst
where d.M is the geodesic distance the manifold and dgraph is the shortest path in the AST. We trained HypeCodeNet variants with fixed curvature values ranging from near-zero (Euclidean limit) strongly negative, and measured both distortion and downstream Fl performance.
Table 17: Analysis of Embedding Distortion (ówg) and Model Performance (FI) as a function of Manifold Curvature (c). Results indicate clear correlation: as the curvature becomes negative (up to an optimal point), the geometric distortion decreases, and task performance improves. This empirically verifies the "naturalness" of negative curvature for ASTs.
Curvature (c)
Avg. Distortion (5avg) 4
F1-score t
-0 0.01 (Near Euclidean)
0.241
0.924
-0.50
0.183
0.932
-1.00 (Ours)
0.152
0.940
2.00
0.165
0.937
5.00
0.210
0.929
The results in Table 17 provide the requested distortion analysis. We observe that:
1. Inverse Correlation:
There is strong inverse correlation between distortion and perfor- mance. The curvature setting (e 11.0) that minimizes distortion (0.152) corresponds exactly to the peak performance (0.940).
2. Optimal Curvature: While negative curvature is beneficial, excessively curved space
(c = -5.0) increases distortion. This is likely because extreme curvature imposes overly strict constraints on node separation that may mismatch the specific branching factor of standard code ASTs, confirming that e -1 the ideal operating point for this domain.
L.3 TRAINING DYNAMICS OF CURVATURE ANNEALING
To address the request for an analysis the curvature annealing process, we tracked the evolution of the learnable curvature parameter and the validation loss across training epochs on the Big- CloneBench dataset. Our annealing strategy initializes near 0 (mimicking Euclidean space) and allows to evolve towards a stable negative value.
Table 18
presents the training dynamics. We observe three distinct phases:
1. Euclidean Phase (Epochs 0-1): The model operates in near-flat regime (c RJ -10-5). The loss decreases rapidly as the model learns local syntactic features (e.g., token embed- dings), similar to standard Euclidean GNN.
2. Transition Phase (Epochs 2-3): As becomes more negative, we observe a second sharp drop in validation loss. This correlation indicates that the activation of the hyperbolic geometry allows the model resolve hierarchical ambiguities that were difficult to capture in the flat space.
3. Stable Hyperbolic Phase (Epochs 4+): The curvature stabilizes around as -1.02, closely matching the canonical curvature -1. This learned stability suggests that the data "prefers" this specific geometric configuration without manual forcing.
L.4 SEMANTIC AUGMENTATION AND GRAPH UNIVERSALITY
The reviewer correctly noted that while ASTs are hierarchical, program reasoning often requires understanding semantic relationships (e.g., data flow) that may form non-tree structures. To test the universality of HypeCodeNet and address the "semantic-syntactie decoupling" concern, we con- ducted experiment integrating Data Flow Graph (DFG) edges.
We constructed a Hybrid Graph where edges = EAST U FDFG. EDRG connects variable usage nodes to their definitions, creating cycles and shortcuts in the graph. We compared the performance of Euclidean and Hyperbolic models on this augmented structure.
37

---

Published as a conference paper at ICLR 2026
Table 18: Evolution of the learnable curvature e and Validation Loss during training. The "Delta Loss" highlights the significant performance gain specifically attributable to the transition into the hyperbolic regime (Epochs 2-3).
Epoch
Curvature value (c)
Validation Loss
Phase Description
0
-0.00001
0.682
Initialization (Near- Euclidean)
I
-0.005
0,415
Syntactic Learning
2
-0,254
0.302
Hyperbolic Transition
3
-0.990
0.215
Hierarchical Alignment
4
-0.985
0.208
Fine-tuning
5
-1.021
0.205
Convergence
Table 19: Performance impact of integrating semantic Data Flow Graph (DFG) edges on Code Clone Detection. HypeCodeNet (+DFG) outperforms the pure AST version, demonstrating that hyperbolic space can effectively model semantic "shorteuts" in addition to the syntactic hierarchy. Crucially, the geometric advantage (Euclidean vs. Hyperbolic) remains significant even with semantic aug- mentation.
Geometry
Graph Structure
Fl-score
Rel. Imp.
AST Only AST + DFG
0.923
Euclidean
0.931
+0.8%
AST Only AST + DFG
0.940
Hyperbolic (Ours)
0.946
+0.6%
The results in Table 19 demonstrate two key findings:
Robustness to Non-Tree Edges: Adding DFG edges improves HypeCodeNet's perfor- mance (0.940 + 0.946). This confirms that our manifold-aware attention mechanism can handle semantic cycles without breaking the hierarchical embedding structure. The "short- cuts" provided by DFG edges are naturally modeled as secants or distinct geodesics within the Poincaré ball.
2. Geometric Dominance: Even when Euclidean models are enhanced with semantic edges (Euclidean AST+DFG: 0.931), they still underperform the base HypeCodeNet on pure ASTs (0.940). This suggests that while semantic augmentation is beneficial, the limitation of Euclidean space in representing the underlying backbone hierarchy remains the primary bottleneck, which HypeCodeNet effectively resolves.
L.5 STATISTICAL SIGNIFICANCE OF GEOMETRIC GAINS
To confirm that the performance improvements reported in manifold ablation (Table 16 and semantic augmentation (Table experiments are statistically robust and not artifacts of random initialization, we conducted paired t-tests. We trained 5 runs for each major model variant on the BigCloneBench dataset with different random seeds.
Table 20 summarizes the results. The v-values indicate that the performance gap between the Pure Lorentz model and the Product Manifold is statistically significant (p < 0.01), reinforcing the con- clusion that fully hyperbolic space the superior inductive bias for this task. Furthermore, the improvement from integrating DFG edges, while numerically smaller, is also statistically signifi- cant, validating the model's capability utilize semantic information.
Table 20: Statistical significance testing (Paired t-test) for key comparative results. All comparisons are made against the standard HypeCodeNet (Lorentz, AST-only) model. Results show that the superiority over Euclidean/Product manifolds is highly significant.
Model Comparison (A vs. B)
Mean A F1
p-value
Significance
HypeCodeNet Euclidean Baseline
+1.7%
2.3 10-5
*** (High)
HypeCodeNet vs. Product Manifold
+0.7%
8.1 x 10-3
(Medium)
HypeCodeNet (+DFG) vs. HypeCodeNet
+0.6%
1.5 10-2
(Significant)
38

---

Published as a conference paper at ICLR 2026
L.6 THEORETICAL ALIGNMENT: BRANCHING FACTOR AND OPTIMAL CURVATURE
Finally, to provide the "formal quantification of the curvature-hierarchy correlation" requested by the reviewer, we analyze the relationship between the dataset's intrinsic graph properties and the model's 's learned curvature.
According to the theory of hyperbolic tree embeddings (Sala et al., 2018), for a regular tree with branching factor b, the minimum curvature c required to embed the tree with arbitrarily low distortion satisfies the scaling relationship Ç x A In b, where Ç is the scale of the embedding. Conversely, for fixed scale, the optimal curvature matches the tree's expansion rate.
We calculated the average branching factor Dany of the ASTs in the BigCloneBench dataset, found to be approximately 3.1 (Table 15). In our training dynamics experiment (Table T8), the learnable curvature naturally converged to x -1.02. Substituting these values into the capacity bound im- plies an effective hierarchy expansion rate of e ev1.02 x 2.75, which aligns closely with the empirical branching factor of 3.1.
This theoretical alignment confirms that HypeCodeNet is not merely minimizing a loss function, but actively discovering the intrinsic geometric signature of the source code. The convergence of to a value predicted by the AST's branching statistics provides the rigorous justification that hyperbolic geometry indeed the "natural" geometry for these program structures.
M SENSITIVITY ANALYSIS AND MANIFOLD TOPOLOGY
In this section, we perform detailed sensitivity analysis regarding the curvature parameter of the embedding space and investigate the efficacy of alternative manifold topologies, specifically hybrid geometries, for code representation tasks.
M. CURVATURE SENSITIVITY ANALYSIS
The curvature e of the hyperbolic space determines the "steepness" of the hierarchy capacity, While proposed method treats as learnable parameter, we conducted experiments with fixed curva- ture values initialized and frozen at different scales to assess the model's sensitivity to this hyperpa- rameter.
Table 21 presents the Fi-scores the BigCloneBench dataset for various fixed curvature settings.
The results indicate:
The model maintains high performance (F1 >> 0.93) across a wide range of negative curvature values (c E [-0.5, -.0)). consistently outperforming the Euclidean baseline (c = 0, F1=0.923). This suggests that the benefit of hyperbolic geometry is intrin- sic and not dependent
Robustness:
precise hyperparameter tuning.
Optimality of Optimization: The learnable curvature approach, which converges to c R -1 1.022 achieves the highest performance (0.940). This confirms that allowing the geometry adaptively match the intrinsic scale of the dataset yields better results than manual selection.
Degradation at Extremes: Performance drops slightly at extremely high curvature (c -5.0), likely because the excessive distortion of local neighborhoods outweighs the bene- fits of hierarchical separation.
M.2 EVALUATION OF PRODUCT MANIFOLDS
We further explored whether a "mixed-curvature" geometry could offer advantages by combining the strengths of flat and curved spaces. We implemented a Product Manifold configuration, M = Ed/2 x 4d/2, where the embedding space the Cartesian product of a Euclidean subspace and a Lorentz subspace, splitting the feature dimensions equally.
As shown in Table 22 the Product Manifold achieves an Fi-score of 0.933, surpassing the pure Eu- clidean baseline. However, it does not match the performance of the fully hyperbolic Lorentz model (0.940). This result suggests that the Abstract Syntax Tree structure is dominantly hierarchical. By allocating half of the dimensions to flat Fuclidean space, the effective capacity to model this hier- archy is reduced. Therefore, a pure negatively curved manifold serves as a more efficient inductive bias for AST-centric tasks compared to hybrid geometries.
39

---

Published as a conference paper at ICLR 2026
Table 21: Sensitivity analysis of fixed curvature values on BigCloneBench compared to the learn- able strategy. The model is robust to range of negative curvatures, with the learnable parameter achieving the optimal trade-off.
Curvature Strategy
Value (c)
F1-score
Euclidean Baseline
0.0
0.923
0.1
0.929
0.5
0,936
Fixed Hyperbolie
-1.0
0.939
-2.0
0,935
5.00
0.928
Learnable (Ours)
1.02
0.940
Table 22: Performance comparison of manifold topologies. While the Product Manifold improves
upon the pure Euclidean baseline, the Pure Lorentz model remains superior, indicating that maxi- mizing the hyperbolic capacity is crucial for AST-based representation.
Manifold Topology
Components
F1-score
MAP
Euclidean Space
Pure Ed
0.923
0.921
Product Manifold
Id/2 C4/2
0.933
0,930
Lorentz (Ours)
Pure Ld
0.940
0.938
N GENERALIZATION CAPABILITIES
To address the reviewer's inquiry regarding the generalization of HypeCodeNet beyond standard AST-based tasks within single language, we conducted experiments on cross-language transfer and non-AST graph structures.
N.1 CROSS-LANGUAGG TRANSFER LEARNING
The reviewer asked if HypeCodeNet allows for transfer learning between programming languages. We hypothesize that while surface syntax (keywords) varies across languages, the underlying hier- archical structure of logic (nesting, branching) remains universal. Hyperbolic embeddings, which prioritize this structure, should therefore exhibit stronger cross- loossll transferability than Euclidean models.
We evaluated this using a Zero-Shot Cross-Language Clone Detection setup. The model was trained exclusively the BigCloneBench dataset (Java) and then evaluated directly on the POJ-104 dataset (C/C++) without any fine-tuning.
Table 23: Zero-Shot Cross-Language Transfer results (Train on Java -> Test on C/C++). HypeCo- deNet demonstrates significantly lower performance degradation compared to Euclidean baselines, suggesting that the hyperbolic geometry captures universal structural patterns that generalize across languages.
Model
Geometry
Zero-Shot Accuracy (C++)
Transfer Drop
GraRhCodeBERT
Euclidean
0.842
-12.0%
CodeGNN
Euclidean
0.815
-12.0%
HypeCodeNet (Ours)
Hyperbolic
0.894
-8.7 %
Table 23 confirms that HypeCodeNet achieves Zero-Shot Accuracy of 0.894 on C++, outper- forming GraphCodeBERT (0.842). Notably, the performance drop relative to in-domain training smaller for our model (-8.7%) compared the baselines (-12.0%), validating that hyperbolic geometric features are more invariant to language-specine syntactic shifts.
N.2 GENERALIZATION TO NON-AST STRUCTURES: PROGRAM DEPENDENCE GRAPHS
To test the model's ability to reason over non-AST graphs, specifically "interprocedural depen- dency graphs" as requested, we applied HypeCodeNet to Defect Detection task using the Devign dataset. In this task, the input represented not AST, but
Program Dependence Graph
40

---

Published as a conference paper at ICLR 2026
(PDG), which contains both control flow and data flow edges, forming graph with cycles and non-hierarchical connections.
We trained HypeCodeNet to classify functions as vulnerable or non-vulnerable based on their PDG representations.
Table 24: Performance on Defect Detection using Program Dependence Graphs (PDGs). Despite PDGs being less strictly tree-like than ASTs, HypeCodeNet maintains performance advantage. This indicates that the model generalizes well to graphs with mixed hierarchical backbone and cyclic dependencies.
Model
Input Structure
Accuracy
Fl-score
GCN
PDG
61.8
53.2
Devign (Gated GNN)
PDG + Composite
63.4
55.6
HypeCodeNet (Ours)
PDG
65.1
57.4
The results in Table 24 show that HypeCodeNet outperforms the GCN baseline and the special- ized Devign model on PDG structures. While PDGs are not pure trees, they still exhibit significant hierarchical properties (e.g., control dependence backbones). The hyperbolic geometry effectively embeds this backbone while the manifold-awand message passing handles the cyclic data dependen- cies, demonstrating the framework's 's versatility beyond pure AST reasoning.
O MATHEMATICAL PROOFS AND APPROXIMATIONS
This section provides the formal mathematical justifications requested regarding the properties of the exponential and logarithmic maps in the Lorentz model, as well as the derivation of the Euclidean approximation for the geodesic distance.
0.1 INVERSE RELATIONSHIP OF EXPONENTIAL AND LOGARITHMIC MAPS
We demonstrate that the exponential map expş and the logarithmic map log, inverse operations. Let p E Cd be a point on the manifold and E TpLC a tangent vector with norm = ||v|lc- Recall the definition of the exponential map:
u = (u) = cosh( cosh((-cA)pp sinh(V-cA) 1
(47)
V-c
Our goal is to show that log, (u) v. First, we compute the Lorentz inner product (p, u) Since
p). 1/c and (p, u) = 0, we have:
sinh(v-cX) (p,u)2 u)
(p, u) c cosh(VCcN)ppp)--
(48)
V-ch
1
cosh(V-ao)
(49)
Next, we substitute u and (p, u)e into the definition of the logarithmic map:
de u)
logs, (u) Vip, 2)2 (1/c)2 (u clp, cp)
(50)
We evaluate the vector term in the parentheses:
sinh sinh(v-ca) coosh(VFc\pp=
6) - -o co( (oooh(-V-e))) p
clp, up
(51)
V-c1
sinh(v-cc)).
(52)
Now we evaluate the scalar coefficient. The geodesic distance de u) X. For the denominator, using the identity cosh?
sinh? x and noting e 0:
(p. up.u)² - (1/c)²
cosh (v (v-cc) o²
(53)
sinhi(V-c))
(54)
41

---

Published as a conference paper at ICLR 2026
Substituting these back into the log map equation:
A
sinh( -cd)
(0g sinh(v -cX)
(55)
V-cl
(-c) V-cA
-0 1 V-c V-cx
(56)
In our implementation (using Geoopt), the tangent vectors are implicitly scaled or the metric defini- tion absorbs the v-c factor to ensure v is recovered exactly. The structural derivation confirms the linearity and inverse relationship.
0.2 JUSTIFICATION OF EUCLIDEAN APPROXIMATION
We justify the approximation used in Appendix (Eq. 40): -c(u, v)c 2 1+ 7 ugl? as
Let u, U L.- In the ambient space Rd+1, we can write (up, u) where u E RRd. The constraint (u, u)c = 1/c implies -ug + ||ul|2 = 1/c. Thus, un = v //ee² + ||ll For small curvature e 0- (i.e., large radius), we use Taylor expansion:
No -=
uul
(1 c |ull²)
2
(57)
Now, consider the term -o( u) -c( un Dy + u v) approximation for uo, Un:
cugup
c(u v). Substituting the
c-c (1 - ||ul²)
c
-/2 (1 -1-11v11P))
1
Cugio c
(58)
- (1 _ ||||||2 =
(59)
2 - z (1|u|1 + [[v13)
(60)
Substituting this back:
c/u = (-1+ (11u] |v]3) - c(u.v)
(61)
2
= =-1
[||u| -u
(62)
vll
(63)
Since we are interested in the distance term which involves positive arguments for arcosh, we ob- serve that for the negative inner product, c(u, v) s - l v/ Letting k - 0, this
becomes +
v matching the form used to justify the Euclidean limit.
P EXTENDED EXPERIMENTAL EVALUATION
In response to the reviewers' requests for broader evaluation scope and comparisons against mod- ern baselines, we present additional experimental results in this section. We extend our evaluation to the task of Defect Detection using Program Dependence Graphs (PDGs) and analyze the model's cross-lingual generalization capabilities via Multi-Language Cloze Test. Furthermore, a direct comparison with CodeT5+ to position our work within the landscape of Large Language Models (LLMss).
include
P.I DEFECT DETECTION ON DEVIGN DATASET
To demonstrate HypeCodeNet's ability to reason over non-tree graph structures, we evaluated on the Defect Detection task using the Devign dataset. Unlike ASTs, the input here is modeled as Program Dependence Graph (PDG), which contains cyclic data dependencies and control flow edges. The goal is to classify function as vulnerable or safe.
We compare HypeCodeNet against strong encoder-based baselines BERR,, CodeT5-
base) and the much larger CodeTT5+ (16B) model. As shown in Table 25 HypeCodeNet signifi-
42

---

Published as a conference paper at ICLR 2026
cantly outperforms the Euclidean graph baseline (GraphCodeBERT) by 4.6% in accuracy. Remark- ably, it achieves performance competitive with the 16-billion parameter CodeTS+ model (67.8% vs. 68.4%), despite having approximately 188x fewer parameters. This result confirms that the hyper- bolic inductive bias effectively captures the complex, cyclic structural patterns in PDGs, offering highly efficient alternative to massive LLMs for code classification tasks.
Table 25: Performance comparison on the Defect Detection task (Devign dataset). HypeCodeNet demonstrates superior efficiency, surpassing comparable encoder models and approaching the per- formance of the massive CodeTT5+ 16B model.
Model
Parameters
Accuracy (%)
Fl-score
GraphCodeBERT CodeT55-baee
125M
63.2
55.1
220M
64.8
56.9
CodeT5+ (16B)
16B
68.4
60.2
HypeCodeNet (Ours)
85M
67.8
59.6
P.2 MULTI-LANGUAGE GENERALIZATION EVALUATION
To investigate the cross lingual generalization capabilities of HypeCodeNet, we conducted a Masked Language Modeling (MLM) evaluation, also referred as Cloze Test, across six diverse program- ming languages from the CodeXGLUE benchmark: Java, JavaScript, Ruby, Go, PHP, and Python.
We hypothesize that while surface syntax varies significantly across languages, the underlying hi- erarchical structure of logic remains universal. Therefore, hyperbolic model that prioritizes this structural embedding should exhibit stronger generalization, particularly on languages with less training data (e.g., Ruby, Go) compared to resource-rich languages (e.g., Java, Python).
Table 26 presents the comparative results. We observe the following:
1. Overall Improvement:
HypeCodeNet outperforms the strong GraphCodeBBRRT baseline across all six languages, with an average accuracy improvement of 2.7%.
2. Gain in Lew-Resource Settings: The performance gap notably wider for "long-tail" languages. For instance, on Ruby and JavaScript, HypeCodeNet achieves gains of +4,4% and +3.4% respectively, compared
modest +1.1% on Python.
This trend strongly suggests that the hyperbolic geometry provides a more robust inductive bias for code structure, allowing the model to learn effective representations even when data is scarcer or when transferring structural knowledge across language boundaries.
Table 26: Accuracy (%) on the CodeXGLUE Multi-Language Cloze Test (Token-level Masked Language Modeling). HypeCodeNet shows consistent improvements over GraphCodeBERT, with the most significant gains observed in languages other than Python and Java.
Model
Java
JS
Ruby
Go
PHP
Python
Avg.
CodeBERT
68.4
61.2
58.9
64.3
59.7
71.2
63.9
GraphCodeBERT
70.1
63.8
61.4
66.7
62.3
72.8
66.2
CodeT5+ (16B)
76.8
69.7
67.5
72.4
68.1
78.9
72.2
HypeCodeNet (Ours)
71.5
67.2
65.8
69.3
66.1
73.9
68.9
vs. GraphCodeBERT
+1.4
+3.4
+4.4
+2.6
+3.8
+1.1
+2.7
P.3 SCHEMATIC DATA FLOW OF GEODESIC ATTENTION
In response to the request for detailed illustration of the attention mechanism (Suggestion 3), we provide step-by-step schematic description of the data flow within a single HypeCodeNet layer. This process highlights the interplay between manifold-native operations and tangent space aggregations.
The flow for updating a central node e with neighborhood. (V()) proceeds follows:
Step 1. Manifold-to-Tangenn Projection (Log-Map):
Input:
Hyperbolic node features (1) (0)
Cd.
Operation: Map neighbors to the tangent space of u.
Output: Relative Euclidean vectors my-+2
log (hw) dR..
43

---

Published as a conference paper at ICLR 2026
Step 2. Dual-Stream Attention Computation (Tangent Space):
Stream A (Geometric): Compute structural scores based on geodesic distance
do ha).
Stream B (Semantic): Compute content scores via query-key dot products tangent vectors. To handle the zero self-message (My-r = 0), we include the bias term bo:
(WQm + ( (W
Fusion: Combine streams using the learnable gate Ag: (1
softmax Geom +
Sem).
Step 3. Weighted Aggregation:
Operation: Aggregate neighbor vectors using attention weights.
Output:
Update vector A. E Th. C. (a Euclidean vector in the local tangent frame),
Step 4. Tangent-to-Manifold Update (Exp-Map):
Operation: Apply the update along the geodesic defined by A..
Output: New hyperbolic node position 1 (1+1) exps, (Ap) E L.
P.4 EFFICIENCY ANALYSIS: THE "SWEET SPOT" OF GEOMETRIC LEARNING
Finally, we synthesize the comparisons with the large-scale CodeTT5+ model (16B parameters) pre- sented in Tables and 26
While Large Language Models (LLMs) achieve high performance through massive parameter scal- ing, HypeCodeNet demonstrates that a correct geometric inductive bias can achieve comparable results with orders of magnitude greater efficiency. Specifically:
Parameter Efficiency: HypeCodeNet (85M) matches 99.1 % of the Defect Detection ac- curacy of CodeT5+ (16B) while using only 0.5% of the parameters.
Training Resources: HypeCodeNet can be trained on single commodity GPU (e.g., A 100 or even 3090) in hours, whereas tine-tuning 16B model requires multi-node clusters and significantly higher energy costs.
This positions HypeCodeNet as an ideal solution for resource-constrained environments or scenar- ios requiring thgo-thru embedding generation, where deploying a 16B parameter model is infeasible.
P.5 CONVERGENCE ANALYSIS OF THE FRÉCHET MEAN
Reviewer Huu9 raised valid concern regarding the convergence guarantees of the iterative Fréchet mean computation used in our graph pooling layer (Appendix B.3 We provide rigorous justifica- tion below.
Geometric Setting. The Lorentz model C with curvature c < a complete, simply con- nected Riemannian manifold of constant negative sectional curvature K Such manifolds Hadamard manifolds (CAT(0) spaces) and are globally diffeomorphic Fuclidean space the exponential map at any point.
Convexity of the Objective. the variance function
The Fréchet mean μ of finite (h1...., ANr L minimizes
F(P) 2N
Σ C. h.).
(64)
manifolds of non nos-iositive sectional curvature, the squared geodesic distance function H (p. 14) is geodesically convex, and strictly geodesically convex away from the base point hi- Its Riemannian Hessian satisfies
(Hossp d.(,,))))). o) 2||b|
(65)
for all E T.C with equality only along the radial direction from h;. Consequently, F(p) is strictly geodesically convex any bounded subset not containing all points in single geodesic.
Existence and Uniqueness. A strictly geodesically convex function on a complete Hadamard manifold possesses unique global minimizer. Thus, the Fréchet mean of hyperbolic node em- beddings is well-defined and unique.
44

---

Published as a conference paper at ICLR 2026
Convergence of the Iterative Algorithm. We compute the Fréchet mean via Riemannian gradient descent (RGD) with constant step size 1l 1
Pr+1 exp
Σ logu, (h:)
(66)
This update is known as the inductive mean geodesic averaging. On Hadamard manifolds, the inductive mean applied the Fréchet variance function converges exponentially the unique Fréchet mean for any finite set of points and any initialisation. The negative curvature eliminates local minima and saddle points in the objective landscape, ensuring global and rapid convergence in practice (typically < iterations in our experiments, consistent with Geoopt's implementation).
These theoretical guarantees directly follow from the negative curvature of the Lorentz model and not available in Euclidean positively curved spaces.
LLM USAGE STATEMENT
We utilized large language model (LLM) solely for the purpose of refining grammar, punctuation, and phrasing this manuscript. The LLM was not used for generating any of the core scientific content, such as the methodology, experiments, or conclusions presented herein.
45


Reviews:
[{"review_id": "XbROOhkTbP", "reviewer": "Reviewer_4Dzo", "summary": "​​This paper argues that hyperbolic geometry is a better inductive bias for program representations than Euclidean space, because Abstract Syntax Trees (AST) are tree-like and expand exponentially. The authors introduce HypeCodeNet, a Lorentz-model hyperbolic GNN that introduces manifold aware operations to the embedding layer and message passing attention module.  Across code detection, line-level code completion, and link prediction on call graphs, HypeCodeNet achieves SOTA results and exceeds strong Transformer and graph baselines, demonstrating that hyperbolic geometry offers meaningful advantages for modeling hierarchical code structures.", "questions": "1. The visualization in Figures 7 and 8 is compelling.\n   - How many ASTs contributed to the aggregated statistics in Figure 7? (Figure 8 appears to show a single example)\n   - What is the size range of AST(s) in your evaluation (min/max/median nodes and depth)?\n   - Does the distortion advantage hold consistently across different code complexity levels?\n\n2. How does memory consumption scale compared to Euclidean models, especially for very large ASTs?\n\n3. The paper is missing a citation for CodeFORMER, which appears to be your primary baseline.  Liu et al. (2023) \"CodeFORMER: A GNN-Nested Transformer for Source Code Representation\" (MDPI Electronics 12(7):1722). Please confirm this is the correct reference and add it to your bibliography.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper demonstrates careful engineering work to make hyperbolic deep learning stable and practical for code representation:\n\n1. **Consistent improvements across diverse tasks demonstrate general utility and significance.** The approach shows modest but consistent gains over CodeFORMER in semantic understanding (BigCloneBench: +1.2% F1) and code completion (~1% across metrics), while achieving substantial improvements in link prediction (+5.0% AUC, +5.2% Hits@10). This variation suggests the method particularly excels at structural reasoning tasks.\n\n2. **High-quality experimental evaluation.** The authors compare against 9 baselines spanning sequence-based, graph-based, and hybrid architectures across multiple datasets per task. The ablation studies (Section 4.5) provide clear empirical evidence that performance gains stem from hyperbolic geometry rather than other architectural choices.\n\n3. **Original technical contributions beyond existing hyperbolic models.** HypeCodeNet advances beyond prior work through: (i) use of the numerically stable Lorentz model, (ii) geodesic-based attention mechanism, and (iii) curvature annealing for stable training. These design choices address known challenges in hyperbolic deep learning.\n\n4. **Strong empirical validation of theoretical claims.** The distortion analysis (Figure 7, Appendix) provides compelling evidence that hyperbolic embeddings preserve AST hierarchical structure with significantly less distortion than Euclidean alternatives, supporting the paper's central thesis. \n\n5. **Well-written paper with clear presentation.** The overall thesis is compelling and the technical approach is explained systematically. The experimental section is particularly well-organized. (Though Figure 1 could be improved, as noted in the weaknesses section)", "weaknesses": "1. **Insufficient Geometric Analysis of Learned Representations.** While the paper provides compelling empirical evidence that hyperbolic geometry improves performance, it lacks rigorous geometric validation of the learned embeddings. The authors cite Yang et al. (2023) for hyperbolic representation learning, yet that work explicitly cautions that hyperbolic embeddings do not automatically guarantee hierarchical structure and demonstrates cases where geometric properties may not align with semantic hierarchies. While Figure 8 provides valuable qualitative intuition, it requires accompanying quantitative validation. The current visualization alone cannot confirm that the hierarchical structure is preserved beyond visual inspection.\n\n**Suggested quantitative analyses:**\n   - **Distance-to-Origin Analysis:** Provide quantitative measurements of node distances from the origin in the Lorentz model. Specifically, verify that root nodes consistently map near the origin (small $d(v, o_c)$ ) while leaf nodes map toward the boundary, with statistical significance tests across multiple ASTs.\n   - **Hierarchical Structure Validation:** Compute the correlation between graph depth and hyperbolic distance to origin. Following Nickel & Kiela (2018), report Spearman's rank correlation coefficient between tree depth and $\\|h_v\\|_L$.\n   - **Statistical Validation:** Consider adding histograms of distance distributions by tree depth and statistical tests confirming that parent-child pairs maintain consistently smaller distances than arbitrary node pairs at the same depth.\n\n2. **Figure 1 needs significant improvement to effectively convey the paper's core contributions.**\n   \n   **Part (a) - The geometric intuition is too generic and could represent any hierarchical structure.** \n   The authors might consider:\n   - Showing a concrete code snippet (5-10 lines) with its actual AST and node labels displaying real AST node types (e.g., IfStatement, ForLoop, Variable)\n   - Providing a side-by-side comparison with Euclidean embeddings to demonstrate distortion differences\n   \n   **Part (b) - The 'log-aggregate-exp' visualization feels cluttered and unfocused.** \n   To improve clarity, the authors could:\n   - Use progressive panels that track a specific node through each transformation step\n   - Add visual differentiation through color coding (e.g., parent nodes in blue, children in green, message flow as arrows)\n   - Emphasize the tangent plane with shading or perspective to help readers understand why operations must occur in this local Euclidean space\n   \n   While these are suggestions, addressing the abstract nature of the current visualization would significantly strengthen the paper's accessibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tuA9097LF9", "reviewer": "Reviewer_enmz", "summary": "This paper introduces HypeCodeNet, a geometric deep learning framework for code representation learning that operates natively in hyperbolic space, formulated under the Lorentz model.\nThe authors argue that source code’s Abstract Syntax Tree (AST) inherently possesses hierarchical and tree-like structures that are better represented in negatively curved manifolds than in Euclidean space.\nHypeCodeNet integrates manifold-aware components — a hyperbolic embedding layer, a tangent-space message-passing mechanism, and a geodesic-based attention module — trained with Riemannian optimization and curvature annealing.\nAcross three standard benchmarks (clone detection, code completion, link prediction), it consistently outperforms strong Euclidean baselines such as CodeBERT, GraphCodeBERT, and CodeFORMER, supporting the claim that hyperbolic geometry aligns more naturally with program hierarchies.", "questions": "1.\tProvide a formal distortion analysis: derive or empirically approximate the embedding distortion as a function of curvature.\n2.\tInclude ablation across manifold types: compare Lorentz, Poincaré, and product manifolds.\n3.\tAnalyze training dynamics of curvature annealing: curvature vs. epoch plot.\n4.\tExplore semantic augmentation: integrating CFG/DFG edges to test the universality claim.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tStrong conceptual motivation grounded in geometric theory\nThe paper convincingly argues that the exponential volume growth of hyperbolic space matches the hierarchical expansion of ASTs. The “distortion” argument is well supported, referencing Bourgain’s theorem and previous work on low-distortion tree embeddings.\n2.\tTechnically principled formulation\nBy adopting the Lorentz model instead of the unstable Poincaré ball, the authors ensure both numerical stability and Riemannian differentiability, enabling a deep stack of manifold layers with standard GPU parallelization.\n3.\tWell-designed architecture bridging geometry and semantics\nThe “log–aggregate–exp” message-passing paradigm and geodesic-aware attention are elegant and grounded in geometric consistency. The method preserves curvature constraints while incorporating multi-head attention and layer normalization — non-trivial achievements in hyperbolic neural design.", "weaknesses": "Theoretical insufficiency in proving “naturalness” of hyperbolic geometry.\nThe core claim — that “hyperbolic geometry is the natural geometry of code” — remains conceptually persuasive but not theoretically rigorous. No formal quantification of distortion or curvature–hierarchy correlation (e.g., tree embedding distortion bounds). Missing mathematical analysis of curvature c → embedding fidelity or proofs showing convergence of representations to low-distortion manifolds.\nAdding a distortion vs. curvature empirical curve or formal derivation would strengthen the argument substantially.\n\nLimited comparison with non-Euclidean or hybrid geometries.\nThe paper frames hyperbolic geometry as the only alternative, but mixed-curvature or spherical–hyperbolic hybrid embeddings could better capture cross-function semantics.\nA control experiment with mixed curvature or product manifolds (𝔼×ℍ) would clarify whether pure hyperbolic geometry is indeed optimal.\n\nOverlooked semantic–syntactic decoupling.\nThe model tightly couples AST topology with embedding curvature but does not explicitly distinguish between semantic relations and syntactic nesting.\nThis may limit generalization to tasks involving cross-file or semantic code reasoning. Integrating semantic edges (DFG, CFG) or cross-function attention could make the representation more complete.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kNoYH0tuEH", "reviewer": "Reviewer_rLfX", "summary": "The paper proposes HypeCodeNet, a novel hyperbolic Graph Neural Network (GNN) that learns source code representations directly in hyperbolic space, rather than traditional Euclidean embeddings. Motivated by the hierarchical, tree-like nature of Abstract Syntax Trees (ASTs), the authors argue that hyperbolic geometry better captures program structure with low distortion.\nHypeCodeNet operates in the Lorentz model for stability and integrates manifold-aware message passing, curvature annealing, and Riemannian optimization. Across three tasks: code clone detection, code completion, and function call link prediction, the model achieves strong gains over GraphCodeBERT, UniXcoder, and CodeFORMER. Ablations further confirm that performance improvements stem mainly from the hyperbolic geometry rather than architectural tweaks.", "questions": "Curvature tuning:\nHow sensitive is model performance to the final curvature value? Would a mixed-curvature or adaptive manifold (e.g., product manifolds combining Euclidean + hyperbolic subspaces) perform better?\n\nGeneralization beyond AST-based tasks:\nCan HypeCodeNet generalize to non-AST graph structures, such as interprocedural dependency graphs or text-conditioned code generation? Have you explored transfer learning between programming languages?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Originality (Novel geometric framing):\nThe paper introduces a foundational geometric shift in code representation, from Euclidean to hyperbolic space, supported by clear theoretical intuition. It’s the first to present an end-to-end hyperbolic framework for code reasoning, filling a notable gap in the literature.\n\nQuality (Sound formulation & strong empirical results):\nThe proposed model is mathematically rigorous, leveraging Lorentz manifolds, log/exp maps, Riemannian Adam, and curvature annealing for stability. Results across multiple benchmarks (BigCloneBench, CodeXGLUE, GitHub Java Corpus) show consistent SOTA performance, with up to 3–5% improvement over strong baselines. Ablations convincingly isolate geometry as the key performance driver.\n\nClarity & Significance:\nThe paper is well-organized, with detailed geometric explanations and intuitive visualizations (e.g., Figure 1). Its results suggest non-Euclidean geometry may be a next paradigm for program representation, opening a new research direction bridging geometric deep learning and code understanding.", "weaknesses": "Reproducibility & implementation accessibility:\nThe paper does not mention public release of code or datasets, and implementation details (e.g., curvature annealing schedule, manifold dimensionality tuning) are deferred to appendices. Reproducibility may be difficult without explicit scripts or pretrained models.\n\nLimited scope of benchmarks:\nAlthough the model excels on line-level code completion and clone/link tasks, all evaluations remain static-structure-centric. The framework isn’t tested on dynamic or generation tasks (e.g., code repair, test generation), which would test whether hyperbolic embeddings generalize beyond AST reasoning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KJNffqydRw", "reviewer": "Reviewer_Huu9", "summary": "The paper explores the hypothesis that the hierarchical structure of source code, typically represented by Abstract Syntax Trees (ASTs), can be more effectively modeled in hyperbolic space than in traditional Euclidean geometry.\nBased on this intuition, the authors introduce HypeCodeNet, a hyperbolic graph neural network constructed in the Lorentz model, integrating manifold-aware components such as hyperbolic embeddings, tangent-space message passing, and geodesic attention.\nThe model’s performance is evaluated across multiple code understanding benchmarks, including BigCloneBench and POJ-104 for code clone detection, as well as CodeXGLUE for code completion and GitHub Java Call Graphs for link prediction.\nIn these experiments, HypeCodeNet is compared against nine baseline models and results show that HypeCodeNet demonstrates performance comparable to or exceeding that of established baselines across multiple benchmarks, particularly on tasks requiring structural reasoning.", "questions": "**Points to justify**\n\nI do not consider those weaknesses to significantly diminish the overall contribution of the paper.\nHowever, I believe the following points require further justification and clarification.\nIf the authors can adequately address the questions raised below, I am willing to reconsider my evaluation. \n\n1) As “reasoning” has taken on a different connotation in recent LLM literature, the term “Program Reasoning” in the title may be somewhat misleading. It would be clearer to emphasize that the model focuses on encoder-based code comprehension, for example by adopting a title such as “Hyperbolic Representation Learning for Encoder-based Code Comprehension.”\n\n2) While the proposed model makes noteworthy efforts to ensure fair comparison, the baseline models and target tasks used are largely outdated, originating from 2022 or earlier. Moreover, recent advances in code understanding have been increasingly driven by large language models (LLMs), most of which follow a decoder-only paradigm, fundamentally differing from the encoder-based architecture proposed in this paper. Therefore, a discussion on the methodological timeliness, empirical relevance, and potential extensibility of the proposed approach is necessary to position the work within the modern landscape.\n\n3) Incorporating CodeT5+ [1], the successor to CodeT5, as an additional baseline would strengthen the experimental comparison and provide a more up-to-date benchmark context.\n\n4) Regarding the Code Completion task, it appears that the evaluation in this paper functions more as a Cloze Test, since it involves predicting a few masked tokens rather than performing true line-level completion. In fact, CodeXGLUE also treats this as a Cloze Test when evaluating encoder-only models such as CodeBERT, while Code Completion is evaluated using decoder-based models like CodeGPT. If there is a specific reason for evaluating under the “Code Completion” split of CodeXGLUE despite this distinction, please provide further justification. Regardless of this reasoning, aligning the task naming with the corresponding dataset would improve clarity and consistency.\n\n5) The geodesic distance appears to be incorrectly defined. For a vector $u \\in \\mathcal{L}_c^d$, $d_c(u,u)=\\frac{1}{(-c)^{\\frac{1}{2}}}\\arcosh(-c<u,u>_L)=\\frac{1}{(-c)^{\\frac{1}{2}}}\\arcosh(-1)$. However, $\\arcosh$ is defined only for $x \\ge 1$. Also, $<p, log_p^c(h)>_{\\mathcal{L}} \\neq 0$. Thus, I suspect that $<x, x>_{\\mathcal{L}}$ should be $-\\frac{1}{c}$ in the definition of $\\mathcal{L}_c^d$.\n\n6) Including a proof, or at least an outline of the key steps, demonstrating that the exponential map is the inverse of the logarithmic map is necessary for mathematical completeness and clarity.\n\n7) Equation (40), which approximates $-c<u,v>_L to 1+\\frac{-c}{2} \\left\\| u_E - v_E \\right\\| ^2$, should be further justified.\n\n**Suggestions**\n\n1) As an upper bound reference, it would be valuable to additionally report the performance of a recent LLM-based baseline, even if only with a smaller model variant.\n\n2) More comprehensive experimentation would strengthen the paper. Reporting additional results on the Defect Detection task from CodeXGLUE and on Cloze Test experiments across languages beyond Java and Python would provide stronger empirical support. If such experiments are infeasible, clarifying the reasons would be appreciated.\n\n3) It would be helpful to include a diagram, perhaps in the appendix, illustrating how attention is computed within each layer.\n\n\n4) The term \"central node\" is used but not clearly defined. If it does not carry a specific meaning, consider replacing it with a clearer phrase such as \"node to update\" or explicitly defining it for clarity.\n\n5) It would be preferable to avoid the excessive use of bold text, as it may visually interfere with paragraph structure. Using italics or underlines for emphasis would improve readability and consistency.\n\n6) Since the initial vector $z_v^E \\in \\mathbb{R}^d$ and $T_{o_c}\\mathcal{L}_c^d \\in \\mathbb{R}^{d+1}$, the dimensionality increases. Therefore, the operation described should be referred to as an injection or extension rather than a projection.\n\n7) On p.15, if $q_v^{(k)}=0$, there appears to be no need to include the bias term or the matrices $W_{Q,k}^{(l)}$ and $W_{K,k}^{(l)}$ in the computation. In this case, it would be equivalent to performing a learnable weighted sum over $m_{u \\to v}^{(l)}$.\n\n8) The section formatting is inconsistent. For instance, Section 3.2 introduces points as (1), (2), and (3), but later paragraphs switch to different numbering styles or omit numbering altogether, while the description of the output layer appears separately in Section 3.4. The numbering and formatting should be standardized for consistency, maintaining a uniform scheme or using sub-subsections if hierarchical structure is intended.\n\n**Typo**\n- p.3, line 126: \"base point\" should be written as ``base point''.\n- Appendix A: The letter 'A' in the title should also be enclosed as `A'.\n- Markdown-style emphases such as *text* or **text** appear in the appendix and should be removed for formal consistency.\n\n[1] Wang, Y., Le, H., Gotmare, A., Bui, N., Li, J., & Hoi, S. (2023, December). CodeT5+: Open Code Large Language Models for Code Understanding and Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 1069-1088).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Well-formulated hyperbolic representation architecture\nThe proposed model presents a mathematically well-grounded extension of GNNs into hyperbolic space, specifically tailored to capture hierarchical tree structures.\nUnlike prior works that primarily relied on exponential mapping, this paper explicitly defines the logarithmic map and integrates it into a novel geodesic attention mechanism, providing both theoretical and empirical justification for its design.\n\n2. Strong empirical results across multiple benchmarks\nThe model is comprehensively evaluated against nine baselines, spanning sequence-based, graph-based, and hybrid architectures, across three tasks and four datasets.\nHypeCodeNet consistently surpasses or matches the strongest baselines, demonstrating its robustness and generality across code understanding scenarios.\n\n3. Rich analytical evaluation\nThrough extensive ablation studies, the authors show that the proposed structure performs effectively even with extremely low-dimensional embeddings (e.g., 32 dimensions), and can outperform GraphCodeBERT with significantly fewer parameters.\nThis suggests that hyperbolic geometry provides an efficient inductive bias for hierarchical data.", "weaknesses": "1. Limited input flexibility due to reliance on AST parsing\nThe model requires input to be parsed into a valid AST graph.\nConsequently, it cannot directly process unstructured code snippets, natural language prompts, or free-form textual descriptions.\nThis constraint implies that the method may need additional components to handle real-world data where parsing is incomplete or ambiguous, limiting the model’s applicability in broader scenarios.\n\n2. Relatively slow training and inference\nAs reported in Appendix J, despite having fewer parameters, HypeCodeNet involves multiple nonlinear hyperbolic operations and iterative computations.\nThis results in $1.5-2\\times$ slower training and approximately 20% longer inference time compared to baselines.\nWhile this trade-off is understandable given the geometric complexity, it reduces throughput and may hinder deployment in latency-sensitive environments.\n\n3. Lack of theoretical discussion on gradient convergence\nThe paper empirically demonstrates stable training and overall convergence, supported by techniques such as Riemannian gradient clipping and curvature annealing.\nHowever, it does not provide a formal proof of convergence, nor does it include a theoretical analysis of convergence behavior. Also, the convergence bounds of the iterative Fr\\’echet mean computation remain unspecified.\nWhile the empirical results indicate reliable convergence in practice, the lack of theoretical guarantees leaves the model’s stability only experimentally supported and may limit its applicability in more rigorous settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)