Intrinsic Lorentz Neural Network
Authors: Xianglong Shi
Ziheng Chen
Yunhan Jiang
Nicu Sebe
Venue: ICLR 2026 Poster (2026)
Keywords: hyperbolic; lorentz model
Primary area: learning on graphs and other geometries & topologies

Abstract: Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. 
Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz batch normalization that couples gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while reducing training time. We additionally proposed a gyro-additive bias for the FC output, a Lorentz patch-concatenation operator that aligns the expected log-radius across feature blocks via a digamma-based scale, and a Lorentz dropout layer.
Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-art performance and computational cost among hyperbolic models and consistently surpasses strong Euclidean baselines.

Published as a conference paper at ICLR 2026
INTRINSIC LORENTZ NEURAL NETWORK
Xianglong Shi Ziheng C Yunhan Jiang", , Nicu Sebe² Universitty Science and Technology of China, University of Trento, "Peking University x1 shi@mail. ust edu.cn, ziheng_ ch@l 63. com,
ABSTRACT
Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying extrinsic parameterizations. To address it, we propose the Intrinsic Lorentz Neural Network (ILNN), fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces novel point-to- to-heerplaaaeeee fully connected layer (FC), replacing traditional Euclidean affine logits with closed-fore hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions re- spect the inherent curvature. Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz. batch normalization that couples gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while re- ducing training time. We additionally proposed a gyro-additive bias for the FC output, Lorentz patch- patch-concatenation operator that aligns the expected log-radius across feature blocks via a digamma higamma-based scale. and Lorentz dropout layer. Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-arr performance and computational cost among hyperbolic models and consistently surpasses strong Euclidean baselines. The code is available at this url.
INTRODUCTION
Hierarchical structures exist across a wide range of machine learning applications, including computer vision (Khrulkov et al., 2020; Ghadimi Atigh et al.. 2022; Bdeir et al., 2024; Pal et al., 2025; Shi et al., 2025; Wang et al., 2026), natural language processing (Ganea et al., 2018; Tifrea et al., 2019; Yang et al., 2024a), knowledge-gras -gaah reasoning (Nickel & Kiela, 2017; Balazevic et al., 2019; Welz et al., 2025), graph learning (Chami et al., 2019; Yang et al., 2022: Li al., 2024), brain signal decoding (Li et al., 2026), and genomics tasks (Khan et al., 2025; Zhou & Sharpee, 2021; Yang et al., 2025). While Euclidean neural networks often incur high distortion or require excessive dimensionality when embedding hierarchical or scale-free structures (Nickel & Kiela, 2018), hyperbolic geometry, characterized by constant negative curvature, offers exponential representational capacity and enables more compact embeddings (Ganea et al., 2018; Shimizu al., 2020). Consequently, hyperbolic neural networks (HNNs) have demonstrated notable success in applications with hierarchical structure (Ganea et al., 2018; Chami al., 2019; Shimizu al., 2020; Gulcehre al., 2019; Li al., 2024; Fan et al., 2024; Leng et al., 2024; Bdeir et al., 2024; Pal et al., 2025; Chen et al., 2025a; He et al., 2025).
Early hyperbolic neural networks predominantly operated in the Poincaré model because its gyrovec- tor formalism makes many neural primitives straightforward define (Ganea et al., 2018: Shimizu et al., 2020). However, the unit-ball constraint and boundary saturation render the Poincaré ball more susceptible numerical instabilities than the Lorentz model, motivating a recent shift toward Lorentz neural networks (LNNs) that exhibit improved optimization stability (Bdeir et al., 2024). Despite this trend leading to the emergence of excellent work (He et al., 2024; Fan et al., 2024; Liang et al., 2024), recent advanced LNNs (Bdeir et al., 2024; Khan et al., 2025) architectures remain only partially
"Equal contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
intrinsic: they mix Euclidean affine transformations with manifold operations or rely extrinsic parameterizations that compromise geometric consistency. For example, the Lorentz fully connected layer (LFC) (Bdeir et al., 2024) that applies Euclidean mappings to Lorentz vector, then extracts the space part from mapping output and accordingly calculates the time part to form the output Lorentz vector. Normalization further illustrates this tension. Lorentz batch normalization (LBN) (Bdeir et al., 2024) recenter features but either ignore gyro-variance, while GyroBN (Chen et al., 2025a:c) offers gyrogroup-basee control yet can still be computationally heavy due to depending on Fréchet-type statistics. These compromises limit the representational power and efficiency of Lorentz neural networks.
An effective way to mitigate partially intrinsic designs is the -ohye------eeeea formulation, which has shown success spherical, hyperbolic, and matrix geometries (Lebanon & Lafferty, 2004a; Shimizu et al., 2020; Bdeir et al., 2024; Chen et al., 2024a;b; Nguyen et al., 2025). Inspired by this, we introduce II.NN, a fully intrinsic hyperbolic network in which every operation, parameter, and update is defined inside the Lorentz model. At its core is a Point-to-hyperplane Lorentz Fully Connected (PLFC) layer that replaces Euclidean affine transformation with closed-form Lorentzian distances to learned hyperplanes, yielding curvature-aware, PLFC, we develop intrinsic components: CyroLBN, a batch normalization that couples gyro-centering with variance-controlled gyro-scaling and outperforms LBN and GyroBN while reducing wall-clock time. Furthermore, we introduced log-radius algin Lorentz patch-concatenation to build the stable CNN module, gyro-additive bias, and Lorentz dropout layer to further improve its performance. By designing intrinsic geometric operations, ILNN maintains mathematical consistency and geometric interpretability throughout the network.
interrerrreee decision functions. Surrounding
The main contributions of our work can be summarized as follows:
1. We propose an intrinsic hyperbolic neural network that eliminates reliance on extrinsic Euclidean operations, thus fully harnessing hyperbolic geometry.
2. We introduce novel
parentz,uull connected layerPPLFC), which replaces traditional affine transformations with intrinsic hyperbolic distances, significantly enhancing representational fidelity.
3. We introduce a GyroLBN, Lorentz batch normalization that combines gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while reducing training time.
4. Extensive experiments demonstrate the effectiveness of ILNN, achieving ofte--of-the-rr art per- formance on DXFAR-10/100 datasets and genomic benchmarks (TEB and GUE), consistently outperforming Fuclidean and existing hyperholic counterparts.
2 RELATED WORK
Hyperbolic embeddings. large body of work demonstrates that negatively curved representations can efficiently encode hierarchical and scale-free structure across modalities. Foundational results on hyperbolic embeddings (eg. Poincaré embeddings) show strong benefits for symbolic data with latent trees (Nickel & Kiela, 2017), and early hyperbolic neural architectures extend core layers and classifiers to the Poincaré model for language tasks (Ganea et al., 2018). On graphs, hyperbolic GNNs capture hierarchical neighborhoods and outperform Euclidean counterparts on link prediction and node classification (Chami et al., 2019); knowledge graphs likewise benefit from multi-relational hyperbolic embeddings (Balazevic et al., 2019). In vision, hyperbolic image embeddings improve retrieval and classification under class hierarchies (Khrulkov et al., 2020), and fully hyperbolic CNNs (HCNN) generalize convolution, normalization, and MLR directly in the Lorentz model, yielding strong encoder-side gains (Bdeir et al., 2024). Hyperbolic modeling has also been adapted to genomics, where hyperbolic genome embeddings leverage evolutionary signal to surpass Euclidean baselines across diverse benchmarks (Khan et al., 2025). In this paper, target these two application fronts, vision and genomics.
Hyperbolic neural networks. Designing fully hyperbolic neural networks means replacing every Euclidean building block with operations that are intrinsic to a negatively curved manifold, without shuttling features back and forth between Euclidean and hyperbolic spaces. Early work the Poincaré ball showed how to lift common layers and primitives (linear/FC maps, activations, concate- nation, MLR) using gyrovector-space calculus and Möbius operations (Ganea et al., 2018; Shimizu
2

---

Published as a conference paper at ICLR 2026
et al., 2020), while concurrent efforts formulated end-to-end hyperbolic models beyond simple projection heads (Chen et al., 2022). Due to the Poincaré ball's greater susceptibility to numerical instability, subsequent work tried to generalize network components to the Lorentz model (Gulcehre et al., 2019; Chami et al., 2019; Fan et al., 2023). Beyond classification heads, attention mecha- nisms (Gulcehre et al., 2019), graph convolutional layers (Chami et al., 2019), and fully hyperholic generative models (Qu & Zou, 2022) were also developed in this setting. A major advance came with HCNN, which introduced Lorentz-native formulations of convolution, batch normalization, and multinomial logistic regression, bringing vision encoders fully into the hyperbolic domain (Chen al., 2022; Bdeir al., 2024; Yang et al., 2024b). But it still remains partially intrinsic, mixing Euclidean operations with hyperbolic ones. Our work follows this intrinsic-firsl principle and departs in key aspects: a point-to-hyperplane fully connected layer in the Lorentz model with closed-form h-distances for logits and GyroLBN, a Lorentz batch normalization that couples gyro-centering with gyro-scaling.
Normalization in HNNs. Normalization layers are central stable and efficient training, yet extending them to curved spaces remains challenging. A general Riemannian batch normalization (RBN) based on the Fréchet mean was first made practical by differentiable solvers, enabling manifold- aware centering and variance control; however, its iterative nature can be slow in practice (Lou et al., 2020). Recently, Bdeir et al. (2024) introduced Lorentz Batch Normalization (LBN), which uses the closed-form Lorentzian centroid (Law et al., 2019) for re-centering and a principled tangent-space rescaling. Concurrently, Gyrogroup Batch Normalization ( with gyro-structure, performing gyro-centering and (GyroBN) generalizes RBN to manifolds gyro-scaling that better preserve distributional shape under gyro-operations (Chen et al., 2025a). While LBN provides efficient Lorentzian re-centering and global rescaling, it does not explicitly align batch statistics under eyro-operttions: GyroBN performs such alignment but is computationally heavier because relies on Fréchet means. Motivated by these trade-offs, propose CyroLBN, which aligns batch statistics under gyro-operations within the Lorentz model while retaining the speed and practicality of closed-form re-centering.
BACKGROUND
There are five isometric hyperbolic models that one can work with (Cannon et al., 1997). We adopt the Lorentz (hyperboloid) model due to numerical stability (Mishne et al., 2023).
Lorentz model. The Lorentz model embeds hyperbolic space as the upper sheet of a two-sheeted hyperboloid in dimeensiiioa Minkowski space. It is defined as
LE { f E x) R: zo > 0}.
where K < 0 denotes the constant sectional curvature, and 1), == -Joll + Σ Trys is the Lorentz inner product. Following Ratcliffe (2006), we write x = (Io, x.) with time coordinate 15 and space coordinates v, R" The origin is 0 = (K-1/2,0 ........), 0). The closed-form squared distance is given by dz(2z y) = a| y 12 A 2(x 3) c. Other Riemannian operators are presented in Appendix B, such as exponential and logarithmic maps, and parallel transport.
Figure 1: 2-dimensional Lorentz model in the 3-dimensional Minkowski space.
Gyrovector space. It forms the algebraic foundation for the hyperbolic space, as the vector space for the Euclidean space (Ungar, 2022). A gyrovector space has gyroaddition and scalar gyromultiplication, corresponding to the vector addition and scalar multiplication in Euclidean space. Recently, Chen et al. (2025c) proposed the gyroaddition and gyromultiliication over the Lorentz model:
x + y = Exp,, (PT (L)g)()))) Yz L
(1)
O σ Expy (t (Log(())) VE R, Yz E Lj-
(2)
3

---

Published as a conference paper at ICLR 2026
Particularly, the inverse is Cr := (-1) xr= -x,]. which satisfies @ (@z) (@x) @ z 0. Eqs. and admit closed-form expressions, which are more efficient.
INTRINSIC LORENTZ NEURAL NETWORK
We introduce the Intrinsic Lorentz Neural Network (ILNN), whose components are entirely defined by the Lorentz geometry. Specifically, we propose (i) point-to-hyperplatt fully connected (FC) layer, (ii) GyroLBN, a Lorentz batch normalization layer, and (ii) several other intrinsic modules, including log-radius concatenation, Lorentz activations, and Lorentz dropout.
4.1
LORENTZ FULLY FULLY-CONNECTER CONNECTED LAYER
FC layers perform an affine transformation defined by y = Ax b, which can be element-wisely expressed as y/2 a&æ by. where æ, a₃ R" and 6y- R. Geometrically, this transformation can be interpreted as mapping the input vector æ to an output score yk. representing either the coordinate value the signed distance relative to hyperplane passing through the origin and orthogonal the k-th coordinate axis in the output space Rn (Shimizu et al., 2020; Chen et al., 2025b). Motivated by this geometric interpretation, in this section, we first derive the Lorentz multinomial logistic regression (Lorentz MLR) to obtain the signed distance, and subsequently present the formulation of the point-to-hyperyy hyperplane Lorentz fully connected (PLFC) layer.
Lorentz MLR. In Euclidean space, the multinomial logistic regression (MLR) for class E (I...., C.....,Cee.. can formulated as the logits the Euclidean MLR classifier using the distance from instances to hyperplanes describing the class region (Lebanon & Lafferty, 2004b), which can be written as
ply = x) x exp(uu(x)), vo(x) sign((a P.)) (a.ld((z, Ha..p. ae R" (3)
where Ha..p. = (xc E lk: (a., x - paP } is the Euclidean hyperplane of class e and .(...) denotes Euclidean distance. Once we have the Lorentz hyperplane and closed-form point-hyperplane distance, we can extend Euclidean MLR to the Lorentz model.
In Lorentz model L following Bdeir al. (2024), for p E Lik and E TTpLkk the hyperplane passing through p and perpendicular to w is given by Hw.p = f E LL (w, Φ) = 0}, where w should satisfy the condition (w, w) 0. To eliminate this condition,
E TTLL is parameterized by a vector Σ E ToL = [0, az/ /|||||||, where a E R and z E R". As w E TpL z is parallel transport p. the Lorentz hyperplane defined by
Hz.0 fx LL cosh (V- (z, x.) sinh sinh(V--Kin (V-Ka) Ka)
||||| x4, =),
(4)
where and z represent the distance and orientation to the origin, respectively. Due to the = ToL and the sign-preet greeerreng property of parallel transported, this construction naturally satisfies (w, un) > O. Then for x E L. the distance to a Lorentz hyperplane given by
cosh (V- (z. ma) (V-K
de(ar Ma,,)
sinh -
(5)
V-K
V cash (V- Ka)=1l (sinh(( |=
Substituting Eq. and Eq. into Eq. 3, we obtain that the Lorentz MLR's output logit corresponding to class given by
Usciae (x) = V-K sign(aw)) sinh -1
(V-K Oc
(6)
Be
cosh (V-Ka) (2, (2, xx) - sinh (V-Ka),
B. /|cosh (-Ka) x (sinh (V-Ka) ||
4

---

Published as a conference paper at ICLR 2026
Lorentz fully-connected layer (PLFC). Shimizu al. (2020); Chen al. (2025b) interpreted the Euclidean FC layer as an operation that transforms the input a via ex (2), treating the output yıu the signed distance from the hyperplane Ho ,o passing through the origin and orthogonal to the k-th axis of the output space I" which can be written
d' (3 H2()))) (a)),,
1.......7 m.
where (-,-) denote the signed distance in Euclidean space. We now collect the above ingredients into an intrinsic FC layer that maps an input x E LK to an output = (yo, E LL Let { (zx, as ) 1 be learnable hyperplane parameters and define Us (x(x))= = Uz (x) as in Eq. 6. Matching each us (xxx)) (z) to the signed /i-distance from y to the k-th coordinate hyperplane Ho _0 = { (#o, Z1,...,
"a....,mm.. Fmm) E LK | (e(*), 2) = 2
k constraint.
m) fixes the spatial coordinates, while the time coordinate follows from the hyperboloid
Theorem 1 (PLFC layer). Let a E L, Z = {ze CIR" and a {ax]E=1 C.R. The point-to-hyperplanee
Lorentz fully connected layer PLFCK: L + LR is
(x) = 1=z2ia( (2), k= m...., m,
(8a)
Me,k = V-K sinld V K vx)).
(8b)
(V--)- 11
(8c)
where ys (Je.1..... ..smm) = Av + with A, = and bias = a.
)" In the flat-space limit K +0 equation 8 reduces to the Euclidean affine map
It can be shown that the signed distance from to cach Lorentz hyperplane passing through the origin and orthogonal to the k-th coordinate axis is given by us (w), as detailed in the Appendix E.I, thereby fulfilling the properties described above.
Gyro-bias. A learnable offset LEL can be added intrinsically via the gyroaddition, <- y b. yielding the final PLFC output.
Discussion. The previous Lorentz fully connected layer (LFC) used HCNN was formed by
y [voo(xx, v) - 1/K, ((Wx,v)
where x E C& and Wx is a standard matrix vector product in RR. This Wx is defined using the ambient linear structure of the Minkowski space, not any operation the Lorentz manifold so itself only partially intrinsie. The PLFC depends only on Lorentzian operations, avoids tangent-space lincarisation, and enjoys closed-for form gradients, making it an efficient and curvature-consistet replacement Eaclidean FC layers in hyperbolic networks. More detailed discussions are in Appendix D and a theoretical advantage of PLFC over LFC is shown in Appendix E.2.
4.2 GYROGROUP LORENTZ BATCH NORMALIZATION (GYROLBN)
Batch Normalization (BN) facilitates training by normalizing batch statistics. Recently, Bdeir et al. (2024) proposed LBN, which uses Lorentzian centroid to efficiently compute the batch mean. However, their approach fails to normalize sample statisties. Besides, Chen et al. (2025a:c) extended BN into manifolds based on gyrogroup structures, referred to GyroBN (Chen et al., 2025a). Although it can normalize sample mean/variance on Lorentz spaces, the involved Fréchet mean could be inefficient. Therefore, we propose GyroL.NN to combine the gyrogroup normalization with I Lorentzian centroid & radius statistics, retaining GyroBN's effectiveness while eliminating computationally expensive Fréchet mean.
Batch centering and dispersion. Generally, the Fréchet mean must be solved iteratively, which signif- icantly delays the training speed. Therefore, following Law et al. (2019); Bdeir et al. (2024), with æ E Lk. N 2 0, >i i-1 V/: > 0 and a batch of features B = {#; E Lx im is given by. we define the batch mean by Lorentzian centroid which can he calculated efficiently in closed form (Law et al., 2019):
Σ P:E;
#B
(9)
V-K 1I mællle
which solves mingggtt En (n pB). Moreover, the mean not weighted, which gives i
As for dispersion, we adopt the Fréchet variance x² RR defined as the expected squared Lorentzian distance hetween point 2; and the mean μo, and given by o Im dz 11) (Kobler al., 2022).
5

---

Published as a conference paper at ICLR 2026
Normalization map. Denote learned scale 7 E R>u (per-channel) and learned bias B E Lk: For each sample v, the GyroLBN output
Biasing
Scaling
Centering
ViS,, 3.
30
7 VoB te
OpB
> O.
(10)
where denotes left gyroaddition Eq. denotes gyro inverse, and @ denotes gyro scalar product Eq. 2. Unless otherwise stated, during inference, we maintain per-channel running statisties by updating the Lorentzian centroid (mean) and dispersion (variance) with momentum and, at test time, substitute these for batch stats.
Discussion. Gyrol BBN unifies the gymggroup normalization paradigm of GyroBN with the efficient Lorentz statistics of LBN, differing from GyroBN only in the choice of statisties by replacing Fréchet-based batch estimates with closed-form Lorentzian centroid and variance while retaining the same gyrogroup centering and scaling scheme. Compared to Fréchet variance used generic Riemannian 2022), the mean-radius statistic avoids iterative solvers and proved numerically stable in our settings. It therefore (i) remains fully intrinsic, (ii) avoids iterative Fréchet solvers (important for large batches and 2D convolutions), and (iii) integrates naturally with gyro- additivee residual/bias layers used in encoder. Compared LBN and Fréchet-base based GyroBN (Lou et al., 2020), GyroLBN consistently reduced wall-clock time while improving accuracy our settings, as shown Table
(Lou et al., 2020; Chen et al..
4.3 OTHER LORENTZ MODULES
Log-radius concatenation.
When concatenating Lorentz patches, each with (1 + d) coordinates (one time and spatial), naively stacking the Nd spatial components biases the resulting feature norm toward higher di- mensions: the expected radius grows with N d, skewing subsequent layers. We introduce a log-radius-a preserviin concatenation that makes the expected log spatial radius invariant to the number of concatenated blocks. Let v E R denote the spatial part of a block and assume its radius factorizes as ||v|| = T with T ~ x Then
Ellog [|v]]] = log + 2 (1b( %1) + log 2).
where v is the digamma function. To keep B[log] ||]|]] constant across dimensions, we scale each block's spatial part by
n.n = exp ((())
with n = Nd the total post-concat spatial dimension and n. given per-window tensors (l1,..., .(x) (times) and (uI..... uu) with u, E R" (spaces), we form the scaled space ū; and recompute single time coordinate that keeps the output the hyperboloid:
the per-block spatial dimension. Concretely,
(13 )
where A: > 0 sets the origin time via to = /k (in practice, with K -1, 1). The final concatenated vector is [t', u....., ūN] E R R- This log-radius alignment (i) is parameter-free free, (ii) is robust to heavy-tailed radii because it matches geometric means, (iii) preserves the Lorentz constraint by design, and (iv) avoids domination by any single wide block, improving stability as kernel size or channel count grows.
Lorentz convolutional layer. We use channel- feature map representations throughout HCNNs, and add the Lorentz model's 's time component as an additional channel dimension, following Bdeir et al. (2024), hyperbolic feature map can be define as an ordered set of n-dimensional hyperbolic vectors, where every spatial position contains vector that can be combined with its neighbors
The convolutional layer can be formulated concatenation of values within its receptive field (Shimizu et al., 2020). Then, we extend this definition by replacing the Euclidean FC and concatenation with our PLFC and log-radius- preserving concatenation.
matrix multiplication between linearized kernel and the
Let {I w L H.W be input hyperbolic feature map and let H1 x W denotes the kernel size with R.W stride 6. For each spatial location (h, w) we gather the patch {I 5h, w+&w I h, C LK pad with the origin
(VI/(-K), ..... 0) and concatenate these HW vectors by the log log-radius scheme introduced above. We defined the Lorentz convolutional layer as
y PLFC(LogCa(((#w) 488. E LL) R.
(11)
where h' and w' denote the starting position, and LogCat denotes our log-radduu concatenation.

---

Published as a conference paper at ICLR 2026
Lorentz dropout. To regularize features without leaving the manifold, adopt a dropout operator that acts on Lorentz coordinates and then reprojects the hyperboloid. Concretely, during training, we apply an elementwise Bernoulli mask to the current representation x E LX (probability p of zeroing cach entry), yielding & in ambient R=+!, Because naive masking can violate the hyperboloid constraint and the positivity of the time component, we immediately map back via a projection p that restores (z, z), = - and xo > O. At evaluation, the operator the identity. In practice, the variant "log-exp" " scheme that applies a log map to Tol-k. performs Euclidean dropout, and then returns via the exponential map, because the nonlinearity of expo couples all coordinates so that masking any tangent component perturbs the entire point after mapping back to the manifold. It is parameter- free, numerically stable, and compatible with subsequent intrinsic layers (e.g.. GyroLBN and PLFC), since its output again lies on Lj--
observe that the "mask & project" scheme outperforms
Following Bdeir et al. (2024), we define activations directly the Lorentz model by acting on the spatial coordinates only and then recomputing the time coordinate from the hyperboloid constraint. For example, given x = (mo, 2x) L and an elementwise nonlinearity activation function ReLU, Lorentz ReI.U can he defined as
Lorentz activation.
L- -eReLU(x)=
[V = + |[ReLU(z.)]B ReLU(x.)]
EXPERIMENT
We evaluate ILNN on image classification dataset CIFAR-10/100 (Krizhevsky et al., 2009) and genomics (TEB (Khan et al., 2025), GUE (Zhou et al., 2023)) and compare against Euclidean and multiple hyperbolic baselines under matched training recipes. For fairness, cach Euclidcan backbone is translated to the hyperbolic setting via one-to-one module replacement, keeping depth/width, parameter count, and schedule as close as possible. All models are implemented in PyTorch (Paszke et al., 2019) with 32-bit precision. We fix curvature to K = 1 and train with Riemannian optimizers from Geoopt (Kochurov et al., 2020): RiemannianSGD for CIFAR and GiemannianAdai for genomics. Unless otherwise stated, results are averaged over five random seeds and reported as mean + std; classification uses accuracy on LLFAR-10/100 and MCC on genomics. All experiments are conducted on NVIDIA A100 SUGB GPUs. Additional configuration details (batch size, learning rate) appear in Appendix F.
5.1 IMAGE CLASSIFICATION
Experimental setup. Following the evaluation setup of HCNN (Bdeir et al., 2024) and symmetric space (Nguyen al., 2025), we benchmark ILNN on CIFAR-10 and CIFAR-100. For cach method, we instantiate ResNet-18 backbone (He et al., 2015) in the corresponding geometry: The original ResNet-18 with BN and linear classifier (Euclidean); Euclidean encoder hyperbolic head (Poincaré or Lorentz); all layers fully to the target manifold (HCNN (Bdeir et al., 2024) and our ILNN). The relative distortion from input manifold to classifier (Bdeir et al., 2024) is comparable across (6.- 0.26 0.26 for CIFAR-10: 10; 0.23 for CIFAR- 100).
Main results. Table summarizes test accuracy as msantsd over five runs ILNN attains the highest average accuracy on both datasets, 95.36% on CIFAR-10 and 78.41% on CIFAR-100. exceeding the Euclidean ResNet-18 baseline by +0.22 and +0.69 percentage points (pp). respectively. Relative to the strongest prior hyperbolic competitor (HCNN-Lorentzz)
Table 1:
Classification accuracy (%) of ResNet- 18 models. We estimate the mean and standard deviation from five runs. The best performance is highlighted in bold (higher is better).
CIFAR-10 (6,m 0.26)
CIFAR-100 100 (5.u 0.23)
Euclidean (He et al., 2015)
95.14 + 0.12
77.72 + 0.15
Hybrid Poincaré (Guo et al., 2022)
95.04 + 0.13
77.19 0.50
Poincaré ResNet (Van Spengler et al., 2023)
94.51 1 0.15
76.60 0.32
(Fan al., 2023)
81.72 1 7.84
44.35 1 2.93
Ahelidean-Poincar--- (Ganea al., 2018)
95.14 0.11
77.78 0.09
Muelidean-Poincare (Nguyen et al., 2025)
95.23 + 0.08
77.78 0.15
ILNN also improves by +0.22
Hybrid Lorentz (Bdeir al., 2024)
94.98 + 0.12
78.03 + 0.21
pp on CIFAR-10 and +0.34
HCNN Lorentz (Bdeir et al., 2024)
95.14 + 0.08
78.07 + 0.17
pp on CIFAR-100. Although slightly larger standard deviation than
ILNN (ours)
95.36 + 0.13
78.41 + 0.23
ILNN exhibits
HCNN-Lorentz, on CIFAR-IO, even the conservative lower bound of ILNN (95.23%) exceeds the upper bound of HCNN (95.22%) on CIFAR-IOK 100, the lower bound of ILNN is specitically 78.24%, equal to the upper bound of HCNN. These gains demonstrate that our intrinsic approach preserves the geometry of the Lorentz model without distortion: by combining point-to-hyperplane FC and GyroBN within fully negative-curvature space, IN leverages the native manifold structure more effectively than others.
7

---

Published as a conference paper at ICLR 2026
Visualization. We visualize embeddings by mapping network outputs from the Lorentz model the Poincaré ball and, in parallel, applying the logarithmic map at the origin view them in the tangent plane; colors indicate output label prediction. In Figure 2 (CIFAR-1O) 10), the ILNN embeddings form ten compact clusters, whose areas are visibly smaller than the corresponding clusters produced by HCNN. Only marginal colour bleeding occurs at the boundaries, indicating that the decision margins learned by the point-lo-hyperplnee FC induce margins the harder CIFAR-100 task, as
aligned with the data geometry. The effect becomes even more pronounced shown in Figure 3. Despite packing 100 classes into the same 2D manifold, ILNN still yields dense colour "islands" with clear gaps in between, whereas HCNN exhibits overlapping clouds and several mixed-colour zones. These visual trends are consistent with the quantitative improvements in Table 1.
5.2 GENOMIC CLASSIFICATION
Experimental setup. We evaluate on the most challenging subsets of the TEB and GUE genomics bench- marks, specifically those on which the published HCNN (Khan et al., 2025) does not surpass Euclidean convolutional baseline. For every dataset, we instantiate four models that share the same convolutional stem classifier width: our comparisons, the Euclidean CNN uses standard convolutions, Euclidean BN, and a linear classifier; HCNN-S employs Lorentz modules with a single global curvature K across the network; HCNN-M uses Lorentz modules with layer-wise curvatures K; and IINNN is fully intrinsic, fcaturing GyroBN and a point-to-hyperplang FC with gyro-bias, while fixing global curvature K = 11 All hyperbolic models are optimized with Riemannian We measure performance by the Matthews correlation coefficient (MCC).
10-3 "); the Euclidean baseline uses Adam with an identical schedule.
Main results. Table shows that ILNN achieves best score on every task. On the two TEB pseudogene sets, it improves over the Euclidean baseline by +9.6 and +13.0 pp. and still exceeds the stronger HCNN-S by +2.0 and 8.8 pp. respectively. demonstrating clear gains where previous hyperbolic models were already competitive. The advantage is even more striking GUE. For Covid- variant elassification, HCNN collapses (MCC 36.7/14.8) much lower than the Euclidean model, whereas ILNN matches and slightly surpasses the Euclidean score (64.8 vs. 63.6). On promoter-related tasks, ILNN consistently raises the bar: Tata core-promoter detection jumps from 79.9 (best prior) to 83.9, and the most difficult "all" split goes from 67.2 to 70.7. Across the board, ILNN tightens the worst-case gap between hyperbolic and Euclidean models and converts several of HCNN pndfo--erformance into clear wins. These results confirm that fully intrinsic design choices, point-to- to-hyo hyperplane FC, GyroLBN, and other proposed components, translate into tangible accuracy gains on real-world genomic data.
Table 2:
Model performance (MCC) all real-world genomics datasets, averaged over five random seeds (mean + standard deviation). The two highest-scoring models are in bold, denotes that the result was reproduced under the same setting.
Model
Benchmark
Task
Dataset
Euclidean CNN
Hyperbolic HCNN-S
Hyperbolic HCNN-M
Hyperbolic ILNN
processed
60.66-aa2
68.30
65.41-5.54
70.26-0.22
Pseudogenes
unprocessed
51.94+209
56. 3
58.36-1.0
64.90-n.74
Covid Variant Classification
Covid
63.62a134
36.7 199.69
1
64.76x0.54
78.26+285
79.54=1.6
79.87=2.0
83.90
GUB
Core Promoter Detection
notata
66.60s107
66.5 66.52a0.28
25.955
72.59.0.99
all
66.472074
65.26
67. 67.16.0.55
70.89
tala
78.5843.99
79.74.2 66
78.77
83.2611.90
Promoter Detection
notata
10.81
39.8668.76
90.28s0.37
92.48
88. 38,00+a.9
87. 87.60=0.5
87.93
91.34+n.38
5.3 GRAPII
Experimental setup. To further evaluate the effectiveness of our method, we extend it to three widely- used graph datasets (AIRPORT, CORA and PUBMED), which exhibit intricate topological and hierarchical relationships, making them an ideal testbed for evaluating the effectiveness of hyperbolic networks (e.g.. HGNN, HGCN, HGAT, HAN, HNN++ and Hypformer). We choose the Hypformer as baseline and replace the Linear layer in Hypformer.
The quantitative results are summarized in Table 3. Overall, hyperholic models consistently outperform their Fuclidean counterparts on all three benchmarks, confirming that negatively curved representa-
Main results.
8

---

Published as a conference paper at ICLR 2026
Figure 2: Embedding visualization of CIFAR-10 dataset in Poincaré and Tangent Space. Colors represent labels. HCNN (94.98, left) and ILNN (95.48, right).
-00
400
Figure 3: Embedding visualization of CIFAR-100 dataset in Poincaré and Tangent Space. Colors represent labels. HCNN (77.67, left) and II.NN (78.64, right).
tions are well-suited for graphs with rich hicrarchical structure. Among existing methods, HAN, HNN++ and SGFormer constitute the strongest baselines, while Hypformer further improves their performance. Building on this backbone, Hypformer-PLFC achieves the best results on all datasets, reaching 96.03%=0.34 on AIRPORT, 0.19 on CORA, and $2.52%-1 0.333 on PUBMED. Compared with original Hypformer, this corre- sponds to absolute gains of +1.03, +-0.68, and 1.22 percentage points, respectively. The standard deviations remain small, and the confidence intervals of Hypforme++PLFC are largely separated from those of competing methods AIRPORT and PUBMED, indicating that the observed gains statistically stable. These results demonstrate that replacing Euclidean linear classifier Hypformer with our hyperbolic point-to- to-hyperplane PLFC layer provides a simple yet effective way to enhance graph representation learning on graph benchmarks.
85.68%1.00 1.119
5.4 ABLATION STUDY
To better understand the con- tributions of each architee- tural component, we conduct an ablation study on our two main innovations: the point- to-hyperplane fully connected head (PLFC) and the Gy- rogroup Lorentz Batch Nor- malization (GyroLBN). We compare them against their Lorentz counterparts (LFC, LBN) as well GyroBN, mea- suring both classification accu- racy across CIFAR-10 and ge- nomics tasks, and training ef- ficiency disentangle the ef- fects of the classifier design and the normalization strategy, and to assess whether their combination leads to comple- mentary improvements.
Table 3: Testing results (Accuracy) on AIRPORT, CORA and PUBMED. The best results are in bold, respectively.
Models
AIRPORT
CORA
PEB
GCN (Kipf & Welling, 2017)
81.4 0.6
81.3 0.3
78.1 0.2
GAT (Veličković al., 2017)
81.5 11.50.5 0.3
83.0 0.7
7..
SGC (Wu al., 2019)
82.1 0.5
90.140.2
78.7
HGNN (Liu et al., 2019)
84.7
7.8
7
HGCN (Chami al., 2019)
89.3 0031100 1.2
76.5
78.0 1.0
HGAT (Chami et al., 2019)
89.6 1.0
1.4 0.7
GraphFormer (Ying al., 2021)
88.11
0.3
73.3
GraphTrans (Wu et al., 2021)
94.3
77.6
0.7
GraphGPS (Rampášek al., 2022)
9.9
1.4
72.8 1.4
FPS-T (Cho et al., 2023)
9.6
8.7
0.6
HAN (Gulcehre et al., 2019)
9.69
8
79.0 0.6
HNN+ (Shimizu al., 2020)
0.3
82.8 0.6
0.44
F-HNN (Chen al., 2022)
93.0
81.0 + 0.7
77.5 0.8
NodeFormer (Wu et al., 2022)
8.6
82.2 0.9
79.9 1.0
SGFormer (Wu al., 2023)
92.9 0.5
83.2 + 0.9
80.0 0.8
Hypformer (Yang et al., 2024b)
95.0 1. 0.5
85.0 0.3
81.3 0.3
Hypformer+PL.FC
96.0 - 0.3
85.7 + 0.2
82.5 0.3

---

Published as a conference paper at ICLR 2026
Table 4: Ablation on PLFC and GyroLBN. Fit time denotes the training time of the CIFAR-10 dataset epoch.
Model
Benchmark
Task
Dataset
LFC LBN
LFC
PLFC
PLFC
GyroLBN
GyroBN
GyroLBN
CIFAR-10
95.14 14.00
95.19
95.28s0.17
95.36+0.13
tata
78.26+2
81,33=3.19
10.89
83,90
Core Promoter Detection
notata
66.60=1.07
71.92+0:52
72.22+0.82
72.59
all 66.4710.74
69.74.1.3
70.14
70.89
78.58+3.3
80,46+0.99
81.16
83.26s1.90
Promoter Detection
notata all 88.00a0.39
+0.51
91.88-1.01
91.67+0.49
92.48
90.28
91.02a0.56
91.34.0.38
Fit Time(s)
125
314
169
PLFC vs. LFC. Holding
the normalizer fixed, replacing the Lorentz fully connected head with the hoitt-o--ypeerplane fully connected head consistently improves accuracy. With GyroLBN, PLFC outperforms LFC on CIFAR- (95.36 vs. 95.19, +0.17) and on six genomics subsets, with gains of +2.57 on tata core-promoter detection (83.90 vs. 81.33), +0.67 on notata (72.59 vs. 71.92), and +0.97 on the all split (70.89 vs. 69.74); for promoter detection, the improvements are +2.80 (tata: 83.26 vs. 80.46), +0.60 (notata: 92.48 91.88), and +1.06 (all: 91.34 vs. 90.28). Although the PLFC variant with GyroBN also performs strongly, exceeding the LFC baseline by more than +3 on all genomics subsets and matching or surpassing on CIFAR- 10 (95.28 vs. 95.14). These results indicate that decision functions based on point-to-hypen -hhpepla-yeee distance provide a more effective inductive bias than affine logits in both vision and genomics.
GyroLBN vs. LBN, GyroBN. Under the same FC, GyroLBN improves over LBN across all tasks. With the LFC head, CIFAR- 10 increases from 95.14 to 95.19 genomics, the gains range from +1.07 to +5.32 (c.g.. core-pro pore-promttrr notata: 71.92 vs. 66.60). With PLFC head, GyroLBN superior GyroBN on every dataset, including notata in core core-pronee promoter detection, where it reaches 72.59 versus 72.22. The improvements span +0.32 to +3.01 on genomics and +0.08 on CIFAR-10. Training time measurements show that GyroLBN attains these gains with favorable efficiency, running faster than PLFC. with GyroBN (169 314 s) and even faster than LBN for the comparison of LFC between Gyrol .BN and LBN (125 better accuracy and efficiency trade-off than both LBN and GyroBN our settings.
142 s). Overall, GyroL .NN offers
Furthermore, to ensure a fair comparison with GyroBN, we further vary the number of Fréchet mean iterations used by the normalizers (1, 2, 5, 10, and a fixed-point solve denoted by co; see Appendix G). Across all iteration cases, PIFC+ Gyrol .BN remains the best- best-perforiing configuration CIFAR-10 and on every genomies split, exhibiting the effectiveness of GyroLBN again.
CONCLUSION
This study presented the Intrinsic Lorentz Neural Network, an architecture whose computations remain entirely within the Lorentz model of hyperbolic space. We introduced a point-to-hyperplanee fully connected layer that converts signed hyperbolic distances into logits, logether with GyroLBN and log-radius concatenation for numerically stable normalization and feature aggregation. Integrated into coherent network, these components yield superior performance on CIFAR- 10, CIFAR- 100, and two challenging genomics benchmarks, surpassing Euclidean, hybrid, and prior hyperbolic baselines while preserving competitive training cost. The results underscore the value of keeping every layer intrinsic to the manifold and provide practical building blocks for future work on representation learning in negatively curved geometries.
REPRODUCIBILITY STATEMENT
All theoretical results are established under explicit conditions. Experimental details are given in Appendix F. The code will be released upon acceptance.
ETHICS STATEMENT
This work only publicly available datasets and does not involve human subjects sensitive information. We identify no specific ethical concern..
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by EU Horizon project ELLIOT (No. 101214398) and by the FIS project GUIDANCE (No. FIS2023-03251), We acknowledge CINECA for awarding perfo-mnrrrm computing resources under the ISCRA initiative, and the EurolIPC Joint Undertaking for granting access to Leonardo at CINECA, Italy.
REFERENCES
Ivana Balazevic, Carl Allen, and Timothy Hospedales, Multi-relational poincaré graph embeddings. Advances in neural information processing systems, 32, 2019.
Ahmad Bdeir, Kristian Schwethelm, and Niels Landwehr. Fully hyperbolic convolutional neural networks for computer vision. In International Conference Learning Representations (ICLR), 2024.
James W. Cannon, William J. Floyd, Richard Kenyon, and Walter R. Parry. Hyperbolic geometry. In Silvio Levy (ed.), Flavors of Geometry, volume 31 of MSRI Publications. Mathematical Sciences Research Institute, 1997.
Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances neural information processing systems, 32, 2019.
Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Fully hyperbolic neural networks. In Proceedings the 60th Annual Meeting the Association for Computational Linguistics (Volume Long Papers), pp. 5672-5686, 2022.
Ziheng Chen, Yue Song. Gaowen Liu, Ramana Rao Kompella, Xiao-Jun Wu, and Nicu Sebe. Riemannian multinomial logistics regression SPD neural networks. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024a.
Ziheng Chen, Yue Song, Rui Wang, Xiao-Jun Wu, and Nicu Sebe. RMLR: Extending multinomial logistic regression into general geometries. Advances in Neural Information Processing Systems, 2024b.
Ziheng Chen, Yue Song, Xiao-Jun Wu, and Nicu Sebe. Gyrogroup batch normalization. In International Conference on Learning Representations (ICLR), 2025a.
Ziheng Chen, Xiao-Jun Wu. Bernhard Schölkopf, and Nicu Sebe. Building transformation layers for Riemannian neural networks, 2025b.
Ziheng Chen, Xiao-Jun Wu, and Nicu Sobe. Riemannian batch normalization: A gyro approach. arXiv preprint arXiv:2509.00 07115, 2025c.
Sungjun Cho, Seunghyuk Cho, Sungwoo Park, Hankook Lee, Honglak Lee, and Moontae Lee. Curve your attention: Mixed-curvature transformers for graph representation learning. arXiv preprint arXiv2230,0040S: 2023.
Xiran Fan, Chun-Har Yang, and Baba C. Vemuri. Horospherical decision boundaries for large margin classifica- tion in hyperholic space. arXiv preprint arXiv:2302. 06807, 2023.
Xiran Fan, Minghua Xu, Huiyuan Chen, Yuzhong Chen, Mahashweta Das, and Hao Yang. Enhancing hyperbolic knowledge graph embeddings via lorentz transformations. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 4575 4589,,59,,40 2024.
Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. Advances in neurul information processing sysiems, 31, 2018.
Mina Ghadimi Atigh, Julian Schoep, Erman Acar, Nanne van Noord, and Pascal Mettes. Hyperbolic image segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4429-4439 4439, 2022.
Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, et al. Hyperbolic attention networks. In ICLR, 2019.
Y. Guo, X. Wang, Y. Chen, and S. X. Yu. Clipped hyperbolic classifiers are super-hyperbolic classifiers. In 2022 IEEE/CVP Conference Computer Vision and Pattern Recognition (CVPR), pp. 1-10, Los Alamitos, CA, USA, jun 2022. IEEE Computer Society. doi: 10.1 119//CCP55288.202222222 011P556688.02220001111 2022.00010.
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015.
11

---

Published as a conference paper at ICLR 2026
Neil He, Menglin Yang, and Rex Ying. Lorentzian residual neural networks. arXiv preprint arXiv:2412.14695. 2024.
Neil Hc, Rishabh Anand, Hiren Madhu, Ali Maatouk, Smita Krishnaswamy, Leandros Tassiulas, Menglin Yang, and Rex Ying. Helm: Hyperbolic large language models via mixture-of-curvauuee experts. arXiv preprint arXiv:2505.24722. 2025.
Raiyan R Khan, Philippe Chlenski, and Itsik Pe'er. Hyperbolic genome embeddings. arXiv preprint WrXiv2250.22164 2025.
Andrei Khrulkow, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky. Hyperbolic image embeddings. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6418-6428, 2020.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
Reinmar J. Kohler, Jun-ichiro Hirayama, and Motoaki Kawanabe. Controlling the fréchet variance improves batch normalization on the symmetric positive definite manifold. In ICASSP 2022 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3863-3867, 2022. doi: 10.1109/ ICASSP43922.2001 202.9997222222966622
Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in pytorch. arXiv preprint arXiv:2005.02888 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Marc T. Law, Renjie Liao, Jake Snell, and Richard S. Zemel. Lorentzian distance learning for hyperbolic representations. Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 3672-3681, PMIR, 2019.
Guy Lebanon and John Lafferty. Hyperplane margin classifiers on the multinomial manifold. Proceedings of the twenry-first first international conference on Machine learning, 2004a.
Guy Lebanon and John Lafferty. Hyperplane margin classifiers on the multinomial manifold. Proceedings of the twenty-firt international conference on Machine learning, pp. 66, 2004h.
Zhiying Leng, Tolga Birdal, Xiaohui Liang, and Federico Tombari. Hypersdfusion: Bridging hierarchical structures in language and geometry for enhanced 3d text2shape generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2024.
Shanglin Li, Chu Shiwen, Okan Koç, Yi Ding, Qibin Zhao, Motoaki Kawanabe, and Ziheng Chen. HEEGNet: Hyperbolic embeddings for EEG. In The Fourteenth International Conference on Learning Representations, 2026.
Yancong Li, Xiaoming Zhang, Ying Cui, and Shuai Ma. Hyperbolic graph neural network for temporal knowledge graph completion. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation ELRA and ICCL.
2024), pp. 8474 8486, Torino, Italia, 2024.
Qiuyu Liang, Weihua Wang, Feilong Bao, and Guanglai Gao. Fully hyperbolic rotation for knowledge graph embedding. arXiv preprint arXiv:2411.03622, 2024.
Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic graph neural networks. In NeurIPS, pp. 8230-8241, 2019.
Aaron Lou, Nina Miolane, Tien-Minh Le, Xavier Pennee, Susan Holmes, and Quoc Gu. Differentiating through the fréchet mean. Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pp. 6393 6403, 2020.
Gal Mishne, Zhengchao Wan, Yusu Wang, and Sheng Yang. The numerical stability of hyperbolic representation learning. In International Conference on Machine Learning, pp. 14925-24944 PMLR, 2023.
Xuan Son Nguyen, Shuo Yang, and Aymeric Histace. Neural networks on symmetric spaces of noncompact type. In The Thirteenth International Conference on Learning Representations, 2025.
Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. In
Advonces Neural Information Processing Systems, volume 30, 2017.
12

---

Published as a conference paper at ICLR 2026
Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geom- ctry. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of PMLR, 2018.
Machine Learning Research, pp.
Avik Pal. Max van Spengler, Guido Maria D' Amely di Melendugno, Alessandro Flaborea, Fabio Galasso, and Pascal Mettes. Compositional entailment learning for hyperbolic vision-language models. The Thirteenth International Conference on Learning Representations, 2025.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, begh-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPSS, 2019.
Eric Qu and Dongmian Zou. Lorentzian fully hyperbolic generative adversarial network. arXiv preprint
arXiv:2201. /2825 2022.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. In NeurlPS, volume 35, pp. 15501-14515 2022.
John G. Ratcliffe. Foundations of Hyperbolic Manifolds, volume 149 of Graduate Texts in Mathematics. Springer, New York, edition, 2006.
Xianglong Shi, Silin Cheng. Sirui Zhao, Yunhan Jiang, Enhong Chen, Yang Liu, and Sebastien Ourselin. Lihe: Linguistic instance-split apperbolic-euclideee framework for generalized weakly- referring expression comprehension. arXiv preprint 1xXi:2511..2000 2025.
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks+++. arXiv preprint
Stti:220060022110 2020.
Alexandru Tifrea, Gary Béeigneul, and Octavian Oetavian-Eugen Ganca. Poincaré glove: Hyperbolic word embeddings. arXiv preprint arXiv: 1810.0660,,,6 2019.
Abraham A. Ungar. Analytic Hyperbolic Geometry and Albert Einstein's 's Special Theory of Relativity. World Scientific, 2008.
Abraham Albert Ungar. Analytic Hyperbolic Geometry in N Dimensions: An Introduction. CRC Press, Boca Raton, 2014.
Abraham Albert Ungar. Analytic Hyperbolic Geometry and Albert Einstein's 's Special Theory of Relativity (Second Edition). World Scientific, 2022.
Max Van Spengler, Erwin Berkhout, and Pascal Mettes. Poincare resnet. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5419-5428. 2023.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint 17 10903, 2017.
Rui Wang. Yuting Jiang, Xiaoqing Luo, Xiao-Jun Wu, Nicu Sebe, and Ziheng Chen. Wasserstein-aligned hyperbolic multi-view clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026.
Simon Welz, Lucie Flek, and Akbar Karimi. Multi-hop reasoning for question answering with hyperbolic representations. Xiviv preprint ar arxi::2007.03611, 2025.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty. Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In ICML, pp. 6861-6871. PMLR, 2019.
Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. Nodeformer: A scalable graph structure learning transformer for node classification. In NewrlPS, volume 35, pp. 27387-27401, 2022.
Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie. Haitian Jiang, Yatao Bian, and Junchi Yan. Simplifying and empowering transformers for harge-graph representations. In NeurlPS, volume 36, 2023.
Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and lon Stoica. Representing long-range context for graph neural networks with global attention. In NewrlPS, volume 34, pp. 13266-13279. 2021.
Jiaqi Yang. Wenting Chen, Xiaohan Xing, Sean He, Xiaoling Luo, Xinheng Lyu, Linlin Shen, and Guoping Qiu. Hysurvpred: Multimodal hyperbolic embedding with angle-aware hierarchical contrastive learning and uncertainty constraints for survival prediction. arXiv preprint arXiv:2503. 12.5500...866. 13862, 2025.
13

---

Published as a conference paper at ICLR 2026
Menglin Yang. Min Zhou, Zhihao Li, Jiahong Liu, Lujia Pan, Hui Xiong, and Irwin King. Hyperbolic graph neural networks: A review methods and applications. arXiv preprint arXiv:2202.13852, 2022.
Menglin Yang. Aosong Feng, Bo Xiong, Jihong Liu, Irwin King. and Rex Ying. Hyperbolic fine-tuning for large language models. arXiv preprint arXiv:2410.04010 2024a.
Menglin Yang. Harshit Verma, Delvin Ce Zhang, Jiahong Liu, Irwin King, and Rex Ying. Hypformer: Exploring efficient transformer fully in hyperbolic space. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.
2024b.
Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, Yan Liu. Do transformers really perform badly for graph representation? In NeurIPS, 2021.
Yuansheng Zhou and Tatyana Sharpee. Hyperbolic geometry of gene expression. iScience, 24(3):102225, 2021. doi: 10.1001/jisci.2211...
Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, Han Liu. Dnabert-2: Efficient foundation model and benehmark for multi spetiis genome, arXiv preprint arXiv:2306. 15006, 2023.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Use of large language models
16
Operations in the Lorentz model
16
Hyperbolic gyrovector structures
17
C.I Gyrogroups
17
C.2 From gyrogroups gyrovector spaces
17
C.3 Hyperbolic gyrovector spaces
17
Discussion about Intrinsic
18
Proofs
19
E.1 Proof of Theorem
19
E.2 Proof of Theorem
20
F
Implementation Details
21
E.I Datasets
21
Settings
22
More Ablation Studies
22
15

---

Published as a conference paper at ICLR 2026
A USE OF LARGE LANGUAGE MODELS
We large language models aid polish writing.
B OPERATIONS IN THE LORENTZ MODEL
Setup and notation a negative curvature < 0. Let (t-) denote the Minkowski bilinear form with signature +-....... +) on R*+1, and write || ||c = +) e for the induced (Riemannian) tangent
vectors. n--dimensional hyperbolic space the Lorentz (hyperboloid) model is
LK := =xXER"th (x, x) R. 20 }}
and we use 0 (vーx the pole ("origin"). When no confusion can arise, || denotes the Fuclidean norm in a tangent space.
Distance
Fo E Lx, the geodesic distance inherited from Minkowski space is 1
Dz(x,y) = V-K cosh -1 *(K (x)
A useful identity for computations is the "Lorentzian chord" expression (squared distance)
(x.)) ||x ylli 2 K
(x,
Specializing the pole
Dz 0) = logo (x)]1:
de(x: K
2 2xx
2x0
K
V-K
and, equivalently.
Dc(x,0)=
cosh (v- K zo))
V-K
Tangent space
The tangent space x E Li the Linkowski-orthogona complement x.
T&L {v e R
(v,x) x) 0}.
Restricted to T&L the metric is positive definite and coincides with the Riemannian metric of L
Exponential and logarithmic maps For E TxLK.
expx (z) cosh(a) x + sinh(a) a V-K ||2||c.
The inverse map log L Tx sends y L
cosh ((3) (y 3x).
log K (y)
B K (x,y)c.
82
At the pole these simplify
exp K (z)
cooh((- -K [|xll). sinh(V K [|z|]]
V-K
[iz]|
0,
y y= ō.
log (y)= = cushi ((3)
3: K (0x))
(y - -0),
otherwise,
V
with the convention a/||z|| when
Parallel transport Transporting E TxLK along the geodesic from x to y yields
P KT (v) --- (log (y), v)2 (log (y) + logy (x)) = v dc(x,y)
(y, v)
(x + yy-
-K (x. y)
16

---

Published as a conference paper at ICLR 2026
Lorentzian centroid and average pooling Given points ; Xm LK with nonnegative weights V, (not zero), the weighted Fréchet mean with respect to the squared Lorentzian distance, minpery - de (x,, (4), is obtained in closed form by
> Mi
M V
E 1 uxx:le
In neural architectures, an "average pooling" over hyperbolic receptive field can be implemented by taking this Lorentzian centroid of the features in the field.
C HYPERBOLIC GYROVECTOR STRUCTURES
C.I GYROGROUPS
We recall the algebraic notion of a gyrogroup, which extends the concept of group settings where associativity relaxed and corrected by gyrations (Ungar, 2008; 2014).
Definition 1 (Gyrogroup). Let G be nonempty set endowed with binary operation D: G x G - G. The pair (G, is gyrogroup if. for all a, b, E G. the following axioms hold:
(GI) There exists an element E G such that e a (left identity).
(G2) For each a there exists Go. E G such that Ga a = (left inverse).
(G3) There exists map gyr (a, b): G + G (the gyration generated by a, b) such that
c c) = (a @ b) @ gyrla, (left gyroassociatve law).
(G4) gyrla, b] gvrle b] (left reduction law). in addition
+ b gyr(a,
Vo,be
then (G, @) is called gyrocommutative.
Gyrogroups generalize groups: when all gyrations are the identity map. (G, 1) reduces to a usual group. The lack of strict associativity is compensated by the gyration operators, which encode curvature induced nonlincarity.
C.2 FROM GYROGROUPS TO GYROVECTOR SPACES
To model both addition and scalar multiplication in curved geometries, gyrogroups be enriched gyrovector spaces (Ungar, 2008; Chen al., 2025c).
Definition (Gyrovector space). Let (G, ()) be gyrocommutative gyrogroup and let O: RR x G - G be map called scalar multiplication. The triple (G, @,, () is gyrovector space if. for all a, b, and s,
(VI) 1 =a
(V2)
(V3) (st) (
(V4) byrla,b(( =0 L gyrla, b)(c).
(V5) gyr(s a @ a] is identity map on G.
These axioms mirror the familiar properties of vector spaces, with gyrations accounting for the deviation from linearity. In particular, (V2) and (V3) play the role of distributivity and associativity for scalar multiplication, while (V4) (V5) guarantee consistent interaction between gyrations and scaling.
C.3 HYPERBOLIC GYROVECTOR SPACES
The gyro-structure over the hyperbolic space can he defined by its Riemannian operators (Ganea et al., 2018: Chen al., 2025c). Given hyperbolic model H& along with v, H& and t R, the gyroaddition and gyromultiplication defined
@n Exp, (PT.4z (Loge v)),
t 1o Exp, Log. z), gyr(2, v|z Sw (z @x v) ew (x oH (y @ z)),
(12)
where denotes the origin in H
17

---

Published as a conference paper at ICLR 2026
On the Poincaré ball Pk. such gyro-structure is known the Möbius gyrovector space (Ungar. 2022, Ch. 6.14):
x GM (1 2K (a, v) x x K]|==|²)
2K (7, + K² |=/|||1
tanh ((tanh-'(( (v
OM
V[k
where Omr -1 DM -r is the gyroinverse and O is the gyro identity: 0x Va E PRR As shown by Chen et al. (2025c, Props. 24-25), the Lorentz gyroaddition and gyromultiplication also admit closed-fore expressions:
0,
y
D-KN KN | DAKN
(13)
Otherwise.
A. +Ag
0,
cosh (t0) sinh (t0)
$ GL x
(14)
Otherwise,
VIKi
2x
Here, cosh -1 (V[
x4),
A, = ab²
2K b8bsr
Kany A, b(a² Kre)) with the following
notation:
1 Vi b= 1+ VKKlu
M2 |,= n, I|u= 8ay (o),
(15)
D = a² d²t² 2K 2Kabszy + K² KRnnney
N = ann, + 2abazy + bz
In particular, the gyro identity is o and the gyroinverse Dr.rr -1 1r. [rr, -x.]
D DISCUSSION ABOUT INTRINSIC
In our paper, "intrinsic Lorentz" refers to using only operations that are well-defined on the Lorentz model itself, rather than on its ambient Minkowski space. We define layer (F' LK + L intrinsic if
input, output, and intermediate states lie on some IL (they always satisfy (z, - xu 0). 2. it is expressed entirely in terms of the Lorentzian geometry: (-, (--)). L. the induced distance du, and operators derived from them (exp/log maps, parallel transport, gyroaddition/gyrosett without ever using arbitrary Euclidean linear maps Lorentz vectors in the ambient space.
Lorentzian centroids, etc.),
Under this definition, the previous Lorentz fully connected layer (LFC) used in HCNN is not intrinsic. Its update has the form
= [VooWWx, v) 1/K @(Wx, v)]
where x E CR. Wx is standard matrix-vector product in I2" and the operation
We(x) + We(() bl
@ Wa, >a +b)
(16)
This W' is defined using the ambient linear structure of the Minkowski space, not any operation on the Lorentz manifold itsclf. Because the core transformation is an ambient Minkowski multiplication rather than Lorentzian/geodesic operation, this layer is only partially intrinsic.
By contrast, PLFC constructed entirely from Lorentz-geometry primitives that have closed-form definitions on L2. Each logit the signed Lorentzian distance from the input point learned Lorentz hyperplane, and the output y is then recovered in closed form as the unique point on LR whose signed distances to a set of coordinate hyperplanes equal these logits. All steps (hyperplane parameterization, point-to-hye to-hyperpaane distance, reconstruction of y) expressed only via the Lorentzian inner product and distance; ambient Euclidean affine map Wx b is ever applied. In this sense, PIFFC is intrinsic Lorentz FC layer.
18

---

Published as a conference paper at ICLR 2026
E PROOFS
E. PROOF OF THEORRM
Proof. We work in the hyperboloid model LR = {æ= = [x,] E 1m
x) = 1/K, 31. > 0} with
K < 0 and Minkowski bilinear form {[T,, [y,, y-l) = -xty + (a., y.). Let = [(-K) 1/22 .0] he
basepoint. Denote by e (k) R the k-th Euclidean basis vector in the spatial block and e [0. e (1))
Lorentz coordinate hyperplanes. The k-th spatial axis is the geodesic through 0 in the direction e (k) The Lorentz coordinate hyperplane through O orthogonal this axis, given
Definition (Lorentz hyperplane containing O and orthogonal the k-th axis)
H H20h) = =1x [et
LR
(e( x) = (eel)) Ian =0]. x.)
(17)
This the special case the Lorentz hyperplane He,a Eq. 4 with a = and z (x)) (hence || 1), for which cosh(v-Ko = and sinh(v = 0, giving (z, xx.) 2., O.
With Definition 3, the preparation for constructing y in Eq. is complete.
Derivation of y. Let x L and y 1yt, v.]" E IL be the input and output of the PLFC layer, respectively, As Eq. 8a, for 1, .,..., m we define the scores (x)) = Usgingg (a) via the Lorentz MLR logits in Eq. 6.
To endow y with the desired property that the signed hyperbolic distance from y to the k-th coordinate hyperplane equals (xx)) we impose the simultancous system
dt(yy MT ,u) U/ (x)),
h == 1,...,,
(18)
Using Eqs. and 5 with a hhint-oo-hyerpphe distance specializes to
z (k) (so that the denominator equals 1), unsigned
dely, i ())
1 |sinh (V-K yo
V-K
Orienting H. by the unit normal (N) and recalling sign convention used Eq. 6. the signed distance therefore
de H. ( .) V sinh (V - K Be,k))
(19)
Substituting Eq. into Eq. 18 yields, for each ki
V sinh (V-K y..)) vx(x))
(20)
and hence
yo,k sints Mullv--Kin (x(x)). = L....,m,
(21)
which is exactly Eq. 8b. Collecting these coordinates gives y. = (Ja.1...., m) Finally, the time coordinate y: is fixed by the hyperboloid constraint (y, y) 1/ K, i.c.
- vE 1
y1 = V (-K--1 + 1ly. ll..
(22)
with the positive root chosen remain the top sheet, which is Eq. 8c. Thus Eq. 8 follows.
Confirmation of the existence of y. For any real scores en (x).. Eq. 21 yiclds real ya.k because sinh is entire. Since K > 0, have (-K)-1- l|., l 0, so ys in Eq. 22 real and strictly positive. Therefore y = [y+, y.] E L always exists and lies on the correct sheet. Moreover, Eq. 20 guarantees that the signed distance from y each coordinate hyperplane HK is exactly (x)),, required.
Flat-space limit. As + 0 we have sinh (V-K = V O(K3/2), hence Uo,A - vx (æ) from Eq. 21. Using Eq. and expansion cosh V-Ka) 1 + O(K), sinh (V-Ka) V-K + 3(x3/2). one obtains vn (a: - (z) u.) as. so the spatial part reduces to the Euclidean affine map with row vectors As 2 and bias by ak, as stated below Theorem
19

---

Published as a conference paper at ICLR 2026
E.2 PROOF OF THEOREM 2
Theorem 2 (Margin preservation and contraction of PLFC and LFC). Fix a curvature K O. For any input let the penultimate layer produce
u(x) = (ur( um (m)) E ER",
and let denote the true class. Define the pre-logit margin
(x)) ue(a)
max ui (I)). s
Consider two Lorentz output layer designs that signed geodexic distances from the output point y(x) to the coordinate hyperplanes logits:
PI.FC head (intrinsic). The spatial coordinates are
PLPC
(x)
sinh (vinh(--Ks un(x)).
V-K
and the signed Loventzian distance the k-th coordinate hyperplane is
PLFC
(x)
asinh ( (v-K Ye.k
PLFC (x)).
V-K
LFC head (extrinsic/ linear). The spatial coordinates are taken directly as
L.F Ns,k (a)) us (x),
LFC.
and the signed Lorentzian distance to the same hyperplane is
(z)
asinh (V-K ux(x))
Define the distance-hased margins
A PLPC (x) ( == de LLFF
max dj j
PLPC (x),
A LPC (x) := de (2F)(())
max LFC
(x)).
jfc
Then, for every sample x: (Margin preservation of PLFC)
A PLLC (x) = A(x).
(Margin contraction of LFC) The LFC head preserves the sign of the margin and contracts its magnitude:
sign (A (x)) = sigu (A(x)),
( s A())),
with strict inequality A LFC (x) A(x) whenever A(x) O.
Proof. Define
A(t)
- asinh(v
<< 0.
By the point to hyperplane distance formula the Lorentz model, for any Lorentz point (yo, ys) the signed distance to the k-th coordinate hyperplane is exactly h(y,,)).
(1) PLFC preserves the margin. For the PLFC head we have
Ba.k
(x)
1
sinh (V (x)):
and thus
PLPC de
(ar) hy
PLFC (x))
asinh R V-K sinh (annhv-Kuaa ux(z)))
asinh (inn
sintiv-- (V-K ux ux)
V-K
V-K
Since sinh: : IR is bijection with inverse asinh, we have asinh(sinh = z for all z E R, hence
dk (I) = (x))
PLFC
Consequently,
A PLFC de PLFC (x)
PLPC (x) ue(x) max u, maxu, (x(x) A(x),
max d
20

---

Published as a conference paper at ICLR 2026
which proves (I).
1.FC ( (x)),, hence (2) LFC contracts the margin. For the LFC head we have ya.k
dx ( h(xxx)))
We first record two basic properties of h. Differentiating.
d dt \ asinh (v-Kt) V-R
VR
(t)
VT+
so (t)
and (t) S 1 with (f) and only if f Therefore, is strictly increasing and
1 1-Lipschitz.
Let j" be any index of a maximizer of the competing pre-logits:
j
arg max (z))
Since is strictly increasing, same index maximizes the distance-based logits:
max LFC
(a)) max (z)) = h(uy- (z))-
J
Hence
((x) un(x)) uy u.. (z),
A L (z) = h(w.(x)) h (z)).
If (()) then (rr(3) My* (a:) and consequently now (22) and, without loss generality, set
A LFC (x) 0, so the conclusion holds trivially. Suppose
u.( == uy- (x),
* b.
Assume b; the b is analogous by symmetry. Using the fundamental thcorem of calculus,
h(a) h(b) = J. dt)
Because h (+) > 0 for all we have h(a) h(b) > 0, sign(h(a) h(b)) sign(a 6). Moreover, since ('(t) 1 everywhere and (t()) for all the integrand strictly less than subset of [b, a] with posilive measure whenever * b. Thus
<< h(a) l(6)) 1." (1()) f("
1di b.
Taking absolute values yields
Ih(a) h(b)) < la--10 -/.-
Substituting back ue(x) and b uy (z), obtain
sign (A LFC (A (x)) = sign (A(x)),
AA LPC
(x) = [h(a) h(b) < la - - ->()))
Combining with the (xx) = case, this gives
LAFF (x) |A()))
with strict incquality whenever A ((T) o which proves (2).
F IMPLEMENTATION DETAILS
F.I DATASETS
these datasets exhibit hierarchical class relations and high hyperbolicity (low 8ret) as shown in Table 5, making the use of hyperbolic models well-motivated.
Image Datasets For image classification, we adopt the standard benchmarks CIFAR-10 and CIFAR- 100(Krzheessky et al., 2009). CIFAR-10 and CIFAR-100 each contain 60,000 32 x 32 color images drawn from 10 and 100 classes, respectively. Following the PyTorch setup and (Bdeir al., 2024), we 50,000 images for training and 10,000 testing. CIFAR-10 10 and CIFAR- 100 are standard proxies for visual object recognition whose categories naturally admit semantic hierarchies (e.g., animal - mammal + dog + specific breed). In CIFAR- 100, this is made explicit by grouping the 100 fine- grained classes into 20 coarse superclasses in the original dataset design. This hierarchical structure has been extensively verified in prior hyperbolic vision work Bdeir al. (2024); Nguyen al. (2025) CTEAR-10W 10/100
21

---

Published as a conference paper at ICLR 2026
Gene Datasets For genomic sequence classification, evaluate the Transposable Elements Benchmark (TEB) (Khan al., 2025) and the Genome Understanding Evaluation (GUE) (Zhou et al., 2023) suite. TEB a curated, multi-species collection of binary classification tasks spanning seven vransposale-clemen across retrotransposons. DNA transposons, and pseudogenes; we follow the authors' relcased preprocessing and data partitions. GUE aggregates 28 datasets covering seven biologically meaningful tasks-incluuding transcription-facton binding, epigenetic mark prediction, promoter and splice-site detection-with sequences ranging roughly from 70 to 1000 base pairs originating from yeast, mouse, human, and viral genomes. Unless otherwise noted, we adopt the official hiti/iaaliatioottestt splits and report the Matthews correlation coefficient (MCC) as our primary metric. The TEB and GUE suites are constructed directly from natural genomic sequences and inherit the biological hierarchies of their domains. TEB tasks span multiple transposable-elemee across retrotransposons, DNA transposons, and pseudogenes, which themselves in multi-level taxonomic hicrarchy (orders + superfamilies + families). GUE aggregates datasets for transcription-fae factor binding, promoter and core-promoter detection, splice-site prediction, and COVID-varian classification across several species. These tasks are all manifestations of hierarchical regulatory structure (e.g., motifs - modules - promoters gene expression).
familics
families
Table 5: Hyperbolicity values of the datasets used in experiments ( (6ie))-
Benchmark
Task
Dataset
5rel
CIFAR-10
0.26
CIFAR
Image classification
CIFAR-100
0.23
TEB
Pseudogenes
processed 0.19
Pseudogenes
unprocessed
0.16
Covid variant classification
covid
0.42
all 0.23
GUE
Core Promoter detection
notata
0.21
tata
0.14
all 0.26
Promoter detection
notata 0.26
tata 0.14
F.2 SETTINGS
Table 6 summarizes the hyperparameters used to train the model. We additionally note a dataset-specific choice regarding normalization statistics. On CIFAR-I0/IC we enable running statistics: we maintain per-channel running statistics by updating the Lorentzian centroid (mean) and dispersion (variance) with momentum term and, at test time, substitute these running estimates for the batch statistics. contrast, on the TEB GUE genomic suites, we disable running statistics entirely, because enabling them consistently led to a collapse of MCC after a few dozen epochs. For these genomic datasets, we therefore compute statistics on-the-fly fly from cach evaluation batch (i.c., no moving averages used at test time). Compared with natural images, the genomic tasks exhibit stronger distributional non-stationarity (heterogeneous sequence lengths and tasks) and higher batch-to-batch variability. Under these conditions, momentum- lag behind the true data distribution; in a Lorentzian normalization layer, a biased centroid and underestimated dispersion can over- or normalze timelike fcatures, shrinking margins and destabilizing optimization.
running estimates accumulate bias and
G MORE ABLATION STUDIES
Table 7 above expands the ablation to the number of Fréchet mean iterations used by each normalizer while keeping all other components fixed. Across all GUE datasets and both tasks, WTFC+Gyroll BNN achieves the hest accuracy under every iteration budget, and the relative ordering methods is unchanged as the budget increases. Moving from to 2 and iterations yields modest but consistent gains, whereas 10 steps and the fixed-point solution (cells with gray background, denoted by oc) offer only marginal improvements at higher computational cost. These trends indicate that the advantage of GyroLBN stems from the normalization rule itself rather than from merely computing tighter Fréchet mean. In practice, allocating small budget two five iterations recovers nearly all of the attainable accuracy while preserving efficiency, making PLFC+GyroLBN the most offective and economical choice in our setting.
22

---

Published as a conference paper at ICLR 2026
Table 6: Summary of hyperparameters used in different datasets.
Hyperparameter
CIFAR-10& 100
TEB
GUE
Epochs
200
150
150
Batch size
128
256
256
Learning rate (LR)
le-1
8e-4
9e-4
Drop LR epochs
60, 120, 160
100,130 130
100,130
Drop LR gamma
0.2
0.1
0.1
Weight decay
5e-4
6e-3
5e-3
Optimizer
(Riemannian/SGiD
(Riemannian)Adam
(Riemannian)Adaa
Floating point precision
32 bit
32 bit
32 bit
GPU type
RTX A100
RTX A100
RTX A100
Num. GPUs
Hyperbolic curvature K
-1
-1
Dropout rate
0.05
0.05
0.05
Table 7: Ablation study on the number Fréchet mean iterations. The symbol oo indicates that iterations performed until convergence, which is the setting used in main ablation experiments.
Model
Benchmark
Task
Dataset Iteration
LFC LBN
LFC
PLFC GyroBN
PLFCC
GyroLBN
GyroLBN
1
80.09
2
81.38
lata
5
78.26.2.8
81.33
79.47a221
83.90s0.3
10
81.27at.75
O
80.89
1
71.9920.6
71,0620.49
Core Promoter Detection
notata
5
66.60.1.07
71.92ea
7 1.,65.0.7
42.59.0.9
10
71.12+1.7
Pc
72.22-1.44
1
70.47
2
70 70.42.0.99
all
5
66.47.074
69.74+1.a
70. 70.75.0.45
70.89
10
69.81 100.77
70. 14.0.5
GUE
81.71
2
80.19
78.58+3.99
80.46+0.99
81.69+3.00
83.26x1.90
10
79. 79.79
16.19
1
92.15a0.0
2
91.89.0.7
Promoter Detection
notata
5
1
91.88
92. 19..9...4
92.48
10
92.07
8
91.67.0.56
1
90.45s0.72
2
91.01=0.9
5
38.0040.9
90.28
90.80se.33
91.3440.38
10
18.08.
02.20.66
23


Reviews:
[{"review_id": "gX9LZx4nHk", "reviewer": "Reviewer_qw2r", "summary": "This work proposes to learn representation and classification with all computations in the Lorentz model for hierarchical and long-tail data using point-to-hyperplane layers and intrinsic normalization, tested on image and genomic datasets.", "questions": "In the datasets mentioned, where exactly do you observe the latent hierarchical structure and long-tail distribution? Could you explain why hyperbolic methods are effective for addressing latent hierarchical structures and long-tail distributions? Additionally, what specifically constitutes the hierarchical structure in this context?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper's main contribution proposes a new point-to-hyperplane fully connected layer (PLFC) that utilizes intrinsic Lorentzian distance for logits to match the data's latent hierarchy. However, this approach is not originally developed by the authors and lacks novelty. Furthermore, the mechanism by which this method achieves matching with the data's latent hierarchy remains unclear and insufficiently explained.", "weaknesses": "(1) The paper claims to focus on real-world data with latent hierarchical structure and long-tail distribution. However, I did not find clear evidence that CIFAR-10, CIFAR-100, or the TEB and GUE genomics benchmarks possess such properties.\n\n(2) The PLFC method referenced by the authors as originating from HNN++ had been previously proposed in works on fully hyperbolic neural networks and the hypformer paper (\"Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space\"), yet the authors do not cite or compare with these relevant methods.\n\n(3) Most other modules in the paper appear to be adopted from existing literature; I do not observe any genuine innovation.\n\n(4) It remains unclear to me how the proposed method specifically addresses latent hierarchical structure and long-tail distribution in the data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UvzttlmeuB", "reviewer": "Reviewer_9oRv", "summary": "This paper proposes Intrinsic Lorentz Neural Network which is a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. The paper introduces several intrinsic modules including a novel point-to-hyperplane fully connected layer and GyroLBN. Several other components including a gyroadditive bias for the FC output, a Lorentz patch-concatenation operator and a Lorentz dropout layer are also introduced. Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-art performance.", "questions": "Besides the weaknesses, I have the following additional questions,\n\n1. The authors use CIFAR-10 and CIFAR-100 in the experiments to demonstrate the benefits of the intrinsic approach. However, it is unclear what kind of hierarchical structure these two datasets possess. Therefore, it is difficult to conclude that the intrinsic approach preserves the geometry of the Lorentz model without distortion. Could the authors consider using datasets with explicit hierarchical structures to better support their claim?\n\n2. The authors mentioned fully connected layer and convolutional layer, I am wondering if the same intrinsic approach can be applied to build other kind of neural network layers such as attention layers？", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed Intrinsic Lorentz Neural Network is well motivated as existing hyperbolic architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations.\n\n2. The proposed point-to-hyperplane Lorentz fully connected layer is new in the literature of hyperbolic neural network which replaces traditional affine transformations with intrinsic hyperbolic distance.\n\n3. The paper is also generally well-written and extensive experimental results show the improvements of the proposed Intrinsic Lorentz Neural Network.", "weaknesses": "1. The experiments are only conducted on CIFAR-10/100  and two genomic benchmarks, it would be interesting to show if the proposed Intrinsic Lorentz Neural Network can outperform existing hyperbolic neural networks on large-scale datasets such as ImageNet and datasets with long-tail class distributions. \n\n2. The authors argue that intrinsic designs are preferable compared with partially intrinsic or extrinsic parameterizations, but there is a lack of enough evidence showing that the claim is true. There are quantitative results but the performance of ILNN is very similar to that of Hybrid Lorentz. Therefore, it is not clear if such intrinsic designs can bring significant benefits. \n\n3. In table 3, the authors analyze training efficiency to disentangle the effects of the classifier design and the normalization strategy, but there is no comparison of training time of the proposed Intrinsic Lorentz Neural Network with other hyperbolic neural networks mentioned in table 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vl60Q6mbbn", "reviewer": "Reviewer_98q5", "summary": "This paper proposes the intrinsic Lorentz neural network (ILNN), which is claimed to fully operate within the Lorentz model. The main architecture is a point-to-hyperplane Lorentz fully connected layer, along with which the paper introduces Lorentz versions of batch normalization (GyroLBN) and dropout, etc. Experiments on CIFAR-10/100, TEB, and GUE datasets show that ILNN achieves competitive results. Ablation studies are also reported to confirm the contributions of PLFC and GryoLBN.", "questions": "1. How is “intrinsic” formally defined? How are existing Lorentz-model methods not intrinsic? Why is “intrinsic Lorentz” beneficial? These have to be defined carefully and mathematically.\n2. The result of Theorem 1 is of course interesting. But can you formulate a theorem to show the benefit of your specific design, or benefit of using an intrinsic Lorentz design?\n3. Can you introduce some stronger baselines for TEB and GUE? \n4. Can you test your model on some better-known datasets where hyperbolic space has proved effective?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper addresses an important topic of building neural architecture that respects hyperbolic geometry while avoiding the inconsistent mixture of Euclidean and hyperbolic components.\n2. The experiments on vision and genomics tasks show potential interdisciplinary applications of hyperbolic neural networks.\n3. On TEB and GUE datasets, ILNN achieves significant improvement on several tasks.\n4. The paper is clearly structured, with detailed mathematical formulas and reasonable ablation studies.", "weaknesses": "1. The paper claims that its method is “Intrinsic Lorentz”, yet provides no rigorous definition of what that means. If “intrinsic” means depending only on manifold operations, then computing directly in Minkowski coordinates is still extrinsic.\n2. On CIFAR-10/100, the gain over Euclidean model is very limited (< 0.5%), and most of the selected baseline models are not even as good as the standard Euclidean model…\n3. On TEB and GUE, there should be more baseline methods (some from what is reviewed on page 2) than only Euclidean CNN and HCNN. Maybe other tasks of TEB and GUE can be included as well. Actually, other than using CIFAR, TEB and GUE, why not use some datasets that have been widely used and clearly have hierarchical structures?\n4. The use of the “gyro-” terminology is conceptually problematic. The review of gyrovector spaces on page 3 is inaccurate: equations (1) and (2) describe standard Riemannian composition rules using exponential and logarithmic maps, not Ungar’s gyrovector algebra. The Lorentz model does not form a gyrovector space in the algebraic sense, unlike the Poincaré ball.\n5. Theorem 1 mixes a definition (the formulation of the PLFC layer) and a theorem statement. These should be clearly separated for readability and mathematical rigor.\n6. Some reference items appear multiple times. For instance, the paper by Bdeir appeared three times, but they are actually the same one. This is not the only entry. Please carefully check it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YOaZQhho2Y", "reviewer": "Reviewer_UjqT", "summary": "This paper introduces the Intrinsic Lorentz Neural Network (ILNN), which is a hyperbolic NN architecture that keeps all computations inside the Lorentz space instead of mixing Euclidean operations with manifold operations. They propose point-to-hyperplane Lorentz fully connected (PLFC) that computes the space-dimension of the output as signed distance to hyperplanes, and GyroLBN that re-centers and scales the output with efficient statistics. They show improved in CIFAR datasets and genomics datasets, with ablations that show effectiveness of PLFC and GyroLBN against LBN, GyroBN, and LFC.", "questions": "1. Does the improvement brought by intrinsic Lorentz operations sustain beyond genomic data, since the improvement on the CIFAR datasets is marginal?\n2. Could the authors clarify on discrepancy between their definition of LFC and the corresponding papers (as in weakness 1)?\n3. One property of LFC is that it encompasses all Lorentz transformations. Can similar properties be proved for PLFC?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper demonstrates substantial improvements over HCNN and Euclidean CNN in genomics tasks\n2. The visualization in Figure 2/3 shows better separated clusters than HCNN\n3. Using intrinsic hyperbolic operations instead of relying implicitly on Euclidean operations is an interesting direction that could result in more principled model designs", "weaknesses": "1. The authors claimed that LFC \"applies Euclidean mappings to spatial components while treating the time-like coordinate separately\" on line 58, which isn't correct. Fully hyperbolic linear layers proposed in prior works (https://arxiv.org/abs/2303.15919, https://arxiv.org/abs/2105.14686, https://arxiv.org/abs/2407.01290) operates on the entire Lorentz vector and the compute the time dimension. This brings into question of whether the authors correctly implemented the baselines for their subsequent experiments. \n2. The improvement in vision tasks is incredibly marginal in both the main results and the ablations\n3. Because the improvement in CIFAR datasets are marginal, the performance of the proposed method is essentially entirely dependent on the genomic dataset. \n4. Some of the citations are repeated, which causes confusion", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)