Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability
Authors: Ambarish Moharil
Venue: ICLR 2026 Poster (2026)
Keywords: Graph Meta Networks
Temporal Hyperbolic Embeddings
Neural Weights as Data
Primary area: learning on graphs and other geometries & topologies
TL;DR: We encode MLP training traces as parameter graphs embedded in the Poincaré ball, processed by hyperbolic attention with recurrent kernel evolution. Outputs predict links and signed weights, with Riemannian optimization refining temporal embeddings.

Abstract: This paper investigates how multilayer perceptrons (MLPs) can be represented in non-Euclidean spaces, with emphasis on the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting analysis to single checkpoints, we construct temporal parameter-graphs across $T$ snapshots of the optimization process. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based meta learning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter-graphs in the Poincaré ball and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, it preserves functional equivalence across time and recovers the network’s latent geometry. Experiments on regression and classification tasks with trained MLPs show that hyperbolic temporal representations expose how structure emerges during training, offering intrinsic explanations of self-organisation in a given model training environment.

Published as a conference paper at ICLR 2026
TEMPORAL GEOMETRY OF DEEP NETWORKS: HYPER- BOLIC REPRESENTATIONS OF TRAINING DYNAMICS FOR INTRINSIC EXPLAINABILITY
Ambarish Moharil
Data Governance Department
Jheronimus Academy of Data Science Eindhoven University of Technology
wwwoaril@et.. nl
ABSTRACT
Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an op- timization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our repre- sentation features the Poincaré model of hyperbolic geometry. We aim to cap- ture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints per established measure- based explainability methods construct temporal parameter graphs, i.e., snapshots over time T steps of the optimization/rraiing process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing the idea that many complex networks admit embeddings in hidden metric spaces where dis- tances correspond to connection likelihood, we present geometric and temporal graph-based metalcarning framework for obtaining dynamic hyperbolic represen- tations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while main- taining equivariance to stahin--naaphot neuron permutations and invariance to permutations of past snapshots. In doing the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments regression and classification tasks with trained MLPs show strong meta-netwon performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's 's self-organization.
INTRODUCTION
The central problem that we address in this work is the geometric evolution of neural networks, specifically how the weighted topology of deep neural network (DNN) model reorganizes during training and how this self-organization shapes the (neural network) function. We argue that studying the weighted topology is key to figure out the role of neural weights, that is the parameters used for any decision, are not just incidental outputs of some optimization process, but also the structures through which generalization, compression, and transfer learning arise (Han et al. 2016 Li et al.) 2020). Our focus this work not only to investigate the performance of hyperbolic and temporal neta-networks, but also to expose how neural network's 's structure forms and shifts during training. A temporal geometric view makes it possible to track the emergence of hierarchy, the consolidation of functional modules, and the redistribution of influence across layers by studying the evolution of its parameter graph. This perspective offers intrinsic explanations grounded in the model's 's own dynamics and seff-organization, rather than relying post-hoc hoc probes for explanation.
recent approach has been to treat neural networks as data objects themselves and to train meta- networky that take neural parameters inputs predict the generalization performance of the un-

---

Published as a conference paper at ICLR 2026
derlying model (accuracy, R², etc.), classifying the implicit neural representations (INRs) of the
underlying neural model or to generate weights for transfer learning across domains (Unterthinrr
et al.
2021 Schhürholt et al. 2021] Dupont et al. 2022, Navon et al. 2024 Zhou et al. 2023b You et al. 2020 Lim et al. 2023 Kofinas et al. 2024). Early works on the topic flattened weights and biases into single tensor, but this discarded structural relations and ignored the underlying geo- metric priors of the weight space, such as funcctinaa invariance and equivariance to permutations (Hecht-Nielsen 1990, Kofinas et al. 2024). Later work enforced data symmetries, such as permu- lation equivariance, ensuring that reordering the neurons did not affect the network functionality (Zhou et al.) 2023a bj Navon et al.] 2024; Hecht-Nielsen, [990).. These methods, although robust, often require archifecture-specffic tuning and Euclidean representations, making it challenging to interpret the network's is internal self-organization.
A complementary direction has leveraged graph-theoretic and network-science perspectives. Here, neurons and biases become network nodes, whereas the model weights are tied to the existence of an edge link between two nodes, with meta-learning being performed on these parameter graphs (You) ct al. 2020 Lim et al. 2023 Kofinas et al.] 2024). These methods have performed well for INR classification and ceneralization prediction tasks, but usually operate single checkpoints (along the training trajectory) and make zero-shot predictions, leaving the temporal traces of optimization underused (Lim et al. 2023 Kofinas et al.] (024).. On the other hand, few studies complex network graphs in physical domains (Simas & Rocha, [2014] Allard et al.] (017) have considered how geometric priors could inform edge weight dynamics; a link that can be crucial in making deep
learning interpretable.
Network science points two priors of particular relevance: small-world effects and modularity (Song et al. 2005; Kim et al. 20075; Wei et al. 2013 Fronczak et al.) 2024). They also often admit embeddings in metric spaces where the distances reflect the connection probabilities (Allard et al. 2017 Krioukov et al., 2010). If parameter graphs display similar tendencies, embedding them in metric spaces with distance-biased learning can yield compact, interpretable representations. Most existing methods use Euclidean embeddings (Lim et al.] |023) Kofinas et al. 2024). but these require high dimensions and distort structure when projected (Nickel et al., 20144; Nickel & Kicla (017)) On the other hand, hyperbolic spaces preserve hierarchical relations with low distortion (Gromov) 1987]. yet no prior work has embedded neural parameter graphs within hyperbolic spaces while preserving permutation symmetry. The closest example addresses static graphs of multilayer brain networks (Guillemaud et al. 2025).
We close this gap by developing a geometric temporal hyperbolic framework for parameter graphs that treats training itself as an object of study. Conceptually, we view the optimization of a neural network trajectory across weighted parameter graphs evolving in negatively curved space, and we argue that intrinsic explanations should be grounded in this evolution rather than in isolated checkpoints. Methodologically, we (a) construct a temporal hyperbolic meta network (GTH-GMN) that (b) models MLP parameter traces as permutation equivariant parameter graphs, (c) embedding hyperbolic attention with meta evolution of the attention kernels over time (Pareja et al., 2019 Li et al. 2024). The architecture is designed to respect the symmetries of weight space: snapshot and invariant to permutations of past snapshots, so that functionally equivalent networks share the same representation, while their latent geometry can still be recovered. A signed weight regressor links geodesic distance to edge magnitude through power law prior (Allard et al. 2017), decoupling magnitude from polarity and importing scale free inductive biases from network sci- ence (Song al. 2005 Kim et al.) 20076 Wei et al. 2013 Fronczak al. 2024 Empirically, we show on INR classification, CIFAR-10 generalization prediction, and sinusoid regression, that this temporal hyperbolic encoder matches heavily approaches strong Euclidean and tensor-based baselines, while producing compact embeddings whose radial and angular structure tracks the self- organization of the underlying networks. In this way, the method offers a concrete route toward in-
them in the Poincaré ball, and (d) coupling distance-biand
equivariant to the neuron permutations within each
trinsic explainability through the geometry of parameter graph trajectories (Unterthiner et al. 2021
Schürholt et al. 202T Dupont et al., 2022 Navon et al. 2024 Zhou et al.
2023b) You et 2020
Lim et al.] 2023 Kofinas et al. 2024 Allard et al. 2017 Krioukov et al. 20100] Nickel et al
2014
Nickel & Kiela 2017
Gromov
1987 Guillemaud et al. 2025 Pareja et al. 2019 Li ct al. 2024).
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Neural Meta Networks. Neural networks can themselves be treated data. Early meta-network
approaches flattened parameters or relied on simple statistics, which ignored neuron permutation
symmetry and had limited cross-architectuuee generalization (Unterthiner et al., [202T] Eilertsen et al.]
2020 Dupont et al., 2022] De Luigi et al. 2023). Subsequent works developed permutation-
equivariant meta-networks that respect weight-space symmetries (Schürholt al.
2021 Navon
et al. 2024 Zhou et al.) 2023a b) Maron et al. 2019). Recently, Graph-based approaches have
extended this view by modeling parameters as graphs (Lim et al., [023] Kofinas al. (024). Graph
Meta Networks (GMN) construct parameter graphs to learn graph neural networks (GNNs) that remain equivariant to neural DAG automorphisms, covering CNNs, Transformers, and multilayer perceptrons (MLPs) (Lim et al. (023). These wraph-based approaches model individual neurons and biases as parameter graph nodes, where edge existence depends on the weight between adja- cent nodes, enabling GNNs or relational transformers to process architectures with varying depth, width, and skip connections (Lim et al., 2023; Kotinas et al. 2024). Other work has used higher- level graphs for NAS, federated learning, and parameter sharing (Liu et al. 2018 You et al. 2020
Litany et al.) 2022), though often at a coarser level of abstraction. GNN provides a foundation of
permutation-equivaaiann operators for nodes and edges (Hamilton 2020 Maron et al.. 2019 Kim jet al. 202T] Lim et al.] (023). Our contribution builds on this trajectory but addresses two gaps. First, most approaches rely on single checkpoints and do not exploit the temporal traces generated during optimization. Second, embeddings are almost always Euclidean, which limits their capac- ity to represent hierarchical or heavy-tailed structures. We therefore focus on temporal parameter graphs of MLPs, embedding them in hyperbolic space to capture both the evolving geometry and functional symmetries over the course of training.
Hyperbolic GNNs. Poincaré and Lorentz embeddings first established that hierarchical relations admit compact, faithful representations in negatively curved spaces (Nickel & Kiela 2017 2018) (
For background on hyperbolic geometry, refer to Appendix [B). Building on this insight, hyperbolic
neural models extended message passing, attention, and diffusion
-uclideid manifolds for
static graphs (Ganea et al., 2018 Chami et al., 2019: Zhang et al.) 2019 Wen et al. (024).. The
key challenge that hyperbolic spaces are not vector spaces, so Euclidean modules do not apply verbatim (Yang et al., [203)). Two strategies have emerged: design fully manifold counterparts for core layers (Chen et al. 2021). or lift computations to tangent spaces and map back while enforcing model-specific constraints (Ganea et al. 2018 Chami et al. 2019: Zhang et al.) 2021). Manifold optimization further supports these pipelines through Riemanninn adaptive methods (Sakai & liduka, 2020 In our framework, node embeddings are also updated intrinsically: alongside the Euclidean parameter pass for kernels and regressors, we run a dedicated Riemannian optimizer that updates positions 2() on the Poincaré ball in a geometry-consister second phase.
Temporal Learning in Euclidean and Hyperbolic Spaces. Early approaches to temporal graph learning largely relied on Euclidean neural architectures. Sequence-awe awwee graph models combined recurrent updates with graph convolutions. For example, the spatial operators in convolution- -base long-short term memory (LSTM) networks were replaced with Chebyshev filters so that temporal dynamics and topology could be processed jointly (Seo et al. 2016), (Shi et al. [015 Defferrard et al.) 2017). Later work introduced uncertainty modeling over time by placing latent distributions at cach step (Hajiaaezaaaai et al., 2019), and eventually allowed the graph encoder itself to evolve through recurrent updates, so the parameters of the convolutional layers changed along with the temporal signal (Pareja al.) 2019). However, Euclidean geometry can flatten or obscure the heavy-tailes and hierarchical patterns that naturally arise in many temporal networks (Krioukov et al. 2010: Nickel & Kiela| 2017). This motivated the development of temporal models built directly in hy- perbolic space, where message passing and recurrence are designed to respect the curvature. These methods combine hyperbolic graph layers, hyperbolic recurrent units, and temporal attention mech- anisms, along with geometric consistency losses that stabilize learning (Yang et al.) (023). Other approaches use hyperbolic diffusion operators, dilated causal convolutions, and curved decoders adapted to long- horizon forecasting (Bai et al., 2023). Hyperbolic attention variants compute simi- larity using geodesic distances and perform aggregation using Einstein gyromidpoints. Extensions that incorporate temporal context improve multi-step forecasting and link prediction et al. 2024 2023). J. Continuous time formulations push this further by combining hyperbolic variational encoders with temporal encodings for event stream modeling (Sun et al. 2021a). Across these developments, the unifying ideas are operators that preserve the manifold, mappings that respect curvature, and explicit mechanisms that maintain temporal coherence. Despite this progress, difficulties remain in
3

---

Published as a conference paper at ICLR 2026
scaling hyperbolic temporal models and in adapting curvature over long sequences (Zhang al. 2021 Yang et al. 2023). To address these limitations, we follow the kernel evolution strategy intro- duced in recurrent graph encoders (Pareja et al. 2019). which updates the parameters of the attention kernels directly over time. This avoids storing the entire sequence of hyperbolic embeddings, sig- nificantly reduces memory usage, and enables efficient learning on long optimization traces while maintaining temporal smoothness (Pareja et al. 2019) (related work in Appendix C).
3 RESEARCH METHOD
Figure [ provides an overview of our approach. Given a training trace of an MLP, we encode each checkpoint parameter graph where nodes represent neurons or biases and edges represent weights. The sequence of graphs is embedded in the Poincaré ball and processed with a hyperbolic graph attention network, while temporal evolution is handled through kernel meta-evolution that updates attention parameters recurrently across snapshots. The model outputs link probabilities and signed weight predictions, and node embeddings are refined by intrinsic Riemannian optmization. This yields compact temporal representations that capture both structural symmetries and geometric self-organization of the network over training.
8g44
Geuatondll Popitto Aitaron (i)
X
O O O
DEOO
cuse Amterton
m
fm
reghts Predoo
9" W. ② Su
AA W @ 2
Of (f
W. 2 2n
4
X TTY
tD
Bolan (sto
Malunn I
G
Ghe Tims
Figure Architecture of the Geometric Temporal Hyperbolic Graph Meta-Network (GTH-GMN).
3.1 TEMPORAL GRAPH CONSTRUCTION FROM MIP TRACES
We represent the evolving weight matrices of an MLP as a sequence of graph snapshots of param- eters, ensuring that the permutation symmetries are respected. At each epoch f, individual neurons and biases are represented as nodes (u Vi), while the parameters define the weighted edges (fu,e) E F, with signed label w (). Nodes with zero weights at epoch / do not register adjacency [Lim et al. 2023 Kofinas et al. 2024). For an MLP with L layers and widths
.n.....,,,,,, this yields a snapshot G2 = (X,, E., W.). Input and output layers are fixed so that reordering neurons within hidden layers results only in an equivariant graph structure (Lim al.] (023). The set of nodes includes inputs, hidden neurons, and explicit bias nodes (Lim et al.] [023] Kofinas et al, 2024). Each node feature (I..)) contains a layer label ((u) and type T(u). The neuron edges
connect successive layers, while the bias edges connect a bias node to its layer. Directed pairs are symmetrized by averaging across antiparallel directions, treating missing partners zero, so that
4

---

Published as a conference paper at ICLR 2026
the undirected edge set E, preserves signed magnitudes without spurious artifacts. Each node (z),t) is further endowed with structural signals that remain invariant under permutations. From the edges incident to u compute absolute and signed strength scores and then standardize them across the graph (Unterthiner et al. 2021). The resulting z-scores (5abe (u, 1), 3e8"(u. (u, 1)) quantify how strongly node is embedded in its neighhorhood. Together with f(u) and r(u), they yield the feature vector Fu. [(u), Tuu), gabe (u, t), 5055 (u,t), Edge attributes (Cuv.t) are deliberately minimal: they en- code only whether an edge is a bias connection and whether it spans layers, never leaking functions of Wan,t into the model. Repeating this construction produces temporal sequence {G:) that captures the self-organization of the MLP while remaining invariant to hidden layer permutations.
All descriptors (Iu,,, tue,t) embedded in the Fuclidean tangent space the origin ToT IR", which provides a stable learning backbone. Discrete labels (f(u), (u)) are mapped through learned embeddings, while standardized strength features are passed through a small MLP. A residual block refines the result, producing a n -dimensional representation X,, E To. For edges, the binary con- text Cur.t biss, Aé) is encoded in a bias term bu,, and a gate of near identity Jue.t using an MLP, which modulates attention scores without altering the geometry of the manifold. Thus, normalization, dropout, and mixing remain stable in Euclidean space, while curvature is introduced only in the subsequent hyperbolic message passing layers (Li et al. 2024, ZZhag et al. 2019) Ganea al.] 2018). The final output is a temporal sequence of parameter graphs with node characteris- tics (Xu,t) and edge characteristics (Ewv,z, buv,t: 9uv,t) that respect the permutation symmetry and provide a clean basis for the prediction and regression of semi-supervised link probability.
3.2 HYPERBOLIC GRAPH ATTENTION LAYER: INTUITION AND FORMULATION
The goal of this layer is to compute the updated positions of the nodes on the ball by applying manifold-aware structural self-attention, which respects the curvature of the geometry well as the temporal evolution of parameter graphs. Each parameter graph snapshot G, lives on the Poincaré ball, which naturally encodes hierarchical and heavy-tailed structure. hyperbolic attention layer, therefore, cannot simply apply Euclidean linear maps: every projection, comparison, and aggrega- tion must respect the curvature of the underlying manifold. We follow simple principle throughout: compute in the tangent space, transport to the ball only when geometry matters, and aggregate in model where barycenters (a conifolddccosssterd version of weighted average) are well defined. This yields a stable attention mechanism whose scores and updates have direct geometric meaning (Li ctt al. 2024 2023).
Hyperbolic Affine Maps (Q/K/V on the Ball). Each attention head first computes hyperbolic query, key, and value vectors, which act as the basic attention primitives (cquation 2): the queries encode how a node seeks information, the keys encode how is compared to its neighbors in hyper- bolic space, and the values carry the features that will be aggregated along curved geodesics. Since linear maps are not directly compatible with curved geometry, we realize them as tangent space projections, followed by exponential lifting, which transports the updated tangent vector back onto the manifold along the geodesic defined at that point (Ganea et al. 2018 Li et al. 2024). Given the Euclidean parameters W and b, the hyperbolic affine map (b) is defined as
@ (x) = (x (x)
PT P ( (a) (6)).
(0) W (x) = expo (W log,
(I)
where logo and cxPu move between the tangent space and the ball at the origin, and PT denotes parallel transport. This construction should be viewed as the hyperbolic analogue of linear map plus bias: the matrix W acts in the flat tangent space, while b is transported to the tangent space at & W (x) where it is applied before lifting back onto the ball. Applying & to each node yields the corresponding hyperbolic query, key, and value vectors as follows:
(r,()
(()) wi(r (mmt) (2))),,
(t)) wine gnn (z("), ((())) = (t) wing ggan (2 (=(
Attention Logits and Structural Gating. Attention on the Poincaré ball must measure similarity by geodesic distance, in line with prior hyperbolic graph attention formulations (Zhang et al. 2019 Li et al. 2023 2024). For each directed edge (u + v). we therefore score proximity as the negative hyperbolic distance, since smaller geodesic distances correspond to stronger similarity, and using the
5

---

Published as a conference paper at ICLR 2026
negative ensures that nearer nodes receive exponentially larger attention weights under the softmax (equation 3).
0 - de kgrr))) + ue!
where bue injects edge context such bias indicator or layer separation. Softmax normalization produces the structural attention coefficients in the following manner
expl
(e) Oo Jir
(rit)
(r.t) Qu U-
(4)
E
exp (ritt)
Σ wowww uu(w-+)EEB 0 Swn
Because some edges carry more structural influence (e.g., cross-layer or bias connections), we mod-
ulate these coefficients using gate gao (right part of equation 4). Renormalizing yields a:r.)) which behaves like standard attention but incorporates curvature-awar aware structural reweighting (Li ett al. 2023 2024), which is achieved by multiplying each coefficient by its gate and normalizing across the gated neighbors so that their relative influence is correctly rescaled.
Geometric Aggregation via Einstein Gyromidpoints. Averaging vectors in curved space can- not be done linearly because Euclidean addition does not respect the manifold's geometry: the sum of two points typically lies outside the manifold and does not correspond to any meaningful geodesic midpoint. We therefore use Einstein gyromidpoints, computed in the Klein model where Li et al. 2024). The Poincaré to Klein map (x)
closed-form barycenters exist (Zhang al. 2019 and its inverse are
2/0z
<=2)
R (v) = Vc 1+ /
(5)
+ c|122/1/
1y
with Lorentz factor
(1y) =1/V1 -1y112. For node and head r, the Klein barycenter is
< (mffrt))) Ems(u-ooER (n.)) (x((V)))) (((ll)))
(6)
(..))) (M(U(()))
Mapping back and returning to the tangent space gives the multi-head aggregation (right- partt-parttto equa- tion 7 for computing the node update, which is simply the average of these tangent updates:
H
((:+)) x: (((((((()))))
(())))) logo(m!(:t)
M(O
Σ H
1
This design has critical interpretive consequence: attention uses geodesics to decide "who influ- ences whom," while aggregation respects the global curvature, ensuring that hierarchical structure, hub nodes, and boundary effects are preserved. Each node then receives gated residual update (addition in the tangent space followed by exponential lifting). A projection step enforces a safety margin inside the ball to avoid numerical instability near the boundary (Li et al. 2024).
Temporal Evolution of Attention Kernels. Now, to model temporal smoothness across snap-
shots, meta-evolve the attention parameters using GRU, inspired by (Pareja al. 2019). For
each layer l and head r, we maintain a recurrent state u as follows:
N
1 (t-1)
GRU (pt (t-1) u
(t-1))
Pi
uer
(8)
N
The GRU emits fresh Q/K/V parameters via an MLP:
(6) Lr.k///v = dMuL Wout f.r + bout) E I 3d +3d
(9)
which are then used in the hyperbolic affine maps 11 1- This mechanism allows the attention ker- nels to evolve smoothly over time without storing past embeddings, providing temporal consistency while preserving permutation invariance and limiting memory compact recurrent states.

---

Published as a conference paper at ICLR 2026
3.3 SIGNED WEIGHT REGRESSION
Before introducing the mathematical details of this module, we outline its role in the architecture. A standard link predictor estimates whether an edge should exist, but in weighted networks, also need to recover the actual strength and polarity of each connection. Magnitude reflects how strongly two nodes influence each other, while the sign encodes the direction of that influence. In many real networks, these magnitudes follow heavy-tailed distributions (Song et al., | Krioukov al. 2010 Wei al. 2013, Allard et al. 2017). and hyperbolic distance offers natural geometric proxy for such patterns. The signed regression head therefore serves two purposes: it provides ad- ditional supervision for the temporal encoder, and imposes a geometric prior that ties magnitudes to hyperbolic distances while modeling polarity in the tangent space. In contrast, Euclidean regres- sors or generic MLP decoders (Goyal et al. 2018) ignore curvature and risk losing this interpretive structure.
Magnitude as a Power Law of Hyperbolic Distance. We begin by predicting node-dependent scale and decay factors from the tangent features x(") E Ty = Rd 88 [x( k;
fi((X(()), where fo and fs are MLPs. 8; & K; adjust the local steepness and scale of the magni- oude-distane relation. For each edge (u, v) these corrections are used symmetrically as Sy)
and (ky,,,). k, The slope of the power law is then allowed to vary slightly with local context:
Oue = orp + 50 tanh(oMuP(H
x((), euvl)),
(10)
where Og is global baseline, 8a a small deviation, and ØMLP an MLP (Allard et al. 2017). This design captures mild structural asymmetries between edge types while keeping the decay law inter- pretable and stable. Given these components, the predicted magnitude follows a heavy-tailed power law (Allard et al. 2017):
When
exp(log v + sn + sv) exp( -1- (ku + kb)) (d
(11)
where >> > 0 is global scale and d (= therefore, correspond to smaller hyperbolic distances, while small magnitudes produce larger metric separation (Simas & Rocha, 2014). This establishes direct geometric interpretation of weight strength.
(1) (()) is the Poincaré distance. Large magnitudes,
Polarity in the Tangent Space. Distances on the manifold are positive definite (Simas & Rocha 2014). Polarity is therefore modeled as directional effect in the tangent space, where inner prod- ucts are well defined. We map 2(th into the Tangent Space 21 using the logarithmic map (equation
(()) 1 = -logg (1) (2(")) Ma = W,
Eu
= The
A_00
(12)
X X Iralll
1 c||200
1122
and normalize the source feature Tu by the conformal factor so that directionality is consistent with the Riemannian metric. The sign logit is then given by hyperbolically consistent inner product,
(
tanh (s())
(13)
This construction separates magnitude and polarity in a geometrically faithful manner (equation B A distance-based heavy-tailed law controls magnitude (cquation IT]. while polarity arises from
the angle between transported displacements and conformally normalized source features (equation
12 As result, short hyperbolic distances naturally represent strong connections, whereas longer distances encode weaker or negligible ones, and the directional alignment in the tangent space deter- mines the sign. The module therefore provides an interpretable decomposition of each weight into strength and direction, consistent with the geometry of the Poincaré ball.
3.4 LINK DECODER, SUPERVISION, AND OBJECTIVE
To translate the hyperbolic embeddings into edge predictions, we use the Fermi decoder (1)(t)
(equation (4) (Nickel & Kiela 2017 2018 Wen et al. 2024). which links connectivity directly to the geodesics on the Poincaré ball, for a given time step t:
/ (4) (u, v) = (1 + exp de(2(1) (0)
(14)
7

---

Published as a conference paper at ICLR 2026
Here, the radius R. controls the effective neighborhood boundary and the temperature T controls the sharpness of the tramsition, both learned during training. Under this decoder, nodes that lie close in hyperbolic distance are classified as likely neighbors, and distant nodes unlikely. Training uses binary cross-entropy loss together with dynamic negative sampling from both same-layer and uniform distributions. Moreover, connectivity alone does not describe the strength or direction of in- fluence between two parameters. For this reason, we include a signed weight regressor (Section (33) that predicts the full value of each positive edge. The regression loss encourages the predicted weight to match the true magnitude, while separate classification loss predicts its polarity. The magnitude loss measures the difference between predicted and true weights, while the polarity loss compares the predicted sign logit with the binary label indicating whether the weight is positive or negative. Formal expressions for these losses and their components are provided in Appendix [ This design ties geometric separation to functional strength: large-magnitude edges correspond to smaller hyperbolic distances, and the sign is encoded through directions in the tangent space. As result, the embeddings acquire semantic meaning: hubs drift outward, stronger edges cluster ra- dially, and excitatory versus inhibitory effects appear as angular variation. To maintain coherent geometry across time, we add a small set of regularizers. A slope prior keeps the decay exponents near global baseline, a temporal smoothness term limits abrupt movement between snapshots, and ranking loss enforces that stronger interactions correspond to shorter hyperbolic distances. The overall objective is a weighted sum of the Fermi Dirnc cross- crosspyn the magnitude and sign super- vision terms, and these geometric regularizers. Training follows gentle curriculum that anneals the ranking margin and progressively hardens negative sampling to stabilize learning before introducing difficult examples. Ablations about specific terms can be found in Appendix !
Optimization proceeds in two distinct phases. This separation is necessary because the attention stack and regressors operate in Euclidean space, whereas the node embeddings evolve on a curved manifold; using single update rule would either ignore curvature or force the Euclidean modules to inherit manifold constraints they were not designed for. the first phase. a Euclidean pass updates all kernel weights, regressors, and the link decoder using standard backpropagation. In the second phase, a Riemannian pass refines the node positions 2(() directly on the Poincaré ball. This intrinsic step begins by rescaling Euclidean gradients with the conformal factor X2 = 23/1/|ll=] -c|==|]) so that updates follow the manifold geometry rather than the ambient space (Bridson & Haefliger 1999 Bonnabel| 2013 Nickel & Kiela 2017). To stabilize learning, first and second moments of accumulated in a RAMSGrad style procedure (Bécigneul & Ganea 2019 Sakai & liduka| 2020), which extends adaptive optimization to curved spaces. Each accepted update is applied via the exponential map. projecting the tangent step back onto the manifold. Finally, moment vectors are parallel transported to the new tangent space, keeping the optimizer's memory coherent across iterations. This two-phase design decouples the stability of the Euclidean attention stack from the dynamics of hyperbolic embeddings, ensuring that geometry is respected while long training traces remain stable. We describe the algorithm in Appendix G Overall, this scheme does more than minimize loss: it shapes an embedding space where magnitudes follow radial distance, polarity is encoded in tangent orientation, and temporal smoothness emerges from controlled geodesic motion. Geometry thus becomes a built-in inductive bias for compact and interpretable representations.
these gradients
4 EXPERIMENTAL SETUP AND RESULTS
Classification of Images via INR Traces. This experiment evaluates whether the temporal evolu- tion of the parameter graph of the intrinsic neural representation (INR) of image contains enough structure for a meta-network to infer the class of the underlying image. Crucially, the meta-network never sees the image itself: it only receives the evolution of the INR weights over optimization. We fit each image from MNIST and Fashion-MNIST using a shallow INR, recording its parame- ter values over optimization, yielding a temporal trace of parameter graphs. Each trace contains T E [80 - 100) checkpoints, i.c., sequence of signed mouuettrr-gaphh. Dataset splits are made at the image level to avoid leakage; we use 45,000 images for training. 5,000 for validation, and 10,000 for testing. A spatio-temporal meta-network is pretrained with intrinsic (hyperbolic) optimization on a joint men-prediittoon and signed weight-regression objective over the full kept trace per image, along with other losses in our curriculum; within each checkpoint, edge embeddings are pooled to a snapshot vector, and snapshot vectors are then pooled over time to produce a fixed n-dimensional representation. A small feed-forward classifier is trained on these representations to predict the digit or fashion class. The same encoding procedure is applied at validation and test time, and we report classification accuracy on the held-out test images. Results are shown in Table T
8

---

Published as a conference paper at ICLR 2026
Table 1: INR classification static vs. temporal settings. Average test accuracies (# standard error) on MNIST and Fashion-MNeN INRs. Baselines are reproduced from DWSNets (Navon et al. 2024) and NFNs
Zhou et al. 2023ah. Our spatio-temporal variant uses pooled representations actoss K traces.
Model
MNIST INR
Fashion-MNIST INR
MLP
17.55 + 0.01
19.91 + 0.47
MLP + Perm. aug
29.26 + 0.18
22.76 = 0.13
MLP Alignment
58.98 = 0.52
47.79 = 1.03
Vec (Arch.)
23.69 + 0.10
22.33 + 0.41
Transformer
26.57 + 0.18
26.97 = 0.33
DWSNets
85.71 = 0.57
67.06 = 0.29
HFN
92.5 = 0.07
72.7 + 1.53
NFNNP
92.9 + 0.22
75.6 + 1.07
Ours (Temporal, K-trace pooling)
95.6 + 0.18
80.72 + 0.29
Predicting DNN Generalization from Weights. This experiment tests whether the temporal evo- lution of the parameters of a model contains predictive signal about its test accuracy. More specif- ically, can we recover accuracy-related information solely from a deep neural network's s weight structure? Here, each trial i, corresponds to an MLP trained on a subset of the CIFAR-10 dataset. For each trial i we gather checkpoints {0,,c)=- and the corresponding test accuracy (per-epoch) yr.1, and convert every checkpoint to parameter graph Gt,; with corresponding node/edge features. Traces are encoded over fixed-length (stride-roleed) windows or complete sequences, with last-step supervision, i.e., the intermediate steps evolve while the final step is supervised. We classify the divisions by trial, and the fidelity of the classification is measured using Kendall's r (Zhou et al., 2023a Navon al. 2024 Lim al. 2023). See Results in (Table 2
Table 3: Results of meta-net representations sinusoid-MIPP; Entries report test MSE a downstream MLPregressor trained on the learned representations. We compare with other baselines
Table 2: Kendall's correlation on the CIFAR- 10 benchmark. This table compares the rank cor- relation between predicted and ground-truth ac- curacies of neural architectures (INRs) cap- tured by different meta-network representations. Higher values indicate stronger consistency be- tween the learned representation space and the empirical performance ranking of architectures (Unterthiner al. 2021 Zhou et al. 2023a)
by Lim et al. (2023) Navon et al. 2024
Zhou
et (2023a).
Metanetwork
Test MSE
MLP
7.39 + 0.19
MLP Perm. aug
5.65 + 0.01
Metanetwork
CIFAR-10
MLP + Alignment
4.47 = 0.15
NFNINP
0.934 0.001
INR2Vec (Arch.)
3.86 + 0.32
NFN
0.922 + 0.001
Transformer
5.11 0.12
StatNN
0.915 + 0.002
DWSNets
1.39 = 0.06
GMN
1.13 + 0.08
GTH-GMN (ours)
0.846 + 0.004
GTH-GMN (ours)
1.06 + 0.24
Predicting Sine Wave Frequency. Additionally, we also evaluate our temporal model on the benchmark introduced by (Navon et al. 2024), in which each input network is an MLP trained to fit one-dimensional sinusoidal function of the form +> asin(bz)) with coefficients a, b E IR varying across samples. The goal is to learn meta-network encoder that maps these trained MLP temporal parameter graphs to meaningful representations. while remaining invariant to neuron per- mutations, where representation quality is assessed in a centrastive-learminn framework inspired by SimCLR (Lim et al.) 2023) Navon et al. 2024 Zhou al. 2023a Chen et al.) 2020). We intro- duce a temporal variant: link successive checkpoints per INR. build an edge-centie hyperbolic representation, mean-pool, and predict (a, b) with small ML.P See results in Table 3
9

---

Published as a conference paper at ICLR 2026
Prmcre teek PPAA 1w13
Pemsrr PGG de bw!f
Live
Live
LAWe
Figure 2: Hyperbolic embeddings of parameter graphs from 13-layer MLP trained INR CIFAR-10 10, shown at checkpoints
11 (left) and 97 (right). Embeddings are learned in d = 10 dimensions with 4 HGAT layers and 4 EvolveGCN-H kernels (tangent dim. 128) and projected to the Poincaré disk via PCA. Nodes are colored by layer index, with edges in gray, illustrating the progressive reorganization and separation of layers over training.
DISCUSSION
Interpreting the Results. Across the three experiments Section 4 we aimed to test whether a single temporal hyperbolic meta network can generalize across very different prediction tasks. The results suggest that the evolution of the parameter graph carries a stable and predictive structure. In the INR classification, the encoder achieves average test accuracies of 95.6 = 0.18% on MNIST and 80.72 + 0.29% on Fashion MNIST, revealing that the way INRs self-organize during training encodes class-specific geometric structures that the model is able consistently separate. In the CI- FAR 10 generalization prediction task, we achieve relatively modest Kendall's r = 0.846 + 0.004 compared the neural functional baselines. Our factorization emphasizes global geometric struc- ture and temporal coherence over preserving all fine-graiee fine-grained tensor details. It has been shown that representation learning of neural networks using Autoencoders often exhibits a smoothening bias as MSE reconstruction loss tends to coarse- representations (Meynent et al. (025). In con- trast, pernutationn-equivrrinn neural functionals operate directly on raw weight tensors (Zhou et al.) 2023a) and retain more cocuracy-correlate pretability, This likely contributes to the relatively lower Kendall's T achieved over the CIFAR- 10 dataset (Fu et al. 2021). We see this as trade-off between raw performance over the generaliza- tion prediction task and generating geometric and interpretable embeddings of the underlying DNN. Moreover, over the sinusoid benchmark, our temporal hyperbolic model reaches the 1.06 0.24 test MSE, comparable in mean to GMN's 1.13 + 0.08, though with higher variance. This arises from the interaction of curvature (Fu et al. 2021 and temporal coupling: in the Poincaré ball, the ef- fective step size increases with radius, so small early differences can produce divergent trajectories, and the recurrent kernel further amplifies such variation (Nickel & Kiela [0177] [Bonaaeel 2013 Ganea et al.) 2018). Variability can be reduced with more conservative Riemannian step control, stronger temporal regularization, or light ensembling, which are standard techniques for stabilizing hyperbolic and recurrent models (Ganea et al., 2018 Chen et al. 2024 (Sun et al. 2021b).
micro-structure at the cost of reduced geometric inter-
Implications for Explainability. The embeddings reveal coherent patterns of self-organization: points drift toward the boundary, layers separate in both radial and angular directions, and trajecto- ries evolve smoothly across training (Figure(). These behaviors suggest several potential signals for intrinsic explainability. Radial position and drift may reflect how influence strengthens weakens over time; angular separation may indicate the emergence of functional specialization; geometric proximity and repeated strong connections may reveal central processing hubs; and consistent mag- diiudeddistance alignment may highlight connections that are both geometrically and functionally salient. Conversely, nodes whose trajectories collapse together drift outward uniformly may sig- nal redundancy and pruning opportunities (Nickel & Kiela 2017 Ángeles Serrano et al. 2008). Additionally, Appendix (ection[K]xppaan how our approach can be extended beyond multilayered perceptrons (MLPs).
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
I thank Eindhoven University Technology (TU/e) for providing access to the Umbrella cluster, which supported part of the experimentation. Moreover, would like extend my thanks to Dr. Indika Kumara and Dr. Damian Tamburri, from Tilburg University and Eindhoven University re- spectively for their supervisory contribution during the course of this project. also acknowledge SURF (www.surf.nl) for enabling use of the National Supercomputer Snellius. The use of large language models (LLMs) was restricted exclusively to polishing the text for brevity under space constraints. The author(s) take full responsibility for the research ideation and the experimentation conducted and reported in this work.
REFERENCES
Antoine Allard, M. Angeles Serrano, Guillermo García-Pérez, and Marián Boguñá. The ge- ometric nature of weights in real complex networks. Nature Communications, 8(1):14103, 2017. ISSN 2041-1723. doi: 10. 00.1038//comms141000 URL
ncomms 4103]
L. A. N. Amaral, A. Scala, M. Barthélémy, and E. Stanley, Classes of small-world networks. Pro- ceedings of the National Academy of Sciences, 97(21):11114-111155 2000. doi: 10. 1073/pnas. 200327197. Bttppttts://ww
Qijic Bai, Changli Nie, Haiwei Zhang, Dongming Zhao, and Xiaojie Yuan. Hgwavenet: A hy- perbolic graph neural network for temporal link prediction. In Proceedings of the ACM Web Conference 2023, WWW '23, pp. 532-532. ACM, April 2023. doi: 00.11457353300.553355. URL http://d:/ doi http:////xx..1..rrr11111111.1111
10.
3583455
Danielle Bassett and Ed Bullmore. Small-world brain networks. Neuroscientist, 12(6):512-523
December 2006. ISSN 1073-8584. doi: 10. 10.117/1077338800299138
Silvere Bonnabel. Stochastic gradient descent on riemannian manifolds. IEEE Transactions on Automatic Control, 58(9)) September 2013. ISSN 1558-2523, doi: 10.1109ttac2011.. 2254619. URL http: dx. dww.oxx.oogg.11//d. 1119/1110CCC. 2013. 2254619 2011
Martin R. Bridson and André Haefliger. Metric Spaces of -os-Piiii Curvature, Springer, Berlin, Heidelberg, New York, 1999. doi: 10. 111100978----62-1111 12494-9.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learn- ing: Grids, groups, graphs, geodesics, and gauges, 2021. URL httpt::/aaa....coc////oo//o
13478
Ed Bullmore and Olaf Sporns. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience, 10(3):186-1))) 1988 2009. ISSN 1471-0048, doi: 1038/ 1038//mn2285555555 /rn2555
10.1038/nm2275. URL/tttpa://dol.orr
Gary Bécigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods, 2019.
wtRL/tttss://axiv.coo
Alexander Campbell, Antonio Giuliano Zippo, Luca Passamonti, Nicola Toschi, and Pietro Lio. Dyndepnet: Learning time-varying dependency structures from fmri data via dynamic graph structure learning, 2023.
13513
Ines Chami, Rex Ying, Christopher Ré, and Jure Leskovee. Hyperbolic graph convolutional neural networks, 2019. URL
Honglin Chen, Hsueh-Ti Derek Liu, Alec Jacobson, David I. W. Levin, and Changxi Zheng. Trust- region eigenvalue filtering for projected newton, 2024. URL
2410 10102
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, 2020. URL httph: //ar:iv///vvvv..a w/aaxiivorrg////e/aa/......o.olt abs/ 2002. 05709

---

Published as a conference paper at ICLR 2026
Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou.
Fully hyperbolic neural networks. arXiv preprint arXiv:2105. 14686, 2021.
Reuven Cohen and Shlomo Havlin. Complex Networks: Structure, Robustness and Function. Cam- bridge University Press, Cambridge, UK, 2010. ISBN 9780521841565.
Hejie Cui, Wei Dai, Yanqiao Zhu, Xiaoxiao Li, Lifang He, and Carl Yang. Brainnnexplainer: An interpretable graph neural network framework for brain network based disease analysis, 2021.
URL BttLhhtss//azxiii org. wttp://wwww..g///////.ooooo 05097
Hejie Cui, Wei Dai, Yanqiao Zhu, Xiaoxiao Li, Lifang He, and Carl Yang. Interpretable graph neural
networks for connectome-bedt based brain disorder analysis, 2022. URL httts:://axxiv.or
abs/2207.00813
Luca De Luigi, Adriano Cardace, Riccardo Spezialetti, Pierluigi Zama Ramirez, Samuele Salti, and Luigi Di Stefano. Deep learning on implicit neural representations of shapes. arXiv preprint arX14:2302.055488 2023.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst.. Convolutional neural networks on graphs with fast localized spectral filtering, 2017. URL httpp::/aaax...orr//aa/////
09375
Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point is function and you treat like one. arXiv preprint arXiv:2201. 12204, 2022.
Gabriel Eilertsen, Daniel Jönsson, Timo Ropinski, Jonas Unger, and Anders Ynnerman. Classifying the classifier: dissecting the weight space of neural networks. arXiv preprint arXiv:2002.05688, 2020.
Agata Fronczak, Piotr Fronczak, Mateusz J. Samsel, Kordian Makulski, Michal Lepek, and Maciej J. Mrowiński. Scaling theory of fractal complex networks. Scientific Reports, 14:9079, 2024. doi: 10.1038/s41598- 0248--22--557-22
Xingcheng Fu, Jianxin Li, Jia Wu, Qingyun Sun, Cheng Ji, Senzhang Wang. Jiajun Tan, Hao Peng, and Philip Yu. Ace-hgnn: Adaptive curvature exploration hyperbolic graph neural network. In 2021 IEEE International Conference Data Mining (ICDM), pp. 111-120. IEEE, Decem- ber 2021. doi: 10.11199icdm51623 122.0m2..02220.200000000 URL http: dx.
ICDM5 1 629 2021 00021
Octavian Ganea, Gary Becigneul, and Thomas Hofmann. Hyperbolic neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL pttpes://proceesiiggg.... httpps/s/pppeeedddsss.eersssssss file/ http://aa2aaac111000000000000000
htcc/paper_fils/ppeeeeeeeee files//pper/20188/ www-aaapppppppp
Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dy- namic graphs, 2018.
Mikhael Gromov. Hyperbolic groups. In Essays in group theory, pp. 75-263. Springer, 1987.
Martin Guillemaud, Vera Dinkelacker, and Mario Chavez. Hyperbolic embedding of multilayer 20378
networks, 2025. URL (ttttt:ss//:axiiii
Ehsan Hajiramezanali, Arman Hasanzadeh, Nick Duffield, Krishna R. Narayanan, Mingyuan Zhou, and Xiaoning Qian. Variational graph recurrent neural networks. In Advances in Neural Informa- tion Processing Systems 32 (NeurIPS 2019), 2019.
wwwcxiv.org/abs/1900
09710
William L Hamilton. Graph representation learning. Morgan & Claypool Publishers, 2020.
Keqi Han, Yao Su. Lifang He, Liang Zhan, Sergey Plis, Vince Calhoun, and Carl Yang. Rethinking functional brain connectome analysis: Do graph deep learning models help?, 2025. URLhetps.
//arxiv..rg//aaae. 17207
12

---

Published as a conference paper at ICLR 2026
Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruming, trained quantization and huffman coding, 2016. URL
Abs/1551.0014
Robert Hecht-Nielsen. On the algebraic structure of feedforward network weight spaces. In Ad- vanced Neural Computers, pp. 129-135. Elsevier, 1990.
Xuan Kan, Zimu Li, Hejie Cui, Yue Yu, Ran Xu, Shaojun Yu, Zilong Zhang, Ying Guo, and Carl Yang. R-mixup: Riemannian mixup for biological networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23, pp. 1073-1085. ACM, August 2023, doi: (0.1145/550030055599944 URL http: doi owg..r//1...
Byung-Hoon Kim, Jong Chul Ye, and Jae-Jin Kim. Learning graph represenaatooo of brain connectome with spatio-temporal attention, 2021.
13495
Kim, K.-I. Goh, Salvi, E. Oh, B. Kahng, and Kim. Fractality in complex networks: Crit- ical and supercritical skeletons. Physical Review E, 75(1), January 2007a. ISSN 1550-2376. doi: 10100/physeeee.55.111111 016110
URL http://dxx doi http
J. S. Kim, K.-I. Goh, G. Salvi, E. Oh, B. Kahng, and D. Kim. Fractality in complex net- works: Critical and supercritical skeletons. Physicul Review E, 75:016110, 2007b. doi: 10.103//hysReeE275.061100
Miltiadis Kofinas, Boris Knyazev, Yan Zhang, Yunlu Chen, Gertjan J. Burghouts, Efstratios Gavves, Cees G. M. Snoek, and David W. Zhang. Graph neural networks for learning equivariant repre- sentations of neural networks, 2024. URL httthhttt//aaxttii
abtgg//bb/// 22t./2433... 12143
Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguñá. Hyperbolic geometry of complex networks. Physical Review E, 82(3):036106, 2010. doi: 10. //00//hyyseevE.82.066111
Vito Latora and Massimo Marchiori. Efficient behavior of soald-world networks. Phys. Rev. Lett, 87:198701, Oct 2001.
doi: 10.1003/hy.ReeeLLt......83 URL attps://link aps.
Hao Li, Hao Jiang, Dongsheng Ye, Qiang Wang. Liang Du Yuanyuan Zeng, Liu yuan, Yingxue Wang, and Cheng Chen. Dhgat: Hyperbolic representation learning dynamic graphs via tention networks. Neurocomputting, 568: 127038. 2024. ISSN 0925-2312. doi: https://oi..or//
20.101166jecccm2222331777078 URL https: http:tts/ww.weeeeeeeeeeee..oooo wwtpe/eeettttoccceeeeeeeeee www.sciccc//////cccccccccccccccc artiicle/pi1//09252222222
Jingling Li, Yanchao Sun, Jiahao Su, Taiji Suzuki, and Furong Huang. senenaiioninnngennaaaliiiioooooo in deep learning via tensor methods. 2020. U.LL:ttt.////a.....////////....0
Xiaohan Li, Yuqing Liu, Zheng Liu, and Philip S. Yu. Time-aware hyperbolic graph attention network for session-basse based recommendation, 2023. Utl/hctpss//axxx..oog//////////
03780
Xiaoxiao Li, Yuan Zhou, Nicha Dvomek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H Staib, Pamela Ventola, and James S Duncan. Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis, 74: 74:102233, 2021. ISSN 1361-8415. doi: BRaiNGNN BrainGNN_Pytorch
URL https: http:::///ittbb..o.o htt.t/ub..omm//.llpp///.
Derek Lim, Haggai Maron, Marc T. Law, Jonathan Lorraine, James Lucas. Graph metanetworks for processing diverse neural architectures, 2023.
04501
Or Litany, Haggai Maron, David Acuna, Jan Kautz, Gal Chechik, and Sanja Fidler. Feder- ated learning with heterogeneous architectures using graph hypernetworks. arXiv preprint 2oXiv:2201.08455 2022.
13

---

Published as a conference paper at ICLR 2026
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09000 2018.
Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph
networks. In International Conference on Learning Representations, 2019. URL https:// openreview net/ www.mridddsx22222233333
Léo Meynent, Ivan Melev, Konstantin Schürholt, Göran Kauermann, and Damian Borth. Structure not enough: Leveraging behavior for neural network weight reconstruction, 2025. URL https:
Aviv Navon, Aviv Shamsian, Ethan Fetaya, Gal Chechik, Nadav Dym, and Haggai Maron. Equiv- ariant deep weight space alignment, 2024. URL
Mark Newman. Networks: An Introduction. Oxford University Press, Inc., USA, 2010. ISBN 0199206651.
Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representa- tions, 2017. URL httthttt///aaeee wttpxiii..rr//b//////00/ 08039
Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry, 2018.
03417
Maximilian Nickel, Xueyan Jiang, and Volker Tresp. Reducing the rank in relational factorization models by including observable patterns. Advances in Neural Information Processing Systems, 27, 2014.
Fragkiskos Papadopoulos, Dmitri Krioukov, Marián Boguñá, and Amin Vahdat. Efficient navigation in scale-free networks embedded in hyperbolic metric spaces. arXiv preprint, 2008
Fragkiskos Papadopoulos, Maksim Kitsak, M. Ángeles Serrano, Marián Boguñá, and Dmitri Kri- oukov. Popularity versus similarity in growing networks. Nature, (ew.14487777337--25555 Septeme 2012. ISSN 1476-4687. doi: 10.1033/nature/ 1459. URL htt.hhtp://// /xx dei. ott...oo//...1103///133
nature 45%
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, and Charles E. Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs, 2019. URL
/1902. 10191
Ciyuan Peng. Yuclong Huang, Qichao Dong, Shuo Yu, Feng Xia, Chenggi Zhang, and Yaochu Jin. Biologically plausible brain graph transformer, 2025. URL htttt::t///aattt org/ /
0202.08958
William T. Redman, Juan Bello-Rivas, Maria Fonoberova, Ryan Mohr. Yannis G. Kevrekidis, and Igor Mezić. Identifying equivalent training dynamics. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Pro- cessing Systems, volume 37, pp. 23603-23629, Curran Associates, Inc., 2024. doi: 10.52202/ wew.eeeeeeddnnn.....e.eee...eeee neursss.eeiippc Paper Confereen--ooffeeecceece
079017-0744. URL htt.:p/ee.ppppeeeinn..eeeraaeeep http://appprrfieeepppeeee.
Mikail Rubinov and Olaf Sporns. Complex network measures of brain connectivity: Uses and interpretations. Neurolmage. 5253):1059-1069, 2010. ISSN 1053-8119. doi: https://do..or/100
1016ff.nerroimage2200...
URL pestps:/wwwr sci encedi rect. cwm. com/sciienee/ article/pii/5105533 00055331111990011 0 074X Computational Models of the Brain.
Hiroyuki Sakai and Hideaki liduka. Riemannian adaptive optimiizaionii alga algorithm and its application to natural language processing, 2020. URLLhttpes///arrii.o////////// 00897
W. Scannell, C. Blakemore, and M. P. Young. Analysis of connectivity in the cat cerebral cortex. The Journal of Neuroscience, 15(2):1463-1483 February 1995. ISSN 0270-6474. doi: 10.
|523//NERRSCC.:5502220000 011411555------119999999 URL https: http///oo.ooo./oo////o 1523 1 03-02-01463..00 1995
14

---

Published as a conference paper at ICLR 2026
Konstantin Schürholt, Dimche Kostadinov, and Damian Borth. Self- representation learn- ing on neural network weights for model characteristic prediction. Advances in Neural Informa- fion Processing Systems, 34: 14418-14111493 2021.
Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequeeee modeling with graph convolutional recurrent networks, 2016. URL https://arxiv.
abs /os/1612.. 07659
M. Angeles Serrano, Dmitri Krioukov, and Marián Boguñá. Self-similarity of complex networks and hidden metric spaces. Physical Review Letters, 100(7), February 2008. ISSN 1079-7114. doi: 10.1 http://tyyreeeee..00.07777///11 URL http: doi wrw.lo.rgoo.... 11.3/00..1..PP.0 wtt....ee...eeeeeetttttttt...... 10C. 078701
Xingjian Shi, Zhourong Chen, Hao Wang, Dit- Yan Yeung, Wai kin Wong, and Wang chun Woo. Convolutional Istm network: machine learning approach for precipitation nowcasting, 2015.
httthhtts//aaaae wwt.:oggg//aoo.ooo
abs/1506. 04214
Tiago Simas and Luis M Rocha. Distance closures complex networks, 2014. URL https:
//arxiv.org/ org. abs hwtp://rrrraaaaaal....a.
Chaoming Song. Shlomo Havlin, and Hernán A. Makse. Self-similarity of complex networks. Na- fure, 433:392-395, 2005. doi: 10. 10.00//attur00324..
Li Sun, Zhongbao Zhang, Jiawei Zhang, Feiyang Wang. Hao Peng, Sen Su, and Philip S. Yu. Hy- perbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 4375-4383, 2021a.
Li Sun, Zhongbao Zhang, Jiawei Zhang, Feiyang Wang, Hao Peng, Sen Su, and Philip S. Yu. Hy- perbolic variational graph neural network for modeling dynamic graphs, 2021b. URL https:
//axxiv.org/abs abt/////aa///e 2104. 02228
Bishal Thapaliya, Esra Akbas, Jiayu Chen, Ram Sapkota, Bhaskar Ray, Pranav Suresh, Vince D. Calhoun, and Jingyu Liu. Brain networks and intelligence: A graph neural network based ap- proach to resting state fmri data. Medical Image Analysis, 101:103433, April 2025. ISSN URL http:htpp:/eee doi http://ttt.ooior//....o.//// 1016/J.
1361-8415. doi: 10. mmdiia. 2024. 103433
Abraham Ungar. A gyrovector space approach to hyperbolic geometry. Springer Nature, 2022.
Abraham Ungar. The hyperbolic square and mobius transformations. Banach Journal of Mathe- matical Analysis, 1(1):101-11,, 2007.
Thomas Unterthiner, Daniel Keysers, Sylvain Gelly, Olivier Bousquet, and Ilya Tolstikhin. Predict- ing neural network accuracy from weights, 2021. URL..tttps.//aaxiv.rrrr/////b//
448
Daijun Wei, Bo Wei, Yong Hu, Haixin Zhang, and Yong Deng. A new information dimension of complex networks. arXiv preprint arXiv:1311.3527, 2013.
/tttpp:://axxv.... org/
abs/1311.3527
Lingfeng Wen, Xuan Tang. Mingjie Ouyang, Xiangxiang Shen, Jian Yang, Daxin Zhu, Mingsong Chen, and Xian Wei. Hyperbolic graph diffusion model, 2024. htth::ts:://// htt.:p:://aaxv..../ org/
abs/2306. 07618
Zhihao Wu, Giulia Menichetti, Christoph Rahmede, and Ginestra Bianconi. Emergent complex network geometry, Scientific Reports, 5(1), May 2015. ISSN 2045-2322. doi: 10. 10.1038/srep10073. doi
URL http:///x...oooo///1./oo/1// 10tp://111..03////p11177.. srrpp/07/
Menglin Yang, Min Zhou, Hui Xiong, and Irwin King. Hyperbolic temporal network embedding. IEEE Transactions Knowledge and Data Engineering, 35(11): 1489-11102, 2023. doi: 10.
11099TKDEE.2022.3333338
Jiaxuan You, Jure Leskovee, Kaiming He, and Saining Xie. Graph structure of neural networks, 2020. URL httphhtttt//att www.vvorggaaaao
http://wwwwrg/bb/////oo// httgg//b///0000 06559
15

---

Published as a conference paper at ICLR 2026
Yue Yu, Xuan Kan, Hejie Cui, Ran Xu, Yujia Zheng, Xiangehen Song, Yanqiao Zhu, Kun Zhang, Razieh Nabi, Ying Guo, Chao Zhang, and Carl Yang. Learning task-aware effective brain connec- tivity for fmri analysis with graph neural networks, 2022. URL https::/aaxxi..oeettt /bs/
2211. 00261
Y. Zhang, X. Wang, C. Shi, N. Liu, and G. Song. Lorentzian graph convolutional networks. In Proceedings of the Web Conference 2021 (WWW 22), pp. 2249-126., ACM. 2021.
Yiding Zhang, Xiao Wang, Xunqiang Jiang. Chuan Shi, and Yanfang Ye. Hyperbolic graph attention network, 2019, /ttp:://s/// wttpaaawwwww....aa/aaaaaa// 03046
Allan Zhou, Kaien Yang, Kaylee Burn,, Adriano Cardace, Yiding Jiang, Samuel Sokota, J. Zico Kolter, and Chelsea Finn. Permutation equivariant neural functionals, 2023a. URL https:
//axxiv.crg/abs/22330 wtt..////33.11.. 14040
Allan Zhou, Kaien Yang. Yiding Jiang, Kaylee Burns, Winnie Xu, Samuel Sokota, Zico Kolter, and Chelsea Finn. Neural functional transformers, 2023b. Uttpttss://acxxv. org/abrrgaaas
2305 13546
M. Ángeles Serrano, Dmitri Krioukov, and Marián Boguñá. Self-similarity of complex networks and hidden metric spaces. Physical Review Letters, 100:078701, 2008. doi: 10.1103/PhysRewLet. 100.078701.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A NOTATIONS
Parameter Graphs & Features
G2
Parameter-graph snapshot at epoch /
F,
Node set; edge set at epoch t
wio
Signed edge label on {u, w}
Cuv
Edge context (bias/kernel, interlayer flag)
L, d,
#layers; embedding dim; curvature parameter Layer label of node
((u)
T(U)
Type label (input / neuron bias)
Saba (u)
Incident absolute-weighe sum at Incident signed-weiee sum Standardized node-weight signals Final node feature vector
Sugn (u)
Sabar Suga
Tu
Ve, N₂
Nodes in layer E; their count
Temporal States and Recurrence
Time/epoch index
Tangent-backkone feature of node i t
Mean-pooled tangent for layer
Recurrent state (layer f, head r)
Packed attention parameters (head r)
Hyperbolic Geometry (Poincaré Ball)
B2
d-dimensional Poincaré ball
Geodesic distance on BB
exp,, (-), (.)
Exponential logarithmic maps at z
dx
Conformal factor at point z
PTz-)(()
Parallel transport from
K(-), K K-1(-)
Poincaré<+Kleit Klein coordinate maps
Hyperbolic Attention Block
Attention head index
N (1:9)
(1s)) (10
Query, key, value points on Bd
Softmax temperature (head r)
bwv
Edge bias from context Edge gate from context Attention coefficient (head r)
Juv
(n.t)
m. (r.4)
Head message in tangent space Multi-head combined message
(
Node position update on IBd Neighborhood of node
N(v)
y
Klein coordinate (||yll < 1)
7(v)
Lorentz factor in Klein model
17

---

Published as a conference paper at ICLR 2026
Decoders and Regressors
( (1, u)
Fermi-Dirac link probability
R, T
Link radius; temperature (FD decoder)
(1)
Predicted signed weight; magnitude
Ouu
Disnance-magnitude exponent (edge-wise)
v, B
Magnitude scale; sign-logit scale
Hyperbolic displacement (source u to v)
Conformal direction for sign prediction
Suv
(t)
Sign logit
g
Small positive constant (stability)
Losses and Optimization
LFD
BCE loss for Permi-Dirac link decoder Magnitude regression loss
Cweee
bsign
Sign prediction loss (BCE) Prior/regularizer on Que
La
Lamooth
Temporal smoothness penalty
Lrank
Pairwise ranking loss
E+, E
Positive edges; sampled negatives
Trank
Ranking margin parameter
Eue L, Riem L
Euclidean Riemannian gradients
A;
Intrinsic update step at node
((+1)
Updated node position
n
Learning rate
B THEORETICAL PRELIMINARIES
B.1 RIEMANNIAN MANIFOLD
A Riemannian manifold (M,g is a smooth manifold M endowed with Riemannian metric g,M, which assigns to each tangent space T.M at point E .M positive definite inner product. The tangent space T&M consists all tangent vectors at x and may regarded as the best linear, first-order approximation to the manifold in neighborhood of that point. The metric equips this linear space with a bilinear form (::) T.M x T.M -> TR, from which a norm induced by ||v|| = gV, v) for any v E T.M. With this structure, the manifold acquires the ability to measure lengths, angles, and volumes, thereby extending the familiar tools of Euclidean geometry to curved spaces. If I (a, bj + M is smooth curve, its Riemannian length is defined as L(>) = Ja (t)ll)) dt, quantity that depends on the metric through the induced norm tangent vectors. Among all curves connecting two points æ, E M, geodesics play a central role: they are the locally length-minimizin paths, serving as the natural analogue of straight lines in Euclidean space. The Riemannian distance is accordingly given by ( y) inf, L(7), where the infimum is taken over all smooth curves y with r(a)
and >b) y. At this point, fundamental questions of differential geometry naturally arise. Given a point x E M and a tangent vector e E T.M, how can one move from the linearized tangent space back to the curved manifold? Conversely, given two points Z, y M, how can their displacement be expressed in the linear coordinates of T.M? These questions motivate the introduction of the exponential and logarithmic maps. The exponential map at x, denoted exp, TyM + .M, takes tangent vector v E TrM and returns the point on the manifold obtained by following the geodesic that begins at x in the direction of parameterized provides a principled mechanism for "lifting" linear displacements in T into the curved geometry of M. The logarithmic map, denoted log, M M,M, is defined locally the inverse of the
that unit time corresponds to unit length. In this way, the exponential map
18

---

Published as a conference paper at ICLR 2026
exponential map. For a point y M sufficiently close x, it yields the unique vector M.M such that exp,, = y. Thus, the log map addresses the complementary problem: it represents the displacement between manifold points in the linear framework of the tangent space. Taken together, these constructions endow the Riemannian manifold with a powerful interpretative scheme. Tangent vectors become infinitesimal displacements that may be integrated into finite motions along the manifold via exp2, while finite displacements can be linearized into tangent vectors through log.. This duality shows why exponential and logarithmic maps are often viewed rigorous analogues of addition and subtraction on curved spaces, providing the formal apparatus that underpins the heuristic extension of Euclidean intuition Riemannian geometry.
B.2 POINCARÉ BALL MODEL
A hyperbolic manifold can be described as Riemannian manifold of constant negative curvature e with e < Among the isomorphic realizations of hyperbolic space, three models are commonly employed: the Poincaré ball model, the Lorentz model, and the Klein model. In what follows, we focus on the Poincaré ball model, which offers a particularly convenient formulation for learning problems in hyperbolic geometry.
The Poincaré ball model denoted as (B29")) where the domain IB = {a E R" ||=||² < -1/c] the open n-dimensional ball of radius 1/ 1/vlcl Its Riemannian metric tensor is defined as d = (Xg)² ge. where de is the Fuelidean metric, i.e. the identity matrix In and X = r+o| is the conformal factor that encodes the curvature dependence.
Given two points x, y E B, the induced geodesic distance is given by
1 cosh (x,)) = viel
(1-
2c/ y/|²
(15)
(1 + c a (3) + cllvl13)
Closed-form expressions for the exponential and logarithmic maps in the Poincaré ball model were derived in Ganea et al. (2018). The exponential map at E B applied a tangent vector E T. takes the form
Ilulll 2
(xp = @c tanh
(16)
Vcllle||
where @. denotes Möbius addition. This expression formalizes the process of mapping linear dis- placements in the tangent space to points on the curved manifold along the geodesic through æ.
The logarithmic map, which serves as the inverse of the exponential map, is given by x +e y
2 tanh -1 (Viell - x tbe vll)
log (y)
(17)
Vidls
x ① yl
Here, log produces the unique tangent vector that corresponds to the geodesic displacement from x to
Another key operation in hyperbolic geometry is parallel transport, which allows one move tangent vector from T.B to T, along the geodesic connecting x and y, without altering its length intrinsic orientation. This operation formally defined as
X X
PT (v)
gyrly, - v,
(18)
where gyr(y, -x] is the gyration operator associated with Möbius addition, capturing the non- associative structure of hyperbolic translations Ungar (2007 2022). Taken together, the distance function, exponential and logarithmic maps, and parallel transport furnish the Poincaré ball with the full differential hiffereental-geomettiic apparatus needed for optimization and representation learning in spaces of constant negative curvature.
C ADDITIONAL RELATED WORK
Network science and geometric models The study of complex networks has long emphasized the interplay between efficiency. clustering, and degree heterogeneity. Small-world properties, for in- stance, can be captured through measures of global and local efficiency (I (Latora & Marchiori] 2001)
19

---

Published as a conference paper at ICLR 2026
Geometry itself may emerge from simple generative rules: Aipllcial-complee models yield clus- tered and modular graphs whose spectral dimension is finite, with curvature distributions obeying Gauss-Bonnet relations (Wu et a 2015). Empirical evidence further suggests that hidden met- ric spaces shape both connectivity and weights, as shown by the strong coupling between triangle formation and edge strengths (Allard et al. 2017).
A recurrent observation is the coexistence of heavy-tailed degree distributions and high clustering (Kim et al., 2007a] (eeranno al., 2008; [Amaral et al. 2000), structural combination that classical random-graph or lattice-based models cannot reproduce (Newman 2010 Cohen & Havlin 2010). Renormalization studies reveal that such networks are often self similai under box covering, they remain scale-free, exhibiting fractal organization (Song et al.) (005). This fractality can be traced to critical branching "skeletons," whereas supercritiica skeletons drive small- world behavior (Kim) et al. 2007b). Beyond standard box dimension, informatiiondimennsse
measures have been devel- oped to capture self-similar structure in real and weighted networks (Wei et al.] 2013). A recent synthesis unifies these microscopic and macroscopic exponents, demonstrating robust scaling laws across web, brain, and citation networks (Fronczak et al. 2024).
Such findings motivate latent-space models in which link probabilities decay with hidden distance (Papadopoulos et al. 2008; 2012). The hyperbolic framework of Krioukov et al. (2010) provides a particularly powerful resolution, reconciling hierarchical expansion with geometric growth. In this view, the popularity-simiian through angular ones, thereby unifying hub formation with community structure across diverse real- world systems (Kiioukov et al. 2010; Allard et al. 2017).
model represents degree through radial coordinates and similarity
Neuroscience and Graph Learning Neurobiological work has used graph theory to study how anatomical and functional connectomes are organized, translating empirical association matrices into graphs and then asking what principles explain their structure. Early cortical mapping of the cat showed that laminar hierarchy and nonmetric multidimensional scaling reveal coherent system- level topologies that are only partly explained by simple neighborhood wiring, suggesting additional long-range organizing factors (Scannell et al. 1995). This line of inquiry converged on the view that brain networks combine high clustering with short paths, a small world organization that can be quantified by clustering, path length, small worldness, and efficiency, and related to plausible cost- benefit trade-offss (Bassett& & Bullmore 2006). Reviews then systematized how to construct graphs from imaging data and how measures of segregation, integration, centrality, motifs, assortativity, and robustness jointly characterize architecture, while emphasizing that node definitions, edge weight- ing, thresholding, and null models strongly shape inference (Bullmore & Sporns) 2009 Rubinov & Sporns 2010).
Building on this foundation, learning based approaches seek predictive models that preserve neu- robiological interpretability. BrainGNN encodes region identity with region-of of-interest--warr con- volutions and performs region selection pooling to yield subject-level and cohort-level biomarkers that align with prior findings (Li et al. 202T). BrainNNExplainer complements this with a group- level explainer that learns a sparse global edge mask, improving diagnosis while revealing disorder- specific subgraphs (Cui et al. 2021). Dynamic representations extend beyond static functional con- nectivity: STAGIN learns spatio-temporal graph embeddings with attention across windows of con- nectivity (Kim et al. 2021); IBGNN proposes an interpretable backbone that couples edge weight aware message passing with a shared mask for disorder specific biomarkers (Cui al. 2022)); TBDS learns sparse directed acyclic graphs from BOLD time series and couples them to signed edge graph neural networks (Yu et al.] 2022); and DynDepNet replaces fixed connectivity with time varying dependencies that are optimized jointly with recurrent classifier (Campbell et al., 2023). Paral- lel developments adjust data geometry and architectural bias: R Mixup interpolates correlation and adjacency matrices along log Fuclidean geodesics on the symmetric positive definite manifold to stabilize training and improve generalization (Kan et al., 2023);; BrainRGIN adapts graph isomor- phism style aggregation, region aware pooling, and attention based readouts to predict individual differences with interpretable subnetworks (Thapaliya et al. 2025); and BioBGT encodes hub roles and modular structure within transformer by combining diffusion based node importance with module aware self attention (Peng et al. 2025).
Recent benchmarking advises caution when applying message aggregation functional connec- tomes. Broad comparisons indicate that aggregation can underperform strong non-graph basclines as graph density increases, while hybrids that separate global vectorized patterns from localized
20

---

Published as a conference paper at ICLR 2026
graph structure provide competitive accuracy and clearer interpretation (Han al. 2025 In paral- lel, work on hyperbolic embeddings of multilayer networks, including formulations in the Lorentz model. points toward latent geometric explanations, although current treatments remain focused on static graphs and often rely on stochastic block models to encode topology (Guillemaud et al. 2025)
D THEORETICAL PROPOSITIONS & PROOFS
Setup. Fix an integer I. 2 Let the common vertex set be
V = JÚHÚ Ú. -
1), outputs O, and bias nodes B (6("),... ..... B('.). Let T be nonempty index set of time steps (epochs). For each E T, the parameter snapshot is a directed (or undirected as well), edge-labeled graph
disjoint union of inputs I, hidden layers He (E 1,......
G. = (V. Fs, 0t).
where Es S
is the edge and 0, Et time t. The family (Gi)eeT is called a trace.
assigns real weight to each edge present
Definition 1. A family (Ft))TT with ypt V -> V bijective for each is a snapshot automorphism family for the trace (Gi)eT if. for every t E T.
1. Edge preservation at time t:
(u, v) E E
(ellu),Ge(()) Es for all E.V.
2. Node label preservation at time t: input nodes, output nodes, and bias nodes are fixed points; that is,
(i(i) = Viel,
(.(0) = YoE (b) = Ab
(Lim et al.
2023
Definition 2. Let T denote the set of all traces the fixed vertex set V. An encoder is map
F:T Z
into a representation space Z equipped with a (left) group action of the product group Herr (G.) on Z. The encoder is called snapshot-wise permutation equivariant if. for ev- ery trace (Gt)ET and every snapshot automorphism family (PP))t with se E Aut(G.).
K((pt-Ge)eer Ge)ee)) = d(pi))eT) F(GG))ET).
When Z carries index structure inherited from V at each (e.g., per-node per-edge features over time). p(((()))) is the natural permutation of those indices, acting componentwise in If. in addition, readout P: Z + Y is permatation-imvarian with respect to p. then P F is invariant under all snapshot automorphism families (Bronstein el al.) 2027).
V belongs to the snapshot automorphism family at time t if and only if it fixes all input, output, and bias nodes and permutes only hidden neurons within their own layer; equivalently, By membership, this is exuctly the class of permutations that preserve the time-t edges:
Proposition 1. For any epoch E T. per-snapshot permutation 9t
(u, E. + (pr(4), ((()) E, for all V.
Proof. FiXLET.
(=) If spr belongs to the snapshot automorphism family at time I, then by node-label preservation, it fixes I. O, and B. Moreover, In an MLP DAG with edges only between adjacent layers, the layer index E(h) of a hidden node equals the maximum length of directed path from input to h. Since permutations render this maximal path length unchanged under se i.e., (h)) {(h) which implies that &e maps each hidden layer itself.
graph isomorphism of G, it preserves lengths of directed paths.
(+) Conversely, Pc fixes I, O, and B, maps each hidden layer to itself, and preserves E,, then satisfies the snapshot automorphism conditions at time by definition.
21

---

Published as a conference paper at ICLR 2026
Proposition 2. For any trace (Gr and any family of per-snapshot permutations (Hh)er with 2/ E Au) at each t, the GTH-GML encoder satisfies:
1. Snapshot equivariance: at every time i, relabeling the snapshot by the relabels the time-t. output in the same way
2. Previous-step invariance: the temporal transition from step to t invariant any per-snapshol permutation applied at
Consequently, the full encoder is equivariant to di the current step t, while the update from to is unaffected by any admissible relabeling at
Proof. 1. At snapshot the parameter graph is G2 EV, 0z). Each node u E V carries raw node features
xr(u) = [((),, r(u), Sabs(u). Sagn (een(u)]
where ((u) is the layer index, 7(u) E (in, hid, out, bias} the node type, and Sabs Sugn are standard- ized incident-weight aggregations (as defined in Sec. 3.2) computed node from the coalesced signed weights at time f. Each edge (u, u) E Es carries raw edge features
(u iss v), Al(u,
encoding whether the edge originates at bias node and the layer difference Af(u, u) {(v) - l(u). By construction, these descriptors are structural: they do not depend on arbitrary node indices and invariant under within-layer permutations that fix inputs/outputsbiiass.
Group action on features. Let th E Aut(Gr) be any per-snapshot permutation (automorphism) at time f.. Its action on the node and edge features is the natural reindexing:
[Wa((re] vi(()) (z()),
[xbx(e))] (e(o). oi()) (u,
Equivalently, if Pv, 10, 10 is the permutation matrix induced by th, nodes and Pv. Ps. @ Ps. the induced action on ordered node pairs, then in matrix/tensor form
(x,) = Pg. St.
(e((e)) = Pv. Ct:
(where the second equality is understood as reindexing the first two modes of the edge-feature ten- sor). The values of f. Eabe, Segn and s s_bias. Ae] are preserved because t/, fixes node labels for / U and only permutes hidden nodes within layers; hence and 7 are unchanged, and the incident-weighh aggregations at move (u) without alteration of their numeric value.
Neighborhoods and reindexing. Write (G == {v E V (u,v) Eg } for the (directed or undirected) out-neighborhood time t. Automorphism implies (i((((u)) (N)())) so any neighbor-win wise aggregation satisfies
Σ ((v(())
Σ (V.
f(v)
Σ ((b((u)). vEN.(M)
and likewise for means and other symmetric reductions.
Equivariance of the hyperbolic graph attention block. Let HGAT, denote the per-snapshot hyper- bolic attention block at time f, with shared parameters We supplied by the temporal kernel. Its forward map can abstracted as
(x), HGAT, e6; Wi).
where X, E R V are tangent embeddings and 2 E (B.+)IVI are hyperbolic embeddings. Internally, the block computes nodewise (tangent) projections, hyperbolic queries/keys/value shared Möbius-linear maps, attention logits that depend only on index-symmetric quantities (e.g., (z (u), (x)) and eu(u, v)), neighbor-wise sofumax normalizations, and Mübius-weighted sums over (u). All these operations commute with the reindexing induced by th, Concretely, letting
lye 9dd,(), (z())) (u v), Xx(u). X((r)),
Ouw
22

---

Published as a conference paper at ICLR 2026
one has (o) Luv and hence (1)()))) Quvi messages and updates therefore reindex accordingly. It follows that there exists the induced permutation Pe. such that
HGAT, ( (a())) (ex); Wi) Ps. HGAT, (t, ct: Wt),
i.e. the per-snapshot block is permutation equivariant in our setting on both the tangent and hyper- bolic branches.
2. Fix / E and let da E Aut(G(-1) act on node indices with permutation matrix Pa. Let X E
R V xd. and Z E (Bd+ VI denote the tangent and hyperbolic node embeddings produced at time
t 1. The action of v on embeddings
(Pp. (()
Xx-I(v= (Poze-)(()) - z=1(( +t-1((-'(w))
Euclidean (tangent) pooling is permutatioon-nnvraaae invariant. Define the pooled tangent vector(s)
(t-11)
1
Σ
Xi-1(u) or layerwise Pt (t-1)
Σ
X1-1(u).
V
Hel
WEV
uEH.
Because & fixes JUOU B and only permutes hidden nodes within layers, each He is preserved. Summation is agler-agnost;; hence
Σ (PoX = p0--1. V
Σ (Px = Pe WEHe
(t-1) Al.
Hel
NEV
Einstein gyromidpoint aggregation (Klein computation) and permutation invariance. Since linear averages not geometrically compatible the Poincaré ball, we aggregate values using the Ein- stein (gyro) midpoint aggregation, which we restate here. The computation performed the Klein model, where weighted barycenters admit a closed form. Let Bd Kd be the Poincaré-to -oeeen map.
2/e2())
((=00)= =
20 E E
(19)
1+ 1c20000022
and n nー Kd +
+B B its inverse,
(t)
k (y(00) = Vc
n E Kd.
(20)
0p
We also denote by ()) = 1/V1-- || the Lorentz factor. For node v and head r at time t, the Klein-space barycenter of the incoming neighborhood is
Σ
airil ((((p((tt))) sfoll (ol
16 ((mm(:t)
az (w-w)EEE
(21)
Σ
an (1((11 (1111))))
0A (u-
We then map back to the Poincaré ball and linearize at the origin for the residual path:
m (m))
15- (m((m())))
Bd
((n))
loga m. m((rt))) E ToB
(22)
and average across heads,
H mira
M(4)
(r.t)
(23)
H
Per-snapshot permutation equivariance of the Einstein aggregator Let the E Aut(G,) be any per- snapshot permutation. Automorphism implies {u: (u-v) E E& is sent to {u' (u' + +e(v)) e F.)) with w/ = (u), and snapshot equivariance of attention yields (r.)))
(((())))())) a Apply-
ing equation 21 at node (e) and reindexing the sums by u' = (1) gives
((r() En au-zo (N((((1))) ol (n.))
x ((m(re)))
(rot)
En
((r))) autto (N(( ((()))
23

---

Published as a conference paper at ICLR 2026
Thus the Einstein weighted midpoint is symmetric in the multiset of weighted Klein vectors and therefore commutes with the reindexing induced by Vh mapping back by H and logo preserves this equivariance. Consequently, (t) is exactly the Pv. -reindexing of M
(w)
At time 1. let the hyperbolic node embeddings be Zı-1 (2l--1)) JEEV C B and the tangent JuEV
embeddings be - = 1 ( (u) C RR We summarize these for the temporal kernel using (i) Fuclidean means the tangent space and (ii) Einstein gyromidpoints Klein space, consistent with Eqs. equation T9-equation 23
Define the global (or layerwise) langent summaries
((t-1)
= VV Σ
(i-)(u),
(t-1) Pt
1
Σ Xi-I(u). ueH,
Hr
WEV
These are permutation-invarian because they are simple averages over sets preserved by automor- phisms.
Let and x- be the Poincaré Klein maps in equation With uniform weights, the global Klein barycenter is
equation 20 and >(v) 1/v 1lyl2
Σ WEV
(x(E
m(2u 1221-1))
(Z-1)
2--1 n (x(2L1)))
E weV
(1-()1111))
For layerwise pooling, restrict the sums temporal GRU/MLP operating in the tangent space, we finally use
He obtain zt-1. For compatibility with the
(2)
2 = logo (z+-),,
2 = logo (2(2)
Let vb e Aut(Gr-1) with induced reindexing (Po2(t-1)), = 2(111) In the global Einstein mid-
point, the numerator and denominator are sums over the multiset (s(((2l "))) x( (t-1)))) ")] ")) which are merely reordered by v. Hence
2 (Pg: 2 (z(((((11)),
and applying logo preserves equality: ( ( The same argument holds layerwise because v maps each He to itself. Therefore, the hyperbolic pooled summaries , and (2(2)
(2t-1he he are invariant to any admissible permutation time 1.
Let X,-1 be the tangent embeddings at (logo(²-~))) and let ((t-1) (or {p) ( denote their mean pool(s). Since p(t-1) is an ordinary Euclidean average and layer sets are preserved by automorphisms, (tt-1) is permutation invariant. The Evolve-GCN-H style kernel then consumes only agne-agnostic summaries,
(W,, w.) diw (GR ui-1)) (or with layerwise pools),
where U is learned hidden temporal vector, which proves invariance of (We, ue) to any per- snapshot permutation at time - 1. This completes the previous-step invariance argument, i.e, the temporal kernel renders the module invariant to permutations the previous time step (Bronstein
et al. 2021)
E
EXPERIMENT DETAILS
E. PROBLEM FORMULATION
We study dynamic parameter graphs extracted from the training of base multilayer perceptron
(MLP). A dynamic parameter graph is sequence (G. (V, F., W.))f of T snapshots,
24

---

Published as a conference paper at ICLR 2026
where V is the fixed node set (inputs, hidden units, and biases), E, is the set of undirected edges time E, and W, [wae (u, v) E Er} the associated signed weights. The goal is learn node embeddings {=] E Bd iEV, L 1=,..... T in the d-dimensional Poincaré ball such that:
(i) a Fermi-Dirac decoder (0(0 v) predicts adjacency labels, (ii) a signed regressor @ie estimates Wae and (iii) trajectories {2] ( ). evolve smoothly across time.
Supervision is provided on positive edges, with negative samples generated adaptively, placing the task in the sumi-supervised, temporal graph embedding setting.
E.2 ADDITIONAL DETAILS: CLASSIFICATION OF IMAGES VIA INR TRACES
Base INR architecture and training. For each image E (MNIST, Fashion-MNIST) we train shallow implicit neural representation (INR) to fit pixel intensities as continuous function of coordinates. The INR has three fully connected layers of width 32 with tanh activations, mapping [o, 1J2 (pixel coordinates) to grayscale value æ(u). Weights are initialized with Xavier initial- ization and optimized using Adam (learning rate 5 x 10 for up to 100 epochs, with early stopping if the PSNR of the reconstruction exceeds 40. This produces a sequence of checkpoints {0,,) for image i, where T, E [80, 100] depending on early stopping.
T2
Parameter-graph construction. At each checkpoint the INR parameters 0g are transformed into a signed parameter graph G.,::
nodes correspond to input, hidden, and bias units;
edges connect units across successive layers and from biases to their layer, labeled by the signed weight;
node features include layer index, node type, and scored sums of incident weights (ab- solute and signed);
edge features indicate whether the edge originates from bias node and the layer jump AL.
This yields a temporal trace {G T7 per image.
Trace dataset and splits. Each image contributes one temporal trace. Dataset splits are made strictly the image level to prevent leakage across time. We use 45,000 images for training, 5,000 for validation, and 10,000 for testing. All checkpoints from an image remain in the same split. Each trace contains between 80 and 100 checkpoints, ensuring sufficient temporal depth.
Meta-network pretraining. We pretrain Geometric Temporal Hyperbolic Graph Meta- Network (GTH-GMN) on the training traces. The backbone consists of L 4 Hyperbolic Graph Attention (HGAT) layers, each paired with one EvolveGCN-H temporal kernel, so that the num- ber of HGAT layers and temporal kernels always equal. Embeddings live Bd with dimension d 125 and curvature parameter -1. The optimization alternates between:
1. Euclidean updates of the attention kernels, regressors, and decoder;
2. Riemannian updates of node embeddings with trust-region control.
We use RAMSGrad with learning rate 10-3 and trust radius 0.1.
Losses and curriculum. The training objective combines:
Fermi-Dirac link prediction loss (binary cross-entropy):
signed weight regression (L2 loss on log-magnitude + sign classification);
quadratic prior on the learned distance exponent;
temporal smoothness penalty on successive embeddings;
pairwise logistic ranking loss semi-hard negative pairs.
25

---

Published as a conference paper at ICLR 2026
A gentle curriculum is applied: the ranking margin increases epochs, and mining hardness is annealed to prevent early saturation.
Representation extraction. During pretraining, edge embeddings within each checkpoint are pooled into a snapshot vector by mean aggregation. For downstream classification, snapshot vec- tors are pooled temporally (mean across checkpoints in the trace) to yield a fixed n2-dimensional representation per image.
Classifier and evaluation. An MLP classifier (n 128 + 10, with ReLU) is trained on these representations using cross-entropy loss. The classifier trained on the training set, tuned on the validation set, and evaluated on the test set. Accuracy the test set is reported in Table I
E.3 ADDITIONAL DETAILS: PREDICTING DNN GENERALIZATION FROM WEIGHTS
We evaluate checkpoint traces from implicit neural representation (INR) trials trained for CIFAR- 10 classification. For each trial i collect the ordered checkpoints {0..) together with the corresponding per-epoch test accuracy Vi.t- Using the architecture metadata stored with each trial, reconstruct shape-compatible network skeleton and convert every checkpoint to a parameter- graph G. where nodes and edges encode their respective features. No image content is used; supervision solely the scalar accuracy aligned to checkpoints. Each trace is processed by the
Predicted vs Actual Accuracies
0.8
Cendallle 5.866
0.7
Prodicted Accursty
0.6
0.5
0.b
0.22
0.3
0.4
0.5
0.5
0.7
0.8
Actual Accuracy
Figure 3: Predicted versus actual generalization performance on CIFAR-10 checkpoint traces. Each point corresponds to model trial at given epoch, where predictions are obtained from the learned
GTH-GMN embeddings. The strong monotonic trend (Kendall's = 0.846 + 0,004) indicates that hyperbolic embeddings capture meaningful structure in the parameter dynamics, even if absolute correlation lags slightly behind NEN baselines.
encoder introduced in the previous section over fixed-length, stride-rolled windows or full traces (depending computational constraints). We adopt the last-step supervision regime: intermediate steps are used only to evolve the state, and supervision is applied to the final step target. Dataset splits made strictly by trial (not by checkpoint) prevent leakage across traces. For all experiments, we embed parameter-graphs into hyperbolic ball of dimension d E [80, 125), with four HGAT layers and four EvolveGCN-H temporal kernels of hidden size 128. Hyperbolic embeddings are optimized with the two-phase scheme described in Section 3.4 combining Euclidean updates for kernel parameters with Riemannian gradient steps for node positions. Training employs binary cross-entropy on Fermi-Dirac link logits, signed weight regression, prior on the learned exponent, temporal smoothness penalties, and a ranking curriculum.
We report Kendall's 7 for ranking fidelity, following Zhou et al. (2023a); Navon et al. (204)): Lim ct al. 2023). Results are shown Table 2 and illustrated qualitatively in Figure 3 Our method 6TH-GMMN achieves
0.846 + 0.004, lower than NFN baselines that learn end-to-end from raw weight tensors. This gap is expected, since NFNs exploit stronger task-aligned inductive biases while our approach compresses parameters into aermutation-invariian graphs and trains with ge- ometric proxy losses under last-step supervision, inevitably discarding certain accuracy-correlat cues. Nonetheless, GTH-GMN offers complementary benefit by jointly modeling link structure,
26

---

Published as a conference paper at ICLR 2026
signed weights, and temporal evolution in hyperbolic space, producing embeddings that robust, interpretable, and faithfull to training dynamics.
E.4 ADDITIONAL DETAILS: PREDICTING SINE WAVE FREQUENCY
In addition to supervised objectives, we employ SimCLR-style (Chen et al. 2020) contrastive regularizer adapted to temporal checkpoint traces. This setup treats the optimization trajectory of a neural network analogously to image in classification task, operating in "parameter-time space" rather than pixel space.
For each trial, two correlated views of the same checkpoint subsequence are generated by combining:
(i) Gaussian perturbations of node and edge features, and
iii) Random temporal masking, where subset of checkpoints in the subsequence is dropped with probability p = 0.5.
The encoder processes both views to produce pooled representations, which are learned using a con- trastive loss (Chen al.) 2020). Positive pairs correspond to augmented views of the same check- point subsequence (representing the same underlying function dynamics), while negative pairs are drawn from other trials. This auxiliary objective ensures that the learned geometric representation captures the intrinsic signature of the optimization trajectory.
E.5 ADDITIONAL DETAILS: WEIGHTS VS DISTANCE RELATIONSHIPS
Dstarce loloeee
Figure 4: Relationship between hyperbolic distance d and edge weights w on CIFAR-10 INR traces, shown as log-count hexbins. (Top Left, Bottom Left) Distance versus absolute weight |w| (at / E T), highlighting the concentration of large weights at small distances and the rapid decay at larger dis- tances. (Top Right, Bottom Right) Distance versus log- magnitud log |w| (at t' E T),, which reveals an approximately linear inverse trend, consistent with power-law dependence of weight magni- tudes on hyperbolic distance. Together, these plots illustrate that the hyperbolic geometry learned by GTH-GMN aligns distances with functional weight strengths, capturing both edge existence and graded magnitude geometrically coherent manner.
To probe the strength of the geometric relationship captured by GTH-GMN, we visualize the align- ment between learned hyperbolic distances and underlying edge weights on the CIFAR-10 INR traces. Figure 4 reports hexbinned joint distributions of hyperbolic distance against both |w] and log [w]. The results reveal a clear inverse correlation: large weights tend to concentrate at smaller hyperbolic distances, while edges at larger distances exhibit systematically lower magnitudes. On the logarithmic scale, the relationship is approximately linear, consistent with the power-law pa- rameterization employed in our signed edge-weight regressor. These findings support the claim that temporal hyperbolic embeddings faithfully encode functional structure: the geometry reflects not only the presence connections but also their graded strengths across training, thereby grounding the link between parameter values and hyperholic distance.
27

---

Published as a conference paper at ICLR 2026
F
HYPERPARAMETER INVESTIGATION
Table 5: Architecture and data settings for our Experiments.
Component
Hyperparameter
Range Value
Manifold
Curvature |c]
fixed to 1.0
Embedding dimension d
80 125
HGAT (per layer)
# layers L heads H
4
4
(tuned)
Head temperature
learned, init 0.5 - 1.0
Residual gate
learned scalar E (0,
Geodesic trust radius Tτ
0.05 0.15
EvolveGCN-H
temporal kernels
4 (equal to HGAT layers)
GRU hidden size
64 128
Input to GRU
mean pool in To (prev. snapshot)
Graph construction
Node features
(l, t, sabs s
Edge features
(is bias, A
Temporal window
Sequence length K Stride
64
T (full trace, T'E [80, 100))
4
Contrastive views (SimCLR-style)
Feature noise (Gaussian) Temporal masking
σ = 0.1 - 0.3
drop prob. 0.3 0.5
Temperature (NT-Xent)
0.1 0.5
Hyperparameter Investigation. Tables 5 summarize the settings used our studies. Architec- ture choices fix the manifold curvature at |c0-1.0 and sweep embedding dimension d E [80, 125]; HGAT uses L=4 layers with H E [4, 8). learned head temperatures, a learned residual gate, and per-layer geodesic trust radii тe [0.05, 0.15]. EvolveGCN H mirrors the HGAT depth with GRU size 64-128, while the parameter graph encodes (t, t, sabs sSen) at nodes and (is (is_bias, AE) at edges. Temporal windows use K E [64, T' with stride 4-8, and SimCLR-style augmentations vary fea- ture noise, temporal masking, and NT-Xent temperature. Optimization (Table 6 employs Adam for Euclidean parameters with Ir E [1 1x1003 30 31, weight decay 10 6-10 StepLR, and 30-50 epochs; node positions use Riemannian AMSGrad (RAMSGrad) with matched learning rates, trust radius E [0.05, 0.15). (31, f2) (0.9, 0.99-0.999) and conformal preconditioning. The FD decoder learns radius R and temperature 7 from initialized ranges, negatives mix same-layer and uniform samples, and losses combine FD BCE, signed weight reconstruction, sign BCE, exponent prior, temporal smoothness, and pairwise ranking with reported weights; the same losses are reused in only pass with (*) detached. For throughput, we cap positive edges and chunk FD/distance computations as listed.
G ALGORITHMS
Overview of Algorithms. Algorithm 2 converts a single MLP checkpoint into permuta- tion-invariant parameter graph by instantiating input, neuron, and bias nodes; coalescing directed parameter incidences into undirected edges with context and signed labels; and standardizing node- wise strength features to form G. = (X,, E, W.). Algorithm | then updates node positions in- trinsically on the Poincaré ball: Euclidean gradients are rescaled to Riemannian ones, Adam-si style moments accumulated, a geodesic trust region clips the tangent step, the exponential map applies the update with a safety projection, and first moments are parallel transported. The two procedures independent and can be composed sequentially within each training step.
28

---

Published as a conference paper at ICLR 2026
Table 6: Optimization and loss settings for INR-trace image classification. Ranges correspond to grid or random search over hyperparameters.
Component
Hyperparameter
Range Value
Euclidean params
Optimizer / I.R
Adam, === x 10-3 10-3
(kernels, heads)
Weight decay
10-0
10
LR schedule
StepLR (step 10 20, 0.6 0.8)
Epochs
30 50
Manifold params (node positions 2)
Optimizer
RAMSGrad (Riemannian)
LR / trust radius
Ir =r= 1x 3 10 0.05 0.15
(31, (2)
(0.9, 0.99 - 0.999)
Conformal precond.
divide by x
Link decoder
Fermi-Dires radius R
init 1.5 2.5 (learned)
Temperature
init 0.3 0.7 (learned)
Negatives
Neg. multiplier
3 10x positives
Same-layer ratio
0.3 0.7
Losses (Euclid pass)
FD BCE (links)
weight ramp to 1.0
Signed weight recon
Awree 0.2
0.6
Sign BCE
Asign 0.2 - 0.4
Exponent prior
An
10 10-2
Temporal smoothness
smooth = 0.2 0.6
Ranking (pairs)
Arank 0.5 1.0, margin 0.2 0.5
Losses (z-only pass)
Same terms
as Euclid pass; (*) detached
Practical caps
Pos. edges per step
10,000 20,000
FD distance chunks
20,000 40,000 edges chunk
Optimization Curriculum (FD) decoder acting Poincaré geodesics (Nickel & Kiela 2017 2018 Wen et al. 2024):
We map hyperbolic geometry to link probabilities with Fermi -Drac
3 (u, v) (1 (1+ exp (de()))) ,2(5) )))
(24)
with learned radius R and temperature T. Training uses positives F, and dynamic negatives from mixture of same-layer and uniform samplers (mixture capped 50%). The FD loss is binary cross-entropy,
LED E log (i v)] ((,,)E
[log(1 - (1 v))]
(25)
To capture edge magnitudes and signs, we add the signed regressor of Section
3.3 For (u, v)
E
with weight w and prediction ((tt) from equation 13 Goyal et al. 2018
Pareja et al. 2019).
Lwrec ((,,)EE [1 w [pp]
(26a)
Laign B(a,yEEE- Yau log of (() + (1 vuv) log(1 o(s
(26b)
with (1,2} and Yoe Kiwn > 0). We regularize the exponent via
La = E (,, [Qww
(27)
encourage smooth trajectories (Goyal et al. 2018
Yang et al.
2023
Lsmooth
Σ
d(2
(1-1)
(28)
M
iEV
29

---

Published as a conference paper at ICLR 2026
and enforce distance distance-magnitude monotonicity with logistic pairwise rank loss (Nickel & Kiela
2017].
Fankk = Bu,tul, w logg(1 + exp(dc(2))) 2 de(z!] 2 + 7))
(29)
The total objective is
( = >ED Awreehwiee + Asign Sagnn + Aafa + Amooth Smooohh + Arank franks
(30)
with gentle curriculum: anneal 7 and harden mining over epochs.
Training runs in phases: Euclidean pass updates kernels, regressors, and decoder; Rieman- nian pass updates node positions 20) the Poincaré ball. With conformal factor X
the Riemannian gradient rescales the Euclidean one (Bridson Haefliger
1999
Bonnabel 2013
Nickel & Kiela 2017 2018):
Riem
Fuc L.
(31)
We maintain first and second moments and take an Adam Adam-style tangent step (Bécigneul & Ganea 2019 Sakai & liduka 2020):
4/0 = 1
m
(32)
+
geodesic trust region clips the move, its Euclidean norm. With
2 (t) = exp c0 (4
de = =d 24 (y)
(33)
we scale ( by T/ and accept the intrinsic update
(+1) = exp, (xp_00((4
(34)
followed by projection into the ball. First mosttt vectors are then parallel transported:
m
(++1) PT =. (2--1 (m) (").
(35)
The manifold step optimizes a o-ly objective mirroring equation 25 equation 29 with (*) de- tached. This decouples the hyperbolic graph attention head's stability from embedding motion and enforces controlled geodesic updates near the boundary.
H HYPERBOLIC EMBEDDINGS OF INRS
We visualize hyperbolic embeddings of parameter graphs from a 25-layer INR-MLP at several checkpoints
5] Embeddings are learned with 4 HGAT layers and EvolveGCN-H kernels (tangent dim. 128) in d = 10 and d = 25, then projected to the Poincaré disk via PCA for display. Points colored by layer, and edges are gray. Early snapshots show mixed layers and long inter-layer edges. As training proceeds, layers separate along angular directions, intra-layer bands tighten, and high-strength nodes drift radially inward, indicating hub formation and growing specialization. The smooth drift of clusters across checkpoints reflects temporal kernel meta-evolution rather than abrupt reconfiguration. Because these plots are PCA projections from a higher-dimensional ball, exact angles are not preserved; nevertheless, the progressive stratification and the shortening of within-layer geodesics are consistent across panels and align with the learned distance-magnitude coupling.
I COMPUTATIONAL COST ANALYSIS
We analyze the time and space complexity of GTH-GMN in terms of the size of the parameter graphs and the temporal window length. Let a parameter graph snapshot G, have nodes and E edges. Embeddings live in d dimensional Poincaré ball, the hyperbolic encoder uses H attention heads and L HGAT layers, the temporal kernel has window length K and GRU hidden size h, and each training batch contains E+ Faach positive edges.
30

---

Published as a conference paper at ICLR 2026
Algorithm Intrinsic Riemannian Optimization of Node Positions on the Poincaré Ball
Input: Snapshots {Gr)ī=ı; initial positions {2,"); moments m; (8) trust radius т; tolerance E; curvature c
0, o O; learning rate 7
(t)
Output: Updated positions {=1 {=(")]=
fort = 1 to T do
> after Euclidean parameter update
2: Detach ( compute Euclidean gradient Bug & for losses equation 25
equation 29
=(()
3: Convert to Riemannian gradient with conformal factor X 1-
Riem
Ruc L
4: Update moments (Adam/AMSSrall (Bécigneul & Ganea 2019
Sakai & liduka 20200:
mi + 3im? + (1- (1-31)) V
Riem C
2(
u/ B2 (0) + (1 B22) V Riin L)
5: Compute tangent step:
m (1)
A = -
Vus 4
6: Trial update and geodesic distance:
try = exp_ (A.)
d: = de(z)) 2 ,
7: Trust scaling: 8; = min 1, 7/4 + e)}:
A, + i
8: Accept update and project inside ball:
(++1)
exp co (A.),
(++1) + 1واا (2)g(22+1)/) (1+1)
9: Parallel transport first moment:
(++1)
PT 1o - (0
m;
10: end for
Time Complexity of the Hyperbolic Encoder. For a single HGAT layer G., hyperbolic queries, keys and values are obtained by applying the affine maps 2w. at each node and head. Each such map performs one logarithmic map logo, matrix-vector product W log, (z), one ponential map expo- and a bias transport plus exponential at 2w all on d dimensional vectors. The cost per head and node is therefore Odd d), and aggregated over all nodes and heads this yields
Costokv O(HNd).
(36)
Attention logits and softmax normalization are computed per edge and head. For each directed edge (u, v). the Poincaré distance de (gu, k.)) requires a constant number of inner products and norms in Rd. so (d) operations, followed by scalar exponentiation and normalization. Einstein gyromid- point aggregation in the Klein model combines value vectors from the incoming edges of each node and maps the result back via x= and logn- which again contributes ((d) per edge and head plus (dd per node and head. Summed over all edges, nodes and heads, the attention and aggregation stage costs
+ Cost = O(H))+HN())
(37)
The total cost of a single HGAT layer therefore
CostHGAT, layer OH + HEd),
(38)
and a stack of L layers has
COSTIGAT O (TH
(39)
31

---

Published as a conference paper at ICLR 2026
Algorithm 2 MLP Parameter Graph (single snapshot at epoch ()
Input: Layer widths (rio..... nr.); weights (Wi)é=1 biases {bx)e=1 Output: Snapshot G2 = (X,, E, W.)
1: V < (1))
input nodes
2: for L 4 to L do
3: add bias node 6 to
4: add neuron nodes 8
to V 5: set (f(w), for all newly added u
>> layer index, type
6: end for
7: A_
< 0
D directed edge bag with context + weight
8: for 8 1 to do
9:
for + 1 to - do
for q
n2 do append (1p (f-1))
11:
(6) + iq
(is bias = 0, 0, = 1), w t- We( to A
end for 13: end for
12:
14: for 1 Tlg do
15:
append (b(e) - iq (9)
(is bias 1, A = 0), w + belg)) to A
16: end for
17: end for
18: M < empty map from unordered pairs {u, v} to running sums
19: for all (u-vv, 6, w) E A.,, do
20: k =
21: update M [k].summcce + [[k]. += w; M M[k]eent += 1
end for
23: E <- 9.
W, <-
for all
i
[u, r) in .M do
25: Euv <- M sum__cs /tt/eexx//M/// ent
D coalesced context (mean)
26: w f- M [k] sumk.ssumm..... sum w Mk]..emt
P coalesced label (mean)
27: insert (u, u} into E with context euvi set Willu, o}] +- whv
28: end for
29: for all V:
sub((u) E Je Wilfu, ubl: 5 + Em.Luler Will, vt]
30: Pabs J Jass <- mean/std of s (u) V}
31: Megn; Usgn mean/std of (g (u) u V
32: for all u E do
(4)
Mabs
33:
gabs
abs
abbs
0,
otherwise
(u)
Msga
sgn 0
34:
(u)
Osgn
0,
otherwise
35: end for
36: X, f {Iu [[(u), r(u), Jabs (u), (u), (w)) uEV)
return Gt (X,, E, We)
In the regimes we study, E is much larger than N and d is moderate, so the edge term O(LH Ed) dominates and the cost effectively linear in the number of edges. The hyperbolic operations (exp, log, distance and parallel transport) are all (d) vector calculations and do not change the asymptotic order relative Euclidean graph attention network with the same (L, H, d).
Time Complexity of Temporal Kernel Evolution. Temporal evolution follows the EvolveGCN- scheme (Pareja et al. 2019),, where each HGAT layer maintains GRU with hidden size h over window of length K. A single GRU step processes pooled d dimensional summary and costs (hh hd). For K steps and L layers, the total temporal cost
Costump (T.K() hd)).
(40)
32

---

Published as a conference paper at ICLR 2026
Palscved
Lyye
Liver
Lryen
Lyer
Leee
Lyye
Pelncarn (ALA Barm
Kancare ole (PCA to
Lger
Lyye4
Liyer
Upera
Traerh
Trym
Lyer
Polncand (PCA 25.::
Falncard Dbe IPCA tron 25
Liee's
Lean
Figure 5: Hyperbolic embeddings of parameter graphs from a 25-layer MLP trained as an INR on CIFAR-10, 10, shown at checkpoints. Embeddings are learned in d 10, 25 dimensions with 4 HGAT layers and 4 EvolveGCN Nodes colored by layer index, with edges in gray, illustrating the progressive reorganization and separation of layers over training.
kernels (tangent dim. 128) and projected to the Poincaré disk via PCA.
This term is linear the window length and independent of E. For the configurations used in our experiments, H Ed dominates K (h² fid), so temporal coupling is lower order than the HGAT edge processing.
Time Complexity of the Link Decoder and Signed Regressor. The Fermi-Dirac decoder and signed weight regressor operate on positive edges and their sampled negatives. For each edge we probability, power law magnitude, and
compute Poincaré distance dz(zu, 20), sign logit from a tangent inner product. Each of these involves a constant number of d dimensional vector operations, the per edge is Old). If Frunc positive edges present in the batch and
33

---

Published as a conference paper at ICLR 2026
negatives sampled per positive, then
Costaec = (d(1 - k.) Fotcct))
(41)
In practice Ftt is capped (for example at 10a to 10 and distance computations are per- formed in chunks of at most 40 000 edges to keep buffers bounded. This term is therefore linear in a controlled edge count and does not affect the overall scaling in and K.
Two Phase Optimization Overhead. Each training step consists of a Euclidean pass and Rie- mannian refinement pass. the Fuclidean pass we backpropagate through the HGAT stack, tem- poral kernels and decoders, updating all Fuclidean parameters. In the Riemannian pass we update only the node embeddings on the Poincaré ball. This involves rescaling gradients by the confor- mal factor X applying RAMSGrad updates in the tangent space, mapping back via the exponential map, and maaallel-trassorrinng optimizer moments, all on d dimensional vectors for each node. The cost therefore
CostRiem = O(d)).
(42)
We reuse the same sampled edges and losses and do not recompute the attention stack or temporal kernels in this phase, so the two phase scheme introduces constant factor overhead over a single Euclidean training pass with the same architecture while preserving the linear dependence on E and K.
Space Complexity. Memory consumption is dominated by current node embeddings, attention buffers and temporal states. Storing node embeddings requires ((Nd) memory. Attention coeffi- cients and intermediate messages kept for the current batch of edges, which costs O(EH Ed). The temporal kernels maintain GRU hidden states and pooled summaries of size O(Lh + Kd). In addition, the decoder holds temporary buffers for at most 40 000 edges at a time for distance and loss computation, which is bounded by the edge caps discussed above. Crucially, we never store embeddings for all checkpoints in trace: only the current temporal window and the recurrent state are retained. The overall space complexity is therefore
Space = (NN + EH + Kh),
(43)
rather than O(TNd) in the length T of the full optimization trace.
Scaling Behaviour. Collecting all components, the full time complexity of one training step is OLHND + LHEEd + O(LK(1² + hd)) + (d(( + ()) Fatch)
O(Nd)
(44)
HGAT encoder
temporal kereels
decoder
Riemannian refinement
In our setting, the parameter graphs satisfy E > N and the embedding dimension d moderate. Under these conditions, the edge term LH Ed dominates the node- delendentt terms LH Nd and Nd. Moreover, Ettcch is capped in practice, so the decoder contributes only a controlled linear factor in E.
Under these assumptions, the effective leading dependence simplifies to
EdH + OLLK(h² + hd)) + O(d(1 + (d) K)
(45)
which is linear the number edges E and in the temporal window length K. Hyperbolic opera- tions and the Riemannian refinement introduce only Odd) costs per node or edge, adding modest constant factor relative to a Euclidean graph meta network with the same (I., H.d) d) but not altering the asymptotic scaling in model size.
J ABLATIONS
Design of the ablations. The full objective can be viewed core prediction loss equipped with set of geometric and temporal regularizers. The core part consists of the Fermi-Dirac link loss LED. which teaches the model which edges should exist, and the signed regression head Lwee + Esiaas which aligns the predicted weights with the ground truth values. On top of this, the slope prior L, and the ranking loss Lrank encourage monotone relation between hyperbolic distance and weight
34

---

Published as a conference paper at ICLR 2026
Table 7: Ablation scenarios and their relation to the three guiding questions. Each row indicates which loss terms are active in the corresponding variant.
Variant
Question
LTD
Lweec
Lsign
La
Lrank
Lsmooth
Full objective
No ranking loss
11)
V
V
V
V
X
V
No ranking no smoothness
V
V
V
V
X
X
No signed regression head
X
X
X
magnitude, while the temporal smoothness term L smoohh prevents node embeddings from moving
abruptly between consecutive checkpoints (equation 30
In the ablation study we retain LTD so that the model continues to learn meaningful link structure, and we selectively remove geometric terms examine their individual contribution. Removing frank tests whether enforcing distance-magnituee ordering matters primarily for interpretability or also for INR accuracy. Removing both Lrank and fmmooth probes the importance of temporal regular- ity for maintaining coherent geometry across the optimization trace. Finally, removing the signed regression head (Ewree. Csign. and C.) asks whether explicit supervision on magnitudes and signs is required to bind weights to distance, or whether link prediction alone can sustain the geometric cou- pling. The following ablations on MNIST and Fashion-MNIST evaluate these questions in detail. We conduct ablations the INR-trace image classification task because it is the setting in which all components of the composite objective are jointly active: link prediction, signed weight regres- shaping the
sion, distance-magnitude priors, ranking, and temporal smoothness all contribute learned hyperbolic geometry. This task provides long, dense temporal traces (80-100 checkpoints per image), rich variation in weight magnitudes and signs, and a strong supervisory signal, making it the most sensitive and diagnostic environment for assessing how each loss term influences the geometry. Moreover, this is the setting in which our temporal and geometric meta-network attains high classification accuracy, ensuring that ablations probe a regime where the model operates at full capacity rather than in a degraded or low-signal setting.
Ablation questions.
We design our ablations around three fundamental questions (Table
Does the model still produce meaningful geometry if is never explicitly told that stronger weights should be closer? (No Ranking Loss)
2. If neither local geometric ordering nor temporal continuity is enforced, does the temporal hyperbolic geometry collapse? (No Ranking and Temporal Smoothness)
3. Can hyperbolic distances encode magnitude and polarity purely from link prediction? (No Sign Magnitude)
Each ablation scenario disables specific subset of loss terms in order answer exactly of these questions while keeping the remaining components of the objective intact.
Ablations on geometric objectives (MNIST Fashion-MNIST), We examine four variants of GTH-GMN on the INR classification task in order isolate the role of each geometric loss. The variants are:
The full objective.
A model without the ranking loss (Aynnk = 0).
A model without both ranking and temporal smoothness (Arank Asmoceh 0).
A model without the signed regression head (dropping Lwrecs Leign and L., while keeping the Fermi-Dirac decoder).
For each condition we report Kendall's 7 between predicted and actual accuracies, and the fitted slope of the distance-log [u] relation (Table
35

---

Published as a conference paper at ICLR 2026
Table 8: Ablation Study Table
Condition
Metric
MNIST Value
Fashion-MNIST Value
Full Objective
r
0.946
0.935
Slope (b)
00.677
-0.322
0.811
0.797
No Ranking Loss
Slope (b)
-0.473
-0.189
f
0.765
0.764
No Rank/Smooth
Slope (b)
- -0.365
-0.061
0.
0.726
No Signed Head
Slope (b)
00.211
+-0.125
Full objective. With all losses active, the model exhibits the strongest alignment between its hyperbolic representations and the underlying predictive signal: Kendall's т reaches 0.946 on MNIST and 0.935 on Rashion-MNIST Geometrically, the fistance-weighee relation is sharply pressed. The inverse slope is steepest on MNIST (-0.617) and moderately steep on MaSHIon-MNIST (-0.322), reflecting the dataset-specific geometric requirements. The data shows narrow concen- tration of points around this trend. Large weights consistently lie close to the origin, weaker weights move outward, and the trajectory across checkpoints remains stable. This is the regime in which the geometry, temporal evolution, and predictive signal remain coherent, forming the reference structure against which the ablations can be interpreted (Figures
Predicted cted Actual Accuracies N94ST 1.41
Predicted Actual Accuracies FASHIEN
C8
Figure 6: Kendall 7 over the full objective for the INR classification task on MNIST and Fashion-MNIST datasets. GTHI-GMN with full objective achieves Kendall's of 0.946 MNIST and 0.935 Fash- ion MNIST datasets.
No ranking loss. Removing Crank has negative effect on both accuracy and the learned geometry. Kendall's 7 drops to 0.811 in MNIST and 0.797 in Pashion-MNIST reflecting that the model retains a broad performance gradient, but loses the finer monotonic structure present in the full objective (Figure (). The fitted slope of the distance-log [w] relation (Figure flattens significantly -0.4733 on MNIST and -0. 189 on Mashion-MNISTT, and the hexbins widen, indicating more frequent local inversions where strong and weak edges intermingle. The geometry remains meaningful, but longer enforces a crisp ranking between weight magnitudes.
No ranking and no temporal smoothness. When both Crank and Lamooth are removed, the ge- ometry loses coherence across time as well as within each snapshot. Kendall's 7 declines further 0.765 on MNIST and 0.764 on Fashion MNIST (Figure 10). and the distance-log |ws| slope be- comes much shallower, falling to -0.365 on MNIST and -0.061 on MaSSI The scatter rises sharply, and the expected ordering "stronger weights closer" breaks down in many regions. Without the temporal smoothness term, node embeddings can jump between checkpoints, making the temporal signal harder to follow and reducing the stability of the mapping between weights and their geometric placement. This ablation shows that ranking and smoothness act jointly: one forces local ordering, the other maintains temporal continuity, and removing both produces a visibly unstable geometric structure (Figure 12.
Mishion-MNIST
No signed regression head. Removing the signed regression head produces the strongest degra- dation in the geometry. INR accuracy remains comparatively close to the full model, yet Kendall's
36

---

Published as a conference paper at ICLR 2026
O19 liget OGGIO
Uerance 10g doart sedinn wnw.--nddddeee...
Figure 7: Analysis of the relationship between hyperbolic distance (d) and edge weights for the "full objective" condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIST. For cach datasct, the left image shows the hexbin plot of distance vs. log le| with fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance for MNIST this condition ( (- 0.617d) indicates a strong decay of edge weights, significantly steeper than for Fashion-MNIST (- 0.322d), reflecting distinct geometric influences on model training for each dataset.
the weight magnitude /w]. The fitted slope
Pradicted Actual Accuracies VINS
Predicted Actual Accuracies MASHONN
Preccse
02
0.5
1/3
05
10 07
Figure 8: Kendall over the "No ranking loss" condition for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN without Lank achieves a Kendall's of 0.811 MNIST and 0.797 on Fashion-MNIS" datasets, indicating noticeable degradation the monotonicity of the predictive signal compared to the full objective.
drops to 0.738 on MNIST and 0.726 on MNIST (Figure 11I. The slopes collapse significantly on MNIST (-0.211) and even reverse on Fashion -MNIST (+0.125) (Figure and the hexbins lose almost all discernible structure. Without supervision magnitudes and signs, the model only learns which edges exist; it no longer learns how strong or influential they are, nor how polarity should be represented in tangent directions. Hyperbolic distances therefore reflect connectivity but not graded strength, and the embedding becomes insensitive to functional variation. This confirms that the signed head is the main mechanism that binds the geometry to weight values rather than merely assisting link prediction.
Taken together, these ablations show that the ranking and smoothness losses preserve the temporal and geometric coherence of the embeddings, while the signed head is essential for encoding mag- nitude-distance coupling. The INR classification task remains robust several ablations, yet the auxiliary metrics and geometric plots reveal steady erosion of the hyperbolic structure that supports our explanatory interpretations when each constraint is removed.
K FUTURE WORKS
Our approach of geometrically representing the temporal dynamics of deep neural networks nat- urally extends theoretically beyond MLPs because any neural architecture can be represented as a parameter graph (Lim et al. (023). For CNNs, convolutional kernels give structured edge sets between input and output channel nodes forming multi-graph, with kernel position and related
37

---

Published as a conference paper at ICLR 2026
colcert Nittgee Fader west cora4
Larace il Eogerat wow. -1
Figure 9: Analysis of the relationship between hyperbolic distance (d) and edge weights for the "No ranking loss" condition. The top row displays results for MNIST, while the bottom row shows Fashion-MNIS" For cach datasct, the left image shows the hexbin plot of distance vs. log le] with a fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance vs. the weight magnitude /w]. The fitted slope for MNIST in this condition 0.473d) indicates moderate decay edge weights, which is substantially shallower than the full objective model, while the decay for Fashion-MNIST ( (-0. 0.189d) is notably weaker. This indicates that removing the ranking loss leads to loss of the crisp, monotone geometric ordering.
Prudicted Accus acies MNSST
Prodicted Accuracies FASHION Hencalta
Gande
Pockled
24
Figure 10: Kendall over the "No ranking and no temporal smoothness" condition for the INR classification task MNIST and Fashion-MNIST datasets. GTH-GMN without both Zrant and Lamouth achieves a Kendall's T 0.765 on MNIST and 0.764 on Fashion-MNIST datasets, indicating a further decline in the ordering coherence of the predictive signal.
Pradicted Actival Accuracies.. MNST siened
Predicted Actual Accuracies MSHION
30
Figure 11: Kendall 7 over the "No signed regression head" condition for the INR classification task on MNIST and Fashion-MNIST datasets. GTH-GMN without the signed head achieves Kendall's of 0.738 MNIST and 0.726 Fashion-MNIST datasets, representing the strongest degradation in the geometric ordering of the predictive signal.
structure stored as edge features. For Transformers, attention layers induce subgraphs that connect query, key, value, and output channels, with head and layer indices and normalization parameters encoded node and edge features (Lim et al. 2023). The deeper layers form the MLP parameter graphs and can be modeled as stated in this work for both CNNs and Transformers. Since the hyper- bolic encoder operates only node and edge structure, these architectures require modification
38

---

Published as a conference paper at ICLR 2026
Detance
Dics
CBIOO log POSEOS POTER SPOCO
De loger Modit -1A22C-----EAAAA 1A aron
Figure 12: Analysis of the relationship between hyperbolic distance (d) and edge weights for the "No ranking and no temporal smoothness loss" condition. The top row displays results for MNIST, while the bottom row shows Pashion-MNIST For each dataset, the left image shows the hexbin plot of distance fitted linear regression line (dashed orange). The right image shows the hexbin plot of distance raw weight magnitude jw/. The fitted slope for MNIST in this condition (-0.365d) indicates a substantially shallower decay of edge weights compared to the full model, while the decay for Fashion-MNIST is minimal ( -0.061d), reflecting a loss of coherent geometric structure when both regularization terms are removed.
log [w] with the
Io Dipord
Figure 13: Analysis of the relationship between hyperbolic distance (d) and edge weights the "No signed regression head" condition. The top row displays results for MNIST, while the bottom row shows Fashion- MNIST. For each dataset, the left image shows the hexbin plot of distance vs. log [w] with fitted linear re- gression line (dashed orange). The right image shows the hexbin plot of distance vs. the jw]. The fitted slope for MNIST this condition (0.2211d) near-zero, and the slope for Fashion-MNIST (+0.125d) is positive, indicating that the expected decay of weights with distance has collapsed or reversed. This confirms that the signed head is the primary mechanism binding the hyperbolic geometry to the functional magnitude of the weights.
weight magnitude
to the core method: their parameters simply instantiate richer graph topologies that the same tem- poral geometric machinery process. Apart from that, recent work has shown that optimization trajectories admit coarse equivalence classes through topological conjugacy and Koopman spec- tral analysis (Redman et al. 2024). These dynamical characterizations complement parameter graph approach. A promising future direction to incorporate such global dynamical in- formation into the temporal update module the final task-specific readout, allowing these signals to enrich the geometric structure captured by GTH-GMN through a broader global context.
hyperbolic
39


Reviews:
[{"review_id": "qgGzmFnmm1", "reviewer": "Reviewer_oHUv", "summary": "This submission builds on prior work that seeks to develop methods for identifying the behavior of a random deep neural network (DNN), as well possibly train the DNN. The authors use temporal information (i.e., the optimization history) to learn models with greater accuracy. In addition, the authors embed the snapshots of the DNNs (across training) in a hyperbolic geometry. This can aid in performance and interpretability.", "questions": "1. What exactly was tested in \"Classification of Images via INR traces\"?\n\n2. What is a \"sinusoid–MLP\" and what exactly was being tested in \"Predicting Sine Wave Frequency\"? \n\n3. How might the authors' method be improved with - instead of passing in many optimization graphs - the dynamics of the optimization were first filtered with dynamical systems approaches?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The work was well motivated. The utility of metanetworks was clearly presented and the idea that optimization trajectories could be leveraged for further improvement was good. \n\n2. The authors test on several different experimental set-ups, finding strong performance on in all cases. While they are not always best, their method is not so far off. \n\n3. The visualization of the hyperbolic embedding (Figure 2) was very interesting and points to possibly new ways to understand DNN training.", "weaknesses": "1. I think the biggest weakness of this work is that it is so dense. I found the last paragraph of the Related Works Section and Sec. 3.2-3.4 just full of acronyms, method names, and details. I'm sure that this is partially due to the fact that I am not so familiar with this area, but it felt very difficult to follow.\n\n2. The result of having so much detail crammed into the Methods section was that then there was little room for discussion on the experiments. It's not necessary to have all the model and experimental details, but it wasn't always clear to me what was even really being tested in the 3 experiments in Sec. 4. What exactly was tested in \"Classification of Images via INR traces\"? Which image is being shown to a MLP? if so, this seems to not be so aligned with the motivation in the Introduction. Similarly, I was confused as to what a \"sinusoid–MLP\" is and how the developed method was being used. \n\n3. A minor point, but one way to study the optimization trajectory is to use tools from dynamical systems. These tools can be invariant to node ordering and can extract interpretable and comparable structure (e.g., https://proceedings.neurips.cc/paper_files/paper/2024/hash/2a07348a6a7b2c208ab5cb1ee0e78ab5-Abstract-Conference.html). How might the authors' work be extended/improved by including such dynamical characterization?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8HpAXKW49a", "reviewer": "Reviewer_fCBW", "summary": "This paper proposes a novel framework for studying MLP training dynamics through hyperbolic geometry. The authors construct temporal parameter graphs from checkpoints during training and embed them in the Poincaré ball using hyperbolic graph attention networks (HGAT) with kernel meta-evolution. The approach maintains permutation equivariance and ties edge weights to hyperbolic distances via power-law relationships, providing interpretable geometric representations of network self-organization during training.", "questions": "- Can you provide runtime/memory comparisons with baselines? How does the approach scale with model size?\n- What specific architecture modifications would be needed to extend beyond MLPs? Is this fundamentally intractable?\n- Could you provide ablations showing the contribution of each loss term (Eq. 32)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel geometric perspective: The application of hyperbolic geometry to temporal parameter graphs is original and well-motivated. The connection to network science findings (scale-free, hierarchical organization) provides strong theoretical justification.\n- Temporal modeling: Unlike most prior meta-learning work that analyzes single checkpoints, this work explicitly models training trajectories.\n- Comprehensive methodology: The signed weight regressor with magnitude-distance power law (Eq. 13) and conformal sign prediction (Eq. 15) shows careful design. The two-phase optimization (Euclidean + Riemannian) is well-executed.", "weaknesses": "- Limited performance gains: On CIFAR-10 generalization prediction, the method achieves τ=0.846±0.004, notably below NFN baselines (0.922-0.934). While the authors acknowledge this is \"expected,\" it raises questions about practical utility. The gap suggests the geometric compression may discard accuracy-correlated information that end-to-end methods capture.\n- Architectural limitations: The approach is restricted to MLPs. The authors mention this limitation but don't provide a clear path to extending to CNNs, Transformers, or other architectures that dominate modern deep learning. This severely limits practical applicability and impact.\n- Computational cost not addressed: The paper doesn't discuss training time, memory requirements, or scalability. Hyperbolic operations, Riemannian optimization, and temporal processing likely add significant overhead.\nTables 5-6 mention \"practical caps\" but no runtime comparisons are provided.\n- High variance in some experiments: Sinusoid task: 1.06±0.24 MSE vs GMN's 1.13±0.08 - the 3x higher variance is concerning and poorly explained. The authors attribute this to \"known sensitivities in Riemannian optimization\" but don't investigate mitigation strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8wnLVubh2m", "reviewer": "Reviewer_dcw3", "summary": "This paper models the training process of neural networks as a sequence of evolving graphs. Each of these graphs is embedded in hyperbolic space. The authors introduce a Hyperbolic Graph Meta-Network (GTH-GMN) that learns to encode the temporal evolution of these graphs using hyperbolic attention and recurrent updates. This geometric representation is intended to provide intrinsic explainability by revealing structure in training dynamics. Experiments on small-scale tasks show consistent improvements over prior Euclidean and static graph baselines, along with interpretable visualizations of how networks evolve during training.", "questions": "1. How sensitive are the results to the choice of curvature or manifold model? For example, if the same temporal graph encoder were trained in Euclidean or spherical space, would the geometric patterns and performance differences still hold?\n\n2. Can the authors clarify what specific types of interpretability their method enables? Beyond visualizing trajectories, are there measurable insights (such as neuron redundancy, layer specialization, or early-stopping indicators) that can be extracted from the hyperbolic embeddings?\n\n3. The experiments focus on relatively small MLPs and simple datasets. How might this framework scale to larger architectures, such as Transformers or CNNs, where parameter spaces and symmetries are more complex? Are there computational or conceptual challenges expected in extending the approach?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper's main strength is in its main idea: to think about the training of a neural network not just as a sequence of parameter updates, but as a geometric process that unfolds in hyperbolic space. This view allows the authors to tie together many ideas from disparate fields. The resulting metrics for tracking training seem to capture meaningful structure in how networks evolve, and the visualizations provide some insights into the learning process.", "weaknesses": "I should preface my comments by noting that I am not an expert in the specific technical areas this paper draws on, which is reflected in my relatively low confidence score. I found the paper quite challenging to follow at times, largely due to its dense mathematical notation and the level of background knowledge it assumes from the reader. Despite the impressive mathematical machinery, I am not entirely convinced that the insights gained from this approach justify the conceptual and computational complexity it introduces.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6leVYQTlrM", "reviewer": "Reviewer_3BLp", "summary": "This paper proposes to incorporate temporal and geometric considerations into study neural networks as data points (e.g. in INRs). The authors propose a methodology the process INR trajectories using the Poincare ball model of hyperbolic geometry to use for metanetwork purposes (process NNs as datapoints e.g. for classification of INRs or predicting generalistion from the trained weights themselves)\n\nI have given a 4 but would give a 3 if there was the option.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The paper seems correct and seems to have succeeded in adding temporal and geometric considerations into a framework for meta-networks.\n2. There is a lot of background material involved in the paper. this is both a strength, as there is a lot of technical detail to get right, but also  a weakness (see below).", "weaknesses": "I'd say overall my criticisms of the paper stem from the fact that I found it hard to read. This is by no means my area, but even still I think the authors could have made several improvements to the presentation:\n\n1. The motivation isn't very strong. The paper focuses on the technical problem (of adding temporal and geometric aspects to meta-networks), but not enough of the paper addresses why this is a problem to focus on.\n2. Likewise, the empirical results for the methodology (the proposed GTH-GMN) of this paper are quite bare. There are 6 terms in the loss function, each of which are motivated in section 3: how is the empirical performance of GTH-GMN affected without each of the 6 terms? This is essential for motivating the method. Why do we need two optimisation steps? The paper is missing quite a lot of ablations imo. Relatedly, is it essential to have temporal *and* geometric aspects to the framework: what happens if you remove one (how does performance vary as the number of checkpoints changes)?\n3. The contributions section (last paragraph of section 1) isn't very clear imo. The paragraph reads as if the contributions are methodological, are there any theoretical challenges that the authors needed to overcome in order to create the methodology (or was it mainly a case of applying and combining existing theory into a method). Likewise, I think a background section is missing between the related work and the research method sections in order to provide context for the reader of the most relevant background info (e.g. what is the most relevant methodological baseline and how does it work? Is this GMN, based on the name GTH-GMN?)\n4. How much more expensive is it to have to track the full trajectory that say the final checkpoint?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)