Exploring End-to-end Differentiable Neural Charged Particle Tracking – A Loss Landscape Perspective
Authors: Tobias Kortus
Ralf Keidel
Nicolas R. Gauger
Venue: Accepted by TMLR ()

Abstract: Measurement and analysis of high energetic particles for scientific, medical or industrial applications is a complex procedure, requiring the design of sophisticated detector and data processing systems. The development of adaptive and differentiable software pipelines using a combination of conventional and machine learning algorithms is therefore getting ever more important to optimize and operate the system efficiently while maintaining end-to-end (E2E) differentiability. In this work, we lay the groundwork for E2E differentiable decision focused learning for the application of charged particle tracking using graph neural networks with combinatorial components solving a linear assignment problem for each detector layer. We demonstrate empirically that including differentiable variations of discrete assignment operations allows for efficient network optimization, working better or on par with approaches that lack E2E differentiability. In additional studies, we dive deeper into the optimization process and provide further insights from a loss landscape perspective, providing a robust foundation for future work. We demonstrate that while both methods converge into similar performing, globally well-connected regions, they suffer under substantial predictive instability across initialization and optimization methods, which can have unpredictable consequences on the performance of downstream tasks such as image reconstruction. We also point out a dependency between the interpolation factor of the gradient estimator and the prediction stability of the model, suggesting the choice of sufficiently small values. Given the strong global connectivity of learned solutions and the excellent training performance, we argue that E2E differentiability provides, besides the general availability of gradient information, an important tool for robust particle tracking to mitigate prediction instabilities by favoring solutions that perform well on downstream tasks.

Published in Transactions on Machine Learning Research (07/2025)
Exploring End-to-end Differentiable Neural Charged Particle Tracking A Loss Landscape Perspective
Tobias Kortus
tobias.kortus@rptu.du
Chair for Scientific Computing
University of Kniserslauterr-Lantt (RPTU)
Ralf Keidel
rul.keeddel@rphu.de de
Chair for Scientiffi Computing
University of Kniserslautern-Lane Landav (RPTU)
Nicolas R. Gauger
hicolas.gougerrsscccom......e
Chair for Scientific Computing
University of Kotserslauterr-Landau (RPTU)
On behalf of the Bergen pCT Collaboration
Reviewed on OpenReview:
Abstract
Measurement and analysis of high energetic particles for scientific, medical or industrial applications is a complex procedure, requiring the design of sophisticated detector and data processing systems. The development of adaptive and diffcrentiable software pipelines using a combination of conventional and machine learning algorithuns therefore gelting ever more important to optimize and operate the system efficiently while maintaining end-to-end (E2E) differentiability. In this work, we lay the groundwork for E2E differentiable decision focused learning for the application of charged particle tracking using graph neural networks with combinatorial components, solving linear assignment problem for each detector layer. We demonstrate empirically that including differentiable variations of discrete assignment op- orations allows for efficient network optimization, working better or on par with approaches that lack E2E differentiability. In additional studies, we dive deeper into the optimization process and provide further insights from a loss landscape perspective, providing robust foundation for future work. We demonstrate that while both methods converge into sim- ilar performing, globally well-connected regions, they suffer under substantial predictive instability across initialization and optimization methods, which can have unpredictable consoquences on the performance of downstream tasks such as image reconstruction. We also point out a dependency between the interpolation factor of the gradient estimator and the prediction stability of the model, suggesting the choice of sufficiently small values. Given the strong global connectivity of Icarned solutions and the excellent training performance, we argue that E2E differentiability provides, besides the general availability of gradient in- formation, important tool for robust particle tracking to mitigate prediction instabilities by favoring solutions that perform well on downstream tasks.
1 Introduction
Charged particle tracking [Strandlie Frühwirth 2010) is a central element in analyzing readouts generated by ionizing energy losses particle detectors, with the goal of reconstructing distinct sets of coherent parti- cle tracks from discrete particle cloud measured over subsequent detector layers. Current state-of the-art approaches in particle tracking heavily utilizes geometric deep learning in two-phase predict-then-optimize

---

Published in Transactions on Machine Learning Research (07/2025)
approach (Shlomi et al. 2021 Duarte & Vlimant 2022 Thais et al., 2022 In initial prediction step, relations between detector readouts described as a graph structure are modeled and quantified. Then the predicted relationships are used in a separate and disconnected step to parameterize a discrete optimization problem, generating disconnected track candidates. Separating the initial scoring from the final assignment operations, however, bares the risk of learning suboptimal solutions, that only minimize the intermediate optimization quantity of the scoring task (prediction loss) instead of the final task loss (Mandi et al.) 2023).. This effect is potentially further magnified by integrating particle tracking in entire component-based data processing pipelines" used for reconstruction and analysis, introducing additional complexities due to non- monotonic error dependencies of components on the downstream reconstruction performance, as well as complex error propagation due to entanglement of individual processing steps (Sculley et al. 2015 Nushi et al. 2017) (further discussed in Section 3.2). Providing end-to-end differentiable pipelines, where all in- dividual components allow the propagation of gradient information, could significantly improve the overall performance of roconstruction and analysis and allow the integration of gradicnt- experiment design and optimization (Dorigo et al. 2023 Aehle al., 2023b). Providing end-lo-end differentiable solutions is, however, not trivial, the structured nature of tracks requires discrete, piecewise constant assignment operations where the gradient is undefined. While recent work, aims to bootstrap the identification of tracks to downstream tasks (Stroud et al., 2024 Reuter et al. 2024), thereby achieving end-to-end optimization, the high combinatorial complexity of finding track candidates that satisfy the imposed unique assignment con- straints directly, is a major challenge. The complexity of differentiating particle tracking and reconstruction pipelines in general is further examined and discussed in Aehle et al. (2023b). The fecisionn-ocused learning or predict-and and-anndomeeeee where prediction of intermediate scoring and optimization for structured outputs are directly integrated in the training procedure by embedding the combinatorial solver as component of the network architecture, allowing to optimize the main objective in an E2E manner. This framework has already been proven highly efficient for various fields of applications such as natural language processing, computer vision, or planning and schoduling tasks (Mandi et al., 2023). In this work, we propose and explore a predict-and-optimizz framework for charged particle tracking, providing, to our knowledge, the first fully differentiable charged particle tracking pipeline for high-energy physic[] Further, we aim to provide first insights into the im- portance and usability of end-to-end particle tracking for fully differentiable roconstruction and analysis pipelines analyzing the loss landscape (e.g., tharpness/ffatneee of minima, connectivity of local minima, and similarity of learned models) of the tracking network. We choose this simplification over an analysis of an entire pipeline to reduce the overall complexity, introduced by other components, obscuring the influence of E2E tracking on the overall optimization performance. Further, E2E differentiability is still an active area of research for high-energy physics, with dilferentiability yet to be provided or improved for some components, making a meaningful and robust analysis infeasible. Our main contributions in this paper summarize as follows, serving as a foundation for further work on E2E charged particle tracking:
paradigm (Kotary et al.) 2021 Mandi et al. 2023 provides a general framework
1. We propose an edge-classification architecture for particle tracking, that operates on a filtered line graph representation of an original hit graph, providing strong inductive bias for the dominant effect of particle scattering.
2. We then formalize the generation of track candidates as a layer-wise linear sum assignment problem, for which we provide gradient information based on previous work by Vlastelica et al. (2020).
3. We demonstrate the competitive performance of our proposed network architecture for both tradi- tional and E2E optimization, comparing it with existing tracking algorithms for the Bergen pCT detector protolype.
4. We find significant predictive instabilities between the different training paradigms as well as across random initializations, arguing the importance of E2F differentiability for robust particle tracking,
Proceessing pipelines for, e.g., Bergen pCT project and the ALICE, ATLAS and CMS experiments, depicting the complexity
and varicty of individual chained components, are described in detail in Achle et al. (2023)) Buncic et al. (2015): Akesson et al. 2005); Lange& & the CMSCollaboration) (2011)
"Th source code and data utllied This study are detailed in Appendix including all relevant source code, datasets,
and results necessary to reproduce the findings presented in this paper.
2

---

Published in Transactions on Machine Learning Research (07/2025)
providing necessary tools to constrain the solution space of tracking models to subset optimizing an additional metric of the downstream tasks.
5. Further, we reveal, analyzing the local and global structure of the loss landscape, strong connectiv- ity between the trained representations both for E2E training and optimization of a prediction loss. Together with the high predictive instability, we argue that the strong connectivity of diverse recon- struction policies enables efficient end-to-end optimization in reconstruction and analysis pipelines, favoring tracking solutions that provide beneficial performance on downstream tasks.
6. Finally, we point out a coupling between the interpolation parameter defined by the blackbox gradi- ents and the prediction stability, suggesting the choice of sufficiently small values for optimization.
2 Related Work
Charged particle tracking is well-studied problem, with the majority of advancements originating from de- velopments in high-energy physics research at particle facilities (e-g-, CERN). For the last decades, tracking algorithms have been dominated by conventional model-based pattern recognition and optimization algo- rithms such as Kalman filter (Frühwirth 1987), Hough transform (Hough) 1959), or cellular automaton GGlazov et al., 1993). Here we specifically want to point out existing work by Pusztaszeri et al. (1996) related to our work, using combinatorial optimization on handcrafted features for partiele tracking providing solutions that satisfy unique combination of vertex detector observations using five-dimensional assignment model. With the ever- accelerating advancement and availability of deep learning algorithms, coupled with the inereasing complexity and density of collision events, a significant demand has been imposed on novel reconstruction algorithms, resulting in faster and more precise models. While initial studies mainly relied on basic regression- and classification models based on convolutional newral networks (CNN) or recurvent neural networks (RNNN (Farrell et al.) 2017a b), remarkable progress has been made recently by computationally efficient and well- performrno network architectures based on geometric deep learning (GDL) (Shlomi et al., Thais el al." 2022) leveraging sparse representations of particle events in combination with graph newral networks (GNN). Here, the vast majority of current designs can be categorized into one of two main schemes.
2021
Edge classification tracking (Farrell et al.] 2018 Ju et al. 2020
Duarte & Vlimant 2020
Baranov et al. predicts for each
2019 Heintz et al., 2020: DeZoort et al. 2021 Elabd et al.) 2022
Murnane et al. 2023
graph edge connecting two particle hits in subsequent layers a continuous output score, which is then used
construct track candidates. In object condensation tracking (Kieseler 2020 Qasim et al., 2022 Lieret
& DeZoort. 2023; Lieret et al., 2023). GNN's are trained with multi-objective object condensation loss. In contrast to scoring edges in the graph, object condensation embeds particle hits in an N-dimensional cluster space, where hits belonging to the same track are attracted to a close proximity while hits from different tracks are repelled from each other. While both approaches provide great tracking performances, they require a separate optimization step (e.g.; clustering) to generate final track candidates in a discon- nected step, thus lacking an end-to-end differentiable architecture, preventing gradient flow throughout the whole reconstruction process. Recent progress has been made using reinforcement learning (Kortus et al., 2023) for charged particle tracking, allowing to different through the discrete assignment operation using variants of the policy gradient score function estimator approach, maximizing an objective function (0), defined as the expected future reward obtained by the agent's 's policy. Additional work by c.g., [Stroud et al.) (0224h Reuter et al. (2024) proposed the fusion of tracking with track-fitting procedures (track parameter estimation), avoiding the optimization over discrete assignment operations.
3 Theory and Background
In this work, we focus on reconstructing particle tracks generated in the pixclated proton computed tomogra- phy (pCT) detector protolype, proposed by the Beryen pCT Collaboration {Alme et al., 2020 Aehle et al. 2023a). This detector is designed for generating high-resolution computer tomographic images using accel- erated protons and composed using total of 4644 high-resolution ALPIDE CMOS Pixel sensors (Mager 2016 Aglieri Rinella, 2017) developed for the upgrade of the Inner Tracking System (ITS) of the ALICE experiment at CERN. To capture the residual energy and path of the particle, the detector is arranged in
3

---

Published in Transactions on Machine Learning Research (07/202) 2025)
Figure Schematic representation of the combinatorial complexity of reconstructing particle tracks over multiple subsequent layers. Marked in bold lines is the correct particle track. Marked in dashed lines are only possible next. track segments originating from the previous correct segment. During reconstruction, all possible combinations of segments over all hits are considered.
multi-layer structure, aligned in two tracking and 41 calorimeter layers. Each of the tracking layers, used for capturing the incoming particle path with as little interaction as possible, is separated from each subsequent layer by a 55.18 mm and 39.58 mm air gap respectively. The following calorimeter layers are separated from each other with a 3.5 mm aluminum slate. A detailed description of the detector geometry can be found in Alme et al. (2020). In this detector setup, multiple simultaneous particle trajectories, generated by a 230 MeV scanning pencil beam, are captured distal to patient as discrete readouts of pixel clusters in the sensitive layers which have to be reconstructed into particle tracks under the influence of physical interactions with the detector material, randomly deflecting the particle from its straight path (ref. Appendix C With the high particle density in the detector, the resulting combinatorial explosion of track candidates, where each hit combination in subsequent layers need to be considered as possible partial solutions, requires effi- cient and reliable algorithms that are able to simultancously reconstruct disjoint particle tracks by uniquely connecting particle hits over multiple layers in the detector (ref. Figure 1
3.1 Particle Interactions in Matter
The path of protons at energies relevant for pCT is mainly influenced by interactions of the particles with both electrons and atomic nuclei changing the path of the accelerated particle (Gottschalk) 2018 Groom & Klein 2000). posing additional difficulties in recovering the traversal path of the particle throughout a multi-layer particle detector. In the following, we briefly describe all three relevant effects in increasing order regarding their influence on the particle trajectories:
When interacting with negatively charged clectrons in the atom's outer shell, protons lose a fraction of their initial energy, due to attracting forces between the particles Blochh 1933), causing the particle to stop after a traversal range, proportional the particle's energy. Due to the relatively small mass of the electron compared to the high momentum of the accelerated proton, the proton remains on its original path.
Ionizing energy loss:
Elastic nuclcar interactions: However, when repeatedly interacting clastically with the heavier atomic nuclei, the proton is deflected from its original path due to the repelling forces between nuclei and particle. Integrated over multiple interactions, this effect, also referred to Multiple- Coulomb scattering (MCS), follows an approxinately Gaussian shape proportional to the particle's energy well as the amount and density of material traversed (Gottschalk) 2018
3. Inelastic nuclear interactions: Finally, on rare occasions, particles suffer from inelastic interac- tions with an atomic nucleus. In this case, the proton undergoes destructive process, where the primary can knock out one or multiple secondary particles of various types from the target nucleus. Due to the stochastic nature of the event, the outgoing path of particles is highly complex in its nature and involves significantly larger angles compared to MCS (Gottschalk) 2018). Inelastic inter- actions are therefore extremely difficult to reconstruct while losing any meaningful information for image reconstruction due to its stochastic behavior.
4

---

Published in Transactions on Machine Learning Research (07/2025)
3.2
component-based Processing Pipelines
Modern data processing pipelines are increasingly powered by data-driven machine learning algorithms, requiring the efficient interplay of multiple individually optimized tasks that are strongly coupled and en- tangled with each other (Sculley et al., 2015 Nushi et al.) 2017). The coupling of data-driven algorithms introduces additional challenges that need to be considered in order to ensure operability and maintain- ability while providing reproducible and optimal performance [Sulleyy et al. (2015). While the influences of component-based processing pipelines utilizing machine learning are manyfold, we focus in the following on the following aspects, that influence the initial design and operation, especially benefiting from end-to-end ceinniiaabe/diiffreeetta solutions:
Stochastic nature of optimization: In contrast to conventional model- or rule-based algo- rithms, data-driven algorithms are sensitive to random mechanisms, such as parameter initialization, stochastic optimization, or parallel operations on GPUs, roducing the overall stability of the system.
Error propagation: While individual model performances are often not only affected by prediction instabilitics (Lopez-Martinez et al., 2022), the different sources of errors propagate throughout the entire pipeline, resulting unpredictable error patterns in the final output of the pipeline due to the existent complex entanglements of the downstream processing steps.
Finally, due to the entanglement the individual components, model improvements of individual components are not necessarily bound to an improvement of the overall system performance Nushi et al. (2017). While this seems unintuitive at first, this behavior can be caused by downstream tasks being optimized on the erroncous output signal of the upstream part of the pipeline misalignment of the individual components' optimization metric and overall system performance.
Non-monotonic error:
To tackle the challenges associated with component-based pipelines, different solutions have been developed for mitigating the effect. This includes, for example, the development of modular processing pipelines, aiming to decouple the individual pipeline components to reduce the complex effects entanglements Modi eet al. (2023). Increasing effort has also been put into the development. of F2F--diflerentiable or general optimizable ML pipelines YYu et al.] [022 Hilprechtt et al., 2023), allowing optimize the entirety of components while optimizing the final output of interest. A similar trend, utilizing and-to-end differentiable solutions, can also be found in high-energy physics (Dorigo el al. 2023 Aehle el al. 2023b).
3.3 Loss Landscape Analysis
Optimization of neural networks in high-dimensional parametric spaces is notoriously difficult visualize and understand. While theoretical analysis is complex and usually requires multiple restrictive assumptions, recent work addressed this problem by providing empirical tools for loss landscape analysis providing com- pressed representations for characterizing and comparing different architectures or optimization schemes. We provide in the following section a brief introduction to a subset of methods, used for the analyses in the later scctions. Here, we closcly follow the methodology and taxonomy provided by Yang et al.) (2021) helping to characterize and assess differences and qualities of traditional and end-to-end differentiable particle tracking with graph neural networks by characterizing and comparing the general form of the loss landscape as well as the global connectivity of the landscape.
Loss surfaces: To obtain a general understanding of the smoothness and shape of the loss landscape et al. (2018) proposed a method to visualize two-dimensional loss surfaces along random slices of the loss landscape centered on the estimated parameters 8" according to
f(a B) L(0 av Bn).
(1)
Here, v and n are random vectors, spanning a 2D-slice through the nigh-dimensional loss landscape. While this technique is not fully descriptive and only a partial view of the optimization landscape, Li et al. (018
5

---

Published in Transactions on Machine Learning Research (07/2025)
demonstrates that the qualitative characteristic and behavior of the loss landscape is consistent across dif- ferent randomly selected directions. In our analysis, we use the first. two eigenvectors with the highest eigenvalues of the hessian matrix (Chatzimichallidis et al. 2019 of the trained network parameters 0*, providing the loss surface along the steepest curvature.
Mode connectivity: Garipov et al. 2018 and Draxler ot al. (2018), demonstrated independent work
the existence of non-linear low-energy connecting curves between neural network parameters. While this approach was initially proposed for generating ensembles, Gootmare et al. (2018) and Yang et al. (2021)
demonstrated the usefulness of this approach for comparing different initialization and optimization strategies well as characterizing global loss landscape characteristies. For finding connecting curves of form ehe(t) [0, RRd Garipov et. al. (2018) defines optimization schemes using two modes taa dhs (weights after training), minimizing the integral loss alongside the parameterized curve approximated by minimizing the expoctation randomly sampled points, following a uniform distribution:
[()) = L'' ((vo(t))dt B.-UTO. [A (Vo(t)))
(2)
For quantifying the connectivity of minima generated using two-step and E2F. optimization, we follow the recommended parametrization of the learnable curve used in Yang et al. (2021). defined by Bézier curve with three anchor points (wa, ths and learnable anchor 8), with
(b)(1) = (1- + 21(1 t)0 P
(3)
We additionally evaluate the linear connectivity both as a baseline and measure for mechanistic similari- tics [Neyshabur et al. 2020 Juneja et al. 2022 Lubana et al. 2023
Representational and functional similarities: Analyzing network similaritics in both representations and outputs an effective tool in comparing results of network configurations, providing an estimate of proximity in the loss landscape, especially for globally well-connected minima (Yang et al.] 2021).
Achieving high network similarity is espocially desirable for our use cases, differences in reconstructed tracks might influence results of the downstream tasks, such as image reconstruction for pCT or statistical analysis for HEP experiments.
To gain an insight into the bohavior of the trained networks, we thus quantify both the similarity of learned representations using CKA similarities (Kornblith et al. 2019) and prediction instability using the min-max normalized disagreement (Klabunde et al. 2023).
Linear-CKA Linear central kernel alignment (CKA) is widely used to quantify a correlation- like similarity metric [Kornblihh et al. 2019) to compare learned representations of neural network layers of same or different models. CKA compares representations via the Hilbert-Schmidt Independence Criterion (HSIC) according to
OSTC(K,I) L) HSIC HJSIC K)
CKA(K,
where K X X and L YYT the gram matrices caleulated for the representations of two layers with X E Raxd] and Y E Rnxd2 To cope with the large number of processed nodes and edges processed, we use the batched Linear-CKA leveraging unbiased estimator of the HSIC over a set of minibatches [Nguyen et al. 2020).
Prediction instability Prediction instability or churn captures the average ratio of disagrocment between predictions of different models fi and fz (Fard et al. 2016 Klabunde & Lemmerich 2023 For a general multi-class classifier, the disagreement is defined following

---

Published in Transactions on Machine Learning Research (07/2025)
Hit Graph (9H)
Line Graph (9z)
(..p.A)
particle dirortion
stadlarity encoding
in Gut
Osics
Gg Eak in Git
0g in Gi
Figure 2: Schematic reprosentation of directed hit graph (Gn) and undirocted line graph (91 L(Gn)) using detector readout data simulated for the Bergen pCT protolype detector over multiple detector layers.
d(fi, f2) = Br.fi.al (arg max (a:)) * arg max f2(2))
(5)
This concept naturally translates to binary classifiers as used in this paper, where the argmax is replaced by a threshold function. This metric, however, is difficult to interpret since the value range bounded by
(1 (Err( S d(fi, min(qe:)(()) + (E2),
(6)
where qErr is the error rate of the model fi or f2, respectively (Klabunde & Lemmerich, 2023). We thus the min-max normalized disagroement (Klabunde & Lemmerich 2023
defined
d(fi, min d(fi, f2 f2)
dnorm (2) max d(f, f2) min d(fi, f2)
with min d(Jn f2) 1ummr (fi) (F2)) and max d(fi, min (gEer (J1) (f2r) giving similarities in a bounded interval of [0, 1].
4 Predict-then-track and Predict-and-track Framework
In this section, we introduce our end-to-end differentiable predict-and-track (PAT) and two-step predict- then-track (PTT) approaches, including preprocessing steps directly aimed at the specific data gathered during proton computed tomography. We closely follow existing state-of-the-arl edge-classifying approaches, with modifications match the algorithm to the specific challenges of reconstruction in digital tracking calorimeter, to provide generally applicable blueprint and transferrable results that can be casily adapted to different delector geometries and data structures.
4.1 Hit Graph and Edge (Line) Graph Construction
In this work, parameterize the detector data undirected line graph Gr. (V,, generated from initially generated hit graph GH (VH,EH) containing all the detector readouts of single readout frame, In an initial hit graph structure, we represent all particle hit centroids in readout frame as set of graph vertices UHr and describe possible partial track connections (segments) between two consecutive layers in the detector by edge connections éH, This representation widely used for particle tracking, as allows describing and parameterizing the particle hits and possible partial solutions single sparse structure. Yet. to capture a richer representation of the scattering behavior over combinations of two consecutive segments, providing strong inductive bias for modelling the most likely path of the particle in the detector, under the

---

Published in Transactions on Machine Learning Research (07/2025)
infiuence of clastic nuclear interactions, transform this initial representation into line graph (92), where each edge in Gr is transformed into node in Gr. (ref. Figure 2 Further, edges are generated bet ween nodes UL,6, 2L that share a common vertex in GH, constructing descriptions of track deflections of candidates over three consecutive layers. This transformed representation allows to efficiently aggregate and compare information of scaltering behavior, providing important information on possible track segments one edge feature, that can be aggregated by a single GNN message, as supposed to the more complex aggregation GH over a two-hop neighborhood (ref. Figure 2). To provide machine-readable and differentiable features (w.r.t. simulation output) we parameterize both vertex (VL,s) and edge (EL.sy) feature: of the line-graph representation as
UT, (AEn, AE (1,,, m,, M, R)
sin(w [1
~V Scos)) if = 2k
(8)
et,ij
cos(w / /Soos)
if i 2k +
where AEH the deposited energy the sensitive layers, and (x, y/ 2) is the global position of the particle hit in the detector for the adjacent nodes and j. Further, each edge is parameterized by a positional leverage
encoding, similar to Vaswani et al. (2017). However, instead of using discrete token positions, cosine similarities 8cos between the directional vectors of two track segments in 9и as proposed by Kortus ct al. [2023] : injecting additional information on the relative likelihood of observing a specific transition dynamic of track candidates over segment combinations over three consecutive detector layers under the underlying physical interaction mechanisms. The importance and usefulness of the line-graph transforma- tion, especially its beneficial parameterization of segment pairs, functioning as strong inductive bias by condensing information on particle scattering into single feature, is analyzed and confirmed in detail in Appendix D Here, we highlight the performance gain compared to conventional hit-graph baseline (De- Zoort et al. 2021 parameterized according to the description in (Kortus et al., 2023). Additional results demonstrating the importance of segment information, provided by the positional encoding mechanism, are presented Kortus et. al. 2023).
To reduce the complexity of the graph and to minimize the combinatorial explosion of edges in GL, we remove implausible edges with high angles in Gh, measured orthogonal to the sensitive layer. We find suitable independent thresholds for both calorineter and tracking layer edges as a trade-off between reducing the graph size and minimizing the number of removed true edges required for constructing the particle tracks (sce Appendix A Both thresholds are determined on the train-set and are kept constant for evaluation. Despite the elimination of graph edges, using a line- mne-graph representation still introduces computational overhead for reconstruction. Additional results quantifying the impact of the line-graph representation are presented in Appendix D
4.2 Edge Scoring Architecture
Following the basic idea of track reconstruction using an edge classification scheme (Heintz et al. 2020; DeZoort et al.) 2021], 1 we compose a graph neural network based on the interaction network (IN) architecture Battaglia et al.) 2016 2018 as proposed in (DeZoort et al.) 2021). ; predicting edge scores capturing the likelihood of every possible track segment candidate 9n- However, we use slightly modified architecture to predict node scores on the line graph representation Gr, which directly correspond
the edge scores in GHr (see Figure 3. Following is the formulation of edge and node updates in generic message passing formulation, well the final scoring function based on node representation Gr. In first step, updated edge representations are calculated using a concatenated vector containing the edge attributes, describing
IIn the following, unless otherwise specified, odge- and node-features of the line graph representation Gr. The additional identifyer L omitted in favor of readability, We the continuous energy deposition and position outputs of the Monte-Care simulations here be able calculate gradients w.r.t. position and energy deposition. The parameters can be replaced with the cluster and cluster centroid position accordingly. However, this representation requires additional work to provide differentiable solutions for pixel clustering.
simplified notation for u7.4 and ez.45, where v, and ey defaults to
8

---

Published in Transactions on Machine Learning Research (07 (025)
Rmxe
LSA
Interaction Network
Graph tramsforms
partiche divectin
Layer
RI
R2
LSA
Layer lil
TSA
Subgradieuts
Gradients of recoustruction w.r.t. network pareanters/inplt features
Figure 3: Combination of interaction network style architecture and layerwise combinatorial assignment for particle tracking, providing gradient information using linear interpolations the optimization mapping.
the scattering behavior of the triplet-candidate,, together with the node attributes v/ and vj of adjacent nodes according to
eij Sn,1 (v, (0)
Here ΦR,1 is multilayer feed-forward (FF) network mapping the concatenated input to fixed size repre- sentation with dR,1 2dadd dedgee + Idduiden
In a subsequent step, all node features of G7 are updated in an aggregation step using the current node representation and edge information aggregated from the direct neighborhood of the respective node e according to
U. (1) U
60
(0)
max (7)))
e,
(10)
JEN;
and transformed using& learned mapping, parameterized by ferd-forward network, with do Rddcatto pdindden, Given the dynamic number of incoming edges for each graph vertex, either due to the changing numbers of hit readouts in subsequent layers of GH or the required generalization ability to different particle densities of the proton beam, we use a node-agnostic max-aggregation scheme, providing empirically better generalization abilities compared to non-node-agnoste aggregation schemes (Joshi et al. 2021
To predict for cach edge in [H an edge score, we perform a final rcasoning step, transforming the node network ΦR.2:
embeddings of each node in Gr into a single scalar output using
Sij - (0R2 (v. (1R2(1;(()) ())
(11)
We generate the final edge scores Siy E [0, 1] using a sigmoid activation function as proposed in Heintz et al.
(2020); DeZoort (2021). We augment. the multi-layer feed-forward networks using batch norma normalization Loffe & Szegedy 2015), mitigate the risk of vanishing gradients and smoothen the optimization landscape
Santurkar al. 2018 providing better training and inference performance.
4.3 Track Construction as Differentiable Assignment Problem
Generating discrete track candidates from the continuous edge-score predictions requires the application of discrete (non-differentiaall) algorithm that matches the tracks with the lowest. total cost. Optimally,

---

Published in Transactions on Machine Learning Research (07 (07/2025)
this would require solving multidimenniooall assignment problem (MAP) to minimize the total cost of all assignments over the entire detector geometry. This exact definition is, however, impractical in real-world usage since solving a MAP is NPP-hard [Pierskalla] 1968), requiring approximate solutions. In the related literature, different optimization procedures, such as DBSCAN, union-find or partitioning algorithms, are used frequently (DeZoori al. 2021: Ju et al. 2021 Lieret & DeZoort, 2023) independently of the training procedure, to construct disconnected tracks during inference. In this work, we rely on relaxation of the original MAP formulation to a layer-wise linear sum assignment problem (LSAP), finding a solution y(C) in form of discrete assignments that minimizes
UCC):
arg min yey
Σ MijCiij
(SJ)EEE
Σ Vij 1, jEW VT
s.t.
iEVs
(12)
Σ yij S 1, iEVs
JEVr
i V1 Vs Mi E {0,1},
for every detoctor layer. Here, Gyy E C is the assignment cost for the edge combination i,j, where E Vs is node from the set of source nodes, defined by all particle hits in the detector layer and j Vr is node from the set of target nodes in the adjacent detector layer 1-1 respectively. Further, Yij is the respective assignment policy defining whether the edge should be assigned under the reconstruction policy. We provide, as described below, additional gradient information w.r.t. C &through a lincarized representation of the solver mapping (Vlastelica et al., 2020), enabling the efficient optimization of the combined tracking algorithm, directly minimizing the assignment error. While the results in Section 5.1 demonstrate good performance for the choice of the LSA assignment, we want note here that an increasing amount of detoctor noisc and particle scaltering can significantly influence the performance due to the required assignment without any notion of noise.
Edge-costs and solver: For the application of charged particle tracking we define dense assignment cost matrix C E [ where each entry with existing edge in GH is set as sij entries, we assign the cost matrix infinite cost, marking this assignment as infeasible:
5 For all other
I 1-5i5 sij ef E E c=
C=
(13)
00 if if $ En
The dense cost matrix allows to solve the assignment problem reasonably efficient in O(n²) using the Jonker- Jonker-Volgenant algorithm (LAPJV) (Jonker & Volgenaut, 1987). Similarly, for larger tracking detectors with sparser occupation per area and thus connectivity, this notation can be replaced by sparse cost matrix and solved with the sparse LAPMOD variant of the Jonker- Volgenant algorithm (Volgenant) 1996) respectively.
Blackbox differentiation: For this general formulation of LSAP a wide range of work of predict-and- optimize schemes providing an end-to-end optimization ability to combinatorial optimization problems exist basod on e.g., the interpolation of optimization mappings (Vlastelica et al.] 2020 Sahoo et al.] 2023), con- tinuous relaxations (Amos & Kolter 2017; Elmachtoub & Grigas 2017 [ilder et al. 2019) or methods that bypass the calculation of gradients for the optimizer entirely using surrogate losses (Mulamba et al. 2021
Shah et al.] 2022). For in-depth review and comparison of existing approaches, the mindful reader is referred to Geng ct al. 2023) Geng et al. (0223 demonstrated for all major and reprosentative types of
Equivaaeentyy we could directly predict the edge cost. Gy by the graph neural network instead using the edge score 8ij- However, we chose this particular notation to stay compatible with the output of the peedict-thenopptmmie variant.
10

---

Published in Transactions on Machine Learning Research (07/2025)
end-to-end training mechanisms for bipartite matching problems, similar regret bounds. We thus base our selection of a technique solely on simplicity and generalizability of the approach to other optimizers and use-cases. Vlastelica et al. (2020) defines a general blackbox differentiation scheme for combinatorial solvers of form y(C)= = arg me c(C, y) by considering the lincarization of the solver mapping at the point y(C) according to
Vefa(C:= = - - ( where c (c+ dL dy (v(C))..00))
(14)
Here, y(C) and yx (C') are standard and perturbed-cost output of the combinatorial solver, À is a hyperpa- ramoter controlling the interpolation and dL/dy is the gradient of the reconstruction loss w.r.t. the output of the combinatorial solver. Using a linearized view of the optimization mapping around the input allows remaining with exact solvers without the necessity of using any relaxation of the combinatorial problem.
Cost margins: The usage of discrete assignment does not provide any confidence scores, thus task losses of correctly assigned edges are always exactly zero, even for marginal differences between cost elements, potentially having a noticeable impact on the generalization performance (Rolinek et al.) 2020b a). To enforce larger margins between predicted costs, Rolinek al. (2020bla) introduced ground-truth-indin margins where negatíve positive penalty is added on the predicted costs based the ground truth labels. Later, Sahoo et al. (2023) introduced noise-induced margins, removing the previous ground-truth dependence by adding random noise to the predicted costs. However, for the large number of edges in our use case of particle tracking, we found both methods to be highly unstable, even for small margin factors.
4.4 Network Optimization
While all previous building blocks are shared between the proposed poddict-andtrran and seelict-then-trae framework, the main difference lies in the optimization procedure of the network. We aim to optimize the prediction quality of both networks by reducing a specific loss function minimizing the disagreement between prediction and ground truth, which is defined as a binary label for every edge in Gμ. We treat splitting particle tracks containing secondary particles as noisy labels due to the relatively low production rate and generally short lifetime of most sccondary particles (o.g., clectrons). This avoids the need for computationally costly tracing algorithms that follow the particle generation of the Monte-Carlo (MC) simulation for the creation of the training data and avoids generating assignment rules for complex particle interactions. Given the true edge labels, we minimize for the predict-and-optmmit scheme the hamming loss of the assignments over a batch of N randomly sampled hit graphs according to
N N
1
C((y))NN (f(G))):)) =
Wiy (Ui (1 - (1-)))) + -yts) (1)).
(15)
(i.))EE,
where Dig is the assignment of the k-th edge in hit graph GH and Vij is the ground truth assignment of the track. We perform additional weighting of the reconstruction loss for tracking and calorimeter layer to account for the different scattering behavior. Similarly, we use the binary-cross entropy (BCE) loss of the predicted edge scores for the prodict-then-optimize version for comparison. To reduce the overall memory footprint of the optimization, we implement the gradient calculation of a single batch using gradient accumulation as N individual predictions over single graph. We use for all following studies an RMSProp optimizer (Hinton et al. 2012 with a learning rate of 1e-3, which we found to be significantly more stable than Adam (Kingma & Ba 2015) and more robust to the selection of the learning rate compared to standard SGD. We train all network variants PAT and PTT for 10,000 training iterations, where in each ileration a random minibatch of N graphs is sampled from the training set. In initial experiments, we observed that the E2E architectures showed the tendency to get unstable after reaching a certain reconstruction performance. We thus perform selective "carly stopping", where we used the training checkpoint (evaluated every 100 iterations) with the best validation performance, determined based the purity of the reconstructed tracks.
11

---

Published in Transactions on Machine Learning Research (07/2025)
5
Experimental Results and Analysis
Dataset:
For the studies reported in this work, we rely on MC simulation of detector readout data, which
we generate using GATE (Jan et al., 2004 2011) version 9.2 based on Geant4 (version 11.0.0) Agosttinelli
et al. 2003 Allison et al. 2006 2016). We generate different datasets for training (100,000 particles) and validation (5,000 particles), using 230 McV pencil beam as beam source with water phantoms of various thicknesses between particle beam and detector. For comparability purposes, the test set, containing 10,000 particles in total, is taken from Kortus et al. 20222) A detailed listing of all data sources is provided in Appendix
F We generate for the training and validation simulations hit- and line graphs with 100 primaries per frame FF), according to the description in Section [4.1 which we consider the expected target density for the constructed detector. Further, we generate hit graphs for the lest set with 50, 100 and 150 p² /F to assess the generalization ability to unseen densities.
Hyperparameter settings: We share model hyperparameters for both PAT and PTT framework, doc- umented in detail in Appendix B As we consider the interpolation parameter À an important tunable parameter, potentially impacting the optimization behavior, analyze the effect of the parameter choice the tracking performance. For this study, we select values of 25, 50 and 75 covering range that is well inside the specified value ranges defined for similar applications (Vlastelica et al., 2020 Rolinek et al. 2020b).
Track reconstruction and filtering: During inference, particle tracks are constructed combining the trained reconstruction networks (PTT and PAT) in all configurations with the linear assignment solver, creating unique assignments of particle hits to tracks To remove particle tracks produced by secondary particles involving inclastic nuclear interactions, we apply track filtering scheme similar to [Pettersnn et al. (020)) and Kortus el al. (2023) removing implausible tracks after reconstruction based on physical thresholds. In this work, we limit the track filters to an energy deposition threshold (625 keV in the last reconstructed layer), ensuring the existence of a Bragg peak in the reconstructed track.
Baseline models: As baseline models, providing reference values for quantifying the track reconstruction performance of the proposed architectures, we select both traditional itcrative track follower algorithm (Pettersen et al.) 2 and reinforcement learning (RL) based tracking algorithm (Kortus al. 2023)
The first baseline finds suitable tracks by iteratively searching for track candidates minimizing the total scattering angle. Similarly, the RL-based algorithm aims to learn reconstruction policy functioning as learned heuristic to the algorithm by (Pettersen et al. 2020] by iteratively maximizing the expected likelihood of observing the scattering behavior determined by MCS during training. Similar to our line- graph parameterization in Section 4.1 Kortus et al. (2023) augment the simpler hit-graph features using segment pair information an inductive bias for tracking. Yet, due to the sequential nature of RL, the pairwise information is queried only for the current partial track candidates without explicitly building the full line-graph (sce Kortus et al. 2023 for more details).
Quantification of model performances: To compare the quality of the reconstruction, we quantify the performance based on true positive rate (TPR), false positive rate (FPR) of the assigned edges as well as roconstruction purity (p) and efficiency (c) of entire tracks, defined respectively as
N-iit
filt Nectt total
RTP
NPP
rec,
TPR
TTP + NPN'
FPR-
DFP + TTN
P
N filt
(16)
rec,+/-
Here RTP,, WTN, REN, and upp are the number of true-positive, true-negative, false-negative and false- positive reconstructed edge segments. Further, N is the number of correctly reconstructed partiele tracks after filtering, fiit - is the total number of reconstructed tracks after applying the track filter, and Nerrim is the total number of primary tracks (without inclastic nuclear interactions) in readout frame. To tootl
Vissaaizaaions of selection of reconstructed tracks can be found in Appendix C
The source code for the track follower proposed by Pettersen et is provided software component the Digital Tracking Calorimeter Toolkit.
12

---

Published in Transactions on Machine Learning Research (07 2025)
18-1
99.35
92
Train-set pert.
[%]
99.30
2.4
PAT 1A 231
PAT (A- 75]
PAT 14 50)
Track purity [%]]
911
Track efficiency (%)
82
2.2
81
99.25
90
99.20
2.0
89
80
PAT 11=
253
FAT 21 25
PAT (A
99.15
501
1.8
88
79
PAT CA
5000
10000
5000
10000
5000
10000
5000
10000
Optimisation step
Optimisation step
Optimisation step
Optimisation step
1e-
99.35
92
Val-set pert.
True [%]
99.30
False pos. [36]
2.4
PAT (A 770
PAT (A -0)
PAT 25)
Track purity (%g
91
Track efficiency [%)
B2
2.2
PIT
81
99.25
90
99.20
2.0
89
PAT 1A 25
PAT (4 25)
PAT (A 25)
PAT 1A- 501
PAT 16- 751
1.8
BAT (1-00 500
RAT C4- 75)
79
PAT (1 50
PAT (A 75)
99.10
1.6
5000
10000
5000
10000
10000
5000
10000
Optimisation step
Optimisation
Optimisation step
Optimisation
Figure 4: True positive and false positive rate of assignments logether with purity and efficiency of track reconstruction, evaluated for predict-ande and-track and predict-then-trest as a function of training steps.
provide confidence intervals for the model performance as well as the following analysis of the loss landscapes, we calculate the results over five independently trained networks with different random initializations.
5.1 Training and Inference Performance
Figure 4 visualizes the intermediate model performances for every 100 training iterations, determined by true-positive rate, false-positive rate, purity and efficiency, both on the training and validation dataset. Additionally, marked with horizontal dotted lines the iteration of each model run with the highest vali- dation purity. Noticcably, all model instances (PAT and PTT) display similar training performances with significant improvement of all metrics over the first 1000-2000 training iterations. Continuing from there, the end-to-end trained networks, especially with lambda of 50 and 75, shows slightly improved perfor- mance compared to the model minimizing the prediction loss. However, this performance advantage does not translate to the validation dataset, whore once again all models show almost identical results. This lack of generalization ability can be likely traced back to the absence of cost margins, as described Section [.3 which we had to remove due to the amount of instability introduced by this mechanism. The replacement of this mechanism with equivalent, more stable mochanism is thus likely beneficial.
Further, Figure demonstrates that the end-to-end differentiable versions with an interpolation factor À of 50 and 75 can converge faster than the PTT variant. PAT (A = 50) and (A= = 75) converges on average in 7000 + 1512 and 4780 + 1263 training steps, respectively, while PTT requires on average 7580 + 1536 steps to converge. In contrast, PAT (A = 25) requires with 9080 + 1151 more steps than PPT.
Additional performance results are provided in Table 5.1 Here, the performance of the trained models is compared for different phantom and particle density configurations. In addition, the performance results of traditional track follower procedure, which was developed for the particular use-case of particle tracking in the DTC prototype, are provided as bascline.
Similar to the results provided in Figure 4 all model configurations, trained on the lask- and the prediction- loss, demonstrate near identical reconstruction purity and efficiency over all tested phantom and particle configurations. Analogous to Vlastelica et al. (2020), we find that the exact choice of not critical for the reconstruction quality and only impacts the trainings- and convergence speed with larger X (ref. Figure 4 Further, the graph neural networks outperform the baseline algorithm in almost all performance metrics over
13

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Table 1: Reconstruction performance (purity p and efficiency 6) for 100-200 mm water phantoms and 50-150 primaries per frame p*/ /F. Results marked with * use layer-wise reconstruction scheme, with the initial transition generated using ground truth.(t: higher is better; & lower better)
mm Water Phantom
mm Water Phantom
mm Water Phantom
p-
Algorithm
P [%]] (1)
[X] (1)
P [X] (t)
[%] (t)
p [X] (t)
[%) (f)
30
Track follower Pettersen ef al.
2020
87.1+0.00
78.1+0.00
89.4+0.0
81.1+0.0
90.8+0.0
82.14+0.0
RI-bbase Kortus el 2122
92.510.2
81.520.3
93.7-0.2
84.0:0.4
94.440.2
85,440.4
GNN (BCE. odge score)
94.910.1
83.8+0.0
96.2-0.2
86.9=0.2
96.440.0
88.7400.
GNNPAr (A 25.0)
94.9+-0.1
83.8+0.1
96.2+0.1
87.0=#011
96.5++0.1
88.9+-0.1
GNN (A 50.0)
95.010.2
03.920.2
96.320.2
87.120.2
96.510.1
89.040.1
GNNPAr (A 75.0)
95.0+-0.2
83.9+-0.2
96.3+0.1
87.1+0011
96.5+0000
89.0++0.0
100
Track follower Pettersen
2020
80.6+00.0
71.7:+0.0
84.7=000
76.4:+0.0
85.8++0.0
77.5++0.0
RL-based Kortus et al
85.610.3
75.210.5
88.8
79.0=0.5
$9.510.4
80,810.5
GNNPTT [BCE. edge score)
87.4+0.33
75.24-0.2
91.9-0.3
82.4+0.3
91.7+0.2
83.8++0.1
GNN PAY (A = 25.0)
87,3+0.2
75.0:0.2
91.7=002
82.1=0.3
92.2+0.2
84.4+002
GNN (A = 50.0)
87.4+-0.3
75.1+-0.2
92.0+0.1
82.4+0.2
92.5++0.1
84.6++0.1
GNNPAT (A = 75.0)
87.410.2
75.1
91.921011
82.420.1
92.3:102
81.4:002
150
Track follower Pottersen et sl.
2020
75.6+00.0
67.240.0
80.1=0.1
T2.2=0.0
$2.5+0000
74.6+0000
Ra.-based Kortus et al 2023
80.5+00.4
70.8+0.6
83.
74.4+0066
85.3+0.6
76.9++0.5
GNN ppr (BCE. edge score)
77.520.4
65.010.3
84.920.3
15.320.3
87.2:10.2
79.6::0.2
GNN (A 25.0)
76.
64.3+0.2
84.8=033
75.1+0.3
87.6++0.3
80.1+0..3
GNNPAr (A 50.0)
76.8+0.3
64.4+0.3
85.1+0.2
75.5+0.2
88.1++0.4
80.6:+0.4
GNNEAT (A = 75.0)
76.610.11
61.310011
85.0=0.2
75.420.2
$7.840.2
80,440.2
all configurations, with only the configuration of 150 p* /F and 100 mm water phantom slight outlier in this tendeney. Here, the track follower and RL-based tracker demonstrate a higher reconstruction efficiency, while the reconstruction purity is still higher for the GNN architectures.
5.2 Evaluation of Local and Global Loss Landscapes
To analyze and characterize the training and inference behavior of the end-to-end and two-step tracking approaches and gain an understanding of the effects of end-to-end optimization and its hyperparameters, as well as an intuition regarding the usability and effectiveness of the approach an die--iiferenttiale pro- cessing pipeline (as discussed in Section 3.2), we visualize (Li al.) 2018) and characterize the loss landscape structure, closely following the methodology and taxonomy by Yang et al. (2021). p. We specifically focus on the global connectivity of the loss landscape, as it allows to infer conclusions about the optimization of particle tracking in an isolated setting well as integrated operation in processing pipeline, also considering gradient information from downstream tasks. We further augment the evaluation by additional analysis of functional similarities (Klabunde et al. 2023) of the optimized models, allowing us to quantify the risk predictive instabilities of the reconstruction network as well as potential opportunities for E2E fine- tuming, gained by diverse but connocted minima. All used algorithms are detailed in Scction 3.3 Additional implementation details are listed in Appendix
5.2.1 Local Structure of the Task Loss Surfaces for Aecision-focused Learning
Figure visualizes the two-dimensional loss surfaces (Hamming loss) of the first four runs of all configurations in terms of filled contour maps in logarithmic scale. Noticeably, all loss surfaces show similar and consistent shape and patterns in the local 2D loss landscape around the found minima, both over both random initial- izations and choices of the interpolation parameter X. Projected onto two dimensions, the surfaces show mostly convex structure with wide and flat regions along the minima, supporting the previous findings of good training performance of the E2E training configuration. This pattern also often coincides with good generalization performance (Chaudhari et. al.) 2016). However, Dinh et al. (2017) dernonstrates that the notion flatness is not sufficient to reason about the generalization ability itself. We, argue that in this case, the large flat regions likely are conditioned and defined by the range of sensitivity of prodictions along various parameter configurations. We further strengthen this intuition by comparing the hamming loss with the BCE loss surfaces in Section 5.2.2 demonstrating that the hamming loss surface strongly correlates with
14

---

Published in Transactions on Machine Learning Research (07 (025)
0.50
0.50
0.50
0.50
2 Eigenvector
2 Eigenvectol
2 Eigenvector
PAT (A= 25)
0.25
0.25
0.25
0.25
0.00
0.00
0.00
0.00
-0.25
-0.25
ndd
-0.25
-0.25
-0.50
-0.50
-0.50
-0.50
0.5
0.0
1 Eigenvector
1 Eigenvector
1 Eigenvector
Eigenvettor
0.50
0.50
0.50
0.50
2 Eigenvector
2dd Eigenvector
2nd Eigenvector
2nd Eigenvector
PAT (A 50)
0.25
0.25
0.25
0.25
0.00
0.00
0.00
0.00
-0.25
-0.25
-0.25
-0.25
-0.50
-0.50
-0.50
-0.50
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0,5
1 Eigenvector
1 Eigenvector
1 Eigenvector
1 Eigenvector
0.50
0.50
0,50
0.50
2nd Eigenvector
2nd Eigenvector
2nd Eigenvector
2nd Elgenvector
PAT LA == 75)
0.25
0.25
0.25
0.25
0.00
0.00
0.00
0.00
-0.25
-0.25
-0.25
-0.25
-0.50
-0.50
-0.50
-0.50
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
1 Eigenvector
1 Eigenvector
1 Eigenvector
1 Eigenvector
Figure 5: Two-dimensional loss surfaces of PAT framework in logarithmic scale with annotated contour lines. Marked with * are the trained network parameters.
a flattened version of the BCE-loss. While a significant. amount of the projected loss surface is occupied by a flat, low-loss area, all loss surfaces show a significant loss barrier near the trained network parameters, demonstrating the convergence of the networks to outputs close decision boundaries (sco. Section [4.). Providing an alternative to cost margins, discussed in Section may thus be helpful to improve the generalization performance of the network.
5.2.2 Agreement of Prediction Loss and Task Loss Surfaces
While optimizing prediction and task loss use different loss functions on both intermediate and final outputs (hat generally do not coincide Mandi et al. (2023), Figure 4 and Table 5.1 revealed substantial similarities both during training and inference. Figure indicates that the similarities directly translate, and thus are most likely caused by the similar shape of the projected loss landscape of prediction and task loss. For all tracking networks, trained minimizing the prediction loss, we find similar shapes and pattern for both the prediction loss (BCE) and task loss surface (Hamming). We especially emphasize the existence of minima in the loss surface with strongly correlated shapes, demonstrating the best agreement around the minimum itself While the overall shape of the loss surface suggests generally good agreement between the two losses, the surfaces also show significant differences in relative loss for suboptimal solutions. While this did not seem to be significant. for work, theoretical work indicates increasingly growing regrets for models optimizing the prediction loss if the parametric model class is ill-posed and there is little data available (Hu et al. 2022 Elmachtoub et al.) 2023). For well-specified models, experimental results reported in Geng et al. (022)) further strengthen our findings that regret for recision-focused learning strategies in matching problems closcly line up with the ones obtained with two-step training, minimizing the prediction loss.
8The difference of prediction loss and task loss vanishes for perfect predictions of the two-step approach with perfect confidences (either exactly sero one).
15

---

Published in Transactions on Machine Learning Research (07/202) 2025)
PTT (Hamming)
2 Eigenvector
2 Eigenvector
nd Eigervector
n Eigenvector
0.2
0.2
0.2
0.2
0.0
0.0
0.0
-0.2
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.4
-0.5
0.0
Elgenvector
Eigenvector
Eigenvector
Elgenvector
2 Eigenvertor
2 Eigenvector
2nd Eigenvertor
2nd Elgenvector
PTT (BCE)
0.2
0.2
0.2
0.2
0.0
9
-0.2
-0.2
-0.2
3.33
5:6
-0.4
-0.4
-0.4
-0.4
0.0
Eigenvector
1st Eigenvector
Eigenvector
Eigenvector
Figure 6: Visualization and comparison of task (lop) and prediction loss (bottom) for trained models gener- ated using models optimized with the predict-then-tren (PTT) framework in logarithmic scale with anno- tated contour lines. Marked with * are the trained network parameters.
CKA: Rl-module ( 1)
CKA: O-module )
CKA: R2-module ( t
Mmimmax
of -25
0.90
0.85
19 50
0.7
0.80
0.7
0.6
0.18
0.75
0.16
0.70
0.14
0.3
0.65
A-25/A 2-25/A-50p 50 -
79
PTT
25 1- 79 PTT
A 2
2- 50 79 PIT
A
50 79 PTT
0.8
0.9
0.7
0.6
0.175
0.6
0.4
8
0.150
0.5
0.125
0.2
25
50 -4 50
75 25
25
= - 50
p 25
pit pRt-A 50
A 25
25
50 50
75 25
7-7- 50
ptt-A 25
ptt ptt- 50
ptt pit
50
75 25
25
50
50
75 25
75
ptt 25
pit 2- 50
p2-AA
pll
p2-AA
=
p
50
ptt
Figure 7: Representational and functional similarities, expressed in terms of CKA-similarities and min-max
normalized disagrecment, measured for all combinations of training configuration and random initialization (top row) with compressed representation aggregated over all random initializations (bottom row) (f: higher is better; lower is better)
5.3 Representational and Functional Similarities
The similar training and inference performance (Section 5.1 as well as the strong correlation of prediction and task loss surfaces (Section 5.2) indicate strong connections between the F2F, and two-step training
16

---

Published in Transactions on Machine Learning Research (07/2025)
paradigms. However, previous results lack information about parameter and prediction level similarities and differences, thus providing only limited information of the connectivity in the global loss landscape. We thus continue analyzing the inherent similarities of the networks to get a more profound understanding of existing structural differences in both training paradigms.
Figure T visualizes the representational similarities (measured as average CKA similarities between layer representations) and functional similarities (measured as min-max normalized disagreement in predictions). Here, the first row shows all elementwise similarities with an empty diagonal, and the second row shows group wise similarities across random initializations.
Noticeably, bAT-based models reveal consistently moderate to high functional similarities in all network modules. Especially the first two network layers, representing the message passing and aggregation steps of the GNN architecture, show significantly higher similarities compared to their two-step counterpart. This effect consistent both across random initialization and different combinations of interpolation factors X. The representational similarities in the final network module are comparable for both decision- and two-step learning, despite the two-step models exhibiting lower similarities in the preceding layers.
Overall, combinations of PTT and PAT demonstrate the lowest representational similarities, indicating that both E2E and two-step training can find similar performing network parameters, that, however, show significantly larger differences in their internal representation compared to models trained using the same optimization paradigm.
While the average reconstruction performance across multiple readout frames is unaltered by changes in the learned network representations (ref. Table 5.1 , subtle prediction instability caused by different learned prediction mechanisms can have noticeable impact on downstream tasks. Especially for medical applica- tions such as pCT, robustness and stability of the classifier is essential to assure reproducibility and avoid umpredictable changes in the following downstream tasks. To avoid comparing the outcome of downstream tasks, which is computationally expensive and deponds on multiple intermediate stops, we analyze functional similarities or prediction instability of the tracking as a helpful proxy, complementing the previous results.
As illustrated in Figure 7 both PTT and PaT-based models show substantial functional disagreement in a similar range, with median disagrecment rates around 15%. We want to specifically point out the increase in disagreement for models with higher interpolation values (see Appendix G for the statistical significance analysis), suggesting choosing smaller À values for improved stability of the model. Further, the experiments demonstrate, similar to the representational similarity results, a significant increase in disagreement for combinations containing both PAT and PTT variants. Here, we can observe median disagreement rates of approximately 20%, corresponding to an increase prediction instability of approximately 33%.
Interpretation: We find that the substantial degree of prediction instability and different levels of similar- ily in the learned representalions pose considerable risk, polentially influencing the overall performance, effectiveness, and reliability of entire ceaponent-busedd processing pipelines (see Section 3.2 in the applica- tion domain. While we do not expect significant effects on, e.g., the average reconstruction results of pCT, especially for homogeneous materials, the prediction instability could have a measurable impact on the con- trast and spatial resolution. The instability in dounstream reconstruction remains unquantified and left for future work to maintain the conciseness of this initial study. By propogating information from a downstream lask, E2E oplimization provides on important tool to restrict the solution set and thus control the qualily of learned solutions. While some of the instabilities can be avoided to improve reproducibility, other sources of randomness introduced by processing hardware and implementation, e.g., nondetermimnsstc scatter opera- lion of Py Torch Geometric (Klabunde & Lemmerich 2023) still remain and display a noteworthy source of uncertainty and unreliability, propagated throughout the entire processing pipeline.
5.4 Global Connectivity of Local Minima
Given the findings of Section 5.3] demonstrating substantial disagreements in model predictions, we now aim to quantify the global shape and connectivity of the loss landscape. Table Jadd Table Jummarzz the mode connectivity, comparing connecting Bézier curves between model parameters as specified in Section 5.2 To get a more profound understanding of the global landscape for prediction and task loss, we generate curves
17

---

Published in Transactions on Machine Learning Research (07/2025)
using both the BCE-loss the predicted edge-scores and the Hamming loss on the assigned edges. Here, we take advantage of our findings in Section 5.2.1 suggesting strong agreement between Hamming-loss and BCEE-loss for low-loss values. In addition, we provide the linear connectivity between two models as a baseline. Further, Neyshabur et al.] (2020 Juneja et al. (2022) and Lubana et al. (2023) demonstrate in their work that trained models deficient in linear connectivity, are mechanistic dissimilar" This allows us to gain further insides on the stability of the trained models from a loss landscape perspective, supporting the results in Section 5.3
Table 2: Comparison of minimum, maximum, and average hamming loss and true positive rate for models sampled alongside low-energy connecting linear and Bézier curve (three anchor points; minimizing hamming or smary-ccros-entrops: loss) between two modes of same model type. (1: higher is better; & lower is better)
Hamming loss [xle "] (4)
True positive rate [%] (t)
Model
Curve
Min
Max
Mean
Min
Max
Mean
PAT (A- 25)
Linear
6.64
128.58:110.622
37.73:19.92
75.01=11.77
99.32:10.01
95 95.80:2211
PAT (A- 25)
Bézier (Ham.
6.61+00.07
8.14=0.53
7.11+0.10
99.12--0.09
99.32-+0001
99. 09.26++0.01
PAT (A- 25)
Bézier (BCE)
6.65
0.78=:0.13
7.35
99.18=t0.02
99.32:10.01
0.02410.01
PAT (A- 50)
Linear
6.47+00.12
251-09+
102.44
43.46--222466
71.57+10.21
99.31++.001
95.01+22.39
PAT (A 50)
Bézier (Ham.)
6.46:0.12
8.01=0.40
7.00
19.15=00.05
99.3440.001 0.10
99.28:10.01
PAT (A- 50)
Bézier (BCE)
6.47+0008
7.76=0012
7.35++0.05
99.18=+0.02
99.31+00001
99.21+-0.01
PAT (A 75)
Linear
6.40
133.666333.60
82.49:144.28
63. 13.55
99.34
91.1314.60
PAT (A= 75)
Bézier (Ham.)
6.47+0.05
8.22+0.79
7.02++0.16
99.11+00.12
99.31+0.00
0.02
PAT (A 75)
Bézier (BCE)
6.5140.04
8.12=0072
7.40:0.08
99.16:10.05
99.34++0.00
99.23:10.01
PTT
Linear
6.70+0.07
132.39+
107.21+42.20
63.67+11.94
99.32+0000
87.59+4.31
PTT
Bézier (Ham.)
6.70+10.07
15.3315.54
9.56
97. 07.98
99.32
98.9110.18
PTT
Bézier (BCF)
6.69++0.07
7.60+0.21
7.07++0.06
99.21++0.03
99.32+0.01
09.28+0.01
Table 3: Comparison of minimum, maximum, and average hamming loss and true positive rate for models sampled alongside low-energy connecting lincar and Bézier curve (three anchor points; minimizing hamming or binary-rros--entropy loss) between PTT and PAT modes. (t: higher is better; + lower is better)
Hamming loss [x1e (J)
True positive rate [%] (t)
Model
Curve
Min
Max
Mean
Min
Max
Mean
GNN GNN
Linear
6.60
337.23+169.20
76..42276.26
63.90+17.66 66
99.33+0.01
87.12+8.06
GNN PTT > GNN
Bézier (Ham.)
6.60.10.13
11.32.13.25
8.06
98.65 10.55
99.33.10.01
99.13.10.08
GNNPTT GNNPAT
Bézier (BCE)
6.5910.12
7.75
7.1910..0
99.19.10.03
99.33.10.01
99.26110.01
Supplemented by the results in Table 2 we find that connecting curves with consistently low-loss values
and high true positive rates are existent for models with the same training configurations. However, despite the existence of the excellent connectivity with Bézier curves, the linear connectivity is associated with persistently high losses indicating low mechanistic similarity. Analogous to the functional similarity results in Figure 7 we report increasing losses for higher interpolation values, strengthening the findings in Section pointing towards the importance of the interpolation factor despite similar reconstruction performances.
Despite the lack of linear connectivity, given the marginal difference in non-linear connecting curves between all PAT-trained models, all parameter configurations seem to be strongly interlinked with globally well- connected minima. This properly also holds for the PTT configuration; however, were only able to find low-loss connecting curves by using the BCE-loss as the optimization criterion. Optimizing the Hamming- loss, in contrast, yielded substantially worse results, which we cannot explain.
We find similar results demonstrating strong nonlinear connectivity between trained models of PAT and PTT mixtures, indicating a good global connectivity of the modes without noticeable loss barriers that could indicate more pronounced differences in the optimization process (see Table 3). However, the high loss values
9Trained models using different: mechanisms or representations for predictions.
18

---

Published in Transactions on Machine Learning Research (07 2225))
of the linear connectivity (meamPTT-PAP' 117.42=76.26, vs. e.g., meanprr: 107. 107.24+42.2) demonstrate
noticeable impact of mixing the training paradigms on the mechanistic similarity. In addition, we find similar to the connecting curves of PTT minimizing the Hamming-loss noticeable higher loss values compared to the BCE-loss.
Interpretation: While analyzing the global connectivity of loss landscape does not replace the analysis of DeF--iiffrenlaabblly in an enlire processing pipeline, we argue that good global conneclivily of local loss basins which are interconnected via low-loss non-linear path a good initial indication that E2E particle tracking be efficiently augmented by gradients originated from downstream reconstruction Finding parameter configuration in the low-loss region that minimizes the performance of the downstream task is thus likely effective without getting stuck in local regions, that are seporated by high loss borriers. Further, the low mechanistic, representational and functional similarities (see Section 5.3 and 5.4) suggests that a diverse set of reconstruction policies can be reached in those interconnected regions.
analysis loss.
6 Discussion and Conclusion
In this work, we propose and present first studies on E2E differentiable neural charged particle tracking using graph neural networks. We integrate combinatorial optimization mechanisms used for generating disconnected track candidates from predicted edge scores directly into the training pipeline by leveraging mechanisms from decision-focused learning. By optimizing the whole track reconstruction pipeline in an E2E fashion, we obtain comparable results to two-step training, minimizing the BCE-loss of raw edge scores. However, we demonstrate the predict-and-trace approach can provide gradient information that can be propagated throughoul the reconstruction process. While the usage for simply optimizing the reconstruction performance of the network is limited, providing gradient information is highly valuable for various use- cases such as uncertainty propagation, reconstruction pipeline optimization and allows the construction of complex, multistep architectures, potentially allowing to reduce the combinatorial nature of particle tracking by ensuring unique hit assignment.
By examining the global loss landscape, we observe that, despite similar training behaviors leading to globally well-connected minima with comparable reconstruction performance, there are discernible differences in the learned representations leading to substantial prediction instability. This instability is evident across random initializations and is further exacerbated by comparing models optimized for prediction- and task loss, respectively. The observed prediction instability highlights the importance of E2E differentiable solutions, given that the impact of model instability on separate downstream tasks, such as image reconstruction in PCT, is unprodictable and thus especially crucial for safety-critical applications. Therefore, incorporating the functional requirements of downstreem tasks through an additional loss term appears to be highly beneficial. Given the strong training performance of the F2F, architecture as well as the mostly convex shape of the two-dimensional loss surfaces with globally well-connected minima, the combined optimization of tracking and downstream task loss promises to be feasible and effective. However, as this would have exceeded the scope of this paper, we leave this for future work.
Our analysis demonstrates, beside the similar reconstruction results obtained for different interpolation values and the general understanding that the exact choice of the hyperparameter is non-critical (Vlastelica et al., 2020), coupling of the choice of X on the prediction stability of the network and an inverse dependency on the convergence speed. Thus, seleeting large interpolation constants, improves the informativeness of gradient information, resulting in faster convergence. The gain in convergence speed, however, comes the cost of decreased robustness. Selecting lower values for lambda is thus especially important in critical applications where a consistency of outputs is desirable or even essential.
Moving forward, we plan lo expand the framework by integrating E2E tracking into reconstruction code incorporating auxiliary losses for downstream tasks, especially for the use-case of pCT. Additionally, we will continue extending our studics to explore the out-of-distribution robustness and general robustness to distribution shifts introduced by simulation to reality gaps. Further, we plan extending our work to other potential combinatorial solvers for generating disconnected track candidates from predicted edge scores,
19

---

Published in Transactions on Machine Learning Research (07/2025)
well as investigating additional network architectures to gain more profound understanding of the impact of end-to-end optimization on charged particle tracking.
Members of the Bergen PCT Collaboration
Max Aehle", Johan Alme, Gergely Gábor Barnaföldi", Tea Bodova", Vyacheslav Borshchov", Anthony van den Brink", Mamdouh Chaar", Viljar Eikeland", Gregory Feofilov'. Christoph Garth5, Nicolas R. Gauger", Georgi Genov", Ola Grøttvik", Håvard Helstrup", Sergey Igolkin', Ralf Keidel', Chinorat Kobdaj', Tobias Kortus", Viktor Leonhardt Shruti Mehendale", Raju Ningappa Mulawade', Odd Harald Odland b. George O'Neill,, Gáhor Papp' Thomas Peitzmann", Helge Egil Seime Pettersen", Pierluigi Piersimoni Maksym Protsenko", Max Rauch", At- tiq Ur Rehman", Matthias Richter", Dieter Röhrich", Joshua Santana', Alexander Schilling', Joao Seco P, Arnon Songmoolnak b. Åkos Sudár Jarle Rambo Solie", Ganesh Tambave", Ihor Tymchuk", Kjetil Ullaland", Monika Kofaa-Koa Boris Wagner", RenZheng Xiaob. v, Shiming Yang"; Hiroki Yokoyama"
a) Chair for Scientific Computing, TU Kaiserslautern, 67663 Kaiserslautern, Germany b) Department of Physies and Technology, University of Bergen, 5007 Bergen, Norway; c) Wigner Research Centre for Physics, Budapest, Hungary; d) Research and Production Enterprise "LTU" (RPELTU), Kharkiv, Ukraine; e) Institute for Subatomic Physics, Utrecht Suiversity/Niklae Utrecht, Netherlauds; f) St. Petersburg University, St. Petersburg, Russia; g) Scientific Visualization Lab, TU Kaiserslautern, 67663 Kaiserslauteru, Germany; h) Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences, 5020 Bergen, Norway; Center for Technology and Transfer (ZTT), University of Applied Sciences Worms, Worms, Germany; j) Institute of Science, Suranaree University of Technology, Nakhon Ratchasima, Thailand; k) Department of Oncology and Medical Physics, Haukeland University Hospital, 5021 Bergen, Norway; 1) Institute for Physics, Eötvös Loránd University, I/A Pázmány P. Sétány, H-1117 Budapest, Hungary; m) UniCamillus - Saint Camillus International Universily of Health Sciences, Rome, Ital;; u) Department of Physics, University of Oslo, 0371 Oslo, Norway; o) Department of Biomedical Physies in Radiation Oucology, DKFZ German Cancer Research Center, Heidelberg, Germany; p) Department of Physics and Astronomy, Heidelberg University, Heidelberg, Germany; q) Budapest University of Technology and Feonomies, Budapest, Hungary; r) Department of Diagnostic Physics, Division of Radiology and Nuclear Medicine, Oslo University Hospital, Oslo, Norway; s) Center for Medical and Radiation Physics (CMRP), National Institute of Science Education and Research (NISER), Bhubaneswar, India; t) Biophysics, GSI Helmholtz Center for Heavy lon Research GmbH, Darmstadt, Germany; u) Department of Medical Physics and Biomedical Engineering, University College London, London, UK; v) College of Mechanical & Power Engineering, China Three Gorges University, Yichang, People's Republic of China
Acknowledgments
We thank the anonymous reviewers for their thoughtful comments that greatly improved the final version of this manuscript. This work was supported by the German federal state Rhineland- leg SIVERT), the Research Council of Norway (Norges forskningsrád), and the University of Bergen, grant number 250858. The simulations and training were partly exccuted on the high-performance cluster 'El- wetritsch" at the University of Katserslautern-Lnddee Computing Paaltann-aaltitttee (AHRP). We kindly acknowledge the support of the regional university computing center (RHRK). Tobias Kortus and Nicolas Gauger gratefully acknowledge the funding of the German National High-Parformance Computing (NHR) association for the Center NHR. South- West. The ALPIDE chip was developed by the ALICE collaboration at CERN.
(Forschungskol-
which is part of the "Alliance of High-Performance
References
M. Aehle, J. Alme, G.G. Barnaföldi, T. Bodova, V. Borshchov, A. Van Den Brink, M. Chaar, V. Eike- land, G. Feofilov, C. Garth, N.R. Ganger, G. Genov, O. Grøttvik, H. Helstrup, S. Igolkin, R. Kei- del, C. Kobdaj, T. Kortus, V. Leonhardt, S. Mehendale, R.N. Mulawade, O.H. Odland, G. O'Neill, G. Papp, T. Poitzmann, H.E.S. Pettersen, P. Piersimoni, M. Protsenko, M. Rauch, A. Ur Rehman, M. Richter, D. Röhrich, J. Santana, A. Schilling, J. Seco, A. Songmoolnak, J.R. Solie, G. Tam- bave, I. Tymchuk, K. Ullaland, M. Varga- Köfaragó,,, L. Volz, B. Wagner, S. Wendzel, A. Wiebel, R. Xiao, S. Yang, H. Yokoyama, and S. Zillien. The bergen proton CT system. Journal of In- strumentation, 1822),,, 2023a. ISSN 1748-0221. doi: /0.11088//4880011//8/00////////0
URL
20

---

Published in Transactions on Machine Learning Research (07/2025)
Max Achle, Johan Alme, Gergely Gábor Barnaföldi, Johannes Blühdorn, Tea Bodova, Viatcheslav Bor- shehov, Authony van den Brink, Viljar Nilsen Fikeland, Grigori Feofilov, Christoph Garth, Nicolas R Gauger, Ola Slettevoll Grøttvik, Haavard Helstrup, Sergey Igolkin, Ralf Keidel, Chinorat Kobdaj, Tobias Kortus, Lisa Kusch, Viktor Leonhardt, Shruti Mchendale, Raju Ningappa Mulawade, Odd Harald Odland, George O'Neill, Gábor Papp, Thomas Peitzmann, Helge Egil Seime Pettersen, Pierluigi Piersimoni, Rohit Pochampalli, Maksym Protsenko, Max Rauch, Attiq Ur Rehman, Matthias Richter, Dieter Rochrich, Max Sagebaum, Joshua Santana, Alexander Schilling, Joao Seco, Arnon Songmoolnak, Åkos Sudár, Ganesh Tambave, Ihor Tymchuk, Kjetil Ullaland, Mónika Varga-Köfaragó, Lennart Volz, Boris Wagner, Steffen Wendzel, Alexander Wiebel, RenZheng Xiao, Shiming Yang, and Sebastian Zillien. Exploration of dif- ferentiability in a proton computed tomography simulation framework. Physics in Medicine & Biology, 2023b. URL http://iopscience. iop. org/ wwtgg//rticceee/.. 01w.0////111111000000000000
Gianluca Aglicri Rinella. The ALPIDE pixel sensor chip for the upgrade of the ALICE Inner Tracking System. Nuclear Instruments and Melhods in Physics Research, Section A: Acceleralors, Spectrometers, Detectors and Associated Equipment, 5487 2017. ISSN 01689002. doi: 00.1016/j.imma2211.05000 URL http:// //dx: .ww...dooooooooo org/ 1016/] nima. 201.m....0....0 2016.05 016
S. Agostinclli, J. Allison, K. Amako, J. Apostolakis, H. Araujo, P. Arce, M. Asai, D. Axen, S. Bancrjee, G. Barrand, F. Behner, L. Bellagamba, J. Boudreau, L. Broglia, A. Brunengo, H. Burkhardt, S. Chauvie, J. Chuma, R. Chytracek, G. Cooperman, G. Cosmo, P. Degtyarenko, A. Dell' Acqua, G. Depaola, D. Di- etrich, R. Enami, A. Feliciello, C. Ferguson, H. Fesefeldt, G. Folger, F. Foppiano, A. Forti, S. Garelli, S. Giani, R. Giannitrapani, D. Gibin, J. J. Gomez Cadenas, I. Gonzalez, G. Gracia Abril, G. Greenians, W. Greiner, V. Grichine, A. Grossheim, S. Guatelli, P. Gumplinger, R. Hamatsu, K. Hashimoto, H. Hasui, A. Heikkinen, A. Howard, V. Ivanchenko, A. Johnson, F. W. Jones, J. Kallenbach, N. Kanaya, M. Kawa- bata, Y. Kawabata, M. Kawaguti, S. Kelner, P. Kent, A. Kimura, T. Kodama, R. Kokoulin, M. Kossov, H. Kurashige, E. Lamanna, T. Lampen, V. Lara, V. Lefebure, F. Lei, M. Liendl, W. Lockman, F. Longo, S. Magni, M. Maire, E. Medernach, K. Minamimoto, P. Mora de Freitas, Y. Morita, K. Murakami, M. Nagamatu, R. Nartallo, P. Nieminen, T. Nishimura, K. Ohtsubo, M. Okamura, S. O'Neale, Y. Oohata, K. Paech, J. Perl, A. Pfeiffer, M. G. Pia, F. Ranjard, A. Rybin, S. Sadilov, E. di Salvo, G. Santin, T. Sasaki, N. Savvas, Y. Sawada, S. Scherer, S. Sei, V. Sirotenko, D. Smith, N. Starkov, H. Stoecker, J. Sulkimo, M. Takahata, S. Tanaka, E. Tchorniacv, E. Safai Tchrani, M. Tropcano, P. Truscott, H. Uno, L. Urban, P. Urban, M. Verderi, A. Walkden, W. Wander, H. Weber, J. P. Wellisch, T. Wenaus, D. C. Williams, D. Wright, T. Yamada, H. Yoshida, and D. Zschiesche. GEANT4- A simulation toolkit. Nuclear Instru- ments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, (6663))550-000 2003. ISSN 01689002. doi: 10.1016 (0BGG/S068-90000 (000)-00000(0301366---4
Torsten Åkesson, Paula Ecrola, Vincent Hodberg, Göran Jarlskog, Björn Lundberg, Ulf Mjörnmark, Oxana Smirnova, Sverker Almehed, and ATLAS Collaboration. ATLAS compuling: Technical Design Report, volume ATLAS TDR-017; CERN-LHCC-2005-1 of LHCC Reports; ATLAS Techical Design Reports. CERN, 2005. ISBN 92-9083-250-9. Peer reviewed by a referce team of four people appointed by the CERN scientific committee LHCC.
J. Allison, K. Amako, J. Apostolakis, H. Araujo, P. Arce Dubois, M. Asai, G. Barrand, R. Capra, S. Chauvie, R. Chytracek, G. A.P. Cirrone, G. Cooperman, G. Cosmo, G. Cuttone, G. G. Daquino, M. Donszelmann, M. Dressel, G. Folgor, F. Foppiano, J. Generowicz, V. Grichine, S. Guatelli, P. Gumplinger, A. Heikki- nen, I. Hrivnacova, A. Howard, S. Incerti, V. Ivanchenko, T. Johnson, F. Jones, T. Koi, R. Kokoulin, M. Kossov, H. Kurashige, V. Lara, S. Tarsson, F. Lei, F. Tongo, M. Maire, A. Mantero, B. Mascialino, 1. McLaren, P. Mendez Lorenzo, K. Minamimoto, K. Murakami, P. Nieminen, L. Pandola, S. Parlati, L. Peralta, J. Perl, A. Pfeiffer, M. G. Pia, A. Ribon, P. Rodrigues, G. Russo, S. Sadilov, G. Santin, T. Sasaki, D. Smith, N. Starkov, S. Tanaka, E. Tcherniaev, B. Tomé, A. Trindade, P. Truscott, L. Ur- ban, M. Verderi, A. Walkden, J. P. Wellisch, D. C. Williams, D. Wright, H. Yoshida, and M. Peirgentili. Geant4 developments and applications. IEEE Transactions on Nuclear Science, 53(1))270 278, 2006. ISSN 00189499. doi: 10.1109/TNN 200...00000....2...
J. Allison, K. Amako, J. Apostolakis, P. Arce, M. Asai, T. Aso, E. Bagli, A. Bagulya, S. Banerjee, G. Barrand, B. R. Beck, A. G. Bogdanov, D. Brandt, J. M.C. Brown, H. Burkhardt, Ph Canal, D. Cano-Ott, S. Chauvie,
21

---

Published in Transactions on Machine Learning Research (07 2205))
K. Cho, G. A.P. Cirrone, G. Cooperman, M. A. Girald, G. Cosmo, G. Cuttone, G. Depaola, L. Desorgher, X. Dong, A. Dotti, V. D. Elvira, G. Folger, Z. Francis, A. Galoyan, L. Garnier, M. Gayer, K. L. Genser, V. M. Grichine, S. Guatelli, P. Guèye, P. Gumplinger, A. S. Howard, 1. Hřivnáčová, S. Hwang, S. Incerti, A. Ivanchenko, V. N. Ivanchenko, F. W. Jones, S. Y. Jun, P. Kaitaniemi, N. Karakatsanis, M. Karamitrosi, M. Kelsey, A. Kimura, T. Koi, H. Kurashige, A. Lechner, S. B. Lee, F. Longo, M. Maire, D. Mancusi, A. Mantero, E. Mendoza, B. Morgan, K. Murakami, T. Nikitina, L. Pandola, P. Paprocki, J. Perl, I. Petrović, M. G. Pia, W. Pokorski, J. M. Quesada, M. Raine, M. A. Reis, A. Ribon, A. Ristié Fira, F. Romano, G. Russo, G. Santin, T. Sasaki, D. Sawkey, J. 1. Shin, I. I. Strakovsky, A. Taborda, S. Tanaka, B. Tomé, T. Toshito, H. N. Tran, P. R. Truscott, L. Urban, V. Uzhinsky, J. M. Verbeke, M. Verderi, B. L. Wendt, H. Wenzel, D. H. Wright, D. M. Wright, T. Yamashita, J. Yarba, and H. Yoshida. Recent developments in GEANT4. Nuclear Instruments and Melhods in Physies Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 835:186 225, 2016. ISSN 01689002. doi: 10.0016j.nima.2016.066
Johan Alme, Gergely Gábor Barnaföldi, Rene Barthel, Vyacheslav Borshchov, Tea Bodova, Anthony van den Brink, Stephan Brons, Mamdouh Chaar, Viljar Eikeland, Grigory Feofilov, Georgi Genov, Silje Grimstad, Ola Grøttvik, Hávard Helstrup, Alf Herland, Annar Eivindplass Hilde, Sergey Igolkin, Ralf Keidel, Chi- norat Kobdaj, Naomi van der Kolk, Oleksandr Listratenko, Qasim Waheod Malik, Shruti Mchendale, Ilker Meric, Simon Voigt Nesbø, Odd Harald Odland, Gábor Papp, Thomas Peitzmann, Helge Egil Seime Pettersen, Pierluigi Piersimoni, Maksym Protsenko, Attiq Ur Rehman, Matthias Richter, Dieter Röhrich, Andreas Tefre Samnøy, Joao Seco, Lena Setterdahl, Hesam Shafiee, Øistein Jelmert Skjolddal, Emilie Solheim, Arnon Songmoolnak, Åkos Sudár, Jarle Rambo Solie, Ganesh Tambave, Ihor Tymchuk, Kjetil Ullaland, Håkon Andreas Underdal, Monika Vafga-Köfarraó, Lennart Volz, Boris Wagner, Fredrik Mekki Widerøe, Ren Zheng Xiao, Shiming Yang, and Hiroki Yokoyama. A High-Grand Granularayyy Digital Tracking Calorimeter Optimized for Proton CT. Fromliers in Physicc, 8(Oclober):1 20, 2020. ISSN 2296424X. doi: 10.3389 20009////by.22255622224
Brandon Amos and J. Zico Kolter. Optnet: Differentiable optimization as layer in neural networks. 34th International Conference on Machine Learning, ICML 2017, 191, 2017.
Dmitriy Baranov, Pavel Goncharov, Gennady Ososkov, and Egor Shehavelev. Graph neural network appli- cation to the particle track reconstruction for data from the GEM detector. ATP Conference Proceedings, 2163(Oe,,,rr), 2019. ISSN 15517616. doi: 111003/111111000.
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, and Koray Kavukeuoglu. Interaction networks for learning about objects, relations and physics. Advances in Neural Information Processing Systems, pp. 4509-4517, 2016. ISSN 10495258.
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Gonvaez- Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. pp. 400 2018. URL
F Bloch. Zur Bremsung rasch bewegter Teilchen beim Durchgang durch Materie. Annalen der Physik, 408 (3):285 320, jan 1933. doi: 20.1002/andd..18334055
P Buncic, M Krzewicki, and P Vande Vyvre. Technical Design Report for the Upgrade of the Online-Offline Computing System. Technical report, 2015. URL https://cds. cern. ch/record/2011227
Avraam Chalzinichailidiss Janis Keuper, Franz Josef Pfreundt, and Nicolas R. Gauger. GradVis: Visual- ization and second order analysis of optimization surfaces during the training of deep neural networks. Proceedings of MLHPC 2019: 5th Workshop on Machine Learning in HPC Environments Held in con- junction with SC 2019: The International Conference for High Performance Computing, Networking. Storage and Analysis, pp. 66 66-74, 2019. doi: 10.1109/MI Http///..HCCA55..00.0.00000
22

---

Published in Transactions on Machine Learning Research (07/2025)
Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jen- nifer Chayes, Levent. Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. 5th International Conference on Learning Representations, 1CLR 2017 Conference Track Pro- ceedings, 11 2016. URL attps://arxiss www.aawwww.....aaaaa.o.ooooooore abs/ 1611/1111... 01838v5
Gage DeZoort, Savannah Thais, Javier Duarte, Vesal Razavimaleki, Markus Atkinson, Isobel Ojalvo, Mark Neubauer, and Peter Elmer. Charged Particle Tracking via Blag--Classifying Interaction Networks. Com- puting and Software for Big Science, 5(1):1 13, 2021. ISSN 2510-2036. doi: 0011007/411811--01-0077---- URL
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Proceedings of the 34th International Conference on Machine Learning, pp. 1019-1028 PMLR, 2017. URL httpp:://ppocceedngg...lll mlr.
html ISSN: 2640-3498.
Tommaso Dorigo, Andrea Giammanco, Pictro Vischia, Max Achle, Mateusz Bawaj, Alexey Boldyrev, Pablo de Castro Manzano, Denis Derkach, Julien Donini, Auralee Edelen, Federica Fanzago, Nicolas R. Gauger, Christian Glaser, Atilm G. Baydin, Lukas Heinrich, Ralf Keidel, Jan Kieseler, Claudius Krause, Maxime Lagrange, Max Lamparth, Lukas Layer, Gernot Maier, Federico Nardi, Hclge E.S. Pettersen, Alberto Ramos, Fedor Ratnikov, Dieter Röhrich, Roberto Ruiz de Austri, Pablo Martínez Ruiz del Årbol, Oleg Savchenko, Nathan Simpson, Giles C. Strong, Angela Taliercio, Mia Tosi, Andrey Ustyuzhanin, and Haitham Zaraket. Toward the end-to-end optimization of particle physics instruments with differentiable programming. Reviews in Physics, 10(May):100055, 2023. ISSN 24054283. doi: 10.1016/j.revip.22222. 100085. URL htthh::/tp/oooi wt..oo..//..///// revip. 2023. 100085
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. In Proceedings of the 35th International Conference on Machine Learning, pp. 1309-1318. PMLR, 2018. URL https://prroceedings mlr prress/v80/draxee1116 html ISSN: 2640-3498.
Javier Duarte and Jean Roch Vlimant. Graph neural networks for particle tracking and reconstruction, 2020.
ISSN 23318422.
Javier Duarte and Jean-Roch Vlimant. Graph Neural Networks for Particle Tracking and Reconstruction. pp. 387 436. WORLD SCIENTIFIC, February 2022. doi: (0.1142/97898811234033 0012. URL http:
/11.11////18111233331 _012
Abdelrahman Elabd, Vesal Razavimaleki, Shi Yu Huang, Javier Duarte, Markus Atkinson, Gage DeZoort, Peter Elmer, Scott Hauck, Jin Xuan Hu, Shih Chieh Hsu, Bo Cheng Lai, Mark Neubauer, Isobel Ojalvo, Savannah Thais, and Matthew Trahms. Graph Neural Networks for Charged Particle Tracking on FPGAs. Frontiers in Big Data, 5(March):1 17, 2022. ISSN 2624909X. doi: 1..3389/fdata.22222..2
Adam N. Elmachtoub and Paul Grigas. Smart 'predict, then optimize". Management Science, 68:9-26, 10 2017. ISSN 15265501. doi: 10. 20.1127/mssc.2001 (mnsc.2020.392. URL http:: //axxiv.// htt.:/or///bbb////00000555
Adam N. Elmachtoub, Henry Lam, Haofeng Zhang, and Yunfan Zhao. Estimate-thennoptimiz versus penpladdsstinatiiooopppiaaan versus sample average approximation: A stochastic dominance per-
spective. 4 2023. URL http:::/aarii..oor//bb///2.0...
Facebook Al Research FAIR. fvcore. httppr//ggtaao.ooooccccooeeeeres
2019.
Mahdi Milani Fard, Quentin Cormier, Kevin Canini, and Maya Gupta. Launch and iterate: Reducing prediction churn. Advances in Newral Information Processing Systems, 29, 2016.
Steven Farrell, Dustin Anderson, Paolo Calafiura, Giuseppe Cerati, Lindsey Gray, Jim Kowalkowski, Mayur Mudigonda, Prabhat, Panagiotis Spentzouris, Maria Spiropoulou, Aristcidis Tsaris, Jean Roch Vlimant, and Stephan Zheng. The HEP. TrkX Project: Deep neural networks for HL-LHC online and offline tracking. F.J Web of Conferences, 150:1-12, 2017a. ISSN 2100014X. doi: 10.1051 /eppcooff/011150000000 7150000003
23

---

Published in Transactions on Machine Learning Research (07/2025)
Steven Farrell, Paolo Calafiura, Mayur Mudigonda, Dustin Anderson, Josh Bendavid, Maria Spiropoulon, Jean-roch Vlimant, Stephan Zheng, Giuseppe Cerati, Lindsey Gray, Keshav Kapoor, Jim Kowalkowski, Panagiotis Spentzouris, Aristeidis Tsaris, and Daniel Zurawski. Particle Track Reconstruction with Deep Learning. Deep Learning for Physical Sciences Workshop (NIPS), (0PS(N 2017b.
Steven Farrell, Paolo Calafiura, Mayur Mudigonda, Prabhat, Dustin Anderson, Jean-Roch Vlimant, Stephan Zheng, Josh Bendavid, Maria Spiropulu, Giuseppe Cerati, Lindsey Gray, Jim Kowalkowski, Panagiotis Spentzouris, and Aristeidis Tsaris. Novel deep learning methods for track reconstruction. 2018. URL
htt.b///181000666 06111
R. Frühwirth. Application of Kalman filtering to track and vertex fitting. Nuclear Instruments and Methods Physies Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 262 (2):444 450, 1987. ISSN 0168-9002. doi: https://ddo.org//1.10000 (11000011111000000000 01066-0002(77770887--44 URL https:
//ww. www.s/accciiiic.c....ccaaaaaaaaa
Timur Garipov, Pavel Izmailov, Dmitri Podoprikhin, Dmitry P Vetrov, and Andrew Wilson. Loss surfaces, mode connectivity, and fast ensembling of DNNs. In Advances in Neural Informalion Processing Systems, volume 31. Curran Associates, Inc., 2018
Haoyu Geng, Hang Ruan, Runzhong Wang, Yang Li, Yang Wang, Finvolution Group, Lei Chen, and Junchi paradigm for combinatorial optimization prob-
Yan. Rethinking and benchmarking lems. Proceedings of ACM Conference (Conference (17, 1, 11 2023. URL 07633v2
A. Glazov, 1. Kisel, E. Konotopskaya, and G. Ososkov. Filtering tracks in discrete detectors using a cellular automaton. Nucleor Inst. and Methods in Physics Research, A, (12))))22662-668 1993. ISSN 01689002. doi: 10. 0116/0068-9002(93)300
Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. Using mode connectivity for loss landscape analysis, 2018. URL. hattp://arxiv.rr// orggabs/1806 00977777777777777
Bernard Gottschalk. Radiotherapy Proton Interactions in Matter. Xiv, 2018. ISSN 23318422.
D Groom and S Klein. Passage of particles through matter. European Physical Journal C EUU PHYS J C, 15:163 173, 2000. doi: 111007//BF0283419.
Ancesh Hcintz, Vesal Razavimaleki, Javier Duarte, Gage DeZoort, Isobel Ojalvo, Savannah Thais, Markus Atkinson, Mark Neubauer, Lindsey Gray, Sergo Jindariani, Nhan Tran, Philip Harris, Dylan Rankin, Thea Aarrestad, Vladimir Loncar, Maurizio Pierini, Sioni Summers, Jennifer Ngadiuba, Mia Liu, Edward Kreinar, and Zhenbin Wu. Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs. ( 2020. URL http://arxiv..t www.aaawwww.r.oraaaaaaaaee 2011201111........
Benjamin Hilprecht, Christian Hammacher, Eduardo S. Reis, Mohamed Abdelaal, and Carsten Binnig, Diffml: End-to-end differentiable ml pipelines. Proceedings of the 7th Workshop on Data Management for Fnn- To- Fnd Machine Learning, DEEM 2029, 2023. doi:
URL https:
//dl. //dl.acm.. acm. orwo/ooi//oooooooooooo 0010//1..114///555366000 3595857
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Lecture - rmsprop: Divide the gradient by a running average of its recent magnitude. 2012.
P. V. C. Hough. Machine Analysis of Bubble Chamber Pictures. Conf. Proc. C, 590914:554 558, 1959.
Yichun Hu, Nathan Kallus, and Xiaojie Mao. Fast rates for contextual linear optimization. Man- agement Science, 68, 2022. doi: 10. 1287 wttp:/mmcc.0222224/... 4383. URL hattp://pubsonllnne inforns.orrr...... 1287 (2077/msc..2200 2022, http://mw.a.....t.tooooo//maaaaa
0556-4225..ttpss//aaa arg/000-0003-172--055
Sergey Toffe and Christian Szegody. Batch normalization: Accelerating deep network training by reducing internal covariate shift. 32nd International Conference on Machine Learning, ICML 2015, 1:448-456, 2015.
24

---

Published in Transactions on Machine Learning Research (07/2025)
S Jan, G Santin, D Strul, S Staclens, K Assié, D Autret, S Avner, R. Barbier, M Bardiès, P M Bloomfield, D Brasse, V Breton, P Bruyndonckx, I Buvat, A F Chatziioannou, Y Choi, Y H Chung, C Comtat, L Simon, T Y Song, J.-M Vieira, D Visvikis, R Van De Walle, E Wieërs, and C Morel. GATE -Geant4 Application for Tomographic Emission: a simulation toolkit for PET and SPECT. Phys Med Biol. Phys Med Biol, 49(19):45413 4561, 2004.
S. Jan, D. Benoit, E. Becheva, T. Carlier, F. Cassol, P. Descourt, T. Frisson, L. Grevillot, L. Guigues, L. Maigne, C. Morel, Y. Perrot, N. Rehfeld, D. Sarrut, D. R. Schaart, S. Stute, U. Pietrzyk, D. Visvikis, N. Zahra, and I. Buvat. GATE V6: A major enhancement of the GATE simulation platform enabling modelling of CT and radiotherapy. Physics in Medlccine and Biology, 56( 56(4):881 901, 2011. ISSN 13616560. doi: 10. 10.1088/B 0003-9155550/4/0000
R. Jonker and A. Volgenant. A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing, 38(4):325-34) 340, 1987. ISSN 0010485X. doi: O0.1007/FF2278777
Chaitanya K. Joshi, Quentin Cappart, Louis Martin Rousseau, and Thomas Laurent. Learning TSP requires rethinking generalization, volume 210. Schloss Dagstuhl Leibniz-Zentrum für Informatik, Dagstuhl Pub- lishing, Germany, 2021. ISBN 9783959772112. doi: 10.4230/ 10.4230/LIPIcs /tt.//III......I1...... 20.201..33
Xiangyang Ju, Steven Farrell, Paolo Calafiura, Daniel Murnane, Prabhat, Lindsey Gray, Thomas Klijnsma, Kevin Pedro, Giuseppe Cerati, Jim Kowalkowski, Gabriel Perdue, Panagiotis Spentzouris, Nhan Tran, Jean Roch Vlimant, Alexander Zlokapa, Joosep Pata, Maria Spiropulu, Sitong An, Adam Aurisano, Jeremy Hewes, Aristcidis Tsaris, Kazuhiro Terao, and Tracy Usher. Graph neural networks for particle reconstruction in high energy physics detectors. arXiv, (NeurIPS 2019):1-6, 2020. ISSN 23318422.
Xiangyang Ju, Daniel Murnane, Paolo Calafiura, Nicholas Choma, Sean Conlon, Steven Farrell, Yaoyuan Xu, Maria Spiropulu, Jean Roch Vlimant, Adam Aurisano, Jeremy Hewes, Giuseppe Cerati, Lindsey Gray, Thomas Klijnsma, Jim Kowalkowski, Markus Atkinson, Mark Neubauer, Gage DeZoort, Savannah Thais, Aditi Chauhan, Alex Schuy, Shih Chich Hsu, Alex Ballow, and Alina Lazar. Performance of geometric deep learning pipeline for hl-lhc particle tracking. The European Physical Journal C 2021 81:10, 10 2021. ISSN 1434-6052, doi:
URL atttps://inme htip:://iink.springer springer... com/
article/10..
Jeevesh Juneja, Rachit Bansal, Kyunghyun Cho, João Sedoc, and Naomi Saphra. Linear connectivity reveals generalization strategies. International Conference on Learning Representations, 2022. doi: 10. 00555//AXXIV.2205.1222
Jan Kieseler. Object condensation: one-stage grid-free multi-object reconstruction in physics detectors, graph, and image data. European Physical Jowrnal C, 80(9):1-12, 2020. ISSN 14346052. doi: 10.1140/ Ppic//s0005200200044611 URL http://///ii.rr/1....11.1. http://0...1100///p////000000000
Dicderik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015- Conference Track Proceedings, pp. 1-13, 2015.
Max Klabunde and Florian Lemmerich. On the prediction instability of graph neural networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 13715 LNA1:187 -02, 2023. ISSN 16113349. doi: 10. 11--00079778-30012660000 12/ FIGURES/11. URL ftttp:://iin.sspringee wwtp://rcccc/cccpt////.... 100701...079988901 1001100777-----111-----------0-0
Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of neu- ral network models: A survey of functional and representational measures. 5 2023. URL https:
//arxiv.org/abs/23000 20t.b//2220550002922222 06329v2
Simon Korublith, Mohammad Norouzi, Honglak Lee, and GeolTrey Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 3519-3529. PMLR, 09-15 Jun 2019. URL https://proceeding. mlr. http://oooooi........pe////////p preess//9///oommtttll...
html
25

---

Published in Transactions on Machine Learning Research (07/2025)
Tobias Kortus, Alexander Schilling, Ralf Keidel, Nicolas R Gauger, and on behalf of the Bergen pCT collaboration. Particle Tracking Data: Bergen DTC Prototype, dec 2022. URL https://doi org/10.
5281/zenodo. 7426388
Tobias Kortus, Ralf Keidel, and Nicolas R. Gauger. Towards Neural Charged Particle Tracking in Digital Tracking Calorimeters with Reinforcement Learning. IEEE Transactions Pattern Analysis and Machine Intelligence, (1)1):11220-118333 2023. ISSN 19393539. doi:
James Kotary, Ferdinando Fioretto, Pascal van Hentenryck, and Bryan Wilder. End-to-end constrained optimization learning: A survey. IJCAI International Joint Conference on Artificial Intelligence, pp. 4475-4482, 3 2021. ISSN 10450823. doi: 1449666/jcaii20222 /610. URL https://arxiv..orr wtw..br/g/b.....////./////////
16378v1
David J Lange and (Representing the CMSCollaboratiion) The cms reconstruction software. Journal of Physics: Conference Series, 331:032020, 12 2011. ISSN 1742-6596. doi: 10. 10.1088/1742-6596/
3/ 3/032020. URL https::/iopsccieccc iop wtw.::oppppooooooooooooooooo
iopscience.. wop.pooppooooo. www.ooraaggg//oooooooo 2tt.://c.........000/71777766669 010.0000011222266666666/00000000
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in Neural Information Processing Systems, 2018 N018-Decmmbe(NeurPPP 2018 2018):6389- 6399, 2018. ISSN 10495258.
Kilian Licret and Gage DeZoort An Object Condensation Pipeline for Charged Particle Tracking at the High Luminosity LHC. 2023. URL httt::///arxvv.orr/bbb/220.0
Kilian Lieret, Gage DeZoort, Devdoot Chatterjee, Jian Park, Siqi Miao, and Pan Li. High pileup particle tracking with object condensation. pp. 2023, 12 2023. URL https:://axivv.org/ org. wttggaab//12....0///
Daniel Lopez-Martinez, Alex Yakubovich, Martin Seneviratne, Adam Lelkes, Akshit Tyagi, Jonas Kemp, Ethan Steinberg, N Lance Downing, Ron Li, Keith E. Morse, Nigam H Shah, and Ming-Jun Chen. Instability in clinical risk stratification models using deep learning. Proceedings of Machine Learning Research, 193:552-565 2022.
Ekdeep Singh Lubana, Eric J Bigelow, Robert P Dick, David Krueger, and Hidenori Tanaka. Mechanistic mode connectivity. Proceedings of the 40th International Conference on Machine Learning, 2023.
M. Mager. ALPIDE, the Monolithic Active Pixel Sensor for the ALICE ITS upgrade. Nuclear Instru- ments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Asso- ciated Equipment, 824(2016):434 434, 2016. ISSN 01689002. doi: 1..006/..nima.2155.00000 http:://xxxddi.og//e/// http://dx: doi wwi. www...oooo.....000000.00.......0 1016/ -nima. 2015.09 09.057
URL
Jayanta Mandi, James Kotary, Senne Berden, Victor Bucarey, Universidad O de, Ku Leuven, and Bel- gium Ferdinando Fioretto. Decision-focused learning: Foundations, state of the art, benchmark and future opportunities. 2023. URL
wwwp:///wwwggbbb......e.. 13565v2
Aditya Modi, Jivat Neet Kaur, Maggie Makar, Pavan Mallapragada, Amit Sharma, Enre Kiciman, and Adith Swaminathan. Towards modular machine learning pipelines. In 2023 International Confer- ence on Machine Learning, 7 2023. URL httpt::/t////tt http://wwwwmmmcootttte/ www.://sff...co./aaaeeeeeeeeeeee /tr.///aaaar/rreeerrraapalaaaaaa
novarde-modular-macchin machhaarrmmccin---leaaeeeeeppppp
Philipp Moritz, Robert Nishihara, Stephanic Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zougheng Yang, William Paul, Michael Jordan, and Ion Stoica. Ray: a distributed framework for emerging AI applications. In Proceedings of the 13th USENIX conference on Operating Systems Design and Implementation, OSDI'18, pp. 561 577. USENIX Association, 2018. ISBN 978-1-931971-47-8.
Maxime Mulamba, Jayanta Mandi, Michelangelo Diligenti, Michele Lombardi, Victor Bucarey, and Tias Guns. Contrastive losses and solution caching for predict-and-optimzze IJCAI International Joint Con- ference on Artificial Intelligence, 3:2833 2840, 2021. ISSN 1045-0823. doi: 10.24963/IJCAA.2022//
URL https::/ sttpp:://ww... http://www.......rroooooo///////
26

---

Published in Transactions on Machine Learning Research (07/2025)
Daniel Murnane, Savannah Thais, and Ameya Thete. Equivariant graph neural networks for charged particle tracking, 2023.
Behnam Neyshabur, Hanie Sedghi, Google Brain, and Chiyuan Zhang, What is being transferred in transfer learning? Neural Information Processing Systems, 2020.
Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. ICLR 2021 9th International Conference on Learning Representations, 10 2020. URL
http://wwrr//bbb......... 15327v2
Besmira Nushi, Ece Kamnrr, Eric Horvitz, and Donald Kossmann. On human intellec and machine failures: Troubleshooting integrative machine learning systems. 31st AAAI Conference on Artificial Intelligence, AAAI 2017, pp. 1017-1025, 2017. ISSN 2159-5399. doi: 10. http//aaaa....1111111
Helge E.S. Pettersen, llker Meric, Odd Harald Odland, Hesam Shafiee, Jarle R. Sølie, and Dieter Röhrich. Proton tracking algorithm in a pixel-based range telescope for proton computed lomography. arXiv, 2020.
ISSN 23318422.
William P Pierskalla. The Multidimensionall Assignment Problem. Operations Research, 16(2):422 431, dec 1968. ISSN 0030364X, 15265463. URL httt:ttttt/ww.jseeee org/stable//68768
Jean François Pusztaszeri, Paul E. Rensing, and Thomas M. Liebling. Tracking elementary particles near their primary vertex: A combinatorial approach. Journal of Global Optimization, 9(1):41-64, 1996. ISSN 09255001. doi:
Shah Rukh Qasim, Nadezda Chernyavskaya, Jan Kieseler, Kenneth Long, Oleksandr Viazlo, Maurizio Pierini, and Raheel Nawaz. End-to-end multi-particle reconstruction in high occupancy imaging calorimeters with graph neural networks. European Physical Journal C, 32(8):1 15, 2022. ISSN 14346052. doi: 10.1140/ 10665-7
epic/s10052--02- 022-00652--22-166--777 URL /ttps://doi.org 0040/.1110/000/
Lea Reuter, Giacomo De Pietro, Slavomira Stefkova, Torben Ferber, Valerio Bertacchi, Giulia Casarosa, Luigi Corona, Patrick Ecker, Alexander Glazov, Yubo Han, Martina Laurenza, Thomas Lueck, Ludovico Massaccesi, Suryanarayan Mondal, Bianca Scavino, Stefano Spataro, Christian Wessel, and Laura Zani. End-to-end multi-track reconstruction using graph neural networks at belle ii, 2024. URL https: org/ 2411. 13596
Michal Rolínek, Vit Musil, Anselm Paulus, Marin Vlastelica, Claudio Michaelis, and Georg Martius. Op- timizing rank-based metries with blackbox differentiation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 7617-7627, 2020a. ISSN 10636919. doi: /ttp://CC222222222000000000
Michal Rolinek, Paul Swoboda, Dominik Zietlow, Anselm Paulus, Vit Musil, and Georg Martius. Deep graph matching via blackbox differentiation of combinatorial solvers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 12373
LNCS:407-424 2020b. ISSN 16113349. doi: 10.1007 10.1007/978-3-0330 -90-777----00--000---11 25.
Subham Sahoo, Anselm Paulus, Marin Vlastelica, Vit Musil, Volodymyr Kuleshov, and Georg Martius. Backpropagation through combinatorial algorithms: Identity with projection works. In Proceedings of the Eleventh International Conference on Learning Representations, May 2023. URL httpss// net/forrumid=:J2MR77
Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does batch normalization help optimization? In Advances in Newral Information Processing Systems, volume 31. Curran Associates, Inc., 2018
D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean- Françira Crespo, and Dan Dennison. Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems, 28, 2015.
27

---

Published in Transactions on Machine Learning Research (07/2025)
Sanket Shah, Kai Wang, Bryan Wilder, Andrew Perrault, and Milind Tambe. focused learning without differentiable optimization: Learning locally optimized decision losses. Advances in Neural Infor- mation Processing Systems, 35, 3 2022. ISSN 10495258. URL
Jonathan Shlomi, Peter Battaglia, and Jean-Roch Vlimant. Graph neural networks in partiele physics. Machine Learning: Science and Technology. 2(2):021001, 2021. doi: 00.1088/633--115//bbbb
Are Strandlie and Rudolf Frühwirth. Track and vertex reconstruction: From classical to adaptive methods. Reviews of Modern Physics, 82:1419-1458, 2010. ISSN 00346861. doi:
Samuel Van Stroud, Philippa Duckett, Max Hart, Nikita Pond, Sébastien Rettie, Gabriel Facini, and Tim Scanlon. Transformers for charged particle track reconstruction in high energy physics, 2024. URL
https: //t.//aaavvvv/////aa/eeeeaaaaa http://arxiiooorraaaaaaaa/ 07149
Savannah Thais, Paolo Calafiura, Grigorios Chachamis, Gage DeZoort, Javier Duarte, Sanmay Ganguly, Michael Kagan, Daniel Murnane, Mark S. Neubauer, and Kazuhiro Terao. Graph neural networks in particle physics: Implementations, innovations, and challenges, 2022.
Ashish Vaswani, Noam Shazecr, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 2017- (595 -009,, 2017. ISSN 10495258.
Marin Vlastelica, Anselm Paulus, Vit Musil, Georg Martius, and Michal Rolinek. Differentiation of Blackbox Combinatorial Solvers. 8th. International Conference on Learning Representations, ICLR 2020, pp. 1-19, 2020.
Volgenant. Linear and semi-assignment problems: A core oriented approach. Computers and Operations Research, 23109911,,-,,, 1996. ISSN 03050548. doi:
Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, TA 2019 and the 9th AAAT Symposium on Educational Advances Artificial Intelligence, EAAI 2019, pp. 1658-1666, 2019. ISSN 2159-5399. doi: v33101 htt.aaa31311111111111111 33011658
10.1609/ (660999A/ALL A33.0133.11.33 111001.3301110 1658. URL pittps://dl.. acm wttp://////o...100/.000009
Yaoqing Yang, Liam Hodgkinson, Ryan Theisen, Joe Zou, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. Taxonomizing local versus global structure in neural network loss landscapes. Tiitty-Fifth Conference on Newral Information Processing Systems, 2021.
Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W. Mahoney. Pyhessian: Neural networks through the lens of the hossian. 2020 IEEE International Conference on Big Data (Big Data), pp. 581-590, 2020. URL httpp:////pppp sewanticscholastt
Cttp//rrpuusDDDC 209376531
Gyeong-In Yu, Saeed Amizadeh Microsoft, Sehoon Kim, Artidoro Pagnoni, Ce Zhang, Eth Zurich, Byung- Gon Chun, Markus Weimer Microsoft, Matteo Interlandi Microsoft, Saeed Amizadeh, Markus Weimer, and Matteo Interlandi Windtunnel. Windtunnel: Towards differentiable ml pipelines beyond a single model. Towards Differentiable ML Pipelines Beyond Single Model. PVLDB, 15:11 20, 2022. doi: 10.
4478833485450.34854422 URL https: //gtthub.btt/tt/// hotp//acccooottttmmmimmni/////ee
28

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A Edge Filter Results
In Section 4.1 we mentioned the necessity of performing edge filtering, minimizing the size of both the hit- and line graph representations, improving reconstruction speeds and minimizing memory requirements during training and inference. Selecting sufficient. thresholds, given the stochastic nature of particle scattering material, requires a trade-off between reduced graph size and the number illegitimately removed true edges. Figure shows the fraction of removed number of edges in the graph opposed to the fraction of removed true edges, generated for 100 graphs of the train dataset selected uniformly over all phantom thicknesses. Based on the results in Figure we select the thresholds 0₂ 400 mrad for the edges in the calorimeter layer and 200 mrad for all edges contained in the tracking layer, significantly reducing the number of total edges, while limiting the impact on the final reconstruction quality to a minimum. Error rales for both Og and 0a are presented in Figure
0g = 200
100
True edges rem. (9.)
90
Total edges rem. (0.)
80
True edges rem. (6.)
70
0g = 400mrad
Total edges rem. (8+)
Fraction [%]
93" 2.4%
60
50
40
30
1 200mrad
9. 4.2%
20
10
0
O
200
400
600
800
1000
1200
1400
1600
Edge threshold [mrad]
Figure 8: Fraction total and true edges removed from GH given the edge filter 02 and O.. Measured for 100 sampled graphs from the training dataset with 100p FF and 100, 150 and 200 mm water phantoms.
B Hyperparameters
Table 4: Selected hyperparameters used in the studies in Section 5 All hyperparameters, except À are kept consistent across training runs for PAT and PTT. is selected based on the value ranges in (Vlastelica et al. Rolinek et al. 2020b) as we consider it as one of the main parameters for E2E training.
2020
Name
Value
Description
0a
400 mrad
Filter threshold for edges in the calorimeter layers. Filter threshold for edges in the tracking layer.
0,
200 mrad
Mhidden
(R1)
3
Number network layers
Mhidden
(0)
3
Number of network layers
Mhidden
R2)
3
Number of network layers
dhiaden
16
Hidden size of the network layer
scaling
0.001
Scaling factor of the sinusoidal encoding (see Section 4.1
batch size
32
Number of reconstructed graphs a single batch Learning rate used for parameter updates with RMSProp
Ir
1c-3
(25, 50, 75)
Interpolation factor for blackbox differentiation of LSA layer
29

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
C
Reconstructed Particle Tracks
500
uoqioodz
uoqisod-z
400
wotties (mm)
400
400
300
300
300
200
-298.gg 2o
-2% go
29
y-position [mm]
2010
0-1620
y-position [mm]
2010
1220
y y-position [mm]
20
2010 0- H220
x-position [mm]
x-position [mm]
x-position [mm]
500
400
uogiiod-z
usrisod
400'
400
350
300
300
250
200
-298,. go
- 229 20
22% go
1520
1920
2010 1e20
y-position [mm] x-position [mm]
2010
y-position [mm] x x-position [mm]
2010
y-position [mm] x-position [mm]
Figure 9: Reconstructed particle tracks generated for multiple readout frames with 100mm water phantom and 100 pt /F using PAT (A 25) model. Marked in green correctly reconstructed track segments, marked in red are incorrectly reconstructed track segments and marked in orange are correct reconstructions (same particle ID) following the wrong primary particle.
D Ablation Study: Hit-graph and Line-graphs Representations
While demonstrating good training performance, utilizing line-graph representation for particle track- ing comes at the cost of increased graph size and thus computational overhead over the original hit-graph representation. To verify the necessity and usefulness of this transformation over default hit graph param- eterization, this section provides additional ablation results comparing hit- and line-graph representations in terms of convergence behavior, reconstruction performance and inference speed.
Reference results utilizing hit-graphs are generated for all model configurations outlined in Section by replacing the modified architecture with the original interaction- implementation (Battaglia et al. 2016 2018) used in DeZoort et al. (2021). The assignment operation, described in Section 4.2 is left is. We paramcterize both vertex- (v;) and edge features (eiz) following previous work by [oortus et al. 2023), using energy deposition and global position (AE, x, y, =) and a description of the edge in terms of spherical coordinates (r, 0, o) orthogonal to the detector plane respectively (see Figure 2) Table 5 summa- rizes the average node and edge count (+ standard deviation) for cach evaluated configuration. For every interaction-network configuration operating on hit-graph, the same hyperparameters from Appendix B are utilized except for the hidden dimension dhidden; which is increased to 64. While we found that the
30

---

Published in Transactions on Machine Learning Research (07 (0/5))
interaction network operating on the hit-graph benefits from the increased hidden dimensionality, find similar to DeZoort et al. (2021) that the model saturates around the selected value and further increase in parameter count does not yield any noteworthy improvement.
Table 5: Average node- and edge count of hit- and line-graph representations (# standard deviation), for simulated data with 100, 150 and 200 water phantoms and 5, 100 and 150 primaries per frame (p* /F).
100 mm Water Phantom
150 mm Water Phantom
200 mm Water Phantom
p*/F
Repr.
#nodes
#edges
#nodes
#edges
#nodes
#edges
50
GH
1337=54
4311#497
1074+38
3079=397
815+28
2235+2284
G1
431111497
68328116107
30791397
39584110795
2235-1284
2481517541
100
2675=76
14837+1333 1350
2148+-53
10397+1015
1628+440
7514+-803
Gr.
14837-1350
493910=86354
10397+1015
279974+53359
7514++03
172926+-41226
150
4015180
31532-1211
3224162
21906=1836
2445144
15770=1400
Gr
31532=2413
1611761=232139
2190611836
9089531153476
15770:14000
552870-196190
1.50
TAT
Train-set perf.
True rale (96]
Falue pbs. rate [%)
1.25
PATEA 75)
CA 320
Track effic sency (%)
70
1.00
purity
60
PAT 5:3
PAT (1. 251
PAT (3 731
0.50
50
PAT 12- 751
FAT U.- 50
PAT 11 250
50
PAT I4 501
PAT 1h
PAT
40
FTT
40
0.25
Optimisation step 1e4
Optimisation step le4
Optimisation step 1e4
Optimisation step l64
1.50
Val-set perf.
True 1%6]
98
False pos. rale [66]
1.25
[%6]
80
Track efficiency [%)]
70
70
1.00
60
96
0.75
60
50
0.50
94
40
0.25
Optimisation step 1e4
Optimisation step 1e-1
Optimisation step 1e4
Optimisation step lef
Figure 10: True positive and false positive rate of assignments, as well as purity and efficiency of track reconstruction, evaluated for PAT and PTT with hit-graph representation on validation set as a function of optimisation steps.
Training performance: Figure 10 visualizes analogous to Figure 4 the performance metrics on training and validation as a function of training steps. We find that restricting the readout representation hit-graph severely impedes training performance. Across all model configurations, we find a significant drop across all measured quantities. Especially, end-to-end optimization strategies suffer from the reduced information density encoded in the line-graph representation. This results both in decreased convergence speed and stability, as well as reduced reconstruction quality.
Reconstruction performance: Table D further confirms the previous finding, comparing the reconstruc- tion quality of the hil-graph baseline (Gn) with the line graph (Gr.) results, first presented in Table 5.1 We find a substantial drop in reconstruction purity and efficiency across all tested phantom geometries and particle densitics when the line graph transformation is omitted. This effect is especially pronounced for the end-to-end trained baseline, confirming the necessity of the line-graph transformation and the importance of the direct parameterization of segment-pair features, capturing the scattering behavior of particles, for
31

---

Published in Transactions on Machine Learning Research (07 (025))
Table 6: Reconstruction performance for hit- and line-graph representations, measured purity p and efficiency for water phantoms of 100, 150 and 200 mm thickness and 50, 100 and 150 primaries per frame p²/ DF... The results for GH were previously presented in Table 5.1
(1: higher is better; lower is better)
100 mm Water Phantom
150
Water Phantom
200 Water Phantom
/
Repr.
Algorithm
p [X] (t)
[%] (t)
[%]] (1)
[%] (1)
[%]] (t)
[%] (1)
50
Gst
GNNeTT (BCE, edge score)
81,955.8
71.6=5.3
89.920.8
80.920.7
92.3+0.1
84.8+0.1
GNNPAT (A 25.0)
73.514.11
64.0.3.8
83.9.1..0
75.611.8
88.8122.1
81.7.12.0
GNNPAT (A 50.0)
74.915.7
65.2.5.6
83.0.1..8
74.712.6
87.5.12.5
80.412.3
GNNPAT (A 75.0)
74.8+3.1
65.3=2.7
81.3+3.9
73.1+3.5
84.9+44.5
77.9++4.3
Gr
GNNPrr (BCE. edge score)
94.9++.11
83.8=0.0
96.2++:.2
86.9+0.2
96.4++0.0
88.7++0.1
GNNPAT (A -25.0)
94.920.1
83.8=0.1
96.240.1
87.0=0.1
96.5:0.11
88.920.1
GNNPAT (A =50.0)
95.010.2
83.9.0.2
96.3.10.2
87.110.2
96.5.10.11
89.0.1..1
GNNPAT (A -75.0)
95.010.2
83.9.0.2
96.3110.1
87.110.11
96.5.10.0
89.011.0
100
G
GNNeTT (BCE. edge score)
66.2+7.8
56.0-6.9
79.8+
71.++1.
83.8+00.1
76.2+0.1
GNNPAT (2 =25.0)
53.523.8
45.2-3.5
70.342.6
62.74244
77.013.3
70,243.1
GNNPAT (2 =50.0)
35.7=6.8
46.8=6.4
70.1+3.7
62.543.4
75.243.9
68.543.6
GNNPAT (A =75.0)
54.913.3
46.4.12.9
66.5
39.2.1.00
714.1677
64.91662
91.
GNNeTT (BCE. edge scorc]
87.410.3
75.2=0.2
91.910.3
82.410.3
91.7100.2
83.810.1
GNNEAT (A =25.0)
87.3+0.2
75.0-0.2
91.7++0.2
82.1+0.3
92.2+0022
84,4+0.2
GNNPAT (A =5500.)
87.4+0.3
75.1=0.2
92.0++0.1
82.4+0.2
92.5++0.1
84.6++0.1
GNNPAT () 17755.)
87.420.2
75.1=0.2
91.9+0.1
82.420.1
92.340.2
84.4+0.2
150
Gн
CNNPTT (BCE. odge score)
52.617.8
43.1-6.77
60 00.2.1..0
60.811.8
75.8100.
68.7.00.4
GNNPAT (A 25.0)
40.013.4
32.7=3.1
57 07.182..8
30.212.5
67.113.88
61.113.5
GNNPAT (A (A= =50.0)
42.0+5.7
34.2-5.3
56.9+-4.0
50.1+3.6
65.6+4.55
59.7+4..2
GNNPAT (A (A=75.0
41.022.8
33.6=2.5
53.525.4
47.1
61.547.5
55.826.9
9r.
GNNPTT (BCE. edge score)
77.5=0.4
65.0=0.3
84.9+0.3
75.3=0.3
87.2+0.2
79.6+0.2
GNNPAT (A =25.0)
76.710.2
64.310.2
84.810.3
75.110.3
87.610.3
80.110.3
GNNPAT (1 -50.0)
76.810.3
64.420.3
85.110.2
75.51002
88.110.4
80.610.4
GNNPAT (A = =75.0)
76.6+0.1
64.3-0.1
85.0+00.2
75.4+0.2
87.8+0.2
80.4+0.2
both two-stage and end-to-end approaches. In line with (Kortus al. 2023): highlighting the importance of positional encodings for segment pairs, we specifically credit the significant gain performance the induc- tive bias, introduced by the encodings, simplifying learning the underlying interaction mechanisms defined by multiple Coulomb scattering (see. Section 3.1 This effect further enables the use of smaller models, enhancing parameter efficiency compared to networks that operate on the hit-graph parameterization.
Computational complexity: Figure 11 ooutlines the estimated floating-pois operations (FLOP) of graph and lino-graph interaction networks, operating on various phantom and density configurations T0 We provide averaged results obtained over all readout frames for each given configuration.
Ultimately, Figure 12 presents and compares the average reconstruction times of readout frames for both hit-graph and line-graph representation quantify the runtime penalty of line-graph representations. Both runtimes for the interaction-netwwee are presented for various water phantom geometries and particle densitics. All results are generated on an NVIDIA A100 GPU for all readout frames in the test set, with 10 evaluations for each readout frame.
prediction (including the line-graph transform) and total reconstruction
We find across all configurations that the line- aheegraph interaction-network noticeably impedes the runtime of the predictive phase, both in terms of FLOP and runtimes, due to the significantly incrcased and denser connectivity of the graph (see Table 5). Yet, the overall impact the total reconstruction runtime is manageable for the defined scope, outweighing the increase in runtime by the significant gain in reconstruction quality.
All FL.P estimates calculated using the PlopCountAnallattt tool provided as part
PAIR 2019
32

---

Published in Transactions on Machine Learning Research (07 (025)
100 mm Water Phantom
150 mm Water Phantom
200 mm Water Phantom
Pred. (Gr)
Pred. (Gh)
Fred. (@x)
(@2)
(())
(a.)
GFLOP
50
100
150
50
100
150
50
100
150
Particle density (p (F)
Particle density (p 1/F
Particle density (p [p*/F]
Figure 11: Estimated number of floating-point operations (FLOP) for hit-graph and line-graph interaction networks, evaluated for readout frames of various phantom geometries and particle densities.
100 mm Water Phantom
150 mm Water Phantom
200 mm Water Phantom
Avg. runtime [ms]
80
Pred.
Pred.
Fred.
Gr Reco.
Gr Reco.
Gr Reco.
60
Pred.
G Pred.
Pred.
G Reco.
20
50
100
150
100
150
50
100
150
Particle density [p /F]
Particle density [=-/F] IF]
Particle density [p*/F] /F]
Figure 12: Average runtime of edge-classification charged particle tracking (predictive stage and full recon- struction.) operating hit-graph (GH) and linc-graph (9z) represcntation for readout frames of various water phantom geomelries and particle densities.
E Implementation Details for Loss Landscape Evaluations in Section 5.2
This section intended to provide additional implementation details, specifying the implementation and evaluation of the approaches for evaluating the loss landscape in Section 5.2 and all its subsections.
General remarks: As all experiments require a significant amount network evaluations, often including tens or hundreds of iterations over the data in the training set, we select for Sections 5.2.2 5.3 and 5.4 subset of 100 uniformly selected graphs from the train set to remain with feasible runtimes, while still providing enough data.
Loss surfaces: We generate the loss surfaces following the implementation details described in Yang al. (2021). This includes especially the usage of fixed batch normalization parameters across all sampled a, B configurations. We generate the eigenvectors of the hessian used as the spanning vectors for the loss landscape using an implementation based on the PyHESSIAN framework (Yao et al. 2020 adapted for our use case. All loss surfaces are generated using 50 x 50 parameter values and parallelized on single machine with multiple GPUs using the Ray software framework (Moritz et al.) 2018
Ropresennational and functional similaritics Our implementation for quantifying representational and functional similarities matches the description of (Nguyen el. al. 2020) and (Klabunde & Lemmerich, 2023). However, given the large cardinality and connectivity of the line graph representations, we sample each itcration of the CKA algorithm only a subset of x 1024 features Each combination of networks is evaluated as individual jobs using the SLURM resource manager on the Elwetritsch HPC cluster of the University of Kaiserglautern-Landee
remain with feasible runtimes.
33

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Mode connectivity: We closely follow the implementation for generating connecting, curves provided by (Garipov et al. 2018). However, instead of updating the batch normalization parameters for every we follow similar approach used for generating the loss landscapes in Yang et al. (2021]. We use constant parameters determined by the first curve anchor, providing us with reasonable normalization values. Each curve is optimized using 1000 iterations with batch size of 8 graphs. We found that increasing the training ilerations resulted with worse connectivity results for PTT-based combinations, minimizing the hamming loss. We use the same parallelization strategy as used for calculating representational and functional similarities.
F
Reproducibility
We analyze in this work several trained models well as results based on various evaluation mechanisms, each requiring significant amount of computing resources. We thus provide for transparency all trained models together with the evaluation results and data under Additionally, we provide all the source code for generating the tables and figures used throughout this work, which can be re-run without regenerating any of the required result data. All source code that supports the findings this study will be openly available open source after paper acceptance under https: wtt.ubb.cc//SSa8111-////------ee
127591
G
Statistical Testing for Similarity Scores
Given the seemingly increasing predictive instability of the tracking network with increasing interpolation factor X, we suspect a dependency between those two dependencies. We verify this hypothesis, using Welch's 's t-test, demonstrating that the prediction instability decreases with smaller À values (see. Figure 13]
t-statistic (a b)
p-value (a b)
1.0
50 50
A 50 +2
20
= 75 =
A 75 50
0
75
0.4
ptt
plt pt 50
20
ptt Rt
ptt ppt-
8
50
-A 18
25
PLL-A 50
ptt-A ptt
75
10
8
50
pit att-2 75
ott pot.
ptt-2 75-75-1
7
nit-A
nt-+2
Figure 13: p-values and t-statistic generated for functional similarity values various model combinations under the hypothesis (x-axis) < (y-axis).
34


Reviews:
[{"review_id": "rDfUJu10gu", "reviewer": "Reviewer_ERhZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n The application of end-to-end training of GNN for particle tracking is interesting.  \n\n\n**Weaknesses:**\n\n(1) The proposed methods in section 4 are not clearly discussed, and many parts are confusing. \n\n- The definition of $\\boldsymbol{v}_i$ in Eq.(8) on page 7 is not discussed. \n\n- The cosine similarity $\\boldsymbol{s}_{cos}$ is not clear. \n\n- The function $\\phi_{O}$ in Eq.(10) on page 9 is not given. \n\n- The authors claim that transforms the hit graph into the line graph at first. However,  why do we need this transformation, and how does this transformation influences the results is not clearly discussed. \n\n- The source node set and target node set in the linear sum assignment problem (LSAP) in Eq.(12) on page 9 are not well explained. \n\n- The authors claim a layer-wise linear sum assignment in Eq.(12). However, how to perform the layer-wise LSA is not clear.   How can the LSA perform end-to-end training? \n\n- The argmin of $y(\\mathcal{C})$ above Eq.(14) is not defined.  What is $y$ here and in Eq.(14)? \n\n\n\n(2) The experimental evidence is not significant. \n\nIn Table 5 on page 30, it shows that the Line graph representation influences the performance a lot.  However, whether the baseline methods in comparison in Table 1 are not clearly discussed.  I am not sure whether the improvement comes from the better  Line graph representation or from the proposed end-to-end training.   The evidence supporting the claim is not clear. \n\n(3) The contribution to the end-to-end training of GNN seems marginal. \n\n\n**Typos:**\n\n$f2$ in Eq.(5)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors propose an end-to-end training scheme of graph neural networks for particle tracking. They introduce predict-and-track (PAT) and two-step predict-then-track (PTT) approaches. In addition, linear sum assignment is introduced to determine whether the edge should be assigned under the reconstruction policy.  However, end-to-end training of GNN and linear sum assignment is not new in the GNN and segmentation areas.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "(1) Revision of section 4 according to the comments above.\n\n(2) Clarify the experiments and provide the empirical evidence.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4bt9Oaxbgo", "reviewer": "Reviewer_Pqe2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The paper presents a novel combination of GNN-based edge classification with a differentiable combinatorial solver, effectively bridging the gap between prediction and discrete optimization.\n+ The detailed analysis of both local and global loss landscapes provides valuable insights into the optimization behavior and stability of the trained networks.\n+ By ensuring end-to-end differentiability, the method lays a strong foundation for integration with downstream tasks due to enhanced gradient flow and stability.\n\nWeaknesses:\n- While the end-to-end method shows benefits in training dynamics and loss landscape properties, the improvements in final reconstruction metrics are relatively limited compared to the baseline.\n- The approach relies on a black-box differentiation technique that introduces an interpolation parameter $\\lambda$, which appears sensitive and requires careful tuning, potentially complicating practical applications.\n- Some aspects of the methodology, such as the specifics of the line graph construction and the edge filtering strategy (to mitigate excessive density), are not explained in sufficient detail. It is also unclear why the line graph is chosen to use, in particular using line graphs often introduces a larger and denser graph, which introduces computation overhead. The authors provide some reasons but those sound subjective. It is better to provide an ablation study for the choice of using line graphs.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an end-to-end differentiable framework for charged particle tracking by integrating a GNN based edge classification scheme with a combinatorial optimization step formulated as a layer-wise linear assignment problem (LSAP). By incorporating a black-box differentiation technique, the authors are able to approximate gradients through the otherwise non-differentiable assignment operation, thus enabling true end-to-end training. Extensive experiments on simulated proton computed tomography data compared the end-to-end “predict-and-track” (PAT) approach with the traditional “predict-then-track” (PTT) method. The analysis shows that while reconstruction metrics are similar between the two methods, PAT exhibits faster convergence and more favorable loss landscape properties.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Two ablation studies are expected:\n1. Include ablation studies that explore the impact of different values of $\\lambda$ on training stability, convergence speed, and final performance. This will help in assessing the sensitivity of the black-box differentiation approach.\n2. Include ablation studies on using original graphs v.s. line graphs to explore the impact on the prediction performance and computation complexity.", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LN21qSFnyB", "reviewer": "Reviewer_m8YL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The ideas used in the paper are very interesting to dig into and make for elegant reading. I was quite happy to read through the related work and learn how these disparate ideas were used in the particle tracking problem of the text. \n+ The results bear out in that end to end differentiability is attained by the trick used in the paper to solve a linear assignment or program and it compares well with the two step approach. \n- This is not a familiar area to me (I am a computer vision researcher). However, even with that, it was not easy to get intuition into the problem that is being solved. I think the paper should go into more detail on the nuances of charged particle tracking, as to why end to end modelling is not straight forward in this context, and generally give more background on various domain specific matters (e.g. hit graph vs line graph). That being said, please take this comment with the requisite grain of salt brought on by my lack of domain knowledge. \n- I found the loss landscape analysis angle quite interesting, but it seems a bit weak to extend it to the paper. What is the final outcome or summary from the loss landscape analysis? Similarly, for the other analyses on similarity, etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper examines charged particle tracking through a graph neural network approach. There are a number of pieces in the work \n- End to End optimization with a fully differentiable setup (authors claim that they are the first to do this)\n- Linear assignment to discover tracks (differentiable)\n- Loss landscape analysis\n\nThe loss landscape analysis uses ideas from existing works - such as mode connectivity, and a similarity score between networks. \nFavourable evaluations are shown against two baseline methods - Track follower [2] and an RL based approach [2]. They also look at network similarities across parameters and network connectivity across modes.  \n\n[1] Track follower (Petterson et al 2020)\n[2] RL based approach (Kortus et al 2023)", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address my comments in the previous section. I would hope that we can add more context on the problem itself so as to make it a bit more accessible to people who aren't familiar with the area. \n\nCould you also shed some light on what the loss landscape analysis accomplishes? In the fascinating paper [2], for instance, they propose an ensembling idea from this insight - that we can connect modes through Bezier curves. I do see that the results bear out here too, but it somehow feels as though I am missing the point as to what the final word is on the loss analysis in this paper. \n\n[3] Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs  https://arxiv.org/pdf/1802.10026", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)