Fisher-Rao Sensitivity for Out-of-Distribution Detection in Deep Neural Networks
Authors: Anthony Nguyen
Antonin Bertrand
Sylvie Le Hégarat-Mascle
Emanuel Aldea
Mohammed Nabil EL-KORSO
Renaud LUSTRAT
Franck FLORIN
Venue: ICLR 2026 Poster (2026)
Keywords: Deep Learning
Out-Of Distribution Detection
Information Geometry
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Deep neural networks often remain overconfident on Out-of-Distribution (OoD) inputs. We revisit this problem through Riemannian information geometry. We model the network's predictions as a statistical manifold and find that OoD inputs exhibit higher local Fisher-Rao sensitivity. By quantifying this sensitivity with the trace of the Fisher Information Matrix (FIM), we derive a unifying geometric connection between two common OoD signals: feature magnitude and output uncertainty. Analyzing the limitations of this multiplicative form, we extend our analysis using a product manifold construction. This provides a theoretical framework for the robust additive scores used in state-of-the-art (SOTA) detectors and motivates our final, competitive method.

Published as a conference paper at ICLR 2026
SENSITIVITY FOR OUT-OF- DISTRIBUTION DETECTION IN DEEP NEURAL NET- WORKS
FISHER-RAO
Anthony Nguyen," "I Sylvie Le Hégarat-Mascle, Emanuel Aldea, Renaud Lustrat SATIE Laboratory UMR 8029 Paris-Saclas University CNRS, Gif-sur- Yette, France {anthony higghhonynguuen,n nguuen, sylvie.leeeeeeeeeeeeeeeeeeeeeeee -e-hhgarat, emanuel. ent.uueee..addaaauuiaaattteeeeee renaud. tts
fr,
Antonin Bertrand," + Franck Florin, Mohammed Nabil El-Korso
L2S Laboratory UMR 8506 Paris Saclay University CNRS, Centrale Supelec, Gif-tte-Yvette, France (antonin. cuntonin.bertraree rand, mohammed nabil. ele--orrsseeeeettasssees.......e franck. florinefr wew.aneffr.aaalggooooooo thalesgroup.com com
ABSTRACT
Deep neural networks often remain overconfident Out-of-Distribution (OoD) inputs. We revisit this problem through Riemannian information geometry. We model the network's predictions statistical manifold and find that OoD inputs exhibit higher local Fisher-Rao sensitivity. By quantifying this sensitivity with the trace of the Fisher Information Matrix (FIM), we derive unifying geomet- ric connection between two common OoD signals: feature magnitude and output uncertainty. Analyzing the limitations of this multiplicative form, we extend our analysis using product manifold construction. This provides theoretical frame- work for the robust additive scores used in sfate-of-the-ard (SOTA) detectors and motivates our final, competitive method.
INTRODUCTION
Deep neural networks have demonstrated remarkable success on wide range of tasks. However, their reliability often does not extend beyond the training distribution. A major challenge arises when these models are deployed in real-world settings, where they inevitably encounter inputs outside the training distribution. These OoD samples can cause the network produce predictions that not only erroncous but also overconfident. This fundamental challenge of distinguishing ID data from OoD inputs is critical barrier building truly safe and trustworthy Al systems (Amodei et al., 2016).
A common baseline for OoD detection to use the model's predictive uncertainty. Hendrycks & Gimpel (2016) showed that, average, Ool inputs result in lower softmax confidence when fed into a classifier. At the same time, SOTA classifiers can maintain high accuracy under distribution shift. For example, Ovadia al. (2019) found that the base model's accuracy increases, OoD accuracy also improves.
To approach this problem from new perspective, turn to Information Geometry, A parametrized family distributions forms a statistical manifold endowed with the Fisher-Rao metric, which is the natural Riemannian metric induced by the model (Galott et al. 1990 Amari & Nagaoka 2000). This metric defines the local quadratic form and thus the sensitivity of p(: to parameter perturbations. In this work measure sensitivity via the Fisher metric: summarize sensitivity with the per-input FIM trace. We provide further discussion in Appendix ( about the link hetween sensitivity and curvature to strengthen our motivation.
"Co-first authors with equal contribution and importance Thales Land and Air Systems, BU ARC. Limours, France Thales Research Technology, Palaiscau, France

---

Published as a conference paper at ICLR 2026
Ply|ZID. 8)
p(ulzooD,
So
S
>
0
5 (p(ol=ID.0) 0 E e
S2 {ply|zo.D.0) 0)1 0 e@}
Figure Fisher-Rao sensitivity distinguishes ID from OoD. For fixed input æ, varying the parameters f traces the statistical manifold S_ {p(y]x 0) θ). At the trained parameters i (dotted line), the manifold Sy shows lower local Fisher-Rao sensitivity (left) while So shows higher local Risher-Rao sensitivity (right). This higher sensitivity corresponds to larger size of the elements of the tangent space (blue). See Appendix For intuitive and technical details sensitivity vs. curvature.
We use this geometric viewpoint to mathematically characterize the difference between ID and OoD data. We hypothesize and empirically observe that Fisher higher for OoD inputs. Intuitively, a large trace indicates that small parameter perturbations strongly affect the output, while small trace reflects local stability.
local sensitivity around trained model is
By restricting our analysis to the weights of the final layer, we first establish an analytical expression for the trace, which provides direct correspondence between the theoretical construct of Fisher- Rao sensitivity and the signals feature magnitude and output uncertainty. We then generalize this expression by restricting the trace to more discriminative parameter subspaces constructed via a tensor factorization. Finally, motivated by the limitations of a multiplicative form, we propose a more robust additive score derived from product manifold construction.
In summary, our contributions are:
Connecting Geometric Theory to OoD Signals. Our main result (Theorem 5.1 reveals that, when restricted to the final layer's weights, the intrinsic sensitivity of a classifier neural network decomposes into the product of feature magnitude and output uncertainty, estab- lishing a direct theoretical link between these common signals and Information Geometry.
Additive Detectors:
We expose the limitations of the multiplicative form and propose product manifold construction, leading to an additive score. In doing so, we provide geometric justification for the additive composition found in SOTA methods (e.g., (Wang Igoe et al. 2022)), leading to our final competitive detector.
et al. 2022
Competitive performance. Our additive detector require only single forward pass, with OoD data exposure and with no training or inference-time modifications, i.e., the base model is used as is. It achieves competitive results against related post-hoc baselines.
2 RELATED WORK
OoD detection methods are often divided into two categories: those that modify the training process and those that are post-hoc.
2

---

Published as a conference paper at ICLR 2026
One line of work integrates OoD awareness directly into training. This is done through techniques
such as outlier exposure during training (Hendrycks et al.] 2019; Zhang et al.] 2023 Gao & Lij 2025) data augmentation Zhang et al., 2017| Hendrycks et al. 2022). Other works consider architectural choices that prevent overconfidence (Hein et al. 2019). or methods that shape the feature space, such activation shaping (Sun al. 2022 Djurisic al. (023) or controlling the
model's Lipschitz constant (Liu et al.] [020a] Mukhoti et al. (022)) Others leverage geometrical properties (Haas et al. 2023; Regmi et al. 2024| Nguyen et al. 2024 2025) of the feature space.
While very effective, these methods require specific training protocols or test-time alterations.
Post-hoc methods are widely used due to their flexibility. These can be broadly grouped into two sub-categories. The first involves inference-time modifications, where the network's 's forward pass altered.
A popular approach is activation shaping (e.g.. ReAct (Sun et al. 2021). DICE (Sun & Li 2022) or ASH (Djurisic et al., (023)), which clips or sparsifies feature activations before they are passed to the final layer. While highly effective, these techniques alter the network's computations.
In contrast, our proposed method belongs to the second sub-category of purely analytical methods, providing an OoD score without any modification during inference. Within this analytical category, many methods analyze the geometry of the model's latent feature space, typically by measuring distances to class prototypes. Examples include Mahalanobis (Lee et al. 2018), Deep k-NN (Sun) ct al.) 2022). and ViM (Wang et al. al.] 2022). Other distance approaches adopt a deeper geometric per- spective. This includes IGEOOD (Gomes et al. 2022), which uses the Fisher-Rat Rao geodesic distance on the statistical manifold whose geometry is fundamentally defined by the gradients of the model's log-likelihood. The gradient leveraged geometric tool by other methods in different ways. For instance, ODIN (Liang et al. 2017) perturbs the input using the gradient of the classification loss to amplify the difference in softmax outputs between ID and OoD samples. GradNorm (Huang al.) 2021] uses the gradient norm with respect to a uniform distribution as an OoD score. However, the role of gradients in these methods has been questioned. In their empirical analysis of Grad- Norm, [goe et al. (2022) argue that the method's success comes from a simple combination of two signals: the magnitude of the feature encoding and the statisties of the output distribution. More recently, GradOrth (Behpour et al. 2023) extends this by projecting gradients onto learned sub- space. Similarly, Kwon et al. (2020) use backpropagated gradients as Fisher Kernels to characterize the local geometry of the autoencoder's 's manifold, requiring training -ime regularization to constrain the tangent space. To further contextualize our method, we compare in Appendix Clour first baseline score, the Standard FIM Trace, against GradNorm (Huang et al. 2021) and IGEOOD.
3 INFORMATION-GHOMETT PRELIMINARIES
3.1 STATISTICAL MODEL AND RIEMANNIAN GEOMETRY
Information geometry provides a rigorous framework for analyzing the structure of statistical models by modeling them as Riemannian manifold (Amari) 2016). We briefly recall the key concepts before specializing them to predictive models p(y (z, O) used in deep learning.
Definition 3.1 (Statistical manifold). Let (X, A, be a o-finite measure space and + C m open. Consider an identifiable, y-dominated model (Po Jeco with densities pl-,0) that are Co in E e. Then
s= (p(:0) 1 1 {)
is an m-dimensional C manifold, via the parametrization &p θ S, x(0) - -(:))p(:;0)
By identifiability, the parametrization & one-to-one, so @ serves as global coordinate chart on S. To quantify the effect of parameter perturbations, the manifold is endowed with a Riemannian metric, which defines an inner product on each tangent space, allowing for the measurement of distances and angles. A fundamental result in information geometry is that the Fisher Information Matrix serves as a natural Riemannian metric for a statistical manifold.
Definition 3.2 (Fisher- Rao Metric). The Fisher Information Matrix (Rao 1992) is defined by its components which are given by the following expectation:
@ llogp(u p(u, 0) θ Ologp(u, 0)
(i) = Euup(..0)
(1)
00,
00,
3

---

Published as a conference paper at ICLR 2026
This metric is invariant smooth reparameterization of & and provides intrinsic measure of sta- tistical distinguishability through the quadratic form d0 F(0) de where (-) denotes the transpose operator.
3.2 LOCAL SENSITIVITY VIA THE TRACE OF THE FISHER INFORMATION MATRIX
On the Riemannian statistical manifold (S, F), infinitesimal parameter changes dl yield the squared (statistical) distance ds² d0 F(0) d9, which measures how sensitive p(:, ,) is to infinitesimal changes of the parameters.
Proposition 3.3 (Trace of the FIM). The trace of the FIM F(0), defined as the sum of its diagonal elements, given by the expected squared L2 norm of the score vector:
Tr (F(0) EExpp( .0) [ log puu, 0)1
3.3 FROM FISHER-RAO GEOMETRY
NEURAL NETWORKS
The theoretical framework of information geometry be directly applied deep neural networks. A neural network with a fixed architecture is specified by its parameter vector 0 E R". Each parameter vector defines conditional probability distribution p(y xr 0). For any input z, the set of all such distributions forms a statistical manifold S.,, parametrized by I!:
S = {p(ul 0) | E {
Thus, a specific model a point on the manifold S.. The tangent space at that point, pp,Sa Sa. repre- sents all possible infinitesimal changes dA. The Fisher-Rao metric, which is input-wise, provides natural way to measure, for each input, the geometric and statistical impact of these weight pertur- bations.
4 DEEP LEARNING/OOD BACKGROUND AND NOTATIONS
4.1 DATA, DISTRIBUTIONS, AND ID/OOD MODEL.
Let the random variables X E X C. Rd and Y y = {l..... K} denote the input and label, respectively. We assume an unknown joint distribution pxy X x Y and ID training dataset Dirain = {xx;, y:) Pxy. The ID input marginal is px. To model open-world inputs, we introduce a binary random variable Z E (0, 1} indicating whether test input is ID (Z=0) or OoD (Z=1). The test-time input distribution is mixture with Px (x 0) px and px]2 (x: 1)
qx (z), where qx is an arbitrary (unknown) OoD marginal. We do not assume any parametric form for px or 9x. We denote the sets of ID and OoD inputs by XnD and XOOD respectively.
4.2 NEURAL NETWORK PREDICTOR
A classifier with parameters f E Rm defines the distribution p(y|æ, 0) softmax (go(fo(x)) (z))) where we decompose the network into feature extractor fo X + R and classification head Go : TR D - & producing logits in K We denote the predictor ho = o so- After training on Dirain we obtain and the predicted label ((z) arg max p(y|z,00 0).
Notation at a fixed trained model. We fix a pre-trained classifier with parameters Ô m and conditional distribution p ar 8) over K classes. We adopt the standard setting where the clas- sification head is single linear layer applied to the penultimate representation: let fz(x) E R denote the penultimate feature, and let the predictor be ha(x) W fa(x) + b where W D and b R the parameters of this final layer and constitute subset of the full parameter tor 0. Then p(y (z:, 0) softmax (x)) For convenience write P (x) p(y=k(e, 0) and (>))1=) Ekn Pa (x)).
4

---

Published as a conference paper at ICLR 2026
Information Geometry Notation.
According to Section 3.3 for any X. the input-specific Fisher Information Matrix is defined as the expectation over the conditional distribution:
F (0) 10p-pp110000 Vo p(y)z, 0) Ve log
(3)
Its trace is given by:
Tr (F) By-r((1a.0) V p(y 0)
(4)
To analyze the model's ' sensitivity specifically to its last-layer weights W, we focus on the corre- sponding block of the FIM. This block, denoted Fw,, (0). is defined as:
Fo (0) = Ey-rp-1xx00) [Vw log ply x () (V log p(yl>, 0))
(5)
TRACE OF FIM FOR OoD CHARACTERIZATION
5.1 MOTIVATION
We approach OoD detection by examining the stability induced during optimization. We posit that the training process may encourage the learned parameters 0 to be relatively stable for the ID dis- tribution, phenomenon related to stochastic optimizers where small parameter perturbations have minimal impact predictions for ID data (Maddox et al., 2019). In other words, small perturba- tions of 0 are unlikely to significantly affect predictions for ID inputs. However, since this stability only learned from ID data, it may not hold for OoD inputs. Consequently, the local geometry of the manifold SHOaD exhibit higher sensitivity for TOoD- We quantify this sensitivity with the per-input FIM trace Tr [r(F, (0) introduced above (Figure 1).
In practice, computing the trace of the full FIM over all parameters is intractable for deep neural networks. Therefore, focus the parameters of the final layer, as they directly control how the model translates high-level features into the final prediction (Pearce al. 2021).
5.2 SPECIALIZATION TO PENULTIMATE-LATE SUBSPACES
Let U R be r-dimensional linear subspace of the parameter space. We represent this sub- space with a matrix P E Rmxr whose columns form an orthonormal basis for U. Any parameter perturbation d0 that lies within this subspace can be expressed as d0 Pdox, where da E R" is the coordinate vector in basis P. Substituting into the general Fisher-Rao metric yields, for any x E X, the metric induced on the subspace 1d:
ds² (Pda) Fz (0)((ad)) = da (P da.
(6)
The matrix PT (T) is the restriction of the FIM to 21 and for any X, we have Fw.. (0) Tr(PT PO)
Theorem 5.1 (Closed-Form FIM Trace for the Final Layer). Take U to be the coordinate subspace spanned by the vectorized last-layer weights W. We have the following closed-form expression:
Tx(Fw.((0) = lff()))1 (1 (1-
Proof sketch. For the linear output layer, the gradient factorizes as the product Vw log (ey pg).. Substituting this into the trace, the squared feature norm [|f factors out. The remaining term is the expectation of the squared prediction error [y p.lla]. which corresponds to the multinomial variance - | For full proof, see Appendix
We will refer to equation 7 as the Standard FIM Trace.
Connecting Fisher-Rao Sensitivity to common OoD Signals. This theorem is foundational as it reveals that the geometric notion of Fisher-Rao sensitivity, when applied to a neural network's 's final layer, manifests product of the feature vector's magnitude and the output distribution's 's uncertainty. This provides a bridge between geometry and the design of several existing heuristic- based methods (e.g., GradNorm, Energy-based scores) which combine similar signals.
5

---

Published as a conference paper at ICLR 2026
Score Distributien- Standard
Score Distribution Standard FIM Tracs
6.1255
1.
6.1190
6.
6.0125
Denalty
6,2000
6.0075
0,0000
0.2
6,0023
6.0
0,0000
Score
(a) ResNet-50
(b) VIT B/16
Figure 2: Empirical validation of the Fisher Rao Sensitivity. Score distributions for the Standard FIM Trace (equation 7 evaluated on a ResNet-50 (left) and ViT (right), with ImageNet as ID (blue) and Places365 as OoD (green). The plot shows a visible shift in the OoD distribution towards higher values, empirically confirming that OoD inputs tend to exhibit greater local Fisher-Rao sensitivity. However, the imperfect overlap between the distributions indicates the limitations of this baseline score, motivating the more advanced subspace methods developed in Section 5.3
5.2.1 DISCRIMINATIVE SUBSPACE VIA TENSOR FACTORIZATION
The Standard FIM Trace considers the entire parameter space of the final layer, treating all directions of perturbation equally. However, as suggested by the overlap in Figure 2 not all directions are equally informative for separating ID from OoD data. For instance, it can be uninformative when both the embedding norm and the softmax flatness fail to separate ID from OoD. Our key insight is to more discriminative oower-dimensional subspaces where the separation is enhanced.
To construct such a subspace, we leverage the natural structure the last layer's gradients. For any input x, the gradient of the log-probability with respect the last-layer weights W factorizes into a class-dependent term and a feature-dependent term. Specifically, its vectorized form is a tensor product (Bourbaki] 2006):
vec W logp(y/z 0)) = (e, (p(x)) (z))
(8)
part
part
where ey E RK is the y-th canonical basis vector (See Appendix G This decomposition of the gradient, which forms the basis of the FIM, allows us to decompose the parameter space. We can thus select informative directions for the class and feature spaces.
This motivates defining a subspace using a tensor product basis P @ R, where the columns of Q I Kr. span class subspace and the columns of R Rxxy span feature subspace. The methods for constructing these projection matrices Q and R using PCA and LDA are detailed at the end of Section 5.3 This approach leads to the following result for the FIM trace restricted this specialized subspace.
Proposition 5.2 (Tensor FIM Trace), Let P Q & R and Spg(a) = diag(p,(x)) - (o()) For any x E X, the restricted FIM trace for the last layer is:
T(PT Fw.x
TiTQ Spo(a) Q)
|R Sa fa(x)
(9)
Ug(x)
Mg(x)
See proof in Appendix
5.3
FROM A SINGLE MANIFOLD TO A PRODUCT MANIFOLD
The Standard FIM Trace in equation
9 reveals that local sensitivity manifests a product of predic- tive uncertainty (x) and feature magnitude (z). However, this multiplicative coupling can be suboptimal, weak signal can suppress the other (see Appendix Dand Igoe et al. (2022)..7) T.his

---

Published as a conference paper at ICLR 2026
limitation motivates to decouple these signals, treating them structurally independent sources of geometric sensitivity. To explicitly model the uncertainty and feature space components as or- thogonal axes of variation, the canonical geometric structure to model the parameter space is product manifold.
5.3.1 GEOMETRICAL CONSTRUCTION AND PROPERTIES OF THE ADDITIVE SCORE
To construct this product manifold, we first need refine the geometric description of the feature space. The magnitude term M3(z) only captures the feature's alignment with the learned ID sub- space, which is an incomplete geometric description. more complete picture requires considering the component of the feature that lies orthogonal to this subspace. The Pythagorean theorem pro- vides a natural way for this, decomposing the feature's 's total energy into two orthogonal parts for any orthonormal matrix Rt
[1(())) = || sa())11 11(()))
RR fa(T)1
(10)
Proj. Magnitude
Residual Energy
This decomposition naturally reveals the residual energy, the component of the feature unexplained by the learned subspace. The orthogonality of the projected magnitude and residual energies allows us to treat them as independent geometric signals.
Building this orthogonality assumption, we model the total sensitivity on a product manifold M l xu xu. We define generalized and specialized metrics (F", F" on these three copies of the parameter subspace L/ via conffrrll transformations (see Appendix [F3) based on
the projected FIM. We suppose that Vx X, (x))), (x(2) and (11fg())113 (fg(=))l-MMM(1)))
non-zero. Let
(Ug(2)-N2g(3)
(AM; Aq) (12*), We define:
(0) = (0) (0) = (w. Ma(x)
Ug (x( F. (0) = (11f(() RR fo(x))l) Fw, X (T)(() Ma(x))
(11)
such that their traces are precisely our three signals of interest: Tr(Fw (0)) = Ug(2),
Tr(FW..(0)) = Au Mg(x)() and Tr(Fw.,. (0) = (11f((())) RR fa(x) XyNa(()) For- mally, we evaluate the local sensitivity at the diagonal injection of the learned parameters into the product space, i... the point (0.0,0) M. At this point, the tangent space decomposes into a direct sum of the component tangent spaces TeM = Tyu @ Tzll @ TyU. Consequently, per- turbation vector in M is a tuple of independent perturbations, and the natural metrie on M is the block-diagona tensor representing the direct sum of the component metrics:
(O)
8 (0)
W.
(12)
FW..(()
Its trace, representing the total sensitivity on M, is the sum of the traces of its components:
TT(F...)) = Tr(F(,..()) + Tr(F (0)) Tr(FW (0))
U2(x)
+AM
Mg(x)
+Ay (11g(()) RR (a()))
(13)
Uncertainty
Proj. Magnitude
Residual (1p)
(Aar Xy) are hyperparameters that balance the scales of these contributions as we do not have ac- cess to OOD labels during tuning. Without access to OoD correlations during tuning, adopt the standard principle of indifference: we tune the hyperparameters assuming the components are independent. These determined by minimizing:
C( AMD (Var[A, Wo (x)] Var(Ug(=)))² + (Var[Aar Mg(x)] - VarlUg(>))))
(14)
This objective is 4th-degree polynomial in X, and AM and immediately yields the analytical solu- tion:
Ag1 = Var(U)()))/Varr) (ar((( and |Axl = VarrUu(())/Vaa(MM
(15)

---

Published as a conference paper at ICLR 2026
and the choice of the signs are based the signal contribution analysis detailed in Appendix D In the case where the hyperparameter is strictly negative, the manifold not Riemannian. However get Seeudd-Riemmandin manifold structure where the local sensitivity is still well-defined. See Appendix F
Proposition 5.3 (Variance Balancing under Independence Hypothesis). If we further assume statis- tical independence of the uncertainty U and feature- pracure--pace signals (M, y). minimizing the objective guarantees that each component contributes equally to the total variance of the additive score.
Proof. See proof Appendix
rigorous geometric measure such Fisher-Rao sensitivity must capture the intrinsic local curva- ture of the statistical manifold independently of parameterization. The following proposition shows that our score remains consistent across equivalent models regardless of their specific parameteriza- tion.
Proposition 5.4 (Invariance under Reparameterization). The additive score, when tuned via equa-
tion 14 is invariant under the group of similarity transformations acting on the feature space fo and weights W that preserve the predictive distribution plyle x 0):
1. Orthogonal Transformation: Sa R.fa and W - RW, where R RR (e.g., rota- tion).
2. Classifier Translation: W W C, where C DxX has identical columns (shift of logits).
3. Uniform Scaling: The joint rescaling fa - aff and W - a wW with + 0.
Proof. See proof in Appendix
This invariance holds even when the subspaces Q and are estimated from the data. For instance, applying rotation to the input features results in an equivalent rotation of the estimated eigenvec- tors, leaving the projected norms unchanged.
Subspace Matrices Construction. Our method relies on the construction of two discriminative subspaces from a random chosen calibration set sampled from ID data, D The selection of algo- rithms construct the projection matrices for the feature space and the probability space Q is de- liberately chosen to align with the distinct geometric properties of each space. For the feature space, we construct the matrix using Linear Discriminant Analysis (LDA) followed by an orthonor- malization process. The basis vectors are derived from the feature representations {z)(z)) and their corresponding class labels. The choice LDA is to find a projection that maximizes the inter-class variance while simultaneously minimizing the intra-class variance. Our hypothesis is that feature vectors of ID inputs will exhibit a high projection norm onto this manifold, whereas OoD features, lacking correspondence orthogonal component. For the probability space, we construct the matrix Q using Principal Com- ponent Analysis (PCA) the set of softmax distributions (pa ( 2ED.... This subspace models the typical patterns of predictive uncertainty and inter-class confusion exhibited by the model on ID data. It the natural linear algebraic tool designed analyze such covariance structures: diago- nalizes the covariance matrix to identify the orthogonal directions of maximum variance: it extracts the principal axes of the Fisher Information Matrix (the directions of highest sensitivity) in linear basis. Therefore, the power of this score relies two-stage process: first, learning discrimina- tive subspace to create a meaningful geometric decomposition, and second, tuning the weights AM and A to find the optimal balance between the magnitude and residual signals.
the learned class structure, will be characterized by significant
EXPERIMENTAL RESULTS
We conduct experiments on standard OoD detection benchmarks to validate our geometric frame- work. The evaluation is designed to answer two questions from our theory: 1) Does our final, additive score, motivated by the product manifold construction, achieve competitive performance
8

---

Published as a conference paper at ICLR 2026
against analytic post-hoc baselines? 2) Does this additive formulation provide significant empiri- cal advantage over the simpler multiplicative score obtained from our initial analysis?
Datasets and models. We consider experimental settings. CIFAR: ResNet-18 (He et al, 2016) trained on CIFAR-10/100 (Krizhevsky et al., 2009), ) the OoD datasets are given by the com- plementary CIFAR dataset and SVHN (Netzer al.) 20TT). ImageNet-IK: ResNet-50 or ViT- B/16 (Dosovitskiy et al. (2200) trained on ImageNet-Ie IK (Deng et al. 2009). the OoD datasets in-
clude Places365 (Zhou al., (017). ImageNet-O (Hendrycks al.) 2021). iNaturalist (Van Horn 2018) and SUN (Xiao et al. 2010).
et al.
Table 1: OOD Detection Performance (AUROC/ INR@TPR95) on ImageNet benchmark
VIT-B/16
Method
ImageNet-O)
iNaturalist
Places365
SUN
MSP
85.08/46.89
96.64/87.91
84.76/43.67
86.62/46.11 14
Energy Score
91.72/67. 28
97.85/94.58
89.05/54.21
90.57/61.77 .73
GradNorm
82.23/63.92
96.11/92.90
82.68/52.79
85.57/40.21
GradOrth
93.22/67.99 .88
98.98/98.49
89.78/61.10
93.15/68.68
Mahalanobis Distance
91.94/68.51
98.76/97.88
88.75/57.12
90.07/61.17
ODIN
90.23/67.31 331
98,45/93.45
87.27/49.18 18
91.46/61.33
Deep k-NN
92.21/67.44 .48
98.34/98.37
88.06/53.42
92.56/65,70 70
ViM
92.26/71.34
98.87/98.1
90.83/58.07
93.32/65.
IGEOOD
91.66/66.27
97.89/97.56
87.31/52.12
90.32/60.66
Standard FIM Trace (Ours)
89.03/65.89
96.92/91.34
86.28/51.66
88.56/58.72
Tensor FIM Trace (Ours)
90.12/62.34
97.27/94.18
86.92/55.56
90.23/63.78
Additive FIM Trace (Ours)
92.61/68.25
88/99.88.4
89.51/60.17
93.38/67.99 .98
ResNet-50
Method
ImageNet-O
iNaturalist
Places365
SUN
MSP
58.23/15,98
88.08/50.41
78,11/30.10
79.47 30.18
Energy Score
59.28/15.91
89 89.29/47.31
77.47/29.12 12
79.23/29.2 ,22
GradNorm
60.36/18.98
67.54/28.35
70.96/13.53
69.17/04.13
GradOrth
71.23/23.56
90.12/52.42
82.43/39.64
85.12/40.34
Mahalamobis Distance
69.11/21.43
52.01/39.17
72.71/29.73
54.56/18.23
ODIN
64.34/16.21
89.14/40.65
77.56/31.22
78.31/24.51
Deep k-NN
72.34/22.82
86.97/37.81
70.51/30.25
81.94/24.83
ViM
68.21/22.34
87.19/45.79
79.66/36.94
76.84/27.35
IGEOOD
67.43/22.50
87.18/47.02
82.45/37.47
72.23/05.66 63
Standard FIM Trace (Ours)
61.11/18.62
89.74/49.09
78,92/29.16
83.23/19.
Tensor FIM Trace (Ours)
65.23/21.71
89.89/52.23
81.46/36233
83.78/31.45
Additive FIM Trace (Ours)
71.77/22.41
90.61/54.69
82.93/40.33
85.76/39.95
Baselines. We compare our methods against comprehensive set of purely analytic post-hoc
OoD detection methods, including uncertainty-basee methods such as MSP (Hendrycks & Gimpel, 2016), Energy (Liu et al. 2020b), and gradient-based methods such as ODIN (Liang et al.) 2017). GradNorm (Huang et al. 2021) and GradOrth (Behpour et al. 2023). We include IGEOOD as re- cent related information geometric (Gomes et al. (202)) baseline and related feature-space distance- based methods: Mahalanobis (Lee et al. 2018), Deep K-NN (Sun et al. 2022) and ViM (Wang et al.] 2022) SOTA additive baseline score. We emphasize that VIM also combine feature with logits signals through a virtual class construction and we provide a comparison in Appendix C
Ablation Study. We provide detailed ablation study Appendix A Jusing a ResNet-50 on Ima- geNet. In particular, we highlight the superiority of LDA over PCA for creating the feature subspace and also demonstrate that our method robust the size and the sampling of the calibration set.
Implementation Details. All methods evaluated in this work are post-hoc and do not require training the base model nor inference- time modifications. For our proposed methods, the projection matrices and R constructed using calibration set (50,000 images for ImageNet; 5,000 for CIFAR) sampled from the ID training data. To ensure a fair comparison, the hyperparameters for all baseline methods tuned the same validation set.

---

Published as a conference paper at ICLR 2026
Evaluation Metrics. We report results using standard OoD detection metrics: 1) The Area Under the Receiver Operating Characteristic curve (AUROC) which is our primary metric; 2) The True Negative Rate at 95% True Positive Rate TINRTTPRYS In both cases, higher values indicate better performance.
Performance on Benchmarks. The empirical results serve validate our geometric framework. As shown in T our method is competitive and stable on the CIFAR benchmarks. On the more challenging near- OoD settings (CIFAR-10 vs. CIFAR-IO0 and vice-versa), our proposal achieves competitive results. While some baselines vary substantially across datasets, our additive score re- mains consistently effective, demonstrating a reliability similar to robust methods like ViM. The Additive FIM Trace provides clear improvement over the Standard and Tensor FIM Trace, em- pirically validating the theoretical motivation for moving from a multiplicative to an additive score. As detailed in Table this strong performance extends the large-scalt scale ImageNet-11 K. benchmark. Our additive score is competitive against strong post-hoc methods on both ResNet-50 and ViT-B/16 architectures, confirming that the principles derived from our framework are applicable across dif- ferent models and data.
Table 2: OOD Detection Performance MUROC/TNR@TPRSS CIFAR benchmarks
ID: CIFAR-10
ID: CIFAR-100
Method
vs. SVHN
vs. C-100
vs. C-10
vs. SVHN
MSP
85.48/65.80 800
83.35/36.61
70.33/14.57
74.93/35.6
Energy Score
88.61/67.72
85.57/32.26
70.40/14.77
76.71/36.81
GradNorm
$5.72/63.17
82.11/27.12
70.28/13.99
74.18/19.56
GradOrth
89.11/67.23
86.99/54.24
74.42/14.78
81.92 /37.23
Mahalanobis Distance
89.63/66.2 22
70.53/24.21
58.73/04.35
80.45/24.12
ODIN
87. 87.33/67.09 007
85.74/43.12
71.43/13.26
78.92/36.1
Deep k-NN
88. 88.25/65.01
82.90/26.51
58.63/04.13
78.54/31.37
ViM
89.23/68.64
87.13/53.99
73.80/14.87
79.99/36.45
IGEOOD
90.11/66.43
86.81/44.56
71.41/04.17
77.45/35.34
Standard FIM Trace (Ours)
86.16/66.2 24
84.94/38.60
71.77/14.75
75.35/32.53
Tensor FIM Trace (Ours)
89.06. 68.
86.22/45.81
72.23/14.92
78.17/34.12
Additive FIM Trace (Ours)
91.28/68.88
87.23/54.47
74.38/15.02
80.91/35.33
CONCLUSION
We introduced a novel perspective on OoD detection through the lens of Riemannian information geometry. Our work provides unifying framework that first grounds common OoD signals in the framework local Fisher-Ran sensitivity. More significantly, we extend this framework to provide rigorous derivation for the additive structure. By formalizing the disentanglement of geometric com- ponents on a product manifold, we obtain a practical score that guarantees geometric invariance. Its competitive performance serves as strong validation of our orthogonality hypothesis. Beyond this specific score, our work highlights a larger paradigm shift in uncertainty quantification: moving from global approaches (e.g., Bayesian estimating posterior diversity (Fort al.] [019 Lakshmi- narayanan et al. (017)) to local geometric analysis. We demonstrate that uncertainty information not only captured by the global exploration of the statistical manifold but also encoded locally. Information geometry provides the language to leverage this local information efficiently. This work suggests that deeper understanding of the geometric structures of our models critical step to- wards building more robust, and safe Al systems.
Limitations and perspectives. Our current work focuses on the geometric sensitivity within the parameter space of the final layer. While this provides a direct and analytic view of how features mapped to predictions, OoD phenomena likely manifest in the geometry of deeper layers as well. Extending this sensitivity analysis to intermediate feature representations is natural next step. An- other opportunity for future work lies in the nature of the design choice for our additive score. While it is natural, motivated and justified by the geometric framework of product manifolds, its specific formulation is not unique. A direction for future work, would be to find a single, unified metric that intrinsically improves these geometric aspects.
10

---

Published as a conference paper at ICLR 2026
8 REPRODUCIBILITY STATEMENT
All our experiments are conducted exclusively on publicly available datasets, such as CIFAR-10, CIFAR-100, SVHN, and ImageNet-IK. A detailed description of each dataset and its specific use in our evaluation protocol is provided in Appendix H Our proposed algorithm is explicitly detailed throughout the paper.
For theoretical transparency, complete proof' for all our results are provided in Appendix B We also include supplementary appendices that detail the underlying mathematical concepts in Appendix G and [ and provide comparisons with related work in Appendix C Replicating our results is straightforward, as our methods are post-hoc and require no model retraining. facilitate this process, we will release the complete source code for our experiments on GitHub upon publication shortly.
ETHICS
Our work is motivated by need to create safer, frugal and more trustworthy Al systems. Our primary contribution is to improve the fundamental understanding of OoD behavior by using the theoretical lens of Information Geometry. We believe that a deeper understanding is essential for developing truly reliable machine learning methods. Note that our methods work without any expo- sure to OoD examples or modification of the forward process, either during training or at inference time. Our methods designed to analyze existing pre-trained networks without requiring costly and energy-intensive retraining.
ACKNOWLEDGMENTS
The authors would like to thank the anonymous reviewers for their constructive feedback and valu- able comments that helped improve this manuscript. We acknowledge the support of the Thales company during this study for granting access to their facilities, materials and resources. Finally, A.N. wishes to thank Amandine for reviewing and proofreading the manuscript, and for her support.

---

Published as a conference paper at ICLR 2026
REFERENCES
Shun-ichi Amari. Information geometry and its applications, volume 194. Springer, 2016.
Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2000.
Mouin Ben Ammar, Nacim Belkhir, Sebastian Popescu, Antoine Manzanera, and Gianni Franchi. Neco: Neural collapse based out-of-distribution detection. arXiv preprini arXiv:2310.06823. 2023.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Con- crete problems in ai safety. arXiv preprint arXiv: 1606.0656,, 2016.
Sima Behpour, Thang Long Doan, Xin Li, Wenbin He, Liang Gou, and Liu Ren. Gradorth: A simple yet efficient out-of-distribution detection with orthogonal projection of gradients. Advances in Neural Information Processing Systems, 36:38206-38230, 2023.
Nicolas Bourbaki. Eléments de Mathématique. Groupes et algèbres de Lie: Chapitres 2 et 3. Springer, 2006.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. leee, 2009.
Andrija Djurisic, Nebojša Božič, Oguz Kalkan, Taylan Güngör, and Berk Gökberk. Extremely sim- ple activation shaping for out-of-distribution detection. In International Conference on Machine Learning (ICML), 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 /929, 2020.
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape per- spective. arXiv preprint arXiv: 191200277, 2019.
Sylvestre Gallot, Dominique Hulin, Jacques Lafontaine, et al. Riemannian geometry, volume 2. Springer, 1990.
Heng Gao and Jun Li. Enhaneing ood detection using latent diffusion. Proceedings of the 2025 International Conference on Multimedia Retrieval, pp. 358-367, 2025.
Eduardo Dadalto Camara Gomes, Florence Alberge, Pierre Duhamel, and Pablo Piantanida. Igeood: An information geometry approach to out-of-distribution detection. arXiv preprint arXiv:2203.07798, 2022.
Jarrod Haas, William Yolland, and Bernhard Rabus. Exploring simple, high quality out-of- distribution detection with 12 normalization. arXiv preprint 2xXi:2336604072. 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why ReLU networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-dist examples in neural networks. arXiv preprint 1rXiv:1610.000 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference Learning Representations (ICLR), 2019.
12

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. CVPR, 2021.
Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, and Jacob Stein- hardt. Pixmix: Dreamlike pictures comprehensively improve safety measures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16783-16792, 2022.
Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distribu- tional shifts in the wild. Advances in Neural Information Processing Systems, 34:677-689, 2021.
Conor Igoe, Youngseog Chung, Ian Char, and Jeff Schneider. How useful are gradients for ood detection really?, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Gukyeong Kwon, Mohit Prabhushankar, Dogancan Temel, and Ghassan AlRegib. Backpropagated gradient representations for anomaly detection. In European conference on computer vision, pp- 206-226. Springer, 2020.
Balaji Lakshminarayanann, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Kimin Lec, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690) 2017
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances neural information processing systems, 33:7498-755 -5122, 2020a.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based duitt--diitribution detec- tion. Advances in neural information processing systems, 33:21464-21475, 2020b.
Wesley Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson A simple baseline for bayesian uncertainty in deep learning. Advances in neural information processing systems, 32, 2019.
Jishnu Mukhoti, Andreas Kirsch, Joost Van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deterministic uncertainty: A new simple baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24394, 2023.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, 2011.
Anthony Nguyen, Emanuel Aldea, Sylvie Le Hégarat-Mascles and Renaud Lustrat. Stochastic embeddings : probabilistic and geometric analysis of out-of-distribution behavior. In Sil- via Chiappa and Sara Magliacane (eds.), Proceedings of the Forty-first Conference on Uncer- tainty in Artificial Intelligence, volume 286 of Proceedings of Machine Learning Research, pp. 3200-3220. PMLR, 21-25 Jul 2025. URL https:: fttps::/pproeedinnge mlr pttss//2es////////// nguyen25b. html
Hai Vy Nguyen, Fabrice Gamboa, Reda Chhaibi, Sixin Zhang, Serge Gratton, and Thierry Giac- cone. Combining statistical depth and fermat distance for uncertainty quantification. Advances in Neural Information Processing Systems, 37:30207-30229, 2024.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan. and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
13

---

Published as a conference paper at ICLR 2026
Tim Pearce, Alexandra Brintrup, and Jun Zhu. Understanding softmax confidence and uncertainty. arXiv preprint arXiv:2106.0499 2021.
C Radhakrishna Rao. Information and the accuracy attainable in the estimation of statistical pa- rameters. Breakthroughs in Statistics: Foundations and basic theory, pp. 235-247. Springer, 1992.
Sudarshan Regmi, Bibek Panthi, Yifei Ming, Prashnna K Gyawali, Danail Stoyanov, and Binod Bhattarai. Reweightood: Loss reweighting for distance-based ood detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 131-141, 2024.
Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European conference on computer vision, pp. 691-708. Springer, 2022.
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activa- tions. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International conference on machine learning, pp. 20827-20840. PMLR, 2022.
Grant Van Hom, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 8769-8778 2018.
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang, Vim: Out-of-distribution with virtual- logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4921-4930 2022.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485-348340 3492. IEEE, 2010.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: arXiv: 2017.
Jingyang Zhang, Nathan Inkawhich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards out-of-distribune detection fine-grained environments. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 5531-5540, 2023.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 4066):152-116) 1464, 2017.
14

---

Published as a conference paper at ICLR 2026
A ABLATION STUDY
To validate our design choices and assess the robustness of our proposed method, we conduct series of ablation studies using ResNet-50 model the ImageNet OoD benchmark. We investigate several aspects: the impact of the subspace construction methodology, the sensitivity of our method the size of the calibration set and the hyperparameters for each component's contribution. In each study, we also report the performance of the individual geometric components of our score to understand their respective contributions.
A.1 IMPACT OF SUBSPACE CONSTRUCTION STRATEGY
Our method's effectiveness rests creating discriminative subspace for the features. We compare our proposed supervised strategy. Linear Discriminant Analysis (LDA), against an unsupervised Principal Component Analysis (PCA) and baseline using the identity matrix (i.e., no projection). The performance evaluated using a fixed calibration set of 50000 samples.
Table 3hows that the superiority of the LDA strategy is not just in the final combined score, but in the effectiveness of its geometric components.
With LDA, both the Residual and Projected Norm are strong OoD detectors their own. In con- trast, PCA fails to create useful orthogonal complement; its Residual score is very weak (0.5892), indicating that the directions of maximal variance are necessarily informative for OoD sep- aration. Identity baseline: with no projection, the Residual signal is weak, confirming that our geometric decomposition meaningful when applied to well-chosen subdimensional subspace. This demonstrates that the supervised nature of LDA is important for creating meaningful feature manifold where the orthogonal distance a reliable OoD signal.
Table 3: Ablation on Subspace Construction Strategy. Performance breakdown on a ResNet-50. The supervised LDA strategy unique in its ability to produce highly discriminative Residual signal.
AUROC Contribution
Strategy
Residual
Proj. Norm
Combined
Identity (No Proj.)
50.00
61.07
78.16
PCA
58.92
60.35
76.73
LDA (Ours)
80.23
72.47
82.93
A.2 SENSITIVITY TO CALIBRATION SET SIZE
Our method requires set of ID samples to calibrate the subspaces. We evaluate the performance of our superior LDA-based approach while varying the size the calibration set from 1,500 to 50,000 samples.
The results Table show that performance saturates quickly, confirming the method's practicality. The component-level breakdown offers a deeper insight: the Residual score is highly sensitive to the initial number of samples. With only 1,500 samples, its performance is poor (AUROC 38.62), suggesting that LDA cannot learn generalizable ID manifold from such limited data. However, its effectiveness grows rapidly, becoming a strong signal by 2,500 samples and stabilizing after 10,000. The Projected Norm, while also improving, is less volatile.
15

---

Published as a conference paper at ICLR 2026
Table 4: Sensitivity calibration set size. AUROC performance using ResNet-50, reported as mean + standard deviation over 3 random seeds. Performance improves with larger calibration sets, indicating robustness the random selection
AUROC Contribution
Calib, Size
Uncertainty
Residual
Proj. Norm
Combined
1,500
73.32 = 0.65
37.78 + 0.88
61.31 = 0.91
76.04 = 0.76
2,500
76.38 + 0.48
77.73 0.76
62.43 0.73
81.12 1 0.65
5,000
77.42 + 0.34
79.06 +: 0.43
67.81 0.56
82.00 1: 0.32
10,000
= =
79.96 + 0.25
71.74
82.21 = 0.26
20,000
79.25 = 0.10
80.19 0.14
72.09
82,99 # 0.11
40,000
79.29 = 0.07
81.25 0.09
7.114
82.91 + 0.08
50,000
79.50 = 0.05
81.46 0.06
72.19 0.08
82.97 + 0.04
A.3 SENSITIVITY À PARAMETERS
This figure shows the AUROC performance landscape for OoD detection. Performance is plotted against the magnitudes of two orthogonal feature components: the Projected Norm (how much a sample aligns with the learned ID patterns) and the Residual (the part of the sample that is orthog- onal to those patterns).
The highest performance (the bright yellow diagonal band) is achieved when the projected norm high and the Residual high. This provides empirical validation for our decomposition in the feature space: that separating features into these orthogonal components provides a robust signal for OoD detection.
82
0.08
Residual
78
0.00
0.00
0.00
0.00
0.03
3.42
Projected Norm
Figure 3: AUROC Landscape for ((Arrl Pyy))-
16

---

Published as a conference paper at ICLR 2026
B GEOMETRIC ASSUMPTION FOR ADDITIVITY VIA AN ORTHOGONALITY HYPOTHESIS ON THE PARAMETER MANIFOLD
This appendix provides a more intuitive geometric justification for the additive score equation 13
The Orthogonality Hypothesis. Our central hypothesis that the parameter space M can be de- composed into two functionally orthogonal subspaces: that primarily governs the geometric properties of the feature embeddings fz(z), and another that primarily governs the mapping from features to the statistical manifold of predictions (a()). Intuitively, this orthogonal hypothesis im- plies the existence of distinct directions in the parameter space: those effectively perturbing the feature signal without altering the decision boundary, and vice-versa. Formally, we suppose that the tangent space at decomposes into direct sum of orthogonal subspaces V and Vp:
Tyll = Vj θ Vp
(16)
such that the metric satisfies the orthogonality condition for any u E Vy and e E V2=
gg(u, v) = 0
(17)
This orthogonality is motivated by empirical phenomena: for instance, carefully crafted adversarial perturbation to an input x can induce large change in the output distribution Pa (xx while leaving the geometric properties of the feature vector fz(x) nearly invariant.
Formalizing the Hypothesis with a Composite Metric. We can make this more precise by con- structing a composite Riemannian metric tensor, Grotal. Instead using a single, monolithic Fisher metric, suppose that Gtotal the concatenation of two distinct metric tensor in block matrix, each designed to capture sensitivity along one of the orthogonal axes:
Gp: This metric tensor is constructed to measure sensitivity related to the prediction space. Its design is directly inspired by the original FIM, this is the natural metric for statistical manifold. Its trace, Tr) is defined be our uncertainty signal, (z). It quantifies how parameter perturbations affect the output distribution p1(
2. Gy: This metric tensor is constructed measure sensitivity related the feature space. It is designed to capture how parameter perturbations affect the geometric properties of the feature vector (x). Its trace, Tr(G,), is defined as the weighted sum of our signals: the projected magnitude and the residual energy.
Tr(G,) = AM M2(x) + A, (llf((z) - RR RRT f(())
The necessary mathematical structure to encode this orthogonality is product manifold where each member is endowed wih Gj, Gp respectively. We then naturally obtain our final score the sum of these two components.
The Additive Score as Total Sensitivity. The total sensitivity on the product manifold is the trace of the total metric tensor Grotal- By the linearity of the trace operator:
Tr(Geotal) GT, + Tr(G,).
(18)
Substituting the definitions for the traces of our component metrics, we directly recover the final additive score:
Tr(Guntat) Ux(2)
+ +AM X, (1lf(()) RR fz)
(19)
Prediction Seasitivity
Feature Sensitivity
The additive score is interpreted as the total sensitivity parameter manifold equipped with a composite metric. Thus, this metric is explicitly constructed to reflect plausible hypothesis that OoD phenomena manifest along two independent orthogonal geometric axes: one governing the feature space and one governing the prediction space.
17

---

Published as a conference paper at ICLR 2026
C DETAILED COMPARISON WITH RELATED GRADIENT AND INFORMATION GEOMETRY-BASEE METHODS
This appendix provides detailed comparison between Fisher-Rao sensitivity approach (specif-
ically the Standard FIM Trace) and two closely related methods: GradNorm (Huang et al.) 2021), which also utilizes gradient information, and IGEOOD (Gomes et al. 2022). which also leverages information geometry.
C.I COMPARISON WITH GRADNORM
GradNorm proposes an OoD detection score based on the vector norm of gradients backpropagated from specific loss function. While both GradNorm and our Standard FIM Trace utilize gradi- ent information and offer closed-form solutions for the last layer, their conceptual foundations and mathematical formulations are different.
The Standard FIM Trace measures the local sensitivity of the model p(y|a, 0) to parameter pertur-
bations around the trained weights Ú. It quantifies the magnitude of the Fisher-Rat metric tensor locally, acting proxy for the stability of the optimization landscape at that point for the input x.
GradNorm is motivated by optimization dynamics. It measures the magnitude of the gradient of the Kullback- Leibler divergence between the model's softmax output and fixed uniform distribution u: LUni (x) Dx L. (ullp(-((z, 0)). The score SGradNorm (x) V[VoLun (z))lpp Huang et al. hypothesize that ID inputs, being less uniform, have a larger gradient magnitude towards uniformity.
C.I.1 MATHEMATICAL SETTING
The fundamental difference lies in how the gradients are computed and aggregated. Let W be the last-layer weights and p(-|æ, 0) the softmax output for input x.
The Standard FIM Trace (equation is defined as:
Tr(Fw... (0) (y--p((r)
(or) Vw ply)(y 0)
(20)
This is the expectation of the squared L2 norm of the gradients, where the expectation is taken with respect to the model's 's own predictive distribution Pj (z).
The gradient used in GradNorm is equivalent the average of the gradients of the cross-entropy loss for all labels:
Vw Dw
V (w Ey V logp(ul.,
(21)
K
we consider the 1.2 norm for comparison:
ScradNorm,1al E| Vw logp(ylæ, 0)]1
(22)
This is the squared 1.2 norm of the expectation of the gradients, where the expectation is taken with respect to the uniform distribution u.
Hence, we have the following key distinctions:
1. Jensen Inequality: The FIM Trace calculates E[||g|?]] (the second moment), while Grad- Norm calculates [[]]g]]] 12, By Jensen's inequality, E E|gg] Z || The FIM Trace captures the variance of the gradients, whereas GradNorm captures the magnitude of the mean gradient direction.
2. Base Distribution:
The FIM Trace is intrinsic the model's geometry defined by p(-12, 0), while GradNorm depends on an external reference distribution u.
C.1.2 CL.OSED-FORM FACTORIZATION
Both methods yield a closed-form factorization when restricted to the last layer. Let fx( be the feature extractor output.
18

---

Published as a conference paper at ICLR 2026
Standard FIM Trace: (1) - (1
ExPr,((E))) GradNorm (L1, optimal variant): (lf((z)]lı
Px,((=))))
While both capture similar intuitions (feature magnitude and output uncertainty), the specific metrics used (L3 vs L, for features; Gini impurity vs. LI deviation from uniform for outputs) lead to different behaviors.
C.2 COMPARISON WITH IGEOOD
IGEOOD also utilizes information geometry tools, specifically the Fisher-Rao distance, for OOD detection.
C.2.1 LOCAL SENSITIVITY vs. GEODESIC DISTANCE
The fundamental difference is the distinction between measuring the magnitude of the metric tensor (local sensitivity) and measuring the distance along the manifold induced by that metric (geodesic distance).
FIM Trace: We analyze the geometry locally around the trained parameters 6. The FIM Trace, (O)), is scalar summary of the FIM Fz (0), which acts as the Riemannian metric tensor at θ. It quantifies how rapidly the statistical distance ds² increases due to infinitesimal parameter changes d0.
IGEOOD: IGEOOD utilizes the Fisher-Rao distance, which is the geodesic distance on the sta- tistical manifold. It measures the shortest path between two distinct distributions on the manifold. IGEOOD applies this by comparing the distribution associated with test input to distributions derived from the training data.
C.2.2 METHODOLOGY
FIM Trace: Our method characterized by a focus on the trained model's stability Å. provides computationally efficient, forward-only scores that do not require comparison against stored training data statistics during inference.
IGEOOD:
IGEOOD is essentially distance-based detector (akin to Mahalanobis distance KNN methods) but utilizes the intrinsic Riemannian distance. This involves:
1. Computing class prototypes from the training data. At the logits, this involves optimizing centroids in the Fisher-Rao geometry. At the feature space fØ)),, it involves estimating parameters for Gaussian models.
2. During inference, computing the geodesic distance between the test sample's representation and these stored centroids.
3. The final score is the Rao distance to the nearest centroid or sum distances.
While both methods are grounded information geometry, our approach focuses on the intrinsic local stability of the trained model for given input, whereas IGEOOD focuses on measuring the geodesic distance between the input and the learned distribution of the training data.
C.3 COMPARISON WITH VIM
ViM Wang al. 2022) is highly relevant baseline, sharing fundamental insight with our work: the necessity of combining class-dependent information with information from the feature space. Both methods identify that only relying on one source is potentially not enough for strong OoD detection. ViM addresses this by constructing a virtual logit from the feature residual, expanding the class set to K + While the resulting signals are similar, the motivations and derivations mechanisms differ:
ViM depends on a construction where the norm of the residual ||=P*| treated as new logit. The final score is the softmax probability of this class. Our Additive FIM Trace
19

---

Published as a conference paper at ICLR 2026
is derived from the local sensitivity of the statistical manifold. The additive structure is a theoretical consequence of modeling the total sensitivity on a product manifold, where the prediction space and the orthogonal feature space are treated as independent geometric components.
2. To combine the residual with the original logits, ViM introduces a scaling factor a, deter- mined to match the average magnitude of the maximum logits on the training set. This acts as normalization constant to make the virtual logit comparable. Conversely, our hyper- parameters Am and Ay comes from the principle of indifference, applied to the variances the sensitivity components. Our aim is to equilibrate the contribution of each geometric axis to the total variance of the score, under statistical independence assumption.
3. ViM use PCA on the feature space to define the residual. Our framework employs a Tensor Factorization of the FIM, which motivates the use of LDA for the feature subspace I and PCA for the probability subspace Q
Hence, VIM can be viewed as effective application of the principle of combining feature residuals and logits. Our work provides formal geometric framework for this high-level idea.
D ANALYSIS OF THE STANDARD AND TENSOR FIM TRACE
In this section, we analyze the limitations of the FIM Trace scores derived from the product formu- lation. Although we focus on the Standard FIM Trace for simplicity, the reasoning applies to the Tensor FIM Trace, as both share the same multiplicative structure.
The Standard FIM Trace defined as Tr(Fw.,.00) (0)) = Ilf (2) - (I)())))²). This equation shows that sensitivity is the product of two factors:
1. Magnitude: The magnitude of the feature representation (z)))))))33)
2. Uncertainty: The flatness of the softmax prediction (1 - |lpp (z)
For this score to detect OoD data, we want the product to be higher for Ool) than for ID). However, research suggests these two factors often conflict: Haas et al. (2023) noted that modern architecture learns features of similar magnitude for both ID and OoD inputs (a phenomenon related to Neural Collapse). the magnitudes are similar, the score relies only on uncertainty. which might not be enough for good separation. More importantly, alternative studies (Sun et al. [202T] Ammar al. N guyen et al. 2025) found that OoD inputs generally have lower feature magnitudes than ID inputs. This creates problem for the product formulation:
2023
ID data tends to have High Magnitude and Low Uncertainty.
OoD data tends to have Low Magnitude and High Uncertainty.
Because the Standard FIM Trace and Tensor FIM Trace multiply these factors, the lower magnitude of OoD data counteracts the effect its higher uncertainty. This makes the product score suboptimal for OoD detection.
The following result formalizes this tension between magnitude and uncertainty in the product for- mulation:
Corollary D.1 (Separation conditions for the FIM Trace product). Let the Standard FIM Trace be S(z) = Ilf (2) [Ip) Assume bounds on the feature magnitude:
(15(()) (MID, MD)] for x XID. I (x)1l E [mooD, MoaD] for x E XOOD- (23)
We assume OoD has smaller norms, so MooD S MID (and typically mOoD S mmD).
Assume bounds on the uncertainty (flatness):
1- (=)1 < SID Yz E Xps
1 [1p(())) 2 SooD Ve E XOOD
(24)
We assume OoD is flatter, so CO.D > SID (and MOoD > 0).
Then the bounds the score S(x) are:
max S(x) MID SID, xEXD
min zoDooD
S(x) 2 MooD EOoD-
(25)
20

---

Published as a conference paper at ICLR 2026
For strict separation, the uncertainty gain must outweigh the magnitude loss. Specifically:
COaD
MID
(26)
EID
MooD
this condition holds, then monDeXooD
S(x)
max (
D.1 VISUALIZATION OF THE MAGNITUDE- -UNERTTAINN TRADE-OFF
To validate the theoretical separation conditions discussed in Corollary D.1 and motivate the additive decomposition, we analyze the geometric signal dynamics in controlled environment.
D.1.1 EXPERIMENTAL SETUP FOR GEOMETRIC VISUALIZATION
To isolate the intrinsic behavior of Fisher-Rao sensitivity without the under-perfor muder-pprfformnce of the norm signal induced by complex architectures in large-scale benchmarks, we adopt toy experimental setting. Datasets. use a standard digit recognition setup for OoD analysis:
In-Distribution (ID): The MNIST dataset.
Out-of-Distribution (OoD): The SVHNdataset.
Model Configuration. The architecture consists simple MLP with two hidden layers: 512 and 128 units, ReLU activations. The penultimate layer produces 128-dimensional feature representa- tion Sa (.), which is then fully connected to 10 class logits. Optimization is performed using SGI) minimize the standard CE Loss.
1.1.2 ANALYSIS SIGNAL DYNAMICS
Figure illustrates the joint distribution of feature magnitude (Ma(x) = llfa(z))1) and predictive uncertainty (Ug(z) =
1Standard Traool
Dytamicr: Magnitude vs. Uncertainty
Staristeal Component Compertsen
0.12
6.10
0.00
6.00
6.04
6/22
16C
6.00
Seeaderiyy
Fauture
N00m
(40000
Figure 4: Visualization the Geometric Decomposition. Left: Histogram of the Standard FIM Trace scores showing the imperfect separation between ID (blue) and OoD (red). Middle: Scatter plot of Feature Magnitude vs. Output Uncertainty. We observe distinct geometric regimes: data is clustered in the wwwpmmmmund/llowwuerrreee uncertainty/low-mannien region. Right: Box plots showing the collapse of feature magnitude for OoD data, which dominates the multiplicative score.
region, while OoD data shifts the high-
Why is ID Sensitivity Lower? The scatter plot (middle panel) provides clear geometric expla- nation for why the local Fisher-Rao sensitivity is consistently lower for IDD data: ID samples (blue points) are concentrated at the very bottom of the uncertainty axis. Since the Standard FIM Trace a multiplicative score, this quasi-zero uncertainty acts as an anchor, forcing the total sensitivity to remain low regardless of the high feature magnitude. For OoD samples (red points), the uncer- tainty increases, driving the sensitivity up. However, this effect counteracted by the simultaneous collapse of feature magnitude visible in the box plots the right. This empirical evidence justifies the decoupling by moving to an additive product manifold, preventing the magnitude signal from suppressing the strong uncertainty signal.
21

---

Published as a conference paper at ICLR 2026
E PROOFS OF THEORETICAL RESULTS
Theorem
For the softmax last layer, the gradient of the log-probability w.r. weight Wk, is
i log (, 0)
(1(y=k} px.,(2)) f((x).
(27)
Taking the expectation y p(: xz, 0),
2 @Olgppysa 0)
)9 = f pk,(())(1 (..,oo)(--
(diag F.. By Summing over all (k: i) gives the closed-form trace:
(28)
OWKei
Tr(Fw..:0)) [[ (1 lpo(2))
(29)
Proposition 5.2 Fix an input r and 9 E RT. Let fe(x) E ID the penultimate representation and (T( R the softmax probabilities with entries Pa,a( (xr) ply k| 0). For the last linear
layer= = W [z(x) + b and = softmax(2), the gradient of the log-probability with respect to weight WA, is
i log loggiulæ 0) (1(y=k} Pk,,(x)) fi(x).
(30)
aW,,
Vectorizing the last-layer weights W E DxX the per-class gradient can be written as (x)x == ved Vw log p(y|z, 0)) (e, pb(x)) fz(z),
(31)
where ey is the y-th canonical basis vector in R.K and & denotes the tensor product.
The input-wise Fisher block for W is the second moment of this score under p(. 1;;)):
Fw. Ey (g)(() (z) | = (, (e, - p)(ev - 1) & (fo(z)fo(z)))
(32)
Since Ey diag(pg(x) (x)) and [e, p2(xx) we have
[ey (ey - p) diag(pg(x)) - po(x))p(x=) =: Spa(a)
(33)
Let Q E Rx and R E Dxr,, and set P = Q s R R(DK)x(rarr), Using the mixed-produce
property the tensor product,
(Q s R) (A B)(Q s R) (QT AQ) s (RTBR)
(34)
we obtain
PT Fw. P (QT S (p)(t) Q) @ (R (6( R). Taking traces and using Tr(A 8 B) Tr(AATTr yields
(35)
FW.TFw..H =T(Q= Sp TYR fo())f((z) R).
(36)
Finally, since R (z(z){(()) (RT [a(x)) [
T rank-1,
Ti(R" fe(x)So(x) R) = || fo(x)
(37)
which proves the proposition.
Proposition
5.3 Let be S the additive score. Its developed expression for the variance is: Var(S)= Var(U) + Var(y) + Xh Var(M)
+ 2o,,Co v) + Cov(U, M) + Cov(yy, M)
Under the independence assumption, the covariance terms vanish. Then, the global variance boils down to:
Var(S) Var(U) + X Var(y) + X3 Var(M)
(38)
To ensure that each component contributes equally to this global variance, one must satisfy:
X Var(y) = Var(U) and X3 Var(M) = Var(U).
(39)
This leads to the exact analytical solution minimizing L. Thus, the statistical independence assumption guarantees that minimizing the simple marginal error effectively balances the total variance of the score.
22

---

Published as a conference paper at ICLR 2026
Proposition 5.4 Point 1 and 2 are trivial. We details here the proof for the uniform scaling. Recall that:
Var(Ua)
Var(Ug)
AM
(40)
Var(ya)
Var(Mg)
up to a sign. Consider the rescaling reparameterization: S/4 cfg. where e E RR" assuming the classifier weights adapt (W' = W) to maintain invariant predictions.
1. The uncertainty Ug (based probabilities) remains invariant: U' U.
2. The embedding-basee terms (squared norms) scale quadratically: My = ||cfol1 = c2M
and v/g c² Vja
3. Consequently, their variances scale by c:: Var(M c² (aVar(Mg).
If the Am hyperparameter value were computed on the rescaled data, it would adapt as follows:
Var(U?)
Var(Uş) AM Var( Mg) c²
X1
(41)
V Var(
The scaling factor c² from the feature magnitude is canceled by the 1/c² scaling of the hyperparam- eter:
1 T (a) (c² Aar
Mr
(42)
The same applies to the residual term
F TANGENT SPACE, RIEMANNIAN CURVATURE AND CONFORMAL. TRANSFORMATION
F.1 THE FISHER-RAO METRIC AND THE TANGENT SPACE
Consider statistical manifold $ = {p(- 0) e θ}. Information geometry endows S with the FIM g F(0) its natural (pseudo)- Riemannian metric tensor. The components are given by (0) (0).
We start by fixing local coordinate basis [do],... ,d,p) of the tangent space Ta(S). The Fisher Information Matrix acts as -iiunddn)RRmmaniinn metric on this space, defined by
(d0, d0,) 9:y (0)
(43)
where (.) strictly defined positive or negative. In particular, the squared length of each basis vector is |d0 2 = (0). Summing these contributions gives
Tr(F(0)) = Σ |d0.112.
Thus, the trace aggregates the metric lengths of an orthonormal coordinate frame, and in some sense, measures the "size" of the tangent space. This motivate us to use it as a geometric invariant, quantifying the uncertainty across all directions of the parameter space.
F.2 RIEMANNIAN CURVATURE
Riemannian curvature measures the extent to which the manifold deviates from being locally Eu- clidean.
Formally, curvature involves the derivatives of the metric components, 00,;/00,. These derivatives define the connection on the manifold, often represented by the Christoffel symbols (Gallot et al, 1990). The geometric structure is described by the Riemann curvature tensor, which is rank-4 tensor derived from the Christoffel symbols and their derivatives. Contractions this tensor yield the Ricci tensor and the Scalar curvature. The distinction is:
23

---

Published as a conference paper at ICLR 2026
1. The Metric F(6): Measures the magnitude of the sensitivity at specific point 0. it first-order geometric property.
2. The Curvature R(0): Measures the rate of change of the metric in the neighborhood of 8. It is higher-order geometric property.
F.3 CONFORMAL TRANSFORMATIONS AND PRODUCT MANIFOLDS
Definition F.1 (Conformal and Pseudo-Conformal Transformation). Let (M,9) g)) a Riemannian manifold with metric lensor g. A transformation of g is the construction of new metric
99
where xp - is smooth function. If +ox we say that the transformation is conformal. 1f -0,,0,00 we say that the transformation is psendo-conformal.
the case where =| 0o,0,, 0f, g' is a Poudo-Riemannian metric where the notion of curvature and local sensitivity is well defined (see Appendix
Product Manifold. Given n Riemannian manifolds (M1. 91), (M2, manifold is the Cartesian product M M1 x M2.... x Mn equipped with the natural product Riemannian metric
- 9n), their product
((1 u2,..., )n); (wI, W2, ... wn)) = 9i (11, w1) + (2 W2) + + [an wx)
for tangent vectors (vi, U2, Un), (Wi, W2, w.)) E ((aaaama.. M.
Now for each fix basis {ot (i)
..... ( of the tangent space Tu, M,. The metric tensor of M in the basis {(u] (1) 0, 0.....0) ()....(1%), .................. O)... (0, (n...)))) of T (e M takes the block-
diagonal form
(91(1)) 1)) p.u=l.....k
(02(u)² .ng (1(2)))
GM
p.o= k....
(n)) , c(tii) (gn(c)))
Each block represent a single metric 9: on a manifold Mr
G SOME PROPERTIES OF THE TENSOR PRODUCT
Section 5.2.1 utilizes the tensor product factorize the FIM of the last layer and to define restricted subspaces. This appendix reviews the definition and the key properties used in the derivations for completeness.
G.1 DEFNITION AND VECTORIZATION
Definition (Tensor Product (Bourbaki, 2006 For matrices A E Raxb and B e Ieedd the Tensor
product B E Rueexd is defined as the block matrix:
Bn
B
A B
(44)
0+l
B
The Tensor product is closely related to the vectorization operator, vec(-), which stacks the columns of matrix. For vectors and w:
vec(vu]]
(45)
This property used to express the vectorized gradient of the log-probability with respect to last-layer weights W:
veel Vw pyl 0)) =Cey pp(x)) Sa(x).
(46)
24

---

Published as a conference paper at ICLR 2026
Indeed, recall that the last layer produces logits WWJ f E R with softmax probabilities p = softmax(2) and one-hot vector ey E R for the label y. We use the chain rule on the logits to get:
i OlogPy 1 f) Ph (ey p)n.
(47)
Ozk
And by simple derivation we have:
dzk
= f e
(48)
OW
which is matrix of size D K with the k-th column equal
So the gradient is simply:
Vw log Pg Σ (cp- p)k f f -)
(49)
Finally, by the vectorization identity voc(ab ) b 2 a, we obtain
vec( log Py) = (ey - p) &
(50)
G.2 KEY ALGEBRAIC PROPERTIES
The derivations rely on the following algebraic properties:
Proposition G.1. 1. Mixed-Product Property: If the matrix dimensions are compatible for the
products AC and BD to exist, then:
(A B) & D) (AC) (BD)-
(51)
2. Transpose Property:
B) AT @ BT.
(52)
H DETAILS ON TRAINING AND DATASET
H.1 EXPERIMENTAL DETAILS
For experiments on CIFAR-10 and CIFAR-100, we trained a ResNet-18 architecture from scratch for 200 epochs. The optimization performed using SGD with Nesterov momentum (0.9), weight decay of 5 x 10-4, and batch size of 256. We employed a cosine annealing learning rate schedule defined
Mmin + (710 Thain (1 + cos(Tt//Tmax))
with 0.1, Tuii = O. and Tmax 200. The training objective was the standard cross-entropy loss. Input data was normalized using the following channel-wise statistics:
0.2435, CIFAR-10: (0.4914, 0.4822, 0.4465), a (0.247,,0.2433, 0. 0.2616).
CIFAR-100: = (0.5070, 0.4865, 0.4409), a (0.2673, 0.2564, 0.2761).
H.2 DATASET DETAILS
CIFAR-10 CIFAR-10 (Krizhessky et al., 2009) consists of 10 object classes and contains train- ing set with 50,000 images and a testing set including 10,000 images. Each the images is RGB and of size 32x32 pixels
CIFAR-100 CIFAR-100 (Krizhevsky et al., 2009) consists of 100 object classes and contains training set with 50,000 images and a testing set including 10,000 images. Each of the images is RGB and of size 32x32 pixels
25

---

Published as a conference paper at ICLR 2026
ImageNet-1K Large Scale Visual Recognition Challenge (ILSVRC) 2012 dataset (Deng et al. 2009), commonly referred to as ImageNet- a large-scale benchmark for object recognition. contains 1,000 diverse object classes, with approximately 1.28 million images in its training set and 50,000 images the validation set. The images are high-resolution color photographs, which are typically resized to 224x224 pixels before being fed into model.
SVHN The Street View House Numbers (SVHN) dataset (Netzer et al. 2011 is large-scale
dataset of 600,000 images digits obtained from house numbers in Google Street View, our work, we kept fixed set of 10,000 images that we resized to 32x32.
ImageNet-O
The ImageNet-Outlier dataset (Hendrycks al. 2021) is a test set specifically de- signed for OoD detection for models trained ImageNet-IK. IK. It consists of images that do not belong to any of the original 1,000 ImageNet- IK classes. was created to be a challenging bench- mark testing model's understanding of its learned semantic concepts
Places365
The Places365 dataset (Zhou et al. 2017) is a large-scale scene recognition benchmark. Unlike object-centric datasets like ImageNet, primary goal is train models that can classify the environment or context of an image (e.g., "bakery," "coast," "forest path"). It contains over 1.8 million images categorized into 365 distinct scene classes.
iNaturalist
The iNaturalist (Van Horn et al. 2018) dataset is large collection of photos of plants and animals from all over the world. is created by community of users on the iNaturalist plat- form. Because it includes many different species, it is often used to test model's ability to perform visual classification on the difference between very similar categories, like different species of birds insects.
The SUN (Scene UNderstanding) dataset (Xiao et al. 2010 is collection of images de- signed for training models recognize different environments or scenes. The SUN dataset's goal is to classify the entire context of an image, such as "kitchen," "forest," "beach," or "office." It contains a wide variety of indoor and outdoor scenes.
SUN
26


Reviews:
[{"review_id": "ZKBtuuTIDG", "reviewer": "Reviewer_1qwa", "summary": "The paper proposes a post-hoc OoD detection method based on the per-input trace of the Fisher Information Matrix (FIM) restricted to the final-layer network weights. This acts as a proxy for model sensitivity, which the authors claim is greater for OoD versus ID signals. For a linear softmax head, the authors show that this “standard FIM trace” factorizes into the product of the L2-norm of the final layer features, and a predictive uncertainty term, unifying two well-known OoD signals. They further introduce a more performant “tensor-factorized” variant of this score that involves projecting the final-layer FIM onto a learned subspace, which is constructed from feature and prediction statistics. Finally, they present a more robust additive version of the tensor-factorized score, which they demonstrate is competitive with current SOTA post-hoc OoD detection methods on CIFAR and ImageNet-style benchmarks.", "questions": "Ablation Studies:\n\n1. Impact of subspace construction (A.1): it is mentioned that a partial least squares approach is examined for computing the feature subspace. However, this is not included in the results (Table 3). Furthermore, the baseline (no projection) yields slightly different AUROC contributions for the residual and projected norms, which does not make sense. Have these two been mixed up?\n\n2. Sensitivity to score weights (A.3): it is mentioned that the best AUROC performance is attained when the projected norm is high and residual is low (or vice versa), however the graph shows that this occurs along the diagonal (i.e., when both residual and projected norm are high).\n\nMiscellaneous:\n\n1. The use of notation of the form $X_{\\text{Long Text}}$ is a little awkward. For example, something like $S_{x_{OoD}}$ and $S_{x_{ID}}$ could be replaced by $S_x^O$ and $S_x^I$, while $G_{probability}$ and $G_{feature}$ could be $G_p$ and $G_f$.\n\n2. In lines 912-914, the terms $p_{k,\\hat{\\theta}}(x)(x)$ seem like they should be $p_{k,\\hat{\\theta}}(x)$.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The paper gives an interesting information-geometric justification for the use of two popular OoD signals (feature magnitude, prediction uncertainty). The tensor-factorized subspace approach appears novel and meaningful, and the underlying \"orthogonality hypothesis\" (if further supported) could provide a useful principle for designing new OoD detection methods. The empirical evaluation is thorough, and demonstrates that the final additive version of their OoD score consistently attains performance which is competitive with similar state of the art methods. The ablation studies are also reasonably extensive, although some of the details could benefit from additional clarification (see questions).", "weaknesses": "1. The geometric interpretation of the additive score (section 5.3) reads somewhat like a post-hoc justification. That is, the product manifold and metric seem to be engineered so that the trace reproduces the score, rather than the design of the score being informed by some underlying geometric principle. It would be helpful to clarify whether this product manifold is in some sense “canonical”, and whether it offers any insights/uses beyond re-expressing the additive score in geometric notation.\n\n2. The orthogonality hypothesis (Appendix B) proposes that the tangent subspace corresponding to the final-layer weights can be decomposed into two approximately orthogonal subspaces: one determined by the layer inputs (features) and one by the outputs (class predictions). This is an intriguing and potentially quite insightful claim; however, it does not appear to be supported by much empirical or theoretical evidence. Some further investigation in this direction could strengthen the paper.\n\n3. The paper would benefit from a careful editing pass. There are multiple small typos, some notation issues, and also some potential errors in the ablation studies. Several of these have been highlighted in the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P2mx8Aws2v", "reviewer": "Reviewer_6tV2", "summary": "This paper is presenting a new Out-of-Distribution (OoD) detection method (post-hoc method) grounded in information geometry (Riemannian manifold). The authors model a network's predictive distribution as a statistical manifold and hypothesize that OoD inputs exhibit higher \"local Fisher-Rao sensitivity\" than in-distribution (ID) inputs. The paper's first main contribution is Theorem 5.1, which provides a closed-form expression for this sensitivity obtained from the last layer's statistics. Later authors criticize the multiplicative nature of the score and moving into additive version. Competitive with SOTA methods is shown.", "questions": "weaknesses section have self-contained questions for each item.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- Theorem 5.1 provides an analytical, and geometrically-grounded reason why feature magnitude and output uncertainty are such effective signals for OoD detection. It successfully bridges the gap between abstract geometric theory and common empirical practice.\n- The paper clearly explains why a simple product of (magnitude * uncertainty) is a flawed score: ID inputs are (High * Low) while OoD inputs are (Low * High), and these products can be indistinguishable . \n- ablation studies are rich.", "weaknesses": "- true motivation is empirical rather than quantitative/theoretical.\n- does U(x) provide any benefit?\n- Wang et al 2022 paper has extremely similar formulation on additive score that authors are presenting. Clarification needed. Conceptual overlap is very strong. Isolate contribution from this paper.\n- why PCA is chosen for class subspace Q? how about other methods like unsupervised methods ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zcah3lpkth", "reviewer": "Reviewer_ba3P", "summary": "This work proposes an OOD detection strategy that makes use of information geometric guidance on the last layer of deep neural network model for classification tasks. The strategy implements itself into three different proxies, all based on using trace of Fisher Information Matrix (FIM) as their foundations. The three proxies differ by how the last layer parameter space is decomposed, such that different hyperparameter weights for the traces from each decomposed portions can be applied. The weights, in addition to a couple of projection matrices to decompose the FIM are significant hyperparameters of the proposed OOD detection method. Empirical validations are comprised of ImageNet-based and CIFAR-based scenarios, in which the proposed method is compared to nine other OOD detection methods.", "questions": "The robustness of the additive decomposition of geometric space from last layer may depend on a well-chosen set of relative weights \\lambda's. The sensitivity analysis results in Figure 3 seems to agree on this, and at the same time suggesting some narrow goldilocks zone, which unfortunately seems to be per-dataset, per-model, a posteriori information which requires grid search. Would there be any alternatives that make this hyperparameter search more theory-guided (more post-hoc, if I were to hijack the term)?\n\nThis is mentioned in the limitation. But let me raise it once more. OOD behavior may be also due to factors hidden in the learnable feature (i.e. the non-last-layer of the NN models). How would this information geometric idea extend to the deeper layers?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work in this manuscript proposes a mathematically grounded strategy to approach OOD detection problem for classification tasks. Albeit limited to last layers only, the actual implementation comes in three flavors, making use of trace(FIM) directly, and then decomposing FIM into two pieces, then into three pieces. The last decompostion, which explicitly takes residual orthogonal subspaces seems to make the strategy wholesome and beautifully closed. This gradual improvement makes this work contain educational value as well as researchwise novelty. \n\nThe mathematical complications under the hood of the proposed algorithm are nicely placed in the appendices, in addition to easy-to-read comparison to IGEOOD which also makes use of local geometry of models wrt their parameters. I also appreciate significant portion of work on sensitivity analyses in the appendix.\n\nEmpirical validation section is much richer than a typical theory-oriented manuscript. This is a desirable stance, as the less-mathematically-equipped audience and those who values practical impact much more will find the work more acceptable.", "weaknesses": "The key idea would be much more intuitively understood if theorem 5.1 would accompany the contextualized connection to the strategy found in Eq. 8. I found relevant portions in Appendix D, which was a delight, but at the same time, a dismay that this has to be placed in the appendix rather than places like lines 264-266. Also, the opening of the paragraph on lines 328-332 is personally enjoyed but may throw some readers away due to its mathematical brevity. There are some assumptions simply given in that paragraph without any contextual explanation (e.g. assuming nonzero fraction in line 332), whose meaning may be justified intuitively and concisely.\n\nIt seems that this method relies heavily on a good choice of D_val dataset to calibrate the key parameters \\lambda's, R, and Q. The sensitivity to the size of the D_val is in the appendix, but as the method claims to be an OOD detection method that does not require OOD dataset, I think more thorough analysis (either theoretical or empirical) would provide readers much more insight to this method and OOD detection in general.\n\nPlease correct. Two minor typos I found: \n1) on lines 302-303. \"in Eq. equation 9\". \n2) in the equation for F''' in lines 334-335. L2norm-squared looks weird after the first f_{\\hat{\\theta}}(x) (don't you want it embrace the entire f(x) - RR^T f(x), which is already there?)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wcll4sdr00", "reviewer": "Reviewer_BE7f", "summary": "This paper reframes Out-of-Distribution (OoD) detection using Riemannian information geometry, hypothesizing that OoD inputs exhibit higher \"local Fisher-Rao sensitivity\" than in-distribution data. The authors first provide a novel theoretical proof (Theorem 5.1) that this geometric sensitivity, quantified by the trace of the Fisher Information Matrix (FIM), mathematically decomposes into the product of two common OoD signals: feature magnitude and output uncertainty. Finally, after analyzing this product's limitations, the paper proposes a \"product manifold\" construction to provide a formal geometric justification for robust additive scores, which motivates their final, competitive \"Additive FIM Trace\" method.", "questions": "See weakness section.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper's primary strength is providing a new, unifying geometric language for OoD detection and offers what appears to be the first formal geometric justification for the empirical success of robust additive OoD detectors (like the SOTA method ViM)\n\n* The paper introduces a critical refinement of existing feature-space methods. This is a novel and significant finding that validates a key design choice in its final, competitive \"Additive FIM Trace\" score\n\n* The experimental validation is comprehensive and thorough", "weaknesses": "* The clarity of section 5.3.1 could be improved (I found it difficult to follow and verify). In particular the introduction of the product manifold $M = U \\times U \\times U$ seems unmotivated. Can you add a high-level description of this section? \n\n* The role of the last layer, while clear from a practical standpoint, seems theoretically somewhat arbitrary. Can the authors briefly comment on the role of the remaining parameters (even if the formula become now intractable)? Could they theoretically be added to the statistical manifold? What additional information would this provide? \n\n* The hyper-parameters $\\lambda_M$ and $\\lambda_y$ need to be tuned. Can the authors comment on the range of these parameters for different settings? Are there \"good default\" or the tuning results in widely different values for different settings?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "muohJMY4Ao", "reviewer": "Reviewer_LmQv", "summary": "The paper approaches the OOD detection problem from the perspective of information geometry, using the Fisher–Rao metric. It posits that the *per-sample* trace of the Fisher Information Matrix (FIM) tends to be larger for OOD samples than for in-distribution (ID) ones, meaning that small parameter perturbations cause greater output changes on OOD points. The FIM trace is also interpreted as the product of a \"softmax uncertainty\" term and the magnitude of the penultimate-layer features. Building on this view, the authors propose three OOD detection methods: (1) Standard FIM Trace, using the full last-layer parameter space; (2) Tensor FIM Trace, which projects the two multiplicative components onto subspaces learned from ID data; and (3) Additive FIM Trace, which replaces the product with a weighted sum of these terms and adds a third component capturing the penumltimate features magnitude orthogonal to the learned ID subspace.", "questions": "- Does the metric defined on the product manifold to justify the Additive FIM score retain any important properties of metrics defined above? What motivates calling it a “metric” rather than a heuristic combination of components?  \n- How is the calibration set selected (e.g., randomly sampled, curated subset)? \n- Could the authors discuss the relationship to [1]?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**(S1) Presentation and writing:** The paper is clearly written and well-structured. The Fisher Information framework for OOD detection is effectively introduced and its connection to the first two proposed metrics is well motivated (though the link to the third metric remains less clear; see W1).\n\n**(S2) Analysis of FIM trace metric:** The work offers an interesting perspective on the FIM trace with respect to the penultimate layer parameters, interpreting it as the product of a softmax uncertainty term and the norm of the penultimate layer features. This provides a way to understand the factors influencing OOD detection performance in this and prior works.", "weaknesses": "**(W1) Motivation for the additive FIM trace score:** My main concern is that the motivation and \"derivation\" of the additive FIM trace score do not align with the Fisher–Rao framework developed earlier in the paper. The additive score is introduced by defining a metric on a product of three manifolds, but this construction is no longer Fisher–Rao, lacks a clear statistical interpretation, and is not derived from information-geometric principles. As a result, the proposed formulation appears to be a post hoc way to connect the additive score to the genuinely FIM-based quantities presented in earlier sections.\n\n**(W2) Dependence on a \"calibration set\":** Although the methods are described as \"post-hoc,\" both the Tensor FIM Trace and Additive FIM Trace require access to a calibration set of ID data to fit the projection matrices $Q$ and $R$ (via PCA and/or LDA). This constitutes an additional training phase and requires at least partial access to ID data. Moreover, the performance of these methods is likely sensitive to the specific dataset, model, and selection of the calibration set. The paper, including Appendix A.2 (\"Sensitivity to calibration set size\"), does not clearly specify how this set is chosen or how its composition (not just its size) affects the results. If the selection is random, no error bars are provided to quantify the corresponding variability.\n\n**(W3) Hyperparameters range and clarity:** The two subspace-based methods depend on several hyperparameters: the dimensions of $Q$ and $R$, the weights $\\lambda_M$ and $\\lambda_y$, and the algorithms used to construct $Q$ and $R$ (PCA or LDA). In the experiments, these choices are made via brute-force search, leaving unclear how they should be selected in practice or how robust the results are across different models and datasets. Overall, the discussion of hyperparameters seems to lack transparency. In addition to the ambiguity around the choice of the calibration set (see W2), Appendix A.3 (\"Sensitivity to $\\lambda$ parameters\") is difficult to understand. It refers to the “top-left” and “bottom-right” of Figure 3 as regions of best performance, which does not correspond to the AUC scores actually displayed in the figure.\n\n\n**(W4, minor) Novelty and related work:** I think the paper's claim that it introduces \"a novel perspective on OOD detection through the lens of Riemannian information geometry\" could be at least partially overstated. Previous works did use Fisher Information for OOD detection, one relatively well-cited example is Kwon et al. (2020) [1]. It would help if the authors clarified the difference from [1] and offer a general discussion of information-geometric methods for OOD detection.\n\nOverall, my main concern is that the paper’s main proposed method does not fit coherently within the information-geometric framework developed earlier (see W1). This raises doubts about the usefulness of the framework itself. If the proposed methods demonstrated substantial empirical improvements over SOTA, this could be less concerning. However, since the paper’s primary goal appears to deepen understanding of the mechanisms behind OOD detection, this is a critical weakness.\n\n\n\n## References\n\n[1] Kwon, Gukyeong, et al. \"Backpropagated gradient representations for anomaly detection.\" European conference on computer vision. Cham: Springer International Publishing, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)