Pseudo-Non-Linear Data Augmentation: A Constrained Energy Minimization Viewpoint
Authors: Pingbang Hu
Mahito Sugiyama
Venue: ICLR 2026 Poster (2026)
Keywords: data augmentation
information geometry
energy-based model
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose a simple, information-geometric approach to data augmentation that is learning-free, efficient, controllable, and broadly applicable to structured data.

Abstract: We propose a simple yet novel data augmentation method for general data modalities based on energy-based modeling and principles from information geometry. Unlike most existing learning-based data augmentation methods, which rely on learning latent representations with generative models, our proposed framework enables an intuitive construction of a geometrically aware latent space that represents the structure of the data itself, supporting efficient and explicit encoding and decoding procedures. We then present and discuss how to design latent spaces that will subsequently control the augmentation with the proposed algorithm. Empirical results demonstrate that our data augmentation method achieves competitive performance in downstream tasks compared to other baselines, while offering fine-grained controllability that is lacking in the existing literature.

Published as a conference paper at ICLR 2026
PSEUDO-NONLINEAR DATA AUGMENTATION: A CON- STRAINED ENERGY MINIMIZATION VIEWPOINT
Pingbang Hu University of Illinois Urbana-Champaign pbb0il 1 iinois s. edu
Mahito Sugiyama National Institute of Informatics & SOKENDAL to@nio c.....o......... jip
ABSTRACT
We propose a simple yet novel data augmentation method for general data modal- ities based on energy-based modeling and principles from information geometry. Unlike most existing learning-based data augmentation methods, which rely on learning latent representations with generative models, our proposed framework enables an intuitive construction of a geometrically aware latent space that rep- resents the structure of the data itself, supporting efficient and explicit encoding and decoding procedures. We then present and discuss how to design latent spaces that will subsequently control the augmentation with the proposed algorithm. Em- pirical results demonstrate that our data augmentation method achieves competi- tive performance in downstream tasks compared to other baselines, while offering fine-grained controllability that is lacking in the existing literature.
INTRODUCTION
Data augmentation has advanced considerably in recent years, driven largely by the increasing use of generative models (Kingma & Welling, [014] Chadebec et al., 2022 Antoniou, 2017 Trabucco al. 2024) to meet the demand for larger and more diverse datasets (Feng et al.] 2021 Wong al., 2016). Beyond traditional domains such as images, these methods have been extended to wide range of modalities. Despite their promise, however, generative-model-baee several fundamental challenges. First, data augmentation is most valuable when training data is scarce, yet in such cases, we typically lack a pre-trained foundational model for the target domain. This creates a paradox: before we augment the data, must first train generative model- reintroducing the very problem of limited data. Second, even when suitable foundational models are available, the computationally intensive nature of deep generative methods poses practical obstacles. Since effective augmentation often requires generating data of the same order as the original dataset, the cost of large-scale generation can quickly become prohibitive. Third, augmenting data with gen- erative models raises concerns about their interpretability and controllability (Guidotti et al. 2018)) Consequently, even when these models perform well, the lack of understanding of the underlying transformations of the augmented data makes it difficult to control the generated outputs, which poses a significant risk in the case of high-stakes scenarios (Rudin) 2019).
augmentation faces
In this work, we propose new data augmentation framework that addresses the challenges out- lined above by providing a learning-free, efficient, and controllable algorithm applicable across diverse data modalities. Our approach builds the well-established theory of energy-based mod- els (Xie et al., 2016), together with recent advances in log-linear models on partially ordered sets (posets) (Sugiyama et al. 2016 2017) and information geometry (Amari, 2016; Amari & Nagaoka 2000 Ay et 2017)) Conceptually, our framework resembles an autoencoder (Kingma & Welling, 2014.. We begin by parametrizing data as discrete probability distributions
curved statistical manifold S. The data is then encoded into a chosen "latent space" B C S via forward projec- tion. Within this latent space, simple augmentation procedures informed by the encoded data are applied. Finally, the resulting "augmented representation" is backward projected to the local data space D C S. yielding new augmented data. As our proposed algorithm exploits the duality of the projection in the statistical manifold S, where it is linear in the manifold's 's intrinsic coordinates yet non-linear in the ambient space, we hence term it pseudo-none non-inear- data augmentation.
Work done in part while the author was visiting National Institute of Informatics.

---

Published as a conference paper at ICLR 2026
This design offers three key advantages. First, it is Cearningffree the sub- manifold structure is constructed explicitly, allowing direct control over the properties of the augmented data without the need to train generative model. Second, it is computationally efficient: both forward and backward projections can be formulated as convex programs and solved with efficient first-order methods such as gradient descent. Third, it provides controllability: leveraging prior knowledge about relation- ships among features, one can adjust the choice of S and the sub- uub-manifold of projection to tailor the statistical properties of the augmented data. Our contributions are summarized follows:
We propose novel framework for modeling structured data (e.g.. tensors) within statistical manifold using energy-based models. This framework captures the intrinsic geometry of data and enables the design of geometry -oware algorithms.
We develop the bgeudo-nonlinear data augmentation algorithm under this framework. The method is learning-free, efficient, and controllable, and applies broadly across different data modalities. We empirically validate the effectiveness of our approach, showing that it achieves competitive or superior performance compared to both generative- generative-model-bee soreee-odell-beeeee baselines (e.g., autoencoders) and classical augmentation methods across multiple datasets and modalities.
RELATED WORK
2.1 LEARNING-BASED DATA AUGMENTATION
Data augmentation has proven to be effective enhancing deep learning training by increasing
dataset size, improving model robustness (Rebuffi et al. (021), and introducing implicit regulariza-
tion (Hernández-Gree Garefa & König 2018). These techniques have been applied across various modal-
2022a) and images (Shorten & ities, including text (Shorten et al. [201]] Feng et al.) 2021 Li et al.
Khoshgoftaar 2019 Mumuni & Mumuni 2022 Wang et al. 2017). J Due to the generality and the
popularity, much of the recent progress in data augmentation for general modalities has been driven by advancements in generative models, such as autoencoders (Kingma & Welling, 2014] Chadebed
ett al.. 2022), generative adversarial networks (Antonioul 2017). and diffusion models (Trabucco et al. 2024). Despite the progress, to date, there is no fully satisfactory solution for the two chal- data augmentation, i.e., efficiency and controllability. For example, the design of controllable GANs (Li et al. 2022b; [She et al.) (2021) and efficient flow- based models (Geng al. 2025) remains an active area research, and exploration this direction is largely limited to specilic domains such as images.
lenges mentioned for snnerativeemodel-baae
2.2 LEARNING-FREE DATA AUGMENTATION
Learning-free data augmentation methods (Maharana et al. 2022) that do not rely generative models are particularly appealing because they construct an explicit, low-dimensional "latent space" in which data be represented and augmented with fine-grained and intuitive control and inter- pretability. These latent spaces are typically derived from classical dimension reduction techniques such Principal Component Analysis (PCA) ) Wold et al., 1987) and Singular Value Decomposition (Stw)(St 1993). In general. these methods identify an optimal linear subspace and then per- form augmentations that respect this (Euclidean) geometry of the ambient subspace. Beyond their simplicity and transparency, linear methods also provide useful insights; for instance, PCA reveals principal directions that capture the dominant modes of variation in the data.
A major limitation of linear dimension reduction for augmentation, however, is the difficulty of the inverse problem: reconstructing nigh-dimensional data from -iwddmensional representations with- out a learned decoder is often non-trivial. Some works attempt to circumvent this issue by leverag- ing linear dimension reduction only indirectly for augmentation (Abayomi-Alii et al., 2020) Sirakov et al.) 2024). Other approaches, such as mixup (Zhang et al., 2018). bypass dimension reduction entirely and operate directly in the original data ambient space. While popular in practice due to simplicity, these methods are typically heuristic, application-speciffce and harder to generalize.
Non-linear generalizations of dimension reduction is often referred to as manifold learning (Meilà & Zhang 2024), which provides alternative route. Methods such as t-SNE (Hinton & Roweis, 2002 Van der Maaten & Hinton, 2008). Isomap (Tenenbaum et al. 2000), and UMAP (MeInnes) al. 2018) aim to exploit the manifold hypothesis, which posits that high-dimensional data lie
2

---

Published as a conference paper at ICLR 2026
near lower-dimensional manifold embedded in the ambient space. Their goal is to uncover this manifold and produce a smooth embedding that captures the intrinsic geometry of the data.
In principle, manifold learning could avoid the inverse problem by recovering a low-dimensional manifold with minimal information loss, making it an attractive candidate for data augmentation. In practice, however, this goal is rarely achieved without incorporating learning mechanisms (Duque) ltt al. 2020 Coifman & Lafon 2006] Williams & Seeger 2000; Vladymyrov & Carreira-Perpinan, 2013 Han al.) 2022). A workaround is exploit the latent space learned by a model already trained on a downstream task, as in manifold mixup (Verma el al. 2019). Yet these approaches again sacrifice interpretability and controllability, compared fully learning- free augmentation methods.
3
PRELIMINARY
3.1 DUALLY-FLATNES IN INFORMATION GEOMETRY
Information geometry studies the structure of statistical manifolds S within the space of probability distributions. In this paper, we are primarily concerned with the space of an exponential family {po 0 E RD), where each pe denotes a probability density function parameterized by 0. We focus on the key concept in this field, dually-flatness, in this preliminary, while directing readers to Section A and Amari (016)) for more comprehensive details.
The starting point is the observation that the log functtion (0) (also known as the cumulant generating function in statistics and free energy in physics) of an exponential family with density pe is convex in the natural parameter 0 E RDD This convexity induces a natural coordinate system, 8, on S, defining both the Riemannian metric g (2b)(0) and the Bregman divergence (Bregman 1967) (pe. p&r With these structures, the manifold (S. g) is flat, meaning that any curve ((t) at + (where a, b E IR" P are constants) is a geodesic and lies entirely within S. This flatness is known e-flatness, and the geodesics are referred to as e-geodesics
-eodeesss
The dual structure arises from the Legendre transform (Legendre 1787). which generates the dual function 4² (1)) where 17 RR, is the expectation parameter. This dual function is also convex, giving rise to the expectation coordinate system the dual Riemannian metric g", and also the dual Bregman divergence Da which is the well-known Kullback- Leibler divergence DKI. (Eq. (3) The corresponding flatness is termed m-flatmess, with m- mn-geodesics or dual- naal-geodesics as its geodesics.
Remark 3.1. An e-flat (m-flat) sub-manifold can be defined by forcing linear constraints on the O coordinates (1) coordinates) (Amari 2016 Chapter 2).
Dually-flatness emerges from the interplay between these two structures. Specifically, for any point (distribution) p in S, there is unique point p" on an e-flat sub- manifold B C S that minimizes the dual Bregman divergence D (p, q) = DKL (p. q) (Amari) 2016 Theorem 1.5). This process, known as the m-projection, can be efficiently solved via convex optimization (Section [A.2]. The dual holds when switching e and m. Projection is a central tool in information geometry with profound implications for understanding the geometry of S, which we will use later.
3.2 STATISTICAL MANIFOLD ON POSETS
A set Q a partially ordered set (poset) if is equipped with a partial order "<", relation satisfying the following for all y, E Ω: 1.) x. < x (reflexivity); 2.) x < y and y S. x implies x = y (antisymmetry): 3.) x S y and y S 2 implies x S z (transitivity). We focus on finite posets Ω with a bottom element 1 such that 1 < for all x E Z to prevent some technical difficulties.
Given such a poset Ω consider a discrete random variable X with finite support with its prob- ability mass function p: Ω + R2o. p(x) Pr(X x) for x Ω. For discrete probability distribution p over a poset Ω2 the log-linear model on posets recursively defines 0: Ω + R as log p(x) = Eys, (ιy) for all E Q (Sugiyama et al. 2017]. This model belongs to the expo- nential family, with 0 corresponding to the natural parameters, except for ((L), which coincides with the partition function. Thus, all discrete probability distributions over Ω form a ([ 1)- dimensional dually-flat statistical manifold S = {p: Ω - R2o Exen p(z) = 1}. with the dual coordinate systems (0, 17) depend the poset structure.
3

---

Published as a conference paper at ICLR 2026
Remark 3.2. One can think of (z) I, and the relation between ((x) 'ss on different elements x o's is specified by the poset structure.
( for each element St as specifying the energy (i.e., p(z)) for
PSUEDO-NONLINEAR DATA AUGMENTATION
We first present our proposed framework in Section and the projection algorithms in Section 4.2 then we combine and apply them to data augmentation in Section 4.3 Finally. we discuss two important features of the proposed method regarding controllability and efficiency in Section Throughout this section, we will use positive fensors as our running example (Theorem 4.1
4.1 LOG-LINEAR MODEL ON POSETS FRAMEWORK
Motivated by Theorem 3.2 by associating each element x E with a feature dimension, we can specify the geometry among features-ie., design the poset ttructure-usin prior knowledge or natural structure present the data. The resulting models define an energy-aware, curved statistical space that faithfulll reflects the prescribed relationships among features.
More specifically, given a dataset {2ili=1, we first embed the data into a statistical manifold S by leveraging the log- log-linear model posets, which provides geometric structure induced by the energy-based modeling. The process works three steps: 1.) models each 2; as a real-valued poset; 2.) embeds the real-valued poset into the statistical manifold S by viewing it probability distribution; 3.) computes the corresponding two coordinate representations using the log-linear model on posets. See Figure for an illustration. We now explain each step in detail below.
Data
Poset Ω
Σ1= =1
3
9
(x
Design
Embed yp
Log-Linear (0, n) Model
0.2
8
21
-30.18.1
Value Poset E S2
Probability Distribution po E S
Given structured data, we first design poset that reflects the structure or the relationship between features. The resulting real-valued poset is then embedded into the statistical manifold S as a discrete probability distribution (e(x) via an embedding ş. Finally, the log- log-linear model on posets provides the dually-flat coordinates (8, n) for pe. which can be computed efficiently (Section 3.2).
Figure 1:
Real-Valued Poset. In the typical machine learning pipeline, inputs are often constrained to be vectors or matrices, which fail to accommodate more complex data. In contrast, posets are flexible enough to capture data with structures, including vectors, matrices, tensors. In general, any data structure that admits a natural partial order can be modeled by poset: for instance, D-dimensional vector E IRP (i.e., 1 orrrrd tensor) admits structure that can be modeled by the poset with the partial order being the natural order between positive integers. Similarly, other common data structures, such as matrices or tensors, can be treated in the same way. On the other hand, as noted at the beginning of the section, custom poset structure Ω can also be specified, either with or without the natural structure, to reflect prior knowledge of the relation between features.
[D]
We can then define the real-valued poset, which is a mapping from the poset Ω to the set of real numbers such that each entry (element) of the data structure (poset) x S2 is associated with feature in R. We denote the set of real-valued posets as Ωpp In the D-dimensional vector example, = D). each element x E corresponds to one of the dimensions. Associating a real number to each dimension then corresponds to an element in lgg.
Embedding, To embed the data {ez E Pz which are now modeled as real-valued posets, the statistical manifold S which concerns with discrete probability distributions, we want an embedding ;p: Op + S and its inverse -- : S + Ωg, such that (xen(p(2)))) (x((2)), = 1 for all with dim(S) = D
In other words, s(x;) gives probability mass function of discrete random
'One can also consider the manifold of positive measures of dimension D and avoid the potential scaling issues. For simplicity, we omit this trivial extension in the presentation.
4

---

Published as a conference paper at ICLR 2026
variable over the poset, where (p(2))) is the probability of sampling E S when sampled from ((()), representing the energy of the feature x. From the perspective of energy- based modeling, as noted in Theorem 3.2, yp can be naturally induced, e.g.. for tabular frequency data, customized to reflect the desired energy relationship between features. We note that in the latter case, a joint design of up with the poset structure 52 is often more expressive to describe desirable energy structure.
Dually-Flat Coordinates. Finally, from the log- linear model on posets introduced in Section 3.2 for each 2/ == sp(,,) E S, we associate the dually-flt coordinates ((()) E _D and nn(2() E R R= Such coordinate systems are defined with respect to the underlying poset structure S2 providing representation that captures the prescribed geometric structure among features of the data. We next illustrate the framework with one canonical example, positive tensors, in Theorem 4.1
Example 4.1 (Positive tensor). A d -order tensor T E RRxx.-laaa RDD is a multidimensional array with real entries for every index vector V = (i1,.... id) E [41] X ... x [La] =: Q where for each k. [Rx] := (1....... for a positive integer Ig. Tensors with entries all being positive are called positive tensors, denoted as P E R 20
natural partial order "s" between two index vectors v = (iI,..., Ed), (fi,-...3dd for tensors is that S w if and only is S for k [d]. Finally, for positive tensors, simple embedding yp: TR 20
Figure 2: Natural poset structure of 3rd-or tensors in R333x3
where p == cpPP) Q + R2o such that P = P Ewen Pw for all v E Ω can be defined with a natural empirical inverse (see Theorem 4.3).
Theorem 4.1 applies many common data modalities. For example, a color image can be repre- sented as a 3 -order tensor, where the first two dimensions correspond to height and width, and the third dimension encodes the color channels. Time-series data likewise admit a tensorial poset structure, with the temporal dimension inducing natural ordering among features at discrete time steps. Despite this flexibility, our framework comes with one notable limitation:
Remark 4.2 (Invariance). Because the frumework relies on specifying partial order over the index set, is not naturally equipped to model invariances under index permutations. This can introduce unwanted bias, for instance, in settings such graphical data. Nonetheless, key advantage of being learning-free and fully transparent that the source of such bias is explicit, allowing one to identify and, when desired, mitigate it by appropriately modifying the modeling choices.
To conclude, we emphasize that the notion of energy in our framework acts as a modality potential function that quantifies the stability or plausibility of data configuration under the chosen poset embedding. Once the poset is specified, both the energy function and the induced manifold structure follow directly, yielding clean and computationally convenient representation that sup- ports subsequent computations and algorithm design, which we will discuss next.
4.2 FORWARD AND BACKWARD PROJECTION
Generated Data
O
Ozn
We now demonstrate how to incorporate the log- linear on poset framework with projection the- ory to conduct data augmentation. Our algorithm mimics the architecture of autoencoders, focusing on two building blocks: the encoder Enc(-) and the decoder Dec(.). First, for the encoding step, we formally explain how projection theory can be applied to perform dimension reduction and obtain compact representations within our frame- work. Next, for the decoding step, we introduce our proposed algorithm, termed backward projec- tion, which serves as the inverse of dimension re- duction. Figure provides an intuitive geometric illustration for our proposed algorithms.
4
22
O
S
D
B
O 3 w
Wn
Forward Backward
Figure 3: Illustration of forward and backward projection. Here, w.: latent representation of the original data obtained from forward pro- jection B; w augmented latent represen- tation; w² + z't: backward projection to D, obtained from the original data of the nearest neighbor(s) of w" in the latent space.
Forward Projection. The embedding from Qa to S introduced in Section [.1 maintains the di- mensionaliy. To achieve dimension reduction,
5

---

Published as a conference paper at ICLR 2026
leverage the projection theory: by projecting 2 = (((2)) onto low-dimensional flat sub- manifold called base sub-manifold B _ with dim(B) < dim(S), we obtain the desired encoding Enc == Projg o S2 + B that maps the data to a low-dimensions that the encoding Enc(.) is smooth and well-defined as the projection is unique when B is flat and minimizing either the primal the dual Bregman divergence, depending on either B is e- m-flat.
latent representation. Note
Backward Projection. One of the technical burdens is that the encoding Enc(-) is not invertible, hence a perfect decoding Dec(-) is mathematically impossible, even when Enc(-) only involves simple linear projection in Fuclidean space. Here, we propose simple, geometrically intuitive, and data-centric solution that aims find the inverse of the projection with theoretical guarantees.
The high-level intuition is simple: we assume that similar data will result in similar projections. Hence, given a point in the low-dimensional latent representation space, we try to "project it back" to approximate the original dataset by exploiting the fact that we have access to the inverse of the dataset's 's projection, which is the dataset itself. Specifically, we can artificially create a sub-manifold D around a subset of the dataset that captures the local geometric structure of the dataset around the latent representation, and backward project onto
Formally, assuming that we have access to the embedded dataset {2/ = (p(2;)}?= and their pro- jected result {w, (2)] for some flat base sub-manifold B. To find the inverse of some given point wt E B assuming it comes from the projection on B, we first find w"'s k-nearest neighbors among wt ss obtaining a size index set N C [n]] with |N = k. Then we create flat sub-manifold DD called local data sub-manifold based on the pre-images z!'s of these w; 's, and project w* on 2> to obtain the inverse x'* Projg --1 (w*) == Projo(w"), The construction of D can be arbitrary, in particular, one can easily control the degree of freedom of the result- ing z²: for instance, from Theorem 3.1 given the nearest neighbor z/,, one can define an e-flat
D == {0 Rdim(5) (0)z (0(2).) )), for some 22} by fixing some indexes of 0 to be the corresponding 0-coordinate values of Z/---
Algorithm iin Section B summarizes this procedure, which we termed backward projection. With access to Projg' (.), decoding is simply Dec == 9 o Proje": B - Qg. Algorithm is a geo- metrically intuitive, data-centric algorithm with desirable theoretical guarantees such divergence minimizing when projecting on the constructed local data sub- 1o--manifol D.
4.3 PSUEDO-NONLINEAT DATA AUGMENTATION
With all the building blocks in place, we can now formally describe the proposed data augmentation algorithm, which consists of: I.) encoding, 2.) augmenting, and 3.) decoding.
Encoding. As described in Section 4.2 the encoding Enc == Projg op is simply combination of the embedding followed by a projection. Notation-wise, we write w; == Enc(zi).
Augmenting. To generate an augmented data 2", we first generate new representation w" in the latent space, which in our case, is a pre-specified flat base sub-manifold B. This w* can be generated in various ways, such controlled perturbations of the original representations or linear mixture of two arbitrary original representations.
the decoding Dec == to is simply a combination of backward projection (Algorithm with the inverse of the embedding. Notation-wise, we write
Decoding. As described in Section 4.2,
2" = Dec(w*)= = = ((²*) where
Proje (w=)=""w===>))") == Projp
(w*)
The proposed method integrates the nonlinear forward and backward projections encoding and decoding, which we summarize the above in Algorithm with an illustration given by Figure
Remark 4.3 (Positive tensor). With Algorithm the empirical inverse -1 for the positive tensors in Theorem 4.1 can be defined as the inverse of the average of the scaling among nearest neighbors.
4.4 SUB- MUB-MANIFOLDS FOR POSITIVE TENSORS
It is evidence that the proposed method is learning-free. In this section, we describe how to con- struct flat nub-manifolds (for and D) to control the augmentation process, and how these sub- manifolds naturally admit efficient projection algorithms. For clarity and concreteness, our discussion on the case of positive tensors, while noting that the principles and arguments extend to broader settings for other data modalities, poset structures, and the design of the energy potential.
focus

---

Published as a conference paper at ICLR 2026
Designing Sub-Manifolds. We start by discussing intrinsic trade-off of choosing the dimen- sion of S S we aim to forward project on. Clearly, more information about the data is preserved after forward projection onto B as dim((B) increases. Hence, the quality of the backward projec- tion Projg-() (-) (Algorithm increases along with dim However, in the extreme
when dim(B) (B) Re dim(S), Algorithm 2 becomes less effective as the augmenting step now suffers from the curse of dimensionality. Previous studies such a trade-off of choosing (i) (Sugiyamal et al. 20118; Ghalamkari et al.) 2024) reveal how one should construct flat base sub-manifolds. In particular, the many-body tensor approximation (Ghalamkari et al. 20244] Derun & Sugiyama 2025) aims to capture a hierarchy of mode interactions with different (B) for positive tensors within the log-linear model on posets. Specifically, the Ê-body approximation considers projection on the following sub-manifold:
M₁ == (0 E Rddm(() 0- = 0 for all non (-hody parameters # E E[],
where the Ê-body parameter corresponds body and two-body parameters (Ghalamkari et al. 2024). Intuitively speaking, an é-body parameter captures the interaction among & different modes. Hence, when B .Me, all interactions between modes of orders higher than are neglected. This approach provides principled way of designing the latent space with a clear understanding of what each dimension significs.
non-one indices, acting as generalization of one-
On the other hand, dual-like trade-off exists for the local data sub-manifold D. Recall that the goal of backward projection is to project back to the "local data" space D given by a set V nearest neighbors of generated latent representation. When dim increases, backward projecting onto D has a higher degree of freedom, which is desirable for data augmentation. However, in the extreme dSm(S), the backward projection becomes unconstrained and potentially
case when dDm generates gibberish results. Hence, a natural construction of 'D is to consider the "dual" notion of Me, where we now allow all non ê-body parameters to vary while fixing every f-body parameter to be the average of the 0 values among N:
1 Σ (0(24-)) for all Ê-body parameters x E EQ (2) K
(2 = dddi(())
0,
Σ
MEN
These constructions offer practical design choice for Algorithm while providing the desired properties. For instance, By choosing an appropriate E, both Ms and Mj can capture specific information with desired degree of freedom.
Efficient Projection. For sub-manifolds Mr designed for many-body approximation with B many non-fixed indexes (i.e., f-body parameters), the projection can be efficiently computed via formulating the projection as convex program over B variables that can be solved via gradient descent in polynomial time (in terms of B). We note that the projection in the statistical manifold offers other desirable theoretical guarantees, such as minimizing the KL--divergence (correspond- ing to energy minimization) and uniqueness of the projection. Moreover, in the case of many-bod, approximation, it admits an efficient algorithmic implementation, since the gradient of the corre- sponding convex program has closed-form, which makes the optimization extremely efficient. We refer the reader to Section A.2 for detailed discussion.
5 EXPERIMENTS
The structure of this section is as follows. Firstly, Section 5.1 demonstrates the energy aspect of the proposed framework, validating our motivation. Then, Section 5.2 illustrates the proposed pseudo- nonlinear data augmentation on real-world visual datasets. Finally, apply the pseudo-nonlinear data augmentation on downstream tasks in Section 5.3
5.1 LOG-LINEAR MODEL WITH ENERGY INTERACTION
Setup. We demonstrate our proposed log-lineut model on the posets framework with simple syn- thetic data under controlled feature interactions. The goal of this section is to illustrate both the energy-mode mooggiig perspective of our proposed framework, as well as how designing base sub- manifolds B allows to capture higher-order mode interactions among data.

---

Published as a conference paper at ICLR 2026
In particular, model both the 2-dimensional intensity data and 1-dimensional time-series data as a 2d 2ndl-rder tensor as described in Theorem 4.1 where we apply reshaping to time-series data to capture higher-order mode dependencies across the temporal dimension.
Experiment. Firstly, separable and non-separable 2- dimension intensity data 7. Separable data (Figure 4{a)) corresponds to data without feature interaction, where we generate ji = f() x g(j) for mode and j. On the other hand, non-separable data (Figure[4(b) cor- responds to data with intricate feature interaction, where [(),j) has higher- order interaction between modes i and j such that [ .) is not factoriz- able From Figure 4 we remark two central to our framework: 1.) we can control the order of mode interaction preservation intuitively with ap- propriate sub-manifold design; 2.) even when the base sub-manifold does not have enough capacity to capture the mode interactions (e.g., 16,,--bo 1-body approximation in Figure(((b) it nevertheless captures the essence of the data within its capacity in the energy- menegy-minmmizing sense.
consider
Original
2-bedy Apps.
J-body Appic Smrer
X-bddy Appe.
2-bdy Appi. kmar
Weak mode interaction with separable data.
Appe,
(b) Strong mode interaction with non-separable data.
Figure 4: Mode interaction and approximation.
interesting properties that
Following similar setup, we con- duct case study Figure illus- trate the advantages of our method's induced geometry compared the vanilla ambient space geometry. In particular, we interpolate between separable and non-separable data within both the base sub-manifold (Figure (5a) and ambient space (Fig- (r)). and measure the interaction energy: the discrepancy between the I-body approximation 2-body approximation of the interpolated data. The result is shown in Fig- (t)](c) where see that energy-aware base sub-manifol consistently induces lower interaction energy compared to the ambient geometry, indicating that our method, when operating on top of which, utilizes less energy compared to its ambient space geometry counterpart.
t-0.20
b-0.50
1-200
Energy Awars
(a) Energy-aware interpolation.
tw0.00
two
1-0.50
1w0.55
1-100
120
32 Parameter
24
(b) Ambient space interpolation.
(c) Energy increase.
Figure 5: Interpolation energy in different geometries.
Finally, in Figure [ we consider time series data with temporal de- pendencies emerging from higher- order interactions. A similar trend as Figure 4 emerges: the latent space geometry derived from a sim- ple base sub-manioldd results in a larger error at the place where higher-order mode interactions presented (i.e., high frequency chirp pattern). In contrast, an appropriate base sub-manifold can capture such mode interaction perfectly.
150
Amplitudo
b-boyy Appo.
1.00
bobc Erion
1.00
150
200
250
300
75.
400
Time (t)
Figure 6: Approximation of temporal-deppendet time-series.
5.2
CONTROLLABILITY WITH CHOICES SUB-MANIFOLDS
Setup. As discussed in Section 4.4 constructing the sub-manifold carefully provides the essential controllability. demonstrate this with MNIST (LeCun 1998) and CIFAR (Krizhevsky Hinton 2009 where we first apply the log-linear model on posets for positive tensors (Theorem 4.1 by
Equivalently, separable data is linear in log space where log [(t.j) log f() + log g(j), while non- separable data has non-linear mode interactions log space.
8

---

Published as a conference paper at ICLR 2026
normalizing features to be positive and reshaping the features tensor of suitable dimensions, then apply our proposed aseudo-nonlineat data augmentation afterwards. In particular: 1.) for MNIST, we let B = M and D .Mi, which correspond to preserving shape information; 2.) for CIFAR, we first carefully reshape the colored images to higher-order tensors, and let B = Ms and D = M which correspond preserving fine- grained collective shape and color information.
Experiment. Figures ((a) and 7( show the results of the forward projection, while Figures ( and 7(e) show the results of the backward projection the latent representations sampled from the
kernel density model M fitted the results (8-coordinates) of Figures ((()) and [(d] respectively.
For MNIST, we see that the augmentation results (Figure 7()) with backward projection success- fully reconstruct the digit structures, indicating that the essential shape information indeed pre- served and separated in the latent space B to provide non-trivial neighbor information for construct- ing sufficiently good D for backward projection. Note that the local data sub-manifold 'D has dimension of 767, indicating a high degree of freedom for backward projection.
(a) Forward projection on with (B5
17.
(d) Forward projection on B with dim(1) = 1410.
O
/
2
3
4
5
6 7
9
(b) Backward projection D with dim(D) 767.
(c) Backward projection D with dim(D) 2334.
O
2
3
4
5
6
7
8
9
(c) The closest training data of Figure 7(b)
(I) The closest training data of Figure (
Figure 7: Results of MNIST (Left) and CIFAR- (Right) for Algorithm The first row shows some representative latent representations from the dataset, while the second row shows the backward projection from an augmented latent representation.
More interesting results for CIFAR-10 10 are shown in Figures 7 to 7(t) By our proposed projection- based augmentation method, the fine-grained shape and color information preserved. For stance, the third image, ostrich, successfully preserves the fine-grained shape and color relationship (e.g., colors for eyes and beak, and small pink flowers the background), while the crude shape- to-color information is lost (e.g., colors for the background without shape details shift noticeably). The same trend can be observed consistently, validating the proposed method's s efficacy.
In practice, by carefully reshaping the data into higher-order tensors such that some modes of the ten- sors contain the essential relationship between features that one wishes to control, with many-body approximation, it is possible construct suitable sub-ma manifolds that preserve the chosen informa- tion, providing a controllable augmentation of the original data via simple projection operations.
5.3 CLASSIFICATION PERFORMANCE
Setup. We apply our proposed pseudo-nonlinear data augmentation algorithm on downstream classification tasks across different datasets with various modalities, including image (MNIST and CIFAR-10), 10), audio (Speech Commands (Warden 2018)), and tabular data (Connectionist Bench (Se- jnowski & Gorman 1988). Taiwanese Bankruptcy (Journal 2020). and Wine Quality (Cortez et al. 2009p. For each dataset, we train classifier on both the original training set and an augmented training set, where the augmented portion corresponds to 20% of the original training size and is generated from the training data. The classifiers used are ResNet-18 (He et al. (016) for CIFAR- 10, M5 (Dai et al.) (2017) for SpeechCommands, and simple MLP for the remaining datasets. All of the models are evaluated on 20 randomly bootstrapped test subsets, each containing 50% of the original test data. Further details of the training setup are provided in Section C.1
Experiment. Several baselines compared against our proposed method, including both learning-based and learning-fre methods, including: 1.) pseudo-nonlimear (PNL, ours), 2.) au- loencoder-based augmentation (AE) (Kingma & Welling 2014 Chadebec al.
2022 3.) mixup
Moor recent leamnin-- hased baselines, e.g., diffusion- hased augmentation (Trabucoo et al. computationally infeasible at our scale and can only he applicd in "few-shot" setting.
2024 are often

---

Published as a conference paper at ICLR 2026
Table I: Test accuracy of classifiers trained on different datasets.
Training Set
Dataset
MNIST
CIEAR-IP
Speech Commandss
Connectionis
Bankrupicy
Wine Quality
OG
97.98 + 0.19%
88.57 0.57%
81.48 + 0.50%
88. 10 1.2...... 8.58%
96.51 +: 0.56%
55.001 = 1.69%
OG
97.98 = 0.21%
89.89 0.44%
82.98 + 0.50%
83.24 = 7.06%
96.17 + 0.57%
67.85 = 1.81%
PG
97.91 0.21%
0.48%
84.35
93.81
96.53 0.47%
59.03 1.74%
OGAE
97.97 1 0.25%
88.36 0.46%
83.13 1 0.32%
82,86 7.59%
95.92 1: 0.62%
5T.23 = 1.67%
MG
96.15 = 0.23%
86 60 0.19%
81 $1,85- = 0.61%
89.29 = 1.97%
96.35 + 0.68%
57.76 = 1.67%
MG
97.52 0.30%
88 02 0.39%
83.06 0.54%
91.19
5.06%
96.44 0.53%
58.70 1.74%
(MU) (Zhang et al. 2018) 4.) manifold mixup (MMU) (Verma et al. 2019), and 5.) standard aug- mentation (STD). For images, STD includes standard techniques such as random cropping, flipping. rotations, and affine transformations. For speech, STD combines random volume scaling, time stretching, MelSpectrogram conversion, frequency masking, and time masking (Park et al. 2019). For other data types, STD is implemented as Gaussian noise perturbations.
Results are summarized in Table I where we denote the original dataset as OG, and the augmented dataset using method AG OGAG In most cases, the classifier trained the augmented data achieves a better prediction accuracy compared to the one trained only on OG. One exception is the dataset associated with image modality, where all other baselines perform worse than OG and OGSTD, A potential explanation is that the baseline augmentation algorithms for other baselines act more like regularizers, while OG explicitly forces the classifier learn the robust visual features associated with modality-specific transformation, promoting properties such as rotation, location, and color invariance. Nevertheless, in general, PNL consistently outperforms other baselines on all datasets, demonstrating a competitive performance across modalities.
Importantly, we highlight the stability, a crucial but often overlooked goal in data augmentation. This is best illustrated by the Connectionist Bench dataset, which contains only 208 data points and 60 features, posing a significant challenge for stable training and testing for generalization. From Table 1 the standard deviations in accuracy on this dataset are noticeably higher for OG, OG and OGAE In contrast, our method achieves substantially lower standard deviation of 4.54%, indicating improved consistency across bootstrap testing runs. Other data augmentation baselines, such as OG and MMU also achieve substantial improvement. Still, our method achieves the lowest standard deviation among all, and this trend is consistently presented across all datasets.
5.4 ADDITIONAL. EXPERIMENTS
We conduct series of additional ablation studies and experiments in Sections to Specif- ically, Sections C.2 and assess the robustness of our proposed method and the impact of aug- mentation on downstream task performance. In contrast, Section C.4 provides justification for the necessity of forward projection. Finally, Section C.5 explores the effect different latent space design choices augmentation outcomes, offering insights into how these design decisions can be leveraged to better control the augmentation process.
CONCLUSION
We introduced the pseudo-nonlinear data augmentation framework, which leverages information geometry and energy-base based models to provide learning-free, efficient, and controllable augmen- tation method. Our approach, grounded in the log-linear model on posets, endows data with rich information-geomente
structure induced from the designed energy potential, facilitating both ge- ometric reasoning and principled algorithm design. A key component is the backward projection algorithm, which reverses dimension reduction a geometrically intuitive and data-centric manner.
Through extensive experiments, demonstrated the effectiveness of our method across diverse modalities and datasets. In particular, it enables scalable augmentation for general data types while offering controllability via the design of 1.) the base sub-manifold B, 2.) the local data sub-manifie maniioldo D. and 3.) the poset structure Q. Empirically, our framework outperforms both learning based and learning- frrming-free data augmentation baselines, on common modalities such images and audios.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by JSPS. KAKENHI Grant Number JP25H01 JP25H01124, Japan and JST, CREST Grant Number JPMJCR22D3, Japan.
ETHICS STATEMENT
We have carefully reviewed and adhered the ICLR Code of Ethics. This work does not involve human subjects, personally identifiable information, sensitive data. All datasets used in our ex- periments are publicly available and commonly used the community. We have followed standard practices for dataset preprocessing and model evaluation to avoid introducing unfair bias. To the best of our knowledge, our methodology and findings do not pose potential harm, nor do they raise concerns regarding privacy, security, discrimination, misusc.
REPRODUCIBILIIY STATEMENT
We committed to ensuring the reproducibility of our results. The main paper provides detailed description of the proposed method and the experimental setup. All hyperparameters, training pro- cedures, and evaluation metrics documented in Section C.I] To further facilitate reproducibility,
we provide the source code and instructions for reproducing all experiments, which are publicly available at httts:://gtthbb com/ http://gg.hh.aaaac..eeeeeeeeee/p
REFERENCES
Olusola Oluwakemi Abayomi-Alli, Robertas Damaševičius, Michat Wieczorek, and Marcin Wožniak. Data augmentation using principal component resampling for image recognition by deep learning. In Artificial Intelligence and Soft Computing: 19th International Conference, ICAISC 2020, Zakopane, Poland, October 12-14, 14, 2020, Proceedings, Part Il 19, pp. 49-48. Springer, 2020.
Shun-Ichi Amari. Natural gradient works efficiently in learning. Neural computation, 10(2):251- 276, 1998.
Shun-ichi Amari. Information Geometry and Its Applications, volume 194 of Applied Mathematical Sciences. Springer Japan, 2016. ISBN 978-4-431-559772 978-4-431-55978-8 doi: 10.1007/
978-4-431-55978-8
Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2000.
A Antoniou. Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04848 2017.
Nihat Ay, Jürgen Jost, Höng Vân Lê, and Lorenz Schwachhöfer. Information Geometry, volume 64 of Ergebnisse Der Mathematik Und Ihrer Grenzgebiete 34. Springer International Publishing, 2017. ISBN 978-3-319-56477-77 978-3-319-564787
doi: 14007/978-3-319-5664
Lev M Bregman. The relaxation method of finding the common point of convex sets and its applica- tion to the solution problems convex programming. USSR computational mathematics and mathematical physics, 733):200-217, 1967.
Clément Chadebec, Elina Thibeau-Sutre Ninon Burgos, and Stéphanie Allassonnière. Data aug- mentation in high dimensional low sample size setting using a geometry-based variational au- toencoder. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):2879-2888 2022.
Ronald R Coifman and Stéphane Lafon. Geometric harmonies: novel tool for multiscale out-of- sample extension of empirical functions. Applied and Computational Harmonic Analysis, 21(1): 31-52, 2006.
11

---

Published as a conference paper at ICLR 2026
Paulo Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Wine Quality. UCI Machine Learning Repository, 2009. DOE: https///bb.ogg//.3333333663
Wei Dai, Chia Dai, Shuhui Qu, Juncheng Li, and Samarjit Das. Very deep convolutional neural networks for raw waveforms. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 421-425. IEEE, 2017.
Zhou Derun and Mahito Sugiyama. Optimal submanifold structure in log-linear models. In The 41st Conference on Uncertainty in Artificial Intelligence, 2025. URL Bttts::/oppnzeviie
net/Eorum22G==KCCOVVGG
Andrés Duque, Sacha Morin, Guy Wolf, and Kevin Moon. Extendable and invertible manifold learning with geometry regularized autoencoders. In 2020 IEEE International Conference Big Data (Big Data), pp. 5027-5036. IEEE, 2020.
Steven Y. Feng. Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. survey of data augmentation approaches for NLP. Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computationnal Lin- guistics: ACL-IJCNLP 2021, pp. 968-988, Online, August 2021. Association for Computational Linguisties. doi: 10.18653/v1/2011 htt.s//////22....nnnnssalalllll fin.ings-acl.84.
Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505. 13447, 2025.
Kazu Ghalamkari, Mahito Sugiyama, and Yoshinobu Kawahara. Many-body approximation for non-negative tensors. Advances in Neural Information Processing Systems, 36, 2024.
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):1 122 2018.
Henry Han, Wentian Li, Jiacun Wang, Guimin Qin, and Xianya Qin. Enhance explainability of manifold learning. Neurocomputing, 500:877-895, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Alex Hernández-Gareia and Peter König. Further advantages of data augmentation on convolu- tional neural networks. In Artificial Neural Networks and Machine Learning-ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Pro- ceedings, Part 27, pp. 95-103. Springer, 2018
Geoffrey E Hinton and Sam Roweis. Stochastic neighbor embedding. In S. Becker, S. Thrun, and K. Obermayer (eds.), Advances Neural Information Processing Systems, volume 15. MIT Press, 2002.
Taiwan Economic Journal. Taiwanese Bankruptcy Prediction. UCI Machine Learning Repository, 2020. DOL:
Diederik P Kingma and Max Welling, Auto-encoding variational bayes. In 2nd International Con- ference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Confer- ence Track Proceedings, 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images, Mas- ter's thesis, Department of Computer Science, University of Toronto, 2009.
Yann LeCun. The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998.
John M Lee. Smooth manifolds. Springer, 2012.
Adrien Marie Legendre. Mémoire sur l'intégration de quelques équations aux différences partielles. Imprimerie royale, 1787.
12

---

Published as a conference paper at ICLR 2026
Bohan Li, Yutai Hou, and Wanxiang Che. Data augmentation approaches in natural language pro- cessing: A survey, Ai Open, 3.71-90, 2022a.
Chao Li, Kelu Yao, Jin Wang, Boyu Diao, Yongjun Xu, and Quanshi Zhang. Interpretable generative adversarial networks. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2): 1280- 1288, Jun. 2022b. doi: 10.1609/aaii (0.1609/anai.v3622220015 v3622.2001.
Kiran Maharana, Surajit Mondal, and Bhushankumar Nemade. A review: Data pee-proeeeeee pre-erocessin and data augmentation techniques. Global Transitions Proceedings, 3(1):91-99, 2022.
L. McInnes, J. Healy, and J. Melville. UMAP:
Uniform Manifold Approximation and Projection for Dimension Reduction. ArXiv e-prints, February 2018.
Marina Meilä and Hanyu Zhang. Manifold learning: What, how, and why. Annual Review of Statistics and Its Application, 11, 2024.
Alhassan Mumuni and Fuseini Mumuni. Data augmentation: A comprehensive survey of modern approaches. Array, 16:100258, 2022.
Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. Specaugment: A simple data augmentation method for automatic speech recognition. Interspeech 2019, Sep 2019. doi: 10.2 40.21447//mterspeechho w0t.eeechhhh1111100 2019-2680. URL htttp://dxx doi.
org/10.214377 http//////nneeeeeccheeee 2019- 2680
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Data augmentation can improve robustness. Advances in Neural Information Processing Systems, 34:29935 299488, 2021.
Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv:1609.04747, 2016.
arXiv preprint
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206-22) 222006111,,11 2019.
Terry Sejnowski and R. Gorman. Connectionist Bench (Sonar, Mines vs. Rocks), UCI Machine Learning Repository, 1988. DOI: hutps//bb..rr///2233//5110
Rui She, Pingyi Fan, Xiao- Liu, and Xiaodong Wang. Interpretable generative adversarial networks with exponential function. IEEE Transactions on Signal Processing. 69:3854-366 2021.
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019,
Connor Shorten, Taghi M Khoshgoftaar, and Borko Furht. Text data augmentation for deep learning. Journal of big Data, 8(1):101, 2021.
Nikolay Metodiev Sirakov. Tahsin Shahnewaz, and Arie Nakhmani. Training data augmentation with data distilled by principal component analysis. Electronics, 13(2):282, 2024.
Gilbert W Stewart, On the early history of the singular value decomposition. SIAM review, 35(4): 551-566, 1993.
Mahito Sugiyama, Hiroyuki Nakahara, and Koji Tsuda. Information decomposition on structured space. In 2016 IEEE International Symposium on Information Theory (ISIT), pp. 575-579, 2016.
doi: 10. 10.1 11.11099///T:2211.55511111
Mahito Sugiyama, Hiroyuki Nakahara, and Koji Tsuda. Tensor Balancing Statistical Manifold. In Proceedings of the 34th International Conference on Machine Learning, pp. 3270-3279, PMLR, 2017.
Mahito Sugiyama, Hiroyuki Nakahara, and Koji Tsuda. Legendre decomposition for tensors. Ad-
vances in Neural Information P Processing Systems, 31, 2018.
13

---

Published as a conference paper at ICLR 2026
Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A global geometrie framework for nonlinear dimension reduction. Science, 29065500)2231--2222 2000. doi: (0.1126/scencee299. 5500.2319.
Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov. Effective data aug- mentation with diffusion models. In The Twelfth International Conference on Learning Repre- sentations, 2024. URL
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE, Journal of machine learning research, 9(11), 2008.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz. and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International conference machine learning, pp. (438-6447) PMLR, 2019.
Max Vladymyrov and Miguel Á Carreira-Perpinán, Locally linear landmarks for large-scale mani- fold learning. In Machine Learning and Knowledge Discovery in Databases: European Confer- ence, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part III 13, pp. 256-271. Springer, 2013.
Jason Wang, Luis Perez, et The effectiveness of data augmentation in image classification using deep learning. Convolutional Neural Networks Vis. Recognit, 11(0017):--8, 2017.
Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint /804.0000003200 2018.
Christopher Williams and Matthias Seeger. Using the Nyström method to speed up kernel machines. Advances in neural information processing systems, 13, 2000.
Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37-52, 1987.
Sebastien C Wong, Adam Gatt, Victor Stamatescu, and Mark D McDonnell. Understanding data augmentation for classification: when to warp? In 2016 international conference on digital image computing: techniques and applications (DICTA), pp. 1-6. IEEE, 2016.
Jianwen Xie, Yang Lu, Song-Chur Zhu, and Yingnian Wu. A theory of generative convnet. In International conference on machine learning, pp. 2635-2644, PMLR, 2016.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018
14

---

Published as a conference paper at ICLR 2026
A PROJECTION THEORY IN INFORMATION GEOMETRY
We will assume some familiarity with the basic terminologies for manifold (Leel 2012 Chapter 1, 4). In particular, in this section, explain the main concepts of information geometry used in this study, including natural parameters, expectation parameters, model flatness, and convexity of optimization. In the following, consider only discrete probability distributions for simplicity and refer to Amari (2016) for more general cases.
A.1 (0, COOORDDAEEE AND GEODESICS
Consider S as the space of discrete probability distributions, which is au--uucidean space with the Fisher information matrix G as the metric. This metric measures the distance between two points, iee. discrete probability distributions, in S. Euclidean space, the shortest path between two points is straight line, while in non-Euclidean space, such shortest path is called geodesic. In the space S, two kinds of geodesics can be introduced: e-geodesics and m-geodesics. For two points P1,P2 E S, e- and m-geodesies are defined as
(re logr: = (1 - t) log P1- + log P2 (1),0 (rr 1r (1 ()=t) tpp, 0 OSS1])
respectively, where (tt) is normalization factor to keep T2 to be a distribution.
We can parameterize distributions p E S by parameters known as natural parameters. In Sec- we have described the relationship between distribution and natural parameter vector for a discrete probability distribution over sample space of D elements in the log-linear
tion 3.2 0
E E model. The natural parameter serves coordinate system of S, since any distribution in S is specified by determining 0. Furthermore, we can also specify distribution p by its expectation parameter vector E D-1 which corresponds to expected values of the distribution and an al- ternative coordinate system of S. More explicitly, the definition of the expectation parameter 1j is defined as The Σyze p(y) for x E Ωt and 17.1 1, where p(z) is the probability mass function of p over the sample set Ω2 which is assumed to be poset. The 0-coordinates and n--corddinates are orthogonal with each other, which means that the Fisher information matrix G has the following property, Gu 0m./00,.. and (G- ",,v 00,/0m.. and m-geodesics can also be described using these parameters as follows:
{0, 0r (1 t)0 + t0 115 13. {n T3 -1-t)mp t5t
where 6º and 17" are and coordinnte of distribution p E
A.2 FLATNESS, PROJECTION, AND OPTIMIZATION
A subspace is called e-flat when any e-geodesic connecting two points in subspace is included in the subspace. The vertical descent of an m-geodesic from a point p S onto e-flat subspace B. is called mn-projection. Similarly, e-projection is obtained when we replace all with and m with e. The flatness of subspaces guarantees the uniqueness the projection destination. The projection destination P or p obtained by m- e-projection onto B. or B,, minimizes the following KL. divergence
p arg min (p. P arg min DKL(gg geB...
gEB.
where the KL divergence from discrete distributions p E S to q E S is given
DKL(p. = -E p(x) log p(x)
(3)
q(x)²
where p(x) and g(x) are the probability mass functions p and q, respectively. A subspace with some of its natural parameters fixed at 0 is e-flat (Amari 2016 Chapter 2), which is obvious from the definition of e-flatness. More generally, any subspace resulting from linear constraints on the natural parameter is e-flat. Similarly, any subspace B resulting from linear constraints the expectation parameter is m-flat. When a space e-flat and m-flat at the same time, we say that the space is dually-flat. The set of discrete probability distributions S is dually-flat.
Both e- and m flllatness guarantee that the cost functions to optimized in Eq. (3) are convex. Therefore, m- and e-projection onto e- m-flat subspace be implemented by a gradient
15

---

Published as a conference paper at ICLR 2026
method using second-order gradient. This second-order gradient method is known the natural gradient method (Amari, 1998). The Fisher information matrix G appears by second-order differ- entiation of the KL divergence. For instance, given p and an e-flat subspace B,, the optimization problem p arg min (p: q) can be efficiently solved via gradient descent with second- order derivative by the update rule 0z41
8, G (n- 17"), where G E RD is the Hessian matrix, and (DrI.(P, (P))000 = - Tfp is the derivative of the KL divergence. The updated nat- ural parameters 01+1 can then be used to construct 4++1 Be that is closer to the destination along with the e-geodesic from 9i to p. By repeating this process until convergence, we can always find the global optimal solution. A similar algorithm be implemented for the other case, i.e., p = arg mingeB... (g.(g.))
We make remarks the optimization of many-body approximation (Ghalamkari al. 2024) that we omit in Section 4.4, which a specific case of the above discussion.
Example A.1 (Many-body approximation), For Bg = -M defined Eq.
1. Convexity and uniqueness: The solution of the many-body approximation is always unique, and the objective function of the many-body approximation is convex (Ghalankari et al. 2024 The- orem 1). In particular, the many-body approximation is maximum likelihood estimation that approximates a non-negative lensor, which is regarled as an empirical distribution, by an ex- tended Boltzmann machine without hidden variables.
2. Computational complexity: The computational complexity of the many-body approximation for B. Me with B many non-fixed indexes (i.e., f-body parameters) is B(T) where T the number of iterations of the optimization. This is because the overall complexity is dominated by the update of 0, which includes matrix inversion of G, and the complexity of computing the inverse of an n x n matrix is O(n²). ). As an alternative, one can appeal to first--orrer methods such as gradient descent, which will then reduce the complexity to OT))B), where [B] corresponds to computing the gradient.
Note that this complexity can be reduced if one reshapes tensors so that the size of each mode becomes small. We explore this idea further in Section C.5
B OMITTED DETAILS FROM SECTION 4
We provide the pseudocode for the proposed algorithm in Algorithms I [nd
Algorithm 1: Backward projection
Data: representation w" E B, ep-embedded dataset {=/)== with projection [w:)² on B.
A E N
Result: Backward projected data z'*
N -earrest-Neighboo (k, w", {w, {w
D Consttruct -uurr Sub-Maaifold (
Prooj (w", D) return z"
*One consider 13,, with Eq.
being defined w.r.t. the vycoordinaaee system, and similar remarks hold.
16

---

Published as a conference paper at ICLR 2026
Algorithm 2: Pseudo-non-linear data augmentation
Data: A dataset {2 embedding yp: Qa + S, k E N, flat base sub- sub-manifold B, size m E N
Result: A generated dataset {2ī /==1 of size m
for i 1,...,M 71 do 2 p(z;)
1/ Encoding
0; Pro 12/ B)
1/ w = EEn(=)) = Projg oz(2)
for: j = ..... m do
// Augmenting
" Augment [w B))
for; j I...., do
1/ Decoding // Alggrithhm
2j Back-Proj (wj, {2/1m.. (wi k)
3j 5 (x))
11 x- =Dec(w") = Projg (w"
return {zī}
C OMITTED DETAILS FROM SECTION 5
we provide the details experimental setup omitted in Section [ and Sections to C.5 consists of additional experiments.
In Section C.1
C.3
C.I DETAILS OF EXPERIMENTAL SETUP
In this section, we provide the details of each dataset and other experimental setups with relevant explanations.
First, we summarize the details each dataset in Table and relevant parameters for applying the log- linear model on posets and Algorithm 2
Datasets.
Table 2: Summary of each dataset and the corresponding setups of Algorithm
Datasel
MNIST
CIFAR-10
Speech Commands
Connectionist
Bankrupicy
Quality
Train Size
60,000
60,000
84,848
166
5,455
5,197
Test Size
10,000
10,000
4,890
42
1,364
1,300
Augment Size
Class 10,000
10,000
7,000
32
1,090
1,036
10
10
$5
2
2
7
Feature
3,072
16,000
Y 4,000
60
11
Poset Q
X22
x2* 20
R2 xaxx
p2*x3
Base witw
Mi am)
M, d.no
Mg dao
MM m
Adg (s0)
Mg no)
Local Data
1o7
MG
Mt
M2 cm
Bandwidth
0.05
0.05
0,05
0.05
0.05
0.05 10
Neighbor
2
Note that MNIST holds CC BY-SA 3.0 license, CIFAR-10 is released with a MIT license, and Speech Commands is released with a CC BY 4.0 license. Finally, all the UCI datasets (Connectionist Bench, Taiwanese Bankruptcy, and Wine Quality) are licensed under CC-BY 4.0. We now break each group down and explain it in detail:
1. The first group consists of basic dataset statistics. The first three datasets (MNIST, CIFAR-10, and Speech Commands) come with default train/test split; for the last three UCI datasets, since there is no default train-test split, we take 80% of the whole dataset as the training set, and the remaining 20% as the test dataset. Augment Size reports the size of the augmented data, which roughly 20% of Train Size, off by some rounding errors since we assume we augment the same amount of data for each class.
In all cases, the full training set is used to train the classifier when evaluating the classification performance in Section [.3 and also to train our data augmentation baseline (i.e., autoencoder) for comparison. The only exception is that for MNIST and Speech Commands, choose equal number of samples for every class when doing our pseudo-non min-le data augmentation for implementation convenience.
17

---

Published as a conference paper at ICLR 2026
Finally, due to the extremely high dimensionality of Speech Commands (16000), down- sampled each data to 4000 dimensions in the entire experiment due to the computational con- straint: solving 16000-dimension convex programs is infeasible in terms of the memory require- ment.
2. The second group consists of the poset structure impose on each dataset when applying the log-linear model for positive tensors. Since our ultimate goal is to utilize the many-body ap- proximation (Eqs. [ and (2, by reshaping the feature vector into a high-order tensor, finer hierarchy of projection can be obtained. Hence, in all experiments, we reshape the feature w.rt. the prime-number factorization of the number of features. For instance, an MNIST image is in 20 28828 and we reshape it into a tensor of shape (7, 2, 2, 2,),), giving it -order tensor structure 20 Rxx2x2xxxxx2 For notation convenience, Poset 12, we overload IR indicate the natural poset structure introduced in Theorem 4.1 and compress the repeated prime factors in the exponent. We note that for the Wine Quality dataset, since the feature dimension is originally 11, which a prime, we artificially add dimension by padding 0's, so we get a non-trivial prime factorization.
Next, Base B and Local Data D report the corresponding construction of the base sub-manin aub-manifold and the local data sub-manifold using either Eq. (I) or Eq. w.r.t. the given poset structure of that dataset. The number the parentheses reports the corresponding dimension the constructed Lub-manifolds
Finally, Bandwidth reports the bandwidth parameter we used when fitting the kernel density model M in the generating step, and Neighbor k reports the number of nearest neighbors used in Algorithm 1
Classification Performance. In Section 5.3 evaluate the augmentation methods by training classifiers and evaluating their prediction accuracy. For MNIST and three UCI datasets, we consider the simple 2-layer MLP, while we use the ResNet-18 for the other two datasets. All models are trained by Stochastic Gradient Descent (SGD) (Ruder) (016) with learning rate 0.1, momentum 0.9, and weight decay 5 10-4 A step size scheduler utilized to reduce the learning rate by factor of 0.1 every 30 epochs until convergence.
For the two baseline data augmentation methods:
The standard image augmentation methods include random horizontal flipping and cropping. For other modalities, the standard augmentation method corresponds to adding Gaussian noise, which is sampled from ((00 where a is chosen to be 1/4 of the minimum standard deviations over each feature dimension to make sure that the noise level is reasonable.
Standard method (STD):
Autoencoder (AE): For UCI datasets, we consider a simple two-layer MIP encoder-decoder ar- chitecture, where the dimension of the hidden layer is the same as dim(B) indicated in Table 2 with a bottleneck dimension being 3. For the image datasets, both the encoder and the decoder are based on convolutional layers. The encoder uses series of convolutional layers with increasing feature map sizes to progressively downsample the input image, while the decoder mirrors this structure with transposed convolutional layers to reconstruct the image from the latent representa- tion.
In all experiments, the autoencoder is trained with the Adam optimizer with a learning 10-3 under the mean squared error until convergence.
C.2 IMPACT OF THE SIZE AUGMENTATION
In this section, we conduct additional experiments on the impact of the ratio between the augmented dataset size on the performance. Specifically, we consider the two image datasets (MNIST and CIFAR-10) and the audio dataset (Speech Commands), and vary the size of the original size of the augmented dataset, which is 20% of the original dataset size. The results are presented in Tables to
18

---

Published as a conference paper at ICLR 2026
Table 3: Impacts on the sizes of the augmentation dataset for MNIST.
AG 25%
50%
75%
100%
None
98.14%
98.08%
98.10%
98.12%
STD
92.08%
92.00%
92.02%
92.16%
AE 97.91%
97.86%
97.90%
97.97%
PNL
98.01%
98.07%
97.93%
97.55%
Table 4: Impacts on the sizes of the augmentation dataset for CIFAR-IB 10.
AG
25%
50%
75%
100%
None
89.06%
89.32%
88.63%
88.83%
STD
90.42%
92.00%
90.60%
90.92%
AE 88.56%
88.76%
88.42%
86.49%
PNL
88. 78%
88.32%
88.53%
88.67%
Table 5: Impacts the sizes of the augmentation dataset for Speech Commands.
AG
25%
50%
75%
100%
None
83.65 33.68%
83.26%
84.72%
84.16%
STD
84.34%
83.84%
84.98%
84.25%
AE 82.30%
83.49%
84.72%
85.21%
PNL
84.20%
84.61%
84.56%
84.43%
C.3 SENSITIVITY AND ROBUSTNESS
We examine our proposed method's robustness and sensitivity of the bandwidth used when fitting the kernel density model, and also the number le of the nearest neighbors used in Algorithm [ For an easier visual inspection, we use MNIST in this section.
Bandwidth of Kernel Density Estimation Model. Consider varying the bandwidth we use when fitting the kernel density model, ranging among (0.01, 0.05, 0. 0.2, 0.5). The results are shown in Figure We observe that Algorithm is robust under different bandwidths when working with the kernel density estimation model in the generating step.
19

---

Published as a conference paper at ICLR 2026
0
2
3
4
5
6
7
9
(a) Result with bandwidth 0.01.
0
/
2
3
4
5
6
7
(b) Result with bandwidth 0.05.
0
2
3
4
5
6
7
8
9
(c) Result with bandwidth 0.1.
0
/
2
3
5
6
7
Result with bandwidth 0.2.
0
1
2.
3
(c) Result with bandwidth 0.5.
Figure 8: Augmented data via Algorithm 2
with different kernel density estimation bandwidths.
Number of Nearest Neighbors. Next, we consider ranging i among (1, 4, 16). The results are shown in Figure 9 Observe that when k is small, e.g., 1, the result of Algorithm since the local sub-manifold D in Algorithm iis defined using only the nearest neighbor. When goes up, non-trivial augmentation emerges, robust across different k's.
2 tends to overfit
0
2
3
4
S
6
7
8
S
0
2
3
4
S
6
7
8
S
(a) Result with 1: = 1.
0
2
3
4
5
6
7
8
9
0
2
3
4
5
6
7
8
9
(b) Result with 4.
0
1
2
3
4
5 6
7
8
9
0
2
3
4
5
7
7
8
9
(c) Result with 8
0
2
3
4
5
6
7
8
9
0
2
3
4
5
7
7
8
9
(d) Result with & 16.
Figure 9: (Top) Augmented data via Algorithm with different k's for Algorithm | closest training data.
(Bottom) The
20

---

Published as a conference paper at ICLR 2026
C.4 NECESSITY OF DIMENSION REDUCTION
We demonstrate that dimension reduction, a key building block of our proposed method based on the intuition we have from autoencoder-like models, necessary for Algorithm 2 to work. For easier visual inspection, we use MNIST in this section.
data augmentation methods fall short of high- dimensional data due to the sparsity of the data. Figure [0 shows the results of directly fitting kernel density estimation model on MNIST with 1000 samples.
Direct Fitting. Naive perturbation-
0
f
2
3
4
5
6
7
8
9
0
f
2
3
5
6
8
9
Figure 10: (Top) Augmented data via directly fitting kernel density estimation model with band- width 30. (Bottom) The closest training data.
Observe that even with a large bandwidth (30) introduce variability, we only see a meaningless noisy perturbation on one of the exact training samples, indicating overfitting.
Local Data Sub-manifold. A potential problem related to the necessity of dimension reduction is that, if D captures too much local information about the data (i.c., with low dimension), backward projecting a random latent representation w" E might already suffice to augment the data non-trivial way, without the need for knowing the latent representations of the training dataset. To this end, consider sampling uniformly random latent representations within the empirical range we observed from the latent representations of the training data and perform Algorithm [ are shown in Figure
The results
2
3
4
5
8
9
1
2
3
4
5
4
8
7
(a) Result with h: = 1.
O
2
3
4
6
8
(b) Result with 4.
0
2
3
O
/
2
3
4
5
6
8
9
(c) Result with 8.
0
/
2
3
4 5
6
8
9
(d) Result with & 16.
Figure 1I: (Top) Augmented data on random latent representations via Algorithm 2 with different k's for Algorithm T (Bottom) The closest training data.
21

---

Published as a conference paper at ICLR 2026
For k = 1, Figure [I(a) shows that, similar to Figure ((a) it is possible overfit one of the training data (i.e.. the nearest neighbor of the randomly sampled latent representation). This is not surprising since the base sub-manifold is only of dimension 17, the random latent representation sufficiently close to one of the representations the training data B, their backward projection result should not deviate too much. Furthermore, we observe the fading effect, which intuitively corresponds to misspecification of the energy, indicating that the sampled latent representation is fundamentally different from that of the dataset.
As A: increases, the benefit of getting informative and meaningful latent representations from the original dataset becomes clear. Specifically, we start to see degeneration: from unclear overlappings to collapsing (i.e., only a few pixels are showing). Intuitively speaking, it is because the random latent representation's s nearest neighhors appear to significantly different, hence failing to provide a consistent local data sub-manifold. For instance, in the extreme case when k = 16, the local data sub-mane uub-manifold is completely not informative, resulting in collapsing. Overall, without dimension reduction, we will lose the reference of realistic latent representations provided by the original dataset, which leads to bad performance once we are beyond the trivial overfitting regime.
C.5 CHOICES OF TENSOR STRUCTURE AND CONSTRUCTION OF SUB-MANIFOLDS
In Section 5.2
we consider varying é for Me with the tensor structure IR 20
xxx2xxxTx2X2 on
the MNIST dataset. In this section, we further vary the tensor structure well: in particular, consider the tensor structure of the MNIST image being 20
28228 7xxxxx and IR 20 20
xxxXxxxxxxx
Remark C.1. For notation convenience, we also write their corresponding poset structures as 228228 TTxXxXX1 and 10 Rxx22xxxx2x2, and further write the many-body approximation sub- 20
manifold (Eq. (1) as Me(S2) and its dual (Eq. 2 as (M for particular poset to empha- size the dependency.
Finally, we consider ranging & from 1 to at most 4 where we neglect the degenerate case: for instance, in the case of 288x25 .M2 25x28 = S there are only two modes for a matrix, therefore degen- erates to direct fitting which is not of interest (see Section C.4 Note that throughout this section, we fix the default local data sub-manifold to be D = M Xxxxxxxxx2x2 for consistency.
The results for the finest structure, R 7xxxxxxxx2xx are shown in Figure 12 As l grows, the forward 20 projection results (Top) preserve the structure of the data better, subsequently improving the quality of the augmented data (Bottom). Similar trends can be found in the case of R 7x4x7x4 as shown in
20
Figure I3
If we look at the results when using the original matrix structure 2828x2 (Figure 14 some interest-
ing comparisons can be made. Firstly, we compare the augmentation results for MR 28x28 (Figure 14 (Bottom)) with the finer structures counterparts, e.g., (Figure 12(a) (Bottom)) for M1 (IR. x xx2xxxxx2) one can observe that the results are worse. However, the former requires
more dimension (dimn(M, (IR 26 28x2*) = 55 17 dimmmM, (IXxXx2xxXxxxx22 for the base sub-
manifold. Similarly, the augmentation results with B MR (Figure 13(a) (Bottom)) also achieve better performance with lower base sub-manifold dimension.
22

---

Published as a conference paper at ICLR 2026
3
4
5
6
7
9
(a) Result with base mub-manifold Mi (R 20
[xxxxxxxxx
(dim( 17).
1
3
4
5
6
8
9
(b) Result with base sub-manifold B
M2 (2
xxxxxxxxxx))) (dim = 107).
20
2
3
4
5
7
3
4
5
6
7
8
9
(c) Result with base sub-manifold B = MM
Tx xaxxx 7x2x2)
(dim (B)
327).
2
3
4
S
6
7
8
/
2
3
4
S
6
7
8
9
(d) Result with base sub-manifold B Ma (1 xxxxxxxxxxxxx (dim(B) 592).
Figure 12: (Top) Forward projection on M (R (RXx2x2x7x2xx3
(Bottom) Backward projection
on D = (R 2x2xxxxxxxx
3
4
5
6
7
(a) Result with base sub-manifold Mı (R 7x4x7x4
(dim(B) 19).
/ 2
3 4
5
6
7
8
(b) Result with base sub-manifold B = M2 7x47x4 (dim(B) = 136).
Figure 13: (Top) Forward projection on B Mr ( (Bottom) Backward projection on
D (R (xXxxxxxxxxx
2
3
4
6
7
8
9
Figure 14: (Top) Forward projection on
M, 28x28 20
(Bottom) Backward projection on
D = M (RTxxxxxxxx2
23


Reviews:
[{"review_id": "tJaDxZbmk6", "reviewer": "Reviewer_dmdm", "summary": "This paper proposes a learning-free and geometry-aware data augmentation framework termed Pseudo-Non-Linear Data Augmentation (PNL-DA). The key idea is to view data augmentation as an energy minimization problem on a latent manifold derived from the intrinsic structure of the dataset.\n\nInstead of using neural generators or search-based augmentation (e.g., AutoAugment), the authors construct an information-geometric latent space and perform constrained energy descent to generate perturbed samples that respect the data manifold while introducing non-linear diversity. This results in augmented data that remain semantically valid yet expand the effective data support.\n\nThe paper claims that PNL-DA:\n1. requires no additional network training,\n2. offers fine-grained controllability over augmentation strength via explicit constraints, and\n3. achieves competitive or superior downstream performance across standard image classification benchmarks.", "questions": "1.The paper frames data augmentation as energy minimization on a latent manifold. Could you clarify the exact connection between your “energy” function and the traditional risk minimization objective? Is the energy equivalent to a potential function, or is it empirically constructed?\n\n2. You define “pseudo-non-linear” transformations. In what precise sense are they pseudo non-linear? Are these transformations non-linear in the data space but linear in latent space, or vice versa?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper reframes data augmentation as an energy minimization process under latent-space constraints. This is a refreshing deviation from purely heuristic or search-based augmentations. The geometric viewpoint is conceptually elegant and could inspire follow-up work linking augmentation and manifold regularization.\n\n2. In contrast to most augmentation strategies that involve training auxiliary networks or policy search, this method is deterministic and learning-free once the latent geometry is built. This makes it lightweight and potentially appealing for resource-constrained environments.\n\n3. The energy constraint allows explicit control over perturbation magnitude, which is a desirable property compared to black-box stochastic augmentation policies. The paper demonstrates that tuning this constraint can balance sample diversity and fidelity.\n\n4. Experiments on several vision datasets (CIFAR-10/100, Tiny-ImageNet, SVHN, and others) show that PNL-DA matches or slightly surpasses traditional augmentations and performs competitively against AutoAugment and RandAugment, while using less computational cost.\n\n5. The exposition of the method—particularly the geometric intuition and constrained optimization formulation—is well organized and mathematically sound. The visualizations of latent-space perturbations help the reader grasp the intuition.", "weaknesses": "1. The energy constraint is intuitive, but there is no formal proof that minimizing it preserves label semantics or improves generalization bounds. The paper could benefit from a more rigorous analysis linking the energy formulation to risk minimization.\n\n2. The method introduces a constraint coefficient controlling perturbation scale. The results indicate non-trivial sensitivity, but the paper lacks systematic tuning guidelines or ablation to quantify robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zEd1m3l9Da", "reviewer": "Reviewer_W952", "summary": "This paper proposes a projection-based data augmentation framework that maps data onto a statistical manifold and performs controlled perturbations through forward and backward projection. The main novelty focuses on achieving learning-free, efficient, and controllable augmentation with semantics preserved, by leveraging the geometric structure induced by posets. Experiments on image and speech datasets validate that the method can maintain core structural features while introducing meaningful variation.", "questions": "The backward projection relies on nearest neighbors in the latent space to reconstruct data, which means the augmented samples often stay very close to existing examples. As the dataset grows, will this approach become less scalable and may fail to produce diverse new samples?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method is learning-free and inverse-consistent. In addition, it is controllable with explicit selection of structural attributes.", "weaknesses": "1. Performance gains over simple AE-based augmentation are modest, especially in CIFAR, where improvements are marginal.\n2. Efficiency is claimed relative to diffusion models, but no direct comparison or runtime analysis is provided.\n3. As discussed in the paper, the method requires data to admit a meaningful poset structure, which cannot directly handle permutation-invariant data such as point clouds or general graphs.\n4. The experiments are primarily on images, as argued in line 88, with more different modalities, like graph, video, might be more interesting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HWB1EGsdqq", "reviewer": "Reviewer_ZQPF", "summary": "This paper proposes a new learning-free data augmentation method that is based on energy-based modeling. The authors first assume a poset structure of the data. Using this poset structure, they map the data to a log linear model on the poset. They then associate dually-flat coordinates with the log linear model. This constitutes the embedding of the data which is then used to generate data augmentations as follows. Given a sub-manifold, the authors project the embedding and then backward project using some heuristic assumptions on the structure of the data. Overall, this results in a task-independent data augmentation method.", "questions": "1. Can $z_i$ in principle contain data labeling? As far as I understand, $z_i$ is just data without labels. This makes the data augmentation procedure task-agnostic. This results in a method that cannot incorporate any biases that are beneficial for the downstream task except the choice of poset structure. Can you explain how the poset structure introduces such biases in the case of images?\n\n2. Why do you think the probability distribution that comes after applying $\\phi$ is a good representation of the data? This should be dependent on the properties of data. For example, could you explain why it is a good representation for images? What is added by the dually flat coordinates? Why would the data be in a linear subspace of this manifold?\n\n3. You cite generative-model-based data augmentation suffers from two challenges: efficiency and controllability. Is this the case for MNIST and CIFAR-10? I believe the challenges apply to harder datasets and I don't see why the proposed method scales to high dimensions.\n\nThe following paper is very relevant related work as it has an encoding/decoding scheme over a manifold that is learned.\n\n[1] Yüksel, Oguz Kaan, et al. \"Semantic perturbations with normalizing flows for improved generalization.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is presented clearly. The authors manage to put interesting and novel ideas together. This is an interesting addition to the landscape of data augmentation. The fact that the method is learning-free is very appealing.", "weaknesses": "1. The poset structure does not fit many data domains well. This is acknowledged by the authors. I don't understand why they suggest that images have an inherent partial order. Directed graphs and time series might conform to such partial orderings.\n\n2. It is unclear what the log linear model captures about the data. The authors mention energy-based geometric modeling but more discussion is needed. It seems that there is first a normalization is done (Example 4.1.) and then this is interpreted as a probability distribution. It is unclear what the meaning of this probability distribution is for many data domains.\n\n3. Apart from the total energy in the features, $\\phi(z_i)$ preserves all the information about the data. Similarly, $\\theta$ coordinates should then do the same. It is unclear why the geometry here is better for data augmentation than the direct data space.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iZMNK6XKel", "reviewer": "Reviewer_gGYH", "summary": "This paper proposes a learning-free, geometry-aware framework for data augmentation based on energy-based modeling and information geometry. Unlike deep generative models that rely on trained latent representations, the proposed method constructs a statistical manifold via log-linear models on posets and performs data augmentation through forward and backward projections on low-dimensional submanifolds. The approach, termed Pseudo-Nonlinear Data Augmentation (PNL), combines the linearity of projections with nonlinear geometric curvature, offering both computational efficiency and controllability. Empirical results on images (MNIST, CIFAR-10), speech, and tabular datasets show that PNL performs comparably or better than autoencoder-based and standard augmentation methods, while being faster and interpretable.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a geometric viewpoint on data augmentation grounded in energy minimization and information geometry, distinct from the dominant deep learning paradigms. The formulation via dually-flat manifolds and log-linear models on posets is both elegant and principled.\n\n2. The method requires no model training, avoiding the heavy computational cost and data dependence of generative augmentation. Projections are formulated as convex programs solvable with first-order methods.\n\n3. Augmentation behavior can be explicitly controlled by the choice of submanifold and the poset structure, enabling fine-grained manipulation of data geometry.\n\n4. The method demonstrates applicability to image, speech, and tabular data without domain-specific modifications.\nShows robustness and consistency, especially on small datasets with high variance.", "weaknesses": "1. The baselines are somewhat minimal (autoencoder and standard augmentation).\nComparison to modern generative augmentations (e.g., diffusion-based, mixup, manifold mixup) would strengthen the claims.\n\n2. The method requires defining a partial order (poset) over features, which might be nontrivial or restrictive for unstructured or permutation-invariant data.\n\n3. While the method is theoretically interpretable, more concrete visualization or case studies of controllable augmentation (beyond MNIST/CIFAR) would enhance clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)