BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

## Abstract

Abstract Neural network compression techniques typically require expensive fine-tuning or search procedures, rendering them impractical on commodity hardware. Inspired by recent LLM compression research, we present a general activation-aware factorization framework that can be applied to a broad range of layers. Moreover, we introduce a scalable budgeted rank allocator that allows flexible control over compression targets (e.g., retaining 50% of parameters) with no overhead. Together, these components form BALF, an efficient pipeline for compressing models without fine-tuning. We demonstrate its effectiveness across multiple scales and architectures, from ResNet-20 on CIFAR-10 to ResNeXt-101 and vision transformers on ImageNet, and show that it achieves excellent results in the fine-tuning-free regime. For instance, BALF reduces FLOPs on ResNeXt-101 by 45% with only a 1-percentage-point top-1 accuracy drop.

## 1 Introduction

Deep learning models achieve cutting-edge results in various domains (LeCun et al., 2015), yet their computational and memory demands limit deployment feasibility, motivating the need for model compression methods. Among these, factorization is a strong candidate. Traditional techniques use singular value decomposition (SVD)-based methods to minimize the discrepancy between original and compressed parameters (Jaderberg et al., 2014; Hua et al., 2023). Nonetheless, such techniques often require fine-tuning and/or costly search procedures. For large networks, this is impractical on standard hardware.

Recently, the rise in popularity of large language models (LLMs) has led to the exploration of activation-aware factorization techniques, which seek to lessen the (expected) output distortion at each layer (Wang et al., 2025c; b; Chen et al., 2021), often reducing the need for fine-tuning. These techniques predominantly focus on fully connected layers, which are central to attention-based LLMs (Vaswani et al., 2017).

Here, we propose a unified framework that builds on Wang et al. (2025c) and is compatible with a broad class of layers. Moreover, we introduce a scalable rank allocator that determines how much to compress each layer based on a Lagrangian relaxation. Together, these form BALF, a complete and general fine-tuning-free compression pipeline. Our main contributions are:

- •
Unified framework. We extend activation-aware low-rank factorization (Wang et al., 2025c; b) to a broad class of layers, including (possibly grouped) convolutional layers.
- •
Budget-aware allocation. We introduce a zero-overhead rank allocator that meets user-specified FLOPs or parameter count budgets.
- •
Practicality. BALF runs end-to-end efficiently on commodity hardware (e.g., on an RTX 2070 laptop, it takes less than four minutes to compress most models we tested), and avoids expensive hyperparameter sweeps.
- •
Extensive evaluation. Across vision models from ResNet-20 (He et al., 2016) on CIFAR-10 (Krizhevsky, 2009) to ResNeXt-101 (Xie et al., 2017) and ViT-B/16 (Dosovitskiy et al., 2021) on ImageNet (Deng et al., 2009), our method achieves strong accuracy-compression trade-offs in the fine-tuning-free regime.

## 2 Related Work

Deep learning model compression has been extensively studied. The principal categories of compression techniques include quantization, which employs low-bit storage and operations (Jacob et al., 2018; Frantar et al., 2023; Gholami et al., 2021); pruning, which involves the elimination of parameters or parameter groups from the model (Hoefler et al., 2021; Frantar & Alistarh, 2022); and factorization, which divides model layers into smaller sub-layers to lower overall memory usage and computational demands (Jaderberg et al., 2014; Ben Noach & Goldberg, 2020). This paper focuses on the factorization approach.

Various factorization methods have been extensively studied over time. Among these, the SVD (Golub & Van Loan, 2013) stands out as a straightforward approach. It is directly applicable to fully connected layers, such as those found in transformer architectures, consisting mainly of matrix multiplications.

In convolutional neural networks (CNNs), factorization is usually performed by reshaping parameter tensors into matrices and subsequently decomposing them (Yang et al., 2020), with an alternative research direction employing other kinds of decompositions (Denton et al., 2014; Yin et al., 2021; Phan et al., 2020; Liebenwein et al., 2021; Lebedev et al., 2014).

Existing factorization approaches often require fine-tuning and/or compute-intensive search (Idelbayev & Carreira-Perpiñán, 2020; Yu & Bouganis, 2023). The impossibility of running these on common hardware motivates the need for lighter methods. For example, pruning methods that explicitly aim to be efficient in this regard include Narshana et al. (2023); Murti et al. (2023); Chen et al. (2024); Zhang et al. (2024); Wang et al. (2025a), whereas fine-tuning-free factorization methods are less common—for instance, Yu & Bouganis (2023). Liebenwein et al. (2021); Idelbayev & Carreira-Perpiñán (2020) also report some results in the fine-tuning-free regime.

Recent factorization research, motivated by the computational demands of LLMs, leverages intermediate model features to identify additional redundancies suitable for decomposition (Chen et al., 2021; Wang et al., 2025c; b; Qinsi et al., 2025; Yuan et al., 2025). These methods use a calibration dataset to characterize the distribution of feature tensors and then use that information to guide parameter decompositions. However, they primarily focus on decomposing fully connected layers in the language domain, often overlooking other settings.

In contrast, BALF is able to perform activation-aware compression on a general class of layers in a principled way, including convolutional layers. Together with our efficient rank allocation strategy, we obtain excellent performance in the no-fine-tuning regime in minutes without requiring expensive hardware.

## 3 Preliminaries and Notation

##### The singular value decomposition.

Given a matrix ${\bm{A}}\in\mathbb{R}^{M\times N}$, we denote its SVD by ${\bm{A}}={\bm{U}}{\bm{\Sigma}}{\bm{V}}^{T}$, where ${\bm{U}}\in\mathbb{R}^{M\times M}$ and ${\bm{V}}\in\mathbb{R}^{N\times N}$ are orthogonal matrices, and ${\bm{\Sigma}}=\operatorname{diag}(\bm{\sigma};(M,N))$ is a diagonal matrix with its singular values. Another concept of interest is the $P$-truncated SVD, which is defined by taking only the $P$ largest singular values and their associated singular vectors. In particular, we define $\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{A}}\bigr)={\bm{U}}_{:,:P}{\bm{\Sigma}}_{:P,:P}({\bm{V}}_{:,:P})^{T}$. It can be thought of as an operator that projects a matrix to its rank-$P$ approximation via SVD.

##### Uncentered whitening.

Data whitening (Kessy et al., 2018) (also termed data sphering) is a widely used technique in machine learning and statistics. Here, we are interested in a special case called uncentered whitening. This process transforms data so that its second-moment matrix becomes the identity matrix. Formally, we define an uncentered whitening matrix (UWM) ${\bm{M}}\in\mathbb{R}^{I\times I}$ for the data ${\bm{X}}\in\mathbb{R}^{N\times I}$ as a matrix such that ${\bm{M}}^{T}{\bm{X}}^{T}{\bm{X}}{\bm{M}}=N\overline{{\bm{I}}}_{R}$, where $\overline{{\bm{I}}}_{R}\in\mathbb{R}^{I\times I}$ is the identity on the first $R$ coordinates and zero elsewhere, and $R$ is the rank of ${\bm{X}}$. We further restrict the definition by requiring that $\operatorname{row}({\bm{X}})=\operatorname{range}({\bm{M}})$, which in turn forces the last $I-R$ columns of ${\bm{M}}$ to be zero and makes ${\bm{X}}={\bm{X}}{\bm{M}}{\bm{M}}^{+}$; this assumption and these consequences will be used in later results.

##### Other notation and conventions.

We refer to general tensors as ${\bm{\mathsfit{X}}}$. When some tensor is known to be a matrix or a vector, we may use ${\bm{X}}$ and ${\bm{x}}$, respectively. For matrices, we denote the Moore-Penrose inverse (also termed the pseudoinverse) of ${\bm{M}}$ by ${\bm{M}}^{+}$, and the square root of the pseudoinverse by ${\bm{M}}^{+1/2}$.

Deep learning frameworks, such as PyTorch (Ansel et al., 2024), usually include “reshape” and “permute” operators, denoted in this work by $\mathsf{reshape}\left({\bm{\mathsfit{X}}};(s_{1},\dots,s_{n})\right)$ and $\mathsf{permute}\left({\bm{\mathsfit{X}}};(\pi_{1},\dots,\pi_{n})\right)$, respectively.

When talking about network layers, for simplicity of notation, unless specified, we drop the layer identifiers and denote the computation carried out by a layer by $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})$, where ${\bm{\mathsfit{X}}}$ denotes the layer’s input and ${\bm{\mathsfit{W}}}$ the parameters specific to that layer. In the case of the first layer, ${\bm{\mathsfit{X}}}$ denotes the input to the network (e.g., a multi-channel 2D signal). In general, when discussing factorization techniques, we ignore bias terms because they are unaffected by factorization; only the primary operations (e.g., convolutions or matrix multiplications) are modified.

Throughout this work, $\|\cdot\|_{F}$ refers to the tensor Frobenius norm, i.e., $\|\cdot\|_{F}=\|\mathsf{flatten}(\cdot)\|_{2}$.

We denote the identity function by $\mathrm{id}$. Moreover, when discussing convolutional layers, $B$ will denote the batch size; $H_{x},W_{x},C_{x}$ the height, width, and channels of the input (when $x=i$) and output (when $x=o$); and $H_{k},W_{k}$ the spatial size of the kernel. In this context, we also refer to $G$ as the number of groups in the convolutional layer (Xie et al., 2017).

## 4 Our Framework

Before diving into the factorization scheme, we first restrict our discussion to a particular class of layers that can be expressed as a combination of matrix multiplications and auxiliary functions.

###### Definition 1 .

A layer $f$ is $(\overline{O},\overline{I},\overline{P})$-expressible if it can be expressed as $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})=\overline{O}(\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}}))$, where $\overline{O}$ and $\overline{P}$ are compositions of reshape and permute operators, and $\overline{I}$ is a linear operator. $\overline{O}$ takes a batch of matrices in the form of a third-order tensor and outputs a tensor (possibly a matrix), whereas $\overline{P}$ and $\overline{I}$ both take a tensor (possibly a matrix) and output a batch of matrices in the form of a third-order tensor of compatible shapes. In particular, $\overline{O}:\mathbb{R}^{G\times N\times O}\mapsto\cdot$, $\overline{I}:\cdot\mapsto\mathbb{R}^{G\times N\times I}$, and $\overline{P}:\cdot\mapsto\mathbb{R}^{G\times I\times O}$.

$G$ stands for the number of groups. In the special case where $G=1$, we may simply refer to matrices instead of batches containing a single matrix. Note that in the preceding definition, matrix multiplication is batched, i.e., $G$ independent matrix multiplications are carried out. The necessity of batch semantics in the group dimension will become apparent later. This condition is satisfied by popular layers. In what follows, layers are assumed to operate in batched mode, with $B$ denoting the batch size. We refer to $N$ as the outer dimension.

###### Example 1 .

Fully connected layers are $(\mathrm{id},\mathrm{id},\mathrm{id})$-expressible.

In some architectures (e.g., transformers), linear layers receive inputs with multiple leading independent dimensions—for example, a tensor of shape $(B,L,D)$. In those cases, we flatten the leading dimensions, reshaping the input to $(BL,D)$ and restoring the original shape afterward. We now turn our attention to convolutional layers. Although an ungrouped convolution is a special case of a (possibly grouped) convolution, we present it separately for clarity.

###### Example 2 .

Ungrouped convolutional layers are $(\overline{O},\overline{I},\overline{P})$-expressible, with

$$ $\displaystyle\overline{O}({\bm{\mathsfit{Y}}})$ $\displaystyle=\mathsf{permute}(\mathsf{reshape}({\bm{\mathsfit{Y}}};(B,H_{o},W_{o},C_{o}));(0,3,1,2)),$ $\displaystyle\overline{I}({\bm{\mathsfit{X}}})$ $\displaystyle=\mathsf{reshape}(\mathsf{im2col}({\bm{\mathsfit{X}}});(BH_{o}W_{o},C_{i}H_{k}W_{k})),$ $\displaystyle\overline{P}({\bm{\mathsfit{W}}})$ $\displaystyle=\mathsf{reshape}({\bm{\mathsfit{W}}};(C_{o},C_{i}H_{k}W_{k}))^{T},$ $$

where $\mathsf{im2col}:\mathbb{R}^{B\times C_{i}\times H_{i}\times W_{i}}\to\mathbb{R}^{B\times H_{o}W_{o}\times C_{i}H_{k}W_{k}}$ maps a batch of 2D signals with $C_{i}$ channels into a matrix (see Chellapilla et al. (2006)). The PyTorch equivalent is called $\mathsf{unfold}$(^1^11[https://docs.pytorch.org/docs/stable/generated/torch.nn.Unfold.html](https://docs.pytorch.org/docs/stable/generated/torch.nn.Unfold.html)). It encapsulates the convolution hyperparameters—kernel size, stride, padding, and dilation—but not the number of groups. For brevity, we omit the encapsulated hyperparameters from the notation.

On the other hand, a grouped convolution can be viewed as $G$ separate, parallel convolutions. Each one takes a disjoint slice of $C_{i}/G$ input channels and produces $C_{o}/G$ output channels. Concatenating the $G$ outputs recovers the full $C_{o}$ channels (Krizhevsky et al., 2012; Xie et al., 2017). This also fits our framework.

###### Example 3 .

Grouped convolutional layers are $(\overline{O},\overline{I},\overline{P})$-expressible, with

$$ $\displaystyle\overline{O}({\bm{\mathsfit{Y}}})$ $\displaystyle=\mathsf{permute}(\mathsf{reshape}(\mathsf{permute}({\bm{\mathsfit{Y}}};(0,2,1));(C_{o},B,H_{o},W_{o}));(1,0,2,3)),$ $\displaystyle\overline{I}({\bm{\mathsfit{X}}})$ $\displaystyle=\mathsf{permute}(\mathsf{reshape}(\mathsf{im2col}({\bm{\mathsfit{X}}});(BH_{o}W_{o},G,(C_{i}/G)H_{k}W_{k}));(1,0,2)),$ $\displaystyle\overline{P}({\bm{\mathsfit{W}}})$ $\displaystyle=\mathsf{permute}(\mathsf{reshape}({\bm{\mathsfit{W}}};\,(G,C_{o}/G,(C_{i}/G)H_{k}W_{k}));(0,2,1)).$ $$

Additionally, [Example 2](https://arxiv.org/html/2509.25136v2#Thmexample2) and [Example 3](https://arxiv.org/html/2509.25136v2#Thmexample3) have trivial extensions to higher-dimensional convolutions, achieved by replacing $\mathsf{im2col}$ with its higher-dimensional counterparts.

Low-rank factorization applies naturally to linear layers by decomposing their weight matrices. The main idea of our framework is to extend the notion of factorization to $(\overline{O},\overline{I},\overline{P})$-expressible layers by exploiting their equivalence to (possibly batched) matrix multiplications.

Let us focus on linear layers for a moment. Recall that a low-rank matrix (or its low-rank projection) can be expressed as a combination of a tall and a wide matrix. $\mathcal{T}_{P}\bigl({\bm{W}}\bigr)$ is defined to be a generic operator that returns a low-rank projection of ${\bm{W}}$. If we fix $P$ and approximate ${\bm{X}}{\bm{W}}\approx{\bm{X}}\mathcal{T}_{P}\bigl({\bm{W}}\bigr)=({\bm{X}}{\bm{W}}_{0}){\bm{W}}_{1}$, with ${\bm{W}}\in\mathbb{R}^{I\times O},{\bm{W}}_{0}\in\mathbb{R}^{I\times P},{\bm{W}}_{1}\in\mathbb{R}^{P\times O}$, for sufficiently small $P$, the storage and compute requirements will be less than those of the original layer. By using their $(\overline{O},\overline{I},\overline{P})$ representation, we can extend this to, for instance, convolutional layers. It can be verified that ($\overline{P}{}^{-1}$ denotes the inverse) ${\bm{\mathsfit{X}}}*{\bm{\mathsfit{W}}}\approx{\bm{\mathsfit{X}}}*\overline{P}{}^{-1}\left(\mathcal{T}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\right)=({\bm{\mathsfit{X}}}*{\bm{\mathsfit{W}}}_{0})*{\bm{\mathsfit{W}}}_{1}$ for some ${\bm{\mathsfit{W}}}_{0}\in\mathbb{R}^{P\times C_{i}\times H_{k}\times W_{k}},{\bm{\mathsfit{W}}}_{1}\in\mathbb{R}^{C_{o}\times P\times 1\times 1}$, obtained from decomposing $\overline{P}({\bm{\mathsfit{W}}})\in\mathbb{R}^{C_{o}\times C_{i}H_{k}W_{k}}$. Again, with sufficiently small $P$, this results in storage and compute savings.

###### Definition 1 .

###### Example 1 .

###### Example 2 .

###### Example 3 .

##### SVD-based factorization.

SVD-based factorization is just a specific way of obtaining two low-rank factors from a matrix. In particular, as per the Eckart–Young–Mirsky (EYM) theorem (Golub & Van Loan, 2013), it is the optimal factorization scheme in terms of parameter distortion, defined as

$$ $\ell^{\mathrm{param}}({\bm{\mathsfit{W}}},P)=\|{\bm{\mathsfit{W}}}-\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)\|^{2}_{F}.$ (1) $$

In order to simplify the notation, we define $\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)\equiv\overline{P}{}^{-1}\left(\mathcal{T}^{\mathrm{SVD}}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\right)$. In other words, we extend the matrix low-rank projection notion to arbitrary tensors that represent the weights of $(\overline{O},\overline{I},\overline{P})$-expressible layers. In the case where $G>1$, we decompose each group $\overline{P}({\bm{\mathsfit{W}}})_{g}$ separately, with the constraint that the per-group rank is the same across groups. This latter constraint has practical reasons, which are discussed in [Section A.3.1](https://arxiv.org/html/2509.25136v2#A1.SS3.SSS1).

##### General low-rank projections.

Note that there are many ways one can obtain low-rank factors from a matrix (or a tensor, with our defined extension). A general class of operators can be defined (for a group index $g$) as

$$ $\mathcal{T}^{D}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})_{g}\bigr)=\operatorname{arg}\,\underset{\operatorname{rank}(\overline{P}(\widehat{{\bm{\mathsfit{W}}}})_{g})\leq P}{\min}\;D(\overline{P}({\bm{\mathsfit{W}}})_{g},\overline{P}(\widehat{{\bm{\mathsfit{W}}}})_{g}),$ (2) $$

where $D$ is some penalty function, and $\widehat{{\bm{\mathsfit{W}}}}$ is the approximate tensor. Traditional SVD-based decomposition corresponds to the case where $D$ is the Frobenius norm of the (reshaped) parameters, i.e., it penalizes distance in parameter space (see [Equation 1](https://arxiv.org/html/2509.25136v2#S4.E1)). We use $\mathcal{T}^{D}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)$ to also denote batched computation, where each group’s matrix is decomposed individually. As before, we omit $\overline{P}$ to simplify notation.

### 4.1 Activation-Aware Low-Rank Projections

##### Motivation.

A more natural proxy for task performance is the (expected) distortion in each layer’s outputs. In general, though, we have a calibration dataset (which can be the entire training dataset, a subset of it, etc., and is assumed to consist of i.i.d. samples), used to gather layer activations $({\bm{\mathsfit{X}}}^{(i)})_{i=1}^{B}$, which we then use to estimate the preceding expectation. The (empirical) average activation distortion for a layer $f$ is defined as

$$ $\ell^{\mathrm{activ}}({\bm{\mathsfit{W}}},P)=\frac{1}{B}\sum_{i=1}^{B}\|f({\bm{\mathsfit{X}}}^{(i)};{\bm{\mathsfit{W}}})-f({\bm{\mathsfit{X}}}^{(i)};\mathcal{T}_{P}\bigl({\bm{\mathsfit{W}}}\bigr))\|^{2}_{F}=\frac{1}{B}\|f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})-f({\bm{\mathsfit{X}}};\mathcal{T}_{P}\bigl({\bm{\mathsfit{W}}}\bigr))\|^{2}_{F}.$ (3) $$

Note that the last equality holds when forming a “batch” with the list $({\bm{\mathsfit{X}}}^{(i)})_{i=1}^{B}$. In what follows, our analysis is performed in terms of the intermediate activations ${\bm{\mathsfit{X}}}$ obtained with the calibration dataset, with the expectation that the results carry over to the test dataset. We now aim to find a low-rank projection operator that minimizes [Equation 3](https://arxiv.org/html/2509.25136v2#S4.E3).

##### Which projection is best?

In order to answer that question, we first need to define what “best” should mean. Without loss of generality, fix a group $g$. Clearly, $\operatorname{rank}\left(\overline{I}({\bm{\mathsfit{X}}})\mathcal{T}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)\right)\leq P$. Moreover, by the EYM theorem, the best rank-$P$ approximation of the output of $f$ in terms of the Frobenius norm is $\mathcal{T}^{\mathrm{SVD}}_{P}\bigl(\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})\bigr)$. Hence, a reasonable notion of an ideal scheme is one that factorizes the parameters in a way that is equivalent to factorizing the outputs of the layer; something better is impossible.

###### Definition 2 .

We call a scheme $\mathcal{T}_{P}\bigl(\cdot\bigr)$ optimal if its distortion is equivalent to directly low-rank truncating the outputs of a layer, i.e., an optimal scheme satisfies

$$ $\|\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})-\mathcal{T}^{\mathrm{SVD}}_{P}\bigl(\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})\bigr)\|_{F}=\|\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})-\overline{I}({\bm{\mathsfit{X}}})\mathcal{T}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\|_{F}.$ $$

Recall that each group is factorized separately. Note that a scheme is optimal if and only if it satisfies [Equation 2](https://arxiv.org/html/2509.25136v2#S4.E2) with the distance function set to the output distortion. In particular, SVD-LLM (Wang et al., 2025c) introduces a specific way of “talking in terms of activation distortion” for linear layers, and it is optimal in the sense of [Definition 2](https://arxiv.org/html/2509.25136v2#Thmdefinition2). A more general version of their low-rank activation-aware projection method, from the lens of our framework, can be defined as follows. For any $(\overline{O},\overline{I},\overline{P})$-expressible function $f$, let

$$ $\mathcal{T}^{\mathrm{AA}}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)=\overline{P}{}^{-1}\left({\bm{\mathsfit{M}}}\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{\mathsfit{M}}}^{+}\overline{P}\left({\bm{\mathsfit{W}}}\right)\bigr)\right),$ $$

where ${\bm{\mathsfit{M}}}$ is a batch of whitening matrices for $\overline{I}({\bm{\mathsfit{X}}})$ (where each batch pertains to a group, and the matrix multiplication is batched over the groups dimension), and the pseudoinverse is performed over each group.

Figure: Figure 1: Normalized output distortion (in terms of squared Frobenius norm) with $\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\cdot\bigr)$ and $\mathcal{T}^{\mathrm{SVD}}_{P}\bigl(\cdot\bigr)$ for different values of $P$, across different layer types.
Refer to caption: x1.png

For the layers and their corresponding $(\overline{O},\overline{I},\overline{P})$-tuples mentioned earlier, this corresponds, as discussed in previous examples, to two sequential sub-layers (e.g., two linear or convolutional layers) that yield computational gains with sufficiently low $P$. More details are given in [Appendix A](https://arxiv.org/html/2509.25136v2#A1). Additionally, our scheme remains optimal, as claimed in [Theorem 1](https://arxiv.org/html/2509.25136v2#Thmtheorem1).

###### Theorem 1 .

$\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\cdot\bigr)$ is optimal in the sense of [Definition 2](https://arxiv.org/html/2509.25136v2#Thmdefinition2).

Moreover, the activation distortion that follows from using our framework is readily computable from the artifacts obtained during compression, as shown in [Theorem 2](https://arxiv.org/html/2509.25136v2#Thmtheorem2).

###### Theorem 2 .

Assume that $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})$ is $(\overline{O},\overline{I},\overline{P})$-expressible. Then, the activation distortion (as defined in [Equation 3](https://arxiv.org/html/2509.25136v2#S4.E3)) incurred when low-rank projecting the parameters with $\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\cdot\bigr)$ is $\frac{N}{B}\sum_{g=1}^{G}{\sum_{i=P+1}^{U}\bm{\sigma}_{g,i}^{2}}$, where $\bm{\sigma}$ denotes the batched singular values of ${\bm{\mathsfit{M}}}^{+}\overline{P}({\bm{\mathsfit{W}}})$.

Recall that $B$ is the batch size (number of calibration samples), and $N$ is the outer dimension (see the discussion below [Definition 1](https://arxiv.org/html/2509.25136v2#Thmdefinition1)).
The main result of Wang et al. (2025c) follows as a corollary.

###### Definition 2 .

###### Theorem 1 .

###### Theorem 2 .

##### A simple experiment.

We compare the output distortion of our activation-aware projection with that of standard SVD. In particular, we select three layers of different types, truncate them to various rank ratios, and measure the resulting output distortions on 1024 CIFAR-10 test images (using 1024 training images for calibration). As shown in [Figure 1](https://arxiv.org/html/2509.25136v2#S4.F1), activation-aware factorization yields significantly lower distortion.

### 4.2 Computing the Whitening Matrices

We compute the whitening matrices using an eigendecomposition of the transformed features. Suppose we are given a group $g$. Let ${\bm{X}}=\overline{I}({\bm{\mathsfit{X}}})_{g}\in\mathbb{R}^{N\times I}$. We can then compute the eigenvectors and eigenvalues of $\frac{1}{N}{\bm{X}}^{T}{\bm{X}}$: ${\bm{V}}$ and ${\bm{\Lambda}}$, respectively, and set

$$ ${\bm{M}}={\bm{V}}{\bm{\Lambda}}^{+1/2},\quad{\bm{M}}^{+}={\bm{\Lambda}}^{1/2}{\bm{V}}^{T}.$ $$

This procedure is applied (in vectorized form) independently for each group. Second moments can be accumulated incrementally over minibatches, so the method is memory-efficient. Additional implementation details are provided in [Appendix B](https://arxiv.org/html/2509.25136v2#A2).

### 4.3 Budgeted Rank Allocation

Apart from offering output distortion guarantees, this framework is interpretable in the sense that we can measure the output distortion without additional steps (see [Theorem 2](https://arxiv.org/html/2509.25136v2#Thmtheorem2)). Can that be used to efficiently choose the rank to which each layer should be truncated?

##### The rank allocation problem.

We now turn to the question of how to select the ranks to retain in each layer, $\{P_{l}\}_{l=1}^{L}$. The naive option is to choose a uniform percentage of rank per layer (Wang et al., 2025c). A more reasonable option, used in the past for the traditional SVD scheme, is the energy-based singular value pruning criterion (Yang et al., 2020; Liebenwein et al., 2021). While more expensive approaches have been explored, such as training-based rank learning (Idelbayev & Carreira-Perpiñán, 2020; Qinsi et al., 2025) or NAS-based search (Yu & Bouganis, 2023), our main goal is to find a scalable and efficient method. We can define the analog of the energy metric in our setting as follows (it reads as the energy retained in layer $l$ when keeping up to rank $P_{l}$):

$$ $E_{l}(P_{l})=1-\frac{\|f_{l}({\bm{\mathsfit{X}}}^{l-1};{\bm{\mathsfit{W}}}^{l})-f_{l}({\bm{\mathsfit{X}}}^{l-1};\mathcal{T}^{\mathrm{AA}}_{P_{l}}\bigl({\bm{\mathsfit{W}}}^{l}\bigr))\|^{2}_{F}}{\|f_{l}({\bm{\mathsfit{X}}}^{l-1};{\bm{\mathsfit{W}}}^{l})\|^{2}_{F}}=\frac{\sum_{g=1}^{G}\sum_{i=1}^{P_{l}}\bm{\sigma}_{g,i}^{2}}{\sum_{g=1}^{G}\sum_{i=1}^{U_{l}}\bm{\sigma}_{g,i}^{2}},$ $$

where $l$ denotes the layer index, $U_{l}$ the total number of singular values (per group), and $P_{l}$ the number retained (per group). Energy-based selection then consists of picking the lowest rank such that a certain energy threshold (manually defined by the user) is retained. We propose instead to solve an optimization problem that maximizes the total retained energy globally:

$$ $\underset{{P_{1},\dots,P_{L}}}{\max}\sum_{l=1}^{L}E_{l}(P_{l}),\quad\text{subject to }\sum_{l=1}^{L}C_{l}(P_{l})\leq C_{\max}\;\ \text{ and}\quad P_{l}\in\{1,\dots,U_{l}\},$ $$

where $C_{l}(P_{l})$ is some measure of complexity (e.g., FLOPs or number of parameters when keeping up to rank $P_{l}$ on layer $l$), $U_{l}$ is the total number of singular values for a group on layer $l$, and $C_{\max}$ is the model complexity budget. Our allocator supports FLOPs and the number of parameters as complexity targets. The optimization problem is an instance of the multiple-choice knapsack problem, and it can be formulated as a binary linear program. Unfortunately, it is known to be NP-hard (for a review, see Szkaliczki (2025)), which motivates the need for an alternative.

##### A Lagrangian relaxation.

We solve this limitation by using a Lagrangian relaxation of the problem, whose time complexity is $O((I+D+1)\sum_{l}U_{l})$, where $I$ stands for “number of iterations” and $D$ depends on the network. In practice, using 300 iterations yields excellent results while taking less than 0.2 seconds, even for the larger models with which we experimented. We provide additional details in [Section C.1](https://arxiv.org/html/2509.25136v2#A3.SS1).

##### Advantages.

Our selection approach has several advantages: (1) it has virtually zero additional overhead and does not query the model, unlike brute-force search methods, which can take hours (e.g., Yu & Bouganis (2023)), and (2) it provides explicit control over a complexity budget. This reduces the need for manual hyperparameter sweeps to achieve some compression target: a single run is enough.

### 4.4 Putting It All Together: BALF

Together, these pieces form BALF. An overview can be found in [Algorithm 1](https://arxiv.org/html/2509.25136v2#alg1). The routine NewLayer generates a new layer (consisting of two sequential sub-layers) when given the factorized matrices. In practice, we do not select a manual budget $C_{\max}$, but rather a ratio (e.g., we might want to obtain a model with 50% of its original FLOPs). Additionally, note that we retain the original layer if compressing it to the selected rank is not computationally advantageous.

Figure: Algorithm 1 BALF overview. Group notation is omitted for simplicity.

##### Bounding the output distortion of a model.

In general, providing bounds on the cumulative model distortion incurred when compressing it is challenging. Interestingly, talking in terms of $\ell^{\mathrm{activ}}$ (see [Equation 3](https://arxiv.org/html/2509.25136v2#S4.E3)) allows for tighter bounds.

###### Theorem 3 .

(Informal)
Define a sequential L-layer network

$$ ${\bm{\mathsfit{X}}}^{0}={\bm{\mathsfit{X}}},\quad{\bm{\mathsfit{X}}}^{l}=a_{l}\left(f_{l}({\bm{\mathsfit{X}}}^{l-1};{\bm{\mathsfit{W}}}^{l})+{\bm{b}}^{l}\right)=a_{l}\left(\overline{O}_{l}(\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l}))+{\bm{b}}^{l}\right),$ $$

where $a_{l}$ is an element-wise activation function with Lipschitz constant $A_{l}$. Moreover, let $B_{l}$ be the Lipschitz constant of $\overline{I}_{l}$ (recall that $\overline{I}_{l}$ is linear). We assume all layers are ungrouped. Then,

$$ $\frac{1}{\sqrt{B}}\bigl\|{\bm{\mathsfit{X}}}^{L}-\widehat{{\bm{\mathsfit{X}}}}{}^{L}\bigr\|_{F}\;\leq\;\sum_{l=1}^{L}\Bigl[A_{l}\sqrt{\ell^{\mathrm{activ}}_{l}(P_{l})}\;\prod_{i=l+1}^{L}\|\overline{P}_{i}(\widehat{{\bm{\mathsfit{W}}}}{}^{i})\|_{2}A_{i}B_{i}\Bigr].$ $$

A more detailed statement, its proof, and special cases appear in [Appendix D](https://arxiv.org/html/2509.25136v2#A4). The tightness of the bound depends on the model’s depth and its weights, and it is typically loose in practical deep networks. Even so, it remains conceptually informative.

###### Theorem 3 .

## 5 Experiments

This section presents our experimental results, covering a range of architectures, datasets, and scales. Our code is available at [https://github.com/davidgonmar/BALF](https://github.com/davidgonmar/BALF). On CIFAR-10 (Krizhevsky, 2009), we evaluate ResNet-20 and ResNet-56, while on ImageNet (Deng et al., 2009), we consider ResNet-18 and ResNet-50 (He et al., 2016), ResNeXt-50 (32$\times$4d) and ResNeXt-101 (32$\times$8d) with grouped convolutions (Xie et al., 2017), ViT-B/16 (Dosovitskiy et al., 2021), DeiT-B/16 (Touvron et al., 2021), and MobileNet-V2 (Sandler et al., 2018). For all models, every layer is considered for factorization. Calibration is performed using 1024 images for CIFAR-10 and 8192 images for ImageNet, both uniformly sampled from their respective training datasets. The precise settings and hyperparameters are provided in [Appendix E](https://arxiv.org/html/2509.25136v2#A5).

##### Comparison with factorization approaches.

First, we perform an extensive sweep showcasing the complexity-accuracy curves for FLOP and parameter counts. To highlight which components of BALF are important, each run is performed using different methods (all implemented by us): SVD with the energy criterion, activation-aware SVD with the energy criterion, and BALF (activation-aware decomposition with our rank allocator); see [Section 4.3](https://arxiv.org/html/2509.25136v2#S4.SS3) for an overview.

Figure: Figure 2: Sweep over different compression ratios with different decomposition methods on different models. Each figure shows six curves. Three of them represent the parameters-accuracy curves and the other three represent the FLOPs-accuracy curves. BALF is set with parameter count and FLOP count ratio targets in the parameters-accuracy and FLOPs-accuracy curves, respectively.
Refer to caption: x2.png

[Figure 2](https://arxiv.org/html/2509.25136v2#S5.F2) summarizes the complexity-accuracy trade-offs across the different settings. The qualitative trend is consistent: activation-aware decomposition yields clear gains, with the magnitude of the gap varying by configuration. In contrast, standard SVD degrades accuracy even at modest compression ratios, making it impractical without fine-tuning. Finally, our rank-allocation method enables further low-distortion compression—for example, we can reduce the FLOPs of the ResNeXt-101 model by 50% with only about 1.5 percentage points of top-1 accuracy drop.

##### Comparison with other fine-tuning-free methods.

Additionally, we compare our factorization method with other compression methods in the fine-tuning-free regime.

For ResNet-50, we compare BALF with several pruning methods: DFPC (Narshana et al., 2023), IterTVSPrune (ITVSP) (Murti et al., 2023), and IFM (Chen et al., 2024). For ResNet-20, we compare against the pruning approach FVRCP (He et al., 2020) and the factorization method ALDS (Liebenwein et al., 2021).

For DeiT-B/16, we compare our method with two types of techniques: GTP (Xu et al., 2024), a token-merging approach that reduces inference FLOPs (but not model size), and two pruning methods: DC-ViT (Zhang et al., 2024) and PRACTISE (Wang & Wu, 2023) (using the results reported by Zhang et al. (2024)).

Compressing compact models such as MobileNet-V2 is known to be challenging. For this model, we compare against the factorization methods SVD-NAS (Yu & Bouganis, 2023), ALDS (Liebenwein et al., 2021), and LR-S2 (Idelbayev & Carreira-Perpiñán, 2020) (using the no-fine-tuning results for ALDS and LR-S2 reported by Yu & Bouganis (2023)). We also compare with the pruning-restoration method proposed by Lee et al. (2025), using their L2 variant (denoted L2+REST). Note that SVD-NAS requires an expensive search procedure (on the order of hours), and LR-S2 allocates ranks through a training run, making them substantially more expensive than BALF. [Table 1](https://arxiv.org/html/2509.25136v2#S5.T1) contains the different results. In general, BALF surpasses or is competitive with all the compared methods.

##### BALF is efficient.

On an RTX 2070 laptop, compressing ImageNet models completes in minutes (under four minutes for all models except ResNeXt-101, which takes less than seven). The most expensive steps can be cached, reducing subsequent compression runs to seconds. We provide a comprehensive benchmark and implementation details in [Appendix G](https://arxiv.org/html/2509.25136v2#A7).

**Table 1: Comparison of different methods at different compression configurations. BALF-P-$x$ and BALF-F-$y$ denote our method when set with parameter and FLOP count objectives, respectively. “—” is used for results that were not reported. $\Delta$F% and $\Delta$P% denote the percentage changes in FLOPs and parameter counts relative to the original model.**
| Method | $\Delta$F% | $\Delta$P% | $\Delta$Top-1 pp |
| --- | --- | --- | --- |
| BALF-F-0.5 | $-50.04$ | $-43.86$ | $-13.84$ |
| BALF-P-0.5 | $-28.18$ | $-50.08$ | $-7.78$ |
| BALF-F-0.7 | $-30.00$ | $-21.69$ | $-3.48$ |
| BALF-P-0.7 | $-13.55$ | $-29.94$ | $-1.96$ |
| BALF-F-0.8 | $-20.00$ | $-9.77$ | $-1.65$ |
| BALF-P-0.8 | $-9.03$ | $-21.92$ | $-1.12$ |
| IFM | — | $-5.65$ | $-1.46$ |
| IFM | — | $-20.42$ | $-10.45$ |
| ITVSP | — | $-4.76$ | $-3.02$ |
| ITVSP | — | $-9.98$ | $-10.21$ |
| DFPC | — | $-5.66$ | $-5.78$ |
| DFPC | — | $-10.92$ | $-13.88$ |

Figure: Figure 3: Calibration dataset size vs. accuracy over different compression ratios (in terms of parameter counts) on ResNet-50.
Refer to caption: x8.png

##### Other results.

On CIFAR-10-C (Hendrycks & Dietterich, 2019), BALF remains robust to distribution shift: when whitening matrices were estimated on the standard CIFAR-10 training set and evaluation was performed on CIFAR-10-C, the additional accuracy drop was typically less than 2.5 pp. Moreover, we found that a smaller number of calibration samples (about 256 and 1024 for CIFAR-10 and ImageNet, respectively) often yields only marginally worse results than those reported in this section. For instance, see [Figure 3](https://arxiv.org/html/2509.25136v2#S5.F3) for a sweep on ResNet-50. The remaining results can be found in [Appendix F](https://arxiv.org/html/2509.25136v2#A6).

##### Limitations and future work.

Even when surpassing (or being competitive with) other methods, compressing compact models (like MobileNet-V2) remains difficult. End-to-end speedups are notable, but we believe that implementing specialized operators would yield better gains (see [Section F.3](https://arxiv.org/html/2509.25136v2#A6.SS3) for a discussion). We speculate that combining BALF with expensive search methods (e.g., SVD-NAS) could yield excellent results at the expense of compression time.

## 6 Conclusion

In this work, we present BALF, a principled and general activation-aware factorization framework with a budgeted rank allocator that lets users specify FLOPs or parameter count targets directly. Across diverse architectures and scales, BALF delivers strong accuracy-efficiency trade-offs—often surpassing existing fine-tuning-free approaches—while achieving low factorization time on commodity GPUs. This positions BALF as a practical path to model compression without costly search or fine-tuning.

#### Reproducibility Statement

We have taken several steps to ensure that our results are reproducible. [Appendix E](https://arxiv.org/html/2509.25136v2#A5) provides a detailed description of our experimental setup, and we make our code publicly available at [https://github.com/davidgonmar/BALF](https://github.com/davidgonmar/BALF). The repository includes algorithm implementations, experiment and launch scripts with the exact parameters used, and fixed seeds for every run. It also provides installation instructions and a complete list of dependencies.

#### Acknowledgments

I thank Fernando Sancho Caparrini and Miguel Ángel Martínez del Amor for their valuable comments and suggestions, which helped improve this manuscript. I acknowledge access to computational resources provided by the University of Seville, including an A100 40GB GPU donated through the NVIDIA Hardware Grant, as well as an RTX 4090, and servers managed by the DeepKnowledge group. I am also grateful for the support of the University of Seville through the *Programa de Becas de Iniciación a la Investigación (VIIPPIT-2025-II.1)*.

## References

- Ansel et al. (2024)
Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael Lazos, Mario Lezcano, Yanbo Liang, Jason Liang, Yinghai Lu, CK Luk, Bert Maher, Yunjie Pan, Christian Puhrsch, Matthias Reso, Mark Saroufim, Marcos Yukio Siraichi, Helen Suk, Michael Suo, Phil Tillet, Eikan Wang, Xiaodong Wang, William Wen, Shunting Zhang, Xu Zhao, Keren Zhou, Richard Zou, Ajit Mathews, Gregory Chanan, Peng Wu, and Soumith Chintala.
PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation.
In *29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS ’24)*. ACM, April 2024.
doi: 10.1145/3620665.3640366.
URL [https://docs.pytorch.org/assets/pytorch2-2.pdf](https://docs.pytorch.org/assets/pytorch2-2.pdf).
- Ben Noach & Goldberg (2020)
Matan Ben Noach and Yoav Goldberg.
Compressing Pre-trained Language Models by Matrix Decomposition.
In Kam-Fai Wong, Kevin Knight, and Hua Wu (eds.), *Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing*, pp. 884–889, Suzhou, China, December 2020. Association for Computational Linguistics.
doi: 10.18653/v1/2020.aacl-main.88.
URL [https://aclanthology.org/2020.aacl-main.88/](https://aclanthology.org/2020.aacl-main.88/).
- Chellapilla et al. (2006)
Kumar Chellapilla, Sidd Puri, and Patrice Simard.
High Performance Convolutional Neural Networks for Document Processing.
In Guy Lorette (ed.), *Tenth International Workshop on Frontiers in Handwriting Recognition*, La Baule (France), October 2006. Université de Rennes 1, Suvisoft.
URL [https://inria.hal.science/inria-00112631](https://inria.hal.science/inria-00112631).
- Chen et al. (2021)
Patrick Chen, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh.
DRONE: Data-aware Low-rank Compression for Large NLP Models.
In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), *Advances in Neural Information Processing Systems*, volume 34, pp. 29321–29334. Curran Associates, Inc., 2021.
URL [https://proceedings.neurips.cc/paper_files/paper/2021/file/f56de5ef149cf0aedcc8f4797031e229-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2021/file/f56de5ef149cf0aedcc8f4797031e229-Paper.pdf).
- Chen et al. (2024)
Yiting Chen, Zhanpeng Zhou, and Junchi Yan.
Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory.
In *The Twelfth International Conference on Learning Representations*, 2024.
URL [https://openreview.net/forum?id=4bSQ3lsfEV](https://openreview.net/forum?id=4bSQ3lsfEV).
- Cisse et al. (2017)
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier.
Parseval Networks: Improving Robustness to Adversarial Examples.
In Doina Precup and Yee Whye Teh (eds.), *Proceedings of the 34th International Conference on Machine Learning*, volume 70 of *Proceedings of Machine Learning Research*, pp. 854–863. PMLR, 06–11 Aug 2017.
URL [https://proceedings.mlr.press/v70/cisse17a.html](https://proceedings.mlr.press/v70/cisse17a.html).
- Deng et al. (2009)
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.
ImageNet: A large-scale hierarchical image database.
In *2009 IEEE Conference on Computer Vision and Pattern Recognition*, pp. 248–255, 2009.
doi: 10.1109/CVPR.2009.5206848.
- Denton et al. (2014)
Emily Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus.
Exploiting linear structure within convolutional networks for efficient evaluation.
In *Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1*, NIPS’14, pp. 1269–1277, Cambridge, MA, USA, 2014. MIT Press.
- Dosovitskiy et al. (2021)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.
In *International Conference on Learning Representations*, 2021.
URL [https://openreview.net/forum?id=YicbFdNTTy](https://openreview.net/forum?id=YicbFdNTTy).
- Dunning et al. (2011)
Iain Dunning, Stuart Mitchell, and Michael O’Sullivan.
PuLP: A Linear Programming Toolkit for Python.
Technical report, Department of Engineering Science, The University of Auckland, September 2011.
URL [https://optimization-online.org/?p=11731](https://optimization-online.org/?p=11731).
- Frantar & Alistarh (2022)
Elias Frantar and Dan Alistarh.
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning.
In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), *Advances in Neural Information Processing Systems*, 2022.
URL [https://openreview.net/forum?id=ksVGCOlOEba](https://openreview.net/forum?id=ksVGCOlOEba).
- Frantar et al. (2023)
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh.
OPTQ: Accurate Quantization for Generative Pre-trained Transformers.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=tcbBPnfwxS](https://openreview.net/forum?id=tcbBPnfwxS).
- Gholami et al. (2021)
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer.
A Survey of Quantization Methods for Efficient Neural Network Inference.
*CoRR*, abs/2103.13630, 2021.
URL [https://arxiv.org/abs/2103.13630](https://arxiv.org/abs/2103.13630).
- Golub & Van Loan (2013)
Gene H. Golub and Charles F. Van Loan.
*Matrix Computations - 4th Edition*.
Johns Hopkins University Press, Philadelphia, PA, 2013.
doi: 10.1137/1.9781421407944.
URL [https://epubs.siam.org/doi/abs/10.1137/1.9781421407944](https://epubs.siam.org/doi/abs/10.1137/1.9781421407944).
- He et al. (2020)
Junjie He, Bohua Chen, Yinzhang Ding, and Dongxiao Li.
Feature Variance Ratio-Guided Channel Pruning for Deep Convolutional Network Acceleration.
In *Proceedings of the Asian Conference on Computer Vision (ACCV)*, November 2020.
- He et al. (2016)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep Residual Learning for Image Recognition.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2016.
- Hendrycks & Dietterich (2019)
Dan Hendrycks and Thomas Dietterich.
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations.
In *International Conference on Learning Representations*, 2019.
URL [https://openreview.net/forum?id=HJz6tiCqYm](https://openreview.net/forum?id=HJz6tiCqYm).
- Hendrycks & Gimpel (2023)
Dan Hendrycks and Kevin Gimpel.
Gaussian Error Linear Units (GELUs), 2023.
URL [https://arxiv.org/abs/1606.08415](https://arxiv.org/abs/1606.08415).
- Heroux et al. (2024)
Michael Heroux, Ahmad Abdelfattah, Natalie Beams, Robert Carson, Pieter Ghysels, Tzanio Kolev, Thomas Stitt, Arturo Vargas, Stanimire Tomov, and Jack Dongarra.
MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU architectures.
*Int. J. High Perform. Comput. Appl.*, 38(5):468–490, September 2024.
ISSN 1094-3420.
doi: 10.1177/10943420241261960.
URL [https://doi.org/10.1177/10943420241261960](https://doi.org/10.1177/10943420241261960).
- Hoefler et al. (2021)
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste.
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks.
*Journal of Machine Learning Research*, 22(241):1–124, 2021.
URL [http://jmlr.org/papers/v22/21-0366.html](http://jmlr.org/papers/v22/21-0366.html).
- Hua et al. (2023)
Ting Hua, Xiao Li, Shangqian Gao, Yen-Chang Hsu, Yilin Shen, and Hongxia Jin.
Dynamic Low-rank Estimation for Transformer-based Language Models.
In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), *Findings of the Association for Computational Linguistics: EMNLP 2023*, pp. 9275–9287, Singapore, December 2023. Association for Computational Linguistics.
doi: 10.18653/v1/2023.findings-emnlp.621.
URL [https://aclanthology.org/2023.findings-emnlp.621/](https://aclanthology.org/2023.findings-emnlp.621/).
- Idelbayev & Carreira-Perpiñán (2020)
Yerlan Idelbayev and Miguel Á. Carreira-Perpiñán.
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer.
In *2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 8046–8056, 2020.
doi: 10.1109/CVPR42600.2020.00807.
- Ioffe & Szegedy (2015)
Sergey Ioffe and Christian Szegedy.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.
In Francis Bach and David Blei (eds.), *Proceedings of the 32nd International Conference on Machine Learning*, volume 37 of *Proceedings of Machine Learning Research*, pp. 448–456, Lille, France, 07–09 Jul 2015. PMLR.
URL [https://proceedings.mlr.press/v37/ioffe15.html](https://proceedings.mlr.press/v37/ioffe15.html).
- Jacob et al. (2018)
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko.
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2018.
- Jaderberg et al. (2014)
Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman.
Speeding up Convolutional Neural Networks with Low Rank Expansions.
In *Proceedings of the British Machine Vision Conference*. BMVA Press, 2014.
doi: http://dx.doi.org/10.5244/C.28.88.
- Kessy et al. (2018)
Agnan Kessy, Alex Lewin, and Korbinian Strimmer.
Optimal Whitening and Decorrelation.
*The American Statistician*, 72(4):309–314, 2018.
doi: 10.1080/00031305.2016.1277159.
URL [https://doi.org/10.1080/00031305.2016.1277159](https://doi.org/10.1080/00031305.2016.1277159).
- Krizhevsky (2009)
Alex Krizhevsky.
Learning Multiple Layers of Features from Tiny Images, 2009.
URL [https://api.semanticscholar.org/CorpusID:18268744](https://api.semanticscholar.org/CorpusID:18268744).
- Krizhevsky et al. (2012)
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.
ImageNet Classification with Deep Convolutional Neural Networks.
In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), *Advances in Neural Information Processing Systems*, volume 25. Curran Associates, Inc., 2012.
URL [https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf).
- Lebedev et al. (2014)
Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, I. Oseledets, and Victor S. Lempitsky.
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition.
*CoRR*, abs/1412.6553, 2014.
URL [https://api.semanticscholar.org/CorpusID:15002492](https://api.semanticscholar.org/CorpusID:15002492).
- LeCun et al. (2015)
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.
Deep learning.
*Nature*, 521(7553):436–444, 2015.
doi: 10.1038/nature14539.
URL [https://doi.org/10.1038/nature14539](https://doi.org/10.1038/nature14539).
- Lee et al. (2025)
Keonho Lee, Minsoo Kim, and Dong-Wan Choi.
Training-Free Restoration of Pruned Neural Networks, 2025.
URL [https://arxiv.org/abs/2502.08474](https://arxiv.org/abs/2502.08474).
- Liebenwein et al. (2021)
Lucas Liebenwein, Alaa Maalouf, Dan Feldman, and Daniela Rus.
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition.
In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), *Advances in Neural Information Processing Systems*, 2021.
URL [https://openreview.net/forum?id=BvJkwMhyInm](https://openreview.net/forum?id=BvJkwMhyInm).
- maintainers & contributors (2016)
TorchVision maintainers and contributors.
TorchVision: PyTorch’s Computer Vision library, November 2016.
URL [https://github.com/pytorch/vision](https://github.com/pytorch/vision).
- Murti et al. (2023)
Chaitanya Murti, Tanay Narshana, and Chiranjib Bhattacharyya.
TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=sZI1Oj9KBKy](https://openreview.net/forum?id=sZI1Oj9KBKy).
- Nair & Hinton (2010)
Vinod Nair and Geoffrey E. Hinton.
Rectified linear units improve restricted boltzmann machines.
In *Proceedings of the 27th International Conference on Machine Learning*, ICML’10, pp. 807–814, Madison, WI, USA, 2010. Omnipress.
ISBN 9781605589077.
- Narshana et al. (2023)
Tanay Narshana, Chaitanya Murti, and Chiranjib Bhattacharyya.
DFPC: Data flow driven pruning of coupled channels without data.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=mhnHqRqcjYU](https://openreview.net/forum?id=mhnHqRqcjYU).
- NVIDIA (2025)
NVIDIA.
*cuSOLVER Library*, 2025.
URL [https://docs.nvidia.com/cuda/cusolver/](https://docs.nvidia.com/cuda/cusolver/).
Release 13.0.
- Phan et al. (2020)
Anh-Huy Phan, Konstantin Sobolev, Konstantin Sozykin, Dmitry Ermilov, Julia Gusak, Petr Tichavský, Valeriy Glukhov, Ivan Oseledets, and Andrzej Cichocki.
Stable Low-Rank Tensor Decomposition for Compression of Convolutional Neural Network.
In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), *Computer Vision – ECCV 2020*, pp. 522–539, Cham, 2020. Springer International Publishing.
ISBN 978-3-030-58526-6.
- Qinsi et al. (2025)
Wang Qinsi, Jinghan Ke, Masayoshi Tomizuka, Kurt Keutzer, and Chenfeng Xu.
Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives.
In *The Thirteenth International Conference on Learning Representations*, 2025.
URL [https://openreview.net/forum?id=kws76i5XB8](https://openreview.net/forum?id=kws76i5XB8).
- Sandler et al. (2018)
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.
MobileNetV2: Inverted Residuals and Linear Bottlenecks.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2018.
- Shumaylov et al. (2025)
Zakhar Shumaylov, Vasileios Tsiaras, and Yannis Stylianou.
On Information Geometry and Iterative Optimization in Model Compression: Operator Factorization, 2025.
URL [https://arxiv.org/abs/2507.09428](https://arxiv.org/abs/2507.09428).
- Sui et al. (2024)
Yang Sui, Miao Yin, Yu Gong, Jinqi Xiao, Huy Phan, and Bo Yuan.
ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks, 2024.
URL [https://arxiv.org/abs/2401.10341](https://arxiv.org/abs/2401.10341).
- Szkaliczki (2025)
Tibor Szkaliczki.
Solution Methods for the Multiple-Choice Knapsack Problem and Their Applications.
*Mathematics*, 13(7), 2025.
ISSN 2227-7390.
doi: 10.3390/math13071097.
URL [https://www.mdpi.com/2227-7390/13/7/1097](https://www.mdpi.com/2227-7390/13/7/1097).
- Touvron et al. (2021)
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou.
Training data-efficient image transformers &; distillation through attention.
In Marina Meila and Tong Zhang (eds.), *Proceedings of the 38th International Conference on Machine Learning*, volume 139 of *Proceedings of Machine Learning Research*, pp. 10347–10357. PMLR, 18–24 Jul 2021.
URL [https://proceedings.mlr.press/v139/touvron21a.html](https://proceedings.mlr.press/v139/touvron21a.html).
- Vaswani et al. (2017)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is All you Need.
In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 30. Curran Associates, Inc., 2017.
URL [https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf).
- Wang et al. (2025a)
Dong Wang, Haris Šikić, Lothar Thiele, and Olga Saukh.
Forget the Data and Fine-Tuning! Just Fold the Network to Compress.
In *The Thirteenth International Conference on Learning Representations*, 2025a.
URL [https://openreview.net/forum?id=W2Wkp9MQsF](https://openreview.net/forum?id=W2Wkp9MQsF).
- Wang & Wu (2023)
Guo-Hua Wang and Jianxin Wu.
Practical Network Acceleration With Tiny Sets.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 20331–20340, June 2023.
- Wang et al. (2025b)
Xin Wang, Samiul Alam, Zhongwei Wan, Hui Shen, and Mi Zhang.
SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compression.
In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), *Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)*, pp. 4287–4296, Albuquerque, New Mexico, April 2025b. Association for Computational Linguistics.
ISBN 979-8-89176-189-6.
doi: 10.18653/v1/2025.naacl-long.217.
URL [https://aclanthology.org/2025.naacl-long.217/](https://aclanthology.org/2025.naacl-long.217/).
- Wang et al. (2025c)
Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang.
SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression.
In *The Thirteenth International Conference on Learning Representations*, 2025c.
URL [https://openreview.net/forum?id=LNYIUouhdt](https://openreview.net/forum?id=LNYIUouhdt).
- Wightman (2019)
Ross Wightman.
PyTorch Image Models, 2019.
URL [https://github.com/huggingface/pytorch-image-models](https://github.com/huggingface/pytorch-image-models).
- Xie et al. (2017)
Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and Kaiming He.
Aggregated Residual Transformations for Deep Neural Networks.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, July 2017.
- Xu et al. (2024)
Xuwei Xu, Sen Wang, Yudong Chen, Yanping Zheng, Zhewei Wei, and Jiajun Liu.
GTP-ViT: Efficient Vision Transformers via Graph-Based Token Propagation.
In *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)*, pp. 86–95, January 2024.
- Yang et al. (2020)
Huanrui Yang, Minxue Tang, Wei Wen, Feng Yan, Daniel Hu, Ang Li, Hai Li, and Yiran Chen.
Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification.
*CoRR*, abs/2004.09031, 2020.
URL [https://arxiv.org/abs/2004.09031](https://arxiv.org/abs/2004.09031).
- Yin et al. (2021)
Miao Yin, Yang Sui, Siyu Liao, and Bo Yuan.
Towards Efficient Tensor Decomposition-Based DNN Model Compression With Optimization Framework.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 10674–10683, June 2021.
- Yu & Bouganis (2023)
Zhewen Yu and Christos-Savvas Bouganis.
SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search.
In *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)*, pp. 1503–1512, January 2023.
- Yuan et al. (2025)
Zhihang Yuan, Yuzhang Shang, Yue Song, Dawei Yang, Qiang Wu, Yan Yan, and Guangyu Sun.
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models, 2025.
URL [https://arxiv.org/abs/2312.05821](https://arxiv.org/abs/2312.05821).
- Zhang et al. (2024)
Hanxiao Zhang, Yifan Zhou, and Guo-Hua Wang.
Dense Vision Transformer Compression with Few Samples.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 15825–15834, June 2024.

## Appendix Overview

This appendix is organized as follows:

- •
[Appendix A](https://arxiv.org/html/2509.25136v2#A1) expands on the factorization scheme and provides parameter and FLOP counts for layers before and after low-rank factorization. It includes results for fully connected, ungrouped convolutional, and grouped convolutional layers, and motivates the use of a shared per-group rank for the latter.
- •
[Appendix B](https://arxiv.org/html/2509.25136v2#A2) details the construction of uncentered whitening matrices via eigendecomposition, discusses numerical choices and fallbacks, and describes the incremental estimation of second moments and other details.
- •
[Appendix C](https://arxiv.org/html/2509.25136v2#A3) discusses the specifics of our rank allocator: it specifies the cost model, and presents the previously mentioned Lagrangian relaxation, together with complexity analysis and experimental results on its behavior.
- •
[Appendix D](https://arxiv.org/html/2509.25136v2#A4) states and proves the formal counterpart of [Theorem 3](https://arxiv.org/html/2509.25136v2#Thmtheorem3) (discussed in the main text), including simplified forms for special cases.
- •
[Appendix E](https://arxiv.org/html/2509.25136v2#A5) documents the experimental details of our main experiments, including datasets, models, preprocessing, compression protocols, and other important details.
- •
[Appendix F](https://arxiv.org/html/2509.25136v2#A6) includes additional experiments that answer secondary questions, including reports on calibration dataset size ablations, robustness under distribution shift (on CIFAR-10-C), practical speedups, and additional accuracy-complexity curves that did not appear in the main text due to space constraints.
- •
[Appendix G](https://arxiv.org/html/2509.25136v2#A7) describes the end-to-end practical implementation of BALF at a high level and provides timing and peak memory measurements of its different parts.
- •
[Appendix H](https://arxiv.org/html/2509.25136v2#A8) provides a brief analysis of BALF through the lens of information geometry, following the work of Shumaylov et al. (2025).
- •
[Appendix I](https://arxiv.org/html/2509.25136v2#A9) collects complete proofs for results deferred from the main text, namely [Theorems 2](https://arxiv.org/html/2509.25136v2#Thmtheorem2) and [1](https://arxiv.org/html/2509.25136v2#Thmtheorem1) and auxiliary lemmas.
- •
[Appendix J](https://arxiv.org/html/2509.25136v2#A10) discloses the role of LLMs in this work.

## Appendix A Decomposing the Layers and Complexity After Factorization

In this section, we provide a more thorough discussion of how layers are decomposed. We also provide the parameter counts and FLOP counts before and after decomposition (the latter is used as the cost function in our rank allocator). Given a layer and a rank $P$, our scheme has the same complexity as basic SVD truncation to rank $P$; the difference lies in the (expected) quality of the outputs of the factorized layer. We omit biases for simplicity, as these remain constant in the original and factorized layers. $B$ denotes the batch size. Strictly speaking, the FLOP counts reported below should all be doubled (as it stands, we count each multiply-accumulate as one FLOP), but we omit the doubling factor for readability.

### A.1 Fully Connected Layers

Let the weight be ${\bm{W}}\in\mathbb{R}^{D_{i}\times D_{o}}$. For ${\bm{X}}\in\mathbb{R}^{B\times D_{i}}$, the original layer computes

$$ $f({\bm{X}};{\bm{W}})={\bm{X}}{\bm{W}}.$ $$

It has a total of $D_{i}D_{o}$ parameters, and takes $BD_{i}D_{o}$ FLOPs.

##### After decomposition.

Given a rank $P$, we factor the weights into ${\bm{W}}_{0}\in\mathbb{R}^{D_{i}\times P}$ and ${\bm{W}}_{1}\in\mathbb{R}^{P\times D_{o}}$. The decomposed layer becomes

$$ $f({\bm{X}};{\bm{W}}_{0},{\bm{W}}_{1})=({\bm{X}}{\bm{W}}_{0}){\bm{W}}_{1}.$ $$

The total storage needed is the sum of the storage for each matrix, that is, $P(D_{i}+D_{o})$, while the number of FLOPs is $BP(D_{i}+D_{o})$ (the computation order is algebraically irrelevant, but the order is important for efficiency).

### A.2 2D Convolutional Layers

Let the kernel be ${\bm{\mathsfit{W}}}\in\mathbb{R}^{C_{o}\times C_{i}\times H_{k}\times W_{k}}$ and the (per-item) output spatial size be $H_{o}\times W_{o}$. For an input
${\bm{\mathsfit{X}}}\in\mathbb{R}^{B\times C_{i}\times H_{i}\times W_{i}}$, the original layer computes

$$ $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})={\bm{\mathsfit{X}}}*{\bm{\mathsfit{W}}},$ $$

where $*$ denotes 2D convolution. The total number of parameters is $C_{o}C_{i}H_{k}W_{k}$, and the number of FLOPs is $BH_{o}W_{o}C_{o}C_{i}H_{k}W_{k}$.

##### After decomposition.

Choose a rank $P$. We factor the kernel into

$$ ${\bm{\mathsfit{W}}}_{0}\in\mathbb{R}^{P\times C_{i}\times H_{k}\times W_{k}}\quad\text{and}\quad{\bm{\mathsfit{W}}}_{1}\in\mathbb{R}^{C_{o}\times P\times 1\times 1},$ $$

and the layer computes

$$ $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}}_{0},{\bm{\mathsfit{W}}}_{1})=\big({\bm{\mathsfit{X}}}*{\bm{\mathsfit{W}}}_{0}\big)*{\bm{\mathsfit{W}}}_{1}.$ $$

The total number of parameters is $P(C_{i}H_{k}W_{k}+C_{o})$, and the FLOPs are
$BH_{o}W_{o}P(C_{i}H_{k}W_{k}+C_{o})$. Note that $H_{o}$ and $W_{o}$ subsume the additional convolution hyperparameters (padding, strides, dilation, etc.).

“Regular” convolutions are a special case of grouped convolutions, but we included a separate discussion for the convenience of the reader. We will now discuss the general case.

### A.3 Grouped 2D Convolutional Layers

Let $G$ be the number of groups (assume $C_{i}$ and $C_{o}$ are divisible by $G$, as is generally required in modern frameworks). The grouped kernel has shape

$$ ${\bm{\mathsfit{W}}}\in\mathbb{R}^{C_{o}\times C_{i}/G\times H_{k}\times W_{k}}.$ $$

The original grouped convolution computes

$$ $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})={\bm{\mathsfit{X}}}\;\mathbin{*_{G}}\;{\bm{\mathsfit{W}}},$ $$

with a total of $C_{o}\frac{C_{i}}{G}H_{k}W_{k}$ parameters, and a total of $BH_{o}W_{o}C_{o}\frac{C_{i}}{G}H_{k}W_{k}$ FLOPs.

##### After decomposition.

Given the per-group rank $P$, we decompose the weights into

$$ ${\bm{\mathsfit{W}}}_{0}\in\mathbb{R}^{PG\times\,C_{i}/G\times\,H_{k}\times\,W_{k}}\quad\text{and}\quad{\bm{\mathsfit{W}}}_{1}\in\mathbb{R}^{C_{o}\times\,P\times\,1\times 1},$ $$

and the layer computes two grouped convolutions in sequence:

$$ $f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}}_{0},{\bm{\mathsfit{W}}}_{1})=\big({\bm{\mathsfit{X}}}\;\mathbin{*_{G}}\;{\bm{\mathsfit{W}}}_{0}\big)\;\mathbin{*_{G}}\;{\bm{\mathsfit{W}}}_{1}.$ $$

The total parameter storage across all groups is $P(C_{i}H_{k}W_{k}+C_{o})$, and the FLOPs are $BH_{o}W_{o}P(C_{i}H_{k}W_{k}+C_{o})$.

The intermediate feature map has $PG$ channels in total—$P$ per group—but the parameter and FLOP counts simplify to the expressions above.

#### A.3.1 On the Sharing of Group Ranks

As noted in the preceding analysis and in the main text, each group shares the same rank $P$. This is done for two main reasons.

First, as noted above, the decomposition with the same rank per group results in two sequential (grouped) convolutions. Having different ranks per group would mean that, implementation-wise, we would need $G$ separate convolutions (or at least some kind of bucketing scheme). In practice, this would be very challenging to perform efficiently.

The second reason is that, even if we managed to achieve an optimal implementation, grouped convolutions are, in essence, $G$ smaller convolutions run in parallel. It is expected that the largest group would generally be a bottleneck for the others, as the smaller groups would have to wait for them.

## Appendix B Obtaining the Uncentered Whitening Matrices

In this section, we provide more details about how the uncentered whitening matrices are obtained. We first begin with a discussion of past work.

The naive option to obtain an uncentered whitening matrix is to use the Cholesky decomposition of the second moments, as explained in (Wang et al., 2025c). An issue is that we cannot compute it in the case of singular positive-semidefinite matrices. In their implementation(^2^22[https://github.com/AIoT-MLSys-Lab/SVD-LLM](https://github.com/AIoT-MLSys-Lab/SVD-LLM)), they address this issue by perturbing the matrix with random noise. Later, in Wang et al. (2025b), the authors note this issue and propose the use of an SVD-based whitening matrix, which supports data residing in a subspace (although their analysis does not seem to cover this case). Nevertheless, they show slight end-to-end accuracy gains coming from the SVD-based whitening.

As noted in [Section 4.2](https://arxiv.org/html/2509.25136v2#S4.SS2), we use an eigendecomposition-based scheme, implemented in practice through the $\mathsf{eigh}$ routine in PyTorch (a routine that obtains the eigenvalues and eigenvectors of a Hermitian (symmetric in the case of $\mathbb{R}$) matrix, and is usually faster than calling the corresponding generic eigendecomposition routine). First, we show that the eigendecomposition scheme is a valid approach.

Recall that, for a single group $g$, with ${\bm{X}}=\overline{I}({\bm{\mathsfit{X}}})_{g}\in\mathbb{R}^{N\times I}$, we compute

$$ $[{\bm{V}},{\bm{\Lambda}}]=\mathsf{eigh}(\frac{1}{N}{\bm{X}}^{T}{\bm{X}}),$ $$

and set

$$ ${\bm{M}}={\bm{V}}{\bm{\Lambda}}^{+1/2},\quad{\bm{M}}^{+}={\bm{\Lambda}}^{1/2}{\bm{V}}^{T},$ $$

where

$$ ${\bm{V}}{\bm{\Lambda}}{\bm{V}}^{T}=\frac{1}{N}{\bm{X}}^{T}{\bm{X}}.$ $$

It is easy to show that it satisfies our definition. It follows that

$$ ${\bm{\Lambda}}^{+1/2}{\bm{V}}^{T}{\bm{X}}^{T}{\bm{X}}{\bm{V}}{\bm{\Lambda}}^{+1/2}=N{\bm{\Lambda}}^{+1/2}{\bm{V}}^{T}{\bm{V}}{\bm{\Lambda}}{\bm{V}}^{T}{\bm{V}}{\bm{\Lambda}}^{+1/2}=N\overline{{\bm{I}}}_{R},$ $$

$R$ being the rank of ${\bm{X}}$. Moreover, it is easy to see that $\operatorname{row}({\bm{X}})=\operatorname{range}({\bm{M}})$, as required in our definition (see [Section 3](https://arxiv.org/html/2509.25136v2#S3)). Additionally, the pseudoinverse of ${\bm{\Lambda}}$ (a diagonal matrix) can be obtained efficiently by inverting each nonzero element of the diagonal, while leaving zeros as is. This avoids additional numerical approximation errors when forming ${\bm{M}}$. We do not perform any additional steps on the whitening matrices.

##### Practical implementation.

We use the linear algebra functions (with CUDA tensors) provided in the PyTorch library. During early experimentation, we used the default solver (cuSOLVER (NVIDIA, 2025)), but we noticed that it sometimes crashed for seemingly well-behaved matrices. Because of that, we switched to the MAGMA (Heroux et al., 2024) backend, which behaved much better. Sporadically, the $\mathsf{eigh}$ routine was still unable to converge, even when the matrix was actually symmetric. For that reason, we added a fallback to the regular $\mathsf{eig}$ routine. We observed that the fallback was needed only for a couple of matrices, and only in the ResNeXt-101 model. In none of our experiments did the fallback also fail (so we do not resort to noise addition in any instance, contrasting with Wang et al. (2025c)).

##### The alternative.

An SVD-based UWM, as proposed in SVD-LLM V2 (Wang et al., 2025b), can be obtained in a similar way with the SVD of ${\bm{X}}^{T}{\bm{X}}$. Although it is numerically equivalent, it tends to be slower than our approach.

##### Accumulating the uncentered moments incrementally.

The naive way of obtaining $\frac{1}{N}{\bm{X}}^{T}{\bm{X}}$ in practice is to use a single batch, and then cache activations by passing the batch through the network. However, for some networks with high-dimensional features and/or large batch sizes, the memory requirements might make the process expensive or impossible. A nice property (mentioned in Wang et al. (2025c), but not discussed extensively) is that we can use data batches to accumulate the second moments. Suppose we have $M$ batches, then,

$$ $\frac{1}{N}{\bm{X}}^{T}{\bm{X}}=\frac{1}{N}\begin{bmatrix}{\bm{X}}_{1,:}^{T}\\ \vdots\\ {\bm{X}}_{M,:}^{T}\end{bmatrix}\begin{bmatrix}{\bm{X}}_{1,:}\cdots{\bm{X}}_{M,:}\end{bmatrix}=\frac{1}{N}\left({\bm{X}}_{1}^{T}{\bm{X}}_{1}+\cdots+{\bm{X}}_{M}^{T}{\bm{X}}_{M}\right),$ $$

where $N$ is the total outer dimension after $\overline{I}$ reshapes the activations (e.g., for convolutions, $N=BH_{o}W_{o}$). Here, the indexing denotes the selection of a sub-batch. We use this approach in our implementation.

## Appendix C Rank Allocation Details

This section provides details about the rank allocation strategy discussed in [Section 4.3](https://arxiv.org/html/2509.25136v2#S4.SS3).

Our objective is to select a list $\{P_{l}\}_{l=1}^{L}$ denoting the retained rank in each layer, with the objective of maximizing the global retained energy under some cost constraint. We can model this as a binary linear program in the form of a multiple-choice knapsack problem (for a review, see Szkaliczki (2025)), with decision variables $\{x_{l,t}\}_{l,t=1}^{L,U_{l}}$, where $x_{l,t}\in\{0,1\}$ denotes whether, at layer $l$, $t$ is selected as the maximum retained rank, and $U_{l}$ denotes an upper bound on the rank (in our case, the number of singular values). Furthermore, by establishing a function $C_{l}(P_{l})$ that returns the computational cost (e.g., in FLOPs or number of parameters) of keeping up to rank $P_{l}$ at layer $l$, we can ensure that the complexity of the solution is constrained to some user-set quantity. The binary program can be expressed as:

$$ $\displaystyle\max_{\{x_{l,t}\}}$ $\displaystyle\sum_{l=1}^{L}\sum_{t=1}^{U_{l}}E_{l}(t)\,x_{l,t}$ s.t. $\displaystyle\sum_{t=1}^{U_{l}}x_{l,t}=1,$ $\displaystyle l=1,\dots,L,$ $\displaystyle\sum_{l=1}^{L}\sum_{t=1}^{U_{l}}C_{l}(t)\,x_{l,t}\leq C_{\max},$ $\displaystyle x_{l,t}\in\{0,1\},$ $\displaystyle\forall\,l,t.$ $$

##### Are there other options for the objective function?

Recall that we use the total retained energy, which is normalized, as the objective function. Following the insights of [Theorem 3](https://arxiv.org/html/2509.25136v2#Thmtheorem3) and [Theorem 4](https://arxiv.org/html/2509.25136v2#Thmtheorem4) (although they are not tight for deeper networks), one might speculate that, with appropriate accounting of each layer’s importance, we could achieve better results. During our initial experiments, we also evaluated unnormalized distortion as a metric in place of energy and explored layer-wise weightings based on statistics such as output variance and entropy. None of these alternatives improved performance over the method presented here; instead, they increased complexity and posed additional computational overhead and implementation challenges.

##### Measuring the cost.

Our allocator makes use of a cost function. We refer the reader to [Appendix A](https://arxiv.org/html/2509.25136v2#A1) for the discussion on the cost of regular and factorized layers, both in terms of FLOPs and in terms of the number of parameters. It is clear that not all $P_{l}$ selections result in fewer parameters or FLOPs. Given $P_{l}$ and a criterion, we decompose the layer only when we achieve gains in terms of that criterion (e.g., a reduction in FLOPs or parameters); see [Algorithm 1](https://arxiv.org/html/2509.25136v2#alg1). To reflect this in the cost function, we define it as a piecewise linear function, where the first region is monotonically increasing and represents the region where decomposing the layer results in compression, and the second remains constant, denoting that the action of “keeping rank $P_{l}$” will have the cost of the original layer. Summarizing, we define

$$ $C_{l}(P_{l})=\min\{C_{l}^{\mathrm{orig}},C_{l}^{\mathrm{decomp}}(P_{l})\}.$ $$

We emphasize that this step is important so as to inform the solver that we will not factorize a layer if it is not beneficial.

### C.1 A Scalable Lagrangian Relaxation

The aforementioned program can be solved with publicly available libraries (e.g., PuLP if using Python (Dunning et al., 2011)). The issue is that solving it exactly is known to be NP-hard (see Szkaliczki (2025)). To overcome this, we employ a Lagrangian relaxation. Pseudocode defining our implementation is given in [Algorithm 2](https://arxiv.org/html/2509.25136v2#alg2). It accepts a hyperparameter $I$ denoting the number of iterations to use.

Figure: Algorithm 2 Lagrangian Relaxation for Our Rank Allocator

##### Complexity.

Each of the $I$ outer iterations scans all the candidates once, giving time $O(I\sum_{l}U_{l})$. The initial doubling to find $\lambda_{\max}$ adds an additional $O(D\sum_{l}U_{l})$ factor. Thus, the total time complexity is $O((I+D+1)\sum_{l}U_{l})$ (in the case of a nontrivial solution).

##### Quality of the solutions.

The algorithm yields an approximate solution, but in every experiment we carried out, it produced good results and saturated (approximately) the target budget.

Put simply, the algorithm aims to search for the ideal balance between penalizing the cost and maximizing energy via a parameter $\lambda$; ideal meaning the one that will retain the most energy yet yield a feasible solution. It starts with an interval where it is guaranteed that an optimal $\lambda$ resides (this is guaranteed because of the nature of our optimization problem, where costs and energies are non-decreasing and positive). That interval is then halved every iteration. Its length decreases exponentially with the number of iterations. The final output need not coincide with the actual (“primal”) optimum, but it performs well in practice.

Figure: Figure 4: The effect of the number of iterations on CIFAR-10 models. We fix a compression target of 60% of the original FLOPs. The horizontal axis denotes the number of iterations. The vertical axes denote the accuracy achieved (left) and the number of used FLOPs (right). As one can see, the results stabilize with a relatively small number of iterations.
Refer to caption: x9.png

It is, nevertheless, interesting to visualize how the algorithm behaves w.r.t. the number of iterations. [Figure 4](https://arxiv.org/html/2509.25136v2#A3.F4) compares the number of iterations with two quantities of interest: model accuracy and the actual FLOPs used by the model (when using it with the objective of keeping 60% of the FLOPs).

##### Implementation details.

We implement the algorithm in Python, as even for larger models, it has a negligible runtime (less than 0.2 seconds in all our experiments, see [Appendix G](https://arxiv.org/html/2509.25136v2#A7)). Moreover, although the energy and cost functions are represented in functional form in the algorithm description, in practice, they are precomputed as vectors for each layer.

## Appendix D Bounding the Output Distortion of a Model

In this section, we provide a more thorough analysis of the claim in [Theorem 3](https://arxiv.org/html/2509.25136v2#Thmtheorem3). We provide a more complete description and its proof. The result applies to sequential (possibly nonlinear) networks with $(\overline{O},\overline{I},\overline{P})$-expressible layers (possibly with biases).

###### Theorem 4 .

Define a sequential L-layer network

$$ ${\bm{\mathsfit{X}}}^{0}={\bm{\mathsfit{X}}},\quad{\bm{\mathsfit{X}}}^{l}=a_{l}\left(f_{l}({\bm{\mathsfit{X}}}^{l-1};{\bm{\mathsfit{W}}}^{l})+{\bm{b}}^{l}\right)=a_{l}\left(\overline{O}_{l}(\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l}))+{\bm{b}}^{l}\right),$ $$

where $a_{l}$ is an element-wise activation function with Lipschitz constant $A_{l}$. Moreover, let $B_{l}$ be the Lipschitz constant of $\overline{I}_{l}$ (note that $\overline{I}_{l}$ is linear and thus Lipschitz-continuous). We restrict our analysis to the case of ungrouped layers.
When approximating ${\bm{\mathsfit{W}}}^{l}$ as a low-rank version, we denote the resulting parameter by $\widehat{{\bm{\mathsfit{W}}}}{}^{l}$, and the corresponding cumulatively distorted intermediate activations as $\widehat{{\bm{\mathsfit{X}}}}{}^{l}$.

For each layer $l$, fix the retained rank and let

$$ $\sqrt{\ell^{\mathrm{activ}}_{l}}=\frac{1}{\sqrt{B}}\bigl\|\ f_{l}({\bm{\mathsfit{X}}}^{l-1};{\bm{\mathsfit{W}}}^{l})-f_{l}({\bm{\mathsfit{X}}}^{l-1};\widehat{{\bm{\mathsfit{W}}}}{}^{l})\bigr\|_{F},$ $$

which reads as the (non-cumulative) error introduced by perturbing ${\bm{\mathsfit{W}}}^{l}\to\widehat{\bm{\mathsfit{W}}}{}^{l}$. Then, the total output distortion

$$ $d^{(1:L)}=\frac{1}{B}\bigl\|{\bm{\mathsfit{X}}}^{L}-\widehat{{\bm{\mathsfit{X}}}}{}^{L}\bigr\|^{2}_{F}$ $$

satisfies the bound

$$ $\sqrt{d^{(1:L)}}\;\leq\;\sum_{l=1}^{L}\Bigl[A_{l}\sqrt{\ell^{\mathrm{activ}}_{l}}\;\prod_{i=l+1}^{L}\|\overline{P}_{i}(\widehat{{\bm{\mathsfit{W}}}}{}^{i})\|_{2}A_{i}B_{i}\Bigr].$ $$

We follow the convention that the empty product is equal to 1.

###### Proof.

For brevity, we set ${\bm{W}}^{l}\equiv\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})$ (recall that we assume ungrouped layers).

Note that $\|\overline{O}_{l}(\cdot)\|_{F}=\|\cdot\|_{F}$, as $\overline{O}_{l}(\cdot)$ is a composition of permutations and reshapes. It is also true that $a_{l}\circ\overline{O}_{l}=\overline{O}_{l}\circ a_{l}$, as $a_{l}$ is element-wise.

Then:

$$ $\displaystyle\sqrt{Bd^{(1:L)}}$ $\displaystyle=\|{\bm{\mathsfit{X}}}^{L}-\widehat{{\bm{\mathsfit{X}}}}{}^{L}\|_{F}$ $\displaystyle=\|a_{L}(f_{L}({\bm{\mathsfit{X}}}^{L-1};{\bm{\mathsfit{W}}}^{L})+{\bm{b}}^{L})-a_{L}(f_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1};\widehat{{\bm{\mathsfit{W}}}}{}^{L})+{\bm{b}}^{L})\|_{F}$ $\displaystyle=\|a_{L}(\overline{O}_{L}(\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1}){\bm{W}}^{L})+{\bm{b}}^{L})-a_{L}(\overline{O}_{L}(\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L})+{\bm{b}}^{L})\|_{F}$ $\displaystyle\leq A_{L}\|\overline{O}_{L}(\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1}){\bm{W}}^{L})+{\bm{b}}^{L}-\overline{O}_{L}(\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L})-{\bm{b}}^{L}\|_{F}$ $\displaystyle=A_{L}\|\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1}){\bm{W}}^{L}-\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L}\|_{F}$ $\displaystyle=A_{L}\|\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1}){\bm{W}}^{L}-\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1})\widehat{{\bm{W}}}^{L}+\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1})\widehat{{\bm{W}}}^{L}-\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L}\|_{F}$ $\displaystyle\leq A_{L}\|\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1}){\bm{W}}^{L}-\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1})\widehat{{\bm{W}}}^{L}\|_{F}+A_{L}\|\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1})\widehat{{\bm{W}}}^{L}-\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L}\|_{F}$ $\displaystyle=A_{L}\sqrt{B\ell^{\mathrm{activ}}_{L}}+A_{L}\|\overline{I}_{L}({\bm{\mathsfit{X}}}^{L-1})\widehat{{\bm{W}}}^{L}-\overline{I}_{L}(\widehat{{\bm{\mathsfit{X}}}}{}^{L-1})\widehat{{\bm{W}}}^{L}\|_{F}$ $\displaystyle\leq A_{L}\sqrt{B\ell^{\mathrm{activ}}_{L}}+\|\widehat{{\bm{W}}}^{L}\|_{2}A_{L}B_{L}\|{\bm{\mathsfit{X}}}^{L-1}-\widehat{{\bm{\mathsfit{X}}}}{}^{L-1}\|_{F}$ $\displaystyle=A_{L}\sqrt{B\ell^{\mathrm{activ}}_{L}}+A_{L}B_{L}\|\widehat{{\bm{W}}}^{L}\|_{2}\sqrt{Bd^{(1:L-1)}}.$ $$

By unrolling the recursion and dividing by $\sqrt{B}$, one arrives at the claimed statement.

∎

There are some interesting instances where the bound reduces to simpler expressions. For example, for fully connected layers, $B_{l}=1$. Moreover, for 1-Lipschitz activations, such as ReLU (Nair & Hinton, 2010) or GELU (Hendrycks & Gimpel, 2023), $A_{l}=1$. For instance, on MLPs with ReLU or GELU activations, the bounds reduce to

$$ $\displaystyle\sqrt{d^{(1:L)}}\leq\sum_{l=1}^{L}\left[\sqrt{\ell^{\mathrm{activ}}_{l}}\prod_{i=l+1}^{L}\|\widehat{{\bm{W}}}{}^{i}\|_{2}\right].$ $$

There are some works, for instance Cisse et al. (2017), that aim to constrain the operator norms of the layers of neural networks. These have direct implications for our bounds. However, in everyday networks, as noted in the main text, this bound is informative rather than practical, as it tends to become looser with depth.

###### Theorem 4 .

###### Proof.

## Appendix E Experimental Details

This section provides additional experimental details. All experiments were carried out using the PyTorch library.
We make our code public at [https://github.com/davidgonmar/BALF](https://github.com/davidgonmar/BALF). To ensure reproducibility, we include (1) the library code that implements our methods, as well as the baselines, (2) the Python scripts used to carry out the experiments, (3) the Bash scripts used to call the Python scripts with the specific hyperparameters used, and (4) the dependencies and versions used.

### E.1 CIFAR-10 Experiments

##### Models.

Since there are no widely adopted pre-trained CIFAR-10 ResNet model weights, we pre-train the ResNet-20 and ResNet-56 models ourselves, following standard practice. Our CIFAR-10 ResNet implementation is based on the [fb.resnet.torch](https://github.com/facebookarchive/fb.resnet.torch/blob/master/models/resnet.lua) implementation.

We use SGD with momentum 0.9, weight decay $10^{-4}$, and a batch size of 128. The learning rate is initialized to 0.1 and reduced by a factor of 0.1 at epochs 100 and 150. We save the final model checkpoint at epoch 200. For data augmentation, we apply 4-pixel zero-padding followed by a random crop to $32\times 32$ pixels, a random horizontal flip with a probability of 0.5, and per-channel normalization. Under these settings, the final test accuracies are 91.93% for ResNet-20 and 93.18% for ResNet-56.

##### Compression.

In order to obtain the $k$-calibration dataset, we pick $k$ random images uniformly from the training dataset. For all our main experiments, we use 1024 images.

All factorizable layers are considered for compression. For the CIFAR-10 ResNet models, this includes all the convolutional layers, including shortcut layers, and the final fully connected classifier.

##### Dataset processing.

We do not use preprocessing in our calibration dataset and test dataset, other than normalization, so as to mimic inference behavior.

### E.2 ImageNet Experiments

##### Models.

For all the CNNs, we use the implementation provided by the $\mathsf{torchvision}$ library (maintainers & contributors, 2016). We also use their $\mathsf{V1}$ checkpoints for all the models. For the experiments on vision transformers, we use the implementation and weights provided in the $\mathsf{timm}$ library (Wightman, 2019).

##### Compression.

In order to obtain the $k$-calibration dataset, we pick $k$ random images uniformly from the training dataset. For all our main experiments, we use 8192 images.

All factorizable layers are considered for compression. For the CNNs, this includes all their convolutional layers (including shortcuts) and the linear layers present in the classifier. For the vision transformers, this includes the initial convolutional layer, as well as the linear layers inside attention blocks and MLPs. For the attention blocks, we factorize the merged Q-K-V matrix (implemented in $\mathsf{timm}$ as a single linear layer).

##### Dataset processing.

Following standard inference practice, we first resize images to $256\times 256$ (using bilinear interpolation for all models except for the vision transformers, for which we use bicubic interpolation) and then center-crop them to $224\times 224$. We then normalize them as usual. We do this for both the calibration dataset and the test dataset.

### E.3 General Details

For all our main experiments, we use 300 iterations for our rank allocator (see [Section C.1](https://arxiv.org/html/2509.25136v2#A3.SS1)). For all the compression sweeps in our experiments section, we conduct a full sweep over the compression targets. For the experiments with our allocator, we select compression ratios (w.r.t. parameter and FLOP counts) between 0.1 and 1.0 in steps of 0.05; additionally, we add 0.975. For the other methods (energy-based methods), we manually select an appropriate set of hyperparameters for each model so that they cover a suitable spectrum of compression ratios.

For some comparison results in the experimental section, we used WebPlotDigitizer(^3^33[https://automeris.io/wpd](https://automeris.io/wpd)) to extract numeric values from figures when the values were not reported in tables. We took care to perform the extraction as accurately as possible.

### E.4 Computational Resources

All the experiments presented in the paper were conducted on a laptop with an RTX 2070 and a server with an RTX 4090 (the latter was used mainly for the ImageNet experiments). We note that, for all the models discussed, our framework works and is efficient even on the laptop, but accuracy evaluation is significantly faster on the RTX 4090. Preliminary experimentation that did not make it into the paper was conducted on a variety of hardware configurations, including an A100 40GB and other RTX GPUs.

## Appendix F Additional Experimental Results

This section presents some additional experimental results that aim to answer secondary questions. We also include compression-accuracy curves for three models that were not included in the main text due to space constraints.

### F.1 How Impactful is the Calibration Dataset Size?

Although we used 8192 calibration images for ImageNet and 1024 for CIFAR-10, we noticed that moderate calibration dataset sizes worked well. Here, we carry out a sweep over different calibration dataset sizes and compression ratios with parameter compression targets. We use a ResNet-20 and ResNet-56 on CIFAR-10, and ResNet-18 and ResNet-50 on ImageNet. [Figure 5](https://arxiv.org/html/2509.25136v2#A6.F5) shows the different results. As shown, CIFAR-10 is not particularly sensitive to the size of the calibration dataset. On the other hand, one can easily visualize the need for bigger calibration dataset sizes on ImageNet. This need increases for larger models (in our case, the ResNet-50), and is exacerbated for higher compression ratios. However, after 1024 images, more images yield only marginally better results.

Figure: Figure 5: Calibration size vs. performance across models. Hyperparameters are fixed to those of the main experiments, with calibration sizes swept over ${256,512,1024,2048,4096,8192,16384}$. Ratios indicate the target parameter ratios passed to the allocator.
Refer to caption: x11.png

### F.2 Robustness under Distribution Shift

Our method relies on the computation of (uncentered) whitening matrices (over the training dataset) to estimate the optimal low-rank projections. We rely on the i.i.d. assumption to claim that the statistics of the calibration dataset will be useful for compressing a model in a way that transfers to unseen data. As shown over the training-test pairs on CIFAR-10 and ImageNet, we obtain good results. In general, the i.i.d. assumption is common (although seldom discussed) in methods that use calibration data.

First, we assume that the calibration (a subset of the train dataset, in our case) and the (original) test datasets do satisfy the i.i.d. assumption. Assume we have a third dataset, namely a “corrupted test set”, whose generating distribution differs from that of the calibration and test sets.

Given a fixed compression configuration producing a compressed model (with data obtained from the calibration dataset), there is an accuracy gap (on the test dataset) due to the compression procedure, which we denote by $\Delta^{\text{test-orig}}_{\text{test-comp}}$ (comp stands for “compressed”, and orig for “original”).

Now, if we evaluate our original (non-compressed) model on the corrupted test set, we can measure the accuracy gap as the difference between the accuracy on the original test set and the corrupted test set: $\Delta^{\text{test-orig}}_{\text{test-corr}}$ (corr stands for “corrupted”). For the sake of simplicity, we assume that those are positive (i.e., there are accuracy losses), which is generally true.

We also evaluate our compressed model on the corrupted dataset, and measure $\Delta^{\text{test-orig}}_{\text{test-comp+corr}}$, i.e., the gap between the original test set on our compressed model and the corrupted test set on the compressed model.

We wish to answer the question: How much additional accuracy loss does our compression method induce when evaluated on the corrupted test dataset? Formally, we wish to measure $\Delta^{\text{test-orig}}_{\text{test-comp+corr}}-\Delta^{\text{test-orig}}_{\text{test-corr}}$ (we make the assumption that these are “orthogonal”).

Figure: Figure 6: Compression-accuracy curves on ResNet-20 over different compression ratios on CIFAR-10 and CIFAR-10-C as test datasets (the accuracy on CIFAR-10-C is measured as the average accuracy over all corruption types and severities). The left image uses a FLOPs ratio target and the right one uses a parameter count ratio target.
Refer to caption: x15.png

We measure this quantity on the CIFAR-10 dataset. We take our pre-trained ResNet-20 model on CIFAR-10 and compress it across multiple target ratios using BALF (we compress it with different configurations, including FLOPs targets and parameter count targets). We use 1024 CIFAR-10 training images as the calibration set. Additionally, we use the same hyperparameters as in our main experiments. For each resulting compressed model (and for the original model), we then evaluate: (1) its accuracy on the original CIFAR-10 test set, and (2) its accuracy on each corruption and severity level from CIFAR-10-C (Hendrycks & Dietterich, 2019). Over different compression configurations, we report clean accuracy, CIFAR-10-C mean accuracy (over all the corruption configurations), and the gap between them. Finally, we visualize the differences between clean and corrupted accuracies for both models directly and annotate their gaps. Results are shown in [Figure 6](https://arxiv.org/html/2509.25136v2#A6.F6). This allows us to measure the additional loss in robustness introduced by compression on top of the loss already incurred by distribution shift.

BALF remains surprisingly robust (more than expected, given its direct dependence on the feature distribution), with the compression step causing only small additional drops in accuracy on the corrupted test set.

### F.3 Speedup of Low-Rank Layers in Practice

This section discusses the relationship between our compression results (in terms of FLOPs) and the end-to-end speedup obtained in practice. We use basic PyTorch implementations of low-rank operators. In particular, we substitute each original layer with two sequential layers, as described in [Appendix A](https://arxiv.org/html/2509.25136v2#A1), without any specialized implementation.

We measure the throughput of two ImageNet models (ResNet-50 and ViT-B/16) on an RTX 2070 laptop at different compression ratios. Every batch normalization (BN) (Ioffe & Szegedy, 2015) layer is first removed from the ResNet-50 model so as to mimic the usual inference scenario (where BN layers are folded). We note that our method does not interfere with BN folding in any way.

We found that, in general, compression results translate well to actual speedups (see [Figure 8](https://arxiv.org/html/2509.25136v2#A6.F8) and [Figure 8](https://arxiv.org/html/2509.25136v2#A6.F8)). However, it is to be noted that convolutional layers fare worse than linear layers in this regard, and that the theoretical gains are not fully exploited (which is, in general, a persistent problem in structured compression methods without specialized operators).

We expect that specialized operator implementations, including those found in software for edge devices, will be able to fully exploit the compression in terms of speedup. In fact, Sui et al. (2024), for instance, were able to obtain near-perfect practical speedups from FLOP reductions on ASICs and FPGAs on ResNet-50 with low-rank layers.

Figure: Figure 7: Throughput vs. FLOPs retained on ViT-B/16
Refer to caption: x17.png

We note that this is, in general, a persistent problem with compression methods (see, e.g., Narshana et al. (2023), where, without specialized implementations, the actual speedups are significantly lower than the FLOPs reductions). We leave the development of specialized operators for future work.

### F.4 Additional Complexity-Accuracy Figures

The main text contained complexity-accuracy figures for six models (although experimental results were provided for all nine), but was missing the ResNet-50, DeiT-B/16, and MobileNet-V2 ones because of space constraints. They are provided in [Figure 9](https://arxiv.org/html/2509.25136v2#A6.F9).

Figure: Figure 9: Additional figures with experimental results that did not appear in the main text.
Refer to caption: x19.png

## Appendix G Runtime and Memory Footprint of BALF

Our implementation of BALF is conceptually similar to [Algorithm 1](https://arxiv.org/html/2509.25136v2#alg1), but there are some implementation details that are important to discuss. We will first provide a high-level overview of the implementation details:

- 1.
First, given a calibration dataset, we gather the activation moments. The model runs on the GPU, but the activation moments are offloaded to CPU memory to reduce GPU memory usage and allow compression on lower-end GPUs. Once activation moments are gathered, they can be cached (on disk) for subsequent compression configurations.
- 2.
Then, whitening matrices and factors are computed on the GPU. These are cached on disk to maintain low memory usage and to avoid recomputation in future runs.
- 3.
Our rank allocator uses the singular values (computed in the preceding step) to decide how much to compress each layer.
- 4.
Then, we iterate through the model and substitute the layers with their truncated versions.

**Table 2: Runtime breakdown and memory usage for different models. All timings are in seconds. Peak Mem. denotes the peak CUDA memory usage during the run (in GiB).**
| Model | Act. | Fact.+Whit. | Solver | Replace | Misc | Total | Peak Mem. |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ResNet-18 | 78.706 | 12.664 | 0.024 | 3.345 | 0.005 | 94.744 | 1.16 |
| ResNet-50 | 117.995 | 19.565 | 0.069 | 13.765 | 0.003 | 151.396 | 1.34 |
| MobileNet-V2 | 62.911 | 12.758 | 0.051 | 1.061 | 0.002 | 76.783 | 2.23 |
| ResNeXt-50 (32$\times$4d) | 94.983 | 17.089 | 0.060 | 2.906 | 0.002 | 115.040 | 2.29 |
| ResNeXt-101 (32$\times$8d) | 311.523 | 70.968 | 0.124 | 11.382 | 0.005 | 394.001 | 4.45 |
| ViT-B/16 | 188.084 | 34.659 | 0.064 | 7.694 | 0.003 | 230.503 | 0.80 |
| DeiT-B/16 | 183.044 | 35.961 | 0.067 | 18.332 | 0.002 | 237.406 | 0.80 |

For a thorough examination of the implementation, we refer the reader to our code(^4^44[https://github.com/davidgonmar/BALF](https://github.com/davidgonmar/BALF)). [Table 2](https://arxiv.org/html/2509.25136v2#A7.T2) presents the runtime breakdown and peak memory usage of our method on different models on the ImageNet dataset, evaluated on an RTX 2070 laptop. We use 8192 calibration samples (as in our main experiments), divided into batches of 64 samples. All other parameters match those used in the main experiments. We note that the peak memory usage typically occurs during the computation of activation moments, and that using smaller batch sizes can further reduce it.

The only hyperparameters that are expected to affect the runtime of our method are the number of calibration samples and the number of solver iterations. For the first, it is only expected to affect (linearly) the activation-gathering phase (Act. in [Table 2](https://arxiv.org/html/2509.25136v2#A7.T2)).

Second, the number of solver iterations affects (linearly) the time it takes to produce a solution (Solver in [Table 2](https://arxiv.org/html/2509.25136v2#A7.T2)). Its runtime is negligible (less than 0.15 seconds in the runs of this section, using 300 iterations).

Finally, we reiterate that the most expensive steps (activation moment gathering and factorization/whitening) are cached, allowing one to obtain different models with different compression configurations in a matter of seconds on average, even on lower-end hardware, after the first run. We also note that our implementation was not particularly optimized (as compression time is not a bottleneck in our evaluations), and that further runtime gains are probably attainable.

## Appendix H A Post-Hoc Interpretation via Information Geometry

Recent work by Shumaylov et al. (2025) provides interesting insights into the factorization of neural networks from an information-geometric point of view. BALF has an interesting interpretation through the lens of their framework.

Let $\widehat{{\bm{\mathsfit{W}}}}$ denote the low-rank approximation of a parameter tensor. They posit that general activation-aware factorization approaches solve an optimization problem equivalent to minimizing the squared output mismatch under an i.i.d. Gaussian noise model with identity covariance. Translating that to our framework, we obtain

$$ $\displaystyle\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}}$ $\displaystyle\;\sum_{l=1}^{L}\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})\right\|_{F}^{2}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}}$ $\displaystyle\;\sum_{l=1}^{L}\sum_{g=1}^{G_{l}}\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,:,:}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g,:,:}-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,:,:}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g,:,:}\right\|_{F}^{2}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}}$ $\displaystyle\;\sum_{l=1}^{L}\sum_{g=1}^{G_{l}}\sum_{n=1}^{N_{l}}\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n,:}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g,:,:}-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n,:}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g,:,:}\right\|_{2}^{2}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}}$ $\displaystyle\;D_{\mathrm{{KL}}}\!\bigl(p_{{\bm{\mathsfit{W}}}}\,\|\,p_{\smash{\widehat{{\bm{\mathsfit{W}}}}}\vphantom{{\bm{\mathsfit{W}}}}}\bigr),$ $$

with

$$ $p_{{\bm{\mathsfit{W}}}}(\cdot)=\prod_{l=1}^{L}\prod_{g=1}^{G_{l}}\prod_{n=1}^{N_{l}}\mathcal{N}\!\left(\cdot\,\middle|\,\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g},{\bm{I}}\right),$ $$

and

$$ $p_{\smash{\widehat{{\bm{\mathsfit{W}}}}}\vphantom{{\bm{\mathsfit{W}}}}}(\cdot)=\prod_{l=1}^{L}\prod_{g=1}^{G_{l}}\prod_{n=1}^{N_{l}}\mathcal{N}\!\left(\cdot\,\middle|\,\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g},{\bm{I}}\right).$ $$

In the preceding expressions, the $n$ index corresponds to the outer dimension (e.g., with $BH_{o}W_{o}=N$ for convolutional layers), $g$ to the groups, and $l$ to the layers. The optimization problem generally assumes ranks are fixed ($\widehat{\mathcal{M}}$ denotes a model with layers truncated to fixed ranks, assumed to be given in advance), and it is only concerned with finding the best low-rank projections.

On the other hand, the problem BALF solves can be formulated as

$$ $\displaystyle\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}:C(\widehat{\mathcal{M}})\leq C_{\max}}$ $\displaystyle\;\sum_{l=1}^{L}\frac{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})\right\|_{F}^{2}}{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({{\bm{\mathsfit{W}}}}^{l})\right\|_{F}^{2}}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}:C(\widehat{\mathcal{M}})\leq C_{\max}}$ $\displaystyle\;\sum_{l=1}^{L}\sum_{g=1}^{G_{l}}\frac{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,:,:}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g,:,:}-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,:,:}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g,:,:}\right\|_{F}^{2}}{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})\right\|_{F}^{2}}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in\widehat{\mathcal{M}}:C(\widehat{\mathcal{M}})\leq C_{\max}}$ $\displaystyle\;\sum_{l=1}^{L}\sum_{g=1}^{G_{l}}\sum_{n=1}^{N_{l}}\frac{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n,:}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g,:,:}-\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n,:}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g,:,:}\right\|_{2}^{2}}{\left\|\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({{\bm{\mathsfit{W}}}}^{l})\right\|_{F}^{2}}$ $\displaystyle=\arg\min_{\widehat{{\bm{\mathsfit{W}}}}\in C(\widehat{\mathcal{M}}),\,C(\widehat{\mathcal{M}})\leq C_{\max}}\;D_{\mathrm{{KL}}}\!\bigl(p_{{\bm{\mathsfit{W}}}}\,\|\,p_{\smash{\widehat{{\bm{\mathsfit{W}}}}}\vphantom{{\bm{\mathsfit{W}}}}}\bigr),$ $$

with

$$ $p_{{\bm{\mathsfit{W}}}}(\cdot)=\prod_{l=1}^{L}\prod_{g=1}^{G_{l}}\prod_{n=1}^{N_{l}}\mathcal{N}\!\left(\cdot\,\middle|\,\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n}\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})_{g},\left\|\big(\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})\big)\right\|_{F}^{2}\,{\bm{I}}\right),$ $$

and

$$ $p_{\smash{\widehat{{\bm{\mathsfit{W}}}}}\vphantom{{\bm{\mathsfit{W}}}}}(\cdot)=\prod_{l=1}^{L}\prod_{g=1}^{G_{l}}\prod_{n=1}^{N_{l}}\mathcal{N}\!\left(\cdot\,\middle|\,\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})_{g,n}\overline{P}_{l}(\widehat{{\bm{\mathsfit{W}}}}{}^{l})_{g},\left\|\big(\overline{I}_{l}({\bm{\mathsfit{X}}}^{l-1})\overline{P}_{l}({\bm{\mathsfit{W}}}^{l})\big)\right\|_{F}^{2}\,{\bm{I}}\right).$ $$

There are two main differences. First, BALF also works under the Gaussian assumption, but with the additional nuance that the variances depend on the per-layer output magnitude (in terms of the squared Frobenius norm). Second, the optimization problem is not only concerned with the low-rank projection scheme given a fixed rank per layer; the selection of ranks under some complexity budget is, itself, baked into the optimization problem. (Note that, strictly speaking, BALF solves the problem approximately in practice.)

We believe this point of view might be an interesting avenue for future research.

## Appendix I Missing Proofs

In this section, we provide full proofs for the results in the main text that were not covered in other sections of the appendix. Results are restated for the convenience of the reader.

### I.1 Proof of Theorem 2

See [2](https://arxiv.org/html/2509.25136v2#Thmtheorem2)

###### Proof.

First, observe that

$$ $\|\mathsf{cat}({\bm{A}}_{1},\ldots,{\bm{A}}_{K})\|_{F}^{2}=\sum_{i=1}^{K}\|{\bm{A}}_{i}\|_{F}^{2},$ (4) $$

where $\mathsf{cat}$ denotes the operation that yields a third-order tensor from a batch of matrices (of the same shape). Therefore, for a batch of distortion matrices, we may analyze each term independently and sum their contributions.

Moreover,

$$ $\displaystyle\bigl\|f({\bm{\mathsfit{X}}};{\bm{\mathsfit{W}}})-f\bigl({\bm{\mathsfit{X}}};\mathcal{T}^{\mathrm{AA}}_{P}\bigl({\bm{\mathsfit{W}}}\bigr)\bigr)\bigr\|_{F}$ $\displaystyle=\bigl\|\overline{O}\!\bigl(\overline{I}({\bm{\mathsfit{X}}})\,\overline{P}({\bm{\mathsfit{W}}})\bigr)-\overline{O}\!\bigl(\overline{I}({\bm{\mathsfit{X}}})\,\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\bigr)\bigr\|_{F}$ $\displaystyle=\bigl\|\overline{I}({\bm{\mathsfit{X}}})\,\overline{P}({\bm{\mathsfit{W}}})-\overline{I}({\bm{\mathsfit{X}}})\,\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\bigr\|_{F},$ $$

since $\overline{O}$ is a composition of reshapes and permutes, which preserve the Frobenius norm.

Fix a group index $g$ (omitted from the notation below for simplicity). For reference, the dimensions of the different elements are: $\overline{I}({\bm{\mathsfit{X}}})\in\mathbb{R}^{N\times I}$, ${\bm{M}}\in\mathbb{R}^{I\times I}$, ${\bm{M}}^{+}\in\mathbb{R}^{I\times I}$, and $\overline{P}({\bm{\mathsfit{W}}})\in\mathbb{R}^{I\times O}$.

When using activation-aware factorization and truncating to rank $P$, it follows that

$$ $\displaystyle\ell_{(g)}^{\mathrm{activ}}(P)$ $\displaystyle=\frac{1}{B}\left\|\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})-\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{M}}^{+}\overline{P}({\bm{\mathsfit{W}}})\bigr)\right\|^{2}_{F}$ $\displaystyle\underset{(a)}{=}\frac{1}{B}\left\|\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}\left({\bm{M}}^{+}\overline{P}({\bm{\mathsfit{W}}})-\mathcal{T}^{\mathrm{SVD}}_{P}\bigl({\bm{M}}^{+}\overline{P}({\bm{\mathsfit{W}}})\bigr)\right)\right\|_{F}^{2}$ $\displaystyle=\frac{1}{B}\left\|\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}{\bm{U}}{\bm{\Sigma}}{\bm{V}}^{T}\right\|^{2}_{F}$ $\displaystyle=\frac{1}{B}\operatorname{tr}\left({\bm{V}}{\bm{\Sigma}}{\bm{U}}^{T}{\bm{M}}^{T}\overline{I}({\bm{\mathsfit{X}}})^{T}\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}{\bm{U}}{\bm{\Sigma}}{\bm{V}}^{T}\right)$ $\displaystyle\underset{(b)}{=}\frac{N}{B}\operatorname{tr}\left({\bm{V}}{\bm{\Sigma}}^{2}{\bm{V}}^{T}\right)$ $\displaystyle=\frac{N}{B}\sum_{i=P+1}^{U}\bm{\sigma}_{i}^{2},$ $$

where the SVD is taken of ${\bm{M}}^{+}\overline{P}({\bm{\mathsfit{W}}})$.

For step (a), note that $\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}{\bm{M}}^{+}=\overline{I}({\bm{\mathsfit{X}}})$ (see [Section 3](https://arxiv.org/html/2509.25136v2#S3)).

For step (b), note the following. First, recall from the definition of ${\bm{M}}$ (see [Section 3](https://arxiv.org/html/2509.25136v2#S3)) that ${\bm{M}}^{T}\overline{I}({\bm{\mathsfit{X}}})^{T}\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}=N\overline{{\bm{I}}}_{R}$ (where $R$ is the rank of $\overline{I}({\bm{\mathsfit{X}}})$); and that ${\bm{M}}$ has its last $I-R$ columns set to 0, meaning ${\bm{M}}^{+}$ has its last $I-R$ rows set to 0.

This results in ${\bm{U}}^{T}{\bm{M}}^{T}\overline{I}({\bm{\mathsfit{X}}})^{T}\overline{I}({\bm{\mathsfit{X}}}){\bm{M}}{\bm{U}}=N{\bm{U}}^{T}\overline{{\bm{I}}}_{R}{\bm{U}}=N\overline{{\bm{I}}}_{R}$ (this last equality follows from the fact that ${\bm{M}}^{+}\overline{P}({\bm{\mathsfit{W}}})\in\mathbb{R}^{I\times O}$ has its last $I-R$ rows set to 0 since ${\bm{M}}^{+}$ does).

From the observation in [Equation 4](https://arxiv.org/html/2509.25136v2#A9.E4), it follows that the total distortion is the sum across each group’s distortion

$$ $\ell^{\mathrm{activ}}(P)=\sum_{g=1}^{G}\ell_{(g)}^{\mathrm{activ}}(P)=\frac{N}{B}\sum_{g=1}^{G}{\sum_{i=P+1}^{U}\bm{\sigma}_{g,i}^{2}},$ $$

which completes the proof.

∎

###### Proof.

### I.2 Proof of Theorem 1

###### Lemma 1 .

Let ${\bm{Y}}\in\mathbb{R}^{M\times N}$ satisfy ${\bm{Y}}^{T}{\bm{Y}}=\alpha{\bm{I}}_{N}$ with $\alpha>0$. Then, for any ${\bm{X}}\in\mathbb{R}^{N\times K}$ and any $P\geq 1$,

$$ $\sum_{i=1}^{P}\bm{\sigma}_{i}({\bm{Y}}{\bm{X}})=\sqrt{\alpha}\sum_{i=1}^{P}\bm{\sigma}_{i}({\bm{X}}).$ $$

###### Proof.

We have

$$ $({\bm{Y}}{\bm{X}})^{T}({\bm{Y}}{\bm{X}})={\bm{X}}^{T}({\bm{Y}}^{T}{\bm{Y}}){\bm{X}}=\alpha{\bm{X}}^{T}{\bm{X}}.$ $$

Hence, the (nonzero) eigenvalues of $({\bm{Y}}{\bm{X}})^{T}({\bm{Y}}{\bm{X}})$ are exactly $\alpha$ times those of ${\bm{X}}^{T}{\bm{X}}$, so $\bm{\sigma}_{i}({\bm{Y}}{\bm{X}})=\sqrt{\alpha}\bm{\sigma}_{i}({\bm{X}})$ for all $i$. Summing the top $P$ singular values gives the claim.
∎

See [1](https://arxiv.org/html/2509.25136v2#Thmtheorem1)

###### Proof.

It suffices to prove the result for a single group. Fix a group (its index is omitted from the notation for brevity), and let ${\bm{W}}=\overline{P}({\bm{\mathsfit{W}}})\in\mathbb{R}^{I\times O},{\bm{X}}=\overline{I}({\bm{\mathsfit{X}}})\in\mathbb{R}^{N\times I}$, ${\bm{M}}\in\mathbb{R}^{I\times I}$, and ${\bm{M}}^{+}\in\mathbb{R}^{I\times I}$ denote the items corresponding to said group, and denote the rank of ${\bm{X}}$ by $R$.

We recall the following facts; see [Section 3](https://arxiv.org/html/2509.25136v2#S3). First, ${\bm{M}}$ has its last $I-R$ columns set to 0, which forces ${\bm{M}}^{+}$ to have its last $I-R$ rows set to 0. Hence, ${\bm{M}}{\bm{M}}^{+}={\bm{M}}{\bm{I}}_{:,:R}{\bm{I}}_{:R,:}{\bm{M}}^{+}$. Moreover, from the definition of ${\bm{M}}$, we have ${\bm{M}}^{T}{\bm{X}}^{T}{\bm{X}}{\bm{M}}=N\overline{{\bm{I}}}_{R}$, and therefore, it holds that ${\bm{I}}_{:R,:}{\bm{M}}^{T}{\bm{X}}^{T}{\bm{X}}{\bm{M}}{\bm{I}}_{:,:R}=N{\bm{I}}_{R}$. It is also true that ${\bm{X}}{\bm{M}}{\bm{M}}^{+}={\bm{X}}$.

It follows that

$$ $\sum_{i=1}^{P}\bm{\sigma}^{2}_{i}({\bm{X}}{\bm{W}})=\sum_{i=1}^{P}\bm{\sigma}^{2}_{i}({\bm{X}}{\bm{M}}{\bm{M}}^{+}{\bm{W}})=\sum_{i=1}^{P}\bm{\sigma}^{2}_{i}({\bm{X}}{\bm{M}}{\bm{I}}_{:,:R}{\bm{I}}_{:R,:}{\bm{M}}^{+}{\bm{W}})=N\sum_{i=1}^{P}\bm{\sigma}^{2}_{i}({\bm{M}}^{+}{\bm{W}}),$ $$

where the last equality follows from [Lemma 1](https://arxiv.org/html/2509.25136v2#Thmlemma1).

Hence,

$$ $\displaystyle\|\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})-\mathcal{T}^{\mathrm{SVD}}_{P}\bigl(\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})\bigr)\|^{2}_{F}$ $\displaystyle=\sum_{i=P+1}^{U}\bm{\sigma}_{i}^{2}({\bm{X}}{\bm{W}})$ $\displaystyle=N\sum_{i=P+1}^{U}\bm{\sigma}_{i}^{2}({\bm{M}}^{+}{\bm{W}})$ $\displaystyle=\|\overline{I}({\bm{\mathsfit{X}}})\overline{P}({\bm{\mathsfit{W}}})-\overline{I}({\bm{\mathsfit{X}}})\mathcal{T}^{\mathrm{AA}}_{P}\bigl(\overline{P}({\bm{\mathsfit{W}}})\bigr)\|^{2}_{F},$ $$

where the last equality follows from [Theorem 2](https://arxiv.org/html/2509.25136v2#Thmtheorem2).

Taking square roots completes the proof.
∎

###### Lemma 1 .

###### Proof.

###### Proof.

## Appendix J LLM Usage

In this section, we disclose the role of LLMs in this work.

First, we used LLMs to help identify related work and select comparison baselines (in addition to our manual efforts). The content itself was read by the authors; LLMs primarily assisted with search. We also used them to discuss research ideas and to verify the correctness and coherence of our claims.

LLMs were used to bootstrap, modify, and evaluate code (including checking for correctness). They were particularly useful for iterating on result-visualization scripts (e.g., code that takes experimental outputs and produces plots or table components).

Finally, LLMs were used to evaluate and improve the presentation and writing quality of various drafts (including equation formatting), to suggest missing explanations or experiments, and to assess the overall coherence of the text.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: The theoretical bounds depend on Lipschitz constants. Can't they be too high in practice?
soundness: 3
strengths: Strengths

Conceptual innovation - 
BALF reframes low-rank compression through an activation-centric lens, ensuring that the projection minimizes functional output distortion rather than raw parameter deviation. This distinction aligns compression with representational behavior.

General framework-The authors generalize activation-aware SVD to a more general expressible layer, encompassing dense, convolutional, and grouped convolutional layers in a unified algebraic formulation.

Mathematical rigor: Theorems 1 and 2 show equivalence between activation aware and direct output truncation schemes, with a closed-form expression for activation distortion in terms of singular values.

I especially likes the rank allocator that transforms a combinatorial multiple choice knapsack problem into a linear-time Lagrangian relaxation, which enables control over global compression budgets.

Empirical performance: The improvement is consistent compared to standard SVD baselines, maintaining high accuracy at large compression ratios.

Reproducibility and presentation:
Implementation details, pseudocode, and open-source repository are provided, which enable accessibility for less theory researchers.
summary: The paper suggests a unified and efficient fine-tuning-free compression framework for deep neural networks that combines activation-aware low-rank factorization with a budgeted rank allocation mechanism.

Traditional low-rank factorization techniques reduce model size by truncating singular values of layer weight matrices, but they often require fine-tuning or heuristic search to recover accuracy. BALF addresses this by (1) extending activation-aware decomposition to general layer types (including grouped convolutions) and (2) introducing a scalable, zero-overhead budgeted allocator that meets user-specified parameter or FLOPs constraints without retraining.

The method computes uncentered whitening matrices of layer activations to perform activation aware SVD, minimizing output distortion rather than parameter distortion. A Lagrangian relaxation formulation then determines per-layer rank allocations to meet global resource budgets efficiently.

Empirical results are also provided, outperforming existing fine-tuning-free baselines in both accuracy and runtime. On an RTX 2070, compression of ImageNet-scale models completes in minutes without fine-tuning.
weaknesses: - The proof assumes ungrouped layers and bounded Lipschitz constants without empirical validation.
- The theoretical bounds may be too loose for larger networks.
- The method is not so strong on already-optimized architectures (e.g., MobileNet-V2). The adjustment might be adapted for each architecture.
The description of FLOP estimation and calibration sampling could be expanded for full reproducibility.

---

confidence: 5
contribution: 1
initial_rating: 2
presentation: 2
questions: Please see the Weaknesses.
soundness: 2
strengths: + The motivation of this paper is clear, i.e., to select the optimal ranks using SVD to factorize the matrices of the target DNN models.
+ This paper provides sufficient theoretical analysis for the proposed method.
+ The proposed method shows effectiveness on several CNN models in the experiments.
summary: This paper presents BALF, an efficient pipeline for compressing models using low-rank matrix decomposition without fine-tuning.
weaknesses: - Limited novelty. The idea of compressing neural networks through matrix factorization is long-established, and many prior works have investigated rank selection strategies. In addition, the use of augmented Lagrangian formulations for enforcing low-rank constraints in network compression has already been explored [R1]. The paper does not seem to introduce a fundamentally new principle or formulation.

- Lack of large-scale validation. Although the motivation centers on compressing large models, no experiments are conducted on modern large-scale architectures such as LLMs. Instead, the evaluation focuses on relatively small and dated models (e.g., ResNet, ResNeXt, ViT). Given the large body of existing work applying matrix factorization to such models, the contribution appears incremental.

- Missing comparison with alternative compression methods. The paper does not compare its method against other major compression approaches, such as pruning or quantization, making it difficult to assess the relative performance and practicality of the proposed technique.

- Insufficient baselines in experiments. The set of comparison methods in the experimental section is very limited, and several state-of-the-art approaches, e.g., [R1] mentioned in the related work section, are not included in the empirical evaluation. This weakens the paper’s claim of superiority or generality.

- Unvalidated theoretical claims. Although the paper presents several theoretical results, the lack of strong experimental evidence and comprehensive comparisons makes it hard to verify the practical value of these theoretical contributions.

[R1] Low-rank compression of neural nets: Learning the rank of each layer.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: - Can BALF be composed with post-training quantization?
- When applying BALF to ResNet families, do you fuse BN into Conv before factorization, or do you factorize Conv and then correct BN statistics?
soundness: 3
strengths: - General, principled activation-aware factorization applicable to grouped convolutions, not just FC layers; the method uses whitening to minimize expected layer-output distortion.
- Budget-aware rank allocation: sets layer-wise ranks to meet FLOPs or parameter budgets via a Lagrangian relaxation.
- Broad empirical coverage across CNNs and ViTs, with competitive fine-tuning-free trade-offs.
summary: This paper introduces BALF, a fine-tuning-free compression pipeline that unifies activation-aware low-rank factorization across fully connected, convolutional, and grouped-convolution layers, and couples it with a lightweight budgeted rank allocator based on Lagrangian relaxation that directly targets user-specified FLOPs or parameter budgets. The method estimates uncentered-whitening transforms from a small calibration set, performs truncated SVD in the whitened domain to minimize expected layer-output distortion, and then selects per-layer ranks globally to satisfy a compute/size constraint, replacing layers with efficient two-stage low-rank modules only when they yield real savings.
weaknesses: - Innovation scope: activation-aware factorization is not entirely new; the paper’s novelty lies in generalizing it to conv/grouped-conv and coupling it with an efficient budgeted allocator. This is a strong systems contribution but less of a theoretical breakthrough.
- The model-level bound (Theorem 3 in the main text; formal version in the appendix) is derived for a sequential L-layer network and does not explicitly handle residual/skip connections typical of ResNets. While the empirical section includes ResNet-family models, the paper does not clarify how residual blocks are treated in the theoretical bound (e.g., block-level aggregation or additional terms due to additive skips) nor in the practical error accounting within a residual structure.


Decision: Reject