CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
Authors: XUCHEN FENG
Siyu Liao
Venue: NeurIPS 2025 poster ()
Keywords: Normalizing Flows; Fast Fourier Transform (FFT); Matrix Inversion; Jacobian Determinant
Primary area: deep_learning

Abstract: Normalizing flows are deep generative models that achieve efficient likelihood estimation and sampling through invertible transformations. A key challenge is designing linear layers that enhance expressiveness while enabling efficient computation of the Jacobian determinant and inverse. In this work, we introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition provides a parameter- and computation-efficient formulation, reducing the parameter complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$ by using $m$ diagonal matrices together with $m-1$ circulant matrices, while approximating arbitrary linear transformations.Furthermore, leveraging the Fast Fourier Transform (FFT), our method reduces the time complexity of matrix inversion from $\mathcal{O}(n^{3})$ to $\mathcal{O}(mn \log n)$ and matrix log-determinant from $\mathcal{O}(n^{3})$ to $\mathcal{O}(mn)$, where $n$ is the input dimension. Building upon this, we introduce a novel normalizing flow model called Circulant-Diagonal Flow (CDFlow). Empirical results demonstrate that CDFlow excels in density estimation for natural image datasets and effectively models data with inherent periodicity. In terms of computational efficiency, our method speeds up the matrix inverse and log-determinant computations by $1.17\times$ and $4.31\times$, respectively, compared to the general dense matrix, when the number of channels is set to 96.

CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
Xuchen Feng
Siyu Liao"
School of Integrated Circuits
School of Integrated Circuits
Sun Yat-sen University f Tnngxch@mail2. sysu. edu.
Sun Yat-sen University
1 1iaosy366mail. sysu. edu. cn
Abstract
Normalizing flows are deep generative models that achieve efficient likelihood estimation and sampling through invertible transformations. key challenge designing linear layers that enhance expressiveness while enabling efficient compu- tation the Jacobian determinant and inverse. In this work, we introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition provides a parameter- and computation-efiiee ing the parameter complexity from (n²) to (mn) ((m)) by using diagonal matrices together with m circulant matrices, while approximating arbitrary linear trans- formations. Furthermore, leveraging the Fast Fourier Transform (FFT), method reduces the time complexity of matrix inversion from (nz)) to O (mm log n) and matrix regddetemmnnat from (n²²) O(m)), where is the input dimension. Building upon this, we introduce a novel normalizing flow model called Circulant- Diagonal Flow (CDFlow). Empirical results demonstrate that CDFlow excels in density estimation for natural image datasets and effectively models data with inherent periodicity. In terms of computational efficiency, our method speeds up the matrix inverse and log- log-determinant computations by 1.17x and 4.31x, respec- tively, compared to the general dense matrix, when the number of channels is set 96.
formulation, reduc-
Introduction
Deep generative models have gained significant attention due their potential applications across various domains, including natural language processing [Dong et al. 2024], image restoration and generation Yao al. 2023 robot learning [Chao 2024 and autonomous driving [Guan et al.) 2024 Among these models, normalizing flows hold a distinctive position because of their ability to provide accurate likelihood estimation and efficient sampling.
Flow-based models enable the probabilistic modeling of complex distributions by transforming them into simpler
through series of invertible mappings. Their uniqueness lies in the striet requirements for the construction of these functions: on one hand, the functions must be invertible; on the other hand, the determinant of the corresponding Jacobian matrix should be easy to compute. However, these constraints may limit the expressive power of the models.
To address this challenge. NICE (Dinh et al. 2014) introduced additive coupling layers, which were later extended by RealNVP [Dinh et al. 2016)] to affine coupling layers. These coupling layers have since become fundamental component of most flwedbaeed models. However, the design of these non-lineat layers, which only transform a subset of variables, requires coordination with linear layers to effectively capture dependencies across arbitrary dimensions. As result, recent
Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

efforts have focused on developing specialized linear layer designs, ranging from fixed channel- disrupting operations (Dinh et al. 2014 2016] and even d x d convolutions [Hoogeboom et al., 2019] with learnable parameters. However, dense convolutions can result significant memory overhead as result of the large weight matrices required for high-dimensional inputs. One approach to address this is the construction of reversible butterfly layers [Meng et al., 2022] using butterfly matrices [Dao et al.) 2019], which help maintain the model's expressiveness while reducing the number of parameters.
x convolutions [Kingma and Dhariwal 2018)
Structured matrices and Fourier transform acceleration have been demonstrated in numerous studies [Cheng et al. [013 Ding et al. 2017 Karami et al.) [019 Jensen et al. [022]] to be highly effective in reducing the number of parameters and computational complexity of neural networks. Research by [Huhtanen and Perämäki 2015 demonstrates that any matrix M E R can be represented as the alternating product of a cyclic matrix and diagonal matrix, requiring no more than 2n factors. This factorization structure has demonstrated versatility and effectiveness in domains such as deep neural networks [Moczulski al., 2015 Araujo et al.) 2019) and parameter-effi efficieett fine-tuning Ding et al. 2025 However, these studies have not focused on model reversibility.
Inspired by previous work, we propose a novel normalizing flow model called Cireulant-Diagnnal Flow (CDFlow), which is built on invertible linear layers constructed using circulant and diagonal matrices. This construction preserves the expressive power the model while significantly reducing storage demands, as only the diagonal and circulant elements need to be stored instead of the full n weight matrix. In addition, the distinct properties of circulant and diagonal matrices enable the use of the Fast Fourier Transform (FFT) to enhance computation efficiency (Ding et al. (025). This facilitates faster calculations for both the determinant of Jacobian matrices and the inversion of weight matrices. Overall, the main contributions of our work can be summarized following:
We propose CDFlow, new class of flow-based generative models. The linear layer in CDFlow is designed to have a storage complexity that scales linearly with the input dimen- sion, it only requires storing collection of circulant and diagonal vectors namely, the diagonal elements of diagonal matrices and the eigenvalue vectors of circulant matrices. In practice, the linear layer weights be constructed using small number of circulant and diagonal matrices, enabling efficient transformations across both spatial and channel dimensions.
We leverage the unique properties of circulant and diagonal matrices to significantly reduce computational complexity. key property that the determinant any diagonalizable matrix equals the product of its eigenvalues. Accordingly. by performing an On log FFT once during initialization to precompute the eigenvalues of circulant matrices, we enable (n) log-determinant evaluation at runtime. Additionally, the complexity of matrix inversion is reduced to On log n).
We demonstrate through image generation experiments that our CDFlow performs effectively while also offering superior computational efficiency in terms of both log -og-determinant computation and matrix inverse.
2 Background
2.1 Flow-based Models
The normalizing flow model maps the complex data distribution px (z) to a known simple probability distribution pz (2( via invertible transformation fe + for data generation with Ó being the model parameter. The log-likelihood log po (x) can be obtained according to the change of variables formula:
log po log Pz log det( Ofe dx log pzp log det (J,, (f (2)))
where (.) calculates the Jacobian matrix of the transformation fo(x), and det(-) computes the matrix determinant.

---

It should be noted that fo can be composed of multiple layers in the case of using deep learning models, and each layer needs to be an invertible function guarantee the invertibility of fo:
fo(z) = f1 f.. (1)),,
(2)
6 o (z),
where o is for function composition and there are L layers in the model. According to the chain rule in differentiability, the determinant of fo is now transformed into layer-wise computation:
f 8x
0f dr
Of: 0f:-1
log
log
log
(3)
2=2
Therefore, it is important to have an efficient log-determinant computation when training normalizing flow models. When fa is a linear layer with weight matrix W, its Jabobian matrix is equivalent the weight matrix:
0f: log det W).
log det
44)
0f
where the determinant computation complexity is (
Normalizing flow model can perform data generation using fo by feeding with samples from (z).). Therefore, it is important to have efficient fo to achieve fast data generation. In particular, when fa is a linear layer, its inverse is equivalent to the linear transformation built on top of inverse of weight matrix W:
(x) w- x x
In general, the matrix inverse can take (n²) computation complexity.
Overall, log-determinant computation needs to be fast for training normalizing flow models, while data generation requires fast inverse computation.
Nonlinear Coupling Layer. The coupling layer [Dinh et al. 2014 2016] is a simple and effective invertible transformation, where the Jacobi matrix is triangular matrix. It divides the input into two parts Ta and æs, then transforms 26, using scaling parameter sæ and a translation parameter be that can be nonlinear function of Te, while 2q directly equal to Fa. and the final output concatenates the two parts together:
Za Ta
① + bg. = = concat Zn, 2b)
Coupling layers remain crucial component in most flow-basd models (Kingma and Dhariwal 2018 [Hoogeboom et al. 2019 Meng et al., [022]. However, since coupling layers only transform subset of the variables, they must be combined with linear layers facilitate information fusion across channels. As a result, recent research has increasingly focused on the design linear layers to enhance model performance.
Invertible Linear Layer. A well-designed linear layer can significantly enhance both the coupling layer's ability to learn the target distribution and the overall expressive power of the model. Further- more, the computational cost associated with matrix inversion and determinant computation serves as an important indicator of the efficiency of linear layer design.
Glow Kingma and Dhariwal 2018) pioneered the use of x convolution with learnable parameters linear layer in flow-based models. This method is more effective at capturing correlations between variables compared to simpler channel shuffling operations [Dinh et al. 2014 2016). Additionally, Glow introduces a weight design based LU decomposition, reducing the Jacobian determinant computation from O(n²) to (n).
Similarly. (Hoogeboom et al. 2019 extended convolution to d x d and proposed periodic convolution, leveraging the Fast Fourier Transform (FFT) to accelerate the convolution process. However, the periodic convolution method involves generalized determinant computation, leading to complexity of O(r²). contrast, our proposed method not only accelerates the convolution process using FFT, but also reduces the determinant computation complexity to (n).
3

---

Despite numerous works exploiting special matrix properties to enhance model expressiveness, most focus on improving the efficiency of the Jacobian determinant computation, with limited optimization of the matrix inversion process during the sampling phase of the flow model. In these methods, including the aforementioned works [Kingma and Dhariwal 2018 Hoogeboom et al., 2019 Meng et al. 2022]. the computational complexity for the inverse sampling operation is typically On The Woodbury Transformation [Lu and Huang 2020) achieves a theoretical complexity of O (dn). where d is the dimensionality of smaller matrices (e.g., 8 or 16). Although this is better in theory than the (n log r) complexity of our method, in practice Woodbury requires multiple input transformations, which makes its actual efficiency inferior ours.
2.2 Weight Matrix Construction and Fast Multiplication
general matrix M gax" can be factorized into the alternating product of circulant and diagonal matrices, with the total number of matrices not exceeding 2n - [Huhtanen and Perämäki, 2015). Specifically, it can be written as:
D D2n-D2n--
where D2j-1 and C2j are diagonal and circulant matrix for j 1,2,..., This decomposition can theoretically approximate any dense matrix, and the approximation error can be controlled by adjusting the number of factors. (Ding et al. 2025 have successfully applied such representation to fine-tuning large language models.
Denote the imput vector as x E Ra. The weight matrix W R can be represented by diagonal matrices and m - 1 circulant matrices, thereby m < n. For each E {1,2..... m), the j-th diagonal matrix is defined by the vector d2j-1 R=x1,, and the j-th circulant matrix is defined by the vector C2j E Rax1, Thus, the model weight matrix can be represented as:
W = D2C
diag(d,) circ(c2) x... diag diag(d2m-1).
(10)
where diag(-) and circ(-) represent the construction of diagonal and circulant matrices, respectively. It should also be noted that circulant matrix can be factorized into:
circ (C2)) = F x diag(C2;) x F, C22 = F X C2j.
(11)
where F is the order n discrete Fourier transform matrix. In practice, is better to implement with the frequency domain parameter č2ÿ as weight parameter rather than real valued caj save computation costs in matrix vector product, logarithm determinant, and matrix inversion.
3 Invertible Linear Layer Design
In this section, introduce our invertible linear layer that consist of circulant and diagonal matrices. We discuss the fast algorithms for the matrix vector product, logarithm matrix determinant and matrix inversion. Additionally, we also analyze the theoretical computation cost associated with these operations.
3.1 Matrix Log-determinant
According to Eq. 10 The weight matrix is constructed as an alternating product of diagonal and circulant matrices. The determinant of this matrix can be efficiently computed by leveraging the special properties of these matrices. Recall that determinant of matrix product is equal to product of matrix determinant:
det(W)= = II det D 2j
II det (C2j),
(12)
det( = det (diag (23- (2))-))) d2j-1
(13)
det(C2)) = det(F= (et(diag(()))) det (F)
2
(14)

---

Table Comparisons of computational complexities and parameter counts, where n is the number of input channels. In practice, d is the dimensionality of the smaller matrices used in the Woodbury transformation (typically d 8 or 16), is the number of butterfly layers, and is generally set to 2 for our method. Here h and w denote the spatial dimensions of the feature maps.
Method
Logdet
Inverse
Params
1 x Conv Kingma and Dhariwal 2018
(
(
(n²))
x 1 Conv (LUU) Kingma and Dhariwal
2018
((n)
O
(n
Emerging Hoogeboom al. Periodic Hoogeboom al.
2019
(n)
O(n²)
O(n²)
2019
O(n")
O(n²)
O
Woodbury [Lu and Huang
2020
((nd)
O(nd)
Old(n + h- h-w))
ME- Woodbury [Lu and Huang
g
2020
((nd)
Onnd
O + w))
ButterflyFlow [Meng ct al.
2022
O(n)
Onn))
(
CDFlow (Ours)
(mm)
(mn logn)
O
To compute the logarithm of the matrix determinant of W, we can express into the sum of each matrix determinant:
log det det(W)]
log E2y
ΣΣ
log d22j-1
(15)
i=1
Therefore, the overall log-determinant computation can be finished by simple summation with complexity of O(mm).
3.2 Matrix Inverse
The inversion of the weight matrix for the data sampling process can efficiently implemented by leveraging the decomposition of the weight matrix into alternating diagonal and circulant matrices. This section introduces the inverse calculation process, including its objective, step-by-step procedure, and computational complexity. The objective of the matrix inversion is to recover the input vector from the output y, given the weight matrix W. This achieved by solving the equation:
x= x=W-1 W-1 x,,
(16)
where the inverse of W is represented as:
W- = D 2m x ... DT'
w
(17)
Diagonal Matrices. For each diagonal matrix D its inverse is obtained by taking the reciprocal of its diagonal elements.
D 2 diag(1 (ifag(1//g--1 1/ 1/c2-11 1 1/d5j11),
(18)
which has a computation complexity of (n)).
Circulant Matrices. according to Eq. (T)
For each circulant matrix Cj, its inverse is similar to diagonal matrix
22 F x diag (Czy) F
(19)
x diag(1 (1/62),...... 1/25,) x F.
can be seen that matrix inversion W- can be immediately determined given d2j-1 and caj- Therefore, the complexity of Eq. (T6) is equivalent to matrix vector product complexity, i.c.,
Omm n).
3.3 Generative Modeling with CDFlow
We present our CDFlow model that consists of L. blocks, where each block is formed by stacking K layers of flow modules. Each flow module consists of the ActNorm layer, the invertible convolutional layer, and the coupling layer, working together to transform the input data. The multi-scale structure helps the model learn complex relationships within the data by processing it at different levels of abstraction. Figure iillustrates how the flow module constructed for each layer and how the entire model is repeated and integrated to learn and generate the target data.
5

---

4
ID-Convolution Layer
Element wise multiplication
Flow
x
Coupling Layer
Do
do
Squeeze
CD-Convolution Layer (using circulant and diagooal matrices)
Split
FFT
C1
21
9s
IFFT
Flow
X
D2
y%
Actnorm Layer
Squeeze
Weight Decomposition
x (L-1)
W Dg G D; (dgg(d,) cir((e)) x diag(d.)
Flow Module
Multi-scale Architecture
Figure Architecture the CDFlow Model. The left portion shows a detailed schematic of the proposed CD-Convolution layer, which is constructed using circulant and diagonal matrices. The middle portion illustrates the internal structure of flow module, where the input sequentially passes through an Actnorm layer[Kinge Kingma and Dhariwal 2018), the CD Convolution layer, and coupling layer[Dinh et al. 2016]. The right portion presents the overall CDFlow model architecture, which the flow module is repeated K times and integrated into multi-scale framework (Dihh et al. 2016 to effectively fuse channel-wist wiie information.
Actnorm Layers. The ActNorm layer (Kingma and Dhariwal 2018 is type of normalization technique that uses affine transformations to perform class normalization. Unlike traditional methods like BatchNorm [loffe and Szegedy 2015]. which rely the statistics of the entire batch, the ActNorm layer initializes its parameters based on the statistics of the first batch. Once initialized, these parameters small batch sizes, as avoids the performance degradation that BatchNorm often encounters when dealing with small batches. ActNorm helps in stabilizing the training of normalizing flows and is especially useful for flow-based models that require flexible scaling and translation invariance.
treated learnable variables. This initialization method is advantageous for
Invertible Convolutional Layer. In CDFlow, the CD-Convolution layer leverages special weight matrices, specifically circulant and diagonal matrices, to enable efficient computation. This design significantly reduces the computational cost of matrix inversion and the computation of the Jacobi determinant, which are essential operations for flow-based models. A current limitation of our approach is its reliance A detailed explanation of this design can be found in Section
I convolution, and our future work will extend to d d convolutions.
Coupling Layers. The coupling layer [Dinh et al. 2014 2016)
core component of normalizing flow models. The coupling layer splits the input into two parts: one part is passed through neural network to compute transformation, while the other part left unchanged. This operation enables the model to learn complex transformations while maintaining a simple and computationally efficient structure. The coupling layer combined with the reversible convolutional layer form the flow module, allowing the CDFlow model to capture complex distributions and generate high-quality samples.
Multi-scale architecture. CDFlow incorporates multi-scale architecture Dinh et al. 2016) which improves the model's ability to capture complex and high-dimensional data distributions. Within each block, the split and squeeze operations play an important role:

---

The split operation divides the input tensor along the channel dimension, which allows the model independently process different features. This enhances the model's expressive power by enabling it to focus on diverse aspects of the data.
The squeeze operation reduces the spatial resolution of the data by downsampling the spatial dimensions. This operation allows the model to focus on higher-Ievel features while also reducing computational cost by focusing on the most important data representations.
4 Experiments
In this section, we present the experimental results to evaluate the performance of the proposed method. The experiments are conducted in three main parts:
1. We first evaluate the method on commonly used natural image datasets to demonstrate its general effectiveness under standard conditions.
2. Next, we apply the method to datasets with specialized structures to investigate its perfor- mance on data exhibiting periodic patterns.
3. Finally, we conduct runtime analysis to assess the computational efficiency of the method.
The following subsections present the detailed results and discussions for each of these experiments.
We further investigated the effectiveness of the proposed linear layer within the Flow Matching model, and the results can be found in Appendix C
4.1 Experimental Settings and Model Parameters
Datasets and Baselines. To compare our method with previous approaches, we evaluate its per- formance on the CIFAR-10 (Krizhevsky] [009] and ImageNet [Deng et al. [009] datasets, using bits per dimension (BPD) the evaluation metric. BPD is commonly used measure for assessing the quality of generative models, indicating the number of bits required to encode each dimension of the data. Metries such as IS and FID are not used in this work, as they do not generalize well across different datasets and fail to account for overfitting. For the baseline. we select several models related to CDFlow, including MAF [Papamakarios et al.] [017]. Real NVP [Dinh et al., [0166]. Glow KKngmaa and Dhariwal 2018). Emerging [Hoogeboom et al. 2019). i-ResNet [Behrmann et al. 2019) Woodbury [Lu and Huang 2020 and ButterflyFlow [Megg et 2022
Implementation Details. To facilitate fair comparison with prior architectures, we adopt con- sistent backbone settings for flow-based baselines that share similar multi-scale design, including Glow, Woodbury, Emerging, and ButterflyFlow. On standard images datasets, all of these models are configured with three blocks and 32 flow steps. On the structured dataset, we instead adopt a lighter configuration with two blocks and eight flow steps. For Woodbury, configuration with d, = de = 16, while for ButterflyFlow we use the default setting with butterfly level = 1. For proposed CDFlow, we circulant vector.
adhere the original
2, meaning we use diagonal vectors and
During training, we apply spectral normalization MMyato et al. 2018 to the convolutional layers to ensure stability. Thanks to the structure of circulant and diagonal matrices, the maximum singular value of each weight matrix can be efficiently computed and used to rescale the weights. In addition, we employ a channel-aware learning rate scaling for the CD-Convolution parameters, which prevents abrupt updates to the structured matrices. Empirically, this combination further stabilizes the training of CDFlow.
Hyperparameter Selection Each linear transform in method is decomposed into 2m 1 matrices (i.e., m diagonal and m - circulant). While larger m values increase the approximation power, they also introduce additional parameter and computational costs. The key objective is to balance expressiveness, efficiency, and runtime performance. To this end, we systematically evaluate the effect of varying m model behavior.
As shown in Figure a()). increasing m on CIFAR-10 leads to only marginal improvements in expressiveness, as reflected by BPD values, while significantly increasing the parameter count of the
7

---

10 Convict Farmms and BPD urder dffereet
Foraard
348
Figure Ablation study on the hyperparameter m. Left: parameter counts (K) of the CD-Convolution layers and BPD CIFAR-10. Middle: forward computation time (ms) across different channel sizes. Right: inverse computation time (ms) across different channel sizes.
Table 2: Density estimation results CIFAR-10, ImageNet 32 x 32, and Galaxy, including the corresponding model sizes (Params) and bits per dimension (BPD). Lower BPD indicates better performance.
CIFAR-10
ImageNet 32x
Galaxy
Model
Params
BPD (L)
Params
BPD (1)
Params
BPD (1)
MAF Papamakarios et al. 2017
105.0M
4.31
Real NVP
[Dinh [Din al. 2016
6.4M
3.49
46.2M
4.28
6.38M
2.11
Glow [Kingma and Dharrwal 2018
44.2M
3.36 + 0.002
66.2M
4.09
6.25M
2.02
Emerging Hoogeboom et al.
2019
46.6M
3.34 + 0.002
44.0M
4.09
6.68M
1.98
Residual Flows Chen et al.
2019
25.2M
3.28
47.1M
4.01
5.58M
3.60
i-ResNet [Behrannn et al.
2019
25.0M
3.32 - 0.006
29.7M
4.05
6.88M
2.04
Woodbury Lu and Huang
2020
45,3M
3.42 0.002
45.3M
4.09
6.68M
2.01
i-DenseNet PPerggach-- -haugachi-Daaz cl
2021
24.9M
3.25
47.0M
3.98
6.07M
4.06
ButterllyFlow [Meng ct al.
2022
44.4M
3.33 = 0.003
44.4M
4.09
6.39M
1.95
CDFlow (Ours)
44.2M
3.31 = 0.004
44.2M
4.04
6.25M
1.92
CD- CD-Convolution layers. Figures b)) and (c) further report the runtime measurements for forward and inverse operations. Although larger m values introduce additional latency, our method remains more efficient than the other methods reported in Section 4.4. In addition, we separately evaluated the log-determinant computation time across channel sizes ranging from 16 to 1024, where the runtime remains consistent across different m, with values between 0.095 ms and 0.097 ms.
Based on these results, we adopt m = as the default setting for experiments, including both the main and supplementary studies, unless otherwise specified.
4.2 Density Estimation Standard Images
Standard Image Datasets. To enable direct comparison with prior work [Meng et al. [002]] we evaluate our method on the CIFAR-10 (Krizhessky 2009 and ImageNet-32x32 [Deng et al.] 20099 datasets. For both datasets, training is conducted with unified dequantization and standard data augmentation techniques.
Results. Table presents the quantization results of the model on the CIFAR-IO and ImageNet 32 32 datasets. CDFlow outperforms Glow, Emerging, Woodbury, and ButterflyFlow (all using the same modeling framework) and performs comparably to models like Residual Flows and i-DenseNet. The generated sample results visualized in Figure 3aland Figure 3b
4.3 Density Estimation Structured Images
To demonstrate the capability of CDFlow in modeling periodic structures, we apply it to galaxy images [Ackermann et al. 2018 conducting experiment on this real-world dataset.
Galaxy Datasets. When the boundary pixels of an image are relatively uniform connected, the data can be considered periodic. A typical example of periodic data is galaxy images, which often

---

Original Image
(1)) Ours
Original Image
(1 Original Image
(lI)
Woodbury
Butterfly
(II) Woodbury
(IV) Butterfly
(III Emerging
(IV) Butterfly
(a) CIFAR-10
(b) ImageNet 32 x 32
(c) Galaxy
Figure 3:
Comparison between the original image and outputs from different methods. For clearer and larger visualizations, please refer Appendix D
feature scattered light sources with predominantly dark boundaries. The Galaxy datasets [Ackermann) et al. 2018) is small classification dataset that includes 3,000 merged galaxies and 5,000 non-merged galaxies in the training and test We train CDFlow non-merged galaxy images and compare its performance to other models under the same experimental setup.
Results. Table 2 also presents the BPD results on the galaxy dataset. clear that CDFlow outperforms all other models, achieving 4.95% improvement in BPD compared to the Glow model. In addition, Figure [cl presents a visual comparison between our method and two strong baselines: Emerging and Butterfly. Experimental results demonstrate that proposed CDFlow model can well model data with periodic qualities.
4.4 Runtime
To demonstrate the effectiveness of our method in reducing computational complexity, we separately evaluated the runtime of the forward operation, log-determinant (logdet) computation, and inverse operation. We compared our method with standard
convolution, emerging and periodic convolutions [Hoogeboom et al. 2201), the Woodbury transformation and ME-Wo Woodbury with fixed latent dimension of d 16 [Lu and Huang 2020]. and the butterfly layer with level 10 [Meng et al. 2022]. For fair comparison, all methods were implemented in PyTorch, and all experiments were conducted on an NVIDIA A800 GPU.
The forward operation refers to the runtime required apply the tested layer function and compute the log-determinant, i.e., f(x) and log del(W)) under gradient compu- tation mode. As shown in Figure 4 although the theoretical complexity of the is (n²)) its actual runtime remains relatively stable across varying channel sizes, likely due to GPU parallelism. All baseline methods slower than the x convolution, while method achieves even faster performance. Notably, although the periodic convolution leverages Fourier transforms, its reliance on two-dimensional FFT introduces substantial memory overhead, leading to sharp runtime increases with more channels. In contrast, our method combines circulant and diagonal matrix constructions reduce nog-determinant complexity, and employs an optimized convolutional implementation to accelerate forward computation, as detailed in Sections 2.2 and 3.1
Forward Time.
convolution
In addition, specifically measured the runtime of the log-determinant computation, which depends solely on the size of the weight matrix, i.e., the number of input channels. Figure A illustrates how the runtime of different methods varies with increasing channel dimensions. Our method not only outperforms all baselines but also maintains stable performance without significant growth as the number of channels increases.
The inverse operation refers to the time required to apply the inverse function of the tested layer, i.e., x (y), under no-gradient mode. We first analyze the impact channel number on runtime, as shown in Figure 4 The Emerging convolution suffers from poor parallelism, while the
Inverse Time.

---

Forward Process
Inverse Process
(Batch Size=128. Spatial Size=32x32)
Size=128, Spatial Size=32x32)
10
10
10-
Runtime (ms)
(mz)
10a
109
10-1
96
128
Mumb Channels
Number
Logdet Computation Pracess
Process
(Batch Size- 128, Spatial x333)
(Batch Size- 128, Channel Number-96
10
10a
Runtime (ms)
Burtime
10a
10
16a
10-2
16
32
48
69
00
96
128
256
512
16
32
48
6.4
80
96
128
256
Nu Number of Chennels
Spatial Size
1x1 Conv
Dur
Parodic
Emerging
Butterfly
WTrans
MEWTrans
Figure 4: Comparison of runtime (ms) for forward computation, inverse computation, and log- determinant caleulation under varying numbers of channels, as well inverse computation under different spatial sizes. The results represent the average of 100 runs. Our method consistently outperforms baselines, especially in inverse and log- determmnnnnn computations. The runtime of the Emerging method becomes excessively long at large spatial sizes and is therefore omitted from the corresponding figure.
periodic convolution relies on two-dimensional Fourier transforms, leading to a significant increase in runtime as the number of channels grows. Other baseline methods perform slightly worse than the 1 convolution, possibly due to the lack of optimized inverse implementations and additional transformation overhead. In contrast, our method reduces the inversion complexity O(n log n),, and despite requiring data reshaping, consistently outperforms other baselines, particularly at large channel sizes.
We further evaluate how spatial resolution affects runtime, with the number of channels and batch size fixed. The results show that all methods experience increased runtime with larger spatial sizes. Our method outperforms the convolution at small spatial sizes, and becomes slightly less efficient at larger sizes, since it mainly reduces channel-wise rather than spatial complexity. Nevertheless, our method still clearly outperforms all other baselines except the convolution.
5 Conclusion
In this paper, we introduce the CDFlow model, a novel flow-based generative model. Instead of relying on traditional weight matrices, we employ the interleaved multiplication of circulant and diagonal matrices, leveraging the properties these matrices along with the Fast Fourier Transform (FFT) to efficiently compute the Jacobian determinant and perform the inverse transformation. Our experiments demonstrate that CDFlow excels in density estimation for standard image datasets and performs well on real-world data with periodic properties, outperforming existing baselines.
10

---

Acknowledgments and Disclosure of Funding
This work was financially supported by the National Key R&D Program China (Grant No. 2024YFA121400).
References
S. Ackermann, K. Schawinski, C. Zhang, A. K. Weigel, and M. D. Turp. Using transfer learning to detect galaxy mergers. Monthly Notices of the Royal Astronomical Society, 479(():415-425, 2018.
Alexandre Araujo, Benjamin Negrevergne, Yann Chevaleyre, and Jamal Atif. Understanding and training deep diagonal circulant neural networks. arXiv preprint 190i:::901..0000 2019.
Jens Behrmann, Will Grathwohl, Ricky T. Q. Chen, David Duvenaud, and Joern-Henrik Jacob- sen. Invertible residual networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, edi- tors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 573-582. PMLR, 09-15 Jun 2019. URL ittps://proceeddngs. mlr press/ /vrtsss/v//// /
http//bbhmmann1111 ppress/////bebraanaaa html
Chen-Hao Chao, Wei-Fang Sun, Yen-Chang Hsu, Zsolt Kira, and Chun-Yiles Lee. Training energy-based normalizing flow with matching objectives. Advances Neural Information Processing Systems, 0113886--48851 2023.
Chen-Hao Chao, Chien Feng, Wei-Fang Sun, Cheng-Kuang Lee, Simon See, and Chun- Lee. Maximum entropy reinforcement learning via energy-based normalizing flow. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 2024.
Ricky I. Q. Chen, Jens Behrmann, David Duvenaud, and Jörn-Henrik Jacobsen. Residual flows for invertible generative modeling. In Advances in Neural Information Processing Systems, 2019.
Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, and Shi-Fu Chang. An exploration of parameter redundancy in deep networks with circulant projections. In Proceedings of the IEEE international conference computer vision, pages 2857-2865, 2015.
Tri Dao, Albert Gu, Matthew Eichhorn, Atri Rudra, and Christopher Ré. Learning fast algorithms for linear transforms using butterfly factorizations. Proceedings of machine learning research, 97: 1517-1527, 2019. URL http:: wwt.e/eemaaaaaaiiiieeeeeeeeeaeee
org/ Carpus 76661331
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009, doi: R0110///VV..22000.055
Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, al. Cirenn: accelerating and compressing deep neural networks using block-circulant weight matrices. In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture, pages 395-408. 2017.
Xinyu Ding, Lexuan Chen, Siyu Liao, and Zhongfeng Wang. Parameter-efficient fine-tuning with circulant and diagonal vectors, 2025. URL URLhttpss h/tp://a http:////w////////////////000 00580.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation, 2014. URL https:///xxxixi....t org/ 0aba///41..8111
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp, 2016. URL
wwwp:o/gg.w.....ooo///.. 08803
Runpei Dong, Chumrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, Xiangwen Kong. Xiangyu Zhang, Kaisheng Ma, and Yi.
DreamL.MM: Synergistic multimodal comprehension and creation. In The Twelfth International Conference on Learning Representations, 2024. URL URI htt.tttt///opeerevveeee y011Gvdd9Bw
wwwpeeerrwveeeeee/eeeeeeeeeee net/ wtt.:/iwwwieeeeee/niaei.e.e nwt/forumffif=-

---

Yanchen Guan, Haicheng Liao, Zhenning Li, Jia Hu, Runze Yuan, Yunjian Li, Guohui Zhang, and Chengzhong Xu. World models for autonomous driving: An initial survey. IEEE Transactions on Intelligent Vehicles, pages 1-17, 2024. doi: 10.1 109//TTV.002433388377.
Emiel Hoogeboom, Rianne van den Berg, and Max Welling. Emerging convolutions for generative normalizing flows. In International conference on machine learning, 2019. URL httthtt/paaaiiii
org/abs /bs/ 1901. 11137
Marko Huhtanen and Allan Perämäki. Factoring matrices into the product of circulant and diagonal matrices. Journal of Fourier Analysis and Applications, 21:1018-1033, 2015. URL https
/api..eemntiicccoolaa.. :ot.aaarrrrrr/ooooooii:::::
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network train-
ing by reducing internal covariate shift. ArXiv, abs/1502.03167, 2015. URL https://api
semanticscholar wrggCorpusID::::. 5808102
Kristopher Jensen, Ta-Chu Kao, Jasmine Stone, and Guillaume Hennequin. Scalable bayesian gpfa with automatic relevance determination and discrete noise models. Advances in Neural Information Processing Systems, 14:10613-10626, 2021.
Mahdi Karami, Dale Schuurmans, Jascha Sohl-Dickstein, Laurent Dinh, and Daniel Duckworth. Invertible convolutional flow. Advances in Neural Information Processing Systems, 32, 2019.
Diederik P. Kingma and Prafulla Dhariwal. Glow: generative flow with invertible 1xl convolutions. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, page 10236-10245, Red Hook, NY, USA, 2018. Curran Associates Inc.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
URL https
//ppi.semanttiaeccc.o wwwanticschoolrr..... org/ Corpus ID: 18268744
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
You Lu and Bert Huang. Woodbury transformations for deep generative flows. In Proceedings of the 34th International Conference Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN
Chenlin Meng. Linqi Zhou, Kristy Choi, Tri Dao, and Stefano Ermon. Butterflyflow: Building invertible layers with butterfly matrices. In International Conference on Machine Learning, pages 15360-15375. PMLR, 2022.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In ICI.R, 2018
Marcin Moczulski, Misha Denil, Jeremy Appleyard, and Nando de Freitas. Acde: A structured efficient linear layer. arXiv preprint arXiv: 1.05999 2015.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, page 2335-2344, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Yura Perugachi-Diaze Jakub M. Tomezak, and Sandjai Bhulai. Invertible densenets with concatenated lipswish. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc ISBN 978171384553
Jie-En Yao, Li- Yuan Tsao, Yi-Chen Lo, Roy Tseng, Chia-Che Chang, and Chun-Yi Lee. Local implicit normalizing flow for arbitrary-scale image super-resolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
12

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As described in the abstract and introduction, our paper proposes a new normalization flow model that accelerates matrix inversion and determinant computation through weight decomposition and Fourier transform.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Sections 3.3 and 4.4 discuss limitations, including the current restriction to 1 convolutions and reduced efficiency large spatial inputs.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification: Section presents the mathematical derivation of our approach, including the assumptions and computation process for the transformation and determinant.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Section 4 and the appendix provides sufficient experimental details to allow reproduction, including dataset descriptions, training procedures, and model configurations.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
14

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
We plan to release anonymized version of the code and data upon publica- tion.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccpplliiggudeeeeeeeeeeeee Cooee//ddeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 4.1 and the appendix provides describes training and testing details. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not report error bars, but we provide multiple comparative experiments support the main claims.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
15

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section 4.1 and Section 4.4 include information on the GPU model used and the runtime, offering guidance for reproduction in similar environments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttts::///nntttt http:/rppp....ppppoc//ctttccaaaa
Answer: [Yes]
Justification: This work does not involve any human subjects, sensitive data, or potential risks; it fully conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [No]
Justification: We did not explicitly discuss broader impacts, as this work focuses on thcoretical contribution to machine learning model efficiency, without direct deployment or user interaction components.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
16

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve the release of models or datasets with high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external datasets and baseline models used are cited properly in the main text.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should citet the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will provide documentation and usage instructions for our code and model assets in the supplemental material upon release.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve human subjects, hence IRB approval is not required.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were only used for language editing and polishing purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
19

---

A Impact of Linear Transformation Constraints on Flow-based Model Performance
To further validate the effectiveness of our pro- posed method, we reproduced the experiment de- scribed in Appendix A.5 of [Cao et al. [2023] and included an additional variant based on our Diagonal-Circulant Decomposition (DCD) linear layers. The compared linear layer types include F-type (full matrix), L-type (lowe-triangular), U- type (upper-triae trpanetriangular), LU-type (the product lower and upper triangular matrices), and our pro- posed DCD-type.
5C0
As shown in Fig. the model employing the F-type layers achieved the lowest negative log- likelihood (NLL), consistent with Chao et al. [023)] Although our DCD-type layer does not surpass the fully learnable F-type, it achieves the second-bec best performance, outperforming the LU- type design. This result indicates that DCD retains much of the expressiveness of the unconstrained full matrix while maintaining more efficient and structured parameterization.
Figure 5: NLL comparison among flow-based models constructed with different linear layer types (F, L. LU, and DCD). Our DCD-type layer achieves performance close to the fully learnable F-type and surpasses the LU-type struc- ture.
B Runtime
Specifically, our approach requires transforming the input tensor from four-dimensional structure [B, C, H. W] to two-dimensional matrix [B H W. C). While the runtime reported in Section [.4 omits this transformation, we further measured the total runtime including it to ensure a fair compari- son. As illustrated in Figure [6| our method continues to demonstrate strong computational efficiency even after accounting for the transformation overhead. It is worth noting that the computation of the log-determinant independent of the input tensor, and therefore the reshaping operation has impact on its runtime.
Process
Invrss Process (Batch Stze=128.
(Botch 128. Spatial 32x32
Size322321
30P
2.0
200
10
Mf Chanrel
Buncerof Charrels
Logeet Computation Proceds
Inverse Process
(Botch St20 128, Spotial Sizoo 322x3
[Batch Sie- Charnel N6m
20"
102
2.2
200
207
30
1.1
1.30
750
12
72
75
Figure 6: Comparison of runtime (ms).
20

---

C Additional Experiments on Flow Matching Models
To further validate the effectiveness of the proposed linear layer, conducted experiments within the Flow Matching framework Lipman et al. [0022], which employs 5-layer MLP on 2D toy datasets.
To systematically evaluate the method, we performed experiments on three datasets: Chessboard, Moons, and Circles. Within the original framework, we replaced the linear layers of the standard MLP
with four structured variants comparison: EMLP (Emerging Layer) Hoogeboom et al. (2019).
WMLP (Woodbury Layer) Lu and Huang 2020 BFMLP (Butterfly Layer) Meng et al. 2022 and CDMLP (Ours).
The experimental results are summarized in Table CDMLP achieves the best or near-best perfor- mance across most cases while requiring significantly fewer parameters. For instance, attains the lowest NLL on the Chessboard and Circles datasets and the best MSE and FID on Moons, outperform- ing matching larger models such as MLP and EMLP. In addition, Figures further visualize the learned distributions, showing that the model successfully evolves from Gaussian initialization toward the target distributions. These findings not only confirm the modeling capability of the proposed method within the Flow Matching framework but also highlight its applicability and generality across different data structures.
Table 3: Flow Matching Model Results on 2D-toy Datasets. The best results highlighted in bold.
Dataset
Model
Params
MSE
FID
NLL
MIP Lipman et al. 2022
791.0K
3.796
0.0062
10.6853
EMLP Hoogcboom ct al. 2019
1577.5K
3.797
0.0165
9.357
Chessboard
WMLP
Lu and Huang
2020
40.5K
3.761
0.0042
9,0551
BFMLP
Meng ct al. CDMLP (Ours)
2022
26.
3.789
0.0125
8,2088
8.45K
3.810
0.0115
7.0425
MLP Lipman
2022
791.0K
1.854
0.0022
24.1885
EMLP Hoogeboom et al. 2019
1577.5K
1.855
0.0023
36.9462
Moons
WMLP Lu and Huang
2020
40.5K
1.905
0.0016
47.6328
BFML Meng et al. 2022
26.1K
1.886
0.0031
17.9168
CDMLP (Ours)
8.45K
1.842
0.0016
31.0329
MLP Lipman et al. 2022
791.0K
2.433
0.0019
42.4687
EMLP Hoogeboom
2019
1577.5K
2.517
0.0032
20.5252
Circles
WMLP and Huang
2020
40.5K
2.547
0.0021
26.6795
BFMLPMMnn et
2022
26.1K
2.470
0.0032
21.6716
CDMLP (Ours)
8.45K
2.482
0.0069
15,4922
twdiil
Do Lte
Figure Time evolution of the learned distribution 2D toy datasets (Chessboard, Moons, Circles).
D Visualization of Generation Results on Image Datasets
21

---

Model Likelihood, Hutchinson Estimator, #acc=10
Exact Model Likelihood
2-
0-
0
Model Likelihood, Hutchinson Estimator, #acc=10
Exact Model Likelihood
S
S
4-
4
2-
-2
-2
Model Likelihood, Hutchinson Estimator, #acc=10
Exact Model Likelihood
2-
2
0
0
-4
-4
-4
-2
0
0.005
0.010
0.015 density
0.030
Figure 8: Visualization of model likelihood (left: Hutchinson estimator, right: exact) on 2D toy datasets (Chessboard, Moons, Circles).
22

---

Figure 9: Generation results on the CIFAR-10 dataset.
23

---

Figure 10: Generation results on the ImageNet dataset.
24

---

Figure 11: Generation results on the Galaxy dataset.
25


Reviews:
[{"review_id": "v0WAeeER9m", "reviewer": "Reviewer_zHBj", "summary": "This paper is concerned with the efficient implementation of normalizing flows for sampling from complex probability distributions. These models require the construction of invertible deep networks, with the extra condition that the determinant of the Jacobian matrix should be easy to compute. The inverse is important primarily for the sampling process of the model, while the log-det is important primarily for the training process. Based on this, the authors focus on the efficient implementation of linear transformations (that is, linear layers) whose determinants and inverses are easy to compute. They leverage a result on the approximation of dense matrices by the product of structured matrices: namely the alternating product of diagonal and circulant matrices. The diagonal matrices have easily computable inverses and determinants, and the circulant matrices can be similarly handled using FFTs.", "questions": "My questions are primarily around the theoretical properties of this approach as they pertain to training normalizing flow models.\n\nAs noted in the weaknesses, there is not much discussion on how the choice of $m$ affects the expressivity of the model. Of course, as $m$ increases the linear layer can parameterize a larger family of matrices. When $m$ is small, what kind of matrices are expected to be well-approximated? For instance, the composition of convolutions and pointwise multiplications might do a good job approximating “spatially-varying convolutions,” pseudodifferential operators, or something similar.\n\nThis method is put forth with the idea of enabling efficient matrix inversion and log-determinant calculation. However, these fail when the component (diagonal/circulant) matrices are not well-conditioned. Did you employ regularization in training to ensure that this does not happen? Is it even worthwhile to add penalties to enforce well-conditioned linear layers?\n\nYou have elected to focus on products of diagonal and circulant matrices, particularly since the latter can be easily parameterized using FFTs. However, there are many other types of structured matrices that admit similar efficient $n\\log n$ computations; see the paper “Butterfly factorization” by Li et. al. Do the algorithms and approximations in that paper have any bearing on potential generalizations of your proposed method? A *thorough* discussion on this would make this paper much more significant, even if not much is done experimentally.\n\nThere is a claim on limitations that I do not understand. On pg. 6, you say that “A current limitation of [your] approach is its reliance on 1x1 convolutions.” But your convolutions are implemented directly in the Fourier domain, so it does not seem that you have any control over the spatial extent of the convolutions implemented by the circulant matrices. What did you mean by this statement?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strenghths:\n\nThe results and methods put forth in this work have the strength of being quite simple and easy to understand. The work leverages established results on structured matrix approximation to good effect, yielding a method that I can see being easily adapted by practitioners. The methods based on these results attain good asymptotic complexity, only comparable to the Woodbury method of [Lu and Huang, 2020], which is made up for with superior performance on test datasets and in runtime.\n\nWeaknesses:\n\nThere are some missing references on prior works that use circulant-diagonal linear layers in machine learning applications. For instance, see “ACDC: A structured efficient linear layer” by Moczulski et. al. Papers that use such methods generally weren’t using circulant-diagonal linear layers to implement normalizing flows, so it is not necessary to “compare” as far as experiments go, but it would be best to acknowledge them. Additionally, you set $m=2$ in your experiments, which is presumably quite a bit less than the $m=n$ necessary for exact representation of any dense matrix. Although it is acknowledged that “the approximation error can be controlled by adjusting the number of factors,” not much else is said on the expressivity of approximation as a function of $m$.", "was_revised": true, "final_justification": "Thank you for your rebuttal, and for addressing most of my comments – I hope you incorporate these remarks into the camera-ready version, if accepted. There is a lingering issue with the discussion around 1x1 convolutions. To my understanding, “1x1 convolutions” when represented as circulant matrices are just constant multiples of the identity matrix (in the spatial domain). When there are multiple channels, perhaps there is mixing of channels, but the point still stands. If you parameterize the convolution as a Fourier multiplier, which you do in this work, you lose control over the spatial extent of the convolution filter, so the convolutions are no longer 1x1. Your comment on the transformation being square-shaped doesn’t seem particularly relevant to this. I ask that you please think about this a bit more and make the final version of the paper more clear on this matter.\n\nBased on your response, I am increasing my rating to 5 (accept).", "rebuttal": "We sincerely thank the reviewer for the thoughtful feedback and valuable suggestions, which greatly helped refine both the theoretical foundation and clarity of our work. We address each point below.\n>Weakness1：There are some missing references on prior works……\n\nThank you for pointing this out. We will include a citation to “ACDC: A Structured Efficient Linear Layer” (Moczulski et al.) to acknowledge prior usage of circulant-diagonal layers. While they do not compute the inverse transformations or the log-determinant of the Jacobian, both approaches share the goal of efficient linear transformations.\n\n>Weakness 2 & Question 1：As noted in the weaknesses, there is not much discussion on how the choice of affects the expressivity of the model.\n\nIn practice, we aim to balance expressivity with efficiency in parameter count and runtime. To support the selection of the hyperparameter $m$ (i.e., $m$ diagonal and $m - 1$ circulant) , we conducted additional experiments. \n\nFirst, we replicated standard experiments on the expressivity of structured matrices following the setup in [2]. We added comparisons using circulant and DCD structures. As shown in Table 1, circulant layers achieve fast convergence and closely approximate full matrices. Notably, even with $m=2$, the DCD structure significantly outperforms LU and its components, demonstrating strong approximation capabilities with minimal complexity.\n\nTable 1. Mean NLL ± 95% CI on synthetic data (step 5000 / 10000)\n| Type| 5000 NLL |10000 NLL| \n| ----- | --- | -- | \n| full  | 49.11 ± 0.01 | 48.32 ± 0.12 | \n| tril  | 51.91 ± 0.34 | 50.89 ± 0.18 | \n| triu  | 50.27 ± 0.38 | 49.61 ± 0.73 | \n| trilu | 51.85 ± 0.49 | 51.04 ± 0.22 | \n| circ  | 49.07 ± 0.24 | 48.49 ± 0.39 | \n| dcd   | 50.18 ± 0.38 | 49.21 ± 0.12 |\n\nWe then evaluated the relationship between $m$ and model expressivity, as well as $m$ and runtime performance.\nAs shown in Table 2, increasing $m$ on CIFAR-10 yields only marginal improvements in model performance, while incurring noticeable increases in both parameter size and computation. We also measured forward and backward runtime across different values of $m$, summarized in Table 3 and Table 4, where we observed that runtime increases with $m$, yet still remains faster than existing baselines reported in the original paper.\n\nTable 2. Performance and parameter statistics of CDFlow under different values of $m$ on CIFAR-10.\n| Model | Setting | BPD (CIFAR-10) | #Params | DCD Params |\n|--------|---------|----|---------|------------|\n| CDFlow| m=2 | 3.31  | 44.2 M  | 6.9 K  |\n| CDFlow| m=3 | 3.30  | 44.2 M  | 11.1 K |\n| CDFlow| m=4 | 3.30  | 44.2 M  | 15.3 K |\n\nTable 3. Forward computation time (ms) for CD-Linear layers with different values of m, compared against traditional  convolutions. \n\n| Channels | $m=2$ | $m=3$ | $m=4$ | $m=5$ | $m=6$ |1x1 Conv|\n|---|---|---|---|--|--|---|\n|16 |0.226|0.226|0.326|0.374 |0.424 |0.240|\n|32|0.323|0.405 | 0.547|0.690 |0.837 |0.432 |\n|48|0.379|0.575 | 0.792|1.008 |1.225 |0.477 |\n|64|0.454|0.729 | 1.007|1.287 |1.571 |0.535 |\n|80|0.557|0.909 | 1.267|1.625 |1.983 |0.679|\n|96|0.719|1.194 | 1.676| 2.169 |2.644 |0.758|\n|128 | 0.875|1.434|1.963|2.529|3.144|0.880|\n\nTable 4. Inverse computation time (ms) for CDLinear layers with different $m$ values, compared against $1 \\times 1$ convolutions.\n\n| Channels | $m=2$ | $m=3$ | $m=4$ | $m=5$ | $m=6$ |1x1 Conv |\n|---|---|--|---|---|---|---|\n| 16 | 0.170 | 0.229 | 0.291 | 0.353 | 0.416  | 0.195 |\n| 32 | 0.268 | 0.407 | 0.549 | 0.697 | 0.836  | 0.328 |\n| 48 | 0.348 | 0.566 | 0.782 | 0.993 | 1.234  | 0.428 |\n| 64 | 0.428 | 0.706 | 1.011 | 1.264 | 1.544  | 0.511 |\n| 80 | 0.532 | 0.890 | 1.275 | 1.598 | 1.972  | 0.714 |\n| 96 | 0.693 | 1.168 | 1.653 | 2.135 | 2.633  | 0.757 |\n| 128| 0.849 | 1.397 | 1.940 | 2.492 | 3.054  | 0.935 |\n\nAdditionally, we tested log-determinant computation time as the channel dimension varies from 16 to 1024. As shown, our method maintains stable performance across different $m$, with runtime ranging narrowly from 0.095 ms to 0.097 ms, demonstrating robustness to input dimensionality.\n\nConsidering the trade-off between expressivity and efficiency, we adopt $m = 2$ as the default configuration throughout our experiments unless otherwise specified.\n\n>Question 1: what kind of matrices are expected to be well-approximated?\n\nThis decomposition is particularly effective for representing low-rank matrices. Prior work [1] shows that any $n \\times n$ matrix of rank $k$ can be approximated using a product of $k$ diagonal and $k{-}1$ circulant matrices. Moreover, networks built with such decompositions exhibit strong expressivity: [1] proves that any ReLU network of width $n$ and depth $L$ can be approximated by a diagonal-circulant neural network (DCNN) of depth $(2n{-}1)L$, and DCNNs can represent functions that are not expressible by any rank-$k$ ReLU network, highlighting their superior modeling capacity.\n\nOur method extends this idea by expressing convolutional operations as matrix multiplications and applying structured decomposition to the resulting weight matrices. As a result, any convolution operation that can be formulated via matrix multiplication—including discrete convolutions and their linearized forms—can in principle be approximated within our framework. In particular, circular convolutions are naturally suited, as their Jacobians correspond to circulant matrices.\n\n>Question 2:these fail when the component (diagonal/circulant) matrices are not well-conditioned.\n\nAs discussed in [3], randomly initialized circulant matrices have stable minimum singular values and low condition numbers. Compared to Toeplitz or Gaussian matrices, circulants offer better numerical stability.\nPractically, we carefully design the initialization of both diagonal and circulant components. The diagonal matrices are sampled from a normal distribution centered at 1 with variance scaling inversely with channel size, i.e., $\\mathcal{N}(1, 1/c^2)$, to avoid vanishing or exploding scales. The circulant components are initialized in the frequency domain from $\\mathcal{N}(1, I)$, leveraging the orthogonality of the DFT basis. These schemes result in well-conditioned composite matrices with condition numbers typically close to 1 at the start of training.\n\nIn addition, we apply spectral normalization (Lines 233–234 in the paper) to further stabilize training. This constrains the spectral norm of each weight matrix $W$ to 1, effectively limiting the network’s Lipschitz constant. Since circulant matrices are diagonalizable via the discrete Fourier transform (DFT), their singular values correspond to the magnitudes of their frequency-domain representations. Therefore, spectral normalization can be efficiently implemented by normalizing each frequency-domain vector by its maximum modulus, ensuring both robustness and efficiency.\n\n>Question 3: However, there are many other types of structured matrices that admit similar efficient……\n\nWe thank the reviewer for the insightful suggestion and for pointing us to the Butterfly factorization literature[4] The butterfly factorization approximates an $N \\times N$ matrix $K$ satisfying the complementary low-rank property by a product of sparse matrices. This enables matrix--vector multiplication with $\\mathcal{O}(N \\log N)$ complexity and $\\mathcal{O}(N \\log N)$ parameters.\n\nIn comparison, our diagonal-circulant decomposition requires only $\\mathcal{O}(MN)$ parameters, where $M$ is a small constant. More importantly, one of the major advantages of our method lies in the efficient implementation of matrix inverse and log-determinant computations---both critical operations in flow-based generative models. As shown in Table 1 of the paper, our inverse computation has $\\mathcal{O}(n \\log n)$ complexity due to the structured FFT-based approach, whereas the inverse of butterfly-structured matrices typically incurs $\\mathcal{O}(n^2)$ cost.\n\n>Question 4: There is a claim on limitations that I do not understand……\n\nLastly, we would like to clarify the current limitations of our method. As discussed earlier, our approach is capable of approximating any convolution operation that can be expressed as a matrix multiplication. However, in the context of flow-based models, it is essential to compute both the inverse of the weight matrix and the corresponding Jacobian determinant. These operations are generally only well-defined for square matrices, which constrains our method's applicability to $1 \\times 1$ convolutions or other square-shaped transformations. That said, our ongoing work focuses on extending the framework to support determinant and inverse computations for non-square matrices. We believe this will enable the application of our method to general $d \\times d$ convolutions and potentially to a wider range of structured transformations.\n\nWe sincerely thank the reviewer for the thoughtful feedback and insightful questions. We hope our responses have addressed the concerns regarding model expressivity, training stability, and potential generalizations. We will also revise the final version of the paper to include additional references, including prior works such as ACDC and Butterfly Factorization, to better situate our contribution within the broader literature.\n\n[1] Araujo, Alexandre, et al. \"Understanding and training deep diagonal circulant neural networks.\" ECAI 2020. IOS Press, 2020. 945-952.\n\n[2] Chao, Chen-Hao, et al. \"Training energy-based normalizing flow with score-matching objectives.\" Advances in Neural Information Processing Systems 36 (2023): 43826-43851.\n\n[3] Pan, Victor Y., and Guoliang Qian. \"Condition numbers of random toeplitz and circulant matrices.\" arXiv preprint arXiv:1212.4551 (2012).\n\n[4]Li, Yingzhou, et al. \"Butterfly factorization.\" Multiscale Modeling & Simulation 13.2 (2015): 714-732.\n\n[5]Dao, Tri, et al. \"Learning fast algorithms for linear transforms using butterfly factorizations.\" International conference on machine learning. PMLR, 2019."}, {"review_id": "fpFCbgMpoh", "reviewer": "Reviewer_bzQ3", "summary": "The paper presents and explores a flow architecture, the distinctive feature of which is the usage of the product of interleaving circulant and diagonal matrices as the weights for 1x1 convolutional layers, constituting the non-linear transformation of the coupling layer. The main advantages of this approach are the reduced time complexity for logdet and inverse computation, as well as decreased space consumption.", "questions": "Please see **action items** and **possible action items** in “Weaknesses”. I will be ready to accept the paper if the first three weaknesses are thoroughly addressed. Moreover, I have some smaller questions:\n\n1. Could you please provide experimental results (both regarding sampling quality and runtime efficiency) for different choices of hyperparameter m?\n2. What can be done to represent the non-square weight matrix?\n3. Could you please explain why there is such a large margin between the proposed method and other methods, especially in logdet computation? Some of the methods have the same theoretical complexity, why are they so much slower?\n4. Did you try using the initialization scheme and the placement of non-linearities proposed in [1]? I think it might increase the quality of the final model.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "**Strengths**:\n1. The paper is written very clearly, with all the necessary prerequisites explained in Section 2 (“Background”).\n2. In order to prove the superiority of their method, authors do not only compute the theoretical complexity, but provide experimental time measurements in Section 4.3 as well.\n3. The proposed method, indeed, has a competitive time complexity, both in theory and in practice.\n\n**Weaknesses**:\n\nHere are some conceptual weaknesses of the paper:\n\n1. As the diagonal-circulant neural networks have already been proposed in [1], the novelty of the paper lies exclusively in applying such networks to the construction of a flow model. While the efficiency of this approach is confirmed experimentally, the trade-off between the number of parameters of the flow layers and the quality of the resulting network is not explored. According to [2], this might be a non-negligible trade-off. **Possible action items**: 1) Please check if it would be helpful to rearrange the results from Table 2 into a graph (or two graphs) with number of parameters (or runtime measurements from Figure 3) on one axis and quality on the other. 2) Could you reproduce the experimental setting of Appendix A.5 in [2] with the proposed method?\n2. The experimental section of the paper is scarce. While it is clearly shown that the computation of the Jacobian logdet is much faster than its alternatives, especially for large numbers of channels, which might indeed speed the training up, the sampling quality from Table 2 is average. **Possible action item**: The paper would have benefited much from applying the proposed layers to a real experimental setting, e.g. in a flow matching model or comparing to Woodbury in a setup similar to [3]. Speed up and quality measurements obtained this way would be much more valuable and would help other researchers to choose the appropriate method of flow layer compression.\n3. A couple of words about the presented results: while I understand that the proposed method does not have to be SOTA, as its main goal is efficiency, the results from Table 2 and Figure 2 are not very convincing. In Table 2 the results are presented without confidence intervals. Also, it is unclear why MAF and i-ResNet are not evaluated on ImageNet and even more methods are not evaluated on Galaxy. In Figure 2 the images are too small to see any difference, but to me it seems that images from the proposed method have some strange artifacts which other methods do not have. **Action items**: Please report the results for all methods, add confidence intervals and show larger pictures.\n4. The authors did not mention some very relevant related articles, which is alarming, as their work could have benefited much from using tips for training from [1] (this paper answers the questions of expressivity and efficient training methods of networks with circulant-diagonal layers) and comparing their method to a close in time complexity method from [4] (with both Jacobian determinant and inverse computation complexity being O(nlogn)). Also, similar parameterizations of linear layers as the product of circulant and diagonal matrices were proposed in [5], [6]. **Action item**: Please compare to [4] (please correct me if they are not comparable).\n\nSome small inaccuracies in the manuscript:\n\n1. In equation (3) there are L - 1 terms, while there should be L, with the first one being df_1 / dx.\n2. Lines 125-127: it is written that the proposed complexity O(n log n) is more efficient than O(dn) from Woodbury, where d is a constant, which is not true, especially considering the fact that the proposed complexity is actually O(mn log n), where m is a constant.\n3. In Figure 2, there is a typo in the name of “Woodbury”.\n\n[1] Araujo, Alexandre, et al. \"Understanding and training deep diagonal circulant neural networks.\" ECAI 2020. IOS Press, 2020. 945-952.\n\n[2] Chao, Chen-Hao, et al. \"Training energy-based normalizing flow with score-matching objectives.\" Advances in Neural Information Processing Systems 36 (2023): 43826-43851.\n\n[3] Lee, Jaewoo, et al. \"Differentially private normalizing flows for synthetic tabular data generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 7. 2022.\n\n[4] Karami, Mahdi, et al. \"Invertible convolutional flow.\" Advances in Neural Information Processing Systems 32 (2019).\n\n[5] Cheng, Yu, et al. \"An exploration of parameter redundancy in deep networks with circulant projections.\" Proceedings of the IEEE international conference on computer vision. 2015.\n\n[6] Moczulski, Marcin, et al. \"Acdc: A structured efficient linear layer.\" arXiv preprint arXiv:1511.05946 (2015).", "was_revised": true, "final_justification": "During the rebuttal phase, the authors have resolved W1 and W2 from my initial comment and have promised to report mean and std of all their experiments in the final version of the paper, which was the main point of W3. W4 was also addressed in their comment from the theoretical point of view, although the experimental evaluation (with reproduction of the results of the discussed paper) would add value to their paper (and the code for reproduction would be useful for the community). The authors have also answered my questions, although I have now noticed that I missed the opportunity to ask specifically for the times for circulant layers without diagonal ones (i.e. for m = 1). Overall, I feel that conceptually this work is incremental and lacks originality, as it just mixes together two already well-known concepts. However, the experimental results are solid and convincing, and the authors actively participated in the rebuttal. Thus, I am raising \"quality\" score to 3 and the overall score to 4.", "rebuttal": "We sincerely thank the reviewer for the constructive suggestions and valuable references. Below, we provide detailed responses to each of the raised concerns and questions.\n\n>Weakness 1: Parameter counts and quality\n\n We revised Table 1 to include model parameter counts and added supplementary experiments.\n\nTable 1. Performance comparison of various flow-based models. BPD (bits per dimension) is reported along with the number of model parameters.\n|Model|Params|CIFAR-10 BPD|Params|ImageNet BPD|Params|Galaxy BPD|\n|--|--|--|--|--|--|--|\n|Ours|44.2M|3.31|44.2M|4.04|6.25M|1.92|\n|Butterfly|44.4M|3.33|44.4M|4.09|6.39M|1.95|\n|Emerging|46.6M|3.34|44.0M|4.09|6.68M|1.98|\n|Woodbury|45.3M|3.35|45.3M|4.09|6.68M|2.01|\n|Glow|44.2M|3.35|66.2M|4.09|6.25M|2.02|\n|MAF|105.0M|4.31|—|—|—|—|\n|RealNVP|6.4M|3.49|46.2M|4.28|6.38M|2.11|\n|Residual Flows|25.2M|3.28|47.1M|4.01|5.58M|3.60|\n|i-DenseNet|24.9M|3.25|47.0M|3.98|6.07M|4.06|\n|i-ResNet|25.0M|3.45|29.7M|4.05|6.88M|2.04|\n\nWe reproduced the experiments from A.5 of [1], adding results with circulant and diagonal-circulant-diagonal (DCD) matrices.(Table 2) Circulant matrices converge faster and closely match full matrix performance and DCD variant outperforms LU and its individual L and U components, demonstrating strong expressive power.\n\nTable 2. Mean NLL ± 95% CI on synthetic data (step 5k / 10k)\n|Type|5k NLL|10k NLL|\n|-|-|-|\n|full|49.11±0.01|48.32±0.12|\n|tril|51.91±0.34|50.89±0.18|\n|triu|50.27±0.38|49.61±0.73|\n|trilu|51.85±0.49|51.04±0.22|\n|circ|49.07±0.24|48.49±0.39|\n|dcd|50.18±0.38|49.21±0.12|\n\n>Weakness 2:The experimental section of the paper is scarce.\n\nTo demonstrate broader applicability, we replaced standard linear layers with CDLinear in a 5-layer MLP for flow matching on 2D toy datasets(Table 3). Our method achieves comparable or better MSE, NLL, and FID with significantly fewer parameters.\n\nTable 3. Flow Matching Model Results on 2D-toy Datasets\n|Dataset|Model|Params|MSE↓|FID↓|NLL↓|\n|---|---|---|---|---|---|\n|Chessboard|CDMLP|8.5K|3.81|0.0115|7.0425|\n|Chessboard|MLP|791K|3.796|0.0062|10.6853|\n|Moons|CDMLP|8.5K|1.842|0.0016|31.0329|\n|Moons|MLP|791K|1.854|0.0022|24.1885|\n|Circles|CDMLP|8.5K|2.482|0.0069|15.4922|\n|Circles|MLP|791K|2.433|0.0019|42.4687|\n\nIn addition, [2] did not provide open-source code, which made it difficult to perform a direct comparison between the Woodbury approach and our method under a consistent experimental setup within the limited time available. \n\n>Weakness 3: A couple of words about the presented results.\n\nWe did not report classical confidence intervals, as none of the baseline papers did, reflecting the limited use of statistical testing in deep learning research. Unlike traditional statistical experiments, deep learning models are typically trained multiple times on the same fixed dataset, and the resulting outcomes do not satisfy the assumption of i.i.d. samples. Consequently, conventional tests such as the $t$-test or ANOVA are not applicable. More rigorous comparisons require resampling-based methods as discussed in [3]. However, these methods are often impractical in deep learning scenarios, as they demand a large number of repeated experiments and careful control of confounding variables for each model, making them difficult to incorporate into standard evaluation pipelines.\nAs a result, the common practice is to report the mean and standard deviation across multiple runs to assess model stability. For example, on CIFAR-10, we conducted three trials and achieved a BPD of $3.31 \\pm 0.004$, demonstrating the robustness of our approach.\n\n>Weakness 4: The authors did not mention some very relevant related articles.\n\nThank you for pointing out the missing references. We will add the suggested citations and related discussions.\n\n[4] focuses on symmetric and circulant convolutions for coupling layers, while our method approximates arbitrary convolutions and fully connected layers by decomposing weight matrices into interleaved circulant and diagonal components, enabling fast matrix multiplication.\n\nBesides, [4] github repo is empty, making reproduction and fair comparison infeasible. We therefore focus on theoretical distinctions.\n\nIn terms of complexity, [4] relies on 2D FFTs over spatial dimensions ($\\mathcal{O}(chw \\log(hw))$), while ours uses 1D FFTs over channels ($\\mathcal{O}(chw \\log c)$ for convolution, $\\mathcal{O}(c \\log c)$ for log-determinants). As stated in Lines 83–86, the Jacobian of a linear layer equals the weight matrix $W$, so $\\log|\\det(W)|$ scales with $c$, not $h$ or $w$. \n\nIn summary, [4] tackles a different problem and is not directly comparable to our approach.\n\n>Some small inaccuracies in the manuscript.\n\nThank you for pointing out the issues. We have revised Eq.3 as $\\log |\\partial f_\\theta / \\partial x| = \\log |\\partial f_1 / \\partial x| + \\sum_{i=2}^L \\log |\\partial f_i / \\partial f_{i-1}|$. Additionally, we have clarified the description in Lines 125–127: Although Woodbury's theory has better complexity than ours, in actual operation, they need to transform the input multiple times, so the actual efficiency is not as good as ours. Lastly, we have corrected the typo in the name \"Woodbury\" in Figure 2.\n\n>Q1 : provide experimental results for different choices of hyperparameter m.\n\nOur goal is to balance expressiveness, efficiency, and runtime. We evaluated the impact of varying $m$ on CIFAR-10(Table 4), larger $m$ brings only minor BPD improvements but increases parameter count and computational cost. Runtime results(Tables 5 and 6) confirm that our method remains more efficient than prior alternatives, even as $m$ grows. We further evaluated log-determinant computation across channel sizes from 16 to 1024. The average runtime remains stable (0.095–0.097 ms) across different $m$ values, confirming the scalability of our approach. Therefore, we set $m = 2$ as the default for all experiments unless noted otherwise.\n\nTable 4. Performance and parameter statistics of CDFlow under different values of $m$ on CIFAR-10.\n|Model|m|BPD(CIFAR-10)|DCD Params|\n|--|--|--|--|\n|CDFlow|2|3.31|6.9K|\n|CDFlow|3|3.30|11.1K|\n|CDFlow|4|3.30|15.3K|\n\nTable 5. Forward computation time (ms) for CDLinear layers with different values of $m$, compared against $1 \\times 1$ convolutions. \n\n|Channels|$m=2$|$m=3$|$m=4$|$m=5$|$m=6$|1x1 Conv|\n|--|--|--|--|--|--|--|\n|16|0.226|0.226|0.326|0.374|0.424|0.240|\n|32|0.323|0.405|0.547|0.690|0.837|0.432|\n|48|0.379|0.575|0.792|1.008|1.225|0.477|\n|64|0.454|0.729|1.007|1.287|1.571|0.535|\n|80|0.557|0.909|1.267|1.625|1.983|0.679|\n|96|0.719|1.194|1.676|2.169|2.644|0.758|\n|128|0.875|1.434|1.963|2.529|3.144|0.880|\n\n\nTable 6. Inverse computation time (ms) for CDLinear layers with different $m$ values, compared against $1 \\times 1$ convolutions. \n\n|Channels|$m=2$|$m=3$|$m=4$|$m=5$|$m=6$|1x1 Conv|\n|-|-|-|-|-|-|-|\n|16|0.170|0.229|0.291|0.353|0.416|0.195|\n|32|0.268|0.407|0.549|0.697|0.836|0.328|\n|48|0.348|0.566|0.782|0.993|1.234|0.428|\n|64|0.428|0.706|1.011|1.264|1.544|0.511|\n|80|0.532|0.890|1.275|1.598|1.972|0.714|\n|96|0.693|1.168|1.653|2.135|2.633|0.757|\n|128|0.849|1.397|1.940|2.492|3.054|0.935|\n\n>Q2 : non-square weight matrix.\n\nTo handle non-square weights, we apply a block-wise partitioning strategy [5] to the first circulant matrix only. Specifically, a matrix $\\mathbf{C} \\in \\mathbb{R}^{d_1 \\times d_2}$ is divided into $q_1 = \\lceil d_1/p \\rceil$ by $q_2 = \\lceil d_2/p \\rceil$ blocks of size $p \\times p$, each treated as a circulant matrix. Dimensions not divisible by $p$ are automatically padded by replication. This converts the original multiplication into multiple independent square circulant multiplications, enabling efficient FFT-based computation. The remaining matrices (e.g., diagonal and secondary circulant) remain square for simplicity and efficiency.\n\n>Q3 : a large margin especially in logdet computation.\n\nUpon rechecking our experiments, we found that the previously reported runtime for Emerging was affected by ambiguities in its source code, which caused our measurement to include an extra 1×1 convolution. Similarly, the Butterfly implementation bundled forward propagation time. After careful retesting, we observed that Emerging’s actual average log-determinant runtime is stable at 0.123 ms, while our method achieves 0.107 ms. The slower runtime of Emerging stems from its need to compute two autoregressive convolutional kernels. For Butterfly, the increased latency arises from its decomposition depth scaling with the input dimension; for fairness, we followed the original setting. All other results remain correct. Notably, both WTrans and MEWTrans involve multiple input transformations, whereas our method provides consistently efficient log-determinant computation.\n\n>Q4 : initialization scheme and the placement of non-linearities.\n\nWe did not stack multiple linear layers and therefore did not apply activation functions. However, we agree that stacking linear layers is a promising approach to enhance model expressiveness. Regarding the initialization strategy, we observed no significant performance difference on CIFAR-10 in our experiments.\n\nThank you again for your valuable feedback and for acknowledging the overall technical solidity and impact of our work. We hope that the revisions and clarifications provided here effectively address your concerns and contribute to a more favorable evaluation.\n\n[1] Chao, Chen-Hao, et al. \"Training energy-based normalizing flow with score-matching objectives.\" NeurIPS'23.\n\n[2]  Lee, Jaewoo, et al. \"Differentially private normalizing flows for synthetic tabular data generation.\" AAAI'22.\n\n[3] T. G. Dietterich, \"Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,\" in Neural Computation,1998\n\n[4] Karami, Mahdi, et al. \"Invertible convolutional flow.\" NeurIPS'19.\n\n[5]Ding, Caiwen, et al. \"Circnn: accelerating and compressing deep neural networks using block-circulant weight matrices.\" MICRO’17"}, {"review_id": "SOOx3AsDsv", "reviewer": "Reviewer_fQVx", "summary": "The authors present a new flow architecture that uses a product of diagonal and circulant matrices for the weight matrices. This enables parsimonious representation of weight matrices, and efficient forward, backward, and determinant calculation. The authors also show that this computation allows for good density estimation, and is often computationally more efficient than existing competing architectures.", "questions": "Questions/Comments\n\n - In the abstract, what is meant by \"This decomposition provides a compact representation?\" I don't understand what the phrase \"compact representation\" means here. Could you please explain?\n - In line 59, \"...storing a collection of circulant and diagonal vectors.\" The vectors are not circulant or diagonal, the matrices are, correct?\n - In line 63, \"By utilizing the Fast Fourier Transform (FFT), the determinant computation is simplified to $O(n)$.\" This sentence is not right. The runtime of the determinant computation doesn't follow from the FFT. It follows because you store the eigenvalues of the circulant layers directly.\n - Related to the point above, line 63 \"the determinant of a diagonal matrix equals the product of its diagonal elements, and the determinant of a circulant matrix is the product of its eigenvalues.\" This sentence is very odd. The determinant of every diagonalizable matrix is the product of its eigenvalues. It is not a special property of circulant matrices, and is the reason a diagonal matrix's determinant is the product of its diagonal entries.\n - In the paragraph in lines 189 - 194, I don't understand what is being said. What is \"the invertible convolutional layer?\" Are you referring to the circulant layers? Are you building a specific convolution kernel via the product of diagonal and circulant matrices? How is this similar to the 1x1 convolutions from Dinh et al.? I think this point needs to be clarified considerably, especially given the discussion in section 4.3\n\nTypos:\n - In line 134, there is a missing space between the period and reference.\n - In line 151, the t in The should not be capitalized.\n - In line 166 \"...its inverse is to compute the reciprocal...\" There is a typo here. \n - In line 169 \"is similar to diagonal matrix\" should be \"is similar to a diagonal matrix.\"\n - In line 171, the word \"with\" is unnecessary. \n - Line 203, \"handle complex and complex distributions.\" I don't understand what is meant here. \n - Line 252, there is a space missing after the word model.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n - The paper is well-written and easy to follow.\n - The core idea is simple, elegant and well-executed.\n - The presentation of the mathematics is clear and intuitive.\n - The numerical experiments, especially those presented in Section 4.3, clearly valorizes the work.\n\nWeaknesses:\n - The work has many small typos scattered throughout. Please see the questions section.\n - The experiments show that CDFlow performs well in practice, but the gains in terms of density estimation are modest, at best.", "was_revised": true, "final_justification": "The paper is quite nice. My criticisms of the paper were quite minor, and the authors have addressed them all. My recommendation stands.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive comments. We greatly appreciate the positive feedback on the clarity of our writing, the elegance of our method, and the quality of the mathematical presentation and experiments. Below, we provide point-by-point responses to each of the raised concerns. All suggested modifications have been implemented in the revised manuscript.\n\n>Weaknesses 2: CDFlow performs well in practice, but the gains in terms of density estimation are modest.\n\nWe acknowledge the reviewer’s observation. Indeed, our primary focus is not noly on boosting bits-per-dimension (BPD) or perceptual quality, but also on improving the parameter efficiency and computational complexity of flow-based models. Specifically, our method reduces the parameter count of a dense linear layer from $O(n^2)$ to $O(mn)$, where $m$ is a small constant (set to 2 in our implementation).  Moreover, by leveraging the structure of diagonal and circulant matrices, we reduce the computational complexity of matrix inversion from $O(n^2)$ to $O(n \\log n)$. These gains are especially impactful in large-scale or resource-constrained deployments, and the efficiency has been demonstrated in Section 4.3.\n\n>Question 1: \"The term ‘compact representation’ in the abstract is vague.\"\n\nWe agree and have revised the abstract to improve clarity. The updated phrasing reads:\n “This decomposition provides a parameter- and computation-efficient formulation, reducing the parameter complexity from $\\mathcal{O}(n^2)$ to $\\mathcal{O}(mn)$, while preserving the transformation ability of full linear layers.”\n This avoids ambiguous terms and emphasizes the structural efficiency without overstating representational properties.\n\n>Question 2: In line 59, \"...storing a collection of circulant and diagonal vectors.\" \n\nFor the wording in line 59, we understand the reviewer’s concern. While the matrices involved are circulant or diagonal, our implementation stores only the generating vectors for each: the diagonal elements of the diagonal matrix and the eigenvalue vector of the circulant matrix (i.e., the DFT of its defining vector). Therefore, we retain the term “vectors” intentionally and have clarified this point in the manuscript to avoid confusion.\n\n>Question 3:  in line 63, “By utilizing the Fast Fourier Transform (FFT), the determinant computation is simplified...,”\n\nWith respect to the sentence in line 63, we acknowledge the imprecision in our original phrasing. The actual runtime gain does not come directly from the FFT itself during inference, but from the fact that the eigenvalues of circulant matrices are computed once during initialization using an $O(n \\log n)$ FFT, and are subsequently reused. We have modified the sentence to:  \n\"By performing an $O(n \\log n)$ FFT during initialization, we precompute the eigenvalues of circulant matrices, allowing the log-determinant to be computed at runtime in $O(n)$ time.\" \n\n>Question 4:  The determinant of a diagonal matrix equals the product of its diagonal elements...’ is misleading.\n\nIn a related clarification, the sentence, “the determinant of a diagonal matrix equals the product of its diagonal elements, and the determinant of a circulant matrix is the product of its eigenvalues,” was indeed misleading. As correctly noted, the determinant of any diagonalizable matrix is the product of its eigenvalues. We have revised this to:  \n\"The determinant of a diagonalizable matrix equals the product of its eigenvalues. In our case, the stored diagonal and circulant vectors correspond to these eigenvalues, allowing for efficient log-determinant computation.\"\n\n>Question 5:  Lines 189–194 and the term ‘invertible convolutional layer’ are vague.\n\nThe term was used imprecisely and has now been explicitly tied to the “CD-Convolution layer” illustrated in Figure 1. We revised the sentence to:  \n\"The invertible convolutional layer, referred to as the CD-Convolution layer in Figure 1, leverages special weight matrices—specifically circulant and diagonal matrices—to enable efficient computation.\"\n\nFurther, in response to the comparison with 1×1 convolutions, our method simulates such convolutions by matrix multiplication, where the weight matrix is approximated using the diagonal-circulant decomposition. While 1×1 convolutions (e.g., in Kingma & Dhariwal, 2018) use dense $n \\times n$ kernels requiring $O(n^2)$ parameters, our approach only requires $O(mn)$ parameters. Moreover, log-determinant and inverse operations are significantly more efficient in our design, with $O(n)$ and $O(n \\log n)$ complexities respectively, compared to $O(n^3)$ in standard 1×1 convolutions and $O(n^2)$ in LU-decomposed ones. These theoretical benefits are reflected in the runtime advantages shown in Section 4.3.\n\n>Typos\n\nLastly, we sincerely thank the reviewer for identifying several typos. We have carefully reviewed and corrected the following issues in the revised manuscript:  \n- Line 134: Added missing space.  \n- Line 151: Fixed capitalization of “The.”  \n- Line 166: Corrected grammatical structure.  \n- Line 169: Inserted missing article.  \n- Line 171: Removed unnecessary “with.”  \n- Line 203: Rewrote unclear phrase.  \n- Line 252: Inserted missing space.  \n\nWe believe these revisions improve both the clarity and technical accuracy of the paper. Thank you again for your valuable feedback and for acknowledging the overall technical solidity and impact of our work. We hope the revised version addresses all concerns satisfactorily."}, {"review_id": "g9wLeabaUM", "reviewer": "Reviewer_GMUr", "summary": "This paper presents a new efficient parameterization for linear mixing layers in normalising flow architectures. It relies on a circular-diagonal decomposition of general matrices and leveraging the FFT for efficient matrix inverses and Jacobion computation required in flow models. The resulting speedup is significantly more scalable, where the matrix inversion in the inverse flow direction goes from O(n^3) to O(m n log n) as it is not bottlenecked by the FFT, and the log determinant computation becomes O(mn) due to the simplicity of Jacobians of diagonal and circulant matrices. Empirical results on natural image data with and without periodic features show that it achieve competitive or even superior performance to alternative mixing layer models, while being computationally efficient allowing significant speedup of the training process.", "questions": "- It is not clear how to choose the m value in a systematic way, is this just done heuristically? If there are any heuristic/mathematical guidelines please do include this in the method/discussion\n- Could you clarify the choices of the baselines hyperparameters, e.g. the Woodbury and butterfly settings described in 4.3?\nAre the same baseline hyperparameters for the linear transform layers used in 4.1 and 4.2? This should be stated clearly at the start of 4 if all experiments use similar baseline setups including hyperparameters.\n- A note: In the abstract, I realise nowhere the meaning of “m” is mentioned despite being in the complexity expression (related to number of diagonal and circulant matrices in the decomposition). This should be added for clarity and standalone role of the abstract.\n\nClarification on these points will help me assess whether the framework proposed actually has considered these design and thought points and can help me adjust the score ratings.", "limitations": "It would be good to mention avenues for further work and limitations such as the scaling of m required to achieve sufficient performance in general compared to dense transformations, or some discussion of the trade off between performance and the choice of m. I will be able to boost my review scores if a good discussion aspect is added.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Simple but practical idea that should be easily included in many flow implementation packages and pipelines\n- Proven idea of circulant-diagonal decomposition used in other fields like variational Gaussian processes, e.g. [Jensen, Kristopher, et al. \"Scalable Bayesian GPFA with automatic relevance determination and discrete noise models.\" Advances in Neural Information Processing Systems 34 (2021): 10613-10626.]\n- Competitive performance and evidence of advantages in structured datasets with minimal complexity overhead\n\nWeaknesses:\n- Experiments only consider image data\n- Hyperparameter choice of the method (number of matrices in the decomposition) seems a bit obscure", "was_revised": true, "final_justification": "The authors' rebuttal addresses my concerns and includes novel results from additional experiments to address the image-only concern in terms of experimental evaluations. Improvements in the explanation in the text are proposed as well, and I increase the overall quality (assuming these are executed on).", "rebuttal": "We sincerely thank the reviewer for the detailed comments and constructive suggestions. We greatly appreciate your recognition of the practical value of our method, its relevance to prior structured decompositions (e.g., circulant-diagonal in Gaussian processes), and its empirical performance on structured datasets. Below, we address each of the raised concerns.\n\n>Proven idea of circulant-diagonal decomposition used in other fields……\n\nWe appreciate the reviewer’s insightful remark regarding the broader usage of circulant-diagonal decomposition. As exemplified by [1], this technique has been successfully applied in variational Gaussian processes, further validating its generality and effectiveness. We will add this reference to the Related Work section to acknowledge its relevance and to better position our method within the wider landscape of structured matrix approaches.\n\n>Weakness 1:Experiments only consider image data\n\nTo assess broader applicability, we extended our evaluation to the flow matching task by integrating our CDLinear layer into a 5-layer MLP, forming a backbone we term CDMLP. This replaces standard linear layers with our circulant-diagonal decomposition design. Experiments on synthetic 2D datasets (chessboard, moons, concentric circles) show that CDMLP achieves comparable or better performance (MSE, FID, NLL) than MLP, while reducing parameters by ~99×. We will include these results and corresponding generated samples in the Appendix to highlight the generality of our operator beyond image modeling.\n\nTable 1. Flow matching results on synthetic 2D datasets (chessboard, moons, concentric circles). CDMLP uses our proposed CDLinear layers (with $m=2$). Compared to a vanilla MLP baseline, CDMLP achieves competitive or better accuracy with significantly fewer parameters.\n\n| Dataset       | Method     | Params | MSE ↓ | FID ↓   | NLL ↓    |\n|----|----|----|---|-----|------|\n| Chessboard   | CDMLP   | 8,453  | 3.810  | 0.0115   | 7.0425    |\n| Chessboard    | MLP        | 791,042| 3.796  | 0.0062   | 10.6853   |\n| Moons        | CDMLP   | 8,453  | 1.842  | 0.0016   | 31.0329   |\n| Moons       | MLP        | 791,042| 1.854  | 0.0022   | 24.1885   |\n| Circles      | CDMLP   | 8,453  | 2.482  | 0.0069   | 15.4922   |\n|  Circles      | MLP       | 791,042| 2.433  | 0.0019   | 42.4687   |\n\n>Weakness 2& Question 1: Hyperparameter choice of the method seems a bit obscure\n\nIn our method, each linear transform is decomposed into $2m - 1$ matrices (i.e., $m$ diagonal and $m - 1$ circulant). In our main experiments, we set $m = 2$, which results in 3 structured matrices. This setting reflects a trade-off between model expressivity and efficiency. A larger $m$ increases the approximation power but also incurs higher parameter and computational costs. \nOur objective is to achieve an optimal trade-off among model expressiveness, parameter and computational efficiency, and runtime performance. To this end, we systematically evaluated the influence of varying $m$ on model behavior.\n\nAs summarized in Table 2, increasing $m$ on CIFAR-10 yields only marginal improvements in expressiveness, as indicated by BPD values, while incurring increases in both parameter count and computational cost. Table 3 and Table 4 present the runtime measurements for both forward and reverse operations. Although a larger $m$ introduces additional latency, our method remains more efficient than alternative approaches reported in the original paper.\n\nTable 2. Performance and parameter statistics of CDFlow under different values of $m$ on CIFAR-10.\n| Model | Setting | BPD (CIFAR-10)  | DCD Params |\n|-----|------|--------|------|\n| CDFlow | m=2 | 3.31  | 6.9 K      |\n| CDFlow| m=3 | 3.30   | 11.1 K     |\n| CDFlow | m=4  | 3.30   | 15.3 K     |\n\nTable 3. Forward computation time (in milliseconds) for CDLinear layers with different values of $m$, compared against traditional $1 \\times 1$ convolutions. \n\n| Channels | $m=2$ | $m=3$ | $m=4$ | $m=5$ | $m=6$ |1x1 Conv|\n|----------|-------|-------|-------|-------|--------|-------|\n| 16     | 0.226 | 0.226 | 0.326 | 0.374 | 0.424  | 0.240    |\n| 32     | 0.323 | 0.405 | 0.547 | 0.690 | 0.837  | 0.432    |\n| 48     | 0.379 | 0.575 | 0.792 | 1.008 | 1.225  | 0.477    |\n| 64     | 0.454 | 0.729 | 1.007 | 1.287 | 1.571  | 0.535    |\n| 80     | 0.557 | 0.909 | 1.267 | 1.625 | 1.983  | 0.679    |\n| 96     | 0.719 | 1.194 | 1.676 | 2.169 | 2.644  | 0.758    |\n| 128    | 0.875 | 1.434 | 1.963 | 2.529 | 3.144  | 0.880    |\n\nTable 4. Inverse computation time (in milliseconds) for CDLinear layers with different $m$ values, compared against $1 \\times 1$ convolutions. \n\n| Channels | $m=2$ | $m=3$ | $m=4$ | $m=5$ | $m=6$ |1x1 Conv |\n|----------|-------|------|-------|-------|--------|-------|\n| 16       | 0.170 | 0.229 | 0.291 | 0.353 | 0.416  | 0.195 |\n| 32       | 0.268 | 0.407 | 0.549 | 0.697 | 0.836  | 0.328 |\n| 48       | 0.348 | 0.566 | 0.782 | 0.993 | 1.234  | 0.428 |\n| 64       | 0.428 | 0.706 | 1.011 | 1.264 | 1.544  | 0.511 |\n| 80       | 0.532 | 0.890 | 1.275 | 1.598 | 1.972  | 0.714 |\n| 96       | 0.693 | 1.168 | 1.653 | 2.135 | 2.633  | 0.757 |\n| 128      | 0.849 | 1.397 | 1.940 | 2.492 | 3.054  | 0.935 |\n\nIn addition, we conducted a separate evaluation of the log-determinant computation time across varying channel sizes from 16 to 1024. The proposed method demonstrates consistent runtime performance across different values of $m$, with execution time ranging between 0.095 ms and 0.097 ms.\n\nBased on these observations, we adopt $m = 2$ as the default setting for all experiments, including both the original and supplementary ones, unless otherwise specified.\n\n>Question2: Could you clarify the choices of the baselines hyperparameters\n\nTo ensure fairness, we adopt consistent backbone settings for the subset of flow models that share a similar multi-scale architecture—namely, Glow, Woodbury, Emerging, and ButterflyFlow—each using 3 blocks and 32 flow steps.. For Woodbury, we follow the original configuration for CIFAR-10 with $d_s = \\{16, 16, 8\\}$ and $d_c = \\{8, 8, 16\\}$; for ButterflyFlow, we set $butterfly level=1$ as per its default. Thank you for pointing this out — we will clarify these settings at the beginning of Section 4.\n\n>Question 3: In the abstract, I realise nowhere the meaning of “m” is mentioned……\n\nTo improve clarity and ensure that the abstract is self-contained, we will revise the sentence as follows:  \n\"Furthermore, leveraging the Fast Fourier Transform (FFT), our method reduces the time complexity of matrix inversion from $O(n^3)$ to $O(mn \\log n)$ and matrix log-determinant from $O(n^3)$ to $O(mn)$, where $n$ is the input dimension and $m$ denotes the number of diagonal matrices used in the decomposition, which are combined with $m-1$ circulant matrices to construct each CDLinear layer.\"\n\n>Limits：It would be good to mention avenues for further work and limitations……\n\nAs discussed earlier in our response regarding the choice of $m$, we now aim to further explore the representational capacity of this decomposition scheme. To this end, we reproduced the experiments from Appendix A.5 of [2] and included results using circulant matrices and diagonal-circulant-diagonal ($\\text{DCD}$) matrices. As shown in Table 5, the circulant matrix achieves faster convergence and yields performance closest to the full matrix. Moreover, the performance of the $\\text{DCD}$ method significantly surpasses that of $\\text{LU}$ and the individual $\\text{L}$ and $\\text{U}$ decompositions, highlighting the expressive potential of this class of matrix decompositions.\n\nTable 5. Mean NLL ± 95% CI on synthetic data (step 5000 / 10000)\n| Type  | 5000 NLL     | 10000 NLL    | \n| ----- | ------------ | ------------ | \n| full  | 49.11 ± 0.01 | 48.32 ± 0.12 | \n| tril  | 51.91 ± 0.34 | 50.89 ± 0.18 | \n| triu  | 50.27 ± 0.38 | 49.61 ± 0.73 | \n| trilu | 51.85 ± 0.49 | 51.04 ± 0.22 | \n| circ  | 49.07 ± 0.24 | 48.49 ± 0.39 | \n| dcd   | 50.18 ± 0.38 | 49.21 ± 0.12 |\n\nFrom a theoretical perspective, our decomposition can approximate any linear operator, and since convolutions can be rewritten as matrix multiplications, our method is in principle capable of representing any convolutional transformation. However, in the context of normalizing flows, this capacity is constrained by the need to compute the matrix inverse and log-determinant during inference. When convolution is converted to a matrix multiplication, the resulting weight matrix is typically non-square, making the determinant and inverse undefined in the classical sense.\n\nAddressing this limitation is a key direction for future work. One possible avenue is to explore extensions of determinant and inverse computation to non-square matrices, such as using the the Moore–Penrose pseudoinverse [3] orRadic determinant[4]. By incorporating these tools, we hope to generalize our method to a broader class of transformations, including non-square and structured convolutional layers, thereby expanding the applicability of our framework to arbitrary function classes.\n\nThank you again for your insightful feedback. We hope these clarifications and additions address your concerns and provide a clearer view of the design and potential of our framework. We would greatly appreciate your reconsideration of the manuscript in light of these improvements.\n\n[1]Jensen, Kristopher, et al. \"Scalable Bayesian GPFA with automatic relevance determination and discrete noise models.\" NeurIPS'21: 10613-10626.\n\n[2]Chao, Chen-Hao, et al. \"Training energy-based normalizing flow with score-matching objectives.\" NeurIPS'23: 43826-43851.\n\n[3]Penrose, Roger. \"A generalized inverse for matrices.\" Mathematical proceedings of the Cambridge philosophical society. Vol. 51. No. 3. Cambridge University Press, 1955.\n\n[4]Radic, M. \"A definition of the determinant of a rectangular matrix.\" Glasnik Matematicki 1.21 (1966): 17-22."}]

Author rebuttal:

Meta-review: This paper proposes CDFlow, which replaces dense 1×1 mixing layers with a product of diagonal and circulant matrices to enable efficient computation of invertible layers in normalizing flows. 

The paper received generally positive reviews: one borderline accept (4) and three accepts (5). The main concerns raised during the review and rebuttal phases were:

- **Technical novelty**: While circulant–diagonal networks have been studied previously, reviewers questioned whether the contribution is incremental.
- **Missing related work**: Several relevant papers (e.g., ACDC) were initially omitted.
- **Clarity issues**: Reviewers highlighted unclear discussions on the reliance on 1×1 convolutions versus Fourier-domain mixing, inconsistencies in complexity statements, vague use of terms such as “compact representation,” and imprecise explanations of determinant computation.
- **Experiments**: The experimental section lacked error bars and a deeper parameter–quality–runtime trade-off analysis. Figures were noted to be small and somewhat unclear.

The author rebuttal was thorough and convincing. Reviewers acknowledged that most concerns were resolved in substance, with remaining issues confined to clarity and presentations in experiment ****that can be fixed in the camera-ready version. After carefully reading the reviews and rebuttal discussions, AC agrees with the reviewers’ decision and recommends acceptance.
Decision: Accept (poster)