Unlocking the matrix form of the Quaternion Fourier Transform and Quaternion Convolution: Properties, connections, and application to Lipschitz constant bounding
Authors: Giorgos Sfikas
George Retsinas
Venue: Accepted by TMLR ()

Abstract: Linear transformations are ubiquitous in machine learning, and matrices are the standard way to represent them. In this paper, we study matrix forms of quaternionic versions of the Fourier Transform and Convolution operations. Quaternions offer a powerful representation unit, however they are related to difficulties in their use that stem foremost from non-commutativity of quaternion multiplication, and due to that $\mu^2 = -1$ possesses infinite solutions in the quaternion domain. Handling of quaternionic matrices is consequently complicated in several aspects (definition of eigenstructure, determinant, etc.). Our research findings clarify the relation of the Quaternion Fourier Transform matrix to the standard (complex) Discrete Fourier Transform matrix, and the extend on which well-known complex-domain theorems extend to quaternions. We focus especially on the relation of Quaternion Fourier Transform matrices to Quaternion Circulant matrices (representing quaternionic convolution), and the eigenstructure of the latter. A proof-of-concept application that makes direct use of our theoretical results is presented, where we present a method to bound the Lipschitz constant of a Quaternionic Convolutional Neural Network. Code is publicly available at: https://github.com/sfikas/quaternion-fourier-convolution-matrix.

Published in Transactions on Machine Learning Research (11/2025)
Unlocking the matrix form of the Quaternion Fourier Trans- form and Quaternion Convolution: Properties, connections, and application to Lipschitz constant bounding
Giorgos Sfikas
gsfikas Duniwa.gr
Department of Surveying and Geoinformatics, School of Engineering
University of West Attic
Athens, Greece
George Retsinas
gretsinas@@centrrl.ntuu.
School of Electrical and Computer Engineering National Technical Universily of Athens Athens, Greece
Reviewed on OpenReview:
Abstract
Linear transformations are ubiquitous in machine learning, and matrices are the stan- dard way to represent them. In this paper, we study matrix forms quaternionic ver- sions of the Fourier Transform and Convolution operations. Quaternions offer a powerful representation unit, however they are related to difficulties in their use that stem fore- most from non-commutativity of quaternion multiplication, and due to that n² -1 possesses infinite solutions in the quaternion domain. Handling of quaternionic matri- is consequently complicated in several aspects (definition of eigenstructure, determi- nant, etc.). Our research findings clarify the relation of the Quaternion Fourier Trans- form matrix to the standard (complex) Discrete Fourier Transform matrix, and the exlend on which well-known complex-domain theorems extend to quaternions. We focus espe- cially on the relation of Quaternion Fourier Transform matrices to Quaternion Circulant matrices (representing quaternionic convolution), and the eigenstructure of the latter. A proof-of-concept application that makes direct of theoretical results is presented, where we present method to bound the Lipschitz constant of Quaternionic Convolu- tional Neural Network. Code is publicly available at: https: /gtthub....///ggg///////gg...h.t com/ sfikas/ quaternions cnntternion-foursii
on on-matrix
1 Introduction
Quaternions are Cour-dimensional objects that can be understood as generalizing the concept of complex num- bers. Some of the most prominent applications of quaternions are in the fields of photogrammetry (Förstner & Wrobel, 2016), computer graphics (Vince, 2011), robotics (Daniilidis, 1999; Fresk & Nikolakopoulos, 2013) and quantum mechanics (Adler, 1995; Susskind & Friedman, 2014). Perhaps the most well known use case of quaternions involves representing rotations in three-dimensional space (Kuipers, 1999; Stillwell, 2008; Eater & Sanderson, 2018). This representation is convenient, for example composition of rotations corresponds to multiplication of quaternions, and both actions are non-commutative. Compared to other spatial rota- tion representations such as Euler angles, quaternions are advantageous in a number of respects (Förstner & Wrobel, 2016).
A relatively more recent and perhaps overlooked use of quaternionic analysis traces roots digital signal & image processing (Sangwine, 1996; Sangwine & Le Bihan, 2006; Subakan & Vemuri, 2011; Grigoryan & Agaian, 2014; Miron et al., 2023). The motivation for using quaternionic extensions of standard theoretical

---

Published in Transactions on Machine Learning Research (11/2025)
signal processing tools such as filtering and the Fourier Transform (FT), is that their most typical application on color images involves treating them as three separate monochrome images. As consequence, cross- channel dynamics are ignored in this approach. The solution is to treat each color value as a single, unified object, and one way to do so is by representing multimodal pixel values as quaternions. The caveat of this approach would be that standard tools, filters and methods would have to be redefined and remodeled, a process that is typically not always straightforward (a fact that partly motivates this work). Similar considerations hold for other types of signals, such as acoustic signals or polarimetric imaging (Miron et al., 2023). Recent works move away from the scope set originally by such pyrsically-bastte motivations and extend to other applications, including using quaternion-valued elements as components of Deep Neural Networks. Part of the network may be expressed in terms of quaternions (Rosa et al., 2018; Hsu et al., 2019; Qin et al., 2023) or even all components of the network -inputs, intermediate layers, outputs- as done in Quaternionic Neural Networks. The latter lead in practice to models that can incorporate quaternionic versions of standard layers such as convolution or self-attention, have much less memory requirements than their real-valued counterparts, and perform similarly in terms of generalization (Zhu et al., 2018; Parcollet et al., 2020; Zhang et al., 2021). Parameterized hypercomplex layers build on this paradigm (Zhang et al., 2021), and neural networks can be effectively extended to work with elements of an arbitrary number of dimensions.
In this paper, our focus is on the quaternionic version of the Discrete Fourier transform (DFT) and its relation to Quaternionic Convolution. The Quaternion Fourier Transform (QFT) was originally proposed by Sangwine (1996), and in effect it provides the means to analyze and manipulate the frequency content of multichannel signals. We argue that, while the QFT has been studied by a number of works and has found uses in practical applications (Ell & Sangwine, 2007; Subakan & Vemuri, 2011; Li et al., 2015; Hitzer, 2016), its potential has yet to be exploited in its fullest
From another standpoint, we can frame our work as serving to elucidate the properties of Convolution in H. We contribute to the better undorstanding of the convolutional operator, and provide the means to analyze and manipulate ils spectral properties. Convolution is a ubiquitous operation in data science and machine learning, and its instance in the quaternion domain forms challenging, yet interesting and promising tool that is hitherto underexplored. The extensive use and long history of convolution in the context of neural networks is very well-known (Prince, 2023); while it can be argued that the field is rapidly evolving with non-convolutional backbones such as Transformers or State-Space models (Gu & Dao, 2023), the intrinsic connection of convolution with notions such as invariance and equivariance (Kondor & Trivedi, 2018) or the already known connection to frequency analysis in R (Jain, 1989) (and extended with the current work in HI) are strong pointers in favor of the continued relevance of convolution. The success of hybrids of non-convolution with convolution or convolutional components seems to corroborate this point (Wick et al., 2022; Zhang et al., 2019; Xu et al., 2023; Gu & Dao, 2023; Yu & Wang, 2025). Our work serves to further widen the usefulness and domain of application of convolution.
A matrix representation form for the Quaternion Fourier Transform and Convolution and their propertics are studied in this p We show that a number of properties of the DFT matrix form also do exlend to the quaternionic domain, in most cases appearing as a more complex version of the original (DFT) property. We focus on its relation to Quaternionic Convolution and its circulant form, extend the well-known results for the quttuuateii case (Jain, 1989), and show that the eigenstructure of Quaternionic Circulant matrices is closely connected to the Quaternion Fourier Matrix form.
The major contributions of this paper are the following:
1. We show that convolutions and Fourier transforms in H are represented as matrices with prop- erties that are analogous to their real and complex-domain counterparts; the standard (complex) Fourier matrix can be seen a special case out of an infinite set of Quaternionic Fourier matrices (cf. Propositions 3.1-3.3), which are related through a rotation-like operation (cf. Prop. 3.4).
1 |Concerning prior work, to our knowledge only (Ell et al., 2014, 1.1.1) refer matrix form of the QFT, without proving or discussing any of its properties, its relation to convolution or their implications.
2

---

Published in Transactions on Machine Learning Research (11/2025)
6
4
7
0.7 + 0.0k
0.8 +.3+ + 0.4] ) + 0.2k
0.0 0.0j + 0.0k
7
0.5 0.5 0.5k
0.6 0.81 +.0 + 0.0k
0.0 0.71 0.7] 0.0k
2
2
-0.0 0.60 0.6- 0.6j 0.5k
0.7 + + + 0.0k
0.8 0.31
0.4 0.2k
Matrices of real-valued elements (top left) form a standard representational unit for unidimen- sional data, such as grayscale pixels (boltom left), neural network weights other unidimensional signals. Matrices of guaternion-valuer valued elements (top right) represent arrangement of multidimensional data such as color images (Ell & Sangwine, 2007) 3D spatial rotation (Kerbl et al., 2023) (bottom right).
Figure 1:
We show that circulant, doubly block-circulann matrices and Fourier matrices are "still" related in terms of their properties also the quaternionic domain, albeit a more nuanced manner. In our opinion, it notable that both the left and right QF provide us with different functions of the convolution kernel left eigenvalues (cf. Proposition 3.5). This is result with further theoretical and practical implications, which discuss (cf. Corollary 3.6.2,
example).
3. The literature on the theory and applications of quaternion matrices focuses on right eigenvalues, as the left spectrum is more difficult or impossible to compute in general. Duc to our results (Proposition 3.5), our focus must turn to left eigenvalues 2, which we show how to compute using the QFT; we show that set of N left eigenvalues (out of an infinite number of left eigenvalues of x N-sized matrix) can be used reconstruct Circulant matrix, under specific conditions (ef. Proposition 3.6).
4. We show that sums, multiples and products of quatermionic circulant matrices can be reconstructed using manipulations in the left spectrum (ef. Proposition 3.7).
In terms of application, we show that our results find direct use, replacing and generalizing uses of standard real/complex operators (cf. Scction 4). Two-dimensional convolution kernels are shown to be represented as doubly block-cireulant constructions (cf. Propositions 3.8,3.9). We present prooffof-concept application that makes use of our results. Specifically, a) compute the spectral norm of Quaternionic Convolution and b) set bound on the Lipschitz constant of a Quaternionic Neural Network. These operations are shown to be carried out orders- of-magnitude faster than an implementation that is oblivious to the presented results. This involves a special construction of a matrix that contains 2 x 2 blocks simplex and perplex parts of left cigenvalues (cf. Proposition 4.4, Corollary 4.4.1). This generalizes elegantly from to HI, as perplex parts vanish in C.
propose way to
The paper is structured as follows. We proceed to review preliminaries on quaternions and quaternion matrices in Section 2. This includes a short discussion concerning difficulties handling quaternion matrices and therefore operator matrix forms, and outline the net benefits from such a process. In Section 3, we examine the matrix form of the QFT and Quaternion Convolution and present our theoretical contributions regarding their properties. We showcase the usefulness of the discussed form and its properties in Section 4. We conclude the main paper with Section 5. In the Appendix, have moved supplemental results,
Most hitherto works focus right eigenvalues; Aslaksen (2001) notes, "In general, is difficult talk about eigenvalues of quaternionic matrix. Since work with right vector spaces, must consider right eigenvalues."
3

---

Published in Transactions on Machine Learning Research (11/2025)
short section notation conventions, section on preliminarics on Quaternion algebra, and additional details on models and proofs.
2
Preliminaries on Quaternions and Quaternion Matrices
In this Section, we shall attempt to outline a selection of important definitions, results and key concepts concerning quaternions (Alfsmann el al., 2007; Cheng & Kou, 2016; Ell Sangwine, 2007; Ell et al., 2014; Le Bihan, 2017), and also provide a concise outlook regarding the pros and cons of their uses practice.
2.1 Elements of Quaternions
The structure H represents the division algebra that is formed by quaternions. Quaternions share a position of special importance compared to other algebraic structures, according to Frobenius' theorem (Fraleigh, 2002), every dimtte-dieeniooal (associative) division algebra over R must be isomorphic either to R, or to C, or to H.. Quaternions E HH share the following basic form:
q = + bi + cÿ dk,
(1)
where a, b, c, E IR and j. k are independent imaginary units. Real numbers can be regarded as quaternions with b, e, = 0, and complex numbers can be regarded as quaternions with c, O. Quaternions with zero real part, i.e. a 0, are called pure quaternions. An alternative way to represent quaternions, by writing their real and collective imaginary part separately. In particular:
= S(g) + V()).
(2)
where S(g) and V(g)= = bi + cjj - dk. For all three imaginary units i,j, k, it holds 12 = و² The length or magnitude of quaternion is defined as lal V = / va² + + + di where 7 is the conjugate of 9, defined as 7 = a bi 15-c cj dk. Quatermions with [4l = 1 are named unit quaternions. For any unit pure quaternion μ4, the property 2²² = -1 holds. Exponentials of quaternions ex for x E H can defined through their Taylor series:
1.
(3)
n!
From eq. 3, Fuler's identity extends for quaternions, for a unit and pure:
euo = costl + psin A.
(4)
Note also the caveat that in general eat & e In particular, for a, E R+ and two distinct pure unit quaternions μ, v, we have wua eatus Equality holds however, when quaternions μt, v commute. Hence,
to u8 uB e g = eatt
(5)
since real numbers commute with any quaternion. Furthermore, any quaternion can be written in polar form as:
g g= lglglen0
(6)
Unit pure quaternion j and real angle are called the eigenaxis and eigenangle (or simply axis and angle or phase) of the quaternion (Alexiadis & Daras, 2014). The eigenaxis and eigenangle can be computed as: μ (())/VV)) tan -un--(( -IVV (())))(()))) For pure y, hence S(q) O, we have 0 = 7/2.
In terms of number of independent parameters, or "degrees of freedom" (DOF), note that the magnitude lal corresponds to 1DOF, axis μ to 2DOFs and 0 to 1DOF, summing to total of 4DOF. Quaternion multiplication is in general non-commutative, with:
ij -j k. kjj i, ki -ik j.
(7)
4

---

Published in Transactions on Machine Learning Research (11/2025)
Note the analogy of the above formulae to vector products of 3d standard basis vectors. Indeed, these are generalized with the formula for the product of two generic quaternions P.q H:
pq ((p)S(g) V(p) V(() + (g) + (p)) x V(a),
(8)
where and denote the dot and cross product respectively.
Similarly to quaternion multiplication, quaternionic convolution is non- commutative (f x 9 # g f in general, where f.g E HHN) and conjugation inverses the order of convolution operators ( * - g f). We have left convolution and right convolution:
Quaternionic convolution:
(he f)
Σ n=
hzliffn i].
(9)
N-1
(f ((+)n)) = f1n-ijhnlp [mm))
(10)
n=0
where the difference between the two formulae whether the convolution kernel elements multiply the input from the left or right. A bi-convolution operator can also be defined (please refer to the Appendix for more details).
Quaternion matrices, eigenvalues and eigenuectors: Matrices with quaternion-valued entries will be denoted as A E MNN (see Fig. 1 for a visual reference). An important complication of standard matrix calculus comes with matrix eigenstructure and determinants. Due to multiplication non-commttativity, we now have two distinct ways to define eigenvalues and eigenvectors. In particular, solutions to Ar Xx are left cigenvalues and cigenvectors, while solutions to Ar æλ are right cigenvalues and cigenvectors. The sets of left and right eigenvalues are referred to left and right spectrum, denoted (1(-) and or(()) respectively, The two sets in general different, with differing properties finite matrices, unlike real and complex matrices (cf. Fig. 2). We note here the following important lemmas on the left and right spectrum (Huang & So, 2001); we have
well. Both sets have infinite members for
aippT qA) = (p + gl 1 (A)).
where A E N*N, P.g H., 1 is the identity matrix in N This property does not hold for the right spectrum.
For the right spectrum, we have
or oU AU) = =o)(A), where A, U NNN and U is unitary. Also, e (A( = de for q + 0. These properties do not hold for the left spectrum.
For additional important results, including the circular convolution, the extension of the convolution theorem, the Quaternion SVD and the Cayley-Dickson form, we refer the reader to the Appendix. Other useful results Quaternion matrices can be found in the important treatise of Zhang (1997).
2.2 Discussion on difficulties and benefits associated with Quaternionic representations
Major difficulties in handling quaternionic matrix forms for the FT or the convolution operator involve the following points: a) Multiplication non-commutoatviity Quaternion multiplication is non-commutative, property which is passed on to quaternion matrix multiplication. b) Multiple definitions of convolution and QFTs. There is left-side, a right-side, or even a two-sided Fourier transform for quaternions, well as variations stemming from choices of different FT axes. (An analogous picture holds for quaternion convolution). c) Difficulties with a convolution theorem: A long discussion exists on possible adaptations of the convolution thcorem the quaternion domain (Ell & Sangwine, 2007; Cheng & Kou, 2019; Bahri et al., 2013: Pei et al., 2001; Ell et al., 2014). The convolution theorem cannot be readily applied for any version of the QFT and any version of Quaternion Convolution (Cheng & Kou, 2019). Quaternion convolution theorems
5

---

Published in Transactions on Machine Learning Research (11/2025)
have however been proposed in the literature: Ell & Sangwine (2007) discuss a convolution theorem for the discrete QFT; Bahri et al. (2013) present results for the two-sided continuous 2D QFT. A version with the commutative bicomplex product operator also exists, holding specifically for complex signals transformed w.r.t. to axis j (Ell et al., 2014). d) More compler eigenstructure: Quaternion matrices have a significantly more complex rugenst than real or complex matrices. Due to non-commuttalvvtty of multiplication, left and right eigenvalues are defined, each corresponding respectively. Even worse; the number of the eigenvalues of quaternionic matrix is in general infinite (Zhang, 1997). e) Quaternionic determinants: The issue of defining Quaternion determinant is non-trivial; function possessing all the standard properties of the complex-valued determinant cannot exist in the quaternion domain (Dyson, 1972).
either the problem Ar = Ax or Ar æA
But what can we gain from a matrix form and the properties the constituent quaternionic matrices, espe- cially when moving to the quaternion domain creates all sorts of difficulties? In a nutshell, the motivation is that voctorial signals such as color images or arrangements of rotation representations can be treated in holistic manner, and our analysis opens up the potential to build more powerful models directly in the quaternionic domain. Given signal processing observation model, when Fourier operator of Convo- lutional operator is defined, matrix form allows us to casily adapt the model to the quaternion domain. This bears at least two advantages: a) The model can be easily redefined by replacing operators with their quaternion versions. b) Perhaps more importantly, solution of the model (i.e., find some parameter vector 0 that fits best to observations) involves directly using properties of the Fourier and Circulant matrices (e.g. using results about eigenvalues of a sum of circulant matrices).
Properties of the related non-quaternion operations used in state-of-the-art signal, image processing/v /visinngg sion and learning methods, exploiting the relation of Convolution and Fourier Transform in the real and complex domains. As examples of use, we can mention models in deblurring (Nan et al., 2020) and deconvo- lution (Hidalgo-Gavire Gavira et al., 2019), manipulating convolution layers of neural networks (Sedghi et al., 2018; Singla & Feizi, 2020) or constructing invertible convolutions for normalizing flows (Karami ct al., 2019). These properties are well-kuown for the real and complex domain (Jain, 1989); with the current work, these properties are eztended and adapted from their domain (H), allowing construction of more powerful and expressive models.
complex domain version (C) to the quaternionic
(A) EC
0 (A(A) E H
or (A) H
E RRR.N. NxN
A E N
The whole is more than the sum of its parts: Quaternionic matrix cigenstructure is significantly richer than the eigenstructure of real matrices. The set of all eigenvalues of real-valued matrix forms the spectrum (-) of the matrix; for real-valued (and complex-valued, matrices, it is a subset of C. In quaternionic matrices, we have two distinct spectra, the left spectrum ar(-) and the right spectrum a,(+). corresponding to forms Ax = Ax and Ar = zλ respectively. No known well-posed connection exists that relates the two spectra, and in general they can both be infinite sets. The literature focuses on right. eigenvalues, as no general method cxists to compute the left spectrum. In this paper, our focus turns to the usefulness of the left spectrum, which show how compute efficiently in the case of circulant-structuured matrices.
Figure 2:

---

Published in Transactions on Machine Learning Research (11/2025)
3 Matrix forms: Quaternion Circulant Matrix, Quaternion Fourier Matrix and their connection
The main theme of this section is exploring whether and to what extent well-known properties and relation between circulant matrices, the Fourier matrix, and their eigenstructure generalize the quaternionic do- main. The core of these properties (see for example Jain (1989) for complete treatise) is summarized in expressing the convolution theorem in matrix form, as:
CS - Ag ACAf - G AcF
(11)
where f E CN C CNxN is circulant matrix, G, CN the Fourier transforms of g, f. As the columns of the inverse Fourier matrix A are eigenvectors of any circulant matrix, the convolution expressed by the circulant C' multiplied by the signal f is easily written in eq. 11 as a point-wise product of the transform the convolution kernel Ac (represented as a diagonal matrix) and the transform F. These formulac in practice aid handling circulant matricos, which when used the context of signal and image processing models represent convolutional filters. The most important of these operations are with respect compositions of circulant matrices, as well as operations in the frequency domain.
3.1 Quaternionic Circulant Matrices
Let C E NNN be a quaternionic circulant matrix. This is defined in terms of quaternionic "kernel" vector, denoted kc HH with quaternionic values ke [co Ci ci.--eN-I]'- Then, the element at row and column = 0, 0.1...,,NN - 1 is equal to koli - jlN, where we take the modulo-N of the index in brackets. Hence, a quaternionic circulant matrix bears the following form:
= 0,1,..,,N -
CN-1
CN-1
Co
C
CN-2
CT
(12)
C1
co
For any quaternionic circulant C, from the definition of the quaternionic circulant (eq. 12) and quaternionic convolution, we immediately have: Proposition 3.1. The product Cr implements the quaternionic circular left convolution kc @
N-1
(kc @ x)(d = Σ keli-njja n]xx([],
(13)
R=0
taken for vi E [0, N - 1), where E HN is the signal to be convolued, and Fl denotes modulo-N indewing (Jain, 1989) (i.e., the index "wraps around" with period equal to N).
Proposition 3.2. Quaternionic circulant matrices can be written as matriz polynomials:
C= + ci c2 +... - PN-1
(14)
where kernel [cnc] ... cNl [ E N and P E NxN is the real permutation matria (Strang, 2019) that permutes columns 0 + 1,1 + 2, N N also quaternionic circulant.
The inverse is also straightforward: any such matrix polynomial is
Corollary 3.2.1. Transpose CT and conjugate transpose O are also quaternionic circulant matrices, with kernels equal to (Co CN-1 ..-1.000 C2 ar and (Co CN-1 .2 a) respectively.
Proof. We use pM PT p-1 and PN-a p = I for any Proposition 3.2:
N. We then have, by using
CT co PN +c C PN + PN- 2 ... .....o CN-1P.
(15)
and
"" - + PN-1 1ox-1+ap pN- +... CN
(16)
hence both polynomials over P9 thus quaternionic circulant.

---

Published in Transactions on Machine Learning Research (11/2025)
3.2 Quaternionic Fourier Matrices
We define class of matrices Quaternionic Fourier matrices, shorthanded QN for some pure unit quaternion (termed the "axis" of the transform) and N E N, as follows. The element at row i = 0, ......N-- and column 0.l...., N - of the product of i.j. Hence, write:
equal w zij Nu
where we have used WNP = ---=N raaied to the power
0-0 W Ny 0-1 W Ny
1-0 Ny
w (N-1). Nu 13-11
1-1
Ny
to Ny.
2%
(17)
VN
D-(N-1)
1-(N-1) w N2
(N (N-1)-(N-1) w Np
w Nu
Proposition 3.3 (General properties). For any Qualernionic Fourier matrix we have the following straight- forward properties:
(1) Q'N is square, Vandermonde and symmetric.
(2) 1QN 1 ON = QNIQN1 I, Q'N is unitary.
(3) The product Q' where H.N, equals the left QFT Fi(r):
N-1
FM =
Σ n=0
e -22m na f(n].
(18)
(4) The product Qy Qx*x,, x, where x E H equals the left inverse QPT FT, F7," (x}
N-1
- nu s(m].
(19)
n=0
(5) Q AN where Ax is the (standard, nontquatervionic) Fourier matrix of size N (Jain, 1989; Strang, 2019). The standard DFT comes special case of the QFT, for aris μ and complex-valudd inpul x E CN
(6) QN" (Q Q'x (Q%)
(7) O' = P, where P is a permutation matrix that maps column n to [N - nl..
(see also the Appendix for brief discussion over variants of the Quaternion Fourier Transform).
Proof. These propertics are straightforward generalization from C, and can be confirmed by using the matrix form definition of eq. 17. Let only add short comment on the derivation of 3.3.7. We practically need to prove that each column of Q is perpendicular to exactly that this mapping is given by n + N - (where we use a zero/ pro//moduooN convention, cf. Appendix B). For columns an and Qu we compute the result of the required product at position (u, v) as:
other column of the same matrix, and
W2mN - 'n
a ag
N Σ
p2mN na n -22-N no +
n=0

---

Published in Transactions on Machine Learning Research (11/2025)
which holds since 2 1. This is subsequently equal to
+22nN "n(N-u) 22nN
-22 'nv
aN- ae.
which is the inner product of columns and N N-u4 u. This will result in zero for all pairs of N u except for when v = [N - uI... Hence, the required product will have all-zero columns save for exactly one element, different for each column; this is by definition a permutation matrix (Strang, 2019).
Consequently, and in contrast to what holds in the complex domain, we have infinite number of different Fourier matrices for a given signal length N, one for each different choice of pure unit axis Proposition 3.3 stated that by flipping the sign of the axis we obtain the inverse QFT with respect to the same axis. In general, two arbitrary Fourier matrices are connected via a rolation of their components:
Proposition 3.4. Let Qix, Q'N Quaternionic Fourier matrices with non-collinear azes p.v. We can always find unit E H. such that
QN = =PQNPP
(20)
The required quaternion is 60/2, where { vu + V(v). (u)) and 0 = arccos(V V())..
Proof. For any WNy and WN., we can find H such that WNy = pWN-P"1 Since any pure unit quatermions v. are situated on the unit sphere {t E Ri + Rj + lKk |t| = 1}. we can obtain the one from the other by applying a rotation about the origin. Consequently, there cxists E H so that = pvp- (Stillwell, 2008) (note that WN are unit but not necessarily pure). From there, assuming pure unit μ and real we have the following steps: e pvp to = coso + pup sinf = p(cos0 + usind)p=1 pe"
It suffices to plug = 2x in the previous equation to conclude the required WNn = pWNNp" The same transform can be used for all powers of WNH WNr to any exponent 7, and indeed we have pm) p WNDP-1 WN-WNN ... UNn = wyy.- Since any element of QN can be writlen power W'n, for some natural exponent y, we can write Q = pQ which applies the same transform on all elements simultancously. Intuitively, this action represents a rotation of the unitary disc where all
= pwNnp www.:="wwwx--ppp..
elements of Q" are situaled.
The axis of rotation is given by the cross product of the two pure unit vectors p,v, as it is by definition perpendicular to the plane that u,v form. The product V(v) V(μ) is equal to vu V(v) V(a) 5(1) S(v) = 0, hence we compute & un V(u) V(n). Also, V(u) V(v) = =Iull/coo(()) 0 = (V(() ((n)-V(v)) V(v)).
Let us add a short comment on the intuition behind this relation. We must note that all elements of Q are situated on the same plane in H, which different from the plane for elements of Q (due to the assumption of non-collinearity). However, for any given axis, all planes intersect the origin and all pass from (1,0,0, 0) H, since element [1 equals for any μ and N N. Elements of Q are situated on a unit circle on their corresponding plane, whereupon their position is defined by their relative angle to the line passing between the origin and (1, 0, O, 0). This position is determined by the value of the element WNH: and the power to which this element is raised, so as to give the element of Q% in question. Intuitively, this action represents a rotation of the unitary disc where all elements of Qy are situated (see Fig. 3 for a visual reference of this concept).
3.3 Connection of Circulant and Fourier matrices
The following results then be proved, underpinning the relation between Quaternionic Circulant and Quaternionic Fourier matrices, in particular with respect to the left spectrum of the former:
Proposition 3.5 (Cireulant & Fourier Matrices). For any C NxN that is circulant, and any pure unit / H.
9

---

Published in Transactions on Machine Learning Research (11/2025)
Tna Fatt
-10
Haggery/40
11.3
The relation of the DFT matrix the QFT matrix. In the top row, we see a visualization of Discrete Fourier Transform (DFT) matrix (top left.) and a plot of the elements of its second columm. All elements powers of (top right). In the bottom row, matrix (bottom left), and a plot of the same column. Here, elements are powers of w e - where unitary E III is called the axis of the transform (bottom right). Similar considerations are in place for both matrices, as all elements are powers of w. Again, elements form a disc
Figure 3:
28 and they elegantly form unitary disc the real-complex plane
visualization of 8 x 8 Quaternionic Fourier Transform (DFT)
the rest of the columns that lays on a plane, the orientation of which controlled by μ. For μ i, we obtain the DFT matrix. (For purposes of visualization: a) The top-left plot represents the DFT matrix which complex (C), but is depicted in grayscale in order to emphasize its relation with the QFT matrix as special for μ b) The bottom-right plot shows projection of Hi to space k = 0.)
(1) Any column k 0, 0.1....,N 0,
of the inverse QFT matria Q an eigenvector of C. Column k corresponds to the 1th component. of the veclor of left. eigenvalues X [N ... NN E H. Vector X is equal to the right QFT Fu of the kernel of C.
(2) Ang column k 0, 1,...... corresponds to the tth component of the vector of left eigenvalues R" Ke" ...- 1 HN The conjugate of the vector x² is equal to the left. QFT JH of the kernel of C.
of the inverse QFT matrir Q is an eigenvector of CH. Column k
10

---

Published in Transactions on Machine Learning Research (11/2025)
where transforms denoted with asterisk (*) refer to using unitary coefficient instead of 1/VN:
N-1 flrle -22NN
F 2 p2m 22nN- f(n), F [1] Σ
(21)
Note here that the left spectrum becomes important concerning the circulant matrix eigenstructure. (In general, quaternionic matrices have two different spectra, corresponding to left and right eigenvalues. For real matrices, the two spectra will coincide, however the exact way these two sets connected is largely unknown in the case of true quaternionic matrices (Zhang, 1997; Macias- Virgós et al., 2022)).
Proof. Let ag the hthh column of QN', Its product with quaternionic circulant C' is computed as:
N-1 h|0 n.] w kn
h[1 ná. N NNN
VNCa== =
Enco h|2 N
(22)
Ex 4[N - 1 n N kn
where b is the jth /modulo-N element h; assumed to be the kernel of circulant matrix C. The m clement of Cag then:
N
m (N-1)
h/m n| w. km
Σ hlllww
kn
Emm
(N
N-1
Σ
Σ
Ny
w Nu
m
00
(23)
where we used f = m n and in the last form of equation 23, the term inside the brackets (E h N have the 4thh element of the right-side QFT FR of h. (Note that this is the asymmetric version of the + -km QF ef. Appendix). Outside the brackets, vN "N can be identified the m element of ar; shorthanded as [ax]m. Therefore, Ca# = Aag, so a is an eigenvector of C:; by the previous argument, À is equal to the gth cloment of the right-side QFT. The second part of the theorem, concerning CH,, is dual to the first part.
)
Corollary 3.5.1. For any pure wnit awis H., the conjugates of the eigenvalues A and ret are also left eigencalues of " and C respectively.
Proof. We will prove the corrolary for Nh and C as the case of k and C is dual to the former one. We have Cw = X""
(C X" 0 given non-zero eigenvector w, hence the mullspace of C - X I is N and rank[C" X -X1 < N,
non-trivial. From this, we have rank - X N = where we have used the property rank(A) ") (Zhang, 1997, theorem 7.3). Thus, the nullspace of C - I is nontrivial, and X²¹ an eigenvalue (note that this proof does not link these eigenvalues to specific eigenvectors, however).
Corollary 3.5.2. For any pure unit axis μ E H. the vector of left eigenvalues X is flipped version of X - where the DC component and the (N/2)th component (zero-indezed) remain in place.
Proof. We want to prove that the tth component of X (Xf+) is equal to the [N = component of ~ (X where ordering is w.r.t. modulo-N indexing). But the Xi is left cigenvalue that corresponds to the 4th column of the QFT matrix with axis i as its eigenvector, and also equal to the ith component of the
11

---

Published in Transactions on Machine Learning Research (11/2025)
right QFT by Proposition 3.5. It is equal to En=o finle (m]e-~2nN 12aN"'n while the IN =| component of - is equal to EN= f t22nt=n n(N-i) by definition. Note however that:
N-1
N-1 fnile p t22nN Σ slnle / 1a(N-1)),
Σ Annee ni
Σ
because e 1 for any Z and pure unit μ. Hence X A1 The only components for which N x) are = 0 (DC component) and i N/2, hence for these we have X X."
J
Proposition 3.6. A set of N left eigenvalues that correspond to the eigenvectors columns of QN" for some choice of aris E H. uniquely defines circulant matriz C. The kernel of C is computed by taking the inverse right QFT of the vector of these left eigenvalues.
Note that while the analogous proposition known to hold true for non-quaternionic matrices and the inverse Fourier matrix, it is not completely straightforward to show for quaternionic matrices. For complex matrices it can be proven true by using e-g. the spectral theorem (Strang, 2019), however no such analogous proposition is known concerning quaternionic matrices and their left spectrum.
Proof. We shall proceed by proving the required proposition by contradiction. Let C # HNNN be circulant matrices; all columns of QN" for a choice of axis P are eigenvectors of both C and D due to proposition 3.5. We then have:
N-1
[Cak] =
12
1
km
t=
N-1 Σ
1 km
[Dxx]m
g(ll N
N
kf Nul
WNH
==
for all m E [0, 1), where h, g are the convolution kernels of C, D respectively, and ag corresponds to the btth column of QN"" By the assumption we have set in this proof, we have that left eigenvalues are equal for corresponding cigenvectors for either matrix, hence:
N-1
N-1 h(4 Nu = Σ N
Σ 2=
or, written in a more compact form,
FRr(g
However due to the invertibility of the (either left right-side) QFT, we have h = C = D, which contradicts the assumption. Hence, circulant C' must be unique.
j
Note that an implication this proposition that, while the number of left eigenvalues is in general infinite, picking a random set of N left cigenvalucs does not suffice to reconstruct the original circulant matrix. It must be ensured that the selected left eigenvalues correpond columns of Q where we are free choose the axis μ e H..
Corollary 3. Given (a vector, or ordered set of) N left eigenvnlues A"; we can use the QFT to recon- struct circulant matriz C NNN with X C (C) The corresponding eigenvectors are the columns of the QFT matrix Q The resulting matriz C is unique, in the sense that it is the only matrix E N with this pair of left eigenvalues and eigenvectors.
Proof. This follows from Propositions 3.5 and 3.6: From Proposition 3.5 we have the result that Q contains eigenvectors for any circulant matrix C' e NxN, This allows us to reconstruct the matrix by applying the Quaternionic Fourier Transform over the given vector of left eigenvalues A From Proposition 3.6, we know that there no other quaternionic matrix with the same left eigenvalues A", given axis μ for our eigenvectors (i.e. eigenvectors chosen columns of ")
12

---

Published in Transactions on Machine Learning Research (11/2025)
Corollary 3.6.2. Writing proposition 3.5 in matrix diagonalization form (A = SAS-)) where the columns of S are eigenvectors and A is diagonal matrix of left eigenvalues, is not possible. This would require us to be able to write CQN" = A*; however, the right side of this equation computes right eigenoalues, while proposition 3.5 concerns left eigenvalues.
Proof. The given diagonalization implics that we can rearrange terms to write AS = SA. However, by definition of quaternionic matrix multiplication this form corresponds to Ar æλ, which defines right eigenvalues (Zhang, 1997). For left eigenvalues we would require to write Ar = Ax, and Az = æd general not true, due noncommutativity of H.. Rephrased a proof by contradiction, by the above argument any diagonalization A SAS-! is related to right cigenvalues, and obtaining left eigenvalues by such a form would mean that left and right spectrums coincide. This is known to not be the case in general (Zhang, 1997).
Consequently, and despite an only partial generalization of the corresponding well-known properties to quaternionic circulant matrices, we must note that quaternion circulant matrices have the rather singular property of being relatively easy to numerically compute (part of its) left spectrum:
Corollary 3.6.3. For any Quaternionic Circulant C, any right-side QFT Fir (i.e., with respect to arbitrary pure unit aris (b) of quaternionic convolution kernel lic will result veclor of left eigenvalues for C.
Proof. This stems from Proposition 3.5. Given a circulant matrix E HINNN we obtain kernel ke as its first column and apply right-side QFT (eq. 21). From Proposition 3.5, we know that the resulting vector X equals left eigenvalues of C. (Note that we can change μ will, and obtain different set of left eigenvalues).
Concerning computation of the left spectrum, in general procedure is known to be applicable to generic (non-circulant) quaternionic matrix (Zhang, 1997; Macias- Virgós el al., 2022). Only its right spectrum be fully calculated using a well-defined numerical procedure (Le Bihan & Sangwine, 2003).
C E MNNMN
ko MxN
QFTF R
A c (C))
IQFTF" R
Figure 4: A synopsis of our key results in Propositions 3.5 and 3.9. Quaternion convolution kernel ke (left) is related to doubly-block circulant matrix C' (top). In this work, we show that the left spectrum (right) of any C' can be accessed via applying a QFT over kernel kg. Furthermore, the kernel can be reconstructed exactly via IQFT. This provides a fast and accurate way to access spectral characteristics of convolution, completely bypassing the construction and manipulation of the (quadratically increasing) doubly-block circulant C.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Proposition 3.7 (Eigenstructure of sums, products and inverse of circulant matrices).
Let L, E HNxN circulant matrices, then the following propositions hold. (Analogous results hold for L, K doubly-blon block circulant).
(a) The sums + K and products LK, KL are also circulant. Any sealar product pL or Ly where p E H also results to circulant matrix.
(b) Given some pure unit aris μ HI, let be the tth column of the inverse QFT matrix QN,, and X Ry are left eigenvolues of L, K with respect to the shared eigenvector w. Then, is eigenvector of...
i. L+ K with left eigenvalue equal to N + +
it. pL with left eigenvalue equal to p.
Mt. Lp with left eigenvalue equal to Xprp p.
ule!"" oy if Ne # and equal 0 of k² = =0
LK with left eigenvalue equal to X
v. Provided L-1 exists, (w is on eigenveclor of L-2 with left eigenvalue equal to (A!")-1
Proof. (a) For proving L + K, pL and Lp are circulant, it suffices use their matrix polynomial form (cf. Proposition 3.2). Then summation and scalar product are easily shown to be also matrix polynomials w.r.t. the same permutation matrix, hence also circulant.
For the product LK, we decompose the two matrices as linear combinations of real matrices, writing
LK = (Le + L L + (,,) + K, + Ký + Kgk),
where Les Las L.g, Lk, Ke K2 Kj, K2 NNN and circulant. Computation of this product results in 16 terms, which are all produced by multiplication of real circulant with another real circulant, and multiplication by a quaternion imaginary or real unit. All these operations result to circulant matrices (for the proof that the product of real circulant matrices equals circulant matrix e.g. Jain (1989)), does the summation of these terms. KL is also circulant by symmetry, however not necessarily equal to LK, unlike in the real matrix case.
(b) i. (L KYw = Lw + Kw = NWw + = (X + ktt).. Hence w is an eigenvector of L + K with left eigenvalue equal to X + x!.
ii (pL)w p(L))= (w>")w. Hence an eigenvector of pL with left eigenvalue equal pA!".
iii Let Pa unit quaternion with Pa p/lpl, assuming p * O.
(Lp) = Ip((Lp.)w = 1p) up. P.. = L22
(24)
where z = Ps up represents a rotation of w to vector which is also an eigenvector of L; that is because the ttth element of w is equal to e -Mp22N11 With similar manipulation as in Proposition 3.4, is column of the inverse QFT matrix Q i.e. an inverse QFT matrix with an axis different to the original Thus, due to Proposition 3.5, is an eigenvector of the circulant L, the left eigenvalue of which is Xuure" Continuing eq. 24, we write:
Lp| = p/ p/p zpa = 1p/>p=pP.
P. p. pa =
(Lp) = (2 (XPaPP.") p)w,
(25)
hence w is an eigenvector of Lp with left cigenvalue equal X XPMPP" p.
iv. LKw Lic Xlman." ne w. where q n². n." qu 9/1g/ and we used the right scalar product result of (iii), supposing that q # Hence w an eigenvector of LK with left eigenvalue equal to X; Aguagh kef If
q 0, we cannot use the result of (ii);; the resulting eigenvalue is simply equal to 0 since LKw 0.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Lu = u = Lw( X w(),)= L (- L \-)))(()))--- 7
( = 1-- -X²)=1= where 2 = X1t ((+)
3.4
Quaternionic Doubly-Block Circulant Matrices
Block-circulant matrices are block matrices that are made up of blocks that are circulant. Doubly-block circulant matrices are block-circulant with blocks that are circulant themselves. These latter are useful in representing 2D convolution (Jain, 1989). Results for circulant matrices in general are valid also for doubly- block circulant matrices, where the role of the Quaternionic Fourier Matrix QN H is taken by the Kronecker product QM" @ QN"
Proposition 3.8 (Doubly Block-Cireulant & Fourier Matricos). Let D E HMNXMN be doubly block- circulant, with N x blocks, and each block sized M. Then the product D vve(z), implements the quaternionic circular left 2D convolution kn @ v. The operator vec(x) represents column-wise vectoriza- tion of signal |
M-1N-1 (ND 2 x)|i.]] = Σ Σ
kpli m. - n) [m),
(26)
m== n=0
Proof. A doubly block-cireulant matrix D can be written in block form as:
Do D1 D2
DN-1
DN-1
Do
D1
DN-2
D
: D1 D2 D3
Do
where M is set of N circulant blocks, and for each of the blocks we have D, MxM by definition (note the analogy to eq. 12). Blsck-wise, D is circulant, and each block D, is circulant itself.
Let us denote column u of x as Tu, so vec(x) = [ a ..... Before considering the required product D p-r -eve(()) we can observe that it should be made up of linear combinations of blocks D, and columns T.. Since D, is circulant, it holds that these products represent set of 1D quaternionic circular left convolutions (cf. eq. 42)
M -1
D
Σ m==0
kpli - m, =]mx)mm v), V(U,T) E [O,N N 112.
The required product D - vec(x) can then be written as:
kpli m, n) 2 (m, n)]
knli m, - n a
kpli- m, n)M,Namm
MM- EN-1 kpli m, N -]M.N=mmm n)
which coincides with the vectorized form of the quaternionic circular left 2D convolution kD @ x.
Proposition 3.9. Let R = Qi & Q and the invertible coordinate mapping k < i + Mj. For any C e MNNN that is doubly block-circulant and any pure unit E H,
1) of is an eigenvector of C. Column corresponds to the (i, (1) Any column k = 0, 0.I....., (M - component of the matrix of left eigenvalues A MMN, Matriz X equal the right QFT Fli of the kernel of C.
(2) Ang column = 0, 1......., (M 1)(N 1) of is an eigenvector of H. Column k corresponds the (i.j) component of the matria of left eigenvalues k² MxN, The conjugate of the matria R is equal to the left. QFT Fy. of the kernel of C.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Proof. The proof for the doubly-block circulant 2D case is analogous to the 1D case. Let ag the kth column of R = Q s QN The fth element of Coa is:
1 M-1 N-1
[Cas]e= = VM VN
Σ
Σ
mm yn h(q m. 8 n. M Mi
WNH
m=0 n=0
M-IN-1
Mp. Mp.r)M.N
M NN
W 22 9a Ma WN
VMN
(27)
where we used (() = (q, s) and p m and r = n. Note that WMa and WNz commute between themselves because they share the same axis μ4, but they do not commute with the kernel element h.
The reader may examine Fig. 4 as a visual summary of propositions 3.5 and 3.9.
4 Proof-of- Conceet Application: Fast computation of singular values & Bounding of the Lipschitz Constant
We present a method bound the Lipschitz constant of Neural Network that comprises Quaternionic Convolutions, based on theoretical results. Determination and bounding of the Lipschitz constant has been shown to be a way to control the generalization error (Prince, 2023, Chapter 9, Appendix B). Other important uses include application in Wasserstein GANs (Petzka et al., 2018), (Prince, 2023, Chapter 15) or plug-and-play networks (Ryu et al., 2019) and deep equilibrium (DEQ) networks (Pabbaraju et al., 2021). While the literature on Lipschitz constant bounding or other regularization methods over real-valued operations rich, with many of them focusing on the specifics of convolutional operations (Grishina et al., 2024), the literature on methods applicable H is comparatively not extensive (Altamirano-Gar Gomez & Gershenson, 2024).
The value of the Lipschitz constant depends on the product of the spectral norms of the weight matrices of the network. Given any linear layer, a bound on its spectral norm can be set by projecting the operation matrix onto the set of lincar transforms with a bounded norm (Lefiimmaatis et al., 2013; Sedghi et al., 2018). We are interested in computing and constraining the maximum value of (z))l where fe represents the operator layer l and x the layer input, for all layers in the network. For an arbitrary linear layer, expressed as matrix B E M this leads to dealing with a Rayleigh quotient of the form x" 2" from here we can use the Quaternion SVD (Sangwine & Le Bihan, 2006) to proceed. We analyze as U,E, V qsvd(B) and reconstruct the regularized operation as UÊVH where Σ contains singular values projected in the range [0,c]. cJ. Note that in HII, singular values are related to the right spectrum (cf. Appendix). When the linear layer in question corresponds to Quaternionic Convolution, matrix B is circulant. While can still use the QSVD over B to bound the spectrum, the required related space and time complexity is prohibitive in practice (we refer to this method as "brute-force", see further below for a comparison and discussion). We can instead circumvent computing the "expensive" doubly-block circulant matrix altogether, using our results.
Ba and
We will present the approach as a two-step process. First, we will show how to compute singular values for the convolution operation. Second, will discuss how we can reconstruct. convolution using the clipped singular values.
Fast Computation of Singular Values. We first require the additional lemmas that follow (please refer to the appendix for their proof):
Lemma 4.1. For any pure wnit μ E H. and any N E N, the right spon of the columns of Q is N For any pure unit μ E IHI and any M, N, the right spon of the columns of Qi & Q* is HMN..
Lemma 4.2. Let A NN be block diagonal. The left eigenvalues of each block of A are also left eigenvalues of A. The right eigenvalues of each block of A are also right eigenwalues of A.
16

---

Published in Transactions on Machine Learning Research (11/2025)
The first key result for computing singular values is:
Proposition 4.3. Let C' E HN*N be qualernionic circulant. Then the square norm ||Cu||; given some E IH,, be written as EdEc where c, E IH diagonal matria. All blocks = = are either 1 x 1 or 2 2, and are Hermitian. The 1 x 1 blocks are at most two for ID convolution, and at most four for 2D convolution.
related by bijection, and E {(kc) is block-
Proof. We will treat the case of 1D convolution, and note where the proof changes in non-trivial manner for 2D. We write En=o aRck, as in Lemma 4.2. From there have Cz C C O&CK =En=1 No axck. Then:
*
N-IN- Σ
||xx| (Cx) ==
Σ
Cm a Am X Cn.
(28)
The caveat is that while we know that am, On unit-length pairwise orthogonal, which would have most
terms in the above sum to vanish, it appears we can't commute the Am quaternion scalar terms out of the way. We can however procoeding by writing AM as a sum of simplex and a perplex part, with respect
to axis -y (Ell & Sangwine, 2007), i.e. An = All + X (we have dropped the axis superscript in favor of a more clear notation). Eq. 28 follows up as:
Σ 0=0
Gaa #
+
Σ
Σ
H Gmam (Zmml Am ( + X JanCn
n=0 moin
N-1
N-1
Cn Cn|A"| aan Con + Σ
Σ
H G am (dm
+ 7m ( + X Jn
Cin
N-1
N 11 Σ
N-1
Gp 1212 Cn
m (Am A11 + Am An Jn Ca
Cyn Cral (Jm 2o Am And
m
N-1
N N-1
N-1
|A" cn 2 +
Σ
Σ
Cm (mm am and + Amo" a and )cn +
Σ
Σ
Cm ama (mm An + Amm All ann =
R=0
== myin
n=0 motn
N-1
N-1
Σ
AM +
Σ
Σ
Cr (om am And + Am [a aml H N cnn
=
n=U
n=0 moinn
N-1
+
X + Am am On Anl Cm Am [a am) am
(29)
0=0 m
In the above form, the simplex parts commute with eigenvectors an,am, while the perplex parts conjugate- commute with the same eigenvectors. The reason is that all terms of an eigenvector (column of a QFT matrix) share the same axis p. For n + m, we have Om 1 an as any QFT matrix is unitary (Prop. 3.3), and am; On are different columns of a QFT matrix by assumption. Hence, the second sum vanishes.
The third sum eq. 29 is of special interest, bocause for cach
[0, N 1] there will be most one index m = nt' over which the term will not vanish. That is because a, am is the inner product between on and am (i.e. where the first term is conjugated), so in effect it is a product between a QFT column and another column of the inverse QFT (cf. Proposition 3.3). If and only if indices are n and m [N n]N,, we have T
a am = 1. In other cases, the term vanishes. Consequently, we have:
|C2|
Σ n=U
Dn Plc,11 + Em (m d Am yl
(30)
where we use m [N- nn]x N- Also, recall that all left eigenvalues An are w.r.t. axis -p.
17

---

Published in Transactions on Machine Learning Research (11/2025)
Note that, if and only if = 0 or N/2 the second sum will also vanish; these are the columns of the QFT matrix that contain only real-valued terms. For doubly-block circulant matrix these will be at most four by construction. To see this, we can use Proposition 3.3.7, which in effect says that the inner product of the m and n column of Q will equal to only for m = n = 0 or N/2. If the number of clements is an odd number, only the case m n non-zero elements the diagonal of the permutation matrix P = QQ.
possible. In either case, this coincides with the number of
For 2D convolution and doubly-block circulant matrix of size N x N2 We are then interested in the product (QN & QN)(QN @ QN). This equals to QNQN & QNQN P & P. The result is another permutation matrix, which will have at most 4 non-zero elements in the diagonal, the number of which will depend to whether N is odd or even. For the second term in eq. 30 we need m = ([N - in. N - vnl with (In.n) = - (m) instead of m = [N - n|n of the 1D case, so again we have most one non-vanishing second term for each
obtain analogous result.
[0, N 1].
We can proceed from eq. 30 by rewriting it in the form ||xx||2 H where E' N*N is Hermitian. For example, for N 6 we have:
Poo
2
0
0
0
0
A 1
O
0
Xj A + A A
0
0
|Az1
X
A +
A2 X!
E'
0
0
A
0
A4
||
1/ + A4
|>a
As A1 +
+3
X||
0
|As|?
which, by rearranging indices of coefficients and corresponding rows and columns, we have E' which is also Hermitian. For the previous example for E', we would have:
Aul
0
0
0
0
A
0
0
0
0
0
0
12
A1
A1 1
0
0
=
0
0
As
+
As
X!1
Ass
0
0
0
0
0
12
X
1 + X1 X!
0
0
0
A2 ΛJ +
X!|
D2
Furthermore, E is also block-dingonal, and all of its blocks are of size 2 2, with the exception of at most two (four) 1 x blocks for 1D (2D) convolution. We can then write the required rearranging of coefficients as a multiplication by a permutation matrix P (not unique, and in general + P, P). Hence, ||Cx||2 c p E'Pe' = =chEc, where we have switched places for c, d' for clarity of notation.
Matrix important, because it contains sufficient information to compute the spectrum of the convolution.
The notation E(kc) stresses that we only need the kernel of the convolution to construct E. The structure of the 2 blocks of =\t will be of the form:
Am Xll + T x
Alaa + + 12n
(31)
where || and IL refer to simplex and perplex components w.r.t. axis μ (ef. Appendix, or e.g. Ell & Sangwine (2007)), and terms are left cigenvalucs of the convolution kernel kg. Its 1 blocks are simply equal a gquare-magnitude |mm|2. Indices m,n must form pair as described in the Appendix.
3This is trivially extensible to non-square-shaped filters, fee. MN MN.
18

---

Published in Transactions on Machine Learning Research (11/2025)
Proposition 4.4. All singular values con be computed exactly as either the magnitude of a left eigenvalue of C, or the square-rool of one of the two eigenvalues of the 2 x 2 blocks of E, again each dependent on pair of left eigenvalues of C. The marimum value of ||Cx||? under the constroint ||||| = is given by the square of the largest right eigenvalue = ==((kc)
Proof. The required maximum is equal to the maximum of the Rayleigh quotient. = Ca which obtains x its maximum for the maximum right eigenvalue (Macias- Virgós et al., 2022, Proposition 3.3-3.4). Now, since c, are related through a bijection, maximizing the Rayleigh quotient of CH C is equivalent to maximizing the Rayleigh quotient of E.
As 3 is Hermitian, we know that all of its right eigenvalues are real-valued (and are also left eigenvalues). Due to it being block-diagonal, its right eigenvalues will be the union of the eigenvalues of its blocks, due to Lemma 4.3. As all blocks are either either be: a) equal to the value of the 1 1 block, equal to |Am|?, where An is the left eigenvalue of C that corresponds to the th column of QFT matrix with axis μl, b) equal to the right eigenvalues of the 2 x 2 Hermitian matrix:
1 or 2 x 2 by Proposition 4.1, the sought right eigenvalue will
Am 12 An ll A + x Xl11
ll Am A + A!l
(32)
1
The right eigenvalues for the latter case are real-valued and can in principle be computed through a complex adjoint mapping and the spectral theorem (Zhang, 1997). (Recall that by [Aal we denote the left eigenvalue of C, produced by taking the n element of the right QFT with respect to axis minus y (cf. Proposition 3.3, Lemma 4.3).
J
The above results tell us that we can compute the spectrum of our convolution without building the circulant matrix explicitly. Note that in the non-quaternionic case, E would be diagonal, as off-diagonal terms would vanish, and all x 2 blocks would become diagonal. In H. we require the extra step of computing singular values for the 2 2 blocks.
Bounding the Lipschitz constant. In order to sct bound on the spectral norm, we need to be able to move in the opposite direction from clipped È matrix, to new left eigenvalues, back towards the new kernel. The following corrolary of the previous propositions is a necessary step (please refer to the appendix for its proof):
Corollary 4.4.1. For any matria & such that " = E(kc), matrices C and 4 have the same singular values.
The choice of & is not unique, and ideally we want matrix that will be convenient in the context of clipping and reconstruction. We choose & so that it is block-diagone with block structure identical to that of E.
We pick the 1 x blocks to be equal to [Am], and 2 x 2 blocks equal to X
4. It is ddraightforwan to xll
confirm that this choice fulfills the requirement set in Corollary 4.4.1. Importantly, this choice of & is related through a bijection with the left eigenvalues of kn- Another bijection exists between left eigenvalues of k (cf. corollary 3.6.1) for a given axis, so exact reconstruction of the required clipped convolution is possible.
By making use of the above results, the spectral norm of a given convolutional layer can be clipped as follows: 1) Compute left eigenvalues of the filter using the right QFT, given a choice of μu (following the results of proposition 3.3). 2) Compute block-diagonal elements of Φ (cf. corollary 4.4.1). For 1 x blocks we simply copy the appropriate left eigenvalue. For 2 x 2 blocks we decompose the two left eigenvalues into simplex and perplex parts, as in eq. 31. 3) Compute QSVD for 4; this can be implemented as QSVDs for the 2 2 blocks of T. Clip singular values and reconstruct क. 4) Reconstruct filters using inverse right QFTs (cf. proposition 3.6). 5) Clip spatial range of resulting filters to original range (e.g. (Sedghi et al., 2018, Section 3)). Please see also Fig. 5 for visual summary of the key steps of the process. All convolutions are clipped and roconstructed with this process. The rest of the lincar components are clipped and roconstructed as described in the beginning of this Section.
Another choice can be: E1/2, However will allow us reconstruct. left. eigenvalues only w.r.t their magnitude.
19

---

Published in Transactions on Machine Learning Research (11/2025)
E = {(kc)
kc
E cc(C)
QFI Fi.
IQFT
X E
Figure 5: A visual summary of the procedure that forms the backbone of the Lipschitz constant-bounding method, enabled with our results. Given a quaternionic convolution kernel kc (top left), we compute left cigenvalues X via the QFT with axis μ H. From our left eigenvalues we construct matrix E, then matrix & (see in-text for details). Crucially, matrix & has the same singular values with doubly-block convolutional matrix C, but is much easier to construct and manipulate due to its block diagonal structure, with blocks sized either 1 x 1 or 2 x 2. Spectrum manipulation is thus performed over & to obtain e, and in turn we obtain new left eigenvalues Ar, Finally, we apply IQFT to obtain the quaternionic convolution kernel that corresponds to the manipulated /ccppllateddcllppedd /clipped convolution &g. The result is a method that is more economical in terms of space and time requirements by orders of magnitude, related process that would be oblivious to these results.
Results and Discussion: Complerity of Space and Time Requirements. Direct clipping using QSVD is very expensive both in terms of space and time complexity. The input to QSVD is the doubly-block circulant matrix that corresponds to the convolution filter, and it consists of N quaterniomic elements; our approach does not require computing the doubly-block circulant matrix at all. In terms of time complexity, direct clipping using QSVD requires O the cost of SVD over C. Regarding our method, for the forward and inverse QFT, we need 4 FF'Ss, as each QFT is computed via. standard FFTs (Ell & Sangwine, 2007). Also, we require N2/2 eigenvalue decompositions for the 2 2 blocks of & (depending on how many 1 blocks exist, which in turn depends on whether N is odd even). Hence, total complexity is in the order of O(N2logN).
Computing Singular Values Clipping Time Comparison versus Brute-force QSVD. We have tested runtime of our method versus the brute-force QSVD approach, over a set of random filters. Results can be examined in Table 1, where we compare runtimes using the method we described, versus the brute-force method (direct clipping). Computational times to either only compute singular values, or perform the full clipping and reconstruction process, are also reported. Concerning the size parameter in practical application, note that while small convolutional kernel sizes much as 3 x 3 or 5 x 5 are prevalent in both real-valued and hypercomplex network architectures, when express convolution in terms of circulant or doubly- circulant matrix we need to take into account the zero-padded version of the kernel. In simple terms, when Cr expresses a convolution over input & E H circulant C' must match the dimension of the input, so C E HNxN X Hence, when comparing resource requirements for different sizes of convolution, larger magnitudes for N are more relevant to real-world practice. On operation sizes N 2 128, using the brute-force method is practically impossible due to both space and time constraints. On the contrary, our method scales well on larger sizes. A visualization of clipped and reconstructed filter can be examined in the Appendix.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Runtime comparison for computation of Quaternionic Convolution Singular Values and Spectral Norm Clipping & Reconstruction. Convolution kernel size refers to 2D kernel side length. Average CPU time in milliseconds is reported; an exception are the fields containing asterisk (+), which denote that required runtime is impractical (measured in hours). Our results enable fast and exact carrying out of both tasks.
Table 1:
Required time (in ms) +
Convolution kernel size
4
8
16
32
64
128
Singular Value Computation
ours
4
16
80
210
708
1, 795
brute-force
5
95
1, 2,444
69, 319
*
*
Clipping & Reconstruction
ours
5
20
113
280
1, 119
2, 590
brute-force
8
121
1, 866
150, 000
*
Numerical stability test. We have tested the spectral norm clipping method in terms of numerical stability. To this end, given an "original" quaternionic convolution kernel A E HNN, we apply over it multiple iterations of clipping & reconstruction. (That is, in the sense of starting from a specific kernel, we inject noise, clip & reconstruct, then re-inject noise, clip & reconstruct, and so on). We clip to bound equal to a given percentage of the original maximum singular value. The original kernel is constructed as sample from standard Normal. We have run the experiment over set of different values for the variance of the injected noise (a² = 1,3,5), over different clipping percentages (80,, 90%, 99%), and over different quaternionic convolution sizes (3, 7,9).
We have run the experiment over a total of 100 iterations of clipping & reconstruction, using either the result-enabled method, or the "brute-force" method. In all cases, the numerical difference measured as sum of absolute values between the results of the two methods was steadily under 10-11 (we use the same random soed, and the only difference is between the clipping & reconstruction method applied).
Application on a Neural Network. We test ResNet32 architecture on CIFARIO. We replaced standard convolutional layers with depthwise convolutions and pointwise convolutions (Chollet, 2017) on the quater- nionic domain. We report the results of training both archilecture variants considered Ne3esMet32lsallll and QReeNet32-aargee with and without clipping, in Table 2. We observe that clipping always boosts final performance, regardless of the architecture considered.
Table 2: Comparison of final test accuracy, when setting bound on the Lipschitz constant (w/ clipping) versus not (w/o clipping). Figures for two considered architectures are reported, as statistics over 5 runs. Spectral norms are clipped to value c=
architecture Nee
w/o clipping
w/ clipping 85.80% + 0.38
81.43% + 0.75
NesNeet32eargee
81.78% + 1.04
86.32% + 0.33
To further understand the dynamics of the clipping procedure, we present the loss and accuracy curves in Figure 6 for the case of QResNet32-smoll where the per epoch loss and accuracy are reported for both the with and without clipping variants, showing mean and standard deviation values over 5 runs. As we can see, the clipping impact is mostly realized when the scheduling step occurs (i.e., at 40 epochs), indicating that it enables a finer tuning of the network.
Lipschitz constant bounding tests on CIFARIO. We follow (Sedghi et al., 2018) and test ResNet32 archi- tecture. In our model, 3 convolutional blocks, each one consisted of 5 basic residual blocks, are considered, followed by a fully connected layer. Between each block a downsampling operation is used (with a strided convolution). An initial convolutional layer transforms the 3d image imput into a 16d feature tensor.
For our case, we replaced the convolutional layers of basic residual with depthwise convolutions (Chollet, 2017; DeMagistris et al., 2022). Between the blocks, pointwise convolutions are also added to change the
21

---

Published in Transactions on Machine Learning Research (11/2025)
w/o cli
wib clip
80
1.5
acourscy
60
50
1.0
30
20
30
40
50
60
70
00
10
20
30
40
80 50
60
70
Epochs
opochs
Figure Per-epoch analysis of the impact of using the proposed technique. Training loss (left) and test accuracy (right) curves are reported. Clipping using the method enabled by results always leads to clear performance boost.
number of dimensions and thus each block can safely proceed with depthwise hwise only operations, using the same number of channels. Of course, these depthwise and pointwise operations are replaced with the quaternionic alternatives. Only the first 3 x convolution and the last fully connected layer are retained as they are. This model is referred to as QResNetS2 small and has only 98k parameters. An alternative has also been explored, where wetn each depthwise convolution of the basic residual block, an extra pointwise convolutional layer is intervened. Its functionality is assist in the mixing of the channels, at the cost of greatly increasing the number of parameters. Specifically, this version, dubbed QResNet32-larye has approximately 454k parameters.
Training was performed on CIFAR10, where each model was trained for 80 epochs. Initial learning rate was 0.01 and was subsequently dropped by /10 at 40 and 60 epochs. Clipping is performed every 100 iterations. A summary of architecture and optimization details can be found in the Appendix (Section F).
We show comparisons in Table 2 and Figure 6 in Scction of the main text. Also, understand how clipping may affect the result, we report the singular values of each quaternionic layer in Figure 7, for training the QResNet32-small architecture without clipping. For visualization purposes, each layer is depicted by mean value and the standard deviation of the corresponding singular values of the layer.
We can observe that a) clipping always boosts final performance, regardless of the architecture considered; b) the overall performance of the two architectures is very close. Hence, we contimue our experimentation with one the variants.
Next, we investigated the impact of the clipping value c. The results for a QResNett2-smmall architecture are summarized in Table over 5 runs for each setting. As can see, the choice of the clipping value has impact overall performance. Specifically, choosing a value from 0.25 to 0.5 seems to be more beneficial, while lower or higher values over-"prune" or are too modest to provide notable results.
Table Impact of clipping value e on overall model performance.
0.10
c= 0.25
0.50
c= 1.00
e= 2.00
72.74% + 3.34
82.83% + 1.02 85.8%% + 0.52 85.80% + 0.38 84.21% 0.28
Another aspect that we have tested is the impact of the choice of the axis of the QFT. This acts hyperparanelerr as we need QFT to compute left eigenvalues, and reconstruct the filter from the new left eigenvalues after singular value clipping. In theory, its impact on performance should be minimal. Towards this, we randomly generated total of 20 different runs, with a different value for y on each. Test accuracy curves for are visualized in Figure 8. As we can see, clipping is equally effective, regardless of our choice of the p axis.
22

---

Published in Transactions on Machine Learning Research (11/2025)
2.5
1.5
0.5
0.0
pointwiie a.
biock? Doocr--o9 1-0w-0
porcli--hnn nioc noock--dt
block tiee dioek Bioekl-thy 1ehe
nincs ontee block g
block chee
gord docc (owel thare aadee
wan 20 crchore ceeeeraee soed
ninc bocck aaeeeaa
Figure 7: Distribution of the singular values for each layers, when training a QResNetS2-small architecture. The reported values are for the final model, after 80 epochs.
70
20
40
60
40 spach
80
20
30
60
50
20
10 30 40
40
opocns
epochs
Figure 8: Test accuracy curves during the training for the QRessNet32-smal architecture. A set of 20 random initializations are considered, using different values for axis μ. A random selection of runs and mean standard deviation values for all runs are visualized on the top and bottom rows respectively. Experiments for two different clipping values, c = 0.5 and c = 1.0 are shown, respectively corresponding to the left and right column. Regardless model and clipping parameters, clipping equally effective all cases.
23

---

Published in Transactions on Machine Learning Research (11/2025)
Note also that the non-quaternionie equivalent, with only depthwise convolutions in the residual blocks, is under-perforning, achieving 75.41% test accuracy, even when using more parameters, namely 151 K param- eters in total. We explain this interesting result as follows: Cascades of depthwise convolutions in IR are much less expressive than their counterpart in H. because the former represent scries of convolutions over a single real-valued channel. The latter however, do comprise a form of *channel mixing" under the hood, due to the dcfinition of the Hamilton product. Hence, the quaternionic depthwise layers stand as an interesting trade-off between the completely independent convolutions of real-valued depthwise convolution, and full convolution, taking into account correlation between all possible pairs of channels.
Lipschitz constant bounding tests on an additional backbone and additional benchmarks. In order to further test the validity of our results, we have run additional lests using an adaptation of the VGG16 backbone, following a similar process for its quaternionization as for the aforementioned ResNet variants. In particular, we have run tests on the additional tests Fashion MNIST and CIFAR100, using the large ResNet variant. Results for these sets, as well as results for tests using VGG16 can be examined in Table In all of these cases we have run versions using clipping vs without clipping, as done with our CIFARIO tests. Note also the low variance of reported accuracy w.r.t. different runs, which highlights the stability of the resules-enabled clipping method.
Table 4: Additional results comparing final test accuracy, when setting a bound on the Lipschitz constant (w/ clipping) versus not (w/o clipping). Figures for tests on datascts CIFAR100 and Fashion MNIST are reported, as well as over an additional quaternionized VGG16 backbone (QVGG16). Statistics are reported over runs. Spectral norms are clipped to value 1.0.
datasct / architecture
w/o clipping
w/ clipping
Fashion MNIST/ / QResNet32 large CIFAR100 / -ResNet32-large
89.73% = 0.86
92.36% = 0.27
56.05% + 1.32
61.15% + 0.19
CIFAR10 / QVGG16
90.38% = 0.12
90.80% = 0.02
5 Conclusion
We have presented set of results concerning the properties and relation between the Quaternion Fourier Transform and Quaternion Convolution, with respect to their formulations in terms of Quaternionic matrices. This relation has been well-known to be very close and important the real and complex domains, and already used in wide plethora of learning models and signal processing methods. With our results, we have shown how and to what extend these properties generalize to the domain of quaternions, opening up the possibility to construct analogous models built directly on quaternionic formulations. Given the ubiquitous character of linear transformations, and consequently, matrix representations in virtually every subfield of machine learning, our results pave the way for new, expressive data science models (Qin al., 2023; Guo et al., 2025; Parcollet et al., 2020; Yue et al., 2025; Miao et al., 2024). As proof of concept, we have presented method for bounding of the Lipschitz constant using our theoretical results; the method enables spectral norm bounding in quaternionic NNs, previously impossible in practice. Lipschitz constant bounding and determination is important in a number of important applications, and important in different classes of models, including Wasserstein GANs (Petzka et al., 2018), Plug-n-Play networks (Ryu et al., 2019), or Deep Equilibrium Networks (Pabbaraju et al., 2021); hence, one direct line of research involving our results could envisage generalizations of these applications in quaternionized versions of these models.
Let us refer to two more concrete examples of perspectives or application of our results:
Richer observation models. Assuming an observation model of the form g = h + Hf + + (sce lor example Chantas et al. (2009) or Hidalgo-Gavira et al. (2019)), we can now consider casting all components as quaternionic. Assuming a likelihood model p(glf) x exp( 811- ffll' and a prior p(tffl) x expf- (|]11(1111))) [f12), where & is cast as a convolution, a solution in is known to include operations that require results on addition and multiplication of circulant matrices. Hence, with the current work, this direction can be envisaged, making direct use of e.g. Proposition 3.7.
24

---

Published in Transactions on Machine Learning Research (11/2025)
Applications to adaptations of a wide range of NNs to H. Convolution, linear operation, and has been used in tandem with layers of other types. A good example here is hybrid architectures that have emerged relatively recently. Models like the Convolutional Vision Transformer replace the linear projections for queries, keys, and values with convolutional operations. Other approaches, such as the Conformer for speech recognition, place a dedicated convolution module within the Transformer block capture local correlations alongside the self-attention mechanism's global context (Gulati et al., 2020). The Mamba architecture also includes convolution layer as part of its internal block to extract local patterns before its Structured-State Space Model focuses on long-range dependencies. Other more direct. "hybrids" with traditional convnets also exist, like ConyMamba or VisionMamba (Gu & Dao, 2023; Yu & Wang, 2025). Graph convolution is another widely used example of fusion with convolution (Zhang et al., 2019). With the results we present in Section 4, Lipschitz-conslant bounding on quaternionized versions these architectures is a straightforward step.
Future research will shed more light over these topics.
Acknowledgments
The authors would like to thank the TMLR action editor and the anonymous reviewers for their constructive feedback and suggestions.
Dedication
This paper is dedicated in memory of the late Dr. N.P. Galatsanos.
References
Stephen L. Adler. Quaternionic quantum mechanics and quantum fields, volume 88 of International Series Monographs Physics. Oxford University Press Demand, 1995. 1
Dimitrios S. Alexiadis and Petros Daras. Quaternionic signal processing techniques for automatic evaluation of dance performances from MoCap data. IEEE Trans. on Multimedia, 16(5):1391 1000119991-466,, 2014. 4
Daniel Alfsmann, Heinz G. Göckler, Stephen J. Sangwine, and Todd A. EIL. Hypercomplex algebras in digital signal processing: Benefits and drawbacks. In 15th European Signal Processing Conference, pp. 1322 1326. JEEE, 2007.
Gerardo Altamirano-Gomez and Carlos Gershenson. Quaternion convolutional neural networks: Current advances and future directions. Advances in Applied Clifford Algebras, 34(4):42, 2024. 16
Helmer Aslaksen. Quaternionic determinants. In Mathematical Conversations, pp. 142-156. Springer, 2001.
3
Mawardi Bahri, Ryuichi Ashino, and Rémi Vaillancout. Convolution theorems for Quaternion Fourier Transform: Properties and Applications. In Abstract and Applied Analysis, volume 2013. Hindawi, 2013. 5, 6, 32
Mike Boyle. The Quaternion package: Add support for Quaternions to Python and Numpy. Zenodo, 2018.
29
Giannis Chantas, Nikolaos P. Galatsanos, Rafael Molina, and Aggelos K. Katsaggelos. Variational Bayesian image restoration with product of spatially weighted total variation image priors. IEEF transactions on image processing, 19(2):351 362, 2009. 24
Dong Cheng and Kit Ian Kou. 107.::667..0510, 2016. 4
Properties of Quaternion Fourier Transforms.
arXio preprint
25

---

Published in Transactions on Machine Learning Research (11/2025)
Dong Cheng and Kit Ian Kou. Plancherel theorem and quaternion Fourier transform for square integrable functions. Compler Voriables and Elliptic Equations, (4(2))223-242) 242, 2019. 5, 33
François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251-1258, 2017. 21
Danilo Comminiello, Eleonora Grassucci, Danilo P Mandic, and Aurelio Uncini. Demystifying the hypercom- plex: Inductive biascs in hypercomplex deep learning [hypercomplex signal and image processing)- IEEE Signal Processing Magazine, 41 (13::999 71, 2024. 34
Konstantinos Daniilidis. Hand-eye calibration using dual quaternions. The International Journal of Robolics Research, 18(3):286 298, 1999.
Giorgio DeMagistris, Danilo Comminiello, Christian Napoli, and Janusz T. Starczewski. Visual odome- try with depth-wise separable convolution and quaternion neural networks. In 9th Italian workshop Artificial Intelligence and Robotics. AIRO 2022, 2022. 21
Freeman J. Dyson. Quaternion Determinants. Helveticn Physica Acla, 45(2):289, 1972. 6
Ben Eater and Grant Sanderson, Visualizing quaternions. intro, 2018. [Online; acc.1-7-2023).
Todd A. Ell and Stephen J. Sangwine. Hypercomplex Fourier Transforms of Color Images. IEEE Transac- lions on Image Processing, 16(1):22 35, 2007. 2, 3, 4, 5, 6, 17, 18, 20, 29, 30, 31, 33
Todd A. Ell, Nicolas Le Bihan, and Stephen J. Sangwine. Quaternion Fourier tronsforms for signal and
image processing. John Wiley & Sons, 2014. 2, 4, 5, 6, 32
Wolfgang Förstner and Bernhard P. Wrobel. Photogrammetric Computer Vision, Springer, 2016. 1
John B. Fraleigh. A first course in abstract algebra, 7th, 2002. 4
Emil Fresk and George Nikolakopoulos. Full quaternion based attitude control for a quadrotor. In 2013 European Control Conference (ECC), pp. 3864 3869. IEEE, 2013. 1
Artyom M. Grigoryan and Sos S. Agaian. Retooling of color imaging in the quaternion algebra. Applied Mathemalics and Sciences: An International Journal (MathSJ), 1(3):23 39, 2014.
Ekaterina Grishina, Mikhail Gorbunov, and Maxim Rakhuba. Tight and efficient upper bound on spectral norm of convolutional layers. In European Conference in Computer Vision (ECCV), pp. 19 34. Springer, 2024. 16
Albert Gu and Tri Dao. Mamba: Lincar-time sequence modeling with selective state spaces. arXio preprint arXiv:2312.0075: 2023. 2, 25
Anmol Gulati, James Qin, Chung-Chema Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution Convolution-angmenter transformer for speech recog- nition. arXiv preprint ar Xrx:::000..0 f5.:2005.08100, 2020. 25
Jiujiang Guo, Mankun Zhao, Wenbin Zhang, Tianyi Xu, Linying Xu, Yu Jian, Yu Mei, and Yu Ruiguo. Teds: Joint learning of diachronic and synchronic perspectives in quaternion space for temporal knowledge graph completion. In Proc. of the Infl. Conf. on Machine Learning (TCML), 2025. 24
Natalia Hidalgo-Gavira, Javier Mateos, Miguel Vega, Rafael Molina, and Aggelos K. Katsaggelos. Variational Bayesian blind color deconvolution of histopathological images. IEEE Transactions on Image Processing, 29(1):2026-20) 2036, 2019. 6, 24
Eckhard Hitzer. The quaternion domain Fourier transform and its properties. Advances in Applied Clifford Algebras, 26(3):969 984, 2016. 2
26

---

Published in Transactions on Machine Learning Research (11/2025)
H.W. Hsu, T.Y. Wu, S. Wan, W.H. Wong, and C.Y. Lee. QuatNet: Quaternion based head pose estimation with multiregression loss. IEFE Trons. on Mullimedia, 2444:103351044) 2019. 2
Liping Huang and Wasin So. On left eigenvalues of quaternionic matrix. Linear algebra and its applications, 1-3) 2001.
Anil K. Jain. Fundamentals of digital image processing. Prentice-Hall, Inc., 1989. 2, 6, 7, 8, 14, 15, 32
Mahdi Karami, Jascha Sohl-Dickstein, Dale Schuurmans, Laurent Dinh, and Daniel Duckworth. Invertible convolutional flow. In Advances in Neural Information Processing Systems (NeurIPS), pp. 5635-5645, 2019.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139 2023. 3
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Proc. of the Intl. Conf. on Machine Learning (ICML), pp. 2747-2755. PMLR, 2018. 2
Jack B. Kuipers. Quaternions and Rotation Sequences: A primer with application to orbits, aerospace and virtual realily. Princeton University Press, 1999.
Nicolas Le Bihan. The geometry of proper quaternion random variables. Signal Processing, 1011000616. 2017.
Nicolas Le Bihan and Jérôme Mars. Singular value decomposition of quaternion matrices: new tool for vector-sensor signal processing. Signal processing, 84(7):1177-1199, 2004. 31, 34
Nicolas Le Bihan and Stephen J. Sangwine. Quaternion Principal Component Analysis of Color Images. In Proceedings 2003 International Conference on Image Processing (Cat. No. 03CH87429), volume 1, pp. T-809. IEEE, 2003. 13
Stamatios Lefkimmiatis, John P. Ward, and Michael Unser. Hossian Schatten-norm regularization for lincar inverse problems. IEEE Transactions on Image Processing, 22(5):1873 1888, 2013. 16
Heng Li, Zhiwen Liu, Yali Huang, and Yonggang Shi. Quaternion generic Fourier descriptor for color object recognition. Pattern recognition, 48(12):3895 -903399393333,,3- 2015.
E. Macías Virgós, M.J. Pereira-Sáez, and Ana D. Tarrio-Tobar. Rayleigh quotient and left eigenvalues of quaternionic matrices. Linear and Multilinear Algebra, pp. 1-17, 2022. 11, 13, 19
Yibo Miao, Yinpeng Dong, Jinlai Zhang, Lijia Yu, Xiao Yang, and Xiao-Shan Gao. Improving robustness 3D point cloud recognition from a Fourier perspective. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Sebastian Miron, Julien Flamant, Nicolas Le Bihan, Pierre Chainais, and David Bric. Quaternions in signal and image processing: A comprehensive and objective overview. IEEE Signal Processing Magazine, 40(6): 26-40, 2023. 1,
Yuesong Nan, Yuhui Quan, and Hui Ji. Variational-EM-benn deep learning for noise-blind image deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3626 3635, 2020.
Chirag Pabbaraju, Eara Winston, and J Zico Kolter. Estimating Tipschitz constants of monotone deep equilibrium models. In Proc. of the Intl. Conf. in Learning Representations (ICLR), 2021. 16, 24
Titouan Parcollet, Mohamed Morchid, and Georges Linarès. Quaternion convolutional neural networks for heterogencous image processing. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 8514 8518. IEFF, 2019a. 34
27

---

Published in Transactions on Machine Learning Research (11/2025)
Titouan Parcollet, Mohamed Morchid, Georges Linarès, and Renato De Mori. Real to H-space encoder for speech recognition. Xix preprint orXiv: 190..08043,, 2019b. 34
Titouan Parcollet, Mohamed Morchid, and Georges Linares. survey of quaternion neural networks. Arli- ficial Intelligence Review, 53(4):2957-2982, 2020. 2, 24, 34
Soo-Chang Pei, Jian-Jiun Ding, and Ja-Han Chang. Efficient implementation of quaternion Fourier trans- form, convolution, and correlation by 2-D complex FFT. IEEE Transactions on Signal Processing, 49(11): 2783 2797, 2001. 5
Henning Petzka, Asja Fischer, and Denis Lukovnikov. On the regularization of wasserstein gans. In Proc. of the Intl. Conf. in Learning Representations (ICLR), 2018. 16, 24
Simon J.D. Prince. Understanding Deep Learning- MIT Press, 2023. URL http:://ud book. com. 2, 16
Shaofei Qin, Xuan Zhang, Hongteng Xu, and Yi Xu. Fast quaternion product units for learning disentangled representations in SO3. IEFE Transactions Pattern Analysis and Machine Intelligence, 45(4):4504- 4520, 2023. 2, 24
Stefano Rosa, Giorgio Toscana, and Basilio Bona. Q-PSO: Fast quaternion-based pose estimation from RGB-D images. Journal of Intelligent & Robotic Systems, 92(3-4):465 487, 2018.
Ernest Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Plug-and-play methods provably converge with properly trained denoisers. In Proc. of the Infl. Conf. on Machine Learning (ICML), pp. 5546 5557. PMLR, 2019. 16, 24
Stephen J. Sangwine. Fourier transforms of colour images using quaternion or hypercomplex numbers.
Electronics letters, 32(21))1979-1990, 1996. 1,
Stephen J. Sangwine and Nicolas Le Bihan. Quaternion Singular Value Decomposition based on bidiagonal- ization to a real or complex matrix using quaternion householder transformations. Applied mathematics and computation, 182(7):777-738 2006. 1, 16,
Hanie Sedghi, Vineet Gupta, and Philip M. Long. The singular values of convolutional layers. In Proc. of the Infl. Conf. in Learning Representations (ICLR), 2018. 6, 16, 19, 21
Giorgos Sfikas, George Retsinas, Angelos P. Giotis, Basilis Gatos, and Christophoros Nikou, Keyword spotting with quaternionic resnet: Application spotting in greek manuscripts. In International Workshop on Document Analysis Systems (DAS), pp. 382 396. Springer, 2022. 34
Sahil Singla and Soheil Feizi. Fantastic four: Differentiable and efficient bounds on singular values of convolution layers. In Proc. of the Intl. Conf. in Learning Representations (ICLR), 2020. 6
John Stillwell. Naive Lie theory. Springer Science & Business Media, 2008.
Gilbert Strang. Linear algebra and learning from data. Wellesley-Cambridde Cambridge Press, 2019. 7, 8, 9, 12
Özlem N. Subakan and Baba C. Vemuri. A quaternion framework for color image smoothing and segmen- tation. International Journal of Computer Vision, 91(3):233 250, 2011. 1,
Leonard Susskind and Art Friedman. Quantum mechanics: the theoretical minimum. Basic Books, 2014. 1
John Vince. Quaternions for computer graphics. Springer Science & Business Media, 2011.
Christoph Wick, Jochen Zöllner, and Tobias Grüning, Rescoring sequence-to-sequence models for text line recognition with CTC-prefixes. In International Workshop on Document Analysis Systems (DAS), pp. 260 274. Springer, 2022. 2
Yinshuang Xu, Jiahui Lei, and Kostas Daniilidis. SE3-equivariant convolution and transformer in ray space. Advances in Neural Information Processing Systems (NeurIPS), 36:2463 10000-221000 2510, 2023.
28

---

Published in Transactions on Machine Learning Research (11/2025)
Weihao Yu and Xinchao Wang. Mambaout: Do really need mamba for vision? In IEEE/CVE In- lernational Conference Computer Vision and Pollern Recognition. (CVPR), pp. 4484 4496, 2025. 2, 25
Angxiao Yue, Zichong Wang, and Hongteng Xu. Reqflow: rectified quaternion flow for efficient and high- quality protein backbone generation. In Proc. of the Tril. Conf. on Machine Learning (ICMT., 2025. 24
Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Cheung Hui, and Jie Fu. Beyond fully- connected layers with quaternions: Parameterization of hypercomplex multiplications with 1/n parameters. In Proc. of the Intl. Conf. in Learning Representations (ICLR), 2021.
Fuzhen Zhang. Quaternions and matrices of quaternions. Linear algebra and its applications, 251:21-57 1997. 5, 6, 11, 13, 19, 31, 34
Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. Graph convolutional nelworks: a comprehensive review. Computational Social Networks, 6(1):1- 23, 2019. 2, 25
Xuanyu Zhu, Yi Xu, Hongteng Xu, and Changjian Chen. Quaternion convolutional neural networks.
Proceedings of the European Conference on Computer Vision (ECCV), pp. 631 631-647, 2018. 2, 32
Appendix
A Reconstruction Visualization of Quaternionic Doubly-block Circulant Matrix
In Section 4 ("Proof-of-conceep Application") of the main text, we refer to kernel for our experiments which was produced using the code that follows. We report the code in the interest of reproducibility of the result. We used the "Quaternion" " Python package (Boyle, 2018) (as well as for all implementations required for this paper):
import numpy as np import quaternion
# Using quaternion from
# http:://githll
C_ filter = up. zeros ([N, N], dtype=up. quaterniinnnt
for i in range Ia))
for in range (N):
Iml cp.cos(i)
lm2 up.sin [i)
lm3 cp.cos (.3)
1m4 = sin (1-
C_filter [i, j] np. quaternion (
i4mm//N, j+mm//N, i*jelm3, i*j+lm4
In Figure we show an example result of clipping and reconstruction using our method (the proof-of-concept method of Section applied over a single filter) versus applying directly the QSVD ("brute-force") on the corresponding doubly-block circulant matrix. The test filter used 32 x 32 quaternionic convolution. For reference, the mean (maximum) of the singular values of the original kernel is 3005.2 (238519.9) and the clipping threshold was set to 4000. We have used = (1 + j + k)/ /3 as the QFT axis, following (Ell & Sangwine, 2007).
Both singular values and the reconstruction result are exact using our method. The discrepancy point-to- point difference between our reconstruction and brute-force in the ballpark of 10-11 units of magnitude, at worst (cf. last row in Figure 9). terms of time requirements, our method required 280 milliseconds obtain exact reconstruction, compared to minutes for the brute-force method.
29

---

Published in Transactions on Machine Learning Research (11/2025)
lagit Modulus)
Phase
405
400
10
602
3000
200
600
8003
3000
400
500
803
3300
Asis
keii Moodulss)
200
200
200
05
400
600
800
600
800
1000
2000
200
452
3000
220
400
002
3000
200
403
SC0
500
1000
logll-Mood Modukss
Phase
200
200
0.003
15
600
1.0
600
0003
600
805
05 800
100
3003
255
3030
200 400
1000
Figure 9: Constraining the spectral norm using QSVD directly on the doubly-block circulant matrix (referred to as "brute-force" in text) vs our method. From top to bottom, we see: the original kernel, the clipped kernel using our method, the point-to-point difference between our result and the brute-force method. From left right, we see log- mog-tuuni,udes, phases and axes of respective doubly-block circulant matrices (cf, (Ell & Sangwine, 2007)). The brute-force method is considerably more expensive in terms of space well as time requirements, both by orders of magnitude. For this 32 x 32 kernel, our method required 280 milliseconds to obtain an exact reconstruction, while brute-force took over 2 minutes. (see text of Section 4 in the main text for more details).
B Notation conventions
Throughout the lext, we have used the following conventions. Capital letters M, N will refer to matrix dimensions, input or filter size. On the "Proof-of cooocett Section 4, N refers to the side size the input and output images, or equivalently, the zero-padded kernels. Letters μ,v refer to pure unit quaternions, usually used as axes of QFT. QFT matrix is written as Q referring to the matrix with side x N and reference to axis μ. Letters K refer to left eigenvalues. They will be used with an exponent, Xa, which
30

---

Published in Transactions on Machine Learning Research (11/2025)
is to refer to left eigenvalue with respect to axis μ. When want to state that left eigenvalue is related to specific column of QFT matrix as its eigenvector, we add subscript, X Our "by default" ordering is hence with respect to the order of QF columns (and not e.g. left eigenvalue magnitude). A vector or set of left eigenvalues is denoted boldface, A A conjugate element is denoted by a bar (p). An clement that has been produced after singular value clipping in the sense discussed in Section denoted with a "check" " character on top: É.E.
Indexing, if not stated otherwise, is by default considered to be zero-indexed and "modulo-N" (so index 0 is the same as index NN, or -1 is the same as N - 1 and on). This convention is used in order to match the "N exponents definition of the QFT (eq. 5), which is very important for most of the subsequent theoretical results.
C Additional Results and Notions from Quaternion Algebra
Cagley-Dickson Dickson form and symplectic decomposition: A quaternion may be represented as a complex number with complex real and imaginary parts, in a unique manner. We write
q= A+ Bj,
(33)
with
A A + 924, B 9s + qqú.
An analogous operation can be performed for quaternion matrices, which can be written as a couple of complex matrices (Zhang, 1997). This scheme can be easily generalized to using any other couple of per- pendicular imaginary units μt, #22 instead of i,j. In that, more general case, it is referred to as symplectic decomposition (Ell & Sangwine, 2007), which is essentially change of basis from (1, 2, j. k) to new imaginary units (1, (2, We can in general write
p= pl +p-
where pll = Po P1 and pt = (p2 + P3 P222 + Р2/33 Quaternions pll and p are referred as the parallel / simplex part and the perpendicular / perplex part with respect to some basis (1, M2, M4)) Given symplectic decompositions for quaternions p.q, their parts may commute or conjugate-comment (Ell & Sangwine, 2007, Section 7C). Note the conjugate operator over pll in the second relation:
pllgاl = -ppp!
= =
Quoternion Singular Value Decomposition: (Zhang, 1997) and (Le Bihan & Mars, 2004) have been among the first works to discuss the extension of the SVD into the quaternionic domain, dubbed Singular Value Decom- position for Quaternion matrices or simply Quaternion Singular Value Decomposition (QSVD). (Sangwine & Le Bihan, 2006) have proposed a bidiagonalization- based implementation of QSVD.
The fundamental proposition is that, for any matrix A E MMN, or rank r, there exist two quaternion unitary matrices U and V such that
Σ, V" 0
0 0
A A U-VV = 1
(34)
0
where Σ, is a real diagonal matrix and has r strictly positive entries on its diagonal (Le Bihan & Mars, 2004, Proposition 10). The elements of the diagonal are termed singular values. Matrices U E HMxM and V E HNNN are made up of columns that are cach left and right eigenvectors of S. Note that, perhaps confusingly, the adjectives "left" and "right" are not corresponding to the "left" and "right" spectra of the matrix. Both sets of eigenvectors correspond to singular values which are in turn rather more closely related to the right spectrum. We can see this by considering A A or AAH, which starting from eq. 34 equal to vy²y and U respectively. Consequently, we have A AV V and A = US?, which solve the right eigenvalue problem, Ax = xd.
Hence, the singular values are the non-zero right cigenvalues of either A A or AAH As these must be R. as eigenvalues of Hermitian matrices, they are also left eigenvalues. Left eigenvahues that are not right eigenvalues may very well exist, even for Hermitian matrices (Zhang, 1997, Example 5.3).
31

---

Published in Transactions on Machine Learning Research (11/2025)
D Quaternionic Convolution and Quaternionic Fourier Transform
D.1 Quaternionic Convolution
In the continuous case, quaternionic convolution is defined as (Bahri et al., 2013):
(f*g)(z)= = / f(w)g(z u)du,
(35)
and cross-correlation (Ell et al., 2014) as:
(f)g) J f(w)g(x u)du,
(36)
where V = = for 1D and 2D convolution/correlaiici respectively. Similarity of the above to the well- known non-quaternionie formulae is evident. A number of useful properties of non-quaternionie convolution also hold for quaternionic convolution. These propertics include lincarity, shifting, associativity and distribu- tivity (Bahri et al., 2013). Perhaps unsurprisingly, quaternionic convolution is non non-commutaaiie (f*g + g* in general), and conjugation inverses the order of convolution operators (ʃ * g = g ** f). In practice, discrete versions ID convolution and correlation are employed. In this work we will use the following definitions of 1D quaternion convolution (Ell et al., 2014, Scction 4.1.3):
N Σ
(hL f(
h2lffn i).
(37)
n=0
(f*h))]n]= = Σ fin f|n-l]
(38)
n=0
where the difference between the two formulae is whether the convolution kernel elements multiply the signal from the left or right. Extending to 2D quaternion convolution, two of the formulae correspond precisely to left and right 1D convolution:
N M
(h,, * f) m]
Σ
Σ
hrliffin i,m -1,.
(39)
n=0
N- M
(f + h2+)))n
Σ
Σ
f -n--i,m-)hhppp jl
(40)
n=0 m=0
while a third option available, in which convolution kernel elements multiply the signal from both left and right (Ell et al., 2014):
(hr < hn)m (n, m]
M-
Σ Σ
hrlifff[n - i, - jhhnli...
(41)
R=O m==
This last variant, referred as bi-convolution (Ell al., 2014), has been used to define extension of standard edge detection filters (Sobel, Kirsch, Prewitt) for color images. A variation of bi-convolution has also been used in (Zhu et al., 2018) part of quaternion convolution neural network, locking h2 Fa and adding scaling factor.
We introduce circular variants to the above formulae. Circular left convolution is written as:
(hz, [)
Σ hn i).
(42)
where [ denotes modulo-N indexing (Jain, 1989).
32

---

We shall also employ the short-hand hand notation Fx (g) with X = {L, R2} to denote the left or right QFT with axis μ of a signal g. The inverse transforms will be denoted as Fx", as we can easily confirm that F =x F& is the identity transform. Asymmetric definitions are also useful, and we will use Published in Transactions on Machine Learning Research (11/2025)
D.2 Quaternionic Fourier Transform
For 1D signals we have the definition of the left- and right-side QFT:
N-1
Fulu]
u2nN nu f/1]
(43)
VN
N Amll
FR
=2eN~= nu
(44)
n=
where j is an arbitrary pure unit quaternion that is called the aris of the transform. For 2D) signals, these formulae are generalized to the following definitions (Ell & Sangwine, 2007);
M-
Filu, u]
1
Σ
Σ
22 mmv+N Amx) f(m, m),
(45)
VMN
m=U
n=0
M-1N-1
Filu,
Σ
fn, ml "M
(46)
VMN
Fk denote a transform with a coefficient equal to 1 instead of 1/vN and 1/VMN.
D.3 Convolution theorem in H
In the complex domain, the convolution theorem links together the operations of convolution and the Fourier transform in an elegant manner. For most combinations of adaptation variants of convolution and Fourier transform in the quaternionic domain, similar theorem is not straightforward, if at all possible (e.g. (Cheng & Kou, 2019)). (Ell & Sangwine, 2007) have proved the following formulac for right-side convolution (we have changed equations represent n-dimensional signals in general):
Filf= * h] =
+ F Hu) [u]],
(47)
# hh} VNL
(48)
which by symmetry complemented by the following formulae for left-side convolution:
Fh f) -VN [u vl Fl o] H [u, FI v]},
(49)
F f} VN [ [R + [u]]pp
(50)
On the above formulae, transforms F H are decomposed as:
Fl = Ft P2t Frt = F F2n
" 1 + H2r 11 11 11R22
where we use symplectic dempositions (cf. section C) with respect to the basis (1, p. p2, H12).. Note that μl is an arbitrary pure quaternion conforming to μ 112-
As different transforms are obtained by choosing a different axis we have chosen to denote the axis explicitly in our nota- tion. This formulation is slightly more generic than the one proposed by Ell & Sangwine (2007), and we have the correspondence of F-X (their notation) to F (our notation).
33

---

Published in Transactions on Machine Learning Research (11/2025)
E
Additional Proofs
Lemma 4.1 For any pure unit E H and any N E N, the right span of the columns of Q' is H For any pure unil je E H. and any M, N E N, the right span of the columns of Q @ Q'N is HMN
Proof. Let c E H and let x = Σm= ag (Le Bihan & Mars, 2004, definition 2), where ag is the hthh column of QN In matrix-vector notation, we have x = 2%c But Q% is invertible for any N. (Proposition 3.3), so c c= Q7" x. Thus, for any x E HA, we can compute right linear combination coefficients [er C2 c2... c2]' c.
The proof for Q & 0% is analogous.
(An analogous result holds also for the left span; suffices to take cH = Q x..
Lemma 4.2 Let HN be diagondig The left eigenvalues of each block of are also left eigencolues of A. The right eigenvalues of each block of A are also right eigenvalues of A.
Proof. For a block Aab that situated in the submatrix of within rows and columns a b, take 0 x 0..../" as an eigenvector of A, where r is an eigenvector of Aab.
Corollary 4.4.1 For any matrir & such that & g E(Kc), matrices C and have the same singular values.
Proof. We can write the change of basis in eq. 28 as C CQ's and after we combine it with a rearraging of rows and columns encoded left-right multiplication by permutation matrix P, we have
E = pt C" P (Q" 10
E and C# C are similar matrices, which entails that they have the same right eigenvalues (Zhang, 1997, Section 7). The same holds for 4 and C C, by our assumption that 4 4 & {kkc) The singular values of 4 are the (non-zero) right eigenvalues of 4 " and the singular values of C' are the (non-zero) right cigenvalues of C C, which proves the corollary.
F Model architecture and Optimization details for experiments on CIFAR-10
Dalasels and preprocessing. All initial experiments are conducted on CIFAR-10. Training images are aug- mented with random horizontal flips and random 32 x 32 crops after 4-pixel zero-padding, Inputs are converted to tensors and normalized channel-wise using the widely-adopted ImageNet statistics. Evaluation uses the non-augmented (only channel-wise normalization) pipeline.
Model, We use a CIFAR-style ResNet-32 topology with three stages of residual blocks and increasing channel width. Each stage has 5 residual blocks. Each block applics a 3 3 convolution, batch normalization, and ReLU; downsampling between stages is performed by strided convolutions. The network ends with global average pooling and linear classifier. We then "quaternionize" the model by replacing the typical 3 x 3 convolution on the residual blocks with a depthwise quaternionic one. The input convolution layer is not replaced by a quaternionic one. To re-enable (non-quaternionic) channel mixing, we have a variant where a pointwise real convolution follows each introduced quaternionic depthwise convolution at the cost of more parameters lResNet32--arre variant)..
Comment quaternionization. Regarding the process of "quaternionizationn" i.e. creating architecture that can be perceived as analogous or comparable to real-valued one, we follow previous work (Parcollet et al., 2020; Sfikas et al., 2022; Comminicllo et al., 2024; Parcollet et al., 2019b;a) and use the following trivial mappings from the real-valued to the quaternion domain, back and forth. In particular, assuming an input tensor is sized #XxXC we define a mapping H. "XXXCC + HW XC/4 by keeping coordinates H, W identical in input and output, and rearranging input channels to channels, real and imaginary components of the output. In this manner, channel c, = 4c, + b of the input, where e E [0..C/4 1] and b e [0..3] of the real-valued input is mapped to channel o of the output, and the remainder is used to specify the
34

---

Published in Transactions on Machine Learning Research (11/2025)
exact target channel. By convention, if b = 0 the imput is mapped to the real component, and if b 2, to imaginary components i, k respectively (of course, numerous other conventions may be envisaged, mutually equivalent in practice). This may be a simple enough mapping, however it requires the number of channels to be a multiple of Typically, the input and the output layer may pose a constrain in this regard: the input may be represented in a dimensionality that is not a multiple a dimensionaliit equal to 3), and the output may need be dimensionality that is, again, not a multiple of (e.g. number of classes on classífication task). In the literature, this has been worked around in a number of ways, out of which the most usual ones are either a) zero- pero-padding the imput (or output) with extra channels until they are multiple of 4 b) use a x convolution that acts as linear combination over channels or c) have real-valued convolution for the input (or output). In our implementation, we opt for the latter, to avoid introducing an unnecessary bias on the workflow in terms of comparing the quaternion-val valuud to the real-valued versions of the network. Importantly, the discussed mapping H also a bijection, with an inverse H-1 = -> I #x This mapping is defined in a completely analogous manner H. with real or imaginary channels mapped to the pseudoindex b E [0..3], and the same formula (c, 4c,, is used to map to real-valued indices from quaternion domain indices.
(e.g., color images have
Training protocol. Models are trained for 80 epochs with SGD on cross-cntropy loss (initial lcarning rate 0.1 - momentum 0.8) and a factor of 0.1 at 50% and 75% of training.
of 128. The learning rate follows multi-step schedule, decayed by
35


Reviews:
[{"review_id": "OPxygPKpbo", "reviewer": "Reviewer_3wXS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. Generalization of Matrix Representations to the Quaternion Domain: The paper establishes matrix forms for Quaternionic Convolution and the Quaternion Fourier Transform (QFT), demonstrating that they possess properties analogous to their real and complex counterparts. It reveals that the standard complex Fourier matrix is a special case within an infinite family of QFT matrices, which are interconnected via a rotation-like operation.\n\n2. Elucidation of the Connection between Circulant and QFT Matrices: The work clarifies the nuanced relationship that links quaternionic circulant matrices (representing convolution) and QFT matrices in the quaternion domain. A notable finding is that the left eigenvalues of a convolution kernel are functions of both the left and right QFTs.\n\n3. A Method for Left Eigenvalue Computation and Reconstruction: Addressing the challenge that the left spectrum of a quaternion matrix is generally difficult to compute, the paper introduces an efficient method to calculate a specific set of N left eigenvalues for an N×N circulant matrix using the QFT. It further proves that this set of eigenvalues is sufficient to uniquely reconstruct the original circulant matrix under specific conditions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The core theoretical contributions presented in Section 3 of the paper, and  The authors conducted experiments to verify that their method.", "audience_yn": "Yes", "audience_explanation": "The paper bridges theory with a practical application in deep learning, making it appealing to several key groups within the machine learning research community.", "requested_changes": "The paper presents a valuable contribution by extending the theory of circulant matrices and the Fourier Transform to the quaternion domain and demonstrating its application for regularizing Quaternionic Neural Networks (QNNs). The theoretical proofs are sound, and the empirical results are convincing. However, the work could be made significantly more accessible and its impact broadened with a few adjustments focused on improving exposition and contextualizing the experimental results.\n\n1. Enhancing Intuition for Theoretical Results. The paper is theoretically dense and jumps directly from propositions to formal proofs. This approach may be difficult for a broader machine learning audience to follow, especially for the non-intuitive results stemming from quaternion non-commutativity (e.g., the connection between left eigenvalues and the right QFT in Proposition 3.5). I suggest adding a few sentences or a short paragraph after key propositions (especially 3.4, 3.5, and 4.3) to provide a more high-level, intuitive explanation of the result.\n\n2. Broadening the Introduction and Discussion. The work is primarily framed within the context of QNNs. While this is the direct application, the core contribution—an efficient method to analyze the spectral properties of a convolutional operator—has broader implications. I recommend expanding the introduction and conclusion to better situate the work within the wider field of deep learning. The authors could emphasize that they are solving a general problem of efficiently analyzing convolutional layers, and the quaternion domain is a particularly challenging and interesting instance of this problem. \n\n3. Contextualizing Experimental Performance with Other Regularizers. The experiments convincingly show that applying spectral norm clipping (enabled by the paper's method) improves performance compared to not applying it. However, it is unclear how this method compares to other standard regularization techniques when applied to QNNs, such as weight decay or dropout.\n\n4. Discussing Applicability to Modern Architectures. The experiments are conducted on VGG and ResNet-style architectures. While these are standard, the deep learning landscape is rapidly evolving with architectures like Vision Transformers (ViTs), ConvNeXts, and State Space Models (e.g., Mamba).", "broader_impact_concerns": "There are no concerns about the ethical implications of the work that would require adding a Broader Impact Statement", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "p8OssModyw", "reviewer": "Reviewer_4wny", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work studies the matrix form of the Quaternion Fourier Transform (QFT) and quaternionic convolutions. It focuses on showing how the classical results can extend, in many cases with non-trivial ways, to the quaternionic domain. The paper first defines the family of Quaternionic Fourier matrices $Q_N^\\mu$ (given a pure unit quaternion $\\mu$), and shows how the columns of $Q_N^{-\\mu}$ are eigenvectors of the quaternionic circulant $C$ matrices. These results,  which connect the Quaternionic Fourier matrices with the eigenvalues/eigenvectors of the quaternionic circulant matrices,  are then extended to the doubly block-circulant matrices, which allows the authors to handle cases of 2D convolution. Building on their results, the authors propose a method for computing and bounding the spectral norm of quaternionic convolutional layers without explicitly forming the large doubly-block circulant matrix that densely describes the 2D convolution. Finally, they validate the significant computational gains of their proposed method for bounding the spectral norm of quaternionic 2D convolution, in a ResNet-style architecture trained on CIFAR-10.\n\n**Strengths**:\n- The authors provide a clear formulation of the QFT matrices and their relationship to the DFT. They show that $Q_N^\\mu$ generalizes the complex DFT matrix in the quaternionic domain, with the standard DFT being recovered when $\\mu=i$.\n- The extension of the results to the doubly block-circulant quaternionic matrices is quite helpful since it allows the results to be naturally applied in 2D convolution and all subsequent tasks that utilize them.\n- Such a benefit is shown in the provided proof of concept application of bounding the spectral norm of quaternionic 2D convolutions.\n- The appendix provides a nice overview of the properties of quaternions, making the work accessible to readers who are not familiar with the topic. However, I believe that some of these introductory definitions would be better located at the start of the main text to create a more linear reading flow.\n\n**Weaknesses**:\n- Some of the proofs contain mistakes/typos that make it harder to evaluate their correctness. For example, in equation (12), where is the definition of $h[i]_N$? Also, after equation (12), the authors refer to the $k[i]_N$ without using it. Are $h[i]_N$ and $k[i]_N$ the same thing, and the use of $k$ is a typo?  Similarly, in proposition 3.8, $\\mu$ is defined but never used, while $x$ is used but never defined. While these mistakes might be small, they reduce the confidence of the reader who tries to understand the theoretical formulations.\n- Proposition 3.8 is provided without any proof. For the completeness of the paper, all provided propositions must be accompanied by a proof, even if it is located in the appendix. Similarly some of the corollaries (3.6.1, 3.6.2, 3.7) are not accompanied by a proof, although in this case they might be considered too trivial to be included.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Clarity**: Apart from the mistakes/typos noted in the weaknesses, the paper is well structured and easily readable even to an audience that might not be an expert on the topic.\n\n**Correctness**: Overall, given that the authors address the mistakes/typos, all of the formulations and theoretical results appear sound. In addition to the proofs, the authors showcase how the above results provide actionable tools by describing a proof-of-concept application of spectral bounding of quaternionic convolution in Section 4.", "audience_yn": "Yes", "audience_explanation": "This work provides a significant contribution that can be of interest to the TMLR community since it performs an in-depth analysis of the properties of QFT matrices that can enable practitioners to incorporate quaternionic convolutions in machine learning models more easily.", "requested_changes": "**Critical changes**:\n- The authors should address the two weaknesses regarding the typos in the proofs and the fact that some proposition corollaries are not accompanied by a proof. This is critical for the publication of this work since it affects the correctness of the presented results.\n\n**Minor Changes and Suggestions**:\n- In the proposed application of bounding the spectral norm of the quaternionic convolutions, it would be interesting to also study the numerical stability of the proposed method in comparison with the brute-force method. Such a comparison would be quite interesting since the spectral norm normalization methods are often numerically unstable when incorporated in a learning pipeline.\n- One possible suggestion, mentioned also in the last bullet point of Strengths, is to include some of the preliminary definitions regarding the quaternions at the beginning of the main text, so that the paper has a more linear reading flow.", "broader_impact_concerns": "There are no concerns about the ethical implications of the work that would require adding a Broader Impact Statement", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YGFiJFN1We", "reviewer": "Reviewer_pLGg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a new series of theoretical results regarding the singular values of circulant quaternionic matrices and their relationship to the quaternionic Fourier transform, extending popular results in the complex domain that relate the singular values of a circulant matrix to the Fourier transform of the associated filter. \n\nThe paper leverages these results to propose a new algorithm for efficiently computing the singular values of quaternionic (doubly) block-circulant matrices, which has a significantly smaller complexity than using the quaternionic version of the singular value decomposition (ie without exploiting the block-circulant structure). \n\nThroughout a series of CIFAR10 classification experiments with quaternionic ResNets, the efficient singular value algorithm is used to bound the Lipschitz constant of quaternionic neural networks in a reasonable amount of time, in order to improve the generalization of such networks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "All theoretical results in the paper are supported by proofs, which appear correct to me, although I did not fully verify their correctness, as I'm not an expert on quaternions. The experimental results are convincing as well, demonstrating clear evidence for the impact of spectral norm clipping on the performance of quaternionic neural networks. My only concern regarding these results is the (I believe) unfair comparison with real counterparts: I believe that the authors should compare with non-depth-wise convolutions that can mix channels, as quaternionic networks do mix channels due to the Hamilton product. The current comparison is slightly misleading, as it leads to the conclusion that quaternionic networks are much better than their real counterparts by choosing an unrealistic backbone architecture.", "audience_yn": "Yes", "audience_explanation": "I believe this paper could be of interest to machine learning researchers developing quaternionic neural networks and other algorithms leveraging quaternions.", "requested_changes": "I believe that the presentation in the paper could be greatly improved by polishing the mathematical notation, providing an introduction to quaternions for non-experts, fixing typos and flawed sentences, and moving some technical lemmas/proofs to the appendix. \n\nI also have a concern regarding computing the spectral norm: why not use power methods, which are generally much less expensive than computing all singular values? Is there a hidden assumption about the non-linearity of the network? (I guess some non-linearities could have large Lipschitz constants, and break the constant of the overall network)\nIt would also be good to mention other applications of training networks with a constrained Lipschitz constant beyond improving generalization, such as Wasserstein GANs or convergent plug-and-play/deep equilibrium networks. \n\nNotation:\n- I would use ^\\top instead of ^T for transposing\n- I would explain what \\mathbb{H} is at the beginning of the paper (no more than half a page) and briefly introduce basic concepts around quaternions, ie what is a unit quaternion, how do we write a quaternion etc.\n- I find using uppercase for the Fourier transform of a vector confusing, I would keep any vector lowercase\n- I would write \"j=0..N\" -> \"j=0,1,\\dots,N\"\n- I would use begin{proof} end{proof} for proofs. For example, the beginning of the proof of prop. 3.3 is unclear\n- modulo-N -> modulo-$N$\n- for norms I would use the latex command \"\\|\" instead of \"||\" (openreview doesn't display it well, but it should give two bars on latex)\n- I would use $\\ell$ instead of $l$ \n- I would use \\text{} for rank(.), arccos(.), qsvd(.) etc\n\n\nPresentation:\n- I would move some proofs and technical lemmas to the appendix (for example, lemmas 4.1 and 4.2) - I find the current organization quite dense to read, hindering a global understanding of contributions/ideas. \n- The definition of a layer could be improved at the beginning of page 11. The term \"linearity B\" could be simplified to the \"linear layer\" or \"matrix\" B.\n- Table 1 could be improved, currently it is hard to understand from the caption what is the difference between SV and Full, and its confusing that the time unit is reported in some cell \"hours\" whereas is not explained in others.\n\nTypos/Flawed sentences:\n- the first footnote appears after the full stop, instead of before. There's also some unnecessary blank space before footnotes\n- page 2, remove \"In our opinion, it is notable that \"\n- page 2, contribution 3 could be simplified, it is confusing to refer to \"the previous result\" where no result has been shown so far.\n- page 3, \"quternion\", \"wherever\"\n- page 5, blank space missing \"Notation Conventions)For columns\"\n- page 14, \"Times to just\" -> \"Computational times\n- page 15 \"c = 1.0.\" -> \"c=1.\"\n- Table 1: \"2D kernel side size\" is confusing\n- page 16, I'm not sure what \"To this end we continue out experimentation with the more “interesting\" case of the small version.\" means", "broader_impact_concerns": "I don't have any ethical concerns regarding this work, which is mostly theoretical.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JTbEc98YL6", "reviewer": "Reviewer_QEr5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors investigate the connection between quaternionic circulant matrices and quaternionic Fourier matrices. They then use these results to efficiently constrain the eigenvalues of the quaternionic operators in a quaternion convolutional neural network. This serves as a regularization form that results on improved result using two ResNet-style model sizes on CIFAR10. Computing this projection without the presented results would generally be prohibitive.", "claims_and_evidence_yn": "No", "claims_explanation": "Disclaimer: I am far from an expert in the field. I think I have been assigned this paper due to my past experience with rotation equivariant CNNs, which is unrelated to the work presented in this paper. I thus have a very superficial understanding of the topic of this paper and I am unable to assess the correctness of the theoretical results.\nI will, therefore, focus on evaluating the experimental setup and results, and leave to more knowledgeable reviewers the evaluation of the correctness and relevance of the theoretical results.\n\n1. I appreciate that the authors provide results for multiple random seeds in Fig 3. However, I would suggest rather showing these results by simply adding standard deviations to Tables 2 and 3. This would allow to also see the variation in the baselines.\n2. There is an important lack of details about the experimental setup. I could not find what optimizer was used nor what hyperparameters. I can imagine the results may be different when different optimizers are used.\n3. Although clipping the eigenvalues of the operators seems to improve results, there is no discussion about the alternatives (or lack thereof) to this. In real-valued neural networks, weight decay is generally used to regularize the model weights. A short discussion about whether it is possible or not to use a similar approach with QCNNs would be useful for readers like me.\n4. I miss some comparisons to real-valued models, just as a reference, in terms of performance and running time.\n5. The experiments only include one dataset (CIFAR10) and one architecture (ResNet, even if in two different sizes are used). This undermines the claims about the usefulness of the clipping approach made possible by the results in the paper. I understand there may be computational concerns regarding the extension of experiments, but there are small datasets that could be used (e.g. CUB), and architectures of similar size (VGG or ConvNext). Such extended experiments would be required to sustain the claim about usefulness of the clipping approach.\n6. Other than the proof-of-concept in section 4, I would expect to see a discussion section about other possible applications of the paper’s results and QCNNs in general. In terms of real-valued models, CNNs tend to be replaced by non-(explicitly)-convolutional methods, such as those based on transformers or mamba. Any connection between the paper’s results (or related future work) and these other architectures would be useful.", "audience_yn": "Yes", "audience_explanation": "Although not widely used, quaternion-based neural networks and advances towards making them usable in practice have some potential that may make them interesting to the community.", "requested_changes": "I would like to see a response to each of the raised points:\n1. Adding std to the results, rather than showing several runs as plots.\n2. Adding optimization details.\n3. Adding some discussion about alternative regularizations for QCNNs and their equivalents in real-valued models.\n4. Comparisons to real-valued models.\n5. Adding at least one additional dataset (I suggest CUB or some other small but challenging dataset).\n6. Add a discussion about other possible applications of the proposed theoretical results.", "broader_impact_concerns": "Some discussion about computational requirements would be welcome.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)