L IGHTWEIGHT C ONVOLUTIONAL N EURAL N ETWORKS B Y H YPERCOMPLEX P ARAMETERIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Hypercomplex neural networks have proved to reduce the overall number of parameters while ensuring valuable performances by leveraging the properties of
Clifford algebras.
Recently, hypercomplex linear layers have been further improved by involving efﬁcient parameterized Kronecker products.
In this paper, we deﬁne the parameterization of hypercomplex convolutional layers to develop lightweight and efﬁcient large-scale convolutional models.
Our method grasps the convolution rules and the ﬁlters organization directly from data without requiring a rigidly predeﬁned domain structure to follow.
The proposed approach is ﬂexible to operate in any user-deﬁned or tuned domain, from 1D to n D regardless of whether the algebra rules are preset.
Such a malleability allows processing multidimensional inputs in their natural domain without annexing further dimensions, as done, instead, in quaternion neural networks for 3D inputs like color images.
As a result, the proposed method operates with 1 /n free parameters as regards its analog in the real domain.
We demonstrate the versatility of this approach to multiple domains of application by performing experiments on various image datasets as well as audio datasets in which our method outperforms real and quaternionvalued counterparts.
I NTRODUCTION
Recent state-of-the-art convolutional models achieved astonishing results in various ﬁelds of application by large-scaling the overall parameters amount (Karras et al., 2020; d’Ascoli et al., 2021;
Dosovitskiy et al., 2021).
Simultaneously, quaternion neural networks (QNNs) demonstrated to signiﬁcantly reduce the number of parameters while still gaining comparable performances (Parcollet et al., 2019c;
Grassucci et al., 2021a; Tay et al., 2019).
Quaternion models exploit hypercomplex algebra properties, including the Hamilton product, to painstakingly design interactions among the imaginary units, thus involving 1 / 4 of free parameters with respect to real-valued models.
Furthermore, thanks to the modelled interactions, quaternion networks capture internal latent relations in multidimensional inputs and preserve pre-existing correlations among dimensions.
Therefore, the quaternion domain is particularly appropriate for processing 3 D or 4 D data, such as color images or (up to)
4 -channel signals.
Unfortunately, most common color image datasets contain RGB images and a zero channel has to be padded to the input in order to encapsulate the image in the four quaternion components.
Additionally, while quaternion neural components are widespread and easy to be integrated in pre-existing models, very few attempts have been made to extend models to higher domain orders.
Accordingly, the development of hypercomplex convolutional models for larger multidimensional inputs, such as magnitudes and phases of multichannel audio signals or 16band satellite images, still remains painful.
Recently, a parameterized hypercomplex multiplication (PHM) for fully connected layers have been proposed to generalize hypercomplex multiplications as sum of Kronecker products, going beyond quaternion algebra.
However, no solution exists for convolutional layers, which remain the most employed layers when dealing with multidimensional inputs, such as images and audio signals (Wu et al., 2021; Hershey et al., 2017).
In this paper, we propose a parameterized hypercomplex convolutional (PHC) layer to deﬁne lightweight large-scale neural models admitting any multidimensional input, whichever the number of dimensions.
Our method is ﬂexible to operate in domains from 1 D to n D, where n can be arbitrarily chosen by the user or tuned to let the model performance lead to the most appropriate domain for the given input data.
Such a malleability comes from the ability of the proposed approach to subsume algebra rules to perform convolution regardless of whether these regulations are preset or not.
Thus, neural models endowed with our approach adopt 1 /n of free parameters with respect to their real-valued counterparts, and the amount of parameter reduction is a user choice.
This makes PHC layers adaptable to a plethora of applications and hardware, even on the edge where memory saving is a crucial aspect.
Additionally, the PHC versatility allows processing multidimensional data in its natural domain by simply setting the dimensional hyperparameter n .
For instance, color images can be analyzed in their RGB domain by setting n = 3 without adding any useless information, contrary to standard processing for quaternion networks with the padded zero-channel.
Indeed, PHC-based models are able to grasp the proper algebra from input data, while capturing internal correlations among the image channels and saving 66% of free parameters.
On a thorough empirical evaluation on multiple benchmarks, we demonstrate the ﬂexibility of our method that can be adopted in different domain of applications, from images to audio signals.
We devise a set of convolutional models endowed with PHC layers for large-scale image classiﬁcation and sound event detection tasks, letting them operate in different hypercomplex domain and with various input dimensionality with n ranging from 2 to 16 .
The contribution of this paper is three-fold.
• We introduce a parameterized hypercomplex convolutional (PHC) layer, which aims at building lightweight and more efﬁcient large-scale convolutional models.
The approach grasps the convolution rules directly from data via backpropagation exploiting the Kronecker product properties, thus reducing the number of free parameters to 1 /n .
• We show how the proposed approach can be employed with any kind of multidimensional data by easily changing the hyperparameter n .
Indeed, by setting n = 3 the PHC-based models can process RGB images in their natural domain, while leveraging the properties of hypercomplex algebras, allowing parameters sharing inside the layers and leading to a parameter reduction to 1 / 3 .
To the best of our knowledge, this is the ﬁrst approach that processes color images with hypercomplex-based neural models without adding any padding channel.
As well, multichannel audio signals can be analysed by simply considering n =for standard ﬁrst-order ambisonics (which has 4 microphone capsules), n = 8 for an array of two ambisonics microphones, or even n = 16 if we want to include the information of each channel phase.
• We devise a family of PHC neural networks, redeﬁning common ResNets, VGGs and Sound Event Detection networks (SEDnets), operating in any user-deﬁned domain just by choosing the hyperparameter n , which also drives the number of convolutional ﬁlters.
The rest of the paper is organized as follows.
In Section 2, we recapitulate real and quaternion-valued convolutional layers.
Section 3 theoretically introduces the proposed method and expounds how to process RGB images with n = 3 , while in Section 4 we deﬁne the PHC models and we present the experimental evaluation, including datasets, experiment details and results.
Finally, Sectionreports the related works and in Section 6 we draw conclusions.
R EAL AND Q UATERNION -V ALUED C ONVOLUTIONAL L AYERS
A brief recall on real-valued and quaternion convolutional layers is appropriate to better understand the proposed method.
A generic convolutional layer can be described by
[Equation]
where the input x ∈ R t × s is convolved ( ∗ ) with the ﬁlter tensor W ∈ R s × d × k × k to produce the output y ∈ R d × t , where k is the ﬁlter size.
The bias term b does not heavily inﬂuence the number of parameters, thus the degrees of freedom for this operation are essentially O
( sdk 2 ) .
Quaternion convolutional layers, instead, build the weight tensor W ∈ R s × d × k × k by following the
Hamilton product rule and organize ﬁlters according to it:
science/
-CBBB .
[Equation]

[Equation]
 
[Equation]
  [Equation] where W 0 , W 1 , W 2 , W 3 ∈ R s 4 × d 4 × k × k are the real coefﬁcients of the quaternion weight matrix
W = W 0 + W
ˆ ı + W 2 ˆ  + W 3 ˆ κ and x 0 , x 1 , x 2 , x 3 are the coefﬁcients of the quaternion input x with the same structure.
The imaginary units comply with the property ˆ ı 2 = ˆ 
= ˆ κ 2 = − 1 and with the non-commutative products ˆ ı ˆ  = − ˆ  ˆ ı ; ˆ  ˆ κ = − ˆ κ ˆ  ; ˆ κ ˆ ı = − ˆ ı ˆ κ .
As done for real-valued layers, the bias can be ignored and the degree of freedom computations of the quaternion convolutional layer can be approximated to O
( sdk 2 / 4) .
The lower number of parameters with respect to the real-valued operation is due to the reuse of ﬁlters performed by the
Hamilton product in Eq.2.
Also, sharing the parameter submatrices forces to consider and exploit the correlation between the input components (Parcollet et al., 2019a; Tay et al., 2019; Gaudet &
Maida, 2018).
P ARAMETERIZING H YPERCOMPLEX C ONVOLUTIONS 3. P ARAMETERIZED H YPERCOMPLEX C ONVOLUTIONAL L AYERS
In the following, we delineate the formulation for the proposed parameterized hypercomplex convolutional (PHC) layer.
We also show that this approach is capable of learning the Hamilton product rule when two quaternions are convolved.
The PHC layer is based on the construction, by sum of Kronecker products, of the weight tensor H which encapsulates and organizes the ﬁlters of the convolution.
The proposed method is formally deﬁned as:
[Equation]
whereby H ∈ R s × d × k × k is built by sum of Kronecker products between two learnable matrices.
Here, s is the input dimensionality to the layer, d is the output one, and k is the ﬁlter size.
More concretely,
[Equation]
in which A i ∈ R n × n with i = 1 , ..., n are the matrices that describe the algebra rules and F i ∈
R sn × dn × k × k represents the i -th batch of ﬁlters that are arranged by following the algebra rules to compose the ﬁnal weight matrix.
The core element of this approach is the Kronecker product, which is a generalization of the vector outer product that can be parameterized by n .
The hyperparameter n can be set by the user who wants to operate in a pre-deﬁned real or hypercomplex domain (e.g., by setting n = 2 the PHC layer is deﬁned in the complex domain, or in the quaternion one if n ... is set equal to 4 , as Figure 1 illustrates), or tuned to obtain the best performance from the model.
The matrices A i and F i are learnt during training and their values are reused to build the deﬁnitive tensor H .
The degree of freedom of A i and F i are n
and sdk 2 /n , respectively.
Usually, real world applications employ a large number of ﬁlters in layers ( s, d = 256 , 512 , ... ) and small values for k . Therefore, frequently sdk 2 (cid:29) n 3 holds. Thus, the degrees of freedom for the PHC weight matrix can be approximated to O ( sdk 2 /n ) .
Hence, the PHC layer reduces the number of parameters by 1 /n with respect to a standard convolutional layer in real world problems.
Moreover, when processing multidimensional data with correlated channels, such as color images, rather than mulichannel audio or multisensor signals, PHC layers bring beneﬁts due to the weight sharing among different channels.
This allows capturing latent intra-channels relations that standard convolutional networks ignore because of the rigid structure of the weights (Grassucci et al., 2021a;
Parcollet et al., 2019b).
The PHC layer is able to subsume hypercomplex convolution rules and the desired domain is speciﬁed by the hyperparameter n .
We test this ability with toy problems in
Appendix B. Interestingly, by setting n = 1 a real-valued convolutional layer can be represented too.
Indeed, standard real layers does not involve parameters sharing, therefore the algebra rules are solely described by the single A ∈ R 1 × 1 and the complete set of ﬁlters are included in F s × d × k × k .
We report detailed explanations in Appendix A.
3. P ARAMETERIZED LAYERS FOR COLOR IMAGES
In this section, we describe how the PHC layer can be employed to process color images in hypercomplex domains without needing any additional information to the input.
Different encodes exist to process color images, however, the most common computer vision datasets are comprised of three-channel images in R 3 .
In the quaternion domain, RGB images are enclosed into a quaternion and processed as single elements (Parcollet et al., 2019a).
The encapsulation is performed by considering the RGB channels as the real coefﬁcients of the imaginary units and by padding a zeros channel as the ﬁrst real component of the quaternion.
Here, we propose to leverage the high malleability of PHC layers to deal with RGB images in hypercomplex domains without embedding useless information to the input.
Indeed, the PHC can directly operate in R 3 by easily setting n = 3 and process RGB images in their natural domain while exploiting hypercomplex algebra properties such as parameters sharing.
Indeed, the great ﬂexibility of PHC layers allows the user to choose whether processing images in Ror R 3 .
On one hand, by setting n = 4 , the zeros channel is added to the input even so the layer saves the 75% of free parameters.
On the other hand, by choosing n = 3 the network does not handle any useless information, notwithstanding, it reduces the number of parameters by solely 66% .
This is a trade-off which may depend on the application or on the hardware the user needs.
Furthermore, the domain on which processing images can be tuned by letting the performance of the network indicates the best choice for n .
3. P ARAMETERIZED LAYERS FOR MULTICHANNEL AUDIO
In the following, we expound how the proposed PHC layer can be employed to deal with multichannel audio signals.
For instance, ﬁrst-order Ambisonics (FOA) is composed of 4 microphone capsules, whose magnitude representations can be enclosed in a quaternion (Comminiello et al., 2019; Ricciardi Celsi et al., 2020).
However, the quaternion algebra may be restrictive if more than one microphone is employed for registration or whether the phase information has to be included too.
Indeed, quaternion neural networks badly ﬁt with multidimensional input with more thanchannels.
Conversely, the proposed method can be easily adapted to deal with these additional dimensions by handily setting the hyperparameter n and thus completely leveraging each information in the n -dimensional input.
C ONVOLUTION -B ASED N EURAL M ODELS WITH PHC L AYERS
In this section, we expound neural models endowed with PHC layers and we demonstrate how the proposed method can be employed in various spheres of application.
To this end, we devise deep
PHC models for image classiﬁcation and for sound event detection.
For each task, we empirically substantiate the effectiveness and ﬂexibility of the proposed PHC layer.
In order to be consistent with the literature, we perform each experiment with a real-valued baseline model and then we compare it with its quaternion counterpart and with the proposed PHC network.
Furthermore, we assess the
PHC malleability testing different values of the hyperparameter n , therefore deﬁning PHC models in multiple hypercomplex domains.
4. H YPERCOMPLEX P ARAMETERIZATION FOR I MAGE C LASSIFICATION
To begin with, we test the PHC layer on RGB images and we show how, exploiting the correlations among channels, the proposed method saves parameters while ensuring high performances.
4.1. P ARAMETERIZED H YPERCOMPLEX R ES N ETS
In recent literature, a copious set of high performance in image classiﬁcation is obtained with models having a residual structure.
ResNets (He et al., 2016) pile up manifold residual blocks composed of convolutional layers and identity mappings.
A generic PHC residual block is deﬁned by
[Equation]
whereby H j are the PHC weights of layer j = 1 , 2 in the block, and F is
[Equation]
4.1. P ARAMETERIZED H YPERCOMPLEX VGG S
Another family of popular methods for image classiﬁcation is based on the VGG networks (Simonyan & Zisserman, 2015) that stack several convolutional layers and a closing fully connected classiﬁer.
To completely deﬁne models in the desired hypercomplex domain, we propose to endow the network with PHC layers as convolution components and with Parameterized Hypercomplex
Multiplication (PHM) layers (Zhang et al., 2021) as linear classiﬁer.
The backbone of our PHC
VGG is then
[Equation]
We also test a hybrid model by employing standard real-valued fully connected (FC) layers instead of PHM ones.
However, we believe that it is out of the scope of this paper which aims at exploiting hypercomplex algebras properties to deﬁne highly malleable neural networks.
Nevertheless, for a further comparison, we report these additional experiments in the Appendix B.
4.1. E XPERIMENTAL S ETUP AND E VALUATION
We perform the image classiﬁcation task with ﬁve baseline models.
We consider ResNet18,
ResNet50 and ResNet152 from the ResNet family and VGG16 and VGG19 from the VGG one.
Each hyperparameter is set according to the original papers (He et al., 2016; Simonyan & Zisserman, 2015).
We investigate the performance in four different color images datasets at different scales. We employ SVHN, CIFAR10, CIFAR100, and ImageNet and any kind of data augmentation is applied to these datasets in order to guarantee a fair comparison.
We execute initial experiments with VGGs against Quaternion VGGs and two versions of PHC
VGGs with n equal to 2 and to 4 .
Average and standard deviation accuracy over three runs are reported for SVHN and CIFAR10 datasets in Table 1.
Both the VGG16 and VGG19 PHC-based versions clearly outperform real and quaternion counterparts while being built with more than a half the number of parameters of the baseline.
Additionally, PHC-based models extraordinarily reduce the number of training and inference time required with respect to the quaternion model which operates in a hypercomplex domain as well.
Furthermore, when scaling up the experiment with VGG19, [Table] [Table] the proposed methods are more efﬁcient at inference time with respect to the real-valued VGG19.
Therefore, PHC models can be easily adopted in applications with disk memory limitations, due to the reduction of parameters [Table] and for fast inference problems thanks to the efﬁciency at testing time.
More initial experiments with various networks on other datasets are reported in Appendix B.
The malleability of the proposed approach when dealing with color images is expressed in the opportunity to choose the domain in which operating.
Therefore, we test PHC networks in the complex H( n = 2 ), quaternion H 4 ( n = 4 ) or H 3 ( n = 3 ) domain, where in the latter we do not concatenate any zero padding and process the RGB channels of the image in their natural domain.
In order to implement this experiment, we change the number of ﬁlters in the baseline and in the corresponding compared networks to be divisible by each value of n .
More details about the implementations are provided in Appendix B and in the GitHub repository (link in Section 1). [Table] [Table]
Table 2 presents the the mean and standard deviation accuracy over three runs with different seeds for the ResNet-based models.
We perform extensive experiments and the PHC models with n =always outperform the quaternion counterpart gaining a higher accuracy and being more robust.
This underlines the effectiveness of the PHC architectural ﬂexibility over the predeﬁned and rigid structure of quaternion layers.
Furthermore, our method distinctly far exceeds the corresponding real-valued baselines across the experiments while saving from 50% to 75% parameters.
Focusing on the latter result, the PHC model with n = 3 results to be the most suitable choice in many cases, proving the validity of processing RGB images in their natural domain leveraging hypercomplex algebra.
However, performance with n = 3 and n = 4 are comparable, thus the choice of this hyperparameter may depend on the application or on the hardware employed.
On one hand, n =may sometimes lead to lower performances, nevertheless it allows saving disk memory, as shown in Table 3, thus it may be more appropriate for edge applications.
On the other hand, processing color images with n = 3 may bring higher accuracy even so it requires more parameters.
Therefore, such a ﬂexibility makes PHC models adaptable to a large range of applications.
Likewise, PHC networks with n = 2 gain considerable accuracy scores with respect to the real-valued corresponding models and, due to the larger number of parameters with respect to the PHC with n = 3 , sometimes outperform it too.
Finally, the PHC with n = 4 obtains the overall best accuracy in the largest experiment of this table.
Indeed, considering a ResNet152 backbone on CIFAR100, our method exceeds the real-valued baseline by more than 4% .
This is the empirical proof that, with a very small FLOPs increase, PHC models well scale to large real-world problems by notably reducing the overall number of parameters.
What is more, in Table 3, we report the memory required to store models checkpoints for inference.
Our method crucially reduces the amount of disk memory demand with respect to the heavier real-valued model.
Further, we perform the image classifcation task on the ImageNet dataset.
Since the most valuable choice for n when dealing with RGB images has been proved to be 3 , we test the real-valued
ResNet50 against the PHC counterpart with n = 3 .
We train the models for 300 k iterations with batch size 256 following the recipes in (Wightman et al., 2021).
Table 4 shows that the proposed method achieves comparable, and even slightly superior, performance than the real-valued baseline, while involving 66% fewer parameters.
This proves the robustness of the proposed PHC approach, which can be adopted and implemented in models at different scales. 4. H YPERCOMPLEX P ARAMETERIZATION FOR S OUND E VENT D ETECTION
Sound event detection (SED) is the task of recognizing the sounds classes and at what temporal instances these sounds are active in an audio signal (Mesaros et al., 2021).
We prove that the PHC layer is adaptable to n -dimensional input signals and, due to parameters reduction and hypercomplex algebra, is more performing in terms of efﬁciency and evaluation scores. [Table] [Table]
4.2. P ARAMETERIZED H YPERCOMPLEX SED NETS
Sound Event Detection networks (SEDnets) (Adavanne et al., 2019) are comprised of a core convolutional component which extracts features from the input spectrogram.
The information is then passed to a gated recurrent unit (GRU) module and to a stack of fully connected (FC) layers with a closing sigmoid σ which outputs the probability the sound is in the audio frame.
Formally, the PHC
SEDnet is described by
[Equation]
After the GRU model, We employ standard fully connected layers, that can be also implemented as PHM layers with n = 1 , since the so processed signal loses its multidimensional original structure.
4.2. E XPERIMENTAL S ETUP AND E VALUATION
For sound event detection models we consider the augmented version of the SELDnet (Adavanne et al., 2019; Comminiello et al., 2019) which was proposed as baseline for of the L3DAS21 Challenge Task 2 (Guizzo et al., 2021) and we perform our experiments with the corresponding released dataset 2 .
We consider as our baselines the SEDnet (without the localization part) and its quaternion counterpart.
The L3DAS21 Task 2 dataset contains 15 hours of MSMP B-format Ambisonics audio recordings, divided in 1-minute-long data points, each of which consists of a simulated 3D ofﬁce audio environment where up to 3 acoustic events may overlap.
We perform experiments with multiple conﬁgurations of this dataset.
We ﬁrst test the recordings from one microphone considering the magnitudes only ( 4 channels input), then we test the networks with the signals recorded by two microphones and magnitudes only ( 8 channels input).
In Appendix B we report the experiments and the results up to 16 channels input, in which we consider the phase information too.
We investigate PHC SEDnets in complex, quaternion and octonion domain with n = 2 , 4 , 8 and train each network for 1000 epochs with a batch size of 16 .
Other hyperparameters are set as suggested in the paper (Guizzo et al., 2021) and more details are provided in Appendix B.
To assess the performances of our models we take various metrics into account: the F score on the detection metric, Precision (P) and Recall (R) as suggested in the original paper.
For a more rigorous evaluation, we also compute the SED score and the Error Rate (ER) (Adavanne et al., 2019; Mesaros et al., 2021). [Table]
The proposed parameterized SEDnets distinctly outperform real and quaternion-valued baselines, as reported in Table 5 and Table 6.
Indeed, the PHC SEDnet with n = 2 gains the best results for each score and in both one and two microphone datasets, proving that the weights sharing due to the hypercomplex parameterization is able to capture more information regardless the lower number of parameters.
It is interesting to note that the PHC SEDnet n = 4 , which operates in the quaternion domain, achieves improved scores with respect to the Quaternion SEDnet that follows the rigid predeﬁned algebra rules.
Further, the malleability of PHC layers allows gaining comparable performance with respect to the quaternion baseline even so reducing convolutional parameters by 87% , just setting n = 8 .
In Appendix B, we show additional experimental results of PHC models able to save 94% of convolutional parameters while operating in the sedonion domain by involving n = 16 .
Furthermore, PHC SEDnets are more efﬁcient in terms of time required for training and inference, regardless the equal number of FLOPs.
Table 7 shows that each tested version of the proposed method is faster regards as the real SEDnet and the quaternion one, both at training and at inference time.
Time efﬁciency is crucial in audio applications where networks are usually trained for thousands of epochs and datasets are very large and require protracted computations.
R ELATED W ORKS
While a research ﬁeld deals with scaling up networks (Real et al., 2019), a consistent area in literature aims instead at making these models more efﬁcient and accessible (Tan & Le, 2019; Sandler et al., 2018).
Among the latter, quaternion neural networks (QNNs) have proved to reduce the amount of parameters to 1 / 4 (Parcollet et al., 2019a; 2017).
QNNs ensure high performance thanks to the sharing of weights due to the Hamilton product that preserves correlations among channels in multidimensional inputs (Grassucci et al., 2021c; Tay et al., 2019; Grassucci et al., 2021b).
However, despite the lower number of parameters, QNNs are often slightly slow with respect to real-valued baselines (Hoffmann et al., 2020).
Recently, several attempts have been made to compress neural networks relying on Kronecker product decomposition (Huang et al., 2020; Tang et al., 2021).
These methods gain considerable results in terms of model efﬁciency (Wang et al., 2021).
Lately, a parameterization of hypercomplex multiplications have been proposed to generalize hypercomplex fully connected layers by sum of Kronecker products (Zhang et al., 2021).
The latter method obtains high performance in various natural language processing tasks by also reducing the number of overall parameters.
Lately, other works extended this approach to graph neural networks (Le et al., 2021) and transfer learning (Mahabadi et al., 2021), proving the effectiveness of Kronecker product decomposition for hypercomplex operations.
C ONCLUSION
In this paper, we introduce a parameterized hypercomplex convolutional (PHC) layer which grasps the convolution rule directly from data and can operate in any domain from 1 D to n D, regardless the algebra regulations are preset.
The proposed approach reduces the convolution parameters to 1 /n with respect to real-valued counterparts and allows capturing internal latent relations thanks to parameter sharing among input dimensions.
We show our method is ﬂexible to operate in different ﬁelds of application by performing experiments with images and audio signals.
We also prove the malleability and the robustness of our approach to learn convolution rules in any domain by setting different values for the hyperparameter n from 2 to 16 .
R EPRODUCIBILITY S TATEMENT
One of the objective of proposing methods that allow a reduction of parameters is to make large models more accessible and reproducible, even with a lower budget.
We strongly believe reproducibility should be a key ingredient for recent deep learning papers.
Therefore, we provide detailed explanations in the Appendix A to better understand the proposed method.
Moreover, each experiment reported in the main corpus and in the appendix of this paper is completely reproducible.
We provide the full code at https://anonymous.4open.science/r/HyperNets-CBBB , including models, train conﬁgurations, L3DAS21 preprocessing and information for dataset download and thorough instructions and details are provided in the Appendix B. Notebooks tutorials with toy examples are uploaded too, hoping these may help further research on this topic. CO2 E MISSION R ELATED TO E XPERIMENTS Experiments were conducted using a private infrastructure, which has a carbon efﬁciency of 0. kgCO 2 eq/kWh.
A cumulative of 2000 hours of computation was performed on hardware of type
Tesla V100-SXM2-32GB (TDP of 300W).
Total emissions are estimated to be 267 kgCO 2 eq of which 0 percents were directly offset. Estimations were conducted using the MachineLearning Impact calculator presented in Lacoste et al. More in detail, considering an experiment for the sound event detection (SED) task, according to
Table 7, the real-valued baseline requires approximately 20 hours for training and validation, with a corresponding carbon emissions of 2 . 71 kgCO 2 eq.
Conversely, the proposed PHC model takes approximately 17 hours with a reduction of carbon emissions of 16% , being 2 .
28 kgCO 2 eq.
In conclusion, we believe that the improved efﬁciency of our method with respect to standard models may be a little step towards reducing carbon emissions.
A T HEORETICAL A DD -O NS A. Q UATERNION A LGEBRA
A quaternion q ∈ H is an hypercomplex number of rank 4.
It is deﬁned as the composition of four real coefﬁcients with three imaginary units as follows:
[Equation]
(9) being q 0 , q 1 q 2 , q 3 ∈ R and ˆ ı, ˆ , ˆ κ orthonormal basis in R 3 .
A quaternion with scalar part q 0 equal to 0 is a pure quaternion .
Considering two quaternion q and p , the Hamilton product in Eq.2 can be rewritten in concise form as:
[Equation]
where the bold notation is intended for vectors and the plain notation for scalars.
From the above deﬁnition, it is easy to derive the Hamilton product between two pure quaternions.
It takes the form:
[Equation]
in which − q · p is the scalar part of the new quaternion and q × p its vector part.
Therefore, the vector product between two pure quaternions gives rise to a full quaternion.
As complex numbers, also quaternions have their own complex conjugates that are deﬁned as
[Equation]
The norm of a quaternion is the Euclidean norm in R 4 of its real coefﬁcients, that is
[Equation]
while the inverse is
[Equation]
A. D EMYSTIFYING PHC L AYERS
We provide a formal explanation of the PHC layer to better understand the Kronecker product and how it organizes convolution ﬁlters to reduce the overall number of parameters to 1 /n .
In Eq.10, we show how the PHC layer generalizes from 1 D to n D domains.
When subsuming real-valued convolutions in the ﬁrst line of Eq.10, the Kronecker product is performed between a scalar A and the ﬁlter matrix F , whose dimension is the same as the ﬁnal weight matrix H , that is s × d × k × k .
Considering the complex case with n = 2 in the second line of Eq.10, the algebra is deﬁned in
A 1 and A 2 while the ﬁlters are contained in F 1 and F 2 , each of dimension 1 / 4 the ﬁnal matrix
H .
Therefore, while the size of the weight matrix H remains unchanged, the parameter size is approximately 1 / 2 the real one.
In the last line of Eq.10, we can see the generalization of this process, in which the size of matrices F i , i = 1 , ..., n is reduced proportionally to n .
It is worth noting that, while the parameter size is reduced with growing values of n , the dimension of H remains the same.
[Equation]

[Equation]

H  
[Equation]

[Equation]

[Equation]
B E XPERIMENTAL A DD -O NS B. T OY EXAMPLES : LEARNING THE MATRIX A
We test the receptive ability of the PHC layer in two toy problems building an artiﬁcial dataset.
The ﬁrst task is learning the proper matrix A to build a quaternion convolutional layer which properly follows the Hamilton rule in Eq.2.
We build the dataset by performing a convolution with a matrix of ﬁlters W ∈ H , that are arranged following the regulation in Eq.2, and a quaternion x ∈ H in input.
The target is still a quaternion, named y ∈ H . As shown in Fig. 2 (right), the PHC loss converges very fast, meaning that the layer properly learns the matrix A and the Hamilton convolution. The second toy example is a modiﬁcation of the previous dataset target..
Here, we want to learn the matrix A which describes the convolution among two pure quaternions.
Pure quaternions may be, as an example, an input RGB image and the weights of a hypercomplex convolutional layer.
Figure 2 (left) displays the convergence of the PHC layer loss during training, proving that the proposed method is able of subsuming hypercomplex convolutional rules when dealing with pure quaternions too.. [Figure] L o ss    5 Epochs 0 .0 .0 .0 .0 .0 .
L o ss
[Table]
E XPERIMENTAL A DD -O NS ON I MAGE C LASSIFICATION B.2. I MAGE C LASSIFICATION I MPLEMENTATION D ETAILS
To guarantee the complete reproducibility of our experiments [Table] here we provide more implementation details for the image classiﬁcation task.
The modiﬁed versions of the ResNets are built with an initial convolutional layer with 60 ﬁlters.
Then, the subsequent blocks have 60 , 120 , 240 , 516 ﬁlters.
The number of layers in the blocks depends on the ResNet chosen, whether 18, 50 or 152.
Instead, VGG19 convolution component comprise two 24 , two 72 , four [Table] and eight 648 ﬁlter layers, with batch normalization.
The classiﬁer is composed of three fully connected layers of 648 , 516 and with the number of classes neurons each.
The rest of the hyperparameters are set as suggested in the original papers.
The batch size is ﬁxed to 128 and training is performed via SGD optimizer with momentum equal to 0 .
9 , weight decay 5 e −and a cosine annealing scheduler.
For ResNets, the initial learning rate is set to 0 .
For VGG is equal to 0 .
Models on CIFAR10 and CIFAR100 are trained for 200 epochs whereas on SVHN networks run for 50 epochs.
For the ImageNet dataset, we follow the recipes in (Wightman et al., 2021), so we resize the images for training at 160 × 160 while keeping the standard size of 224 × 224 for validation and test.
We employ a step learning rate decay every 30 epochs with γ = 0 .
the SGD optimizer and an initial learning rate of 0 . 1 with weight decay 0 . 0001 .
The training is performed for 300 k iterations with a batch size of 256 employing 4 GPUs.
B.2. A DDITIONAL E XPERIMENTS ON I MAGE C LASSIFICATION
In the following, we provide additional preliminary experiments with small models equipped with the proposed PHC layer.
Table 8 contains VGG16s with real-valued ﬁnal classiﬁer.
Table 9 reports experiments with ResNet20 where we test also n = 1 to replicate the real-valued model, outperforming it.
Experiments with VGG11 with modiﬁed number of ﬁlters in order to be divided by each value of n is also reported in the same table.
Finally, in Table 10 additional experiment on SVHN and CIFAR10 with ResNet56 and ResNet110, the latter with modiﬁed number of ﬁlters.
PHC-based
[Table] [Table] [Table]
models gain good performance in each test we conduct while reducing the amount of free parameters. Finally, in order to further remove the hypothesis that smaller number of neural parameters leads to higher generalization capabilities
[Table]
we perform experiments with real-valued baselines with a number of parameters reduced by 75% . Table 11 shows that reducing the number of ﬁlters downgrades the performance and thus it is not sufﬁcient to improve the generalization capabilities of a model
[Table]
B. E XPERIMENTAL A DD -O NS ON S OUND E VENT D ETECTION B.3. L3DAS21 D ATASET D ETAILS
The L3DAS21 Task 2 dataset is a collection of 900 1-minute-long data-points sampled at a rate of 32 kHz.
The 14 sounds classes have been selected from the FSD50K dataset and are representative for an ofﬁce sounds: computer keyboard, drawer open/close, cupboard open/close, ﬁnger snapping, keys jangling, knock, laughter, scissors, telephone, writing, chink and clink, printer, female speech, male speech.
In this dataset, the volume difference between the sounds is in the range 0 and 20 dB full scale (dBFS).
Considering the array of two microphones 1 , 2 , the channels order is [W1, Z1,
Y1, X1, W2, Z2, Y2, X2], where WXYZ are the B-format ambisonics channels if the phase (p) information is not considered.
Whether we want to include also this information, the order will be [W1, Z1, Y1, X1, W1p, Z1p, Y1p, X1p, W2, Z2, Y2, X2, W2p, Z2p, Y2p, X2p] up to 16 channels.
In Fig.3, we show the 8 -channel input when considering one microphone and the phase information.
Magnitudes and phases are normalized to be centered in 0 with standard deviation 1 . [Figure]
B.3. SED I MPLEMENTATION D ETAILS
To guarantee the complete reproducibility of our experiments, here we provide more implementation details for the sound event detection task.
The extracted features by the preprocessing are fed to the four-layer convolutional stack with 64 , 128 , 256 , 512 ﬁlters, with batch normalization, ReLU activation, max pooling and dropout (probability 0 . 3 ), with pooling sizes (8 , 2) , (8 , 2) , (2 , 2) , (1 , 1) .
The bidirectional GRU module has three layers, each with an hidden size of 256 .
The tail has is a four-layer fully connected classiﬁer with 1024 ﬁlters alternated by ReLUs and with a ﬁnal dropout and a sigmoid activation function.
The initial learning rate is set to 0 .
Following, we deﬁne the equations of the evaluation metrics considered in Subsection 4.2.
To be consistent with pre-existing literature, we deﬁne True Positives as TP, False Positives as FP and False Negatives as FN.
These are computed according to the detection metric (Guizzo et al., 2021).
Moreover, in order to compute the Error Rate (ER), we consider: S = min( FN , FP ) ,
D = max(0 , FN − FP ) and I = max(0 , FP − FN ) , as in (Adavanne et al., 2019).
[Equation]
whereby N is the total number of active sound event classes in the reference.
[Equation]
For ER and SED score , the lower scores, the better the performances, while for the F score higher values stand for better accuracy.
B.3. A DDITIONAL E XPERIMENTS ON S OUND E VENT D ETECTION
In this section [Table] we provide additional experiments for the sound event detection task.
We conduct a test considering two microphones and the phase information, so to have an input with 16 channels.
For this purposes, we consider as baseline the quaternion model and PHC-based networks with n = 4 , 8 , 16 so to test higher order domains.
Results are shown in Table 12.
[Table]
C
F UTURE
P ERSPECTIVES
In this work, we demonstrate that the proposed parameterized hypercomplex convolutional (PHC) layer is able to consistently reduce the free parameters of neural models while ensuring very high performances thanks to the hypercomplex convolution properties.
Therefore, we believe that these ﬁndings pave the way for novel efﬁcient neural networks.
As an example, a thorough study on pruning PHC-based models may further reduce the number of parameters and lead to extremely lightweight networks.