Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a lightweight convolutional neural network based on the idea of quaternion neural networks. The paper extended the hypercomplex linear layers to convolution neural networks to apply to many multi-dimensional applications. The authors defined the parameterization of hypercomplex convolutional layers so that convolution neural networks can utilize the quaternion algebra to improve the parameter efficiency.
main_review: Strengths:
The paper generalize the quaternion neural networks to CNNs and it shows better accuracy v.s. #parameters trade-off than Quaternion counterparts.

Weakness:
It's not clear how much real speedup/model size reduction can be achieved. The algorithms firstly use Kronecker product to generate the convolution kernel then use it for the normal conv2d. In my understanding, only the number of free parameters is reduced, instead of the practical resource consumption for inference / training.
The comparison is not enough to show the effectiveness of the proposed method. How is the proposed method compared to low-rank based conv kernel decomposition? How is it compared to the depth-wise & point-wise decomposition like in Mobilenets? Without the comparison to other commonly used DNN compression approaches, it's hard to understand the significance of the proposed method.
summary_of_the_review: See Main Review.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Traditionally, networks use real-valued weights. However recent work has shown that in certain domains, weights that obey different algebraic rules can lead to various performance enhancements. The authors introduce a hyper complex convolution block and show that it leads to performance gains and parameter reductions over a variety of baseline models on tasks in various domains. 
main_review: Strengths:
* Results are shown on multiple domains and strong performance is achieved.
* The paper is clearly written.
* The authors provide code making the results easily reproducible by future groups. 

Weaknesses:
* I think the experimental results could be fleshed out. The networks and datasets are fairly small. It would be interesting to see the results pushed to a greater scale. It would be nice to see, for example, some results with a ResNet50 on ImageNet. 
* It isn't immediately clear to me that parameter count is what needs to be cared about the most. Especially with quaternion, there is a substantial FLOP increase. I think comparisons on FLOPs would be most interesting. If the argument is model size, quaternions will still induce an increase activation memory. 


Comments/Questions:
* From my understanding of Deep Complex Networks from Trabelsi, and Deep Quaternion Networks from Gaudet, they argue that a whitening procedure should be used instead of BatchNorm. In AlgebraNets (https://arxiv.org/abs/2006.07360) the authors use BatchNorm, arguing the FLOP expense of whitening is not worth it. Is this something that you experimented with?  Similarly, did you consider other algebras, such as matrix rings?
* Deep Complex Networks, Deep Quaternion Networks, and AlgebraNets all investigate various activation functions beyond ReLU. Is this something that you considered? 
* Other forms of model compression methods such as pruning are quite popular. Have you tried pruning any of these networks for further parameter reductions? 

summary_of_the_review: The authors propose an interesting idea, however I think that the presented comparisons miss a few key comparisons. Specifically, I would have liked to see a measure of FLOPs required to reach given accuracies. Additionally, while the results shown span multiple domains I would have liked to see more ambitious experiments.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes Parametrized Hypercomplex convolutional Layers to replace convolutional layers to attain similar performance using fewer parameters. The PHC layer can subsume hypercomplex convolution rules. The main advantage of the PHC Layer is that they enable choosing an arbitrary $n$ to reduce the number of parameters to $1/n$, whereas this was limited to $4$, $8$, and $16$ with quaternions. Hyperparameter $n$ also drives the number of convolutional filters. The proposed network can represent a real-valued convolutional layer by setting $n = 1$.

The PHC layer differs from the convolutional layer in the way that the weight matrix is obtained. $H$ acts as a weight for input $x$. The $A$ and $F$ matrices which are trainable and used to obtain $H$. Kronecker products of every matrix in the tensors $A$ and $F$ are taken and then these are summed to get $H$.  

Authors have proposed PHC versions of some common neural networks such as ResNet and VGG and have compared the results with real-valued models, Quaternion models, and PHC models (for $n=2$ and $n=4$). 
main_review: Strength:

1. This method significantly improves the time and computational power required for the training of large neural networks while achieving good performance. 
2. The method can be operated in domains from $1D$ to $nD$, where $n$ can be tuned to attain appropriate results on given data. 
3. Neural models redefined with this approach require $1/n$ of free parameters compared to their real-valued counterparts. 

Weakness: 

1. Although the research has many real-world applications, it is lacking novelty. The paper is an extension of (Zhang et al., 2021) for the convolution case. 
2. Many details required to reimplement these models to achieve similar results are missing in the paper. 
3. The choice of $n$ should also be analyzed using FLOPs. 
summary_of_the_review: The paper lacks novelty and is an extension of (Zhang et al., 2021) for the convolution case. This is a major concern for me.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work extends the recently proposed hypercomplex parameterization of neural layers to convolutional layers. Then it describe common deep CNN models (ResNet, VGG, SeDnet) based on this novel paradigm in two different tasks (Image Classification and Sound Event Detection). The introduced method offer better performance with a reduced number of neural parameters as well as faster training and inference than equivalent hypercomplex layers. 
main_review: Summarising this paper is quite straightforward as the contribution is quite simple, yet effective. The concept of hypercomplex parameterization is extended to convolutional layers and common CNN models are also turned into hypercomplex parameterized models. The new model performs better than all baselines with less neural parameters and slightly (always ?) quicker training and inference times compared to hypercomplex models. In practice, the contribution is minimal, but very valuable as the code is given. If the code is open to the community afterwards, it is a win-win situation. Simple, efficient, and works well. 

Seing this concept of hypercomplex parameterization extended is very nice considering that is has changed the efficiency capabilities of hypercomplex networks. Indeed, despite promising results and memory footprint, hypercomplex networks suffered from a lack of generalisation AND a crazy training / inference time compared to real-valued neural networks. This is all almost solved with the hypercomplex parameterization introduced in a previous paper and extended to CNN in this work. 

My only concern, that prevents me from assigning a high rank on the current submission relates to the quality of the writing. I will only provide two examples, but the paper should really be proofread carefully and the English must be improved to meet ICLR requirements. If this is done properly, I don't see why this paper shouldn't be accepted. 
"But the layers saves the 75% of free parameters" / "that consider deep layers [...], thus [...]"

An other smaller comment concerns Section 3.1. It is really weird to have results presented in formal Section. I would encourage the authors to move this to the experimental part instead, or rewrite the two last paragraphs as they currently sound as "look, we tried, it works, but we don't show you the results" even if you give then in Appendix. It is not appropriate to present it in this way and at this stage of the paper. 

**Strengths:**
- Simple idea, yet very effective.
- Definitely impactful for hypercomplex neural network (and thus the concerned community).
- If the code is given, it will be very helpful to the community.
- Good results as expected (since they were already observed with non-convolutional layers). 

**Weaknesses:**
- Writing must be polished.
- Add time and inference for ALL tables as this is a major point in the discussion between hypercomplex networks and parameterised ones !
- Add real-valued models with the exact same number of neural parameters than PHC ones to the tables. This will remove the hypothesis that smaller number of neural parameters leads to higher generalisation capabilities.  

summary_of_the_review: Simple and impactful idea. Results are convincing. It will be an interesting paper to the hypercomplex networks community. Nevertheless, writing must be improved to get accepted (not the outline of the paper but the content).
review_title: null

Meta-review: In general, the reviewers appreciated the elegant concept behind the paper and the good results. However, they also raised considerable reservations about the significance of a method that decreases the parameter count but not necessarily computational efficiency (FLOPS) or memory. While the additional analysis that the authors provided definitely helps to understand the limitations of the method, the reviewers were in the end quite divided on the significance of the results. In addition, all reviewers agreed that the writing was in somewhat rough shape and needed improvement.

In summary, this is definitely a borderline paper, but given the current reviewer assessment, I would recommend that it is not quite ready for publication.
Decision: Reject