Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new neural network paradigm, which aims at gradually disentangling features in the forward propagation. The whole network is constructed with multiple copies of sub-networks. The total information can be maintained rather than compressed during propagation. The proposed method is also evaluated on several typical computer vision tasks and shows competitive performance.
strength_and_weaknesses: **Strength**  
* It is important to explore strong and disentangled representations by designing new neural architectures. The proposed reversible column manner is reasonable to maintain both high- and low-level information.
* The proposed method shows promising experimental performance.

**Weaknesses**
* It is expected to explain the intrinsic difference between the proposed method and previous ones, including but not limited to HRNet and NAS (neural architecture searched) networks. Especially, networks [1,2] searched in the cell-based/-like search space have many in common with the proposed one. A more comprehensive analysis is suggested, not only the provided demonstration from the motivation perspective.
* The shown experimental performance is competitive but the promotion over previous ones is not so evident. I admit the number is almost saturated, but any other advantage this method could bring needs to be demonstrated.
* Real throughput/latency needs to be measured to more accurately validate the model budget, not just FLOPs or #Params. The introduced connections seem to introduce larger latency on real hardware which is not so related to FLOPs numbers.


[1] Liu C, Chen L C, Schroff F, et al. Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 82-92.   
[2] Xie S, Kirillov A, Girshick R, et al. Exploring randomly wired neural networks for image recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 1284-1293.
clarity,_quality,_novelty_and_reproducibility: * **Clarity**: most contents are clear and easy to follow.
* **Quality**: The proposed method is evaluated and studied on substantial benchmarks and settings.
* **Novelty**: The design principles have some in common with previous methods, but some details are new.
* **Reproducibility**: Most implementation details are provided but the code is not available.
summary_of_the_review: The overall idea is good while the main concerns lie in the demonstration of advantages or differences with previous methods and speed analysis.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is mainly about convolutional neural networks for image classification. The authors propose a new paradigm to fuse features from different depths. The are some parallel branches of convolutional layers. Features from each branch are fused through a complex topology as stated in Figure 2(c). Some deep supervised loss are adopted for some branches.
strength_and_weaknesses: The experiments are conducted among many areas, including image classification, object detection, semantic image segmentation and machine translation. Through these experiments, I feel that there are some kinds of competitiveness in this method, but not many. 
For example, in Table 1, it shows this method behaves consistently worse than EfficientNet, and SwinV2 is missing. In Table 9, it shows this method behaves consistently worse than SwinV2. 
clarity,_quality,_novelty_and_reproducibility: The writing is good and clear. There are many details about parameter settings.
summary_of_the_review: Abound experiment are designed, on many topics, and The results of ablation study of feature fuse and deep supervision support the conclusion.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new neural network architecture design based on GLOM (Hinton, 2021) called the Reversible Column Networks (RevCols). RevCols has multiple sub-networks (called columns) of the same structure, between which multi-level reversible units are added. The paper extends RevNets (Gomez et al., 2017) in a recursive way, then splits features into groups and reorganizes groups as columns. The paper leverages two auxiliary heads (Wang et al. 2021) to help disentangle the features. Experiments show that RevCols achieves competitive performances on several vision tasks including image classification, object detection, and semantic segmentation tasks.
strength_and_weaknesses: Strong points.
1. The paper introduces a new CNN-based design with reversible transformations and GLOM architecture, which can inspire future research.
2. The experiments show the competitive performance of the proposed method on several vision tasks.
3. The paper provides detailed information for reproduction.

Weak points.
1. The paper sometimes draws conclusions without enough contextual information/justifications or leaves analysis far away from the conclusions, which makes it difficult for readers to understand, especially in the macro and micro design sections.

2. The paper does not differentiate itself clearly from existing methods, especially from (Hinton, 2021; Wang et al. 2021) and (Chang et al. 2018; Jacobsen et al. 2018; Mangalam et al. 2022).

Minor issues.
Line 9 Section 1, "Researches" -> "Researchers"
The proposed method is sometimes called "RevCols" but sometimes called "RevCol". 

Questions.
1. The difference between the proposed method and GLOM is not clear. The "Relation to previous works" only mentions one difference but does not explain the reason and advantages/disadvantages.
2. Sec. 2.2.1 simplifies the multi-column reversible of Sec. 2.1. Would this still satisfy the reversible criterion?
3. Will code be released?
clarity,_quality,_novelty_and_reproducibility: The paper is generally written well, but it would be better if figures/tables could be arranged to where they are referred to for the first time. In the network design part, the paper sometimes simply describes the design choices without explanations of motivations and justifications or puts them far away. The paper describes its key designs clearly for reproduction. Personally, I feel that it is interesting to combine GLOM architecture with reversible transformations, and this is the key contribution. 
summary_of_the_review: I am on the positive side of this paper. I think it is promising to combine GLOM architecture with reversible transformations. However, I am confused but interested to know the motivation behind it and the detailed analysis between these two works. I am happy to increase my rating if the authors can address this ambiguity.
review_title: null

Decision: Accept: poster