Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper considers a generalization of convolutional neural networks (CNNs) to manifold-valued data such as Symmetric Positive Definite (SPD). This paper proposes a neural architecture search problem of SPD manifold networks. A SPD cell representation and corresponding candidate operation search space is introduced. They demonstrate on drone, action and emotion recognition datasets that their method is performed well compared to SPD approaches.

- It is not clear how the computational graph preserves the geometric structure of SPD manifold.
- It is unclear how to obtain the weights of Frechet Mean w_i of wFM (eq. 3) by backpropagation. 
- Relatively thorough experimental valuations: using 3 datasets comparing with sufficient number of prior
approaches. However, the comparisons of the experiments results are limited to the SPD methods.  It is hard to understand how the generalization of convolutional neural networks (CNNs) to Symmetric Positive Definite (SPD) presented in this paper helps to improve CNN.

review_title: Interesting idea, however, more explanations and comparisons needed. 

final_score: rating: 4: Ok but not good enough - rejection
review_content: 1. "In recent years, plenty of research work has been published in the area of NAS." needs citations.
2. What is "best of both these fields (NAS, SPDNets) " in page 2?
3. The definition of SPD matrix is wrong, this definition is for positive-definite matrix, not for SPD.
4. In Eq. (1), log should be matrix logarithm, and need to be mentioned.
5. In page 3, "There are other efficient methods to compute distance between two points on the SPD manifold" needs to be backed by references. 
6. " Other property of the Riemannian manifoldof our interest is local diffeomorphism of geodesics which is a one-to-one mapping from the pointon the tangent space of the manifold to the manifold", this is not true, geodesic is a function of two points (start and end) and time point (say between 0 and 1), so it can not be a one-one mapping between tngent space to manifold. Authors here meant to say exponential map as defined in Eq. (2). 
7. X in Eq. (2) is not the "reference point", it should be the base point for exponential map (see Chavel's book).
8. The inverse of exp map is not defined everywhere, as exp is LOCAL diffeo., please clarify.
9. Before Eq. (3), sometime->sometimes
10. "This definition is trivially extended tocompute the weighted Riemannian Barycenter" not true, the authors should mention existence and uniqueness of wFM, for example, even if FM exists, wFM may not for some choice of w.
11. What is "valid SPD" in Bimap layer, contrary to "invalid SPD"!
12. what is P in Eq. (4)? I think it is parallel transport but needs to be defined.
13. \epsilon in ReEig layer should be >0.
14. In logEig, essentially you are mapping SPD to space of symmetric matrices (which is "flat"), then why not use Inverse Exp. map? what is the necessity to define logEig map additionally?
15. Same goes for expEig map, what is need given that one can use Exp. map!
16. In weighted pooling layer, why karcher flow is "simple" and recursive method is not? Please clarify.
17. " such that the mixture of all operations still reside on SPD manifolds", here the term mixture is ambiguous!
18. Given that the operations are already defined, definition of SPD cell is NOT novel!
19. The additional search space operations are all trivial extensions of basic operations, which are not the contribution, hence I think the authors need to tone down "Most of these operations are not fully explored for SPD networks" 
20. In 3.2, what the authors meant by "optimize the over parameterized supernet"?
21. In Algorithm 1, do the authors mean Riemannian gradient descent (Absil et al.)?
22. What the authors meant by "Note that the gradient based optimizationforwmust follow the geometry of SPD manifold to update the structured connection weight, and itscorresponding SPD matrix data"? Do the authors mean they need to project the Euclidean gradient found in Eq. (8) to get tangent vectors?
23. The experimental setup is weak, e.g., SPDNet and ManifoldNet need to be compared with same model complexity, otherwise the comparison is not fair! 
review_title: Incremental work using already defined tools

final_score: rating: 4: Ok but not good enough - rejection
review_content: The work focus on finding better SPD Manifold Networks from the view of neural architecture search. They model the network parameterized by a set of architecture parameter and take the problem as a bi-level optimization problem, which is similar with DARTS way. I think the most contribution of this paper is the attempt to apply NAS on a new domain.

1.  The authors introduce a lot of concepts about SPD Network. But they are not meaningful in this paper since the authors just consider operations as the node. From the graph view, SPD Network can be seen as a neural architecture directly even though the operation type is different.

2. My main concern is novelty problem.  The authors also treat SPD Network as a graph in Figure 1 such that the DARTS method can be applied directly. As for Algorithm 1, they also use two types parameters: architecture weights $\alpha$  and operation weight $w$. Then bi-level optimization problem is solved using the same method with DARTS.

 
review_title: The authors apply neural architecture search on SPD Manifold Networks, which is the first attempt in this field.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduces a neural architecture search method for SPD inputs, which extends DARTs from Euclidean space to SPD manifold by defining several SPD cells. Each SPD cell consists of some operations in search space of SPDNets and DARTs. The experiments are conducted on three datasets.

Strengths:
+: The idea of neural architecture search for SPD inputs seems interesting.
+: The proposed SPDNetNAS shows superiority to hand-crafted SPDNets/ManifoldNet and Euclidean NAS (i.e., DARTS and FairDARTS).
+: The paper is clear written.

Weaknesses:
-: The authors would better carefully reconsider motivation and main contribution of this paper. The current work looks like combination of SPDNets and DARTs (with some minor modifications). Then, the first attempt for NAS problem of SPD manifold networks is not an appropriate motivation for such a combination. The authors should clarify why it is necessary to search neural architecture for SPD inputs. In another words, what are drawbacks of traditional SPDNets? 

For Euclidean NAS, the searched architectures aim to have lower model complexity than hand-crafted architectures, while having comparable or higher accuracies. The proposed SPDNetNAS achieves higher accuracies than hand-crafted counterparts, but has 10x times of parameters number (as shown in Table 2). Therefore, it is hard to say that performance gains come from neural architecture search or parameter increasing. Meanwhile, it is not fair for comparison of SPDNetNAS with Euclidean NAS in terms of parameters number or comparison of SPDNetNAS with SPDNets only in terms of performance. 

-: The authors mainly compare with hand-crafted SPDNet and Euclidean NAS. [r1] presents a recurrent model for SPD inputs, which can handle sequential (or temporal) data, e.g., action and emotion recognition. Therefore, what are merits of the proposed SPDNetNAS over [r1] for handling sequential (or temporal) data, except architecture design vs architecture search?
[r1] A Statistical Recurrent Model on the Manifold of Symmetric Positive Definite Matrices. NIPS, 2018.

-: I wonder that why a section of FURTHER STUDY is located in appendix? Could it be integrated into section of Conclusion?

-: It is well known that two-level optimization algorithm for NAS will lead optimization gap, e.g., overfitting or model collapsing. What are strategies used to solve this issue, e.g., early stopping? If source code is not released or no more details are given, the results will be very hard to reproduce.  

review_title: Review for Neural Architecture Search of SPD Manifold Networks

Meta-review: The paper tries to find better semi-positive definite (SPD) manifold networks using neural architecture search. However, as pointed out by the reviewers, the paper has a few weaknesses: (a) it lacks in novelty, (b) it lacks in experiments that are mentioned in the SOTA papers, (c) the experiments should be performed with the same model complexity for fairness.


Decision: Reject