Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Since the existing neural architecture search (NAS) technologies cannot explore the stage-level detection ability required by face detection tasks, this paper proposed a distribution-dependent stage-aware ranking score to evaluate the expressivity of the backbone. Based on the proxy score, they conduct the NAS on SCRFD family detectors and achieve better performance on both accuracy and inference speed. SOTA results are reported in the paper.
strength_and_weaknesses: Strengths:
1, The main theories are well explained, and the math functions are correctly organized. 
2, The proposed DDSAR-score is novel and makes the NAS FD-friendly.

However, I have some concerns:
1, In section 5.3, it is unable to find Table 5.1 in the main manuscript.   
2, There is no ablation study about the proposed stage-aware expressivity score and stage-aware filter sensitivity score. It is better to have some experiments to illustrate their contributions.
3, In Table2, the performance of SCRFD-DDSAR-34GF decreases on the easy and hard subset. It might be better to add some comments on this.

clarity,_quality,_novelty_and_reproducibility: The work is novel and well-illustrated. However, more experiments might be included to support their results.
summary_of_the_review: The proposed method makes the NAS work better in face detection, which is relatively novel. However, insufficient experiments affect the clarity of the manuscript.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors introduce a distribution-dependent stage-aware ranking score (DDSAR-Score) for NAS to search FD-friendly backbone architectures.
The proposed DDSAR-score aims at estimating the stage-level expressivity and identify the importance of each stage which can be used for NAS.
Several modifications are proposed such as: stage-irrelevant term removal, re-combination trick and filter sensitivity score. 
DDSAR score shows its potential in improving the Face detection on WiderFace benchmark.

strength_and_weaknesses: STRENGTH
- The paper is well-motivated.
- The idea of measuring stage-wise expressivity for NAS is interesting.
- Experimental results show improvements on face detection benchmark.

WEAKNESS
1. It is unclear how the authors come up with the configuration for $N^{l’}_{ci}$.
An ablation study for selecting these configurations is recommended.

2. The ablation study for the contributions of each component, i.e. $S^{sae}\_{N\_{ci}}$, $S^{fs}\_{N\_{ci}}$ is missing.
The authors should discuss the contributions of these components to show the advantages of each proposed term.

3. In Section 4.2, Resnet50 backbone is adopted as a reference for ratios ($\lambda_2,\lambda_3,\lambda_4, \lambda_5$).
How these numbers changes if different reference backbones are adopted? How do these values affect the obtained FD backbone?

4. The performance improvements of DDSAR are marginal.
Why does DDSAR improve on hard set but not improve on Easy and Medium sets?

5. The obtained architectures should be presented and discussed.

clarity,_quality,_novelty_and_reproducibility: In general, the paper is well-motivated. The idea of measuring stage-wise expressivity for NAS is interesting.  Experimental results show improvements on face detection benchmark.

summary_of_the_review: In general, the paper is well-motivated. Experiments also show its improvements on FD benchmark.
However, There are some ablation studies on the contributions of the proposed terms are missing as mentioned in Weakness Section.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studied how to search for a lightweight backbone network for face detection using NAS. The key is how to develop stage-aware accuracy predictors of face detection performance for different network architectures. Thus, the paper derived a new stage-aware ranking score from the expressivity of ReLU CNNs, i.e., the lower bound of the maximal number of linear regions, from previous works (e.g., Xiong et al. 2020). Specifically, the paper proposed 1) a stage-aware expressivity score (Eq.4) by removing stage-irrelevant items and adding a fully-connected layer to combine multiple layers; and 2) calibrating the weights (Eq.8) to combine the ranking scores across multiple stages by a dataset with groundtruths and a Resnet50 backbone. The proposed ranking scores, i.e., DDSAR-Score, is integrated in an evolutionary architecture search. This method has been evaluated on the Wider Face dataset and compared favorably with recent NAS methods for face detection.
strength_and_weaknesses: Face detection has been extensively studied in decades with many commercial face detectors running on embedded systems. Search for a lightweight backbone network for face detection using NAS is an interesting and well-motivated topic. The proposed DDSAR-Score reasonably integrates the expressivity of multi-stage ReLU CNNs, which is somewhat novel and empirically works well in an evolutionary architecture search. 

The proposed stage-aware ranking score approximates the expressivity by the lower bound of the maximal number of linear regions, which is fine. The major concern is the gap between the expressivity of ReLU CNN and the face detection performance, e.g. detection rate and false positive rate. Please discuss more about the relation between the network expressivity and detection performance.

Another concern is the generality of the proposed DDSAR-Score, which is derived from the expressivity of ReLU CNNs in previous works. How to extend DDSAR-Score to other network architectures?

clarity,_quality,_novelty_and_reproducibility: The major contribution is to derive the ranking score for \emph{multi-stage} networks from the expressivity of ReLU CNNs, which is modest novel. The technical approach is sound. The proposed method seems not that easy to reproduce by other researchers. The paper is clearly written. 
summary_of_the_review: Overall, this is a descent paper addressing a well-motivated problem. The proposed stage-aware expressivity score is new and works well. Well, face detectors have been integrated in many face recognition systems on embedded platforms. Usually, the face detector is not the bottleneck for either model size or computation.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a distribution-dependent stage-aware ranking score (DDSAR-Score) for efficient face detection (FD), which can explicitly characterize the stage-level expressivity and identify the individual importance of each stage, thus satisfying the model structure design criterion of the FD backbone. As an accuracy predictor, DDSAR-Score measures the quality of sampled backbone architectures stage-wise, which is beneficial to apply the neural architecture search (NAS) technique to FD. 
strength_and_weaknesses: Strength: 
The proposed method is reasonable and feasible. Also, the corresponding theoretical proofs are sufficient. From the reported experimental results, the usefulness of DDSAR-Score can be confirmed.
Weaknesses: 
1.	The literature review related to NAS should be enriched.
2.	The explanation of formulae should be improved. For example, what does $\circ $ mean in the definition of ${{\mathcal{F}}_{\mathcal{N}}}\left( {{x}^{0}};\theta  \right)$?
3.	The strategy for selecting weights in Section 4.2 is described as shallow.
4.	Is the selection of the evolutionary architecture search algorithm a limitation of your work? In other words, can any other search algorithms work? 
5.	How do you define the network search space?
6.	Testing data could be enriched.
7.	More methods should be chosen to testify to your method, especially the latest ones.
8.	Reference should be updated. Most of the cited literature in the current version was published before 2020, which would confuse the readers that your research topic is not essential anymore.

clarity,_quality,_novelty_and_reproducibility: This manuscript is written and organized well. The proposed method is interesting and novel. However, the experiments should be enriched to convince the readers.
summary_of_the_review: Based on the above comments, my decision is Accept.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper explores NAS approaches to search for a lightweight face detection backbone. Though there is plenty of NAS methods for classification models, searching for a face detector backbone is still rare due to the challenges of the multi-stage nature. This paper proposes a new distribution-dependent stage-aware ranking score to enable searching for face detector backbones. Experiment results demonstrate the yielded backbone works well. 

Strength:

- The studied problem, i.e., how to design light-weight face detectors, is important.
- The proposed method is novel.
- The yielded backbone works empirically well in the experiments. 

Weakness:
- Some technical details can be clearer. 
- The performance over existing models are not very significant. 
justification_for_why_not_higher_score: This work proposes a new solution to an important problem. However, the reviewers also share some concerns on (1) the performance advantage is not very significant compared with existing works and (2) some details and ablation experiments are lacking, and also in-depth analysis on the method is lacking. These limit its impact in the community.  
justification_for_why_not_lower_score: All the reviewers give accept ratings. This work proposes a new solution to an important problem. Though the performance improvement is marginal, its method and resulted models should be valuable for the face detection community.
Decision: Accept: poster