Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a spatio-temporal entropy score (STEntr-Score) with a refinement factor to handle the discrepancy of visual information in spatial and temporal dimensions, through dynamically leveraging the correlation between the feature map size and kernel size depth-wisely. Then the proposed entropy-based 3D CNNs (E3D family), can then be efficiently searched by maximizing the STEntr-Score under a given computational budget, via the existing evolutionary algorithm without training the network parameters. Comprehensive comparisons on the popular benchmark datasets show the advances of the proposed method. 
strength_and_weaknesses: The strengths of the paper are: 
+ This paper tried to exploit the important direction for the video action recognition, or generally the video intellengent analysis. 
+ The authors proposed to estimate correlation between feature map and kernel size in different depths, and furthermore, they proposed the Spatio-Temporal Entropy Score. 
+ The proposed STEntrScore is positively correlated with Top1 accuracy which indicates that the proposed spatio-temporal refinement can handle the discrepancy of visual information in spatial and temporal dimensions.
+ The performance on the benchmark datasets is promising for the research in this area.
+ Inference and training speed are appealing 

The weaknesses of the paper are: 
- No limitation discussion in the paper 
- I would like to see some discussions on the affect of the temporal resolution on the performance, such as if we increase or decrease the number of frames we consider in the searching and final architecture, what the performance? 


clarity,_quality,_novelty_and_reproducibility: The paper is presented clearly. Although the search method is not new, however, the consideration of the feature maps, kernel size in different depths and refined by using the proposed Spatio-Temporal Entropy Score shows promising experimental results. I think it will provide useful insights to the community. 
summary_of_the_review: Overall, I think this is a good paper which will have positive contributions to this field.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper aims to find a good 3D CNN architecture for video recognition. This study proposes to use a novel proxy task, maximizing a designed entropy value, to search for the effective architecture. The experiments show the effectiveness of the proposed algorithm.

strength_and_weaknesses: Strength
1. The experiments are solid, showing a good alignment between the proxy task and the video recognition ability.
2. The results on Sth-Sth are good.

Question
1. “Table 9 shows three instantiations of E3D with varying complexity, including E3D-S (1.9G FLOPs), E3D-M (4.7G FLOPs), and E3D-L (18.3G FLOPs).” Wondering the details about how to get E3D-S/M/L – are they searched separately? Or search a model and then manually scale up.
2. Is it possible to get an even larger model (maybe the name would be E3D-XL)? If it is possible, would we expect higher performance?
3. Is it possible to extend this entropy idea to other architectures (eg transformer)?
4. Is it possible to trivially apply other training-free NAS on 3D task? I am a little bit concern that it is not convinced the proposed method is better than previous training-free NAS method.

Weakness
1. The results on Kinetics400 are not as good as the transformer based method (although I understand the transformers are computationally expensive)
2. See question 4: It seems mainly compared with manually designed architectures, but don't compare with other NAS methods.

clarity,_quality,_novelty_and_reproducibility: Clarity: very clear
Quality: high
Novelty: Okay. Training free and entropy idea is explored on 2D, but haven't explore on 3D.
Originality: I didn't see similar works before.
summary_of_the_review: This paper is clear. The story makes sense. The results of this paper is good, and the author shows that the proposed method is better than random search. However, it makes the paper less convincing if don't compare with other NAS based methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a spatio-temporal entropy score to conduct training-free neural architecture search for efficient video recognition. They use a simple network search space with a carefully designed performance estimation score. The searched results achieve state-of-the-art performance on action recognition.
strength_and_weaknesses: ## Strength

1. The derivation of the entropy score is thorough.
1. The visualization of the searched architectures in Figure 1 reveals interesting properties of 3D CNNs.

## Weakness

1. As the search space is the most important part of a NAS method, the authors should give more details such as the lower bound / upper bound performance of the search space.
1. The search happens without considering auxiliary modules like BN, Reslink, and so on. Will the entropy score transfer well with such a gap?
1. Why is Table 2 only compared with most 2D CNN-based methods with ResNet50? Also, the baselines are outdated.

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and the theorems are carefully proven. However, the experiments are not sufficient and the baselines seem outdated.
summary_of_the_review: Baselines are most 2D CNNs-based methods and outdated. So I recommend to weak reject this paper until results on stronger baselines are provided.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a network architecture search method for 3D video recognition network based on a spatial-temporal entropy metric. The results are strong. Though there are some concerns of the reviewers at the beginning, they are largely addressed after rebuttals provided by the authors (although one of the reviewers does not revise the score, he/she does not have concerns any more). Therefore, the meta-reviewer recommends to accept the work. 
justification_for_why_not_higher_score: Only one of the reviewer is enthusiastic about the work. 
justification_for_why_not_lower_score: The concerns of all reviewers have been addressed.
Decision: Accept: poster