Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper gives a significantly improved neural architecture discovery method specialized for convolutional networks. The idea is to train a large super-network and cache common computation and select the best performing subnetwork. The paper also presents several useful ideas to accelerate to training and exploration of convolutional operators at a large scale. This is based on the observation that the overhead of using Fourier transformation is significantly amortized if the transformation is used for exploring convolutions for a wider range of hyperparameters including patch size and dilation factor. In addition the work suggests exploration on multiple tasks at the same tasks for more robust results.
strengths_and_weaknesses: Originality: High

Although essential the ideas in this paper are based on several earlier works (e.g. DARTS or [Wu et al:Tree structured network...]) or the use of Fourier transformations has ben extensively explored in the literature. However the recognition that reusing some of the time-consuming intermediate results make these methods much more appealing in this context is novel and useful.

Quality: Mixed.

The methods are well-motivated and seemingly efficient, but the experimental results lack some more thorough evidence, especially on images of larger size. While the synergy of the applied methods is very high and seems to result in a very efficient methodology, the search space is limited to a very specific type of convolutional deep learning models. The experimental section also lacks more thorough ablation for judging the criticality and 

Clarity: Medium.

The description of the applied methods is extremely formal/mathematical and would benefit from a more intuitive explanations and more figures to introduce the later to the most essential ideas in a more gentle way. Also, the paper is not structured in a very appealing manner and could have been made more accessible by a better top-down explanation which would explain the synergy of the components and more thorough intuitive comparisons with similar approaches.
  
Significance: Medium. Given the fact that transformers are overtaking convolutional networks in many respects, it is hard to judge the future significance of these ideas as transformer-based methods, or even just using transformer-based exploration might render these methods less appealing in the near future. Also the lack of experimental results on benchmarks with larger images (like ImageNet) is concerning and makes it harder to judge the expected impact of the methods.
questions: Could you give more experimental evidence on benchmarks with large images and more classes (esp ImageNet) and other vision tasks.

limitations: I don't see any concerns regarding the societal impact specific to this work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper introduces a NAS method called NASH. This method can be applied to a variety of tasks and achieve good results. It is meaningful that NAS is applied to a variety of tasks, which demonstrates the potential of NAS methods.
strengths_and_weaknesses: # Strengths
+ The tricks used for acceleration are a bit interesting.
+ It makes sense to apply the NAS method to other tasks.

# Weaknesses
+ Table3 is mentioned in many places, but I didn't find it.
+ There are some problems with the experimental part：
1. The FLOPs for each model are not reported, and it is not clear to me whether these methods are comparable.
2.  Is the search space the same for each model?
3. Why only use WRN for experiments? Different structures should be chosen to verify the effectiveness of the method.

+ Frankly speaking, NASH is not a surprising method in terms of method innovation, it is just an application of DARTS to some extent.
questions: Please refer to the weakness in the above part.
limitations: Please refer to the weakness in the above part.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on the direction of searching architectures for diverse tasks. The designed search space contains convolution layers with large and diverse set of kernel sizes and dilation rates. To address the efficiency challenges encountered in weight-sharing, DASH is proposed by using mixture-of-operations based on the Fourier diagonalization of convolution. Extensive experiments on ten tasks suggest the effectiveness of the proposed DASH methods. 
strengths_and_weaknesses: Strengths:
1. This paper explores a new direction of applying NAS in diverse tasks. In their setting, the authors notice the importance of involving convolution layers with large kernel sizes and dilation sizes. To address the efficiency problem in the large search space, mixture-of-operations based on the Fourier diagonalization of convolution is used in their proposed DASH.
2. Extensive experiments validate the importance of involving convolution operations with large kernel sizes and the effectiveness of DASH in ten tasks. 

Weaknesses:
1. For diverse tasks, one important setting is that whether we can directly search architectures from the super-network for different tasks. It seems that the authors did not take this into account. 
2. In section 3.3, grid search is used to search hyper-parameters in both search and retraining stage. Does grid search play a key role in performance? If grid search is also applied to baseline methods, will it improve the performance? 
questions: 1. I 'm wondering the performance of DASH in the designed search space with convolutions of large kernel sizes and dilation sizes in large data sets, like ImageNet. 
2. The other two questions are listed in the weakness part. 
Looking forward to authors' responses.
limitations: Yes.
review_title: null

Meta-review: metareview: This paper proposes a significantly improved neural architecture discovery method specialized for convolutional networks. 

The idea is to train a large super-network and cache common computation and select the best performing subnetwork. 

Also, it presents several useful ideas to accelerate to training and exploration of convolutional operators at a large scale.

While the work is the combination of somewhat well-known approaches, the overall approach is novel, interesting and well-motivated, however its application domain is somewhat limited.

Still, given the convincing experimental results, good execution and significant gains in terms of training time compared to competing appraoches, I propose this paper to be accepted for NeurIPS 2022. 
confidence: Certain
Decision: Accept