Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: 
The paper proposes a method called FNA (fast network adaptation), which takes a pretrained image classification network, and produces a network for the task of object detection/semantic segmentation. The process consists of three phases: Network Expansion, Architecture Adaptation and Parameters Adaptation, and uses the developed parameter remapping scheme twice. Experiments show that it outperforms recent other NAS methods for these two tasks with same or less computation.

Concrete comments
1. The paper's overall method is a novel one, unifying NAS on det/seg tasks, while prior works mostly only focus on one task. It also "eliminates" the need for pretraining each instance of the subnetwork. But no one ever pretrain every classification network for searching on det/seg tasks right? It's an insane amount of computation after all. I'm afraid the emphasis of advantage over prior method here is not very accurate.

2. The concrete parameter remapping scheme is not entirely novel. It is similar to the Net2Net method, while seems more naive than that. It does not preserve the mapping function like Net2Net. It seems like a very coarse effort, since mostly what you do is to copy weights, remove weights or fill in zeros. But it is also interesting to see that this naive method works, and actually beat some of the more advanced alternatives in Section 4. 

3. The results are quite impressive. On segmentation, the adapted model achieves ~1% mIOU improvement using  similar or less iterations and similar size of model with the methods it compared to, and GPU hours' saving is more significant. If the authors faithfully compared with state-of-the-art methods in search det/seg architectures, but I'm not super familiar with this literature. On object detection the method does not improve the model size or accuracy, but reduces the search time a lot compared with DetNAS. Could the authors clarify that you compared with every recent high-performance NAS method on seg/det tasks?

4. Though the improvement over prior methods is good, the experiments lack an apple-to-apple comparison. For example, using exactly the same NAS search method and supernet, and comparing the FNA method with that not using a pretrained model (i.e., directly search on det/seg) could be a good experiment to showcase the importance of adaptation.

Overall I find the method is effective and experiments convincing and I recommend weak accept in my rating. I hope authors can address my concerns in the rebuttal. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper provides a new technique to adapt a source neural network performed well on classification task to image segmentation and objective detection tasks via the author called parameter-remapping trick. The parameter remapping uses weights from the source neural network to the two-stages: architecture adaption phase and parameter adaption phase. The technique results in improvements in both performance and training time.

I like the direction this paper takes, NAS is too expensive and we need faster methods through meta learning/transfer learning. The paper is also clearly organized and written.  To the best of my knowledge, the experiments setting is sensible and the results are good. But I am not in the Computer Vision field and I am not so familiar with NAS, I may missed something. Thus, I am less confident about my rating.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: In this paper, the authors take a MobileNet v2 trained for ImageNet classification, and adapt it either (i) semantic segmentation on Cityscapes, or (ii) object detection on COCO. They do this by first expanding the network into a "supernet" and copy weights  in an ad-hoc manner, then, they perform DARTS-style architecture search before fine-tuning for the task at hand.

There is no TLDR for this paper, and I must admit, on reading the abstract and introduction I wasn't entirely sure what this paper was doing at first. Perhaps I was being slow.

From a narrative perspective, one of the main selling points is not needing to perform any expensive ImageNet pre-training; however, a pre-trained MobileNetv2 is being utilised. While this was off-the-shelf, it still incurred an initial training cost, so it isn't really fair in e.g. Table 4 to put pre-training cost as zero. On a related note, the authors write that this network is used for its "generality". I'd argue that MobileNetv2 is a highly engineered network specialised for mobile computation; a standard ResNet-50 would be more general really.

I would like to see a comparison to a random search, as there are several papers (https://arxiv.org/abs/1902.07638, https://arxiv.org/abs/1902.08142) indicating that this is a very strong baseline. 

As mentioned earlier, the choices for remapping weights seem very ad-hoc. I can't really tell what's going on in Table 5 (why is PR in the NE and PA row?) so the ablation study of how effective this weight mapping is lost on me.  The stuff in Table 6 is pretty interesting however, if convoluted.

I find the odd choices of hyperparameters (tau as 45, gamma as 10, lambda as 9e-3) rather alarming. How important are these? Would this technique work under any other circumstances?

Error bars would be a welcome inclusion, particularly in Table 3 where you have 0.1% separating FNA and MNasnet-92. I appreciate that this can be expensive however.

Pros:
- Some promising results
- Good figures

Cons:
- Ad-hoc design choices
- Not a fair comparison regarding pre-training. 
- Very specific to one network choice
- Lack of error bars or comparison to random search.

I am giving this paper a weak reject, as there is insufficient experimental evidence that the technique works, or generalises beyond Mobilenetv2. I am also concerned about the ad-hoc hyperparmaters or weight-mapping. A comprehensive ablation study, along with error bars, and another choice of seed network would do much to strengthen this paper.





review_title: Official Blind Review #1

Meta-review: Main content: Paper proposes a fast network adaptation (FNA) method, which takes a pre-trained image classification network, and produces a network for the task of object detection/semantic segmentation

Summary of discussion:
reviewer1: interesting paper with good results, specifically without the need to do pre-training on Imagenet. Cons are better comparisons to existing methods and run on more datasets. 
reviewer2:  interesting idea on adapting source network network via parameter re-mapping that offers good results in both performance and training time.
reviewer3: novel method overall, though some concerns on the concrete parameter remapping scheme. Results are impressive
Recommendation: Interesting idea and good results. Paper could be improved with better comparison to existing techniques. Overall recommend weak accept.
Decision: Accept (Poster)