  This work explores how to design a single neural network that is capable of
adapting to multiple heterogeneous tasks of computer vision, such as image
segmentation, 3D detection, and video recognition. This goal is challenging
because network architecture designs in different tasks are inconsistent. We
solve this challenge by proposing Network Coding Propagation (NCP), a novel
"neural predictor", which is able to predict an architecture's performance in
multiple datasets and tasks. Unlike prior arts of neural architecture search
(NAS) that typically focus on a single task, NCP has several unique benefits.
(1) NCP can be trained on different NAS benchmarks, such as NAS-Bench-201 and
NAS-Bench-MR, which contains a novel network space designed by us for jointly
searching an architecture among multiple tasks, including ImageNet, Cityscapes,
KITTI, and HMDB51. (2) NCP learns from network codes but not original data,
enabling it to update the architecture efficiently across datasets. (3)
Extensive experiments evaluate NCP on object classification, detection,
segmentation, and video recognition. For example, with 17\% fewer FLOPs, a
single architecture returned by NCP achieves 86\% and 77.16\% on
ImageNet-50-1000 and Cityscapes respectively, outperforming its counterparts.
More interestingly, NCP enables a single architecture applicable to both image
segmentation and video recognition, which achieves competitive performance on
both HMDB51 and ADE20K compared to the singular counterparts. Code is available
at https://github.com/dingmyu/NCP}{https://github.com/dingmyu/NCP.
