SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 15: Comparison of our method with SOTA NAS methods using MobileNet and ImageNet-1K.
Approach	Top-1 (%)	Search Cost (GPU Days)	Params (M)	Method
MobileNet-V3(1.0)	75.2	288	5.3	MS
PNAS	74.2	224	5.1	MS
DARTS	73.3	4	4.7	OS
PC-DARTS	75.8	3.8	-	OS
SPOS	74.7	8.3	-	OS
GreedyNAS	74.9	7.6	3.8	OS
TE-NAS	75.5	0.17	5.4	ZS
ZiCo	77.1	0.4	-	ZS
Zero-Cost-PT	76.4	0.04	8.0	ZS
Zen-score	76.1	0.5	-	ZS
MeCo (Ours)	77.8	0.08	7.9	ZS
Finally, we demonstrate the limitations of this work and the possible directions of our future work. In Section 5, We propose an optimization method to alleviate the channel-sensitive issue of MeCo. This weight-sampling approach can be further improved in future work. Moreover, we would like to point out that the evaluation of a zero-cost proxy is often tied to the availability of benchmarks. Hench though the proxies are "zero cost", the evaluation of the proxies is strongly dependent on a benchmark, which in the first place is very expensive to create. Finally, although MeCo achieves the highest correlation with the test accuracy on multiple benchmarks, it shows near zero correlations on a few tasks (e.g., TransBench-101-Micro with Autoencoding). We will leave these issues as our future work.
Figure 5: Spearman correlation coefficients of MeCo with different number of samples on NATS-Bench-TSS and NATS-Bench-SSS, respectively.
Normal Cell
Reduction Cell
(a) synflow
Reduction Cell
(b) fisher
Reduction Cell
(c) snip
Reduction Cell
(d) grasp
Normal Cell
Reduction Cell
(e) jacov
Reduction Cell
(f) NTK
Normal Cell
(g) ZiCo
Reduction Cell
(h) MeCo
Figure 6: Cells found by Zero-Cost-PT with all zero-cost proxies on the DARTS-CNN search space using CIFAR-10
(a) synflow
Normal Cell
(b) fisher
Normal Cel
(c) snip
Normal Cell
(d) grasp
Normal Cell
(e) jacov
(f) NTK
Reduction Cell
(g) ZiCo
Normal Cell
(h) MeCo
Reduction Cell
Figure 7: Cells found by Zero-Cost-PT with all zero-cost proxies on the DARTS-CNN search space using CIFAR-100