  This paper introduces EfficientNetV2, a new family of convolutional networks
that have faster training speed and better parameter efficiency than previous
models. To develop this family of models, we use a combination of
training-aware neural architecture search and scaling, to jointly optimize
training speed and parameter efficiency. The models were searched from the
search space enriched with new ops such as Fused-MBConv. Our experiments show
that EfficientNetV2 models train much faster than state-of-the-art models while
being up to 6.8x smaller.
  Our training can be further sped up by progressively increasing the image
size during training, but it often causes a drop in accuracy. To compensate for
this accuracy drop, we propose to adaptively adjust regularization (e.g.,
dropout and data augmentation) as well, such that we can achieve both fast
training and good accuracy.
  With progressive learning, our EfficientNetV2 significantly outperforms
previous models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining on
the same ImageNet21k, our EfficientNetV2 achieves 87.3% top-1 accuracy on
ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while
training 5x-11x faster using the same computing resources. Code will be
available at https://github.com/google/automl/efficientnetv2.
