strong meta vision-transformer architechture pretrain support

#13
by JiangYi - opened

MetaFormer

We are very happy to meet you here and to tell you about our work(MetaFormer). We use a unified framework with meta information to do fine-grained recognition task. We have achieved SOTA performance on INAT18/19/21 and CUB datasets.

###Fine-grained Datasets
Result on fine-grained datasets with different pre-trained model.

Name Pretrain CUB NABirds iNat2017 iNat2018 Cars Aircraft
MetaFormer-0 ImageNet-1k 89.6 89.1 75.7 79.5 95.0 91.2
MetaFormer-0 ImageNet-21k 89.7 89.5 75.8 79.9 94.6 91.2
MetaFormer-0 iNaturalist 2021 91.8 91.5 78.3 82.9 95.1 87.4
MetaFormer-1 ImageNet-1k 89.7 89.4 78.2 81.9 94.9 90.8
MetaFormer-1 ImageNet-21k 91.3 91.6 79.4 83.2 95.0 92.6
MetaFormer-1 iNaturalist 2021 92.3 92.7 82.0 87.5 95.0 92.5
MetaFormer-2 ImageNet-1k 89.7 89.7 79.0 82.6 95.0 92.4
MetaFormer-2 ImageNet-21k 91.8 92.2 80.4 84.3 95.1 92.9
MetaFormer-2 iNaturalist 2021 92.9 93.0 82.8 87.7 95.4 92.8

Results in iNaturalist 2019, iNaturalist 2018, and iNaturalist 2021 with meta-information.

Name Pretrain Meta added iNat2017 iNat2018 iNat2021
MetaFormer-0 ImageNet-1k N 75.7 79.5 88.4
MetaFormer-0 ImageNet-1k Y 79.8(+4.1) 85.4(+5.9) 92.6(+4.2)
MetaFormer-1 ImageNet-1k N 78.2 81.9 90.2
MetaFormer-1 ImageNet-1k Y 81.3(+3.1) 86.5(+4.6) 93.4(+3.2)
MetaFormer-2 ImageNet-1k N 79.0 82.6 89.8
MetaFormer-2 ImageNet-1k Y 82.0(+3.0) 86.8(+4.2) 93.2(+3.4)
MetaFormer-2 ImageNet-21k N 80.4 84.3 90.3
MetaFormer-2 ImageNet-21k Y 83.4(+3.0) 88.7(+4.4) 93.6(+3.3)

We also provide Imagenet 1k/22k and Inaturalist Pretrain models with various resolution, Welcome to use our pretrain models and codebase.

Model zoo

name resolution 1k model 21k model iNat21 model
MetaFormer-0 224x224 metafg_0_1k_224 metafg_0_21k_224 -
MetaFormer-1 224x224 metafg_1_1k_224 metafg_1_21k_224 -
MetaFormer-2 224x224 metafg_2_1k_224 metafg_2_21k_224 -
MetaFormer-0 384x384 metafg_0_1k_384 metafg_0_21k_384 metafg_0_inat21_384
MetaFormer-1 384x384 metafg_1_1k_384 metafg_1_21k_384 metafg_1_inat21_384
MetaFormer-2 384x384 metafg_2_1k_384 metafg_2_21k_384 metafg_2_inat21_384

Repo : https://github.com/dqshuai/MetaFormer
paper link: https://arxiv.org/abs/2203.02751

Sign up or log in to comment