# Multi-View Masked Autoencoder for General Image Representation
## 4 Experiments
### Experimental Results
#### 4.2.1 Architecture Analysis  
To analyze the components of our architecture, we conducted experiments on the masking ratio, loss weight, and decoder depth, which were evaluated on the ImageNet-1K dataset. We pre-trained the model for 200 epochs and fine-tuned the model for 100 epochs. The default setting of our model in Section 4.1.1 is derived from these results. The experimental results on the mask ratio, loss weight, and decoder depth are shown in Table 5.  
Masking ratio: Previous contrastive learning methods adopt strong augmentation, i.e., Gaussian blur and color distortion, due to the representation collapse problem. When the model learns the same representation losing input data diversity, thus resulting in constant output and performance decrement, it is called representation collapse. To avoid this problem, it is necessary to have diverse negative samples, thereby relying on data augmentation. Previous studies have investigated combinations of augmentation, thus showing performance gains depending on which augmentation is used. In addition, our \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **mIOU** \\ \hline MoCo-v3 [32] & 47.3 \\ BeiT [2] & 47.1 \\ CAE [37] & 50.2 \\ SimMIM [36] & **52.8** \\ MAE [23] & 48.1 \\ \hline Ours & 50.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Semantic segmentation results on ADE20K dataset. The best result is shown in bold, and second best results are underlined. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **AP\({}^{box}\)** & **AP\({}^{mask}\)** \\ \hline MoCo-v3 [32] & 47.9 & 42.7 \\ BeiT [2] & 49.8 & 44.4 \\ CAE [37] & 50.0 & 44.0 \\ SimMIM [36] & **52.3** & - \\ MAE [23] & 50.3 & 44.9 \\ \hline Ours & 51.3 & **45.6** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Object detection and segmentation results on COCO dataset. The best result is shown in bold, and second best result is underlined. work overcomes this representation collapse problem to some extent by simply masking a relatively high portion of the image without additional augmentation. We conducted experiments on the mask ratio, as shown in Table 5. Masking 80% of the input patch sequence showed the best performance, while extreme masking (95%) showed the lowest. Masked language models conventionally mask relatively low portions (e.g., 15% [17]) of text tokens, since insufficient context interrupts learning the text representation. However, to eliminate redundancy in the image, a higher masking ratio should be applied in MIM. Our experimental results show that a relatively higher masking ratio (80%) removes the redundancy of image pixels and provides sufficient information to learn image representations at the same time. Note that a higher masking ratio does not necessarily perform well, as an extreme masking ratio (95%) yielded the lowest performance. An extreme masking ratio yielded a performance degradation because it obscures so much of the image that the model does not have sufficient information to learn the image representation. Loss weight: As was aforementioned, our total loss is a weighted sum of two training objectives. We conducted experiments to explore how \(\lambda\), a hyperparameter of the loss weight, affected the model performance by changing the loss weight. Note that when \(\lambda\) was set to zero, the model was the same as the baseline, the MAE. The results show that contrastive loss does affect model performance in a good way. Interestingly, as the loss weight increases, that is, the more the contrastive loss is contributed, the model's performance correspondingly increases. We can say that by adding contrastive loss, the encoder is trained to learn more general and holistic representations. Decoder depth: Since we only used the decoder in the pre-training process, we can flexibly design the decoder, thus adopting an asymmetric encoder-decoder architecture. We conducted experiments on the decoder depth to figure out whether the model would benefit from a shallower decoder depth.