Before revision:  We theoretically study the number of labeled data required, and give a model-free lower bound showing that taking limited downstream data will hurt the performance of self-supervised learning. Furthermore, we take the model structure into account and give a model-dependent lower bound, which gets higher when the model capacity gets larger . Moreover, we conduct several numerical experiments to verify our theoretical results.
After revision:  We theoretically study the number of labeled data required, and give a model-free lower bound showing that taking limited downstream data will hurt the final performance instead. In particular, we prove both model-free and model-dependent lower bound, which gets higher when the model capacity gets larger . Moreover, we conduct several numerical experiments to verify our theoretical results.