Before revision: A structure of a protein has a direct impact on its properties and functions.
After revision: Motivation: Bridging the exponential gap between the number of unlabeled and labeled protein sequences, a couple of works have adopted semi-supervised learning for protein sequence modeling. They pre-train a model with a substantial amount of unlabeled data and transfer the learned representations to various downstream tasks. Nonetheless, the current pre-training methods mostly rely on a language modeling pre-training task and often show limited performances. Therefore, a protein has a direct impact on its properties and functions.