Before revision:  We first pre-train our model to learn the universal representation for both video and language on a large instructional video dataset.
After revision:  Five objectives, including video-text joint, conditioned masked language model (CMLM), conditioned masked frame model (CMFM), video-text alignment, and language reconstruction, are designed to train each of the components. We further develop two pre-training strategies, stage by stage pre-training (StagedP) and enhanced video representation (EnhancedV), to make the training process of the UniVL more effective. The pre-train our model to learn the universal representation for both video and language on a large instructional video dataset.