Before revision:  We curate a massive corpus of Python programs from GitHub to pre-train a BERT model, which we call Code Understanding BERT (CuBERT).
After revision:  We curate a massive , deduplicated corpus of 6M Python files from GitHub, which we use to pre-train a BERT model, which we call Code Understanding BERT (CuBERT).