codeprep (pypi): A toolkit for pre-processing large source code corpora
Keywords: big|large|data|source|code|corpus|machine|learning|pre-processing|nlp|language-modeling|mining-software-repositories|natural-language-processing|source-code-analysis|word-segmentation