added code for cleaning the corpus
added multiple tests, not compiling