preprocess corpora. No of skipped lines still not ok