Some refactoring. Also, read pickles from subdirectory subsets
Added function to clean data in a new file instead of doing it in memory