Refactor all the diff code into a single dir