Basic py.test config
Various behavior for cleaning up cache
Dry run and various fixes