Delete test files after tear down so that the config log thread doesn't create files during deletion