Don't try to remove the watched dir itself