Verify if directory exists before removing