Remove a remote directory that was already deleted locally.