Only delete on success path.