Un-index and remove before rejecting.