Delete key after upsert.