Backout change-server-keeps-old, it's not ready