Stop doing lb_type dispatch in delete_server

This should be done after #772, #773. Closes #767.
It does; I thought we did basically just store the entire thing?