This seems to be checking that none of the new servers have been added to the first load balancer, which is a great thing to test, but seems like this can be asserted without the scale down?

It sort of seems like the scale down test is a separate test (that when a server is deleted, it's removed from the load balancer).  But I'm also wondering about the timing of that.  If enough time has passed, and the servers are deleted, will `self._get_ipv4_address_list_on_servers(active_server_list)` work?  (because all the servers in `active_server_list` might have been deleted by then, and so they will not have any addresses).