github is funny. I could swear it posted an answer to the before.
I am fine with removing the spheres, and i think it should be done here. I was just under the impression you had 0 sized spheres be of some special type. You should check for radius < std::numeric_limits<double>() instead of 0.

Also, since size() == 1 always in this initalization, the if statement can be simplified