Better handle removable singularities in the polynomial bases.

This should enable the use of alpha optimised points once again.