BLD: Fix build issue on some versions of easy_install.

This also needs to be backported to maintenance. Will hold off on merging until we're ready for 0.5.1 to see how we want to handle this.