Because then I'll need to replicate the compatibility checking that pip does (I know pip and setuptools are universal, but if I find a file named pip-1.5-py33-none-any.whl, and I'm on Python 2.7, I'd need to skip it as pip would fail to install it). I could do something simple and "good enough", but that risks odd corner cases and bugs.

I'm actually not sure this is even worth it any more. Now that setuptools no longer uses 2to3, installation from sdist is fast enough that the performance argument is moot. And I'm not sure there is any other real justification for the switch.