Only remove PySide and PyQt if not in egg_info mode