Allow importing pandas without setuptools

Possible solution for https://github.com/pydata/pandas/pull/8107#issuecomment-57017595