BUG: Fix gitdb and async build error explicitly setting build environment

In addition to update the install step of these two projects, this commit
also revert commit r23065 that was disabling the build of PyGithub
and GitPython projects.