Only install build packages if try_pip is true