Repository: scipy/scipy

Issue:
runtests.py overrides `$PYTHONPATH`
I am running a setup where python and python packages are install via [spack](https://github.com/spack/spack). Within this I cloned and build the scipy master. Both `python3 setup.py install --user` (without virtual env) and `python3 setup.py install` (inside a virtual env with `--system-site-packages`) works.

Running `python3 runtests.py`, both without and inside a virtual env, yields:
```
Building, see build.log...
Error: 'numpy' must be installed before running the build.


Build failed! (0:00:00.504368 elapsed)`
```

However `numpy` appears to be present enough for the installation to work and it can also be imported in python interpreter ran in this environment.

#### Reproducing code example:
Below which set up an environment similar to mine and thus should reproduce the error, but I did not test it as would take quite some time to run. It may be better for someone familiar with `runtests.py` to check if there is something inside it which could make it fail to find package which are other wise found by `import`.

This is just to get the gist
```bash
# get spack
git clone https://github.com/spack/spack

# minimally load spack into env
.  spack/share/spack/setup-env.sh
spack compiler add

# install scipy dependencies, will install python 3
DEPS="py-cython  py-pybind11  py-setuptools  py-numpy   py-pythran"
# NOTE: this would take a while. It will not only compile the listed packages, but also basically
# everything they depend on, including things like openblas, zlib and openssl
for p in $DEPS; do
  spack install $p
done

#load modules
## This is the key part: paths to numpy and others are added to
## $PYTHONPATH , $LD_LIBRARY_PATH and $PATH as required,
## but each is under a different prefix.
for p in $DEPS; do
  spack load -r $p
done

which python # gives some very long path inside ~spack/opt/
python3 -c "import numpy" # works

# scipy
git clone https://github.com/scipy/scipy
cd scipy

# optionally, I guess:
python3 setup.py install --user # should work

python3 runtests.py # complains about missing numpy
```

#### Error message:
```
Building, see build.log...
Error: 'numpy' must be installed before running the build.


Build failed! (0:00:00.504368 elapsed)`
```

#### Scipy/Numpy/Python version information:
```bash
python -c "import sys, numpy; print(numpy.__version__, sys.version_info)"
# 1.21.0 sys.version_info(major=3, minor=8, micro=11, releaselevel='final', serial=0)
```
scipy: master


Discussion:
I notice you swap between `python` and `python3`. Are you sure they are the same?
> I notice you swap between `python` and `python3`. Are you sure they are the same?

Yes, they are the same in my cases.

Sorry, this is a bad inconsistency to have, I updated the description to only use `python3`.

The underlying issue is, that [`runtest.py`](https://github.com/scipy/scipy/blob/master/runtests.py#L175) overrides `$PYTHONPATH`.

*Edit*: Specifically for the invocation of `setup.py`, [here](https://github.com/scipy/scipy/blob/master/runtests.py#L392).

Resolving patch:
diff --git a/runtests.py b/runtests.py
--- a/runtests.py
+++ b/runtests.py
@@ -172,7 +172,8 @@ def main(argv):
     if not args.no_build:
         site_dir = build_project(args)
         sys.path.insert(0, site_dir)
-        os.environ['PYTHONPATH'] = site_dir
+        os.environ['PYTHONPATH'] = \
+            os.pathsep.join((site_dir, os.environ.get('PYTHONPATH', '')))
 
     extra_argv = args.args[:]
     if extra_argv and extra_argv[0] == '--':
@@ -389,7 +390,7 @@ def build_project(args):
     # and isn't on the PYTHONPATH. Plus, it has to exist.
     if not os.path.exists(site_dir):
         os.makedirs(site_dir)
-    env['PYTHONPATH'] = site_dir
+    env['PYTHONPATH'] = os.pathsep.join((site_dir, env.get('PYTHONPATH', '')))
 
     log_filename = os.path.join(ROOT_DIR, 'build.log')
     start_time = datetime.datetime.now()


Test patch: