Repository: spyder-ide/spyder

Issue:
Profiler process needs to remove `PYTHONEXECUTABLE` to run in conda env
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [x] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [x] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [x] Reset preferences with ``spyder --reset``
    * [x] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [x] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

If Spyder is set up to use a Python interpreter from some conda environment different from its own, then Profiler does not run correctly on Mac.

This is because the QProcess created in profiler/widgets/main_widget.py seems to have the PYTHONEXECUTABLE environment variable set to the value of Spyder's own python executable, which may be different than the interpreter we are trying to run the Profiler in. This causes sys.path to be set incorrectly when profiling, and thus, if the code you are profiling tries to import any modules that aren't also installed in Spyder's own environment, it crashes.

The following one line corrects this on line 538 of profiler/widgets/main_widget.py:

```
        proc_env = QProcessEnvironment()
        for k, v in os.environ.items():
            proc_env.insert(k, v)
        proc_env.insert("PYTHONIOENCODING", "utf8")
        proc_env.remove('PYTHONPATH')
        proc_env.remove('PYTHONEXECUTABLE') # <----- ADD THIS
```

And then profiling works correctly.

### What steps reproduce the problem?

1. Install Spyder in some conda environment
2. In Spyder's settings, set the Python interpreter to that of another conda environment
3. Run the profiler on any file which has a package installed in the other environment, but not Spyder's native one
4. You will get a popup with a ModuleNotFound error

### What is the expected output? What do you see instead?

For instance, here is the error:

<img width="746" alt="image" src="https://user-images.githubusercontent.com/1020594/221730237-d6c0dbb8-b64d-4c77-bddc-fc1d3e4e4c96.png">
## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 5.4.2  (conda)
* Python version: 3.10.9 64-bit
* Qt version: 5.15.6
* PyQt5 version: 5.15.7
* Operating System: Darwin 21.6.0


### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
# Mandatory:
applaunchservices >=0.3.0     :  0.3.0 (OK)
atomicwrites >=1.2.0          :  1.4.1 (OK)
chardet >=2.0.0               :  5.1.0 (OK)
cloudpickle >=0.5.0           :  2.2.1 (OK)
cookiecutter >=1.6.0          :  2.1.1 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.0.2 (OK)
IPython >=7.31.1;<9.0.0       :  8.10.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.2 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.17.3 (OK)
keyring >=17.0.0              :  23.13.1 (OK)
nbconvert >=4.0               :  7.2.9 (OK)
numpydoc >=0.6.0              :  1.5.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.4 (OK)
pygments >=2.0                :  2.14.0 (OK)
pylint >=2.5.0;<3.0           :  2.16.2 (OK)
pylint_venv >=2.1.1           :  2.3.0 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.7.1;<1.8.0          :  1.7.1 (OK)
pylsp_black >=1.2.0           :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.2.2             :  0.2.2 (OK)
qtawesome >=1.2.1             :  1.2.2 (OK)
qtconsole >=5.4.0;<5.5.0      :  5.4.0 (OK)
qtpy >=2.1.0                  :  2.3.0 (OK)
rtree >=0.9.7                 :  1.0.1 (OK)
setuptools >=49.6.0           :  67.1.0 (OK)
sphinx >=0.6.6                :  6.1.3 (OK)
spyder_kernels >=2.4.2;<2.5.0 :  2.4.2 (OK)
textdistance >=4.2.0          :  4.5.0 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.2.1 (OK)
zmq >=22.1.0                  :  25.0.0 (OK)

# Optional:
cython >=0.21                 :  0.29.33 (OK)
matplotlib >=3.0.0            :  None (NOK)
numpy >=1.7                   :  1.24.2 (OK)
pandas >=1.1.1                :  None (NOK)
scipy >=0.17.0                :  None (NOK)
sympy >=0.7.3                 :  None (NOK)
```



Discussion:
Hey @battaglia01, thanks for reporting and finding a solution to this problem. You said:

> The following one line corrects this on line 538 of profiler/widgets/main_widget.py:

The solution looks good to me, so could you open a pull request for it?

I don't remember if you have done contributions before, but in case you haven't, please read our [Contributing guide](https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md) and make your pull request against our `5.x` branch.

In case you don't feel comfortable with that, I'll make the change for you.
Sure, but that file is different in 5.x and I'm not sure what's going on with it. Used to be this

```
        self.process = QProcess(self)
        self.process.setProcessChannelMode(QProcess.SeparateChannels)
        self.process.setWorkingDirectory(wdir)
        self.process.readyReadStandardOutput.connect(self._read_output)
        self.process.readyReadStandardError.connect(
            lambda: self._read_output(error=True))
        self.process.finished.connect(
            lambda ec, es=QProcess.ExitStatus: self._finished(ec, es))
        self.process.finished.connect(self.stop_spinner)

        # Start with system environment
        proc_env = QProcessEnvironment()
        for k, v in os.environ.items():
            proc_env.insert(k, v)
        proc_env.insert("PYTHONIOENCODING", "utf8")
        proc_env.remove('PYTHONPATH')
        proc_env.remove('PYTHONEXECUTABLE')
        if self.pythonpath is not None:
            logger.debug(f"Pass Pythonpath {self.pythonpath} to process")
            proc_env.insert('PYTHONPATH', os.pathsep.join(self.pythonpath))
        self.process.setProcessEnvironment(proc_env)
```

Now it's this:

```
        self.process = QProcess(self)
        self.process.setProcessChannelMode(QProcess.SeparateChannels)
        self.process.setWorkingDirectory(wdir)
        self.process.readyReadStandardOutput.connect(self._read_output)
        self.process.readyReadStandardError.connect(
            lambda: self._read_output(error=True))
        self.process.finished.connect(
            lambda ec, es=QProcess.ExitStatus: self._finished(ec, es))
        self.process.finished.connect(self.stop_spinner)

        if pythonpath is not None:
            env = [to_text_string(_pth)
                   for _pth in self.process.systemEnvironment()]
            add_pathlist_to_PYTHONPATH(env, pythonpath)
            processEnvironment = QProcessEnvironment()
            for envItem in env:
                envName, __, envValue = envItem.partition('=')
                processEnvironment.insert(envName, envValue)

            processEnvironment.insert("PYTHONIOENCODING", "utf8")
            self.process.setProcessEnvironment(processEnvironment)
```

The whole thing is different now: the QProcessEnvironment object isn't created at all unless pythonpath is not None.

This is strange to me because the code I was talking about above, which is in my 5.4.2 installation from conda, seems to be the same as the 'master' branch, or at least closer to it but not the '5.x' branch. Except it also has these lines:

```
        if not running_in_mac_app(executable):
            env = self.process.processEnvironment()
            env.remove('PYTHONHOME')
            self.process.setProcessEnvironment(env)
```
FWIW, I'm not sure why the 5.x branch looks the way it does, bot those PYTHONHOME and PYTHONEXECUTABLE variables need to be cleared pretty much no matter what (and probably some others). If it inherits those from the calling spyder interpreter, the sys.path will be messed up. I'm not quite sure why the 5.x branch doesn't create the QProcessEnvironment unless pythonpath is None. (I'm also not sure why the 5.4.2 one only clear PYTHONHOME if it's *not* running in the Mac app.)

How do you want me to merge all this?
I think you haven't updated your clone in a while because this is how the file looks in 5.x around the lines you're referencing:

https://github.com/spyder-ide/spyder/blob/2167cb5be41f7897aa3d65eebbd0e5bf7b0d2a6e/spyder/plugins/profiler/widgets/main_widget.py#L531-L539

and master:

https://github.com/spyder-ide/spyder/blob/f0a3df5578c5ccec404c90ee5e3c1fd36567fc4a/spyder/plugins/profiler/widgets/main_widget.py#L531-L539

Resolving patch:
diff --git a/spyder/plugins/profiler/widgets/main_widget.py b/spyder/plugins/profiler/widgets/main_widget.py
--- a/spyder/plugins/profiler/widgets/main_widget.py
+++ b/spyder/plugins/profiler/widgets/main_widget.py
@@ -533,6 +533,7 @@ def start(self, wdir=None, args=None):
             proc_env.insert(k, v)
         proc_env.insert("PYTHONIOENCODING", "utf8")
         proc_env.remove('PYTHONPATH')
+        proc_env.remove('PYTHONEXECUTABLE') # needed on macOS to set sys.path correctly
         if self.pythonpath is not None:
             logger.debug(f"Pass Pythonpath {self.pythonpath} to process")
             proc_env.insert('PYTHONPATH', os.pathsep.join(self.pythonpath))


Test patch: