Repository: spyder-ide/spyder

Issue:
Mac OS - Execute in external system terminal not working
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [ ] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [ ] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [ ] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description
Execution in an external system terminal not working for macOS Big Sur & Spyder 5.1.1 (standalone Mac Installer).

I'm working with miniconda environments and have changed python interpreter to the correct environment. Running the code in a dedicated or open console works just fine. 

### What steps reproduce the problem?

1.  Go to Preferences
2. Under Run -> Console -> Select Execute in an external system terminal
3. Run code
4. External system terminal opens but get a fatal Python error

### What is the expected output? What do you see instead?
Running code in external system terminals works just like running in console within Spyder


### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->

```python-traceback

PASTE TRACEBACK HERE

```
![image](https://user-images.githubusercontent.com/59556820/128757379-2b877247-db6a-4500-a353-3850c5dad9b7.png)

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 5.1.1
* Python version: Python 3.9.5 64-bit
* Qt version: 5.12.10
* PyQt version: 5.12.3
* Operating System name/version: Darwin 20.5.0 

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
PASTE DEPENDENCIES HERE
```



Discussion:
Hey @vancromy, thanks for reporting. We haven't tested this functionality with our Mac installer, so it could be broken indeed.

@mrclary, are you able to reproduce this?
I get a slightly different traceback, but I can reproduce this error.
However, this only occurs if Python Interpreter is set to "Default (i.e. same as Spyder's)". If I use a different environment (e.g. a conda environment), then I can execute successfully in an external terminal.
<details><summary>Same as Spyder Env</summary>

```shell
Last login: Tue Aug 10 21:39:03 on ttys002
/var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_6hvyy2_z.sh ; exit;

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
>> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_6hvyy2_z.sh ; exit;
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = '/usr/local/mdsplus/pydevices:/Users/rclary/Documents/Python/mypython:/Users/rclary/Documents/Python/c2w:/Users/rclary/Documents/Python/c2w_psp:/Users/rclary/Documents/Python/dap_jscopes:/Users/rclary/Documents/Python/vessel:/Users/rclary/Documents/Python/msdb-csv:/usr/local/mdsplus/python'
  program name = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
  sys.base_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
  sys.base_exec_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
  sys.platlibdir = 'lib'
  sys.executable = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
  sys.prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
  sys.exec_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
  sys.path = [
    '/usr/local/mdsplus/pydevices',
    '/Users/rclary/Documents/Python/mypython',
    '/Users/rclary/Documents/Python/c2w',
    '/Users/rclary/Documents/Python/c2w_psp',
    '/Users/rclary/Documents/Python/dap_jscopes',
    '/Users/rclary/Documents/Python/vessel',
    '/Users/rclary/Documents/Python/msdb-csv',
    '/usr/local/mdsplus/python',
    '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/python39.zip',
    '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/python3.9',
    '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000000115c5ae00 (most recent call first):
<no Python frame>
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]
```
</details>
<details><summary>Conda Env</summary>

```shell
Last login: Tue Aug 10 21:40:13 on ttys002
/var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_csral3ro.sh ; exit;

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
>> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_csral3ro.sh ; exit;
P(A1) = 0.0132; P(A2) = 0.0529; P(A3) = 0.0661; P(A4) = 0.8677; Total = 1.0000
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]
```
</details>

What is interesting to me is that my error shows no `PYTHONHOME` set. I would think that this is the reason for the problem, except that the OP error shows the correct `PYTHONHOME` set.
I can poke around and see what the issue might be...
> I get a slightly different traceback, but I can reproduce this error.
> 
> However, this only occurs if Python Interpreter is set to "Default (i.e. same as Spyder's)". If I use a different environment (e.g. a conda environment), then I can execute successfully in an external terminal.
> 
> <details><summary>Same as Spyder Env</summary>
> 
> 
> 
> ```shell
> 
> Last login: Tue Aug 10 21:39:03 on ttys002
> 
> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_6hvyy2_z.sh ; exit;
> 
> 
> 
> The default interactive shell is now zsh.
> 
> To update your account to use zsh, please run `chsh -s /bin/zsh`.
> 
> For more details, please visit https://support.apple.com/kb/HT208050.
> 
> >> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_6hvyy2_z.sh ; exit;
> 
> Could not find platform independent libraries <prefix>
> 
> Could not find platform dependent libraries <exec_prefix>
> 
> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
> 
> Python path configuration:
> 
>   PYTHONHOME = (not set)
> 
>   PYTHONPATH = '/usr/local/mdsplus/pydevices:/Users/rclary/Documents/Python/mypython:/Users/rclary/Documents/Python/c2w:/Users/rclary/Documents/Python/c2w_psp:/Users/rclary/Documents/Python/dap_jscopes:/Users/rclary/Documents/Python/vessel:/Users/rclary/Documents/Python/msdb-csv:/usr/local/mdsplus/python'
> 
>   program name = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
> 
>   isolated = 0
> 
>   environment = 1
> 
>   user site = 1
> 
>   import site = 1
> 
>   sys._base_executable = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
> 
>   sys.base_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
> 
>   sys.base_exec_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
> 
>   sys.platlibdir = 'lib'
> 
>   sys.executable = '/Users/rclary/Applications/Spyder.app/Contents/MacOS/python'
> 
>   sys.prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
> 
>   sys.exec_prefix = '/Users/runner/hostedtoolcache/Python/3.9.5/x64'
> 
>   sys.path = [
> 
>     '/usr/local/mdsplus/pydevices',
> 
>     '/Users/rclary/Documents/Python/mypython',
> 
>     '/Users/rclary/Documents/Python/c2w',
> 
>     '/Users/rclary/Documents/Python/c2w_psp',
> 
>     '/Users/rclary/Documents/Python/dap_jscopes',
> 
>     '/Users/rclary/Documents/Python/vessel',
> 
>     '/Users/rclary/Documents/Python/msdb-csv',
> 
>     '/usr/local/mdsplus/python',
> 
>     '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/python39.zip',
> 
>     '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/python3.9',
> 
>     '/Users/runner/hostedtoolcache/Python/3.9.5/x64/lib/lib-dynload',
> 
>   ]
> 
> Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
> 
> Python runtime state: core initialized
> 
> ModuleNotFoundError: No module named 'encodings'
> 
> 
> 
> Current thread 0x0000000115c5ae00 (most recent call first):
> 
> <no Python frame>
> 
> logout
> 
> Saving session...
> 
> ...copying shared history...
> 
> ...saving history...truncating history files...
> 
> ...completed.
> 
> 
> 
> [Process completed]
> 
> ```
> 
> </details>
> 
> <details><summary>Conda Env</summary>
> 
> 
> 
> ```shell
> 
> Last login: Tue Aug 10 21:40:13 on ttys002
> 
> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_csral3ro.sh ; exit;
> 
> 
> 
> The default interactive shell is now zsh.
> 
> To update your account to use zsh, please run `chsh -s /bin/zsh`.
> 
> For more details, please visit https://support.apple.com/kb/HT208050.
> 
> >> /var/folders/5v/28jqvwxs2cd5fj93gvwnykdrqc926z/T/spyder-rclary/run_spyder_csral3ro.sh ; exit;
> 
> P(A1) = 0.0132; P(A2) = 0.0529; P(A3) = 0.0661; P(A4) = 0.8677; Total = 1.0000
> 
> logout
> 
> Saving session...
> 
> ...copying shared history...
> 
> ...saving history...truncating history files...
> 
> ...completed.
> 
> 
> 
> [Process completed]
> 
> ```
> 
> </details>
> 
> 
> 
> What is interesting to me is that my error shows no `PYTHONHOME` set. I would think that this is the reason for the problem, except that the OP error shows the correct `PYTHONHOME` set.

Thanks for the quick reply @mrclary. For me I'm experiencing the behaviour in a conda env. I'm happy to help you with troubleshooting if there's any logs and things you'd like me to try on my side. :)
> Thanks for the quick reply @mrclary. For me I'm experiencing the behaviour in a conda env. I'm happy to help you with troubleshooting if there's any logs and things you'd like me to try on my side. :)

Do you mean when you set Preferences->Python interpreter to use a Conda environment, you still get the error?

Resolving patch:
diff --git a/spyder/utils/programs.py b/spyder/utils/programs.py
--- a/spyder/utils/programs.py
+++ b/spyder/utils/programs.py
@@ -31,13 +31,12 @@
 
 # Local imports
 from spyder.config.base import (is_stable_version, running_under_pytest,
-                                get_home_dir)
+                                get_home_dir, running_in_mac_app)
 from spyder.config.utils import is_anaconda
 from spyder.py3compat import PY2, is_text_string, to_text_string
 from spyder.utils import encoding
 from spyder.utils.misc import get_python_executable
 
-
 HERE = osp.abspath(osp.dirname(__file__))
 
 
@@ -794,12 +793,14 @@ def run_python_script_in_terminal(fname, wdir, args, interact,
                                         delete=False)
         if wdir:
             f.write('cd {}\n'.format(wdir))
+        if running_in_mac_app(executable):
+            f.write(f'export PYTHONHOME={os.environ["PYTHONPATH"]}\n')
         f.write(' '.join(p_args))
         f.close()
         os.chmod(f.name, 0o777)
 
         def run_terminal_thread():
-            proc = run_shell_command('open -a Terminal.app ' + f.name)
+            proc = run_shell_command('open -a Terminal.app ' + f.name, env={})
             # Prevent race condition
             time.sleep(3)
             proc.wait()


Test patch: