Repository: spyder-ide/spyder

Issue:
MacOS standalone not working due to pygments error
Seems like although the action building and testing the MacOS installer passes there is an error with the generated app. Looking the action log you can see a traceback related with `pygments` (from action execution: https://github.com/spyder-ide/spyder/runs/8079765883?check_suite_focus=true#step:11:17):

```
Run ./test_app.sh -t 60 -d 10 ${DISTDIR}
Traceback (most recent call last):
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/__boot__.py", line 251, in <module>
    _run()
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/__boot__.py", line 174, in _run
    exec(compile(source, path, "exec"), globals(), globals())
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/spyder", line 3, in <module>
    start.main()
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/app/start.py", line 128, in main
    from spyder.config.manager import CONF
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/config/manager.py", line 22, in <module>
    from spyder.config.main import CONF_VERSION, DEFAULTS, NAME_MAP
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/config/main.py", line 21, in <module>
    from spyder.config.appearance import APPEARANCE
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/config/appearance.py", line 15, in <module>
    from spyder.plugins.help.utils.sphinxify import CSS_PATH
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/spyder/plugins/help/utils/sphinxify.py", line 36, in <module>
    from sphinx.application import Sphinx
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/sphinx/application.py", line 32, in <module>
    from sphinx.highlighting import lexer_classes
  File "/Users/runner/work/spyder/spyder/dist/Spyder.app/Contents/Resources/lib/python3.9/sphinx/highlighting.py", line 12, in <module>
    from pygments.formatters import HtmlFormatter, LatexFormatter
  File "pygments/formatters/__init__.pyc", line 132, in __getattr__
  File "pygments/formatters/__init__.pyc", line 28, in _load_formatters
ModuleNotFoundError: No module named 'pygments.formatters.latex'
2022-08-29 21:14:56.679 Spyder[36541:97505] Launch error
2022-08-29 21:14:56.680 Spyder[36541:97505] Launch error
See the py2app website for debugging launch issues

Spyder launched successfully!
Shutting down Spyder...

./test_app.sh: line 72: 36541 Terminated: 15          $1/Spyder.app/Contents/MacOS/Spyder

Spyder shut down successfully in 0s

```

And when launching the installed app you see an error dialog from py2app (I checked this using MacOS 11.6.8 Big Sur):

![imagen](https://user-images.githubusercontent.com/16781833/187513812-c2c6f1be-96ef-4ec0-a739-762696c986e7.png)


Checking seems like `pygments` released version 2.13.0 a couple of weeks ago and PRs which GitHub actions checks runs are older than two weeks ago seem like where able to properly pass the MacOS installer validation test. Could then the new pygments release be the cause of the issue?

What do you think @mrclary ?

Also, I think someone reported something similar on the google group, right @ccordoba12 ?


Discussion:
I confirm Spyder standalone don't run on Monterey either, and obviously, you can mark my own bug report #19237 as a duplicate of this one !
> Also, I think someone reported something similar on the google group, right @ccordoba12 ?

Correct. This seems caused by a an incompatibility between Sphinx and Pygments. I think for now the simplest thing we can do to solve it is to add an upper constraint to Pygments (2.13 was released last August 15th, so that should be the version that introduced this issue).

@mrclary, could you take care of this?
@ccordoba12 should we, for the moment, remove the Spyder.dmg and the link to download it from the Spyder 5.3.3 release page?
Ok, I think that's a good idea given that this is a hard crash, so it'll give a very bad impression to new users.

Please also leave a note in the 5.3.3 release entry saying that the Mac app has a critical error but we'll try to upload it shortly (you can reference this issue if you want too).

Also, you need to check that the 5.3.2 dmg can still be downloaded from our main webpage (i.e. spyder.org).
> I think for now the simplest thing we can do to solve it is to add an upper constraint to Pygments

About this error, I think it's in our side because I couldn't find a similar one in the Sphinx repo. So maybe we're not packaging all Pygments lexers somehow?
> @mrclary, could you take care of this?

If there is not a problem with the Windows or conda installs, then I hesitate to restrict the pygments version because it may be that only the macOS installer (py2app) didn't capture `pygments.formatters.latex`.

I'll take a look. If that is the issue, I can patch that and upload a new artifact (built locally) for 5.3.3. Then we can decide how best to proceed, either a patch or PR to py2app or both.
Just in case, checked on Windows and both standalone and conda version start normally :+1:
So it was definitely py2app not capturing all the components of the package. The quick fix is to force py2app to include the entire package. I'll upload new artifacts shortly.

The long-term fix for this would be to submit a PR for a pygments recipe for py2app. However, given the imminent conda-based installer, I'll forgo that and just submit a PR here for this patch.
Another concern is why the app test erroneously declares that Spyder launched successfully...

Resolving patch:
diff --git a/installers/macOS/packages.py b/installers/macOS/packages.py
--- a/installers/macOS/packages.py
+++ b/installers/macOS/packages.py
@@ -20,6 +20,8 @@
     ImportError: The 'more_itertools' package is required; normally this is
     bundled with this package so if you get this warning, consult the
     packager of your distribution.
+pygments:
+    ModuleNotFoundError: No module named 'pygments.formatters.latex'
 pyls_spyder :
     Mandatory: pyls_spyder >=0.1.1 : None (NOK)
 pylsp_black :
@@ -39,6 +41,7 @@
 PACKAGES = [
     'humanfriendly',
     'pkg_resources',
+    'pygments',
     'pyls_spyder',
     'pylsp_black',
     'setuptools',


Test patch: