Repository: spyder-ide/spyder

Issue:
KeyboardInterrupt shows up when Spyder starts
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

The IDE pops up but crashes before finishing installation.
I started 'bootstrap reset' without success as well.

I add to 'pip install autoped8 yasf' otherwise there was error in lsp_server_logfile.log. This is not in requirements.txt.

python bootstrap.py debug
Executing Spyder from source checkout
Revision 324ca58c9, Branch: master
01. Patched sys.path with C:\Contrib\spyder
02. PyQt5 is detected, selecting
03. Imported Spyder 4.0.0.dev0
    [Python 3.6.5 64bits, Qt 5.9.3, PyQt5 5.9.2 on Windows]
04. Running Spyder
Bootstrap completed in 00:00:00.8743

### Traceback
```python-traceback
  File "C:\Contrib\spyder\spyder\plugins\editor\lsp\client.py", line 191, in on_msg_received
    @Slot()
KeyboardInterrupt
```

## Versions

* Spyder version: 4.0.0.dev0 324ca58c9
* Python version: 3.6.5
* Qt version: 5.9.3
* PyQt5 version: 5.9.2
* Operating System: Windows 10

### Dependencies

```
pygments >=2.0    :  2.2.0 (OK)
sphinx >=0.6.6    :  1.8.1 (OK)
pyls >=0.19.0     :  0.21.2 (OK)
nbconvert >=4.0   :  5.4.0 (OK)
pandas >=0.13.1   :  0.23.4 (OK)
numpy >=1.7       :  1.15.2 (OK)
sympy >=0.7.3     :  1.3 (OK)
cython >=0.21     :  0.28.5 (OK)
qtconsole >=4.2.0 :  4.4.1 (OK)
IPython >=4.0     :  7.0.1 (OK)
matplotlib >=2.0.0:  3.0.0 (OK)
pylint >=0.25     :  2.1.1 (OK)
```



Discussion:
Maybe it is my installation (here at home through pip) because I've tried other revisions without much success (but many different errors). I really don't know what to do. Thanks.
Strangely after closing the error report, the ipython console is working, the menu as well. But as soon as I press a key in the editor, everything freeze.
Here is the full output of --debug. It seems that rope (0.11.0) & jedi (0.3.1) have issues 


Executing Spyder from source checkout
0x. Switching debug mode on
Revision 324ca58c9, Branch: master
01. Patched sys.path with C:\Contrib\spyder
02. PyQt5 is detected, selecting
03. Imported Spyder 4.0.0.dev0
    [Python 3.6.5 64bits, Qt 5.9.3, PyQt5 5.9.2 on Windows]
04. Running Spyder
Bootstrap completed in 00:00:00.8280
Start of MainWindow constructor
appusermodelid: 0
End of MainWindow constructor
*** Start of MainWindow setup ***
  ..core actions
  ..toolbars
  ..tools
  ..sift?
  ..plugin: internal console
    ..internal console: initializing
Creating LSP Manager...
  ..plugin: working directory
Chargement de l'aide en ligne...
adding panel EdgeLine at floating
refresh_panels
panel EdgeLine installed
adding panel IndentationGuide at floating
refresh_panels
panel IndentationGuide installed
adding panel FoldingPanel at left
refresh_panels
panel FoldingPanel installed
adding panel LineNumberArea at left
refresh_panels
panel LineNumberArea installed
adding panel ClassFunctionDropdown at top
refresh_panels
panel ClassFunctionDropdown installed
adding panel ScrollFlagArea at right
refresh_panels
panel ScrollFlagArea installed
ScrollFlagArea visibility changed
refresh_panels
refresh_panels
adding extension CloseQuotesExtension
ScrollFlagArea visibility changed
refresh_panels
EdgeLine visibility changed
refresh_panels
IndentationGuide visibility changed
refresh_panels
LineNumberArea visibility changed
refresh_panels
ClassFunctionDropdown visibility changed
refresh_panels
refresh_panels
Chargement de l'explorateur de structure...
Chargement de l'éditeur...
Introspection Plugin Loaded: rope
Introspection Plugin Loaded: jedi
Introspection Plugin Loaded: fallback
adding panel EdgeLine at floating
refresh_panels
panel EdgeLine installed
adding panel IndentationGuide at floating
refresh_panels
panel IndentationGuide installed
adding panel FoldingPanel at left
refresh_panels
panel FoldingPanel installed
adding panel LineNumberArea at left
refresh_panels
panel LineNumberArea installed
adding panel ClassFunctionDropdown at top
refresh_panels
panel ClassFunctionDropdown installed
adding panel ScrollFlagArea at right
refresh_panels
panel ScrollFlagArea installed
ScrollFlagArea visibility changed
refresh_panels
refresh_panels
adding extension CloseQuotesExtension
ScrollFlagArea visibility changed
refresh_panels
EdgeLine visibility changed
refresh_panels
IndentationGuide visibility changed
refresh_panels
LineNumberArea visibility changed
refresh_panels
refresh_panels
ClassFunctionDropdown visibility changed
refresh_panels
Connecting signal...
Call
C:\Users\Sylvain\.spyder-py3\temp.py
Starting server: pyls --host 127.0.0.1 --port 2098 --tcp
Launching LSP Client...
  ..widgets
Chargement de l'explorateur d'espace de noms...
Loading figure browser...
Chargement du journal d'historique...
Chargement de la console IPython...
Chargement de l'explorateur de fichiers...

[python] LSP-Client <===
{'id': -1, 'method': 'server_ready', 'params': {}}


[python] LSP-Client ===>
initialize
Chargement de l'explorateur de projet...
Initialized fallback
Loading third-party plugins...
Configuration de la fenêtre principale...
Setting up window...
*** End of MainWindow setup ***
FoldingPanel visibility changed
refresh_panels
refresh_panels
adding panel EdgeLine at floating
refresh_panels
panel EdgeLine installed
adding panel IndentationGuide at floating
refresh_panels
panel IndentationGuide installed
adding panel FoldingPanel at left
refresh_panels
panel FoldingPanel installed
adding panel LineNumberArea at left
refresh_panels
panel LineNumberArea installed
adding panel ClassFunctionDropdown at top
refresh_panels
panel ClassFunctionDropdown installed
adding panel ScrollFlagArea at right
refresh_panels
panel ScrollFlagArea installed
ScrollFlagArea visibility changed
refresh_panels
refresh_panels
adding extension CloseQuotesExtension
ScrollFlagArea visibility changed
refresh_panels
EdgeLine visibility changed
refresh_panels
IndentationGuide visibility changed
refresh_panels
LineNumberArea visibility changed
refresh_panels
ClassFunctionDropdown visibility changed
refresh_panels
refresh_panels
FoldingPanel visibility changed
refresh_panels
refresh_panels
adding panel EdgeLine at floating
refresh_panels
panel EdgeLine installed
adding panel IndentationGuide at floating
refresh_panels
panel IndentationGuide installed
adding panel FoldingPanel at left
refresh_panels
panel FoldingPanel installed
adding panel LineNumberArea at left
refresh_panels
panel LineNumberArea installed
adding panel ClassFunctionDropdown at top
refresh_panels
panel ClassFunctionDropdown installed
adding panel ScrollFlagArea at right
refresh_panels
panel ScrollFlagArea installed
ScrollFlagArea visibility changed
refresh_panels
refresh_panels
adding extension CloseQuotesExtension
EdgeLine visibility changed
refresh_panels
IndentationGuide visibility changed
refresh_panels
FoldingPanel visibility changed
refresh_panels
LineNumberArea visibility changed
refresh_panels
ClassFunctionDropdown visibility changed
refresh_panels
refresh_panels
ScrollFlagArea visibility changed
refresh_panels
EdgeLine visibility changed
refresh_panels
IndentationGuide visibility changed
refresh_panels
LineNumberArea visibility changed
refresh_panels
refresh_panels
ClassFunctionDropdown visibility changed
refresh_panels
Connecting signal...
Call
C:\Users\Sylvain\.spyder-py3\temp.py
0
Traceback (most recent call last):
  File "C:\Contrib\spyder\spyder\plugins\editor\lsp\client.py", line 191, in on_msg_received
    @Slot()
KeyboardInterrupt

[python] LSP-Client <===
{'jsonrpc': '2.0', 'id': 1, 'result': {'capabilities': {'codeActionProvider': True, 'codeLensProvider': {'resolveProvider': False}, 'completionProvider': {'resolveProvider': False, 'triggerCharacters': ['.']}, 'documentFormattingProvider': True, 'documentHighlightProvider': True, 'documentRangeFormattingProvider': True, 'documentSymbolProvider': True, 'definitionProvider': True, 'executeCommandProvider': {'commands': []}, 'hoverProvider': True, 'referencesProvider': True, 'renameProvider': True, 'signatureHelpProvider': {'triggerCharacters': ['(', ',']}, 'textDocumentSync': 2, 'experimental': {}}}}


[python] LSP-Client ===>
workspace/didChangeConfiguration
{'python': {'textDocumentSync': {'openClose': True, 'change': 2, 'willSave': False, 'willSaveWaitUntil': False, 'save': {'includeText': True}}, 'hoverProvider': True, 'completionProvider': {'resolveProvider': False, 'triggerCharacters': ['.']}, 'signatureHelpProvider': {'triggerCharacters': ['(', ',']}, 'definitionProvider': True, 'referencesProvider': True, 'documentHighlightProvider': True, 'documentSymbolProvider': True, 'workspaceSymbolProvider': False, 'codeActionProvider': True, 'codeLensProvider': {'resolveProvider': False}, 'documentFormattingProvider': True, 'documentRangeFormattingProvider': True, 'documentOnTypeFormattingProvider': {'firstTriggerCharacter': None, 'moreTriggerCharacter': []}, 'renameProvider': True, 'documentLinkProvider': {'resolveProvider': False}, 'executeCommandProvider': {'commands': []}, 'experimental': {}}}
pythonpython
textDocument/didOpen

[python] LSP-Client ===>
textDocument/didOpen
Errored rope
b''
b'Traceback (most recent call last):\r\n  File "C:\\Program Files\\Python36\\lib\\genericpath.py", line 30, in isfile\r\n    st = os.stat(path)\r\nFileNotFoundError: [WinError 3] Le chemin d\x92acc\xe8s sp\xe9cifi\xe9 est introuvable: \'C:\\\\Program Files\\\\Python36\\\\python36.zip\\\\scipy.special.tests.test_precompute_gammainc\\\\__init__.py\'\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File "plugin_server.py", line 127, in <module>\r\n    plugin = PluginServer(*args)\r\n  File "plugin_server.py", line 27, in __init__\r\n    self.object = self.initialize(*args)\r\n  File "plugin_server.py", line 118, in initialize\r\n    plugin.load_plugin()\r\n  File "C:\\Contrib\\spyder\\spyder\\utils\\introspection\\rope_plugin.py", line 70, in load_plugin\r\n    imp.find_module(submod)\r\n  File "C:\\Program Files\\Python36\\lib\\imp.py", line 286, in find_module\r\n    if os.path.isfile(file_path):\r\n  File "C:\\Program Files\\Python36\\lib\\genericpath.py", line 30, in isfile\r\n    st = os.stat(path)\r\nKeyboardInterrupt\r\n'
Errored jedi
b''
b'Traceback (most recent call last):\r\n  File "plugin_server.py", line 127, in <module>\r\n    plugin = PluginServer(*args)\r\n  File "plugin_server.py", line 27, in __init__\r\n    self.object = self.initialize(*args)\r\n  File "plugin_server.py", line 118, in initialize\r\n    plugin.load_plugin()\r\n  File "C:\\Contrib\\spyder\\spyder\\utils\\introspection\\jedi_plugin.py", line 47, in load_plugin\r\n    jedi.preload_module(lib)\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\__init__.py", line 494, in preload_module\r\n    Script(s, 1, len(s), None).completions()\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\__init__.py", line 179, in completions\r\n    completions = completion.completions()\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\completion.py", line 103, in completions\r\n    return sorted(completions, key=lambda x: (x.name.startswith(\'__\'),\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\completion.py", line 41, in filter_names\r\n    len(like_name)\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\classes.py", line 388, in __init__\r\n    super(Completion, self).__init__(evaluator, name)\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\api\\classes.py", line 70, in __init__\r\n    self.module_path = self._module.py__file__()\r\n  File "C:\\Program Files\\Python36\\lib\\site-packages\\jedi\\evaluate\\context\\module.py", line 134, in py__file__\r\n    return os.path.abspath(self._path)\r\n  File "C:\\Program Files\\Python36\\lib\\ntpath.py", line 552, in abspath\r\n    return normpath(path)\r\n  File "C:\\Program Files\\Python36\\lib\\ntpath.py", line 487, in normpath\r\n    if path.startswith(special_prefixes):\r\nKeyboardInterrupt\r\n'
textDocument/didChange

[python] LSP-Client ===>
textDocument/didChange
Please revert to the Rope 0.10.7 and Jedi 0.12.1 and try again.
No it does not work, same issue. Thanks for the try.
This looks like an issue with the old completion architecture, but only on Windows because I can't reproduce it on Linux.

It should be fixed by PR #8004.
@sylvain-bougnoux, please try again with our latest master.
It does not!
But actually it sounds more subtle as I can test 2 pcs (home & work). Work PC is fine, while home PC is not. I've obviously checked that I have exactly the same packages, I do.
Very strange.
Could it be a bug with the video board?
Eventually I found a version (about 2 months ago) that works on my home PC  - therefore it seems it is code issue - but which one ?

I will check the path.

What about this error ?
`[12344:15568:1011/202209.266:ERROR:service_manager_context.cc(252)] Attempting to run unsupported native service: C:\Program Files\Python36\lib\site-packages\PyQt5\Qt\bin\content_renderer.service.exe`

I've tried to reinstall PyQt5 but no success. Hum. 
> What about this error ?

It definitely seems an error in Qt/QWebEngine:

https://github.com/pyinstaller/pyinstaller/issues/3652

so it's probably related to your video card drivers.

What is your PyQt5 version?
Last one 5.11.3, but I've tried with others, same issue with spyder (maybe not with **this** error).

I did `pip uninstall PtQt5` then reinstall. Is there a better way, such as removing the cache, clean the folder by hand... ?
Please downgrade to 5.9.2 and try again.
@sylvain-bougnoux What is you version of `python-language-server`?

I cannot manage to make introspection works if the version is higher than 0.19, either on my pip or Anaconda install on Windows 10 and python 3.6.
Downgrading python-language-server to 0.19.0 works (whatever PyQt5), merci JS. 
FYI
@ccordoba12 still crash with PyQt5=5.9.2
@jnsebgosselin python-language-server was 0.21.2, **but at work it works, very strange**.
@sylvain-bougnoux Glad it worked, because this means I am not alone with this problem! I think I would like to leave this opened until we can find a fix to be able to use the latest `python-language-server` or that we understand better why it is not working. 

>  @jnsebgosselin python-language-server was 0.21.2, but at work it works, very strange.

I think this means this is not going to be easy to fix... I'm almost hoping that it doesn't work on @ccordoba12's Windows box too lol

@ccordoba12 Do you think there is a link with those 2 tests we have skipped on AppVeyor in PR https://github.com/spyder-ide/spyder/pull/7792? Maybe the error were legit after all?
I'm asking about PyQt version because when I install the `python-language-server`, it force a downgrade to PyQt 5.6. But I just tried using this version of PyQt and introspection doesn't kick off either with `python-language-server>0.19`.

@sylvain-bougnoux At works, Spyder doesn't crash, but does introspection works?
Here is what I did:
- Reinstalled Anaconda.
- Made a fresh new Python 3.7 env.
- Installed Spyder
- conda install -c conda-forge python-language-server
- Upgraded PyQt to 5.9.2
- Added my cloned spyder repo to my Windows PYTHONPATH
- Added my new env and env/Script to my Windows PATH
- Removed my pip install from m y Windows PATH
- Started Spyder in debug mode

Then, I can see that pyls kicks of quite rapidly, but then, Spyder becomes irresponsible and I need to hard kill it. I also get this traceback:

```python
Traceback (most recent call last):
  File "C:\Users\User\spyder\spyder\utils\syntaxhighlighters.py", line 227, in highlightBlock
    self.highlight_block(text)
  File "C:\Users\User\spyder\spyder\utils\syntaxhighlighters.py", line 524, in highlight_block
    match = self.PROG.search(text, match.end())
KeyboardInterrupt
Connecting signal...
```
Then,
- Reverting to `conda install -c conda-forge python-language-server==0.19`
- Upgrade PyQt to 5.9.2 again

Makes everything works fine again


Interesting fact:  Spyder, Ipython console, and pyls start waaaaay faster on my Anaconda install than on my pip install. Both are using `python-language-server==0.19`.
This seems a problem with the latest pyls:

https://github.com/palantir/python-language-server/issues/434

and that's why using older versions makes it work again.
> At works, Spyder doesn't crash, but does introspection works?

@jnsebgosselin Good point, actually **it does not**. Downgrading to 0.19.0 does not make it work either. I must have missed an installation step?
> 
> 
> > At works, Spyder doesn't crash, but does introspection works?
> 
> @jnsebgosselin Good point, actually **it does not**. Downgrading to 0.19.0 does not make it work either. I must have missed an installation step?

You need to make sure, if you restarted Spyder, that there is no background processes running in the Windows task manager, or else, introspection won't work. See Issue #7796.
@jnsebgosselin hum, when I close spyder all Python instance disappear. But when I relaunch it I got 4 instances (see below), with a mix of 32 & 64 version (!). Does introspection should work then? For me it does not, whatever the version (0.19 or 0.21.2).

![taskmgr_2018-10-15_16-52-08](https://user-images.githubusercontent.com/6706662/46958685-b2e9ae00-d09a-11e8-84cc-6484d0455d94.png)


I think it should work...

The problem is when there is a background process remaining in the `Background processes` section, not the `Apps` section. This happens for me after closing Spyder. When I restart Spyder afterward if one background process remains, introspection is broken.

Also, if you are running on master, be sure to get the latest version, because a lot of things were fixed in the latest PR.

Try starting Spyder in debug mode. Sometimes, it is working better, but I do not know why. Also this allow  `pyls.exe` to close correctly, so that there is no remaining processes that block further access to it afterward.

Finally, make sure that `pyls` is pulling the right version of the `pyls.exe`. If you have more than one Python installed on your machine, it can pull the wrong one. Check out your Windows `Path` to make sure.
On a site note, introspection is working a lot better on my Anaconda install than on my pip install... It is not even comparable. I do not know why.
I've checked 'Background processes' as well (& window processes) no python instance.

**But** I updated master (I was 8 days behind). **It crashed.** 
I downgraded pls to 0.19.0. It does not crashed and pls is working... Which is overall what was expected.

Currently
- Closing spyder does not make python left over.
- Launching is much faster.
- I have no idea if pls is fully working. At least pop up & completion works for some symbols.
- `pycodestyle `became incredibly verbose.

This is good news!

> - Closing spyder does not make python left over.
> - Launching is much faster.

Excellent, this seems to reinforce the hypotheses that the issue I've reported in #7796 is specific to my installation.

> - I have no idea if pls is fully working. At least pop up & completion works for some symbols.

My personal experience with this since the new PLS has been implemented is that this is a on or off issue. It works or it doesn't. But I may be wrong.

> - pycodestylebecame incredibly verbose.

I think @ccordoba12  is working to fix this in PR spyder-ide/spyder#7734
We need to disable `pycodestyle` and only leave `pyflakes`. Those were and will continue to be our defaults. However, that's not going to be addressed in #7734 but in a future PR.
The easy way is to configure `pycodestyle `ourselves as described in 
http://pycodestyle.pycqa.org/en/latest/intro.html
Personnally I created `.pycodestyle` with

```
[pycodestyle]
ignore = E111,E114,E225,E226,E228,E231,E241,E261,E262,E265,E302,E305,E401,E501,E701
```
This could be created by spyder, possibly with optional ignore.
I can confirm the problem specifically starts with 0.21.0 on Windows, since 0.20 is unaffected as well in my testing while 0.21.0 produced the crash on startup. I also observed the ``pip`` install and likely background processes reported by @jnsebgosselin and provide more detail over in that thread and #8153 .
On my laptop, with 0.21.0, Spyder simply crashes. With 0.20, it doesn't, but I can't get introspection to work... only with 0.19 am I able to make this work, either with my pip or Anaconda install.
I don't believe I ever got it working on 0.20, but I'm not 100% sure. The problem on 0.19.0, when it occurred, outwardly appeared the same as on 0.20 (no introspection/completion at all on startup), but it only happened some of the time even in the same env, versions and even console window, particularly after I'd just upgraded/downgraded/reinstalled pyls with ``pip``. I haven't tried it with Anaconda to avoid the overhead of a whole other Anaconda env on my relatively modest-sized SSD Anaconda's installed on.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -226,7 +226,7 @@ def run(self):
     # Python 3
     'pyqt5<=5.12;python_version>="3"',
     # Pyls with all its dependencies
-    'python-language-server[all]==0.19.0',
+    'python-language-server[all]>=0.19.0',
     # Required to get SSH connections to remote kernels
     'pexpect;platform_system!="Windows"',
     'paramiko;platform_system=="Windows"'
diff --git a/spyder/plugins/editor/lsp/transport/consumer.py b/spyder/plugins/editor/lsp/transport/consumer.py
--- a/spyder/plugins/editor/lsp/transport/consumer.py
+++ b/spyder/plugins/editor/lsp/transport/consumer.py
@@ -73,7 +73,7 @@ def expect_windows(self):
         continue_reading = True
         while continue_reading:
             try:
-                buffer += self.socket.recv(1024)
+                buffer += self.socket.recv(1)
                 if b'\r\n\r\n' in buffer:
                     split = buffer.split(b'\r\n\r\n')
                     if len(split) == 2:


Test patch: