Repository: spyder-ide/spyder

Issue:
Outline doesn't show anything except the file. 
### What steps will reproduce the problem?

<!--- You can use Markdown here --->

The surprising part is that the problem is not always present. Sometimes I do have the classes, function and section (#%%) properly displayed, and I can quickly navigate my files by clicking on the corresponding function in the outline. Sometimes, the outline only shows the name of the file, e.g. loader.py.

For the same file on a different laptop using macOS or windows and spyder 4.2.5 instead of 5.0.0, the outline is working.

I don't see any error message which could help debug this. This post is mostly to raise awareness of this problem and check if anyone else is experiencing the same behavior. 

<details>

```

Package                       Version    Location
----------------------------- ---------- --------------------------------------
alabaster                     0.7.12
appdirs                       1.4.4
apptools                      5.1.0
arrow                         1.0.3
astroid                       2.4.2
async-generator               1.10
atomicwrites                  1.4.0
attrs                         20.3.0
autopep8                      1.5.4
autoreject                    0.2.2
Babel                         2.9.0
backcall                      0.2.0
bcrypt                        3.2.0
binaryornot                   0.4.4
black                         20.8b1
bleach                        3.2.2
certifi                       2020.12.5
cffi                          1.14.4
chardet                       4.0.0
click                         7.1.2
cloudpickle                   1.6.0
colorama                      0.4.4
configobj                     5.0.6
configparser                  5.0.1
cookiecutter                  1.7.2
cryptography                  3.3.1
cycler                        0.10.0
decorator                     4.4.2
defusedxml                    0.6.0
diff-match-patch              20200713
docutils                      0.16
entrypoints                   0.3
envisage                      5.0.0
flake8                        3.8.4
future                        0.18.2
h5py                          3.1.0
helpdev                       0.7.1
idna                          2.10
imageio                       2.9.0
imagesize                     1.2.0
importlib-metadata            3.4.0
importlib-resources           5.1.0
inflection                    0.5.1
intervaltree                  3.1.0
ipykernel                     5.5.3
ipython                       7.22.0
ipython-genutils              0.2.0
isort                         5.7.0
jedi                          0.17.2
Jinja2                        2.11.2
jinja2-time                   0.2.0
joblib                        1.0.0
jsonschema                    3.2.0
jupyter-client                6.1.12
jupyter-core                  4.7.1
jupyterlab-pygments           0.1.2
keyring                       21.8.0
kiwisolver                    1.3.1
lazy-object-proxy             1.4.3
MarkupSafe                    1.1.1
matplotlib                    3.3.3
mayavi                        4.7.2
mccabe                        0.6.1
meshio                        4.3.8
mistune                       0.8.4
mne                           0.22.0
mypy-extensions               0.4.3
nbclient                      0.5.1
nbconvert                     6.0.7
nbformat                      5.1.2
nest-asyncio                  1.4.3
NeuroDecode                   0.9.dev0   c:\users\mathieu\documents\neurodecode
numpy                         1.19.5+mkl
numpydoc                      1.1.0
opencv-python                 4.4.0
packaging                     20.8
pandas                        1.2.1
pandocfilters                 1.4.3
paramiko                      2.7.2
parso                         0.7.0
pathspec                      0.8.1
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        8.1.0
pip                           21.0.1
pluggy                        0.13.1
poyo                          0.5.0
prompt-toolkit                3.0.18
psutil                        5.8.0
ptyprocess                    0.7.0
PyAudio                       0.2.11
pycodestyle                   2.6.0
pycparser                     2.20
pydocstyle                    5.1.1
pyface                        7.2.0
pyflakes                      2.2.0
Pygments                      2.8.1
pylint                        2.6.0
pyls-black                    0.4.6
pyls-spyder                   0.3.2
pylsl                         1.14.0
PyNaCl                        1.4.0
pyparsing                     2.4.7
PyQt5                         5.12.3
PyQt5-sip                     12.8.1
pyqtgraph                     0.11.1
PyQtWebEngine                 5.12.1
pyrsistent                    0.17.3
pyserial                      3.5
python-dateutil               2.8.1
python-jsonrpc-server         0.4.0
python-language-server        0.36.2
python-slugify                4.0.1
pytz                          2020.5
pyvista                       0.27.4
pywin32                       300
pywin32-ctypes                0.2.0
pyxdf                         1.16.3
pyzmq                         22.0.3
QDarkStyle                    3.0.2
qstylizer                     0.1.10
QtAwesome                     1.0.2
qtconsole                     5.0.3
QtPy                          1.9.0
regex                         2020.11.13
requests                      2.25.1
rope                          0.18.0
scikit-learn                  0.24.1
scipy                         1.6.0
scooby                        0.5.6
setuptools                    54.2.0
simplejson                    3.17.2
six                           1.15.0
snowballstemmer               2.1.0
sortedcontainers              2.3.0
sounddevice                   0.4.1
Sphinx                        3.4.3
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        1.0.3
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.4
spyder                        5.0.0
spyder-kernels                2.0.1
testpath                      0.4.4
text-unidecode                1.3
textdistance                  4.2.0
threadpoolctl                 2.1.0
three-merge                   0.1.1
tinycss                       0.4
toml                          0.10.2
tornado                       6.1
traitlets                     5.0.5
traits                        6.1.1
traitsui                      7.1.1
typed-ast                     1.4.2
typing-extensions             3.7.4.3
ujson                         4.0.2
urllib3                       1.26.2
VTK                           9.0.1
watchdog                      1.0.2
wcwidth                       0.2.5
webencodings                  0.5.1
wheel                         0.36.2
wrapt                         1.12.1
xgboost                       1.3.3
xlrd                          2.0.1
XlsxWriter                    1.3.7
yapf                          0.30.0
zipp                          3.4.0
```
</details>

Screenshot of the issue:

![image](https://user-images.githubusercontent.com/73893616/113753576-89131e80-970e-11eb-85a6-b1bcae927cc7.png)

While on a second computer on macOS Big Sur with spyder 4.2.5:

<img width="266" alt="Screenshot 2021-04-06 at 19 32 56" src="https://user-images.githubusercontent.com/73893616/113754085-21a99e80-970f-11eb-9df0-bea5d5ec15e9.png">

The file loader.py is 600+ lines, so I am not going to copy-paste it. The structure should be clear from the second screenshot: a few classes which inherit from Exception, a few sections with functions defined in each.






Discussion:
Hi @MathieuSch-101,

Can you please post a screenshot of the behavior you are having?  Also can you give us a reproducible code for us to reproduce it?

Thanks for reporting!
@steff456 I added the screenshot taken from both my computers on the same file (same git repo). Note that the issue is not raised by this file only, I just chose it as it shows the problem clearly as it contains both sections, functions, and classes. 
I experience the same problem. When the outline disappears and I reset the spyder's setting, restart spyder I get the outline (after displaying the pane). Then I go through the settings, and apply all my changes. The outline is still here. Sometimes after restarting spyder the outline does not show anymore. Could it have to do with some settings?
@sphh I tried your suggestion of reseting to factory default. It didn't change anything for me, still nothing in the outline except the file name.
Strange, for me, it worked every time I did it.

When I reset to factory default, I had to restart spyder, because I have some settings, which require this. I believe, you have restarted spyder as well.
@sphh Yes, it automatically restarts anyway.
@sphh @MathieuSch-101, thanks for the reply, do you know which exact steps can reproduce this issue? Does it occur when Spyder is opened or after some time of running?
Ha, I found out when it occurs!

This is, what I did:
1. Reset to factory defaults
2. Restart
3. Dismiss tour
4. Show outline pane
3. Open an existing file
4. Restart
5. Open an existing project
6. Restart
7. Dismiss Kite installation
8. Restart
9. The outline has gone!
Well, I just tried it again, and the outline is still here. Strange. I don't envy you at all tracking down this problem!!!
@andfoy For my case, on the Windows PC which shows the problem, it is recurrent. Today, during several spyder sessions, it never worked at any point.

It is a bit difficult to recall as I worked during the past month on 3 different computers, but I believe I already saw that issue before today. I do not remember if it was always on this specific computer. However, it does mean I already observed the issue with spyder 4.x versions. 

If you have debugging tools, methods I could use to produce some useful information, please give me guidelines and I will try to help as much as possible. 
One way to reliably switch of the outline is to disable the untick the "Enable Kite Provider" and untick "Enable Language Server Protocol (LSP) provider" (leaving "Enable Fallback provider" ticked).

The funny thing is, even if you untick the LSP, the ouline remains there, but it does not update anymore.

That might have been the reason for me loosing the outline after a restart, because I always unticked Kite and LSP, since I do not want Kite. But I admit, that I did not know, what I do, when I unticked the LSP. Could you please add a tooltip with a description, what those four possible options do?
One more observation: I just lost my outline again, probably due to triggering the issue #15129. I could get it back by going into the preferences and
1. Untick the LSB provider (and the Text snippet provider for good measure)
2. "Apply" the changes.
3. Tick the LSB and Text snippet provider to enable them.
4. "OK" to apply the changes.
5. Restart spyder and the outline is back.

Note: I always have the Kite disabled.
@sphh @MathieuSch-101, could you please start Spyder with debug messages active: ``spyder --debug-output file --debug-info verbose`` (which generates a `spyder-debug.log` file on the pwd of the console that is used to execute spyder), and post the log here when the error happens?
Voilà:

With LSP enabled, showing the outline:
[spyder-debug.log](https://github.com/spyder-ide/spyder/files/6267943/spyder-debug.log)

and here with LSP disabled, not showing the outline:
[spyder-debug-nooutline.log](https://github.com/spyder-ide/spyder/files/6267945/spyder-debug-nooutline.log)

@andfoy It took me a bit longer this morning as I don't usually start spyder with this command but with a shortcut to 'spyder\app\start.py' instead. Some old habit. Anyway, I finally found the debug log file.

During the 2/3 minute sessions, I pressed on the different buttons and options of the outline (which aside from the one showing all files and the one showing the absolute path; are doing absolutely nothing). 

[spyder-debug.log](https://github.com/spyder-ide/spyder/files/6270563/spyder-debug.log)

@andfoy @steff456 One more issue which is most likely linked. When the outline doesn't work, the functionality Ctrl+click on a function name doesn't work. It does not open the file in which the function is located. This is true for both my files and for official libraries, e.g. numpy, pandas, mne. For instance:

```
import mne

epochs = mne.Epochs(raw, events, event_id=event_dict['audio'], picks=['eeg', 'eog', 'ecg'],
                    tmin=tmin, tmax=tmax, reject=None,
                    proj=True, baseline=(None, 0), preload=True)
```

Ctrl+Hovering over Epochs does show the underline; but Ctrl+Click on Epochs does nothing. On the other computer where the outline works, Ctrl+Click does open the file containing the selected object/function and place the cursor at the beginning of the object/function as expected. 

Here is a log where I Ctrl+clicked on a few functions from official libraries: [spyder-debug.log](https://github.com/spyder-ide/spyder/files/6271851/spyder-debug.log)



I've experienced the same - when the LSP starting hangs the Outline just shows filenames and Ctrl + click doesn't work. I did dissmis Kite suggestion when this happened.

Upon restart LSP is ready and outline works as expected.
@gerazov Unticking both 'Enable Kite Prover' and 'Enable Language Server Protocol (LSP) Prover' and restarting spyder doesn't solve the issue on my side.
I have also faced this issue several times. One thing works for me every time is to restart the LSP.

Click on the `LSP Python: ready` on the status bar below and click on `Restart Python Language Server`. Then it works.

This is just a work-around and need a fix
@varadpatil No luck either. restarting the language server does not restore either the outline or the Ctrl+Click on my side.
@MatthieuDartiailh, please run again Spyder in debug mode and once the error appears, close it, go to

    C:\Users\<your user>\.spyder-py3\lsp_logs

and upload here the two log files present there.
@ccordoba12 In case the @ was not targeting the right person, here are the 2 logs after:

1. Opening spyder (a few .py files were open in the editor, the outline was open and not working)
2. import os
3. import mne
4. Ctrl+Click on a few functions from mne in one of the files.

[server_python_288.log](https://github.com/spyder-ide/spyder/files/6291613/server_python_288.log)
[transport_python_288.log](https://github.com/spyder-ide/spyder/files/6291614/transport_python_288.log)
Thanks @MathieuSch-101 for your help. I see some errors in the server log file that could be causing your issues. Just to clarify:

- Are the outline and "go to definition" not working at all? Or do they work for a time and then stop working?
- Could you download and install our [Windows installer](https://github.com/spyder-ide/spyder/releases/download/v5.0.0/Spyder_64bit_full.exe) and try things with it instead? I saw that you're using a custom Python installation and we don't tend to support those because it's very easy to break them.
@andfoy, as you can see in the server logs, we're generating some errors on it by sending several `didChangeConfiguration` requests before we send it the `initialize` one.

That's something that we could easily fix in the Python language server, but not on others. So I'd say we need to fix that on our side.
This could also be related to the fact that the Jedi and Parso versions we depend on don't support Python 3.9:

https://github.com/palantir/python-language-server/issues/898#issuecomment-770437338
@ccordoba12 The outline and the Ctrl+Click do not work at all. However, the other day, I stumbled upon a key combination by mistake which opens a function in the Help window. It was on one of the functions from my files, and the docstring was properly displayed. So I guess, whatever this key combination was, the Help window did find properly the file/function.

I understand that you don't usually support custom Python installation; but as I like to keep things clean on my PC, it is a very simple and minimalist installation. Python 3.9.1 followed by pip upgrade for pip, setuptools and wheels, and finally by py -m pip install spyder which then got all the dependencies as Jedi and Parso. In my case, I have Jedi 0.17.2 and Parso 0.7.0.

To avoid having a double installation of Spyder + files which might be hard to remove, I would prefer to avoid using the Windows installer on this PC. I could however reproduce my pip installation on a second PC, confirm that the outline problem occurs, and try with the window installer. I'll keep you updated tomorrow or Tuesday. 

Finally, I know debugging without being able to reproduce the problem is a pain. If someone is willing to take the time, I could grant access to my PC.
@ccordoba12 On a second PC, with windows working through bootcamp (MacBook Pro..), a similar install with Python 3.9.1 and a slightly outdated spyder version, 4.2.3, the outline and Ctrl+Click are working fine.

[pip list spyder 4.2.3.txt](https://github.com/spyder-ide/spyder/files/6294944/pip.list.spyder.4.2.3.txt)

After updating to spyder 5.0.0 with `py -m pip install --upgrade spyder`, both the outline and the Ctrl+Click are now broken. 

[pip list spyder 5.0.0.txt](https://github.com/spyder-ide/spyder/files/6294943/pip.list.spyder.5.0.0.txt)

I then uninstalled spyder with pip and used the installer. No Dekstop shortcut was created, and I assumed the correct way to start spyder is through the file `Spyder.launch.pyw`. It requested Python v3.7, so I installed Python 3.7.9. Once launched, the outline and the Ctrl+Click are broken as well.
One more point, I also updated spyder to 5.0.0 on the mac OS Big Sur side which has Python 3.8.6. PIP does show some errors which I believe are due to the dependencies typo for spyder-kernels. 

<img width="998" alt="Screenshot 2021-04-12 at 15 06 31" src="https://user-images.githubusercontent.com/73893616/114398945-ab8ab900-9ba0-11eb-877b-fb1215c9f6e8.png">

Spyder does launch fine anyway, and the Outline and Cmd+Click are broken as well.

Also having this problem (outline pane shows .py files but unable to expand). Using Ubuntu 20.04 spyder updated from 4.2.5 to version 5.0 using conda commands suggested in Spyder's update popup window. (None of the advice above regarding checking / unchecking services and restarting works for me). I can try to collect a log as suggested above if that's useful.
![image](https://user-images.githubusercontent.com/2997028/114464054-4096ad80-9bb3-11eb-9467-d2786b93f350.png)
Enabling LSP and restarting Spyder works for me to get the outline back, but functions haven't been showing for me for several months, and I just noticed something about it: 
Functions will show if and only if they're _**at least double nested**_ inside of other functions.  Here's a little demo and the associated outline. 
> a similar install with Python 3.9.1 and a slightly outdated spyder version, 4.2.3

Please try with 4.2.4 and 4.2.5, and let us know if it's still working.

> No Dekstop shortcut was created, and I assumed the correct way to start spyder is through the file Spyder.launch.pyw

It creates an entry in the Windows menu.

> PIP does show some errors which I believe are due to the dependencies typo for spyder-kernels.

And this is why we don't support pip. You're using an old pip version (i.e. < 21) that didn't install the right requirements.
If I remember correctly, I had the same pip error using the very latest pip version. It was easy to sort (@MathieuSch-101):
`pip3 install --update spyder-kernels`
or
`pip install --update spyder-kernels`
(as it said in the error message).

@ccordoba12: If the requirements are automatically generated, the reason might as well be #15091 (just a guess …)
@sphh, no, they aren't. That's an error with the pip solver (or lack of it in versions less than 21).
In which case it is possibly also present in:
```
pip3 --version
pip 21.0.1 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)
```
which was installed on Feb 14, so before upgrading spyder. Just so that you are aware …
@ccordoba12 @sphh Let's not drift from the original issue. I should have upgraded pip first on the macOS side, my bad. The original issue does occur anyway even when using the **official Spyder Windows installer with Python 3.7.9**. 

-------------------------------------

On the windows side, on the laptop I can use as a sandbox, I completely removed any versions of Python and all libraries (PATH, temporary folders, ...) **and Kite**; to try new installation/setups.

1. Python 3.7.9 installed (for all users, C:\Program Files\Python37) followed by Spyder 5.0.0 through the windows installer (for all users, C.\Program Files\Spyder). **The outline is working.**
2. I now add Kite, login, and restart spyder. **The outline is now broken.**

If you perform the same clean installation and steps, this problem is perfectly reproducible. I could repeat this with more up-to-date Python version 3.9.x and a pip installation of spyder, i.e. this is not a problem specific to a custom pip installation.

---------------------------------------

Next, cleanup of everything again and reinstallation of Python 3.7.9 similarly. 

1. Upgrade pip, wheel, setuptools.
2. Install spyder 4.2.5 `py -m pip install spyder==4.2.5`. **The outline is working.**
3. Add Kite, login, and restart spyder. **The outline is working.**
4. Remove Kite and temporary directories.
5. Upgrade to spyder 5.0.0 `py -m pip install --upgrade spyder`. **The outline is working (and prettier than with 4.2.5).**
6. Add Kite, login, and restart spyder. **The outline is now broken.**
7. Without uninstalling Kite, roll back to spyder 4.2.5 `py -m pip uninstall spyder` and `py -m pip install spyder==4.2.5`. **The outline is working.**

For this last step, I didn't do `py -m pip install --force-reinstall spyder==4.2.5` as it raises a PermisionError for a temporary file in AppData.
Ok, thanks a lot for digging deeper into this @MathieuSch-101! So the problem is with Kite and Spyder 5, which narrows down things a lot for us.

@dalthviz, are you able to reproduce that?
@ccordoba12 @MathieuSch-101 I was able to reproduce this with the installer.

@MathieuSch-101, as a workaround, could you disable Kite by unchecking the `Enable Kite Provider` checkbox under `Completion and linting > General > Providers` in the preferences dialog and see if that helps? Also, you can check if that works without restarting but first you need do a modification in the file and save it to update the outline.

@ccordoba12 maybe some sort of race condition between the LSP and Kite providers is causing the response with the outline to get lost and cause the outline to only show the filename?



@dalthviz Unchecking `Enable Kite Provider` does solve both issues: Outline is restored and Ctrl+Click on a function does work. 
Hey, I'm experiencing the same issue... but none of the above suggestions helped... (Spyder 5.0.1, Windows)
(e.g. restarting the LSP or disabling Kite support)

However, I found that for me the problem only appears if I **select a file inside the Project folder**!
Once I copy the very same file out of the project-folder and open it again, the outline appears just as it should.

(this behaviour persists even if I delete and re-create the project)
I initially wasn't able to get the outline pane to function using the above tips. I then "upgraded" in the Anaconda navigator, which sent me back to 5.0.0 - the outline is now working. It's worth noting that before I installed 5.0.1, I had installed 5.0.0 and experienced the same issue. 

Resolving patch:
diff --git a/spyder/plugins/completion/plugin.py b/spyder/plugins/completion/plugin.py
--- a/spyder/plugins/completion/plugin.py
+++ b/spyder/plugins/completion/plugin.py
@@ -874,7 +874,6 @@ def send_request(self, language: str, req_type: str, req: dict):
         providers = self.available_providers_for_language(language.lower())
         slow_provider_count = sum([self.provider_speed[p] for p in providers])
 
-
         # Start the timer on this request
         if req_type in self.AGGREGATE_RESPONSES and slow_provider_count > 2:
             if self.wait_for_ms > 0:
@@ -1046,12 +1045,13 @@ def match_and_reply(self, req_id: int):
         language = request_responses['language'].lower()
         req_type = request_responses['req_type']
 
+        available_providers = self.available_providers_for_language(
+            language)
+        sorted_providers = self.sort_providers_for_request(
+            available_providers, req_type)
+
         if req_type in self.AGGREGATE_RESPONSES:
             # Wait only for the available providers for the given request
-            available_providers = self.available_providers_for_language(
-                language)
-            sorted_providers = self.sort_providers_for_request(
-                available_providers, req_type)
             timed_out = request_responses['timed_out']
             all_returned = all(source in request_responses['sources']
                                for source in sorted_providers)
@@ -1066,7 +1066,14 @@ def match_and_reply(self, req_id: int):
                 if all_returned or any_nonempty:
                     self.skip_and_reply(req_id)
         else:
-            self.skip_and_reply(req_id)
+            # Any empty response will be discarded and the completion
+            # loop will wait for the next non-empty response.
+            # This should fix the scenario where Kite does not have a
+            # response for a non-aggregated request but the LSP does.
+            any_nonempty = any(request_responses['sources'].get(source)
+                               for source in sorted_providers)
+            if any_nonempty:
+                self.skip_and_reply(req_id)
 
     def skip_and_reply(self, req_id: int):
         """


Test patch: