Repository: spyder-ide/spyder

Issue:
Problem in autocompletion of class attributes in Spyder 4 editor 
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [x] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [x] Reset preferences with ``spyder --reset``
    * [x] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [x] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

On Spyder 3.6.6, it was possible to watch the attributes and methods of a loaded object in the editor (thanks to rope and jedi). With python 4.0.0, it is not possible to visualize them. I'm not sure if this is a problem from Kite App or another change in the Spyder 4.0.0 new **Completion and linting** functionalities. I tried several configurations (by changing Completion and Linting parameters) to retrieve my previous Spyder user experience but without success. I don't see what's the problem here.

Note : I'm aware and tried to configure   _Adding libraries from PYTHONPATH to the Kite index_ . But it does not seem to be the problem.


### What steps reproduce the problem?

Simply create file myclass.py with a class with one attribute:

```python
class MyClass():
    def __init__(self,param):
        self.param=param
```

Then import myclass and create an object A with param=10, for instance:
```python
import myclass
A=myclass.MyClass(10)
A.
```

### What is the expected output? What do you see instead?
On Spyder 4.0.0 editor autocompletion after the dot shows 'ArithmeticError; AssertionError;...' while
in Spyder 3.6 editor autocompletion suggested my usual methods and attribute loaded in My_Class.
Here _My_Class.param_.

PS: I did not produce a screenshot since autocompletion suggestions disappear when **prt scr** 


## Versions

* Spyder version: 3.3.6 and 4.0.0
* Python version: 3.7.3
* Qt version: 5.9.7
* PyQt version: 5.9.2
* Operating System name/version: Ubuntu 18.04.3

## Dependencies
cloudpickle >=0.5.0          :  1.0.0 (OK)
pygments >=2.0               :  2.4.0 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
nbconvert >=4.0              :  5.5.0 (OK)
sphinx >=0.6.6               :  2.0.1 (OK)
pylint >=0.25                :  2.3.1 (OK)
psutil >=0.3                 :  5.6.2 (OK)
qtawesome >=0.5.7            :  0.5.7 (OK)
qtpy >=1.5.0                 :  1.7.1 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
zmq >=17                     :  18.0.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
numpydoc >=0.6.0             :  0.9.1 (OK)
spyder_kernels >=1.8.1;<2.0.0:  1.8.1 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
intervaltree                 :  None (OK)
watchdog                     :  None (OK)
keyring                      :  None (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pympler                      :  None (OK)
sympy >=0.7.3                :  1.4 (OK)
cython >=0.21                :  None (NOK)
IPython >=4.0                :  7.5.0 (OK)
matplotlib >=2.0.0           :  3.1.0rc2 (OK)
pandas >=0.13.1              :  None (NOK)
numpy >=1.7                  :  1.16.3 (OK)
scipy >=0.17.0               :  1.2.1 (OK)
pyls >=0.31.2;<0.32.0        :  0.31.2 (OK)
xdg >=0.26                   :  0.26 (OK)
rtree >=0.8.3                :  0.9.3 (OK)


Discussion:
I can reproduce this on latest master if I don't save the new file. However, If I save the file in the same directory as myclass.py then the the completions are returned as expected.

@groussea Are you saving the file in which you import `myclass.py`?

I tried on Master and with a saved file. I get a similar result. 
After some trial/error tests, I deactivate **code snippets** and **fallback completions**.
It is difficult to figure out what are exactly the effects of these option on the completion. Fallback completion seems to be the most undesirable effect even for saved files in PYTHONPATH or working directory.
I presume this effect is significant for users working with local functions and modules.
In issue #10797 @ccordoba12 said:

>@andfoy is going to change the current behavior, so fallback completions are requested only on demand. That should fix most if not all of these annoying issues.

This would help here as well. I like fallback completions but they shouldn't be offered unless explicitly asked for. 

Agreed. They are very annoying but can be very useful if requested on demand.
I can confirm this problem still exist in 4.1.3.
A minimal example would be:
```python
d = {}
d.
```
even when requesting manually completion nothing happens.
the later version of spyder 4.1.3 is absolute garbage. i am thankful i started with spyder 3. something or probabaly would have given up on Python itself. why in the world does autocomplete stop right away? in spyder 3.X it would be a problem if spyder was left open for a long time untouched. A simple rstart of spyder would solve the issue. now.. it just stops like deer in car lights. neither do the error highlighters work. This is making life unnecessarily hard. i am going to go back to spyder3 . I just wanted to vent since i see this problem has been here for a while and no one seems to be doing anything towards it.
oh and.. when opening a variable or a dataframe from the variable explorer, the some other variable opens.. what in the name of everything that's beautiful, i s the point of a variable explorer where i cannot explore the variable..or did you just mean variable discoverer?
to reproduce these error just open any excel workbook. nothing really has to be done  to reproduce the errors.  like magic, they will appear. and the error highlighters wont go away even after fixing the error in the code. 
i apologize if i seem to harsh. I love working with spyder but ever since v4 has come, my life has become a hell. please fix this.
@venkatgmrit05, please don't post random comments about your frustrations with Spyder 4 in other people's issues. That's simply not polite.

About the issues you're having with code completion and linting, many people have reported the same problem but we couldn't find a way to reproduce it until very recently (a bit more than a month ago). So we didn't know how to fix it.

After that, we've put a ton of effort on making that part of our codebase more robust and error proof, so (hopefully) those problems will be much less frequent after we release version 4.1.4 next week.

I didn't understand your problems with the Variable Explorer, so please open a new issue about them and don't forget to upload screenshots or (better yet) animated gifs (that you can create with the Licecap program) for us to be able to reproduce them.
Random variable opening is being tracked here: #13105

@venkatgmrit05 If you can add something reproducible to that issue that would be helpful.
> Random variable opening is being tracked here: #13105

Thanks @bcolsen! I'll try to merge PR #10843 (that should fix that problem) for 4.1.4 too.
I agree with you. I do apologize.  Thank you for working on the fix. I am
looking forward to it. I appreciate the work you guys do on Spyder.
Regards,
Uma Venkat, Karanam
MSME, SUNY Buffalo
+16476793681
 Linkedin: https://www.linkedin.com/in/umavenkat-karanam-0568341b/


On Thu, Jun 25, 2020 at 5:22 PM Carlos Cordoba <notifications@github.com>
wrote:

> @venkatgmrit05 <https://github.com/venkatgmrit05>, please don't post
> random comments about your frustrations with Spyder 4 in other people's
> issues. That's simply not polite.
>
> About the issues you're having with code completion and linting, many
> people have reported the same problem but we couldn't find a way to
> reproduce it until very recently (a bit more than a month). So we didn't
> know how to fix it.
>
> After that, we've put a ton of effort on making that part of our codebase
> more robust and error proof, so (hopefully) those problems will be much
> less frequent after we release version 4.1.4 next week.
>
> I didn't understand your problems with the Variable Explorer, so please
> open a new issue about them and don't forget to upload screenshots or
> (better yet) animated gifs (that you can create with the Licecap program)
> for us to be able to reproduce them.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/spyder-ide/spyder/issues/11118#issuecomment-649824323>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACEMZTPAK5RETZDUAO7BK2LRYO5Y5ANCNFSM4J32CYLQ>
> .
>


Resolving patch:
diff --git a/external-deps/python-language-server/pyls/plugins/pycodestyle_lint.py b/external-deps/python-language-server/pyls/plugins/pycodestyle_lint.py
--- a/external-deps/python-language-server/pyls/plugins/pycodestyle_lint.py
+++ b/external-deps/python-language-server/pyls/plugins/pycodestyle_lint.py
@@ -78,5 +78,13 @@ def error(self, line_number, offset, text, check):
             'message': text,
             'code': code,
             # Are style errors really ever errors?
-            'severity': lsp.DiagnosticSeverity.Warning
+            'severity': _get_severity(code)
         })
+
+
+def _get_severity(code):
+    # Are style errors ever really errors?
+    if code[0] == 'E' or code[0] == 'W':
+        return lsp.DiagnosticSeverity.Warning
+    # If no severity is specified, why wouldn't this be informational only?
+    return lsp.DiagnosticSeverity.Information
diff --git a/external-deps/python-language-server/pyls/workspace.py b/external-deps/python-language-server/pyls/workspace.py
--- a/external-deps/python-language-server/pyls/workspace.py
+++ b/external-deps/python-language-server/pyls/workspace.py
@@ -258,7 +258,7 @@ def jedi_script(self, position=None):
         env_vars.pop('PYTHONPATH', None)
 
         environment = self.get_enviroment(environment_path, env_vars=env_vars) if environment_path else None
-        sys_path = self.sys_path(environment_path, env_vars=env_vars) + extra_paths
+        sys_path = self.sys_path(environment_path, env_vars=env_vars) + extra_paths + [os.path.dirname(self.path)]
         project_path = self._workspace.root_path
 
         kwargs = {


Test patch:
diff --git a/external-deps/python-language-server/test/fixtures.py b/external-deps/python-language-server/test/fixtures.py
--- a/external-deps/python-language-server/test/fixtures.py
+++ b/external-deps/python-language-server/test/fixtures.py
@@ -44,6 +44,15 @@ def workspace(tmpdir):
     return ws
 
 
+@pytest.fixture
+def workspace_other_root_path(tmpdir):
+    """Return a workspace with a root_path other than tmpdir."""
+    ws_path = str(tmpdir.mkdir('test123').mkdir('test456'))
+    ws = Workspace(uris.from_fs_path(ws_path), Mock())
+    ws._config = Config(ws.root_uri, {}, 0, {})
+    return ws
+
+
 @pytest.fixture
 def config(workspace):  # pylint: disable=redefined-outer-name
     """Return a config object."""
diff --git a/external-deps/python-language-server/test/plugins/test_completion.py b/external-deps/python-language-server/test/plugins/test_completion.py
--- a/external-deps/python-language-server/test/plugins/test_completion.py
+++ b/external-deps/python-language-server/test/plugins/test_completion.py
@@ -334,3 +334,26 @@ def test_jedi_completion_environment(workspace):
     completions = pyls_jedi_completions(doc._config, doc, com_position)
     assert completions[0]['label'] == 'loghub'
     assert 'changelog generator' in completions[0]['documentation'].lower()
+
+
+def test_document_path_completions(tmpdir, workspace_other_root_path):
+    # Create a dummy module out of the workspace's root_path and try to get
+    # completions for it in another file placed next to it.
+    module_content = '''
+def foo():
+    pass
+'''
+
+    p = tmpdir.join("mymodule.py")
+    p.write(module_content)
+
+    # Content of doc to test completion
+    doc_content = """import mymodule
+mymodule.f"""
+    doc_path = str(tmpdir) + os.path.sep + 'myfile.py'
+    doc_uri = uris.from_fs_path(doc_path)
+    doc = Document(doc_uri, workspace_other_root_path, doc_content)
+
+    com_position = {'line': 1, 'character': 10}
+    completions = pyls_jedi_completions(doc._config, doc, com_position)
+    assert completions[0]['label'] == 'foo()'