Repository: spyder-ide/spyder
Pull request: PR: Fix underscore completions
<!--- Make sure to read the Contributing Guidelines:                   --->
<!--- https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md --->
<!--- and follow PEP 8, PEP 257 and Spyder's code style:               --->
<!--- https://github.com/spyder-ide/spyder/wiki/Dev:-Coding-Style      --->

## Description of Changes


* [x] Wrote at least one-line docstrings (for any new functions)
* [x] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->


<!--- Explain what you've done and why --->




### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes #10528


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct: @goanpeca 
<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
Completions are not working for underscore variables
If a method or variable starts with `_`, completions are not working:

```python
def _test1():
    pass


_test2 = 2
__test3 = 1

_*
```

Press `<tab>` on the cursor (*) and none of `_test1`, `_test2` `__test3` appear.

Hints from issue discussion:
It was working on v4.0.0b4

<img width="370" alt="Screen Shot 2019-10-29 at 07 10 29" src="https://user-images.githubusercontent.com/3627835/67765917-47f7e480-fa1b-11e9-8ae5-02c74187e743.png">

Please take a look at this @goanpeca. Most probably the sorting algorithm for completion entries is now dropping underscores.
Will do.

Commit message: Check code and add test for underscore completions

Patch to review:
diff --git a/spyder/plugins/editor/widgets/completion.py b/spyder/plugins/editor/widgets/completion.py
index 40969ba3fbc..63304b64610 100644
--- a/spyder/plugins/editor/widgets/completion.py
+++ b/spyder/plugins/editor/widgets/completion.py
@@ -30,25 +30,26 @@
 
 class CompletionWidget(QListWidget):
     """Completion list widget."""
-    ITEM_TYPE_MAP = {CompletionItemKind.TEXT: 'text',
-                     CompletionItemKind.METHOD: 'method',
-                     CompletionItemKind.FUNCTION: 'function',
-                     CompletionItemKind.CONSTRUCTOR: 'constructor',
-                     CompletionItemKind.FIELD: 'field',
-                     CompletionItemKind.VARIABLE: 'variable',
-                     CompletionItemKind.CLASS: 'class',
-                     CompletionItemKind.INTERFACE: 'interface',
-                     CompletionItemKind.MODULE: 'module',
-                     CompletionItemKind.PROPERTY: 'property',
-                     CompletionItemKind.UNIT: 'unit',
-                     CompletionItemKind.VALUE: 'value',
-                     CompletionItemKind.ENUM: 'enum',
-                     CompletionItemKind.KEYWORD: 'keyword',
-                     CompletionItemKind.SNIPPET: 'snippet',
-                     CompletionItemKind.COLOR: 'color',
-                     CompletionItemKind.FILE: 'filenew',
-                     CompletionItemKind.REFERENCE: 'reference',
-                     }
+    ITEM_TYPE_MAP = {
+        CompletionItemKind.TEXT: 'text',
+        CompletionItemKind.METHOD: 'method',
+        CompletionItemKind.FUNCTION: 'function',
+        CompletionItemKind.CONSTRUCTOR: 'constructor',
+        CompletionItemKind.FIELD: 'field',
+        CompletionItemKind.VARIABLE: 'variable',
+        CompletionItemKind.CLASS: 'class',
+        CompletionItemKind.INTERFACE: 'interface',
+        CompletionItemKind.MODULE: 'module',
+        CompletionItemKind.PROPERTY: 'property',
+        CompletionItemKind.UNIT: 'unit',
+        CompletionItemKind.VALUE: 'value',
+        CompletionItemKind.ENUM: 'enum',
+        CompletionItemKind.KEYWORD: 'keyword',
+        CompletionItemKind.SNIPPET: 'snippet',
+        CompletionItemKind.COLOR: 'color',
+        CompletionItemKind.FILE: 'filenew',
+        CompletionItemKind.REFERENCE: 'reference',
+        }
     ICON_MAP = {}
 
     sig_show_completions = Signal(object)
@@ -120,6 +121,7 @@ def show_list(self, completion_list, position, automatic):
         if not isinstance(completion_list[0], dict):
             self.is_internal_console = True
         self.completion_list = completion_list
+
         # Check everything is in order
         self.update_current(new=True)
 
@@ -179,7 +181,7 @@ def update_list(self, current_word, new=True):
             else:
                 completion_label = completion[0]
 
-            if not self.check_can_complete(completion_label, current_word):
+            if not self.check_can_complete(completion_label, current_word, completion):
                 continue
             item = QListWidgetItem()
 
@@ -347,14 +349,25 @@ def is_up_to_date(self, item=None):
             filter_text = completion
         return self.check_can_complete(filter_text, current_word)
 
-    def check_can_complete(self, filter_text, current_word):
+    def check_can_complete(self, filter_text, current_word, params=None):
         """Check if current_word matches filter_text."""
         if not filter_text:
             return True
+
         if not current_word:
             return True
-        if not filter_text[0].isalpha():
-            current_word = filter_text[0] + current_word
+
+        # https://github.com/spyder-ide/spyder/commit/96ff608af2059865f74c225c065c0102a4da3f45#diff-a020df4c2a52279746bacc37a5fc8ce7R332
+        # if not filter_text[0].isalpha():
+        #     current_word = filter_text[0] + current_word
+        # if params:
+        #     print('\n')
+        #     print('filter_text', [filter_text])
+        #     print('insert_text', [params.get('insertText')])
+        #     print('currentword', [current_word])
+        #     print(params)
+        #     print('\n')
+
         return to_text_string(filter_text).lower().startswith(
                 to_text_string(current_word).lower())
 
diff --git a/spyder/plugins/editor/widgets/tests/test_introspection.py b/spyder/plugins/editor/widgets/tests/test_introspection.py
index 349c8fe9aa2..95663553e9c 100644
--- a/spyder/plugins/editor/widgets/tests/test_introspection.py
+++ b/spyder/plugins/editor/widgets/tests/test_introspection.py
@@ -270,9 +270,9 @@ def test_automatic_completions_parens_bug(lsp_codeeditor, qtbot):
     assert "onesee" in [x['label'] for x in sig.args[0]]
 
 
-@pytest.mark.slow
-@pytest.mark.first
-@flaky(max_runs=5)
+# @pytest.mark.slow
+# @pytest.mark.first
+# @flaky(max_runs=5)
 def test_completions(lsp_codeeditor, qtbot):
     """Exercise code completion in several ways."""
     code_editor, _ = lsp_codeeditor
@@ -284,6 +284,34 @@ def test_completions(lsp_codeeditor, qtbot):
     # Set cursor to start
     code_editor.go_to_line(1)
 
+    # Complete dunder imports from _ --> import 
+    qtbot.keyClicks(code_editor, 'from _')
+    with qtbot.waitSignal(code_editor.lsp_response_signal, timeout=30000):
+        code_editor.document_did_change()
+
+    # press tab and get completions
+    with qtbot.waitSignal(completion.sig_show_completions,
+                          timeout=10000) as sig:
+        qtbot.keyPress(code_editor, Qt.Key_Tab)
+    assert "__future__" in [x['label'] for x in sig.args[0]]
+    code_editor.set_text('')  # Delete line
+    code_editor.go_to_line(1)
+
+    # Complete underscore variables
+    qtbot.keyClicks(code_editor, '_foo = 1;_foom = 2;_fo')
+    with qtbot.waitSignal(code_editor.lsp_response_signal, timeout=30000):
+        code_editor.document_did_change()
+
+    # press tab and get completions
+    with qtbot.waitSignal(completion.sig_show_completions,
+                          timeout=10000) as sig:
+        qtbot.keyPress(code_editor, Qt.Key_Tab)
+    completions = [x['label'] for x in sig.args[0]]
+    assert "_foo" in completions
+    assert "_foom" in completions
+    code_editor.set_text('')  # Delete line
+    code_editor.go_to_line(1)
+
     # Complete import mat--> import math
     qtbot.keyClicks(code_editor, 'import mat')
     with qtbot.waitSignal(code_editor.lsp_response_signal, timeout=30000):


Code review comments:
diff_hunk: @@ -347,14 +349,25 @@ def is_up_to_date(self, item=None):
             filter_text = completion
         return self.check_can_complete(filter_text, current_word)
 
-    def check_can_complete(self, filter_text, current_word):
+    def check_can_complete(self, filter_text, current_word, params=None):
         """Check if current_word matches filter_text."""
         if not filter_text:
             return True
+
         if not current_word:
             return True
-        if not filter_text[0].isalpha():
-            current_word = filter_text[0] + current_word
+
+        # https://github.com/spyder-ide/spyder/commit/96ff608af2059865f74c225c065c0102a4da3f45#diff-a020df4c2a52279746bacc37a5fc8ce7R332
path: spyder/plugins/editor/widgets/completion.py
text: @author:
@user1 @user2 @user3 do you know what was the purpose of this code?

```python
        if not filter_text[0].isalpha():
            current_word = filter_text[0] + current_word
```

That is currently breaking completions for variables starting on underscore `_`

I would like to understand what is/was the use case so I can fix around it. I commented the code and added some tests.

Thanks

@author:
I am not sure if this is new or not
https://github.com/spyder-ide/spyder/commit/96ff608af2059865f74c225c065c0102a4da3f45#diff-a020df4c2a52279746bacc37a5fc8ce7R332

@user1:
This existed before my changes (I just renamed the variables for clarity), and I avoided changing it since I wasn't sure either.

It doesn't make much sense to me, so I'm :+1: on removing it.

cc @user4