Repository: spyder-ide/spyder

Issue:
Completions sometimes insert extraneous text
## Problem Description
When using a completion, sometimes extraneous text is inserted into the buffer.

### What steps reproduce the problem?
See the video below:

![spyder-bad-insert](https://user-images.githubusercontent.com/3752225/69360018-a1c88480-0c3e-11ea-88e3-2701a02093a2.gif)

### What is the expected output? What do you see instead?
__Expected:__ `this_message` should be inserted into the buffer.

__Actual:__ In this case, `this_message$is` was inserted into the buffer, where the `$` denotes the final location of the cursor.

This behavior seems intermittent. Sometimes the correct text is inserted, sometimes the extraneous text is inserted.

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->
Python 3.7.0 64-bit | Qt 5.9.6 | PyQt5 5.9.2 | Darwin 19.0.0 

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```
cloudpickle >=0.5.0          :  1.2.2 (OK)
pygments >=2.0               :  2.4.2 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
sphinx >=0.6.6               :  2.2.1 (OK)
pylint >=0.25                :  2.4.4 (OK)
psutil >=0.3                 :  5.6.5 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
zmq >=17                     :  18.1.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
numpydoc >=0.6.0             :  0.9.1 (OK)
spyder_kernels >=1.8.0;<2.0.0:  1.8.0 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
watchdog                     :  None (OK)
keyring                      :  None (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pympler                      :  None (OK)
sympy >=0.7.3                :  None (NOK)
cython >=0.21                :  None (NOK)
IPython >=4.0                :  7.9.0 (OK)
matplotlib >=2.0.0           :  None (NOK)
pandas >=0.13.1              :  None (NOK)
numpy >=1.7                  :  None (NOK)
scipy >=0.17.0               :  None (NOK)
pyls >=0.31.0;<0.32.0        :  0.31.1 (OK)
applaunchservices >=0.1.7    :  0.2.1 (OK)
rtree >=0.8.3                :  0.8.3 (OK)
```




Discussion:
Thanks for reporting this one @its-dhung , however, as you mention this issue is difficult to reproduce; at least I couldn't reproduce it on my machine by trying the aforementioned completion scenario
What do you think about this one @ccordoba12 
I'm leaving this for 4.1.0 to try to find a reproducible use case.
I have exactly the same bug.

## Version

Spyder 4.0.0 with Kite, Python 3.6.9, Fedora 31.

## Dependencies

cloudpickle >=0.5.0          :  1.2.2 (OK)
pygments >=2.0               :  2.5.2 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
sphinx >=0.6.6               :  2.3.1 (OK)
pylint >=0.25                :  2.4.4 (OK)
psutil >=0.3                 :  5.6.7 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
zmq >=17                     :  18.1.1 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
numpydoc >=0.6.0             :  0.9.2 (OK)
spyder_kernels >=1.8.1;<2.0.0:  1.8.1 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
atomicwrites >=1.2.0         :  1.3.0 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
intervaltree                 :  None (OK)
watchdog                     :  None (OK)
keyring                      :  None (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pympler                      :  None (OK)
sympy >=0.7.3                :  1.5 (OK)
cython >=0.21                :  0.29.14 (OK)
IPython >=4.0                :  7.10.2 (OK)
matplotlib >=2.0.0           :  3.1.2 (OK)
pandas >=0.13.1              :  0.25.3 (OK)
numpy >=1.7                  :  1.18.0 (OK)
scipy >=0.17.0               :  1.4.1 (OK)
pyls >=0.31.2;<0.32.0        :  0.31.4 (OK)
xdg >=0.26                   :  0.26 (OK)
I may have a hint of the error origin. I have a class with a method `self._corr_fig` without docstring (not sure if it does matter). When I hit tab after having written `self._cor`, instead of some additional characters, I get `self._corr_fig($1)$0`. My suspicion is that some of the last written characters go into this $0 sometimes.
@andfoy, please take a look at this one.
> I may have a hint of the error origin. I have a class with a method self._corr_fig without docstring (not sure if it does matter). When I hit tab after having written self._cor, instead of some additional characters, I get self._corr_fig($1)$0. My suspicion is that some of the last written characters go into this $0 sometimes.

I think this is a separate issue, which I can reproduce
@melsophos, PR #11400 should fix your aforementioned issue
@its-dhung @ccordoba12  I'm still unable to reproduce this issue:

![Peek 24-01-2020 15-45](https://user-images.githubusercontent.com/1878982/73102774-9dc81580-3ec0-11ea-8f8e-2ae9ebd0af3f.gif)


@andfoy I can reproduce this reliably.

![Jan-27-2020 20-41-37](https://user-images.githubusercontent.com/3752225/73235969-7afa6300-4145-11ea-9d21-2a16378987c1.gif)

In the video above, the first 2 usages of the completion `json` result in extra text being inserted. In the 3rd usage, the right text is inserted after moving the cursor around. Not sure if moving the cursor around has anything to do with the root cause though.

Let me know if there's any info I could be logging to help you debug.
@dalthviz, could you reproduce this one?
@ccordoba12 I was able to reproduce this by changing the number of characters for automatic completion to 1 and decreasing the show automatic completions keyboard idle to 50:

![image](https://user-images.githubusercontent.com/16781833/73295948-abd6a880-41d6-11ea-8bd4-a960f5a4266a.png)

![extra](https://user-images.githubusercontent.com/16781833/73296077-f7895200-41d6-11ea-9079-c6fc4a0e5403.gif)

Seems like the extra characters are the ones typed before the automatic request for completion is done and shown in the completion widget for the first typed letter(s). So for example, if I type `j` a request will be made and the completion for `json` will complete `son` next to the `j`, but then the rest of the elements typed like `s` will not be taken into account in the completion and will end up at the end of the completion insert:
```python
import j|  # Autocompletion request is made
---------------------------------------------------------------------------------------------
import js|  # 's' is typed and the completion widget pops up but it's actually showing  
            # completions for 'j'. Accepting the completion would insert elements at 
            # 'import j|'
---------------------------------------------------------------------------------------------
import json|s  # Insertion at 'j|'. End cursor before '|s'

```

 

Great work @dalthviz! So this is sort of a race condition in the completion widget.

I think the problem is that Spyder takes some time to display the completion widget, so the additional characters the user types are not passed to the widget. And until the widget appears, they are not passed to the editor either.

A possible solution would be to display the completion widget immediately when users select `1` as the minimum number of characters to show completions. @its-dhung, what do you think?
If multiple requests occur before displaying a widget, how about displaying the widget only for the last request?

I'm not sure my code is the solution about this problem. 
I changed ```send function of _maybe_send_locked of plugin.py of completion```  to my code.
I've been using the following code for months.
```python
        def send():
            max_req_id = max(
                [key for key, item in self.requests.items()
                 if item['req_type'] == self.requests[req_id]['req_type']],
                default=-1)

            del self.requests[req_id]

            # Response only to recent requests.
            if req_id == max_req_id:
                self.gather_and_send(request_responses)
```
> If multiple requests occur before displaying a widget, how about displaying the widget only for the last request?

That makes a lot of sense, thanks @ok97465!

@dalthviz, please take a look at @ok97465's patch to see if it solves this issue.
@ccordoba12 the patch of @ok97465 fixes the issue for me 🎉 

@ok97465 could you open a PR with your patch, please? or if you prefer I could create the PR, whatever works best for you.

Again, thanks for the fix suggestion and feedback @ok97465 !

Resolving patch:
diff --git a/spyder/plugins/completion/plugin.py b/spyder/plugins/completion/plugin.py
--- a/spyder/plugins/completion/plugin.py
+++ b/spyder/plugins/completion/plugin.py
@@ -139,8 +139,18 @@ def _maybe_send_locked(self, req_id):
         request_responses = self.requests[req_id]
 
         def send():
+            # Needed to prevent the send of completions for old requests
+            # See spyder-ide/spyder#10798
+            max_req_id = max(
+                [key for key, item in self.requests.items()
+                 if item['req_type'] == self.requests[req_id]['req_type']]
+                or [-1])
+
             del self.requests[req_id]
-            self.gather_and_send(request_responses)
+
+            # Response only to recent requests.
+            if req_id == max_req_id:
+                self.gather_and_send(request_responses)
 
         wait_for = set(source for source
                        in self.WAIT_FOR_SOURCE[request_responses['req_type']]


Test patch: