Repository: spyder-ide/spyder

Issue:
Autoformat files on save when running them causes lines addition to code
# Editor replicates lines of code at end of script
I have already written about this on stackoverflow (https://stackoverflow.com/questions/68472061/spyder-suddenly-copies-code-to-end-of-file), but at the time I could not reproduce the bug. Here is what I know today with an example to reproduce everything.

With certain script constellations, the last couple of lines of a script are replicated at the end of the script without user intervention when pressing F-5 (Run script). This can lead to a lot of time lost, e.g. when the main function is replicated at the end of a script. 

### Steps to Reproduce the Problem

1. Copy the following script to the spyder editor. Note that the code is not yet formatted and there are a couple of long lines towards then end.
```python
# -*- coding: utf-8 -*-
"""
Created on Wed Aug 25 15:19:45 2021

@author: tobia
"""

import statistics
from edatest import Edatest


def get_quartiles(l, method):

    l = sorted(l)
    q0, q4 = min(l), max(l)
    q2 = statistics.median(l)
    if method == "MS":
        p1 = int((len(l) + 1) / 4 + 0.5)
        q1 = l[p1 - 1]
        p1 = int(3 * (len(l) + 1) / 4) - 1
        q3 = l[p1]
    else:
        len_odd = len(l) % 2 == 1
        mid_point = len(l) // 2

        left = l[0:mid_point]
        if len_odd:
            right = l[mid_point + 1 :]
        else:
            right = l[mid_point:]

        if (method == "T") and (len(l) % 2 == 1):  # odd number of elements
            left.append(q2)
            right = [q2] + right

        # print(f"left: {left}, right: {right}")
        q1 = statistics.median(left)
        q3 = statistics.median(right)

    return [q0, q1, q2, q3, q4]


if __name__ == "__main__":
    Test = Edatest()
    Test.assert_equals(get_quartiles([4, 1, 7, 8, 3, 6, 5, 2], "T"), [1, 2.5, 4.5, 6.5, 8])
    Test.assert_equals(get_quartiles([4, 1, 7, 8, 3, 6, 5, 2], "MS"), [1, 2, 4.5, 7, 8])
    Test.assert_equals(get_quartiles([2, 6, 1, 8, 5, 9, 7, 4, 3], "MM"), [1, 2.5, 5, 7.5, 9])
    Test.assert_equals(
    get_quartiles([2, 6, 1, 8, 5, 9, 7, 4, 3], "MS"), [1, 3, 5, 7, 9]
    )
    Test.assert_equals(get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "T"), [1, 3, 5.5, 8, 10])
    Test.assert_equals(get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "MM"), [1, 3, 5.5, 8, 10])
    Test.assert_equals(get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "MS"), [1, 3, 5.5, 8, 10])
    Test.assert_equals(get_quartiles([3, 9, 11, 2, 4, 1, 8, 6, 10, 5, 7], "T"), [1, 3.5, 6, 8.5, 11])
    Test.assert_equals(get_quartiles([3, 9, 11, 2, 4, 1, 8, 6, 10, 5, 7], "MM"), [1, 3, 6, 9, 11])

``` 
2. Press F-5 -> the script is saved, formatted and executed
3. Press F-5 again: the script is again executed and terminates with an error message
```
File "C:\Users\tobia\OneDrive\Documents\Python Scripts\edabit\quartiles\quartiles.py", line 63
    )
    ^
SyntaxError: unmatched ')'
```
because someone added a couple of lines to the end of the script. In this case it is the following lines:
```python
        get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "T"), [1, 3, 5.5, 8, 10]
    )
    Test.assert_equals(
        get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "MM"), [1, 3, 5.5, 8, 10]
    )
    Test.assert_equals(
        get_quartiles([10, 3, 1, 8, 6, 4, 7, 5, 2, 9], "MS"), [1, 3, 5.5, 8, 10]
    )
    Test.assert_equals(
        get_quartiles([3, 9, 11, 2, 4, 1, 8, 6, 10, 5, 7], "T"), [1, 3.5, 6, 8.5, 11]
    )
    Test.assert_equals(
        get_quartiles([3, 9, 11, 2, 4, 1, 8, 6, 10, 5, 7], "MM"), [1, 3, 6, 9, 11]
    )
```
which are the ~15 last lines of the script, which were just copied to the end.
4. You can go back by pressing Ctrl-Z twice, and try again from there.

It is interesting to note that one can observe this behaviour in real-time when watching the scroll bar at the right side of the window. Its size changes twice, presumably the first time when the code is reformatted, and then again, when the last couple of lines are replicated.
## Versions

Spyder version: 5.0.5
Python 3.8.10 64-bit | Qt 5.9.7 | PyQt5 5.9.2 | Windows 10 

I have observed this behaviour also in previous versions of spyder, but could only reproduce it now.

### Dependencies

For the script above to run edatest needs to be installed (https://pypi.org/project/edatest/), however this is not central to the bug, only required to run the script.


Discussion:
Hi @divingtobi thank you for the feedback! Is this file in a OneDrive directory (with syncing available)? If you test the same script outside the OneDrive directory/sync this behavior still happens? Also, what are the preferences you have selected regarding source code autoformatting on save? Let us know!
Hi #dalthviz - thanks a lot for the response. The file is indeed in a OneDrive directory. The effect only happens when synchronization is on, switching synchronization off stops it from happening.

I set the preferences to "Autoformat files on Save" using "black".

![image](https://user-images.githubusercontent.com/52105524/130847520-966830b7-4b9d-4be4-a28d-eff1b537a2c6.png)

Thanks a lot!
Hi all. I'm having the same problem, but I'm not using OneDrive, and it's specifically when I format using Black.

### Steps to recreate:
1. Create a python file with anything in it, e.g.
```
if __name__ == "__main__":
    print("Hello World")
```
2. Save the file locally e.g. C:\Users\\<user\>\Desktop
3. In Preferences --> Code style and formatting, change the code formatting provider to `black`.
4. Run `black` on the file via the drop-down menu or `CTRL`+`ALT`+`I`.
5. Observe file contents as:
```
if __name__ == "__main__":
    print("Hello World")
)
```

The amount of copied text seems to depend on the size of the file. e.g.:
```
def useless_function():
    pass


if __name__ == "__main__":
    print("Hello World")

```
becomes:
```
def useless_function():
    pass


if __name__ == "__main__":
    print("Hello World")
rld")
```

This content recreation doesn't occur when I use `black` through a console (i.e. not using Spyder) or if I use `autopep8` instead of `black` as the formatting provider.
Hi @ahurlbatt I was able to reproduce this and as you say without having the file synced with any service :+1: 

Thanks for the reproducible example ! 
Pinging @ccordoba12 and @andfoy just in case (maybe they know better what could be happening)
@ahurlbatt, your bug seems like a duplicate of #16180 and it'll be fixed in our next version (5.1.2), to be released shortly.

This bug only happens when you run an unsaved file and auto-formatting on save is activated.
@ccordoba12 Unfortunately not. I have auto-formatting on save turned off specifically because I noticed this. The example I give above happens when triggering auto-format manually on a saved file, without running it. 
@ahurlbatt, please read issue #16180, it's the same problem you're describing. As I said, this bug is totally different.
@ccordoba12 You're right. Apologies! 
While investigating an issue in a custom plugin, I stumbled across some potential race condition in autosave + autoformat when running a script. I am able to trigger that race condition (not completely reliable) which leads to leads to the error described in this issue. PR #16505 should fix it. 
Hi @rear1019 thanks for digging into this and proposing a solution!

Resolving patch:
diff --git a/spyder/plugins/editor/widgets/codeeditor.py b/spyder/plugins/editor/widgets/codeeditor.py
--- a/spyder/plugins/editor/widgets/codeeditor.py
+++ b/spyder/plugins/editor/widgets/codeeditor.py
@@ -539,6 +539,7 @@ def __init__(self, parent=None):
         self.folding_supported = False
         self.is_cloned = False
         self.operation_in_progress = False
+        self.formatting_in_progress = False
 
         # Diagnostics
         self.update_diagnostics_thread = QThread()
@@ -1149,6 +1150,8 @@ def process_symbols(self, params):
     def document_did_change(self, text=None):
         """Send textDocument/didChange request to the server."""
         self.text_version += 1
+        # Cancel formatting
+        self.formatting_in_progress = False
         text = self.get_text_with_eol()
         if self.is_ipython():
             # Send valid python text to LSP
@@ -1672,6 +1675,9 @@ def format_document_or_range(self):
     def format_document(self):
         if not self.formatting_enabled:
             return
+        if self.formatting_in_progress:
+            # Already waiting for a formatting
+            return
 
         using_spaces = self.indent_chars != '\t'
         tab_size = (len(self.indent_chars) if using_spaces else
@@ -1693,6 +1699,7 @@ def format_document(self):
         self.document().setModified(True)
         self.sig_start_operation_in_progress.emit()
         self.operation_in_progress = True
+        self.formatting_in_progress = True
 
         return params
 
@@ -1700,6 +1707,9 @@ def format_document(self):
     def format_document_range(self):
         if not self.range_formatting_enabled or not self.has_selected_text():
             return
+        if self.formatting_in_progress:
+            # Already waiting for a formatting
+            return
 
         start, end = self.get_selection_start_end()
         start_line, start_col = start
@@ -1736,13 +1746,15 @@ def format_document_range(self):
         self.document().setModified(True)
         self.sig_start_operation_in_progress.emit()
         self.operation_in_progress = True
+        self.formatting_in_progress = True
 
         return params
 
     @handles(CompletionRequestTypes.DOCUMENT_FORMATTING)
     def handle_document_formatting(self, edits):
         try:
-            self._apply_document_edits(edits)
+            if self.formatting_in_progress:
+                self._apply_document_edits(edits)
         except RuntimeError:
             # This is triggered when a codeeditor instance was removed
             # before the response can be processed.
@@ -1757,11 +1769,13 @@ def handle_document_formatting(self, edits):
             self.document().setModified(True)
             self.sig_stop_operation_in_progress.emit()
             self.operation_in_progress = False
+            self.formatting_in_progress = False
 
     @handles(CompletionRequestTypes.DOCUMENT_RANGE_FORMATTING)
     def handle_document_range_formatting(self, edits):
         try:
-            self._apply_document_edits(edits)
+            if self.formatting_in_progress:
+                self._apply_document_edits(edits)
         except RuntimeError:
             # This is triggered when a codeeditor instance was removed
             # before the response can be processed.
@@ -1776,6 +1790,7 @@ def handle_document_range_formatting(self, edits):
             self.document().setModified(True)
             self.sig_stop_operation_in_progress.emit()
             self.operation_in_progress = False
+            self.formatting_in_progress = False
 
     def _apply_document_edits(self, edits):
         """Apply a set of atomic document edits to the current editor text."""


Test patch: