Repository: spyder-ide/spyder
Pull request: PR: Prevent segfault when modifying the editor content
<!--- Make sure to read the Contributing Guidelines:                   --->
<!--- https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md --->
<!--- and follow PEP 8, PEP 257 and Spyder's code style:               --->
<!--- https://github.com/spyder-ide/spyder/wiki/Dev:-Coding-Style      --->

## Description of Changes


* [ ] Wrote at least one-line docstrings (for any new functions)
* [ ] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->
* [ ] Included a screenshot or animation (if affecting the UI, see [Licecap](https://www.cockos.com/licecap/))


<!--- Explain what you've done and why --->
Try to detect when a block becomes invalid



### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes https://github.com/spyder-ide/spyder/issues/18520


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct:

<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
Segmentation fault crash from the Editor
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

Head of C stack:

```
Thread 0 Crashed:: CrBrowserMain Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff80434300e __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff8043791ff pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff8042872c8 raise + 26
3   libsystem_platform.dylib      	    0x7ff80438edfd _sigtramp + 29
4   ???                           	  0x5300440068004d ???
5   QtGui.so                      	       0x1060ac96b meth_QTextBlock_firstLineNumber(_object*, _object*) + 75
6   Python                        	       0x1038e2d7b cfunction_call + 90
7   Python                        	       0x103897688 _PyObject_MakeTpCall + 127
8   Python                        	       0x10398e528 call_function + 279
9   Python                        	       0x103987d1c _PyEval_EvalFrameDefault + 28216
10  Python                        	       0x10389807d _PyFunction_Vectorcall + 192
11  Python                        	       0x10389acb4 method_vectorcall + 365
12  sip.cpython-39-darwin.so      	       0x105826a07 sip_api_call_procedure_method + 199
13  QtWidgets.so                  	       0x1087833d7 sipQWidget::paintEvent(QPaintEvent*) + 103
14  QtWidgets                     	       0x108bda548 QWidget::event(QEvent*) + 1096
```

Hapened when coming back to spyder after modyfying a file outside of it that was opened inside spyder

### Traceback
```python-traceback
Fatal Python error: Segmentation fault

Thread 0x0000700012d52000 (most recent call first):
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 316 in wait
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 574 in wait
  File "/usr/local/lib/python3.9/site-packages/jupyter_client/channels.py", line 134 in _async_run
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/tasks.py", line 256 in __step
  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 183 in step
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py", line 80 in _run
  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 111 in _run_once
  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 75 in run_until_complete
  File "/usr/local/lib/python3.9/site-packages/jupyter_client/channels.py", line 112 in run
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x0000700011d4f000 (most recent call first):
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/selectors.py", line 562 in select
  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 98 in _run_once
  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 64 in run_forever
  File "/usr/local/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 199 in start
  File "/usr/local/lib/python3.9/site-packages/jupyter_client/threaded.py", line 220 in run
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 930 in _bootstrap

Thread 0x0000700010d4c000 (most recent call first):
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 312 in wait
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 574 in wait
  File "/usr/local/lib/python3.9/site-packages/IPython/core/history.py", line 829 in run
  File "/usr/local/lib/python3.9/site-packages/IPython/core/history.py", line 58 in needs_sqlite
  File "/usr/local/lib/python3.9/site-packages/decorator.py", line 232 in fun
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 930 in _bootstrap

Current thread 0x000000010b892600 (most recent call first):
  File "/Users/quentinpeter/pyscripts/spyder/spyder/plugins/editor/panels/scrollflag.py", line 248 in paintEvent
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/utils.py", line 333 in create_window
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/mainwindow.py", line 2028 in main
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/start.py", line 266 in main
  File "/Applications/Spyder.app/Contents/MacOS/Spyder", line 83 in <module>
```

## Versions

* Spyder version: 6.0.0.dev0 6270a5576 (pip)
* Python version: 3.9.12 64-bit
* Qt version: 5.12.10
* PyQt5 version: 5.12.3
* Operating System: Darwin 21.4.0

### Dependencies

```
# Mandatory:
applaunchservices >=0.1.7     :  0.3.0 (OK)
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.31.1;<8.0.0       :  7.34.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.4.0 (OK)
keyring >=17.0.0              :  23.5.0 (OK)
nbconvert >=4.0               :  6.5.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.11.2 (OK)
pylint >=2.5.0;<3.0           :  2.12.2 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0          :  1.4.1 (OK)
pylsp_black >=1.2.0           :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0     :  3.0.3 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0      :  5.3.1 (OK)
qtpy >=2.1.0                  :  2.1.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  60.10.0 (OK)
sphinx >=0.6.6                :  4.3.2 (OK)
spyder_kernels >=2.3.1;<2.4.0 :  3.0.0.dev0 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.6 (OK)
zmq >=22.1.0                  :  23.2.0 (OK)

# Optional:
cython >=0.21                 :  0.29.30 (OK)
matplotlib >=3.0.0            :  3.5.1 (OK)
numpy >=1.7                   :  1.23.0 (OK)
pandas >=1.1.1                :  1.4.1 (OK)
scipy >=0.17.0                :  1.8.0 (OK)
sympy >=0.7.3                 :  1.10.1 (OK)
```

Hints from issue discussion:
@impact27, any ideas on how to reproduce this?
I managed to (kinda) reproduce it:

1. open a file and add ~100 lines with breakpoints, errors, warnings, todo, and other things that would create scrollflags
2. Open the same file with another program and delete all the lines
3. go back to spyder

Doing that it crashes sometimes 
So the problem is that Spyder can't update the file when it's modified externally and has a lot of linting?
The problem is that in `File "spyder/plugins/editor/panels/scrollflag.py", line 248 in paintEvent`, `firstLineNumber` is called on a block that doesn't exist anymore. The code looks like:
```python
for block in dict_flag_lists[flag_type]:
    if not block.isValid():
        continue
    block_line = block.firstLineNumber()
```
So `block.isValid()` is either not a good test or something happens between these two statements?


That's typical with Qt: if an object is removed and the app tries to access it again, then it generates an error or segfault. I think to solve this we should emit a signal when the file is reloaded so that the `dict_flag_lists` is cleaned after that and we can't perform the `block.isValid()` call.
Another one:
```
Fatal Python error: Segmentation fault
Current thread 0x00000001188c9600 (most recent call first):
  File "/Users/quentinpeter/pyscripts/spyder/spyder/plugins/editor/panels/scrollflag.py", line 226 in paintEvent
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/utils.py", line 333 in create_window
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/mainwindow.py", line 2028 in main
  File "/Users/quentinpeter/pyscripts/spyder/spyder/app/start.py", line 266 in main
  File "/Applications/Spyder.app/Contents/MacOS/Spyder", line 83 in <module>
```
```
Thread 0 Crashed:: CrBrowserMain Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff81055b00e __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff8105911ff pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff81049f2c8 raise + 26
3   libsystem_platform.dylib      	    0x7ff8105a6dfd _sigtramp + 29
4   ???                           	  0x5300440068004d ???
5   QtWidgets                     	       0x111da8939 0x111bde000 + 1878329
6   QtWidgets                     	       0x111da935d QPlainTextEdit::blockBoundingGeometry(QTextBlock const&) const + 29
7   QtWidgets.so                  	       0x11181cdef meth_QPlainTextEdit_blockBoundingGeometry(_object*, _object*) + 111
```

This time the code is:
```python
for block in dict_flag_lists[flag_type]:
    if not block.isValid():
        continue
    geometry = editor.blockBoundingGeometry(block)
```
> That's typical with Qt: if an object is removed and the app tries to access it again, then it generates an error or segfault. I think to solve this we should emit a signal when the file is reloaded so that the `dict_flag_lists` is cleaned after that and we can't perform the `block.isValid()` call.

That makes sense
To reproduce more easily:
Open internal console:
```
>>> spy.window.editor.new()
>>> editor = spy.window.editor.get_current_editor()
>>> editor.set_text(("\n" * 10 + "d") * 100)
>>> editor.set_text("") # or editor.clear()
```
(Doesn't work all the time)

Commit message: pep8

Patch to review:
diff --git a/spyder/plugins/editor/panels/scrollflag.py b/spyder/plugins/editor/panels/scrollflag.py
index 5a761f24935..2310b576859 100644
--- a/spyder/plugins/editor/panels/scrollflag.py
+++ b/spyder/plugins/editor/panels/scrollflag.py
@@ -20,6 +20,7 @@
 # Local imports
 from spyder.api.panel import Panel
 from spyder.plugins.completion.api import DiagnosticSeverity
+from spyder.plugins.editor.utils.editor import block_safe
 
 
 REFRESH_RATE = 1000
@@ -221,7 +222,7 @@ def paintEvent(self, event):
             if editor.verticalScrollBar().maximum() == 0:
                 # No scroll
                 for block in dict_flag_lists[flag_type]:
-                    if not block.isValid():
+                    if not block_safe(block):
                         continue
                     geometry = editor.blockBoundingGeometry(block)
                     rect_y = ceil(
@@ -233,7 +234,7 @@ def paintEvent(self, event):
             elif last_line == 0:
                 # Only one line
                 for block in dict_flag_lists[flag_type]:
-                    if not block.isValid():
+                    if not block_safe(block):
                         continue
                     rect_y = ceil(first_y_pos)
                     painter.drawRect(rect_x, rect_y, rect_w, rect_h)
@@ -243,7 +244,7 @@ def paintEvent(self, event):
                     # If the file is too long, do not freeze the editor
                     next_line = 0
                     for block in dict_flag_lists[flag_type]:
-                        if not block.isValid():
+                        if not block_safe(block):
                             continue
                         block_line = block.firstLineNumber()
                         # block_line = -1 if invalid
diff --git a/spyder/plugins/editor/utils/editor.py b/spyder/plugins/editor/utils/editor.py
index 1b6b127d54d..afcf3296a74 100644
--- a/spyder/plugins/editor/utils/editor.py
+++ b/spyder/plugins/editor/utils/editor.py
@@ -51,6 +51,23 @@ def drift_color(base_color, factor=110):
             return base_color.lighter(factor + 10)
 
 
+def block_safe(block):
+    """
+    Check if the block is safe to work with.
+
+    A BlockUserData must have been set on the block while it was known safe.
+
+    If an editor is cleared by editor.clear() or editor.set_text() for example,
+    all the old blocks will continue to report block.isValid() == True
+    but will raise a Segmentation Fault on almost all methods.
+
+    One way to check is that the userData is reset to None or
+    QTextBlockUserData. So if a block is known to have setUserData to
+    BlockUserData, this fact can be used to check the block.
+    """
+    return block.isValid() and isinstance(block.userData(), BlockUserData)
+
+
 class BlockUserData(QTextBlockUserData):
     def __init__(self, editor, color=None, selection_start=None,
                  selection_end=None):
diff --git a/spyder/plugins/editor/widgets/codeeditor.py b/spyder/plugins/editor/widgets/codeeditor.py
index d18c33548ea..5c959d8e6d0 100644
--- a/spyder/plugins/editor/widgets/codeeditor.py
+++ b/spyder/plugins/editor/widgets/codeeditor.py
@@ -2579,7 +2579,11 @@ def __mark_occurrences(self):
         extra_selections = self.get_extra_selections('occurrences')
         first_occurrence = None
         while cursor:
-            self.occurrences.append(cursor.block())
+            block = cursor.block()
+            if not block.userData():
+                # Add user data to check block validity
+                block.setUserData(BlockUserData(self))
+            self.occurrences.append(block)
             selection = self.get_selection(cursor)
             if len(selection.cursor.selectedText()) > 0:
                 extra_selections.append(selection)
@@ -2623,7 +2627,11 @@ def highlight_found_results(self, pattern, word=False, regexp=False,
             selection = TextDecoration(self.textCursor())
             selection.format.setBackground(self.found_results_color)
             selection.cursor.setPosition(pos1)
-            self.found_results.append(selection.cursor.block())
+            block = selection.cursor.block()
+            if not block.userData():
+                # Add user data to check block validity
+                block.setUserData(BlockUserData(self))
+            self.found_results.append(block)
             selection.cursor.setPosition(pos2, QTextCursor.KeepAnchor)
             extra_selections.append(selection)
         self.set_extra_selections('find', extra_selections)


Code review comments:
diff_hunk: @@ -51,6 +51,23 @@ def drift_color(base_color, factor=110):
             return base_color.lighter(factor + 10)
 
 
+def block_safe(block):
+    """
+    Check if the block is safe to work with.
+
+    A BlockUserData must have been set on the block while it was known safe.
+
+    If an editor is cleared by editor.clear() or editor.set_text() for example,
+    all the old blocks will continue to report block.isValid() == True
+    but will raise a Segmentation Fault on almost all methods.
+
+    One way to check is that the userData is reset to None or
+    QTextBlockUserData. So if a block is known to have setUserData to
+    BlockUserData, this fact can be used to check the block.
path: spyder/plugins/editor/utils/editor.py
text: @user1:
```suggestion
    One way to check if a block is valid is that the userData is reset to None
    or QTextBlockUserData. So if a block is known to have setUserData to
    BlockUserData, this fact can be used to check the block.
```

diff_hunk: @@ -51,6 +51,23 @@ def drift_color(base_color, factor=110):
             return base_color.lighter(factor + 10)
 
 
+def block_safe(block):
+    """
+    Check if the block is safe to work with.
+
+    A BlockUserData must have been set on the block while it was known safe.
path: spyder/plugins/editor/utils/editor.py
text: @user1:
```suggestion
    A BlockUserData must have been set on the block while it was known to be
    safe.
```

diff_hunk: @@ -51,6 +51,23 @@ def drift_color(base_color, factor=110):
             return base_color.lighter(factor + 10)
 
 
+def block_safe(block):
+    """
+    Check if the block is safe to work with.
+
+    A BlockUserData must have been set on the block while it was known safe.
+
+    If an editor is cleared by editor.clear() or editor.set_text() for example,
+    all the old blocks will continue to report block.isValid() == True
+    but will raise a Segmentation Fault on almost all methods.
path: spyder/plugins/editor/utils/editor.py
text: @user1:
```suggestion
    all old blocks will continue to report block.isValid() == True, but will
    raise a segmentation fault on almost all methods.
```

diff_hunk: @@ -51,6 +51,23 @@ def drift_color(base_color, factor=110):
             return base_color.lighter(factor + 10)
 
 
+def block_safe(block):
path: spyder/plugins/editor/utils/editor.py
text: @user1:
Let's rename this function to `is_block_safe` for clarity.

@author:
done