Repository: spyder-ide/spyder

Issue:
Editor becomes sluggish when displaying errors
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [x] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

The whole program becomes so slow after some time, that it is not even possible to save the file and close Spyder.

This is obviously related to how many errors are displayed beside the line numbers.

If I start the program, it gives you a snappy response, even if errors are displayed, but after some time the editor becomes sluggish. The main Spyder process uses 100% of one CPU core. If I manage to modify my code to be error-free, the sluggish response goes away - until the next error is displayed. The high CPU load appears after the errors are displayed beside the line numbers but before they are displayed in the flag area near the scrollbar.

Sometimes if you make an edit, which causes a lot of errors, Spyder does not respond at all. Then it is not even possible to save the file and close Spyder. When that happens I watched the system monitor and found not only that the main Spyder process uses 100% of one CPU core but also its memory consumption increases continuously.

## Set-up

I have only two panes open:
- Editor, split horizontally to show two files. (The behaviour described above appears if I have the same file shown in both editors. I don't know for sure, if it also appears if two different files are shown.)
- Outline

## Preferences

My settings from `Completion and linting` are as follows (only the active items are shown).
- General
    [x] Enable code completion
    [x] Show completion details
    [x] Show completions on the fly
    [x] Enable Language Server Protocol server
    [x] Enable text snippets provider
- Linting
    [x] Enable basic linting
    [x] Underline errors and warnings
- Introspection
    [x] Enable Go to definition
    [x] Follow imports when going to a definition
    [x] Show calltips
- Code style and formatting
    [x] Show vertical line at that length (79)
    [x] Enable hover hints
- Docstring style
- Advanced
- Other languages
    - [None]
- Snippets
    - [Default]



## Additional Information

See also #15631. The PR #16206 improved the situation quite a lot.

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 5.1.3
* Python version: 3.8.10 64-bit
* Qt version: 5.12.9
* PyQt version: 5.12.3
* Operating System name/version: Linux Mint 20.2

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  1.6.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.27.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.17.2 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.0.1 (OK)
nbconvert >=4.0               :  6.1.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
parso >=0.7.0;<0.9.0          :  0.7.0 (OK)
pexpect >=4.4.0               :  4.6.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.5.1 (OK)
pygments >=2.0                :  2.9.0 (OK)
pylint >=2.5.0;<2.10.0        :  2.9.6 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.2.2;<1.3.0          :  1.2.2 (OK)
pylsp_black >=1.0.0           :  None (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.0.3 (OK)
qtconsole >=5.1.0             :  5.1.1 (OK)
qtpy >=1.5.0                  :  1.9.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  57.5.0 (OK)
sphinx >=0.6.6                :  4.1.2 (OK)
spyder_kernels >=2.1.1;<2.2.0 :  2.1.1 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  1.0.2 (OK)
xdg >=0.26                    :  0.26 (OK)
zmq >=17                      :  22.2.0 (OK)

# Optional:
cython >=0.21                 :  None (OK)
matplotlib >=2.0.0            :  3.4.3 (OK)
numpy >=1.7                   :  1.21.2 (OK)
pandas >=1.1.1                :  1.3.2 (OK)
scipy >=0.17.0                :  1.7.1 (OK)
sympy >=0.7.3                 :  1.8 (OK)
```



Discussion:

Resolving patch:
diff --git a/spyder/plugins/editor/utils/editor.py b/spyder/plugins/editor/utils/editor.py
--- a/spyder/plugins/editor/utils/editor.py
+++ b/spyder/plugins/editor/utils/editor.py
@@ -70,15 +70,11 @@ def __init__(self, editor, color=None, selection_start=None,
         # Add a reference to the user data in the editor as the block won't.
         # The list should /not/ be used to list BlockUserData as the blocks
         # they refer to might not exist anymore.
-        # This prevent a segmentation fault.
+        # This prevents a segmentation fault.
         if editor is None:
             # Won't be destroyed
             self.refloop = self
             return
-        # Destroy with the editor
-        if not hasattr(editor, '_user_data_reference_list'):
-            editor._user_data_reference_list = []
-        editor._user_data_reference_list.append(self)
 
     def _selection(self):
         """
diff --git a/spyder/plugins/editor/widgets/codeeditor.py b/spyder/plugins/editor/widgets/codeeditor.py
--- a/spyder/plugins/editor/widgets/codeeditor.py
+++ b/spyder/plugins/editor/widgets/codeeditor.py
@@ -137,10 +137,10 @@ class CodeEditor(TextEditBaseWidget):
     # linting results arrive, according to the number of lines in the file.
     SYNC_SYMBOLS_AND_FOLDING_TIMEOUTS = {
         # Lines: Timeout
-        500: 500,
-        1500: 1200,
-        2500: 3200,
-        6500: 4500
+        500: 350,
+        1500: 800,
+        2500: 1200,
+        6500: 1800
     }
 
     # Custom signal to be emitted upon completion of the editor's paintEvent
@@ -473,6 +473,7 @@ def __init__(self, parent=None):
         # Code Folding
         self.code_folding = True
         self.update_folding_thread = QThread()
+        self.update_folding_thread.finished.connect(self.finish_code_folding)
 
         # Completions hint
         self.completions_hint = True
@@ -541,6 +542,9 @@ def __init__(self, parent=None):
 
         # Diagnostics
         self.update_diagnostics_thread = QThread()
+        self.update_diagnostics_thread.run = self.set_errors
+        self.update_diagnostics_thread.finished.connect(
+            self.finish_code_analysis)
         self._diagnostics = []
 
         # Editor Extensions
@@ -1210,9 +1214,6 @@ def process_code_analysis(self, diagnostics):
         self._diagnostics = diagnostics
 
         # Process diagnostics in a thread to improve performance.
-        self.update_diagnostics_thread.run = self.set_errors
-        self.update_diagnostics_thread.finished.connect(
-            self.finish_code_analysis)
         self.update_diagnostics_thread.start()
 
     def cleanup_code_analysis(self):
@@ -1244,7 +1245,13 @@ def set_errors(self):
     def underline_errors(self):
         """Underline errors and warnings."""
         try:
+            # Clear current selections before painting the new ones.
+            # This prevents accumulating them when moving around in or editing
+            # the file, which generated a memory leakage and sluggishness
+            # after some time.
+            self.clear_extra_selections('code_analysis_underline')
             self._process_code_analysis(underline=True)
+            self.update_extra_selections()
         except RuntimeError:
             # This is triggered when a codeeditor instance was removed
             # before the response can be processed.
@@ -1255,8 +1262,8 @@ def underline_errors(self):
     def finish_code_analysis(self):
         """Finish processing code analysis results."""
         self.linenumberarea.update()
-        self.underline_errors()
-        self.update_extra_selections()
+        if self.underline_errors_enabled:
+            self.underline_errors()
         self.sig_process_code_analysis.emit()
         self.sig_flags_changed.emit()
 
@@ -1280,6 +1287,9 @@ def _process_code_analysis(self, underline):
             them can't.
         """
         document = self.document()
+        if underline:
+            first_block, last_block = self.get_buffer_block_numbers()
+
         for diagnostic in self._diagnostics:
             if self.is_ipython() and (
                     diagnostic["message"] == "undefined name 'get_ipython'"):
@@ -1295,27 +1305,25 @@ def _process_code_analysis(self, underline):
                 'severity', DiagnosticSeverity.ERROR)
 
             block = document.findBlockByNumber(start['line'])
-            data = block.userData()
+            text = block.text()
 
             # Skip messages according to certain criteria.
             # This one works for any programming language
-            if 'analysis:ignore' in block.text():
+            if 'analysis:ignore' in text:
                 continue
 
             # This only works for Python.
             if self.language == 'Python':
-                if NOQA_INLINE_REGEXP.search(block.text()) is not None:
+                if NOQA_INLINE_REGEXP.search(text) is not None:
                     continue
 
+            data = block.userData()
             if not data:
                 data = BlockUserData(self)
 
             if underline:
                 block_nb = block.blockNumber()
-                first, last = self.get_buffer_block_numbers()
-
-                if (self.underline_errors_enabled and
-                        first <= block_nb <= last):
+                if first_block <= block_nb <= last_block:
                     error = severity == DiagnosticSeverity.ERROR
                     color = self.error_color if error else self.warning_color
                     color = QColor(color)
@@ -1327,7 +1335,7 @@ def _process_code_analysis(self, underline):
 
                     # Don't call highlight_selection with `update=True` so that
                     # all underline selections are updated in bulk in
-                    # finish_code_analysis or update_decorations.
+                    # underline_errors.
                     self.highlight_selection('code_analysis_underline',
                                              data._selection(),
                                              underline_color=block.color)
@@ -1881,8 +1889,6 @@ def handle_folding_range(self, response):
         # Update folding in a thread
         self.update_folding_thread.run = functools.partial(
             self.update_and_merge_folding, extended_ranges)
-        self.update_folding_thread.finished.connect(
-            self.finish_code_folding)
         self.update_folding_thread.start()
 
     def update_and_merge_folding(self, extended_ranges):
@@ -3013,14 +3019,7 @@ def show_hint_for_completion(self, word, documentation, at_point):
     def update_decorations(self):
         """Update decorations on the visible portion of the screen."""
         if self.underline_errors_enabled:
-            # Clear current selections before painting the new ones.
-            # This prevents accumulating them when moving around in the file,
-            # which generated a memory leak and sluggishness in the editor
-            # after some time.
-            self.clear_extra_selections('code_analysis_underline')
-
             self.underline_errors()
-            self.update_extra_selections()
 
         # This is required to update decorations whether there are or not
         # underline errors in the visible portion of the screen.


Test patch: