Repository: spyder-ide/spyder
Pull request: PR: Improve folding updates and line displacement on the editor
<!--- Make sure to read the Contributing Guidelines:                   --->
<!--- https://github.com/spyder-ide/spyder/blob/master/CONTRIBUTING.md --->
<!--- and follow PEP 8, PEP 257 and Spyder's code style:               --->
<!--- https://github.com/spyder-ide/spyder/wiki/Dev:-Coding-Style      --->

## Description of Changes
This PR aims to improve the code folding updating logic to be more robust in order to prevent breakages and strange behaviour.

* [x] Wrote at least one-line docstrings (for any new functions)
* [ ] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->
* [x] Included a screenshot or animation (if affecting the UI, see [Licecap](https://www.cockos.com/licecap/))


<!--- Explain what you've done and why --->

#### **Moving a folded region line down**

![Peek 10-11-2020 19-04](https://user-images.githubusercontent.com/1878982/98748565-f604d100-2387-11eb-8e07-9fd928d2f21e.gif)

#### **Moving a folded region line up**

![Peek 10-11-2020 19-06](https://user-images.githubusercontent.com/1878982/98748598-0321c000-2388-11eb-9fc9-61f12b2539c7.gif)

#### **Moving a whole folded region down**

![Peek 10-11-2020 19-08](https://user-images.githubusercontent.com/1878982/98748704-32d0c800-2388-11eb-885f-57e7dcce797e.gif)

#### **Moving a whole region up**

![Peek 10-11-2020 19-09](https://user-images.githubusercontent.com/1878982/98748769-58f66800-2388-11eb-985e-a4b90254b87e.gif)

#### **Moving a line down inside a folded region**

![Peek 10-11-2020 19-11](https://user-images.githubusercontent.com/1878982/98748878-922ed800-2388-11eb-8bb0-2f67d9b3bb50.gif)

#### **Moving a selection down inside a folded region**

![Peek 10-11-2020 19-12](https://user-images.githubusercontent.com/1878982/98748949-bc809580-2388-11eb-8aa5-e3c6e3a29ae8.gif)

#### **Moving a line up inside a folded region**

![Peek 10-11-2020 19-14](https://user-images.githubusercontent.com/1878982/98749151-12edd400-2389-11eb-98ec-2694ab8e1ad2.gif)

#### **Moving a selection up inside a folded region**

![Peek 10-11-2020 19-15](https://user-images.githubusercontent.com/1878982/98749217-2dc04880-2389-11eb-8e22-0c974acdf473.gif)

#### **Inserting newlines before a folded region**

![Peek 11-11-2020 00-20](https://user-images.githubusercontent.com/1878982/98773569-33cd1e00-23b7-11eb-8676-f235d35b3659.gif)

#### **Deleting newlines before a folded region**

![Peek 11-11-2020 00-46](https://user-images.githubusercontent.com/1878982/98773667-66771680-23b7-11eb-912a-a895333f4c9c.gif)

#### **Unfold nested regions**

![Peek 24-11-2020 12-55](https://user-images.githubusercontent.com/1878982/100133224-7c271a00-2e54-11eb-9d59-4ef4944d5c62.gif)


### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes #13779
Fixes #11360


### Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the [Developer Certificate of Origin](https://developercertificate.org)
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

<!--- TYPE YOUR USER/NAME AFTER THE FOLLOWING: --->
I certify the above statement is true and correct: @andfoy

<!--- Thanks for your help making Spyder better for everyone! --->


Problem statement:
Code Folding Occasionally Breaks
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

The problem is that, occasionally, the arrow for code folding collapse and expansion disappears. This renders the code there inaccessible from the editor. The editor still grays out the function name, but it will not be expandable. The arrow only seems to disappear after collapsing a code block/function/class. This doesn't happen everytime that I collapse the code block. I don't know the exact order of clicks and where those clicks take place that make this issue manifest.



## Versions

* Spyder version: 4.0.1 
* Python version: 3.7.1
* Qt version: 5.9.6
* PyQt5 version: 5.9.2
* Operating System: Windows 10

### Dependencies

```
atomicwrites >=1.2.0         :  1.3.0 (OK)
chardet >=2.0.0              :  3.0.4 (OK)
cloudpickle >=0.5.0          :  1.2.2 (OK)
diff_match_patch >=20181111  :  20181111 (OK)
intervaltree                 :  None (OK)
IPython >=4.0                :  7.11.1 (OK)
jedi =0.14.1                 :  0.14.1 (OK)
nbconvert >=4.0              :  5.6.1 (OK)
numpydoc >=0.6.0             :  0.9.2 (OK)
pexpect >=4.4.0              :  4.7.0 (OK)
pickleshare >=0.4            :  0.7.5 (OK)
psutil >=0.3                 :  5.6.7 (OK)
pygments >=2.0               :  2.5.2 (OK)
pylint >=0.25                :  2.4.4 (OK)
pyls >=0.31.2;<0.32.0        :  0.31.2 (OK)
zmq >=17                     :  18.1.0 (OK)
qdarkstyle >=2.7             :  2.7 (OK)
qtawesome >=0.5.7            :  0.6.0 (OK)
qtconsole >=4.6.0            :  4.6.0 (OK)
qtpy >=1.5.0                 :  1.9.0 (OK)
rtree >=0.8.3                :  0.8.3 (OK)
sphinx >=0.6.6               :  2.3.1 (OK)
spyder_kernels >=1.8.1;<2.0.0:  1.8.1 (OK)
watchdog                     :  None (OK)
cython >=0.21                :  None (NOK)
matplotlib >=2.0.0           :  3.1.1 (OK)
numpy >=1.7                  :  1.18.1 (OK)
pandas >=0.13.1              :  None (NOK)
scipy >=0.17.0               :  1.3.2 (OK)
sympy >=0.7.3                :  None (NOK)
```

Hints from issue discussion:
Could you please provide a video/animation of this with Licecap?
I will try to do this. I won’t be able to demonstrate the process of it breaking, but I can demonstrate it’s broken state once it happens again.

Sent from my iPhone

> On Jan 20, 2020, at 2:21 PM, Gonzalo Peña-Castellanos <notifications@github.com> wrote:
> 
> ﻿
> Could you please provide a video/animation of this with Licecap?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

>it’s broken state once it happens again.

Please do! thanks
I just wanted to come here to double down on this bug report.  I'm having the identical issue.

It happened to me multiple times within 30 minutes.  I was writing functions, each with their own multi-line doc strings, and I would collapse the doc strings in on themselves.  Then I would flesh out the function and collapse it when I moved onto the next func.

Spyder seems to indicate a collapsed code block by "graying out" the first line of the fold but I was having all sorts of issues with this feature.

1.  Sometimes the lines would regain their color and show that they aren't collapsed.
2.  Sometimes it wouldn't offer me the option to collapse AT the doc string -- only above or below.
3.  Sometimes, as detailed above, entire folds will be lost to the aether.

I'll pay a bit more attention when I sit down to code today and monitor the exact input before any instance of this bug, should it arise.  I've got a hunch it doesn't play well with the alt+up / alt+down shortcut for shifting sections of code up and down to different rows.  But I'm currently lacking documentation.  Just wanted to, at the very least, share that this isn't an isolated incident.
Could you please provide a video/animation of this with Licecap and/or screenshots?
> Could you please provide a video/animation of this with Licecap and/or screenshots?

![documentation](https://user-images.githubusercontent.com/60125364/72833137-2a21d080-3c54-11ea-9a54-28d93ee140ef.png)

Took a screenshot.  This process can be reliably replicated.
@andfoy could you take a look a t this one?
I think the "errors" are due to our syntax error-based folding, it makes best effort folding detection, rather than provide a full solution for such cases. Therefore, we cannot do much besides asking users to have correct syntax code
@andfoy, that's not the problem. The problem is that when the user has folded regions in the editor and introduces new changes to it, it's not possible to correctly unfold those regions afterwards.

We haven't considered this case before and I think it requires to maintain in sync what's folded in the editor and new folding info coming from the PyLS.
>  I think it requires to maintain in sync what's folded in the editor and new folding info coming from the PyLS.

Actually, that's the case in https://github.com/spyder-ide/spyder/blob/48540b37fd2a27fbbbdd73b3ab8b1000b9c68e20/spyder/plugins/editor/panels/codefolding.py#L207

Commit message: Fix issues with backspace

Patch to review:
diff --git a/binder/environment.yml b/binder/environment.yml
index f86c4f4792a..4862cbfe9e5 100644
--- a/binder/environment.yml
+++ b/binder/environment.yml
@@ -43,6 +43,7 @@ dependencies:
 # NOTE: There's no need to set constraints for spyder-kernels here
 # because Spyder uses the subrepo for it when started in Binder.
 - spyder-kernels
+- textdistance >=4.2.0
 - three-merge >=0.1.1
 - watchdog >=0.10.3
 
diff --git a/requirements/conda.txt b/requirements/conda.txt
index 1f52cf60943..50de4914466 100644
--- a/requirements/conda.txt
+++ b/requirements/conda.txt
@@ -39,5 +39,6 @@ sphinx >=0.6.6
 # NOTE: There's no need to set a version for spyder-kernels
 # here because we're using a subrepo for it to run our tests.
 spyder-kernels
+textdistance >=4.2.0
 three-merge >=0.1.1
 watchdog >=0.10.3
diff --git a/setup.py b/setup.py
index cde7d56b9d7..35762a9a2aa 100644
--- a/setup.py
+++ b/setup.py
@@ -218,6 +218,7 @@ def run(self):
     'setuptools>=39.0.0',
     'sphinx>=0.6.6',
     'spyder-kernels>=1.10.0,<1.11.0',
+    'textdistance>=4.2.0',
     'three-merge>=0.1.1',
     'watchdog>=0.10.3'
 ]
diff --git a/spyder/dependencies.py b/spyder/dependencies.py
index 04a69dd8d3f..5de28585148 100644
--- a/spyder/dependencies.py
+++ b/spyder/dependencies.py
@@ -63,6 +63,7 @@
 SETUPTOOLS_REQVER = '>=39.0.0'
 SPHINX_REQVER = '>=0.6.6'
 SPYDER_KERNELS_REQVER = '>=1.10.0;<1.11.0'
+TEXTDISTANCE_REQVER = '>=4.2.0'
 THREE_MERGE_REQVER = '>=0.1.1'
 # None for pynsist install for now
 # (check way to add dist.info/egg.info from packages without wheels available)
@@ -222,6 +223,10 @@
      'package_name': "spyder-kernels",
      'features': _("Jupyter kernels for the Spyder console"),
      'required_version': SPYDER_KERNELS_REQVER},
+    {'modname': 'textdistance',
+     'package_name': "textdistance",
+     'features': _('Compute distances between strings'),
+     'required_version': TEXTDISTANCE_REQVER},
     {'modname': "three_merge",
      'package_name': "three-merge",
      'features': _("3-way merge algorithm to merge document changes"),
diff --git a/spyder/plugins/editor/panels/codefolding.py b/spyder/plugins/editor/panels/codefolding.py
index 85f6d6c5f76..4c48dfd6f43 100644
--- a/spyder/plugins/editor/panels/codefolding.py
+++ b/spyder/plugins/editor/panels/codefolding.py
@@ -17,9 +17,11 @@
 """
 
 # Standard library imports
-import sys
-from math import ceil
+import bisect
 from intervaltree import IntervalTree
+from math import ceil
+import sys
+import uuid
 
 # Third party imports
 from qtpy.QtCore import Signal, QSize, QPointF, QRectF, QRect, Qt
@@ -27,6 +29,7 @@
 from qtpy.QtGui import (QTextBlock, QColor, QFontMetricsF, QPainter,
                         QLinearGradient, QPen, QPalette, QResizeEvent,
                         QCursor)
+import textdistance
 
 # Local imports
 from spyder.plugins.editor.api.decoration import TextDecoration, DRAW_ORDERS
@@ -36,6 +39,197 @@
 import spyder.utils.icon_manager as ima
 
 
+class FoldingRegion:
+    """Internal representation of a code folding region."""
+
+    def __init__(self, text, fold_range):
+        self.text = text
+        self.fold_range = fold_range
+        self.id = str(uuid.uuid4())
+        self.index = None
+        self.nesting = 0
+        self.children = []
+        self.status = False
+        self.parent = None
+
+    def delete(self):
+        for child in self.children:
+            child.parent = None
+
+        self.children = []
+        if self.parent is not None:
+            self.parent.remove_node(self)
+            self.parent = None
+
+    def add_node(self, node):
+        node.parent = self
+        node.nesting = self.nesting + 1
+        children_ranges = [c.fold_range[0] for c in self.children]
+        node_range = node.fold_range[0]
+        new_index = bisect.bisect_left(children_ranges, node_range)
+        node.index = new_index
+        for child in self.children[new_index:]:
+            child.index += 1
+        self.children.insert(new_index, node)
+
+    def remove_node(self, node):
+        for child in self.children[node.index + 1:]:
+            child.index -= 1
+
+        self.children.pop(node.index)
+        for idx, next_idx in zip(self.children, self.children[1:]):
+            assert idx.index < next_idx.index
+
+    def clone_node(self, node):
+        self.id = node.id
+        self.index = node.index
+        self.nesting = node.nesting
+        self.parent = node.parent
+        self.children = node.children
+        self.status = node.status
+
+        for child in self.children:
+            child.parent = self
+
+        if self.parent is not None:
+            self.parent.replace_node(self.index, self)
+
+    def replace_node(self, index, node):
+        self.children[index] = node
+        node.parent = self
+
+    def __repr__(self):
+        return str(self)
+
+    def __str__(self):
+        return '({0}, {1}, {2}, {3})'.format(
+            self.fold_range, self.text, self.id, self.status)
+
+
+class FoldingStatus(dict):
+    """
+    Code folding status storage.
+
+    This dictionary subclass is used to update and get the status of a
+    folding region without having to deal with the internal representation.
+    """
+
+    def values(self):
+        values = dict.values(self)
+        return [x.status for x in values]
+
+    def __getitem__(self, key):
+        value = dict.__getitem__(self, key)
+        return value.status
+
+    def __setitem__(self, key, value):
+        if isinstance(value, FoldingRegion):
+            dict.__setitem__(self, key, value)
+        else:
+            region = dict.__getitem__(self, key)
+            region.status = value
+
+
+def merge_interval(parent, node):
+    """Build code folding tree representation from interval tree."""
+    match = False
+    start, end = node.fold_range
+    while parent.parent is not None and not match:
+        parent_start, parent_end = parent.fold_range
+        if parent_end <= start:
+            parent = parent.parent
+        else:
+            match = True
+
+    if node.parent is not None:
+        node.parent.remove_node(node)
+        node.parent = None
+
+    parent.add_node(node)
+    return node
+
+
+def merge_folding(ranges, current_tree, root):
+    """Compare previous and current code folding tree information."""
+    folding_ranges = []
+    for starting_line, ending_line, text in ranges:
+        if ending_line > starting_line:
+            starting_line += 1
+            ending_line += 1
+            folding_repr = FoldingRegion(text, (starting_line, ending_line))
+            folding_ranges.append((starting_line, ending_line, folding_repr))
+
+    tree = IntervalTree.from_tuples(folding_ranges)
+    changes = tree - current_tree
+    deleted = current_tree - tree
+    adding_folding = len(changes) > len(deleted)
+
+    deleted_iter = iter(sorted(deleted))
+    changes_iter = iter(sorted(changes))
+    deleted_entry = next(deleted_iter, None)
+    changed_entry = next(changes_iter, None)
+    non_merged = 0
+
+    while deleted_entry is not None and changed_entry is not None:
+        deleted_entry_i = deleted_entry.data
+        changed_entry_i = changed_entry.data
+        dist = textdistance.jaro_winkler.normalized_similarity(
+            deleted_entry_i.text, changed_entry_i.text)
+
+        if dist >= 0.80:
+            # Copy folding status
+            changed_entry_i.clone_node(deleted_entry_i)
+            deleted_entry = next(deleted_iter, None)
+            changed_entry = next(changes_iter, None)
+        else:
+            if adding_folding:
+                # New symbol added
+                non_merged += 1
+                changed_entry = next(changes_iter, None)
+            else:
+                # Symbol removed
+                deleted_entry_i.delete()
+                non_merged += 1
+                deleted_entry = next(deleted_iter, None)
+
+    if deleted_entry is not None:
+        while deleted_entry is not None:
+            # Symbol removed
+            deleted_entry_i = deleted_entry.data
+            deleted_entry_i.delete()
+            non_merged += 1
+            deleted_entry = next(deleted_iter, None)
+
+    if changed_entry is not None:
+        while changed_entry is not None:
+            non_merged += 1
+            changed_entry = next(changes_iter, None)
+
+    if non_merged > 0:
+        tree_copy = IntervalTree(tree)
+        tree_copy.merge_overlaps(
+            data_reducer=merge_interval,
+            data_initializer=root)
+    return tree, root
+
+
+def collect_folding_regions(root):
+    queue = [(x, 0, -1) for x in root.children]
+    folding_status = FoldingStatus({})
+    folding_regions = {}
+    folding_nesting = {}
+    folding_levels = {}
+    while queue != []:
+        node, folding_level, folding_nest = queue.pop(0)
+        start, end = node.fold_range
+        folding_regions[start] = end
+        folding_levels[start] = folding_level
+        folding_nesting[start] = folding_nest
+        folding_status[start] = node
+        queue = [(x, folding_level + 1, start) for x in node.children] + queue
+    return folding_regions, folding_nesting, folding_levels, folding_status
+
+
 class FoldingPanel(Panel):
 
     """
@@ -171,6 +365,8 @@ def __init__(self, highlight_caret_scope=False):
         self._display_folding = False
         self._key_pressed = False
         self._highlight_runner = DelayJobRunner(delay=250)
+        self.current_tree = IntervalTree()
+        self.root = FoldingRegion(None, None)
         self.folding_regions = {}
         self.folding_status = {}
         self.folding_levels = {}
@@ -192,49 +388,14 @@ def update_folding(self, ranges):
         """Update folding panel folding ranges."""
         if ranges is None:
             return
-        new_folding_ranges = {}
-        for starting_line, ending_line in ranges:
-            if ending_line > starting_line:
-                new_folding_ranges[starting_line + 1] = ending_line + 1
-
-        past_folding_regions = dict(self.folding_regions.copy())
-        self.folding_regions = new_folding_ranges
-        folding_status = {line: False for line in self.folding_regions}
-
-        if len(folding_status) == len(self.folding_status):
-            # No folding lines were introduced before/after
-            self.folding_status = dict(
-                zip(folding_status.keys(), self.folding_status.values()))
-        else:
-            # Line difference computation is done in order
-            # to detect if folding regions were not modified.
-            differ = self.editor.differ
-            diff, previous_text = self.editor.text_diff
-            current_text = self.editor.toPlainText()
-            prev_offsets = self.__compute_line_offsets(previous_text)
-            current_lines = self.__compute_line_offsets(current_text, True)
-
-            for line in past_folding_regions:
-                offset = prev_offsets.get(line)
-                if offset:
-                    new_offset = differ.diff_xIndex(diff, offset)
-                    new_line = current_lines.get(new_offset)
-                    if new_line and new_line in self.folding_regions:
-                        try:
-                            folding_status[new_line] = self.folding_status[line]
-                        except KeyError:
-                            pass
-            self.folding_status = folding_status
-        # Compute region nesting level
-        self.folding_levels = {start_line: 0 for start_line in self.folding_regions}
-        self.folding_nesting = {start_line: -1 for start_line in self.folding_regions}
-        tree = IntervalTree.from_tuples(self.folding_regions.items())
-        for (start, end) in self.folding_regions.items():
-            nested_regions = tree.envelop(start, end)
-            for region in nested_regions:
-                if region.begin != start:
-                    self.folding_levels[region.begin] += 1
-                    self.folding_nesting[region.begin] = start
+
+        self.current_tree, self.root = merge_folding(
+            ranges, self.current_tree, self.root)
+
+        folding_info = collect_folding_regions(self.root)
+
+        (self.folding_regions, self.folding_nesting,
+         self.folding_levels, self.folding_status) = folding_info
         self.update()
 
     def sizeHint(self):
@@ -285,6 +446,8 @@ def paintEvent(self, event):
         # on the folding panel.
         super(FoldingPanel, self).paintEvent(event)
         painter = QPainter(self)
+        document = self.editor.document()
+
         if not self._display_folding and not self._key_pressed:
             if any(self.folding_status.values()):
                 for info in self.editor.visible_blocks:
@@ -657,10 +820,13 @@ def _on_key_pressed(self, event):
         Override key press to select the current scope if the user wants
         to deleted a folded scope (without selecting it).
         """
-        delete_request = event.key() in [Qt.Key_Backspace,
-                                         Qt.Key_Delete]
+        delete_request = event.key() in {Qt.Key_Delete, Qt.Key_Backspace}
+        cursor = self.editor.textCursor()
+        if cursor.hasSelection():
+            if event.key() == Qt.Key_Return:
+                delete_request = True
+
         if event.text() or delete_request:
-            cursor = self.editor.textCursor()
             self._key_pressed = True
             if cursor.hasSelection():
                 # change selection to encompass the whole scope.
@@ -669,11 +835,10 @@ def _on_key_pressed(self, event):
                 positions_to_check = (cursor.position(), )
             for pos in positions_to_check:
                 block = self.editor.document().findBlock(pos)
-                start_line = block.blockNumber() + 1
+                start_line = block.blockNumber() + 2
                 if (start_line in self.folding_regions and
                         self.folding_status[start_line]):
                     end_line = self.folding_regions[start_line]
-                    self.toggle_fold_trigger(block.next())
                     if delete_request and cursor.hasSelection():
                         tc = TextHelper(self.editor).select_lines(start_line, end_line)
                         if tc.selectionStart() > cursor.selectionStart():
@@ -687,8 +852,6 @@ def _on_key_pressed(self, event):
                         tc.setPosition(start)
                         tc.setPosition(end, tc.KeepAnchor)
                         self.editor.setTextCursor(tc)
-                        self.folding_regions.pop(start_line)
-                        self.folding_status.pop(start_line)
             self._key_pressed = False
 
     @staticmethod
diff --git a/spyder/plugins/editor/widgets/base.py b/spyder/plugins/editor/widgets/base.py
index c14820c7fb1..440bee07372 100644
--- a/spyder/plugins/editor/widgets/base.py
+++ b/spyder/plugins/editor/widgets/base.py
@@ -681,6 +681,50 @@ def __move_line_or_selection(self, after_current_line=True):
         start_pos, end_pos = self.__save_selection()
         last_line = False
 
+        # Unfold any folded code block before moving lines up/down
+        folding_panel = self.panels.get('FoldingPanel')
+        fold_start_line = cursor.blockNumber() + 1
+        block = cursor.block().next()
+
+        if fold_start_line in folding_panel.folding_status:
+            fold_status = folding_panel.folding_status[fold_start_line]
+            if fold_status:
+                folding_panel.toggle_fold_trigger(block)
+
+        if after_current_line:
+            # Unfold any folded region when moving lines down
+            fold_start_line = cursor.blockNumber() + 2
+            block = cursor.block().next().next()
+
+            if fold_start_line in folding_panel.folding_status:
+                fold_status = folding_panel.folding_status[fold_start_line]
+                if fold_status:
+                    folding_panel.toggle_fold_trigger(block)
+        else:
+            # Unfold any folded region when moving lines up
+            block = cursor.block()
+            offset = 0
+            if self.has_selected_text():
+                ((selection_start, _),
+                 (selection_end)) = self.get_selection_start_end()
+                if selection_end != selection_start:
+                    offset = 1
+            fold_start_line = block.blockNumber() - 1 - offset
+
+            # Find the innermost code folding region for the current position
+            enclosing_regions = sorted(list(
+                folding_panel.current_tree[fold_start_line]))
+
+            if len(enclosing_regions) > 0:
+                innermost_region = enclosing_regions[-1]
+                fold_start_line = innermost_region.begin
+                block = self.document().findBlockByNumber(fold_start_line)
+
+                if fold_start_line in folding_panel.folding_status:
+                    fold_status = folding_panel.folding_status[fold_start_line]
+                    if fold_status:
+                        folding_panel.toggle_fold_trigger(block)
+
         # ------ Select text
         # Get selection start location
         cursor.setPosition(start_pos)
diff --git a/spyder/plugins/editor/widgets/codeeditor.py b/spyder/plugins/editor/widgets/codeeditor.py
index 23b74356075..01b8029fb24 100644
--- a/spyder/plugins/editor/widgets/codeeditor.py
+++ b/spyder/plugins/editor/widgets/codeeditor.py
@@ -1742,13 +1742,21 @@ def handle_folding_range(self, response):
         """Handle folding response."""
         try:
             ranges = response['params']
+            if ranges is None:
+                return
+
             folding_panel = self.panels.get(FoldingPanel)
 
             # Update folding
             text = self.toPlainText()
             self.text_diff = (self.differ.diff_main(self.previous_text, text),
                               self.previous_text)
-            folding_panel.update_folding(ranges)
+            extended_ranges = []
+            for start, end in ranges:
+                text_region = self.get_text_region(start, end)
+                extended_ranges.append((start, end, text_region))
+
+            folding_panel.update_folding(extended_ranges)
 
             # Update indent guides, which depend on folding
             if self.indent_guides._enabled and len(self.patch) > 0:


Code review comments:
diff_hunk: @@ -36,6 +39,197 @@
 import spyder.utils.icon_manager as ima
 
 
+class FoldingRegion:
+    """Internal representation of a code folding region."""
+
+    def __init__(self, text, fold_range):
+        self.text = text
+        self.fold_range = fold_range
+        self.id = str(uuid.uuid4())
+        self.index = None
+        self.nesting = 0
+        self.children = []
+        self.status = False
+        self.parent = None
+
+    def delete(self):
+        for child in self.children:
+            child.parent = None
+
+        self.children = []
+        if self.parent is not None:
+            self.parent.remove_node(self)
+            self.parent = None
+
+    def add_node(self, node):
+        node.parent = self
+        node.nesting = self.nesting + 1
+        children_ranges = [c.fold_range[0] for c in self.children]
+        node_range = node.fold_range[0]
+        new_index = bisect.bisect_left(children_ranges, node_range)
+        node.index = new_index
+        for child in self.children[new_index:]:
+            child.index += 1
+        self.children.insert(new_index, node)
+
+    def remove_node(self, node):
+        for child in self.children[node.index + 1:]:
+            child.index -= 1
+
+        self.children.pop(node.index)
+        for idx, next_idx in zip(self.children, self.children[1:]):
+            assert idx.index < next_idx.index
+
+    def clone_node(self, node):
+        self.id = node.id
+        self.index = node.index
+        self.nesting = node.nesting
+        self.parent = node.parent
+        self.children = node.children
+        self.status = node.status
+
+        for child in self.children:
+            child.parent = self
+
+        if self.parent is not None:
+            self.parent.replace_node(self.index, self)
+
+    def replace_node(self, index, node):
+        self.children[index] = node
+        node.parent = self
+
+    def __repr__(self):
+        return str(self)
+
+    def __str__(self):
+        return '({0}, {1}, {2}, {3})'.format(
+            self.fold_range, self.text, self.id, self.status)
+
+
+class FoldingStatus(dict):
+    """
+    Code folding status storage.
+
+    This dictionary subclass is used to update and get the status of a
+    folding region without having to deal with the internal representation.
+    """
+
+    def values(self):
+        values = dict.values(self)
+        return [x.status for x in values]
+
+    def __getitem__(self, key):
+        value = dict.__getitem__(self, key)
+        return value.status
+
+    def __setitem__(self, key, value):
+        if isinstance(value, FoldingRegion):
+            dict.__setitem__(self, key, value)
+        else:
+            region = dict.__getitem__(self, key)
+            region.status = value
+
+
+def merge_interval(parent, node):
+    """Build code folding tree representation from interval tree."""
+    match = False
+    start, end = node.fold_range
+    while parent.parent is not None and not match:
+        parent_start, parent_end = parent.fold_range
+        if parent_end <= start:
+            parent = parent.parent
+        else:
+            match = True
+
+    if node.parent is not None:
+        node.parent.remove_node(node)
+        node.parent = None
+
+    parent.add_node(node)
+    return node
+
+
+def merge_folding(ranges, current_tree, root):
+    """Compare previous and current code folding tree information."""
+    folding_ranges = []
+    for starting_line, ending_line, text in ranges:
+        if ending_line > starting_line:
+            starting_line += 1
+            ending_line += 1
+            folding_repr = FoldingRegion(text, (starting_line, ending_line))
+            folding_ranges.append((starting_line, ending_line, folding_repr))
+
+    tree = IntervalTree.from_tuples(folding_ranges)
+    changes = tree - current_tree
+    deleted = current_tree - tree
+    adding_folding = len(changes) > len(deleted)
+
+    deleted_iter = iter(sorted(deleted))
+    changes_iter = iter(sorted(changes))
+    deleted_entry = next(deleted_iter, None)
+    changed_entry = next(changes_iter, None)
+    non_merged = 0
+
+    while deleted_entry is not None and changed_entry is not None:
+        deleted_entry_i = deleted_entry.data
+        changed_entry_i = changed_entry.data
+        dist = textdistance.jaro_winkler.normalized_similarity(
+            deleted_entry_i.text, changed_entry_i.text)
+
+        if dist >= 0.80:
path: spyder/plugins/editor/panels/codefolding.py
text: @user1:
What's this `0.80` means?

@author:
That is 80% of similiarity between the strings

diff_hunk: @@ -681,6 +681,50 @@ def __move_line_or_selection(self, after_current_line=True):
         start_pos, end_pos = self.__save_selection()
         last_line = False
 
+        # Unfold any folded code block before moving lines up/down
+        folding_panel = self.panels.get('FoldingPanel')
path: spyder/plugins/editor/widgets/base.py
text: @user1:
This class (`TextEditBaseWidget`) is shared between the editor and the Internal console. And the Internal console doesn't have panels.

So I'd suggest to redefine `__move_line_or_selection` in `CodeEditor` because all the code you added only applies to it. Of course, you can call `super().__move_line_or_selection()` there to call what you need from this class.