Repository: spyder-ide/spyder

Issue:
Pandas Series index wrong in Variable Explorer
## Problem Description

When viewing a Series in the variable explorer, sorting the series or the index does not change the index displayed

### What steps reproduce the problem?

1. Double Click a Pandas Series in the Variable Explorer to Open the floating pane
2. Sort the series either up or down on values, or up or down on index
3. Index does not change

### What is the expected output? What do you see instead?

Expected output--index order will change when the series is sorted in various ways.

Actual output:
![image](https://user-images.githubusercontent.com/30508983/90320804-632bd600-df01-11ea-8f72-bad0be03d4ab.png)


### Paste Traceback/Error Below (if applicable)

```python-traceback

NONE

```

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

Spyder 4.1.4  
Python 3.7.6 64-bit  
Qt 5.9.6  
PyQt5 5.9.2  
Windows 10  

### Dependencies

```

# Mandatory:
atomicwrites >=1.2.0           :  1.3.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.3.0 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.12.0 (OK)
jedi =0.17.1                   :  0.17.1 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  0.9.2 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso =0.7.0                   :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.6.7 (OK)
pygments >=2.0                 :  2.5.2 (OK)
pylint >=1.0                   :  2.4.4 (OK)
pyls >=0.34.0;<1.0.0           :  0.34.1 (OK)
qdarkstyle >=2.8               :  2.8 (OK)
qtawesome >=0.5.7              :  0.6.1 (OK)
qtconsole >=4.6.0              :  4.6.0 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
rtree >=0.8.3                  :  0.9.3 (OK)
sphinx >=0.6.6                 :  2.3.1 (OK)
spyder_kernels >=1.9.2;<1.10.0 :  1.9.2 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  18.1.1 (OK)

# Optional:
cython >=0.21                  :  0.29.13 (OK)
matplotlib >=2.0.0             :  3.2.1 (OK)
numpy >=1.7                    :  1.17.4 (OK)
pandas >=0.13.1                :  1.0.3 (OK)
scipy >=0.17.0                 :  1.5.0 (OK)
sympy >=0.7.3                  :  1.5.1 (OK)

# Spyder plugins:
spyder_terminal                :  0.3.1 (OK)

```



Discussion:
Hey @eafpres, thanks for reporting. I think this an important bug and we'll try to fix it in our next release.
The problem exists in Pandas Series and Pandas DataFrames in the same form.
The problem does not exist for lists.
There is a variant of the problem for NumPy arrays--they don't sort at all.
Here is a reproducible example:

`
\# -*- coding: utf-8 -*-
"""
Created on Sun Aug 16 12:00:16 2020

@author: Blaine Bateman (eafpres)
"""
\#
\#%% libraries
\#
import pandas as pd
import numpy as np
\#
\#%% create data
\#
my_series = pd.Series(np.random.sample(1000))
my_list = list(np.random.random_integers(0, 1000, 1000))
my_dataframe = pd.DataFrame({'reals' : my_series,
                             'ints' : my_list})
my_array_1D = np.array(my_series)
my_array_2D = np.array(my_dataframe)
\#
\# now for each object:
\#  open Variable explorer
\#  click on the column or index headins
\#  list sorts data and index correctly
\#  Pandas Series and Pandas DataFrame do not sort index correctly
\#  NumPy objects do not sort at all
\#
`
Do you know what is the latest version that does not have this issue?

> Do you know what is the latest version that does not have this issue?

In 4.1.2, Pandas Series and DataFrames sort correctly.
In 4.1.2, NumPy objects don't sort at all (as in 4.1.4)
In 4.1.2, Python Lists sort erratically:

![image](https://user-images.githubusercontent.com/30508983/90551777-14737b80-e14f-11ea-9bc5-e47ad04b45d0.png)

In the above, the list is 1000 element long. Sorting on the index should put 999 or 0 at the top.   
Scrolling down in the list and sorting produces different results depending on the location:
![image](https://user-images.githubusercontent.com/30508983/90552069-821fa780-e14f-11ea-84dd-9d596647eec5.png)


Resolving patch:
diff --git a/spyder/plugins/variableexplorer/widgets/dataframeeditor.py b/spyder/plugins/variableexplorer/widgets/dataframeeditor.py
--- a/spyder/plugins/variableexplorer/widgets/dataframeeditor.py
+++ b/spyder/plugins/variableexplorer/widgets/dataframeeditor.py
@@ -366,6 +366,10 @@ def data(self, index, role=Qt.DisplayRole):
                          "an error ocurred while trying to do it")
         return to_qvariant()
 
+    def recalculate_index(self):
+        """Recalcuate index information."""
+        self.df_index_list = self.df.index.tolist()
+
     def sort(self, column, order=Qt.AscendingOrder):
         """Overriding sort method"""
         if self.complex_intran is not None:
@@ -397,6 +401,8 @@ def sort(self, column, order=Qt.AscendingOrder):
                     QMessageBox.critical(self.dialog, "Error",
                                          "SystemError: %s" % to_text_string(e))
             else:
+                # Update index list
+                self.recalculate_index()
                 # To sort by index
                 self.df.sort_index(inplace=True, ascending=ascending)
         except TypeError as e:
@@ -580,7 +586,6 @@ def sortByColumn(self, index):
         if self.sort_old == [None]:
             self.header_class.setSortIndicatorShown(True)
         sort_order = self.header_class.sortIndicatorOrder()
-        self.sig_sort_by_column.emit()
         if not self.model().sort(index, sort_order):
             if len(self.sort_old) != 2:
                 self.header_class.setSortIndicatorShown(False)
@@ -589,6 +594,7 @@ def sortByColumn(self, index):
                                                    self.sort_old[1])
             return
         self.sort_old = [index, self.header_class.sortIndicatorOrder()]
+        self.sig_sort_by_column.emit()
 
     def contextMenuEvent(self, event):
         """Reimplement Qt method."""
@@ -1338,6 +1344,8 @@ def _sort_update(self):
 
         Uses the model of the dataTable as the base.
         """
+        # Update index list calculation
+        self.dataModel.recalculate_index()
         self.setModel(self.dataTable.model())
 
     def _fetch_more_columns(self):


Test patch:
diff --git a/spyder/plugins/variableexplorer/widgets/tests/test_dataframeeditor.py b/spyder/plugins/variableexplorer/widgets/tests/test_dataframeeditor.py
--- a/spyder/plugins/variableexplorer/widgets/tests/test_dataframeeditor.py
+++ b/spyder/plugins/variableexplorer/widgets/tests/test_dataframeeditor.py
@@ -83,6 +83,22 @@ def generate_pandas_indexes():
 # =============================================================================
 # Tests
 # =============================================================================
+def test_dataframemodel_index_sort(qtbot):
+    """Validate the data in the model for index when sorting."""
+    ds = Series(numpy.arange(10))
+    editor = DataFrameEditor(None)
+    editor.setup_and_check(ds)
+    index = editor.table_index.model()
+
+    index.sort(-1, order=Qt.AscendingOrder)
+    assert data_index(index, 0, 0, Qt.DisplayRole) == '0'
+    assert data_index(index, 9, 0, Qt.DisplayRole) == '9'
+
+    index.sort(-1, order=Qt.DescendingOrder)
+    assert data_index(index, 0, 0, Qt.DisplayRole) == '9'
+    assert data_index(index, 9, 0, Qt.DisplayRole) == '0'
+
+
 def test_dataframe_to_type(qtbot):
     """Regression test for spyder-ide/spyder#12296"""
     # Setup editor