Repository: spyder-ide/spyder

Issue:
Error when trying to sort dataframe with duplicate columns
## Description

**What steps will reproduce the problem?**

1. 
2. 
3. 

**What is the expected output? What do you see instead?**


**Please provide any additional information below**

  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/dataframeeditor.py", line 562, in sortByColumn
    if not self.model().sort(index, sort_order):
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/dataframeeditor.py", line 312, in sort
    kind='mergesort')
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 3196, in sort_values
    str(by))
ValueError: Cannot sort by duplicate column cv1


## Version and main components

* Spyder Version: 3.2.3 
* Python Version: 3.5.3
* Qt Versions: 5.7.1, PyQt5 5.7 on Linux

## Dependencies
```
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.20.3 (OK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
psutil >=0.3     :  5.3.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  None (NOK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.2 (OK)

```

Error when trying to sort dataframe with duplicate columns
## Description

**What steps will reproduce the problem?**

1. 
2. 
3. 

**What is the expected output? What do you see instead?**


**Please provide any additional information below**

  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/dataframeeditor.py", line 562, in sortByColumn
    if not self.model().sort(index, sort_order):
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/dataframeeditor.py", line 312, in sort
    kind='mergesort')
  File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 3196, in sort_values
    str(by))
ValueError: Cannot sort by duplicate column cv1


## Version and main components

* Spyder Version: 3.2.3 
* Python Version: 3.5.3
* Qt Versions: 5.7.1, PyQt5 5.7 on Linux

## Dependencies
```
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.20.3 (OK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
psutil >=0.3     :  5.3.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  None (NOK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.2 (OK)

```



Discussion:
Please post the code you used to generate the Dataframe you were trying to sort.
Sorry.
I've made dtaframe with 2 identical columns. and when i tried to sort it in variable preview - error occured.
Thanks for reporting. We'll try to fix this in a future release.
pandas does not allow to sort `DataFrame` on a duplicate column:
```python
In[10]: df
Out[10]: 
   A  A
0  1  1
1  2  2
2  3  3

In[11]: df.sort_values(by='A')
Out[11]: Traceback (most recent call last):
  File "<ipython-input-9-4119ec283a31>", line 1, in <module>
    df.sort_values(by='A')
  File "C:\...\site-packages\pandas\core\frame.py", line 3294, in sort_values
    str(by))

ValueError: Cannot sort by duplicate column A
```
@ccordoba12 how are such things handled in spyder? Should the error be catched and then the sorting is not performed? Or should a specific error message be displayed to the user? 
@Prikers, I think we should display a message for this because users don't know that Pandas can't perform this operation.
Please post the code you used to generate the Dataframe you were trying to sort.
Sorry.
I've made dtaframe with 2 identical columns. and when i tried to sort it in variable preview - error occured.
Thanks for reporting. We'll try to fix this in a future release.
pandas does not allow to sort `DataFrame` on a duplicate column:
```python
In[10]: df
Out[10]: 
   A  A
0  1  1
1  2  2
2  3  3

In[11]: df.sort_values(by='A')
Out[11]: Traceback (most recent call last):
  File "<ipython-input-9-4119ec283a31>", line 1, in <module>
    df.sort_values(by='A')
  File "C:\...\site-packages\pandas\core\frame.py", line 3294, in sort_values
    str(by))

ValueError: Cannot sort by duplicate column A
```
@ccordoba12 how are such things handled in spyder? Should the error be catched and then the sorting is not performed? Or should a specific error message be displayed to the user? 
@Prikers, I think we should display a message for this because users don't know that Pandas can't perform this operation.

Resolving patch:
diff --git a/spyder/widgets/variableexplorer/dataframeeditor.py b/spyder/widgets/variableexplorer/dataframeeditor.py
--- a/spyder/widgets/variableexplorer/dataframeeditor.py
+++ b/spyder/widgets/variableexplorer/dataframeeditor.py
@@ -315,6 +315,10 @@ def sort(self, column, order=Qt.AscendingOrder):
                     self.df.sort(columns=self.df.columns[column-1],
                                  ascending=ascending, inplace=True,
                                  kind='mergesort')
+                except ValueError as e:
+                    # Not possible to sort on duplicate columns #5225
+                    QMessageBox.critical(self.dialog, "Error",
+                                         "ValueError: %s" % to_text_string(e))
                 self.update_df_index()
             else:
                 self.df.sort_index(inplace=True, ascending=ascending)


Test patch:
diff --git a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
--- a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
+++ b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
@@ -17,18 +17,20 @@
 import os
 
 # Third party imports
-from pandas import DataFrame, date_range, read_csv
+from pandas import DataFrame, date_range, read_csv, concat
+from qtpy import PYQT4
 from qtpy.QtGui import QColor
-from qtpy.QtCore import Qt
+from qtpy.QtCore import Qt, QTimer
 import numpy
 import pytest
 
 # Local imports
 from spyder.utils.programs import is_module_installed
+from spyder.utils.test import close_message_box
 from spyder.widgets.variableexplorer import dataframeeditor
 from spyder.widgets.variableexplorer.dataframeeditor import (
     DataFrameEditor, DataFrameModel)
-from spyder.py3compat import PY2
+from spyder.py3compat import PY2, PY3
 
 FILES_PATH = os.path.dirname(os.path.realpath(__file__))
 
@@ -228,5 +230,22 @@ def test_dataframeeditor_with_datetimeindex():
     assert data(dfm, 2, 1) == '2015-01-03 00:00:00'
 
 
+@pytest.mark.skipif(PYQT4 and PY3,
+                    reason="It generates a strange failure in another test")
+def test_sort_dataframe_with_duplicate_column(qtbot):
+    df = DataFrame({'A': [1, 3, 2], 'B': [4, 6, 5]})
+    df = concat((df, df.A), axis=1)
+    editor = DataFrameEditor(None)
+    editor.setup_and_check(df)
+    dfm = editor.dataModel
+    QTimer.singleShot(1000, lambda: close_message_box(qtbot))
+    editor.dataModel.sort(1)
+    assert [data(dfm, row, 1) for row in range(len(df))] == ['1', '3', '2']
+    assert [data(dfm, row, 2) for row in range(len(df))] == ['4', '6', '5']
+    editor.dataModel.sort(2)
+    assert [data(dfm, row, 1) for row in range(len(df))] == ['1', '2', '3']
+    assert [data(dfm, row, 2) for row in range(len(df))] == ['4', '5', '6']
+
+
 if __name__ == "__main__":
     pytest.main()