Repository: spyder-ide/spyder

Issue:
Nan value in dataframe editor crash Spyder
## Description of your problem

1. Run the following script:
```
import numpy as np
import pandas as pd

np_array = np.zeros(100)
np_array[10:] = np.nan
dataframe = pd.DataFrame(np_array)
```

2. Go to the Variable explorer
3. Double click on the ```dataframe``` variable
4. Close the table editor window

**What is the expected output? What do you see instead?**
Spyder completely crashes. This is not always reproducible but opening and closing table editor several times triggers the crash.

## Versions and main components

* Spyder Version: 3.1.2
Python 3.5.2 64bits
Qt 5.7.0
PyQt5 5.7
* Operating system: Kubuntu 16.04

## Dependencies

jedi >=0.8.1     :  0.10.0 (OK)
matplotlib >=1.0 :  1.5.3 (OK)
nbconvert >=4.0  :  5.1.1 (OK)
numpy >=1.7      :  1.12.0 (OK)
pandas >=0.13.1  :  0.18.1 (OK)
pep8 >=0.6       :  1.7.0 (OK)
psutil >=0.3     :  5.1.3 (OK)
pyflakes >=0.6.0 :  1.5.0 (OK)
pygments >=2.0   :  2.2.0 (OK)
pylint >=0.25    :  1.6.5 (OK)
qtconsole >=4.2.0:  4.2.1 (OK)
rope >=0.9.4     :  0.9.4-1 (OK)
sphinx >=0.6.6   :  1.5.2 (OK)
sympy >=0.7.3    :  None (NOK)




Discussion:
I can't reproduce this. How did you install Qt/PyQt?
I don't know how I installed it. All with standard packages and/or pip.

I did some further investigations: 
- The format (in variable explorer) has to be ```%d```. Which I used before for a special dataframe.
- There are exceptions in the internal console.

```
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/dataframeeditor.py", line 277, in data
    return to_qvariant(self._format % value)
ValueError: cannot convert float NaN to integer
```
@jitseniesen, any ideas?
Yes, I understand it after markusr's explanation. The error message complains that the format `%d` is for integers and cannot be applied to NaN.

An easy solution for us would be to use the default format (which is `%.3g`) in this case. Would that be acceptable from a user experience point of view?
> An easy solution for us would be to use the default format (which is %.3g) in this case. Would that be acceptable from a user experience point of view?

@jitseniesen Are you suggesting that non-NaN values are shown as ```%d``` and NaN values as ```%.3g``` (if  ```%d``` is set as the input format)?

Yes
This sounds like a good solution. I also gave it some thoughts but I couldn't come up with better solution.

Anyhow better than the application crash...
@jitseniesen, could you create a PR against the `3.1.x` branch for this? It seems an easy fix but I fail to understand what we should exactly do to fix it :-)

Resolving patch:
diff --git a/spyder/widgets/variableexplorer/dataframeeditor.py b/spyder/widgets/variableexplorer/dataframeeditor.py
--- a/spyder/widgets/variableexplorer/dataframeeditor.py
+++ b/spyder/widgets/variableexplorer/dataframeeditor.py
@@ -43,6 +43,9 @@
 # Used to convert bool intrance to false since bool('False') will return True
 _bool_false = ['false', '0']
 
+# Default format for data frames with floats
+DEFAULT_FORMAT = '%.3g'
+
 # Limit at which dataframe is considered so large that it is loaded on demand
 LARGE_SIZE = 5e5
 LARGE_NROWS = 1e5
@@ -83,7 +86,7 @@ class DataFrameModel(QAbstractTableModel):
     ROWS_TO_LOAD = 500
     COLS_TO_LOAD = 40
     
-    def __init__(self, dataFrame, format="%.3g", parent=None):
+    def __init__(self, dataFrame, format=DEFAULT_FORMAT, parent=None):
         QAbstractTableModel.__init__(self)
         self.dialog = parent
         self.df = dataFrame
@@ -274,7 +277,12 @@ def data(self, index, role=Qt.DisplayRole):
             else:
                 value = self.get_value(row, column-1)
                 if isinstance(value, float):
-                    return to_qvariant(self._format % value)
+                    try:
+                        return to_qvariant(self._format % value)
+                    except (ValueError, TypeError):
+                        # may happen if format = '%d' and value = NaN;
+                        # see issue 4139
+                        return to_qvariant(DEFAULT_FORMAT % value)
                 else:
                     try:
                         return to_qvariant(to_text_string(value))


Test patch:
diff --git a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
--- a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
+++ b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
@@ -20,6 +20,7 @@
 from pandas import DataFrame, date_range, read_csv
 from qtpy.QtGui import QColor
 from qtpy.QtCore import Qt
+import numpy
 import pytest
 
 # Local imports
@@ -156,6 +157,19 @@ def test_dataframemodel_get_bgcolor_with_object():
     a = dataframeeditor.BACKGROUND_MISC_ALPHA
     assert colorclose(bgcolor(dfm, 0, 1), (h, s, v, a))
 
+def test_dataframemodel_with_format_percent_d_and_nan():
+    """
+    Test DataFrameModel with format `%d` and dataframe containing NaN
+
+    Regression test for issue 4139.
+    """
+    np_array = numpy.zeros(2)
+    np_array[1] = numpy.nan
+    dataframe = DataFrame(np_array)
+    dfm = DataFrameModel(dataframe, format='%d')
+    assert data(dfm, 0, 1) == '0'
+    assert data(dfm, 1, 1) == 'nan'
+
 def test_change_format_emits_signal(qtbot, monkeypatch):
     mockQInputDialog = Mock()
     mockQInputDialog.getText = lambda parent, title, label, mode, text: ('%10.3e', True)