Repository: spyder-ide/spyder

Issue:
Hard crash when opening Numpy array in VarExp  with one or more np.object fields
## Description

### What steps will reproduce the problem?

<!--- You can use Markdown here --->

@ccordoba12 I get a consistently reproducible (including under ``--safe-mode``) Spyder hard crash to desktop on the latest ``master``, trying to open a Numpy record array (``np.void`` type) on which at least one of the sub-arrays is of the ``np.object`` type. Can be easily repro'd by attempting to open e.g. `` x = np.array([("spam", 2), ("eggs", 4)], dtype=[('x', np.object), ('y', int)])``in the Variable Explorer. The exception is:

```python-traceback
  File "C:\Users\C. A. M. Gerlach\Documents\dev\SpyderDev\spyder\spyder\plugins\variableexplorer\widgets\arrayeditor.py", line 279, in data
    / (float(self.vmax) - self.vmin))
ValueError: could not convert string to float: 'fsd'
```

Curiously, what actually triggers the hard crash, though, appears to be something to do with the resulting error dialog as if I enable ``--debug`` mode I don't get the Qt crash, just the exception printed to the terminal, and the crash happens a few hundred ms after the error dialog appears. The console messages (debug off) that appear simultanious to the hard crash are:

```
QWidget::repaint: Recursive repaint detected
QBackingStore::endPaint() called with active painter on backingstore paint device
```
   
 
## Proposed solution

 The root cause of the exception that initially triggers the error dialog (which actually causes the crash itself) is Spyder supporting record arrays just fine, but not any object arrays they contain.

Therefore, the simplest way to handle this would just be to check the types of the contained fields for np.void arrays to ensure none of them are ``np.object``  and either exclude them from the popup list at bottom left of the ArrayEditor (possibly with a warning on opening the array listing the object fields that could not be viewed), or show an error dialog when trying to switch to them (just like we do for object arrays opened from the other viewer/editors). If all the array's fields are np.object, then it we make it behave exactly as if the array itself were np.object (error dialog).



## Versions

* Spyder version: 4.0.0.dev0 6a918f8b5
* Python version: 3.7.2
* Qt version: 5.9.6
* PyQt5 version: 5.9.2
* Operating System: Windows 8.1

### Dependencies

```
pygments >=2.0    :  2.3.1 (OK)
qdarkstyle >=2.6.4:  2.6.5 (OK)
sphinx >=0.6.6    :  1.8.4 (OK)
pyls >=0.19.0     :  0.21.5 (OK)
nbconvert >=4.0   :  5.4.0 (OK)
pandas >=0.13.1   :  0.24.1 (OK)
numpy >=1.7       :  1.15.4 (OK)
sympy >=0.7.3     :  1.3 (OK)
cython >=0.21     :  0.29.4 (OK)
qtconsole >=4.2.0 :  4.4.3 (OK)
IPython >=4.0     :  7.2.0 (OK)
matplotlib >=2.0.0:  3.0.2 (OK)
pylint >=0.25     :  2.2.2 (OK)
```



Discussion:

Resolving patch:
diff --git a/spyder/plugins/variableexplorer/widgets/arrayeditor.py b/spyder/plugins/variableexplorer/widgets/arrayeditor.py
--- a/spyder/plugins/variableexplorer/widgets/arrayeditor.py
+++ b/spyder/plugins/variableexplorer/widgets/arrayeditor.py
@@ -17,17 +17,18 @@
 from __future__ import print_function
 
 # Third party imports
+import numpy as np
 from qtpy.compat import from_qvariant, to_qvariant
 from qtpy.QtCore import (QAbstractTableModel, QItemSelection, QLocale,
                          QItemSelectionRange, QModelIndex, Qt, Slot)
 from qtpy.QtGui import QColor, QCursor, QDoubleValidator, QKeySequence
 from qtpy.QtWidgets import (QAbstractItemDelegate, QApplication, QCheckBox,
-                            QComboBox, QDialog, QDialogButtonBox, QGridLayout,
-                            QHBoxLayout, QInputDialog, QItemDelegate, QLabel,
-                            QLineEdit,  QMenu, QMessageBox, QPushButton,
-                            QSpinBox, QStackedWidget, QTableView, QVBoxLayout,
+                            QComboBox, QDialog, QGridLayout, QHBoxLayout,
+                            QInputDialog, QItemDelegate, QLabel, QLineEdit,
+                            QMenu, QMessageBox, QPushButton, QSpinBox,
+                            QStackedWidget, QTableView, QVBoxLayout,
                             QWidget)
-import numpy as np
+from spyder_kernels.utils.nsview import value_to_display
 
 # Local imports
 from spyder.config.base import _
@@ -165,6 +166,10 @@ def __init__(self, data, format="%.6g", xlabels=None, ylabels=None,
             self.dhue = None
             self.bgcolor_enabled = False
 
+        # Deactivate coloring for object arrays
+        if self._data.dtype.name == 'object':
+            self.bgcolor_enabled = False
+
         # Use paging when the total size, number of rows or number of
         # columns is too large
         if size > LARGE_SIZE:
@@ -251,28 +256,39 @@ def get_value(self, index):
         return self.changes.get((i, j), value)
 
     def data(self, index, role=Qt.DisplayRole):
-        """Cell content"""
+        """Cell content."""
         if not index.isValid():
             return to_qvariant()
         value = self.get_value(index)
+        dtn = self._data.dtype.name
+
+        # Tranform binary string to unicode so they are displayed
+        # correctly
         if is_binary_string(value):
             try:
                 value = to_text_string(value, 'utf8')
-            except:
+            except Exception:
                 pass
+
+        # Handle roles
         if role == Qt.DisplayRole:
             if value is np.ma.masked:
                 return ''
             else:
-                try:
-                    return to_qvariant(self._format % value)
-                except TypeError:
-                    self.readonly = True
-                    return repr(value)
+                if dtn == 'object':
+                    # We don't know what's inside an object array, so
+                    # we can't trust value repr's here.
+                    return value_to_display(value)
+                else:
+                    try:
+                        return to_qvariant(self._format % value)
+                    except TypeError:
+                        self.readonly = True
+                        return repr(value)
         elif role == Qt.TextAlignmentRole:
             return to_qvariant(int(Qt.AlignCenter|Qt.AlignVCenter))
-        elif role == Qt.BackgroundColorRole and self.bgcolor_enabled \
-          and value is not np.ma.masked:
+        elif (role == Qt.BackgroundColorRole and self.bgcolor_enabled
+                and value is not np.ma.masked):
             try:
                 hue = (self.hue0 +
                        self.dhue * (float(self.vmax) - self.color_func(value))
@@ -280,7 +296,7 @@ def data(self, index, role=Qt.DisplayRole):
                 hue = float(np.abs(hue))
                 color = QColor.fromHsvF(hue, self.sat, self.val, self.alp)
                 return to_qvariant(color)
-            except TypeError:
+            except (TypeError, ValueError):
                 return to_qvariant()
         elif role == Qt.FontRole:
             return to_qvariant(get_font(font_size_delta=DEFAULT_SMALL_DELTA))
@@ -455,7 +471,6 @@ def load_more_data(self, value, rows=False, columns=False):
                     new_selection.append(part)
                 self.selectionModel().select
                 (new_selection, self.selectionModel().ClearAndSelect)
-
         except NameError:
             # Needed to handle a NameError while fetching data when closing
             # See isue 7880
@@ -644,8 +659,17 @@ def setup_and_check(self, data, title='', readonly=False,
             return False
         if not is_record_array:
             dtn = data.dtype.name
-            if dtn not in SUPPORTED_FORMATS and not dtn.startswith('str') \
-               and not dtn.startswith('unicode'):
+            if dtn == 'object':
+                # If the array doesn't have shape, we can't display it
+                if data.shape == ():
+                    self.error(_("Object arrays without shape are not "
+                                 "supported"))
+                    return False
+                # We don't know what's inside these arrays, so we can't handle
+                # edits
+                self.readonly = readonly = True
+            elif (dtn not in SUPPORTED_FORMATS and not dtn.startswith('str')
+                    and not dtn.startswith('unicode')):
                 arr = _("%s arrays") % data.dtype.name
                 self.error(_("%s are currently not supported") % arr)
                 return False
@@ -667,7 +691,8 @@ def setup_and_check(self, data, title='', readonly=False,
         if is_record_array:
             for name in data.dtype.names:
                 self.stack.addWidget(ArrayEditorWidget(self, data[name],
-                                                   readonly, xlabels, ylabels))
+                                                       readonly, xlabels,
+                                                       ylabels))
         elif is_masked_array:
             self.stack.addWidget(ArrayEditorWidget(self, data, readonly,
                                                    xlabels, ylabels))
@@ -683,7 +708,7 @@ def setup_and_check(self, data, title='', readonly=False,
         self.arraywidget = self.stack.currentWidget()
         if self.arraywidget:
             self.arraywidget.model.dataChanged.connect(
-                                                    self.save_and_close_enable)
+                self.save_and_close_enable)
         self.stack.currentChanged.connect(self.current_widget_changed)
         self.layout.addWidget(self.stack, 1, 0)
 
@@ -828,7 +853,7 @@ def accept(self):
 
     def get_value(self):
         """Return modified array -- this is *not* a copy"""
-        # It is import to avoid accessing Qt C++ object as it has probably
+        # It is important to avoid accessing Qt C++ object as it has probably
         # already been destroyed, due to the Qt.WA_DeleteOnClose attribute
         return self.data
 


Test patch:
diff --git a/spyder/plugins/variableexplorer/widgets/tests/test_arrayeditor.py b/spyder/plugins/variableexplorer/widgets/tests/test_arrayeditor.py
--- a/spyder/plugins/variableexplorer/widgets/tests/test_arrayeditor.py
+++ b/spyder/plugins/variableexplorer/widgets/tests/test_arrayeditor.py
@@ -53,6 +53,23 @@ def setup_arrayeditor(qbot, data, title="", xlabels=None, ylabels=None):
 # =============================================================================
 # Tests
 # =============================================================================
+def test_object_arrays(qtbot):
+    """Test that object arrays are working properly."""
+    arr = np.array([u'a', 1, [2]], dtype=object)
+    assert_array_equal(arr, launch_arrayeditor(arr, "object array"))
+
+
+def test_object_arrays_display(qtbot):
+    """
+    Test that value_to_display is being used to display the values of
+    object arrays.
+    """
+    arr = np.array([[np.array([1, 2])], 2], dtype=object)
+    dlg = setup_arrayeditor(qtbot, arr)
+    idx = dlg.arraywidget.model.index(0, 0)
+    assert u'[Numpy array]' == dlg.arraywidget.model.data(idx)
+
+
 def test_type_errors(qtbot):
     """
     Verify that we don't get a TypeError for certain structured arrays.