Repository: spyder-ide/spyder

Issue:
AttributeError when attempting to view dom.minidom.Element object in Variable Explorer
## Description
variable explorers list view crashes on opening nested xml elements

**What steps will reproduce the problem?**

1. extract content.xml from *.odt, load it into xml.dom.minidom object
2. variable2View = dom.getElementsByTagName(...) becomes a dom.minicompat.NodeList
3. open variable2View, and try to open one of the contained cells 

**What is the expected output? What do you see instead?**
expected to see content of the xml node, got a crash instead

**Please provide any additional information below**

  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 513, in createEditor
    readonly=readonly)
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 1308, in setup
    readonly=readonly, remote=remote)
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 1260, in __init__
    title)
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 1145, in __init__
    minmax=minmax)
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 110, in __init__
    self.set_data(data)
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 158, in set_data
    self.set_size_and_type()
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 178, in set_size_and_type
    for index in range(start, stop) ]
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 178, in <listcomp>
    for index in range(start, stop) ]
  File "/usr/local/lib/python3.5/dist-packages/spyder/widgets/variableexplorer/collectionseditor.py", line 78, in __getitem__
    return getattr(self.__obj__, key)
AttributeError: _localName


## Version and main components

* Spyder Version: 3.2.4 
* Python Version: 3.5.2
* Qt Versions: 5.9.1, PyQt5 5.9 on Linux

## Dependencies
```
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.19.2 (OK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.5 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.11.0 (OK)
psutil >=0.3     :  5.4.0 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  None (NOK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.2.1 (OK)
pylint >=0.25    :  1.7.4 (OK)

```

AttributeError when attempting to edit non-writeable or non-settable attributes of certain (e.g. many common Pandas) objects in CollectionsEditor
## Problem Description

Certain Python objects, especially many common ones from ``pandas``( like ``Period``, ``Categorical``, ``Interval``, the various Indexes,  etc), have attributes that Spyder thinks it can edit directly, but are in fact not writable or not settable. Attempting to edit these attributes e.g by double-clicking them will result in one of the two below ``AttributeError``s. 

This is essentially the general case of the error in #5999 , which in that specific case was fixed by adding full support for timedeltas in the Variable Explorer and thus not allowing the viewing of the arbitrary object with attributes that could trigger the proximate issue. While we may not be able to officially support all pandas objects until Spyder 4, it would be better to handle errors more gracefully.

### What steps will reproduce the problem?

1. Create an object that has attributes that can't be written for whatever reason. e.g.

```python
test_period = pd.Period("2017-01")

test_categorical = pd.Categorical([1, 2, 3, 9])
test_interval = pd.Interval(0, 5)

test_catindex = pd.CategoricalIndex([0, 1, 5, 7])
test_floatindex = pd.Float64Index([2.4, 8.3, 92.0, 323.])
```

2. Doubleclick a directly editable attribute (string, numeric, datetime, etc.) in the CollectionsEditor view of that object to edit it, and enter a different value than the original (for example, ``day``, ``month`` etc. in ``Period`` or ``nbytes``, ``ndim`` etc. in ``Categorical``). 

3. Hit enter to attempt to save the attribute value, and the error will occur.

### Traceback

First case (non-writeable attribute, in this example in ``pandas.Period``)

```python-traceback
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 626, in setModelData
    self.set_value(index, value)
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 387, in set_value
    index.model().set_value(index, value)
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 342, in set_value
    self._data[ self.keys[index.row()] ] = value
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 83, in __setitem__
    setattr(self.__obj__, key, value)
AttributeError: attribute 'day' of 'pandas._libs.period._Period' objects is not writable
```

Second case (Attribute that can't be set, in this example in pandas.Categorical

```python-traceback
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 626, in setModelData
    self.set_value(index, value)
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 387, in set_value
    index.model().set_value(index, value)
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 342, in set_value
    self._data[ self.keys[index.row()] ] = value
  File "C:\Anaconda3\lib\site-packages\spyder\widgets\variableexplorer\collectionseditor.py", line 83, in __setitem__
    setattr(self.__obj__, key, value)
AttributeError: can't set attribute
```

## Package Versions

* Spyder 3.2.7
* Python 3.6.4
* Qt 5.6.2
* PyQt5 5.6
* Windows 8.1

### Dependencies

```
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  0.22.0 (OK)
numpy >=1.7      :  1.14.0 (OK)
sphinx >=0.6.6   :  1.7.1 (OK)
rope >=0.9.4     :  0.10.7 (OK)
jedi >=0.9.0     :  0.11.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
sympy >=0.7.3    :  1.1.1 (OK)
cython >=0.21    :  0.27.3 (OK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.2.1 (OK)
pylint >=0.25    :  1.8.2 (OK)
```



Discussion:
Thanks for reporting, but unfortunately your description is not complete. Please post the Python code you used to get `variable2View` along with a simple odt file we can use in our end to reproduce this problem.

Thanks!
Thanks for getting back to me, and my apologies for the incomplete information.
Instead of the odt file, I attached the content.xml extracted from the odt and had to rename it the txt...

[content226_uifz_block6.txt](https://github.com/spyder-ide/spyder/files/1463566/content226_uifz_block6.txt)

```python
file = "content226_uifz_block6.txt"
if __name__ == '__main__':
    with open(file) as f:
        read_data = f.read()
        
    content = xml.dom.minidom.parseString(read_data)
    tables = content.getElementsByTagName("table:table")
    # the crash happens when opening the tables variable with VariableExplorer right after the above line
```

Generally speaking, the script does what I want it to do, extracting strings from the cells within tables[0] if I don't stop it. And sorry for dumping tables into writer documents, I know it is odd ;-)
![screenshot_2017-11-11_09-16-33](https://user-images.githubusercontent.com/11660472/32687626-12b69364-c6c1-11e7-8112-16a370fcc41c.png)
![screenshot_2017-11-11_09-16-15](https://user-images.githubusercontent.com/11660472/32687627-12d48748-c6c1-11e7-8fbd-1cbdc5d18977.png)


I've reproduced and fixed this by just catching the error where it occurs, and you are now able to view the ``dom.minidom.Element`` just fine. PR incoming shortly. More complete code to reproduce:

```python
from xml.dom.minidom import parseString

file = "Data/content226_uifz_block6.txt"
with open(file) as f:
    read_data = f.read()

    content = parseString(read_data)
    tables = content.getElementsByTagName("table:table")
```


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -218,7 +218,7 @@ def run(self):
 #==============================================================================
 EXTLIST = ['.mo', '.svg', '.png', '.css', '.html', '.js', '.chm', '.ini',
            '.txt', '.rst', '.qss', '.ttf', '.json', '.c', '.cpp', '.java',
-           '.md', '.R', '.csv', '.pyx', '.ipynb']
+           '.md', '.R', '.csv', '.pyx', '.ipynb', '.xml']
 if os.name == 'nt':
     SCRIPTS += ['spyder.bat']
     EXTLIST += ['.ico']
diff --git a/spyder/widgets/variableexplorer/collectionseditor.py b/spyder/widgets/variableexplorer/collectionseditor.py
--- a/spyder/widgets/variableexplorer/collectionseditor.py
+++ b/spyder/widgets/variableexplorer/collectionseditor.py
@@ -77,21 +77,33 @@ def __len__(self):
         return len(get_object_attrs(self.__obj__))
 
     def __getitem__(self, key):
-        """Get the attribute corresponding to given key."""
-        # Fix #6284 where pandas MultiIndex returns NotImplementedError
-        # Due to NA checking not being supported on a multiindex.
+        """Get the attribute corresponding to the given key."""
+        # Catch NotImplementedError to fix #6284 in pandas MultiIndex
+        # due to NA checking not being supported on a multiindex.
+        # Catch AttributeError to fix #5642 in certain special classes like xml
+        # when this method is called on certain attributes.
+        # Catch TypeError to prevent fatal Python crash to desktop after
+        # modifying certain pandas objects. Fix issue #6727 .
+        # Catch ValueError to allow viewing and editing of pandas offsets.
+        # Fix issue #6728 .
         try:
             attribute_toreturn = getattr(self.__obj__, key)
-        except NotImplementedError:
+        except (NotImplementedError, AttributeError, TypeError, ValueError):
             attribute_toreturn = None
         return attribute_toreturn
 
     def __setitem__(self, key, value):
         """Set attribute corresponding to key with value."""
+        # Catch AttributeError to gracefully handle inability to set an
+        # attribute due to it not being writeable or set-table.
+        # Fix issue #6728 . Also, catch NotImplementedError for safety.
         try:
             setattr(self.__obj__, key, value)
-        except TypeError:
+        except (TypeError, AttributeError, NotImplementedError):
             pass
+        except Exception as e:
+            if "cannot set values for" not in str(e):
+                raise
 
 
 class ReadOnlyCollectionsModel(QAbstractTableModel):


Test patch:
diff --git a/spyder/widgets/variableexplorer/tests/dom_element_test.xml b/spyder/widgets/variableexplorer/tests/dom_element_test.xml
new file mode 100644
--- /dev/null
+++ b/spyder/widgets/variableexplorer/tests/dom_element_test.xml
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<note>
+  <to>Carlos</to>
+  <from>Pierre</from>
+  <heading>Spyder</heading>
+  <body>Spyder is an amazing IDE!</body>
+</note>
diff --git a/spyder/widgets/variableexplorer/tests/test_collectioneditor.py b/spyder/widgets/variableexplorer/tests/test_collectioneditor.py
--- a/spyder/widgets/variableexplorer/tests/test_collectioneditor.py
+++ b/spyder/widgets/variableexplorer/tests/test_collectioneditor.py
@@ -7,13 +7,15 @@
 # ----------------------------------------------------------------------------
 
 """
-Tests for collectionseditor.py .
+Tests for the Variable Explorer Collections Editor.
 """
 
 # Standard library imports
-import os  # Example module for testing display inside collecitoneditor
+import os  # Example module for testing display inside CollecitonsEditor
+from os import path
 import copy
 import datetime
+from xml.dom.minidom import parseString
 try:
     from unittest.mock import Mock
 except ImportError:
@@ -24,6 +26,7 @@
 import pandas
 import pytest
 from flaky import flaky
+from qtpy.QtCore import Qt
 from qtpy.QtWidgets import QWidget
 
 # Local imports
@@ -32,6 +35,13 @@
     LARGE_NROWS, ROWS_TO_LOAD)
 
 
+# =============================================================================
+# Constants
+# =============================================================================
+# Full path to this file's parent directory for loading data
+LOCATION = path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
+
+
 # =============================================================================
 # Utility functions
 # =============================================================================
@@ -43,6 +53,19 @@ def data_table(cm, n_rows, n_cols):
     return [[data(cm, i, j) for i in range(n_rows)] for j in range(n_cols)]
 
 
+# =============================================================================
+# Pytest Fixtures
+# =============================================================================
+@pytest.fixture
+def nonsettable_objects_data():
+    """Rturn Python objects with immutable attribs to test CollectionEditor."""
+    test_objs = [pandas.Period("2018-03"), pandas.Categorical([1, 2, 42])]
+    expected_objs = [pandas.Period("2018-03"), pandas.Categorical([1, 2, 42])]
+    keys_test = [["_typ", "day", "dayofyear", "hour"],
+                 ["_typ", "nbytes", "ndim"]]
+    return zip(test_objs, expected_objs, keys_test)
+
+
 # =============================================================================
 # Tests
 # ============================================================================
@@ -179,7 +202,7 @@ def test_edit_mutable_and_immutable_types(monkeypatch):
     """
     Test that mutable objs/vals are editable in VarExp; immutable ones aren't.
 
-    Regression test for #5991 .
+    Regression test for issue #5991 .
     """
     MockQLineEdit = Mock()
     attr_to_patch_qlineedit = ('spyder.widgets.variableexplorer.' +
@@ -278,7 +301,7 @@ def test_view_module_in_coledit():
     """
     Test that modules don't produce an error when opening in Variable Explorer.
 
-    Also check that they are set as readonly. Regression test for #6080 .
+    Also check that they are set as readonly. Regression test for issue #6080 .
     """
     editor = CollectionsEditor()
     editor.setup(os, "module_test", readonly=False)
@@ -289,7 +312,7 @@ def test_notimplementederror_multiindex():
     """
     Test that the NotImplementedError when scrolling a MultiIndex is handled.
 
-    Regression test for #6284.
+    Regression test for issue #6284 .
     """
     time_deltas = [pandas.Timedelta(minutes=minute)
                    for minute in range(5, 35, 5)]
@@ -309,7 +332,7 @@ def test_editor_parent_set(monkeypatch):
     """
     Test that editors have their parent set so they close with Spyder.
 
-    Regression test for #5696 .
+    Regression test for issue #5696 .
     """
     # Mocking and setup
     test_parent = QWidget()
@@ -353,5 +376,99 @@ def test_editor_parent_set(monkeypatch):
         assert mock_class.call_args[1]["parent"] is test_parent
 
 
+def test_xml_dom_element_view():
+    """
+    Test that XML DOM ``Element``s are able to be viewied in CollectionsEditor.
+
+    Regression test for issue #5642 .
+    """
+    xml_path = path.join(LOCATION, 'dom_element_test.xml')
+    with open(xml_path) as xml_file:
+        xml_data = xml_file.read()
+
+    xml_content = parseString(xml_data)
+    xml_element = xml_content.getElementsByTagName("note")[0]
+
+    col_editor = CollectionsEditor(None)
+    col_editor.setup(xml_element)
+    col_editor.show()
+    assert col_editor.get_value()
+    col_editor.accept()
+
+
+def test_pandas_dateoffset_view():
+    """
+    Test that pandas ``DateOffset`` objs can be viewied in CollectionsEditor.
+
+    Regression test for issue #6729 .
+    """
+    test_dateoffset = pandas.DateOffset()
+    col_editor = CollectionsEditor(None)
+    col_editor.setup(test_dateoffset)
+    col_editor.show()
+    assert col_editor.get_value()
+    col_editor.accept()
+
+
+def test_set_nonsettable_objects(nonsettable_objects_data):
+    """
+    Test that errors trying to set attributes in ColEdit are handled properly.
+
+    Unit regression test for issues #6727 and #6728 .
+    """
+    for test_obj, expected_obj, keys in nonsettable_objects_data:
+        col_model = CollectionsModel(None, test_obj)
+        indicies = [col_model.get_index_from_key(key) for key in keys]
+        for idx in indicies:
+            assert not col_model.set_value(idx, "2")
+            # Due to numpy's deliberate breakage of __eq__ comparison
+            assert all([key == "_typ" or
+                        (getattr(col_model.get_data().__obj__, key)
+                         == getattr(expected_obj, key)) for key in keys])
+
+
+@flaky(max_runs=3)
+@pytest.mark.no_xvfb
+def test_edit_nonsettable_objects(qtbot, nonsettable_objects_data):
+    """
+    Test that errors trying to edit attributes in ColEdit are handled properly.
+
+    Integration regression test for issues #6727 and #6728 .
+    """
+    for test_obj, expected_obj, keys in nonsettable_objects_data:
+        col_editor = CollectionsEditor(None)
+        col_editor.setup(test_obj)
+        col_editor.show()
+        qtbot.waitForWindowShown(col_editor)
+        view = col_editor.widget.editor
+        indicies = [view.model.get_index_from_key(key) for key in keys]
+
+        for _ in range(3):
+            qtbot.keyClick(view, Qt.Key_Right)
+        last_row = -1
+        rows_to_test = [index.row() for index in indicies]
+        for row in rows_to_test:
+            for _ in range(row - last_row - 1):
+                qtbot.keyClick(view, Qt.Key_Down)
+            qtbot.keyClick(view, Qt.Key_Space)
+            qtbot.keyClick(view.focusWidget(), Qt.Key_Backspace)
+            qtbot.keyClicks(view.focusWidget(), "2")
+            qtbot.keyClick(view.focusWidget(), Qt.Key_Down)
+            last_row = row
+
+        qtbot.wait(100)
+        # Due to numpy's deliberate breakage of __eq__ comparison
+        assert all([key == "_typ" or (getattr(col_editor.get_value(), key)
+                    == getattr(expected_obj, key)) for key in keys])
+
+        col_editor.accept()
+        qtbot.wait(200)
+        # Same reason as above
+        assert all([key == "_typ" or (getattr(col_editor.get_value(), key)
+                    == getattr(expected_obj, key)) for key in keys])
+        assert all([getattr(test_obj, key)
+                    == getattr(expected_obj, key) for key in keys])
+
+
 if __name__ == "__main__":
     pytest.main()
diff --git a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
--- a/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
+++ b/spyder/widgets/variableexplorer/tests/test_dataframeeditor.py
@@ -476,7 +476,7 @@ def test_dataframeeditor_edit_bool(qtbot, monkeypatch):
         qtbot.waitForWindowShown(dialog)
         view = dialog.dataTable
 
-        qtbot.keyPress(view, Qt.Key_Right)
+        qtbot.keyClick(view, Qt.Key_Right)
         for test_str in test_strs:
             qtbot.keyClick(view, Qt.Key_Space)
             qtbot.keyClick(view.focusWidget(), Qt.Key_Backspace)