Repository: pyvista/pyvista

Issue:
PolyData faces array is not updatable in-place and has unexpected behavior
### Describe the bug, what's wrong, and what you expected.

When accessing `PolyData.faces` (and likely other cell data), we cannot update the array in place. Further, there is some unexpected behavior where accessing `PolyData.faces` will override existing, modified views of the array.

### Steps to reproduce the bug.

```python 
>>> import pyvista as pv
>>> mesh = pv.Sphere()
>>> f = mesh.faces
>>> f
array([  3,   2,  30, ..., 840,  29,  28])
>>> a = f[1:4]
>>> a
array([ 2, 30,  0])
>>> b = f[5:8]
>>> b
array([30, 58,  0])
>>> f[1:4] = b
>>> f[5:8] = a
>>> f
array([  3,  30,  58, ..., 840,  29,  28])
>>> assert all(f[1:4] == b) and all(f[5:8] == a)
>>> mesh.faces  # access overwrites `f` in place which is unexpected and causes the check above to now fail
>>> assert all(f[1:4] == b) and all(f[5:8] == a)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-82-08205e08097f> in <cell line: 13>()
     11 assert all(f[1:4] == b) and all(f[5:8] == a)
     12 mesh.faces  # access overwrites `f` in place
---> 13 assert all(f[1:4] == b) and all(f[5:8] == a)

AssertionError: 
 ```

### System Information

```shell
--------------------------------------------------------------------------------
  Date: Thu May 26 11:45:54 2022 MDT

                OS : Darwin
            CPU(s) : 16
           Machine : x86_64
      Architecture : 64bit
               RAM : 64.0 GiB
       Environment : Jupyter
       File system : apfs
        GPU Vendor : ATI Technologies Inc.
      GPU Renderer : AMD Radeon Pro 5500M OpenGL Engine
       GPU Version : 4.1 ATI-4.8.13

  Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:12:38)
  [Clang 11.0.1 ]

           pyvista : 0.35.dev0
               vtk : 9.1.0
             numpy : 1.22.1
           imageio : 2.9.0
           appdirs : 1.4.4
            scooby : 0.5.12
        matplotlib : 3.5.2
           IPython : 7.32.0
          colorcet : 3.0.0
           cmocean : 2.0
        ipyvtklink : 0.2.2
             scipy : 1.8.0
        itkwidgets : 0.32.1
              tqdm : 4.60.0
            meshio : 5.3.4
--------------------------------------------------------------------------------
```


### Screenshots

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct


Discussion:
Considering we are purely using VTK for this interface through the `vtk_to_numpy` function, I'm inclined to think this may be an upstream bug where the array doesn't interface completely

https://github.com/pyvista/pyvista/blob/eaa71a640264765c8a9b40d485b6ccdb970c87b0/pyvista/core/pointset.py#L762 

or perhaps there is a better way to access the Polys from the VTK object?
A pure VTK demonstration of the issue:

```py
import vtk
from vtkmodules.util.numpy_support import vtk_to_numpy

# https://github.com/pyvista/pyvista/issues/2705
sphere = vtk.vtkSphereSource()
sphere.Update()
mesh = sphere.GetOutput()


f = vtk_to_numpy(mesh.GetPolys().GetData())

>>> f
array([  3,   2,  30, ..., 840,  29,  28])
>>> a = f[1:4]
>>> a
array([ 2, 30,  0])
>>> b = f[5:8]
>>> b
array([30, 58,  0])
>>> f[1:4] = b
>>> f[5:8] = a
>>> f
array([  3,  30,  58, ..., 840,  29,  28])

assert all(f[1:4] == b) and all(f[5:8] == a), 'if this fails, we have more serious problems'

# access overwrites `f` in place which is unexpected and causes the check above to now fail
vtk_to_numpy(mesh.GetPolys().GetData())
assert all(f[1:4] == b) and all(f[5:8] == a), 'well, this is not expected'
```
Well, this was a joy to track down. The short answer: Use `vtkCellArray.GetConnectivityArray()` instead of `vtkCellArray.GetData()`.

The long explanation: A while ago, `vtkCellArray` was changed so that it can store either 32 bit or 64 bit connectivity data. This enabled saving memory when VTK is compiled with 64 bit ids (`vtkIdType`) AND if the mesh has less than 2 billion connectivity entries. This meant that `vtkCellArray` can store either a 32 bit or 64 bit array for connectivity. However, `GetData()` returns a `vtkIdTypeArray` which is usually compiled to be 64 bit so `vtkCellArray` could no longer return its internal array in this method. So, the placeholder implementation now makes a copy from its internal array to return. So changing this array does not actually affect the connectivity. The issue shown here is a side effect of this implementation. `vtkCellArray.GetConnectivityArray()` actually returns the internal array and can be safely changed to affect the connectivity. pyvista and VTK Python classes have to be changed to use `vtkCellArray.GetConnectivityArray()` and `vtkCellArray.GetOffsetsArray()`. Also don't use `vtkCellArray.SetCells()`. Use `vtkCellArray.SetData()` instead.
Unfortunately for PyVista, this is not an easy fix as `GetConnectivityArray()` is not a direct stand-in for `GetData()`. 

`GetData()` array has connectivity offsets (`GetOffsetsArray()`) embedded in the array, whereas `GetConnectivityArray()` does not. This is quite nice for `vtkPolyData.GetPolys()` if and only if all of the polys are the same size (e.g., 3 for triangulated mesh), but if there are mixed polys, this is pretty tough to work with when trying to update the array in-place:

```py
import numpy as np
import pyvista as pv
from pyvista import _vtk

vertices = np.array([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0], [0.5, 0.5, -1]])

# mesh faces
faces = np.hstack(
    [
        [4, 0, 1, 2, 3],  # square
        [3, 0, 1, 4],  # triangle
        [3, 1, 2, 4],  # triangle
    ]
)

surf = pv.PolyData(vertices, faces)
```

```py
>>> _vtk.vtk_to_numpy(surf.GetPolys().GetData())
array([4, 0, 1, 2, 3, 3, 0, 1, 4, 3, 1, 2, 4])
```

```py
>>> _vtk.vtk_to_numpy(surf.GetPolys().GetConnectivityArray())
array([0, 1, 2, 3, 0, 1, 4, 1, 2, 4])
```

```py
>>> _vtk.vtk_to_numpy(surf.GetPolys().GetOffsetsArray())
array([ 0,  4,  7, 10])
```

In PyVista, we designed the `.cells`, `.faces`, `.strips`, `.lines`, etc. APIs around the assumption of having the connectivity offsets embedded in the array. This means we'd have to redesign these accessors entirely (breaking a lot of downstream code) just to enable this in-place operation. Further, changing the API to require users to deal with both connectivities and offsets as separate arrays will likely be too much of a burden.

------

> the placeholder implementation now makes a copy from its internal array to return. So changing this array does not actually affect the connectivity.

@berkgeveci, I'm curious if you think this copy could/will be fixed upstream in VTK so that `GetData()` could work with in-place operations?
Ah I see. Unfortunately, what you want is not possible. The underlying structure for `vtkCellArray` changed and it does not store the number of vertices per cell anymore. This is the main reason why `GetData()` creates a new array (something that I forgot). For backwards compatibility. But it has the side effect of this array being "detached" from the actual topology of the mesh. There are good reasons for this change:

- Most other codes and OpenGL prefer the new structure so there is more likelihood that VTK will be binary compatible with them.
- When we get rid of the vertex count and always have the offsets, we can have random access at no additional memory cost. In the past, for polydata, the offsets were built when needed to save memory.

Note that there is no loss of information here. You can simply recover the number of vertices of a cell by doing `offset[cellId+1]-offset[cellId]`. To make this work, the offsets array actually is of size `ncells+1` so that you can get the number of vertices for the last cell.

I understand that this will be a backwards incompatible change for PyVista but there is no other way. You can still support

```
# mesh faces
faces = np.hstack(
    [
        [4, 0, 1, 2, 3],  # square
        [3, 0, 1, 4],  # triangle
        [3, 1, 2, 4],  # triangle
    ]
)

surf = pv.PolyData(vertices, faces)
```

but it will have to create the connectivity and offsets array under the covers (or let VTK do it). But if you want to avoid the deep copy, you have to support `pv.PolyData(vertices, offsets, connectivity)`, which has to create a `vtkCellArray` and call `SetData()` on it.
I suggest avoiding the use of `GetData()` all together.

Also note that there is a bug in how `SetData()` works currently and when called from Python, it does not cause `vtkCellArray()` to hold a reference to the underlying numpy buffer. Unless you are keeping a reference to that object from the Python side, the VTK object will hold a reference to a junk pointer once the numpy array goes out of scope. I am working on a fix for this issue.


Patch:
diff --git a/pyvista/core/pointset.py b/pyvista/core/pointset.py
--- a/pyvista/core/pointset.py
+++ b/pyvista/core/pointset.py
@@ -13,7 +13,7 @@
 import pyvista
 from pyvista import _vtk
 from pyvista.utilities import abstract_class
-from pyvista.utilities.cells import CellArray, create_mixed_cells, get_mixed_cells
+from pyvista.utilities.cells import CellArray, create_mixed_cells, get_mixed_cells, numpy_to_idarr
 
 from .._typing import BoundsLike
 from ..utilities.fileio import get_ext
@@ -751,12 +751,32 @@ def lines(self, lines):
 
     @property
     def faces(self) -> np.ndarray:
-        """Return a pointer to the faces as a numpy array.
+        """Return the connectivity array of the faces of this PolyData.
+
+        The faces array is organized as::
+
+           [n0, p0_0, p0_1, ..., p0_n, n1, p1_0, p1_1, ..., p1_n, ...]
+
+        where ``n0`` is the number of points in face 0, and ``pX_Y`` is the
+        Y'th point in face X.
+
+        For example, a triangle and a quadrilateral might be represented as::
+
+           [3, 0, 1, 2, 4, 0, 1, 3, 4]
+
+        Where the two individual faces would be ``[3, 0, 1, 2]`` and ``[4, 0, 1, 3, 4]``.
 
         Returns
         -------
         numpy.ndarray
-            Array of face indices.
+            Array of face connectivity.
+
+        Notes
+        -----
+        The array returned cannot be modified in place and will raise a
+        ``ValueError`` if attempted.
+
+        You can, however, set the faces directly. See the example.
 
         Examples
         --------
@@ -773,8 +793,21 @@ def faces(self) -> np.ndarray:
                [4, 1, 2, 5, 4],
                [4, 3, 4, 7, 6],
                [4, 4, 5, 8, 7]])
+
+        Set the faces directly. The following example creates a simple plane
+        with a single square faces and modifies it to have two triangles
+        instead.
+
+        >>> mesh = pv.Plane(i_resolution=1, j_resolution=1)
+        >>> mesh.faces = [3, 0, 1, 2, 3, 3, 2, 1]
+        >>> mesh.faces
+        array([3, 0, 1, 2, 3, 3, 2, 1])
+
         """
-        return _vtk.vtk_to_numpy(self.GetPolys().GetData())
+        array = _vtk.vtk_to_numpy(self.GetPolys().GetData())
+        # Flag this array as read only to ensure users do not attempt to write to it.
+        array.flags['WRITEABLE'] = False
+        return array
 
     @faces.setter
     def faces(self, faces):
@@ -1576,11 +1609,33 @@ def _check_for_consistency(self):
 
     @property
     def cells(self) -> np.ndarray:
-        """Return a pointer to the cells as a numpy object.
+        """Return the cell data as a numpy object.
+
+        This is the old style VTK data layout::
+
+           [n0, p0_0, p0_1, ..., p0_n, n1, p1_0, p1_1, ..., p1_n, ...]
+
+        where ``n0`` is the number of points in cell 0, and ``pX_Y`` is the
+        Y'th point in cell X.
+
+        For example, a triangle and a line might be represented as::
+
+           [3, 0, 1, 2, 2, 0, 1]
+
+        Where the two individual cells would be ``[3, 0, 1, 2]`` and ``[2, 0, 1]``.
 
         See Also
         --------
         pyvista.DataSet.get_cell
+        pyvista.UnstructuredGrid.cell_connectivity
+        pyvista.UnstructuredGrid.offset
+
+        Notes
+        -----
+        The array returned cannot be modified in place and will raise a
+        ``ValueError`` if attempted.
+
+        You can, however, set the cells directly. See the example.
 
         Examples
         --------
@@ -1590,13 +1645,25 @@ def cells(self) -> np.ndarray:
 
         >>> import pyvista
         >>> from pyvista import examples
-        >>> hex_beam = pyvista.read(examples.hexbeamfile)
-        >>> hex_beam.cells[:18]  # doctest:+SKIP
-        array([ 8,  0,  2,  8,  7, 27, 36, 90, 81,  8,  2,  1,  4,
-                8, 36, 18, 54, 90])
+        >>> grid = examples.load_hexbeam()
+        >>> grid.cells[:18]
+        array([ 8,  0,  2,  8,  7, 27, 36, 90, 81,  8,  2,  1,  4,  8, 36, 18, 54,
+               90])
+
+        While you cannot change the array inplace, you can overwrite it. For example:
+
+        >>> grid.cells = [8, 0, 1, 2, 3, 4, 5, 6, 7]
 
         """
-        return _vtk.vtk_to_numpy(self.GetCells().GetData())
+        # Flag this array as read only to ensure users do not attempt to write to it.
+        array = _vtk.vtk_to_numpy(self.GetCells().GetData())
+        array.flags['WRITEABLE'] = False
+        return array
+
+    @cells.setter
+    def cells(self, cells):
+        vtk_idarr = numpy_to_idarr(cells, deep=False, return_ind=False)
+        self.GetCells().ImportLegacyFormat(vtk_idarr)
 
     @property
     def cells_dict(self) -> dict:
diff --git a/pyvista/demos/logo.py b/pyvista/demos/logo.py
--- a/pyvista/demos/logo.py
+++ b/pyvista/demos/logo.py
@@ -148,7 +148,7 @@ def plot_logo(
     v_grid_atom = atomize(v_grid)
     v_grid_atom['scalars'] = v_grid_atom.points[:, 0]
     v_grid_atom_surf = v_grid_atom.extract_surface()
-    faces = v_grid_atom_surf.faces.reshape(-1, 5)
+    faces = v_grid_atom_surf.faces.reshape(-1, 5).copy()
     faces[:, 1:] = faces[:, 1:][:, ::-1]
     v_grid_atom_surf.faces = faces
     plotter.add_mesh(


Test patch:
diff --git a/tests/test_grid.py b/tests/test_grid.py
--- a/tests/test_grid.py
+++ b/tests/test_grid.py
@@ -146,6 +146,14 @@ def test_init_from_arrays(specify_offset):
     assert np.allclose(cells, grid.cells)
     assert np.allclose(grid.cell_connectivity, np.arange(16))
 
+    # grid.cells is not mutable
+    assert not grid.cells.flags['WRITEABLE']
+
+    # but attribute can be set
+    new_cells = [8, 0, 1, 2, 3, 4, 5, 6, 7]
+    grid.cells = [8, 0, 1, 2, 3, 4, 5, 6, 7]
+    assert np.allclose(grid.cells, new_cells)
+
 
 @pytest.mark.parametrize('multiple_cell_types', [False, True])
 @pytest.mark.parametrize('flat_cells', [False, True])
diff --git a/tests/test_polydata.py b/tests/test_polydata.py
--- a/tests/test_polydata.py
+++ b/tests/test_polydata.py
@@ -82,6 +82,15 @@ def test_init_from_arrays():
     with pytest.warns(Warning):
         mesh = pyvista.PolyData(vertices.astype(np.int32), faces)
 
+    # array must be immutable
+    with pytest.raises(ValueError):
+        mesh.faces[0] += 1
+
+    # attribute is mutable
+    faces = [4, 0, 1, 2, 3]
+    mesh.faces = faces
+    assert np.allclose(faces, mesh.faces)
+
 
 def test_init_from_arrays_with_vert():
     vertices = np.array([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0], [0.5, 0.5, -1], [0, 1.5, 1.5]])