Repository: scikit-hep/awkward

Issue:
Raise warning on setattr for assigning arrays to records
### Description of new feature

Take the following rather surprising but currently correct behavior:
```
>>> import awkward as ak
>>> a = ak.zip({"x": ak.Array([[1], [2, 3], [4, 5, 6]]), "y": ak.Array([[10], [20, 30], [40, 50, 60]])})
>>> a.x = a.x *3
>>> a.x
<Array [[3], [6, 9], [12, 15, 18]] type='3 * var * int64'>
>>> a["x"]
<Array [[1], [2, 3], [4, 5, 6]] type='3 * var * int64'>
```
It is surprising to users that `a.x` and `a["x"]` are different, but expected because the array set by `setattr` ends up masking the underlying array already in the layout of `a`. The recommended way to assign arrays to existing records is to use `setitem` as shown:
```
>>> a["y"] = a.y *3
>>> a.y
<Array [[30], [60, 90], [120, 150, 180]] type='3 * var * int64'>
>>> a["y"]
<Array [[30], [60, 90], [120, 150, 180]] type='3 * var * int64'>
```
then the `getattr` and `getitem` methods behave the same.

I don't see a lot of legitimate use cases for using `setattr` with array arguments. In these cases I think we should raise a warning or error.


Discussion:
I think this is reasonable. In general, I think it is poor practice to store any non-array data on an `ak.Array`, so this would be a nice default.
(We've been talking about this on Slack.) I agree: just telling people "use this, not that" is insufficient because not everybody's listening on the same channel. Since one can do `a.x` to get an array, it's very natural to expect `a.x = array-like` to set an array. This must be prevented because it gets very confusing downstream, as Nick pointed out.

We also need to be able to stash data on an ak.Array Python object (at the very least in informal/debugging situations), so we can't exclude general `__setattr__`. It's an expert/developer-level feature. What needs to be prevented is a `a.x = array-like` that looks like an `a["x"] = array-like`.

How about ~~these two rules~~ this one rule:

   * If the name of the attribute assigned with `__setattr__` starts with an underscore, it should be allowed.
   * ~~If the thing being assigned is not array-like, it should be allowed. "Array-like" would be defined as: ak.Array or ak.Record, number/bool (corresponding to our [list of primitive types](https://github.com/scikit-hep/awkward/blob/49a3ff13ef90b8778a80573211d58c544729eaa5/src/awkward/_v2/types/numpytype.py#L67-L81)), string or bytestring, or an object with a `dtype` and `shape`.~~

On second thought, making a choice based on the type of the object is too complicated, and would therefore be hard to describe. So just this: if you attach something to an ak.Array or ak.Record, it has to be a recognized mutable property or start with an underscore.

This doesn't catch users whose field values start with underscores... but maybe those should also be refused in `__getattr__`, allowing `a["_x"]` but not `a._x`. (Is that already true?) There's a restriction on what field names can be used in `__getattr__`; the simplest thing would be for it to be exactly the same as the restriction on what field names can be used in `__setattr__`.

And it should be an exception, rather than a warning. Warnings are useful if the only context we want to allow this in is informal debugging, but maybe we'll need to assign caches or something in production.

Subclasses of ak.Array and ak.Record should be able to override this `__setattr__`. (And I don't see anything preventing them, following normal Python semantics.)
I think we don't need to be too restrictive here. As long as the defaults are sane, then if a user decides to push the limit, they'll ultimately hit a bug and report.

I'm happy to promote this to an exception. I agree that this should not look at the object type. 
It might be worth explaining here why `setattr` can't just call internally `setitem` on array arguments. The usual corner case is `array.attr.subattr = ...` where in the current scheme `array.attr` is a shallow copy and hence the `setitem` doesn't actually propagate to `array`. Perhaps with enough backreferences this could be arranged, but it would not be trivial I imagine. Jim may have more reason to disallow this.
When I last thought about it, I thought that it would be impossible, given how Python is eager. But... maybe if those arrays returned via `.attr` carry some backward-pointing metadata, it might be possible. But then we'd have a problem of sliced arrays owning the arrays they were sliced from, which would have a surprising effect on memory footprint: if you slice out one field of an array and `del` the rest, the memory wouldn't go away!

I think it's best to keep it simple, though all of the pathways in which someone could do something that they very clearly didn't intend should be blocked.

Resolving patch:
diff --git a/src/awkward/_v2/highlevel.py b/src/awkward/_v2/highlevel.py
--- a/src/awkward/_v2/highlevel.py
+++ b/src/awkward/_v2/highlevel.py
@@ -1044,7 +1044,7 @@ def __setitem__(self, where, what):
     def __delitem__(self, where):
         """
         Args:
-            where (str): Field name to add to records in the array.
+            where (str): Field name to remove from records in the array.
 
         Removes a field from records in the array.
         """
@@ -1062,6 +1062,9 @@ def __delitem__(self, where):
 
     def __getattr__(self, where):
         """
+        Args:
+            where (str): Attribute name to lookup
+
         Whenever possible, fields can be accessed as attributes.
 
         For example, the fields of an `array` like
@@ -1089,15 +1092,7 @@ def __getattr__(self, where):
              keyword.
 
         Note that while fields can be accessed as attributes, they cannot be
-        *assigned* as attributes: the following doesn't work.
-
-            array.z = new_field
-
-        Always use
-
-            array["z"] = new_field
-
-        to add a field.
+        *assigned* as attributes. See #ak.Array.__setitem__ for more.
         """
         if where in dir(type(self)):
             return super().__getattribute__(where)
@@ -1115,6 +1110,45 @@ def __getattr__(self, where):
             else:
                 raise ak._v2._util.error(AttributeError(f"no field named {where!r}"))
 
+    def __setattr__(self, name, value):
+        """
+        Args:
+            where (str): Attribute name to set
+
+        Set an attribute on the array.
+
+        Only existing public attributes e.g. #ak.Array.layout, or private
+        attributes (with leading underscores), can be set.
+
+        Fields are not assignable to as attributes, i.e. the following doesn't work:
+
+            array.z = new_field
+
+        Instead, always use #ak.Array.__setitem__:
+
+            array["z"] = new_field
+
+        or #ak.with_field:
+
+            array = ak.with_field(array, new_field, "z")
+
+        to add or modify a field.
+        """
+        if name in dir(type(self)) or name.startswith("_"):
+            super().__setattr__(name, value)
+        elif name in self._layout.fields:
+            raise ak._v2._util.error(
+                AttributeError(
+                    "fields cannot be set as attributes. use #__setitem__ or #ak.with_field"
+                )
+            )
+        else:
+            raise ak._v2._util.error(
+                AttributeError(
+                    "only private attributes (started with an underscore) can be set on arrays"
+                )
+            )
+
     def __dir__(self):
         """
         Lists all methods, properties, and field names (see #__getattr__)


Test patch:
diff --git a/tests/v2/test_1511-set-attribute.py b/tests/v2/test_1511-set-attribute.py
new file mode 100644
--- /dev/null
+++ b/tests/v2/test_1511-set-attribute.py
@@ -0,0 +1,21 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import pytest  # noqa: F401
+import awkward as ak  # noqa: F401
+import numpy as np
+
+
+def test():
+    record = ak._v2.contents.RecordArray(
+        [ak._v2.contents.NumpyArray(np.arange(10))], ["x"]
+    )
+    array = ak._v2.Array(record)
+
+    with pytest.raises(AttributeError):
+        array.x = 10
+
+    with pytest.raises(AttributeError):
+        array.not_an_existing_attribute = 10
+
+    array._not_an_existing_attribute = 10
+    assert array._not_an_existing_attribute == 10