Repository: scikit-hep/awkward

Issue:
Adding fields to a record array drops its parameters
A recent change has caused the following code:
```python
import awkward as ak

a = ak.zip({"x": [1, 2, 3], "y": [2, 3, 4]}, with_name="asdf")
print(ak.type(a))
a["z"] = ak.Array([4, 5, 6])
print(ak.type(a))
```

to no longer output:
```
3 * asdf["x": int64, "y": int64]
3 * asdf["x": int64, "y": int64, "z": int64]
```
but rather
```
3 * asdf["x": int64, "y": int64]
3 * {"x": int64, "y": int64, "z": int64}
```

The change was introduced in https://github.com/scikit-hep/awkward-1.0/pull/606/files#diff-63b9cf483f8a04b770807d6cabd4098708d35ed6d124e8d1c969fcca657d9b32R852-R866
I think there's a case to be made for dropping parameters in some cases, but here is one of the rare instances where awkward appears to be letting us mutate the array (though under the hood it swaps the layout to a new one) and that distinction is probably worth keeping parameters.


Discussion:
Better link: https://github.com/scikit-hep/awkward-1.0/pull/606/files#diff-63b9cf483f8a04b770807d6cabd4098708d35ed6d124e8d1c969fcca657d9b32L845-R865
Well, `getitem_fields` _should_ drop parameters:

```python
array_of_records[["just", "these", "fields"]]
```

turns a record array into a record array of fewer fields, so it should no longer allow the same methods (or have the same identity as an `asdf` type of object), which might require a full set of fields.

But you've demonstrated that `ak.with_field` is losing parameters and it should not. Having _more_ fields (or the same number of fields, with one of its values replaced) is not a problem for having the same set of methods (or the same type-identity). The `ak.with_field` function is calling a `Content::setitem_field` method:

https://github.com/scikit-hep/awkward-1.0/blob/ea2d88c8df7fe2cfd3a7f8b157846e7c18d0aa1c/src/awkward/operations/structure.py#L661-L669

But... the `RecordArray::setitem_field` method _does_ return a RecordArray with the same `parameters_`:

https://github.com/scikit-hep/awkward-1.0/blob/ea2d88c8df7fe2cfd3a7f8b157846e7c18d0aa1c/src/libawkward/array/RecordArray.cpp#L429-L472

That's weird. I'll have to go into deep debugging mode to figure out where it's being lost. (Not here: https://github.com/scikit-hep/awkward-1.0/blob/ea2d88c8df7fe2cfd3a7f8b157846e7c18d0aa1c/src/awkward/highlevel.py#L1058-L1069 .) There shouldn't be a point where an array is sliced with its full set of fields, which would do nothing except drop parameters, but that's the sort of thing I'd be looking for.
Ok, I `git bisect`ed to find that PR was the culprit, but only guessed that line change to be the relevant piece of the PR.

Resolving patch:
diff --git a/src/awkward/operations/structure.py b/src/awkward/operations/structure.py
--- a/src/awkward/operations/structure.py
+++ b/src/awkward/operations/structure.py
@@ -658,24 +658,39 @@ def with_field(base, what, where=None, right_broadcast=False, highlevel=True):
             what, allow_record=True, allow_other=True
         )
 
-        def getfunction(inputs):
-            nplike = ak.nplike.of(*inputs)
-            base, what = inputs
-            if isinstance(base, ak.layout.RecordArray):
-                if not isinstance(what, ak.layout.Content):
-                    what = ak.layout.NumpyArray(nplike.repeat(what, len(base)))
-                return lambda: (base.setitem_field(where, what),)
-            else:
-                return None
-
         keys = base.keys()
         if where in base.keys():
             keys.remove(where)
+
         if len(keys) == 0:
             # the only key was removed, so just create new Record
-            out = (ak.layout.RecordArray([what], [where]),)
+            out = (ak.layout.RecordArray([what], [where], parameters=base.parameters),)
+
         else:
-            base = base[keys]
+
+            def getfunction(inputs):
+                nplike = ak.nplike.of(*inputs)
+                base, what = inputs
+                if isinstance(base, ak.layout.RecordArray):
+                    if not isinstance(what, ak.layout.Content):
+                        what = ak.layout.NumpyArray(nplike.repeat(what, len(base)))
+                    if base.istuple and where is None:
+                        recordlookup = None
+                    elif base.istuple:
+                        recordlookup = keys + [where]
+                    elif where is None:
+                        recordlookup = keys + [str(len(keys))]
+                    else:
+                        recordlookup = keys + [where]
+                    out = ak.layout.RecordArray(
+                        [base[k] for k in keys] + [what],
+                        recordlookup,
+                        parameters=base.parameters
+                    )
+                    return lambda: (out,)
+                else:
+                    return None
+
             out = ak._util.broadcast_and_apply(
                 [base, what],
                 getfunction,
@@ -683,6 +698,7 @@ def getfunction(inputs):
                 right_broadcast=right_broadcast,
                 pass_depth=False,
             )
+
         assert isinstance(out, tuple) and len(out) == 1
 
         if highlevel:


Test patch: