internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: scikit-hep/awkward

Issue:
`ak.zip` converts `RegularArray`s to `ListOffsetArray64`s
### Version of Awkward Array

1.4.0

### Description and code to reproduce

Here is a reproducer:
```python3
import awkward as ak
import numpy as np

parameters = ak.from_numpy(np.arange(8 * 8).reshape(-1, 8), regulararray=True)
array = ak.zip((parameters, parameters))
assert array.layout.purelist_isregular
```

This is caused by the change to `ak.zip` in 585ca1a0d0c6f8d8473c3f177f7ccf8546cea9a6


Discussion:

Resolving patch:
diff --git a/src/awkward/operations/structure.py b/src/awkward/operations/structure.py
--- a/src/awkward/operations/structure.py
+++ b/src/awkward/operations/structure.py
@@ -477,6 +477,7 @@ def zip(
     with_name=None,
     highlevel=True,
     behavior=None,
+    right_broadcast=False,
 ):
     """
     Args:
@@ -496,6 +497,8 @@ def zip(
             a low-level #ak.layout.Content subclass.
         behavior (None or dict): Custom #ak.behavior for the output array, if
             high-level.
+        right_broadcast (bool): If True, follow rules for implicit
+            right-broadcasting, as described in #ak.broadcast_arrays.
 
     Combines `arrays` into a single structure as the fields of a collection
     of records or the slots of a collection of tuples. If the `arrays` have
@@ -639,9 +642,8 @@ def getfunction(inputs, depth):
         layouts,
         getfunction,
         behavior,
-        right_broadcast=False,
+        right_broadcast=right_broadcast,
         pass_depth=True,
-        regular_to_jagged=True,
     )
     assert isinstance(out, tuple) and len(out) == 1
     out = out[0]


Test patch:
diff --git a/tests/test_1012-zip-regular-arrays.py b/tests/test_1012-zip-regular-arrays.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1012-zip-regular-arrays.py
@@ -0,0 +1,19 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+from __future__ import absolute_import
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+def test():
+    parameters = ak.from_numpy(np.arange(3 * 3).reshape(-1, 3), regulararray=True)
+    array = ak.zip((parameters, parameters))
+    assert array.layout.purelist_isregular
+
+    assert ak.to_list(array) == [
+        [(0, 0), (1, 1), (2, 2)],
+        [(3, 3), (4, 4), (5, 5)],
+        [(6, 6), (7, 7), (8, 8)],
+    ]
Repository: scikit-hep/awkward

Issue:
Left & right broadcasting scenarios
### Which documentation?

Python docstrings

### What needs to be documented?

The scenarios in which left and right broadcasting occurs, namely that the *entire* layout must be regular for right-broadcasting, could perhaps be more explicitly stated.


Discussion:

Resolving patch:
diff --git a/src/awkward/operations/structure.py b/src/awkward/operations/structure.py
--- a/src/awkward/operations/structure.py
+++ b/src/awkward/operations/structure.py
@@ -1335,9 +1335,9 @@ def broadcast_arrays(*arrays, **kwargs):
     Awkward Array's broadcasting manages to have it both ways by applying the
     following rules:
 
-       * If a dimension is regular (i.e. #ak.types.RegularType), like NumPy,
+       * If all dimensions are regular (i.e. #ak.types.RegularType), like NumPy,
          implicit broadcasting aligns to the right, like NumPy.
-       * If a dimension is variable (i.e. #ak.types.ListType), which can
+       * If any dimension is variable (i.e. #ak.types.ListType), which can
          never be true of NumPy, implicit broadcasting aligns to the left.
        * Explicit broadcasting with a length-1 regular dimension always
          broadcasts, like NumPy.


Test patch:
Repository: scikit-hep/awkward

Issue:
`ak.packed` should pack `Record`s
### Description of new feature

Currently `ak.packed` only packs the array of a `Record`. This means that we keep more items than necessary to reconstruct the record.


Discussion:

Resolving patch:
diff --git a/src/awkward/operations/structure.py b/src/awkward/operations/structure.py
--- a/src/awkward/operations/structure.py
+++ b/src/awkward/operations/structure.py
@@ -2323,7 +2323,7 @@ def _pack_layout(layout):
         return layout
 
     elif isinstance(layout, ak.layout.Record):
-        return layout
+        return ak.layout.Record(layout.array[layout.at : layout.at + 1], 0)
 
     # Finally, fall through to failure
     else:
@@ -2354,6 +2354,7 @@ def packed(array, highlevel=True, behavior=None):
     - #ak.layout.BitMaskedArray becomes an #ak.layout.IndexedOptionArray if it contains records, stays a #ak.layout.BitMaskedArray otherwise
     - #ak.layout.UnionArray gets projected contents
     - #ak.layout.VirtualArray gets materialized
+    - #ak.layout.Record becomes a record over a single-item RecordArray
 
     Example:
 


Test patch:
diff --git a/tests/test_0912-packed.py b/tests/test_0912-packed.py
--- a/tests/test_0912-packed.py
+++ b/tests/test_0912-packed.py
@@ -181,7 +181,8 @@ def test_record():
     a = ak.layout.NumpyArray(np.arange(10))
     b = ak.layout.NumpyArray(np.arange(10) * 2 + 4)
     layout = ak.layout.RecordArray([a, b], None, 5)
-    first = layout[0]
-    packed = ak.packed(first, highlevel=False)
-    assert ak.to_list(packed) == ak.to_list(first)
-    assert len(packed.array) == len(layout)
+    record = layout[4]
+    packed = ak.packed(record, highlevel=False)
+    assert ak.to_list(packed) == ak.to_list(record)
+    assert len(packed.array) == 1
+    assert ak.to_list(packed.array) == [(4, 12)]
Repository: scikit-hep/awkward

Issue:
Awkward array to numpy structured array fills in None with garbage
### Version of Awkward Array

1.4.0

### Description and code to reproduce

**Code:**
```python3
import awkward as ak
tab = ak.Array({"A": [1,2,3], "B": [4, None, 5]})
print(tab.to_numpy())
```
**Output:**
```
[(1,               4) (2, 0) (3,               5)]
```
**Expected Output:**
Something like

```
[(1,               4) (2, --) (3,               5)]
```
Specifically, I expected to get a masked array, or failing that, an error.

If I construct the array using pyarrow instead:
```python3
import pyarrow as pa
import awkward as ak

tab = pa.table(
    {
        "A": pa.array([1, 2, 3], type=pa.int32()),
        "B": pa.array([4, None, 6], type=pa.int32()),
    }
)
aktab = ak.from_arrow(tab)
print(aktab.to_numpy())
```
I'm able to get what looks like uninitalized memory (changes each run). Presumably this is an artifact of how pyarrow constructs nullable arrays, and awkward re-using the buffer.
```
[(1,     4) (2, 32607) (3,     6)]
```


Discussion:

Resolving patch:
diff --git a/src/awkward/operations/convert.py b/src/awkward/operations/convert.py
--- a/src/awkward/operations/convert.py
+++ b/src/awkward/operations/convert.py
@@ -347,8 +347,21 @@ def to_numpy(array, allow_missing=True):
             len(contents[0]),
             dtype=[(str(n), x.dtype) for n, x in zip(array.keys(), contents)],
         )
+
+        mask = None
         for n, x in zip(array.keys(), contents):
+            if isinstance(x, numpy.ma.MaskedArray):
+                if mask is None:
+                    mask = numpy.ma.zeros(
+                        len(array), [(n, np.bool_) for n in array.keys()]
+                    )
+                if x.mask is not None:
+                    mask[n] |= x.mask
             out[n] = x
+
+        if mask is not None:
+            out = numpy.ma.MaskedArray(out, mask)
+
         return out
 
     elif isinstance(array, ak.layout.NumpyArray):


Test patch:
diff --git a/tests/test_1066-to_numpy-masked-structured-array.py b/tests/test_1066-to_numpy-masked-structured-array.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1066-to_numpy-masked-structured-array.py
@@ -0,0 +1,20 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+from __future__ import absolute_import
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+@pytest.mark.skipif(
+    ak._util.py27,
+    reason="np.ma.zeros(..., structured dtype) fails for NumPy " + str(np.__version__),
+)
+def test():
+    a = ak.to_numpy(ak.Array({"A": [1, 2, 3], "B": [4, None, 5]}))
+    assert a["A"].data.tolist() == [1, 2, 3]
+    assert a["A"].mask.tolist() == [False, False, False]
+    assert a["B"].data[0] == 4
+    assert a["B"].data[2] == 5
+    assert a["B"].mask.tolist() == [False, True, False]
Repository: scikit-hep/awkward

Issue:
`with_name` fails for `ak._v2.Record`
### Version of Awkward Array

278dccee

### Description and code to reproduce

```python
import awkward._v2 as ak

ak.Record({'x': 10.0}, with_name="X")
```


Discussion:

Resolving patch:
diff --git a/src/awkward/_v2/record.py b/src/awkward/_v2/record.py
--- a/src/awkward/_v2/record.py
+++ b/src/awkward/_v2/record.py
@@ -193,3 +193,29 @@ def to_list(self, behavior=None):
 
     def deep_copy(self):
         return Record(self._array.deep_copy(), copy.deepcopy(self._at))
+
+    def recursively_apply(
+        self,
+        action,
+        depth_context=None,
+        lateral_context=None,
+        keep_parameters=True,
+        numpy_to_regular=True,
+        return_array=True,
+        function_name=None,
+    ):
+
+        out = self._array.recursively_apply(
+            action,
+            depth_context,
+            lateral_context,
+            keep_parameters,
+            numpy_to_regular,
+            return_array,
+            function_name,
+        )
+
+        if return_array:
+            return Record(out, self._at)
+        else:
+            return None


Test patch:
diff --git a/tests/v2/test_1400-with_name-record.py b/tests/v2/test_1400-with_name-record.py
new file mode 100644
--- /dev/null
+++ b/tests/v2/test_1400-with_name-record.py
@@ -0,0 +1,10 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+def test():
+    record = ak._v2.with_name(ak._v2.Record({"x": 10.0}), "X")
+    assert ak._v2.parameters(record) == {"__record__": "X"}
Repository: scikit-hep/awkward

Issue:
`ak.from_parquet` returns empty array when columns are specified
### Version of Awkward Array

HEAD

### Description and code to reproduce

```python
>>> taxi = ak.from_parquet(
    "https://pivarski-princeton.s3.amazonaws.com/chicago-taxi.parquet",
    columns=["trip.km"],
)
>>> taxi.type.show()
7728 * {
    
}

```


Discussion:
NB: the `docs` branch will need modifying after this fix to re-enable strict mode for the 10-minutes-... notebook. Right now, it's allowed to fail in order to keep builds working.

Resolving patch:
diff --git a/src/awkward/_v2/operations/ak_from_parquet.py b/src/awkward/_v2/operations/ak_from_parquet.py
--- a/src/awkward/_v2/operations/ak_from_parquet.py
+++ b/src/awkward/_v2/operations/ak_from_parquet.py
@@ -132,16 +132,18 @@ def metadata(
         ):
             list_indicator = "list.element"
             break
+
+    subform = ak._v2._connect.pyarrow.form_handle_arrow(
+        parquetfile_for_metadata.schema_arrow, pass_empty_field=True
+    )
     if columns is not None:
+        subform = subform.select_columns(columns)
 
-        form = ak._v2._connect.pyarrow.form_handle_arrow(
-            parquetfile_for_metadata.schema_arrow, pass_empty_field=True
-        )
-        subform = form.select_columns(columns)
+    # Handle empty field at root
+    if parquetfile_for_metadata.schema_arrow.names == [""]:
+        column_prefix = ("",)
     else:
-        subform = ak._v2._connect.pyarrow.form_handle_arrow(
-            parquetfile_for_metadata.schema_arrow, pass_empty_field=True
-        )
+        column_prefix = ()
 
     metadata = parquetfile_for_metadata.metadata
     if scan_files and not path_for_schema.endswith("/_metadata"):
@@ -196,7 +198,9 @@ def metadata(
         else:
             col_counts = None
 
-    parquet_columns = subform.columns(list_indicator=list_indicator)
+    parquet_columns = subform.columns(
+        list_indicator=list_indicator, column_prefix=column_prefix
+    )
 
     return parquet_columns, subform, actual_paths, fs, subrg, col_counts, metadata
 


Test patch:
diff --git a/tests/v2/test_1619-from-parquet-empty-field.py b/tests/v2/test_1619-from-parquet-empty-field.py
new file mode 100644
--- /dev/null
+++ b/tests/v2/test_1619-from-parquet-empty-field.py
@@ -0,0 +1,31 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import os
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+pytest.importorskip("pyarrow.parquet")
+
+
+def test_no_extension(tmp_path):
+    array = ak._v2.Array(
+        [
+            [
+                {"x": 1, "y": 1.1},
+                {"x": 2, "y": 2.2},
+                {"x": 3, "y": 3.3},
+            ],
+            [
+                {"x": 1, "y": 1.1},
+                {"x": 2, "y": 2.2},
+            ],
+        ]
+    )
+    path = os.path.join(tmp_path, "array-no-ext.parquet")
+
+    ak._v2.to_parquet(array, path, extensionarray=False)
+
+    result = ak._v2.from_parquet(path, columns=["x"])
+    assert result.fields == ["x"]
Repository: scikit-hep/awkward

Issue:
Inconsistency in ak.concatenate when dim length is zero
### Version of Awkward Array

main@8c9bf49

### Description and code to reproduce

Sorry, it's me again with another edge case...

```python
import awkward._v2 as ak
import numpy as np

a0 = ak.to_regular(ak.Array(np.empty((5, 0))), 1)
a1 = ak.to_regular(ak.Array(np.empty((8, 0))), 1)

a0.type
# ArrayType(RegularType(NumpyType('float64'), 0), 5)

a1.type
# ArrayType(RegularType(NumpyType('float64'), 0), 8)

ak.concatenate([a0, a1]).type
# ArrayType(RegularType(NumpyType('float64'), 0), 8)
```

I'd expect this to return an array of shape `(13, 0`) instead of `(8, 0)`. 



Discussion:

Resolving patch:
diff --git a/src/awkward/_v2/contents/regulararray.py b/src/awkward/_v2/contents/regulararray.py
--- a/src/awkward/_v2/contents/regulararray.py
+++ b/src/awkward/_v2/contents/regulararray.py
@@ -707,7 +707,7 @@ def mergemany(self, others):
         elif all(x.is_RegularType and x.size == self.size for x in others):
             parameters = self._parameters
             tail_contents = []
-            zeros_length = 0
+            zeros_length = self._length
             for x in others:
                 parameters = ak._v2._util.merge_parameters(
                     parameters, x._parameters, True


Test patch:
diff --git a/tests/v2/test_1644-concatenate-zeros-length.py b/tests/v2/test_1644-concatenate-zeros-length.py
new file mode 100644
--- /dev/null
+++ b/tests/v2/test_1644-concatenate-zeros-length.py
@@ -0,0 +1,12 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+def test():
+    x = ak._v2.from_numpy(np.empty((5, 0), dtype=np.float64), regulararray=True)
+    y = ak._v2.from_numpy(np.empty((8, 0), dtype=np.float64), regulararray=True)
+    z = ak._v2.concatenate([x, y], axis=0)
+    assert len(z) == 13
Repository: scikit-hep/awkward

Issue:
`ak.Array` does not copy the behaviour of a wrapped array
### Version of Awkward Array

1.10.0

### Description and code to reproduce

See title!


Discussion:

Resolving patch:
diff --git a/src/awkward/highlevel.py b/src/awkward/highlevel.py
--- a/src/awkward/highlevel.py
+++ b/src/awkward/highlevel.py
@@ -198,6 +198,7 @@ def __init__(
 
         elif isinstance(data, Array):
             layout = data._layout
+            behavior = ak._util.behavior_of(data, behavior=behavior)
 
         elif isinstance(data, np.ndarray) and data.dtype != np.dtype("O"):
             layout = ak.operations.from_numpy(data, highlevel=False)


Test patch:
diff --git a/tests/test_1709-ak-array-constructor-behavior.py b/tests/test_1709-ak-array-constructor-behavior.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1709-ak-array-constructor-behavior.py
@@ -0,0 +1,20 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import pytest  # noqa: F401
+import awkward as ak  # noqa: F401
+
+
+numpy = ak.nplike.Numpy.instance()
+
+
+class MyBehavior(ak.Array):
+    ...
+
+
+def test():
+    behavior = {"MyBehavior": MyBehavior}
+    array = ak.with_parameter([1, 2, 3], "__array__", "MyBehavior", behavior=behavior)
+    assert isinstance(array, MyBehavior)
+
+    shallow_copy = ak.Array(array)
+    assert isinstance(shallow_copy, MyBehavior)
Repository: scikit-hep/awkward

Issue:
Iterating over Records and `ak.from_iter`
Currently, `ak.Record.__iter__` returns an iterator over its _field names_, which may be what a user expects if a Record were a [Mapping/MutableMapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) like a dict, but it's not. It's probably for the best that it's not, because we don't want `keys`, `values`, `items` methods, particularly since we're trying to consolidate the nomenclature of field names as "fields", rather than "keys". And we especially don't want the non-vectorized `__eq__` and `__ne__` that Mapping has.

To throw a little more confusion in, `ak.Record.__len__` always returns 1 and not the number of fields (the length of what it iterates over). That's a weird enough choice to be considered a bug.

Since v2 is coming up and we can make little backward-incompatible changes like this, perhaps we should just ensure that v2 `ak.Record` has no `__iter__` and `__len__` methods. If they're really needed for something, we can be driven by the use-case as to whether `ak.Record` iteration should be over field names, field-value pairs, or something else. At the very least, `__len__` should return the length of the `__iter__` iterable.

Related to this is how `ak.Record` is treated by `ak.from_iter`: it naively iterates, which means it gets the field names:

```python
>>> import awkward as ak
>>> ak._v2.from_iter(ak._v2.Record({"a": 1, "b": 2}))
<Array ['a', 'b'] type='2 * string'>
>>> ak._v2.from_iter([ak._v2.Record({"a": 1, "b": 2})])
<Array [['a', 'b']] type='1 * var * string'>
```

Although iteration is a slow way to build an array and coming from a collection of `ak.Record` is suggestive that it's being used when a faster way is possible, the above is almost certainly not what the user had in mind. Changing v2 `ak.from_iter` to recognize `ak.Record` and build an equivalent thing is an improvement that would be independent of fixing the `ak.Record.__iter__` and `__len__` behavior, but both should be done.

**Edit:** also check `ArrayBuilder.append`; make sure it's consistent with `ak.from_iter` (it might be how `ak.from_iter` is implemented...)


Discussion:
I am inclined to agree that we should not push the record-as-mapping protocol here. We'd start getting into the weeds with how NumPy defines e.g. `__contains__` and how a pure-Python mapping should etc. So, :+1: from me on making `Record` less clever.

Resolving patch:
diff --git a/src/awkward/highlevel.py b/src/awkward/highlevel.py
--- a/src/awkward/highlevel.py
+++ b/src/awkward/highlevel.py
@@ -1657,8 +1657,8 @@ def is_tuple(self):
     def _ipython_key_completions_(self):
         return self._layout.array.fields
 
-    def __iter__(self):
-        yield from self._layout.array.fields
+    # Disable iteration (and prevent old-style iteration from being used)
+    __iter__ = None
 
     @property
     def type(self):


Test patch:
Repository: scikit-hep/awkward

Issue:
Pickling of `Array`s should use `Form`'s dict representation
### Version of Awkward Array

main

### Description and code to reproduce

...


Discussion:

Resolving patch:
diff --git a/src/awkward/highlevel.py b/src/awkward/highlevel.py
--- a/src/awkward/highlevel.py
+++ b/src/awkward/highlevel.py
@@ -1381,7 +1381,7 @@ def __getstate__(self):
             behavior = None
         else:
             behavior = self._behavior
-        return form, length, container, behavior
+        return form.to_dict(), length, container, behavior
 
     def __setstate__(self, state):
         if isinstance(state[1], dict):
@@ -1997,7 +1997,7 @@ def __getstate__(self):
             behavior = None
         else:
             behavior = self._behavior
-        return form, length, container, behavior, packed.at
+        return form.to_dict(), length, container, behavior, packed.at
 
     def __setstate__(self, state):
         if isinstance(state[1], dict):


Test patch:
Repository: scikit-hep/awkward

Issue:
`ak.fill_none(axis=None)` does nothing
### Version of Awkward Array

main

### Description and code to reproduce

The `apply` function for this case does not return (or recurse)


Discussion:

Resolving patch:
diff --git a/src/awkward/operations/ak_fill_none.py b/src/awkward/operations/ak_fill_none.py
--- a/src/awkward/operations/ak_fill_none.py
+++ b/src/awkward/operations/ak_fill_none.py
@@ -107,8 +107,8 @@ def maybe_fillna(layout):
 
     if axis is None:
 
-        def action(layout, depth, depth_context, **kwargs):
-            layout = maybe_fillna(layout)
+        def action(layout, continuation, **kwargs):
+            return maybe_fillna(continuation())
 
     else:
 


Test patch:
diff --git a/tests/test_1823-fill-none-axis-none.py b/tests/test_1823-fill-none-axis-none.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1823-fill-none-axis-none.py
@@ -0,0 +1,15 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import numpy as np  # noqa: F401
+import pytest  # noqa: F401
+
+import awkward as ak  # noqa: F401
+
+
+def test():
+    array = ak.Array([None, [1, 2, 3, [None, {"x": [None, 2], "y": [1, 4]}]]])
+
+    assert ak.fill_none(array, -1.0, axis=None).to_list() == [
+        -1.0,
+        [1, 2, 3, [-1.0, {"x": [-1.0, 2], "y": [1, 4]}]],
+    ]