Repository: scikit-hep/awkward

Issue:
Segmentation fault with nested pair-wise comparison
### Version of Awkward Array

1.9.0 and 1.10.0

### Description and code to reproduce

Used as part of coffea package: 

```python
from coffea.nanoevents import NanoEventsFactory
from coffea.nanoevents.schemas import TreeMakerSchema
import awkward as ak

events = NanoEventsFactory.from_root(
    "root://cmseos.fnal.gov//store/user/lpcsusyhad/ExoEMJAnalysis2020/Run2ProductionV20Skimmed/Summer20UL16/GJets_DR-0p4_HT-100To200_TuneCP5_13TeV-madgraphMLM-pythia8/0_RA2AnalysisTree.root",
    schemaclass=TreeMakerSchema,
    treepath="TreeMaker2/PreSelection",
    entry_start=10000,
    entry_stop=20000).events()

jets = events.Jets[events.Jets.pt > 100]
tracks = events.Tracks[events.Tracks.pt > 1.0]

_, tracks = ak.unzip(ak.cartesian([jets.eta, tracks], nested=True))
tracks['deltaR'] = jets.delta_r(tracks)

t_index = ak.argmin(tracks.deltaR, axis=-2)
j_index = ak.local_index(jets.pt, axis=-1)
_, t_index = ak.unzip(ak.cartesian([j_index, t_index], nested=True))
tracks = tracks[j_index == t_index]

tracks = tracks[tracks.deltaR < 0.4]
```

The above segmentation faults at different point for different input files. Usually around the `ak.cartesian` calls, but also when adding variables to the array. Segmentation fault was observed for both awkward 1.9.0 and awkward 1.10.0, but never occurred for awkward 1.8.0

Reduction of inner dimension in ragged array produces incorrect offsets
### Version of Awkward Array

main

### Description and code to reproduce

This *might* not be a bug, depending upon what we define the behavior of `ListArray` to be w.r.t `starts[i] == stops[i]`. The following reproducer produces a result layout with `starts[1] == stops[1] == 3`. The leaf `NumpyArray` has a length of 1, so unless we define out-of-bounds starts to be valid iff. they are equal to stops, then we're not doing the right thing here.


```python
import numpy as np
import awkward as ak

array = ak.Array(
    ak.contents.ListOffsetArray(
        ak.index.Index64(
            np.array(
                [0, 1, 1],
                dtype=np.int64,
            )
        ),
        ak.contents.ListOffsetArray(
            ak.index.Index64(
                np.array(
                    [0, 1],
                    dtype=np.int64,
                )
            ),
            ak.contents.NumpyArray(np.arange(1)),
        )
    )
)
# [
#     [
#         [0],
#     ],
#     [],
# ]

print(ak.sum(array, axis=1).layout)

```


Discussion:
Hi @yimuchen, could you provide a copy of the file to test against? I don't have auth to read from the given XRootD server.

You can create a file `events.pickle.txt` with:
```python3
import pickle
form, length, container = ak.to_buffers(events)
data = (form.tojson(), length, container)
with open("events.pickle.txt", "wb") as f:
    pickle.dump(data, f)
```
Of course, here is a 100-event file that crashes similarly. This `ak.to_buffers` call was performed using awkward 1.8.0

[events_awkward1.9.0_issue.pickle.txt](https://github.com/scikit-hep/awkward/files/9775067/events_awkward1.9.0_issue.pickle.txt)


This is definitely a bug, and it's being triggered by [one of our kernels](https://github.com/scikit-hep/awkward/blob/b315e6519eb78c8083e088e3ca065cfdb3af5d31/src/cpu-kernels/awkward_ListOffsetArray_reduce_nonlocal_outstartsstops_64.cpp#L33), where we're writing to an out-of-bounds value during the computation of 
```python3
ak.argmin(tracks.deltaR, axis=-2)
```
I think the kernel is behaving properly (although `k` is computed to be `>= outlength`) but we're just passing in either the wrong value for `outlength`, or the wrong array of `gaps`.


My approach to fixing this is currently "write a load of documentation comments on the reduction logic until I understand what is going wrong". This is not a particularly fast process; if e.g. @ianna can easily spot the root-cause then feel free to open a PR whilst I take my time ;)
In this instance, it looks like the issue lies in the 
```python
while k < outlength:
    outstarts[k] = lendistincts + 1
    outstops[k] = lendistincts + 1
    k = k + 1
```
portion of  `awkward_ListOffsetArray_reduce_nonlocal_outstartsstops_64`. My initial understanding is that this except is responsible for padding trailing empty-lists.
OK, I *think* we want to set `outstarts[k] = maxdistinct + 1` instead of `lendistincts + 1` - `lendistincts = outlength * maxcount[0]` whereas `maxdistinct` gives the index of the final column.

Resolving patch:
diff --git a/src/awkward/contents/listoffsetarray.py b/src/awkward/contents/listoffsetarray.py
--- a/src/awkward/contents/listoffsetarray.py
+++ b/src/awkward/contents/listoffsetarray.py
@@ -1481,7 +1481,6 @@ def _reduce_next(
 
         branch, depth = self.branch_depth
         globalstarts_length = self._offsets.length - 1
-        parents_length = parents.length
 
         if not branch and negaxis == depth:
             (
@@ -1493,27 +1492,12 @@ def _reduce_next(
                 nextstarts,
             ) = self._rearrange_prepare_next(outlength, parents)
 
-            gaps = ak.index.Index64.empty(outlength, self._nplike)
-            assert gaps.nplike is self._nplike and parents.nplike is self._nplike
-            self._handle_error(
-                self._nplike[
-                    "awkward_ListOffsetArray_reduce_nonlocal_findgaps_64",
-                    gaps.dtype.type,
-                    parents.dtype.type,
-                ](
-                    gaps.data,
-                    parents.data,
-                    parents_length,
-                )
-            )
-
             outstarts = ak.index.Index64.empty(outlength, self._nplike)
             outstops = ak.index.Index64.empty(outlength, self._nplike)
             assert (
                 outstarts.nplike is self._nplike
                 and outstops.nplike is self._nplike
                 and distincts.nplike is self._nplike
-                and gaps.nplike is self._nplike
             )
             self._handle_error(
                 self._nplike[
@@ -1521,13 +1505,11 @@ def _reduce_next(
                     outstarts.dtype.type,
                     outstops.dtype.type,
                     distincts.dtype.type,
-                    gaps.dtype.type,
                 ](
                     outstarts.data,
                     outstops.data,
                     distincts.data,
                     distincts.length,
-                    gaps.data,
                     outlength,
                 )
             )


Test patch:
diff --git a/tests/test_1784-reduce-leading-sublist.py b/tests/test_1784-reduce-leading-sublist.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1784-reduce-leading-sublist.py
@@ -0,0 +1,23 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import numpy as np  # noqa: F401
+import pytest  # noqa: F401
+
+import awkward as ak  # noqa: F401
+
+
+def test():
+    small = ak.Array(
+        ak.contents.ListOffsetArray(
+            ak.index.Index64(np.array([0, 0, 2])),
+            ak.contents.ListOffsetArray(
+                ak.index.Index64(np.array([0, 2, 4])),
+                ak.contents.NumpyArray(np.array([1, 2, 3, 4], dtype=np.float32)),
+            ),
+        )
+    )
+
+    reduced = ak.sum(small, axis=-2)
+    assert reduced.tolist() == [[], [4, 6]]
+    assert np.asarray(reduced.layout.starts).tolist() == [0, 2]
+    assert np.asarray(reduced.layout.stops).tolist() == [0, 4]
diff --git a/tests/test_1791-reduce-trailing-sublist.py b/tests/test_1791-reduce-trailing-sublist.py
new file mode 100644
--- /dev/null
+++ b/tests/test_1791-reduce-trailing-sublist.py
@@ -0,0 +1,133 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+import numpy as np  # noqa: F401
+import pytest  # noqa: F401
+
+import awkward as ak  # noqa: F401
+
+
+def test():
+    array = ak.Array(
+        ak.contents.ListOffsetArray(
+            ak.index.Index64(
+                np.array(
+                    [0, 1, 1],
+                    dtype=np.int64,
+                )
+            ),
+            ak.contents.ListOffsetArray(
+                ak.index.Index64(
+                    np.array(
+                        [0, 1],
+                        dtype=np.int64,
+                    )
+                ),
+                ak.contents.NumpyArray(np.arange(1)),
+            ),
+        )
+    )
+
+    reduced = ak.sum(array, axis=1)
+    # We currently get a ListArray here. Ensure that the start/stops are correct
+    assert np.asarray(reduced.layout.starts).tolist() == [0, 1]
+    assert np.asarray(reduced.layout.stops).tolist() == [1, 1]
+
+
+def test_review_examples():
+    array = ak.Array(
+        ak.contents.ListOffsetArray(
+            ak.index.Index64([0, 4, 4, 6]),
+            ak.contents.ListOffsetArray(
+                ak.index.Index64([0, 3, 3, 5, 6, 8, 9]),
+                ak.contents.NumpyArray([2, 3, 5, 7, 11, 13, 17, 19, 23]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]
+
+    array = ak.Array(
+        ak.contents.ListOffsetArray(
+            ak.index.Index64(np.array([0, 4, 4, 6]) + 2),
+            ak.contents.ListOffsetArray(
+                ak.index.Index64(np.array([123, 321, 0, 3, 3, 5, 6, 8, 9]) + 1),
+                ak.contents.NumpyArray([9999, 2, 3, 5, 7, 11, 13, 17, 19, 23]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]
+
+    array = ak.Array(
+        ak.contents.ListArray(
+            ak.index.Index64([0, 4, 4]),
+            ak.index.Index64([4, 4, 6]),
+            ak.contents.ListArray(
+                ak.index.Index64([0, 3, 3, 5, 6, 8]),
+                ak.index.Index64([3, 3, 5, 6, 8, 9]),
+                ak.contents.NumpyArray([2, 3, 5, 7, 11, 13, 17, 19, 23]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]
+
+    array = ak.Array(
+        ak.contents.ListArray(
+            ak.index.Index64([0, 100, 4]),
+            ak.index.Index64([4, 100, 6]),
+            ak.contents.ListArray(
+                ak.index.Index64([0, 1000, 3, 5, 6, 8]),
+                ak.index.Index64([3, 1000, 5, 6, 8, 9]),
+                ak.contents.NumpyArray([2, 3, 5, 7, 11, 13, 17, 19, 23]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]
+
+    array = ak.Array(
+        ak.contents.ListArray(
+            ak.index.Index64([0, 100, 6]),
+            ak.index.Index64([4, 100, 8]),
+            ak.contents.ListArray(
+                ak.index.Index64([0, 1000, 3, 5, 60, 80, 6, 8]),
+                ak.index.Index64([3, 1000, 5, 6, 80, 90, 8, 9]),
+                ak.contents.NumpyArray([2, 3, 5, 7, 11, 13, 17, 19, 23]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]
+
+    array = ak.Array(
+        ak.contents.ListArray(
+            ak.index.Index64([0, 100, 6]),
+            ak.index.Index64([4, 100, 8]),
+            ak.contents.ListArray(
+                ak.index.Index64([9, 1000, 3, 5, 60, 80, 6, 8]),
+                ak.index.Index64([12, 1000, 5, 6, 80, 90, 8, 9]),
+                ak.contents.NumpyArray([20, 30, 50, 7, 11, 13, 17, 19, 23, 2, 3, 5]),
+            ),
+        )
+    )
+    assert ak.Array(ak.prod(array, axis=-2), check_valid=True).tolist() == [
+        [182, 33, 5],
+        [],
+        [391, 19],
+    ]