Repository: scikit-hep/awkward

Issue:
Can't convert a pyarrow struct with a fixed-size list
Hi there,
Thanks for the library.

I'm trying to use `from_arrow` to convert a pyarrow struct containing fixed-size lists and it's currently failing. I used the following snippet to test:

```py
import pyarrow as pa
import awkward as ak
def test():
    struct = pa.struct([pa.field('x', pa.list_(pa.float64(), 2)), pa.field('y', pa.list_(pa.float64(), 2))])

    arr = pa.array([{'x': [1.0, 2.0], 'y': [3.0, 4.0]}], type=struct)
    assert ak.from_arrow(arr).tolist() == [{"x": [1.0, 2.0]}, {"y": [3.0, 4.0]}]
```

It fails at this assertion https://github.com/scikit-hep/awkward-1.0/blob/main/src/awkward/operations/convert.py#L2767 and I believe the reason is that pyarrow's type is `pyarrow.lib.FixedSizeListType` which is a subclass of `pyarrow.lib.DataType` thus matching it as a null data type.


Discussion:
Thanks for the report! Yes, `pyarrow.lib.FixedSizeListType` is new to me; it will need to be added as a case to convert. The Awkward equivalent is `ak.layout.RegularArray`, so this will be straightforward. (The testing might need to restrict to a limited set of pyarrow versions if it's very new.)

Resolving patch:
diff --git a/src/awkward/operations/convert.py b/src/awkward/operations/convert.py
--- a/src/awkward/operations/convert.py
+++ b/src/awkward/operations/convert.py
@@ -2655,6 +2655,16 @@ def popbuffers(array, tpe, buffers):
             out = ak.layout.ListOffsetArray64(offsets, content)
             # No return yet!
 
+        elif isinstance(tpe, pyarrow.lib.FixedSizeListType):
+            assert tpe.num_buffers == 1
+            mask = buffers.pop(0)
+            content = popbuffers(array.values, tpe.value_type, buffers)
+            if not tpe.value_field.nullable:
+                content = content.content
+
+            out = ak.layout.RegularArray(content, tpe.list_size)
+            # No return yet!
+
         elif isinstance(tpe, pyarrow.lib.UnionType):
             if tpe.mode == "sparse":
                 assert tpe.num_buffers == 2


Test patch:
diff --git a/tests/test_0906-arrow-fixed-size-list-type.py b/tests/test_0906-arrow-fixed-size-list-type.py
new file mode 100644
--- /dev/null
+++ b/tests/test_0906-arrow-fixed-size-list-type.py
@@ -0,0 +1,37 @@
+# BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
+
+from __future__ import absolute_import
+
+import pytest  # noqa: F401
+import numpy as np  # noqa: F401
+import awkward as ak  # noqa: F401
+
+pyarrow = pytest.importorskip("pyarrow")
+
+
+def test():
+    struct = pyarrow.struct(
+        [
+            pyarrow.field("x", pyarrow.list_(pyarrow.float64(), 2)),
+            pyarrow.field("y", pyarrow.list_(pyarrow.float64(), 2)),
+        ]
+    )
+    array = pyarrow.array(
+        [
+            {"x": [1.1, 2.1], "y": [3.1, 4.1]},
+            {"x": [1.2, 2.2], "y": [3.2, 4.2]},
+            {"x": [1.3, 2.3], "y": [3.3, 4.3]},
+        ],
+        type=struct,
+    )
+    as_awkward = ak.from_arrow(array)
+
+    assert ak.to_list(as_awkward) == [
+        {"x": [1.1, 2.1], "y": [3.1, 4.1]},
+        {"x": [1.2, 2.2], "y": [3.2, 4.2]},
+        {"x": [1.3, 2.3], "y": [3.3, 4.3]},
+    ]
+    assert (
+        str(as_awkward.type)
+        == '3 * {"x": option[2 * ?float64], "y": option[2 * ?float64]}'
+    )