Repository: rapidsai/cudf
Pull request #18768: Create a pylibcudf Column from a python iterable
Author: login: Matt711
Labels: name: feature request
description: New feature or request
name: Python
description: Affects Python cuDF API.
name: non-breaking
description: Non-breaking change
name: cudf-polars
description: Issues specific to cudf-polars
name: pylibcudf
description: Issues specific to the pylibcudf package
Created: 2025-05-12T20:11:01Z | Merged: 2025-05-21T00:32:23Z

## Description
<!-- Provide a standalone description of changes in this PR. -->
<!-- Reference any issues closed by this PR with "closes #1234". -->
<!-- Note: The pull request title will be included in the CHANGELOG. -->
Closes #18745 and Closes #18819 and contributes to #18534 #15132. I replaced as many pyarrow  calls as I could with features supported in this PR. The only notable case I skipped is conversions to pylibcudf Columns from pyarrow string arrays. To support creating pylibcudf columns from string lists, I think we'd need #17192. Nevertheless that case will be covered by #18564 using `pl.Series` instead of `pa.array`.

In the future, when we support #17192, we can use `plc.Column.from_list([...])` instead of `plc.Column(pl.Series(...))`.
## Checklist
- [ ] I am familiar with the [Contributing Guidelines](https://github.com/rapidsai/cudf/blob/HEAD/CONTRIBUTING.md).
- [ ] New or existing tests cover these changes.
- [ ] The documentation is up to date with these changes.


Linked issue #1234: [FEA] Create a single RAPIDS library
**Is your feature request related to a problem? Please describe.**
Hi all,

I would like to share with you the following idea:

So just as we have the following imports:

> import pandas as pd
> import tensorflow as tf
> import matplotlib as plt

why do not create a rapids library that could be imported as follows?

> import rapids as rs

or

> from rapids import cudf

I am aware that there are pros about having each library separate, but I think there are also pros about having a single library, such us

- easier for the data scientists to install: conda install rapids
- it creates a kind of identity around rapids (we do not think on cudf or cuml as independent identities, but as a whole).
- we might adopt 'rs' as a shortcut for rapids (currently, I have only found the following use of 'rs' import rhinoscriptsyntax as  rs)

Please, share your thoughts about the above.

Thanks!

P.S.: There are also cons about having a huge library, but I think the pros exceed the cons.
Linked issue comments: user: login: kkraus14
created_at: 2019-07-03T22:56:24Z
body: This is out of scope for now, possibly to be revisited in the future.
author_association: COLLABORATOR

Diff:
diff --git a/python/cudf_polars/cudf_polars/dsl/expressions/boolean.py b/python/cudf_polars/cudf_polars/dsl/expressions/boolean.py
index 0d12319e9d9..2a34bfbc8c3 100644
--- a/python/cudf_polars/cudf_polars/dsl/expressions/boolean.py
+++ b/python/cudf_polars/cudf_polars/dsl/expressions/boolean.py
@@ -10,8 +10,6 @@
 from functools import partial, reduce
 from typing import TYPE_CHECKING, Any, ClassVar
 
-import pyarrow as pa
-
 import pylibcudf as plc
 
 from cudf_polars.containers import Column
@@ -164,11 +162,9 @@ def do_evaluate(
             if is_finite:
                 # NaN is neither finite not infinite
                 to_search.append(float("nan"))
-            haystack = plc.interop.from_arrow(
-                pa.array(
-                    to_search,
-                    type=plc.interop.to_arrow(needles.obj.type()),
-                )
+            haystack = plc.Column.from_iterable_of_py(
+                to_search,
+                dtype=needles.obj.type(),
             )
             result = plc.search.contains(haystack, needles.obj)
             if is_finite:
diff --git a/python/cudf_polars/cudf_polars/dsl/expressions/datetime.py b/python/cudf_polars/cudf_polars/dsl/expressions/datetime.py
index 2b1816f36d4..fc89eb5b1a8 100644
--- a/python/cudf_polars/cudf_polars/dsl/expressions/datetime.py
+++ b/python/cudf_polars/cudf_polars/dsl/expressions/datetime.py
@@ -153,7 +153,9 @@ def do_evaluate(
                 plc.strings.convert.convert_datetime.from_timestamps(
                     column.obj,
                     self.options[0],
-                    plc.interop.from_arrow(pa.array([], type=pa.string())),
+                    plc.Column.from_iterable_of_py(
+                        [], dtype=plc.DataType(plc.TypeId.STRING)
+                    ),
                 )
             )
         if self.name is TemporalFunction.Name.Week:
@@ -161,8 +163,8 @@ def do_evaluate(
                 plc.strings.convert.convert_datetime.from_timestamps(
                     column.obj,
                     format="%V",
-                    input_strings_names=plc.interop.from_arrow(
-                        pa.array([], type=pa.string())
+                    input_strings_names=plc.Column.from_iterable_of_py(
+                        [], dtype=plc.DataType(plc.TypeId.STRING)
                     ),
                 ),
                 plc.types.DataType(plc.types.TypeId.INT8),
@@ -173,8 +175,8 @@ def do_evaluate(
                 plc.strings.convert.convert_datetime.from_timestamps(
                     column.obj,
                     format="%G",
-                    input_strings_names=plc.interop.from_arrow(
-                        pa.array([], type=pa.string())
+                    input_strings_names=plc.Column.from_iterable_of_py(
+                        [], dtype=plc.DataType(plc.TypeId.STRING)
                     ),
                 ),
                 plc.types.DataType(plc.types.TypeId.INT32),
diff --git a/python/pylibcudf/pylibcudf/column.pyx b/python/pylibcudf/pylibcudf/column.pyx
index 397bcb974bd..802061187fb 100644
--- a/python/pylibcudf/pylibcudf/column.pyx
+++ b/python/pylibcudf/pylibcudf/column.pyx
@@ -57,8 +57,10 @@ from .utils cimport _get_stream
 from .gpumemoryview import _datatype_from_dtype_desc
 from ._interop_helpers import ArrowLike, ColumnMetadata
 
+import array
 import functools
 import operator
+from typing import Iterable
 
 __all__ = ["Column", "ListColumnView", "is_c_contiguous"]
 
@@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
+        raise TypeError(f"Unsupported scalar type: {type(current).__name__}")
+
+    return depth, type(current)
+
+
+def _flatten_nested_list(obj: list, depth: int) -> tuple[list, tuple[int, ...]]:
+    """Flatten a nested list and compute the shape"""
+    shape = _infer_shape(obj, depth)
+
+    flat = [None] * functools.reduce(operator.mul, shape)
+    _flatten(obj, flat, 0)
+    return flat, shape
+
+
+def _infer_shape(obj: list, depth: int) -> tuple[int, ...]:
+    shape = []
+    current = obj
+
+    for i in range(depth):
+        if not current:
+            raise ValueError("Cannot infer shape from empty list")
+
+        shape.append(len(current))
+
+        if i < depth - 1:
+            first = current[0]
+            if not all(
+                isinstance(sub, list) and len(sub) == len(first) for sub in current
+            ):
+                raise ValueError("Inconsistent inner list shapes")
+            current = first
+
+    return tuple(shape)
+
+
+def _flatten(obj: list, out: list, offset: int) -> int:
+    if not isinstance(obj[0], list):
+        out[offset:offset + len(obj)] = obj
+        return offset + len(obj)
+    for sub in obj:
+        offset = _flatten(sub, out, offset)
+    return offset
+
+
+def _python_typecode_from_dtype(dtype: DataType) -> str:
+    """The Python type string."""
+    return {
+        type_id.INT8: 'b',
+        type_id.INT16: 'h',
+        type_id.INT32: 'i',
+        type_id.INT64: 'q',
+        type_id.UINT8: 'B',
+        type_id.UINT16: 'H',
+        type_id.UINT32: 'I',
+        type_id.UINT64: 'Q',
+        type_id.FLOAT32: 'f',
+        type_id.FLOAT64: 'd',
+        type_id.BOOL8: 'b'
+    }[dtype.id()]
+
+
+def _typestr_from_dtype(dtype: DataType) -> str:
+    """The array interface type string."""
+    return {
+        type_id.INT8: "|i1",
+        type_id.INT16: "<i2",
+        type_id.INT32: "<i4",
+        type_id.INT64: "<i8",
+        type_id.UINT8: "|u1",
+        type_id.UINT16: "<u2",
+        type_id.UINT32: "<u4",
+        type_id.UINT64: "<u8",
+        type_id.FLOAT32: "<f4",
+        type_id.FLOAT64: "<f8",
+        type_id.BOOL8: "|b1",
+    }[dtype.id()]
+
+
 def _prepare_array_metadata(
     iface: dict,
 ) -> tuple[int, int, tuple[int, ...], tuple[int, ...] | None, DataType]:
@@ -265,10 +358,10 @@ cdef class Column:
         cdef _ArrowColumnHolder result
         cdef unique_ptr[arrow_column] c_result
         if hasattr(arrow_like, "__arrow_c_device_array__"):
-            schema, array = arrow_like.__arrow_c_device_array__()
+            schema, d_array = arrow_like.__arrow_c_device_array__()
             c_schema = <ArrowSchema*>PyCapsule_GetPointer(schema, "arrow_schema")
             c_device_array = (
-                <ArrowDeviceArray*>PyCapsule_GetPointer(array, "arrow_device_array")
+                <ArrowDeviceArray*>PyCapsule_GetPointer(d_array, "arrow_device_array")
             )
 
             result = _ArrowColumnHolder()
@@ -289,9 +382,9 @@ cdef class Column:
                 tmp.children(),
             )
         elif hasattr(arrow_like, "__arrow_c_array__"):
-            schema, array = arrow_like.__arrow_c_array__()
+            schema, h_array = arrow_like.__arrow_c_array__()
             c_schema = <ArrowSchema*>PyCapsule_GetPointer(schema, "arrow_schema")
-            c_array = <ArrowArray*>PyCapsule_GetPointer(array, "arrow_array")
+            c_array = <ArrowArray*>PyCapsule_GetPointer(h_array, "arrow_array")
 
             result = _ArrowColumnHolder()
             with nogil:
@@ -831,6 +924,69 @@ cdef class Column:
             f"Cannot convert object of type {type(obj)} to a pylibcudf Column"
         )
 
+    @staticmethod
+    def from_iterable_of_py(obj: Iterable, dtype: DataType | None = None) -> Column:
+        """
+        Create a Column from a Python iterable of scalar values or nested iterables.
+
+        Parameters
+        ----------
+        obj : Iterable
+            An iterable of scalar values (e.g., int, float, bool) or a nested iterable.
+        dtype : DataType | None
+            The data type of the elements. If not specified, the type is inferred.
+
+        Returns
+        -------
+        Column
+            A Column containing the data from the input iterable.
+
+        Raises
+        ------
+        TypeError
+            If the input contains unsupported scalar types.
+        ValueError
+            If the iterable is empty and dtype is not provided.
+
+        Notes
+        -----
+        - Only scalar types int, float, and bool are supported.
+        - Nested iterables must be materialized as lists.
+        """
+
+        # TODO: Investigate when we can avoid full list materialization.
+        obj = list(obj)
+
+        if not obj:
+            if dtype is None:
+                raise ValueError("Cannot infer dtype from empty iterable object")
+            return Column(dtype, 0, None, None, 0, 0, [])
+
+        if dtype is None:
+            depth, py_dtype = _infer_list_depth_and_dtype(obj)
+            dtype = DataType.from_py(py_dtype)
+        else:
+            depth, _ = _infer_list_depth_and_dtype(obj)
+
+        flat, shape = _flatten_nested_list(obj, depth)
+
+        buf = array.array(_python_typecode_from_dtype(dtype), flat)
+        mv = memoryview(buf).cast("B")
+
+        iface = {
+            "data": (mv.obj.buffer_info()[0], False),
+            "shape": shape,
+            "typestr": _typestr_from_dtype(dtype),
+            "strides": None,
+            "version": 3,
+        }
+
+        class ArrayInterfaceWrapper:
+            def __init__(self, iface):
+                self.__array_interface__ = iface
+
+        return Column.from_array_interface(ArrayInterfaceWrapper(iface))
+
     cpdef DataType type(self):
         """The type of data in the column."""
         return self._data_type
diff --git a/python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py b/python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py
new file mode 100644
index 00000000000..6131fd5858c
--- /dev/null
+++ b/python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py
@@ -0,0 +1,168 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.
+
+import array
+import operator
+from functools import reduce
+
+import pyarrow as pa
+import pytest
+from utils import assert_column_eq
+
+import pylibcudf as plc
+
+DTYPES = [
+    ("int", int, pa.int64()),
+    ("float", float, pa.float64()),
+    ("bool", bool, pa.bool_()),
+]
+
+SHAPES = [
+    (6,),
+    (2, 3),
+    (2, 2, 3),
+    (2, 2, 2, 3),
+]
+
+
+@pytest.fixture(params=SHAPES, ids=lambda x: f"shape={x}")
+def shape(request):
+    return request.param
+
+
+@pytest.fixture(params=DTYPES, ids=lambda x: x[0])
+def dtype_info(request):
+    return request.param
+
+
+def generate_list_data(shape, dtype):
+    def gen_values(n):
+        if dtype is bool:
+            return [(i % 2) == 0 for i in range(n)]
+        else:
+            return [dtype(i) for i in range(n)]
+
+    def nest(values, shape):
+        if not shape:
+            return values[0]
+        size = len(values) // shape[0]
+        return [
+            nest(values[i * size : (i + 1) * size], shape[1:])
+            for i in range(shape[0])
+        ]
+
+    return nest(gen_values(reduce(operator.mul, shape)), shape)
+
+
+def test_from_list(shape, dtype_info):
+    _, py_type, pa_type = dtype_info
+    data = generate_list_data(shape, py_type)
+    expect_type = pa_type
+    for _ in range(len(shape) - 1):
+        expect_type = pa.list_(expect_type)
+
+    expect = pa.array(data, type=expect_type)
+
+    got = plc.Column.from_iterable_of_py(
+        data, dtype=plc.interop.from_arrow(pa_type)
+    )
+
+    assert_column_eq(expect, got)
+
+
+def test_from_list_empty_without_dtype_raises():
+    with pytest.raises(
+        ValueError, match="Cannot infer dtype from empty iterable object"
+    ):
+        plc.Column.from_iterable_of_py([])
+
+
+def test_from_list_irregular_shapes_raises():
+    data = [[1], [2, 3], [], [4]]
+    with pytest.raises(ValueError, match="Inconsistent inner list shapes"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_dicts_raises():
+    data = [[{"a": 1}], [{"b": 2}]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_strings_raises():
+    data = [["a", "b"], ["c", "d"]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_none_raises():
+    data = [[None], [None]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_zero_dimensional_list_with_dtype():
+    got = plc.Column.from_iterable_of_py(
+        [], dtype=plc.DataType(plc.TypeId.STRING)
+    )
+    expect = pa.array([], type=pa.string())
+    assert_column_eq(expect, got)
+
+
+@pytest.mark.parametrize(
+    "factory, pa_type",
+    [
+        (lambda: range(5), pa.int64()),
+        (lambda: (x for x in range(5)), pa.float32()),
+        (lambda: array.array("q", range(5)), pa.int64()),
+        (lambda: filter(lambda x: x % 2 == 0, range(10)), pa.int16()),
+        (lambda: map(int, [1.1, 2.2, 3.3]), pa.int64()),
+        (lambda: reversed([1, 2, 3, 4]), pa.int64()),
+        (lambda: tuple(range(4)), pa.int32()),
+    ],
+)
+def test_from_builtin_iterables(factory, pa_type):
+    materialized = list(factory())
+    got = plc.Column.from_iterable_of_py(
+        iter(materialized), dtype=plc.interop.from_arrow(pa_type)
+    )
+    expect = pa.array(materialized, type=pa_type)
+    assert_column_eq(expect, got)
+
+
+def test_from_custom_iterable():
+    class Foo:
+        def __init__(self, values):
+            self.values = values
+
+        def __iter__(self):
+            return iter(self.values)
+
+    list_data = [1, 2, 3]
+    got = plc.Column.from_iterable_of_py(
+        Foo(list_data), dtype=plc.DataType(plc.TypeId.INT64)
+    )
+    expect = pa.array(list_data, type=pa.int64())
+    assert_column_eq(expect, got)
+
+
+def test_from_custom_generator():
+    class Foo:
+        def __init__(self, stop):
+            self.i = 0
+            self.stop = stop
+
+        def __iter__(self):
+            return self
+
+        def __next__(self):
+            if self.i < self.stop:
+                val = self.i
+                self.i += 1
+                return val
+            raise StopIteration
+
+    got = plc.Column.from_iterable_of_py(
+        Foo(4), dtype=plc.DataType(plc.TypeId.INT64)
+    )
+    expect = pa.array([0, 1, 2, 3], type=pa.int64())
+    assert_column_eq(expect, got)
diff --git a/python/pylibcudf/pylibcudf/types.pyi b/python/pylibcudf/pylibcudf/types.pyi
index c91a95414bd..abe6b8c6275 100644
--- a/python/pylibcudf/pylibcudf/types.pyi
+++ b/python/pylibcudf/pylibcudf/types.pyi
@@ -79,6 +79,7 @@ class DataType:
     def __init__(self, type_id: TypeId, scale: int = 0): ...
     def id(self) -> TypeId: ...
     def scale(self) -> int: ...
+    def from_py(self, type: type) -> DataType: ...
 
 def size_of(t: DataType) -> int: ...
 
diff --git a/python/pylibcudf/pylibcudf/types.pyx b/python/pylibcudf/pylibcudf/types.pyx
index 4dd61ebb05e..d814f1f5a43 100644
--- a/python/pylibcudf/pylibcudf/types.pyx
+++ b/python/pylibcudf/pylibcudf/types.pyx
@@ -93,6 +93,41 @@ cdef class DataType:
         ret.c_obj = dt
         return ret
 
+    @staticmethod
+    def from_py(typ: type) -> DataType:
+        """
+        Construct a DataType from a Python type.
+
+        Parameters
+        ----------
+        typ : type
+            A Python type (eg. int, str, list)
+
+        Returns
+        -------
+        DataType
+            The corresponding pylibcudf DataType.
+
+        Raises
+        ------
+        TypeError
+            If the Python type is not supported.
+        """
+        if typ is bool:
+            return DataType(type_id.BOOL8)
+        elif typ is int:
+            return DataType(type_id.INT64)
+        elif typ is float:
+            return DataType(type_id.FLOAT64)
+        elif typ is str:
+            return DataType(type_id.STRING)
+        elif typ is list:
+            return DataType(type_id.LIST)
+        elif typ is dict:
+            return DataType(type_id.STRUCT)
+        else:
+            raise TypeError(f"Cannot infer DataType from Python type {typ}")
+
 cpdef size_t size_of(DataType t):
     """Returns the size in bytes of elements of the specified data_type.
 


Reviews:
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T20:25:48Z
user: login: Matt711
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T17:02:11Z
user: login: mroeschke
body: I think you can generalize `list` here to be iterable and just tackle https://github.com/rapidsai/cudf/issues/18819 in this PR as well.

So instead of `Column.from_list` you can create `Column.from_iterable_of_py`
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T17:41:22Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T17:45:58Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T17:50:21Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T16:46:07Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T16:48:06Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T16:52:01Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T16:58:18Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T17:04:32Z
user: login: mroeschke
body: A few non-blocking comments that would be nice to look at in a follow up, so approving. 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T17:05:51Z
user: login: mroeschke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T19:15:51Z
user: login: Matt711
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T11:38:36Z
user: login: Matt711
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T11:39:02Z
user: login: Matt711
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T12:29:34Z
user: login: Matt711
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T12:31:00Z

Review comments:
diff_hunk: @@ -462,9 +462,10 @@ def add_file_paths(
         Each path is repeated according to the number of rows read from it.
         """
         (filepaths,) = plc.filling.repeat(
-            # TODO: Remove call from_arrow when we support python list to Column
+            # TODO: Remove call from_arrow when we support python list[str] to strings Column
+            # Needs https://github.com/rapidsai/cudf/issues/17192
             plc.Table([plc.interop.from_arrow(pa.array(map(str, paths)))]),
-            plc.interop.from_arrow(pa.array(rows_per_path, type=pa.int32())),
+            plc.Column.from_list(rows_per_path, dtype=plc.DataType(plc.TypeId.INT32)),
path: python/cudf_polars/cudf_polars/dsl/ir.py
user: login: mroeschke
body: Just FYI, I'm just using `pl.Series` for this in https://github.com/rapidsai/cudf/pull/18564/files#diff-58a9bfc6f3a54d841c1a798c95354d72b05555be355adec331ca5d5d1c67ad16R465
created_at: 2025-05-12T20:25:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +127,84 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    scalar_type = type(current)
+    if scalar_type not in (int, float, bool):
+        raise TypeError(f"Unsupported scalar type: {scalar_type}")
+
+    return depth, scalar_type
+
+
+def _flatten_nested_list(obj: list, depth: int) -> tuple[list, tuple[int, ...]]:
+    """Flatten a nested list to a flat list of scalars and compute shape."""
+    if depth == 1:
+        shape = (len(obj),)
+        return obj, shape
+
+    sublists = obj
+    n_outer = len(sublists)
+    if n_outer == 0:
+        raise ValueError("Cannot determine shape from empty outer list")
+
+    flat = []
+    shapes = []
+
+    for sub in sublists:
+        sub_flat, sub_shape = _flatten_nested_list(sub, depth - 1)
+        flat.extend(sub_flat)
path: python/pylibcudf/pylibcudf/column.pyx
user: login: Matt711
body: Maybe preallocate the correct size first?
created_at: 2025-05-14T17:02:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +127,98 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    scalar_type = type(current)
+    if scalar_type not in {int, float, bool}:
+        raise TypeError(f"Unsupported scalar type: {scalar_type}")
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: ```suggestion
    if not isinstance(current, (int, float, bool)):
        raise TypeError(f"Unsupported scalar type: {type(current).__name__}")
```
created_at: 2025-05-15T17:45:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -831,6 +923,78 @@ cdef class Column:
             f"Cannot convert object of type {type(obj)} to a pylibcudf Column"
         )
 
+    @staticmethod
+    def from_list(obj: list, dtype: DataType | None = None) -> Column:
+        """
+        Create a Column from a Python list
+
+        Parameters
+        ----------
+        obj : list
+            A list of scalar values (e.g., int, float, bool)
+            or a nested list of such values.
+        dtype : DataType | None
+            The data type of the elements. If not specified,
+            the type is inferred from the list.
+
+        Returns
+        -------
+        Column
+            A Column containing the data from the input list.
+
+        Raises
+        ------
+        TypeError
+            If the input is not a list or contains unsupported scalar types.
+        ValueError
+            If the list is empty and no dtype is specified.
+            If the nested lists are irregular (i.e., have inconsistent shapes).
+
+        Notes
+        -----
+        - Only scalar types int, float, and bool are supported.
+        - The input list may be multi-dimensional, and will be interpreted
+          as a C-contiguous array with nested list columns.
+
+        Examples
+        --------
+        >>> import pylibcudf as plc
+        >>> plc.Column.from_list([1, 2, 3])
+        >>> plc.Column.from_list([[1, 2], [3, 4]])
+        >>> plc.Column.from_list([[1.0, 2.0], [3.0, 4.0]], dtype=plc.DataType.FLOAT64)
+        """
+        if not obj:
+            if dtype is None:
+                raise ValueError(
+                    "Cannot infer dtype from empty list, please specify dtype"
+                )
+            return Column(dtype, 0, None, None, 0, 0, [])
+        if dtype is None:
+            depth, py_dtype = _infer_list_depth_and_dtype(obj)
+            dtype = DataType.from_py(py_dtype)
+        else:
+            depth, _ = _infer_list_depth_and_dtype(obj)
+
+        flat, shape = _flatten_nested_list(obj, depth)
+
+        import array
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: Could we import this at the top?
created_at: 2025-05-15T17:50:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: For a future PR, we should to check that all elements are the same type e.g. `Column.from_iterable_of_py([1, 1.0, True])` should probably raise
created_at: 2025-05-20T16:46:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: Also that `None` is probably an acceptable input as a missing value.
created_at: 2025-05-20T16:48:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
+        raise TypeError(f"Unsupported scalar type: {type(current).__name__}")
+
+    return depth, type(current)
+
+
+def _flatten_nested_list(obj: list, depth: int) -> tuple[list, tuple[int, ...]]:
+    """Flatten a nested list and compute the shape"""
+    shape = _infer_shape(obj, depth)
+
+    flat = [None] * functools.reduce(operator.mul, shape)
+    _flatten(obj, flat, 0)
+    return flat, shape
+
+
+def _infer_shape(obj: list, depth: int) -> tuple[int, ...]:
+    shape = []
+    current = obj
+
+    for i in range(depth):
+        if not current:
+            raise ValueError("Cannot infer shape from empty list")
+
+        shape.append(len(current))
+
+        if i < depth - 1:
+            first = current[0]
+            if not all(
+                isinstance(sub, list) and len(sub) == len(first) for sub in current
+            ):
+                raise ValueError("Inconsistent inner list shapes")
+            current = first
+
+    return tuple(shape)
+
+
+def _flatten(obj: list, out: list, offset: int) -> int:
+    if not isinstance(obj[0], list):
+        out[offset:offset + len(obj)] = obj
+        return offset + len(obj)
+    for sub in obj:
+        offset = _flatten(sub, out, offset)
+    return offset
+
+
+def _python_typecode_from_dtype(dtype: DataType) -> str:
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: Maybe in the future, these should be attributes of `pylibcudf.DataType`
created_at: 2025-05-20T16:52:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
+        raise TypeError(f"Unsupported scalar type: {type(current).__name__}")
+
+    return depth, type(current)
+
+
+def _flatten_nested_list(obj: list, depth: int) -> tuple[list, tuple[int, ...]]:
+    """Flatten a nested list and compute the shape"""
+    shape = _infer_shape(obj, depth)
+
+    flat = [None] * functools.reduce(operator.mul, shape)
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: Nit: It would be nice to only need to allocate this if we have a nested list, but can be looked at in a follow up
created_at: 2025-05-20T16:58:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,168 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.
+
+import array
+import operator
+from functools import reduce
+
+import pyarrow as pa
+import pytest
+from utils import assert_column_eq
+
+import pylibcudf as plc
+
+DTYPES = [
+    ("int", int, pa.int64()),
+    ("float", float, pa.float64()),
+    ("bool", bool, pa.bool_()),
+]
+
+SHAPES = [
+    (6,),
+    (2, 3),
+    (2, 2, 3),
+    (2, 2, 2, 3),
+]
+
+
+@pytest.fixture(params=SHAPES, ids=lambda x: f"shape={x}")
+def shape(request):
+    return request.param
+
+
+@pytest.fixture(params=DTYPES, ids=lambda x: x[0])
+def dtype_info(request):
+    return request.param
+
+
+def generate_list_data(shape, dtype):
+    def gen_values(n):
+        if dtype is bool:
+            return [(i % 2) == 0 for i in range(n)]
+        else:
+            return [dtype(i) for i in range(n)]
+
+    def nest(values, shape):
+        if not shape:
+            return values[0]
+        size = len(values) // shape[0]
+        return [
+            nest(values[i * size : (i + 1) * size], shape[1:])
+            for i in range(shape[0])
+        ]
+
+    return nest(gen_values(reduce(operator.mul, shape)), shape)
+
+
+def test_from_list(shape, dtype_info):
+    _, py_type, pa_type = dtype_info
+    data = generate_list_data(shape, py_type)
+    expect_type = pa_type
+    for _ in range(len(shape) - 1):
+        expect_type = pa.list_(expect_type)
+
+    expect = pa.array(data, type=expect_type)
+
+    got = plc.Column.from_iterable_of_py(
+        data, dtype=plc.interop.from_arrow(pa_type)
+    )
+
+    assert_column_eq(expect, got)
+
+
+def test_from_list_empty_without_dtype_raises():
+    with pytest.raises(
+        ValueError, match="Cannot infer dtype from empty iterable object"
+    ):
+        plc.Column.from_iterable_of_py([])
+
+
+def test_from_list_irregular_shapes_raises():
+    data = [[1], [2, 3], [], [4]]
+    with pytest.raises(ValueError, match="Inconsistent inner list shapes"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_dicts_raises():
+    data = [[{"a": 1}], [{"b": 2}]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_strings_raises():
+    data = [["a", "b"], ["c", "d"]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_none_raises():
+    data = [[None], [None]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_zero_dimensional_list_with_dtype():
+    got = plc.Column.from_iterable_of_py(
+        [], dtype=plc.DataType(plc.TypeId.STRING)
+    )
+    expect = pa.array([], type=pa.string())
+    assert_column_eq(expect, got)
+
+
+@pytest.mark.parametrize(
+    "factory, pa_type",
+    [
+        (lambda: range(5), pa.int64()),
+        (lambda: (x for x in range(5)), pa.float32()),
+        (lambda: array.array("q", range(5)), pa.int64()),
+        (lambda: filter(lambda x: x % 2 == 0, range(10)), pa.int16()),
+        (lambda: map(int, [1.1, 2.2, 3.3]), pa.int64()),
+        (lambda: reversed([1, 2, 3, 4]), pa.int64()),
+        (lambda: tuple(range(4)), pa.int32()),
+    ],
+)
+def test_from_builtin_iterables(factory, pa_type):
+    materialized = list(factory())
+    got = plc.Column.from_iterable_of_py(
+        iter(materialized), dtype=plc.interop.from_arrow(pa_type)
path: python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py
user: login: mroeschke
body: Do we need to always call `iter` here? Is `array.array` not iterable?
created_at: 2025-05-20T17:04:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
path: python/pylibcudf/pylibcudf/column.pyx
user: login: mroeschke
body: Another note: I realized for the `LiteralColumn` pyarrow removal I'll need to support an iterable of strings. I'll probably add support for it in https://github.com/rapidsai/cudf/pull/18564 unless there's something else I would need to consider supporting strings
created_at: 2025-05-20T19:15:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,168 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.
+
+import array
+import operator
+from functools import reduce
+
+import pyarrow as pa
+import pytest
+from utils import assert_column_eq
+
+import pylibcudf as plc
+
+DTYPES = [
+    ("int", int, pa.int64()),
+    ("float", float, pa.float64()),
+    ("bool", bool, pa.bool_()),
+]
+
+SHAPES = [
+    (6,),
+    (2, 3),
+    (2, 2, 3),
+    (2, 2, 2, 3),
+]
+
+
+@pytest.fixture(params=SHAPES, ids=lambda x: f"shape={x}")
+def shape(request):
+    return request.param
+
+
+@pytest.fixture(params=DTYPES, ids=lambda x: x[0])
+def dtype_info(request):
+    return request.param
+
+
+def generate_list_data(shape, dtype):
+    def gen_values(n):
+        if dtype is bool:
+            return [(i % 2) == 0 for i in range(n)]
+        else:
+            return [dtype(i) for i in range(n)]
+
+    def nest(values, shape):
+        if not shape:
+            return values[0]
+        size = len(values) // shape[0]
+        return [
+            nest(values[i * size : (i + 1) * size], shape[1:])
+            for i in range(shape[0])
+        ]
+
+    return nest(gen_values(reduce(operator.mul, shape)), shape)
+
+
+def test_from_list(shape, dtype_info):
+    _, py_type, pa_type = dtype_info
+    data = generate_list_data(shape, py_type)
+    expect_type = pa_type
+    for _ in range(len(shape) - 1):
+        expect_type = pa.list_(expect_type)
+
+    expect = pa.array(data, type=expect_type)
+
+    got = plc.Column.from_iterable_of_py(
+        data, dtype=plc.interop.from_arrow(pa_type)
+    )
+
+    assert_column_eq(expect, got)
+
+
+def test_from_list_empty_without_dtype_raises():
+    with pytest.raises(
+        ValueError, match="Cannot infer dtype from empty iterable object"
+    ):
+        plc.Column.from_iterable_of_py([])
+
+
+def test_from_list_irregular_shapes_raises():
+    data = [[1], [2, 3], [], [4]]
+    with pytest.raises(ValueError, match="Inconsistent inner list shapes"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_dicts_raises():
+    data = [[{"a": 1}], [{"b": 2}]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_strings_raises():
+    data = [["a", "b"], ["c", "d"]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_none_raises():
+    data = [[None], [None]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_zero_dimensional_list_with_dtype():
+    got = plc.Column.from_iterable_of_py(
+        [], dtype=plc.DataType(plc.TypeId.STRING)
+    )
+    expect = pa.array([], type=pa.string())
+    assert_column_eq(expect, got)
+
+
+@pytest.mark.parametrize(
+    "factory, pa_type",
+    [
+        (lambda: range(5), pa.int64()),
+        (lambda: (x for x in range(5)), pa.float32()),
+        (lambda: array.array("q", range(5)), pa.int64()),
+        (lambda: filter(lambda x: x % 2 == 0, range(10)), pa.int16()),
+        (lambda: map(int, [1.1, 2.2, 3.3]), pa.int64()),
+        (lambda: reversed([1, 2, 3, 4]), pa.int64()),
+        (lambda: tuple(range(4)), pa.int32()),
+    ],
+)
+def test_from_builtin_iterables(factory, pa_type):
+    materialized = list(factory())
+    got = plc.Column.from_iterable_of_py(
+        iter(materialized), dtype=plc.interop.from_arrow(pa_type)
path: python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py
user: login: Matt711
body: No I think we can remove it
created_at: 2025-05-21T11:38:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
+        raise TypeError(f"Unsupported scalar type: {type(current).__name__}")
+
+    return depth, type(current)
+
+
+def _flatten_nested_list(obj: list, depth: int) -> tuple[list, tuple[int, ...]]:
+    """Flatten a nested list and compute the shape"""
+    shape = _infer_shape(obj, depth)
+
+    flat = [None] * functools.reduce(operator.mul, shape)
+    _flatten(obj, flat, 0)
+    return flat, shape
+
+
+def _infer_shape(obj: list, depth: int) -> tuple[int, ...]:
+    shape = []
+    current = obj
+
+    for i in range(depth):
+        if not current:
+            raise ValueError("Cannot infer shape from empty list")
+
+        shape.append(len(current))
+
+        if i < depth - 1:
+            first = current[0]
+            if not all(
+                isinstance(sub, list) and len(sub) == len(first) for sub in current
+            ):
+                raise ValueError("Inconsistent inner list shapes")
+            current = first
+
+    return tuple(shape)
+
+
+def _flatten(obj: list, out: list, offset: int) -> int:
+    if not isinstance(obj[0], list):
+        out[offset:offset + len(obj)] = obj
+        return offset + len(obj)
+    for sub in obj:
+        offset = _flatten(sub, out, offset)
+    return offset
+
+
+def _python_typecode_from_dtype(dtype: DataType) -> str:
path: python/pylibcudf/pylibcudf/column.pyx
user: login: Matt711
body: Yes I agree
created_at: 2025-05-21T11:39:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,168 @@
+# Copyright (c) 2025, NVIDIA CORPORATION.
+
+import array
+import operator
+from functools import reduce
+
+import pyarrow as pa
+import pytest
+from utils import assert_column_eq
+
+import pylibcudf as plc
+
+DTYPES = [
+    ("int", int, pa.int64()),
+    ("float", float, pa.float64()),
+    ("bool", bool, pa.bool_()),
+]
+
+SHAPES = [
+    (6,),
+    (2, 3),
+    (2, 2, 3),
+    (2, 2, 2, 3),
+]
+
+
+@pytest.fixture(params=SHAPES, ids=lambda x: f"shape={x}")
+def shape(request):
+    return request.param
+
+
+@pytest.fixture(params=DTYPES, ids=lambda x: x[0])
+def dtype_info(request):
+    return request.param
+
+
+def generate_list_data(shape, dtype):
+    def gen_values(n):
+        if dtype is bool:
+            return [(i % 2) == 0 for i in range(n)]
+        else:
+            return [dtype(i) for i in range(n)]
+
+    def nest(values, shape):
+        if not shape:
+            return values[0]
+        size = len(values) // shape[0]
+        return [
+            nest(values[i * size : (i + 1) * size], shape[1:])
+            for i in range(shape[0])
+        ]
+
+    return nest(gen_values(reduce(operator.mul, shape)), shape)
+
+
+def test_from_list(shape, dtype_info):
+    _, py_type, pa_type = dtype_info
+    data = generate_list_data(shape, py_type)
+    expect_type = pa_type
+    for _ in range(len(shape) - 1):
+        expect_type = pa.list_(expect_type)
+
+    expect = pa.array(data, type=expect_type)
+
+    got = plc.Column.from_iterable_of_py(
+        data, dtype=plc.interop.from_arrow(pa_type)
+    )
+
+    assert_column_eq(expect, got)
+
+
+def test_from_list_empty_without_dtype_raises():
+    with pytest.raises(
+        ValueError, match="Cannot infer dtype from empty iterable object"
+    ):
+        plc.Column.from_iterable_of_py([])
+
+
+def test_from_list_irregular_shapes_raises():
+    data = [[1], [2, 3], [], [4]]
+    with pytest.raises(ValueError, match="Inconsistent inner list shapes"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_dicts_raises():
+    data = [[{"a": 1}], [{"b": 2}]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_strings_raises():
+    data = [["a", "b"], ["c", "d"]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_list_nested_none_raises():
+    data = [[None], [None]]
+    with pytest.raises(TypeError, match="Unsupported scalar type"):
+        plc.Column.from_iterable_of_py(data)
+
+
+def test_from_zero_dimensional_list_with_dtype():
+    got = plc.Column.from_iterable_of_py(
+        [], dtype=plc.DataType(plc.TypeId.STRING)
+    )
+    expect = pa.array([], type=pa.string())
+    assert_column_eq(expect, got)
+
+
+@pytest.mark.parametrize(
+    "factory, pa_type",
+    [
+        (lambda: range(5), pa.int64()),
+        (lambda: (x for x in range(5)), pa.float32()),
+        (lambda: array.array("q", range(5)), pa.int64()),
+        (lambda: filter(lambda x: x % 2 == 0, range(10)), pa.int16()),
+        (lambda: map(int, [1.1, 2.2, 3.3]), pa.int64()),
+        (lambda: reversed([1, 2, 3, 4]), pa.int64()),
+        (lambda: tuple(range(4)), pa.int32()),
+    ],
+)
+def test_from_builtin_iterables(factory, pa_type):
+    materialized = list(factory())
+    got = plc.Column.from_iterable_of_py(
+        iter(materialized), dtype=plc.interop.from_arrow(pa_type)
path: python/pylibcudf/pylibcudf/tests/test_column_from_iterable.py
user: login: Matt711
body: No I think we can remove it
created_at: 2025-05-21T12:29:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -127,6 +129,97 @@ cdef class OwnerMaskWithCAI:
         return self.cai
 
 
+def _infer_list_depth_and_dtype(obj: list) -> tuple[int, type]:
+    """Infer the nesting depth and final scalar type."""
+    depth = 0
+    current = obj
+
+    while isinstance(current, list) and current:
+        current = current[0]
+        depth += 1
+
+    if not current and depth == 0:
+        raise ValueError("Cannot infer dtype from empty input")
+
+    if not isinstance(current, (int, float, bool)):
path: python/pylibcudf/pylibcudf/column.pyx
user: login: Matt711
body: I'll add support an iterable of strings in a follow up PR. I think we need to do the construction differently because the leaf is a strings column.
created_at: 2025-05-21T12:31:00Z
author_association: CONTRIBUTOR

PR comments:
user: login: copy-pr-bot[bot]
created_at: 2025-05-12T20:11:05Z
body: Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message [here](https://docs.gha-runners.nvidia.com/cpr/auto-sync).

author_association: NONE
user: login: Matt711
created_at: 2025-05-15T19:01:25Z
body: > I think you can generalize `list` here to be iterable and just tackle #18819 in this PR as well.
> 
> So instead of `Column.from_list` you can create `Column.from_iterable_of_py`

Are you thinking we should just convert the iterable to a list first?

Edit: I just called `list(obj)` and noted the limitation in the doc string
author_association: CONTRIBUTOR
user: login: mroeschke
created_at: 2025-05-15T20:27:06Z
body: > Are you thinking we should just convert the iterable to a list first?

I think you'll only need to do this if the iterable is a generator (`isinstance(obj, collections.abc.Generator)`), otherwise I think you can replace your `list` checks with `collections.abc.Iterable`. For flattening, you can flatten the iterable into a `list` though
author_association: CONTRIBUTOR
user: login: Matt711
created_at: 2025-05-19T12:02:41Z
body: > > Are you thinking we should just convert the iterable to a list first?
> 
> I think you'll only need to do this if the iterable is a generator (`isinstance(obj, collections.abc.Generator)`), otherwise I think you can replace your `list` checks with `collections.abc.Iterable`. For flattening, you can flatten the iterable into a `list` though

I've gone with calling `list(obj)` up front for now, since we need to materialize the iterable as a list/tuple to infer the shape and flatten the iterable in the nested case anyway. I added a TODO for investigating when we can avoid full list materialization (which I think we can for 1d iterables only).
author_association: CONTRIBUTOR
user: login: mroeschke
created_at: 2025-05-21T00:32:18Z
body: /merge
author_association: CONTRIBUTOR