Repository: narwhals-dev/narwhals (python)

Issue:
[Bug]: Summing empty Pandas DataFrame
### Describe the bug

`ValueError` when attempting to sum an empty Pandas DataFrame

### Steps or code to reproduce the bug

```py
import narwhals as nw
import pandas as pd
from narwhals import narwhalify

df = pd.DataFrame(
    {
        "name": ["John", "Doe", "Jane"],
    },
)


@narwhalify
def bla(df):
    df = df.filter(nw.col("name") == "Boo")
    df.with_columns(nw.col("name").sum())


bla(df)
```

### Expected results

An empty DataFrame

### Actual results

```
ValueError: Length of values (1) does not match length of index (0)
```

### Please run narwhals.show_version() and enter the output below.

```shell
System:
    python: 3.12.8 (main, Dec  6 2024, 19:42:06) [Clang 18.1.8 ]
executable: ...
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
     narwhals: 1.20.1
       pandas: 2.2.3
       polars: 1.18.0
         cudf:
        modin:
      pyarrow:
        numpy: 2.2.1
```


### Relevant log output

_No response_

Pull request:
fix: scalar reductions on empty inputs
<!--
# Thanks for contributing a pull request! 
## Please make sure you see our contribution guidelines: https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md
-->

## What type of PR is this? (check all applicable)

- [ ] 💾 Refactor
- [ ] ✨ Feature
- [x] 🐛 Bug Fix
- [ ] 🔧 Optimization
- [ ] 📝 Documentation
- [ ] ✅ Test
- [ ] 🐳 Other

## Related issues

- Related issue #1711 
- Closes #1711 

## Checklist

- [x] Code follows style guide (ruff)
- [x] Tests added
- [ ] Documented the changes

## If you have comments or can explain your changes, please do so below

When performing a scalar reduction on an empty DataFrame, Polars, pandas, and PyArrow generated disparate result sets with default options. For example, when computing the sum of an empty column (via Narwhals)
- Polars returns a value of 0
- pandas raises a ValueError (on reconstruction of the Series, computationally it also produces 0 as the scalar)
- PyArrow returns null (default scalarreduction options requires at least 1 observed value to produce observed result)

pandas & PyArrow backends now produce output that is consistent with Polars.

Furthermore, there is a difference in the result set when using `.select` vs `.with_columns` where the former may return a value even if the the input was empty whereas the latter will return an empty DataFrame since its input was empty. Both Polars and PyArrow backends exhibited this behavior so pandas does the same as well now.

---

See the following example for how the behavior of `.sum` was changed.

```python
import narwhals as nw
import pandas as pd
import polars as pl
import pyarrow as pa
from itertools import product

@nw.narwhalify
def nw_select(df):
    return (
        df
        .filter(nw.col("name") == "Boo")
        .select(nw.col("value").sum())
    )

@nw.narwhalify
def nw_with_columns(df):
    return (
        df
        .filter(nw.col("name") == "Boo")
        .with_columns(res=nw.col("value").sum())
    )

pl_df = pl.DataFrame(
    {
        "name": ["John", "Doe", "Jane"],
        "value": [10, 5, 20],
    },
)
pd_df = pl_df.to_pandas()
pa_df = pa.Table.from_pandas(pd_df)


for func, df in product([nw_select, nw_with_columns], [pl_df, pd_df, pa_df]):
    print(f' {func.__name__} → {(type(df).__module__.split(".")[0])} '.center(50, '\N{box drawings light horizontal}'))
    try:
        print(func(df))
    except ValueError as e:
        print(e)
    print()
```

Old behavior
- pandas errors when reconstructing a series from a scalar (index is empty, but passed data is not)
- pyarrow returns null where Polars returns a value
```
─────────────── nw_select → polars ───────────────
shape: (1, 1)
┌───────┐
│ value │
│ ---   │
│ i64   │
╞═══════╡
│ 0     │
└───────┘

─────────────── nw_select → pandas ───────────────
Length of values (1) does not match length of index (0)

────────────── nw_select → pyarrow ───────────────
pyarrow.Table
value: int64
----
value: [[null]]

──────────── nw_with_columns → polars ────────────
shape: (0, 3)
┌──────┬───────┬─────┐
│ name ┆ value ┆ res │
│ ---  ┆ ---   ┆ --- │
│ str  ┆ i64   ┆ i64 │
╞══════╪═══════╪═════╡
└──────┴───────┴─────┘

──────────── nw_with_columns → pandas ────────────
Length of values (1) does not match length of index (0)

─────────── nw_with_columns → pyarrow ────────────
pyarrow.Table
name: string
value: int64
res: null
----
name: []
value: []
res: [0 nulls]
```

New behavior
- `select` operations will produce a consistent scalar output even if the input was empty
    - `all` → `True`
    - `any` → `False`
    - `sum` → `0`
    - `max` → `Null` or `NaN` if non-nullable
    - `min` → `Null` or `NaN` if non-nullable
    - `mean` → `Null` or `NaN` if non-nullable
- `with_columns` returns empty if the input was empty
```
─────────────── nw_select → polars ───────────────
shape: (1, 1)
┌───────┐
│ value │
│ ---   │
│ i64   │
╞═══════╡
│ 0     │
└───────┘

─────────────── nw_select → pandas ───────────────
   value
0      0

────────────── nw_select → pyarrow ───────────────
pyarrow.Table
value: int64
----
value: [[0]]

──────────── nw_with_columns → polars ────────────
shape: (0, 3)
┌──────┬───────┬─────┐
│ name ┆ value ┆ res │
│ ---  ┆ ---   ┆ --- │
│ str  ┆ i64   ┆ i64 │
╞══════╪═══════╪═════╡
└──────┴───────┴─────┘

──────────── nw_with_columns → pandas ────────────
Empty DataFrame
Columns: [name, value, res]
Index: []

─────────── nw_with_columns → pyarrow ────────────
pyarrow.Table
name: string
value: int64
res: null
----
name: []
value: []
res: [0 nulls]
```

Patch:
diff --git a/narwhals/_arrow/series.py b/narwhals/_arrow/series.py
index 37dce5db..ecc5b084 100644
--- a/narwhals/_arrow/series.py
+++ b/narwhals/_arrow/series.py
@@ -304,7 +304,9 @@ class ArrowSeries(CompliantSeries):
     def sum(self: Self, *, _return_py_scalar: bool = True) -> int:
         import pyarrow.compute as pc
 
-        return maybe_extract_py_scalar(pc.sum(self._native_series), _return_py_scalar)  # type: ignore[no-any-return]
+        return maybe_extract_py_scalar(  # type: ignore[no-any-return]
+            pc.sum(self._native_series, min_count=0), _return_py_scalar
+        )
 
     def drop_nulls(self: Self) -> ArrowSeries:
         import pyarrow.compute as pc
@@ -474,12 +476,16 @@ class ArrowSeries(CompliantSeries):
     def any(self: Self, *, _return_py_scalar: bool = True) -> bool:
         import pyarrow.compute as pc
 
-        return maybe_extract_py_scalar(pc.any(self._native_series), _return_py_scalar)  # type: ignore[no-any-return]
+        return maybe_extract_py_scalar(  # type: ignore[no-any-return]
+            pc.any(self._native_series, min_count=0), _return_py_scalar
+        )
 
     def all(self: Self, *, _return_py_scalar: bool = True) -> bool:
         import pyarrow.compute as pc
 
-        return maybe_extract_py_scalar(pc.all(self._native_series), _return_py_scalar)  # type: ignore[no-any-return]
+        return maybe_extract_py_scalar(  # type: ignore[no-any-return]
+            pc.all(self._native_series, min_count=0), _return_py_scalar
+        )
 
     def is_between(
         self, lower_bound: Any, upper_bound: Any, closed: str = "both"
diff --git a/narwhals/_pandas_like/utils.py b/narwhals/_pandas_like/utils.py
index 8d67aa65..1a888ed7 100644
--- a/narwhals/_pandas_like/utils.py
+++ b/narwhals/_pandas_like/utils.py
@@ -293,6 +293,8 @@ def native_series_from_iterable(
     """Return native series."""
     if implementation in PANDAS_LIKE_IMPLEMENTATION:
         extra_kwargs = {"copy": False} if implementation is Implementation.PANDAS else {}
+        if len(index) == 0:
+            index = None
         return implementation.to_native_namespace().Series(
             data, name=name, index=index, **extra_kwargs
         )


Test patch:
diff --git a/tests/expr_and_series/reduction_test.py b/tests/expr_and_series/reduction_test.py
index b1dcad23..3b579d9f 100644
--- a/tests/expr_and_series/reduction_test.py
+++ b/tests/expr_and_series/reduction_test.py
@@ -6,6 +6,7 @@ import pytest
 
 import narwhals.stable.v1 as nw
 from tests.utils import Constructor
+from tests.utils import ConstructorEager
 from tests.utils import assert_equal_data
 
 
@@ -59,3 +60,84 @@ def test_scalar_reduction_with_columns(
     df = nw.from_native(constructor(data))
     result = df.with_columns(*expr).select(*expected.keys())
     assert_equal_data(result, expected)
+
+
+def test_empty_scalar_reduction_select(constructor: Constructor) -> None:
+    data = {
+        "str": [*"abcde"],
+        "int": [0, 1, 2, 3, 4],
+        "bool": [True, False, False, True, False],
+    }
+    expressions = {
+        "all": nw.col("bool").all(),
+        "any": nw.col("bool").any(),
+        "max": nw.col("int").max(),
+        "mean": nw.col("int").mean(),
+        "min": nw.col("int").min(),
+        "sum": nw.col("int").sum(),
+    }
+
+    df = nw.from_native(constructor(data)).filter(str="z")
+
+    result = df.select(**expressions)
+    expected = {
+        "all": [True],
+        "any": [False],
+        "max": [None],
+        "mean": [None],
+        "min": [None],
+        "sum": [0],
+    }
+    assert_equal_data(result, expected)
+
+
+def test_empty_scalar_reduction_with_columns(constructor: Constructor) -> None:
+    from itertools import chain
+
+    data = {
+        "str": [*"abcde"],
+        "int": [0, 1, 2, 3, 4],
+        "bool": [True, False, False, True, False],
+    }
+    expressions = {
+        "all": nw.col("bool").all(),
+        "any": nw.col("bool").any(),
+        "max": nw.col("int").max(),
+        "mean": nw.col("int").mean(),
+        "min": nw.col("int").min(),
+        "sum": nw.col("int").sum(),
+    }
+
+    df = nw.from_native(constructor(data)).filter(str="z")
+    result = df.with_columns(**expressions)
+    expected: dict[str, list[Any]] = {
+        k: [] for k in chain(df.collect_schema(), expressions)
+    }
+    assert_equal_data(result, expected)
+
+
+def test_empty_scalar_reduction_series(constructor_eager: ConstructorEager) -> None:
+    data = {
+        "str": [*"abcde"],
+        "int": [0, 1, 2, 3, 4],
+        "bool": [True, False, False, True, False],
+    }
+    df = nw.from_native(constructor_eager(data), eager_only=True).filter(str="z")
+    result_s = {
+        "all": [df["bool"].all()],
+        "any": [df["bool"].any()],
+        "max": [df["int"].max()],
+        "mean": [df["int"].mean()],
+        "min": [df["int"].min()],
+        "sum": [df["int"].sum()],
+    }
+    expected = {
+        "all": [True],
+        "any": [False],
+        "max": [None],
+        "mean": [None],
+        "min": [None],
+        "sum": [0],
+    }
+
+    assert_equal_data(result_s, expected)