Repository: narwhals-dev/narwhals (python)

Issue:
[Bug]: `is_duplicated` method results in different outputs for pyarrow vs. polars
### Describe the bug

As I was exploring narwhals, I noticed `df.is_duplicated()` produces unexpected results for pyarrow tables. When null values are present in a a table, `is_duplidated` defaults to `True` for pyarrow tables. See reproducible example.


### Steps or code to reproduce the bug



``` python
import polars as pl
import pyarrow as pa
import narwhals as nw

url_data = "https://raw.githubusercontent.com/great-expectations/gx_tutorials/main/data/yellow_tripdata_sample_2019-01.csv"

data = {
    "col1": [1, 2, 3],
    "col2": ["one", None, None]
}

eagr_df = pl.DataFrame(data)
arrw_df = pa.Table.from_pydict(data)
arrw_df_from_pl = eagr_df.to_arrow()
```

``` python
@nw.narwhalify
def add_dupl_col(df: nw.typing.IntoFrame):
    df = df.with_columns(is_duplicated=df.is_duplicated())
    return df.to_polars()
```

``` python
(
    add_dupl_col(eagr_df),
    add_dupl_col(arrw_df),
    add_dupl_col(arrw_df_from_pl),
)
```

    (shape: (3, 3)
     ┌──────┬──────┬───────────────┐
     │ col1 ┆ col2 ┆ is_duplicated │
     │ ---  ┆ ---  ┆ ---           │
     │ i64  ┆ str  ┆ bool          │
     ╞══════╪══════╪═══════════════╡
     │ 1    ┆ one  ┆ false         │
     │ 2    ┆ null ┆ false         │
     │ 3    ┆ null ┆ false         │
     └──────┴──────┴───────────────┘,
     shape: (3, 3)
     ┌──────┬──────┬───────────────┐
     │ col1 ┆ col2 ┆ is_duplicated │
     │ ---  ┆ ---  ┆ ---           │
     │ i64  ┆ str  ┆ bool          │
     ╞══════╪══════╪═══════════════╡
     │ 1    ┆ one  ┆ false         │
     │ 2    ┆ null ┆ true          │
     │ 3    ┆ null ┆ true          │
     └──────┴──────┴───────────────┘,
     shape: (3, 3)
     ┌──────┬──────┬───────────────┐
     │ col1 ┆ col2 ┆ is_duplicated │
     │ ---  ┆ ---  ┆ ---           │
     │ i64  ┆ str  ┆ bool          │
     ╞══════╪══════╪═══════════════╡
     │ 1    ┆ one  ┆ false         │
     │ 2    ┆ null ┆ true          │
     │ 3    ┆ null ┆ true          │
     └──────┴──────┴───────────────┘)


### Expected results

Same output for polars and pyarrow tables

### Actual results

See above

### Please run narwhals.show_version() and enter the output below.

```shell
System:
    python: 3.11.10 (main, Sep  7 2024, 01:03:31) [Clang 15.0.0 (clang-1500.3.9.4)]
executable: /Users/sinan/personal/explore-narhwals/.venv/bin/python
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
     narwhals: 1.25.0
       pandas: 2.2.3
       polars: 1.21.0
         cudf: 
        modin: 
      pyarrow: 19.0.0
        numpy: 2.2.2
```

### Relevant log output

```shell

```

Pull request:
fix: raise NotImplementedError for DataFramte.is_duplicated for PyArrow when there is more than one column
closes #1947 


<!--
# Thanks for contributing a pull request! 
## Please make sure you see our contribution guidelines: https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md
-->

## What type of PR is this? (check all applicable)

- [ ] 💾 Refactor
- [ ] ✨ Feature
- [ ] 🐛 Bug Fix
- [ ] 🔧 Optimization
- [ ] 📝 Documentation
- [ ] ✅ Test
- [ ] 🐳 Other

## Related issues

- Related issue #\<issue number\>
- Closes #\<issue number\>

## Checklist

- [ ] Code follows style guide (ruff)
- [ ] Tests added
- [ ] Documented the changes

## If you have comments or can explain your changes, please do so below


Patch:
diff --git a/narwhals/_arrow/dataframe.py b/narwhals/_arrow/dataframe.py
index 72c18880..71f15c1b 100644
--- a/narwhals/_arrow/dataframe.py
+++ b/narwhals/_arrow/dataframe.py
@@ -694,6 +694,16 @@ class ArrowDataFrame(CompliantDataFrame, CompliantLazyFrame):
         from narwhals._arrow.series import ArrowSeries
 
         columns = self.columns
+        if (
+            len(columns) > 1
+            and pc.any(
+                [pc.any(pc.is_null(self._native_frame[col])).as_py() for col in columns]
+            ).as_py()
+        ):
+            # blocked on https://github.com/apache/arrow/issues/32204
+            msg = "DataFrame.is_duplicated with more than one column when nulls are present is not yet supported for PyArrow."
+            raise NotImplementedError(msg)
+
         index_token = generate_temporary_column_name(n_bytes=8, columns=columns)
         col_token = generate_temporary_column_name(
             n_bytes=8, columns=[*columns, index_token]
diff --git a/narwhals/utils.py b/narwhals/utils.py
index aa3e00d9..3311a0bb 100644
--- a/narwhals/utils.py
+++ b/narwhals/utils.py
@@ -1076,7 +1076,7 @@ def validate_native_namespace_and_backend(
     if native_namespace is not None and backend is None:  # pragma: no cover
         if emit_deprecation_warning:
             msg = (
-                "`native_namespace` is deprecated, please use `pass_through` instead.\n\n"
+                "`native_namespace` is deprecated, please use `backend` instead.\n\n"
                 "Note: `native_namespace` will remain available in `narwhals.stable.v1`.\n"
                 "See https://narwhals-dev.github.io/narwhals/backcompat/ for more information.\n"
             )


Test patch:
diff --git a/tests/frame/is_duplicated_test.py b/tests/frame/is_duplicated_test.py
index 481fb772..cb0ee4ea 100644
--- a/tests/frame/is_duplicated_test.py
+++ b/tests/frame/is_duplicated_test.py
@@ -1,5 +1,9 @@
 from __future__ import annotations
 
+from contextlib import nullcontext as does_not_raise
+
+import pytest
+
 import narwhals.stable.v1 as nw
 from tests.utils import ConstructorEager
 from tests.utils import assert_equal_data
@@ -12,3 +16,18 @@ def test_is_duplicated(constructor_eager: ConstructorEager) -> None:
     result = nw.concat([df, df.head(1)]).is_duplicated()
     expected = {"is_duplicated": [True, False, False, True]}
     assert_equal_data({"is_duplicated": result}, expected)
+
+
+def test_is_duplicated_with_nulls(constructor_eager: ConstructorEager) -> None:
+    context = (
+        pytest.raises(NotImplementedError)
+        if "pyarrow_table" in str(constructor_eager)
+        else does_not_raise()
+    )
+    data = {"col1": [1, 2, 3], "col2": ["one", None, None]}
+    df_raw = constructor_eager(data)
+    df = nw.from_native(df_raw, eager_only=True)
+    with context:
+        result = df.is_duplicated()
+        expected = {"is_duplicated": [False, False, False]}
+        assert_equal_data({"is_duplicated": result}, expected)