Repository: narwhals-dev/narwhals (python)

Issue:
Mark `pipe` as supported in api-completeness
This page makes it look like we don't support `pipe` for any backend: https://narwhals-dev.github.io/narwhals/api-completeness/dataframe/

However, we do 😎 That's because we implement it directly in `narwhals/dataframe.py` - can we special-case this one so it shows as completed for all backends?

file to look at: https://github.com/narwhals-dev/narwhals/blob/main/utils/generate_backend_completeness.py

contributing guide: https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md

Pull request:
docs: Add DataFrame.pipe to API completeness table
<!--
# Thanks for contributing a pull request! 
## Please make sure you see our contribution guidelines: https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md
-->

## What type of PR is this? (check all applicable)

- [ ] 💾 Refactor
- [ ] ✨ Feature
- [ ] 🐛 Bug Fix
- [ ] 🔧 Optimization
- [x] 📝 Documentation
- [ ] ✅ Test
- [ ] 🐳 Other

## Related issues 
 
- Closes #1006 

## Checklist

- [X] Code follows style guide (ruff)
- [N/A] Tests added 
- [X] Documented the changes

## If you have comments or can explain your changes, please do so below.



Patch:
diff --git a/.github/workflows/mkdocs.yml b/.github/workflows/mkdocs.yml
index 37b18841..354dafd4 100644
--- a/.github/workflows/mkdocs.yml
+++ b/.github/workflows/mkdocs.yml
@@ -27,6 +27,6 @@ jobs:
           path: .cache
           restore-keys: |
             mkdocs-material-
-      - run: pip install -r docs/requirements-docs.txt -e . pandas polars
+      - run: pip install -r docs/requirements-docs.txt -e . pandas polars pyarrow
 
       - run: mkdocs gh-deploy --force
diff --git a/.gitignore b/.gitignore
index 3825a68a..8b9adeb8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -32,5 +32,8 @@ tpch/data/*
 # VSCode
 .vscode/
 
+# IntelliJ IDEA / PyCharm
+.idea/
+
 # MacOS
 .DS_Store
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index f2ed79c6..25051474 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -91,7 +91,8 @@ If you add code that should be tested, please add tests.
 ### 6. Running tests
 
 To run tests, run `pytest`. To check coverage: `pytest --cov=narwhals`.
-To run tests on the docset-module, use `pytest narwhals --doctest-modules`.
+To run tests on the doctests, use `pytest narwhals --doctest-modules`.
+To run unit tests and doctests at the same time, run `pytest tests narwhals --cov=narwhals --doctest-modules`.
 
 If you want to have less surprises when opening a PR, you can take advantage of [nox](https://nox.thea.codes/en/stable/index.html) to run the entire CI/CD test suite locally in your operating system.
 
diff --git a/docs/api-reference/series.md b/docs/api-reference/series.md
index c016b566..9868e7b9 100644
--- a/docs/api-reference/series.md
+++ b/docs/api-reference/series.md
@@ -57,6 +57,7 @@
         - to_list
         - to_numpy
         - to_pandas
+        - to_native
         - unique
         - value_counts
         - zip_with
diff --git a/docs/basics/dataframe.md b/docs/basics/dataframe.md
index 3cd2c884..41be8aeb 100644
--- a/docs/basics/dataframe.md
+++ b/docs/basics/dataframe.md
@@ -4,7 +4,7 @@ To write a dataframe-agnostic function, the steps you'll want to follow are:
 
 1. Initialise a Narwhals DataFrame or LazyFrame by passing your dataframe to `nw.from_native`.
     All the calculations stay lazy if we start with a lazy dataframe - Narwhals will never automatically trigger computation without you asking it to.
-   
+
     Note: if you need eager execution, make sure to pass `eager_only=True` to `nw.from_native`.
 
 2. Express your logic using the subset of the Polars API supported by Narwhals.
@@ -21,6 +21,7 @@ Just like in Polars, we can pass expressions to
 `DataFrame.select` or `LazyFrame.select`.
 
 Make a Python file with the following content:
+
 ```python exec="1" source="above" session="df_ex1"
 import narwhals as nw
 from narwhals.typing import FrameT
@@ -34,6 +35,7 @@ def func(df: FrameT) -> FrameT:
         a_std=nw.col("a").std(),
     )
 ```
+
 Let's try it out:
 
 === "pandas"
@@ -60,7 +62,16 @@ Let's try it out:
     print(func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="df_ex1"
+    import pyarrow as pa
+
+    table = pa.table({"a": [1, 1, 2]})
+    print(func(table))
+    ```
+
 Alternatively, we could have opted for the more explicit version:
+
 ```python
 import narwhals as nw
 from narwhals.typing import IntoFrameT
@@ -75,6 +86,7 @@ def func(df_native: IntoFrameT) -> IntoFrameT:
     )
     return nw.to_native(df)
 ```
+
 Despite being more verbose, it has the advantage of preserving the type annotation of the native
 object - see [typing](../api-reference/typing.md) for more details.
 
@@ -84,6 +96,7 @@ In general, in this tutorial, we'll use the former.
 
 Just like in Polars, we can pass expressions to `GroupBy.agg`.
 Make a Python file with the following content:
+
 ```python exec="1" source="above" session="df_ex2"
 import narwhals as nw
 from narwhals.typing import FrameT
@@ -93,6 +106,7 @@ from narwhals.typing import FrameT
 def func(df: FrameT) -> FrameT:
     return df.group_by("a").agg(nw.col("b").mean()).sort("a")
 ```
+
 Let's try it out:
 
 === "pandas"
@@ -119,12 +133,21 @@ Let's try it out:
     print(func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="df_ex2"
+    import pyarrow as pa
+
+    table = pa.table({"a": [1, 1, 2], "b": [4, 5, 6]})
+    print(func(table))
+    ```
+
 ## Example 3: horizontal sum
 
 Expressions can be free-standing functions which accept other expressions as inputs.
 For example, we can compute a horizontal sum using `nw.sum_horizontal`.
 
 Make a Python file with the following content:
+
 ```python exec="1" source="above" session="df_ex3"
 import narwhals as nw
 from narwhals.typing import FrameT
@@ -134,6 +157,7 @@ from narwhals.typing import FrameT
 def func(df: FrameT) -> FrameT:
     return df.with_columns(a_plus_b=nw.sum_horizontal("a", "b"))
 ```
+
 Let's try it out:
 
 === "pandas"
@@ -160,6 +184,14 @@ Let's try it out:
     print(func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="df_ex3"
+    import pyarrow as pa
+
+    table = pa.table({"a": [1, 1, 2], "b": [4, 5, 6]})
+    print(func(table))
+    ```
+
 ## Example 4: multiple inputs
 
 `nw.narwhalify` can be used to decorate functions that take multiple inputs as well and
@@ -169,6 +201,7 @@ For example, let's compute how many rows are left in a dataframe after filtering
 on a series.
 
 Make a Python file with the following content:
+
 ```python exec="1" source="above" session="df_ex4"
 from typing import Any
 
@@ -201,3 +234,12 @@ Let's try it out:
     s = pl.Series([1, 3])
     print(func(df, s.to_numpy(), "a"))
     ```
+
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="df_ex4"
+    import pyarrow as pa
+
+    table = pa.table({"a": [1, 1, 2, 2, 3], "b": [4, 5, 6, 7, 8]})
+    a = pa.array([1, 3])
+    print(func(table, a.to_numpy(), "a"))
+    ```
diff --git a/docs/basics/column.md b/docs/basics/series.md
similarity index 84%
rename from docs/basics/column.md
rename to docs/basics/series.md
index 62951cb4..f5d8dee1 100644
--- a/docs/basics/column.md
+++ b/docs/basics/series.md
@@ -47,6 +47,14 @@ def my_func(df: FrameT) -> FrameT:
     print(my_func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="ex1"
+    import pyarrow as pa
+
+    table = pa.table({"a": [-1, 1, 3], "b": [3, 5, -3]})
+    print(my_func(table))
+    ```
+
 ## Example 2: multiply a column's values by a constant
 
 Let's write a dataframe-agnostic function which multiplies the values in column
@@ -86,6 +94,14 @@ def my_func(df: FrameT) -> FrameT:
     print(my_func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="ex2"
+    import pyarrow as pa
+
+    table = pa.table({"a": [-1, 1, 3], "b": [3, 5, -3]})
+    print(my_func(table))
+    ```
+
 Note that column `'a'` was overwritten. If we had wanted to add a new column called `'c'` containing column `'a'`'s
 values multiplied by 2, we could have used `Expr.alias`:
 
@@ -123,6 +139,14 @@ def my_func(df: FrameT) -> FrameT:
     print(my_func(df).collect())
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="ex2.1"
+    import pyarrow as pa
+
+    table = pa.table({"a": [-1, 1, 3], "b": [3, 5, -3]})
+    print(my_func(table))
+    ```
+
 ## Example 3: finding the mean of a column as a scalar
 
 Now, we want to find the mean of column `'a'`, and we need it as a Python scalar.
@@ -156,5 +180,13 @@ def my_func(df: nw.DataFrame) -> float | None:
     print(my_func(df))
     ```
 
+=== "PyArrow"
+    ```python exec="true" source="material-block" result="python" session="ex2"
+    import pyarrow as pa
+
+    table = pa.table({"a": [-1, 1, 3], "b": [3, 5, -3]})
+    print(my_func(table))
+    ```
+
 Note that, even though the output of our function is not a dataframe nor a series, we can
 still use `narwhalify`.
diff --git a/docs/extending.md b/docs/extending.md
index 13b93387..f6829ba3 100644
--- a/docs/extending.md
+++ b/docs/extending.md
@@ -4,11 +4,13 @@
 
 Currently, Narwhals supports the following libraries as inputs:
 
-- pandas
-- Polars
-- cuDF
-- Modin
-- PyArrow
+| Library  | 🔗 Link 🔗 |
+| ------------- | ------------- |
+| ️Polars 🐻‍❄️ | [github.com/pola-rs/polars](https://github.com/pola-rs/polars) |
+| pandas 🐼 |  [github.com/pandas-dev/pandas](https://github.com/pandas-dev/pandas) |
+| cuDF | [github.com/rapidsai/cudf](https://github.com/rapidsai/cudf) |
+| Modin | [github.com/modin-project/modin](https://github.com/modin-project/modin) |
+| PyArrow ⇶ | [arrow.apache.org/docs/python](https://arrow.apache.org/docs/python/index.html) |
 
 If you want your own library to be recognised too, you're welcome open a PR (with tests)!
 Alternatively, if you can't do that (for example, if you library is closed-source), see
diff --git a/docs/installation.md b/docs/installation.md
index ac7d2341..2f801254 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -28,6 +28,7 @@ they are not required dependencies - Narwhals only ever uses what the user passe
 
 - [pandas](https://pandas.pydata.org/docs/getting_started/install.html)
 - [Polars](https://pola-rs.github.io/polars/user-guide/installation/)
+- [PyArrow](https://arrow.apache.org/docs/python/install.html)
 
 ### Simple example
 
@@ -38,6 +39,7 @@ from __future__ import annotations
 
 import pandas as pd
 import polars as pl
+import pyarrow as pa
 import narwhals as nw
 from narwhals.typing import IntoFrame
 
@@ -48,13 +50,19 @@ def my_function(df_native: IntoFrame) -> list[str]:
     return column_names
 
 
-df_pandas = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
-df_polars = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
+data = {"a": [1, 2, 3], "b": [4, 5, 6]}
+df_pandas = pd.DataFrame(data)
+df_polars = pl.DataFrame(data)
+table_pa = pa.table(data)
 
 print("pandas output")
 print(my_function(df_pandas))
+
 print("Polars output")
 print(my_function(df_polars))
+
+print("PyArrow output")
+print(my_function(table_pa))
 ```
 
 If you run `python t.py` then your output should look like the above. This is the simplest possible example of a dataframe-agnostic
diff --git a/docs/roadmap_and_related.md b/docs/roadmap_and_related.md
index 5e464683..43580db6 100644
--- a/docs/roadmap_and_related.md
+++ b/docs/roadmap_and_related.md
@@ -23,3 +23,9 @@ this includes Ibis and Vaex. See [extending](extending.md) for details.
 ### Array API
 
 Array counterpart to the DataFrame API, see [here](https://data-apis.org/array-api/2022.12/index.html).
+
+### PyCapsule Interface
+
+Allows C extension modules to safely share pointers to C data structures with Python code and other C modules, encapsulating the pointer with a name and optional destructor to manage resources and ensure safe access, see [here](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) for details. 
+
+Narwhals supports exporting a DataFrame via the Arrow PyCapsule Interface.
diff --git a/mkdocs.yml b/mkdocs.yml
index b0aaf106..045a6a67 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -8,7 +8,7 @@ nav:
   - Installation and quick start: installation.md
   - Tutorial:
     - basics/dataframe.md
-    - basics/column.md
+    - basics/series.md
     - basics/complete_example.md
   - Pandas-like concepts:
     - other/pandas_index.md
diff --git a/narwhals/dataframe.py b/narwhals/dataframe.py
index 16b869ff..21c9c80b 100644
--- a/narwhals/dataframe.py
+++ b/narwhals/dataframe.py
@@ -395,14 +395,16 @@ class DataFrame(BaseFrame[FrameT]):
         Convert this DataFrame to a pandas DataFrame.
 
         Examples:
-            Construct pandas and Polars DataFrames:
+            Construct pandas, Polars (eager) and PyArrow DataFrames:
 
             >>> import pandas as pd
             >>> import polars as pl
+            >>> import pyarrow as pa
             >>> import narwhals as nw
             >>> df = {"foo": [1, 2, 3], "bar": [6.0, 7.0, 8.0], "ham": ["a", "b", "c"]}
             >>> df_pd = pd.DataFrame(df)
             >>> df_pl = pl.DataFrame(df)
+            >>> df_pa = pa.table(df)
 
             We define a library agnostic function:
 
@@ -410,7 +412,7 @@ class DataFrame(BaseFrame[FrameT]):
             ... def func(df):
             ...     return df.to_pandas()
 
-            We can then pass either pandas or Polars to `func`:
+            We can then pass any supported library such as pandas, Polars (eager), or PyArrow to `func`:
 
             >>> func(df_pd)
                foo  bar ham
@@ -422,6 +424,13 @@ class DataFrame(BaseFrame[FrameT]):
             0    1  6.0   a
             1    2  7.0   b
             2    3  8.0   c
+            >>> func(df_pa)
+               foo  bar ham
+            0    1  6.0   a
+            1    2  7.0   b
+            2    3  8.0   c
+
+
         """
         return self._compliant_frame.to_pandas()
 
@@ -434,10 +443,12 @@ class DataFrame(BaseFrame[FrameT]):
 
             >>> import pandas as pd
             >>> import polars as pl
+            >>> import pyarrow as pa
             >>> import narwhals as nw
             >>> df = {"foo": [1, 2, 3], "bar": [6.0, 7.0, 8.0], "ham": ["a", "b", "c"]}
             >>> df_pd = pd.DataFrame(df)
             >>> df_pl = pl.DataFrame(df)
+            >>> df_pa = pa.table(df)
 
             We define a library agnostic function:
 
@@ -445,12 +456,14 @@ class DataFrame(BaseFrame[FrameT]):
             ...     df = nw.from_native(df)
             ...     return df.write_csv()
 
-            We can then pass either pandas or Polars to `func`:
+            We can pass any supported library such as pandas, Polars or PyArrow to `func`:
 
             >>> func(df_pd)  # doctest: +SKIP
             'foo,bar,ham\n1,6.0,a\n2,7.0,b\n3,8.0,c\n'
             >>> func(df_pl)  # doctest: +SKIP
             'foo,bar,ham\n1,6.0,a\n2,7.0,b\n3,8.0,c\n'
+            >>> func(df_pa)  # doctest: +SKIP
+            'foo,bar,ham\n1,6.0,a\n2,7.0,b\n3,8.0,c\n'
 
             If we had passed a file name to `write_csv`, it would have been
             written to that file.
diff --git a/narwhals/series.py b/narwhals/series.py
index 9d21058f..f34217c9 100644
--- a/narwhals/series.py
+++ b/narwhals/series.py
@@ -22,7 +22,7 @@ class Series:
     """
     Narwhals Series, backed by a native series.
 
-    The native dataframe might be pandas.Series, polars.Series, ...
+    The native series might be pandas.Series, polars.Series, ...
 
     This class is not meant to be instantiated directly - instead, use
     `narwhals.from_native`, making sure to pass `allow_series=True` or
@@ -85,6 +85,45 @@ class Series:
         ca = pa.chunked_array([self.to_arrow()])
         return ca.__arrow_c_stream__(requested_schema=requested_schema)
 
+    def to_native(self) -> Any:
+        """
+        Convert Narwhals series to native series.
+
+        Returns:
+            Series of class that user started with.
+
+        Examples:
+            >>> import pandas as pd
+            >>> import polars as pl
+            >>> import narwhals as nw
+            >>> s = [1, 2, 3]
+            >>> s_pd = pd.Series(s)
+            >>> s_pl = pl.Series(s)
+
+            We define a library agnostic function:
+
+            >>> @nw.narwhalify
+            ... def func(s):
+            ...     return s.to_native()
+
+            We can then pass either pandas or Polars to `func`:
+
+            >>> func(s_pd)
+            0    1
+            1    2
+            2    3
+            dtype: int64
+            >>> func(s_pl)  # doctest: +NORMALIZE_WHITESPACE
+            shape: (3,)
+            Series: '' [i64]
+            [
+                1
+                2
+                3
+            ]
+        """
+        return self._compliant_series._native_series
+
     def scatter(self, indices: int | Sequence[int], values: Any) -> Self:
         """
         Set value(s) at given position(s).
@@ -224,7 +263,7 @@ class Series:
             + "─" * length
             + "┐\n"
             + f"|{header}|\n"
-            + "| Use `narwhals.to_native()` to see native output |\n"
+            + "| Use `series.to_native()` to see native output |\n"
             + "└"
             + "─" * length
             + "┘"
diff --git a/narwhals/stable/v1.py b/narwhals/stable/v1.py
index 1f103aae..1cd73125 100644
--- a/narwhals/stable/v1.py
+++ b/narwhals/stable/v1.py
@@ -342,7 +342,7 @@ class Series(NwSeries):
     """
     Narwhals Series, backed by a native series.
 
-    The native dataframe might be pandas.Series, polars.Series, ...
+    The native series might be pandas.Series, polars.Series, ...
 
     This class is not meant to be instantiated directly - instead, use
     `narwhals.from_native`, making sure to pass `allow_series=True` or
diff --git a/pyproject.toml b/pyproject.toml
index 0196124f..f2109e11 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -99,6 +99,7 @@ force-single-line = true
 docstring-code-format = true
 
 [tool.pytest.ini_options]
+testpaths = ["tests"]
 filterwarnings = [
   "error",
   'ignore:distutils Version classes are deprecated:DeprecationWarning',
diff --git a/utils/check_api_reference.py b/utils/check_api_reference.py
index 1bf1f086..0f95dfed 100644
--- a/utils/check_api_reference.py
+++ b/utils/check_api_reference.py
@@ -27,6 +27,7 @@ SERIES_ONLY_METHODS = {
     "zip_with",
     "item",
     "scatter",
+    "to_native",
 }
 
 # TODO(Unassigned): make dtypes reference page as well
diff --git a/utils/generate_backend_completeness.py b/utils/generate_backend_completeness.py
index b2cb9df2..1d6ee417 100644
--- a/utils/generate_backend_completeness.py
+++ b/utils/generate_backend_completeness.py
@@ -38,6 +38,8 @@ BACKENDS = [
 
 EXCLUDE_CLASSES = {"BaseFrame", "Then", "When"}
 
+DIRECTLY_IMPLEMENTED_METHODS = ["pipe"]
+
 
 def get_class_methods(kls: type[Any]) -> list[str]:
     return [m[0] for m in inspect.getmembers(kls) if not m[0].startswith("_")]
@@ -50,7 +52,9 @@ def parse_module(module_name: str, backend: str, nw_class_name: str) -> list[str
             module_,
             predicate=lambda c: inspect.isclass(c) and c.__name__.endswith(nw_class_name),
         )
-        methods_ = get_class_methods(class_[0][1]) if class_ else []
+        methods_ = (
+            get_class_methods(class_[0][1]) if class_ else []
+        ) + DIRECTLY_IMPLEMENTED_METHODS
 
     except ModuleNotFoundError:
         methods_ = []


Test patch:
diff --git a/tests/series_only/to_native_test.py b/tests/series_only/to_native_test.py
new file mode 100644
index 00000000..269348ea
--- /dev/null
+++ b/tests/series_only/to_native_test.py
@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+from typing import Any
+
+import narwhals.stable.v1 as nw
+
+data = [4, 4, 4, 1, 6, 6, 4, 4, 1, 1]
+
+
+def test_to_native(constructor_eager: Any) -> None:
+    orig_series = constructor_eager({"a": data})["a"]
+    nw_series = nw.from_native(constructor_eager({"a": data}), eager_only=True)["a"]
+    result = nw_series.to_native()
+    assert isinstance(result, orig_series.__class__)