Repository: narwhals-dev/narwhals
Pull request #2826 (2025-07-13 20:21:56)

test: Use pytest-xdist in GHA
## What type of PR is this? (check all applicable)

- [ ] 💾 Refactor
- [ ] ✨ Feature
- [ ] 🐛 Bug Fix
- [ ] 🔧 Optimization
- [ ] 📝 Documentation
- [x] ✅ Test
- [x] 🐳 Other

## Related issues

- Related issue #2818

## Checklist

- [ ] Code follows style guide (ruff)
- [ ] Tests added
- [ ] Documented the changes

## If you have comments or can explain your changes, please do so below

Opening as draft to see what else is affected by pyproject.toml configs.
Also, probably it's better to _not_ have it by default but enabling it in GHA. Thoughts on this?

FBruzzesi: > I'd quite like it with `--numprocesses=logical` if possible? 🙏

Sure, I just wanted to check the runtimes.

What I wonder is if it's ok to have it as opt-in vs opt-out.

Pros for being opt-in are:
- you have full control (can set a env variable for the repo)
- if you need to run a single test you don't get any overhead
- if you need to use a breakpoint in a test, that works without any issue
- opt-out requires to edit a file, while opt-in can easily be done in the terminal
dangotbanned: https://github.com/narwhals-dev/narwhals/pull/2826#issuecomment-3067296675

You can opt-out in quite a few places btw


In `altair`, I have it disabled for the the VSCode `pytest` integration, partially for this point - but also because it has been buggy for me when enabled

> if you need to run a single test you don't get any overhead

```jsonc
# <...>.code-workspace
{
	"settings": {
        "python.testing.pytestArgs": [
			"--numprocesses=1",
         ],
    }
}
```

I think you can do env var and possibly some other files for whatever reason 😂 
FBruzzesi: > No worries if you wanna keep the PR scoped to just CI but ...

I personally would prefer that yes. Adding a `-n logical` in the CLI seems quite straightforward, while deactivating it to quickly jump in a test via an inline breakpoint feels too much work for a quick debugging 

> It seems we have the _briefest-of-mentions_ of `pytest-xdist` in the docs already 😄

Yes this came up during a open source sprint. Honestly I don't remember why we decided to not have it by default. It could be that GHA runners had 2 workers at the time, and the speedup was irrelevant if any, and contributors can always do that locally.

> Maybe if all goes well, we could document/link to the different ways to config?

Sure, I am not sure what other config options it has 😄


Patch:
diff --git a/.github/workflows/extremes.yml b/.github/workflows/extremes.yml
index a836ed00..b3439714 100644
--- a/.github/workflows/extremes.yml
+++ b/.github/workflows/extremes.yml
@@ -5,6 +5,10 @@ on:
   push:
     branches: [main]
 
+env:
+  PY_COLORS: 1
+  PYTEST_ADDOPTS: "--numprocesses=logical"
+  
 jobs:
   minimum_versions:
     strategy:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 27a084ec..556ae808 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -166,6 +166,15 @@ If you add code that should be tested, please add tests.
     which the pandas API just doesn't support, and even then, it's accompanied by a big warning.
   - Don't use inplace methods, unless you're creating a new object and are sure it's safe to modify
     it. In particular, you should never ever modify the user's input data.
+  - Please remember that `assign`, `drop`, `reset_index`, and `rename`, though seemingly harmless, make
+    full copies of the input data.
+
+    - Instead of `assign`, prefer using `with_columns` at the compliant level.
+    - For `drop` and `reset_index`, use `inplace=True`, so long as you're only modifying a new object which
+      you created. Again, you should never modify user input. This may need updating if/when it's
+      deprecated/removed, but please keep it for older pandas versions
+      https://github.com/pandas-dev/pandas/pull/51466/files.
+    - Instead of `rename`, prefer `alias` at the compliant level.
 
 - Polars:
 
diff --git a/docs/overhead.md b/docs/overhead.md
index 5420cec2..b0435a7c 100644
--- a/docs/overhead.md
+++ b/docs/overhead.md
@@ -9,8 +9,6 @@ Sometimes it's even negative, because of how careful we are in Narwhals
 to avoid unnecessary copies and index resets. Here are timings from the
 TPC-H queries, comparing running pandas directly vs running pandas via Narwhals:
 
-![Comparison of pandas vs "pandas via Narwhals" timings on TPC-H queries showing neglibile overhead](https://github.com/user-attachments/assets/bbd6fcaf-5c25-46a6-8c03-9ce42efca787)
-
 [Complete code to reproduce](https://www.kaggle.com/code/marcogorelli/narwhals-vs-pandas-overhead-tpc-h-s2).
 
 ## Plotly's story
diff --git a/narwhals/_duration.py b/narwhals/_duration.py
index d1fe4a6d..8e678be7 100644
--- a/narwhals/_duration.py
+++ b/narwhals/_duration.py
@@ -32,7 +32,7 @@ TimedeltaKwd: TypeAlias = Literal[
 ]
 
 PATTERN_INTERVAL: re.Pattern[str] = re.compile(
-    r"^(?P<multiple>\d+)(?P<unit>ns|us|ms|mo|m|s|h|d|q|y)\Z"
+    r"^(?P<multiple>-?\d+)(?P<unit>ns|us|ms|mo|m|s|h|d|q|y)\Z"
 )
 MONTH_MULTIPLES = frozenset([1, 2, 3, 4, 6, 12])
 QUARTER_MULTIPLES = frozenset([1, 2, 4])
diff --git a/narwhals/_ibis/expr_dt.py b/narwhals/_ibis/expr_dt.py
index 35cc9602..57ec3e97 100644
--- a/narwhals/_ibis/expr_dt.py
+++ b/narwhals/_ibis/expr_dt.py
@@ -5,7 +5,11 @@ from typing import TYPE_CHECKING, Any, Callable
 from narwhals._compliant import LazyExprNamespace
 from narwhals._compliant.any_namespace import DateTimeNamespace
 from narwhals._duration import Interval
-from narwhals._ibis.utils import UNITS_DICT_BUCKET, UNITS_DICT_TRUNCATE
+from narwhals._ibis.utils import (
+    UNITS_DICT_BUCKET,
+    UNITS_DICT_TRUNCATE,
+    timedelta_to_ibis_interval,
+)
 from narwhals._utils import not_implemented
 
 if TYPE_CHECKING:
@@ -87,7 +91,7 @@ class IbisExprDateTimeNamespace(
         if unit in {"y", "q", "mo", "d", "ns"}:
             msg = f"Offsetting by {unit} is not yet supported for ibis."
             raise NotImplementedError(msg)
-        offset = interval.to_timedelta()
+        offset = timedelta_to_ibis_interval(interval.to_timedelta())
         return self.compliant._with_callable(lambda expr: expr.add(offset))
 
     def replace_time_zone(self, time_zone: str | None) -> IbisExpr:
diff --git a/narwhals/_ibis/utils.py b/narwhals/_ibis/utils.py
index 5809ecdd..aabab260 100644
--- a/narwhals/_ibis/utils.py
+++ b/narwhals/_ibis/utils.py
@@ -10,6 +10,7 @@ from narwhals._utils import isinstance_or_issubclass
 
 if TYPE_CHECKING:
     from collections.abc import Mapping
+    from datetime import timedelta
 
     import ibis.expr.types as ir
     from ibis.common.temporal import TimestampUnit
@@ -233,3 +234,7 @@ def narwhals_to_native_dtype(  # noqa: C901, PLR0912
         raise NotImplementedError(msg)
     msg = f"Unknown dtype: {dtype}"  # pragma: no cover
     raise AssertionError(msg)
+
+
+def timedelta_to_ibis_interval(td: timedelta) -> ibis.expr.types.temporal.IntervalScalar:
+    return ibis.interval(days=td.days, seconds=td.seconds, microseconds=td.microseconds)
diff --git a/narwhals/_pandas_like/dataframe.py b/narwhals/_pandas_like/dataframe.py
index 9cad81a3..e60b41a7 100644
--- a/narwhals/_pandas_like/dataframe.py
+++ b/narwhals/_pandas_like/dataframe.py
@@ -557,7 +557,10 @@ class PandasLikeDataFrame(
             for left_key, right_key in zip(left_on, right_on)
             if right_key != left_key
         ]
-        return result_native.drop(columns=extra)
+        # NOTE: Keep `inplace=True` to avoid making a redundant copy.
+        # This may need updating, depending on https://github.com/pandas-dev/pandas/pull/51466/files
+        result_native.drop(columns=extra, inplace=True)  # noqa: PD002
+        return result_native
 
     def _join_full(
         self, other: Self, *, left_on: Sequence[str], right_on: Sequence[str], suffix: str
@@ -584,17 +587,17 @@ class PandasLikeDataFrame(
             key_token = generate_temporary_column_name(
                 n_bytes=8, columns=(*self.columns, *other.columns)
             )
-            return (
-                self.native.assign(**{key_token: 0})
-                .merge(
-                    other.native.assign(**{key_token: 0}),
-                    how="inner",
-                    left_on=key_token,
-                    right_on=key_token,
-                    suffixes=("", suffix),
-                )
-                .drop(columns=key_token)
+            result_native = self.native.assign(**{key_token: 0}).merge(
+                other.native.assign(**{key_token: 0}),
+                how="inner",
+                left_on=key_token,
+                right_on=key_token,
+                suffixes=("", suffix),
             )
+            # NOTE: Keep `inplace=True` to avoid making a redundant copy.
+            # This may need updating, depending on https://github.com/pandas-dev/pandas/pull/51466/files
+            result_native.drop(columns=key_token, inplace=True)  # noqa: PD002
+            return result_native
         return self.native.merge(other.native, how="cross", suffixes=("", suffix))
 
     def _join_semi(
@@ -628,18 +631,18 @@ class PandasLikeDataFrame(
             columns_to_select=list(right_on),
             columns_mapping=dict(zip(right_on, left_on)),
         )
-        return (
-            self.native.merge(
-                other_native,
-                # TODO(FBruzzesi): See https://github.com/modin-project/modin/issues/7384
-                how="left" if implementation.is_pandas() else "outer",
-                indicator=indicator_token,
-                left_on=left_on,
-                right_on=left_on,
-            )
-            .loc[lambda t: t[indicator_token] == "left_only"]
-            .drop(columns=indicator_token)
-        )
+        result_native = self.native.merge(
+            other_native,
+            # TODO(FBruzzesi): See https://github.com/modin-project/modin/issues/7384
+            how="left" if implementation.is_pandas() else "outer",
+            indicator=indicator_token,
+            left_on=left_on,
+            right_on=left_on,
+        ).loc[lambda t: t[indicator_token] == "left_only"]
+        # NOTE: Keep `inplace=True` to avoid making a redundant copy.
+        # This may need updating, depending on https://github.com/pandas-dev/pandas/pull/51466/files
+        result_native.drop(columns=indicator_token, inplace=True)  # noqa: PD002
+        return result_native
 
     def _join_filter_rename(
         self, other: Self, columns_to_select: list[str], columns_mapping: dict[str, str]
diff --git a/pyproject.toml b/pyproject.toml
index 0f98e37c..5a1fcd52 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -54,8 +54,9 @@ tests = [
   "covdefaults",
   "pytest",
   "pytest-cov",
-  "pytest-randomly",
   "pytest-env",
+  "pytest-randomly",
+  "pytest-xdist",
   "hypothesis",
 ]
 extra = [  # heavier dependencies we don't necessarily need in every testing job


Test patch:
diff --git a/.github/workflows/pytest-pyspark.yml b/.github/workflows/pytest-pyspark.yml
index 563a8266..a5002393 100644
--- a/.github/workflows/pytest-pyspark.yml
+++ b/.github/workflows/pytest-pyspark.yml
@@ -6,6 +6,7 @@ on:
 
 env:
   PY_COLORS: 1
+  PYTEST_ADDOPTS: "--numprocesses=logical"
 
 jobs:
 
diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml
index 2ae018ba..8b73a0e5 100644
--- a/.github/workflows/pytest.yml
+++ b/.github/workflows/pytest.yml
@@ -7,6 +7,7 @@ on:
 
 env:
   PY_COLORS: 1
+  PYTEST_ADDOPTS: "--numprocesses=logical"
 
 jobs:
   pytest-39:
diff --git a/.github/workflows/random_ci_pytest.yml b/.github/workflows/random_ci_pytest.yml
index 825ad898..8e6c9c3d 100644
--- a/.github/workflows/random_ci_pytest.yml
+++ b/.github/workflows/random_ci_pytest.yml
@@ -5,6 +5,7 @@ on:
 
 env:
   PY_COLORS: 1
+  PYTEST_ADDOPTS: "--numprocesses=logical"
 
 jobs:
   tox:
diff --git a/tests/expr_and_series/dt/offset_by_test.py b/tests/expr_and_series/dt/offset_by_test.py
index 5d1c6f7b..3707ff2c 100644
--- a/tests/expr_and_series/dt/offset_by_test.py
+++ b/tests/expr_and_series/dt/offset_by_test.py
@@ -95,6 +95,41 @@ data_dst = {"a": [datetime(2020, 10, 25, tzinfo=timezone.utc)]}
                 datetime(2023, 1, 2, 2, 4, 14, 715123),
             ],
         ),
+        (
+            "-2us",
+            [
+                datetime(2021, 3, 1, 12, 34, 56, 49010),
+                datetime(2020, 1, 2, 2, 4, 14, 715121),
+            ],
+        ),
+        (
+            "-2000ns",
+            [
+                datetime(2021, 3, 1, 12, 34, 56, 49010),
+                datetime(2020, 1, 2, 2, 4, 14, 715121),
+            ],
+        ),
+        (
+            "-7h",
+            [
+                datetime(2021, 3, 1, 5, 34, 56, 49012),
+                datetime(2020, 1, 1, 19, 4, 14, 715123),
+            ],
+        ),
+        (
+            "-13d",
+            [
+                datetime(2021, 2, 16, 12, 34, 56, 49012),
+                datetime(2019, 12, 20, 2, 4, 14, 715123),
+            ],
+        ),
+        (
+            "-3y",
+            [
+                datetime(2018, 3, 1, 12, 34, 56, 49012),
+                datetime(2017, 1, 2, 2, 4, 14, 715123),
+            ],
+        ),
     ],
 )
 def test_offset_by(