Package: narwhals
NarwhalsExtremely lightweight compatibility layer between Polars, pandas, cuDF, and Modin.Seamlessly support all four, without depending on any of them!✅Just usea subset ofthe Polars API, no need to learn anything new✅No dependencies(not even Polars), keep your library lightweight✅ SeparateLazyand Eager APIs✅ Use PolarsExpressionsNote: this is work-in-progress, and a bit of an experiment, don't take it too seriously.Installationpip install narwhalsOr just vendor it, it's only a bunch of pure-Python files.UsageThere are three steps to writing dataframe-agnostic code using Narwhals:usenarwhals.to_polars_apito wrap a pandas, Polars, cuDF, or Modin dataframe
in the Polars APIuse the subset of the Polars API defined inhttps://github.com/MarcoGorelli/narwhals/blob/main/narwhals/spec/__init__.py.useDataFrame.to_nativeto return an object to the user in their original
dataframe flavour. For example:if you started with pandas, you'll get pandas backif you started with Polars, you'll get Polars backif you started with Modin, you'll get Modin backif you started with cuDF, you'll get cuDF back (and computation will happen natively on the GPU!)ExampleHere's an example of a dataframe agnostic function:fromtypingimportTypeVarimportpandasaspdimportpolarsasplfromnarwhalsimporttranslate_frameAnyDataFrame=TypeVar("AnyDataFrame")defmy_agnostic_function(suppliers_native:AnyDataFrame,parts_native:AnyDataFrame,)->AnyDataFrame:suppliers,pl=translate_frame(suppliers_native)parts,_=translate_frame(parts_native)result=(suppliers.join(parts,left_on="city",right_on="city").filter(pl.col("color").is_in(["Red","Green"]),pl.col("weight")>14,).group_by("s","p").agg(weight_mean=pl.col("weight").mean(),weight_max=pl.col("weight").max(),))returnresult.to_native()You can pass in a pandas, Polars, cuDF, or Modin dataframe, the output will be the same!
Let's try it out:suppliers={"s":["S1","S2","S3","S4","S5"],"sname":["Smith","Jones","Blake","Clark","Adams"],"status":[20,10,30,20,30],"city":["London","Paris","Paris","London","Athens"],}parts={"p":["P1","P2","P3","P4","P5","P6"],"pname":["Nut","Bolt","Screw","Screw","Cam","Cog"],"color":["Red","Green","Blue","Red","Blue","Red"],"weight":[12.0,17.0,17.0,14.0,12.0,19.0],"city":["London","Paris","Oslo","London","Paris","London"],}print("pandas output:")print(my_agnostic_function(pd.DataFrame(suppliers),pd.DataFrame(parts),))print("\nPolars output:")print(my_agnostic_function(pl.DataFrame(suppliers),pl.DataFrame(parts),))print("\nPolars lazy output:")print(my_agnostic_function(pl.LazyFrame(suppliers),pl.LazyFrame(parts),).collect())pandas output:
    s   p  weight_mean
0  S1  P6         19.0
1  S2  P2         17.0
2  S3  P2         17.0
3  S4  P6         19.0

Polars output:
shape: (4, 3)
┌─────┬─────┬─────────────┐
│ s   ┆ p   ┆ weight_mean │
│ --- ┆ --- ┆ ---         │
│ str ┆ str ┆ f64         │
╞═════╪═════╪═════════════╡
│ S1  ┆ P6  ┆ 19.0        │
│ S3  ┆ P2  ┆ 17.0        │
│ S4  ┆ P6  ┆ 19.0        │
│ S2  ┆ P2  ┆ 17.0        │
└─────┴─────┴─────────────┘

Polars lazy output:
shape: (4, 3)
┌─────┬─────┬─────────────┐
│ s   ┆ p   ┆ weight_mean │
│ --- ┆ --- ┆ ---         │
│ str ┆ str ┆ f64         │
╞═════╪═════╪═════════════╡
│ S1  ┆ P6  ┆ 19.0        │
│ S3  ┆ P2  ┆ 17.0        │
│ S4  ┆ P6  ┆ 19.0        │
│ S2  ┆ P2  ┆ 17.0        │
└─────┴─────┴─────────────┘Magic! 🪄ScopeDo you maintain a dataframe-consuming library?Is there a Polars function which you'd like Narwhals to have, which would make your job easier?If, I'd love to hear from you!Note: this isnota "Dataframe Standard" project. It just translates a subset of the Polars
API to pandas-like libraries.Why "Narwhals"?Because they are so awesome.