Subset Observations - rows

Reshaping Data – Change layout,sorting, renaming

Polars Cheat Sheet
Open in ColabGeneral
Install

Summarize Data
Count number of rows with each unique value ofvariable
df["groups"].value_counts()

Filter: Extract rows that meet logical criteria.
df.f lter(pl.col("random") > 0.5)
df.f lter((pl.col("groups")"B")& (pl.col("random") > 0.5))

# of rows in DataFrame

Append rows of DataFramespl.concat([df, df2])len(df)# ordf.height

Tuple of # of rows, # of columns in DataFrame

pip install polars

Sampledf.shape

Import# Randomly select fraction of rows.
df.sample(frac=0.5)# of distinct values in a column

import polars as plCreating/reading DataFrames
Create DataFrame

df["groups"].n_unique()# Randomly select n rows.
df.sample(n=2)

Select rst and last rows
# Select f rst n rows
df.head(n=2)

Append columns of DataFramespl.concat([df, df3], how="horizontal")

Basic descriptive and statistics for each column
df.describe()# Select last n rows.
df.tail(n=2)

Aggregation functions
df = pl.DataFrame({"nrs": [1, 2, 3, None, 5],"names": ["foo", "ham", "spam", "egg", None],"random": [0.3, 0.7, 0.1, 0.9, 0.6],"groups": ["A", "A", "B", "C", "B"],})

Read CSV
df = pl.read_csv("https: j.mp/iriscsv",
has_header=True)

Read parquet

Subset Variables - columns
Gather columns into rows

Select multiple columns with speci c names

df.select([# Sum values
pl.sum("random").alias("sum"),

df.melt(
id_vars="nrs",
value_vars=["names", "groups"])# Minimum value
pl.min("random").alias("min"),# Maximum value
pl.max("random").alias("max"),# orpl.col("random").max().alias("other_max"),

df.select(["nrs", "names"])

Select columns whose name matches regex
df.select(pl.col("^n.*$"))

Subsets - rows and columns# Standard deviation
pl.std("random").alias("std dev"),

Spread rows into columns# Variancepl.var("random").alias("variance"),

df.pivot(values="nrs", index="groups",
columns="names")

df = pl.read_parquet("path.parquet",columns=["select", "columns"]# Medianpl.median("random").alias("median"),Order rows by values of a column

Expressions
Polars expressions can be performed in sequence.
This improves readability of code.
df \.f lter(pl.col("nrs") < 4) \.groupby("groups") \.agg(pl \.all() \.sum())

Download at polars.diebold.io

Select rows 2-4
df[2:4,]

Select columns in positions 1 and 3 ( rst column is0)df[ , [1, 3]]

Select rows meeting logical condition, and only thespeci c columns
df[df["random"] > 0.5, ["names", "groups"]]# low to highdf.sort("random")# Meanpl.mean("random").alias("mean"),# high to low
df.sort("random", reverse=True)

df.rename({"nrs": "idx"})# Quantile
pl.quantile("random", 0.75) \.alias("quantile_0.75"),# orpl.col("random").quantile(0.75) \.alias("other_quantile_0.75"),

Drop columns from DataFrame# First valuepl.f rst("random").alias("f rst"),Rename the columns of a DataFramedf.drop(["names", "random"])])

By Franz Diebold (diebold.io), inspired by Pandas Cheat SheetGroup DataHandling Missing Data

Rolling Functions

Window Functions
Window functions allow to group by several
columns simultaneously

Drop rows with any column having a null value
Group by values in column named "col", returning aGroupBy object

df.drop_nulls()

df.groupby("groups")

All of the aggregation functions from above can beapplied to a group as well
df.groupby(by="groups").agg([# Sum values
pl.sum("random").alias("sum"),

The following rolling functions are available
df.select([# Rolling maximum value
pl.col("random") \.rolling_max(window_size=2) \.alias("rolling_max"),

Replace null values with given value# Rolling mean value
pl.col("random") \.rolling_mean(window_size=2) \.alias("rolling_mean"),

df.f ll_null(42)# Rolling median value
pl.col("random") \.rolling_median(window_size=2, min_periods=2) \.alias("rolling_median"),# Minimum value
pl.min("random").alias("min"),# Maximum value
pl.max("random").alias("max"),# orpl.col("random").max().alias("other_max"),# Standard deviation
pl.std("random").alias("std_dev"),# Variancepl.var("random").alias("variance"),# Medianpl.median("random").alias("median"),Replace null values using forward strategy
df.f ll_null(strategy="forward")# Rolling minimum value
pl.col("random") \.rolling_min(window_size=2) \.alias("rolling_min"),

Other ll strategies are "backward", "min", "max","mean", "zero" and "one"
Replace oating point NaN values with given value# Rolling standard deviation
pl.col("random") \.rolling_std(window_size=2) \.alias("rolling_std"),

df.f ll_nan(42)Make New Columns# Rolling sum values
pl.col("random") \.rolling_sum(window_size=2) \.alias("rolling_sum"),# Meanpl.mean("random").alias("mean"),# Quantile
pl.quantile("random", 0.75) \.alias("quantile_0.75"),# orpl.col("random").quantile(0.75) \.alias("other_quantile_0.75"),# First valuepl.f rst("random").alias("f rst"),])

Additional GroupBy functions
df.groupby(by="groups").agg([# Count the number of values in each grouppl.count("random").alias("size"),# Sample one element in each grouppl.col("names").apply(lambda group_df: group_df.sample(1)),

Download at polars.diebold.io

Combine Data Sets

Inner Join
Retains only rows with a match in the other set.
df.join(df4, on="nrs")# ordf.join(df4, on="nrs", how="inner")

Left Join
Retains each row from "left" set (df).
df.join(df4, on="nrs", how="left")# Rolling variancepl.col("random") \.rolling_var(window_size=2) \.alias("rolling_var"),

Add a new columns to the DataFrame
df.with_column((pl.col("random") * pl.col("nrs")) \.alias("product"))# Rolling quantile
pl.col("random") \.rolling_quantile(quantile=0.75, window_size=2,min_periods=2) \.alias("rolling_quantile"),Add several new columns to the DataFrame
df.with_columns([(pl.col("random") * pl.col("nrs")) \.alias("product"),
pl.col("names").str.lengths() \.alias("names_lengths"),])# Rolling skew
pl.col("random") \.rolling_skew(window_size=2) \.alias("rolling_skew"),# Rolling custom function
pl.col("random") \.rolling_apply(function=np.nanstd, window_size=2) \.alias("rolling_apply"),

Add a column at index 0 that counts the rows
df.with_row_count()])

df.select(["names","groups",pl.col("random").sum().over("names") \.alias("sum_by_names"),pl.col("random").sum().over("groups") \.alias("sum_by_groups"),])

Outer Join
Retains each row, even if no other matching row exists.
df.join(df4, on="nrs", how="outer")

Anti Join
Contains all rows from df that do not have a match in df4.
df.join(df4, on="nrs", how="anti")])

By Franz Diebold (diebold.io), inspired by Pandas Cheat Sheet