Question: How would you fill_null in a DataFrame by sampling the rest of the column for a unique value each time a null is occurred? Example: Suppose I have this DataFrame: >>> daf=pl.DataFrame({"a":[1,2, None, None, 5, 6, None, None, None, 10], "b":[None, "two", "three", None, "five", "six", "seven", None, "nine", "ten"]}) >>> daf shape: (10, 2) ┌──────┬───────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞══════╪═══════╡ │ 1 ┆ null │ │ 2 ┆ two │ │ null ┆ three │ │ null ┆ null │ │ 5 ┆ five │ │ 6 ┆ six │ │ null ┆ seven │ │ null ┆ null │ │ null ┆ nine │ │ 10 ┆ ten │ └──────┴───────┘  I'd like to fill the nulls in all columns by sampling the rest of the column. For example, the nulls in "a" should get a random value from [1,2,5,6,10], and the nulls in "b" should get a random value from ["two", "three", "five", "six", "seven", "nine", "ten"]. But there shouldn't be str in "a" or i64 in "b". And the value used to fill the null shouldn't be the same for each null in a column. I was able to do it by looping over the columns using a standard Python for loop: >>> for c in daf.columns: ... repl=daf[c].drop_nulls().sample(daf[c].len(), with_replacement=True) ... daf=daf.with_columns(pl.col(c).fill_null(repl)) ... >>> daf shape: (10, 2) ┌─────┬───────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═══════╡ │ 1 ┆ two │ │ 2 ┆ two │ │ 1 ┆ three │ │ 2 ┆ six │ │ 5 ┆ five │ │ 6 ┆ six │ │ 5 ┆ seven │ │ 10 ┆ two │ │ 5 ┆ nine │ │ 10 ┆ ten │ └─────┴───────┘  but I was trying to figure out how to do it using expressions, and couldn't get it. Is there a way to do this in a more compact way, using the Polars expression syntax?

Answer: Solution This seems to work. It's based on your original idea, it just does the same operations to each column purely with expressions. daf.select( pl.col("*").fill_null( pl.col("*").drop_nulls().sample(pl.len(), with_replacement=True) ) )  ┌─────┬───────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═══════╡ │ 1 ┆ two │ │ 2 ┆ two │ │ 6 ┆ three │ │ 10 ┆ six │ │ 5 ┆ five │ │ 6 ┆ six │ │ 1 ┆ seven │ │ 2 ┆ two │ │ 2 ┆ nine │ │ 10 ┆ ten │ └─────┴───────┘  Explanation pl.col("*") (or pl.all()) just selects each column individually and applies the subsequent operation to it. Later on it retains the same structure, because for example this works: >>> df = pl.DataFrame({"a": [1, 2, 3], "b": [100, 200, 300]}) >>> df.select(pl.col("*").mul(pl.col("*").mean())) shape: (3, 2) ┌─────┬─────────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞═════╪═════════╡ │ 2.0 ┆ 20000.0 │ │ 4.0 ┆ 40000.0 │ │ 6.0 ┆ 60000.0 │ └─────┴─────────┘  If I use a LazyFrame and explain, this is what it does under the hood: >>> daf.select(pl.col("*").fill_null(pl.col("*").drop_nulls().sample(pl.len(), with_replacement=True))).explain() SELECT [col("a").fill_null([col("a").drop_nulls().sample([len()])]), col("b").fill_null([col("b").drop_nulls().sample([len()])])] FROM DF ["a", "b"]; PROJECT 2/2 COLUMNS; SELECTION: "None"  What was a bit surprising to me that the fill_null part actually works. The documentation for fill_null is not very clear on the fact that you can pass an expression to it that returns something else than a single value and it works this way. Also one would think that using 1 or 100 instead of pl.len() would result to the same, but no. 1 returns the same sample for each null replacement, and value any other than 1 or pl.len() gives a shape error. It seems to do a table-sized or operation and replace every null value with the value from the corresponding spot from the other side (which can still be null), which is kind of interesting. >>> daf.select(pl.col("*").fill_null(pl.col("*").reverse())) shape: (10, 2) ┌──────┬───────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞══════╪═══════╡ │ 1 ┆ ten │ │ 2 ┆ two │ │ null ┆ three │ │ null ┆ seven │ │ 5 ┆ five │ │ 6 ┆ six │ │ null ┆ seven │ │ null ┆ three │ │ 2 ┆ nine │ │ 10 ┆ ten │ └──────┴───────┘