eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following dataframe:  Sample ID 'Deinococcus soli' Cha et al. 2014 16SrX (Apple proliferation group) 16SrXII (Stolbur group) 0 C1day1_barcode01 21 1 0 1 C1day21_barcode19 22 0 0 2 C3day1_barcode03 13 0 0 3 C3day14_barcode15 14 2 2 4 T1day21_barcode22 19 1 1  This is my desired output:  Sample ID C1day1_barcode01 C1day21_barcode19 C3day1_barcode03 C3day14_barcode15 T1day21_barcode22 0 'Deinococcus soli' Cha et al. 2014 21 22 13 14 19 1 16SrX (Apple proliferation group) 1 0 0 2 1 2 16SrXII (Stolbur group) 0 0 0 2 1  I have used transpose and I tried inserting a new index & renaming the columns as well as resetting the index but in either cases I get something like this: 0 Sample ID 0 1 2 3 4 0 Sample ID C1day1_barcode01 C1day21_barcode19 C3day1_barcode03 C3day14_barcode15 T1day21_barcode22 1 'Deinococcus soli' Cha et al. 2014 21 22 13 14 19 2 16SrX (Apple proliferation group) 1 0 0 2 1 3 16SrXII (Stolbur group) 0 0 0 2 1  or Sample ID index C1day1_barcode01 C1day21_barcode19 C3day1_barcode03 C3day14_barcode15 T1day21_barcode22 0 'Deinococcus soli' Cha et al. 2014 21 22 13 14 19 1 16SrX (Apple proliferation group) 1 0 0 2 1 2 16SrXII (Stolbur group) 0 0 0 2 1  Though I tried renaming index to Sample ID I'm unable to remove the "Sample ID" title of the index even by setting index.name = None. What do I do?

Answer: set_index on the "Sample ID" column to set it aside, transpose to reshape, rename_axis to exchange the axis names, and reset_index to move back the IDs as column: out = (df.set_index('Sample ID').T .rename_axis(index='Sample ID', columns=None) .reset_index() )  Alternative: col = 'Sample ID' out = (df.set_index(col).T .rename_axis(index=col, columns=None) .reset_index() )  Output:  Sample ID C1day1_barcode01 C1day21_barcode19 C3day1_barcode03 C3day14_barcode15 T1day21_barcode22 0 'Deinococcus soli' Cha et al. 2014 21 22 13 14 19 1 16SrX (Apple proliferation group) 1 0 0 2 1 2 16SrXII (Stolbur group) 0 0 0 2 1
Question: I am making an Android app in Python using briefcase from BeeWare that must start a service. And I have this code... This is the relevant code from file MainActivity.java: package org.beeware.android; import com.chaquo.python.Kwarg; import com.chaquo.python.PyException; import com.chaquo.python.PyObject; import com.chaquo.python.Python; import com.chaquo.python.android.AndroidPlatform; public class MainActivity extends AppCompatActivity { public static MainActivity singletonThis; protected void onCreate(Bundle savedInstanceState) { singletonThis = this; ... start Python } public void startMyService() { Intent intent = new Intent(this, MyService.class); startService(intent); }  And this is the relevant code from app.py that my intuition came up with: from chaquopy import Java class Application(toga.App): ...UI code here def start_tcp_service(self, widget): msg = 'START pressed !' print(msg); self.LogMessage(msg) self.CallJavaMethod('startMyService') def CallJavaMethod(self, method_name): MainActClass = Java.org.beeware.android.MainActivity MainActivity = MainActClass.singletonThis method = getattr(MainActivity, method_name) method()  Now, when I try to run the project with briefcase run android -u on my Android phone, through the USB debugging bridge, I get the error:  E/AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.myapp/org.beeware.android.MainActivity}: com.chaquo.python.PyException: ModuleNotFoundError: No module named 'chaquopy'  It seems that there isn't any module with name chaquopy. I tried to install it with pip, but it is not found. But then, how can I access the MainActivity methods from Python? What is the correct module to include? I found here some documentation that says "The java module provides facilities to use Java classes and objects from Python code.". I tried to import java bun this is not found either... It seems that on this page it tells how to access Java from Python, but I don’t understand all that is there, because this is my first interaction with Java and Android...

Answer: I found it! It goes like this... from org.beeware.android import MainActivity class Application(toga.App): # ...UI code here def start_tcp_service(self, widget): msg = 'START pressed!' print(msg); self.LogMessage(msg) self.CallJavaMethod('startMyService') def CallJavaMethod(self, method_name): MainActInst = MainActivity.singletonThis method = getattr(MainActInst, method_name) method()
Question: I have a large polars dataframe that I need to unpivot. This dataframe contains lots of null values (at least half). I want to drop the nulls while unpivoting the dataframe. I already tried to unpivot the dataframe first and then filter it with drop_nulls() or similar approaches. However, this is too memory-intensive (on a machine with about 1TB RAM). Is there any way in which I can filter the dataset already during the process of unpivot? Any help is appreciated! Sample data. # in reality, this dataset has about 160k rows and columns # (square matrix), and is about 100GB df = { "A": [None, 2, 3], "B": [None, None, 2], "C": [None, None, None], "names": ["A", "B", "C"] } df = pl.DataFrame(df) df.unpivot(index = "names", variable_name = "names_2", value_name = "distance")  Output. shape: (9, 3) ┌───────┬─────────┬──────────┐ │ names ┆ names_2 ┆ distance │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞═══════╪═════════╪══════════╡ │ A ┆ A ┆ null │ │ B ┆ A ┆ null │ │ C ┆ A ┆ null │ │ A ┆ B ┆ 2 │ │ B ┆ B ┆ null │ │ C ┆ B ┆ null │ │ A ┆ C ┆ 3 │ │ B ┆ C ┆ 2 │ │ C ┆ C ┆ null │ └───────┴─────────┴──────────┘  This could then be filtered (e.g. using df = df.drop_nulls()), but I would like to get this desired result directly from the unpivot. Expected output. shape: (3, 3) ┌───────┬─────────┬──────────┐ │ names ┆ names_2 ┆ distance │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞═══════╪═════════╪══════════╡ │ A ┆ B ┆ 2 │ │ A ┆ C ┆ 3 │ │ B ┆ C ┆ 2 │ └───────┴─────────┴──────────┘ 

Answer: Very likely you can make the operation more efficient in runtime and memory consumption by using pl.LazyFrames and polars' streaming engine. By using pl.LazyFrames, the melt / unpivot and filter / drop_nulls won't be operations eagerly, but first aggregated into a query plan. When collecting the lazy DataFrame (i.e. materialising it into a pl.DataFrame), the query plan can be optimised, taking into account subsequent operations. Streaming will enable the processing to not done all-at-once, but executed in batches, ensuring that the processed batches don't grow larger-than-memory. ( df # convert to pl.LazyFrame .lazy() # create query plan .unpivot( index="names", variable_name="names_2", value_name="distance" ) .drop_nulls() # collect pl.LazyFrame while using streaming engine .collect(streaming=True) )  Note. Tentative tests on my machine give large improvements in runtime and memory consumptions.
Question: Is there a way to do something like this using FastAPI: @app.post("/instance/new", tags=["instance"]) async def MyFunction(condition): if condition: response = {"key": "value"} return response else: return some_big_plain_text  The way it is coded now, the JSON is returned fine, but some_big_plain_text is not human friendly. If I do: @app.post("/instance/new", tags=["instance"], PlainTextResponse) I get an error when returning a JSON response.

Answer: It is just needed to add a FastAPI Response: from fastapi import FastAPI, Response @app.post("/instance/new", tags=["instance"]) async def MyFunction(condition): if condition: response = {"key": "value"} return response else: return Response(content=some_big_plain_text, media_type="text/plain")
Question: I'm trying to create an enditor where the Footer contains the usual bindings on the left and some status information on the right, for example the line number. The Footer in textual is very simple so I thought to extend it, but I'm unable to see both my label and the binding of the base Footer. This is my code: class MyFooter(Footer): DEFAULT_CSS = """ MyFooter { .right-label { text-align: right; } } """ def compose(self) -> ComposeResult: for widget in super().compose(): yield widget yield Label("This is the right side label", id="right-label")  To test it, you can use the first example of the tutorial: from textual.app import App, ComposeResult from textual.widgets import Header, Footer,Label class MyFooter(Footer): DEFAULT_CSS = """ MyFooter { .right-label { text-align: right; } } """ def compose(self) -> ComposeResult: """Create child widgets for the footer.""" for widget in super().compose(): yield widget yield Label("This is the right side label", id="right-label") class StopwatchApp(App): """A Textual app to manage stopwatches.""" BINDINGS = [("d", "toggle_dark", "Toggle dark mode")] def compose(self) -> ComposeResult: """Create child widgets for the app.""" yield Header() yield MyFooter() def action_toggle_dark(self) -> None: """An action to toggle dark mode.""" self.dark = not self.dark if __name__ == "__main__": app = StopwatchApp() app.run() 

Answer: I'd recommend solving this by laying out multiple widgets, instead of overriding the Footer class. The Footer widget uses dock: bottom; layout: grid; grid-columns: auto, which makes this a little tricky. But you can wrap the Footer in a fixed-sized container, and lay out your label next to that. from textual.app import App, ComposeResult from textual.widgets import Header, Footer, Label from textual.containers import Horizontal class StopwatchApp(App): """A Textual app to manage stopwatches.""" BINDINGS = [("d", "toggle_dark", "Toggle dark mode")] CSS = """ Horizontal#footer-outer { height: 1; dock: bottom; } Horizonal#footer-inner { width: 75%; } Label#right-label { width: 25%; text-align: right; } """ def compose(self) -> ComposeResult: """Create child widgets for the app.""" yield Header() with Horizontal(id="footer-outer"): with Horizontal(id="footer-inner"): yield Footer() yield Label("This is the right side label", id="right-label") def action_toggle_dark(self) -> None: """An action to toggle dark mode.""" self.dark = not self.dark if __name__ == "__main__": app = StopwatchApp() app.run()
Question: I used to run a collection of Fortran 95 subroutines from Python by compiling it via f2py. In the Fortran source I have a module with my global variables:  MODULE GEOPLOT_GLOBALS IMPLICIT NONE INTEGER, PARAMETER :: N_MAX = 16 INTEGER, PARAMETER :: I_MAX = 18 INTEGER, PARAMETER :: J_MAX = 72 ... END MODULE GEOPLOT_GLOBALS  The compiled file has the name "geoplot.cpython-312-darwin.so" and is in a subfolder named "geo". When using f2py in numpy 1.26, I could do this: import geo.geoplot as geo maxN = geo.geoplot_globals.n_max maxI = geo.geoplot_globals.i_max maxJ = geo.geoplot_globals.j_max  Now, with numpy 2.0.1, I do the same but get the error message AttributeError: module 'geo.geoplot' has no attribute 'geoplot_globals'  Which can be confirmed by listing the __dict__ attribute or using the getmembers module: They all list the Fortran subroutines and modules which contain source code, except for the geoplot_globals module which contains only variable declarations. So my question is: How am I supposed to access global Fortran variables from Python when using numpy 2.0? And please do not suggest to write all to a file in Fortran only to read it in Python. There should be a more direct way.

Answer: This must be a bug in f2py. See here: https://github.com/numpy/numpy/issues/27167 What got me unstuck is an ugly workaround: I simply added some useless dummy code to the module, like that: MODULE GEOPLOT_GLOBALS USE MOD_TYPES IMPLICIT NONE INTEGER, PARAMETER :: N_MAX = 16 INTEGER, PARAMETER :: I_MAX = 18 INTEGER, PARAMETER :: J_MAX = 72 ... CONTAINS SUBROUTINE DUMMY (UNSINN) INTEGER :: UNSINN OPEN(UNIT=29, FILE="FOR29.txt", STATUS = 'UNKNOWN') WRITE(29,"(I8)") UNSINN CLOSE (29) END SUBROUTINE DUMMY END MODULE GEOPLOT_GLOBALS  Now the before missing module is carried over into Python and can be accessed in the usual way. >>> print(geo.geoplot_globals.__doc__) n_max : 'i'-scalar i_max : 'i'-scalar j_max : 'i'-scalar ...
Question: This is an extension to this accepted answer. My DataFrame: import pandas as pd df = pd.DataFrame( { 'a': [-3, -1, -2, -5, 10, -3, -13, -3, -2, 1, 2, -100], 'b': [1, 2, 3, 4, 5, 10, 80, 90, 100, 99, 1, 12] } )  Expected output:  a b 5 -3 10 6 -13 80 7 -3 90 8 -2 100  Logic: a) Selecting the longest streak of negatives in a. b) If for example there are two streaks that has same size, I want the one that has a greater sum of b. In df there are two streaks with size of 4 but I want the second one because sum of b is greater. My Attempt: import numpy as np s = np.sign(df['a']) df['g'] = s.ne(s.shift()).cumsum() df['size'] = df.groupby('g')['g'].transform('size') df['b_sum'] = df.groupby('g')['b'].transform('sum')  Edit 1: I have provided an extra df to clarify the point. I want the negative streaks under any circumstance. In this df the positive streak is longer and its b is greater but I still want the last two rows which is the longest negative streak: df = pd.DataFrame( { 'a': [-0.65, 11, 18, 1, -2, -3], 'b': [1, 20, 30000, 4322, 300, 3] } ) #output 4 -2.00 300 5 -3.00 3  This is my attempt to get this output but if there are no negative rows in a dataframe then it throws an error: df['sign'] = np.sign(df.a) df['g'] = df.sign.ne(df.sign.shift()).cumsum() df = df.loc[df.a.lt(0)] out = df[df.g.eq(df.groupby('g')['b'].agg(['size', 'sum']) .query('size == size.max()')['sum'].idxmax())] 

Answer: You can keep the same logic, just add one extra filtering step (e.g. with query) to get all max sizes, before getting the idxmax of sum of "b": # negative numbers m = df['a'].lt(0) # form groups g = m.ne(m.shift()).cumsum() out = df[g.eq(df[m] .groupby(g)['b'].agg(['size', 'sum']) .query('size == size.max()') ['sum'].idxmax())]  Output:  a b 5 -3 10 6 -13 80 7 -3 90 8 -2 100  Intermediate: df.groupby(g)['b'].agg(['size', 'sum']) size sum a 1 4 10 2 1 5 3 4 280 4 2 100 5 1 12  Or, using your approach (note that this wouldn't guarantee a unique group if two or more have the max length and the same sum of b): s = np.sign(df['a']) g = df.groupby(s.ne(s.shift()).cumsum()) s1 = g['a'].transform('size') s2 = g['b'].transform('sum') out = df[s1.eq(s1.max()) & s2.eq(s2.max())]
Question: I'm looking to unit test all my AWS Lambda code in my project using pytest. Due to how I have to configure the directory structure to work with infrastructure as code tooling, each Lambda sits within it's own CloudFormation stack, I've got a pretty non-standard directory structure. I'm unable to get pytest to run all tests across all my Lambda functions - ideally I'd like this to work by just running 'pytest' in the root directory of the project. The directory structure is as follows (it's worth noting that changing the structure is not an option): - Project_Directory - stack1 - product.template.yaml - src - lambda1 - lambda_function.py - requirements.txt - tests - __init__.py - test_functions - __init__.py - test_lambda1.py - stack2 - product.template.yaml - src - lambda2 - lambda_function.py - requirements.txt - tests - __init__.py - test_functions - __init__.py - test_lambda2.py - conftest.py - pytest.ini  Each test_lambda.py file imports from the lambda_function.py file as follows: from src.lambdax.lambda_function import func1, func2 When only a single stack is in the project directory pytest has no issue picking up the tests. However when a second stack is added pytest fails with the following error: ModuleNotFoundError: No module named 'tests.test_functions.test_lambda2' Also, when running pytest directly against each stack directory the tests run with no issue. That is running pytest stack1 and pytest stack2. Expectation: Running 'pytest' from project_directory yields all tests for all lambdas. I've tried adding testpaths = stack1 stack2 and testpaths = stack1/tests stack2/tests to pytest.ini to no success. What am I missing here, I'm guessing maybe some module namespace collision but I'm not sure how to resolve it! Any advice on this issue is much appreciated! Edit: This definitely looks to be a tests module namespace collision. After modifying the tests directory in stack2 to be called tests2 all tests run as expected. I'm still keen for advice here, I'd really like to avoid enforcing each stack to have a different name for the tests directory!

Answer: Managed to solve this after painstakingly going through different approachs. The solution is a combination of using the new(ish) importlib import mode and having some custom sys.path manipulation in conftest.py. Configure pytest.ini as this: [pytest] addopts = --import-mode=importlib  Add the following code into conftest.py: def add_path(directory): src_path = os.path.abspath(directory) if src_path not in sys.path: sys.path.insert(0, src_path) add_path(os.path.join(os.path.dirname(__file__), "stack1")) add_path(os.path.join(os.path.dirname(__file__), "stack2"))  The above conftest.py code could definitely be written a bit nicer using PathLib. I'll leave that as an exercise to the reader! Another nice consequence to using the importlib import mode is that the __init__.py files are no longer required in the tests and test_functions directories in each stack!
Question: In a Python BeeWare project, I want to use toga.Canvas to draw some horizontal rectangles, but I don't know from where to get the Canvas width. I can't find any documentation for the toga.Canvas() dimensions on the internet...  def redraw_canvas(self): x = 4; y = 4; for i in range(7): with self.canvas.context.Fill(color=self.clRowBkg) as fill: fill.rect(x, y, 100, self.row_height) y += self.row_height + 4 

Answer: There doesn't appear to be any way to get this information from the public API, but there are some non-public properties used in the Toga Canvas example:  canvas.layout.content_width canvas.layout.content_height  These should work for any Toga widget, not just Canvas.
Question: I have a polars dataframe that contains a datetime column. I want to convert this column to strings in the format %Y%m. For example, all dates in January 2024 should be converted to "202401". from datetime import datetime import polars as pl data = { "ID" : [1,2,3], "dates" : [datetime(2024,1,2),datetime(2024,1,3),datetime(2024,1,4)], } df = pl.DataFrame(data)  I have tried using strftime. However, the following AttributeError is raised. AttributeError: 'Expr' object has no attribute 'strftime' 

Answer: Note that pl.Expr.dt.strftime is available under the pl.Expr.dt namespace. Hence, it is called on the dt attribute of an expression and not the expression directly. df.with_columns( pl.col("dates").dt.strftime("%Y%m") )  shape: (3, 2) ┌─────┬────────┐ │ ID ┆ dates │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪════════╡ │ 1 ┆ 202401 │ │ 2 ┆ 202401 │ │ 3 ┆ 202401 │ └─────┴────────┘
Question: Consider a simple question using Scipy.optimize: Maximize(xy) s.t x^2+y^2=200. The right code is this : import numpy as np from scipy.optimize import minimize def objective(var_tmp): x, y = var_tmp return -x * y def constraint(var_tmp): x, y = var_tmp return 200 - (x ** 2 + y ** 2) initial_guess = [1, 1] constraints = {'type': 'ineq', 'fun': constraint} result = minimize(objective, initial_guess, constraints=constraints) optimal_x, optimal_y = result.x optimal_value = -result.fun print(f"Optimal x: {optimal_x}") print(f"Optimal y: {optimal_y}") print(f"Maximum xy value: {optimal_value}")  Which gives the right answer 100. However, if the constraint is written as follows: def constraint(var_tmp): x, y = var_tmp if x ** 2 + y ** 2 <= 200: return 1 return -1  It will give infinity as an answer. Why is the case?

Answer: By default, SciPy uses SLSQP to minimize a problem which has constraints. (Several other minimizers have support for constraints; see the "Constrained Minimization" section of the minimize() documentation.) SLSQP requires that its constraints be differentiable. Here is a passage from the SLSQP paper showing this. In this context, f is the function you are minimizing, and g is your equality and inequality constraints.  ...where the problem functions f : R^n -> R^1 and g : R^n -> R^m are assumed to be continuously differentiable and have no specific structure  Source: Kraft D (1988), A software package for sequential quadratic programming. Tech. Rep. DFVLR-FB 88-28, DLR German Aerospace Center — Institute for Flight Mechanics, Koln, Germany. page 8, section 2.1.1 (I mentioned before that SciPy has multiple minimize methods which can handle constraints. However, I checked, and none of them seem to be able to handle non-differentiable constraints.)
Question: I have a dataframe with empty values in some rows like this: ID Date Price Curr A Jan 21 (10,0) USD B Aug 8 (10,0) USD C Sep 29 (10,0) USD settle Aug 24 ( ,)  where the last row has 2 empty values in Price and Curr columns. How can I either replace the empty values with nan so I can dropna() or drop the rows that contain empty values to get a dataframe like: ID Date Price Curr A Jan 21 (10,0) USD B Aug 8 (10,0) USD C Sep 29 (10,0) USD  sample: data = { "ID": ["A", "B", "C", "settle"], "Date": ["Jan 21", "Aug 8", "Sep 29", "Aug 24"], "Price": [(10,0), (10,0), (10,0), ()], "Curr": ["USD", "USD", "USD", ""] } df = pd.DataFrame(data) 

Answer: To drop the rows that are empty, you can try something like this: import pandas as pd data = { "ID": ["A", "B", "C", "settle"], "Date": ["Jan 21", "Aug 8", "Sep 29", "Aug 24"], "Price": [(10, 0), (10, 0), (10, 0), ()], "Curr": ["USD", "USD", "USD", ""] } df = pd.DataFrame(data) # Filter out rows where Price is an empty tuple df = df[df['Price'].apply(lambda x: len(x) > 0)] # Convert the 'Price' tuples to floats by taking the first element of each tuple df['Price'] = df['Price'].apply(lambda x: float(x[0])) print(df) print(df.dtypes) # Output ID Date Price Curr 0 A Jan 21 10.0 USD 1 B Aug 8 10.0 USD 2 C Sep 29 10.0 USD ID object Date object Price float64 Curr object dtype: object
Question: how can i select last item of list in paths column after applying the str.split("/") function? dataNpaths = pl.scan_csv("test_data/file*.csv", has_header=True, include_file_paths = "paths").collect() dataNpaths.with_columns(pl.col("paths").str.split("/").alias("paths"))  >>> dataNpaths.with_columns(pl.col("paths").str.split("/").alias("paths")) shape: (30, 5) ┌──────────┬──────────┬──────────┬──────────┬────────────────────────────┐ │ Column1 ┆ Column2 ┆ Column3 ┆ Column4 ┆ paths │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 ┆ f64 ┆ list[str] │ ╞══════════╪══════════╪══════════╪══════════╪════════════════════════════╡ │ 0.603847 ┆ 0.509877 ┆ 0.091579 ┆ 0.43821 ┆ ["test_data", "file1.csv"] │ │ 0.572299 ┆ 0.817647 ┆ 0.087951 ┆ 0.397217 ┆ ["test_data", "file1.csv"] │ │ 0.886123 ┆ 0.159805 ┆ 0.766246 ┆ 0.083915 ┆ ["test_data", "file1.csv"] │ │ 0.142208 ┆ 0.413847 ┆ 0.043408 ┆ 0.147779 ┆ ["test_data", "file1.csv"] │ │ 0.105215 ┆ 0.924754 ┆ 0.309823 ┆ 0.724407 ┆ ["test_data", "file1.csv"] │ │ … ┆ … ┆ … ┆ … ┆ … │ │ 0.381675 ┆ 0.849887 ┆ 0.498281 ┆ 0.733085 ┆ ["test_data", "file3.csv"] │ │ 0.697427 ┆ 0.950464 ┆ 0.999596 ┆ 0.645253 ┆ ["test_data", "file3.csv"] │ │ 0.49979 ┆ 0.172414 ┆ 0.679287 ┆ 0.091804 ┆ ["test_data", "file3.csv"] │ │ 0.668585 ┆ 0.640259 ┆ 0.932463 ┆ 0.579558 ┆ ["test_data", "file3.csv"] │ │ 0.077462 ┆ 0.802565 ┆ 0.966791 ┆ 0.29297 ┆ ["test_data", "file3.csv"] │  but neither of these approaches worked dataNpaths.with_columns(pl.col("paths").str.split("/")[-1].alias("paths")) dataNpaths.with_columns(pl.col("paths").str.split("/",-1).alias("paths")) 

Answer: You should the List accessor .list which is similar to .str: dataNpaths.with_columns(pl.col("paths").str.split("/").list[-1])  .alias('paths') is superfluous as you are using the old column name not creating a new column. Alternatively as the last element of the List is needed, the same result can be obtained using a .list method: dataNpaths.with_columns(pl.col("paths").str.split("/").list.last())  Details of the comprehensive .list methods are given in LINK. These include .slice, .contains, .head, .tail etc.