eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I noticed that it's possible to use df.rename(columns=str.lower), but not df.rename(columns=str.replace(" ", "_")).  Is this because it is allowed to use the variable which stores the method (str.lower), but it's not allowed to actually call the method (str.lower())? There is a similar question, why the error message of df.rename(columns=str.replace(" ", "_")) is rather confusing – without an answer on that.  Is it possible to use methods of the .str accessor (of pd.DataFrame().columns) inside of df.rename(columns=...)? The only solution I came up so far is df = df.rename(columns=dict(zip(df.columns, df.columns.str.replace(" ", "_"))))  but maybe there is something more consistent and similar to style of df.rename(columns=str.lower)? I know df.rename(columns=lambda x: x.replace(" ", "_") works, but it doesn't use the .str accessor of pandas columns, it uses the str.replace() of the standard library. The purpose of the question is explore the possibilities to use pandas str methods when renaming columns in method chaining, that's why df.columns = df.columns.str.replace(' ', '_') is not suitable to me.   As an df example, assume: df = pd.DataFrame([[0,1,2]], columns=["a pie", "an egg", "a nut"]) 

Answer: df.rename accepts a function object (or other callable). In the first case, str.lower is a function. However, str.replace(" ", "_") calls the function and evaluate to the result, although, in this case, the call is not correct so it raises an error. But you don't want to pass the result of calling the function, you want to pass the function. So something like def space_to_dash(col): return col.replace(" ", "_") df.rename(columns=space_to_dash)  Or, use a lambda expression: df.rename(columns=lambda col: col.replace(" ", "_"))  Note, df.rename(columns=str.lower) doesn't use the .str accessor either, it uses the built-in str method. So I think you are confused. Now, you can use the .str accessor on the column index object, so: df.columns.str.replace(" ", "_")  But then you would need to do what you already said you didn't want to do: df.columns = df.columns.str.replace(" ", "_")  It is important to point out, this mutates the original dataframe object in place as opposed to df.rename, which returns a new dataframe object. It isn't clear why you want to use the .str accessor, is that the reason?
Question: My problem next: I can't execute query, where need to get list of Accounts with their Records by account_id as sum of Records. My query next: subq = ( select( func.sum(Record.amount).label('amount') ) .filter(Record.account_id == Account.uuid) .subquery() ) accounts = ( await self.session.execute( select(Account) .join(subq, Account.uuid == Record.account_id) .filter(Account.creator_id == uuid) # uuid = it's User ID (Creator) ) ).scalars().all()  Also, Record table looks like this: [ { "type": "income", "amount": 2000, "uuid": "cf876a3d-3395-4f5f-82b5-b496a66e107c" }, { "type": "income", "amount": 5000, "uuid": "fe25274d-111f-410c-a18c-04c73cbcc9db" }, { "type": "expense", "amount": 3000, "uuid": "7a151849-fb8e-47dc-96a0-50fef12233e8" }, { "type": "expense", "amount": 750, "uuid": "dd90988a-cd53-4125-8017-2e2ed05ec48c" } ]  Desired result is: [ { "uuid": "38528eff-61d8-4210-8f9b-7739414dff4a", "name": "newAccount", "creator_id": "15fvh85j-012f-41bh-9km5-e22c8t78e321", "is_private": false, "amount": 25000 }, { "uuid": "86256mdjv-40m7-96v3-4de1-1e946b356gs8", "name": "personalAccount", "creator_id": "867becf0-c86a-4551-af81- "is_private": false, "amount": 53500 } ]  But I got only the same without amount. What I do wrong?

Answer: I found the answer that solved my problem, this using join and mappings().all():  accounts = ( await self.session.execute( select( Account.uuid, Account.name, Account.creator_id, Account.is_private, func.coalesce(func.sum(Record.amount), 0).label('amount') ) .filter(Account.creator_id == uuid) .join(Record, Record.account_id == Account.uuid, isouter=True) .group_by(Account.uuid) ) ).mappings().all()
Question: I want to make the Python fastapi work with MSYS2. I work from the MSYS2 MinGW x64 shell. I have the following installed using the commands: pacman -S mingw-w64-x86_64-python pacman -S mingw-w64-x86_64-python-pip pacman -S mingw-w64-x86_64-python-fastapi  and I also ran pip install fastapi  which gave the output imelf@FLORI-LENOVO-93 MINGW64 /c/Users/imelf/Documents/NachhilfeInfoUni/Kadala/Pybind11 $ pip install fastapi Requirement already satisfied: fastapi in c:/software/msys64/mingw64/lib/python3.11/site-packages (0.109.0) Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from fastapi) (2.5.3) Requirement already satisfied: starlette<0.36.0,>=0.35.0 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from fastapi) (0.35.0) Requirement already satisfied: typing-extensions>=4.8.0 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from fastapi) (4.9.0) Requirement already satisfied: annotated-types>=0.4.0 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi) (0.6.0) Requirement already satisfied: pydantic-core==2.14.6 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from pydantic!=1.8,!=1.88.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi) (2.14.6) Requirement already satisfied: anyio<5,>=3.4.0 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from starlette<0.36.0,>=0.35.0->fastapi) (4.2.0) Requirement already satisfied: idna>=2.8 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from anyio<5,>=3.4.0->starlette<0.36.0,>=0.35.0->fastapi) (3.6) Requirement already satisfied: sniffio>=1.1 in c:/software/msys64/mingw64/lib/python3.11/site-packages (from anyio<5,>=3.4.0->starlette<0.36.0,>=0.35.0->fastapi) (1.3.0)  Now I have this file test.py with the following content: from fastapi import FastAPI meine_coole_rest_api = FastAPI() @meine_coole_rest_api.get("/") async def wurzel_pfad(): return {"coole_nachricht" : "Fast API works"}  Trying to launch it with as described in this tutorial fastapi dev test.py  gives the error message imelf@FLORI-LENOVO-93 MINGW64 /c/Users/imelf/Documents/NachhilfeInfoUni/Kadala/Pybind11 $ fastapi dev test.py bash: fastapi: command not found  Why does he not recognize the command? How do I install fastapi correctly?

Answer: fastapi doesn't ship with the CLI anymore by default, you can install it using pip install fastapi-cli
Question: I have a string that I want to do regex substitutions on: string = 'ptn, ptn; ptn + ptn'  And a list of strings: array = ['ptn_sub1', 'ptn_sub2', '2', '2']  I want to replace each appearance of the regex pattern 'ptn' with a corresponding item from array. Desired result: 'ptn_sub1, ptn_sub2; 2 + 2' I tried using re.finditer to iterate through the matches and substitute each time, but this caused the new string to get mangled since the length of the string changes with each iteration. import re matches = re.finditer(r'ptn', string) new_string = string for i, match in enumerate(matches): span = match.span() new_string = new_string[:span[0]] + array[i] + new_string[span[1]:]  Mangled output: 'ptn_sptn_s2, ptn2tn + ptn' How can I do these substitutions correctly?

Answer: As stated, one just have to use the "callback" form of re.sub - but it is actually simple enough: import re string = 'ptn, ptn; ptn + ptn' array = ['ptn_sub1', 'ptn_sub2', '2', '2'] array_iter = iter(array) result = re.sub("ptn", lambda m: next(array_iter), string)
Question: I am using the Python library Sympy to calculate the partial fraction decomposition of the following rational function: Z = (6.43369157032015e-9*s^3 + 1.35203404799555e-5*s^2 + 0.00357538393743079*s + 0.085)/(4.74334912634438e-11*s^4 + 4.09576274286244e-6*s^3 + 0.00334241812250921*s^2 + 0.15406018058983*s + 1.0) I am calling the method apart() like this: Z_f = apart(Z, full=True).doit() And this is the result: -1.30434068814287e+23/(s + 85524.0054884464) + 19145168.0/(s + 774.88576677949) + 91.9375/(s + 40.7977016133126) + 2.59375/(s + 7.79746609204661)  As you can see from the result, these are the residuals:  -3.60418953263334e+22 -4789228.25000000 11.0468750000000 0.787109375000000  Note: I identify them like this: Z_f_terms = Z_f.as_ordered_terms()  and then extract them in a for cycle. The problem is that by using other tools I get different residuals. By using GNU Octave's residue() I get these residuals:  133.6 1.0776 0.39501 0.56426  Via Wolphram Alpha I get the same residuals as Octave see here the results (please wait until you see the partial fraction expansion) . Why isn't the Sympy library not working as expected? The poles are always correct. Only the residuals are not as I expect. Thanks for any suggestion.

Answer: The problem is to do with floats. Presumably something in the algorithm used by apart(full=True) or RootSum.doit() is numerically unstable for approximate coefficients. If you convert the coefficients to rational numbers before computing the RootSum then evalf gives the expected result: In [27]: apart(nsimplify(Z), full=True).evalf() Out[27]: 133.599202650992 1.07757928431867 0.395006955518971 0.564264854137341 ──────────────────── + ─────────────────── + ──────────────────── + ──────────────────── s + 85524.0054884464 s + 774.88576677949 s + 40.7977016133126 s + 7.79746609204661 In [29]: apart(Z, domain="QQ", full=True).evalf() # rational field Out[29]: 133.599202650994 1.07757928431867 0.395006955518971 0.564264854137341 ──────────────────── + ─────────────────── + ──────────────────── + ──────────────────── s + 85524.0054884464 s + 774.88576677949 s + 40.7977016133126 s + 7.79746609204661  Note that the general algorithm and approach used for full=True is really intended for finding an exact representation of the decomposition by avoiding radicals. It is not intended for floats. Probably though normal apart without full=True should compute this decomposition.
Question: I have this dataframe: import polars as pl df = pl.DataFrame({ 'file':['a','a','a','a','b','b'], 'ru':['fe','fe','ev','ev','ba','br'], 'rt':[0,0,1,1,1,0], })  shape: (6, 3) ┌──────┬─────┬─────┐ │ file ┆ ru ┆ rt │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞══════╪═════╪═════╡ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ ev ┆ 1 │ │ a ┆ ev ┆ 1 │ │ b ┆ ba ┆ 1 │ │ b ┆ br ┆ 0 │ └──────┴─────┴─────┘  I'd like to replace the values in "ru" and "rt" within the same group defined by "file" with the values of the first row in the group if the first "rt" value is 0. The desired output would look as follows. shape: (6, 3) ┌──────┬─────┬─────┐ │ file ┆ ru ┆ rt │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞══════╪═════╪═════╡ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ b ┆ ba ┆ 1 │ │ b ┆ br ┆ 0 │ └──────┴─────┴─────┘  How can I achieve that?

Answer: Get first column values within each "file" group. This can be achieved using window functions (using pl.Expr.over in polars). df.with_columns( pl.col("ru").first().over("file"), pl.col("rt").first().over("file"), )  Polars also accepts multiple column names in pl.col and will evaluate the expressions independently of each other (just as above). df.with_columns( pl.col("ru", "rt").first().over("file") )  shape: (6, 3) ┌──────┬─────┬─────┐ │ file ┆ ru ┆ rt │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞══════╪═════╪═════╡ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ b ┆ ba ┆ 1 │ │ b ┆ ba ┆ 1 │ └──────┴─────┴─────┘  Add condition to take use first column values. To only use the first values within each group, if the first column value of "rt" within the group is 0, we can use a pl.when().then().otherwise() construct. For the condition we again use window functions. df.with_columns( pl.when( pl.col("rt").first().over("file") == 0 ).then( pl.col("ru", "rt").first().over("file") ).otherwise( pl.col("ru", "rt") ) )  shape: (6, 3) ┌──────┬─────┬─────┐ │ file ┆ ru ┆ rt │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞══════╪═════╪═════╡ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ a ┆ fe ┆ 0 │ │ b ┆ ba ┆ 1 │ │ b ┆ br ┆ 0 │ └──────┴─────┴─────┘
Question: My organization uses special codes for various employee attributes. We are migrating to a new system and I have to map these codes to a new code based on certain logic. Here is my mappings df Mappings: State Old_Mgmt New_Mgmt Old_ID New_ID New_Site 01 A001 A100 0000 0101 123 01 A002 A100 0000 0102 01 A003 A105 0000 0103 123 02 A001 A100 0000 0101  And here is EmployeeData: State Management ID Site 01 A001 0000 456 01 A002 0000 987 02 A002 0000 987 ....  The logic for the mapping is to go through each row of EmployeeData and if there is a match for State, Management, and ID, then it will update to the corresponding New_ value. However for Site, it will update the Site ID only if New_Site is not blank/NaN. This mapping will modify the original dataframe. Based on the above mapping the new EmployeeData would be: State Management ID Site 01 A100 0101 123 (modified this row) 01 A100 0102 987 (modified this row) 02 A002 0000 987 ....  My initial thought process was to do something like this: for i,r in EmployeeData.iterrows(): # For each employee row # Create masks for the filters we are looking for mask_state = Mappings['State'] == r['State'] mask_mgmt = Mappings['Old_Mgmt'] == r['Management'] mask_id = Mappings['Old_ID'] == r['ID'] # Filter mappings for the above 3 conditions MATCH = Mappings[mask_state & mask_mgmt & mask_id] if MATCH.empty: # No matches found print("No matches found in mapping. No need to update. Skipping.") continue MATCH = MATCH.iloc[0] # If a match is found, it will correspond to only 1 row EmployeeData.at[i, 'Management'] = MATCH['New_Mgmt'] EmployeeData.at[i, 'ID'] = MATCH['New_ID'] if pd.notna(MATCH['New_Site']): EmployeeData.at[i, 'Site'] = MATCH['New_Site']  However this seems fairly inefficient because I have to filter mappings for every row. If only 1 column was being mapped, I would do something like: # Make a dict mapping Old_Mgmt -> New_Mgmt MGMT_MAPPING = pd.Series(Mappings['New_Mgmt'].values,index=Mappings['Old_Mgmt']).to_dict() mask_state = Mappings['State'] = r['State'] EmployeeData.loc[mask_state, 'Management'] = EmployeeData.loc[mask_state, 'Management'].replace(MGMT_MAPPING)  But that would not work for my situation since I need to map multiple values

Answer: Left-merge and update: EmployeeData.update(EmployeeData .rename(columns={'Management': 'Old_Mgmt', 'ID': 'Old_ID'}) .merge(Mappings.rename(columns={'New_Mgmt': 'Management', 'New_ID': 'ID', 'New_Site': 'Site'}), how='left', on=['State', 'Old_Mgmt', 'Old_ID'], suffixes=('_old', None)) .replace('', None)[EmployeeData.columns] )  Updated EmployeeData:  State Management ID Site 0 01 A100 0101 123 1 01 A100 0102 987 2 02 A002 0000 987
Question: I am looping through the groups of a pandas groupby object to print the (sub)dataframe for each group. The headings are printed for each group. Here are some of the (sub)dataframes, with column headings "MMSI" and "ShipName":  MMSI ShipName 15468 109080345 OYANES 3 [19%] 46643 109080345 OYANES 3 [18%] MMSI ShipName 19931 109080342 OYANES 2 [83%] 48853 109080342 OYANES 2 [82%] MMSI ShipName 45236 109050943 SVARTHAV 2 [11%] 48431 109050943 SVARTHAV 2 [14%] MMSI ShipName 21596 109050904 MR:N2FE [88%] 49665 109050904 MR:N2FE [87%] MMSI ShipName 13523 941500907 MIKKELSEN B 5 [75%] 45711 941500907 MIKKELSEN B 5 [74%]  Web searching shows that pandas.io.formats.style.Styler.hide_columns can be used to suppress the headings. I am using Python 3.9, in which hide_columns is not recognized. However, dir(pd.io.formats.style.Styler) shows a hide method, for which the doc string gives this first example: >>> df = pd.DataFrame([[1,2], [3,4], [5,6]], index=["a", "b", "c"]) >>> df.style.hide(["a", "b"]) # doctest: +SKIP 0 1 c 5 6  When I try hide() and variations thereof, all I get is an address to the resulting Styler object: >>> df.style.hide(["a", "b"]) # doctest: +SKIP <pandas.io.formats.style.Styler at 0x243baeb1760> >>> df.style.hide(axis='columns') # https://stackoverflow.com/a/69111895 <pandas.io.formats.style.Styler at 0x243baeb17c0> >>> df.style.hide() # Desparate random trial & error <pandas.io.formats.style.Styler at 0x243baeb1520>  What could cause my result to differ from the doc string? How can I properly use the Styler object to get the dataframe printed without column headings? I am using pandas 2.0.3 with Spyder 5.4.3.

Answer: I don’t know about Styler, but you can print the data without headers like this. import pandas as pd df = pd.read_csv(r"/home/bera/GIS/data/Pandas_testdata/flights.csv") for month, subframe in df.groupby("month"): print(subframe.to_string(header=False)) print("\n")  Output: 3 1949 April 129 15 1950 April 135 27 1951 April 163 39 1952 April 181 51 1953 April 235 63 1954 April 227 75 1955 April 269 87 1956 April 313 99 1957 April 348 111 1958 April 348 123 1959 April 396 135 1960 April 461 7 1949 August 148 19 1950 August 170 31 1951 August 199 43 1952 August 242 55 1953 August 272 67 1954 August 293 79 1955 August 347 91 1956 August 405 103 1957 August 467 115 1958 August 505 127 1959 August 559 139 1960 August 606
Question: For the following df I would like to extract columns with "dates": import pandas as pd df = pd.DataFrame([["USD", 12.3, 1, 23.33, 33.1],["USD", 32.1, 2, 34.44, 23.1]],columns= ['currency', '1999-07-31', 'amount', '1999-10-31', '2000-01-31'])     currency 1999-07-31 amount 1999-10-31 2000-01-31     USD 12.3 1 23.33 33.1   USD 32.1 2 34.44 23.1    Current code: datetime_types = ["datetime", "datetime64", "datetime64[ns]", "datetimetz"] dates = df.columns.to_frame().select_dtypes(include=datetime_types)  Current output: dates.to_string() 'Empty DataFrame\nColumns: []\nIndex: [currency, 1999-07-31, amount, 1999-10-31, 2000-01-31]'  Desired output: [1999-07-31, 1999-10-31, 2000-01-31] 

Answer: You can't use select_dtypes in this context, this function filters based on the dtypes of the columns, not their names. You can however use pd.to_datetime with errors='coerce' and notna to form a boolean indexer: out = df.loc[:, pd.to_datetime(df.columns, errors='coerce', format='mixed') .notna()]  Output:  1999-07-31 1999-10-31 2000-01-31 0 12.3 23.33 33.1 1 32.1 34.44 23.1  If you just want the column names: cols = df.columns[pd.to_datetime(df.columns, errors='coerce', format='mixed').notna()]  Output: Index(['1999-07-31', '1999-10-31', '2000-01-31'], dtype='object')
Question: In the dataframe I define below I want to use the features ID and ID2 to fill the cells of features val1 and val2 with values. I want all ID and ID2 cominations to have the same values for the features val1 and val2.  df = pd.DataFrame({'ID':[0,0,0,1,1,1], 'DATE':['2021', '2022', '2023', '2021', '2022', '2023'], 'ID2':[23, 34, 54, 321, 1244, 1244], 'val1':[np.nan, 200, 300, np.nan, 234, np.nan], 'val2':[55555, 66666, 77777, 88888, 99999, np.nan], 'val3':['A', 'F', 'W', 'T', 'I', 'O']}) #expected result print(pd.DataFrame({'ID':[0,0,0,1,1,1], 'DATE':['2021', '2022', '2023', '2021', '2022', '2023'], 'ID2':[23, 34, 54, 321, 1244, 1244], 'val1':[np.nan, 200, 300, np.nan, 234, 234], 'val2':[55555, 66666, 77777, 88888, 99999, 99999], 'val3':['A', 'F', 'W', 'T', 'I', 'O']})) 

Answer: If need first non missing value per groups in columns val1 and val2 use GroupBy.transform with GroupBy.first: df[['val1','val2']] = df.groupby(['ID','ID2'])[['val1','val2']].transform('first') print (df) ID DATE ID2 val1 val2 val3 0 0 2021 23 NaN 55555.0 A 1 0 2022 34 200.0 66666.0 F 2 0 2023 54 300.0 77777.0 W 3 1 2021 321 NaN 88888.0 T 4 1 2022 1244 234.0 99999.0 I 5 1 2023 1244 234.0 99999.0 O
Question: I have a polars dataframe with two columns, one contains lists of string and the other one string. I want to apply the following expression to both columns. However, for some reason isinstance(x, list) doesn't work properly. def process_column(column_name: str, alias_name: str) -> pl.Expr: return ( pl.col(column_name).map_elements( lambda x: " ".join(x) if isinstance(x, list) else x ) .str.to_lowercase() .str.split(by="-") .list.join(" ") .alias(alias_name) )  Here is a sample dataframe df = pl.DataFrame({ "lists": [["hello", "World"], ["polars", "IS", "fast"]], "strings": ["foo-hello", "bOO"] })  Applying process_column to the "strings" column gives the expected result. df.with_columns(process_column("strings", "processed_string"))  However, for the "lists" column a SchemaError is raised. df.with_columns(process_column("lists", "processed_lists"))  polars.exceptions.SchemaError: invalid series dtype: expected `String`, got `list[str]`  I tried using map_elements with return_dtype=pl.String. This doesn't return an error, but the output is wrong.

Answer: I'd start by checking what is going on when applying pl.Expr.map_elements to a column of type List[pl.String] as follows. def check(x): print(x) return x df.with_columns( pl.col("lists").map_elements(check) )  shape: (2,) Series: '' [str] [ "hello" "World" ] shape: (3,) Series: '' [str] [ "polars" "IS" "fast" ] shape: (2, 2) ┌──────────────────────────┬───────────┐ │ lists ┆ strings │ │ --- ┆ --- │ │ list[str] ┆ str │ ╞══════════════════════════╪═══════════╡ │ ["hello", "World"] ┆ foo-hello │ │ ["polars", "IS", "fast"] ┆ bOO │ └──────────────────────────┴───────────┘  This suggests that the function passed to pl.Expr.map_elements doesn't receive native python lists, but pl.Series objects instead. Hence, we should replace lambda x: " ".join(x) if isinstance(x, list) else x  with lambda x: x.str.concat(" ").item() if isinstance(x, pl.Series) else x  to obtain the expected result. def process_column(column_name: str, alias_name: str) -> pl.Expr: return ( pl.col(column_name) .map_elements( lambda x: x.str.concat(" ").item() if isinstance(x, pl.Series) else x, return_dtype=pl.String ) .str.to_lowercase() .str.split(by="-") .list.join(" ") .alias(alias_name) ) df.with_columns(process_column("lists", "processed_lists"))  Note. I've also added the parameter return_dtype=pl.String to silence a warning about calling map_elements without specifying return_dtype possibly leading to unpredictable results. shape: (2, 3) ┌──────────────────────────┬───────────┬─────────────────┐ │ lists ┆ strings ┆ processed_lists │ │ --- ┆ --- ┆ --- │ │ list[str] ┆ str ┆ str │ ╞══════════════════════════╪═══════════╪═════════════════╡ │ ["hello", "World"] ┆ foo-hello ┆ hello world │ │ ["polars", "IS", "fast"] ┆ bOO ┆ polars is fast │ └──────────────────────────┴───────────┴─────────────────┘
Question: I have my dataframe set up like this    Cus_ID Cus_Type Cost birthdate     123 Owner 50 01 Jan 1980   123 Spouse 50 10 Feb 1982   123 Father 300 01 Dec 1950   125 Owner 20 30 Jan 1990   125 Spouse 30 15 Jul 1994   125 Mother 100 06 Sep 1970    I am trying to do a groupby where I group by Cus_ID but return the cus_type owner such that the final product looks like this:    Cus_ID Cus_Type Cost birthdate     123 Owner 400 01 Jan 1980   125 Owner 150 30 Jan 1990    I have a solution to work around it such as doing the groupby cus_id, summing the cost column then adding a column called cus_type on the new dataframe populated with cus_type then do a left join by cus_id and cus_type to the original DF to get the birth date. Is there a better way to do it than the solution that I currently have?

Answer: Sort the rows to have the Owner first or last, then use a custom groupby.agg: out = (df.sort_values(by='Cus_Type', key=lambda s: s.eq('Owner')) .groupby('Cus_ID', as_index=False) .agg({'Cus_Type': 'last', 'Cost': 'sum', 'birthdate': 'last'}) )  Another way that is more generic if you have many columns: d = dict.fromkeys(df.columns, 'last') d['Cost'] = 'sum' out = (df.sort_values(by='Cus_Type', key=lambda s: s.eq('Owner')) .groupby('Cus_ID', as_index=False).agg(d) ) # OR out = (df.assign(flag=df['Cus_Type'].eq('Owner')) .groupby('Cus_ID', as_index=False) .agg({'flag': 'idxmax', 'Cost': 'sum'}) ) out = out.join(df.drop(columns=['Cus_ID', 'Cost']) .reindex(out.pop('flag')).reset_index(drop=True))  Output:  Cus_ID Cus_Type Cost birthdate 0 123 Owner 400 01 Jan 1980 1 125 Owner 150 30 Jan 1990