eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a larger tkinter app that I wanted to dinamically set the topmost attribute. I am able to achieve what I want, but everytime I check the state of topmost, the selected menu bar on the screen goes invisible. To reproduce this, consider the MRE below and upon running the code, click the "menu" button on the menu bar, the cascade opens and the exit button shows, watch it vanish after the check_topmost function runs, but the "menu" button is still pressed somehow. Commenting out either the line that checks the attribute or the line that sets it as True stops the behaviour import tkinter as tk def check_topmost(): print(app.attributes('-topmost')) # comment this app.after(1000, check_topmost) app = tk.Tk() menu_bar = tk.Menu(app) sub_menu = tk.Menu(menu_bar, tearoff=0) sub_menu.add_command(label = 'exit', command = app.destroy) menu_bar.add_cascade(label = "menu", menu = sub_menu) app.config(menu = menu_bar) app.attributes('-topmost', 1) # comment this app.after(1000, check_topmost) app.mainloop()  What am I doing wrong?

Answer: This is not a fix for your issue, but rather a workaround. It seems like the topmost attribute is really weird... First, it is platform specific. Second, once set, it doesn't just change. If you set "-topmost", 1 for another window both will have the same attribute. From the documentation I found "-topmost gets or sets whether this is a topmost window". I assume, that once you call the attribute, it not only checks the attribute, but reassigns the value, therefore giving you the weird behavior with the drop-down menu. Interestingly, if you get all attributes at the same time by app.attributes(), your drop-down menu is not affected. So you can use this as workaround to check the attribute and only set it if necessary. Most likely, this is not the best solution, but it is the best I could come up with. import tkinter as tk def set_topmost(): app2.attributes('-topmost', 1) print('window 1', app.attributes()) print('window 2', app2.attributes()) # as you can see both windows can have the same topmost attribute app.attributes('-topmost', 0) # try the behavior after commenting this out def check_topmost(): print(app.attributes()) # this does not affect your window/drop-down menu att = str(app.attributes()) # making it a string as workaround if "'-topmost', 1" in att: print('already set as topmost') else: print('switch topmost') app.attributes('-topmost', 1) # this will still disrupt your drop down menu app.after(5000, check_topmost) # i set it to 5 seconds to better see the effects app = tk.Tk() menu_bar = tk.Menu(app) sub_menu = tk.Menu(menu_bar, tearoff=0) sub_menu.add_command(label = 'exit', command = app.destroy) menu_bar.add_cascade(label = "menu", menu = sub_menu) app.config(menu = menu_bar) app.attributes('-topmost', 1) # comment this app2 = tk.Toplevel() app2.geometry('500x500') btn = tk.Button(app, text='set topmost', command=set_topmost) btn.pack() app.after(1000, check_topmost) app.mainloop()
Question: I have a pandas line of code that gives me a future deprecation warning as stated in the title and I can't find in the pandas documentation how to modify it in order to remove the warning. The line of code is the following: df['temp_open']=df['temp_open'].replace('',method='ffill')  Any help would be greatly appreciated. I tried to fill blanks and it works, but I would like to get rid of the warning.

Answer: You can do this instead : df["temp_open"] = df["temp_open"].replace("", None).ffill()  And if you want to keep the nulls (if any) untouched, you can use : df["temp_open"] = ( df["temp_open"].replace("", None).ffill().where(df["temp_open"].notnull()) )  Output : print(df) temp_open 0 A 1 A 2 NaN 3 B 4 C 5 C  Used input : df = pd.DataFrame({"temp_open": ["A", "", None, "B", "C", ""]})
Question: In dplyr package of R, there's the option .keep = "unused" when creating new columns with the function mutate() (which is their equivalent of assign). An example, for those who haven't used it: > head(iris) Sepal.Length Sepal.Width Petal.Length Petal.Width Species 1 5.1 3.5 1.4 0.2 setosa 2 4.9 3.0 1.4 0.2 setosa 3 4.7 3.2 1.3 0.2 setosa 4 4.6 3.1 1.5 0.2 setosa 5 5.0 3.6 1.4 0.2 setosa 6 5.4 3.9 1.7 0.4 setosa # any column used in creating `new_col` is dropped afterwards automatically > mutate(.data = head(iris), new_col = Sepal.Length + Petal.Length * Petal.Width, .keep = "unused") Sepal.Width Species new_col 1 3.5 setosa 5.38 2 3.0 setosa 5.18 3 3.2 setosa 4.96 4 3.1 setosa 4.90 5 3.6 setosa 5.28 6 3.9 setosa 6.08  I say they are equivalent, but there doesn't appear to be the option for doing this with assign in the Pandas documentation so I assume it doesn't exist. I was curious about creating a way of doing something similar then. One way I can think of to do this is to create a list of names beforehand, and drop them afterwards, like this: from sklearn import datasets import pandas as pd used_columns = ['sepal length (cm)', 'petal length (cm)', 'petal width (cm)'] iris = pd.DataFrame(datasets.load_iris().data, columns=datasets.load_iris().feature_names) iris.assign(new_col = lambda x: x['sepal length (cm)'] + x['petal length (cm)'] * x['petal width (cm)']).drop(used_columns, axis=1)  or iris.assign(new_col = lambda x: x[used_columns[0]] + x[used_columns[1]] * x[used_columns[2]]).drop(used_columns, axis=1)  Which seems ~fine~, but requires a separate list, and with the first one, keeping two things updated, and with the second, the cognitive load of keeping track of what the nth list item is in my head. So I was curious if there's another way I'm not aware of of doing this, that would be easier to maintain? Both of the ones above seem not very Pythonic? Research I've done: I did a bunch of googling around this, with no luck. It seems there's plenty of ways of dropping columns, but none I've found seem particularly well-suited to this type of situation. Any help you could provide would be much appreciated! Answers which use other Python packages (e.g. janitor) are okay too.

Answer: I've never used R but based on the definition of unused and AFIK, to simulate the same behaviour in pandas, you will need to pop each column from a copy of the original DataFrame :  "unused" retains only the columns not used in ... to create new columns. This is useful if you generate new columns, but no longer need the columns used to generate them. DataFrame.pop(item) returns item and drops from frame. Raises KeyError if not found.   ( iris.copy().assign( new_col= lambda x: x.pop('sepal length (cm)') + x.pop('petal length (cm)') * x.pop('petal width (cm)')) )  Output :  sepal width (cm) new_col 0 3.5 5.38 1 3.0 5.18 2 3.2 4.96 3 3.1 4.90 4 3.6 5.28 .. ... ... 145 3.0 18.66 146 2.5 15.80 147 3.0 16.90 148 3.4 18.62 149 3.0 15.08 [150 rows x 2 columns]
Question: I have this table here and i'm trying to take the value and device category from each row in each column so I can have data like below. series: [{ name: 'engaged_sessions', data: [{ name: 'Desktop', y: 7765, }, { name: 'Mobile', y: 388 },... name: 'event_count', data: [{ name: 'Desktop', y: 51325, }, { name: 'Mobile', y: 4349 },...  And basically go through each column taking the device category and value into a list of dictionaries Heres the pivot table,  engaged_sessions event_count new_users total_revenue total_users device_category Desktop 7765 51325 6593 9 8021 Mobile 388 4349 795 0 412 Smart Tv 2 38 1 250 9 Tablet 87 111 37 0 97  I've tried using a for loop and put each iteration into a list but it wasn't quite right. The closest I've gotten is with to_dict() method and I think that has the best bet so far. This question here (Pandas to_dict data structure, using column as dictionary index) is very similar but I'm trying group by each column and if I use groupby(df.cloumns) or groupby(['column'],['column']) it gives me objects with numbers in it but no reference to what they are

Answer: If df contains the pivoted dataframe from your question you can do: out = [] for c in df: out.append( {"name": c, "data": [{"name": k, "y": v} for k, v in df[c].to_dict().items()]} ) print(out)  Prints: [ { "name": "engaged_sessions", "data": [ {"name": "Desktop", "y": 7765}, {"name": "Mobile", "y": 388}, {"name": "Smart Tv", "y": 2}, {"name": "Tablet", "y": 87}, ], }, { "name": "event_count", "data": [ {"name": "Desktop", "y": 51325}, {"name": "Mobile", "y": 4349}, {"name": "Smart Tv", "y": 38}, {"name": "Tablet", "y": 111}, ], }, { "name": "new_users", "data": [ {"name": "Desktop", "y": 6593}, {"name": "Mobile", "y": 795}, {"name": "Smart Tv", "y": 1}, {"name": "Tablet", "y": 37}, ], }, { "name": "total_revenue", "data": [ {"name": "Desktop", "y": 9}, {"name": "Mobile", "y": 0}, {"name": "Smart Tv", "y": 250}, {"name": "Tablet", "y": 0}, ], }, { "name": "total_users", "data": [ {"name": "Desktop", "y": 8021}, {"name": "Mobile", "y": 412}, {"name": "Smart Tv", "y": 9}, {"name": "Tablet", "y": 97}, ], }, ]
Question: I need to restructure a .csv file that follows this structure: a b 1 2 3 4 5 6 ...  Into this new structure: a b a b a b ... 1 2 3 4 5 6 ...  I don't know how to do this efficiently without iterating over each row. Is there a more efficient way of doing this in Python?

Answer: Here is one possible solution: import pandas as pd # read the .csv to a dataframe df = pd.read_csv('your_file.csv') # create a new df new_df = pd.DataFrame([df.to_numpy().ravel()], columns=df.columns.to_list() * len(df)) print(new_df)  Prints:  a b a b a b 0 1 2 3 4 5 6   To save the new dataframe: new_df.to_csv("out.csv", index=False)
Question: I am trying to extract the tuple items which consist of another embedded tuple without using the index. items = [(('a', 'b'), 'a'), (('a', 'b'), 'b'), (('a', 'b'), 'c'), (('a', 'b'), 'd')] # Currently accessing using index for xy,z in items: print(xy[0],xy[1],z) # Trying without index but getting error for x, y, z in items: print(x,y,z) for item in items: x, y, z = item print(x,y,z)  Error: ValueError: not enough values to unpack (expected 3, got 2)  Is it possible?

Answer: Yes, you just need to wrap the nested tuple to have an extra layer of unpacking: for (x, y), z in items: print(x,y,z)  Note how (x, y) causes the inner tuple to be unpacked.
Question: I work through Python tutorial and ran against a sample in https://docs.python.org/3/tutorial/errors.html in section 8.9. Raising and Handling Multiple Unrelated Exceptions which doesn't work by me: $ python Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> excs = [OSError('error 1'), SystemError('error 2')] >>> raise ExceptionGroup('there were problems', excs) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'ExceptionGroup' is not defined >>>  Why? Isn't a ExceptionGroup a built-in exception? The compiler doesn't give any errors and IDE pops up the documentation for this class... The next thought was - I have to import something: >>> from builtins import ExceptionGroup Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name 'ExceptionGroup' from 'builtins' (unknown location)  What's wrong?

Answer: According to the official docs, ExceptionGroup was only added in Python 3.11. Make sure you check your Python version
Question: I am trying to format the dataframe using https://pandas.pydata.org/docs/reference/api/pandas.io.formats.style.Styler.format.html. But, I am not getting the desired result: >>> df = pd.DataFrame([[np.nan, 1.534231, 'A'], [2.32453251, np.nan, 3.0]]) >>> df.style.format(na_rep='MISS', precision=3) <pandas.io.formats.style.Styler object at 0x7f0b32eff740> >>> print(df.head()) 0 1 2 0 NaN 1.534231 A 1 2.324533 NaN 3.0 >>> print(pd.__version__) 1.5.3  with python --version Python 3.12.0rc3  What am I doing wrong here?

Answer: Styler.format is used to apply as specific formatting to a pandas DataFrame. So the data present in the dataframe doesn't change but when you display it in a specific environment, the styling will be applied to the data. So when you write the following code, >>> df = pd.DataFrame([[np.nan, 1.534231, 'A'], [2.32453251, np.nan, 3.0]]) >>> df.style.format(na_rep='MISS', precision=3) <pandas.io.formats.style.Styler object at 0x7f0b32eff740>  Notice the last line, where a Styler object is returned. You can hold this Styler object in another variable as follows. Now, to display this formatted df, let's make the environment into a HTML web browser environment. As both Styler and pandas.DataFrame have a to_html() method, lets convert the df into html code and view into in the web browser. >>> import webbrowser >>> with open('str.html', 'w') as f: s=df.style.format(na_rep='MISS', precision=3) s.to_html(f) >>> webbrowser.open_new_tab('str.html') True  The above code pops up a new tab with the page 'str.html' stored locally, where you can easily view the formatted df.
Question: I have a decorator that looks for an optional kwargs item, foo, with all other *args and **kwargs passed through to the decorated function. I want the typing hints to specify which type that specific item must be, if it is present.  For the following snippet, how can I create the typing partial constraint on FnParams such that IFF foo item is present, then its type MUST be str ? from typing import Callable, ParamSpec, TypeVar, TypedDict, Required, NotRequired, Unpack, cast, Any FnRet = TypeVar("FnRet") # how can this be partially constrained to _required_ type `str`, # if `foo` item is present? FnParams = ParamSpec("FnParams") FnType = Callable[FnParams, FnRet] def decorator(fn: FnType) -> FnType: def wrapper(*args: FnParams.args, **kwargs: FnParams.kwargs) -> FnRet: # type: ignore[type-var] if "foo" in kwargs: assert type(kwargs["foo"]) == str return cast(FnRet, fn(*args, **kwargs)) return wrapper ## Should pass: @decorator def ok_1() -> int: return 42 class RequiredFooKwArgs(TypedDict): foo: Required[str] @decorator def ok_required_foo(*_args: Any, **kwargs: Unpack[RequiredFooKwArgs]) -> int: return 42 class NotRequiredFooKwArgs(TypedDict): foo: NotRequired[str] @decorator def ok_not_required_foo(*_args: Any, **kwargs: Unpack[NotRequiredFooKwArgs]) -> int: return 42 ## The Badies: class BadTypeRequiredFooKwArgs(TypedDict): foo: Required[int] # <--- this should be flagged as the wrong type @decorator def bad_type_required_foo(*_args: Any, **kwargs: Unpack[BadTypeRequiredFooKwArgs]) -> int: return 42 class BadTypeNotRequiredFooKwArgs(TypedDict): foo: NotRequired[int] # <--- this should be flagged as the wrong type @decorator def bad_type_not_required_foo(*_args: Any, **kwargs: Unpack[BadTypeNotRequiredFooKwArgs]) -> int: return 42   When I type check this code with mypy 1.5.1, I get: mypy --enable-incomplete-feature=Unpack ~/tmp.py Success: no issues found in 1 source file  What I'd like is to augment FnParams with a foo field type constraint and then have some python type-checker fail the static analysis.

Answer: It appears this isn't available yet: https://peps.python.org/pep-0612/#concatenating-keyword-parameters However, there is room for this in the future.
Question: I am new to Python and I am quite confused about the following code: Example 1: n = 1 m = n n = 2 print(n) # 2 print(m) # 1  Example 2: names = ["a", "b", "c"] visitor = names names.pop() print(names) # ['a', 'b'] print(visitor) # ['a', 'b']  Example 1 shows that n is 1 and m is 1. However, example 2 shows that names is ['a', 'b'], and visitor is also ['a', 'b']. To me, example 1 and example 2 are similar, so I wonder why the results are so different? Thank you.

Answer: Integers are not mutable. So the only way to change a variable that is assigned to an integer is through assignment. When you assign m=n, m is assigned to the same address that n currently is assigned. Both m and n refer to the address of the integer 1. m will continue to point to 1 even if n is assigned to a different value be it an integer or something else entirely. This behavior based on assignment is common to both mutable and immutable types. By contrast, lists are mutable so you can modify by means that are not only assignment. When you assign visitor = names, visitor points to the same list that names is assigned to, just like the above case. And were you to change the value of names through assignment visitor would not be impacted. The difference emerges if you mutate the list. If you call names.pop() for example this mutates the list and visitor (and any other variable that was assigned to this list) will see the mutation. In your case you mutated the list with a pop operation, there are others which we won't go into here.
Question: I have a DataFrame named airtep_df with columns 'AIRTEP' (Int64), 'AIRTEP_qc' (Int64), and 'UTCTime'. I'm trying to create a plot from this data, but the time tick labels on the x-axis are overlapping, making the plot unreadable. I attempted to rotate the tick labels using plt.xticks(rotation=45), but the plot disappeared. Can someone explain what went wrong and provide a solution for formatting the time tick labels with datetime correctly? This picture shows the details about the data frame I'm using:  This is the script I used: import matplotlib.pyplot as plt import pandas as pd index = (airtep_df['AIRTEP'] != 9999) & (airtep_df['AIRTEP_qc'] == 1) selected_y = airtep_df.loc[index,'AIRTEP'] y = selected_y.values selected_x = time_df.loc[index,'UTCTime'] x = selected_x.values plt.plot(x[30],y[30])  #This is the result I have, the the x-axis are overlapping: #Then I trid to rotate the xticks plt.xticks(rotation=45)  This is the result I have, as the picture:  How can I plot datetime data and format it such that the labels are not overlapping?

Answer: Datetime-formatted rotated x-axis tick labels You could try something like: import numpy as np import pandas as pd ​ import matplotlib.dates as mdates import matplotlib.pyplot as plt ​ ​ ### Sample data generation date_rng = pd.date_range(start="2016-01-01", end="2016-01-02", freq="H") ​ temps = np.random.randint(17, 41, size=(len(date_rng))) ​ qc_values = np.ones(len(date_rng), dtype=int) ​ airtep_df = pd.DataFrame( {"UTCTime": date_rng, "AIRTEP": temps, "AIRTEP_qc": qc_values} ) ​ print(airtep_df) ​ ​ ### Plotting plt.figure(dpi=300) plt.plot(airtep_df.UTCTime, airtep_df.AIRTEP, marker="o", linestyle="-") ​ # Rotating the x-tick labels plt.xticks(rotation=45) ​ # Adjusting the date format ax = plt.gca() ax.xaxis.set_major_formatter(mdates.DateFormatter("%H:%M")) ​ plt.tight_layout() plt.xlabel("Time") plt.ylabel("Temperature (℃)") plt.grid(True, which="both", linestyle="--", linewidth=0.5) plt.show()  which produces:  UTCTime AIRTEP AIRTEP_qc 0 2016-01-01 00:00:00 30 1 1 2016-01-01 01:00:00 27 1 2 2016-01-01 02:00:00 37 1 3 2016-01-01 03:00:00 36 1 4 2016-01-01 04:00:00 18 1 5 2016-01-01 05:00:00 26 1 6 2016-01-01 06:00:00 27 1 7 2016-01-01 07:00:00 20 1 8 2016-01-01 08:00:00 27 1 9 2016-01-01 09:00:00 30 1 10 2016-01-01 10:00:00 37 1 11 2016-01-01 11:00:00 32 1 12 2016-01-01 12:00:00 29 1 13 2016-01-01 13:00:00 29 1 14 2016-01-01 14:00:00 23 1 15 2016-01-01 15:00:00 24 1 16 2016-01-01 16:00:00 37 1 17 2016-01-01 17:00:00 19 1 18 2016-01-01 18:00:00 19 1 19 2016-01-01 19:00:00 26 1 20 2016-01-01 20:00:00 35 1 21 2016-01-01 21:00:00 29 1 22 2016-01-01 22:00:00 24 1 23 2016-01-01 23:00:00 21 1 24 2016-01-02 00:00:00 39 1
Question: I'm trying to scale down a set of numbers to feed into a DP subset-sum algorithm. (It blows up if the numbers are too large.) Specifically, I need to find the largest power of 10 I can divide into the numbers without losing precision. I have a working routine but since it will run often in a loop, I'm hoping there's a faster way than the brute force method I came up with. My numbers happen to be Decimals. from decimal import Decimal import math def largest_common_power_of_10(numbers: list[Decimal]) -> int: """ Determine the largest power of 10 in list of numbers that will divide into all numbers without losing a significant digit left of the decimal point """ min_exponent = float('inf') for num in numbers: if num != 0: # Count the number of trailing zeros in the number exponent = 0 while num % 10 == 0: num //= 10 exponent += 1 min_exponent = min(min_exponent, exponent) # The largest power of 10 is 10 raised to the min_exponent return int(min_exponent) decimal_numbers = [Decimal("1234"), Decimal("5000"), Decimal("200")] result = largest_common_power_of_10(decimal_numbers) assert(result == 0) decimal_numbers = [Decimal(470_363_000.0000), Decimal(143_539_000.0000), Decimal(1_200_000.0000)] result = largest_common_power_of_10(decimal_numbers) assert(result == 3) divisor = 10**result # Later processing can use scaled_list scaled_list = [x/divisor for x in decimal_numbers] assert(scaled_list == [Decimal('470363'), Decimal('143539'), Decimal('1200')]) reconstituted_list = [x * divisor for x in scaled_list] assert(reconstituted_list == decimal_numbers) 

Answer: This can be done all at once with the math library if your list is all integers. import math def largest_common_power_of_10(numbers): # get the greatest power of 10 that divides all numbers in list gcd_nums = math.gcd(*numbers) gcd = [x for x in range(1, len(str(gcd_nums))) if gcd_nums % math.pow(10, x) == 0] if len(gcd) == 0: return 0 else: gcp = max(gcd) return gcp  Examples: numbers = [11230000,125,44500000] largest_common_power_of_10(numbers) # 0 numbers = [11230000,1540,44500000] largest_common_power_of_10(numbers) # 1 numbers = [11230000,1540000,44500000] largest_common_power_of_10(numbers) # 4  Note: This may only work with python 3.9 and greater because math.gcd() starrted accepting lists in that release.
Question: I have a set of pairs that can contain the result of mutliple "N choose 2" combinations: inputs = { ('id1', 'id2'), ('id1', 'id3'), ('id1', 'id4'), ('id2', 'id3'), ('id2', 'id4'), ('id3', 'id4'), ('id3', 'id5'), ('id4', 'id5'), ('id5', 'id6'), }  And I would like to reverse those combinations, like this: recombinations = [ ('id1', 'id2', 'id3', 'id4'), ('id3', 'id4', 'id5'), ('id5', 'id6'), ]  I managed to do it using brute-force: ids = list(sorted( {i for i in itertools.chain(*inputs)} )) excludes = set() recombinations = {tuple(i) for i in map(sorted, inputs)} for i in range(3, len(ids)+1): for subset in itertools.combinations(ids, i): for j in range(i-1, len(subset)): combs = set(itertools.combinations(subset, j)) if all(tup in recombinations for tup in combs): recombinations.add(subset) excludes = excludes.union(combs) for tup in excludes: recombinations.remove(tup) print(recombinations)  {('id1', 'id2', 'id3', 'id4'), ('id3', 'id4', 'id5'), ('id5', 'id6')}  Is there a smarter way to do it? Or some optimizations that I can add to the code?

Answer: Using the networkx library, this is quite simple, since it has a function that does exactly what you're asking for: finding all maximal cliques in a graph. Here: import networkx as nx G = nx.Graph() pairs_of_connected_nodes = [ ('id1', 'id2'), ('id1', 'id3'), ('id1', 'id4'), ('id2', 'id3'), ('id2', 'id4'), ('id3', 'id4'), ('id3', 'id5'), ('id4', 'id5'), ('id5', 'id6') ] G.add_edges_from(pairs_of_connected_nodes) maximal_cliques = list(nx.find_cliques(G)) for clique in maximal_cliques: print(clique)  Output: ['id3', 'id4', 'id2', 'id1'] ['id3', 'id4', 'id5'] ['id6', 'id5']  Of course if your assignment is to implement the Bron-Kerbosch algorithm yourself, you'd have to code it up - but then asking on StackOverflow kind of defeats the purpose, unless you have a specific problem with your solution that you need help with? If you're just asking for a review of your code, ask on Code Review Stack Exchange, but expect to get told to use networkx as well.