eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm attempting to create a subclass of pathlib.Path that will do some manipulation to the passed string path value before passing it along to the base class. class MyPath(Path): def __init__(self, str_path): str_path = str_path.upper() # just representative, not what I'm actually doing super().__init__(str_path)  However, when I try to use this: foo = MyPath("/path/to/my/file.txt") bar = foo / "bar"  I get the following error: TypeError: unsupported operand type(s) for /: 'MyPath' and 'str' I am using Python 3.12 which I understand to have better support for subclassing Path

Answer: MyPath("foo") / "bar' goes through several steps:  It invokes MyPath("foo").__truediv__("bar"), which Invokes MyPath("foo").with_segments("bar"), which Invokes MyPath(MyPath("foo"), "bar"), which Raises a TypeError because you overrode MyPath.__init__ to only accept a single additional positional argument, which Causes MyPath.__truediv__ to catch a TypeError and subsequently return NotImplemented  Your __init__ method must be careful to accept multiple path components, not just a single string, and it must not assume that each argument is an actual string, but rather objects that implement the os.PathLike interface. (In particular, don't assume that a path component has an upper method. Call os.fspath on the component first to retrieve a str representation that does have an upper method.) Something like import os class MyPath(Path): def __init__(self, *paths): super().__init__(*(os.fspath(x).upper() for x in paths))  and thus % py312/bin/python -i tmp.py >>> bar MyPath('/PATH/TO/MY/FILE.TXT/BAR')  Alternately, as others have alluded to, you can override with_segments to, for example, combine the segments yourself and pass the single result to MyPath, but I see no reason to restrict MyPath.__init__'s signature as you currently do.
Question: I want to take a font size (14) and using pygame_widgets.button.Button() scale it to match the font size up to a maximum almost every question I've seen on here has been the other way around at the moment I'm unfamiliar with the maths that would be used but I would greatly appreciate the help import pygame import pygame_widgets from pygame_widgets.button import Button pygame.init() font_size = 14 screen = pygame.display.set_mode((640, 480)) font = pygame.font.SysFont('Arial', font_size) Wide = 60 # Math High = 20 # Math button = Button( screen, # surface to display 10, # Button's top left x coordinate 10, # Button's top left y coordinate Wide, # Button's Width High, # Button's Height text="Button", fontSize=font_size, inactiveColour=(255, 255, 255), hoverColour=(245, 245, 245), pressedColour=(230, 230, 230), radius=2, # curve on the button corners onClick=lambda: print("Click") ) while True: events = pygame.event.get() for event in events: if event.type == pygame.QUIT: pygame.quit() sys.exit() screen.fill((200, 200, 200)) pygame_widgets.update(events) pygame.display.update() 

Answer: For scaling a button's size dynamically based on its font size you can use a simple proportional scaling method. I've implemented it in code done below: import pygame import pygame_widgets from pygame_widgets.button import Button import sys pygame.init() screen = pygame.display.set_mode((640, 480)) base_font_size = 14 current_font_size = 14 max_font_size = 40 # Max font size for scaling # Button size at base font size (14) base_button_width = 60 base_button_height = 20 def scale_button_size(font_size, max_size): """ Scales the button dimensions proportionally to font size """ scale_factor = font_size / base_font_size width = min(base_button_width * scale_factor, max_size) height = min(base_button_height * scale_factor, max_size) return int(width), int(height) # Get scaled button dimensions Wide, High = scale_button_size(current_font_size, max_font_size) # Create button with scaled dimensions button = Button( screen, 10, # Button's top-left x coordinate 10, # Button's top-left y coordinate Wide, # Button's width High, # Button's height text="Button", fontSize=current_font_size, inactiveColour=(255, 255, 255), hoverColour=(245, 245, 245), pressedColour=(230, 230, 230), radius=2, onClick=lambda: print("Click") ) while True: events = pygame.event.get() for event in events: if event.type == pygame.QUIT: pygame.quit() sys.exit() screen.fill((200, 200, 200)) pygame_widgets.update(events) pygame.display.update()
Question: I have a Pandas dataframe that is setup like so: Code StartDate EndDate A 2024-07-01 2024-08-03 B 2024-08-06 2024-08-10 C 2024-08-11 2024-08-31  I have a part of my code that iterates through each day (starting from 2024-07-01) and I am trying to return the corresponding Code given a date (with a fallback if the date does not fall within any StartDate/EndDate range). My original idea was to do something like: DAYS = DAY_DF['Date'].tolist() # Just a list of each day for DAY in DAYS: code = False for i,r in df.iterrows(): if r['StartDate'] <= DAY <= r['EndDate']: code = r['Code'] break if not code: # `Code` is still False code = 'Fallback_Code'  But this seems very inefficient to iterate over each row in the dataframe especially because I have a lot of records in my dataframe. Here are some example inputs and the resulting code output: 2024-07-03 -> 'A' 2024-08-04 -> 'Fallback_Code' 2024-08-10 -> 'B' 2024-08-11 -> 'C' 

Answer: A possible solution, which converts the StartDate and EndDate columns to datetime format (to allow for comparison of dates). It then checks if a specific date (e.g., 2024-07-03) falls within any of the date ranges defined by StartDate and EndDate. If it does, it retrieves the first corresponding Code from those rows; if not, it returns Fallback code. df['StartDate'] = pd.to_datetime(df['StartDate']) df['EndDate'] = pd.to_datetime(df['EndDate']) date = '2025-07-03' # input example m = df['StartDate'].le(date) & df['EndDate'].ge(date) df.loc[m, 'Code'].iloc[0] if m.any() else 'Fallback code'   To get the codes for a list of dates, we might use the following: dates = ['2024-07-03', '2024-08-04', '2024-08-10', '2024-08-11'] m = lambda x: df['StartDate'].le(x) & df['EndDate'].ge(x) {date: df.loc[m(date), 'Code'].iloc[0] if m(date).any() else 'Fallback code' for date in dates}   Output: {'2024-07-03': 'A', '2024-08-04': 'Fallback code', '2024-08-10': 'B', '2024-08-11': 'C'}
Question: I would like to generate/enumerate all possible lists of non-negative integers such that the algorithm will generate lists like the following at some point [1] [24542,0] [245,904609,848,24128,350,999]  In other words, for all possible non-negative integers, generate all possible lists which contain that many non-negative integers. I have figured that the trick for a list with two numbers is to enumerate their values diagonally like this    first value\second value 0 1 2 3     0 0 (this will be generated first) 2 (this third etc.) 5 9   1 1 (this second) 4 8    2 3 7     3 6       def genpair(): x = 0 y = 0 yield x,y maxx = 0 while True: maxx += 1 x = maxx y = 0 while x >= 0: yield x,y x -= 1 y += 1 gen = genpair() for i in range(10): print(next(gen))  But does the same trick (or another) also make this work for lists of arbitrary length?

Answer: One of infinitely many ways to do this: Imagine a number line with cells 1, 2, 3, and up to infinity. Now think of a binary number representation, with bits indicating if there is a "break" at the cell border. So, 1 -> [1] 10 -> [2] 11 -> [1,1] 100 -> [3] 101 -> [2, 1] 110 -> [1, 2]  Note how number of bits is the same as the sum of the list, and number of positive bits indicates the number of list elements. Code would look somewhat like: def list_gen(n): res = [] counter = 0 while n: counter += 1 n, flag = divmod(n, 2) if flag: res.append(counter) counter = 0 return res
Question: I have a python JSON data and need to iterate through the key pair values and check if the value is of certain type and perform an operation. here is the example: total = 0 grades = { 'Math': 90, 'Science': None, 'English': 85, 'History': 'A', 'Art': 88 } for grade in grades.values(): total += grade  i need to check if the value is a number then add to the total.

Answer: You can check the variable type using isinstance() like this for grade in grades.values(): if isinstance(grade, (int, float)): # Check if the grade is a number total += grade  This will add the values only if the variable is of type int or float
Question: I have pandas series that I want to split into two: one with all the entries of the original series where index contains a certain word and the other with all the remaining entries. Getting a series of entries which do contain a certain word in their index is easy: foo_series = original_series.filter(like = "foo")  But how do I get the rest?

Answer: You could drop those indices from the original Series: foo_series = original_series.filter(like = "foo") non_foo_series = original_series.drop(foo_series.index)  Or use boolean indexing: m = original_series.index.str.contains('foo') foo_series = original_series[m] non_foo_series = original_series[~m]  Example: # input original_series = pd.Series([1, 2, 3, 4], index=['foo1', 'bar1', 'foo2', 'bar2']) # outputs # foo_series foo1 1 foo2 3 dtype: int64 # non_foo_series bar1 2 bar2 4 dtype: int64
Question: I need all the rows that have null in one of the predefined columns. I basically need this but i have one more requirement that I cant seem to figure out. Not every column needs to be checked. I have a function that returns the names of the columns that need to be checked in a list. Assume this is my dataframe: data = pl.from_repr(""" ┌───────┬───────┬─────┬───────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ bool │ ╞═══════╪═══════╪═════╪═══════╡ │ abc ┆ null ┆ u ┆ true │ │ def ┆ abc ┆ v ┆ true │ │ ghi ┆ def ┆ null┆ true │ │ jkl ┆ uvw ┆ x ┆ true │ │ mno ┆ xyz ┆ y ┆ null │ │ qrs ┆ null ┆ z ┆ null │ └───────┴───────┴─────┴───────┘ """)  Doing data.filter(polars.any_horizontal(polars.all().is_null())) gives me all rows where any of the columns contain null. Sometimes it's fine for column c to contain a null so let's not check it. what I want is this: ┌───────┬───────┬─────┬───────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ bool │ ╞═══════╪═══════╪═════╪═══════╡ │ abc ┆ null ┆ u ┆ true │ │ mno ┆ xyz ┆ y ┆ null │ │ qrs ┆ null ┆ z ┆ null │ └───────┴───────┴─────┴───────┘  Row 3 is not shown even though there is a null value in column c. columns = ["a", "b", "d"] data.filter(polars.any_horizontal(polars.all(*columns).is_null()))  This gives me polars.exceptions.SchemaError: invalid series dtype: expected 'Boolean', got 'str' I thought maybe the columns aren't aligned or somethig because data has more columns than what the filter uses, so i did this. columns = ["a", "b", "d"] # notice `.select(columns)` here data.select(columns).filter(polars.any_horizontal(polars.all(*columns).is_null()))  But is still get the same error. How do I get the full rows of data that contain a null in one of ["a", "b", "d"] columns

Answer: If you want exclude some columns you can use .exlude(): import polars as pl data.filter(pl.any_horizontal(pl.exclude("c").is_null()))  ┌─────┬──────┬─────┬──────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ bool │ ╞═════╪══════╪═════╪══════╡ │ abc ┆ null ┆ u ┆ true │ │ mno ┆ xyz ┆ y ┆ null │ │ qrs ┆ null ┆ z ┆ null │ └─────┴──────┴─────┴──────┘  Or you can just use column names by using .col(): import polars as pl cols = ["a","b","d"] data.filter(pl.any_horizontal(pl.col(cols).is_null()))  shape: (3, 4) ┌─────┬──────┬─────┬──────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ bool │ ╞═════╪══════╪═════╪══════╡ │ abc ┆ null ┆ u ┆ true │ │ mno ┆ xyz ┆ y ┆ null │ │ qrs ┆ null ┆ z ┆ null │ └─────┴──────┴─────┴──────┘  If you want to be really flexible, you can use selectors, for example .selectors.exclude(): import polars.selectors as cs data.filter(pl.any_horizontal(cs.exclude("c").is_null()))  shape: (3, 4) ┌─────┬──────┬─────┬──────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ bool │ ╞═════╪══════╪═════╪══════╡ │ abc ┆ null ┆ u ┆ true │ │ mno ┆ xyz ┆ y ┆ null │ │ qrs ┆ null ┆ z ┆ null │ └─────┴──────┴─────┴──────┘
Question: I'm trying to read a 16-bit grayscale PNG with OpenCV. This image is stored on a network share. When I try to load the image with cv.imread() nothing is returned: import cv2 as cv print(img_paths[0]) print(type(img_paths[0])) print(img_paths[0].exists()) img = cv.imread(img_paths[0].as_posix(), cv.IMREAD_ANYDEPTH) print(type(img)) >>> M:\path\to\Meß-ID 1\images\img1.png >>> <class 'pathlib.WindowsPath'> >>> True >>> <class 'NoneType'>  As you can see the file certainly exists. It even is loadable with PIL: from PIL import Image img = Image.open(img_paths[0]) print(img) >>> <PIL.PngImagePlugin.PngImageFile image mode=I;16 size=2560x300 at 0x27F77E95940>  And only when I copy the image to the directory of the .ipynb I'm working in, openCV is able to load the image: import shutil shutil.copy(img_paths[0], "./test_img.png") img = cv.imread("./test_img.png", cv.IMREAD_ANYDEPTH) print(type(img)) >>> <class 'numpy.ndarray'>  Why is OpenCV refusing to load the image?

Answer: Yes, OpenCV on Windows has issues with encodings. Usually, anything non-ASCII in the path string can cause you trouble. Shortest possible solution: im = cv.imdecode(np.fromfile(the_path, dtype=np.uint8), cv.IMREAD_UNCHANGED)
Question: I am trying to recreate the charts on this website: https://bumps.live/torpids/2022. I am using matplotlib and am running into an issue when drawing the logos, which I have recreated with the MWE below. I am drawing two semicircles next to each other, and the result is as expected when they are around the same size as the axis, but when they are much smaller there is a loss of precision and the semicircles no longer take up the space of half a circle. The radius of both semicircles is 0.1 in the following two figures, but the first figure has axis limits from -0.15 to 0.15 and the second figure has limits from -10 to 10 (I have zoomed in on the right figure). When using plt.show and zooming in, this issue does not occur.   I am guessing that matplotlib has defined the wedges to a suitable degree of accuracy assuming that no one is zooming in a lot, although as I am zooming in, this is not sufficient. I asked ChatGPT and it suggested adding theresolution=100 kwarg to my wedges, but this appears to be deprecated or something as that gives an error. I am using python 3.12.3 and matplotlib 3.9.1.post1. I will need to produce around 350 logos, and I would be willing to add them as SVGs if the performance is good enough as a backup plan, but ideally I would like to understand how to fix this issue. import matplotlib.pyplot as plt from matplotlib.patches import Wedge fig, ax = plt.subplots(figsize=(5, 5)) radius = 0.1 left_semicircle = Wedge((0, 0), radius, 0, 180, color="r") right_semicircle = Wedge((0, 0), radius, 180, 360, color="b") ax.add_patch(left_semicircle) ax.add_patch(right_semicircle) axis_limit = 0.15 axis_limit = 10 ax.set_xlim((-axis_limit, axis_limit)) ax.set_ylim((-axis_limit, axis_limit)) ax.set_aspect('equal') plt.savefig(f"TwoSemicircles_{axis_limit}.pdf") 

Answer: When you set the color, you are setting both facecolor (which is the color of the inside of the shape) and the edgecolor (which is the color of the outline). Matplotlib then draws the outline with a default line width of 1 point. That linewidth is preserved for the interactive zoom in plt.show (so remains too small to notice), but in your pdf the linewidth is a fixed proportion of the page size. Instead you should just set the facecolor left_semicircle = Wedge((0, 0), radius, 0, 180, facecolor="r") right_semicircle = Wedge((0, 0), radius, 180, 360, facecolor="b")  then Matplotlib does not draw the edge.
Question: I was trying this new free-thread version of the interpreter, but find out that it actually takes longer than the GIL enabled version. I did observe that the usage on the CPU increase a lot for the free-thread interpreter, is there something I misunderstand about this new interpreter? Version downloaded: python-3.13.0rc2-amd64 Code: from concurrent.futures import ThreadPoolExecutor from random import randint import time def create_table(size): a, b = size table = [] for i in range(0, a): row = [] for j in range(0, b): row.append(randint(0, 100)) table.append(row) return table if __name__ == "__main__": start = time.perf_counter() with ThreadPoolExecutor(4) as pool: result = pool.map(create_table, [(1000, 10000) for _ in range(10)]) end = time.perf_counter() print(end - start, *[len(each) for each in result])  python3.13t takes 56sec python3.13 takes 26sec python3.12 takes 25sec 

Answer: The primary culprit appears to be the randint module, as it is a static import and appears to share a mutex between threads. Another problem is that you're only able to process 4 tables at a time. Since you want to create 10 tables in total, you'll be running batches of 4-4-2. Here is the code with the randint problem addressed by replacing it with a SystemRandom instance per thread: from concurrent.futures import ThreadPoolExecutor from random import SystemRandom import time def create_table(size): a, b = size table = [] random = SystemRandom() for i in range(0, a): row = [] for j in range(0, b): row.append(random.randint(0, 100)) table.append(row) return table if __name__ == "__main__": start = time.perf_counter() with ThreadPoolExecutor(4) as pool: result = pool.map(create_table, [(1000, 10000) for _ in range(10)]) end = time.perf_counter() print(end - start, *[len(each) for each in result])  And here is some code that achieves the same thing, but is more flexible with the thread creation and avoids unnecessary inter-thread communication: import threading from random import SystemRandom import time def create_table(obj, result: list[list[int]]): a, b = obj print(f"Starting thread {threading.current_thread().name}") random = SystemRandom() result[:] = [[random.randint(0, 100) for j in range(b)] for i in range(a)] print(f"Finished thread {threading.current_thread().name}") if __name__ == "__main__": start = time.perf_counter() obj = (1000, 10000) results: list[list[list[int]]] = [] threads: list[threading.Thread] = [] for _ in range(4): result: list[list[int]] = [] thread = threading.Thread(target=create_table, args=(obj, result)) thread.start() threads.append(thread) results.append(result) for thread in threads: thread.join() print([len(r) for r in results]) end = time.perf_counter() print(end - start)
Question: I have a pl.DataFrame with a column comprising lists like this: import polars as pl df = pl.DataFrame( { "symbol": ["A", "A", "B", "B"], "roc": [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6], [0.7, 0.8]], } ) shape: (4, 2) ┌────────┬────────────┐ │ symbol ┆ roc │ │ --- ┆ --- │ │ str ┆ list[f64] │ ╞════════╪════════════╡ │ A ┆ [0.1, 0.2] │ │ A ┆ [0.3, 0.4] │ │ B ┆ [0.5, 0.6] │ │ B ┆ [0.7, 0.8] │ └────────┴────────────┘  Further, I have a regular python list weights = [0.3, 0.7] What's an efficient way to multiply pl.col("roc") with weights in a way where the first and second element of the column will be multiplied with the first and second element of weights, respectively? The expected output is like this: shape: (4, 3) ┌────────┬────────────┐──────────────┐ │ symbol ┆ roc │ roc_wgt │ │ --- ┆ --- │ --- │ │ str ┆ list[f64] │ list[f64] │ ╞════════╪════════════╡══════════════╡ │ A ┆ [0.1, 0.2] │ [0.03, 0.14] │ = [0.1 * 0.3, 0.2 * 0.7] │ A ┆ [0.3, 0.4] │ [0.09, 0.28] │ = [0.3 * 0.3, 0.4 * 0.7] │ B ┆ [0.5, 0.6] │ [0.15, 0.42] │ = [0.5 * 0.3, 0.6 * 0.7] │ B ┆ [0.7, 0.8] │ [0.21, 0.56] │ = [0.7 * 0.3, 0.8 * 0.7] └────────┴────────────┘──────────────┘ 

Answer: Update: Broadcasting of literals/scalars for the List type was added in 1.10.0 df.with_columns(roc_wgt = pl.col.roc * weights)  shape: (4, 3) ┌────────┬────────────┬──────────────┐ │ symbol ┆ roc ┆ roc_wgt │ │ --- ┆ --- ┆ --- │ │ str ┆ list[f64] ┆ list[f64] │ ╞════════╪════════════╪══════════════╡ │ A ┆ [0.1, 0.2] ┆ [0.03, 0.14] │ │ A ┆ [0.3, 0.4] ┆ [0.09, 0.28] │ │ B ┆ [0.5, 0.6] ┆ [0.15, 0.42] │ │ B ┆ [0.7, 0.8] ┆ [0.21, 0.56] │ └────────┴────────────┴──────────────┘   As of Polars 1.8.0 list arithmetic has been merged. Follow on work will add support for broadcasting of literals (and scalars).  https://github.com/pola-rs/polars/issues/8006  It can be added as a column for now. (df.with_columns(wgt = weights) .with_columns(roc_wgt = pl.col.roc * pl.col.wgt) )  shape: (4, 4) ┌────────┬────────────┬────────────┬──────────────┐ │ symbol ┆ roc ┆ wgt ┆ roc_wgt │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ list[f64] ┆ list[f64] ┆ list[f64] │ ╞════════╪════════════╪════════════╪══════════════╡ │ A ┆ [0.1, 0.2] ┆ [0.3, 0.7] ┆ [0.03, 0.14] │ │ A ┆ [0.3, 0.4] ┆ [0.3, 0.7] ┆ [0.09, 0.28] │ │ B ┆ [0.5, 0.6] ┆ [0.3, 0.7] ┆ [0.15, 0.42] │ │ B ┆ [0.7, 0.8] ┆ [0.3, 0.7] ┆ [0.21, 0.56] │ └────────┴────────────┴────────────┴──────────────┘  Broadcasting of literals works for the Array datatype as of 1.8.0 dtype = pl.Array(float, 2) df.with_columns(roc_wgt = pl.col.roc.cast(dtype) * pl.lit(weights, dtype))  shape: (4, 3) ┌────────┬────────────┬───────────────┐ │ symbol ┆ roc ┆ roc_wgt │ │ --- ┆ --- ┆ --- │ │ str ┆ list[f64] ┆ array[f64, 2] │ ╞════════╪════════════╪═══════════════╡ │ A ┆ [0.1, 0.2] ┆ [0.03, 0.14] │ │ A ┆ [0.3, 0.4] ┆ [0.09, 0.28] │ │ B ┆ [0.5, 0.6] ┆ [0.15, 0.42] │ │ B ┆ [0.7, 0.8] ┆ [0.21, 0.56] │ └────────┴────────────┴───────────────┘
Question: I am facing a strange behaviour with my code. I don't understand why the subplot at the top left has a different space between the imshow and the colorbar compared to the subplot at the top right. And also I don't understand why the colorbar at the bottom is not aligned with the one at the top right. Can you explain this ? import matplotlib.pyplot as plt import numpy as np matrix = np.random.rand(100, 100) mosaic = "AB;CC" fig = plt.figure(layout="constrained") ax_dict = fig.subplot_mosaic(mosaic) img = ax_dict['A'].imshow(matrix, aspect="auto") fig.colorbar(img, ax=ax_dict['A']) img = ax_dict['B'].imshow(matrix, aspect="auto") fig.colorbar(img, ax=ax_dict['B']) img = ax_dict['C'].imshow(matrix, aspect="auto") fig.colorbar(img, ax=ax_dict['C']) plt.show()  

Answer: I posted this issue on Matplotlib github page and get an answer from jklymak. This is not the most elegant way to do it, but it works. However, if you want to place the colorbar at the bottom you have to adapt the code, updating the value in the inset_axes and also the orientation of the colorbar. img = ax['A'].imshow(matrix, aspect='auto') fig.colorbar(img, cax=ax['A'].inset_axes([1.02, 0, 0.05, 1])) img = ax['B'].imshow(matrix, aspect='auto') fig.colorbar(img, cax=ax['B'].inset_axes([1.02, 0, 0.05, 1])) img = ax['C'].imshow(matrix, aspect='auto') fig.colorbar(img, cax=ax['C'].inset_axes([1.02, 0, 0.05, 1]))