eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I checked python operator precedence (This one grammar is more detailed and more appropriate for the actual Python implementation)  Operators in the same box group left to right (except for exponentiation and conditional expressions, which group from right to left). ** Exponentiation [5] if – else Conditional expression  I can understand exponentiation that 2**3**2 is equal to 2**(3**2). But Conditional expression  conditional_expression ::= or_test ["if" or_test "else" expression]  is not one binary operator. I can't give one similar example as **. Could you give one example of "group from right to left" for if Conditional expression?

Answer: a if b else c if d else e means a if b else (c if d else e) and not (a if b else c) if d else e
Question: Using numpy, one can subset an array with one boolean array per dimension like: In [10]: aa = np.array(range(9)).reshape(-1, 3) In [11]: aa Out[11]: array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) In [12]: conditions = (np.array([True, True, False]), np.array([True, False, True])) In [13]: aa[np.ix_(*conditions)] Out[13]: array([[0, 2], [3, 5]])  Is there a way to do this in Pandas? I've looked in their docs https://pandas.pydata.org/docs/user_guide/indexing.html#boolean-indexing but didn't find it. (I would have posted 4 relevant links, but then the automatic question checks think I've posted code that is not properly formatted.) This https://github.com/pandas-dev/pandas/issues/11290 github issue is close, but I want to pick entire rows and columns.

Answer: You should be able to directly use boolean indexing with iloc (or loc): df = pd.DataFrame(aa) out = df.iloc[conditions]  Note that conditions should be a tuple of arrays/lists/iterables, if not you should convert it: out = df.iloc[tuple(conditions)]  Output:  0 2 0 0 2 1 3 5
Question: I have a DataFrame that looks something like this:    C1 C2     10 10   20 10   30 16   5 23   6 23   8 10   4 10   2 10    I would like to calculate the mean value in column C1 depending on the values in column C2. The mean value is to be calculated over all values in column C1 until the value in column C2 changes again. The result table should look like this:    C1 C2     15 10   30 16   5.5 23   4.67 10   

Answer: Use GroupBy.mean by consecutive values created by compared Series.shifted values with Series.cumsum, last remove first level and get original order of columns by DataFrame.reindex: out =(df.groupby([df['C2'].ne(df['C2'].shift()).cumsum(),'C2'],sort=False)['C1'] .mean() .droplevel(0) .reset_index() .reindex(df.columns, axis=1)) print (out) C1 C2 0 15.000000 10 1 30.000000 16 2 5.500000 23 3 4.666667 10  How it working: print (df.assign(compared=df['C2'].ne(df['C2'].shift()), cumsum=df['C2'].ne(df['C2'].shift()).cumsum())) C1 C2 compared cumsum 0 10 10 True 1 1 20 10 False 1 2 30 16 True 2 3 5 23 True 3 4 6 23 False 3 5 8 10 True 4 6 4 10 False 4 7 2 10 False 4  Thank you @ouroboros1 for another easier solution with GroupBy.agg: out = (df.groupby(df['C2'].ne(df['C2'].shift()).cumsum(), as_index=False) .agg({'C1': 'mean', 'C2': 'first'})) print (out) C1 C2 0 15.000000 10 1 30.000000 16 2 5.500000 23 3 4.666667 10
Question: I am trying to detect a large stain using OpenCV's SimpleBlobDetector following this SO answer. Here is the input image:  I first tried working with params.filterByArea = False, which allowed to detect the large black stain:  However, smaller spots also ended up being detected. I therefore toggled params.filterByArea = True hoping to enforce a criterion on object area. However, when setting params.filterByArea = True with params.minArea = 10 the largest stain is no longer identified:  I tried using other minArea parameters to no avail, even trying a minArea of 0, which should be equivalent to no filtering at all. What am I missing here?

Answer: opencv maxArea param has 5000 as default value. Increasing it could detect blobs bigger than that import cv2 import numpy as np def show_keypoints(im, keypoints): im_key = cv2.drawKeypoints(im, keypoints, np.array([]), (0, 0, 255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS) cv2.imshow("Keypoints", im_key) cv2.waitKey(0) im_path = "test.png" im_m = cv2.imread(im_path, cv2.IMREAD_GRAYSCALE) params = cv2.SimpleBlobDetector_Params() params.filterByArea = True params.minArea = 10000 params.maxArea = 50000 detector = cv2.SimpleBlobDetector_create(params) keypoints = detector.detect(im_m) show_keypoints(im_m, keypoints)  To print all detector params with default values use # a map suitable to get configuration from a yaml file param_map = {"blobColor": "blob_color", "filterByArea": "filter_by_area", "filterByCircularity": "filter_by_circularity", "filterByColor": "filter_by_color", "filterByConvexity": "filter_by_convexity", "filterByInertia": "filter_by_inertia", "maxArea": "max_area", "maxCircularity": "max_circularity", "maxConvexity": "max_convexity", "maxInertiaRatio": "max_inertia_ratio", "maxThreshold": "max_threshold", "minArea": "min_area", "minCircularity": "min_circularity", "minConvexity": "min_convexity", "minDistBetweenBlobs": "min_dist_between_blobs", "minInertiaRatio": "min_inertia_ratio", "minRepeatability": "min_repeatability", "minThreshold": "min_threshold", "thresholdStep": "threshold_step"} for k in param_map: print(f"{k:<21}: {params.__getattribute__(k)}")  result blobColor : 0 filterByArea : True filterByCircularity : False filterByColor : True filterByConvexity : False filterByInertia : False maxArea : 7000.0 maxCircularity : 3.4028234663852886e+38 maxConvexity : 3.4028234663852886e+38 maxInertiaRatio : 3.4028234663852886e+38 maxThreshold : 220.0 minArea : 500.0 minCircularity : 0.5 minConvexity : 0.20000000298023224 minDistBetweenBlobs : 10.0 minInertiaRatio : 0.10000000149011612 minRepeatability : 2 minThreshold : 50.0 thresholdStep : 10.0
Question: I have a dataframe whose index is a multiindex where axes[0] is the date, and axis[1] is the rank. Rank starts with 1 and ends at 100, but there can be a variable number of ranks in between as below. Here are the ranks dx = pd.DataFrame({ "date": [ pd.to_datetime('2025-02-24'), pd.to_datetime('2025-02-24'), pd.to_datetime('2025-02-24'), pd.to_datetime('2025-02-24'), pd.to_datetime('2025-02-25'), pd.to_datetime('2025-02-25'), pd.to_datetime('2025-02-25'), pd.to_datetime('2025-02-26'), pd.to_datetime('2025-02-26'), pd.to_datetime('2025-02-26'), pd.to_datetime('2025-02-26'), pd.to_datetime('2025-02-26') ], "rank": [0.0,1.0,2.0,100.0,0.0,1.0,100.0,0.0,1.0,2.0,3.0,100.0], "value": [2.3, 2.5, 2.4, 2.36, 2.165, 2.54, 2.34, 2.12, 2.32, 2.43, 2.4, 2.3] }) dx.set_index(["date", "rank"], inplace=True)  I want to plot this df, and df.plot() works fine creating a reasonable x-axis. However, I want to add a grid or vertical lines at all the rank=1, and all the rank=100(different color). I tried this :  fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(30, 5)) dx.plot(ax=axes[0]) axes[0].tick_params('x', labelrotation=90) xs = [x for x in dx.index if x[1]==0] for xc in xs: axes[0].axvline(x=xc, color='blue', linestyle='-')  but get this error: ConversionError: Failed to convert value(s) to axis units: (Timestamp('2025-02-24 00:00:00'), 0.0)  I also want to only show x labels for rank=0, and not all of them. Currently, if i set label rotation to 90, it results in that but not sure if this is the best way to ensure that. axes[0].tick_params('x', labelrotation=90)  So looking for 2 answers  How to set vertical lines at specific points with this type of multiindex How to ensure only certain x labels show on the chart 

Answer: With a categorical axis, plt will use an integer index "under the hood". Here, since you are using a lineplot, it tries to come up with a reasonable step: dx.plot(ax=axes[0]) axes[0].get_xticks() # array([-2., 0., 2., 4., 6., 8., 10., 12.])  With a barplot, you would get the more logical: dx.plot.bar(ax=axes[0]) axes[0].get_xticks() # array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])  You can use Axes.set_xticks and Axes.set_xticklabels to fix this. E.g., ticks = range(len(dx)) # only label at rank 0 labels = [f"{x[0].strftime('%Y-%m-%d')}, {int(x[1])}" if x[1] == 0 else '' for x in dx.index] axes[0].set_xticks(ticks=ticks) axes[0].set_xticklabels(labels=labels, rotation=90)  It's easier to see now that we need the appropriate index matches for Axes.axvline. We can apply np.nonzero to Index.get_level_values and then add the lines in a loop: def add_vlines(ax, rank, color): indices = np.nonzero(dx.index.get_level_values('rank') == rank)[0] for index in indices: ax.axvline(x=index, color=color, linestyle='dotted') add_vlines(axes[0], 1, 'blue') add_vlines(axes[0], 100, 'red')  Output:
Question: I would like to implement something similar to this OCaml in Python: let example = fun v opt_n -> let fltr = fun i -> i mod 2 = 0 in let fltr = match opt_n with | None -> fltr | Some n -> fun i -> (i mod n = 0 && fltr n) in fltr v  This is easily composable/extendable, I can add as many predicates as I want at runtime. This is of course a simplified example, in real life I have many optional inclusion/exclusion sets, and predicate checks for membership. Doing this the naive way in Python fails: def example(v: int, opt_n=None): """ doesn't work! """ # doesn't need to be a lambda, an explicitely defined function fails too fltr = lambda i: i % 2 == 0 if opt_n is not None: # fails miserably -> maximum recursion depth exceeded fltr = lambda i: fltr(i) and i % opt_n == 0 return fltr(v) example(10, 5)  This is annoying because it seems that since fltr can only appear once on the left side of the assignment, I have to inline the initial fltr in every case afterward: def example(v: int, opt_n=None, opt_m=None): """annoying but works""" fltr = None # some inital filters pred_0 = lambda _: True # do some real checks ... pred_1 = lambda _: True # do some real checks ... if opt_n is not None: # fltr is inlined, only appears on left side, now it works fltr = lambda i: pred_0(i) and pred_1(i) and opt_n % 2 == 0 if opt_m is not None: # much repetition fltr = lambda i: pred_0(i) and pred_1(i) and opt_n % 3 == 0 if fltr is None: # inlined again fltr = lambda i: pred_0(i) and pred_1(i) return fltr(v)  Is there any way to fix my mess, maybe I am missing something, and/or what is the recommended way to compose predicates in Python?

Answer: When you write fltr = lambda i: fltr(i) and i % opt_n == 0  fltr remains a free variable inside the lambda expression, and will be looked up when the function is called; it's not bound to the old definition of fltr in place when you evaluate the lambda expression. You need some way to do early binding; one option is to bind the current value of fltr to a new variable that's local to the lambda expression, namely another parameter: fltr = lambda i, cf=fltr: cf(i) and i % opt_n == 0  Not the cleanest solution. If you don't mind an additional function call, you can define an explicit composition function: def pred_and(f, g): return lambda x: f(x) and g(x)  then use that to compose the old fltr with another predicate to produce a new filter. def example(v: int, opt_n=None): fltr = lambda i: i % 2 == 0 if opt_n is not None: # fails miserably -> maximum recursion depth exceeded fltr = pred_and(fltr, lambda i: i % opt_n == 0) return fltr(v))  (I don't know OCaml very well, but pred_and is somewhat like the use of an applicative functor in Haskell, e.g. pred_and = liftA2 (&&).)
Question: Consider the following DB: from sqlalchemy import String, select, create_engine from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, Session class Base(DeclarativeBase): pass class User(Base): __tablename__ = "user_account" name: Mapped[str] = mapped_column(String(30)) surname: Mapped[str] = mapped_column(String(30)) def __repr__(self): return f"User(name={self.name!r}, surname={self.surname!r})" engine = create_engine("sqlite+pysqlite:///test3.sqlite3", echo=True) Base.metadata.create_all(engine) with Session(engine) as session: user1 = User( name="Mario", surname="Rossi" ) user2 = User( name="Mario", surname="Bianchi", ) user3 = User( name="Giovanni", surname="Bianchi", ) session.add_all([user1, user2, user3]) session.commit()  Now suppose I have a list of users I want to find: users = [("Mario", "Rossi"), ("Giovanni", "Bianchi")]  Then I would run: names = [name for name, _ in users] surnames = [surname for _, surname in users] with Session(engine) as session: stmt = select(User).where(User.name.in_(names)).where(User.surname.in_(surnames)) print(session.execute(stmt).scalars().all())  which returns: [User(name='Mario', surname='Rossi'), User(name='Mario', surname='Bianchi'), User(name='Giovanni', surname='Bianchi')]  but "Mario Bianchi" was not in the list of input users I had. How can I concatenate IN statements in order to select only pairwise correspondence? That is, if I have varA IN (el1, el2) AND varB IN (el3, el4), I do not wat to select entries with varA==el1 AND varB==el4

Answer: You need to use more explicit OR Boolean condition sqlalchemy.or_(conditions) ensures that each (name, surname) pair is checked explicitly. from sqlalchemy import or_ with Session(engine) as session: stmt = select(User).where( or_(*[(User.name == name) & (User.surname == surname) for name, surname in users]) ) print(session.execute(stmt).scalars().all())  This works across all databases, including SQLite. PostgreSQL-Specific Optimization If you were using PostgreSQL, we can use tuple_ for a more elegant and optimized query: from sqlalchemy.sql.expression import tuple_ with Session(engine) as session: stmt = select(User).where(tuple_(User.name, User.surname).in_(users)) print(session.execute(stmt).scalars().all())
Question: Currently, I want to change the Name of the Pico 2W device displayed on the nRF Connect app. I am using Micro Pico in VsCode. I tried many things but the Name is always "N/A". import bluetooth from time import sleep bluetooth.BLE().active(True) ble = bluetooth.BLE() device_name = 'PicoW_Device' advertising_data = b'\x02\x01\x06\x03\x03\x0F\x18\x09' + bytes(device_name, 'utf-8') ble.gap_advertise(100, adv_data=advertising_data) print(f"The Name of the device is currently: {device_name}") while True: sleep(1)  I tried to change the name, but it always displayed "N/A"

Answer: The problem is how you constructed the advertising_data. Try this device_name = 'PicoW_Device' advertising_data = bytearray() advertising_data += b'\x02\x01\x06' # Flags advertising_data += bytearray([1 + len(device_name), 0x09]) # Name length and Complete Local Name AD type advertising_data += device_name.encode('utf-8') # Ensures the name is encoded into bytes ble.gap_advertise(100, adv_data=bytes(advertising_data)) # Converts the bytearray to bytes
Question: I am learning asyncio callbacks. My task is- I have a message dict, message codes are keys, message texts are values. In coro main I have to create a number of asynchronous tasks (in my case 3 tasks), each task wraps a coro which prints one message. Also I have to add a callback to each task. Callback must print a code associated with a message printed by the coroutine wrapped by the task. The question is- how to pass code to callback? The staright solution is to add name to each task with the value of a code, but I dont want to go this way. I decided to use ContextVar for this purpose. So I create a global context variable, set() the value to the variable equal to code. Then I try to get() the context variable value from a callback but receive an Exception LookupError: <ContextVar name='msg_code' at 0x000001C596D94F40>. That's my code: import asyncio from contextvars import ContextVar msg_dict = { 'code1': 'msg1 by code1', 'code2': 'msg2 by code2', 'code3': 'msg3 by code3' } msg_code = ContextVar('msg_code') async def print_msg(code): await asyncio.sleep(0.5) msg_code.set(code) print(f'Message: {msg_dict[code]}') def callback_code(*args): code = msg_code.get() print(f'Code: {code}') async def main(): tasks = [asyncio.create_task(print_msg(code)) for code in msg_dict.keys()] [task.add_done_callback(callback_code) for task in tasks] await asyncio.gather(*tasks) asyncio.run(main())  I found that add_done_callback() also has keyword argument context= but I can't find any examples of how to pass task's context to a callback.

Answer: No tricks are needed, just specify the context. async def main(): tasks = [] for code in msg_dict: ctx = copy_context() # note: import copy_context from contexvars task = asyncio.create_task(print_msg(code), context=ctx) task.add_done_callback(callback_code, context=ctx) tasks.append(task) await asyncio.gather(*tasks)
Question: I'm facing an issue while migrating from MoviePy v1 to MoviePy v2. In v1, I could apply a transparency mask to an ImageClip, making certain areas of the clip transparent. However, in MoviePy v2, the same approach doesn't seem to work. Expected Behavior The ImageClip should become transparent in areas defined by the mask. Observed Behavior In MoviePy v2, the mask is just overlayed on top of image. Working Code (MoviePy v1) This code works perfectly fine in MoviePy 1.0.3:  from moviepy.editor import ImageClip, CompositeVideoClip image_clip = ImageClip("mask.png").set_duration(5) mask_clip = ImageClip("blueDark.png", ismask=True).set_duration(5) image_clip = image_clip.set_mask(mask_clip) video = CompositeVideoClip([image_clip], bg_color=(90, 90, 90)) video.preview()  Non-Working Code (MoviePy v2) Here’s the MoviePy v2 equivalent, where I updated the function and parameter names according to the new version: import moviepy as mpy image_clip = mpy.ImageClip("mask.png").with_duration(5) mask_clip = mpy.ImageClip("blueDark.png", is_mask=True).with_duration(5) image_clip = image_clip.with_mask(mask_clip) video = mpy.CompositeVideoClip([image_clip], bg_color=(90, 90, 90)) video.preview()  Additional Details Versions Installed: MoviePy v1: pip install moviepy==1.0.3 pygame MoviePy v2: pip install -U moviepy I have attached both mask.png and blueDark.png in case someone wants to reproduce the issue. blueDark.png  mask.png 

Answer: Turns out it works when I use RGBA image instead of greyscale image for the mask. Even though MoviePy docs explicitly mention that mask should always be greyscale image, but it seems to work with RGBA images only.
Question: I want to build a Django template hierarchy like so:  root.html |_ root-dashboard.html |_ root-regular.html  root.html shall have an if statement: {% if style == "dashboard" %} {# render some elements in a certain way #} {% else %} {# render those elements in a different way #} {% endif %}  And root-dashboard.html and root-regular.html should individually extend root.html by setting style: # root-dashboard.html {% extend 'root.html' with style='dashboard'%} # root-regular.html {% extend 'root.html' with style='regular'%}  (with above is not an actual valid syntax, its just something similar I want) And a view can use either root-dashboard.html or root-regular.html to show the content in one style or the other. How do I achieve this without the view having to set the style context?

Answer: Define a {% block … %} template tag [Django-doc] instead. In root.html, you don't use an if, but: {% block render_item %} {# render those elements in a different way #} {% endblock %}  then in your root-dashboard.html, you use: # root-dashboard.html {% extend 'root.html' %} {% block render_item %} {# render some elements in a certain way #} {% endblock %}  The idea is similar to the dynamic binding concept [wiki] in object-oriented programming, and is usually better than using if conditions: the latter is not extensible, and thus limits later modifications.
Question: I am in a situation where I have some time series data, potentially looking like this: { "t": [1, 2, 5, 6, 7], "y": [1, 1, 1, 1, 1], }  As you can see, the time stamp jumps from 2 to 5. For my analysis, I would like to fill in zeros for the time stamps 3, and 4. In reality, I might have multiple gaps with varying lengths. I'd like to fill this gap for all other columns. I'd also really like to keep my data in a LazyFrame since this is only one step in my pipeline. I don't think that .interpolate is really addressing my issue, nor is fill_null helpful here. I managed to achieve what I want, but it looks too complex:  # Dummy, lazy data. lf = pl.LazyFrame( { "t": [1, 2, 5, 6, 7], "y": [1, 1, 1, 1, 1], } ) lf_filled = lf.join( pl.Series( name="t", values=pl.int_range( start=lf.select("t").first().collect().item(0, 0), end=lf.select("t").last().collect().item(0, 0) + 1, eager=True, ), ) .to_frame() .lazy(), on="t", how="right", ).fill_null(0)  The output is correct and I am never collecting any more data than the two values needed for start and end. This looks like there should be a better way to do this. Happy to hear other suggestions :)

Answer: I think your approach is sensible, there's just no need for an intermediate collect: lf.join( lf.select(pl.int_range(pl.col.t.first(), pl.col.t.last()+1)), on="t", how="right" ) .fill_null(0)  An alternate approach that might be a bit more efficient is to use an asof-join with no tolerance: lf.select(pl.int_range(pl.col.t.first(), pl.col.t.last()+1)) .join_asof(lf, on="t", tolerance=0) .fill_null(0)
Question: url ="https://www.dsebd.org/top_20_share.php" r =requests.get(url) soup = BeautifulSoup(r.text,"lxml") table = soup.find("table",class_="table table-bordered background-white shares-table") top = table.find_all("th") header = [] for x in top: ele = x.text header.append(ele) df = pd.DataFrame(columns= header) print(df) row1 = table.find_all("tr") row2 =[] for r1 in row1[1:]: ftd= r1.find_all("td")[1].find("a", class_="ab1").text.strip() data=r1.find_all("td")[1:] r2 = [ele.text for ele in data] r2.insert(0,ftd) l= len(df) df.loc[l]= r2 print(df) df.to_csv("top_20_share_value_22.csv")  How to see / make data visible after converting to csv and view via excel? I have gone through above mentioned code.

Answer: The script itself works and CSV can be easily imported into Excel. Alternatively, export the data directly .to_excel('your_excle_file.xlsx') and open it in Excel.  Since you are already operating with pandas, just use pandas.read_html which BeautifulSoup uses in the background to scrape the tables. import pandas as pd df_list = pd.read_html('https://www.dsebd.org/top_20_share.php', match='TRADING CODE') # first table only df_list[0].to_csv("first_top_20_share_value_22.csv") # all three tables in concat pd.concat(df_list, ignore_index=True).to_csv("all_top_20_share_value_22.csv")
Question: I'm reading lines from a file and trying to match them with regex, but it's failing despite the regex matcher looking right. When comparing the line to what it should be as a string declaration, python is saying that they are not equal. It's looking like this is being caused by a non utf-8 encoding on my file but not sure how to fix this as I'm not sure exactly which encoding is being used. This is a simplified version of the code I'm using to debug: fp = open('tree.txt', 'r') lines = [line.strip() for line in fp.readlines()] fp.close() for line in lines: print(f'|{line}| vs |{line}|') print(line == "[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT") print(line.encode('utf-8')) print("[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT".encode('utf-8'))  My output once scanning the line in the file I'm expecting looks like this |[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT| vs |[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT| False b'\x00[\x00I\x00N\x00F\x00O\x00]\x00 \x00i\x00o\x00.\x00j\x00i\x00t\x00p\x00a\x00c\x00k\x00:\x00m\x00o\x00d\x00u\x00l\x00e\x002\x00:\x00j\x00a\x00r\x00:\x002\x00.\x000\x00-\x00S\x00N\x00A\x00P\x00S\x00H\x00O\x00T\x00' b'[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT'  I'm generating tree.txt by doing mvn dependency:tree > tree.txt on Windows 11 from VSCode terminal, if that's any clue to what kind of encoding is being used lol. Is there a way to convert line into a string with this utf-8 encoding b'[INFO] io.jitpack:module2:jar:2.0-SNAPSHOT', agnostic of its current encoding? I did try opening the file with fp = open('tree.txt', 'r', encoding='utf-8') but that did not work.

Answer: The pattern of nulls in the output says that this file is encoded in big-endian UTF-16. Open it with encoding='utf-16be'. You might also want to figure out why Maven is producing output in UTF-16.