eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am testing whether OSMesa functions properly, but I encountered the following error. How can this error be resolved? The full error message: --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[1], line 2 1 from OpenGL import GL ----> 2 from OpenGL import osmesa 4 ctx = osmesa.OSMesaCreateContext(osmesa.OSMESA_RGBA, None) 5 if ctx: File ~/anaconda3/envs/flame/lib/python3.8/site-packages/OpenGL/osmesa/__init__.py:2 1 from OpenGL.raw.osmesa._types import * ----> 2 from OpenGL.raw.osmesa.mesa import * File ~/anaconda3/envs/flame/lib/python3.8/site-packages/OpenGL/raw/osmesa/mesa.py:34 30 OSMesaGetCurrentContext = _p.GetCurrentContext 32 @_f 33 @_p.types(OSMesaContext,GLenum, OSMesaContext) ---> 34 def OSMesaCreateContext(format,sharelist): pass 36 @_f 37 @_p.types(OSMesaContext,GLenum, GLint, GLint, GLint, OSMesaContext) 38 def OSMesaCreateContextExt(format, depthBits, stencilBits,accumBits,sharelist ): pass File ~/anaconda3/envs/flame/lib/python3.8/site-packages/OpenGL/raw/osmesa/mesa.py:9, in _f(function) 7 def _f( function ): 8 return _p.createFunction( ----> 9 function,_p.PLATFORM.OSMesa, 10 None, 11 error_checker=None 12 ) AttributeError: 'GLXPlatform' object has no attribute 'OSMesa'  Minimal reproducible example: from OpenGL import osmesa ctx = osmesa.OSMesaCreateContext(osmesa.OSMESA_RGBA, None)  My environment as follows: pyrender==0.1.45 OpenGL==3.1.0 

Answer: To resolve this, the environment variable PYOPENGL_PLATFORM should be set to osmesa before importing pyrender. The pyrender documentation suggests to do this either in the shell when executing the program: PYOPENGL_PLATFORM=osmesa python render.py  or by adding the following lines at the top of the program: import os os.environ["PYOPENGL_PLATFORM"] = "osmesa"  Otherwise, it is as the error indicates: the GLX platform doesn't provide OSMesa functionality.  Full code example: import os os.environ["PYOPENGL_PLATFORM"] = "osmesa" from OpenGL import osmesa ctx = osmesa.OSMesaCreateContext(osmesa.OSMESA_RGBA, None) if ctx: print("OSMesa is working correctly")
Question: I have a numpy array of numpy arrays and I need to add a leading zero to each of the inner arrays: a = [[1 2] [3 4] [5 6]] --> b = [[0 1 2] [0 3 4] [0 5 6]] Looping through like this: for item in a: item = np.insert(item, 0, 0) doesn't help. Numpy.put() flattens the array, which I don't want. Any suggestions how I accomplish this? Thanks

Answer: np.insert - insert values along the given axis before the given indices. If axis is None then array is flattened first. import numpy as np a = np.array([[1, 2], [3, 4], [5, 6]]) b = np.insert(a, 0, 0, axis=1) print(b)  Result: [[0 1 2] [0 3 4] [0 5 6]]
Question: It is a pretty common occurrence to have leading and trailing NaN values in a table or DataFrame. This is particularly true after joins and in timeseries data. import numpy as np import pandas as pd df1 = pd.DataFrame({ 'a': [1, 2, 3, 4, 5, 6, 7], 'b': [np.NaN, 2, np.NaN, 4, 5, np.NaN, np.NaN], }) Out[0]: a b 0 1 NaN 1 2 2.0 2 3 NaN 3 4 4.0 4 5 5.0 5 6 NaN 6 7 NaN  Let's remove these with dropna. df1.dropna() Out[1]: a b 1 2 2.0 3 4 4.0 4 5 5.0  Oh no!! We lost all the missing values that showed in column b. I want to keep the middle (inner) ones. How do I strip the rows with leading and trailing NaN values in a quick, clean and efficient way? The results should look as follows: df1.stripna() # obviously I'm not asking you to create a new pandas method... # I just thought it was a good name. Out[3]: a b 1 2 2.0 2 3 NaN 3 4 4.0 4 5 5.0   Some of the answers so far are pretty nice but I think this is important enough functionality that I raised a feature request with Pandas here if anyone is interested. Let's see how it goes!

Answer: Another way, that might be a little more readable is using the pd.Series.first_valid_index and pd.Series.last_valid_index with index slicing using loc: df1.loc[df1['b'].first_valid_index():df1['b'].last_valid_index()]  Output:  a b 1 2 2.0 2 3 NaN 3 4 4.0 4 5 5.0  And, this should be really fast. Using @LittleBobbyTables input dataframe. %timeit df1.loc[df1['b'].ffill().notna()&df1['b'].bfill().notna()]   24.2 ms ± 610 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  verus: %timeit df1.loc[df1['b'].first_valid_index():df1['b'].last_valid_index()]   1.43 ms ± 34.3 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
Question: I am super confused how neither of these work. Can someone help me understand what's going on and why it prints "BAD" and "Value does not exist"? from enum import Enum class EventType(Enum): USER_LOGIN = 1, USER_LOGOUT = 2, @classmethod def has_value(cls, value): return value in cls._value2member_map_ eventType = 2 if not EventType.has_value(eventType): print("BAD") else: print("GOOD") if eventType in EventType.__members__.values(): print("Value exists") else: print("Value does not exist") 

Answer: As @msanford said in the comments, remove the trailing commas from your values -- they are creating tuples.
Question: I am trying to calculate the RLC tank transfer function using LCAPY. I know what the answer should be but I want to do it with code. The issue is there isn't a current() or voltage() function in LCAPY and only impedance() and transfer(). Here is the code so far: import lcapy from lcapy import s, expr from IPython.display import display, Math from sympy import symbols, collect, sqrt import sympy as sp # Define the circuit netlist with components netlist = """ Iin 1 0 Cp 1 0 C Rp 1 0 R Lp 1 0 L """ # Create a Circuit object from the netlist circuit = lcapy.Circuit(netlist) s_model = circuit.s_model() # Calculate the transfer function H = I_out / I_in # H(s) = transfer from node 1 (current source) to ground (node 0) # Calculate the output current I_out I_out = s_model.current(1, 0) # Current through resistor Rp at node 1 # Calculate the input current I_in from the current source I_in = s_model.Iin(1) # Current from the current source at node 1 # H(s) = I_out / I_in H_final = I_out / I_in # Display the transfer function H display(Math(sp.latex(H_final)))  I was hoping to get the following: 

Answer: You don't need a voltage here, since both currents will be proportional to it. You just need the resistance (effectively the impedance of the resistor) and the total impedance (resistor+capacitor+inductor) between the two lines. If you want your frequency response then you will need to specify this explicitly. Since they end at the same nodes and I(in)=I(total), writing Z for combined impedance of resistor, capacitor, inductor: V = I(total).Z = I(resistor).R  Hence, H = abs( I(resistor) / I(total) ) = abs( Z / R )  Run the following in, e.g., a jupyter notebook: import lcapy from lcapy import s, j, omega from IPython.display import display, Math, Latex import sympy as sp # Define the circuit netlist with components netlist = """ Iin 1 0 Cp 1 0 C Rp 1 0 R Lp 1 0 L """ circuit = lcapy.Circuit(netlist) S = circuit.s_model() # Calculate the transfer function # H = S.resistance( 1, 0 ) / S.impedance( 1, 0 ) # Note that resistance() actually returns abs(Z*Z)/R H = S.impedance( 1, 0 ) / circuit.Rp.R # H = I(resistor) / I(total) = Z(total) / Z(resistor) A = H( j * omega ).magnitude display(Math(sp.latex( A )))  Output:  Note: in the (commented-out) version of H, resistance() does not return R; it appears to actually return what I would call abs(Z2)/R, where Z is the complex impedance (which is returned correctly); hence the transfer function looks "the wrong way up".
Question: np.where is great for pandas since it's a vectorized way to change the column based on a condition. But while it seems to work great with np native types, it doesn't play nice with dates. This works great: >>> df1 = pd.DataFrame([["a", 1], ["b", np.nan]], columns=["name", "num"]) >>> df1 name num 0 a 1.0 1 b NaN >>> np.where(df1["num"] < 2, df1["num"], np.nan) array([ 1., nan])  But this doesn't: >>> df2 = pd.DataFrame([["a", datetime.datetime(2024,1,2)], ["b", np.nan]], columns=["name", "date"]) >>> df2 name date 0 a 2024-01-02 1 b NaT >>> np.where(df2["date"] < datetime.datetime(2024,3,1), df2["date"], np.nan) Traceback (most recent call last): File "<python-input-10>", line 1, in <module> np.where(df2["date"] < datetime.datetime(2024,3,1), df2["date"], np.nan) ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ numpy.exceptions.DTypePromotionError: The DType <class 'numpy.dtypes.DateTime64DType'> could not be promoted by <class 'numpy.dtypes._PyFloatDType'>. This means that no common DType exists for the given inputs. For example they cannot be stored in a single array unless the dtype is `object`. The full list of DTypes is: (<class 'numpy.dtypes.DateTime64DType'>, <class 'numpy.dtypes._PyFloatDType'>) >>>  What is the proper vectorized way to do the latter operation?

Answer: The best answer is to use Series.where: df2['out'] = df2['date'].where(df2["date"] < datetime.datetime(2024,3,1))  As a second best answer, you can use NaT. numpy.where returns an array with a single dtype, you should not use NaN as an empty value but NaT: np.where(df2["date"] < datetime.datetime(2024,3,1), df2["date"], pd.NaT)  Output: array([1704153600000000000, NaT], dtype=object) Note the the output values are integers (datetimes are integers). If you want to assign a column: df2['out'] = pd.to_datetime(np.where(df2["date"] < datetime.datetime(2024,3,1), df2["date"], pd.NaT))  Output:  name date out 0 a 2024-01-02 2024-01-02 1 b NaT NaT
Question: Consider the following example: import polars as pl pl.DataFrame(pl.Series("x", ["1, 0", "2,3", "5 4"])).with_columns( pl.col("x").str.split(",").list.eval(pl.element().str.split(" ")) )  shape: (3, 1) ┌────────────────────┐ │ x │ │ --- │ │ list[list[str]] │ ╞════════════════════╡ │ [["1"], ["", "0"]] │ │ [["2"], ["3"]] │ │ [["5", "4"]] │ └────────────────────┘  I want to flatten the elements of the column, so instead of being a nested list, the elements are just a list. How do I do that?

Answer: You can use Expr.explode(), Expr.list.explode(), or Expr.flatten() to return one row for each list element, and using it inside of Expr.list.eval() lets you expand each row's nested lists instead of exploding the series itself. import polars as pl df = pl.DataFrame(pl.Series("x", ["1, 0", "2,3", "5 4"])) print(df.with_columns( pl.col("x") .str.split(",") .list.eval(pl.element().str.split(" ")) .list.eval(pl.element().flatten()) )) # You can also combine it with your existing .list.eval() print(df.with_columns( pl.col("x") .str.split(",") .list.eval(pl.element().str.split(" ").flatten()) ))
Question: Let's say I have a plot that looks like so: import numpy as np df = sns.load_dataset('iris') dfm = pd.melt(df, id_vars=["species"]) dfm = dfm.query('variable in ["sepal_length", "sepal_width"]') sns.stripplot(data=dfm, x="species", y="value", hue="variable", dodge=True) plt.legend(bbox_to_anchor=(1.05, 1), loc=2)   Let's also say there's another column with important info in my data, such as "potency". dfm['potency'] = np.random.randint(1, 6, dfm.shape[0]) I would like to highlight the potency corresponding to each point in my plot with darkening colors (high potency -> darker). Is this possible? I know that hue='potency' would do just this but then I could not use dodge to separate the data into sepal_width and and sepal_length chunks.

Answer: You could overlay multiple stripplots with different alphas: start, end = dfm['potency'].agg(['min', 'max']) for k, v in dfm.groupby('potency'): sns.stripplot(data=v.assign(variable=v['variable']+f' / potency={k}'), x="species", y="value", hue="variable", dodge=True, alpha=(k-start+1)/(end-start+1) ) plt.legend(bbox_to_anchor=(1.05, 1), loc=2)  Output:  A variant with multiple colors in a paired palette: pal = sns.color_palette("Paired") for i, (k, v) in enumerate(dfm.groupby('potency')): sns.stripplot(data=v.assign(variable=v['variable']+f' / potency={k}'), x="species", y="value", hue="variable", dodge=True, palette=pal[2*i:2*(i+1)] ) plt.legend(bbox_to_anchor=(1.05, 1), loc=2)  Output:
Question: My code is as follows: with concurrent.futures.ThreadPoolExecutor() as executor: length = len(self.seq) futures = [None] * length results = [] for i in range(length): f,args,kwargs = self.seq[i] future = executor.submit(f, *args, **kwargs) futures[i] = future for f in concurrent.futures.as_completed(starts): results.append(f.result())  I'd like the return values in results to be ordered by the order of the functions they were returned from in self.seq, but my code orders them in the order of whichever returns first. How can I do this? For clarification, .as_completed() is not what I'm looking for. I want to maintain the order of the original function list.

Answer: The simplest answer is to pass a counter argument along with the rest of your args and kwargs, where you set counter to be i. You then need to modify f to return the counter along with whatever other values its supposed to return. Your results can then be sorted by the counter. Alternatively, create a results array of the right size, and as each result comes back, the counter value tells you where to put it in the array.  If you do not have access to f, then just write a wrapper around it: def f_wrapper(*args, *, counter, **kwargs): result = f(*args, **kwargs) return counter, result  Likewise, you'll write future = executor.submit(f_wrapper, *args, **kwargs | dict(counter=i))
Question: I have a pandas dataframe with a group structure where the value of interest, val, is guaranteed to be sorted within the group. However, there are missing values in val which I need to bound. The data I have looks like this: group_id id_within_group val 1 1 3.2 1 2 4.8 1 3 5.2 1 4 NaN 1 5 7.5 2 1 1.8 2 2 2.8 2 3 NaN 2 4 5.4 2 5 6.2  I now want to create a lower bound, max_prev which is the maximum value within the group for the rows before the current row, whereas min_next is the minimum value within the group for the rows after the current row. It is not possible to just look one row back and ahead, because there could be clusters of NaN. I don't need to take care of the edge cases of the first and last row within group. The desired output would hence be group_id id_within_group val max_prev min_next 1 1 3.2 NaN 4.8 1 2 4.8 3.2 5.2 1 3 5.2 4.8 7.5 1 4 NaN 5.2 7.5 1 5 7.5 5.2 NaN 2 1 1.8 NaN 2.8 2 2 2.8 1.8 5.4 2 3 NaN 2.8 5.4 2 4 5.4 2.8 6.2 2 5 6.2 5.4 NaN  How can I achieve this in a reasonable fast way?

Answer: You could use a custom groupby.transform with ffill/bfill+shift: g = df.groupby('group_id')['val'] df['max_prev'] = g.transform(lambda x: x.ffill().shift()) df['min_next'] = g.transform(lambda x: x[::-1].ffill().shift()) # or df['min_next'] = g.transform(lambda x: x.bfill().shift(-1))  If your values are not sorted, add a cummax/cummin: g = df.groupby('group_id')['val'] df['max_prev'] = g.transform(lambda x: x.ffill().cummax().shift()) df['min_next'] = g.transform(lambda x: x[::-1].ffill().cummin().shift())  Output:  group_id id_within_group val max_prev min_next 0 1 1 3.2 NaN 4.8 1 1 2 4.8 3.2 5.2 2 1 3 5.2 4.8 7.5 3 1 4 NaN 5.2 7.5 4 1 5 7.5 5.2 NaN 5 2 1 1.8 NaN 2.8 6 2 2 2.8 1.8 5.4 7 2 3 NaN 2.8 5.4 8 2 4 5.4 2.8 6.2 9 2 5 6.2 5.4 NaN
Question: I have some hard time understanding, how a 3D point from a scene can be translated into a 2d Image. I have created a scene in Blender where my camera is positioned at P_cam(0|0|0) at is looking between the x and y axis (rotation of x=90° and z=-45°). I spawned a test cube at pos_c (5|5|0). The img I see looks like this:  I now want to create the same img with native python. Therefore I use opencv to map 3D points to 2D points. I created my camera matrix (with (0|0) refering to the middle of the img) cx = width/2 #principal point is our image center cy = height/2 camera_matrix = np.array([[fx, 0, cx], [0, fy, cy], [0, 0, 1]], np.float32)  I then use openvcs projectPoints method to transform my cubes world coordinates into 2d img points. I apply no transformation in form of the tvec. But i know that i need the rotation vector and thats the tricky part for me. I tried to use the vector with the same values as i did in blender, but it did not work. cam_rot_in_deg = (90, 0, -45) cam_rot_in_rad = np.radians(cam_rot_in_deg) rvec = np.array([[cam_rot_in_rad]], np.float32).reshape((3,1))  The image I render looks like this:  Within in the image I also visualized the coordinate axis, so I can see that the cube is drawn at the correct position within the world, but my camera perspective is completly off. What am I missing? Thank you so much I tried numerous rotation angles, but I did not came accros the desired solution.

Answer: Your issue is here: cam_rot_in_deg = (90, 0, -45) cam_rot_in_rad = np.radians(cam_rot_in_deg) rvec = np.array([[cam_rot_in_rad]], np.float32).reshape((3,1))  OpenCV's "rvec" is an axis-angle representation, not Euler angles. What you did there is Euler angles. That's no good. Instead of rvecs (and tvecs), you can just deal with matrices. I'd recommend 4x4 for 3D affine transforms. They're trivial to compose using matrix multiplication, which is @ for numpy arrays. That way you can compose any individual primitive rotations around any axes you like. OpenCV has the cv.Rodrigues() function that converts back and forth between axis-angle vector and (3x3) rotation matrix. For convenience, you should define utility functions that build matrices for translation (translation along each axis), scaling (factor, or factor per dimension), rotation (from axis and angle). Another convenience function you'd probably need will take a 4x4 transform matrix apart into its rotation and translation parts. Since slicing the rotation part is trivial already, that function should also call Rodrigues because then the function can turn a transform matrix into rvec and tvec, which is commonly used within OpenCV. A function to do the forward thing (rtvec to 4x4) just does the opposite. Euler angles are generally unwieldy (gimbal lock), so they aren't used as a medium for anything, merely for initialization. They're confusing too, because everyone has their own conventions for the order of the rotations, and even which axes there are.
Question: I am running into an issue that saving a numpy array after indexing results in much slower saving. A minimal reproducible example can be seen below: import time import numpy as np def mre(save_path): array = np.zeros((245, 233, 6)) start = time.time() for i in range(1000): with open(save_path + '/array1_' + str(i), "wb") as file: np.save(file, array) end = time.time() print(f"No indexing: {end - start}s") array2 = array[:,:,[0,1,2,3,4,5]] start = time.time() for i in range(1000): with open(save_path + '/array2_' + str(i), "wb") as file: np.save(file, array2) end = time.time() print(f"With indexing: {end - start}s") print("Arrays are equal: ", np.array_equal(array, array2))  Which results in: No indexing: 2.9975574016571045s With indexing: 10.408239126205444s Arrays are equal: True  So according to numpy the arrays are equal, but still the resulting saving times are significantly slower. Does anyone have an idea as to why this is?

Answer: Have you tried to use the numpy.ascontiguousarray() function ? This function is useful when working with arrays that have a non-contiguous memory layout, as it can improve performance by ensuring that the data is stored in contiguous memory locations. Example array2 = np.ascontiguousarray(array[:,:,[0,1,2,3,4,5]])  Output No indexing: 6.80817985534668s With indexing (contiguous copy): 6.550203800201416s
Question: Let' say I have a value array values = np.array([0.0, 1.0, 2.0, 3.0, 4.0])  and an index array indices = np.array([0,1,0,2,2])  Is there a vectorized way to sum the values for each unique index in indices? I mean a vectorized version to get sums in this snippet: sums = np.zeros(np.max(indices)+1) for index, value in zip(indices, values): sums[index] += value  Bonus points, if the solution allows values (and in consequence sums)to be multi-dimensional. EDIT: I benchmarked the posted solutions: import numpy as np import time import pandas as pd values = np.arange(1_000_000, dtype=float) rng = np.random.default_rng(0) indices = rng.integers(0, 1000, size=1_000_000) N = 100 now = time.time_ns() for _ in range(N): sums = np.bincount(indices, weights=values, minlength=1000) print(f"np.bincount: {(time.time_ns() - now) * 1e-6 / N:.3f} ms") now = time.time_ns() for _ in range(N): sums = np.zeros(1 + np.amax(indices), dtype=values.dtype) np.add.at(sums, indices, values) print(f"np.add.at: {(time.time_ns() - now) * 1e-6 / N:.3f} ms") now = time.time_ns() for _ in range(N): pd.Series(values).groupby(indices).sum().values print(f"pd.groupby: {(time.time_ns() - now) * 1e-6 / N:.3f} ms") now = time.time_ns() for _ in range(N): sums = np.zeros(np.max(indices)+1) for index, value in zip(indices, values): sums[index] += value print(f"Loop: {(time.time_ns() - now) * 1e-6 / N:.3f} ms")  Results: np.bincount: 1.129 ms np.add.at: 0.763 ms pd.groupby: 5.215 ms Loop: 196.633 ms 

Answer: Another possible solution, which:  First, creates an array of zeros b with a length equal to the number of unique elements in the indices array  It then uses the np.add.at function to accumulate the values from the values array into the corresponding positions in b as specified by the indices array.   b = np.zeros(1 + np.amax(indices), dtype=values.dtype) np.add.at(b, indices, values)  Output: array([2., 1., 7.])
Question: Imagine I want to run this function: def main(): pass  at these scheduled times (not every random 3 hours): import schedule schedule.every().day.at("00:00").do(main) schedule.every().day.at("03:00").do(main) schedule.every().day.at("06:00").do(main) schedule.every().day.at("09:00").do(main) schedule.every().day.at("12:00").do(main) schedule.every().day.at("15:00").do(main) schedule.every().day.at("18:00").do(main) schedule.every().day.at("21:00").do(main) while True: schedule.run_pending() time.sleep(1)  How can I make this run main() immediately when the script is run at some time between the scheduled start times, say at 19:42. This works fine but if the something happens like the power goes out and I'm not there to restart the script and miss a scheduled run, I'd like it to execute the function as soon as the script is run again even if it's not at one of the scheduled times. This is Windows machine if it matters.

Answer: I think you need just to add main() call before the while loop, with check of current time import datetime # other schedules ... schedule.every().day.at("21:00").do(main) now = datetime.datetime.now() if 0 <= now.hour <= 20: main() # <-- this will be called once, at script start if current time is between 00:00 and 21:00 while True: schedule.run_pending() time.sleep(1)
Question: I have this function: async_session = contextvars.ContextVar("async_session") async def get_async_session() -> AsyncGenerator[AsyncSession, None]: async with async_session_maker() as session: try: _token = async_session.set(session) yield session finally: async_session.reset(_token)  This fails with: ValueError: <Token var=<ContextVar name='async_session' at 0x7e1470e00e00> at 0x7e14706d4680> was created in a different Context  How can this happen? AFAICT the only way for the Context to be changed is for a whole function call. So how can the context change during a yield? This function is being used as a FastAPI Depends in case that makes a difference - but I can't see how it does. It's running under Python 3.8 and the version of FastAPI is equally ancient - 0.54.

Answer: For anyone who comes after me, the actual problem is that my generator function is not async (the example in the question is misleading). FastAPI runs async and sync dependencies in different contexts to avoid the sync dependencies holding up the async loop thread, which is why it is then cleaned up in the wrong context.