eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I learn python asyncio and now struggling with Conditions. I wrote a simple code: import asyncio async def monitor(condition: asyncio.Condition): current_task_name = asyncio.current_task().get_name() print(f'Current task {current_task_name} started') async with condition: await condition.wait() print(f'Condition lock released for task {current_task_name}, start doing smthg') await asyncio.sleep(2) print(f'Current task {current_task_name} finished') async def setter(condition: asyncio.Condition): print(f'Setter starts') async with condition: await asyncio.sleep(1) condition.notify_all() # Пробуждаем все задачи print('Setter finished') print('Setter unlocked') async def main(): cond = asyncio.Condition() async with asyncio.TaskGroup() as tg: tg.create_task(setter(cond)) [tg.create_task(monitor(cond)) for _ in range(3)] # tg.create_task(setter(cond)) asyncio.run(main())  But the code catches a DeadLock after Setter unlocked message. I cant realise why. But if I swap task creation order in main and create monitor tasks prior to creation setter task- all good. please, I am a very new in asyncio, so please, in simple words...

Answer: In your original code, the setter task is started before the monitor tasks. When setter calls notify_all(), no monitor tasks are yet waiting so no tasks are notified. Later, the monitors reach condition.wait() but remain stuck because no further notifications are sent. When you reverse the order, starting monitor tasks first, they begin waiting on the condition before the setter notifies. As a result, they are properly awakened and the deadlock is avoided. To fix the issue, ensure that all tasks that should wait on the condition are created and running before calling notify_all().
Question: This script: import numpy as np a = np.linspace(-2.5, 2.5, 6, endpoint=True) b = np.vstack((a, a)).T c = np.array([2, 1]) print(b*c)  produces: [[-5. -2.5] [-3. -1.5] [-1. -0.5] [ 1. 0.5] [ 3. 1.5] [ 5. 2.5]]  which is my desired output. Can it be produced directly with from a and c? Trying a*c and c*a fails due to ValueError: operands could not be broadcast together error. Trying different dimensions of c, e.g. [[2, 1]] fails, too.  I found that the simplest way to go is to define a differently, i.e. a = np.linspace([-2.5, -2.5], [2.5, 2.5], 6, endpoint=True). I can now write a*c, a*(2, 1), etc. Since it changes my initial post, all answers below remain valid.

Answer: With the original (6,) and (2,) arrays, einsum may make the outer product more explicit: In [331]: a=np.linspace(-2.5, 2.5, 6, endpoint=True); c=np.array([1,2]) In [332]: np.einsum('i,j->ij',a,c) Out[332]: array([[-2.5, -5. ], [-1.5, -3. ], [-0.5, -1. ], [ 0.5, 1. ], [ 1.5, 3. ], [ 2.5, 5. ]])  np.outer(a,c) and np.multiply.outer(a,c) also do this.
Question: We can check if a unittest.mock.Mock has any call with some specified arguments. I now want to test that some of the arguments are correct, while I do not know about the other ones. Is there some intended way to do so? I could manually iterate over Mock.mock_calls, but is there a more compact way? import random from unittest import mock def mymethod(a, handler): handler(a, random.random()) def test_mymethod(): handler = mock.Mock() mymethod(5, handler) handler.assert_any_call(5, [anything]) # How do I achieve this? 

Answer: You can assert the wildcard argument to be unittest.mock.ANY, which has its __eq__ method overridden to unconditionally return True so that the object is evaluated equal to anything: from unittest.mock import ANY def test_mymethod(): handler = mock.Mock() mymethod(5, handler) handler.assert_any_call(5, ANY) # passes  Demo: https://ideone.com/8Kbywo
Question: I have a dataframe with weekly product sales    product_id week_number sales     A1 1 1000   A1 2 2000   A1 3 3000   A2 1 8000   A2 2 4000   A2 3 2000    I want to add a column that identifies rows where the total sales were the highest for the given product:    product_id week_number sales product_max     A1 1 1000 FALSE   A1 2 2000 FALSE   A1 3 3000 TRUE   A2 1 8000 TRUE   A2 2 4000 FALSE   A2 3 2000 FALSE    Since A1 had its highest sales on week 3, that row is tagged as True. But week 3 wasn't the highest for A2. (And so for A2, week 1 is tagged as True.) I know I could write a loop to do this and cycle through each of the product IDs one by one, but I am wondering if there is a way to do this with a different function - possibly Pandas Groupby? Thank you!

Answer: Answer df['product_max'] = ( df.groupby('product_id')['sales'].transform('max') .eq(df['sales']) )  df  product_id week_number sales product_max 0 A1 1 1000 False 1 A1 2 2000 False 2 A1 3 3000 True 3 A2 1 8000 True 4 A2 2 4000 False 5 A2 3 2000 False   Example Code import pandas as pd data = {'product_id': ['A1', 'A1', 'A1', 'A2', 'A2', 'A2'], 'week_number': [1, 2, 3, 1, 2, 3], 'sales': [1000, 2000, 3000, 8000, 4000, 2000]} df = pd.DataFrame(data)
Question: I have two numpy 'arrays': 1st is a 2D array with shape (N, 2) 2nd is a 3D "matrix" with shape (N, M, 2) where M can be different from 0 to N-1 Code example import numpy as np a = np.array([[1, 2], [3, 4]]) # (N=2,2) shape b = np.array([np.array([[1, 2]]), np.array([[1, 2], [3, 4]])], dtype=object) # (N=2,(1,2), 2) shape diff = [_ - b[i] for i, _ in enumerate(a)] # <-- this is an expected behavior diff1 = a - b print(diff) # <-- [array([[0, 0]]), array([[2, 2], [0, 0]])] print(diff1) # <-- [[array([[ 0, -1]]) array([[ 1, 0], [-1, -2]])] # [array([[ 2, 1]]) array([[ 3, 2], [ 1, 0]])]] ???  Is it possible to perform element-by-element subtraction in numpy (without loop or list comprehension)? It looks like I need something like np.subtract(a, b, axis=0) but np.subtract doesn't understand axis argument.  The answer of @Subir Chowdhury works fine. ~ real code with real M, N def std_dev_lc(flows: arrays): vectors: arrays = [flow.mean(axis=0) for flow in flows] std_dev = [np.linalg.norm(v - a, axis=1).mean() for v, a in zip(vectors, flows)] return np.array(vectors), np.array(std_dev) def std_dev_cs(flows: arrays): vectors: arrays = [flow.mean(axis=0) for flow in flows] flatten_flows = np.concatenate(flows, axis=0) vectors_rep = np.repeat(vectors, [len(flow) for flow in flows], axis=0) diff_flat = vectors_rep - flatten_flows norms_flat = np.linalg.norm(diff_flat, axis=1) norms = np.split(norms_flat, np.cumsum([len(flow) for flow in flows]))[:-1] return np.array(vectors), np.array([norm.mean() for norm in norms]) M, N = 30, 300 _flows: arrays = [randn(randint(1, M), 2).astype(np.float32) for _ in range(N)] print('1', timeit(lambda: std_dev_lc(_flows), number=500)) # <- 3.34 print('2', timeit(lambda: std_dev_cs(_flows), number=500)) # <- 2.54 

Answer: Another solution is using np.concatenate and np.split: import numpy as np a = np.array([[1, 2], [3, 4]]) b_list = [np.array([[1, 2]]), np.array([[1, 2], [3, 4]])] # Flatten `b` into a single array b_concat = np.concatenate(b_list, axis=0) a_repeat = np.repeat(a, [len(b) for b in b_list], axis=0) diff_flat = a_repeat - b_concat # Split back into original structure split_indices = np.cumsum([len(b) for b in b_list])[:-1] # Restore original structure diff_final = np.split(diff_flat, split_indices) print(diff_final)
Question: I am trying to optimize a matrix given some bounds and constraints. When I run the minimize function with only the bounds everything works fine, but when adding the constraints I get the following error: Exception has occurred: IndexError SLSQP Error: the length of bounds is not compatible with that of x0.  Below is a nonsense, but minimal, example of the code not working: from scipy.linalg import norm from scipy.optimize import minimize import numpy as np def objectiveFunction(guess, targetVector) -> float: guess.shape = (int(np.sqrt(guess.size)), int(np.sqrt(guess.size))) attempt = np.matmul(np.array(targetVector).transpose(), guess) assert(type(norm(attempt - targetVector)) == float) return norm(attempt - targetVector) def rowSumConstraint(guess) -> float: guess.shape = (int(np.sqrt(guess.size)), int(np.sqrt(guess.size))) guess = guess.sum(axis = 1) ones = np.ones(guess.size) assert(type(norm(guess - ones)) == float) return norm(guess - ones) size = 3 target = np.random.rand(size) initialGuess = np.random.rand(size, size).flatten() _bounds = tuple([(0,1) for _ in range(len(initialGuess))]) _constraints = [{'type': 'eq', 'fun': rowSumConstraint}] print(len(_bounds)) print(len(initialGuess)) res = minimize(fun = objectiveFunction, x0 = initialGuess, args = (target), bounds = _bounds, constraints = _constraints)  The console output from the print statements is reasonably both 9. The code runs fine using only the bounds, but when the constraints are added, the bounds throw an error. I am at a loss, thank you in advance. Edit: This was solved by letting all inputs to minimize be lists, and then converting them to arrays as necessary in the objective and constraint functions. I have no idea why this works or why it would matter to minimize.

Answer: The minimize() function generally assumes that when it evaluates f(x), then x is not modified. If you modify it, you may get strange behavior. This is a problem, because this modifies the argument passed to it: guess.shape = (int(np.sqrt(guess.size)), int(np.sqrt(guess.size)))  This is why removing the constraint helps - it evaluates the constraint while setting up the problem, which changes the value of x0. It's probably why converting the arrays to lists and back helps, because that introduces a copy. A simple way to fix this is to copy the array before modifying it. def objectiveFunction(guess, targetVector) -> float: guess = guess.copy() guess.shape = (int(np.sqrt(guess.size)), int(np.sqrt(guess.size))) attempt = np.matmul(np.array(targetVector).transpose(), guess) assert(type(norm(attempt - targetVector)) == float) return norm(attempt - targetVector) def rowSumConstraint(guess) -> float: guess = guess.copy() guess.shape = (int(np.sqrt(guess.size)), int(np.sqrt(guess.size))) guess = guess.sum(axis = 1) ones = np.ones(guess.size) assert(type(norm(guess - ones)) == float) return norm(guess - ones)
Question: I am building the MCP server application to connect some services to LLM . I use the MCP Python SDK https://github.com/modelcontextprotocol/python-sdk One of things i want to implement is authorisation of a user with the token.  I see it must be possible somehow. Most of tutorials about MCP are related to STDIO kind of a server run. My will be SSE. There is my code: from mcp.server.fastmcp import FastMCP from fastapi import FastAPI, Request, Depends, HTTPException app = FastAPI() mcp = FastMCP("SMB Share Server") @mcp.tool() def create_folder(parent_path: str, name: str) -> str: """Create new subfolder in the specified path""" return f"Folder {name} created in {parent_path}" app.mount("/", mcp.sse_app())  How can i read Authorization header in case if it is sent by the client? I tried to use approaches of FastAPI - setting dependency, adding request:Request to arguments but this doesn't work. Is there a way?

Answer: If someone is still looking for this. There is the solution. from mcp.server.fastmcp import FastMCP from fastapi import FastAPI, Request import subprocess import shlex # Global variable to keep a token a for a request auth_token = "" app = FastAPI() mcp = FastMCP("Server to manage a Linux instance") @app.middleware("http") async def auth_middleware(request: Request, call_next): auth_header = request.headers.get("Authorization") if auth_header: # extract token from the header and keep it in the global variable global auth_token auth_token = auth_header.split(" ")[1] response = await call_next(request) return response def require_auth(): """ Check access and raise an error if the token is not valid. """ if auth_token != "expected-token": raise ValueError("Invalid token") return None def run_cli(command: str, cwd: str = None) -> str: """ Execute a CLI command using subprocess.""" if cwd == "": cwd = None command_list = shlex.split(command) run_result = subprocess.run( command_list, cwd=cwd, capture_output=True, text=True, check=False, ) success = run_result.returncode == 0 return f"STDOUT: {run_result.stdout}\n\nSTDERR: {run_result.stderr}\nRETURNCODE: {run_result.returncode}\nSUCCESS: {success}" @mcp.tool() def cli_command(command: str, work_dir: str | None = "") -> str: """ Execute command line cli command on the Linux server. Arguments: command - command to execute. work_dir - workdir will be changed to this path before executing the command. """ require_auth() # we have to add this inside each tool method return run_cli(command, work_dir) app.mount("/", mcp.sse_app())  But as i understand there will be better ways to do this soon because developers of that python sdk have some ideas how to support this
Question: I am trying to create a simple clock app, and for that, I am using time. I want to update the time label Imports import toga from toga.style import Pack from toga.style.pack import COLUMN, ROW import time  code  self.time_label=toga.Label("00:00:00",style=Pack(padding=10,font_size=50))  I directly call the update_time() function in the startup method  self.Update_time()  self.time_label is needed to change to the current time the text will change inside the Update_time() function  def Update_time(self): current_time= time.strftime('%I:%M:%S') self.time_label.text=current_time  Now, I encounter that it only displays the time and does not change to the current time For example, if I run the program at 9:22:56 it displays 9:22:56 but when it becomes 9:22:57, or 9:23:00, it still shows 9:22:56.

Answer: If you only call the update method once, then of course it will only update once. To update it regularly, you'll need to run an async task. I think the startup method is called before the asyncio event loop is running, so instead you could override the on_running method, like this: async def on_running(self): while True: self.Update_time() await asyncio.sleep(1)
Question: i currently have a model fit using statsmodel OLS formula and I am trying to save this model to ADLS blob storage. '/mnt/outputs/' is a mount point I have created and I am able to read and write other files from this directory. import statsmodels.formula.api as smf fit = smf.ols(formula=f"Pressure ~ {cat_vars_int} + Speed + dose_time:Speed + Speed:log_curr_speed_time", data=df_train).fit() path = f'/mnt/outputs/Models/20240406_M2.pickle' fit.save(path)  However I am getting this error when I am saving. I am trying to write a new file not read an existing file, so i am not sure why i am getting this error. Any help would be great, thanks! FileNotFoundError: [Errno 2] No such file or directory: '/mnt/outputs/Models/20240406_M2.pickle' 

Answer: Default the mount point will be under dbfs context, whenever you reference files without spark you need prefix path with /dbfs. So, save the file giving path like below. path = f'/dbfs/mnt/outputs/Models/20240406_M2.pickle' fit.save(path)  and whenever accessing via spark context give like below. spark.read.csv("dbfs:/path_to_file")  Listing files. Dbutils display(dbutils.fs.ls(mount_point))  Output:  Python OS module os.listdir("/dbfs/"+mount_point)   Learn more about handling files in databricks here.
Question: Assignment:  Write a funcion cycle_sublist(lst, start, step) where:  lst is a list start is number that satisfies: 0 <= start < len(lst) step is the amount we increase your index each iteration  without using: slicing, importing, list comprehension, built-in functions like map and filter. The function works in this way: We start to iterate over the list of items when we get back to start or cross it again. So for example: cycle_sublist([1], 0, 2) -> [1] cycle_sublist([6, 5, 4, 3], 0, 2) -> [6, 4] cycle_sublist([7, 6, 5, 4, 3], 3, 1) -> [4, 3, 7, 6, 5] cycle_sublist([4, 3, 2, 5, 1, 6, 9], 2, 2) -> [2, 1, 9, 3] cycle_sublist([4, 3, 2, 5, 1, 6, 9], 5, 3) -> [6, 3, 1]   My problem is detecting when I have completed a cycle. I tried to:  Check my previous step and current steps and check it against start. The problem is there are some cases where it fails. Count my steps and checking if I had crossed the start.  None of those worked. Here is my code - with the missing logic for detecting the cycle: def cycle_sublist(lst,start,step): index = start length = len(last) cycle_complete = False res = [] while True: index = index % length if index >= length else index if ...: cycle_complete = True if cycle_complete and index >= start: break res.append(lst[index]) index += step return res  If you can I'd like to ask you to answer with the algorithm to detect the cycle only so I can write the code myself.

Answer: This is my try based on your try. I got something like this: def cycle_sublist(lst,start,step): index = start length = len(lst) res = [] while index < length + start: res.append(lst[index % length]) index += step return res print(cycle_sublist([1], 0, 2)) # [1] print(cycle_sublist([6, 5, 4, 3], 0, 2)) # [6, 4] print(cycle_sublist([7, 6, 5, 4, 3], 3, 1)) # [4, 3, 7, 6, 5] print(cycle_sublist([4, 3, 2, 5, 1, 6, 9], 2, 2)) # [2, 1, 9, 3] print(cycle_sublist([4, 3, 2, 5, 1, 6, 9], 5, 3)) # [6, 3, 1]  Basically, there is not much of a difference between your and my code. You were trying to keep index in valid range which made it more difficult to detect when cycle was over. Change I introduced is that I let index change in each iteration by step and not trying to keep it in valid range since I used remainder when indexing list. I guess you know, but remainder goes up to length - 1 so it will always be in valid range. Additionally, I removed some unused variables. Little addition, not sure if it's helpful. If you want to cycle multiple times under conditions you set, you can change while condition to while index < n * length + start: where n is how many times you want to cycle through list. Also, if you want to include starting element if final iteration sets on him, change < to <= in while condition.
Question: I have a function like so: def get_foo(a: object, b: tuple, c: int,) -> dict: .....  When I do $ ruff format myfile.py, my function is changed to def get_foo( a: object, b: tuple, c: int, ) -> dict: ....  How do I stop this behaviour? Update: @STerliakov I implemented your solution but received this warning. $ ruff format test.py warning: The isort option `isort.split-on-trailing-comma` is incompatible with the formatter `format.skip-magic-trailing-comma=true` option. To avoid unexpected behavior, we recommend either setting `isort.split-on-trailing-comma=false` or `format.skip-magic-trailing-comma=false`. 1 file reformatted  I can't locate isort.split-on-trailing-comma to set it to false to avoid the conflict. How do i fix this issue?

Answer: That happens due to the trailing comma in your arguments list. This behavior is intentional. You can disable it globally using skip-magic-trailing-comma. E.g. in pyproject.toml that would be [tool.ruff.format] skip-magic-trailing-comma = true  Enabling this setting is incompatible with import sorter default configuration. To ignore trailing commas in your imports as well, set isort.split-on-trailing-comma to false: [tool.ruff.lint.isort] split-on-trailing-comma = false
Question: Say we have an array: a = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33], [41, 42, 43] ]) a[[1, 3], [0, 2]] = 0  So we want to set zeros to 0th and 2nd element at both 1st and 3rd rows. But what we get is: [[11 12 13] [ 0 22 23] [31 32 33] [41 42 0]]  Why not: [[11 12 13] [ 0 22 0] [31 32 33] [0 42 0]]  ?

Answer: In import numpy as np a = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33], [41, 42, 43] ]) a[[1, 3], [0, 2]] = 0 a  the last statement is equivalent to (edited) a[1, 0] = 0 a[3, 2] = 0  It gives the following: a = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33], [41, 42, 43] ]) a[1, 0] = 0 a[3, 2] = 0 a # array([[11, 12, 13], # [ 0, 22, 23], # [31, 32, 33], # [41, 42, 0]])  In numpy's documentation, this is referenced as "advanced indexing"  Advanced indexing is triggered when the selection object, obj, is a non-tuple sequence object, an ndarray (of data type integer or bool), or a tuple with at least one sequence object or ndarray (of data type integer or bool).  In your case, your selection object is a tuple with at least one sequence object.
Question: After running the code train(train_data, EPOCHS)  I'm getting error "AttributeError: module 'tensorflow.python.distribute.input_lib' has no attribute 'DistributedDatasetInterface'"  and after traching the error, it points at line 15--> of def train(data, EPOCHS): # Loop through epochs for epoch in range(1, EPOCHS+1): print('\n Epoch {}/{}'.format(epoch, EPOCHS)) progbar = tf.keras.utils.Progbar(len(data)) # Creating a metric object r = Recall() p = Precision() # Loop through each batch for idx, batch in enumerate(data): # Run train step here loss = train_step(batch) yhat = siamese_model.predict(batch[:2]) #<--- line 15 r.update_state(batch[2], yhat) p.update_state(batch[2], yhat) progbar.update(idx+1) print(loss.numpy(), r.result().numpy(), p.result().numpy()) # Save checkpoints if epoch % 10 == 0: checkpoint.save(file_prefix=checkpoint_prefix) 

Answer: Your error happens cuz there's mismatch between how you're trying to make prediction and what your dataset type allows. Tensorflow has changed some of its APIs across versions, please refer this this will help you to gain some knowledge about how actually perform distributed training Distributed training with TensorFlow Try this maybe it can solve your problem Try a direct model call The simplest fix is to just call your model directly instead of using the predict method # Change this line yhat = siamese_model.predict(batch[:2]) # To this yhat = siamese_model(batch[:2], training=False)  This works because it bypasses the predict() machinery that's causing the error. Also refer this github issue as well issue 61900
Question: I have a dataframe that looks like this: test = pd.DataFrame( {'onset': [1,3,18,33,35,50], 'duration': [2,15,15,2,15,15], 'type': ['Instr', 'Remember', 'SocTestString', 'Rating', 'SelfTestString', 'XXX'] } )  I want to create a new dataframe such that when type contains "TestString",  two new rows are created below that row, such that the row is is now split into three rows with (for example) SocTestString_1, SocTestString_2, SocTestString_3 for those three rows, change duration columns to the value 5 for those three rows, also change the onset column such that it is the onset value of the previous column + 5  The final dataframe should look like this: test_final = pd.DataFrame( {'onset': [1,3,18,23,28,33,35,40,45,50], 'duration': [2,15,5,5,5,2,5,5,5,15], 'type': ['Instr', 'Remember', 'SocTestString_1', 'SocTestString_2', 'SocTestString_3', 'Rating', 'SelfTestString_1', 'SelfTestString_2', 'SelfTestString_3', 'XXX'] })  How may I accomplish this?

Answer: You could use str.contains to identify the target rows, then Index.repeat to duplicate them, finally boolean indexing and groupby.cumcount to update the new rows: N = 3 # number of rows to create # identify target rows m = test['type'].str.contains('TestString') # repeat them out = test.loc[test.index.repeat(m.mul(N-1).add(1))] # divide duration out.loc[m, 'duration'] /= N # compute the cumcount cc = out.loc[m].groupby(level=0).cumcount() # increment the onset out.loc[m, 'onset'] += cc*5 # add the suffix out.loc[m, 'type'] += '_'+cc.add(1).astype(str) # optionally, reset the index out.reset_index(drop=True, inplace=True)  NB. this assumes that the original index does not have duplicated indices. Output:  onset duration type 0 1 2 Instr 1 3 15 Remember 2 18 5 SocTestString_1 3 23 5 SocTestString_2 4 28 5 SocTestString_3 5 33 2 Rating 6 35 5 SelfTestString_1 7 40 5 SelfTestString_2 8 45 5 SelfTestString_3 9 50 15 XXX  Intermediates (without updating the original columns and resetting the index):  onset duration type m cc cc*5 _{cc+1} 0 1 2 Instr False <NA> <NA> NaN 1 3 15 Remember False <NA> <NA> NaN 2 18 15 SocTestString True 0 0 _1 2 18 15 SocTestString True 1 5 _2 2 18 15 SocTestString True 2 10 _3 3 33 2 Rating False <NA> <NA> NaN 4 35 15 SelfTestString True 0 0 _1 4 35 15 SelfTestString True 1 5 _2 4 35 15 SelfTestString True 2 10 _3 5 50 15 XXX False <NA> <NA> NaN