eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to add routes from a file and I don't know the actual arguments beforehand so I need to have a general function that handles arguments via **kwargs. To add routes I am using add_api_route as below: from fastapi import APIRouter my_router = APIRouter() def foo(xyz): return {"Result": xyz} my_router.add_api_route('/foo/{xyz}', endpoint=foo)  Above works fine. However enrty path parameters are not fixed and I need to read them from a file, to achieve this, I am trying something like this: from fastapi import APIRouter my_router = APIRouter() def foo(**kwargs): return {"Result": kwargs['xyz']} read_from_file = '/foo/{xyz}' # Assume this is read from a file my_router.add_api_route(read_from_file, endpoint=foo)  But it throws this error: {"detail":[{"loc":["query","kwargs"],"msg":"field required","type":"value_error.missing"}]}  FastAPI tries to find actual argument xyz in foo signature which is not there. Is there any way in FastAPI to achieve this? Or even any solution to accept a path like /foo/... whatever .../?

Answer: This will generate a function with a new signature (I assume every parameter is a string): from fastapi import APIRouter import re import inspect my_router = APIRouter() def generate_function_signature(route_path: str): args = {arg: str for arg in re.findall(r'\{(.*?)\}', route_path)} def new_fn(**kwargs): return {"Result": kwargs['xyz']} params = [ inspect.Parameter( param, inspect.Parameter.POSITIONAL_OR_KEYWORD, annotation=type_ ) for param, type_ in args.items() ] new_fn.__signature__ = inspect.Signature(params) new_fn.__annotations__ = args return new_fn read_from_file = '/foo/{xyz}' # Assume this is read from a file my_router.add_api_route( read_from_file, endpoint=generate_function_signature(read_from_file) )  However I am sure there is a better way of doing whatever you are trying to do, but I would need to understand your problem first
Question: In VSCode, with Python 3.9 and black==22.6.0, I have a project structure like: --- root ------src ---------module0.py ---------module1.py ------tests ---------test_folder0 ------------test_file0.py ------------test_file1.py ---------test_folder1 ---------etc.  In pyproject.toml I can't get the extend-exlude part to actually exclude my test files. I've tried multiple different ways, both for the entire tests folder as well as for the test_whatever.py files but nothing seems to work, even though my various attempts have been validated by https://regex101.com/. The simplest example: [tool.black] line-length = 200 target-version = ['py39'] include = '\.pyi?$' extend-exclude = '''.*test_.*'''  Either my regex is wrong (or black requires some modifications) or VSCode is ignoring my project's configuration or idk.

Answer: Maintainer of Black here :wave: OK, so I actually missed a few points in my comments. To address the main question, this is 100% expected behaviour. Your regex is fine. The thing is that when you ask VSCode to format your file on save, it calls Black passing the filepath to your current file (you just saved) directly. If you open the "Output" tab in the bottom panel and then save a Python file, you'll notice something like this: ./venv/bin/python -m black --safe --diff --quiet ./tests/data/nothing-changed.py  --include, --exclude, and --extend-exclude only affect files and directories that are discovered by Black itself. You might be wondering, huh, Black can look for files to format? Yes, it does when you run black . or give Black any other directory. The flipside of this is that these options do nothing to files given as an argument to Black. If you want to keep Format on Save enabled, your only recourse is to use --force-exclude which is similar to --extend-exclude but it's always enforced. You can either configure --force-exclude in pyproject.toml or via VSCode's Black arguments setting (preferably for the current workspace only). The difference between putting it in pyproject.toml and configuring VSCode to pass extra options to Black is well, when it's applied. If it's in pyproject.toml it will always be enforced, even when you're not using VSCode and instead are using a bash shell or whatever. This can be useful if you're using pre-commit (which passes files to Black directly just like VSCode) or similar, but can be annoying otherwise. (if you do choose to force exclude project-wide via pyproject.toml, you can format force-excluded files by either piping it in or temporarily clearing --force-exclude on the CLI, e.g. black --force-exclude='' file_you_want_to_format_even_though_it_is_force_excluded.py)
Question: I would like to convert several TIFF files with 32-bit pixel depth into 8-bit pixel depth TIFFs while retaining metadata and TIFF tags. The 32-bit TIFFs are four-dimensional ImageJ-hyperstacks with TZYX axes (i.e. time, z-depth, y-coordinate, x-coordinate) and values in the range of [0, 1]. I can convert to 8-bit and copy over the metadata (using a very small sample image created in ImageJ): import numpy as np import tifffile infile32 = "test.tif" with tifffile.TiffFile(infile32) as tif: imagej_metadata = tif.imagej_metadata a = tifffile.imread(infile32) print(a.shape) a = np.round(255 * a) a = a.astype(np.uint8) tifffile.imwrite("test_py-8bit.tif", a, imagej=True, metadata = imagej_metadata)  >>> (4, 3, 10, 11)  However, the pixel resolution (how many micrometers are in 1 pixel) is wrong, the "z" axis (a.shape[1]) is wrongly recognized as color channel, and the "time" axis (a.shape[0]) is wrongly recognized as z. If I do this process manually in ImageJ, this problem does not occur, so I suspect the TIFF tags are necessary. I'd like a programmatic way to do it, so I can run the script on a cluster over hundreds of files. Looking at the documentation of tifffile, I know it's possible to also extract tags: with tifffile.TiffFile(infile32) as tif: for page in tif.pages: for tag in page.tags: tag_name, tag_value = tag.name, tag.value  But how can I pass these tags to tifffile.imwrite?

Answer: Copy the resolution and resolutionunit properties and add the axes order of the image array to the imagej_metadata dict: import numpy import tifffile with tifffile.TiffFile('imagej_float32.tif') as tif: data = tif.asarray() imagej_metadata = tif.imagej_metadata imagej_metadata['axes'] = tif.series[0].axes resolution = tif.pages[0].resolution resolutionunit = tif.pages[0].resolutionunit del imagej_metadata['hyperstack'] imagej_metadata['min'] = 0 imagej_metadata['max'] = 255 tifffile.imwrite( 'imagej_uint8.tif', numpy.round(255 * data).astype(numpy.uint8), imagej=True, resolution=resolution, resolutionunit=resolutionunit, metadata=imagej_metadata, )
Question: EDIT: Other answers than the one I provided are welcome! Consider the following function: from pathlib import Path from typing import Union def func(path: Union[str, Path]) -> None: """My super function. Parameters ---------- path : str | Path path to a super file. """ pass  When documenting with sphinx, I would like to cross-ref both str and Path with intersphinx. But obviously, it does not work for the latter since it is referenced as pathlib.Path in the objects.inv file. Is there a way to tell intersphinx/sphinx that Path is from the pathlib module? Without resorting to: path : str | `pathlib.Path`  or path : str | `~pathlib.Path`  which does not render nicely in a python interpreter, e.g. IPython.

Answer: numpydoc can do this through x-ref aliases: https://numpydoc.readthedocs.io/en/latest/ In the configuration conf.py: numpydoc_xref_param_type = True numpydoc_xref_aliases = { "Path": "pathlib.Path", }  It might try to match other words from the parameters types of the Parameters, Other Parameters, Returns and Yields sections. They can be ignored by adding the following to the configuration conf.py: numpydoc_xref_ignore = { "of", "shape", }  numpydoc also offers other useful tools, such as docsting validation, which can be configured in conf.py with the keys described here: https://numpydoc.readthedocs.io/en/latest/install.html#configuration
Question: For management of my config files, I'm using Hydra which requires passing additional arguments using a plus and then an equal sign between the argument and its value, e.g. python evaluate.py '+model_path="logs/fc/version_1/checkpoints/epoch=1-step=2.ckpt"'  Above I'm also using quotes to escape the equal signs in the value. I want to pass this in vscode to launch.json to the args field; however, I don't know how to do it properly because typically the argument and value parts are separated by a space and not an equal sign as for Hydra. So the following doesn't work: { "version": "0.2.0", "configurations": [ { "name": "Python: Current File", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", "args" : ["+model_path", "logs/fc/version_1/checkpoints/epoch=1-step=2.ckpt"] } ] }  How should I change args to get it right?

Answer: This may not be the most elegant solution but it works if we pass everything as a single argument (no comma inbetween) like this: "args" : ["+model_path='logs/fc/version_7/checkpoints/epoch=19-step=3700.ckpt'"]
Question: This example function returns a dictionary of keras tensors: import pandas as pd import tensorflow as tf def create_input_tensors(data: pd.DataFrame) -> Dict[str,tf.keras.engine.keras_tensor.KerasTensor]: """Turns each dataframe column into a keras tensor and returns them as a dict""" tensors = {} for name, column in data.items(): tensors[name] = tf.keras.Input(shape=(1, ), name=name, dtype=float32) return tensors  I do not know how to correctly type hint the return value. Running the code snippet yields the following exception: Exception has occurred: AttributeError module 'keras.api._v2.keras' has no attribute 'engine'  Googling this exception did not help. Running type(tensors['year']) in the debugger at the end of the function to see what type one of the elements in the return dictionary is (year is one of the columns in data) yields <class 'keras.engine.keras_tensor.KerasTensor'>. I have issues with this specific function, but also generally when trying to type hint functions that handle any kind of keras object. An answer that is applicable to these similar problems would be much appreciated.

Answer: This works like a charm for me: import typing from keras.engine.keras_tensor import KerasTensor def f() -> typing.Dict[str, KerasTensor]: return {"a": tf.keras.Input(shape=(1, ),)} f()
Question: i have a dataframe of 2 columns. i tried converting it into a dictionary using df2.set_index('pay').T.to_dict('list'). As there are duplicated keys, some columns were omitted. Is there any way to resolve this issue or an alternative method?     pay score     500 1   700 4   1000 5   700 3     I would like to achieve this dictionary. {'0': [500, 1], '1': [700, 4], '2': [1000, 5], '3' [700, 3]}

Answer: IIUC use: d = df2.T.to_dict('list') print (d) {0: [500, 1], 1: [700, 4], 2: [1000, 5], 3: [700, 3]}
Question: Lets say I have a function that return a lot of variables: def func(): return var_a, var_b, var_c, var_d, var_e, var_f, var_g, var_h, var_i,  Using this function results in very long lines var_a, var_b, var_c, var_d, var_e, var_f, var_g, var_h, var_i = func()  Ideally, I would like to use the line breaker \, e.g. a = var \ + var \ + var \ + var \ + var  However, I don't think this is possible with the result of a function (i.e. unpacking tuple). Are there methods to do so? Or should I find another way to return fewer variables? Do you have any other style suggestions?

Answer: You can wrap the variables in round brackets (var_a, var_b, var_c, var_d, var_e, var_f, var_g, var_h, var_i) = func()
Question: I drew this flower of life by buffering points to polygons. I wanted each overlapping region to be its own polygon, so I used union and polygonize on the lines.  I have filtered the polygons by area to eliminate sliver polygons, and now I'd like to filter them again and am stuck. I only want to keep the circles that are complete, so the first circle at 0,0 and the first level of surrounding rings (or petals). I want circles like this:  I am wondering if I can filter by centroid location, something like: complete_polys = [polygon for polygon in filtered_polys if centroid[i].x < 4] complete_polys = [polygon for polygon in complete_polys_x if centroid[i].x > -4]  Obviously this doesn't work, and I don't even know if it is possible. Perhaps this is the wrong approach entirely, and maybe snap() or clip_by_rect() might be better options? Thanks in advance for your insight and help. Here's the code to generate the circles: import matplotlib.pyplot as plt from shapely.geometry import Point, LineString from shapely.ops import unary_union, polygonize from matplotlib.pyplot import cm import numpy as np def plot_coords(coords, color): pts = list(coords) x, y = zip(*pts) # print(color) plt.plot(x,y, color='k', linewidth=1) plt.fill_between(x, y, facecolor=color) def plot_polys(polys, color): for poly, color in zip(polys, color): plot_coords(poly.exterior.coords, color) x = 0 y = 0 h = 1.73205080757 points = [# center Point(x, y), # first ring Point((x + 2), y), Point((x - 2), y), Point((x + 1), (y + h)), Point((x - 1), (y + h)), Point((x + 1), (y - h)), Point((x - 1), (y - h)), # second ring Point((x + 3), h), Point((x - 3), h), Point((x + 3), -h), Point((x - 3), -h), Point((x + 2), (h + h)), Point((x - 2), (h + h)), Point((x + 2), (-h + -h)), Point((x - 2), (-h + -h)), Point((x + 4), y), Point((x - 4), y), Point(x, (h + h)), Point(x, (-h + -h)), #third ring Point((x + 4), (h + h)), Point((x - 4), (h + h)), Point((x + 4), (-h + -h)), Point((x - 4), (-h + -h)), Point((x + 1), (h + h + h)), Point((x - 1), (h + h + h)), Point((x + 1), (-h + -h + -h)), Point((x - 1), (-h + -h + -h)), Point((x + 5), h), Point((x - 5), h), Point((x + 5), -h), Point((x - 5), -h)] # buffer points to create circle polygons circles = [] for point in points: circles.append(point.buffer(2)) # unary_union and polygonize to find overlaps rings = [LineString(list(pol.exterior.coords)) for pol in circles] union = unary_union(rings) result_polys = [geom for geom in polygonize(union)] # remove tiny sliver polygons threshold = 0.01 filtered_polys = [polygon for polygon in result_polys if polygon.area > threshold] print("total polygons = " + str(len(result_polys))) print("filtered polygons = " + str(len(filtered_polys))) colors = cm.viridis(np.linspace(0, 1, len(filtered_polys))) fig = plt.figure() ax = fig.add_subplot() fig.subplots_adjust(top=0.85) plot_polys(filtered_polys, colors) ax.set_aspect('equal') plt.show() 

Answer: Is this what you wanted? I used the x^2 + y^2 = r^2 circle formula to filter. complete_polys = [polygon for polygon in filtered_polys if (polygon.centroid.x**2 + polygon.centroid.y**2 < 4**2)] plot_polys(complete_polys, colors)
Question: I have a torch tensor which I need to convert to a byte object so that I can pass it to starlette's StreamingResponse which will return a reconstructed image from the byte object. I am trying to convert the tensor and return it like so: def some_unimportant_function(params): return_image = io.BytesIO() torch.save(some_img, return_image) return_image.seek(0) return_img = return_image.read() return StreamingResponse(content=return_img, media_type="image/jpeg")  The below works fine on regular byte objects and my API returns the reconstructed image: def some_unimportant_function(params): image = Image.open(io.BytesIO(some_image)) return_image = io.BytesIO() image.save(return_image, "JPEG") return_image.seek(0) return StreamingResponse(content=return_image, media_type="image/jpeg")  Using PIL library for this what am I doing wrong here?

Answer: Converting PyTorch Tensor to the PIL Image object using torchvision.transforms.ToPILImage() module and then treating it as PIL Image as your second function would work. Here is an example. def some_unimportant_function(params): tensor = # read the tensor from disk or whatever image = torchvision.transforms.ToPILImage()(tensor.unsqueeze(0)) return_image = io.BytesIO() image.save(return_image, "JPEG") return_image.seek(0) return StreamingResponse(content=return_image, media_type="image/jpeg")
Question: I have a python list like this, arr = [110, 60, 30, 10, 5]  What I need to do is actually find the difference of every number with all the other numbers and then find the average of all those differences. So, for this case, it would first find the difference between 110 and then all the remaining elements, i.e. 60, 30, 10, 5, and then it will find the difference of 60 with the remaining elements, i.e. 30, 10, 5 and etc. After which, it will compute the Average of all these differences. Now, this can easily be done with two For Loops but in O(n^2) time complexity and also a little bit of "messy" code. I was wondering if there was a faster and more efficient way of doing this same thing?

Answer: I'll just give the formula first: n = len(arr) out = np.sum(arr * np.arange(n-1, -n, -2) ) / (n*(n-1) / 2) # 52  Explanation: You want to find the mean of a[0] - a[1], a[0] - a[2],..., a[0] - a[n-1] a[1] - a[2],..., a[1] - a[n-1] ...  there, your `a[0]` occurs `n-1` times with `+` sign, `0` with `-` -> `n-1` times `a[1]` occurs `n-2` times with `+` sign, `1` with `-` -> `n-3` times ... and so on
Question: I was using playwright to scrape pages using Python. I know how to do the same using a script, but I was trying this in an interactive mode. from playwright.sync_api import Playwright, sync_playwright, expect import time def run(playwright: Playwright) -> None: browser = playwright.chromium.launch(headless=False) context = browser.new_context() page = context.new_page() page.goto("https://www.wikipedia.org/") context.close() browser.close() with sync_playwright() as playwright: run(playwright)  I tried to do this in interactive mode as: >>> from playwright.sync_api import Playwright, sync_playwright, expect >>> playwright = sync_playwright() >>> browser = playwright.chromium.launch(headless=False)  But this gave me an error: Traceback (most recent call last): File "C:\Users\hpoddar\AppData\Local\Programs\Python\Python310\lib\idlelib\run.py", line 578, in runcode exec(code, self.locals) File "<pyshell#2>", line 1, in <module> AttributeError: 'PlaywrightContextManager' object has no attribute 'chromium' 

Answer: Use the .start() method: >>> from playwright.sync_api import Playwright, sync_playwright, expect >>> playwright = sync_playwright().start() >>> browser = playwright.chromium.launch(headless=False) >>> page = browser.new_page()  Alternatively, if you just want an interactive browser, and don't care about an interactive shell, you can also use the wait_for_timeout function instead (only applicable on Page objects) and set the timeout to a high value: from playwright.sync_api import sync_playwright with sync_playwright() as playwright: browser = playwright.chromium.launch(headless=False) page = browser.new_page() page.wait_for_timeout(10000)
Question: I'm experimenting with Matplotlib animated charts currently. Having an issue where, using a public dataset, the data isn't animating. I am pulling data from a public CSV file following some of the guidance from this post (which has to be updated a bit for things like the URL of the data). I've tested my Matplotlib installation, and it displays static test data without issue in a variety of formats. Not sure if this is a problem in Pandas getting the data across or something I've missed with the animation. Code: import matplotlib.animation as ani import matplotlib.pyplot as plt import numpy as np import pandas as pd url = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_global.csv' df = pd.read_csv(url, delimiter=',', header='infer') df_interest = df.loc[ df['Country/Region'].isin(['United Kingdom', 'US', 'Italy', 'Germany']) & df['Province/State'].isna()] df_interest.rename( index=lambda x: df_interest.at[x, 'Country/Region'], inplace=True) df1 = df_interest.transpose() df1 = df1.drop(['Province/State', 'Country/Region', 'Lat', 'Long']) df1 = df1.loc[(df1 != 0).any(1)] df1.index = pd.to_datetime(df1.index) print(df1) color = ['red', 'green', 'blue', 'orange'] fig = plt.figure() plt.xticks(rotation=45, ha="right", rotation_mode="anchor") # rotate the x-axis values plt.subplots_adjust(bottom=0.2, top=0.9) # ensuring the dates (on the x-axis) fit in the screen plt.ylabel('No of Deaths') plt.xlabel('Dates') def buildmebarchart(i=int): plt.legend(df1.columns) p = plt.plot(df1[:i].index, df1[:i].values) # note it only returns the dataset, up to the point i for i in range(0, 4): p[i].set_color(color[i]) # set the colour of each curve animator = ani.FuncAnimation(fig, buildmebarchart, interval=50) plt.show()  Result (a static graph with no data):  UPDATE: It appears to be this line of pandas code that might be the culprit: p = plt.plot(df1[:i].index, df1[:i].values)  The df1[:i].index and .values are empty when I insert a breakpoint to print them out.

Answer: Actually, your code is fine. I can successfully run it as a script without making any changes. Here's my environment: python 3.8.5 pandas 1.1.3 matplotlib 3.3.2  But I suspect, that you are working in Jupyter-Notebook. In this case you have to make some changes. First, set up matplotlib to work interactively in the notebook. You can use ether notebook or nbAgg as a backend (see The builtin backends for more info): %matplotlib notebook  Next, deactivate the last line of code: #plt.show() <- comment or delete this line  Should work. Take into account that I use Jupyter-Notebook version 6.1.4. If the problem persists, update the description to reflect your environment.  P.S. Try ipympl as an alternative backend.
Question: I have an array with approximately 12000 length, something like array([0.3, 0.6, 0.3, 0.5, 0.1, 0.9, 0.4...]). Also, I have a column in a dataframe that provides values like 2,3,7,3,2,7.... The length of the column is 48, and the sum of those values is 36. I want to distribute the values, which means the 12000 lengths of array is distributed by specific every value. For example, the first value in that column( = 2) gets its own array of 12000*(2/36) (maybe [0.3, 0.6, 0.3]), and the second value ( = 3) gets its array of 12000*(3/36), and its value continues after the first value(something like [0.5, 0.1, 0.9, 0.4]) and so on.

Answer: import pandas as pd import numpy as np # mock some data a = np.random.random(12000) df = pd.DataFrame({'col': np.random.randint(1, 5, 48)}) indices = (len(a) * df.col.to_numpy() / sum(df.col)).cumsum() indices = np.concatenate(([0], indices)).round().astype(int) res = [] for s, e in zip(indices[:-1], indices[1:]): res.append(a[round(s):round(e)]) # some tests target_pcts = df.col.to_numpy() / sum(df.col) realized_pcts = np.array([len(sl) / len(a) for sl in res]) diffs = target_pcts / realized_pcts assert 0.99 < np.min(diffs) and np.max(diffs) < 1.01 assert all(np.concatenate([*res]) == a)