eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have several subplots, axs, some of them with embedded inset axes. I would like to get the data plotted in the insets by iterating over the main axes. Let's consider this minimal reproducible example: fig, axs = plt.subplots(1, 3) x = np.array([0,1,2]) for i, ax in enumerate(axs): if i != 1: ins = ax.inset_axes([.5,.5,.4,.4]) ins.plot(x, i*x) plt.show()   Is there a way of doing something like data = [] for ax in axs: if ax.has_inset(): # "asking" if ax has embedded inset ins = ax.get_inset() # getting the inset from ax line = ins.get_lines()[0] dat = line.get_xydata() data.append(dat) print(data) # [array([[0., 0.], # [1., 0.], # [2., 0.]]), # array([[0., 0.], # [1., 2.], # [2., 4.]])] 

Answer: You could use get_children and a filter to retrieve the insets: from matplotlib.axes import Axes def get_insets(ax): return [c for c in ax.get_children() if isinstance(c, Axes)] for ax in fig.axes: print(get_insets(ax))  Output: [<Axes:label='inset_axes'>] [] [<Axes:label='inset_axes'>]  For your particular example: data = [] for ax in fig.axes: for ins in get_insets(ax): line = ins.get_lines()[0] dat = line.get_xydata() data.append(dat)  Output: [array([[0., 0.], [1., 0.], [2., 0.]]), array([[0., 0.], [1., 2.], [2., 4.]])]
Question: I have a dataframe look like this, pd.DataFrame([list(range(8))+[np.nan]*2, [np.nan]*len(range(10)), range(10), [np.nan]*2+list(range(8)), range(10)]) Out[31]: 0 1 2 3 4 5 6 7 8 9 0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN NaN 1 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 2 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 3 NaN NaN 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 4 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0  I want to select the row with all NaN, that is the second in this case and fill it with the previous row.  0 1 2 3 4 5 6 7 8 9 0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN NaN 1 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN NaN 2 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 3 NaN NaN 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 4 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0  How to do that? Thanks a lot.

Answer: IIUC, you can do: mask = df.isna().all(axis=1) df.loc[mask, :] = df.loc[mask.shift(-1).fillna(False), :].values print(df)  Prints:  0 1 2 3 4 5 6 7 8 9 0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN NaN 1 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 NaN NaN 2 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 3 NaN NaN 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 4 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
Question: Here is an example of my dataframe: text name1 name2 name3 count_name barbie and ken live in a house barbie ken sophie 2 bond is preparing a bond movie bond james NaN 2 homer likes donuts homer bart NaN 1 where does mary live peter NaN NaN 0 i am travelling with john john NaN NaN 1 barbie ken barbieken barbie ken NaN 2  The last column "count_name" shows what I want to obtain. I want to count the number of times each "name1", "name2" and "name3" appear in column "text". I tried this code following a previous question (which probably would not give me exactly what I wanted) (Count occurrences of a string ocurring in multiple columns at the same time): import pandas as pd df['count_name'] = pd.MultiIndex.from_arrays(df[['text', 'name1', 'name2', 'name3']].to_numpy().T).value_counts()  but I obtain a column full of NaNs.

Answer: Using a regex: import re cols = ['name1', 'name2', 'name3'] # or # cols = list(df.filter(regex=r'name\d+')) pat = df[cols].stack().groupby(level=0).agg('|'.join) df['count_name'] = [len(re.findall(p, t)) for t, p in zip(df['text'], pat)]  To account for full words only: df['count_name'] = [len(re.findall(fr'\b(?:{p})\b', t)) for t, p in zip(df['text'], pat)]  Output:  text name1 name2 name3 count_name 0 barbie and ken live in a house barbie ken sophie 2 1 bond is preparing a bond movie bond james NaN 2 2 homer likes donuts homer bart NaN 1 3 where does mary live peter NaN NaN 0 4 i am travelling with john john NaN NaN 1  If you just want to count the full words, then use a set: import re cols = ['name1', 'name2', 'name3'] S = df[cols].agg(set, axis=1)-{np.nan} df['count_name'] = [sum(1 for w in t.split() if w in s) for t, s in zip(df['text'], S)]  Output:  text name1 name2 name3 count_name 0 barbie and ken live in a house barbie ken sophie 2 1 bond is preparing a bond movie bond james NaN 2 2 homer likes donuts homer bart NaN 1 3 where does mary live peter NaN NaN 0 4 i am travelling with john john NaN NaN 1 5 barbie ken barbieken barbie ken NaN 2
Question: I have 2 arrays of different lengths. For each element in array_1, I want to find the position of the matching element in array_2. This includes duplicates in array_1. For example: array_1 = np.array([555, 641, 1000, 641, 4, 641]) array_2 = np.array([4, 555, 641, 1000])  The desired output would be: out = [1,2,3,2,0,2]  For every element in array_1 there is a matching element in array_2. array_2 is unique and sorted. In reality, each array has around a million terms. The solution below works, but it is too slow for what I need: out = [np.where(array_2 == x) for x in array_1] 

Answer: Assuming that all unique items are present in both arrays. if array_2 is sorted, then it's not even needed to use it, numpy.unique is sufficient. out = np.unique(array_1, return_inverse=True)[1]  Output: array([1, 2, 3, 2, 0, 2]) If array_2 is not sorted, then one needs a bit of post-processing with numpy.argsort: # let's take a non sorted example array_2 = np.array([555, 641, 4, 1000]) uniq, idx = np.unique(array_1, return_inverse=True) out = np.argsort(array_2)[idx]  Output: array([0, 1, 3, 1, 2, 1])
Question: Basically, I want to say that I have a type T which has a T.__sub__(self, other: T) -> T defined. I can currently make it typecheck as if it has a T.__sub__(self, other: Any) -> Any defined, but only works if T is a class I defined myself.  I'm trying to type the following class from typing import TypeVar, Generic from dataclasses import dataclass T = TypeVar('T') @dataclass class Consumption(Generic[T]): first: T second: T def difference(self) -> T: return self.second - self.first  However, my LSP (pyright) complains that "Operator "-" not supported for types "T@Consumption" and "T@Consumption"". I've inspired myself in the typing library and SupportsAbs implementation to try and circumvent this in the following way from typing import TypeVar, Generic, Protocol from dataclasses import dataclass from abc import abstractmethod E = TypeVar('E') class SupportsSub(Generic[E], Protocol): @abstractmethod def __sub__(self, other: E) -> E: pass  With this class, I can express that the difference method is well-typed in the following way T = TypeVar('T') @dataclass class Consumption(Generic[T]): first: T second: SupportsSub[T] def difference(self) -> T: return self.second - self.first  but I lose the ability to say that second is actually also of type T. I've tried to define T such that it supports being subtracted with itself, but "TypeVar bound type cannot be generic". T = TypeVar('T', bound=SupportsSub["T"])  I can just exclude the generic in SupportsSub to make it type-check, but then it also allows other methods which are not well-typed T = TypeVar('T', bound=SupportsSub) def shouldnt_type(a: T, b: int) -> int: return a - b  Another problem with this approach is that I can't have a Consumption[float], since float doesn't inherit from SupportsSub. Therefore, pyright will complain about the following. class Other: water: Consumption[float]  I've had similar problems with pyright for example warning me that PySide.QtCore.QDate doesn't support the < operator, when in fact it does. Is it currently possible to express this with Python's typing?

Answer: Check out typing.Self. That way you can specify that the value being subtracted must be the same type as self.  from typing import Protocol, TypeVar, Self, Generic from dataclasses import dataclass E = TypeVar('E') class SupportsSub(Protocol): @abstractmethod def __sub__(self, other: Self) -> Self: pass T = TypeVar('T', bound=SupportsSub) @dataclass class Consumption(Generic[T]): first: T second: T def difference(self) -> T: return self.second - self.first def shouldnt_type(a: T, b: int) -> T: return a - b # doesn't type
Question: I had CSV file with x and y coordinates, as well as variable values for three different time steps as follows: x, y, var_t1, var_t2, var_t3, 1, 1, 8, 8, 6 1, 2, 6, 1, 2 2, 1, 5, 3, 7 2, 2, 7, 2, 6  I have learned to create a NetCDF file with the following method: import xarray as xr xr.Dataset.from_dataframe(df.set_index(['x', 'y'])).to_netcdf('filename.nc')  This results in a NetCDF with x and y as dimensions, and I get 3 different variables. My goal was to create a NetCDF with x, y and t as dimensions with a single variable. I managed to achieve this but I feel like I did it in a very complicated fashion. My solution was to play with the CSV file and make it 3 times longer, while adding a "t" column to represent time steps: x, y, t, var_t1, var_t2, var_t3, 1, 1, 0, 8, 0, 0 1, 2, 0, 6, 0, 0 2, 1, 0, 5, 0, 0 2, 2, 0, 7, 0, 0 1, 1, 1, 0, 8, 0 1, 2, 1, 0, 1, 0 2, 1, 1, 0, 3, 0 2, 2, 1, 0, 2, 0 1, 1, 2, 0, 0, 6 1, 2, 2, 0, 0, 2 2, 1, 2, 0, 0, 7 2, 2, 2, 0, 0, 6  Now when I apply import xarray as xr xr.Dataset.from_dataframe(df.set_index(['x', 'y', 't'])).to_netcdf('filename.nc')  I get a NetCDF with x, y, t dimensions and a single variable for each different time (i.e. when t = 1, only var_t2 != 0). Would there be a way to achieve this in a much simpler way, in case I encounter a similar problem in the future? This was easy to do with only 3 time steps, but I would be in trouble with tens or thousands of time steps. Thank you!

Answer: Say you have the dataframe df: >> df x y var_t1 var_t2 var_t3 0 1 1 8 8 6 1 1 2 6 1 2 2 2 1 5 3 7 3 2 2 7 2 6  You can set x,y as an index, convert it to xarray, merge the variables var_t1... to a new dimension and set new_times as the coordinates of the time dimension: >> ds = df.set_index(["x", "y"]).to_xarray() >> ds <xarray.Dataset> Dimensions: (x: 2, y: 2) Coordinates: * x (x) int64 1 2 * y (y) int64 1 2 Data variables: var_t1 (x, y) int64 8 6 5 7 var_t2 (x, y) int64 8 1 3 2 var_t3 (x, y) int64 6 2 7 6 >> new_times = range(3) >> ds_result = ds.to_array(dim="time").assign_coords(time=new_times) >> ds_result <xarray.DataArray (time: 3, x: 2, y: 2)> array([[[8, 6], [5, 7]], [[8, 1], [3, 2]], [[6, 2], [7, 6]]]) Coordinates: * x (x) int64 1 2 * y (y) int64 1 2 * time (time) int64 0 1 2
Question: I am trying to apply different value to a df from another column using: df['url']= np.where(df['client'] == 'xyz', "/s?k={query}&s=relevanceblender&page=%s".format(query=df['keyword']), "other")  however query is replaced by all values of df['keyword'], not only the row in question. thanks for your help.

Answer: Assuming this input: df = pd.DataFrame({'client': ['abc', 'abc', 'xyz', 'xyz'], 'keyword': ['kw1', 'kw2', 'kw3', 'kw4'] })  You could use: df['url'] = np.where(df['client'] == 'xyz', df['keyword'].apply("/s?k={}&s=relevanceblender&page=%s".format), 'other')  Notice how {query} was changed to {}. Or, if you cannot change the formatting string: df['url'] = np.where(df['client'] == 'xyz', df['keyword'].apply(lambda x: "/s?k={query}&s=relevanceblender&page=%s".format(query=x)), 'other')  Output:  client keyword url 0 abc kw1 other 1 abc kw2 other 2 xyz kw3 /s?k=kw3&s=relevanceblender&page=%s 3 xyz kw4 /s?k=kw4&s=relevanceblender&page=%s
Question: I cannot understand what parameter offset did in this function and what's kind of values accept? I try use integers but there is no effect. from pyrogram import Client, filters import time app = Client( "my_account", api_id=api_id, api_hash=api_hash, ) async def main(): async with app: bot_results = await app.get_inline_bot_results( "some_bot_name", query="some_query", offset="???" ) app.run(main()) 

Answer: The documentation states:  offset (str, optional) – Offset of the results to be returned.  Which is not super helpful but from the Telegram Docs shows that it is used for pagination:  offset - If the user scrolls past the first len(results) results, and next_offset field is set, the inline query should be repeated with this offset.  Checking the string documentation and [this] GitHub issue, it seems that the offset should be a string of an int like: "0" or "100". Make sure the returned results' length is more than 1 and set the offset to "1".
Question: I have been wrote a code and the result was a report which you can seen blow. the code is about the number of people who survived or died in titanic. my question is what is "Support" in this report?  precision recall f1-score support 0 0.78 0.87 0.82 154 1 0.79 0.67 0.72 114 accuracy 0.78 268 macro avg 0.79 0.77 0.77 268 weighted avg 0.78 0.78 0.78 268  Now I found an explanation about "Support" on the internet which is: "Support is the number of actual occurrences of the class in the dataset. It doesn’t vary between models, it just diagnoses the performance evaluation process." I didn't understood what is "actual occurrences" means as well. I would be grateful if someone could explain these definitions to me with an example.

Answer: support is how many samples are in each class. In your case, 154 samples are in class 0, and 114 samples are in class 1. The total number of samples is 268. It uses the ground truth labels, which represent the actual class of each sample. You might be interested in seeing how these values can be manually calculated - see https://stackoverflow.com/a/76789551/21896093
Question: I'm trying to set the level and format for the loggers used by the Quart module the way I did it successfully for other 'foreign' loggers:  by running basicConfig and implicitly setting up the root-logger or later by running logging.getLogger("urllib3.connectionpool").setLevel(logging.INFO) to get and modify an existing logger  However those approaches don't work for the loggers spawned by Quart. Neither are those affected by basicConfig nor can I set the level. Output will always look like this: [2023-07-28 16:17:12 +0200] [1254610] [INFO] Running on http://0.0.0.0:5432 (CTRL + C to quit)  Setting breakpoints in logging/__init__.py let the program break on log messages by hypercorn.error (so it seems to use the same module), but setting the level like this logging.getLogger("hypercorn.error").setLevel(logging.WARNING)  doesn't have any effect. The doc says I should use dictConfig, so I've added dictConfig({ 'version': 1, 'loggers': { 'quart.app': {'level': 'ERROR'}, 'hypercorn.error': {'level': 'ERROR'}, }, })  .. no effect I found https://github.com/pgjones/hypercorn/issues/120, and tried logger = logging.getLogger("hypercorn.error") logger.addHandler(my_own_handler) logger.setLevel(logging.WARNING) logger.propagate = False  but also without effect. What else can I try?

Answer: I had the same issue recently, and it was a real headache to find how to solve it, but here is my solution: First, for clarity, I defined a function that takes a logger as an input and that patches the logger how I want it. import logging def patch_logger(logger_: logging.Logger): logger_.handlers = [] # Clear any existing handlers logger_.addHandler(file_handler) logger_.addHandler(console_handler) logger_.propagate = False logger_.setLevel(level) return logger_  Then later when running my app, I subclassed the HypercornLogger to have it patch its own loggers: from hypercorn.config import Config from hypercorn.logging import Logger as HypercornLogger from hypercorn.asyncio import serve class CustomLogger(HypercornLogger): def __init__(self, *args, **kwargs) -> None: super().__init__(*args, **kwargs) if self.error_logger: patch_logger(self.error_logger) if self.access_logger: patch_logger(self.access_logger) app_config = Config() app_config.logger_class = CustomLogger  The patch_logger function can really be anything you want to change to your logger, what I did is just an example.
Question: I am currently designing a REST API with FastAPI and using the generated openapi.json specification to generate a client. The client generator I am currently trying to use is limited to OpenAPI 3.0.x. The generator is complaining about "null" being generated as a possible type for a parameter, which makes sense as that was only introduced in OpenAPI 3.1.0 The offending part of the specification: "name": "someParameter", "in": "query", "required": false, "schema": { "anyOf": [ { "type": "array", "items": { "type": "string" }, "minItems": 3, "maxItems": 50 }, { "type": "null" } ],  This is being generated from the following endpoint: @router.get("/{itemId}") async def readItem(someParameter: Annotated[Optional[list[str]], Query(title="...", description="...", min_length=3, max_length=50)] = None)  I am customizing FastAPI to use the OpenAPI 3.0.1 spec like this: def customOpenAPI(): openapiSchema = get_openapi( title = app.title, openapi_version = "3.0.1", version = app.version, summary = app.summary, description = app.description, routes = app.routes ) app.openapi_schema = openapiSchema return app.openapi_schema app.openapi = customOpenAPI  A possible solution would be to restructure the endpoint like this: @router.get("/{itemId}") async def readItem(someParameter: Annotated(list[str], Query(title="...", description="...", min_length=3, max_length=50, nullable=True)] = None)  But then I would be losing the Optional typehint which I would like to keep for code readability purposes. The same problem also applies to Fields in my models: class SomeModel(BaseModel) someField: Optional[str] = None  Which I would have to reformat to class SomeModel(BaseModel) someField: str = Field(nullable=True, default=None)  Is there any way to get FastAPI to generate the correct OpenAPI 3.0.1 specification while keeping the Optional typehint in my code?

Answer: Looks like they don't support it in the code base as described here  The version string of OpenAPI. FastAPI will generate OpenAPI version 3.1.0, and will output that as the OpenAPI version. But some tools, even though they might be compatible with OpenAPI 3.1.0, might not recognize it as a valid. So you could override this value to trick those tools into using the generated OpenAPI. Have in mind that this is a hack. But if you avoid using features added in OpenAPI 3.1.0, it might work for your use case. This is not passed as a parameter to the FastAPI class to avoid giving the false idea that FastAPI would generate a different OpenAPI schema. It is only available as an attribute. Example from fastapi import FastAPI app = FastAPI() app.openapi_version = "3.0.2"   The only way to configure it for version 3.0.1 is to use the get_openapi(...) function post-process all the fields and then override it in the app. I am suffering from the same problem.
Question: I have a python project and I am configuring latest version of ruff for that project for linting and formating purpose. I have the below settings in my pyproject.toml file: [tool.ruff] select = ["E", "F", "W", "Q", "I"] ignore = ["E203"] # Allow autofix for all enabled rules (when `--fix`) is provided. fixable = ["ALL"] unfixable = [] # restrict Line length to 99 line-length = 99  The ruff check command with autofix feature (--fix) of ruff identifies that the lines are long with E501 errors, but it does not format that code to wrap to next line to maintain the line-length restriction. Is there something I need to enable or do to ensure that ruff fixes this? Or is this not possible in ruff currently? Please help. I tried going through the documentation to find anything, but I am clueless what to do here.

Answer: It seems like Ruff has released Ruff Python Formatter as part of v0.0.289 and is currently in alpha state - https://github.com/astral-sh/ruff/blob/main/crates/ruff_python_formatter/README.md We are currently using v0.0.280 which does not have this feature so we used a combination of Black and Ruff as per our project requirements.
Question: Let's say I have a class AmbiguousClass which has two attributes, a and b (let's say they are both int, but it could be more general). They are related by some invertible equation, so that I can calculate a from b and reciprocally. I want to give the user the possibility to instantiate an AmbiguousClass by providing either a or b, depending on what is easier for them. Note that both ways to instantiate the variable can have the same signature, so this is not the typical polymorphism example. Providing none or both should result in an error/warning. My best guess was something like this: class AmbiguousClass(): def __init__(self, a=None, b=None): #if no parameter is provided, we raise an error (can not instantiate) if a is None and b is None: raise SomeCustomError() #if both a and b are provided, this seems redundant, a warning is raised elif a is not None and b is not None: warnings.warn(f"are you sure that you need to specify both a and b?") self.a = a self.b = b # if a is provided, we calculate b from it elif a is not None: self.a = a self.b = calculateB(self.a) # if b is provided: elif b is not None: self.b =b self.a = calculateA(self.b)  And then, the user would have to instantiate the class by specifying which keyword he is providing: var1, var2 = AmbiguousClass(a=3), AmbiguousClass(b=6) However, this feels a bit clunky, especially if the user decides to provide an argument without providing a keyword (it will by default be a, but that is not clear from the user's point of view and might lead to unexpected behaviors). How can I do this within the __init__ function in a way that is clear and will avoid unexpected behavior?

Answer: I would enforce using custom constructors that only take a single parameter and clearly indicate which parameter is provided, thus avoiding ambiguity. class AmbiguousClass: def __init__(self, a, b, _is_from_cls=False): if not _is_from_cls: raise TypeError( "Cannot instantiate AmbiguousClass directly." " Use classmethods 'from_a' or 'from_b' instead." ) self.a = a self.b = b @classmethod def from_a(cls, a): b = calculateB(a) return cls(a=a, b=b, _is_from_cls=True) @classmethod def from_b(cls, b): a = calculateA(b) return cls(a=a, b=b, _is_from_cls=True) var1 = AmbiguousClass.from_a(a=3) var2 = AmbiguousClass.from_b(b=6) var3 = AmbiguousClass(3, 6) # will raise a TypeError var4 = AmbiguousClass(3, 6, True) # will not raise a TypeError but the user explicitly overwrites the default parameter.