eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to animate a patch.Rectangle object using matplotlib. I want the said object to move along a path.Arc. A roundabout way to do this would be (approximately) : import numpy as np from matplotlib import pyplot as plt from matplotlib import animation import matplotlib.patches as mpat fig, ax = plt.subplots() ax.set(xlim=(0, 10), ylim=(0, 10)) # generate the patch patch = mpat.Rectangle((5, 5), 1, 4) patch.rotation_point = 'center' # generate the path to follow path_to_follow = mpat.Arc((5, 5), 2, 2) ax.add_patch(path_to_follow) def init(): patch.set(x=5, y=5) ax.add_patch(patch) return patch, def animate(i, ax): new_x = 5 + np.sin(np.radians(i)) - 0.5 # parametric form for the circle new_y = 5 + np.cos(np.radians(i)) - 2 patch.set(x=new_x, y=new_y, angle=90-i) return patch, anim = animation.FuncAnimation(fig, animate, init_func=init, fargs=[ax], frames=360, interval=10, blit=True) plt.show()  The rectangle follows a circle, but a parametric one. Would it be possible to make it follow any path? In other words, I would like to know if there are other simpler methods to do this (make my patch follow my path, here a circle), and if that could be generalized to other path. Thanks in advance ! I searched into the matplotlib doc for a methods which gives the parametric form for a given path (but apparently there is not), or for a methods which directly move a patch along a path (obviously, there was not).

Answer: Here is one way to use matplotlib.path.Path to generate a path, whose vertices can be obtained using the method cleaned, to move a patch along it. I have tried to showcase how blue and red colored Rectangles can be moved along a (blue) linear path and a (red) circular path, respectively: import numpy as np from matplotlib import pyplot as plt from matplotlib import animation, path import matplotlib.patches as mpat fig, ax = plt.subplots() ax.set(xlim=(0, 10), ylim=(0, 10)) # generate a linear path path1 = np.column_stack((np.arange(500)/50, np.arange(500)/50)) # generate a circular path circle = path.Path.circle(center=(5, 5), radius=1) path2 = circle.cleaned().vertices[:-3] # create patches patch1 = mpat.Rectangle((0, 0), 1, 3) patch2 = mpat.Rectangle((0, 0), 1, 3, color='red', fill=None) # plot path vertices plt.scatter(x=path1[:, 0], y=path1[:, 1], s=2) plt.scatter(x=path2[:, 0], y=path2[:, 1], color='red', s=2) def init(): patch1.set(x=0, y=0) patch2.set(x=5, y=6) ax.add_patch(patch1) ax.add_patch(patch2) return [patch1, patch2] def animate(i, ax): j = i % 500 # path1 has shape (500, 2) k = (i % 16) # path2 has shape (16, 2) patch1.set(x=path1[j][0], y=path1[j][1], angle=-j) patch2.set(x=path2[k][0], y=path2[k][1], angle=-k) return [patch1, patch2] anim = animation.FuncAnimation(fig, animate, init_func=init, fargs=[ax], frames=360, interval=100, blit=True) plt.show()
Question: I want in class B to automatically add the decorator _preCheck to all methods that have been inherited from class A. In the example b.double(5) is correctly called with the wrapper. I want to avoid to manually re-declare (override) the inherited methods in B but instead, automatically decorate them, so that on the call to b.add(1,2) also _preCheck wrapper is called. Side note:  I need to have a reference to the Instance of B in the wrapper (in my example via self) I want to avoid editing the Base class A. If possible, I want to encapsulate the decoration mechanism and initialization of it in the derived class B  class A(object): def __init__(self, name): self.name = name def add(self, a, b): return a + b class B(A): def __init__(self, name, foo): super().__init__(name) self.foo = foo def _preCheck(func): @wraps(func) def wrapper(self, *args, **kwargs) : print("preProcess", self.name) return func(self, *args, **kwargs) return wrapper @_preCheck def double(self, i): return i * 2 b = B('myInst', 'bar') print(b.double(5)) print(b.add(1,2))  Based on How can I decorate all inherited methods in a subclass I thought a possible solutions might be to ad the following snippet into B's init method:  for attr_name in A.__dict__: attr = getattr(self, attr_name) if callable(attr): setattr(self, attr_name, self._preCheck(attr))  However, I get the following error. I suspect the 2nd argument comes from the 'self'. . TypeError: _preCheck() takes 1 positional argument but 2 were given  There exist solutions to similar problems where they either initialize the subclasses from within the base class : Add decorator to a method from inherited class? Apply a python decorator to all inheriting classes

Answer: Decorators need to be added the class itself not the instance: from functools import wraps class A(object): def __init__(self, name): self.name = name def add(self, a, b): return a + b class B(A): def __init__(self, name, foo): super().__init__(name) self.foo = foo def _preCheck(func): @wraps(func) def wrapper(self, *args, **kwargs) : print("preProcess", self.name) return func(self, *args, **kwargs) return wrapper @_preCheck def double(self, i): return i * 2 for attr_name in A.__dict__: if attr_name.startswith('__'): # skip magic methods continue print(f"Decorating: {attr_name}") attr = getattr(A, attr_name) if callable(attr): setattr(A, attr_name, B._preCheck(attr)) b = B('myInst', 'bar') print(b.double(5)) print(b.add(1,2))  Out: Decorating: add preProcess myInst 10 preProcess myInst 3
Question: I have a big array of integers and second array of arrays. I want to create a boolean mask for the first array based on data from the second array of arrays. Preferably I would use the numpy.isin but it clearly states in it's documentation: The values against which to test each value of element. This argument is flattened if it is an array or array_like. See notes for behavior with non-array-like parameters. Do you maybe know some performant way of doing this instead of list comprehension? So for example having those arrays: a = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]]) b = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]])  I would like to have result like: np.array([ [True, True, False, False, False, False, False, False, False, False], [False, False, True, True, False, False, False, False, False, False], [False, False, False, False, True, True, False, False, False, False], [False, False, False, False, False, False, True, True, False, False], [False, False, False, False, False, False, False, False, True, True] ]) 

Answer: Try numpy.apply_along_axis to work with numpy.isin: np.apply_along_axis(lambda x: np.isin(a, x), axis=1, arr=b)  returns array([[[ True, True, False, False, False, False, False, False, False, False]], [[False, False, True, True, False, False, False, False, False, False]], [[False, False, False, False, True, True, False, False, False, False]], [[False, False, False, False, False, False, True, True, False, False]], [[False, False, False, False, False, False, False, False, True, True]]])  I will update with an edit comparing the runtime with a list comp EDIT: Whelp, I tested the runtime, and wouldn't you know, listcomp is faster timeit.timeit("[np.isin(a,x) for x in b]",number=10000, globals=globals()) 0.37380070000654086  vs timeit.timeit("np.apply_along_axis(lambda x: np.isin(a, x), axis=1, arr=b) ",number=10000, globals=globals()) 0.6078917000122601  the other answer to this post by @mozway is much faster: timeit.timeit("(a == b[...,None]).any(-2)",number=100, globals=globals()) 0.007107900004484691  and should probably be accepted.
Question: I have a (m,n,3) array data and I want to filter its values with a (m,n) mask to receive a (x,3) output array. The code below works, but how can I replace the for loop with a more efficient alternative? import numpy as np data = np.array([ [[11, 12, 13], [14, 15, 16], [17, 18, 19]], [[21, 22, 13], [24, 25, 26], [27, 28, 29]], [[31, 32, 33], [34, 35, 36], [37, 38, 39]], ]) mask = np.array([ [False, False, True], [False, True, False], [True, True, False], ]) output = [] for i in range(len(mask)): for j in range(len(mask[i])): if mask[i][j] == True: output.append(data[i][j]) output = np.array(output)  The expected output is np.array([[17, 18, 19], [24, 25, 26], [31, 32, 33], [34, 35, 36]]) 

Answer: import numpy as np data = np.array([ [[11, 12, 13], [14, 15, 16], [17, 18, 19]], [[21, 22, 13], [24, 25, 26], [27, 28, 29]], [[31, 32, 33], [34, 35, 36], [37, 38, 39]], ]) mask = np.array([ [False, False, True], [False, True, False], [True, True, False], ]) output = data[mask]
Question: I'm using python3 and and i have data set. That contains the following data. I'm trying to get the desire value from this data list. I have tried many ways but unable to figure out how to do that. slots_data = [ { "id":551, "user_id":1, "time":"199322002", "expire":"199322002" }, { "id":552, "user_id":1, "time":"199322002", "expire":"199322002" }, { "id":525, "user_id":3, "time":"199322002", "expire":"199322002" }, { "id":524, "user_id":3, "time":"199322002", "expire":"199322002" }, { "id":553, "user_id":1, "time":"199322002", "expire":"199322002" }, { "id":550, "user_id":2, "time":"199322002", "expire":"199322002" } ] # Desired output # [ # {"user_id":1,"slots_ids":[551,552,553]} # {"user_id":2,"slots_ids":[550]} # {"user_id":3,"slots_ids":[524,525]} # ]  I have tried in the following way and obviously this is not correct. I couldn't figure out the solution of this problem : final_list = [] for item in slots_data: obj = obj.dict() obj = { "user_id":item["user_id"], "slot_ids":item["id"] } final_list.append(obj) print(set(final_list)) 

Answer: Lots of good answers here. If I was doing this, I would base my answer on setdefault and/or collections.defaultdict that can be used in a similar way. I think the defaultdict version is very readable but if you are not already importing collections you can do without it. Given your data: slots_data = [ { "id":551, "user_id":1, "time":"199322002", "expire":"199322002" }, { "id":552, "user_id":1, "time":"199322002", "expire":"199322002" }, #.... ]  You can reshape it into your desired output via: ## ------------------- ## get the value for the key user_id if it exists ## if it does not, set the value for that key to a default ## use the value to append the current id to the sub-list ## ------------------- reshaped = {} for slot in slots_data: user_id = slot["user_id"] id = slot["id"] reshaped.setdefault(user_id, []).append(id) ## ------------------- ## ------------------- ## take a second pass to finish the shaping in a sorted manner ## ------------------- reshaped = [ { "user_id": user_id, "slots_ids": sorted(reshaped[user_id]) } for user_id in sorted(reshaped) ] ## ------------------- print(reshaped)  That will give you: [ {'user_id': 1, 'slots_ids': [551, 552, 553]}, {'user_id': 2, 'slots_ids': [550]}, {'user_id': 3, 'slots_ids': [524, 525]} ]
Question: I am trying to parse this website using selenium but I fail to find the buttons of the cookie popup which I need to confirm to proceed. I know that I first need to load the page and then wait for the cookie popup to appear, although that should be well handled by the sleep function. The html of the buttons inside the popup looks like this (retrieved via firefox F12). I am trying to simply click the "OK" button: <button role="button" data-testid="uc-customize-button" style="border: 1px solid rgb(0, 139, 2);" class="sc-gsDKAQ hWcdhQ">Einstellungen oder ablehnen</button> <div class="sc-bBHxTw foBPAO"></div> <button role="button" data-testid="uc-accept-all-button" style="border: 1px solid rgb(0, 139, 2);" class="sc-gsDKAQ fILFKg">OK</button>  I have tried to simply find the buttons via the xpath of the "OK" button: from time import sleep from selenium import webdriver from selenium.webdriver.common.by import By # adapting: https://stackoverflow.com/questions/64032271/handling-accept-cookies-popup-with-selenium-in-python BASE_URL = "https://www.immowelt.at/suche/wohnungen/mieten" driver = webdriver.Firefox() driver.get(BASE_URL) sleep(10) # cannot locate by XPATH driver.find_element( By.XPATH, "/div/div/div[2]/div/div[2]/div/div/div/button[2]" ).click() # weird thing is, I also cannot retrieve any of the buttons (although I know they exist) buttons = driver.find_elements(By.TAG_NAME, "button")  Any ideas why I can't find the button?

Answer: By inspecting the HTML we can see that the button is inside a shadow root.  So in order to be able to interact with the button we must first select the parent of the shadow root, which is the div element with id=usercentrics-root, and then we can select the button and click it: driver.execute_script('''return document.querySelector("#usercentrics-root").shadowRoot.querySelector("button[data-testid='uc-accept-all-button']")''').click()  Alternatively, a more human readable code: shadow_parent = driver.find_element(By.CSS_SELECTOR, '#usercentrics-root') outer = driver.execute_script('return arguments[0].shadowRoot', shadow_parent) outer.find_element(By.CSS_SELECTOR, "button[data-testid='uc-accept-all-button']").click()
Question: Update: This was fixed by pull/5837 shape: (3, 3) ┌─────────────────────┬─────────┬──────────────┐ │ dt ┆ seconds ┆ duration0 │ │ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ f64 ┆ duration[μs] │ ╞═════════════════════╪═════════╪══════════════╡ │ 2022-12-14 00:00:00 ┆ 1.0 ┆ 1µs │ │ 2022-12-14 00:00:00 ┆ 2.2 ┆ 2µs │ │ 2022-12-14 00:00:00 ┆ 2.4 ┆ 2µs │ └─────────────────────┴─────────┴──────────────┘   I want to add a duration in seconds to a date/time. My data looks like import polars as pl df = pl.DataFrame( { "dt": [ "2022-12-14T00:00:00", "2022-12-14T00:00:00", "2022-12-14T00:00:00", ], "seconds": [ 1.0, 2.2, 2.4, ], } ) df = df.with_columns(pl.col("dt").cast(pl.Datetime))  Now my naive attempt was to to convert the float column to duration type to be able to add it to the datetime column (as I would do in pandas). df = df.with_columns(pl.col("seconds").cast(pl.Duration).alias("duration0")) print(df.head()) ┌─────────────────────┬─────────┬──────────────┐ │ dt ┆ seconds ┆ duration0 │ │ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ f64 ┆ duration[μs] │ ╞═════════════════════╪═════════╪══════════════╡ │ 2022-12-14 00:00:00 ┆ 1.0 ┆ 0µs │ │ 2022-12-14 00:00:00 ┆ 2.2 ┆ 0µs │ │ 2022-12-14 00:00:00 ┆ 2.4 ┆ 0µs │ └─────────────────────┴─────────┴──────────────┘  ...gives the correct data type, however the values are all zero. The documentation is kind of sparse on the topic, any better options?

Answer: Update: The values being zero is a repr formatting issue that has been fixed with this commit. pl.duration() can be used in this way: df.with_columns( pl.col("dt").str.to_datetime() + pl.duration(nanoseconds=pl.col("seconds") * 1e9) )  shape: (3, 2) ┌─────────────────────────┬─────────┐ │ dt ┆ seconds │ │ --- ┆ --- │ │ datetime[μs] ┆ f64 │ ╞═════════════════════════╪═════════╡ │ 2022-12-14 00:00:01 ┆ 1.0 │ │ 2022-12-14 00:00:02.200 ┆ 2.2 │ │ 2022-12-14 00:00:02.400 ┆ 2.4 │ └─────────────────────────┴─────────┘
Question: Given a Polars DataFrame, is there a way to replace a particular value by "null"? For example, if there's a sentinel value like "_UNKNOWN" and I want to make it truly missing in the dataframe instead.

Answer: Update: Expr.replace() has also since been added to Polars. df.with_columns(pl.col(pl.String).replace("_UNKNOWN", None))  shape: (4, 3) ┌──────┬──────┬─────┐ │ A ┆ B ┆ C │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞══════╪══════╪═════╡ │ a ┆ null ┆ 1 │ │ b ┆ d ┆ 2 │ │ null ┆ e ┆ 3 │ │ c ┆ f ┆ 4 │ └──────┴──────┴─────┘   You can use .when().then().otherwise() pl.col(pl.String) is used to select all "string columns". df = pl.DataFrame({ "A": ["a", "b", "_UNKNOWN", "c"], "B": ["_UNKNOWN", "d", "e", "f"], "C": [1, 2, 3, 4] }) df.with_columns( pl.when(pl.col(pl.String) == "_UNKNOWN") .then(None) .otherwise(pl.col(pl.String)) # keep original value .name.keep() )
Question: I need to detect the language of text sent in chat, and I am faced with 2 problems:  the length of the message the errors that may be in it and the noise (emoji etc...)  For the noise, I clean the message and that works fine, but the length of the message is a problem. For example, if a user writes "hi", Fasttext detects the language as Dutch text, but Google Translate detects it as English. And most likely it is a message in English. I try to train my own Fasttext model, but how can I adjust the model to have better results with short strings? Do I need to train the model with the dictionary of a lot of languages to get a better result? I use Fasttext because it's the most accurate language detector. Here is an exemple of the problem with Fasttext: # wget https://dl.fbaipublicfiles.com/fasttext/supervised-models/lid.176.bin import fasttext text = "Hi" pretrained_lang_model = "lid.176.bin" model = fasttext.load_model(pretrained_lang_model) predictions = model.predict(text, k=2) print(predictions) # (('__label__de', '__label__en'), array([0.51606238, 0.31865335])) 

Answer: I have found a way to have better results. If you sum all probabilities of all languages on different detectors like fastText and lingua, and add a dictionary-based detection for short texts, you can have very good results (for my task, I also made a fastText model trained on my data).
Question: I'm trying to use the python package aitextgen in google Colab so I can fine-tune GPT. First, when I installed the last version of this package I had this error when importing it. Unable to import name '_TPU_AVAILABLE' from 'pytorch_lightning.utilities'  Though with the help of the solutions given in this question I could pass this error by downgrading my packages like this: !pip3 install -q aitextgen==0.5.2 !pip3 install -q torchtext==0.10.0 !pip3 install -q torchmetrics==0.6.0 !pip3 install -q pytorch-lightning==1.4.0rc0  But now I'm facing this error when importing the aitextgen package and colab will crash! /usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory warn(f"Failed to load image Python extension: {e}")  Keep in mind that the error is in importing the package and there is not a bug in my code. To be more clear I have this error when I just import aitextgen like this: import aitextgen  How can I deal with this error?

Answer: It seems that it is due to your CUDA version (it can be the cuDNN version too) not matching the supported version by tf, torch, or jax. As of Aug 2023, If your CUDA or cuDNN versions are +12, try downgrading them. You can find your CUDA version with nvcc --version and your cuDNN version via apt list --installed | grep cudnn. And you can downgrade your cuDNN with this (there could be other methods too): sudo apt-get install libcudnn8=8.8.1.3-1+cuda11.8 sudo apt-get install libcudnn8-dev=8.8.1.3-1+cuda11.8
Question: I installed djangorestframework as shown below: pip install djangorestframework -jwt  Then, I used rest_framework_jwt.views as shown below: from rest_framework_jwt.views import ( obtain_jwt_token, refresh_jwt_token, verify_jwt_token ) ... path('auth-jwt/', obtain_jwt_token), path('auth-jwt-refresh/',refresh_jwt_token), path('auth-jwt-verify/', verify_jwt_token), ...  But, I got the error below:  ImportError: cannot import name 'ugettext' from 'django.utils.translation'  So, how can I solve the error?

Answer: Upgrading djangorestframework-jwt will solve the error: pip install djangorestframework-jwt --upgrade
Question: I'm working with some data frames that contain Unix epochs in ms, and would like to display the entire timestamp series as a date. Unfortunately, the docs did not help me find a polars native way to do this, and I'm reaching out here. Solutions on how to do this in Python and also in Rust would brighten my mind and day. With pandas, for example, such things were possible: pd.to_datetime(pd_df.timestamp, unit="ms") # or to convert the whole col pd_df.timestamp = pd.to_datetime(pd_df.timestamp, unit="ms")  I could loop over the whole thing and do something like I'm doing here for a single entry in each row. datetime.utcfromtimestamp(pl_df["timestamp"][0] / 1000).strftime("%Y-%m-%d")  If I were to do this in Rust, I would then use something like chrono to convert the ts to a date. But I don't think looping over each row is a good solution. For now, as the best way I have found to help me is to convert pd_df = pl_df.to_pandas() and do it in pandas.

Answer: Adapting the answer of @jqurious. Polars has a dedicated from_epoch function for this: (pl.DataFrame({"timestamp": [1397392146866, 1671225446800]}) .with_columns( pl.from_epoch("timestamp", time_unit="ms") ) )  shape: (2, 1) ┌─────────────────────────┐ │ timestamp │ │ --- │ │ datetime[ms] │ ╞═════════════════════════╡ │ 2014-04-13 12:29:06.866 │ ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤ │ 2022-12-16 21:17:26.800 │ └─────────────────────────┘
Question: I have the following time: time = datetime.timedelta(days=1, hours=4, minutes=5, seconds=33, milliseconds=623)  Is it possible, to convert the time in milliseconds?  Like this: 101133623.0 

Answer: I found a possibility to solve the problem import datetime time = datetime.timedelta(days=1, hours=4, minutes=5, seconds=33, milliseconds=623) result = time.total_seconds()*1000 print(result)
Question: I am trying to add a common label in a matplotlib's subplots, but I am having some troubles. I am using python 3.10 and matplotlib 3.5.1 There is a minimal working example illustrating the problem: import matplotlib.pyplot as plt fig, axs = plt.subplots(3, 2, figsize=(8, 12), sharex=True, sharey=True) fig.supxlabel('Example of supxlabel') fig.supylabel('Example of supylabel') fig.subplots_adjust(wspace=0, hspace=0) plt.savefig('test.pdf', bbox_inches='tight', pad_inches=0)  This code generates the following figure:  Note the huge ugly margins above 'Example of supxlabel' and right to 'Example of supylabel'. I tried to use the option constrained_layout=True, along with fig.set_constrained_layout_pads, but it didn't solve my problem. I know that the problem can be solved using the option x, y, va and ha of supxlabel and supylabel, but I have many figures to generate and cannot realistically find and set the values of these options manually.

Answer: Edit: simplest is plt.subplots(layout='constrained'), works very nicely for all sorts of manipulations; below is if full control is desired, which 'constrained' (from what I can tell) won't permit.  My current workaround is y = np.sqrt(height) / 100 * 2.2, which seems to work well at least for height from 5 to 25. If using subplots_adjust(bottom=), a constant adjustment factor seems to work, like y -= .04. I've opened an Issue. width = 10 height = 20 y = np.sqrt(height) / 100 * 2.2 fig, _ = plt.subplots(23, 1, figsize=(width, height)) fig.supxlabel("supxlabel", y=y)