eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Let's say I have a Python script which is designed to wait for a specific amount of time in milliseconds and then press a key on the keyboard. Should I use threading.Timer() or should I just do it with time.sleep()? Which one is more accurate in terms of pressing the key on time and which one is more efficient? Also, please let me know if there are better and more accurate ways to do this.

Answer: threading.Timer() runs in a separate thread, which means it won't block your main program. However, it can be less precise due to thread scheduling overhead and context switching. time.sleep() is more accurate for short durations (milliseconds range) since it doesn't involve thread creation overhead. However, it blocks the entire program while waiting. A third option worth considering is using Python's asyncio, which can provide good timing accuracy without blocking.
Question: I would like to generate a stacked bar chart (ideally in seaborn), but happy to go with the native pandas plotting functionality. Let me introduce some test data to make things clear. In [353]: import pandas as pd In [354]: import seaborn as sns In [355]: df = pd.DataFrame({"Cat":["A", "B","A","B","A","B","A","B","C"], "Time":[0,0,1,1,0,0,1,1,1], "ID":[0,0,0,0,1,1,1,1,1]}) In [356]: df Out[356]: Cat Time ID 0 A 0 0 1 B 0 0 2 A 1 0 3 B 1 0 4 A 0 1 5 B 0 1 6 A 1 1 7 B 1 1 8 C 1 1 In [357]: df.groupby(["ID","Cat"]).count() Out[357]: Time ID Cat 0 A 2 B 2 1 A 2 B 2 C 1 In [358]:  What I would like to see here, is on the x-axis the ID's where on the y axis I get the count (column Time) stacked by the variable Cat, e.g. for ID 1, I want to see a stacked coloured bar where the sizes are 2, 2 and 1. I've tried the following without succeeding: df.groupby(["ID","Cat"]).count().plot(kind="bar", stacked=True, x="ID")  as it seems it can't handle the multi-axis. Any help much appreciated! EDIT This EDIT is to add the trouble I'm having with the legenbox being placed outside of the plotting window. The real code I'm using p = so.Plot(df.astype({"Time": "category"}),x='Time', color='Category').add(so.Bar(), so.Count(), so.Stack()) fig, ax = plt.subplots(figsize=(2560/120, 1335/120)) today = dt.datetime.today().strftime("%Y%m%d") plt.grid() p.on(ax).save(f"{today}_barchart_sources_{c}.png") plt.close()  

Answer: You would need to unstack the Cat (also only aggregate Time): (df.groupby(['ID', 'Cat'])['Time'].count().unstack('Cat') .plot(kind='bar', stacked=True) )  Output:  Alternatively, with seaborn's object interface, for which you don't need to pre-aggregate the data. See Stack for more examples: import seaborn.objects as so (so.Plot(df.astype({'ID': 'category'}), x='ID', color='Cat') .add(so.Bar(), so.Count(), so.Stack()) )  NB. converting ID to category to avoid having a numeric x-axis. Output:
Question: I have a 3D image with vector components (i.e., a mapping from R3 to R3). My goal is to replace each vector with the vector of maximum norm within its 3x3x3 neighborhood. This task is proving to be unexpectedly challenging. I attempted to use scipy.ndimage.generic_filter, but despite its name, this filter only handles scalar inputs and outputs. I also briefly explored skimage and numpy's sliding_window_view, but neither seemed to provide a straightforward solution. What would be the correct way to implement this? Here's what I ended up writing. It's not very elegant and pretty slow, but should help understand what I'm trying to do. import numpy as np import matplotlib.pyplot as plt def max_norm_vector(data): """Return the vector with the maximum norm.""" data = data.reshape(-1, 3) norms = np.linalg.norm(data, axis=-1) idx_max = np.argmax(norms) return data[idx_max] if __name__ == '__main__': # Load the image range_ = np.linspace(-5, 5, 30) x, y, z = np.meshgrid(range_, range_, range_, indexing='ij') data = 1 - (x ** 2) # Compute the gradient grad = np.gradient(data) grad = np.stack(grad, axis=-1) # Stack gradients along a new last axis # grad = grad[:5, :5, :5, :] # Crop the gradient for testing max_grad = np.zeros_like(grad) for i in range(1,grad.shape[0]-1): for j in range(1,grad.shape[1]-1): for k in range(2,grad.shape[2]-1): max_grad[i, j, k] = max_norm_vector(grad[i-1:i+2, j-1:j+2, k-1:k+2,:]) # Visualization fig = plt.figure(figsize=(12, 6)) # Plot original data ax1 = fig.add_subplot(121, projection='3d') ax1.scatter(x.ravel(), y.ravel(), z.ravel(), c=data.ravel(), cmap='viridis', alpha=0.5) ax1.set_title('Original Data') # Plot maximum gradient vectors ax2 = fig.add_subplot(122, projection='3d') # Downsample for better performance step = 3 x_down = x[::step, ::step, ::step] y_down = y[::step, ::step, ::step] z_down = z[::step, ::step, ::step] max_grad_down = max_grad[::step, ::step, ::step] ax2.quiver(x_down.ravel(), y_down.ravel(), z_down.ravel(), max_grad_down[:, :, :, 0].ravel(), max_grad_down[:, :, :, 1].ravel(), max_grad_down[:, :, :, 2].ravel(), length=0.1, color='red', alpha=0.7) ax2.set_title('Maximum Gradient Vectors') plt.tight_layout() plt.show() 

Answer: DIPlib has this function implemented: dip.SelectionFilter(). This is how you'd use it: grad = ... # OP's grad array norm = dip.Norm(grad) out = dip.SelectionFilter(grad, norm, dip.Kernel(3, "rectangular"), mode="maximum")  You can cast the dip.Image object out to a NumPy array with np.asarray(out) (no copy of the data will be made). NumPy functions will accept the dip.Image object as input, but many functions in scikit-image expect the input array to have a .shape method or similar, which will fail if you don't do the cast explicitly. Install the package with pip install diplib. Disclaimer: I'm an author of DIPlib, but I didn't implement this function.
Question: I am currently working on a 3D surface plot using Plotly in Python. Below is the code I have so far: import numpy as np import plotly.graph_objects as go # Definition of the domain x = np.linspace(-5, 5, 100) y = np.linspace(-5, 5, 100) X, Y = np.meshgrid(x, y) # Definition of the function, avoiding division by zero Z = np.where(X**2 + Y**2 != 0, (X * Y) / (X**2 + Y**2), 0) # Creation of the interactive graph fig = go.Figure(data=[go.Surface(z=Z, x=X, y=Y, colorscale='Viridis')]) # Add title and axis configurations fig.update_layout( title='Interactive graph of f(x, y) = xy / (x^2 + y^2)', scene=dict( xaxis_title='X', yaxis_title='Y', zaxis_title='f(X, Y)' ), ) # Show the graph fig.show()  I would like to add the plane (y = x) to this plot. However, I am having trouble figuring out how to do this. Can anyone provide guidance on how to add this plane to my existing surface plot? Any help would be greatly appreciated!

Answer: To add '𝑦=𝑥' to your 3D surface plot in Plotly, you can define a separate surface for this plane and add it to the figure using another 'go.Surface' object. In Plotly, the Surface plot requires explicit values for 𝑍 to display a plane. To achieve the 𝑦=𝑥 plane across a given 𝑍 range, we need to set both 𝑋 and 𝑌 to represent 𝑦=𝑥 over a flat range of 𝑍. Here is the new code that I modified, import numpy as np import plotly.graph_objects as go # Definition of the domain x = np.linspace(-5, 5, 100) y = np.linspace(-5, 5, 100) X, Y = np.meshgrid(x, y) # Definition of the function, avoiding division by zero Z = np.where(X**2 + Y**2 != 0, (X * Y) / (X**2 + Y**2), 0) # Creation of the main surface plot fig = go.Figure(data=[go.Surface(z=Z, x=X, y=Y, colorscale='Viridis')]) # Define the y = x plane x_plane = np.linspace(-5, 5, 100) y_plane = x_plane # y = x X_plane, Z_plane = np.meshgrid(x_plane, np.linspace(-1, 1, 2)) # Z range can be adjusted as needed Y_plane = X_plane # Since y = x # Add the y = x plane to the plot fig.add_trace(go.Surface(z=Z_plane, x=X_plane, y=Y_plane, colorscale='Reds', opacity=0.5)) # Add title and axis configurations fig.update_layout( title='3D Surface Plot with Plane y = x', scene=dict( xaxis_title='X', yaxis_title='Y', zaxis_title='f(X, Y)' ), ) # Show the graph fig.show()  and the output is
Question: I have a directory called "Data" and inside it I have 35 other directories with another bunch of directories each. I need to check if these last directories have .txt files and, if so, I want to get the name of the specific directory that is one of the aforementioned 35. After this, I want to use the pandas library to generate a "yes/no" spreadsheet with "yes" for the directories (one of the 35) that have .txt files and "no" for the directories (one of the 35) that do not have .txt files. For now, I could write the following as a test: import os w=[] w=os.listdir(r'C:\Users\Name\New\Data') tot=len(w) a=0 while a!=tot: print(w[a]) a=a+1  Which gives me the names of the 35 main directories I am interested in (Folder1, Folder2, Folder3, ..., Folder35) AND for root, dirs, files in os.walk(r'C:\Users\Name\New\Data'): for file in files: if file.endswith('.txt'): print(root)  But it results in a list with the whole path, like "C:\Users\Name\New\Data\Folder1\Folder1-1", and what I really need is to compare the name "Folder1" to the entries of the aforementioned list. How can I check if the element in "w[]" corresponds to the name in "root"?

Answer: To check if each main directory contains any .txt files in its subdirectories, we can combine the logic you've started with and streamline it to match only the main directory name (one of the 35). Here's the code to achieve this and generate a yes/no spreadsheet using pandas. import os import pandas as pd main_dir = r'C:\Users\Name\New\Data' main_folders = os.listdir(main_dir) results = {folder: "No" for folder in main_folders} for root, dirs, files in os.walk(main_dir): if any(file.endswith('.txt') for file in files): main_folder_name = os.path.basename(os.path.dirname(root)) if main_folder_name in results: results[main_folder_name] = "Yes" df = pd.DataFrame(list(results.items()), columns=['Folder', 'Has_txt_file']) output_path = r'C:\Users\Name\New\output.xlsx' df.to_excel(output_path, index=False)  I hope this will help you a little.
Question: I have table which has string such as id | url 1 | /myapi/1241/ 2 | /myapi/ 3 | /myapi/1423/ 4 | /myapi/  Now I want to filter them like this below myModel.Objects.filter(url="/myapi/****/")  Is it possible , or is there any method to do this?

Answer: Filter with the __regex lookup [Django-doc]: MyModel.objects.filter(url__regex=r'/myapi/\d+/')
Question: I tried to load an excel file on Pandas and process it. I use large excel files. For example I'm working on a file with 50 columns. I'd like to see it on spider but when I use Print functions, everytime a part of columns will be hidden with 3 points. See the attached picture. I see someone suggest use Jupiter, is this the only solution? I tried to use print, display and tabulate but nothing worked. I expect someone to give me simple suggestion. I'm not a professional developer. #EXAMPLE FOR QUESTION import pandas as pd df = pd.DataFrame({'Jan21': [22, 20, 25, 30, 4, 8, 12, 10]}) df['Feb21'] = df.Jan21*3 df['Mar21'] = df.Jan21*0.2 df['Apr21'] = df.Jan21*1.23 df['Mag21'] = df.Jan21*2.5 df['Giu21'] = df.Jan21*3 df['Lug21'] = df.Jan21*4.2 df['Ago21'] = df.Jan21*0.25 df['Set21'] = df.Jan21*2.5 df['Ott21'] = df.Jan21*2.7 df['Nov21'] = df.Jan21*3 df['Dic21'] = df.Jan21*0.33 df['Jan22'] = df.Jan21*1.89 df['Feb22'] = df.Jan21*2.65 print ("Monthly sales") print (df)  For example, if I process this simple code, in the end you will see that only few columns of dataframe will be visible. I tried Jupiter but it also had the same problem.

Answer: To see all columns, you can adjust the pandas display settings to avoid truncated output. 1. Option 1: Adjust pandas display options You can change the display option for maximum columns. pd.set_option("display.max_columns", None) print("Monthly sales") print(df)  2. Option 2: Use to_string() for full dataframe display You can use to_string() to show entire dataframe without truncated columns. print("Monthly sales") print(df.to_string())  I hope this will help you a little.
Question: When I search for this topic, I get answers that do not match what I want to do. Let's say I have a table like this:    Item N1 N2 N3 N4     Item1 1 2 4 8   Item2 2 3 6 7   Item3 4 5 7 9   Item4 1 5 6 7   Item5 3 4 7 8    I would like to one-hot encode this to get:    Item 1 2 3 4 5 6 7 8 9     Item1 1 1 0 1 0 0 0 1 0   Item2 0 1 1 0 0 1 1 0 0   Item3 0 0 0 1 1 0 1 0 1   Item4 1 0 0 0 1 1 1 0 0   Item5 0 0 1 1 0 0 1 1 0    Is this feasible at all? I am now in the process of coding some sort of loop to go through each line but I decided to ask if anyone knows a more efficient way to do this.

Answer: Use melt and crosstab. tmp = df.melt('Item') result = pd.crosstab(tmp['Item'], tmp['value']).reset_index().rename_axis(None, axis=1) Item 1 2 3 4 5 6 7 8 9 0 Item1 1 1 0 1 0 0 0 1 0 1 Item2 0 1 1 0 0 1 1 0 0 2 Item3 0 0 0 1 1 0 1 0 1 3 Item4 1 0 0 0 1 1 1 0 0 4 Item5 0 0 1 1 0 0 1 1 0
Question: I have the following string: my_string='11AB2AB33' I'd like to write this string in a list, so 'AB' is a single element of this list in the following way: ['1', '1', 'AB', '2', 'AB', '3', '3']  I tried to do it by list(my_string)  but the result wasn't what I expected: ['1', '1', 'A', 'B', '2', 'A', 'B', '3', '3']  I also tried partition method: list(my_string.partition('AB'))  and also didn't get expected result ['11', 'AB', '2AB33'] 

Answer: You can use re.findall with an alternation using the pipe | matching either AB or a non whitespace character \S If you also want to match spaces you can use a . instead of \S You can see the matches here on regex101. my_string='11AB2AB33' print(re.findall(r'AB|\S', my_string))  Output ['1', '1', 'AB', '2', 'AB', '3', '3']   If you want to use split, and you only have characters A-Z or digits 0-9, you could use non word boundary to get a positon where directly on the right is either AB or a digit. You can see the matches here on regex101 my_string='11AB2AB33' print(re.split(r"\B(?=AB|\d)", my_string))  Output ['1', '1', 'AB', '2', 'AB', '3', '3']
Question: I know there is a formula for n-th even triangle, but it's just a matter of interest for me to try to write a recursive algorithm. def even_triangle(n): value = n * (n + 1) // 2 if value % 2 == 0: return value return even_triangle(n + 1) for i in range(1, 12): print(f"{i})", even_triangle(i))  I tried to write a function which could compute the n-th even triangle number (OEIS A014494), but it turned out that it returns the previous result if the n-th triangle number is odd, but not the next even triangle number as expected. Output: 1) 6 2) 6 3) 6 4) 10 5) 28 6) 28 7) 28 8) 36 9) 66 10) 66 11) 66  What I expect: 1) 6 2) 10 3) 28 4) 36 5) 66 6) 78 7) 120 8) 136 9) 190 10) 210 11) 276 

Answer: The reason you get duplicates follows from your recursive call: return even_triangle(n + 1)  When this gets executed, then consequently even_triangle(n) === even_triangle(n + 1), which is not what you ever want to have (since it enforces the duplicate value). When using recursion, you would typically make a recursive call on a simpler version of the problem (so with a lesser value of n) and then "upgrade" that result with some manipulation to make it suitable for the current value of n. In this particular case we can see that results come in pairs of standard triangular numbers, then skipping two triangular numbers, and then again a pair, ...etc. That makes me look at a solution where you determine whether the n-th even triangular number is the first one of such a pair, or the second one. If it is the second one, the difference with the preceding n-th even triangular number (got from recursion) is the distance between normal triangular numbers, otherwise the distance is greater. In short, we can find this recursive relation: def even_triangle(n): if n == 0: return 0 elif n % 2 == 0: return even_triangle(n - 1) + 2 * n else: return even_triangle(n - 1) + 6 * n  As the difference between the two recursive calls is just the factor 2 versus 6, you could produce that factor from n % 2 (i.e. 2 + (n % 2) * 4) and merge those two recursive return statements. The base case can also be merged into it with an and operator: def even_triangle(n): return n and even_triangle(n - 1) + (2 + (n % 2) * 4) * n  As you have already noted yourself, there is a closed formula for this, so using recursion is not the most efficient here.
Question: Would it be possible given an array A, bad row indices and bad column indices to use slicing to make a new array that does not have these rows or columns? This can be done with np.delete as follows: import numpy as np A=np.random.rand(20,16) bad_col=np.arange(0,A.shape[1],4)[1:] bad_row=np.arange(0,A.shape[0],4)[1:] Anew=np.delete(np.delete(A,bad_row,0),bad_col,1) print('old shape ',A.shape) print('new shape ',Anew.shape)  I also know that you can use slicing to select certain columns and rows from an array. But I'm wondering if it can be used to exclude certain column and rows? and if not what the best way besides np.delete to do that. EDIT: Based on comments, it might not be possible with slicing in place. How about creating a new array with advanced indexing? It can be done with the following code but slow, looking for a faster alternative: good_col = [i for i in range(A.shape[1]) if i==0 or i % 4 != 0] good_row=[ i for i in range(A.shape[0]) if i==0 or i % 4 != 0] Anew2=A[good_row,:][:,good_col] print('new shape ',Anew2.shape)  Thank you

Answer: You cannot remove items of an array without either moving all items (which is slow for large arrays) or creating a new one. There is no other solution. In Numba or Cython, you can directly create a new array with one operation instead of 2 so it should be about twice faster for large arrays. It should be even faster for small arrays because Numpy functions have a significant overhead for small arrays. Numpy views are either contiguous or strided. There is no way to use a variable stride along a given axis. This has been defined that way for sake of performance. Thus, if you want to select only columns and rows with an even ID, you can (because there is a constant stride for each axis that can be set for the resulting view). However, you cannot select all rows/columns avec an ID that is not divisible by 4 for example (because there are no view that can be built with a constant stride). Note that if you try to cheat by creating new dimension and then flatten the view, then Numpy will create a copy (because there is no other way).
Question: I have a large python code base. I want to get a feel of how a BaseClass is subclassed by 'grepping-out' the name of the sub class and the functions in the class, but only for classes that inherit from SomeBaseClass. So if we have multiple files that have multiple classes, and some of the classes look like: class SubClass_A(BaseClass): def foo(self): ... async def goo(self): ... class AnotherClass: ... class SubClass_B(BaseClass): def foo(self): ... async def goo(self): ...  If we apply some sort of scrip to this code, it would output: File: /path/to/file class SubClass_A(BaseClass): def foo(self): async def goo(self): class SubClass_B(BaseClass): def foo(self): async def goo(self):  There can be many classes in the same file some of them may be a BaseClass. Essentially, this is like folding lines of code in an IDE. Now, I tried using sed to do this, but I'm not an expert in it, and sed won't be able to print the file path. However, I know awk can print the file path, but I don't know awk! Argh. I thought about writing a python program to do this, but this problem seems like something a nifty sed/awk program can do. Thanks for the help.

Answer: Using any awk, this may be what you're trying to do: $ awk '/^class/{ f=/\(BaseClass)/ } f && $1 ~ /^(class|def|async)$/' file class SubClass_A(BaseClass): def foo(self): async def goo(self): class SubClass_B(BaseClass): def foo(self): async def goo(self):
Question: How do I replace '.' in a dataframe if it's the only value in a cell without also replacing it if it's part of a decimal number? Here's the dataframe    id     2.2222   .   .   3.2   1.0    I tried this but it removes all decimals df = pd.DataFrame({'id':['2.2222','.','.','3.2','1.0']}) df['id'] = df['id'].str.replace('.','',regex=False)  Unwanted result:    id     22222         32   10    This is the desired result:    id     2.2222         3.2   1.0   

Answer: You could use .loc to do the replace: df.loc[df['id'] == '.', 'id'] = ''
Question: I am quite new to Python. I've been working on a web-scraping project that extracts data from various web pages, constructs a new HTML page using the data, and sends the page to a document management system The document management system has some XML-based parser for validating the HTML. It will reject it if XML special characters appear in text within HTML tags. For example: <p>The price of apples & oranges in New York is > the price of apples and oranges in Chicago</p>  will get rejected because of the & and the >. I considered using String.replace() on the HTML doc before sending it, but it is not broad enough, and I don't want to remove valid occurrences of characters like & and >, such as when they form part of a tag or an attribute Could someone please suggest a solution to replacing the XML special characters with, for example, their english word equivalents (eg: & -> and)? Any help you can provide would be much appreciated

Answer: BeautifulSoup tames unruly HTML and presents it as unbroken HTML. You can use it to fix references like this. from bs4 import BeautifulSoup doc = """<body> <p>The price of apples & oranges in New York is > the price of apples and oranges in Chicago</p> </body>""" soup = BeautifulSoup(doc, features="lxml") print(soup.prettify())  Outputs <html> <body> <p> The price of apples &amp; oranges in New York is &gt; the price of apples and oranges in Chicago </p> </body> </html>  Note that HTML itself is not necessarily XML compliant and there may be other reasons why an HTML document would not pass an XML validator.
Question: I have created the following pandas dataframe: ds = {'col1':[1.0,2.1,2.2,3.1,41,5.2,5.0,6.1,7.1,10]} df = pd.DataFrame(data=ds)  The dataframe looks like this: print(df) col1 0 1.0 1 2.1 2 2.2 3 3.1 4 41.0 5 5.2 6 5.0 7 6.1 8 7.1 9 10.0  I need to create a random 80% / 20% partition of the dataset and I also need to create a field (called buildFlag) which shows whether a record belongs to the 80% partition (buildFlag = 1) or belongs to the 20% partition (buildFlag = 0). For example, the resulting dataframe would like like:  col1 buildFlag 0 1.0 1 1 2.1 1 2 2.2 1 3 3.1 0 4 41.0 1 5 5.2 0 6 5.0 1 7 6.1 1 8 7.1 1 9 10.0 1  The buildFlag values are assigned randomly. Can anyone help me, please?

Answer: SOLUTION (PANDAS + NUMPY) A possible solution, which:  First, using np.random.choice to randomly choose 80% of df indices without replacement.  The df.index.isin function then checks each row's index to see if it was selected.  Finally, np.where assigns a 1 to the Flag column for selected indices and a 0 for the others.   df.assign(Flag=np.where( df.index.isin(np.random.choice( df.index, size=int(0.8 * len(df)), replace=False)), 1, 0))   SOLUTION (PANDAS + SKLEARN) Alternatively, we can use scikit-learn's train_test_split function:  First, it randomly splits the df's indices into two groups: 80% for training and 20% for testing, as specified by test_size=0.2.  The training indices are extracted using [0]. The df.index.isin method then checks which indices belong to the training set, producing a boolean array.  Finally, this boolean array is converted to integers (1 for True and 0 for False) using .astype(int).   from sklearn.model_selection import train_test_split df.assign(Flag = df.index.isin( train_test_split(df.index, test_size=0.2, random_state=42)[0]).astype(int))   Ouput:  col1 Flag 0 1.0 0 1 2.1 1 2 2.2 1 3 3.1 1 4 41.0 1 5 5.2 1 6 5.0 1 7 6.1 1 8 7.1 1 9 10.0 0