eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am attempting to invoke an AWS lambda function asynchronously within another Lambda function using the boto3 SDK. The invocation is done using the following code snippet: lambda_client = boto3.client('lambda') response = lambda_client.invoke( FunctionName='async_function:alias', InvocationType="Event", Payload=json.dumps({'id': '101932092', 'type': 'type', 'sub_type': 'subtype'}) )  The issues I am encountering is that the invoking function sometimes times out(15 minutes) at the above code block. The behavior occurs intermittently and there are no clear patterns. I have ruled out concurrency and throttling issues on the invoked function by checking the relevant metrics. However, even though the invoke call is supposed to put the event in an event queue for asynchronous processing (as per AWS Lambda documentation), the invoking function times out without providing a success or error response. Any insights or suggestions for trouble shooting this would be greatly appreciated.

Answer: The most likely reason for this intermittent connectivity is that your Lambda function has been configured for VPC access and you have chosen a mix of private and public subnets. The fix is to configure the Lambda function for private subnets only or, if your Lambda functions only need to reach AWS services, then configure VPC Endpoints for the AWS services that you need access to. The reason that the Lambda function fails intermittently is that it runs in a private subnet sometimes and in a public subnet at other times, depending on placement decisions made by the Lambda service. When the Lambda function executes in a public subnet, it has no network route to the internet or to AWS services. The reasons for this are:  the Lambda function has a private IP but does not have a public IP the default route for traffic in a public subnet is the Internet Gateway, which drops traffic from private IPs (because they're not routable on the internet) the default route for traffic in a private subnet, if you set it up correctly to reach the internet, is a NAT or NAT gateway which allows private IP traffic to be NATed to a public IP (the public IP of the NAT device) and hence that traffic can reach the internet  Also, see Why can't an AWS lambda function inside a public subnet in a VPC connect to the internet?
Question: I have a small class hierarchy with similar methods and __init__(), but slightly different static (class) read() methods. Specifically will the child class need to prepare the file name a bit before reading (but the reading itself is the same): class Foo: def __init__(self, pars): self.pars = pars @classmethod def read(cls, fname): pars = some_method_to_read_the_file(fname) return cls(pars) class Bar(Foo): @classmethod def read(cls, fname): fname0 = somehow_prepare_filename(fname) return cls.__base__.read(fname0)  The problem here is that Bar.read(…) returns a Foo object, not a Bar object. How can I change the code so that an object of the correct class is returned?

Answer: cls.__base__.read binds the read method explicitly to the base class. You should use super().read instead to bind the read method of the parent class to the current class. Change: return cls.__base__.read(fname0)  to: return super().read(fname0)
Question: I'm encountering an issue when trying to access the 'google' provider in Django-allauth. I'm getting either a MultipleObjectsReturned or ObjectDoesNotExist exception. I have followed the documentation and tried various troubleshooting steps, but the problem persists. Here is the code snippet from my views.py file: from allauth.socialaccount.models import SocialApp from django.core.exceptions import MultipleObjectsReturned, ObjectDoesNotExist def home(request): try: google_provider = SocialApp.objects.get(provider='google') except MultipleObjectsReturned: print("Multiple 'google' providers found:") for provider in SocialApp.objects.filter(provider='google'): print(provider.client_id) raise except ObjectDoesNotExist: print("No 'google' provider found.") raise return render(request, 'home.html', {'google_provider': google_provider})  I have also imported the necessary modules SocialApp and MultipleObjectsReturned/ObjectDoesNotExist from allauth.socialaccount.models and django.core.exceptions respectively. I have checked my database, and there is only one entry for the 'google' provider in the SocialApp table. The client_id is correctly set for this provider. Despite taking these steps, I am still encountering the mentioned errors. I have made sure to restart my Django server after any code or database changes.

Answer: you probably have 2 instance of of your google account set up. one in your settings.py and the other one in your django admin console. best you delete anyone you think you should delete and make migrations and migrate #settings.py (you could delete this and leave the one in the admin) SOCIALACCOUNT_PROVIDERS = { 'google': { # For each OAuth based provider, either add a ``SocialApp`` # (``socialaccount`` app) containing the required client # credentials, or list them here: 'APP': { 'client_id': '123', 'secret': '456', 'key': '' } } }
Question: Given the below dataframe, if rows are grouped by first name and last name, how can I find and create new rows for a group that does not have a row for every type in the types list. So in the example below, two new rows would be created for Bob Jack that are missing from the original dataframe: one with type 'DA' and another with type 'FA', the value columns can be set to 0. data = { 'First Name': ['Alice', 'Alice', 'Alice', 'Alice', 'Bob', 'Bob'], 'Last Name': ['Johnson', 'Johnson', 'Johnson', 'Johnson', 'Jack', 'Jack'], 'Type': ['CA', 'DA', 'FA', 'GCA', 'CA', 'GCA'], 'Value': [25, 30, 35, 40, 50, 37] } types = ['CA', 'DA', 'FA', 'GCA'] df = pd.DataFrame(data) 

Answer: One way to do this is to create a dataframe which is all the combinations of names and types, then left join that to the original dataframe. This will create a df with all combinations, with NaN values where there was a missing entry in the original data. That can then be filled with 0. Note that because the value column gets NaN values in it, it is converted to type float. You can convert that back to int if desired using astype({'Value': int}) in the chain: out = (df[['First Name', 'Last Name']] .drop_duplicates() .merge(pd.Series(types, name='Type'), how='cross') .merge(df, on=['First Name', 'Last Name', 'Type'], how='left') .fillna(0) # use this astype if you need `Value` to be an int .astype({'Value': int}) )  Output (with the astype to convert Value back to int):  First Name Last Name Type Value 0 Alice Johnson CA 25 1 Alice Johnson DA 30 2 Alice Johnson FA 35 3 Alice Johnson GCA 40 4 Bob Jack CA 50 5 Bob Jack DA 0 6 Bob Jack FA 0 7 Bob Jack GCA 37
Question: I am working on a Django python project with a postgres db hosted with render.com. The code works fine on server and my imac. I recently got a Macbook Pro M3 (running sonoma). I have replicated the exact same setup and environment however when I try to run the code locally, I get Python(40505,0x1704f7000) malloc: double free for ptr 0x1368be800 Python(40505,0x1704f7000) malloc: *** set a breakpoint in malloc_error_break to debug  The setup is the exact same on the other device and it works fine there. Here is the link to repo https://github.com/moreshk/django-postgres Any assistance would be useful. Setup https://github.com/moreshk/django-postgres on my new device. Setup virtual environment, any dependencies and installed requirements and .env file. Would have expected it to run fine locally. Other Django Python projects seem to work fine, except this one which has a postgres db with render.com When I try to run the code locally, I get the below error: python manage.py runserver Watching for file changes with StatReloader Performing system checks... System check identified no issues (0 silenced). Python(40505,0x1704f7000) malloc: double free for ptr 0x1368be800 Python(40505,0x1704f7000) malloc: *** set a breakpoint in malloc_error_break to debug 

Answer: if you have installed PostgreSQL with Homebrew, brew upgrade postgresql and ensure you have version 14.10_1 https://github.com/Homebrew/homebrew-core/issues/155651#issuecomment-1827988313 psql(6636,0x10f1de600) malloc: *** error for object 0x7f916b00bc00: pointer being freed was not allocated psql(6636,0x10f1de600) malloc: *** set a breakpoint in malloc _error break to debug
Question: I am facing an issue with drawing triangle borders using Matplotlib in Python. I want to create a specific pattern, but I'm encountering unexpected behavior. I need assistance in identifying and resolving the problem. this is my code import numpy as np import matplotlib.pyplot as plt N = 5 A = np.array([(x, y) for y in range(N, -1, -1) for x in range(N + 1)]) t = np.array([[1, 1], [-1, 1]]) A = np.dot(A, t) # I have defined a triangle fig = plt.figure(figsize=(10, 10)) triangle = fig.add_subplot(111) X = A[:, 0].reshape(N + 1, N + 1) Y = A[:, 1].reshape(N + 1, N + 1) for i in range(1, N + 1): for j in range(i): line_x = np.array([X[i, j + 1], X[i, j], X[i - 1, j]]) line_y = np.array([Y[i, j + 1], Y[i, j], Y[i - 1, j]]) triangle.plot(line_y,line_x, color='black', linewidth=1) plt.show()  but I am getting this image, as u can see,  At corner extra lines are coming, as i encircled it. I dont want this extra line, i tried to solved it using loop, eventhough one extra line will keep remain for i in range(6): if i == N-1 : for j in range(i-1): line_x = np.array([X[i , j+1], X[i, j],X[i-1, j]]) line_y = np.array([Y[i, j+1], Y[i, j], Y[i-1, j]]) triangle.plot(line_y, line_x, color='black', linewidth=1) pass else: for j in range(i): line_x = np.array([X[i , j+1], X[i, j],X[i-1, j]]) line_y = np.array([Y[i, j+1], Y[i, j], Y[i-1, j]]) triangle.plot(line_y,line_x, color='black', linewidth=1) pass  plt.show() kindly resolve the issue

Answer: The lines you want to delete belong to the triangules generated in the first (i = 1, j = 0) and last (i = N, J = N - 1) iterations of the nested for loops: import numpy as np import matplotlib.pyplot as plt N = 5 A = np.array([(x, y) for y in range(N, -1, -1) for x in range(N + 1)]) t = np.array([[1, 1], [-1, 1]]) A = np.dot(A, t) # I have defined a triangle fig = plt.figure(figsize=(10, 10)) triangle = fig.add_subplot(111) X = A[:, 1].reshape(N + 1, N + 1) Y = A[:, 0].reshape(N + 1, N + 1) for i in range(1, N + 1): for j in range(i): if i == 1: # Lower right triangle line_x = np.array([X[i, j + 1], X[i, j]]) line_y = np.array([Y[i, j + 1], Y[i, j]]) elif j == N - 1: # Upper right triangle line_x = np.array([X[i, j], X[i - 1, j]]) line_y = np.array([Y[i, j], Y[i - 1, j]]) else: line_x = np.array([X[i, j + 1], X[i, j], X[i - 1, j]]) line_y = np.array([Y[i, j + 1], Y[i, j], Y[i - 1, j]]) triangle.plot(line_x, line_y, color='black', linewidth=1) plt.show()
Question: How can I modify the capturing group in pandas df.replace()? I try to add thousands separators to the numbers within the string of each cell. This should happen in a method chain. Here is the code I have so far: import pandas as pd df = pd.DataFrame({'a_column': ['1000 text', 'text', '25000 more text', '1234567', 'more text'], "b_column": [1, 2, 3, 4, 5]}) df = (df.reset_index() .replace({"a_column": {"(\d+)": r"\1"}}, regex=True))  The problem is that I don't know how to do something with r"\1", e.g., str(float(r"\1")) doesn't work. Expected output:  index a_column b_column 0 0 1,000 text 1 1 1 text 2 2 2 25,000 more text 3 3 3 1,234,567 4 4 4 more text 5 

Answer: You can use replace in your pipe, looking for a point preceded by a digit and followed by some multiple of 3 digits using this regex: (?<=\d)(?=(?:\d{3})+\b)  That can then be replaced by a comma (,). df = (df .reset_index() .replace({ 'a_column' : { r'(?<=\d)(?=(?:\d{3})+\b)' : ',' } }, regex=True) )  Output:  index a_column b_column 0 0 1,000 text 1 1 1 text 2 2 2 25,000 more text 3 3 3 1,234,567 4 4 4 more text 5 5 5 563 and 45 and 9 text 6  Note I added an extra row to your df to show that you don't get commas where you shouldn't.
Question: I'm encountering an issue with Streamlit where I'm trying to allow the user to modify text using st.text_input and then display the modified text when a button is clicked. However, the modified text is not persisting as expected in the session state. Here's a simplified version of the code: import streamlit as st # Initialize session state if 'text' not in st.session_state: st.session_state.text = "original" if st.button("show"): # Allow the user to modify the text st.session_state.text = st.text_input("Edit Text", value=st.session_state.text) # Display the modified text st.markdown(st.session_state.text) if st.button("show again"): # Display the modified text st.markdown(st.session_state.text)  Despite using st.text_input to modify the text, the "show again" button still displays the original text, not the modified one. I've tried using st.text_area as well, but the issue persists. Why is the modified text not persisting in the session state as expected? How to make it persist as expected?

Answer: The problem is happening due to two things:  The script is rerun every time a button is clicked 'text' is changed when the first button is clicked, so the update lags behind in the session state. This is even listed as a case in the documentation under Buttons to modify st.session_state  section.  There are two ways to fix the issue. Option 1: Change the way text is initialized in the session state We can use get since st.session_state is like a dict (or an if-else block) to initialize the text key so that on script re-run, the text value persists across "sessions"; if it's the first time, its value will be 'original' but for the subsequent runs, it will be whatever it was in the previous session. import streamlit as st # Initialize session state st.session_state.text = st.session_state.get('text', 'original') if st.button("show"): # Allow the user to modify the text st.text_input("Edit Text", key='text') # <--- set using the key kwarg # Display the modified text st.markdown(st.session_state.text) if st.button("show again"): # Display the modified text st.markdown(st.session_state.text)  Option 2: Use callback function Another option is to define a callback function which is activated on button click (to show again). The function updates the session state with the current value of text, so that it persists through the subsequent script re-runs. import streamlit as st def update_text(value): # <--- define callback function st.session_state.text = value # Initialize session state if 'text' not in st.session_state: st.session_state.text = 'original' if st.button("show"): # Allow the user to modify the text st.text_input("Edit Text", key='text') # Display the modified text st.markdown(st.session_state.text) if st.button("show again", on_click=update_text, args=[st.session_state.text]): # <--- invoke the callback # Display the modified text st.markdown(st.session_state.text)
Question: I have a page like this: ... <div class="myclass"> <p> text 1 to keep<span>text 1 to remove</span>and keep this too. </p> <p> text 2 to keep<span>text 2 to remove</span>and keep this too. </p> <div>  I.e.: I want to remove all <span> tags from any <p> element from bs4 (BeautifulSoup in Python3). Currently this is my code: from bs4 import BeautifulSoup ... text = "" for tag in soup.find_all(attrs={"class": "myclass"}): text += tag.p.text  And of course I get all text in spans too... I read I should use unwrap() or decompose() but I really do not understand how to use them in practice in my use-case... All similar Q/A do not help...

Answer: You can try: from bs4 import BeautifulSoup html_text = """\ <div class="myclass"> <p> text 1 to keep<span>text 1 to remove</span>and keep this too. </p> <p> text 2 to keep<span>text 2 to remove</span>and keep this too. </p> <div>""" soup = BeautifulSoup(html_text, "html.parser") for span in soup.select("p span"): span.replace_with(" ") # or span.extract() soup.smooth() print(soup.prettify())  Prints: <div class="myclass"> <p> text 1 to keep and keep this too. </p> <p> text 2 to keep and keep this too. </p> <div> </div> </div>
Question: I have the following issue with Polars's LazyFrame "Structs" (pl.struct) and "apply" (a.k.a. map_elements) in with_columns The idea here is trying to apply a custom logic to a group of values that belong to more than one column I have been able to achieve this using DataFrames; however, when switching to LazyFrames, a KeyError is raised whenever I try to access a column in the dictionary sent by the struct to the function. I'm looping through columns, one by one, in order to apply different functions (mapped elsewhere to their names, but in the examples below I'll just use the same one for simplicity)  Working DataFrame implementation  my_df = pl.DataFrame( { "foo": ["a", "b", "c", "d"], "bar": ["w", "x", "y", "z"], "notes": ["1", "2", "3", "4"] } ) print(my_df) cols_to_validate = ("foo", "bar") def validate_stuff(value, notes): # Any custom logic if value not in ["a", "b", "x"]: return f"FAILED {value} - PREVIOUS ({notes})" else: return notes for col in cols_to_validate: my_df = my_df.with_columns( pl.struct([col, "notes"]).map_elements( lambda row: validate_stuff(row[col], row["notes"]) ).alias("notes") ) print(my_df)   Broken LazyFrame implementation  my_lf = pl.DataFrame( { "foo": ["a", "b", "c", "d"], "bar": ["w", "x", "y", "z"], "notes": ["1", "2", "3", "4"] } ).lazy() def validate_stuff(value, notes): # Any custom logic if value not in ["a", "b", "x"]: return f"FAILED {value} - PREVIOUS ({notes})" else: return notes cols_to_validate = ("foo", "bar") for col in cols_to_validate: my_lf = my_lf.with_columns( pl.struct([col, "notes"]).map_elements( lambda row: validate_stuff(row[col], row["notes"]) ).alias("notes") ) print(my_lf.collect())  (Ah, yeah, do notice that individually executing each iteration does work, so it's not making any sense to me why the for loop breaks) my_lf = my_lf.with_columns( pl.struct(["foo", "notes"]).map_elements( lambda row: validate_stuff(row["foo"], row["notes"]) ).alias("notes") ) my_lf = my_lf.with_columns( pl.struct(["bar", "notes"]).map_elements( lambda row: validate_stuff(row["bar"], row["notes"]) ).alias("notes") )  I have found a workaround using pl.col instead to achieve my desired result, but I would like to know whether Structs can be used the same way with LazyFrames right as I did with DataFrames, or it's actually a bug in this Polars version I'm using Polars 0.19.13, BTW. Thank you for your attention

Answer: It's more of a general "gotcha" with Python itself: Official Python FAQ It breaks because col ends up with the same value for every lambda One approach is to use a named/keyword arg: lambda row, col=col: validate_stuff(row[col], row["notes"])  shape: (4, 3) ┌─────┬─────┬───────────────────────────────────┐ │ foo ┆ bar ┆ notes │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str │ ╞═════╪═════╪═══════════════════════════════════╡ │ a ┆ w ┆ FAILED w - PREVIOUS (1) │ │ b ┆ x ┆ 2 │ │ c ┆ y ┆ FAILED y - PREVIOUS (FAILED c - … │ │ d ┆ z ┆ FAILED z - PREVIOUS (FAILED d - … │ └─────┴─────┴───────────────────────────────────┘
Question: If there is a long-running background task, cancel() method does not work as I expected (does not work at all). And I cannot find a way to actually stop the task, is that even possible or am I missing something about asyncio work? Documentation says: "Task.cancel() does not guarantee that the Task will be cancelled." But as far as I understand it is due to possibility of coroutine handling the CancelledError exception and suppressing cancelation which is not my case. I tried to run sample code beneath and got infinite loop running forever, cancel() there does not work. My understanding here is that hitting that await asyncio.sleep(1) on the next asyncio event loop iteration should trigger task cancellation, but it is not happening. How can I make already awaited background_task stop? async def background_task(): while True: print('doing something') await asyncio.sleep(1) async def main(): task = asyncio.create_task(background_task()) await task task.cancel() print('Done!') asyncio.run(main())  The output would be: doing something doing something doing something doing something doing something doing something ... 

Answer: Turns out its not possible in my case. If you await a long-running task - the caller will wait for task completion and therefore block (thanks to @mkrieger1 for the explanation in comments). To actually stop the task, you should either modify it and add some event or flag like @zShadowSkilled mentioned, or run it with asyncio.wait_for() by specifying a timeout.
Question: I am trying to install pycryptodome-3.19.0 and pycryptodomex-3.19.0 on Windows 11 and Python 3.10 in venv. Whl files were downloaded from pypi manually: pycryptodome-3.19.0-pp310-pypy310_pp73-win_amd64.whl pycryptodomex-3.19.0-pp310-pypy310_pp73-win_amd64.whl  Trying to install any of them I have an error: ERROR: pycryptodome(x)-3.19.0-pp310-pypy310_pp73-win_amd64.whl is not a supported wheel on this platform  But why? There are most suitable whls availavle on pypi Whls are for windows and for python 3.10. Why it can't be installed?

Answer: You've downloaded a PyPy wheel, but you're trying to install it on CPython. Look for a wheel tagged with cp instead of pp. Run pip debug -v to see a list of supported compatibility tags for your Python installation / platform.
Question: In Haskell we can use rankN types like so: rankN :: (forall n. Num n => n -> n) -> (Int, Double) rankN f = (f 1, f 1.0)  Is the same thing possible in python with mypy? I tried the following code in python 3.10.2 with mypy 1.7.1: I = TypeVar("I", int, float) def rankN(f: Callable[[I], I]) -> tuple[int, float]: return (f(1), f(1.0))  This produces the following errors, implying that f is specializing to float: Incompatible return value type (got "tuple[float, float]", expected "tuple[int, float]") [return-value] Argument 1 has incompatible type "float"; expected "int" [arg-type]  I'm not necessarily expecting this to work since the magic syntax in the Haskell case is the nested forall, but I don't know if there is a similar way to convey this to mypy if it is possible at all.

Answer: I’m not familiar with mypy, but my guess is that you can (and must) represent this as a protocol with a generic method, in order to scope the type variable to be per method call, rather than per invocation of rankN. from typing import Protocol, TypeVar class UnaryNumeric(Protocol): I = TypeVar("I", int, float) def __call__(self, input: I) -> I: pass def rankN(f: UnaryNumeric) -> tuple[int, float]: return (f(1), f(1.0))  Now rankN(lambda x: abs(x)) is accepted, while rankN(lambda x: str(x)) is rejected. I was surprised that rankN(lambda x: x / 2) is also accepted, since int isn’t really a subtype of float, but I guess it floats like a duck.
Question: I can produce the list [0, 2, 6, 12, 20, 30, 42, 56, 72, 90] using the following code: x = [] y = 0 for i in range(2,21,2): x.append(y) y += i  However I'm not sure how to convert this into list comprehension syntax of the form [expression for value in iterable if condition ]

Answer: You can assign to y inside the comprehension, using an assignment expession, i.e. using :=: y = 0 x = [y := y + i for i in range(0,20,2)]  Alternatively, you can make use of the fact that these are doubles of triangular numbers, and then you don't need y (but multiplication): x = [i * (i + 1) for i in range(10)]