eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: import pytest from moto import mock_s3 @pytest.fixture(scope='module') def s3(): with mock_s3(): os.environ['AWS_ACCESS_KEY_ID'] = 'test' os.environ['AWS_SECRET_ACCESS_KEY'] = 'test' os.environ['AWS_DEFAULT_REGION'] = 'us-east-1' s3 = boto3.resource('s3') s3.create_bucket(Bucket='test_bucket') yield s3  This code was working, but is now throwing an exception Cannot import name mock_s3 from moto. What am I doing wrong?

Answer: Simply replace your import of mock_s3 with from moto import mock_aws and use with mock_aws(): instead. Moto was recently bumped to version 5.0, and you were probably running 4.x before. https://github.com/getmoto/moto/blob/master/CHANGELOG.md If you check the change log, you will see that an important breaking change was made:  All decorators have been replaced with a single decorator: mock_aws
Question: I am doing some web scraping using playwright-python>=1.41, and have to launch the browser in a headed mode (e.g. launch(headless=False). For CI testing, I would like to somehow cache the headed interactions with Chromium, to enable offline testing:  First invocation: uses Chromium to make real-world HTTP transactions Later invocations: uses Chromium, but all HTTP transactions read from a cache  How can this be done? I can't find any clear answers on how to do this.

Answer: It might solve your problem using HAR-file recording:  Run the first test while recording a HAR-file Storing the HAR-file as an artifact, in your repo or similar in your CI environment Running test again with recorded HAR-file  Here is how to do that with playwright==1.41.1 and pytest-playwright==0.3.3: import pathlib import pytest from playwright.sync_api import Browser, Playwright CACHE_DIR = pathlib.Path(__file__).parent / "cache" @pytest.fixture(name="example_har", scope="session") def fixture_example_har(playwright: Playwright) -> pathlib.Path: har_file = CACHE_DIR / "example.har" with ( playwright.chromium.launch(headless=False) as browser, browser.new_page() as page, ): page.route_from_har(har_file, url="*/**", update=True) page.goto("https://example.com/") return har_file def test_caching(browser: Browser, example_har: pathlib.Path) -> None: with browser.new_context(offline=True) as context: page = context.new_page() page.route_from_har(example_har, url="*/**") page.goto("https://example.com/")
Question: I am currently using the seaborn library in Python to create a facetted stacked barplot from a pandas dataframe named averages with columns ['Name', 'Period', 'Value', 'Solver']. Here is the code I use to create the plot I want. p = so.Plot(data = averages, x = 'Period', y = 'Value', color = 'Name').add(so.Bar(), so.Stack(), suptitle='Inventory levels') p = p.facet(col='Solver', order=['spse', 'mp2', 'mels'])  I am searching for a way to add a general title to the plot i.e. a title above each subplot, like the function matplotlib.pyplot.suptitle function does for example. I know that the function seaborn.objects.Plot.label has a title= option, but when I use it, this puts the same title above each subplot of the facetted graph.

Answer: You can provide an existing Matplotlib figure or axes for drawing the plot using so.Plot.on. This gives you access to the underlying matplotlib.figure.Figure object to which the suptitle can be added. import matplotlib.pyplot as plt import seaborn.objects as so import pandas as pd df = pd.DataFrame({ "x": [1, 2, 3, 4, 5, 1, 2, 3, 4, 5], "y": [1, 2, 3, 4, 5, 1, 2, 4, 8, 16], "group": ["a", "a", "a", "a", "a", "b", "b", "b", "b", "b"], }) fig = plt.Figure() fig.suptitle("Suptitle") ( so.Plot(df, x="x", y="y") .add(so.Line()) .facet(col="group") .on(fig) .plot() )
Question: I'm trying to create a density plot with a gradient fill in python like this:  I've attempted to do so using this code: plt.figure(figsize=(6, 1)) sns.kdeplot(data=df, x='Overall Rating', fill=True, palette='viridis') ...and sns.kdeplot(data=df, x='Overall Rating', fill=True, cmap='viridis')  Neither work, both outputting this plot:  I've searched all over for a method to do this in python, but no luck. I've tried implementing the methods from this answer by @Diziet Asahi but can't wrap my head around it. Any help would really be appreciated!

Answer: You just need to grab the ax: ax = sns.kdeplot(...) and then execute the inner part of the for loop in @Diziet Asahi's solution with that ax. import matplotlib import matplotlib.pyplot as plt import seaborn as sns import numpy as np sns.set_theme(style='white') iris = sns.load_dataset('iris') ax = sns.kdeplot(data=iris, x='petal_length', fill=True) sns.despine() cmap = 'turbo' im = ax.imshow(np.linspace(0, 1, 256).reshape(1, -1), cmap=cmap, aspect='auto', extent=[*ax.get_xlim(), *ax.get_ylim()], zorder=10) path = ax.collections[0].get_paths()[0] patch = matplotlib.patches.PathPatch(path, transform=ax.transData) im.set_clip_path(patch) plt.show()
Question: In the numpy documentation for the qr factorization (https://numpy.org/doc/stable/reference/generated/numpy.linalg.qr.html), the default mode is 'reduced'. I know the theory of the 'complete' mode and expected it to be the default. What does the mode 'reduced' mean mathematically? I could not find any information about that in the documentation.

Answer: Let's summarize the answer given by @stéphane-laurent, so people can find the results more easily: Let , the QR-decomposition is defined as . If , the values of  can be dropped, because the last  rows of  are zero. I.e. we multiply the last  columns of  with zeros. Therefore, we can just drop these columns to get the same result. In order for the dimensions to work out, we also need to remove the rows that are zero (i.e. the last  rows) of . We end up with , where  and .
Question: Apparently I have the opposite of everyone else's problem... I would like to take the mean of a pandas dataframe, and I would like to have the result return NaN if there are ANY NaNs in the frame. However, it seems like neither np.mean nor np.nanmean do this. Example code: b = pd.DataFrame([[1,2],[math.nan,4]]) print(b) print(np.mean(b)) print(np.nanmean(b))  Result:  Expected Result: 

Answer: It can be done using pandas DataFrame.mean() function : b.mean(axis=None,skipna=False)  According to the documentation of numpy, the reason that you don't get nan from mean function is that when numpy sees the input of mean() is not multiarray.ndarray, it tries to use mean function from that data type if possible. if type(a) is not mu.ndarray: try: mean = a.mean except AttributeError: pass else: return mean(axis=axis, dtype=dtype, out=out, **kwargs)  In your case it is like calling the mean() function of pandas with axis=None. Which is like running the following code: b.mean(axis=None) # 2.333333  If you are insisting on using numpy.mean() to calculate mean, you can convert your DataFrame to numpy array before callin np.mean(). array = b.to_numpy() np.mean(array) # nan
Question: Imagine I have a dataframe like this: With lists of elements in a single string. data = {'Col1': ["apple, banana, orange", "dog, cat", "python, java, c++"], 'Col2': ["banana, lemon, blueberry", "bird, cat", "R, fortran"] } df = pd.DataFrame(data) df  How can I create a Col3 with the intersection of elements in Col1 and Col2 Expected output: data = {'Col1': ["apple, banana, orange", "dog, cat", "python, java, c++"], 'Col2': ["banana, lemon, blueberry", "bird, cat", "R, fortran"], 'Col3': ["banana", "cat", NA] } df = pd.DataFrame(data) df 

Answer: Using a list comprehension and set intersection: df['Col3'] = [', '.join(set(a.split(', ')) & set(b.split(', '))) for a,b in zip(df['Col1'], df['Col2'])]  Output:  Col1 Col2 Col3 0 apple, banana, orange banana, lemon, blueberry banana 1 dog, cat bird, cat cat 2 python, java, c++ R, fortran  If you want NAs on empty intersections: df['Col3'] = [x if (x:=', '.join(set(a.split(', ')) & set(b.split(', ')))) else pd.NA for a,b in zip(df['Col1'], df['Col2'])]  Output:  Col1 Col2 Col3 0 apple, banana, orange banana, lemon, blueberry banana 1 dog, cat bird, cat cat 2 python, java, c++ R, fortran <NA>
Question: When utilizing Telethon to send Telegram messages with monospace or HTML text in Python, the parse_mode option (for example parse_mode='markdown' or parse_mode='html') works well during initial message sending. However, when attempting to edit a previously sent message, the parse_mode option does not appear to be available. How can I effectively edit a message and incorporate monospace or HTML-formatted text? Example: Sending a message containing monospace string sent_message = client.send_message(To_id, "the authentication code `8632496`", parse_mode='markdown')  Then editing the message and sending new text containing a monospace string client(EditMessageRequest( peer=To_id, id=sent_message.id, message="the new authentication code `9832798237`" ))  the result:  I am aware that using entities is an option. For instance: entities = [MessageEntityCode(offset=9, length=18)] # specifying a code block entity client(EditMessageRequest( peer=To_id, id=sent_message.id, message='This is a monospace string.', no_webpage=True, entities=entities ))  However, my text is long and contains multiple monospace strings originating from various sources, for example: text = f"something {variable1}, something {variable2}" Hence, identifying the starting point and length of each monospace string is challenging. I'd greatly appreciate any guidance on how to approach this issue, including methods or best practices for effectively managing monospace or HTML text when editing Telegram messages using Telethon.

Answer: client(EditMessageRequest( is a old, not preferred way of editing a message.  You should use client.edit_message method, in your example: client.edit_message(tst_id, first.id, 'Oeps, the code is `1234789`')  This way, you don't even need to specify parse_mode it's the same as the message your editing.  Full code to test this: from telethon import TelegramClient import asyncio import time client = TelegramClient('anon', '1234567', '0123456789abcdef0123456789abcdef') client.start() async def main(): tst_id = 1234 first = await client.send_message(tst_id, "the authentication code `8632496`", parse_mode='markdown') time.sleep(3) second = await client.edit_message(tst_id, first.id, 'Oeps, the code is `1234789`') loop = asyncio.get_event_loop() loop.run_until_complete(main())   Result: message changes after 3 seconds:
Question: So using the following example: import numpy as np X = np.array([[1,10,np.nan,0],[2,10,np.nan,0],[3,10,np.nan,0]])  I can drop the all-NaN or all-0 columns like so:  X = X[:,~np.all(np.isnan(X), axis=0)] X = X[:,~np.all(X==0, axis=0)]  I would also like to drop the zero variance columns, is there a good way to do so using the same concise notation? Thanks!

Answer: You can. import numpy as np X[:, np.var(X, axis=0) != 0] array([[ 1., nan], [ 2., nan], [ 3., nan]])  This calculates the variance of each column (axis=0) and then returns the ones not equal to 0.
Question: I need to login to aruba appliance and get report data using the following api at aruba-airwave endpoint. I have this code so far: import xml.etree.ElementTree as ET # libxml2 and libxslt import requests # HTTP requests import csv import sys # --------------------------------------------------------------------------- # Constants # --------------------------------------------------------------------------- # Login/password for Airwave (read-only account) LOGIN = 'operator' PASSWD = 'verylongpasswordforyourreadonlyaccount' # URL for REST API LOGIN_URL = 'https://aruba-airwave.example.com/LOGIN' AP_LIST_URL = 'https://aruba-airwave.example.com/report_detail?id=100' # Delimiter for CSV output DELIMITER = ';' # HTTP headers for each HTTP request HEADERS = { 'Content-Type' : 'application/x-www-form-urlencoded', 'Cache-Control' : 'no-cache' } # --------------------------------------------------------------------------- # Fonctions # --------------------------------------------------------------------------- def open_session(): """Open HTTPS session with login""" ampsession = requests.Session() data = 'credential_0={0}&credential_1={1}&destination=/&login=Log In'.format(LOGIN, PASSWD) loginamp = ampsession.post(LOGIN_URL, headers=HEADERS, data=data) return {'session' : ampsession, 'login' : loginamp} def get_ap_list(session): output=session.get(AP_LIST_URL, headers=HEADERS, verify=False) ap_list_output=output.content return ap_list_output def main(): session=open_session() get_ap_list(session) main()  I get this error: get() takes no keyword arguments  any ideas what I am doing wrong here or suggestion for another method?

Answer: The problem you are running into is because your open_session() function returns a dict and a dicts args are typically for accessing a key within the dict. since your session parameter is a dict in this case, if you want to use the elements in session, you need to access them by their key that you defined in open_session(). # session['session'].get(include your arguments for the get request here) or # session['login'].get() # although the data attributed to your already obtained session # should be handled by the session hence storing the login result # isn't really necessary unless you are checking to see if the # request was successful but that can be done other ways without # actually storing the result. def get_ap_list(session): output=session['session'].get(AP_LIST_URL, headers=HEADERS, verify=False) ap_list_output=output.content return ap_list_output  I haven't verified that this code works however it does address the error you are currently receiving.
Question: I want my Service Bus triggered Python Azure Function to make the message it has retrieved from the queue available on the queue again to be retried. I want this to happen without having to raise an exception in the function. However as per the documentation:  In non-C# functions, exceptions in the function results in the runtime calls abandonAsync in the background. If no exception occurs, then completeAsync is called in the background.  the completeAsync is going to be called if the function returns successfully. I am aware that the Service Bus Receiver has a function that allows you to abandon a message so that it is made available on the queue again. But I don't believe this can work in a situation when the Azure Function has already retrieved the message for you. Also, the ServiceBusMessage object that is received in the Azure function doesn't seem to have a method that lets you 'abandon' or otherwise return the message to the queue. Most other questions regarding this issue online seem to be specific to C#, so I've not had much look trying to find an answer. My Azure function is calling a third party API that unfortunately is prone to erroring which is outside of my control. However, when looking at error reports, I don't want this Azure Function to be always be at the top of the list of failing functions. Having to raise an exception in my function when this API call fails seems excessive when it is not an unexpected outcome. I would like to return the function as a success, since from my perspective there is nothing wrong with the function, but the message on the queue needs to be tried again.

Answer: The short version is - this can't be done from the Python language worker at this time. Currently, only the .NET Function workers (both in-proc and isolated) are able to bind to the ServiceBusMessageActions parameter needed to explicitly complete messages. The only way for other language workers to complete a message is for processing to succeed or for it to fail enough times that the delivery count is exceeded or one of the other service-side DLQ actions are triggered and the message is dead-lettered. Work is planned to bring the message actions functionality to the other language workers but, to my knowledge, there has not yet been a timeline shared.
Question: I am learning to use a python bytecode preprocessing mechanism. However it complains when processing these lines of code: def q_sample(self, x_start, t, noise=None): noise = default(noise, lambda: torch.randn_like(x_start)) return (extract_into_tensor(self.sqrt_alphas_cumprod, t, x_start.shape) * x_start + extract_into_tensor(self.sqrt_one_minus_alphas_cumprod, t, x_start.shape) * noise)  It said RuntimeError: Preprocessing issues detected: Nonlocal variables are not supported but q_sample() defined in /opt/dev/training/stable_diffusion/ldm/models/diffusion/ddpm.py:467 defines nonlocal variable: x_start  I don't understand why x_start could be nonlocal here. If yes, how can I modify it to be a not Nonlocal variable? Thanks!

Answer: Because there is a nested function(the lambda) which has access to a variable(the x_start) from the outer function(the q_sample())'s namespace. To solve that you can make the x_start local to the lambda by passing it as the default parameter of the lambda like: def q_sample(self, x_start, t, noise=None): noise = default(noise, lambda x_start=x_start: torch.randn_like(x_start))  Here is simplified code for inspection: def outer(a): return lambda: print(a) closure_function = outer(10) print(closure_function.__closure__) print(closure_function.__code__.co_freevars)  output: (<cell at 0x102c422c0: int object at 0x10422e718>,) ('a',)
Question: According to documentation here if I ask, just for place_id, the call should be for free. I tried various versions of call, but all of them either did not work or gave me not just id, but also other basic info such as address, location, rating, etc. Bard told me that it is ok and as I asked just for id, the calls will really cost nothing. However it is not possible to see cost of each call and based on whole day billing I am afraid I paid also for these calls. Should this be really free and even though address, location, etc. should be priced as basic info, it is actually given for free or should I change sth in my code? My code: def find_place(query, api_key): base_url = "https://maps.googleapis.com/maps/api/place/textsearch/json" params = { "query": query, "inputtype": "textquery", "fieldMask": "place_id", # Only retrieve the place ID "key": api_key # Include API key } response = requests.get(base_url, params=params) return response.json() 

Answer: Finally I found out how to do that properly. Now the response is only the needed field(s): def find_place(query, api_key): # Define the API endpoint url = 'https://places.googleapis.com/v1/places:searchText' # Define the headers headers = { 'Content-Type': 'application/json', 'X-Goog-Api-Key': api_key, # Replace 'API_KEY' with your actual Google Places API key 'X-Goog-FieldMask': 'places.id' } # Define the data payload for the POST request data = { 'textQuery': query } # Make the POST request response = requests.post(url, headers=headers, json=data) # Check if the request was successful if response.status_code == 200: # Process the response print(response.json()) else: print(f"Error: {response.status_code}, {response.text}") return json.loads(response.text)
Question: I have a df something like below Name Series ============================= A A1 B B1 A A2 A A1 B B2  I need to convert the series to a list which should be assigned to each Name like a dict or json obj as something like below { "A": ["A1", "A2"], "B": ["B1", "B2"] }  So far I have tried using groupby, but it just groups everything a separate dict test = df.groupby("Series")[["Name"]].apply(lambda x: x)  The above code gives an output as a df like  Series Name A 0 A1 2 A2 3 A1 B 1 B1 4 B2  Any help is much appreciated Thanks,

Answer: First drop_duplicates to ensure having , then groupby.agg as list: out = df.drop_duplicates().groupby('Name')['Series'].agg(list).to_dict()  Or with unique: out = df.groupby('Name')['Series'].agg(lambda x: x.unique().tolist()).to_dict()  Output: {'A': ['A1', 'A2'], 'B': ['B1', 'B2']} If you have other columns, ensure to only keep those of interest: out = (df[['Name', 'Series']].drop_duplicates() .groupby('Name')['Series'].agg(list).to_dict() )  sorting the lists: out = (df.groupby('Name')['Series'] .agg(lambda x: sorted(x.unique().tolist())).to_dict() )  Example: # input Name Series 0 A Z1 1 B B1 2 A A2 3 A Z1 4 B B2 # output {'A': ['A2', 'Z1'], 'B': ['B1', 'B2']}
Question: I was trying to create different arrays using Numpy's fromfunction(). It was working fine until I faced this issue. I tried to make an array of ones using fromfunction()(I know I can create it using ones() and full()) and here is the issue : array = np.fromfunction(lambda i, j: 1, shape=(2, 2), dtype=float) print(array)  Surprisingly, the output of this function is this : 1  Which is expected to be : [[1. 1.] [1. 1.]]  When I change the input function by adding zero times i, It works just fine. array = np.fromfunction(lambda i, j: i*0 + 1, shape=(2, 2), dtype=float) print(array)  The output of this code is : [[1. 1.] [1. 1.]]  My main question is how does fromfunction() actually behaves? I passed the same function with 2 different representations and the output is completely different.

Answer: The callable function is passed two arrays, not repeatedly called with two numbers: i=[ [ 0.0, 0.0 ], [ 1.0, 1.0 ] ] j=[ [ 0.0, 1.0 ], [ 0.0, 1.0 ] ]  It returns what it is asked to provide ... just ONCE ... to be the ultimate result of np.fromfunction. So an input of 1 returns just 1 whereas an input of i*0+1 returns [ [ 0.0, 0.0 ], [ 1.0, 1.0 ] ] * 0 + 1  which is the (broadcast) array [ [ 1.0, 1.0 ], [ 1.0, 1.0 ] ]
Question: I'm encountering an issue where Django doesn't seem to be reading environment variables when running in a Docker-compose environment. Strangely, the environment variables work fine for PostgreSQL but not for Django. I've tried both the env_file and environment options in my Docker-compose file, and I've also experimented with django-dotenv without success. Here's a snippet of my code: print(f" {os.environ.get('SECRET_KEY')} {os.environ.get('DEBUG')} {os.environ.get('DATABASE_NAME')} ") # The above print statement outputs None for all environment variables SECRET_KEY = os.environ.get("SECRET_KEY") # ...  version: '3.9' services: db: # ... web: build: ./src command: gunicorn myapp.wsgi:application --access-logfile /logs/gunicorn/access.log --error-logfile /logs/gunicorn/error.log --bind 0.0.0.0:8000 volumes: # ... # env_file: #not working # - .env environment: - SECRET_KEY=${SECRET_KEY} - DATABASE_NAME=${DATABASE_NAME} # ... (other environment variables) nginx: # ...  SECRET_KEY=3df-81ioo^5cx(p9cl$)s%m3mlu3t7*yh#1lr5h0po4_sab3*5 DATABASE_NAME=mydb DATABASE_USER=postgres DATABASE_PASS=postgres DATABASE_HOST=db DATABASE_PORT=5432 DEBUG=0 

Answer: I believe I've identified the root cause of the issue in my Dockerfile. The problem lies in the sequence of commands during the Docker image build process. Specifically, I observed that the Dockerfile attempts to run python manage.py collectstatic --noinput and python manage.py migrate before Docker Compose copies the environment variables. This results in errors during the build process. To address this, I propose a solution: remove the collectstatic and migrate commands from the Dockerfile and instead include them in the Docker Compose configuration. Adjust the command parameter in the docker-compose.yml (in web part):  command: "sh -c 'python manage.py migrate && python manage.py collectstatic --noinput && gunicorn -c conf/gunicorn.py techgeeks.wsgi:application'"  Here is the previous version of my Dockerfile: # set work directory WORKDIR /app # set environment variables ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 # install dependencies # RUN pip install --upgrade pip COPY . . RUN pip install --upgrade pip RUN pip install -r requirements.txt # The following two lines caused issues, so they were removed RUN python manage.py collectstatic --noinput RUN python manage.py migrate