eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I installed the GUI version of Stable Diffusion here. With it I was able to make 512 by 512 pixel images using my GeForce RTX 3070 GPU with 8 GB of memory:  However when I try to do the same thing with the command line interface, I run out of memory: Input: >> C:\SD\stable-diffusion-main>python scripts/txt2img.py --prompt "a close-up portrait of a cat by pablo picasso, vivid, abstract art, colorful, vibrant" --plms --n_iter 3 --n_samples 1 --H 512 --W 512 Error: RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 6.13 GiB already allocated; 0 bytes free; 6.73 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF If I reduce the size of the image to 256 X 256, it gives a result, but obviously much lower quality. So part 1 of my question is why do I run out of memory at 6.13 GiB when I have 8 GiB on the card, and part 2 is what does the GUI do differently to allow 512 by 512 output? Is there a setting I can change to reduce the load on the GPU? Thanks a lot, Alex

Answer: This might not be the only answer, but I solved it by using the optimized version here. If you already have the standard version installed, just copy the "OptimizedSD" folder into your existing folders, and then run the optimized txt2img script instead of the original: >> python optimizedSD/optimized_txt2img.py --prompt "a close-up portrait of a cat by pablo picasso, vivid, abstract art, colorful, vibrant" --H 512 --W 512 --seed 27 --n_iter 2 --n_samples 10 --ddim_steps 50 It's quite slow on my computer, but produces 512 X 512 images! Thanks, Alex
Question: I want to extract all key-value pairs from JSON file, I loaded it as a Python dictionary. I created this function below that stores all values. However, I am struggling to put them inside a list to store them like that. Any support is very appreciated.  json_example = {'name': 'TheDude', 'age': '19', 'hobbies': { 'love': 'eating', 'hate': 'reading', 'like': [ {'outdoor': { 'teamsport': 'soccer', } } ] } } # My code - Extract values def extract_values(dct, lst=[]): if not isinstance(dct, (list, dict)): lst.append(dct) elif isinstance(dct, list): for i in dct: extract_values(i, lst) elif isinstance(dct, dict): for v in dct.values(): extract_values(v, lst) return lst # Extract keys def matt_keys(dct): if not isinstance(dct, (list, dict)): return [''] if isinstance(dct, list): return [dk for i in dct for dk in matt_keys(i)] return [k+('_'+dk if dk else '') for k, v in dct.items() for dk in matt_keys(v)]  Current output: ['TheDude', '19', 'eating'...]  Desired output: [('name': 'TheDude'), ('age', '19'), ..., ('hobbies_love', 'eating'), ... , ('hobbies_like_outdoor_teamsport', 'soccer')]  Also if there is a more efficient or cleaner way to extract this, then it would be great.

Answer: Issues: Your recursive function currently does not pass the key as part of the function call. Also, you will need to deal with nesting when trying to create the key. Hints: We can assemble a list of all the keys that lead to a particular value (e.g. ['hobbies', 'love']) and then join the keys into a single string (e.g. hobbies_love). Solution: Here is your code with the changes implemented: def extract_values(dct, lst=[], keys=[]): if not isinstance(dct, (list, dict)): lst.append(('_'.join(keys), dct)) elif isinstance(dct, list): for i in dct: extract_values(i, lst, keys) elif isinstance(dct, dict): for k, v in dct.items(): keys.append(k) extract_values(v, lst, keys) keys.remove(k) return lst x = extract_values(json_example) print(x)  Output: The above code will produce the following desired output: [('name', 'TheDude'), ('age', '19'), ('hobbies_love', 'eating'), ('hobbies_hate', 'reading'), ('hobbies_like_outdoor_teamsport', 'soccer')]
Question: I have this data  ID A B C 0 0 True False False 1 1 False True False 2 2 False False True  And want to transform it into  ID group 0 0 A 1 1 B 2 2 C   I want to use the column names as value labels for the category column. There is a maximum of only one True value per row.  This is the MWE #!/usr/bin/env python3 import pandas as pd df = pd.DataFrame({ 'ID': range(3), 'A': [True, False, False], 'B': [False, True, False], 'C': [False, False, True] }) result = pd.DataFrame({ 'ID': range(3), 'group': ['A', 'B', 'C'] }) result.group = result.group.astype('category') print(df) print(result)  I could do df.apply(lambda row: ...magic.., axis=1). But isn't there a more elegant way with pandas' own tools?

Answer: You can use melt then a lookup based on the column where the values are true to get the results you are expecting df = df.melt(id_vars = 'ID', var_name = 'group') df.loc[df['value'] == True][['ID', 'group']]
Question: Input: import random I = 0 z = [] while I < 6: y = random. Choices(range(1,50)) if y in z: break z += y I += 1 print(z)  Output: [8, 26, 8, 44, 31, 22] I'm trying to make a Lotto numbers generator but I can't make the code to generate 6 numbers that do not repeat. As you can see in the Output 8 is repeating. I'm not clear why in the If statement the code does not check if the random y variable is already in the z list.

Answer: Use random.sample: >>> from random import sample >>> sample(range(1, 50), k=6) [40, 36, 43, 15, 37, 25]  It already picks k unique items from the range.
Question: I'm splitting a string with some separator, but want the separator matches as well: import re s = "oren;moish30.4.200/-/v6.99.5/barbi" print(re.split("\d+\.\d+\.\d+", s)) print(re.findall("\d+\.\d+\.\d+", s))  I can't find an easy way to combine the 2 lists I get: ['oren;moish', '/-/v', '/barbi'] ['30.4.200', '6.99.5']  Into the desired output: ['oren;moish', '30.4.200', '/-/v', '6.99.5', '/barbi'] 

Answer: From the re.split docs:  If capturing parentheses are used in pattern, then the text of all groups in the pattern are also returned as part of the resulting list.  So just wrap your regex in a capturing group: print(re.split(r"(\d+\.\d+\.\d+)", s))
Question: In Flet on Windows, I'm running the calc demo and trying to modify properties of the application window in Python. How do I change the size of the Flet window in code and specify that it should not be user resizable? (Ideally this post should be tagged with 'Flet' but the tag doesn't exist yet as the project's in it's infancy and I don't have the 1500 points required to created it.)

Answer: You can use this as an example import flet as ft def main(page: ft.Page): page.window.width = 200 # window's width is 200 px page.window.height = 200 # window's height is 200 px page.window.resizable = False # window is not resizable page.update() ft.app(target=main)
Question: I'm trying to migrate a setuptools-based project from the legacy setup.py towards modern pyproject.toml configuration. At the same time I want to keep well established workflows based on pip-compile, i.e., a requirements.in that gets compiled to a requirements.txt (for end-user / non-library projects of course). This has important benefits as a result of the full transparency:  100% reproducible installs due to pinning the full transitive closure of dependencies. better understanding of dependency conflicts in the transitive closure of dependencies.  For this reason I don't want to maintain the dependencies directly inside the pyproject.toml via a dependencies = [] list, but rather externally in the pip-compiled managed requirements.txt. This makes me wonder: Is there a way to reference a requirements.txt file in the pyproject.toml configuration, without having to fallback to a setup.py script?

Answer: In setuptools 62.6 the file directive was made available for dependencies and optional-dependencies. Use dynamic metadata: [project] dynamic = ["dependencies"] [tool.setuptools.dynamic] dependencies = {file = ["requirements.txt"]}  Note that the referenced file will use a requirements.txt-like syntax; each line must conform to PEP 508, so flags like -r, -c, and -e are not supported inside this requirements.txt. Also note that this capability is still technically in beta. Additionally:  If you are using an old version of setuptools, you might need to ensure that all files referenced by the file directive are included in the sdist (you can do that via MANIFEST.in or using plugins such as setuptools-scm, please have a look on [sic] Controlling files in the distribution for more information). Changed in version 66.1.0: Newer versions of setuptools will automatically add these files to the sdist.  If you want to use optional-dependencies, say, with a requirements-dev.txt, you will need to put an extra group, as follows (credit to Billbottom): [project] dynamic = ["dependencies", "optional-dependencies"] [tool.setuptools.dynamic] dependencies = {file = ["requirements.txt"]} optional-dependencies = {dev = { file = ["requirements-dev.txt"] }}  However:  Currently, when specifying optional-dependencies dynamically, all of the groups must be specified dynamically; one can not specify some of them statically and some of them dynamically.
Question: I need to upgrade the following code to some equivalent code based on psycopg version 3: import psycopg2 from sqlalchemy import create_engine engine = create_engine('postgresql+psycopg2://', creator=connector)  This psycopg2 URL worked like a charm, but: import psycopg # v3.1 from sqlalchemy import create_engine engine = create_engine('postgresql+psycopg://', creator=connector)  (I also tried the 'psycopg3' word without success) returns: Traceback (most recent call last): File "/tmp/ipykernel_1032556/253047102.py", line 1, in <cell line: 1> engine = create_engine('postgresql+psycopg://', creator=connector) File "<string>", line 2, in create_engine File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/util/deprecations.py", line 309, in warned return fn(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/create.py", line 534, in create_engine entrypoint = u._get_entrypoint() File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/url.py", line 661, in _get_entrypoint cls = registry.load(name) File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/util/langhelpers.py", line 343, in load raise exc.NoSuchModuleError( NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgresql.psycopg  So, how to properly create an SQLAlchemy engine based on psycopg (v3.x)? My sqlalchemy version is: '1.4.35' (tried version 1.4.40 but face an AttributeError: module 'sqlalchemy' has no attribute 'dialects' error). psycopg3 doc: https://www.psycopg.org/psycopg3/docs/api/ sqlalchemy doc: https://docs.sqlalchemy.org/en/14/core/engines.html

Answer: Just an update to the current answer: Sqlalchemy 2.0 has been released and it does support psycopg3. You need to upgrade to 2.0 to use it. Note the connection string will have to be changed from postgresql to postgresql+psycopg or SqlAlchemy will (at time of writing) try using psycopg2. See docs here for more info: https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#module-sqlalchemy.dialects.postgresql.psycopg
Question: I want to get a PDF file from s3 and then return it to the frontend from FastAPI backend. This is my code: @router.post("/pdf_document") def get_pdf(document : PDFRequest) : s3 = boto3.client('s3') file=document.name f=io.BytesIO() s3.download_fileobj('adm2yearsdatapdf', file,f) return StreamingResponse(f, media_type="application/pdf")  This API returns 200 status code, but it does not return the PDF file as a response.

Answer: As the entire file data are already loaded into memory, there is no actual reason for using StreamingResponse. You should instead use Response, by passing the file bytes (use BytesIO.getvalue() to get the bytes containing the entire contents of the buffer), defining the media_type, as well as setting the Content-Disposition header, so that the PDF file can be either viewed in the browser or downloaded to the user's device. For more details and examples, please have a look at this answer, as well as this and this. Related answer can also be found here. Additionally, as the buffer is discarded when the close()method is called, you could also use FastAPI/Starlette's BackgroundTasks to close the buffer after returning the response, in order to release the memory. Alternatively, you could get the bytes using pdf_bytes = buffer.getvalue(), then close the buffer using buffer.close() and finally, return Response(pdf_bytes, headers=.... Example from fastapi import Response, BackgroundTasks @app.get("/pdf") def get_pdf(background_tasks: BackgroundTasks): buffer = io.BytesIO() # BytesIO stream containing the pdf data # ... background_tasks.add_task(buffer.close) headers = {'Content-Disposition': 'inline; filename="out.pdf"'} return Response(buffer.getvalue(), headers=headers, media_type='application/pdf')  To have the PDF file downloaded rather than viewed in the borwser, use: headers = {'Content-Disposition': 'attachment; filename="out.pdf"'}
Question: I want to hide this warning UserWarning: pandas only support SQLAlchemy connectable(engine/connection) ordatabase string URI or sqlite3 DBAPI2 connectionother DBAPI2 objects are not tested, please consider using SQLAlchemy and I've tried import warnings warnings.simplefilter(action='ignore', category=UserWarning) import pandas  but the warning still shows. My python script read data from databases. I'm using pandas.read_sql for SQL queries and psycopg2 for db connections. Also I'd like to know which line triggers the warning.

Answer: I tried this and it doesn't work. import warnings warnings.filterwarnings('ignore')  Therefore, I used SQLAlchemy (as the warning message wants me to do so) to wrap the psycopg2 connection. I followed the instruction here: SQLAlchemy for psycopg2 documentation A simple example: import psycopg2 import sqlalchemy import pandas as pd conn = sqlalchemy.create_engine(f"postgresql+psycopg2://{user}:{pw}@{host}:{port}/{db}") query = "select count(*) from my_table" pd.read_sql(query, conn)  The warning doesn't get triggered anymore.
Question: @dataclass class Stock: symbol: str price: float = get_price(symbol)  Can a dataclass attribute access to the other one? In the above example, one can create a Stock by providing a symbol and the price. If price is not provided, it defaults to a price which we get from some function get_price. Is there a way to reference symbol? This example generates error NameError: name 'symbol' is not defined.

Answer: You can use __post_init__ here. Because it's going to be called after __init__, you have your attributes already populated so do whatever you want to do there: from typing import Optional from dataclasses import dataclass def get_price(name): # logic to get price by looking at `name`. return 1000.0 @dataclass class Stock: symbol: str price: Optional[float] = None def __post_init__(self): if self.price is None: self.price = get_price(self.symbol) obj1 = Stock("boo", 2000.0) obj2 = Stock("boo") print(obj1.price) # 2000.0 print(obj2.price) # 1000.0  So if user didn't pass price while instantiating, price is None. So you can check it in __post_init__ and ask it from get_price.  There is also another shape of the above answer which basically adds nothing more to the existing one. I just added for the records since someone might attempt to do this as well and wonder how is it different with the previous one: @dataclass class Stock: symbol: str price: InitVar[Optional[float]] = None def __post_init__(self, price): self.price = get_price(self.symbol) if price is None else price  You mark the price as InitVar and you can get it with a parameter named price in the __post_init__ method.
Question: I am trying to show the user a Modal after they display a button, which contains a dropdown select menu from which they can choose multiple options. This code has functioned in the past, but is not causing an exception. Specifically: [2022-09-02 22:30:47] [ERROR ] discord.ui.view: Ignoring exception in view <TestButtonView timeout=180.0 children=1> for item <Button style=<ButtonStyle.primary: 1> url=None disabled=False label='Test' emoji=None row=None> Traceback (most recent call last): File "C:\Users\adria\PycharmProjects\sblBot\venv\lib\site-packages\discord\ui\view.py", line 425, in _scheduled_task await item.callback(interaction) File "C:\Users\adria\PycharmProjects\sblBot\main.py", line 1131, in test_button_callback await interaction.response.send_modal(TestModal()) File "C:\Users\adria\PycharmProjects\sblBot\venv\lib\site-packages\discord\interactions.py", line 852, in send_modal await adapter.create_interaction_response( File "C:\Users\adria\PycharmProjects\sblBot\venv\lib\site-packages\discord\webhook\async_.py", line 220, in request raise HTTPException(response, data) discord.errors.HTTPException: 400 Bad Request (error code: 50035): Invalid Form Body In data.components.0.components.0: Value of field "type" must be one of (4,).  I have reduced my code to the minimal reproducible example of my issue. Here is the code for the Modal: class TestModal(discord.ui.Modal, title='Test'): def __init__(self, **kw): super().__init__(**kw) select = discord.ui.Select( placeholder='Select a tier.', options=[discord.SelectOption(label='test')] ) async def on_submit(self, interaction: discord.Interaction): await interaction.response.defer()  And here is the code for the view with the button (the f): class TestButtonView(discord.ui.View): def __init__(self, **kw): super().__init__(**kw) self.add_buttons() def add_buttons(self): test_button = discord.ui.Button(label='Test', style=discord.ButtonStyle.blurple) async def test_button_callback(interaction: discord.Interaction): await interaction.response.send_modal(TestModal()) test_button.callback = test_button_callback self.add_item(test_button)  And finally, the command to send the button view: @client.command(hidden=True) async def test(ctx): await ctx.send(view=TestButtonView()) 

Answer: ui.Modal does only support items with type 4, which is a ui.TextInput. Means ui.Select is not a supported item (yet).  See Component Types table: Discord API Documentation The error is quite inaccurate, but it's intended that the error handling is not done better because of future purposes.  See here: Add better errors for incorrect items added to modal
Question: I am trying to use the R reticulate package to convert a plotly graph to a static image. I am using save_image/kaleido. Link to documentation for save_image / kaleido Initial setup: install.packages("reticulate") reticulate::install_miniconda() reticulate::conda_install('r-reticulate-test', 'python-kaleido') reticulate::conda_install('r-reticulate-test', 'plotly', channel = 'plotly') reticulate::use_miniconda('r-reticulate-test')  Here is my (buggy) attempt: > library(plotly) > p <- plot_ly(x = 1:10) > save_image(p,"test.png") No trace type specified: Based on info supplied, a 'histogram' trace seems appropriate. Read more about this trace type -> https://plotly.com/r/reference/#histogram Error in py_run_string_impl(code, local, convert) : NameError: name 'sys' is not defined >  My query is : How do I fix the error that the name 'sys' is not defined? Funnily, if I do : > reticulate::repl_python() Python 3.10.6 (/root/.local/share/r-miniconda/envs/r-reticulate-test/bin/python) Reticulate 1.26.9000 REPL -- A Python interpreter in R. Enter 'exit' or 'quit' to exit the REPL and return to R. >>> import sys >>> exit > save_image(p,"test.png") No trace type specified: Based on info supplied, a 'histogram' trace seems appropriate. Read more about this trace type -> https://plotly.com/r/reference/#histogram >  then it works and produces the picture that I am seeking. Can someone tell me why I need to invoke repl_python, then import sys and exit it ? How can I fix this ? I need this since I need to create an automated script to create graphs.

Answer: As @Salim B pointed out there is a workaround documented to call import sys in Python before executing save_img(): p <- plot_ly(x = 1:10) reticulate::py_run_string("import sys") save_image(p, "./pic.png")
Question: I’m trying to create a Vertex AI endpoint with Monitoring enabled that can trigger a Vertex AI pipeline execution when one of the deployed models drops its performance. However, Vertex AI does not provide any built-in feature to do it. Is there a method to capture the alert thrown by Vertex AI Monitoring and trigger the Pipeline?

Answer:  The Vertex AI Model Monitoring jobs are logged as part of Cloud Logging 1. You can react to those loggings using log-based alerts 2. For that, you need to configure a notification channel to PubSub 3 Based on those PubSub message you can trigger a Cloud Function 4 The Cloud Function can initiate the Vertex AI Pipeline run to re-train the model 5  https://cloud.google.com/vertex-ai/docs/model-monitoring/using-model-monitoring#cloud-logging-info https://cloud.google.com/logging/docs/alerting/log-based-alerts https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.notificationChannels#NotificationChannel https://cloud.google.com/functions/docs/calling/pubsub https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.pipelineJobs/create
Question: I am trying to delete labels from projects, by using the UpdateProjectRequest. It says that it needs an 'update_mask', but I do not know how to created this mask. It is a google.protobuf.field_mask_pb2.FieldMask https://cloud.google.com/python/docs/reference/cloudresourcemanager/latest/google.cloud.resourcemanager_v3.types.UpdateProjectRequest What i have tried is: from google.cloud import resourcemanager_v3 from google.protobuf import field_mask_pb2 def sample_update_project(project_id): client = resourcemanager_v3.ProjectsClient() update_mask = field_mask_pb2.FieldMask(paths=["labels.newkey:None"]) resourcemanager_v3.Project=f"projects/{project_id}" operation = client.update_project(update_mask=update_mask) print("Waiting for operation to complete...") response = operation.result() print(response)  Thanks

Answer: As you already mentioned, when you want to delete all labels, it requires an update mask for the labels field. Reviewing this documentation, please note that if a request is provided, this should not be set. Also, you will find a code example that might help you. Additionally, you should import filed_mask_pb2 if you will update data with a FieldMask use as follows: from google.protobuf import field_mask_pb2  Having the main question in mind, one of two methods can be used to remove a label: Use the read-modify-write pattern to completely remove the label, which removes both the key and the value. To do this, perform these steps:  Use the resource's get() function to read the current labels. Use a text editor or programmatically change the returned labels to add or remove any necessary keys and their values. Call the patch() function on the resource to write the changed labels.  To keep the key and remove the value, set the value to null, but reviewing your code, you are currently using None. Also, consider the following documentation:Creating and managing labels and Method: projects.patch UPDATE Working again with your main question, here is a code that it was already tested to delete labels as you requested: import googleapiclient.discovery def sample_update_project_old(project_id): manager = googleapiclient.discovery.build('cloudresourcemanager', 'v1') request = manager.projects().get(projectId=project_id) project = request.execute() del project['labels']['key'] # replace 'key' with your actual key value request = manager.projects().update(projectId=project_id, body=project) project = request.execute() sample_update_project_old("your-project-id")