eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to updates some psycopg2 code to psycopg3. I'm trying to do a selection based on a set of values passed from Python (joining with an existing table). Without the join, a simplified example is: with connection.cursor() as cur: sql = "WITH sources (a,b,c) AS (VALUES %s) SELECT a,b+c FROM sources;" data = (('hi',2,0), ('ho',5,2)) cur.execute(sql, (data,) ) print(cur.fetchone());  I get an error ProgrammingError: syntax error at or near "'("(hi,2,0)","(ho,5,2)")'" LINE 1: WITH sources (a,b,c) AS (VALUES '("(hi,2,0)","(ho,5,2)")') S...  The psycopg2 code used extras.execute_values instead, which is not available in psycopg3. Is there a way to pass the values for an intermediate table using psycopg3?

Answer: One way to do it: import psycopg con = psycopg.connect("dbname=test host=localhost user=postgres") with con.cursor() as cur: rs = [] sql = "SELECT %s, %s + %s" data = [('hi',2,0), ('ho',5,2)] cur.executemany(sql, data, returning=True ) while True: rs.append(cur.fetchone()) if not cur.nextset(): break print(rs) [('hi', 2), ('ho', 7)]  From here psycopg cursor classes:  executemany( ... )   Note   Using the usual fetchone(), fetchall(), you will be able to read the records returned by the first query executed only. In order to read the results of the following queries you can call nextset() to move to the following result set.   A typical use case for executemany(returning=True) might be to insert a bunch of records and to retrieve the primary keys inserted, taken from a PostgreSQL sequence. In order to do so, you may execute a query such as INSERT INTO table VALUES (...) RETURNING id. Because every INSERT is guaranteed to insert exactly a single record, you can obtain the list of the new ids using a pattern such as:  cur.executemany(query, records) ids = [] while True: ids.append(cur.fetchone()[0]) if not cur.nextset(): break   Warning   More explicitly, fetchall() alone will not return all the values returned! You must iterate on the results using nextset().  UPDATE data_combined = [y for x in data for y in x] data_combined ['hi', 2, 0, 'ho', 5, 2] qry = sql.Composed( [sql.SQL("select a, b + c from ( "), sql.SQL('VALUES '), sql.SQL(",").join( sql.SQL("({})").format(sql.SQL(',').join(sql.Placeholder() * len(data[0]))) * len(data)), sql.SQL(") as t(a, b, c)")]) print(qry.as_string(con)) select a, b + c from ( VALUES (%s,%s,%s),(%s,%s,%s)) as t(a, b, c) cur.execute(qry, data_combined) cur.fetchall() [('hi', 2), ('ho', 7)]  Used sql.Composed to build up a query with a variable number of VALUES and placeholders. Combined the tuple of tuples into a flat list and passed it to the query.
Question: As the title suggests, I would like to create a list in a list from a second list. For this I prepared the following example: list1 = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] list2 = [3, 4, 2, 6] solution = [[1, 2, 3], [4, 5, 6, 7], [8, 9], [10, 11, 12, 13, 14, 15]]  I have already found a solution, but I find it very cumbersome and I am sure there is a faster and easier solution to my problem. list1 = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] list2 = [3, 4, 2, 6] j = 0 solution= [] for i in list2: solution.append(list1[j:i+j]) j += i  Thanks a lot already guys

Answer: One possible solution is using itertools.islice: from itertools import islice list1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] list2 = [3, 4, 2, 6] out = iter(list1) out = [list(islice(out, v)) for v in list2] print(out)  Prints: [[1, 2, 3], [4, 5, 6, 7], [8, 9], [10, 11, 12, 13, 14, 15]]   Another using := operator: list1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] list2 = [3, 4, 2, 6] out = 0 out = [list1[out:(out:=out+a)] for a in list2] print(out)
Question: I'm new to Python, so, I'm sorry if my question may seems dumb. I'm trying to convert GIF files to APNG using this tool. As it says in the bottom of this page, to convert GIF to APNG you must execute the following command: $ vertopal convert GIF_INPUT_FILE --to apng  It works fine when I run the command in the terminal, but I cannot implement it in my Python code. Here is my code: import subprocess discord_sticker_converter = subprocess.call("vertopal convert funny cat.gif --to apng") print(discord_sticker_converter)  This is what is printed in the output: usage: vertopal [options] <command> [<args>] vertopal: error: unrecognized arguments: cat.gif 2  How should I call the convert command properly?

Answer: Let's talk about your command: "vertopal convert funny cat.gif --to apng"  The name "funny cat.gif" has space in it. subprocess calls shlex.split() to split this string into a list of string tokens and the result is: >>> shlex.split("vertopal convert funny cat.gif --to apng") ['vertopal', 'convert', 'funny', 'cat.gif', '--to', 'apng']  As you can see, the file name is broken into 2 tokens: "funny" and "cat.gif". That is why it failed. To fix this situation, you need to properly quote your file name: >>> shlex.split("vertopal convert 'funny cat.gif' --to apng") ['vertopal', 'convert', 'funny cat.gif', '--to', 'apng']  The function subprocess.call() is flexible: it can take a single string and try to split it, or it can take a list of strings. That means both of these will work: subprocess.call("vertopal convert 'funny cat.gif' --to apng") subprocess.call(['vertopal', 'convert', 'funny cat.gif', '--to', 'apng'])  The first form is more compact and is what you would do in the terminal. The second form is more explicit. Which form to use is your choice. As for me, I will choose the second form because the Zen of python said  Explicit is better than implicit.
Question: Given the following directory structure: app/ └── array/ ├── test/ │ ├── __init__.py │ └── test_array.py ├── __init__.py └── functions.py  When I run pytest from the app directory, I get the following error: ERROR collecting array/test/test_array.py _______________________________________ ImportError while importing test module '/path/to/app/array/test/test_array.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) E ModuleNotFoundError: No module named 'array.test'; 'array' is not a package  Simply renaming the module to arrays works, but I don't understand what's wrong with array.

Answer: Because it collides with the stdlib module of the same name.
Question: I have Ubuntu 22.04 with python 3.10. When I try to open jupyter notebook from terminal this error occurrs: Traceback (most recent call last): File "/home/anaconda3/lib/python3.10/site-packages/notebook/services/sessions/sessionmanager.py", line 9, in <module> import sqlite3 File "/home/anaconda3/lib/python3.10/sqlite3/__init__.py", line 57, in <module> from sqlite3.dbapi2 import * File "/home/anaconda3/lib/python3.10/sqlite3/dbapi2.py", line 27, in <module> from _sqlite3 import * ImportError: /home/anaconda3/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so: undefined symbol: sqlite3_trace_v2 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/anaconda3/bin/jupyter-notebook", line 5, in <module> from notebook.notebookapp import main File "/home/anaconda3/lib/python3.10/site-packages/notebook/notebookapp.py", line 83, in <module> from .services.sessions.sessionmanager import SessionManager File "/home/anaconda3/lib/python3.10/site-packages/notebook/services/sessions/sessionmanager.py", line 12, in <module> from pysqlite2 import dbapi2 as sqlite3 ModuleNotFoundError: No module named 'pysqlite2'  When I checked sqlite3, it is intalled in home/anaconda3/lib/python3.10/sqlite3/ and it contains dbapi2.py. Should I somehow reorganize the folders? PS: When I tried pip install pysqlite2 another error occurred: ERROR: Could not find a version that satisfies the requirement pysqlite2 (from versions: none) ERROR: No matching distribution found for pysqlite2

Answer: It looks like you performed a manual installation of Jupyter (and required dependencies) directly in your home directory. I'm assuming /home/anaconda3/ is your home directory, and anaconda3 is just a badly chosen name. Rename the lib subdirectory: mv lib lib_aside  You might want to do something similar for include/ or bin/ directories directly in your home as well (but check first what is in bin/ ; if that is all Python/Jupyter related, or that there's software you actually need that is not default installable). Comment out any PYTHONPATH setting in your .bashrc that points to that lib/ directory (and subdirectories). Comment out any PATH alteration to the bin/ in your home directory as well. If you don't need it, comment-out the section with Conda stuff as well. Install Jupyter from your package manager. E.g. sudo apt install python3-notebook  Start a new terminal, and test: which python which jupyter  These should point to /usr/bin Start your notebook: jupyter notebook
Question: I'm facing an issue which my dag cannot be imported, but cannot figure out why: from airflow.sensors.sql import SqlSensor import pendulum from airflow.decorators import task,dag @dag( dag_id = "database_monitor", schedule_interval = '*/10 * * * *', start_date=pendulum.datetime(2023, 7, 16, 21,0,tz="UTC"), catchup=False,) def Pipeline(): check_db_alive = SqlSensor( task_id="check_db_alive", conn_id="evergreen", sql="SELECT pg_is_in_recovery()", success= lambda x: x == False, poke_interval= 60, #timeout = 60 * 2, mode = "reschedule", ) @task() def alert_of_db_inrecovery(): import requests # result = f"Former primary instance is in recovery, task_instance_key_str: {kwargs['task_instance_key_str']}" data = {"@key":"kkll", "@version" : "alertapi-0.1", "@type":"ALERT", "object" : "Testobject", "severity" : "MINOR", "text" : str("Former primary instance is in recovery") } requests.post('https://httpevents.systems/api/sendAlert',verify=False,data=data) check_db_alive >> alert_of_db_inrecovery dag = Pipeline()  I get this error:  AttributeError: '_TaskDecorator' object has no attribute 'update_relative' 

Answer: You need to call the Python task flow operator i.e change check_db_alive >> alert_of_db_inrecovery to check_db_alive >> alert_of_db_inrecovery() check correct code from airflow.sensors.sql import SqlSensor import pendulum from airflow.decorators import task, dag @dag( dag_id="database_monitor", schedule_interval='*/10 * * * *', start_date=pendulum.datetime(2023, 7, 16, 21, 0, tz="UTC"), catchup=False, ) def Pipeline(): check_db_alive = SqlSensor( task_id="check_db_alive", conn_id="evergreen", sql="SELECT pg_is_in_recovery()", success=lambda x: x == False, poke_interval=60, # timeout = 60 * 2, mode="reschedule", ) @task def alert_of_db_inrecovery(): import requests # result = f"Former primary instance is in recovery, task_instance_key_str: {kwargs['task_instance_key_str']}" data = {"@key": "kkll", "@version": "alertapi-0.1", "@type": "ALERT", "object": "Testobject", "severity": "MINOR", "text": str("Former primary instance is in recovery") } requests.post('https://httpevents.systems/api/sendAlert', verify=False, data=data) check_db_alive >> alert_of_db_inrecovery() dag = Pipeline()  Ref: https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html
Question: I want to either click or select a single row in a table, retrieve the corresponding row's data, and then process it further with handlers. But the documentation is slim at the moment, and I have never used Quasar or Vue.js before. Here's my attempt: from nicegui import ui columns = [ {'name': 'title', 'label': 'Title', 'field':'title', 'required': True, 'align': 'left'}, {'name': 'author', 'label': 'Author', 'field':'author', 'align': 'left'}, {'name': 'year', 'label': 'Year', 'field':'year', 'sortable': True, 'align': 'left'}, ] rows = [ {'title': 'Some title A', 'url' : 'https://example.com/search?q=A', 'author': 'Alice', 'year': 2023}, {'title': 'Some title B', 'url' : 'https://example.com/search?q=B', 'author': 'Bob', 'year': 2022}, {'title': 'Some title C', 'url' : 'https://example.com/search?q=C', 'author': 'Carol', 'year': 2021} ] table = ui.table(title='Example table', columns=columns, rows=rows, row_key='name', pagination=5, selection='single', on_select=lambda e: ui.notify(e.selection)) table.add_slot('body', r''' <q-tr :props="props" @row-click="rowClick"> <q-td> <a :href="props.row.url">{{ props.row.title }}</a> </q-td> <q-td>{{ props.row.author }}</q-td> <q-td>{{ props.row.year }}</q-td> </q-tr> ''') table.on('rowClick', lambda *args: print(args), [None, None, None]) # c.f. nicegui issues #664, #672, #1095 ui.run()  The code above does not work. There are no exceptions, but nothing is happening if I click on a row (and the check boxes to select a row are not shown; I might have disabled them inadvertently using my template). If i I had to choose, I would prefer to handle a click event.

Answer: Here is a working example table: table = ui.table(title='Example table', columns=columns, rows=rows, row_key='title', pagination=5, selection='single') table.add_slot('body-cell-title', r'<td><a :href="props.row.url">{{ props.row.title }}</a></td>') table.on('rowClick', lambda e: print(e.args))  It looks like your body template broke the row-click event, but I'm not 100% sure. Anyway, using the "body-cell-title" slot, you don't need to write the template for the whole body, but for the "title" cell only. And note that, when using features like filters and selection, the row_key should match a column name with unique values. Here I'm using "title".
Question: I have this array: my_array = np.arange(1216800).reshape(2, 100, 78, 78)  The shape now is: (2, 100, 78, 78) and I want to reorder to : (100, 78, 78, 2). I tried something like: my_array[:, :, 2, :], my_array[:, :, :, 3] = my_array[:, :, :, 3], my_array[:, :, 2, :].copy()  to swap first those columns, but I am receiving the same array. I saw this but whatever I try, I am having the same array.

Answer: Here is the code that you want: import numpy as np my_array = np.arange(1216800).reshape(2, 100, 78, 78) reordered_array = np.transpose(my_array, (1, 2, 3, 0)) print(reordered_array.shape) # Output: (100, 78, 78, 2)
Question: In my code, I have certain values like my_list = [725.998474, 0.0, 0.0, 0.0, 0.0, 789.507934, 792.585388, 801.612916, 799.38916, 809.280518, 809.186036, 811.899414, .... , 412.314528]  In my code, I want to interpolate the points where the list is 0.0 because they are outliers. But it is not working as interpolation only works for empty values. How can I erase those 0.0 values and do interpolation?

Answer: Using the data you provided, we can create a "cleaned" list of the x and y data using numpy. You said that the values equal to 0 are the outliers, but checking equality with floating point numbers can lead to issues, so I used np.isclose. With the outliers removed, you can interpolate the cleaned data. import numpy as np from scipy.interpolate import make_interp_spline import matplotlib.pyplot as plt plt.close("all") y = np.array([725.998474, 0.0, 0.0, 0.0, 0.0, 789.507934, 792.585388, 801.612916, 799.38916, 809.280518, 809.186036, 811.899414, 412.314528]) x = np.arange(len(y)) outliers = np.isclose(y, 0) y_clean = y[~outliers] x_clean = x[~outliers] spline = make_interp_spline(x_clean, y_clean) y_interped = spline(x) fig, ax = plt.subplots() ax.plot(x_clean, y_clean, ".", label="cleaned", zorder=3) ax.plot(x, y_interped, label="interpolated") ax.legend() ax.set_xlabel("x") ax.set_ylabel("y") fig.show()   If, as @Reinderien suggested, your actual condition is that values below 100, for example, are outliers, then you could change it to that condition (i.e. outliers = y < 100).
Question: Is it possible to access the parent class name within a static class? For example, how to I print the parent class name in the bar method below? class Static: def bar(self): parent_name = ??? print(parent_name) class A: object = Static() def foo(self): A.object.bar() class B: object = Static() def foo(self): B.object.bar() A().foo() B().foo() 

Answer: You're looking for the __set_name__ method. It is called on every static attribute of a class when the class is finished constructing. One of the parameters is the "owner" class (A or B), so all you need is to store that reference. Note that this requires a separate Static instance for each parent class, but you're doing that already. class Static: def bar(self): print(self.parent_name) def __set_name__(self, owner, name): self.parent_name = owner.__name__ class A: object = Static() def foo(self): A.object.bar() class B: object = Static() def foo(self): B.object.bar() A().foo() B().foo()   I personally avoid these magic methods when possible, so you might also consider taking @JonSG's advice and put the owner class in the Static constructor instead. This way the relationship is much clearer, at the cost of adding them after-the-fact. class Static: def __init__(self, owner): self.parent_name = owner.__name__ def bar(self): print(self.parent_name) class A: def foo(self): A.object.bar() A.object = Static(A) class B: def foo(self): B.object.bar() B.object = Static(B) A().foo() B().foo()  Alternatively, one might leverage a metaclass to do some of the work. This has the potential to also simplify the implementations of A() and B() class Static: def __init__(self, parent_name) -> None: self.parent_name = parent_name def bar(self): print(self.parent_name) class AB_Meta(type): def __init__(cls, name, bases, dct): cls.object = Static(name) class A(metaclass=AB_Meta): def foo(self): A.object.bar() class B(metaclass=AB_Meta): def foo(self): B.object.bar() A().foo() B().foo()  Should give you: A B
Question: I want to merge records of my dataframe if dates are same.Here in the below example I want to merge date (13,14,15), (25,26), (30,31) together as there are continuous dates. I want to break the merging of record if there is any single day break.  cust date description CUST123 2020-06-13 observed increased loss rate CUST123 2020-06-13 cut performed job CUST123 2020-06-14 working tight area CUST123 2020-06-15 production shut neighbouring app CUST123 2020-07-17 loss pressure slow gain trip CUST123 2020-08-25 established circulation load CUST123 2020-08-26 performed sticky test CUST123 2020-08-28 job meeting prior low energy CUST123 2020-08-30 performed maintenance service CUST123 2020-08-31 reconnected control line  expected output  cust date description CUST123 2020-06-13 observed increased loss rate cut performed job working tight area production shut neighbouring app CUST123 2020-07-17 loss pressure slow gain trip CUST123 2020-08-25 established circulation load performed sticky test CUST123 2020-08-28 job meeting prior low energy CUST123 2020-08-30 performed maintenance service reconnected control line 

Answer: In order to merge records of a dataframe if dates are same, you could do: merged_df = df.groupby(['cust', 'date'])['description'].apply(' '.join).reset_index()  which outputs: cust date description 0 CUST123 2020-06-13 observed increased loss rate cut performed job 1 CUST123 2020-06-14 working tight area 2 CUST123 2020-06-15 production shut neighbouring app 3 CUST123 2020-07-17 loss pressure slow gain trip 4 CUST123 2020-08-25 established circulation load 5 CUST123 2020-08-26 performed sticky test 6 CUST123 2020-08-28 job meeting prior low energy 7 CUST123 2020-08-30 performed maintenance service 8 CUST123 2020-08-31 reconnected control line   EDIT: If you want to merge the consecutive dates, keeping the first date of the consecutive range, you could it like this: # Sort DataFrame by 'date' (in case 'df' is not already sorted) df.sort_values('date', inplace=True) # Initialize variables merged_data = [] prev_row = None # Loop through the rows for _, row in df.iterrows(): if prev_row is None or row['cust'] != prev_row['cust'] or (row['date'] - prev_row['date']).days > 1: merged_data.append({'cust': row['cust'], 'date': row['date'], 'description': row['description']}) else: merged_data[-1]['description'] += ' ' + row['description'] prev_row = row # Create merged DataFrame merged_df = pd.DataFrame(merged_data) print(merged_df)  Output: 0 CUST123 2020-06-13 observed increased loss rate cut performed job... 1 CUST123 2020-07-17 loss pressure slow gain trip 2 CUST123 2020-08-25 established circulation load performed sticky ... 3 CUST123 2020-08-28 job meeting prior low energy 4 CUST123 2020-08-30 performed maintenance service reconnected cont...
Question: I have a DataFrame, which have 3 numeric columns A,B,C. I need to extract only those rows where values in all these 3 columns A,B,C is more than 40% of their row average. df = pd.DataFrame([['AA',10,8,12],['BB',10,2,18],['CC',10,6,14]], columns=['ID','A', 'B', 'C']) print(df) ID A B C 0 AA 10 8 12 1 BB 10 2 18 2 CC 10 6 14  I mean the following: For Row 1, the mean of A,B,C is 30/3=10, and I want that in Row 1 all values, be it A or B or C should be more than 40% of 10, i.e; 4. Similarly, for Row 2 and Row 3. In case even one element is less than that, we remove that row. My attempt: I used any() function, but that does't help me when involving average of columns. I always get empty DF. df = df[(df[['A','B','C']] > (0.4*df[['A','B','C']].mean(axis=1))).all(1)] print(df) ID A B C  I was expecting this:  ID A B C 0 AA 10 8 12 2 CC 10 6 14  The average of all rows is 10, so if I would have hardcoded it, it would work, like this: df[(df[['A','B','C']] > 0.4*10).all(1)]  How can I do this dynamically? Thanks.

Answer: Another possible solution: a = df[['A', 'B', 'C']].values df[(a > 0.4*(a.mean(1)[:, None])).all(1)]  Output:  ID A B C 0 AA 10 8 12 2 CC 10 6 14