eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following User model in Django's models.py: class User(AbstractBaseUser): username = models.CharField(max_length=30, unique=True, primary_key=True) full_name = models.CharField(max_length=65, null=True, blank=True) email = models.EmailField( max_length=255, unique=True, validators=[EmailValidator()] )  When trying to update a username via shell python manage.py shell: userobj = User.objects.get(username="username1") userobj.username = user.lower() userobj.save()  It seems that it is trying to create a new user, for which the UNIQUE constraint of the e-mail is violated: django.db.utils.IntegrityError: UNIQUE constraint failed: data_app_user.email  Any solutions? Thank you!

Answer: This is one of the reasons not to use a username as primary key. Indeed, Django uses the primary key as token to check if two items are the same. If the primary key changes, and you save it, it will first try to update the user with that primary key, and if that fails, make an insert. This can thus produce several unwanted scenarios: one is that it duplicates data, in case the new primary key does not exist, but perhaps even worse: it can result in copying the data of the user to a record of the "new" username, and thus essentially overriding data of that user. As a general rule of thumb, it is better to consider primary keys "blackbox values". Yes, an AutoField works with an integer, but it is better to assume that it is a token where we don't know the details. The fact that it is an integer, is merely an implementation detail. In most database courses, one learns that the primary key is the (subset of) column(s) that makes a record unique, and from a database perspective, that is the right approach, but it is a "Djangoism" that the primary keys use a uniform type, and often don't store real information. This is also useful if you would later work with a GenericForeignKey [Django-doc], so that means there are essentially two options: using an AutoField, or a UUIDField, but regardless, it is better not to store information in the primary key. If you really want to insist, you can update the primary key with: User.objects.filter(pk='username1').update(pk='username2') Which will make a query that looks like: UPDATE app_name_user SET username = 'username2' WHERE username = 'username1' But that is probably not a good idea either, since now you omit validation, signals, and all other "mechanics" that Django has put in place. So I would strongly advise to rewrite the model and don't make username the primary key. You can still set it unique=True [Django-doc].
Question: I am trying to use Selenium to connect to existing instance of Firefox - the documentation says to use something like this options=webdriver.FirefoxOptions() options.binary_location = r'C:\Program Files\Mozilla Firefox\firefox.exe' webdriver_service = Service(r'c:\tmp\geckodriver.exe') driver = webdriver.Firefox(service = webdriver_service, service_args=['--marionette-port', '2828', '--connect-existing'])  However, I get the error  driver = webdriver.Firefox(service = webdriver_service, service_args=['--marionette-port', '2828', '--connect-existing']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: WebDriver.__init__() got an unexpected keyword argument 'service_args'  I see other questions about "unexpected keyword argument", they say latest versions of Selenium has other ways of passing arguments through Options. I tried options.add_argument('--marionette-port') options.add_argument('2828') options.add_argument('--connect-existing')  But it still seems to create a new instance of Firefox I have started firefox with the following arguments "C:\Program Files\Mozilla Firefox\firefox.exe" -marionette -start-debugger-server 2828 How do I fix this?  These are my versions Python version python --version Python 3.12.2  Selenium version pip show selenium Name: selenium Version: 4.18.1  Geckodriver version geckodriver --version geckodriver 0.34.0 (c44f0d09630a 2024-01-02 15:36 +0000)  Firefox 123.0 (64-bit) Windows 11

Answer: You should put the service_args parameter in the Service class and not on Firefox. I believe this should work options=webdriver.FirefoxOptions() options.binary_location = r'C:\Program Files\Mozilla Firefox\firefox.exe' webdriver_service = Service(r'c:\tmp\geckodriver.exe', service_args=['--marionette-port', '2828', '--connect-existing']) driver = webdriver.Firefox(service = webdriver_service)
Question: I sorted four similar lists. List d consistently takes much longer than the others, which all take about the same time: a: 33.5 ms b: 33.4 ms c: 36.4 ms d: 110.9 ms  Why is that? Test script (Attempt This Online!): from timeit import repeat n = 2_000_000 a = [i // 1 for i in range(n)] # [0, 1, 2, 3, ..., 1_999_999] b = [i // 2 for i in range(n)] # [0, 0, 1, 1, 2, 2, ..., 999_999] c = a[::-1] # [1_999_999, ..., 3, 2, 1, 0] d = b[::-1] # [999_999, ..., 2, 2, 1, 1, 0, 0] for name in 'abcd': lst = globals()[name] time = min(repeat(lambda: sorted(lst), number=1)) print(f'{name}: {time*1e3 :5.1f} ms') 

Answer: As alluded to in the comments by btilly and Amadan, this is due to how the Timsort sorting algorithm works. Detailed description of the algorithm is here. Timsort speeds up operation on partially sorted arrays by identifying runs of sorted elements.  A run is either "ascending", which means non-decreasing: a0 <= a1 <= a2 <= ... or "descending", which means strictly decreasing: a0 > a1 > a2 > ... Note that a run is always at least 2 long, unless we start at the array's last element.  Your arrays a, b and c each consist of just one run. The array d has 1 million runs. The reason why the descending run cannot be >= is to make the sort stable, i.e. keep the order of equal elements:  The definition of descending is strict, because the main routine reverses a descending run in-place, transforming a descending run into an ascending run. Reversal is done via the obvious fast "swap elements starting at each end, and converge at the middle" method, and that can violate stability if the slice contains any equal elements. Using a strict definition of descending ensures that a descending run contains distinct elements.  Python 3.11 has slightly improved version of timsort, sometimes called powersort, but it uses the same run detection and thus has the same performance.
Question: I have a problem using FastAPI, trying to dynamically define routes. It seems that the final route/handler defined overrides all the previous ones (see image below) Situation: I have a lot of Pydantic models as a list (imported from models.py: simplified in example), and would like to create a GET endpoint for each model. Something like this (my app is more complex — there is a reason it is structured as it is — but I've simplified it down to this, in order to find the problem, which still occurs): # models.py class Thing(BaseNode): value: str class Other(BaseNode): value: str model_list = [Thing, Other] # app.py from models import model_list app = FastApi() # Iterate over the models list and define a get function for model in models_list: @app.get(f"/{model.__name__.lower()}") def get() -> str: print(model.__name__) return f"Getting {model.__name__)   I'm obviously doing something wrong (or FastAPI references the handler function in some weird way... by module/function name(?) so the get() function is being re-defined for each iteration?) Any ideas, or better way to structure this? (I can't manually write out a new function for each model!) Thanks! UPDATE: Update: If I define the get() function inside another function, it works: for model in models_list: def get(model): def _get(): print("Getting", model) return f"Getting {model.__name__}" return _get app.get(.get(f"/{model.__name__.lower()}", name=f"{model.__name__}.View")(get(model)) 

Answer: Most likely the issue lies in the fact that you are using the same model reference for every endpoint created, and hence, it is always assigned the last value given in the for loop. You should rather use a helper function that returns an inner function, as demonstrated in Option 2 of this answer. Example from fastapi import FastAPI, APIRouter, Request from pydantic import BaseModel class One(BaseModel): value: str class Two(BaseModel): value: str app = FastAPI() models = [One, Two] def create_endpoint(m_name: str): async def endpoint(request: Request): print(request.url) return f"You called {m_name}" return endpoint for m in models: app.add_api_route(f"/{m.__name__.lower()}", create_endpoint(m.__name__), methods=["GET"])
Question: I am running into a weird import issue with Airflow. I want to create a module from which others can import. I also want to run unit tests on this module. However, I noticed that as soon as you import anything from the airflow package, it will try and run Airflow. Example: # myfile.py from airflow import DAG print("Hello world")  Then run it with python myfile.py, results in: (.venv) c:\Users\Jarro\Development\airflow-tryout-import>python myfile.py WARNING:root:OSError while attempting to symlink the latest log directory Traceback (most recent call last): File "c:\Users\Jarro\Development\airflow-tryout-import\myfile.py", line 1, in <module> from airflow import DAG File "C:\Users\Jarro\Development\airflow-tryout-import\.venv\Lib\site-packages\airflow\__init__.py", line 68, in <module> settings.initialize() File "C:\Users\Jarro\Development\airflow-tryout-import\.venv\Lib\site-packages\airflow\settings.py", line 559, in initialize configure_orm() File "C:\Users\Jarro\Development\airflow-tryout-import\.venv\Lib\site-packages\airflow\settings.py", line 237, in configure_orm raise AirflowConfigException( airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///C:\Users\Jarro/airflow/airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`.  Aside from the error itself, I am actually way more concerned that it seems I am not able to import things from Airflow, without there being side-effects (such as database initializations). Am I going all wrong about this? Is there another way I can import things from Airflow without these side effects, for example for typing purposes?

Answer: This is happening because of how impoorts work in python. You're importing from the airflow package which has an __init__ file. If you inspect it, there's a piece of code in it that does all of the airflow init stuff. lines 67-68 in __init__py: if not os.environ.get("_AIRFLOW__AS_LIBRARY", None):  settings.initialize() If you just want to import things for typing like you said, you can set the _AIRFLOW__AS_LIBRARY env variable to anything and it will not initialize.
Question: Having a Pandas dataframe with sample hierarchical data i.e. data = pd.DataFrame({ "manager_id": ["A", "A", "B", "A", "C", "A", "B"], "employee_id": ["B", "C", "C", "D", "E", "E", "E"] } )  Given that the data consists of all the descendent relationships for each manager. For example in each manager id (e.g. "A"), the employee id comprises both the employees (e.g. "B") directly managed by manager "A" and the employees managed by employee "B" (e.g. "C", "D"). How can this be represented into a directed graph in networkx? The edges of the graph should be [('A', 'B'), ('A', 'D'), ('B', 'C'), ('C', 'E')] 

Answer: Read the DataFrame as a directed graph with from_pandas_edgelist, then sort the nodes with topological_sort and only keep the edge with the parent that has the greatest topological index per child: G = nx.from_pandas_edgelist(data, source='manager_id', target='employee_id', create_using=nx.DiGraph) # topological order order = {n: i for i,n in enumerate(nx.topological_sort(G))} # {'A': 0, 'B': 1, 'D': 2, 'C': 3, 'E': 4} # for each node, only keep the parent that has the greatest topological index parents = {} for source, target in G.edges(): p = parents.setdefault(target, source) if order[source] > order[p]: parents[target] = source # parents # {'C': 'B', 'E': 'C', 'B': 'A', 'D': 'A'} # remove shorter edges G.remove_edges_from(G.edges - set(zip(parents.values(), parents.keys()))) # or # G = G.edge_subgraph(list(zip(parents.values(), parents.keys())))  Output:  Graph before filtering:  Variant You can also compute an order with topological_generations to have the same number for nodes of the same generation. order = {n: i for i, l in enumerate(nx.topological_generations(G)) for n in l} # {'A': 0, 'B': 1, 'D': 1, 'C': 2, 'E': 3}   In link to your other question, you could also compute the relative generation difference and only keep the edges with a generation difference of 1 between the two nodes: G = nx.from_pandas_edgelist(data, source='manager_id', target='employee_id', create_using=nx.DiGraph) order = {n: i for i, l in enumerate(nx.topological_generations(G)) for n in l} # {'A': 0, 'B': 1, 'D': 1, 'C': 2, 'E': 3} # compute the relative generation difference # keep the edges with a difference of 1 keep = [e for e in G.edges if order[e[1]]-order[e[0]] == 1] # [('A', 'B'), ('A', 'D'), ('B', 'C'), ('C', 'E'), ('F', 'G')] G = G.edge_subgraph(list(zip(parents.values(), parents.keys())))
Question: I am writing a site on Django, the purpose of this site is to create a test to assess students' knowledge I need help with outputting options for answers to a question I keep the questions in a list and the answer options in a nested list for example: questions = [ "question1","question2","question3"] answers = [[ "answer1","answer2",answer3 ],["answer1","answer2",answer3,answer4 ] , ["answer1","answer2",answer3 ]]` and I need this data to be displayed in the following format: question1 answer1 answer2 answer3 question2 answer1 answer2 answer3 answer4 question3 answer1 answer2 answer3  here is my code, it does not work very correctly, I know, I have not yet figured out how to implement it in Django tags  {% for question in questions %} <p>{{question}}</p> <ul> {% for answer in answers %} {% for current in answer %} <li><input type="radio" id="option{{ forloop.parentloop.counter }}_{{ forloop.counter }}" name="answer{{ forloop.parentloop.counter }}" value="{{ current }}">{{ current }}</li> {% endfor %} {% endfor %} </ul> {% endfor %} 

Answer: Perform the joining in the view: def my_view(request): questions = ['question1', 'question2', 'question3'] answers = [ ['answer11', 'answer12', 'answer13'], ['answer21', 'answer22', 'answer23', 'answer24'], ['answer31', 'answer32', 'answer33'], ] return render(request, 'some_template.html', {'qas': zip(questions, answers)}) and in the template work with: {% for question, answers in qas %} <p>{{question}}</p> <ul> {% for answer in answers %} <li><input type="radio" id="option{{ forloop.parentloop.counter }}_{{ forloop.counter }}" name="answer{{ forloop.parentloop.counter }}" value="{{ current }}">{{ current }}</li> {% endfor %} </ul> {% endfor %} That being said, what you do is primitive obsession [refactoring.guru]: expressing data in terms of lists, strings, etc. If data has a certain structure, it makes more sense to define a dedicated class for it, and add logic to it. For example to render the id="" for the option, and logic to parse back data.
Question: I'm attempting to determine the time based on the timezone specified in each row using Polars. Consider the following code snippet: import polars as pl from datetime import datetime from polars import col as c df = pl.DataFrame({ "time": [datetime(2023, 4, 3, 2), datetime(2023, 4, 4, 3), datetime(2023, 4, 5, 4)], "tzone": ["Asia/Tokyo", "America/Chicago", "Europe/Paris"] }).with_columns(c.time.dt.replace_time_zone("UTC")) df.with_columns( tokyo=c.time.dt.convert_time_zone("Asia/Tokyo").dt.hour(), chicago=c.time.dt.convert_time_zone("America/Chicago").dt.hour(), paris=c.time.dt.convert_time_zone("Europe/Paris").dt.hour() )  In this example, I've computed the time separately for each timezone to achieve the desired outcome, which is [11, 22, 6], corresponding to the hour of the time column according to the tzone timezone. Even then it is difficult to collect the information from the correct column. Unfortunately, the following simple attempt to dynamically pass the timezone from the tzone column directly into the convert_time_zone function does not work: df.with_columns(c.time.dt.convert_time_zone(c.tzone).dt.hour()) # TypeError: argument 'time_zone': 'Expr' object cannot be converted to 'PyString'  What would be the most elegant approach to accomplish this task?

Answer: The only way to do this which fully works with lazy execution is to use the polars-xdt plugin: df = pl.DataFrame( { "time": [ datetime(2023, 4, 3, 2), datetime(2023, 4, 4, 3), datetime(2023, 4, 5, 4), ], "tzone": ["Asia/Tokyo", "America/Chicago", "Europe/Paris"], } ).with_columns(pl.col("time").dt.replace_time_zone("UTC")) df.with_columns( result=xdt.to_local_datetime("time", pl.col("tzone")).dt.hour(), )  Result: Out[6]: shape: (3, 3) ┌─────────────────────────┬─────────────────┬────────┐ │ time ┆ tzone ┆ result │ │ --- ┆ --- ┆ --- │ │ datetime[μs, UTC] ┆ str ┆ i8 │ ╞═════════════════════════╪═════════════════╪════════╡ │ 2023-04-03 02:00:00 UTC ┆ Asia/Tokyo ┆ 11 │ │ 2023-04-04 03:00:00 UTC ┆ America/Chicago ┆ 22 │ │ 2023-04-05 04:00:00 UTC ┆ Europe/Paris ┆ 6 │ └─────────────────────────┴─────────────────┴────────┘  https://github.com/pola-rs/polars-xdt If you don't need lazy execution, then as other answers have suggested, you can iterate over the unique values of your 'time_zone' column
Question: I want to calculate rolling_sum, but not over x rows above the current row, but over the x rows below the current row. My solution is to sort the dataframe with descending=True before applying the rolling_sum and sort back to descending=False. My solution: import polars as pl # Dummy dataset df = pl.DataFrame({ "Date": [1, 2, 3, 4, 5, 1, 2, 3, 4, 5], "Close": [-1, 1, 2, 3, 4, 4, 3, 2, 1, -1], "Company": ["A", "A", "A","A", "A", "B", "B", "B", "B", "B"] }) # Solution using sort twice ( df .sort(by=["Company", "Date"], descending=[True, True]) .with_columns( pl.col("Close").rolling_sum(3).over("Company").alias("Cumsum_lead") ) .sort(by=["Company", "Date"], descending=[False, False]) )  Is there a better solution? With better I mean:  more computational efficient and/or less code / easier to read  Thanks! EDIT: I just thought of one other solution which is avoids sorting / reversing the column altogether: using shift ( df .with_columns( pl.col("Close") .rolling_sum(3) .shift(-2) .over("Company").alias("Cumsum_lead")) ) 

Answer: You can avoid sorting the rows and instead reverse the specific column twice using pl.Expr.reverse. ( df .with_columns( pl.col("Close") .reverse().rolling_sum(3).reverse() .over("Company").alias("Cumsum_lead") ) )  For readability, this could also be wrapped into a helper function. def rolling_sum_lead(expr: pl.Expr, window_size: int) -> pl.Expr: return expr.reverse().rolling_sum(window_size).reverse() ( df .with_columns( rolling_sum_lead(pl.col("Close"), 3).over("Company").alias("Cumsum_lead") ) )  Note. On my machine, this takes 124 µs ± 5.67 µs per loop in contrast to 205 µs ± 6.9 µs per loop for the solution using pl.DataFrame.sort.
Question: How do I pass in a parameter name into a function as an argument? def edit_features_retry(editType,data): AGOLlayer.edit_features(editType=data) edit_features_retry("adds", datalayer)  error: TypeError: edit_features() got an unexpected keyword argument 'editType'  If I do it this way as a string: def edit_features_retry(editType,data): AGOLlayer.edit_features(editType=data) edit_features_retry(adds, datalayer)  error: NameError: name 'adds' is not defined 

Answer: You can use dictionary unpacking. Here's a simple example: def f(a = 1, b = 2): print(f"{a = }, {b = }") f(b=5) # output: "a = 1, b = 5" kwargs = {"b": 5} f(**kwargs) # output: "a = 1, b = 5"  Which can be applied to your scenario: def edit_features_retry(editType, data): AGOLlayer.edit_features(**{editType: data}) edit_features_retry("adds", datalayer)
Question: I created a separate Python file for numerical calculations when implementing a quantum circuit. Now, I'm trying to use IBM Quantum Lab to apply this code to a circuit using Qiskit. %matplotlib inline import math import decomposition_2qubit as d2 import numpy as np from qiskit import QuantumCircuit, execute, Aer from qiskit.visualization import plot_histogram from qiskit.extensions import * from qiskit.quantum_info import Statevector matrix = (1/2)*np.array([ [1,1,1,1], [1,-1,1,-1], [1,1,-1,-1], [1,-1,-1,1]]) qc = QuantumCircuit(2,2) #qc.x(0) #qc.x(1) qc.barrier() d2.twoqubit_to_single(qc, matrix) ket = Statevector(qc) ket.draw('latex') qc.draw('mpl')  However, when I try to import the decomposition_2qubit.py file, I encounter this error. How can I resolve this? Traceback (most recent call last): Cell In[1], line 4 import decomposition_2qubit as d2 ModuleNotFoundError: No module named 'decomposition_2qubit' Use %tb to get the full traceback.  I have the 'decomposition_2qubit.py' file located in the same folder as the current Jupyter Notebook file I am using. Additionally, since IBM Quantum Lab appears to use Jupyter Notebook as its base, I tried searching for solutions when encountering such errors in Jupyter Notebook, but unfortunately, I couldn't resolve the issue.

Answer: You are probably facing an issue with the file path. As suggested here, you can either specify your own import path explicitly or just prepend path modifiers like .. and/or /.
Question: This is my DataFrame: import pandas as pd df = pd.DataFrame( { 'a': ['x', 'x', 'x', 'x', 'y', 'y', 'y', 'y', 'z', 'z', 'z', 'p', 'p', 'p', 'p'], 'b': [1, -1, 1, 1, -1, 1, 1, -1, -1, -1, -1, 1, 1, 1, 1] } )  And this the expected output. I want to create column c:  a b c 0 x 1 first 1 x -1 first 2 x 1 first 3 x 1 first 4 y -1 second 5 y 1 second 6 y 1 second 7 y -1 second 11 p 1 first 12 p 1 first 13 p 1 first 14 p 1 first  Groups are defined by column a. I want to filter df and choose groups that either their first b is 1 OR their second b is 1. I did this by this code: df1 = df.groupby('a').filter(lambda x: (x.b.iloc[0] == 1) | (x.b.iloc[1] == 1))  And for creating column c for df1, again groups should be defined by a and then if for each group first b is 1 then c is first and if the second b is 1 then c is second. Note that for group p, both first and second b is 1, for these groups I want c to be first. Maybe the way that I approach the issue is totally wrong.

Answer: A generic method that works with any number of positions for the first 1: d = {0: 'first', 1: 'second'} s = (df.groupby('a')['b'] .transform(lambda g: g.reset_index()[g.values==1] .first_valid_index()) .replace(d) ) out = df.assign(c=s).dropna(subset=['c'])  Notes:  if you remove the replace step you will get an integer in c if you use map in place of replace you can ignore the positions that are not defined as a dictionary key  Output:  a b c 0 x 1 first 1 x -1 first 2 x 1 first 3 x 1 first 4 y -1 second 5 y 1 second 6 y 1 second 7 y -1 second 11 p 1 first 12 p 1 first 13 p 1 first 14 p 1 first  Example from comments: df = pd.DataFrame({'a': ['x', 'x', 'x', 'x', 'y', 'y', 'y', 'y', 'z', 'z', 'z', 'p', 'p', 'p', 'p'], 'b': [1, -1, 1, 1, -1, 1, 1, -1, -1, -1, 1, 1, 1, 1, 1]}) d = {0: 'first', 1: 'second'} s = (df.groupby('a')['b'] .transform(lambda g: g.reset_index()[g.values==1] .first_valid_index()) .map(d) ) out = df.assign(c=s).dropna(subset=['c']) a b c 0 x 1 first 1 x -1 first 2 x 1 first 3 x 1 first 4 y -1 second 5 y 1 second 6 y 1 second 7 y -1 second 11 p 1 first 12 p 1 first 13 p 1 first 14 p 1 first  You can also only filter the rows with: m1 = df.groupby('a').cumcount().le(1) m2 = df['b'].eq(1) out = df.loc[df['a'].isin(df.loc[m1&m2, 'a'])]