eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following code fragment: import seaborn import matplotlib.pyplot as plt plt.bar(df['Col1'], df['Col2'], width = 0.97, color=seaborn.color_palette("coolwarm", df['Col1'].shape[0], 0.999))  Now, my bars are colored in the blue-red spectrum (as given by the parameter coolwarm). How can I change the distribution between these two colors and their order, for example, to get 80% of all bars red and only the rest (i.e., 20%) in blue? (Now it is 50-50% ratio)

Answer: There's no built in way to do this using seaborn or the matplotlib colormaps, but here's a solution that seems to do the trick import seaborn as sns import matplotlib.pyplot as plt import pandas as pd import numpy as np ## function that generates a color palette def gen_palette(n_left, n_right, cmap_name="coolwarm", desat=0.999): """ n_left: number of colors from "left half" of the colormap n_right: number of colors from the "right half" cmap_name: name of the color map to use (ideally one of the diverging palettes) return: palette, list of RGB-triples """ palette_1 = sns.color_palette(palette=cmap_name, n_colors=2 * n_left, desat=desat)[:n_left] palette_2 = sns.color_palette(palette=cmap_name, n_colors=2 * n_right, desat=desat)[n_right:] return palette_1 + palette_2 ## generate example data N = 20 rng = np.random.default_rng(seed=42) y_vals = 10 * rng.random(N) df = pd.DataFrame( {"Col1": np.arange(N), "Col2": y_vals} ) ## build the color palette with the desired blue-red split n_red = round(0.8 * N) palette = gen_palette(N - n_red, n_red) ## plot plt.bar(df['Col1'], df['Col2'], width=0.97, color=palette)
Question: so i am running this code def login_or_join(request): if request.method == "POST": option = request.POST.get("option") print('post request recieved') if option == "1": return login_screen(request) if option == '2': return in_game(request) return render(request,"login_or_join.html")  and def login_screen() looks like this def login_screen(request): if request.method == "POST": username = request.POST.get("username") password = request.POST.get("password") print(username) print(password) user = authenticate(request, username=username, password=password) print(user) if user is not None: return redirect('join_lobby') else: return render(request,'login_page.html',) return render(request, 'login_page.html')  Whenever I click "option 1" it runs login_screen but in a way I don't want it to. it seems to just follow that request.method == "POST" and prints username and password immediately, meaning it is setting username and password immediately making any log in attempt wrong. But I don't want it to set those (or print them) until I've pressed the button on the next page. Further more when I hit "enter" or "log in" it doesn't reshow the page with the error message, it just goes back login_or_join(). I feel like I am taking crazy pills as I've been working on this website for a while and this is the first time I'm having this kind of issue. I've tried messing with it but feel I've been staring at it too long. Any help would be appreciated!

Answer: I guess the problem is that you are calling login_screen with request as parameter. But that variable is the parameter received on login_or_join so is the POST request with the option, neither a GET nor a POST with the username and password. I think in this case you can just name the url and do: if option == "1": return redirect(login_screen_name) if option == "2": return redirect(in_game_screen_name)
Question: I have a series of airflow DAGs which re-use some of the task dependencies. For example DAG 1: T1 >> T2 DAG 2: T1 >> T2 >> T3 DAG 3: T1 >> T2 >> T3 >> [T4, T5, T6] >> T7  I would like to store the dependencies from DAG 1 (which in this model, are being used by every other DAG) and re-use them when declaring the dependencies for the other DAGs, like so: def dag_1_dependencies(): T1 >> T2 DAG 2: dag_1_dependencies() >> T3 DAG 3: dag_1_dependencies() >> T3 >> [T4, T5, T6] >> T7  The problem is that dependencies themselves aren't a value, so I can't return them with a method. Calling dag_1_dependencies() does nothing. Is there a way to circumvent this?

Answer: If tasks t1 and t2 are always the same tasks, you can generate these tasks + dependencies outside the DAG. To add additional tasks/dependencies following t2, you need a reference to that object to configure the additional dependencies. For example: def generate_t1_t2() -> BaseOperator: """Generates tasks + dependencies and returns the last task so that additional dependencies can be set.""" t1 = EmptyOperator(task_id="t1") t2 = EmptyOperator(task_id="t2") t1 >> t2 return t2 with DAG(dag_id="dag2", start_date=datetime(2025, 1, 1), schedule=None): last_task = generate_t1_t2() t3 = EmptyOperator(task_id="t3") last_task >> t3  The function generate_t1_t2 returns the last operator in the chain (t2), which allows configuring additional dependencies such as last_task >> t3. Your full question can therefore be written as: from datetime import datetime from airflow import DAG from airflow.models import BaseOperator from airflow.operators.empty import EmptyOperator def generate_t1_t2() -> BaseOperator: """Generates tasks + dependencies and returns the last task so that additional dependencies can be set.""" t1 = EmptyOperator(task_id="t1") t2 = EmptyOperator(task_id="t2") t1 >> t2 return t2 with DAG(dag_id="dag1", start_date=datetime(2025, 1, 1), schedule=None): generate_t1_t2() with DAG(dag_id="dag2", start_date=datetime(2025, 1, 1), schedule=None): last_task = generate_t1_t2() last_task >> EmptyOperator(task_id="t3") with DAG(dag_id="dag3", start_date=datetime(2025, 1, 1), schedule=None): last_task = generate_t1_t2() ( last_task >> EmptyOperator(task_id="t3") >> [EmptyOperator(task_id="t4"), EmptyOperator(task_id="t5"), EmptyOperator(task_id="t6")] >> EmptyOperator(task_id="t7") )
Question: I am trying to make dashboard in flask by connecting it with SQL server and getting these errors. I confirm there are no null values and I checked by removing the column as well from query but still not working. Code is - import pandas as pd import pyodbc from flask import Flask, render_template, jsonify app = Flask(__name__) # SQL Server Connection Details conn_str = ( "DRIVER={SQL Server};" "SERVER=xyz;" "DATABASE=xyz;" "UID=xyz;" "PWD=xyz;" ) # Fetch Data from SQL Server def fetch_data(): try: conn = pyodbc.connect(conn_str) query = """ SELECT TicketDate, Technician, Open_Tickets, Closed_Tickets, Created_Today, Closed_Today, Created_Hourly FROM Technician_Ticket_Stats """ df = pd.read_sql(query, conn) conn.close() # Debugging logs print("Fetched data successfully:") print(df.head()) df['TicketDate'] = df['TicketDate'].astype(str) # Convert date for JSON return df.to_dict(orient="records") except Exception as e: print("Error fetching data:", e) return [] @app.route("/") def index(): return render_template("index.html") @app.route("/get_data") def get_data(): try: data = fetch_data() return jsonify(data) except Exception as e: return jsonify({"error": str(e)}), 500 if __name__ == "__main__": app.run(host='127.0.0.1', port=8050, debug=True)here 

Answer: Add this line so that if there are NaT values in the 'TicketDate' column, it converts to None rather than throwing an error. df['TicketDate'] = df['TicketDate'].fillna(pd.NaT).apply( lambda x: x.strftime('%Y-%m-%d') if pd.notna(x) else None )
Question: I have a FastAPI setup of the form: class Foo(sqlmodel.SQLModel, table=True): id: typing.Optional[int] = sqlmodel.Field(primary_key=True) data: str @pydantic.field_validator("data", mode="before") def serialize_dict(cls, value): if isinstance(value, dict): return json.dumps(value) return value @app.post("/foos") def create_foo(foo: Foo, session: sqlmodel.Session = fastapi.Depends(get_session)): session.add(foo) session.commit() return fastapi.Response()  I then POST { "data": { "bar": 5 } }  to /foos. However, this is throwing a SQL exception because the data value couldn't be bound. After putting in some logging statements, I discovered that foo.data is a dict and not a str. In addition, I confirmed that my validator is never called. Since SQLModel inherits from pydantic.BaseModel, I would have thought I could use such a validator. What am I missing? This is sqlmodel 0.0.23 with pydantic 2.10.6.

Answer: This is one of the oldest issues with SQLModel (see e.g. this github issue from 2022). Validators are not called on models with table=True.
Question: I would like to rotate an image in opencv before putting it into mediapipe. I took a transpose using numpy but, although the type is still numpy.uint8, mediapipe complains, import cv2 import numpy as np import mediapipe as mp image_file_name = "IMG_0237.JPG" cvImage = cv2.imread(image_file_name) cvImage = np.transpose(cvImage, axes=[1,0,2]) #without this line code runs fine print(cvImage.dtype) mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=cvImage)  returns, uint8 --------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[4], line 9 7 cvImage = np.transpose(cvImage, axes=[1,0,2]) #without this line code runs fine 8 print(cvImage.dtype) ----> 9 mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=cvImage) TypeError: __init__(): incompatible constructor arguments. The following argument types are supported: 1. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint8]) 2. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.uint16]) 3. mediapipe.python._framework_bindings.image.Image(image_format: mediapipe::ImageFormat_Format, data: numpy.ndarray[numpy.float32]) Invoked with: kwargs: image_format=<ImageFormat.SRGB: 1>, data=array([[[176, 223, 255], [176, 223, 255],  I tried commenting out the transpose line and the code runs fine. I checked my types and they seem to be good too. As a bonus question, if anyone knows of an AI model that gives a face and hair (i.e. a selfie) mask without mediapipe that would be most welcome.

Answer: The issue here is that ndarray.transpose does not actually move anything in memory. It simply resets the strides so that accesses APPEAR to be using new locations. mediapipe, on the other hand, requires that the input array be physically contiguous in memory. Your array isn't. You can use np.ascontiguousarray to force it to rearrange the contents. Is numpy.transpose reordering data in memory?
Question: Among other descriptive statistics, I want to get some quantiles out of my pandas DataFrame. I can get the quantiles I want a couple of different ways, but I can't find the right way to do it with aggregate. I'd like to use aggregate because it'd be tidy and maybe computationally efficient to get all my stats in one go. rng = np.random.default_rng(seed=18860504) df = pd.DataFrame({ "dummy": 1, "bell": rng.normal(loc=0, scale=1, size=100), "fish": rng.poisson(lam=10, size=100), "cabin": rng.lognormal(mean=0, sigma=1.0, size=100), }) quants = [x/5 for x in range(6)] quantiles = pd.DataFrame({ "quantile" : [f"q{100*q:02n}" for q in quants], "bell" : df.groupby("dummy")["bell"].quantile(quants), "fish" : df.groupby("dummy")["fish"].quantile(quants), }) print(quantiles)  Output:  quantile bell fish dummy 1 0.0 q000 -2.313461 4.0 0.2 q020 -0.933831 7.0 0.4 q040 -0.246860 9.0 0.6 q060 0.211076 10.0 0.8 q080 0.685958 13.0 1.0 q100 3.017258 20.0  I'd like to get these quantiles using groupby().agg(), ideally with programmatically named columns like "bell_q90". Here's an example of the aggregate syntax that feels natural to me: df.groupby("dummy").agg( bell_med=("bell", "median"), bell_mean=("bell", "mean"), fish_med=("fish", "median"), fish_mean=("fish", "mean"), # fish_q10=("fish", "quantile(0.1)"), # nothing like it # fish_q10=("fish", "quantile", 0.1), # nothing like it # fish_q10=("fish", "quantile", kwargs({"q":0.1}), # nothing like it )  I can imagine generating the columns by iterating over quants and a list of named columns, using Series.agg and than stitching them together, but this seems like a hack. (For example, it would require me to do my "normal" aggregation first and then add quantiles on afterwards.) my_aggs = dict() for q in quants: for col in ["bell", "fish"]: my_aggs[f"{col}_q{100*q:03n}"] = df.groupby("dummy")[col].quantile(q) print(pd.DataFrame(my_aggs)) # numbers equivalent to those above  Is there a better way?

Answer: You could use a function factory to simplify the syntax: def quantile(q=0.5, **kwargs): def f(series): return series.quantile(q, **kwargs) return f df.groupby('dummy').agg( bell_med=('bell', 'median'), bell_mean=('bell', 'mean'), fish_med=('fish', 'median'), fish_mean=('fish', 'mean'), bell_q10=('bell', quantile(0.1)), fish_q10=('fish', quantile(0.1)), )  If you have many combinations, you could also combine this with a dictionary comprehension and parameter expansion: df.groupby('dummy').agg(**{'bell_med': ('bell', 'median'), 'bell_mean': ('bell', 'mean'), 'fish_med': ('fish', 'median'), 'fish_mean': ('fish', 'mean'), }, **{f'{c}_q{100*q:02n}': (c, quantile(q)) for q in [0.1] # add more if needed for c in ['bell', 'fish'] } )  Output:  bell_med bell_mean fish_med fish_mean bell_q10 fish_q10 dummy 1 -0.063454 -0.058557 10.0 9.92 -1.553682 6.0
Question: I am trying to optimize a quantum circuit using scipy.optimize.minimize and Qiskit Runtime's Estimator, running on an IBM Quantum real device. However, I am encountering the following error: AttributeError: 'ParameterExpression' object has no attribute 'name'  The error occurs inside the minimize function call, specifically when `estimator.run() is executed in cost_func_estimator. Here is the relevant code: from scipy.optimize import minimize from qiskit_ibm_runtime import Estimator, Session objective_func_vals = [] with Session(backend=backend) as session: estimator = Estimator(mode=session) estimator.options.default_shots = 1000 estimator.options.dynamical_decoupling.enable = True estimator.options.dynamical_decoupling.sequence_type = "XY4" estimator.options.twirling.enable_gates = True estimator.options.twirling.num_randomizations = "auto" result = minimize( cost_func_estimator, init_params, args=(candidate_circuit, qubitOp, estimator), method="COBYLA", tol=1e-2, callback=callback, ) save_progress(result.x, objective_func_vals, name_saved_file) print(result)  The cost_func_estimator function is called within minimize and runs a job in the estimator: def cost_func_estimator(params, ansatz, hamiltonian, estimator): isa_hamiltonian = hamiltonian.apply_layout(ansatz.layout) pub = (ansatz, isa_hamiltonian, params) job = estimator.run([pub]) # The error occurs here results = job.result()[0] cost = results.data.evs return cost  I expected scipy.optimize.minimize to optimize the parameters of my quantum circuit using Qiskit Runtime’s Estimator on an IBM Quantum real device. I tried: - Ensuring `init_params` contains only numerical values, not `ParameterExpression`. - Using `bind_parameters()` to assign values before optimization. - Updating `qiskit` and `qiskit-ibm-runtime` to the latest version.  Despite these attempts, the error persists. I expected estimator.run() to execute without issues, but it fails with AttributeError: 'ParameterExpression' object has no attribute 'name'. How can I resolve this issue?

Answer: It was a bug in the QPY serialization. qiskit 1.4.1 and qiskit 2.0.0 have fixes for it.
Question: I have tried looking this up but I only find answers related to getting the toggle command to work at all, not about the specific issue I have. I was previously using Notepad++. For Python, when I would toggle the comment of two lines simultaneously, I would get the following behaviour: # Line 1 Line 2  After toggle: Line 1 # Line 2  Future toggles alternate between the two states above. I have not found a way to do this in VS Code (Ctrl + / ). The behaviour I see is # Line 1 Line 2  After toggle: # # Line 1 # Line 2  Toggle again: # Line 1 Line 2  The behaviour of Notepad++ is quite useful, but I can't find any discussion of it or how to get VSCode to behave similarly. The VSCode way is not useless, but the Notepad++ version is much more useful as I can always just use a comment command if I want to simply add a comment and not swap which lines are commented. Does anyone know how I can replicate the Notepad++ behaviour in VSCode?

Answer: Shift+ Alt+ I then Ctrl + /. You can bind them using extension multi-command
Question: The results.csv file is hard to read since the variables, intermediates etc. are shown in a format like "i1000" which probably means "intermediate number 1000". Is there any way to make the results show the proper symbols/characters that I used in the variable declaration? I've used the m.Array function to declare variables, I'm not sure if I can assign names to them with indexes such as Temperature[1,2,3]

Answer: Use a list comprehension to define custom names for an array of variables. Here is an example: from gekko import GEKKO m = GEKKO(remote=False) # Create an array of variables #t = m.Array(m.Var,3,lb=-1000,ub=1000) t = [m.Var(lb=-1000,ub=1000,name=f'T{i+1}') for i in range(3)] # Create intermediate variables i1 = m.Intermediate(t[0] + 10, name='T4') i2 = m.Intermediate(t[2] / 100, name='T5') # Example equations m.Equation(t[0]+t[1]==10) m.Equation(t[1]+2*t[2]==50) m.Equation(t[0]-6*t[2]==10) m.solve() m.open_folder()  The results.csv file is now more readable with: t4, -5.5000000000E+01 t5, -1.2500000000E-01 t1, -6.5000000000E+01 t2, 7.5000000000E+01 t3, -1.2500000000E+01  APMonitor solution engine converts all variable names to lowercase. One other suggestion is to avoid starting names with reserved function names such as abs, sin, cos, and tan.
Question: I'm using Polars to process a DataFrame so I can save it as JSON. I know I can use the method .write_json(), however, I would like to add a new level to the JSON. My current approach: import polars as pl df = pl.DataFrame({ "id": [1, 2, 3, 4, 5], "variable1": [15, 25, 5, 10, 20], "variable2": [40, 30, 50, 10, 20], }) ( df.write_json() )  Current output: '[{"id":1,"variable1":15,"variable2":40},{"id":2,"variable1":25,"variable2":30},{"id":3,"variable1":5,"variable2":50},{"id":4,"variable1":10,"variable2":10},{"id":5,"variable1":20,"variable2":20}]'  But I would like to save it in this way, with the "Befs" key, so each "Befs" contains every record of the DataFrame. Desired output: { "Befs": [ { "ID ": 1, "variable1": 15, "variable2": 40 }, { "ID ": 2, "variable1": 25, "variable2": 30 } ] }  I have tried using .pl.struct() , but my attemps make no sense: ( df .select( pl.struct( pl.lit("Bef").alias("Bef"), pl.col("id"), pl.col("variable1"), pl.col("variable2") ) ) .write_json() ) 

Answer: The write_json() function always returns the data in a row-oriented format, in which the root element is a list, and each row contains a mapping of column_name -> row_value As a hacky workaround you could use write_ndjson() instead, given that its root element is a dictionary (for each line), but for that to match your desired output you'll have to implode everything into a single row and wrap it around a struct. df.select(Bef=pl.struct(pl.all()).implode()).write_ndjson())
Question: I need a PySpark UDF with a for loop to create new columns but with conditions based on the iterator value. def test_map(col): if x == 1: if col < 0.55: return 1.2 else: return 0.99 elif x == 2: if col < 0.87: return 1.5 else: return 2.4 etc. test_map_udf = F.udf(test_map, IntegerType())  And then iterate: for x in range(1, 10): df = df.withColumn(f"new_value_{x}", test_map_udf(F.col(f"old_value_{x}"))  But it errors out because test_map doesn't know what x is when it runs, and you can't pass x to test_map_udf. Should I create a regular Python function that takes x, and that function calls the UDF?

Answer: You can pass x as arguments to udf. def test_map(x, col): ... test_map_udf = F.udf(test_map, IntegerType()) for x in range(1, 10): df = df.withColumn(f"new_value_{x}", test_map_udf(F.lit(x), F.col(f"old_value_{x}"))
Question: For the input string as shown below, I am trying to substitute UK1/ after Street and every , and skip hyphen to create expected output shown below. Input = Street1-2,4,6,8-10 Expected output = StreetUK/1-2,UK/4,UK/6,UK/8-10 With the regex pattern I have trouble substitute for each captured group. How can I capture all the required groups for every , and sub required string.  replacements = [] pattern = r"(Street)?(?:\d+)(((,)?(?:\d+))*[-]?(?:\d+))*" def replacement(x): replacements.append(f"{x.group(1)}{'UK'}/") input = 'Street1-2,4,6,8-10' m = re.sub(pattern, replacement, input) print(m, [''.join(x) for x in replacements] )  The above code just prints ['StreetUK/'] but not as expected.

Answer: You can search using this regex: (Street|,)(?=\d)  and replace with: \1UK/  RegEx Demo Code: import re input = 'Street1-2,4,6,8-10' print(re.sub(r'(Street|,)(?=\d)', r'\1UK/', input))  Output: StreetUK/1-2,UK/4,UK/6,UK/8-10  If you want to do these replacements only for input starting with Street then use: print(re.sub(r'(Street|,)(?=\d)', r'\1UK/', input) if input.startswith("Street") else input)  RegEx Explanation:  (Street|,): Match Street or comma in capture group #1 (?=\d): Make sure there is a digit after the current position \1UK/: is replacement to put back-reference of group #1 back followed by UK/ in the substituted string
Question: Let's say I have a file like this: import pandas as pd df = pd.DataFrame( { "A": [" a: yu", "b: stuff ", "c: more_stuff"], "B": [4, 5, 6], "C": [7, 8, 9], } ) df["A"] = ( df["A"] .str.strip() .str.replace(":", "") .str[0] ) new_df = pd.melt( df, id_vars=["A"] ) print(new_df)  If I then run ruff format --diff play_line_breaks.py  I get -df["A"] = ( - df["A"] - .str.strip() - .str.replace(":", "") - .str[0] -) +df["A"] = df["A"].str.strip().str.replace(":", "").str[0] -new_df = pd.melt( - df, - id_vars=["A"] -) +new_df = pd.melt(df, id_vars=["A"]) print(new_df)  So, the ruff formatter would convert my multiline statements into a single line. I find the multiline version far more readable and would like to keep it. Is there any setting in ruff that would allow me to say "don't touch any line breaks"? The best I could find are skip-magic-trailing-comma = false (in my pyproject.toml)  which does not impact the output from above though or wrapping the two statements like this # fmt: off < all code after df assignment > # fmt: on  That works, but I find it rather cumbersome to do this for all the statements I have in my code base. Are there any smarter ways of doing this?

Answer: It seems there is no option available at the moment in ruff. See: ruff formatter: one call per line for chained method calls. Workaround:  Seems like putting # fmt: skip next to the closing parentheses works as a quick hack. import pandas as pd df = pd.DataFrame( { "A": [" a: yu", "b: stuff ", "c: more_stuff"], "B": [4, 5, 6], "C": [7, 8, 9], } ) df["A"] = ( df["A"] .str.strip() .str.replace(":", "") .str[0] ) # fmt: skip new_df = pd.melt( df, id_vars=["A"] ) # fmt: skip print(new_df)
Question: I need to add the package "modAL" (uppercase AL) as a dependency to a package. Since packages names are (seemingly) case insensitive in pyproject.toml, it falls back to "modal" which is another package. How can I add modAL and not modal as a dependency? I can install modAL separately (it works), but I want to add it as a regular dependency and I have no clue how to do it.

Answer: The comment by juanpa.arrivillega solves my problem:  [...] python package names are case insensitive, and on pypi, the "modAL" package seems to be distributed as "modAL-python" – Commented 2025-03-14 at 16:56:12Z  I just replaced "modAL" by "modAL-python" and it works as expected.