eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a Python service that uses Python's virtual threads (threading.Thread) to handle requests. There is a shared singleton functionality that all threads are trying to access, which is protected using threading.Lock. g_lock = threading.Lock() def my_threaded_functionality(): try: g_lock.acquire() # ... Do something with a shared resource ... finally: g_lock.release()  In the docs of threading.Lock.acquire, there is no mentioning of fairness, whereas, in asyncio's asyncio.Lock.acquire, they mention that the lock is fair. As I want to prevent starvation of threads, and want to preserve the order of the tasks the same as they have arrived, I would go for asyncio's Lock if they didn't mention that the locks are not thread safe. The question is whether it should be an issue also with Python's virtual "threads".

Answer:  Python's virtual threads (threading.Thread)  CPython threads are native threads, not virtual threads, the concept of virtual threads doesn't exist in CPython. asyncio's Lock is not thread-safe, you cannot use it for multithreaded synchronization, only threading.Lock is safe for multithreaded access. you can serialize access to this resource with a threadpool of 1 thread, it has a queue internally and guarantees fairness (first-in-first-out), don't use locks. as a bonus you can use loop.run_in_executor to await it in your eventloops. my_pool = concurrent.futures.ThreadPoolExecutor(max_workers=1) def non_thread_safe_task(): return "not thread-safe" async def my_threaded_functionality_async(): my_loop = asyncio.get_running_loop() result = await my_loop.run_in_executor(my_pool, non_thread_safe_task) def my_threaded_functionality(): result = my_pool.submit(non_thread_safe_task).result()  concurrent.futures.ThreadPoolExecutor spawns threads lazily, so it is okay to have it in the global scope, it doesn't create a thread if it is not used, but i'd rather wrap the whole thing in a class. Note: sending work to other threads and back adds roughly 10-50 microseconds of latency, only use it if you must guarantee order, otherwise just use a threading.Lock g_lock = threading.Lock() def my_threaded_functionality(): with g_lock: # ... Do something with a shared resource ...  there's also an async version to lock threading.Lock in an eventloop (which also has this extra 10-50 microseconds of overhead ... , i'd probably use the 1 worker thread_pool if you are in async code, that's also multithreaded)
Question: I'm trying to view LLVM debug messages for loop vectorization using Numba and llvmlite. I want to see the loop vectorization "LV:" debug output (e.g., messages like LV: Checking a loop in ...) so I can analyze the vectorization decisions made by LLVM. https://numba.readthedocs.io/en/stable/user/faq.html#does-numba-vectorize-array-computations-simd I'm using a conda environment with the following YAML file. This makes sure I’m using the llvmlite build from the numba channel (which should have LLVM built with assertions enabled): name: numbadev channels: - defaults - numba dependencies: - python>=3.12.9 - numba::numba - numba::llvmlite - intel-cmplr-lib-rt  Running conda list shows: # Name Version Build Channel python 3.13.2 hf623796_100_cp313 llvmlite 0.44.0 py313h84b9e52_0 numba numba 0.61.2 np2.1py3.13hf94e718_g1e70d8ceb_0 numba ...  This is my script debug_loop_vectorization.py: import llvmlite.binding as llvm llvm.set_option('', '--debug-only=loop-vectorize') import numpy as np from numba import jit @jit(nopython=True, fastmath=True) def test_func(a): result = 0.0 for i in range(a.shape[0]): result += a[i] * 2.0 return result # Trigger compilation. a = np.arange(1000, dtype=np.float64) test_func(a) # Print the LLVM IR print(test_func.inspect_llvm(test_func.signatures[0]))  When I run the script from a Linux terminal I see the LLVM-IR but no lines starting with "LV:...". How can I get LLVM loop vectorization debug output? Edit: I've tried to call the script from the Linux terminal like this: conda activate numbadev cd /PathToMyFile/ python debug_loop_vectorization.py  or like this: LLVM_DEBUG=1 python debug_loop_vectorization.py 2>&1 | grep '^LV:'  Edit: In llvmlite/conda-recipes/llvmdev, bld.bat contains a cmake argument "DLLVM_ENABLE_ASSERTIONS=ON". https://github.com/numba/llvmlite/blob/main/conda-recipes/llvmdev/bld.bat#L32 build.sh doesn't contain "_cmake_config+=(-DLLVM_ENABLE_ASSERTIONS:BOOL=ON)" anymore. https://github.com/numba/llvmlite/blob/main/conda-recipes/llvmdev/build.sh Edit: LLVM debug messages have been disabled in the Linux build unintentionally and should be enabled in LLVMlite v0.45.0 on the Numba channel.

Answer: On Linux, a workaround solution for tests is to use Numba 0.60.0. Indeed, it is based on LLVMlite v0.43.0 which is build with an LLVM version having assertions. On Linux, newer versions (e.g. Numba 0.61.2 and LLVMlite v0.44.0) do not embed an LLVM supporting assertions (apparently due to a code cleaning). Thus it must be build manually with . One solution with newer version is to recompile LLVMlite and its embedded LLVM locally so assertions are enabled. This can be done by setting -DLLVM_ENABLE_ASSERTIONS:BOOL=ON in CMAKE_ARGS before the build. Alternatively, you can add this line in the conda-recipes/llvmdev/build.sh file. On Windows, this works well by default so far. There is nothing to do.
Question: I have a complex nested dictionary structure and I need to remove elements based on the values in a nested dictionary. My dictionary looks like this: my_dict = { 'item1': {'name': 'Apple', 'price': 1.0, 'category': {'id': 1, 'name': 'Fruit'}}, 'item2': {'name': 'Banana', 'price': 0.5, 'category': {'id': 1, 'name': 'Fruit'}}, 'item3': {'name': 'Carrot', 'price': 0.75, 'category': {'id': 2, 'name': 'Vegetable'}}, 'item4': {'name': 'Broccoli', 'price': 1.5, 'category': {'id': 2, 'name': 'Vegetable'}} }  I want to filter this dictionary to only include items belonging to the 'Fruit' category. I tried the following code: new_dict = {} for key, value in my_dict.items(): if value['category']['name'] == 'Fruit': new_dict[key] = value print(new_dict)  This works, but I'm wondering if there's a more concise or Pythonic way to achieve this, perhaps using dictionary comprehension or a filtering function like filter().

Answer: As a dictionary comprehension A dictionary comprehension can be used to create dictionaries from arbitrary key and value expressions. new_dict2 = { key: value for key, value in my_dict.items() if value['category']['name'] == 'Fruit' } new_dict2 == new_dict # True  Using filter() The filter() function is used to:  Construct an iterator from those elements of iterable for which function is true.  The dict.items() returns an iterable where each element is a tuple of length 2. We can supply each item to a lambda function, where item[0] will be the key and item[1] the value. filter() returns an iterator of the tuples which match the condition. We can wrap this in dict() to get a dictionary (in the same way that dict([("key1", "value1"), ("key2", "value2")]) returns {'key1': 'value1', 'key2': 'value2'}). new_dict3 = dict( filter( lambda item: item[1]['category']['name'] == 'Fruit', my_dict.items() ) ) new_dict3 == new_dict # True  Most Pythonic way Achieving the nebulous goal of Pythonicness (Pythonicity?) is always somewhat subjective. I think a dictionary comprehension is clean and neat but it can be hard to see what it's doing, especially if the dict is deeply nested or the condition is complex. It's probably clearest if you wrap it in an appropriately-named function so you can see what's going on. I've added type annotations for clarity: def find_fruit(d: dict[str, dict]) -> dict[str, dict]: def is_fruit(key: str, value: dict) -> bool: return value["category"]["name"] == "Fruit" return {key: value for key, value in d.items() if is_fruit(key, value)} fruit_dict = find_fruit(my_dict) new_dict == fruit_dict # True  This is fundamentally the same as the first approach but easier on the eyes.
Question: I am trying to use keras-tuner to tune hyperparameters, like !pip install keras-tuner --upgrade import keras_tuner as kt from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Flatten from tensorflow.keras.optimizers import Adam def build_model(hp): model = Sequential([ Flatten(input_shape=(28, 28)), Dense(units= hp.Int('units', min_value = 16, max_value = 64, step = 16), activation='relu'), Dense(units = hp.Int('units', min_value = 8, max_value = 20, step = 2), activation='softmax') ]) model.compile( optimizer=Adam(learning_rate=hp.Float('learning_rate', min_value=1e-4, max_value=1e-2, sampling='LOG')), loss='sparse_categorical_crossentropy', metrics=['accuracy'] ) return model # Create a RandomSearch Tuner tuner = kt.RandomSearch( build_model, objective='val_accuracy', max_trials=10, executions_per_trial=2 ) # Display a summary of the search space tuner.search_space_summary()  shows Search space summary Default search space size: 2 units (Int) {'default': None, 'conditions': [], 'min_value': 16, 'max_value': 64, 'step': 16, 'sampling': 'linear'} learning_rate (Float) {'default': 0.0001, 'conditions': [], 'min_value': 0.0001, 'max_value': 0.01, 'step': None, 'sampling': 'log'}  However, when checking the search_space_summary() output, only the 1st Dense layer is shown in the summary, while the information about the 2nd Dense layer, i.e., Dense(units = hp.Int('units', min_value = 8, max_value = 20, step = 2), activation='softmax'), is not seen. Did I misconfigured something or it is supposed to yield the output like that? Could anyone help me to understand why it outputs the summary like this?

Answer: Each hyperparameter must have a unique name. This is also listed in the docs. In your case, both layer units parameters are called units . You should rename them to something like units_1 and units_2, for example.
Question: I'm trying to insert/make a new column with these attributes in lines of then("") but i get columnNotFoundError instead for the lines of .then(). But these does not exists yet, what is wrong with the code?  df_2023 = df_2023.with_columns( pl.when(pl.col("Tillatt totalvekt opp til og med 3500").eq("X")) .then("Opp til og med 3500") .when(pl.col("Tillatt totalvekt 3501-7500").eq("X")) .then("3501-7500") .when(pl.col("Tillatt totalvekt over 7500").eq("X")) .then("Over 7500") .otherwise(pl.lit(None).cast(pl.String)) .alias("Tillatt totalvekt") ) df_2023.select("Tillat totalvekt").head() *THE ERROR* *ColumnNotFoundError: Opp til og med 3500*  I also tried with: pl.when(pl.col("Tillatt totalvekt opp til og med 3500") == "X") .then("Opp til og med 3500")  I could really use some help, read also the Polars documentation for polars.when, but it was for no help to solve this problem. Thanks in advance!

Answer: The then construct assumes strings are column names. I guess "Opp til og med 3500" is a literal value in your code and not a column name. Use pl.lit('....') to define a literal value explicitly, that way polars won't consider it as a column name. In your case, .then(pl.lit("Opp til og med 3500")) (same for other then). Some more details: https://github.com/pola-rs/polars/issues/13805
Question: I have a deep learning mode which I am running in the jit transformed manner by: my_function_checked = checkify.checkify(model.apply) model_jitted = jax.jit(my_function_checked) err, pred = model_jitted({"params": params}, batch, training=training, rng=rng) err.throw()  The code is compiling fine, but now I want to debug the intermediate values after every few steps, save the arrays, and then compare them with pytorch tensors. For this, I need to repeatedly save the arrays. The easiest way to do this is to use any IDE's inbuilt debugger and evaluate the save expression after every few steps. But jax.jit transformed code doesn't allow external debuggers. But, I can do this after disabling the jit. Should I be expecting any discrepancies between the two runs? Can I assume that the values in jit and non-jit runs will remain same?

Answer: In general when comparing the same JAX operation with and without JIT, you should expect equivalence up to typical floating point rounding errors, but you should not expect bitwise equivalence, as the compiler may fuse operations in a way that leads to differing float error accumulation.
Question: I'm using Pandera to define a schema for a pandas DataFrame where the index represents calendar dates (without time). I want to type-annotate the index as holding datetime.date values. Here's what I tried: # mypy.ini [mypy] plugins = pandera.mypy  # schema.py from datetime import date import pandera as pa from pandera.typing import Index class DateIndexModel(pa.DataFrameModel): date: Index[date]  But running mypy gives the following error: error: Type argument "date" of "Index" must be a subtype of "bool | int | str | float | ExtensionDtype | <30 more items>" [type-var] Found 1 error in 1 file (checked 1 source file)  I know that datetime64[ns] or pandas.Timestamp work fine, but I specifically want to model just dates without time. Is there a type-safe way to do this with Pandera and mypy? Any workaround that lets me enforce date-only index semantics (with or without datetime.date) while keeping mypy happy? Colab example notebook: https://colab.research.google.com/drive/1AdiztxHlyvEMo6B3CzYnvzlnh6a0GfUQ?usp=sharing

Answer: TL;DR use Index[pa.engines.pandas_engine.Date] Pandera as of now does not support datetime.date series data type, but it has a semantic representation of a date type column for each library (pandas, polars, pyarrow etc). Date type for pandas.DataFrames is pa.engines.pandas_engine.Date , for the others you can see the API docs. From the pandera documentation:  class pandera.engines.pandas_engine.Date(to_datetime_kwargs=None) Semantic representation of a date data type.  # schema.py import pandera as pa from pandera.typing import Index class DateIndexModel(pa.DataFrameModel): date: Index[pa.engines.pandas_engine.Date]
Question: I wrote the following simple test: deeptest.py from deepdiff import DeepDiff, Delta dict1 = {'catalog': {'uuid': 'e95fb23c-57d2-495f-8ab5-2c6b3152bcee', 'metadata': {'title': 'Catalog', 'last-modified': '2025-04-10T16:00:34.033789-05:00', 'version': '1.0', 'oscal-version': '1.1.2'}, 'controls': [{'id': 'ac-1', 'title': 'Access Control', 'parts': [{'id': 'ac-1_stmt', 'name': 'statement', 'prose': 'Access control text.'}]}]}} dict2 = {} diff = DeepDiff(dict1, dict2) print(diff) delta = Delta(diff) print(f'delta {delta}')  On the console I observe: $ python python/deep_test.py {'dictionary_item_removed': ["root['catalog']"]} delta <Delta: {"dictionary_item_removed":{"root['catalog']":{"uuid":""}}}>  My question/issue is that the delta should be the entirety if dict1, but not all of it is shown...why?

Answer: A DeepDiff returns an object that has already calculated the difference of the 2 items. The format of the object is chosen by the view parameter. By default it uses view=’text’, but there is also tree view, which is more complicated and detailed, and pretty() method. You can read about it in the documentation But the thing you probably means is verbose_level option Higher verbose level shows you more details. For example verbose level 1 shows what dictionary item are added or removed. And verbose level 2 shows the value of the items that are added or removed too. from deepdiff import DeepDiff, Delta dict1 = {'catalog': {'uuid': 'e95fb23c-57d2-495f-8ab5-2c6b3152bcee', 'metadata': {'title': 'Catalog', 'last-modified': '2025-04-10T16:00:34.033789-05:00', 'version': '1.0', 'oscal-version': '1.1.2'}, 'controls': [{'id': 'ac-1', 'title': 'Access Control', 'parts': [{'id': 'ac-1_stmt', 'name': 'statement', 'prose': 'Access control text.'}]}]}} dict2 = {} diff = DeepDiff(dict1, dict2, verbose_level=2) print(diff) delta = Delta(diff) print(f'delta {delta}')  Output: {'dictionary_item_removed': {"root['catalog']": {'uuid': 'e95fb23c-57d2-495f-8ab5-2c6b3152bcee', 'metadata': {'title': 'Catalog', 'last-modified': '2025-04-10T16:00:34.033789-05:00', 'version': '1.0', 'oscal-version': '1.1.2'}, 'controls': [{'id': 'ac-1', 'title': 'Access Control', 'parts': [{'id': 'ac-1_stmt', 'name': 'statement', 'prose': 'Access control text.'}]}]}}} delta <Delta: {"dictionary_item_removed":{"root['catalog']":{"uuid":""}}}>
Question: I know that to activate virtualenv it's just .venv/Scripts/activate.ps1 but I was wondering if there's a way of having powershell do it automatically? Existing ones just talk about activating it, but not how to have Powershell do it automatically  virtualenv in PowerShell? How to activate virtualenv using PowerShell?  I am looking for something like this but in Powershell Automating virtualenv activation/deactivation in zsh

Answer: Add something like the following to your PowerShell $PROFILE file: $ExecutionContext.SessionState.InvokeCommand.LocationChangedAction = [Delegate]::Combine( $ExecutionContext.SessionState.InvokeCommand.LocationChangedAction, [EventHandler[System.Management.Automation.LocationChangedEventArgs]] { # Look for a virtual-environment activation script relative to the # new working directory and, if found, execute it. if ($script = Get-Item -ErrorAction Ignore -LiteralPath ./.venv/Script/activate.ps1) { Write-Verbose -Verbose "Activating virtual environment in $PWD..." & $script } } )  Note:  This installs an event handler that executes whenever the current location (directory) is changed in the current session and executes a script located in ./.venv/Script/activate.ps1 relative to the new current directory, if present.  Note: The script's success output, if any, will not print to the display by default (though errors and output from other streams (other than the success one) would); to ensure that it prints, use & $script | Out-Host   No attempt is made to deactivate a previously active virtual environment or to detect if a subdirectory of a directory containing a virtual environment is changed to, though doing these things would be possible with additional effort.  Through use of [Delegate]::Combine(), the code preserves any preexisting event handler.  If you can assume that there is none, you can omit this call and assign the script block ({ ... }) directly to $ExecutionContext.SessionState.InvokeCommand.LocationChangedAction
Question: I am trying to make a button that increments a variable with Tkinter, but when I 'call' (I know it isn't really calling) a function with command, i can not use return variable, as there is nowhere to return the variable to. Are there alternative ways to do this? Here is my code: import tkinter as tk variable = 0 root = tk.Tk() def variable_incrementer(): global variable variable += 1 # Not return here click_btn = tk.Button(root, text="Click me", command=variable_incrementer) click_btn.pack() root.mainloop() 

Answer: You don't need to return from the call-back, Instead, you can update the label directly and you used .push() but actually Tkinter widgets uses .grid() , .pack() or .place() to display them? here is the updated code: import tkinter as tk variable = 0 root = tk.Tk() label = tk.Label(root, text=f"Count: {variable}") label.pack() def variable_incrementer(): global variable variable += 1 label.config(text=f"Count: {variable}") click_btn = tk.Button(root, text="Click me", command=variable_incrementer) click_btn.pack() root.mainloop()
Question: I have the following dataframe: import polars as pl df = pl.DataFrame({ 'ID': [1, 1, 5, 5, 7, 7, 7], 'YEAR': [2025, 2025, 2023, 2024, 2020, 2021, 2021] }) shape: (7, 2) ┌─────┬──────┐ │ ID ┆ YEAR │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪══════╡ │ 1 ┆ 2025 │ │ 1 ┆ 2025 │ │ 5 ┆ 2023 │ │ 5 ┆ 2024 │ │ 7 ┆ 2020 │ │ 7 ┆ 2021 │ │ 7 ┆ 2021 │ └─────┴──────┘  Now I would like to get the unique number of years per ID, i.e. shape: (7, 3) ┌─────┬──────┬──────────────┐ │ ID ┆ YEAR ┆ UNIQUE_YEARS │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ u32 │ ╞═════╪══════╪══════════════╡ │ 1 ┆ 2025 ┆ 1 │ │ 1 ┆ 2025 ┆ 1 │ │ 5 ┆ 2023 ┆ 2 │ │ 5 ┆ 2024 ┆ 2 │ │ 7 ┆ 2020 ┆ 2 │ │ 7 ┆ 2021 ┆ 2 │ │ 7 ┆ 2021 ┆ 2 │ └─────┴──────┴──────────────┘  So I tried df.with_columns(pl.col('YEAR').over('ID').alias('UNIQUE_YEARS')) but this gives the wrong result. So I came up with df.join(df.group_by('ID').agg(pl.col('YEAR').unique().len().alias('UNIQUE_YEARS')), on='ID', how='left')  which does gives correct result! But it looks a bit clunky, and I wonder if there is a more natural way using with_columns and over?

Answer: You can use Expr.n_unique: out = df.with_columns( pl.col('YEAR').n_unique().over('ID').alias('UNIQUE_YEARS') )  Output: shape: (7, 3) ┌─────┬──────┬──────────────┐ │ ID ┆ YEAR ┆ UNIQUE_YEARS │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ u32 │ ╞═════╪══════╪══════════════╡ │ 1 ┆ 2025 ┆ 1 │ │ 1 ┆ 2025 ┆ 1 │ │ 5 ┆ 2023 ┆ 2 │ │ 5 ┆ 2024 ┆ 2 │ │ 7 ┆ 2020 ┆ 2 │ │ 7 ┆ 2021 ┆ 2 │ │ 7 ┆ 2021 ┆ 2 │ └─────┴──────┴──────────────┘  Similarly, groupby can take .n_unique() instead of your .unique().len().
Question: So I do not know how to save the csv which this code creates to a specific folder in my directory. Any help would be appreciated! #Store rows which do not conform to the relationship in a new dataframe subset = df[df['check_total_relationship'] == False]` subset.to_csv('false_relationships.csv', index=False, header=True, encoding='utf-8')` 

Answer: Just specify the folder in your directory. subset.to_csv('output_data/false_relationships.csv', index=False, header=True, encoding='utf-8')  Or you can specify the absolute path subset.to_csv('/absolute-path/output_data/false_relationships.csv', index=False, header=True, encoding='utf-8')  If you need to join paths you can use os.path import os BASE_DIR = os.path.dirname(os.path.abspath(__file__)) filepath = 'files/one.txt' request_path = os.path.join(BASE_DIR, filepath)
Question: I want to automatically start the debugging session instead of having it starting as paused. I found out that adding this code makes it work but it feels hackish to me: context.add_init_script("setTimeout(window.__pw_resume, 500)")  Without the setTimeout, it won't work. Am I doing anything wrong?

Answer: __pw_resume isn’t defined immediately, so a short delay is needed. As of now, there’s no official setting to start unpaused, so this workaround necessary.