eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am wanting to use the Pydantic (Version: 2.6.1) aliases so that I can use a Python keyword ('from') when creating JSON. I thought this would work: from pydantic import BaseModel, Field class TestModel(BaseModel): from_str: str = Field(serialization_alias="from") test_obj = TestModel(from_str="John") print("TestModel: ", test_obj) test_json = test_obj.model_dump(by_alias=True) print("Test JSON:", test_json) deserialized_obj = TestModel.model_validate(test_json) print("Deserialized: ", deserialized_obj)  This correctly loads into a dict with the right alias ('from'). But it gives me this error in the call to model_validate: pydantic_core._pydantic_core.ValidationError: 1 validation error for TestModel from_str Field required [type=missing, input_value={'from': 'John'}, input_type=dict]  I thought it would also use the serialization alias when converting back to a TestModel, but it doesn't seem so. I do need it to be able to round-trip. Any help is appreciated. I have tried fiddling with validation_alias as well, but then I get issues when calling model_dump.

Answer: I was able to 'fix' this by setting up the class as follows: class TestModel(BaseModel): model_config = ConfigDict(populate_by_name=True) from_str: str = Field(alias="from")  Where populate_by_name allows you to access the class variable via it's name, but serialization and validation use the alias.
Question: I am using the htmx extension "response-targets" via "hx-ext" in a python django project. The Problem "hx-target-error" will not work when "hx-target" is also defined, but works when it is the only target. Environment htmx 1.9.10, response-targets 1.9.10, python 3.12, django 5.0.1 Example html: <div hx-ext="response-targets"> <button hx-get={% ur "appname:endpoint" %} hx-target-error="#id-fail" hx-target="#id-success" type="submit" > Press Me! </button> </div>  hx-target I have tried with various other types of targets such as "next td", "next span", etc. All of them produce a valid target value "request.htmx.target" in the endpoint defined in views.py. hx-target-error I have also tried various alternatives supported in response-targets such as "hx-target-*", "hx-target-404", etc with no change in results. I verified that "response-targets" is installed correctly and being used correctly because when "hx-target" is removed from the "button", then "hx-target-error" works. Error generation in views.ph @login_required def endpoint(request) return HttpResponseNotFound("Not found 404")  Logs Bad Request: /testbed/endpoint/ 127.0.0.1 - [27/Feb/2024] "GET /testbed/endpoint/ HTTP/1.1" 404  response-targets extension https://htmx.org/extensions/response-targets/

Answer: Resolved: The solution is to move the "hx-ext" attribute <div hx-ext="response-targets"> to a higher encompassing level than any DOM elements referenced by either "hx-target" or "hx-target-error". In my case, "hx-target-error" pointed to a div 'id' outside of the div containing "hx-ext". Thank you @guigui42 for suggesting this solution Example incorrect usage: <tr> <td> <div hx-ext="response-targets"> <button hx-get={% url "testbed:rebound" %} hx-headers='{"custom": "{{ test.id }}"}' hx-target-error="#GET-{{ test.id }}-fail" hx-target="#GET-{{ test.id }}-ok" class="btn btn-sm btn-primary testbed-btn" > GET {{ test.msg }} </button> </div> </td> <td> <span id="GET-{{ test.id }}-ok" /> <span class="text-danger" id="GET-{{ test.id }}-fail" /> </td> </tr>  Example correct usage: <tr hx-ext="response-targets"> <td> <button hx-get={% url "testbed:rebound" %} hx-headers='{"custom": "{{ test.id }}"}' hx-target-error="#GET-{{ test.id }}-fail" hx-target="#GET-{{ test.id }}-ok" class="btn btn-sm btn-primary testbed-btn" > GET {{ test.msg }} </button> </td> <td> <span id="GET-{{ test.id }}-ok" /> <span class="text-danger" id="GET-{{ test.id }}-fail" /> </td> </tr>
Question: I'm unable to test file download with Selenium (python), after Chrome update to the version '122.0.6261.70'. Previously running Chrome with the '--allow-running-insecure-content' arg did a trick. The same is suggested over the net. On some sites one additional arg is suggested: '--disable-web-security'. But both change nothing for me (the warning keeps appearing). Does anybody know if something has been changed between the 121 and 122 versions? Is there some arg or pref that I'm missing? Warning image for the reference:   Driver creation (simplified): from selenium import webdriver from selenium.webdriver.chrome.options import Options options = Options() for arg in ["--allow-running-insecure-content", "--disable-web-security"]: options.add_argument(arg) driver = webdriver.Chrome(options=options) 

Answer: Okay, so found two solutions:  --unsafely-treat-insecure-origin-as-secure=* This is an experimental flag that allows you to list which domains to treat as secure so the download is no longer blocked. --disable-features=InsecureDownloadWarnings This is a more stable flag that disables the insecure download blocking feature for all domains.  -- This is what worked for me: from selenium import webdriver from selenium.webdriver.chrome.options import Options chrome_options = Options() chrome_options.add_argument("--window-size=1920,1080") chrome_options.add_argument("--allow-running-insecure-content") # Allow insecure content chrome_options.add_argument("--unsafely-treat-insecure-origin-as-secure=http://example.com") # Replace example.com with your site's domain chrome_options.add_experimental_option("prefs", { "download.default_directory": download_path, "download.prompt_for_download": False, "download.directory_upgrade": True, "safebrowsing.enabled": True }) driver = webdriver.Chrome(options=chrome_options)
Question: I would like to package some legacy code to a hello Python package containing only one module (hello.py) file in the top-level directory alongside with some data in a folder called my_data without changing the folder structure: hello/ |-hello.py |-pyproject.toml |-my_data/ |-my_data.csv  Packaging the Python source code with the following pyproject.toml file is surprisingly simple (without any prior knowledge on packaging), but running pip install . -vvv fails to copy the data: [project] name = "hello" version = "0.1" [tool.setuptools] py-modules = ['hello'] [tool.setuptools.package-data] hello = ['my_data/*']  The content of hello.py could be minimal: def hello: print('Hello, world!')  I tried multiple variants of this pyproject.toml file according to the documentation on http://setuptools.pypa.io/en/stable/userguide/datafiles.html as well as a related question on Specifying package data in pyproject.toml, but none of them would result in copying of the my_data/ folder directly into the site-packages folder (which is intended, but probably bad practice). I also found documentation suggesting to use a MANIFEST.in> graft my_data  but also this doesn't result in the data to be installed alongside the code.

Answer: The package-data configuration of setuptools can be used when you have a package. But instead you seem to have a a single file-module. In other words package-data is incompatible with the directory structure that you have. I suggest rearranging the files like the following: hello/ |-pyproject.toml |-hello/ |-__init__.py # <----- previously named `hello.py` |-my_data.csv  # pyproject.toml diff ... [tool.setuptools] - py-modules = ['hello'] + packages = ['hello'] [tool.setuptools.package-data] - hello = ['my_data/*'] + "" = ['*.csv'] # <--- "" (empty string) means "all packages".
Question: I'm using mkdocs & mkdocstring to build my documentation and including code examples in the docstrings. I'm also using doctest (via pytest --doctest-modules) to test all those examples. Option 1 - format for documentation If I format my docstring like this:  """ Recursively flattens a nested iterable (including strings!) and returns all elements in order left to right. Examples: -------- ``` >>> [x for x in flatten([1,2,[3,4,[5],6],7,[8,9]])] [1, 2, 3, 4, 5, 6, 7, 8, 9] ``` """  Then it renders nicely in the documentation but doctest fails with the error: Expected: [1, 2, 3, 4, 5, 6, 7, 8, 9] ``` Got: [1, 2, 3, 4, 5, 6, 7, 8, 9]  That makes sense as doctest treats everything until a blank line as expected output and aims to match is exactly Option 2 - format for doctest If I format the docstring for doctest without code blocks:  """ Recursively flattens a nested iterable (including strings!) and returns all elements in order left to right. Examples: -------- >>> [x for x in flatten([1,2,[3,4,[5],6],7,[8,9]])] [1, 2, 3, 4, 5, 6, 7, 8, 9] """  then doctest passes but the documentation renders    [x for x in flatten([1,2,[3,4,[5],6],7,[8,9]])][1, 2, 3, 4, 5, 6, 7, 8, 9]    Workaround? - add a blank line for doctest If I format it with an extra blank line before the end of the codeblock:  """ Recursively flattens a nested iterable (including strings!) and returns all elements in order left to right. Examples: -------- ``` >>> [x for x in flatten([1,2,[3,4,[5],6],7,[8,9]])] [1, 2, 3, 4, 5, 6, 7, 8, 9] ``` """  Then doctest passes but  there is a blank line at the bottom of the example in the documentation (ugly) I need to remember to add a blank line at the end of each example (error prone and annoying)  Does anyone know of a better solution?

Answer: Patching the regex that doctest uses to identify codeblocks solved this problem. Documenting it here for those who stumble across this in the future ... As this is not something I want to do regularly in projects(!), I created pytest-doctest-mkdocstrings as a pytest plugin to do this for me and included some additional sanity-checking, configuration options etc. pip install pytest-doctest-mkdocstrings pytest --doctest-mdcodeblocks --doctest-modules --doctest-glob="*.md"  For those who are looking here for the answer in code to use yourself, the required change is:  _MD_EXAMPLE_RE = re.compile( r""" # Source consists of a PS1 line followed by zero or more PS2 lines. (?P<source> (?:^(?P<indent> [ ]*) >>> .*) # PS1 line (?:\n [ ]* \.\.\. .*)*) # PS2 lines \n? # Want consists of any non-blank lines that do not start with PS1. (?P<want> (?:(?![ ]*$) # Not a blank line (?![ ]*```) # Not end of a code block (?![ ]*>>>) # Not a line starting with PS1 .+$\n? # But any other line )*) """, re.MULTILINE | re.VERBOSE, ) doctest.DocTestParser._EXAMPLE_RE = _MD_EXAMPLE_RE  Specifically I have included (?![ ]*```) # Not end of a code block in the identification of the "want"
Question: I want to add a column new_column to an existing dataframe df. I know this looks like a duplicate of Add new column to polars DataFrame but the answer to that questions, as well as the answers to many similar questions, don't really add a column to an existing dataframe. They create a new column with another dataframe. I think this can be fixed like this: df = df.with_columns( new_column = pl.lit('some_text') )  However, rewriting the whole dataframe just to add a few columns, seems a bit of a waste to me. Is this the right approach?

Answer: Your question suggests that you think that when you do df = df.with_columns( new_column = pl.lit('some_text') )  that you're copying everything over to some new df which would be really inefficient. You're right that that would be really inefficient but that isn't what happens. A DataFrame is just a way to organize pointers to the actual data. The hierarchy is that you have, at the top, DataFrames. Within a DataFrame are Serieses which are how columns are represented. Even at the Series level, it's still just pointers, not data. It is made up of one or more chunked arrays which fit the apache arrow memory model. When you "make a new df" all you're doing is organizing pointers, not data. The data doesn't move or copy. Conversely consider pandas's inplace parameter. It certainly makes it seem like you're modifying things in place and not making copies.  inplace does not generally do anything inplace but makes a copy and reassigns the pointer  https://github.com/pandas-dev/pandas/issues/16529#issuecomment-323890422 The crux of the issue is that in pandas everything you do makes a copy (or several). In polars, that isn't the case so even when you assign a new df that new df is just an outer layer that points to data. The data doesn't move, nor is it copied unless you specifically execute an operation that does. That said, there are methods which will insert columns without requiring you to use the df=df... syntax but they don't do anything different under the hood as using the preferred assignment syntax.
Question: As long as the list isn't sorted, keep replacing it with a merge of an iterator with itself. Is that (equivalent to) one of the commonly known sorting algorithms, just implemented weirdly, or is it something new? from random import shuffle from heapq import merge from itertools import pairwise # Create test data a = list(range(100)) shuffle(a) # Sort while any(x > y for x, y in pairwise(a)): it = iter(a) a = list(merge(it, it)) print(a)  Attempt This Online! heapq.merge does merge two inputs in the "standard" way, but it's an implementation detail and its inputs are supposed to be sorted, which they aren't here (and they're also not independent, as they use the same source). To eliminate it being an implementation detail, let merge actually be this (the standard way, always comparing the two "current" values, yielding the smaller one and fetching a replacement for it): def merge(xs, ys): none = object() x = next(xs, none) y = next(ys, none) while (x is not none) and (y is not none): if x <= y: yield x x = next(xs, none) else: yield y y = next(ys, none) if x is not none: yield x if y is not none: yield y yield from xs yield from ys 

Answer: It's indeed bubble sort implemented weirdly. Bubble sort repeatedly does this until the list is sorted:  Slide a two-item window over the list. At each position, sort the two items in the window, then slide the window one position to the right. This leaves the smaller item behind, the larger item remains in the window, and the next item enters the window.  The merge with the same iterator twice does effectively the same thing. Its "two-item window" is the two "current values", the x and y in the question's merge implementation. Like in bubble sort, the smaller one is "left behind" (gets yielded, becomes the next item in the output). The larger one remains in the "window" (remains x or y). And the next item from the input list enters the window (becomes the other one of x and y). Thinking of Knuth's "Beware ... I have only proved it ... not tried it", let's also try it, checking experimentally whether the behavior is indeed identical to ordinary bubble sort. Sorting a shuffled list with 1000 elements both with my question's sort and with bubble sort, and recording the state of the list after each "round". They're identical for both sorts. from random import shuffle from heapq import merge from itertools import pairwise # Create test data a = list(range(1000)) shuffle(a) def mysort_steps(a): steps = [] while any(x > y for x, y in pairwise(a)): it = iter(a) a = list(merge(it, it)) steps.append(a[:]) return steps def bubblesort_steps(a): steps = [] n = len(a) for i in range(n - 1): swapped = False for j in range(n - 1 - i): if a[j] > a[j+1]: a[j], a[j+1] = a[j+1], a[j] swapped = True if not swapped: break steps.append(a[:]) return steps print(mysort_steps(a[:]) == bubblesort_steps(a[:]))  Attempt This Online!
Question: I encountered this question during an interview and can't think of a solution. This is the problem, suppose you had a dataset as follows (it goes beyond time 2 but this is just a sample to work with): import pandas as pd data = pd.DataFrame({ 'time': [1, 1, 1, 2, 2, 2], 'names': ["Andy", "Bob", "Karen", "Andy", "Matt", "Sim"], 'val': [1, 2, 3, 5, 6, 8] })  Write a function to calculate the mean of values up till each time point but don't count duplicate names. That is, for time 1 the mean is (1+2+3)/3, for time 2 I don't include Andy's first value of '1' I only include the most recent value so the mean for time 2 is (2+3+5+6+8)/5. I have tried creating two dictionaries, one that stores the 'time' count and the other keeping track of 'names' and 'values' but I don't know how to proceed from there or how to come up with an efficient solution so I am not recalculating means at each step (this was another requirement for the interview). It doesn't have to be a pandas solution, the data form can be anything you prefer. I just presented it as a pandas df.

Answer: IIUC, you want to compute the mean of the values up to the current time, while considering only the last seen duplicates (if any). If so, here is one potential option that uses boolean indexing inside a for-loop to build the expanding windows : # uncomment if necessary # data.sort_values("time", inplace=True) to_keep = "last" # duplicate means = {} for t in data["time"].unique(): window = data.loc[data["time"].le(t)] m = ~window["names"].duplicated(to_keep) means[t] = window.loc[m, "val"].mean()  Output (means) : { # time|mean 1: 2.0, # (1+2+3)/3 2: 4.8, # (2+3+5+6+8)/5 }
Question: Is there a way to call .agg() without grouping first? I want to perform standard aggregations but only want one row in the response, rather than separate rows for separate groups. I could do something like df.with_columns(dummy_col=pl.lit("dummy_col")).group_by('dummy_col').agg(<aggregateion>)  but I'm wondering if there's a way without the dummy stuff

Answer: When all expressions within a select context are aggregations, the resulting dataframe only has a single row. import polars as pl df = pl.DataFrame({ "a": [1, 2, 3, 4, 5], "b": [0, 0, 1, 1, 1], })  df.select(pl.col("a").mean(), pl.col("b").first())  shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ i64 │ ╞═════╪═════╡ │ 3.0 ┆ 0 │ └─────┴─────┘
Question: About Selenium with Python... What is the difference between from seleniumbase import Driver (seleniumbase) and from selenium import webdriver (selenium, seleniumwire)? What is the difference in "use cases"? I see only constructor difference: seleniumbase Driver is harder to set options, then manipulaton with object are same. from seleniumbase import Driver driver = Driver(uc=False, headless=True, proxy=proxy, incognito=None, user_data_dir=None, extension_dir=None, binary_location=None)  from selenium import webdriver as Driver driver = Driver.Chrome(options=options) 

Answer: It sounds like you're trying to compare this: from seleniumbase import Driver driver = Driver()  with: from selenium import webdriver driver = webdriver.Chrome()  The seleniumbase driver has more methods than the regular selenium one. The seleniumbase driver methods also have auto-selector detection, smart waiting, special assertion methods, allow truncated URLs, and support the TAG:contains("TEXT") selector. That means you can do this: from seleniumbase import Driver driver = Driver() driver.open("seleniumbase.io/simple/login") driver.type("#username", "demo_user") driver.type("#password", "secret_pass") driver.click('a:contains("Sign in")') driver.assert_exact_text("Welcome!", "h1") driver.assert_element("img#image1") driver.highlight("#image1") driver.click_link("Sign out") driver.assert_text("signed out", "#top_message") driver.quit()  There are some other differences, such as the way options are passed. SeleniumBase options are passed as args into the Driver() definition for the Driver() Manager format (there are many other formats, such as SB(), BaseCase, etc.) SeleniumBase also has a UC Mode option, which has special methods for letting your bots bypass CAPTCHAs that block regular Selenium bots: from seleniumbase import Driver driver = Driver(uc=True) driver.uc_open_with_reconnect("https://top.gg/", 6) driver.quit()  Here's a CAPTCHA-bypass example where clicking is required: from seleniumbase import Driver driver = Driver(uc=True) driver.uc_open_with_reconnect("https://seleniumbase.io/apps/turnstile", 3) driver.uc_switch_to_frame("iframe") driver.uc_click("span.mark") driver.sleep(3) driver.quit()
Question: I'm trying to run Gemma on Keras with this model https://www.kaggle.com/models/keras/gemma/frameworks/keras/variations/gemma_instruct_7b_en And I'm reproducing the Example available on "Model Card" on above page. When I run this code: gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_instruct_7b_en") gemma_lm.generate("Keras is a", max_length=30) # Generate with batched prompts. gemma_lm.generate(["Keras is a", "I want to say"], max_length=30)  I get this error: --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[2], line 1 ----> 1 gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_instruct_7b_en") 2 gemma_lm.generate("Keras is a", max_length=30) 4 # Generate with batched prompts. AttributeError: module 'keras_nlp.models' has no attribute 'GemmaCausalLM'  How can I fix this?

Answer: Try restarting the kernel. Note that the same behavior occurs with the 2B Gemma model, i.e. gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_2b_en"). I am not sure why restarting the kernel works, but it's worth noting that this doesn't happen in Google Colab.
Question: I have two pandas DataFrames: import pandas as pd data1 = { 'score': [1, 2], 'seconds': [1140, 2100], } data2 = { 'prize': [5.5, 14.5, 14.6, 21, 23, 24, 26, 38, 39, 40, 50], 'seconds': [840, 1080, 1380, 1620, 1650, 1680, 1700, 1740, 2040, 2100, 2160], } df1 = pd.DataFrame.from_dict(data1) df2 = pd.DataFrame.from_dict(data2) Output: df1 score seconds 0 1 1140 1 2 2100 Output: df2 prize seconds 0 5.5 840 1 14.5 1080 2 14.6 1380 3 21.0 1620 4 23.0 1650 5 24.0 1680 6 26.0 1700 7 38.0 1740 8 39.0 2040 9 40.0 2100 10 50.0 2160  For each value in seconds column from df1, I would like to get the match (or the closest to) row from df2 and also the closest 2 rows above and below the match. The seconds columns contains only sorted unique values. As result, I expect this: Output: result prize seconds 0 5.5 840 1 14.5 1080 # closest match to 1140 2 14.6 1380 3 21.0 1620 7 38.0 1740 8 39.0 2040 9 40.0 2100 # match 2100 10 50.0 2160 

Answer: You can use a merge_asof to identify the closest value to each value in df1, then a rolling.max to extend the selection to the neighboring N rows: N = 2 # number of surronding rows to keep s1 = df1['seconds'].sort_values() s2 = df2['seconds'].sort_values().rename('_') keep = pd.merge_asof(s1, s2, left_on='seconds', right_on='_', direction='nearest')['_'] out = df2[s2.isin(keep) .rolling(2*N+1, center=True, min_periods=1) .max().astype(bool)]  NB. if the seconds are already sorted, you can skip the .sort_values(). Output:  prize seconds 0 5.5 840 1 14.5 1080 2 14.6 1380 3 21.0 1620 7 38.0 1740 8 39.0 2040 9 40.0 2100 10 50.0 2160  Intermediates:  prize seconds closest isin(keep) rolling.max 0 5.5 840 NaN False True 1 14.5 1080 1140.0 True True 2 14.6 1380 NaN False True 3 21.0 1620 NaN False True 4 23.0 1650 NaN False False 5 24.0 1680 NaN False False 6 26.0 1700 NaN False False 7 38.0 1740 NaN False True 8 39.0 2040 NaN False True 9 40.0 2100 2100.0 True True 10 50.0 2160 NaN False True