eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: If I create an attribute named ϵ for a Python object using a dot expression, the name is converted to ε without any warning. Is this expected? If it is, where can I find a reference about this? Python 3.12.5 (main, Aug 7 2024, 13:49:14) [GCC 14.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> a = type('',(),{})() >>> a.ϵ = 100 >>> hasattr(a,'ϵ') False >>> hasattr(a,'ε') True >>> setattr(a,'ϵ',200) >>> hasattr(a,'ϵ') True >>> a.ϵ 100 >>> 

Answer: Those two are the same identifier in normal form NFKC. >>> unicodedata.normalize("NFKC", 'ε') == unicodedata.normalize("NFKC", 'ϵ') True  And as the docs say,  All identifiers are converted into the normal form NFKC while parsing; comparison of identifiers is based on NFKC.  So – yes, it's expected.
Question: Example Given the file ./mylib/minimalistic_repro.py class Foo: def __init__(self, param): self.param = param class Bar: def __init__(self, param): self.param = param def foobar(par1, par2): return par1.param + par2.param  and its stub ./mylib/minimalistic_repro.pyi from typing import TypeAlias FooBar: TypeAlias = Foo | Bar class Foo: param: int def __init__(self, param: int) -> None: ... class Bar: param: int def __init__(self, param: int) -> None: ... def foobar(par1: FooBar, par2: FooBar) -> int: ...  Note the type alias on the third line: FooBar: TypeAlias = Foo | Bar mypy --strict mylib, flake8 mylib and ruff check --select E,F,B,SIM all pass. When running stubtest however: python -m mypy.stubtest mylib I get the following error: error: mylib.minimalistic_repro.FooBar is not present at runtime …  My current workaround is to use an allowlist (stubtest --generate-allowlist).  Question(s) ● Is there a “better” way to avoid this “error”? / … ● … Am I doing something fundamentally wrong? … ● … and if not: Might this be worth a feature request?  Other approaches ● Of course I could declare def foobar(par1: Foo | Bar, par2: Foo | Bar), but my actual task (writing type hints for a third party pypi package) requires a union of up to 18 types. ● I got the above example to run with stubtest by placing the FooBar type alias definition in a .py file (tp_aliases.py) and then reimporting. This approach failed to work in the case of with my actual pypi package type hinting task (tp_aliases.py is not part of the pypi package).

Answer: stubtest is complaining because it thinks your FooBar is a public API symbol, which might cause type checkers/IDE autocomplete to make incorrect assumptions and suggestions. The "correct" way to fix it is to make it private; that is, precede the name with an underscore: _FooBar: TypeAlias = Foo | Bar def foobar(par1: _FooBar, par2: _FooBar) -> int: ...  For classes and functions, you can alternatively use typing.type_check_only: # type_check_only is not available at runtime and can only be used in stubs from typing import type_check_only @type_check_only def this_function_is_not_available_at_runtime() -> None: ... @type_check_only class AndSoDoesThisClass: ...  There's also the command line option --ignore-missing-stub which will suppress all errors of this kind.
Question: I have the following Python code for loading the given page in selenium and taking a screenshot of the table. from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from PIL import Image import io import os chrome_options = Options() chrome_options.add_argument("--headless") chrome_options.add_argument("--window-size=1920x1080") chrome_options.add_argument("--force-device-scale-factor=1") driver = webdriver.Chrome(options=chrome_options) path = os.path.abspath("table_minimal.html") driver.get(f"file://{path}") table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.TAG_NAME, "table"))) png = table.screenshot_as_png im = Image.open(io.BytesIO(png)) im.save("table.png") driver.quit()  Here is the table_minimal.html: <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <link href="https://fonts.googleapis.com/css2?family=Montserrat:wght@400;700&display=swap" rel="stylesheet"> <style> table { font-family: 'Montserrat'; font-size: 20px; } </style> </head> <body> <table><tr><td>Text in the table</td></tr></tbody></table> </body> </html>  The above produces this image:  If I remove the --headless flag then I get this instead:  So first one does not use the custom font, the second does but detects incorrect table width. Is there some settings I am missing to make this work correctly? Preferably in headless mode, but now I am curious why it does not work in normal mode as well...

Answer: Note: Changed the font to Matemasie to make it stand out more in the screenshot.  The issue is caused by the screenshot being taken before the custom font has loaded. There are some ways to wait for that to finish before continuing, eg:  Selenium ChromeDriver "get" doesn't reliably load @import fonts  time.sleep(2) also works on such a simple DOM :)  Adding that to the code, and using table.screenshot() to take the screenshot gives us: path = os.path.abspath("table_minimal.html") driver.get(f"file://{path}") while True: script = '''return document.fonts.status;''' loaded = driver.execute_script(script) if loaded == 'loaded': break time.sleep(.5) table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.TAG_NAME, "table"))) table.screenshot('./test3.png') driver.quit()  Gives the expected result, with both --headless and without:
Question: import polars as pl df = pl.DataFrame( {"name": list("abcdef"), "age": [21, 31, 32, 53, 45, 26], "country": list("AABBBC")} ) df.group_by("country").agg( pl.col("name").sort_by("age").first().alias("age_sort_1"), pl.col("name").sort_by("age").get(2).alias("age_sort_2"), # OutOfBoundsError: index out of bounds # pl.col("name").sort_by("age").arr.get(2, null_on_oob=True).alias("age_2"), # SchemaError: invalid series dtype: expected `FixedSizeList`, got `str` pl.col("name").sort_by("age").last().alias("age_sort_-1") )  As shown in the code above, I want to get the name in each country whose age is in a specific order. However, Expr.get does not provide the null_on_oob parameter. How to automatically fill in null when an out-of-bounds situation occurs? In addition, the .arr.get method provides the null_on_oob parameter, but reports an error SchemaError: invalid series dtype: expected "FixedSizeList", got "str". I don’t know what this error refers to and how to solve it. ps: The above code uses the repeated code pl.col("name").sort_by("age") many times. Is there a more concise method?

Answer: There's an open issue related to your question now - polars.Expr.take returns null if ComputeError: index out of bounds. For now you can either use shift() (maintain_order = True just to make it more readable): exp = pl.col.name.sort_by("age") ( df .group_by("country", maintain_order = True).agg( exp.first().alias("age_sort_1"), exp.shift(-2).first().alias("age_sort_2"), exp.last().alias("age_sort_-1"), ) ) ┌─────────┬────────────┬────────────┬─────────────┐ │ country ┆ age_sort_1 ┆ age_sort_2 ┆ age_sort_-1 │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ str ┆ str │ ╞═════════╪════════════╪════════════╪═════════════╡ │ A ┆ a ┆ null ┆ b │ │ B ┆ c ┆ d ┆ d │ │ C ┆ f ┆ null ┆ f │ └─────────┴────────────┴────────────┴─────────────┘  Or, just aggregate your data into list, and then use .list.get() which allows you to use null_on_oob parameter: ( df .group_by("country").agg( pl.col.name.sort_by("age") ) .with_columns( pl.col.name .list.get(i, null_on_oob = True).alias(f"age_sort_{c}") for i, c in [(0, 1), (2, 2), (-1, -1)] ).drop("name") )  Alternatively you can use list.gather() to get the list of 3 elements you need, convert it to struct via list.to_struct() method and then unnest() it to columns: ( df .group_by("country").agg( pl.col.name.sort_by("age") ) .with_columns( pl.col.name .list.gather([0,2,-1], null_on_oob = True) .list.to_struct(fields=["age_sort_1","age_sort_2","age_sort_-1"]) ).unnest("name") )
Question: I am considering introducing my organization to Poetry for Python, and I came across this claim:  Avoid using the Poetry tool for new projects. Poetry uses non-standard implementations of key features. For example, it does not use the standard format in pyproject.toml files, which may cause compatibility issues with other tools.  --Modern Good Practices for Python Development Is this true? I didn't immediately turn up anything searching. What does Poetry do that is nonstandard, in the pyproject.toml or anywhere else?

Answer: Update: Poetry 2.0.0 is released and has support for pyproject.toml project settings!  Yes, Poetry as of writing still uses its own [tool.poetry.dependencies] table in pyproject.toml. This is in conflict with PEP621, which specifies among other things that a project's dependencies ought to be listed in [project.dependencies] instead. There is an issue tracking this here https://github.com/python-poetry/poetry/issues/3332 and it seems that as of writing support for PEP621 is on the way, with a draft pull request passing checks here: https://github.com/python-poetry/poetry/pull/9135. The main downside of this is that other tools do not understand which packages your poetry project depends on. I believe the built packages that you upload to PyPI etc do not suffer from this issue, because poetry during the build process creates correct package metadata with a fixed set of version-pinned dependencies, so pip install your-poetry-package would work just fine to install the dependencies. In conclusion, in poetry version 2.0, some time soon poetry might be compatible with pyproject.toml standards.
Question: I'm trying to make a simple password app on my desktop from a code I wrote. I've done this before with no troubles on another simple app. This is my setup: from setuptools import setup APP = ['main.py'] OPTIONS = { 'argv_emulation': True, 'iconfile':"logo.png", 'packages': ['tkinter', 'random', 'json', 'pyperclip'] } setup( app=APP, options={'py2app': OPTIONS}, setup_requires=['py2app'], )  This is the last few lines of code that run in my terminal if I run Python 3 setup.py py2app /Users/cjsander/Desktop/Password/build/bdist.macosx-10.9-universal2/python3.12- standalone/app/collect/typing_extensions-4.12.2.dist-info/METADATA error: [Errno 17] File exists: '/Users/cjsander/Desktop/Password/build/bdist.macosx-10.9- universal2/python3.12-standalone/app/collect/packaging-24.1.dist-info'  The Dist is never created, and even when I move folders that I attempt to run the code from or delete all previous Dist and Build folders this error is produced. I've tried to move the file location, delete previous DIST and BUILD folders, uninstalled and reinstalled py2app including not updating the install. I've also tried on other code files and the same thing happens. All resulted with the same error, even when the file doesn't exist.

Answer: This is a known issue in setuptools >= 70.0.2. Pin 70.3.0 to get past it. For example in a pyproject.tml: [tool.poetry.dependencies] python = "^3.12" pyqt5 = "^5.15.10" setuptools = "==70.3.0" # add this line
Question: I'm trying to reduce the margins of a page using borb, that works, but my Table then is not taking the full width of the page. Please note that not only Table is shifted and not covering the full width, HorizontalRule as well etc... What's the fix for that ? from decimal import Decimal from borb.pdf import Document, Page, SingleColumnLayout, PageLayout, Paragraph, Image, PDF def main(): # Create document pdf = Document() # Add page page = Page() pdf.add_page(page) # create PageLayout page_layout: PageLayout = SingleColumnLayout(page) for i in [Decimal(0), Decimal(16), Decimal(32), Decimal(64)]: page_layout._margin_left = i page_layout._margin_right = i page_layout._margin_bottom = i page_layout._margin_top = i # Add Paragraph page_layout.add(Paragraph(f"padding set at {int(i)}")) page_layout.add( FixedColumnWidthTable( number_of_columns=3, number_of_rows=3 ) ) with open("output.pdf", "wb") as out_file_handle: PDF.dumps(out_file_handle, pdf) if __name__ == "__main__": main()  Here's what I would like.

Answer: disclaimer: I am the author of borb The "issue" you are running into is that borb applies a sensible default for margins (on Image objects). Looking at the (constructor) code in Image:  margin_bottom=margin_bottom if margin_bottom is not None else Decimal(5), margin_left=margin_left if margin_left is not None else Decimal(5), margin_right=margin_right if margin_right is not None else Decimal(5), margin_top=margin_top if margin_top is not None else Decimal(5),  So you explicitly need to pass margin_left=Decimal(0) (and other) to have an Image whose total distance to the page-boundary is exactly as specified. Update (for adding Table objects): The main issue is that PageLayout also keeps track of the available width, which is used by a FixedColumnWidthTable but not by the Image. So, your code needs to change that as well. from decimal import Decimal from borb.pdf import Document, Page, SingleColumnLayout, PageLayout, Paragraph, Image, PDF, FixedColumnWidthTable def main(): # Create document pdf = Document() # Add page page = Page() pdf.add_page(page) # create PageLayout page_layout: PageLayout = SingleColumnLayout(page) for i in [Decimal(0), Decimal(16), Decimal(32), Decimal(64)]: # change margin # IMPORTANT: you are not supposed to do this after having created # the PageLayout page_layout._margin_left = i page_layout._margin_right = i page_layout._margin_bottom = i page_layout._margin_top = i page_layout._column_widths = [Decimal(595) - i - i] # Add Paragraph page_layout.add(Paragraph(f"padding set at {int(i)}")) page_layout.add( FixedColumnWidthTable( number_of_columns=2, number_of_rows=3 ).add(Paragraph("Lorem")) .add(Paragraph("Ipsum")) .add(Paragraph("Dolor")) .add(Paragraph("Sit")) .add(Paragraph("Amet")) .add(Paragraph("Consectetur")) ) with open("output.pdf", "wb") as out_file_handle: PDF.dumps(out_file_handle, pdf) if __name__ == "__main__": main()  Which outputs the following PDF:
Question: I built sending several txs in one jito bundle with Node.js. I want convert this code to python. When I use jito bundle, I should serialize transactions. When I use Node.js, I can use transaction.serialize(). But when I use python, I used transaction.serialize(), it occurs error.  'solders.transaction.VersionedTransaction' object has no attribute 'serialize'  How can I serialize VersionedTransaction in Solana with python?

Answer: It looks like VersionedTransaction implements bytes at https://github.com/kevinheavey/solders/blob/c10419c29b7890e69572f0160e5e74406814048b/python/solders/transaction.pyi#L121, so you should be able to serialize the transaction with: bytes(transaction)
Question: On Django 4.x Code is working as expected from django.db import models class Project(models.Model): rough_data = models.OneToOneField( "Data", related_name="rough_project", on_delete=models.SET_NULL, null=True, blank=True, ) final_data = models.OneToOneField( "Data", related_name="final_project", on_delete=models.SET_NULL, null=True, blank=True, ) class Data(models.Model): pass data, created = Data.objects.update_or_create( rough_project=project, defaults=data )  On Django 5.x: ValueError: The following fields do not exist in this model: rough_project  I do not see any changes related to this in changelog In Django 4.2 and below it worked as one line with update_or_create and in 5.0 it stopped working, so instead I need to do something like this: data = getattr(project, "rough_data", None) if data: # update fields here else: # create Data object 

Answer: In Django 4.x, it did not raise an exception but also did not save the relationship if the object was created — so it was actually not working as expected.  Django ticket: https://code.djangoproject.com/ticket/34586 Django PR that fixed1 it to raise exception: https://github.com/django/django/pull/17112/files  It appears to work as expected only if project.save() is called afterwards. 1You could fix it to proactively save the relationship instead, by subclassing QuerySet and overriding the create method, and then specifying Data objects to use your Manager: class CreateSaveReverseOneToOneFieldsQuerySet(models.QuerySet): def create(self, **kwargs): """ Create a new object with the given kwargs, saving it to the database and returning the created object. Also save objects in reverse OneToOne fields — see https://stackoverflow.com/questions/78863608/django-5-update-or-create-reverse-one-to-one-field. """ obj = self.model(**kwargs) self._for_write = True obj.save(force_insert=True, using=self.db) # Save reverse OneToOne fields reverse_one_to_one_fields = frozenset(kwargs).intersection(self.model._meta._reverse_one_to_one_field_names) for field_name in reverse_one_to_one_fields: getattr(obj, field_name).save() return obj class DataManager(models.manager.BaseManager.from_queryset(CreateSaveReverseOneToOneFieldsQuerySet)): pass class Data(models.Model): objects = DataManager()
Question: I was making a simple Input field in FastHTML. A function MakeInput returns the input field as I wish to clear the input field after processing the form. def MakeInput(): return Input( placeholder='Add new Idea', id='title', hx_swap_oob='true' ) @route('/todos') def get(): form = Form( Group( MakeInput(), Button('Save',hx_post='/',target_id='todo-list',hx_swap='beforeend') ) ) return Titled( 'List', Card( Ul(*todos(),id='todo-list'), header=form ) ) @route('/') def post(todo:Todo): return todos.insert(todo), MakeInput()  But when the final HTML is rendered, the attribute becomes hx-swap:oob='true'. This causes problem for the htmx part to replace the existing element, and creates a new element. for eg: Actual Result It should be hx-swap-oob='true'. Is there is issue with the way I am passing the attributes in FastHTML functions?

Answer: This is a bug that was fixed in version 0.3.1, see the following issue: https://github.com/AnswerDotAI/fasthtml/issues/254 Update to version 0.3.1 or higher and you should be fine.
Question: I have a script in which I go through and parse a large collection of PDFs. I noticed that when I tried to parse a particular PDF, the script just stalls forever. But it doesn't throw up an error and as far as I can tell, the PDF is not corrupted. I can't tell what the issue is, but I can see that it happens on page 4. Is there a way to find out what is causing this issue, or to just skip the PDF if it is taking longer than one minute to parse? For reference, here is the PDF: https://go.boarddocs.com/fl/palmbeach/Board.nsf/files/CTWGW9459021/$file/22C-001R_2ND%20RENEWAL%20CONTRACT_TERRACON.pdf from PyPDF2 import PdfReader doc = "somefile.pdf" doc_text = "" try: print(doc) reader = PdfReader(doc) for i in range(len(reader.pages)): print(i) page = reader.pages[i] text = page.extract_text() doc_text += text except Exception as e: print(f"The file failed due to error {e}:") doc_text = "" 

Answer: You should not use PyPDF2 any more unless really required and switch to pypdf instead - see the note on PyPI as well: https://pypi.org/project/PyPDF2/ Running the corresponding migrated code with the latest release does not show any performance issues: from pypdf import PdfReader doc = "78867160.pdf" doc_text = "" try: print(doc) reader = PdfReader(doc) for i, page in enumerate(reader.pages): print(i) text = page.extract_text() doc_text += text except Exception as e: print(f"The file failed due to error {e}:") doc_text = ""
Question: I'm trying to scrape URLs from a dynamically allocated webpage that requires continuous scrolling to load all the content into the DOM. My approach involves running window.scrollTo(0, document.body.scrollHeight); in a loop using Selenium's execute_script function. After each scroll, I compare the number of URLs loaded before and after the scroll. If the number of URLs doesn't change, I assume the end of the page has been reached and break the loop. However, the script assumes that all content has been loaded into the DOM, even though I know new content is being loaded within the given timeout. Below is my code: def _scroll_page_to_bottom(self, timeout: int): # Todo: Fix Bugs while True: urls_before_scroll = self.browser.find_elements( By.XPATH, read_xpath(self.scrape_programs_urls.__name__, "programs_urls") ) self.browser.execute_script("window.scrollTo(0, document.body.scrollHeight);") # Wait for new content to be loaded try: WebDriverWait(self.browser, timeout).until( lambda _: len(self.browser.find_elements( By.XPATH, read_xpath(self.scrape_programs_urls.__name__, "programs_urls")) ) > len(urls_before_scroll) ) except TimeoutException: # If no new content is loaded within the timeout, assume we've reached the end of the page break  Can anyone please guess what could be causing the issue in the above code? Edit: i did some debugging and found the issue is specifically related to scroll functionality when i execute window.scrollTo(0, document.body.scrollHeight); in the console of the browser the page doesn't get scrolled to the bottom either which explains why my code is not working. The site am trying to scrape is https://hackerone.com/opportunities/all/search

Answer: This code below works well in scrolling down the page, try to embed it into your code: ele = driver.find_element(By.XPATH, '//div[contains(@class,"Pane-module_u1-pane__content")]') driver.execute_script('arguments[0].scrollIntoView(false);', ele)
Question: I am working with the following code... import polars as pl df = pl.DataFrame({ 'region': ['GB', 'FR', 'US'], 'qty': [3, 6, -8], 'price': [100, 102, 95], 'tenor': ['1Y', '6M', '2Y'], }) cols_to_set = ['price', 'tenor'] fill_val = '-' df.with_columns([pl.lit(fill_val).alias(c) for c in cols_to_set])  ...with the following output. shape: (3, 4) ┌────────┬─────┬───────┬───────┐ │ region ┆ qty ┆ price ┆ tenor │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ str ┆ str │ ╞════════╪═════╪═══════╪═══════╡ │ GB ┆ 3 ┆ - ┆ - │ │ FR ┆ 6 ┆ - ┆ - │ │ US ┆ -8 ┆ - ┆ - │ └────────┴─────┴───────┴───────┘  Instead of using a list of pl.lit expressions, I thought I could use a single pl.lit(fill_val).alias(cols_to_set). However, this crashes with an error TypeError: argument 'name': 'list' object cannot be converted to 'PyString'  Is there a way to simplify the above and set all columns in cols_to_set to a specific constant value fill_val?

Answer: you could use replace_strict(): df.with_columns(pl.col(cols_to_set).replace_strict(None, None, default = '-'))  ┌────────┬─────┬───────┬───────┐ │ region ┆ qty ┆ price ┆ tenor │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ str ┆ str │ ╞════════╪═════╪═══════╪═══════╡ │ GB ┆ 3 ┆ - ┆ - │ │ FR ┆ 6 ┆ - ┆ - │ │ US ┆ -8 ┆ - ┆ - │ └────────┴─────┴───────┴───────┘