eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following line of code that keeps giving me an error that Engine object has no object execute. I think I have everything right but no idea what keeps happening. It seemed others had this issue and restarting their notebooks worked. I'm using Pycharm and have restarted that without any resolution. Any help is greatly appreciated! import pandas as pd from sqlalchemy import create_engine, text import sqlalchemy import pymysql masterlist = pd.read_excel('Masterlist.xlsx') user = 'root' pw = 'test!*' db = 'hcftest' engine = create_engine("mysql+pymysql://{user}:{pw}@localhost:3306/{db}" .format(user=user, pw=pw, db=db)) results = engine.execute(text("SELECT * FROM companyname;")) for row in results: print(row) 

Answer: There was a change from 1.4 to 2.0. The above code will run fine with sqlalchemy version 1.4 I believe. setting SQLALCHEMY_WARN_20=1 python and running the above code reveals this warning: <stdin>:1: RemovedIn20Warning: The Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)  So the correct way to do the code now is: with engine.connect() as conn: result = conn.execute(stmt)  source here describing the behavior in 1.4 and here describing the behavior in 2.0
Question: In TypeScript you can override type inferences with the as keyword const canvas = document.querySelector('canvas') as HTMLCanvasElement;  Are there similar techniques in Python3.x typing without involving runtime casting? I want to do something like the following: class SpecificDict(TypedDict): foo: str bar: str res = request(url).json() as SpecificDict 

Answer: If I understand you correctly, you're looking for typing.cast: from typing import cast res = cast(dict, request(url))  This will assert to a typechecker that res is assigned to a value that is a dictionary, but it won't have any effects at runtime.
Question: Today I was solving Project Euler's problem #43 Problem and I ran into a somewhat interesting problem. I don't understand why is my code so fast? from itertools import permutations def main(): numbers1_9 = [0,1,2,3,4,5,6,7,8,9] list_of_all_permutations = list(permutations(numbers1_9, 10)) length_of_my_list = len(list_of_all_permutations) number_of_times_it_ran=0 result = [] for n in list_of_all_permutations: number_of_times_it_ran+=1 if n[0] == 0: continue elif n[3] % 2 == 0 and (n[2]+n[3]+n[4]) % 3 == 0 and n[5] % 5 ==0 and int(str(n[4])+str(n[5])+str(n[6])) % 7 == 0 and (n[5]+n[7]-n[6]) % 11 == 0 and int(str(n[6])+str(n[7])+str(n[8])) % 13 == 0 and int(str(n[7])+str(n[8])+str(n[9])) % 17 == 0: temp_list = [] for digits_of_n in n: temp_list.append(str(digits_of_n)) result.append(int("".join(temp_list))) print(f"Added {temp_list}, Remaining: {length_of_my_list-number_of_times_it_ran}") print(f"The code ran {number_of_times_it_ran} times and the result is {sum(result)}") if __name__ == "__main__": main()  I mean it went through the for loop 3,628,800 times, checked all those parameters, and only took a second. Added ['1', '4', '0', '6', '3', '5', '7', '2', '8', '9'], Remaining: 3142649 Added ['1', '4', '3', '0', '9', '5', '2', '8', '6', '7'], Remaining: 3134251 Added ['1', '4', '6', '0', '3', '5', '7', '2', '8', '9'], Remaining: 3124649 Added ['4', '1', '0', '6', '3', '5', '7', '2', '8', '9'], Remaining: 2134649 Added ['4', '1', '3', '0', '9', '5', '2', '8', '6', '7'], Remaining: 2126251 Added ['4', '1', '6', '0', '3', '5', '7', '2', '8', '9'], Remaining: 2116649 The code ran 3628800 times, and the result is 16695334890  This is the output. The code finished in 1.3403266999521293 seconds.

Answer: It runs so quickly because of the short-circuiting feature of logical operators. The first three conditions in the if statement are easy to calculate, and they filter out the vast majority (around 97%) of all the permutations, so you hardly ever need to execute the more expensive operations like int(str(n[4])+str(n[5])+str(n[6])). So when you have a bunch of conditions that you're connecting with and, and the order that they're tested doesn't matter for the logic, you should put the ones that are easiest to test or are most likely to fail first.
Question: I am trying to setup a simple 3D Engine in pyOpenGL. My current goal was to achieve a 2D rectangle being displayed to the screen, which isn't working at all. (nothing is being rendered to the screen, no exception is being thrown by the program.) The render method I use is following:  @staticmethod def render(model: RawModel): glBindVertexArray(model.get_vao_id()) glEnableVertexAttribArray(0) glDrawArrays(GL_TRIANGLES, 1, model.get_vertex_count()) glDisableVertexAttribArray(0) glBindVertexArray(0)  I suppose something goes wrong with the glDrawArrays() method, because of how I bound my Buffer data: @classmethod def bind_indices_buffer(cls, attribute_number: int, data: list): data = numpy.array(data, dtype='float32') vbo_id = glGenBuffers(1) cls.__vbos.append(vbo_id) glBindBuffer(GL_ARRAY_BUFFER, vbo_id) glBufferData(GL_ARRAY_BUFFER, data, GL_STATIC_DRAW) glVertexAttribPointer(attribute_number, 3, GL_FLOAT, False, 0, 0) glBindBuffer(GL_ARRAY_BUFFER, 0) 

Answer: The problem is here: glVertexAttribPointer(attribute_number, 3, GL_FLOAT, False, 0, 0) glVertexAttribPointer(attribute_number, 3, GL_FLOAT, False, 0, None)  The type of the lase argument of glVertexAttribIPointer is const GLvoid *. So the argument must be None or ctypes.c_void_p(0), but not 0.
Question: My aim is to get the font path from their common font name and then use them with PIL.ImageFont. I got the names of all installed fonts by using tkinter.font.families(), but I want to get the full path of each font so that I can use them with PIL.ImageFont. Is there any other way to use the common font name with ImageFont.truetype() method?

Answer: I'm not exactly sure what you really want - but here is a way to get a list of the full path to all the fonts on your system and their names and weights: #!/usr/bin/env python3 import matplotlib.font_manager from PIL import ImageFont # Iterate over all font files known to matplotlib for filename in matplotlib.font_manager.findSystemFonts(): # Avoid these two trouble makers - don't know why they are problematic if "Emoji" not in filename and "18030" not in filename: # Look up what PIL knows about the font font = ImageFont.FreeTypeFont(filename) name, weight = font.getname() print(f'File: {filename}, fontname: {name}, weight: {weight}')  Sample Output File: /System/Library/Fonts/Supplemental/NotoSansLepcha-Regular.ttf, fontname: Noto Sans Lepcha, weight: Regular File: /System/Library/Fonts/ZapfDingbats.ttf, fontname: Zapf Dingbats, weight: Regular File: /System/Library/Fonts/Supplemental/Zapfino.ttf, fontname: Zapfino, weight: Regular File: /System/Library/Fonts/Supplemental/NotoSansMultani-Regular.ttf, fontname: Noto Sans Multani, weight: Regular File: /System/Library/Fonts/Supplemental/NotoSansKhojki-Regular.ttf, fontname: Noto Sans Khojki, weight: Regular File: /System/Library/Fonts/Supplemental/Mishafi Gold.ttf, fontname: Mishafi Gold, weight: Regular File: /System/Library/Fonts/Supplemental/NotoSansMendeKikakui-Regular.ttf, fontname: Noto Sans Mende Kikakui, weight: Regular File: /System/Library/Fonts/MuktaMahee.ttc, fontname: Mukta Mahee, weight: Regular File: /Users/mark/Library/Fonts/JetBrainsMonoNL-Italic.ttf, fontname: JetBrains Mono NL, weight: Italic ... ...
Question: I am trying to update my Python CI environment and am working on package management right now. I have several reasons that I do not want to use Poetry; however, one nice feature of poetry is the fact that it automatically updates the pyproject.toml file. I know that pip-tools can create a requirements.txt file from the pyproject.toml file; however, is there any feature within virtualenv or pip-tools that will enable an automatic update of the pyproject.toml file when you install a package with pip to your virtual environment?

Answer: A standard tool-agnostic add command does not exist. It is being discussed here: https://discuss.python.org/t/poetry-add-but-for-pep-621/22957 I do not know if there is such a feature in pip-tools. I am pretty sure it does not exist in virtualenv, that would be quite out of scope. Your can always adopt a "dev workflow tool" (PDM, Hatch, Poetry*) if you want it.  *: Poetry uses a non-standard notation for the project metadata, but you seem to be currently using the standard notation (aka PEP621) so this could be a step backward if you were to migrate to Poetry.
Question: I have a Python (3.8) metaclass for a singleton as seen here I've tried to add typings like so: from typing import Dict, Any, TypeVar, Type _T = TypeVar("_T", bound="Singleton") class Singleton(type): _instances: Dict[Any, _T] = {} def __call__(cls: Type[_T], *args: Any, **kwargs: Any) -> _T: if cls not in cls._instances: cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls]  In the line: _instances: Dict[Any, _T] = {}  MyPy warns: Mypy: Type variable "utils.singleton._T" is unbound I've tried different iterations of this to no avail; it's very hard for me to figure out how to type this dict. Further, the line: def __call__(cls: Type[_T], *args: Any, **kwargs: Any) -> _T:  Produces: Mypy: The erased type of self "Type[golf_ml.utils.singleton.Singleton]" is not a supertype of its class "golf_ml.utils.singleton.Singleton" How could I correctly type this?

Answer: This should work: from __future__ import annotations import typing as t _T = t.TypeVar("_T") class Singleton(type, t.Generic[_T]): _instances: dict[Singleton[_T], _T] = {} def __call__(cls, *args: t.Any, **kwargs: t.Any) -> _T: if cls not in cls._instances: cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls]  Rough explanations:  _T = TypeVar("_T", bound="Singleton") is not correct - Singleton is type(type(obj)) where obj: _T = Singleton.__call__(...). In proper usage, the argument to bound= can only be type(obj) or some union typing construct, not type(type(obj). Type variable "_T" is unbound indicates that you need to make Singleton generic with respect to _T to bind _T. The erased type of self ... error message is telling you that you've "erased" the type checker's inferred type* of cls. Technically speaking, __call__ is the same on a metaclass as any other instance method - the first argument is simply the type of the owning class. In the current static typing system, however, a metaclass's instance method's first argument is not in concordance with type[...].   *The inferred type is explicitly Self in the following: import typing as t Self = t.TypeVar("Self", bound="A") class A: def instancemethod(arg: Self) -> None: pass @classmethod def classmethod_(arg: type[Self]) -> None: pass    Runtime is important too, so the final sanity check is to make sure you've actually implemented a singleton using this metaclass: class Logger(metaclass=Singleton): pass >>> print(Logger() is Logger()) True
Question: I am trying to pick up the package polars from Python. I come from an R background so appreciate this might be an incredibly easy question. I want to implement a case statement where if any of the conditions below are true, it will flag it to 1 otherwise it will be 0. My new column will be called 'my_new_column_flag' I am however getting the error message  Traceback (most recent call last): File "", line 2, in File "C:\Users\foo\Miniconda3\envs\env\lib\site-packages\polars\internals\lazy_functions.py", line 204, in col return pli.wrap_expr(pycol(name)) TypeError: argument 'name': 'int' object cannot be converted to 'PyString'  import polars as pl import numpy as np np.random.seed(12) df = pl.DataFrame( { "nrs": [1, 2, 3, None, 5], "names": ["foo", "ham", "spam", "egg", None], "random": np.random.rand(5), "groups": ["A", "A", "B", "C", "B"], } ) print(df) df.with_columns( pl.when(pl.col('nrs') == 1).then(pl.col(1)) .when(pl.col('names') == 'ham').then(pl.col(1)) .when(pl.col('random') == 0.014575).then(pl.col(1)) .otherwise(pl.col(0)) .alias('my_new_column_flag') )  Can anyone help?

Answer: pl.col selects a column with the given name (as string). What you want is a column with literal value set to one: pl.lit(1) df.with_columns( pl.when(pl.col('nrs') == 1).then(pl.lit(1)) .when(pl.col('names') == 'ham').then(pl.lit(1)) .when(pl.col('random') == 0.014575).then(pl.lit(1)) .otherwise(pl.lit(0)) .alias('my_new_column_flag') )  PS: it may look more natural to use predicate for your flat (and cast it to int if you want it to be 0/1 instead of true/false):  df.with_columns( ((pl.col("nrs") == 1) | (pl.col("names") == "ham") | (pl.col("random") == 0.014575)) .alias("my_new_column_flag") .cast(int) )
Question: If I have a series of numbers in a DataFrame with one column, e.g.: import pandas as pd data = [4, 5, 6, 7, 8, 9, 10, 11] pd.DataFrame(data)  Which looks like this (left column = index, right column = data): 0 4 1 5 2 6 3 7 4 8 5 9 6 10 7 11  How do I make it into one sequence number, so (4 5 6 7 8 9 10 11) in python or pandas ? because i want to put that into xml file so it looks like this  <Or> <numbers> <example>4 5 6 7 8 9 10 11</example> </numbers> </Or> 

Answer: You can use a f-string with conversion of the integers to string and str.join: text = f''' <Or> <numbers> <example>{" ".join(s.astype(str))}</example> </numbers> </Or>'''  Output:  <Or> <numbers> <example>4 5 6 7 8 9 10 11</example> </numbers> </Or>
Question: I am developing a fastapi inside a docker container in windows/ubuntu (code below). When I test the app outside the container by running python -m uvicorn app:app --reload in the terminal and then navigating to 127.0.0.1:8000/home everything works fine: { Data: "Test" }  However, when I docker-compose up I can neither run python -m uvicorn app:app --reload in the container (due to the port already being used), nor see anything returned in the browser. I have tried 127.0.0.1:8000/home, host.docker.internal:8000/home and localhost:8000/home and I always receive: { detail: "Not Found" }  What step am I missing? Dockerfile: FROM python:3.8-slim EXPOSE 8000 ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONUNBUFFERED=1 COPY requirements.txt . RUN python -m pip install -r requirements.txt WORKDIR /app COPY . /app RUN adduser -u nnnn --disabled-password --gecos "" appuser && chown -R appuser /app USER appuser CMD ["gunicorn", "--bind", "0.0.0.0:8000", "-k", "uvicorn.workers.UvicornWorker", "app:app"]  Docker-compose: version: '3.9' services: fastapitest: image: fastapitest build: context: . dockerfile: ./Dockerfile ports: - 8000:8000 extra_hosts: - "host.docker.internal:host-gateway"  app.py: from fastapi import FastAPI app = FastAPI() @app.get("/home") #must be one line above the function fro the route def home(): return {"Data": "Test"} if __name__ == '__main__': import uvicorn uvicorn.run(app, host="127.0.0.1", port=8000) 

Answer: The issue here is that when you specify host="127.0.0.1" to uvicorn, that means you can only access that port from that same machine. Now, when you run outside docker, you are on the same machine, so everything works. But since a docker container is (at least to some degree) a different computer, you need to tell it to allow connections from outside the container as well. To do this, switch to host="0.0.0.0", and then you should be able to access your dockerized API on http://localhost:8000.
Question: I want to read a SQLite database file (database.sqlite) using polars package. I tried following unsuccessfully: import sqlite3 import polars as pl conn = sqlite3.connect('database.sqlite') df = pl.read_sql("SELECT * from table_name", conn) print(df)  Getting following error: AttributeError: 'sqlite3.Connection' object has no attribute 'split'  Any suggestions?

Answer: From the docs, you can see pl.read_sql accepts connection string as a param, and you are sending the object sqlite3.Connection, and that's why you get that message. You should first generate the connection string, which is url for your db db_path = 'database.sqlite' connection_string = 'sqlite://' + db_path  And after that, you can type the updated next line, which gave you problems: df = pl.read_sql("SELECT * from table_name", connection_string)
Question: Yolov8 and I suspect Yolov5 handle non-square images well. I cannot see any evidence of cropping the input image, i.e. detections seem to go to the enge of the longest side. Does it resize to a square 640x604 which would change the aspect ratio of objects making them more difficult to detect? When training on a custom dataset starting from a pre-trained model, what does the imgsz (image size) parameter actually do?

Answer: Modern Yolo versions, from v3 onwards, can handle arbitrary sized images as long as both sides are a multiple of 32. This is because the maximum stride of the backbone is 32 and it is a fully convolutional network. But there are clearly two different cases for how input images to the model are preprocessed: Training An example. Let's say you start a training by: from ultralytics.yolo.engine.model import YOLO model = YOLO("yolov8n.pt") results = model.train(data="coco128.yaml", imgsz=512)  By printing what is fed to the model (im) in trainer.py you will obtain the following output: Epoch GPU_mem box_loss cls_loss dfl_loss Instances Size 0%| | 0/8 [00:00<?, ?it/s] torch.Size([16, 3, 512, 512]) 1/100 1.67G 1.165 1.447 1.198 226 512: 12%|█▎ | 1/8 [00:01<00:08, 1.15s/it] torch.Size([16, 3, 512, 512]) 1/100 1.68G 1.144 1.511 1.22 165 512: 25%|██▌ | 2/8 [00:02<00:06, 1.10s/it] torch.Size([16, 3, 512, 512])  So, during training, images have to be reshaped to the same size in order to be able to create mini-batches as you cannot concatenate tensors of different shapes. To preserve the aspect ratio of the images, in order to avoid distortion, they are usually "letterbox'ed". imgsz selects the size of the images to train on. Prediction Now, let's have a look at prediction. Let's say you select the images under assets as source and imgsz 512 by from ultralytics.yolo.engine.model import YOLO model = YOLO("yolov8n.pt") results = model.predict(stream=True, imgsz=512) # source already setup  By printing the original image shape (im0) and the one fed to the model (im) in predictor.py you will obtain the following output: (yolov8) ➜ ultralytics git:(main) ✗ python new.py Ultralytics YOLOv8.0.23 🚀 Python-3.8.15 torch-1.11.0+cu102 CUDA:0 (Quadro P2000, 4032MiB) YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs im0s (1080, 810, 3) im torch.Size([1, 3, 512, 384]) image 1/2 /home/mikel.brostrom/ultralytics/ultralytics/assets/bus.jpg: 512x384 4 persons, 1 bus, 7.4ms im0s (720, 1280, 3) im torch.Size([1, 3, 288, 512]) image 2/2 /home/mikel.brostrom/ultralytics/ultralytics/assets/zidane.jpg: 288x512 3 persons, 2 ties, 5.8ms Speed: 0.4ms pre-process, 6.6ms inference, 1.5ms postprocess per image at shape (1, 3, 512, 512)  You can see that the longest image side is reshaped to 512. The short side is reshaped to the closest multiple of 32 while maintaining the aspect ratio. As you are not feeding multiple images at the same time you don't need to reshape images into the same shape and stack them, making it possible to avoid padding.
Question: Update: pl.cut was removed from Polars. Expression equivalents were added instead: .cut() .qcut()  How can I use it in select context, such as df.with_columns? To be more specific, if I have a polars dataframe with a lot of columns and one of them is called x, how can I do pl.cut on x and append the grouping result into the original dataframe? Below is what I tried but it does not work: df = pl.DataFrame({"a": [1, 2, 3, 4, 5], "b": [2, 3, 4, 5, 6], "x": [1, 3, 5, 7, 9]}) df.with_columns(pl.cut(pl.col("x"), bins=[2, 4, 6]))  Thanks so much for your help.

Answer: From the docs, as of 2023-01-25, cut takes a Series and returns a DataFrame. Unlike many/most methods and functions, it doesn't take an expression so you can't use it in a select or with_column(s). To get your desired result you'd have to join it to your original df. Additionally, it appears that cut doesn't necessarily maintain the same dtypes as the parent series. (This is most certainly a bug) As such you have to cast it back to, in this case, int. You'd have: df=df.join( pl.cut(df.get_column('x'),bins=[2,4,6]).with_columns(pl.col('x').cast(pl.Int64())), on='x' ) shape: (5, 5) ┌─────┬─────┬─────┬─────────────┬─────────────┐ │ a ┆ b ┆ x ┆ break_point ┆ category │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ f64 ┆ cat │ ╞═════╪═════╪═════╪═════════════╪═════════════╡ │ 1 ┆ 2 ┆ 1 ┆ 2.0 ┆ (-inf, 2.0] │ │ 2 ┆ 3 ┆ 3 ┆ 4.0 ┆ (2.0, 4.0] │ │ 3 ┆ 4 ┆ 5 ┆ 6.0 ┆ (4.0, 6.0] │ │ 4 ┆ 5 ┆ 7 ┆ inf ┆ (6.0, inf] │ │ 5 ┆ 6 ┆ 9 ┆ inf ┆ (6.0, inf] │ └─────┴─────┴─────┴─────────────┴─────────────┘
Question: I have a collection of polars expressions being used to generate features for an ML model. I'd like to add a poission cdf feature to this collection whilst maintaining lazy execution (with benefits of speed, caching etc...). I so far have not found an easy way of achieving this. I've been able to get the result I'd like outside of the desired lazy expression framework with: import polars as pl from scipy.stats import poisson df = pl.DataFrame({"count": [9,2,3,4,5], "expected_count": [7.7, 0.2, 0.7, 1.1, 7.5]}) result = poisson.cdf(df["count"].to_numpy(), df["expected_count"].to_numpy()) df = df.with_columns(pl.Series(result).alias("poisson_cdf"))  However, in reality I'd like this to look like: df = pl.DataFrame({"count": [9,2,3,4,5], "expected_count": [7.7, 0.2, 0.7, 1.1, 7.5]}) df = df.select( [ ... # bunch of other expressions here poisson_cdf() ] )  where poisson_cdf is some polars expression like: def poisson_cdf(): # this is just for illustration, clearly wont work return scipy.stats.poisson.cdf(pl.col("count"), pl.col("expected_count")).alias("poisson_cdf")  I also tried using a struct made up of "count" and "expected_count" and apply like advised in the docs when applying custom functions. However, my dataset is several millions of rows in reality - leading to absurd execution time. Any advice or guidance here would be appreciated. Ideally there exists an expression like this somewhere out there? Thanks in advance!

Answer: It sounds like you want to use .map_batches() df.with_columns( pl.struct("count", "expected_count") .map_batches(lambda x: poisson.cdf(x.struct.field("count"), x.struct.field("expected_count")) ) .alias("poisson_cdf") )  shape: (5, 3) ┌───────┬────────────────┬─────────────┐ │ count | expected_count | poisson_cdf │ │ --- | --- | --- │ │ i64 | f64 | f64 │ ╞═══════╪════════════════╪═════════════╡ │ 9 | 7.7 | 0.75308 │ │ 2 | 0.2 | 0.998852 │ │ 3 | 0.7 | 0.994247 │ │ 4 | 1.1 | 0.994565 │ │ 5 | 7.5 | 0.241436 │ └───────┴────────────────┴─────────────┘
Question: When writing a DataFrame to a csv file, I would like to append to the file, instead of overwriting it. While pandas DataFrame has the .to_csv() method with the mode parameter available, thus allowing to append the DataFrame to a file, None of the Polars DataFrame write methods seem to have that parameter.

Answer: To append to a CSV file for example - you can pass a file object e.g. import polars as pl df1 = pl.DataFrame({"a": [1, 2], "b": [3 ,4]}) df2 = pl.DataFrame({"a": [5, 6], "b": [7 ,8]}) with open("out.csv", mode="a") as f: df1.write_csv(f) df2.write_csv(f, include_header=False)  >>> from pathlib import Path >>> print(Path("out.csv").read_text(), end="") a,b 1,3 2,4 5,7 6,8