eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Both Pydantic and Dataclass can typehint the object creation based on the attributes and their typings, like these examples: from pydantic import BaseModel, PrivateAttr, Field from dataclasses import dataclass # Pydantic way class Person(BaseModel): name : str address : str _valid : bool = PrivateAttr(default=False) #dataclass way @dataclass class PersonDataclass(): name : str address : str _valid : bool = False bob = Person(name="Bob", address="New York") bobDataclass = PersonDataclass("Bob", "New York")  With this code, I can get typehint on object creation (see screenshots below): pydantic typehint on object creation dataclass typehint on object creation Not only that, but the object's attributes also get documented.  I studied the code of pydantic to try to achieve the same result, but I couldn't. The code that I tried was this: class MyBaseModelMeta(type): def __new__(cls, name, bases, dct): def new_init(self : cls, /, name : str, address : str): self.name = name self.address = address self._valid = False dct["__init__"] = new_init dct["__annotations__"] = {"__init__": {"name": str, "address": str, "_valid": bool}} return super().__new__(cls, name, bases, dct) class MyBaseModel(metaclass=MyBaseModelMeta): def __repr__(self) -> str: return f"MyBaseModel: {self.__dict__}" class MyPerson(MyBaseModel): pass myBob = MyPerson("Bob", "New York")  My class works (the dynamic init insertion works) but the class and object get no typehint. my class works but it doesn't get typehinted What am I doing wrong? How can I achieve the typehints?

Answer: @Daniil Fajnberg is mostly correct, but depending on your type checker you can can use the dataclass_transform(Python 3.11) or __dataclass_transform__ early adopters program decorator. Pylance and Pyright (usually used in VS-Code) at least work with these. You can only mimic the behaviour of dataclasses that way though, I don't think you're able to define that your Metaclass adds extra fields. :/ Edit: At least pydantic uses this decorator for their BaseModel: https://pydantic-docs.helpmanual.io/visual_studio_code/#technical-details If you dig through the code of pydantic you'll find that their ModelMetaclass is decorated with __dataclass_transform__
Question: I have a python project with all dependencies and versions managed by pyproject.toml file. One of these dependencies is a git reference: [project] name = 'my_package' version = '1.0.0' dependencies = [ 'my_dependency @ git+https://github.com/some_user/some_repo.git' ]  In order to improve version management after some time I started to use tags to specify exact "version". Like this: dependencies = [ 'my_dependency @ git+https://github.com/some_user/some_repo.git@v.1.2.0' ]  But this is still not enough. Ideally I want something flexible, open for minor or patch version increase. Like this: dependencies = [ 'my_dependency >= 1.2 @ git+https://github.com/some_user/some_repo.git' ]  To be clear: I want pip to look for different versions in the entire repo history and take one's that match the condition. According to semver. In this particular case with >= 1.2 it should take 1.2.1, 1.2.2, 1.3, 1.157.256 or any other commit with version in pyproject.toml (or at least git tag) greater or equal to 1.2. Is this possible? Can pip manage versions so well for git repositories?

Answer: Unfortunately it's not possible. As specified here https://peps.python.org/pep-0508/ or here https://pip.pypa.io/en/stable/reference/requirement-specifiers/, you can't use of version requirements with url based dependencies. Your second approach about using tags is the one you need.
Question: New to FastAPI  Getting a "value is not a valid list (type=type_error.list)" error Whenever I try to return {"posts": post} @router.get('', response_model = List[schemas.PostResponseSchema]) def get_posts(db : Session = Depends(get_db)): print(limit) posts = db.query(models.Post).all() return {"posts" : posts}  Although it works if I return posts like this: return posts  Here is my response model: class PostResponseSchema(PostBase): user_id: int id: str created_at : datetime user : UserResponseSchema class Config: orm_mode = True  And Model: class Post(Base): __tablename__ = "posts" id = Column(Integer, primary_key=True, nullable=False) title = Column(String, nullable=False) content = Column(String, nullable = False) published = Column(Boolean, server_default = 'TRUE' , nullable = False) created_at = Column(TIMESTAMP(timezone=True), nullable = False, server_default = text('now()')) user_id = Column(Integer, ForeignKey("users.id", ondelete = "CASCADE"), nullable = False ) user = relationship("User")  what am I missing here?

Answer: your response expects to be the list by this line of code of yours: @router.get('', response_model = List[schemas.PostResponseSchema])  but your response return {"posts" : posts} is object. so you have to return posts because it is a list of objects as your response expects. otherwise, if you want to use return {"posts": posts} just change router.get('', response_model = List[schemas.PostResponseSchema]) to router.get('') then you will get something like this: {"posts": [......]}  inside [] will be a list of posts.
Question: I have an ordered list of entities, numbered in a broken sequence: [1, 2, 3, 6, 7, 11, 17, 18, 19]  I'd like to break the list where there's a gap, and collect the results in a new list: [[1, 2, 3], [6, 7], [11], [17, 18, 19]]  I have the feeling there's a name for what I want to do and probably a nice library function for it - but I can't think of it. Can anyone shine some light before I possibly reinvent a wheel?  edit: Thanks, folks, but I was asking if there's a name for this operation and an existing algorithm, not for implementations - this is what I came up with: def group_adjoining(elements, key=lambda x: x): """Returns list of lists of contiguous elements :key: function to get key integer from list element """ if not elements: return elements result = [[elements[0]]] for a, b in zip(elements, elements[1:]): if key(a) + 1 == key(b): result[-1].append(b) else: result.append([b]) return result 

Answer: Plain itertools.groupby approach: from itertools import groupby lst = [1, 2, 3, 6, 7, 11, 17, 18, 19] out = [] for _, g in groupby(enumerate(lst), lambda x: x[0] - x[1]): out.append([v for _, v in g]) print(out)  Prints: [[1, 2, 3], [6, 7], [11], [17, 18, 19]]
Question: I'm following a tutorial for creating a Flask app with Flask-SQLAlchemy. However, it has started raising an error when creating the database. How do I create the database? from flask import Flask from flask_sqlalchemy import SQLAlchemy db = SQLAlchemy() def create_app(): app = Flask(__name__) app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///project.db" db.init_app(app) from . import models create_database(app) return app def create_database(app): if not path.exists("website/project.db"): db.create_all(app=app) print("created database")  The line db.create_all(app=app) gives me this error: SQLAlchemy.create_all() got an unexpected keyword argument 'app' 

Answer: Flask-SQLAlchemy 3 no longer accepts an app argument to methods like create_all. Instead, it always requires an active Flask application context. db = SQLAlchemy() def create_app(): app = Flask(__name__) app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///project.db" db.init_app(app) from . import models with app.app_context(): db.create_all() return app  There is no need for that create_database function. SQLAlchemy will already not overwrite an existing file, and the only time the database wouldn't be created is if it raised an error.
Question: I would like to programmatically determine if a particular Python script is run a testing environment such as  GitHub action Travis CI Circle CI  etc. I realize that this will require some heuristics, but that's good enough for me. Are certain environment variables always set? Is the user name always the same? Etc.

Answer: An environment variable is generally set for each CI/CD pipeline tool. The ones I know about: os.getenv("GITHUB_ACTIONS") os.getenv("TRAVIS") os.getenv("CIRCLECI") os.getenv("GITLAB_CI")  Will return true in a python script when executed in the respective tool environment. e.g:  os.getenv("GITHUB_ACTIONS") == "true" in a Github Action workflow. os.getenv("CIRCLECI") == "true" in a CircleCI pipeline. ...  PS: If I'm not mistaken, to identify the python script is being executed in Jenkins or Kubernetes Service host, the behavior isn't the same.
Question: Given a certain table of type     A B C     t r 1   t r 1   n j 2   n j 2   n j 2     I would like to group on A and B and only take the number of rows specified by C So the desired output would be     A B C     t r 1   n j 2   n j 2     I am trying to achieve that through this function but with no luck df.groupby(['A', 'B']).agg(lambda x: x.head(df.C)) 

Answer: You can use groupby.cumcount and boolean indexing: out = df[df['C'].gt(df.groupby(['A', 'B']).cumcount())]  Or with a classical groupby.apply: (df.groupby(['A', 'B'], sort=False, as_index=False, group_keys=False) .apply(lambda g: g.head(g['C'].iloc[0])) )  output:  A B C 0 t r 1 2 n j 2 3 n j 2  Intermediates for the groupby.cumcount approach:  A B C cumcount C > cumcount 0 t r 1 0 True 1 t r 1 1 False 2 n j 2 0 True 3 n j 2 1 True 4 n j 2 2 False
Question: I went through different stackoverflow questions and finally posting it because I couldnt solve one of the issues I am facing. I have a dataframe like below A B C group1 group1_c 12 group1 group1_c 12 group1 group1_c 12 group1 group1_c 1 group1 group1_c 12 group1 group1_c 12  I have to match two rows together and whenever the value matches, I cumsum it. To do this, df['cumul'] = df['C'].eq(df.groupby(['A','B'])['C'].shift(1).ffill()).groupby([df['A'],df['B']).cumsum()  Once I do this, A B C Cumul group1 group1_c 12 0 group1 group1_c 12 1 group1 group1_c 12 2 group1 group1_c 1 2 group1 group1_c 12 3 group1 group1_c 12 3  Whereas I want to reset if the condition is not met.Expected solution A B C Cumul group1 group1_c 12 0 group1 group1_c 12 1 group1 group1_c 12 2 group1 group1_c 1 0 group1 group1_c 12 0 group1 group1_c 12 1  Please advice Thank you

Answer: If need count groups per consecutive values of C column use Series.ne with Series.shift and cumulative sum, last use counter by GroupBy.cumcount: df['cumul'] = df.groupby(df['C'].ne(df['C'].shift()).cumsum()).cumcount() print (df) A B C cumul 0 group1 group1_c 12 0 1 group1 group1_c 12 1 2 group1 group1_c 12 2 3 group1 group1_c 1 0 4 group1 group1_c 12 0 5 group1 group1_c 12 1  If need per A, B groups also add both groups: print (df) A B C 0 group1 group1_c 12 1 group1 group2_c 12 <-changed groups 2 group1 group2_c 12 <-changed groups 3 group1 group1_c 1 4 group1 group1_c 12 5 group1 group1_c 12 s = df['C'].ne(df['C'].shift()).cumsum() df['cumul'] = df.groupby([df['A'],df['B'], s]).cumcount() df['cumul1'] = df.groupby(df['C'].ne(df['C'].shift()).cumsum()).cumcount() print (df) A B C cumul cumul1 0 group1 group1_c 12 0 0 1 group1 group2_c 12 0 1 2 group1 group2_c 12 1 2 3 group1 group1_c 1 0 0 4 group1 group1_c 12 0 0 5 group1 group1_c 12 1 1  Alternative solution: s = df[['A','B','C']].ne(df[['A','B','C']].shift()).any(axis=1).cumsum() df['cumul'] = df.groupby(s).cumcount()
Question: it seems impossible to set a regex constraint with a __root__ field like this one: class Cars(BaseModel): __root__: Dict[str, CarData]  so, i've resorted to doing it at the endpoint: @app.post("/cars") async def get_cars(cars: Cars = Body(...)): x = cars.json() y = json.loads(x) keys = list(y.keys()) try: if any([re.search(r'^\d+$', i) is None for i in keys]): raise ValidationError except ValidationError as ex: return 'wrong type' return 'works'  this works well in that i get wrong type returned if i dont use a digit in the request body. but i'd like to return something similar to what pydantic returns but with a custom message: { "detail": [ { "loc": [ "body", "__root__", ], "msg": "hey there, you can only use digits!", "type": "type_error.???" } ] } 

Answer: You can pass your own error string by using raise ValidationError("Wrong data type"). Hope it helps.
Question: I want to create something like a combo box with django model, but i don't find any field type to do that. something like this: enter image description here

Answer: Simply you can do this in models.py: class Student(models.Model): select_gender = ( ('Male', 'Male'), ('Female', 'Female'), ('Other', 'Other'), ) student_name = models.CharField(max_length=100) student_gender = models.CharField(max_length=8, choices=select_gender)  In forms.py file, do this: class StudentForm(forms.ModelForm): class Meta: model = Student fields = '__all__' widgets = { 'student_name' : forms.TextInput(attrs={'class':'form-control'}), 'student_gender' : forms.Select(attrs={'class':'form-control'}) }  This is the way you can do.
Question: I am trying to write json object into a particular column of my sql table variant_str = response.json() print(con.cursor().execute('INSERT INTO TEST_TABLE (JSON_STR) VALUES (?)', (variant_str,)))  Here, the variant_str is of type dict. I get an error that: snowflake.connector.errors.ProgrammingError: 252004: Failed processing pyformat-parameters; 255001: Binding data in type (dict) is not supported.  The table's ddl where I am trying to load the data is this: create or replace TABLE TEST_TABLE ( JSON_STR VARIANT ); 

Answer: It is possible to use ? as the placeholder for parameter binding: import snowflake.connector snowflake.connector.paramstyle='qmark'  Insert using INSERT INTO ... SELECT PARSE_JSON(...) con.cursor().execute('INSERT INTO TEST_TABLE (JSON_STR) SELECT PARSE_JSON(?)' , (variant_str))  where variant_str is a valid JSON string
Question: Wup, I'm trying to deploy a localhost web using Django but I get the following error: TemplateDoesNotExist at / templates/index.html Request Method: GET Request URL: http://127.0.0.1:8000/ Django Version: 4.1.2 Exception Type: TemplateDoesNotExist  I was looking tutorials and docs but I didn't found the answer. Here is my settings.py ALLOWED_HOSTS = ['*'] DEBUG = True ROOT_URLCONF = 'FortyTwop.urls' SECRET_KEY = 'This is a secret key' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': ["templates"], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, }, ]  also urls.py from django.urls import path from . import views urlpatterns = [ path('', views.index, name='index') ]  And views.py from email import message from http.client import HTTPResponse from django.shortcuts import render # Create your views here. def index(request): return render(request, 'index.html')  My path tree: . ├── FortyTwop │ ├── __init__.py │ ├── admin.py │ ├── apps.py │ ├── migrations │ │ └── __init__.py │ ├── models.py │ ├── templates │ │ └── index.html │ ├── tests.py │ ├── urls.py │ └── views.py └── manage.py  What Im doing bad or what I didn't have in the settings?, this project is just for test.

Answer: templates/ is the base path you've configured in the settings, so you'll want def index(request): return render(request, 'index.html')  Furthermore, TEMPLATE_DIRS hasn't been a thing since Django 1.8 or so, and you're on Django 4.x. See these instructions.
Question: E.g. if I have import polars as pl df = pl.DataFrame({'a': [1,2,3], 'b': [4,5,6]})  how would I find the cumulative sum of each row? Expected output:  a b 0 1 5 1 2 7 2 3 9  Here's the equivalent in pandas: >>> import pandas as pd >>> pd.DataFrame({'a': [1,2,3], 'b': [4,5,6]}).cumsum(axis=1) a b 0 1 5 1 2 7 2 3 9  but I can't figure out how to do it in polars

Answer: cum_sum_horizontal() generates a struct of cum_sum values. df.select(pl.cum_sum_horizontal(pl.all()))  shape: (3, 1) ┌───────────┐ │ cum_sum │ │ --- │ │ struct[2] │ ╞═══════════╡ │ {1,5} │ │ {2,7} │ │ {3,9} │ └───────────┘  Which you can unnest() df.select(pl.cum_sum_horizontal(pl.all())).unnest('cum_sum')  shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 5 │ │ 2 ┆ 7 │ │ 3 ┆ 9 │ └─────┴─────┘  The code for cum_sum_horizontal is here. As it stands, it just calls cum_fold() df.select(pl.cum_fold(pl.lit(0, pl.UInt32), lambda x, y: x + y, pl.all()))  shape: (3, 1) ┌───────────┐ │ cum_fold │ │ --- │ │ struct[2] │ ╞═══════════╡ │ {1,5} │ │ {2,7} │ │ {3,9} │ └───────────┘
Question: I am wondering if there's a way to do forward filling by columns in polars. df = pl.DataFrame( { "id": ["NY", "TK", "FD"], "eat2000": [1, 6, 3], "eat2001": [-2, None, 4], "eat2002": [None, None, None], "eat2003": [-9, 3, 8], "eat2004": [None, None, 8] } ); df  shape: (3, 6) ┌─────┬─────────┬─────────┬─────────┬─────────┬─────────┐ │ id ┆ eat2000 ┆ eat2001 ┆ eat2002 ┆ eat2003 ┆ eat2004 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ f64 ┆ i64 ┆ i64 │ ╞═════╪═════════╪═════════╪═════════╪═════════╪═════════╡ │ NY ┆ 1 ┆ -2 ┆ null ┆ -9 ┆ null │ │ TK ┆ 6 ┆ null ┆ null ┆ 3 ┆ null │ │ FD ┆ 3 ┆ 4 ┆ null ┆ 8 ┆ 8 │ └─────┴─────────┴─────────┴─────────┴─────────┴─────────┘  I would like to do the equivlanet of .ffill(axis=1) in pandas. pl.from_pandas(df.to_pandas().ffill(axis=1))  shape: (3, 6) ┌─────┬─────────┬─────────┬─────────┬─────────┬─────────┐ │ id ┆ eat2000 ┆ eat2001 ┆ eat2002 ┆ eat2003 ┆ eat2004 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ f64 ┆ f64 ┆ i64 ┆ f64 │ ╞═════╪═════════╪═════════╪═════════╪═════════╪═════════╡ │ NY ┆ 1 ┆ -2.0 ┆ -2.0 ┆ -9 ┆ -9.0 │ │ TK ┆ 6 ┆ 6.0 ┆ 6.0 ┆ 3 ┆ 3.0 │ │ FD ┆ 3 ┆ 4.0 ┆ 4.0 ┆ 8 ┆ 8.0 │ └─────┴─────────┴─────────┴─────────┴─────────┴─────────┘ 

Answer: You can use the new coalesce Expression to fold columns horizontally. If you place the coalesce expressions in a with_columns context, they will be run in parallel. ( df .with_columns(pl.col("^eat.*$").cast(pl.Int64)) .with_columns( pl.coalesce("eat2004", "eat2003", "eat2002", "eat2001", "eat2000"), pl.coalesce("eat2003", "eat2002", "eat2001", "eat2000"), pl.coalesce("eat2002", "eat2001", "eat2000"), pl.coalesce("eat2001", "eat2000"), ) )  shape: (3, 6) ┌─────┬─────────┬─────────┬─────────┬─────────┬─────────┐ │ id ┆ eat2000 ┆ eat2001 ┆ eat2002 ┆ eat2003 ┆ eat2004 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════════╪═════════╪═════════╪═════════╪═════════╡ │ NY ┆ 1 ┆ -2 ┆ -2 ┆ -9 ┆ -9 │ │ TK ┆ 6 ┆ 6 ┆ 6 ┆ 3 ┆ 3 │ │ FD ┆ 3 ┆ 4 ┆ 4 ┆ 8 ┆ 8 │ └─────┴─────────┴─────────┴─────────┴─────────┴─────────┘  Couple of notes. I first cast the eatXXXX columns to the same type. (In the DataFrame constructor, eat2002 is of type Float64 because of the way Polars initializes an all-null column that is not supplied with an explicit datatype). I've written out the list of coalesce Expressions for demonstration, but the list of expressions can be generated with a Python list comprehension. eat_cols = [col_nm for col_nm in reversed(df.columns) if col_nm.startswith('eat')] ( df .with_columns(pl.col("^eat.*$").cast(pl.Int64)) .with_columns( pl.coalesce(eat_cols[idx:]) for idx in range(0, len(eat_cols) - 1) ) )
Question: When using __qualname__ as part of the python logger formatter for classes, I am getting mypy error 'Name "qualname" not defined'. I can suppress it with inline # type: ignore, but wonder if there are more proper ways to do this, or if there are ways to have mypy recognize __qualname__.

Answer: The issue linked by @SilvioMayolo and its duplicate were closed in October 2023 as completed. __qualname__ (and __module__, for that matter) can now be used as-is in class bodies: (playground) class C: foo: ClassVar[str] = __qualname__ + 'bar'  This has been possible since 1.9.0.
Question: Just like many other people I run into the problem of an instance not being bound to a Session. I have read the SQLAlchemy docs and the top-10 questions here on SO. Unfortunatly I have not found an explanation or solution to my error. My guess is that the commit() closes the session rendering the object unbound. Does this mean that In need to create two objects, one to use with SQL Alchemy and one to use with the rest of my code? I create an object something like this: class Mod(Base): __tablename__ = 'mod' insert_timestamp = Column(DateTime, default=datetime.datetime.now()) name = Column(String, nullable=False)  Then I add it to the database using this function and afterwards the object is usesless, I cannot do anything with it anymore, always getting the error that it is not bound to a session. I have tried to keep the session open, keep it closed, copy the object, open two sessions, return the object, return a copy of the object. def add(self, dataobjects: list[Base]) -> bool: s = self.sessionmaker() try: s.add_all(dataobjects) except TypeError: s.rollback() raise else: s.commit() return True  This is my session setup:  self.engine = create_engine(f"sqlite:///{self.config.database['file']}") self.sessionmaker = sessionmaker(bind=self.engine) Base.metadata.bind = self.engine  My last resort would be to create every object twice, once for SQL Alchemy and once so I can actually use the object in my code. This defeats the purpose of SQL Alchemy for me.

Answer: ORM entities are expired when committed. By default, if an entity attribute is accessed after expiry, the session emits a SELECT to get the current value from the database. In this case, the session only exists in the scope of the add method, so subsequent attribute access raises the "not bound to a session" error. There are a few ways to approach this:  Pass expire_on_commit=False when creating the session. This will prevent automatic expiry so attribute values will remain accessible after leaving the add function, but they may be stale. Create the session outside of the add function and pass it as an argument (or return it from add, though that's rather ugly). As long as the session is not garbage collected the entities remain bound to it. Create a new session and do mod = session.merge(mod) for each entity to bind it to the new session.  Which option you choose depends on your application.
Question: Is there a clean way to resolve environment variables or %VARIABLES% in general purely with the Path class without having to use a fallback solution like os.path.expandvars or "my/path/%variable%".fortmat(**os.environ)? The question How to use win environment variable "pathlib" to save files? doesn't provide an answer. It's exactly the opposite. It suggests using os.path.expandvars.

Answer: This comment from an answer of the linked thread clearly states that it does not provide anything equivalent to os.path.expandvars. Going through the documentation for the module (as of Python 3.10) does not make any references to environment variables. This issue on the CPython issue tracker was closed under the same rationale - various comments noted that any method along "expandvars() works with string, not with path" instances. So in short, no.