eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: this is my database schema.  I defined my Schema like this: from pydantic import BaseModel class Userattribute(BaseModel): name: str value: str user_id: str id: str  This is my model: class Userattribute(Base): __tablename__ = "user_attribute" name = Column(String) value = Column(String) user_id = Column(String) id = Column(String, primary_key=True, index=True)  In a crud.py I define a get_attributes method. def get_attributes(db: Session, skip: int = 0, limit: int = 100): return db.query(models.Userattribute).offset(skip).limit(limit).all()  This is my GET endpoint: @app.get("/attributes/", response_model=List[schemas.Userattribute]) def read_attributes(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)): users = crud.get_attributes(db, skip=skip, limit=limit) print(users) return users  The connection to the database seems to work, but a problem is the datatype: pydantic.error_wrappers.ValidationError: 7 validation errors for Userattribute response -> 0 value is not a valid dict (type=type_error.dict) response -> 1 value is not a valid dict (type=type_error.dict) response -> 2 value is not a valid dict (type=type_error.dict) response -> 3 value is not a valid dict (type=type_error.dict) response -> 4 value is not a valid dict (type=type_error.dict) response -> 5 value is not a valid dict (type=type_error.dict) response -> 6 value is not a valid dict (type=type_error.dict)  Why does FASTApi expect a dictionary here? I don´t really understand it, since I am not able to even print the response. How can I fix this?

Answer: Pydantic 2 changed how models gets configured, so if you're using the most recent version of Pydantic, see the section named Pydantic 2 below. SQLAlchemy does not return a dictionary, which is what pydantic expects by default. You can configure your model to also support loading from standard orm parameters (i.e. attributes on the object instead of dictionary lookups): class Userattribute(BaseModel): name: str value: str user_id: str id: str class Config: orm_mode = True  You can also attach a debugger right before the call to return to see what's being returned. Since this answer has become slightly popular, I'd like to also mention that you can make orm_mode = True the default for your schema classes by having a common parent class that inherits from BaseModel: class OurBaseModel(BaseModel): class Config: orm_mode = True class Userattribute(OurBaseModel): name: str value: str user_id: str id: str  This is useful if you want to support orm_mode for most of your classes (and for those where you don't, inherit from the regular BaseModel). Pydantic 2 Pydantic 2 has replaced the internal Config class with a model_config field: from pydantic import ConfigDict class OurBaseModel(BaseModel): model_config = ConfigDict(from_attributes=True)  This works in the same way as the old orm_mode.
Question: I'd like to convert a OmegaConf/Hydra config to a nested dictionary/list. How can I do this?

Answer: See OmegaConf.to_container(). Usage snippet: >>> conf = OmegaConf.create({"foo": "bar", "foo2": "${foo}"}) >>> assert type(conf) == DictConfig >>> primitive = OmegaConf.to_container(conf) >>> show(primitive) type: dict, value: {'foo': 'bar', 'foo2': '${foo}'} >>> resolved = OmegaConf.to_container(conf, resolve=True) >>> show(resolved) type: dict, value: {'foo': 'bar', 'foo2': 'bar'}
Question: I'm using async_engine. When I try to execute anything: async with self.async_engine.connect() as con: query = "SELECT id, name FROM item LIMIT 50;" result = await con.execute(f"{query}")  I'm getting: Exception has occurred: ObjectNotExecutableError Not an executable object: 'SELECT id, name FROM item LIMIT 50;'   This question was asked before by user @stilmaniac but it is now deleted from SO. I found it in Google Search cache, here is copy. I have the same issue so I'm reasking it, but the original version is below:  I'm trying to create tables from metadata as follows: Base = declarative_base() properties = Table( 'properties', Base.metadata, # ... Column('geolocation', Geography(geometry_type='POINT', srid=4326)), # ... ) engine = create_async_engine("postgresql+asyncpg://user:password@postgres/") async with engine.begin() as conn: await conn.run_sync(Base.metadata.create_all)  Gives me the following error: sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'CREATE INDEX "idx_properties_geolocation" ON "properties" USING GIST ("geolocation")'  Considering this doc Versions:  OS: macOS 11.4 ARM SQLAlchemy: 1.4.22 Python: 3.6 

Answer: As the exception message suggests, the str 'SELECT id, name FROM item LIMIT 50;' is not an executable object. To make it executable, wrap it with sqlalchemy.text. from sqlalchemy import text async with self.async_engine.connect() as con: query = "SELECT id, name FROM item LIMIT 50;" result = await con.execute(text(query))  async.connection.execute requires that its statement argument  [...] is always an object that is in both the ClauseElement and Executable hierarchies, including:  Select  Insert, Update, Delete  TextClause and TextualSelect  DDL and objects which inherit from DDLElement    The synchronous connection.execute method permits raw strings, but this is deprecated in v1.4 and has been removed in SQLAlchemy 2.0.
Question: We have a FastApi app and using httpx AsyncClient for testing purposes. We are experiencing a problem where the unit tests run locally fine but fail on the CI server (Github Actions). After further research we have come across this proposed solution by setting raise_server_exceptions=False to False. client = TestClient(app, raise_server_exceptions=False)  However this is for the sync client. We are using the async client. @pytest.fixture async def client(test_app): async with AsyncClient(app=test_app, base_url="http://testserver") as client: yield client  The AsyncClient does not support the raise_app_exceptions=False option. Does anyone have experience with this? Thanks

Answer: The problem is caused by FastApi version. You can use fastapi==0.65.0 and even without the ASGITransport object and the raise_app_exceptions=False flag you will be able to run the tests which are checking for custom exception raising. Also the fastapi version should be frozen in the requirements file. You can read more here
Question: I am trying to read some data which may sometimes have erroneous and bad rows, so as always I passed error_bad_lines=False but the console keeps throwing the deprecation warning on every run. Why is this feature deprecated and is there any other alternative for skipping bad lines?

Answer: Read the documentation:  Deprecated since version 1.3.0: The on_bad_lines parameter should be used instead to specify behavior upon encountering a bad line instead.  So, replace: df = pd.read_csv(..., error_bad_lines=False)  with: df = pd.read_csv(..., on_bad_lines='skip')
Question: I'm trying to Create New conda environment by 'Anaconda Prompt' usnig yml File in Windows 10. So here is the steps i made through: 1. using cd command i changed the directory to dir which my yml file located. (suppose my yml file is in c:/Users/<USER NAME>/.jupyter ) 2. Then i used conda env create -f Python 310.yml command to create new conda env. and what i got is:  SpecNotFound: Invalid name, try the format: user/package  Now I don't know how can I solve this problem and exactly what is the meaning of this error.  Appendix my Python 310.yml file contains these stuff: 

Answer: issue solved by changing contents of Python 310.yml and renaming yml file to Python310.yml. Here is the final .yml file content: name: Python3.9 channels: - defaults dependencies: - numpy - pandas - matplotlib - pip - python=3.9.* - python-dateutil - pytz - scikit-learn - scipy - statsmodels - xlrd - openpyxl - lxml - html5lib - beautifulsoup4 - jupyter - pip: - pmdarima - tensorflow - keras prefix: C:\Users\Shayan\Anaconda3\envs\Python3.9
Question: I was creating a long list of strings like this: tlds = [ 'com', 'net', 'org' 'edu', 'gov', ... ]  I missed a comma after 'org'. Python automatically concatenated it with the string in the next line, into 'orgedu'. This became a bug very hard to identify. There are already many ways to define multi-line strings, some very explicit. So I wonder is there a way to disable this particular behavior?

Answer: The right Platonic thing to do is to modify the linter. But I think life is too short to do so, in addition to the fact that if the next coder does not know about your modified linter, his/her life would be a living hell. There should not be shame in ensuring that the input, even if hardcoded, is valid. If it was for me, I would implement a manual workaround like so: tlds = [ 'com', 'net', 'org' 'edu', 'gov', ] redone = ''.join(tlds) chunk_size = 3 tlds = [ redone[i:i+chunk_size] for i in range(0, len(redone), chunk_size) ] # Now you have a nice `tlds` print(tlds)  You can forget commas, write two elements on the same line, or even in the same string all you want. You can invite unwary code collabs to mess it too, the text will be redone in threes (chunk_size) later on anyways if that is OK with your application. EDIT: Later to @Jasmijn 's note, I think there is an alternative approach if we have a dynamic size of entries we can use the literal input like this: tlds = ['''com net org edu gov nl co.uk'''] # This way every line is an entry by its own as seen directly without any quotations or decorations except for the first and last inputs. tlds = '\n'.split(tlds)
Question: I'm trying to start up this Project on my Mac https://github.com/realsuayip/django-sozluk It works on my Windows machine, but I got this Error on my Mac:  unexpected character "." in variable name near "127.0.0.1 192.168.2.253\nDJANGO_SETTINGS_MODULE=djdict.settings_prod\n\n\nSQL_ENGINE=django.db.backends.postgresql\nSQL_PORT=5432\nDATABASE=postgres\nSQL_HOST=db\n\nSQL_DATABASE=db_dictionary\nSQL_USER=db_dictionary_user\nSQL_PASSWORD=db_dictionary_password\n\n\nEMAIL_HOST=eh\nEMAIL_PORT=587\nEMAIL_HOST_USER=eh_usr\nEMAIL_HOST_PASSWORD=pw" furkan@MacBook-Air-von-Furkan gs %  Any help would be much appreciated!

Answer: (Sorry about the answer - I don't yet have the rep to comment) Just want to add a note on to the answer by D.Mo - I had the same error this morning, and adding quotes around the values in my .env file did seem to resolve the issue. Though I then noticed that in the documentation for these env files, Docker mentions  There is no special handling of quotation marks. This means that they are part of the VAL.  Just wanted to point that out in case anyone ran into issues with this. I will likely just keep this change locally until others in my team experience the same problem - unless someone can confirm that the ENV values should now have ""s around them. FWIW I could not find a way to disable Docker Compose V2 (I'm on Arch, Docker v20.10.9) - docker-compose disable-v2 isn't a valid command for me (see here for what I assume is the (imo silly) reason).  Edit - I ended up reverting to a previous version of docker-compose as the constant workarounds I had to implement weren't fun. I did so after seeing this - I ended up looking around yay -U /var/cache/pacman/pkg/docker-compose- and tab-complete gave me a list of cached versions. I went with 1.29.2-1 and things have been working perfectly again since then. Will just have to see what happens in future updates etc.
Question: I am training 2 autoencoders with 2 separate input paths jointly and I would like to randomly set one of the input paths to zero. I use tensorflow with keras backend (functional API). I am computing a joint loss (sum of two losses) for backpropagation. A -> A' & B ->B' loss => l2(A,A')+l2(B,B') networks taking A and B are connected in latent space. I would like to randomly set A or B to zero and compute the loss only on the corresponding path, meaning if input path A is set to zero loss be computed only by using outputs of only path B and vice versa; e.g.: 0 -> A' & B ->B' loss: l2(B,B') How do I randomly set input path to zero? How do I write a callback which does this?

Answer: Maybe try the following: import random def decision(probability): return random.random() < probability  Define a method that makes a random decision based on a certain probability x and make your loss calculation depend on this decision. if current_epoch == random.choice(epochs): keep_mask = tf.ones_like(A.input, dtype=float32) throw_mask = tf.zeros_like(A.input, dtype=float32) if decision(probability=0.5): total_loss = tf.reduce_sum(reconstruction_loss_a * keep_mask + reconstruction_loss_b * throw_mask) else: total_loss = tf.reduce_sum(reconstruction_loss_a * throw_mask + reconstruction_loss_b * keep_mask) else: total_loss = tf.reduce_sum(reconstruction_loss_a + reconstruction_loss_b)  I assume that you do not want to set one of the paths to zero every time you update your model parameters, as then there is a risk that one or even both models will not be sufficiently trained. Also note that I use the input of A to create zero_like and one_like tensors as I assume that both inputs have the same shape; if this is not the case, it can easily be adjusted. Depending on what your goal is, you may also consider replacing your input of A or B with a random tensor e.g. tf.random.normal based on a random decision. This creates noise in your model, which may be desirable, as your model would be forced to look into the latent space to try reconstruct your original input. This means precisely that you still calculate your reconstruction loss with A.input and A.output, but in reality your model never received the A.input, but rather the random tensor. Note that this answer serves as a simple conceptual example. A working example with Tensorflow can be found here.
Question: I'm trying to move data from SQL to Mongo. Here is a challenge I'm facing, if any child object is empty I want to remove parent element. I want till insurance field to be removed. Here is what I tried: def remove_empty_elements(jsonData): if(isinstance(jsonData, list) or isinstance(jsonData,dict)): for elem in list(jsonData): if not isinstance(elem, dict) and isinstance(jsonData[elem], list) and elem: jsonData[elem] = [x for x in jsonData[elem] if x] if(len(jsonData[elem])==0): del jsonData[elem] elif not isinstance(elem, dict) and isinstance(jsonData[elem], dict) and not jsonData[elem]: del jsonData[elem] else: pass return jsonData  sample data { "_id": "30546c62-8ea0-4f1a-a239-cc7508041a7b", "IsActive": "True", "name": "Pixel 3", "phone": [ { "Bill": 145, "phonetype": "xyz", "insurance": [ { "year_one_claims": [ { "2020": 200 }, { }, { }, { }, { } ] }, { "year_two_claims": [ { }, { }, { }, { }, { } ] }, ] } ], "Provider": { "agent": "aaadd", } }  Results should look like that  { "_id": "30546c62-8ea0-4f1a-a239-cc7508041a7b", "IsActive": "True", "name": "Pixel 3", "phone": [ { "Bill": 145, "phonetype": "xyz", "insurance": [ { "year_one_claims": [ { "2020": 200 }, ] }, ] } ], "Provider": { "agent": "aaadd", } } 

Answer: Your if statements are kind of confusing. I think you are looking for a recursion: import json # define which elements you want to remove: to_be_deleted = [[], {}, "", None] def remove_empty_elements(jsonData): if isinstance(jsonData, list): jsonData = [new_elem for elem in jsonData if (new_elem := remove_empty_elements(elem)) not in to_be_deleted] elif isinstance(jsonData,dict): jsonData = {key: new_value for key, value in jsonData.items() if (new_value := remove_empty_elements(value)) not in to_be_deleted} return jsonData print(json.dumps(remove_empty_elements(jsonData), indent=4))  Edit/Note: from Python3.8 you can use assignements (:=) in comprehensions Output: { "_id": "30546c62-8ea0-4f1a-a239-cc7508041a7b", "IsActive": "True", "name": "Pixel 3", "phone": [ { "Bill": 145, "phonetype": "xyz", "insurance": [ { "year_one_claims": [ { "2020": 200 } ] } ] } ], "Provider": { "agent": "aaadd" } }
Question:  We have a pyramid web application. We use SQLAlchemy@1.4 with Zope transactions.  In our application, it is possible for an error to occur during flush as described here which causes any subsequent usage of the SQLAlchemy session to throw a PendingRollbackError. The error which occurs during a flush is unintentional (a bug), and is raised to our exception handling view... which tries to use data from the SQLAlchemy session, which then throws a PendingRollbackError. Is it possible to "recover" from a PendingRollbackError if you have not framed your transaction management correctly? The SQLAclhemy documentation says to avoid this situation you essentially "just need to do things the right way". Unfortunately, this is a large codebase, and developers don't always follow correct transaction management. This issue is also complicated if savepoints/nested transactions are used. def some_view(): # constraint violation session.add_all([Foo(id=1), Foo(id=1)]) session.commit() # Error is raised during flush return {'data': 'some data'} def exception_handling_view(): # Wired in via pyramid framework, error ^ enters here. session.query(... does a query to get some data) # This throws a `PendingRollbackError`  I am wondering if we can do something like the below, but don't understand pyramid + SQLAlchemy + Zope transactions well enough to know the implications (when considering the potential for nested transactions etc). def exception_handling_view(): # Wired in via pyramid framework, error ^ enters here. def _query(): session.query(... does a query to get some data) try: _query() except PendingRollbackError: session.rollback() _query() 

Answer: Instead of trying to execute your query, just try to get the connection: def exception_handling_view(): try: _ = session.connection() except PendingRollbackError: session.rollback() session.query(...)  session.rollback() only rolls back the innermost transaction, as is usually expected — assuming nested transactions are used intentionally via the explicit session.begin_nested(). You don't have to rollback parent transactions, but if you decide to do that, you can: while session.registry().in_transaction(): session.rollback()
Question: I'm using opencv-python and when I execute the following code: index = 0 cap = cv2.VideoCapture(video_path) while True: offset = cap.get(cv2.CAP_PROP_POS_MSEC) print(cv2.__version__, index, offset) ok, frame = cap.read() if not ok: break index += 1  I get the following output: 3.4.7 0 0.0 3.4.7 1 33.36666666666667 3.4.7 2 66.73333333333333 3.4.7 3 100.10000000000001 3.4.7 4 133.46666666666667  If I execute this code on version 3.4.8.29, I get the following output: 3.4.8 0 0.0 3.4.8 1 0.0 3.4.8 2 33.36666666666667 3.4.8 3 66.73333333333333 3.4.8 4 100.10000000000001  And if I execute it on version 4.5.2.52 I get: 4.5.2 0 0.0 4.5.2 1 0.0 4.5.2 2 0.0 4.5.2 3 0.0 4.5.2 4 0.0  The question is first of all, which one is the correct one? It seems like 3.4.7 is correct, but it also seems to be changing randomly between versions. And also how can I modify the other versions to get the proper result, same as 3.4.7

Answer: I read the OpenCV docs and they said: "Reading / writing properties involves many layers. Some unexpected result might happens along this chain. Effective behaviour depends from device hardware, driver and API Backend." (source: https://docs.opencv.org/3.4.15/d4/d15/group__videoio__flags__base.html#gaeb8dd9c89c10a5c63c139bf7c4f5704d) So in other words OpenCV does not guaranty consistent and reliable behavior of this function. Also I installed openCV 4.5.2.52 and applied your script to one of my '.mp4' videos. Then I got the same result as you had for openCV version 3.4.8.29. So I think the behaviour you experience is not a 'bug', but rather the unreliable behavior of this function. As work around you can compute the "offset" by dividing the frame number by the FPS count (see code below). Then you have more control over the behaviour and maybe more consisted results. index = 0 cap = cv2.VideoCapture(video_path) fps = cap.get(cv2.CAP_PROP_FPS) while True: offset = cap.get(cv2.CAP_PROP_POS_MSEC) ok, frame = cap.read() if not ok: break # CAP_PROP_POS_MSEC print("CAP_PROP_POS_MSEC: ", index, offset) # Devide fps by frame number offset = cap.get(cv2.CAP_PROP_POS_FRAMES) / fps * 1000 print("cv2.CAP_PROP_POS_FRAMES", index, offset) index += 1
Question: Goal I would like to compute the 3D volume integral of a numeric scalar field. Code For this post, I will use an example of which the integral can be exactly computed. I have therefore chosen the following function:  In Python, I define the function, and a set of points in 3D, and then generate the discrete values at these points: import numpy as np # Make data. def function(x, y, z): return x**y**z N = 5 grid = np.meshgrid( np.linspace(0, 1, N), np.linspace(0, 1, N), np.linspace(0, 1, N) ) points = np.vstack(list(map(np.ravel, grid))).T x = points[:, 0] y = points[:, 1] z = points[:, 2] values = [function(points[i, 0], points[i, 1], points[i, 2]) for i in range(len(points))]  Question How can I find the integral, if I don't know the underlying function, i.e. if I only have the coordinates (x, y, z) and the values?

Answer: A nice way to go about this would be using scipy's tplquad integration. However, to use that, we need a function and not a cloud point. An easy way around that is to use an interpolator, to get a function approximating our cloud point - we can for example use scipy's RegularGridInterpolator if the data is on a regular grid: import numpy as np from scipy import integrate from scipy.interpolate import RegularGridInterpolator # Make data. def function(x,y,z): return x*y*z N = 5 xmin, xmax = 0, 1 ymin, ymax = 0, 1 zmin, zmax = 0, 1 x = np.linspace(xmin, xmax, N) y = np.linspace(ymin, ymax, N) z = np.linspace(zmin, zmax, N) values = function(*np.meshgrid(x,y,z, indexing='ij')) # Interpolate: function_interpolated = RegularGridInterpolator((x, y, z), values) # tplquad integrates func(z,y,x) f = lambda z,y,x : my_interpolating_function([z,y,x]) result, error = integrate.tplquad(f, xmin, xmax, lambda _: ymin, lambda _:ymax,lambda *_: zmin, lambda *_: zmax)  In the example above, we get result = 0.12499999999999999 - close enough!
Question: Consider: course_db = Course(title='Databases') course_db.save()  Coming from a C++ background, I would expect (course_db = Course(title='Databases')) to behave like it would in C++, that is, assign Course(title='Databases') to course_db and return the assigned object so that I can use it as part of a larger expression. For example, I would expect the following code to do the same thing as the code above: (course_db = Course(title='Databases')).save()  This assumption got support from some quick Google searches using terms like "assignment operator in Python", e.g. this article. But when I tried this, I got a syntax error. Why can't I do this in Python, and what can I do instead?

Answer: You should do some more research about the differences between statements and expressions in Python. If you are using Python 3.8+, you can use the := operator: In [1]: class A: ...: def save(self): ...: return 1 ...: In [2]: (a := A()).save() Out[2]: 1 In [3]: a Out[3]: <__main__.A at 0x7f074e2ddaf0>