eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to use mypy with SQLAlchemy. In order to validate/modify specific column value (email in this case), SQLAlchemy official document provides hybrid_property decorator. The problem is, mypy doesn't recognize EmailAddress class constructor properly, it gives: email_address.py:31: error: Unexpected keyword argument "email" for "EmailAddress"; did you mean "_email"?  How can I tell mypy to recognize these columns? from typing import TYPE_CHECKING from sqlalchemy import Column, Integer, String from sqlalchemy.ext.declarative import declarative_base # I don't even like the following statements just for setter if TYPE_CHECKING: hybrid_property = property else: from sqlalchemy.ext.hybrid import hybrid_property Base = declarative_base() class EmailAddress(Base): __tablename__ = "email_address" id = Column(Integer, primary_key=True) _email = Column("email", String) @hybrid_property def email(self): return self._email @email.setter def email(self, email): self._email = email EmailAddress(email="foobar@example.com") # email_address.py:31: error: Unexpected keyword argument "email" for "EmailAddress"; did you mean "_email"?  I'm using following packages: SQLAlchemy==1.4.35 mypy==0.942 mypy-extensions==0.4.3 sqlalchemy2-stubs==0.0.2a22 

Answer: OK, it seems like I finally found a way to solve the problem. This reminds me of uncooperative behaviors between dataclass/property decorators discussed in here. I end up with splitting EmailAddress class into 2:  Use @dataclass decorator on base class in order to indicate constructor options. Override email property so that mypy doesn't complain redef.  from dataclasses import dataclass from typing import TYPE_CHECKING, Optional from sqlalchemy import Column, Integer, String, Table from sqlalchemy.orm import registry mapper_registry: registry = registry() # I don't even like the following statements just for setter if TYPE_CHECKING: hybrid_property = property else: from sqlalchemy.ext.hybrid import hybrid_property @dataclass @mapper_registry.mapped class EmailAddressBase: __tablename__ = "email address" id: int = Column(Integer, primary_key=True) email: Optional[str] = None class EmailAddress(EmailAddressBase): _email = Column("email", String) @hybrid_property def email(self): return self._email @email.setter def email(self, email): self._email = email email = EmailAddress(email="foobar@example.com") print(email.email)
Question: I'm trying to use the code from the Teachable Machine website: from keras.models import load_model from PIL import Image, ImageOps import numpy as np # Load the model model = load_model('keras_model.h5') # Create the array of the right shape to feed into the keras model # The 'length' or number of images you can put into the array is # determined by the first position in the shape tuple, in this case 1. data = np.ndarray(shape=(1, 224, 224, 3), dtype=np.float32) # Replace this with the path to your image image = Image.open('<IMAGE_PATH>') #resize the image to a 224x224 with the same strategy as in TM2: #resizing the image to be at least 224x224 and then cropping from the center size = (224, 224) image = ImageOps.fit(image, size, Image.ANTIALIAS) #turn the image into a numpy array image_array = np.asarray(image) # Normalize the image normalized_image_array = (image_array.astype(np.float32) / 127.0) - 1 # Load the image into the array data[0] = normalized_image_array # run the inference prediction = model.predict(data) print(prediction)  but when running the code, I get the following error: ModuleNotFoundError: No module named 'tensorflow.compat' I tried running the code on two separate machines, uninstalling and re-installing tensorflow, pip, keras, nothing seemed to help. I'm using Python 3.9 and tensorflow 2.8.0

Answer: This just happened to me but I figured it out. Your .py script filename is the same with one of the files of the tensorflow library. You can just rename your python script and it will work fine.
Question: I need to install this specific Python version, to prepare a developer environment, because I'm maintaining a system with multiple libraries based on python 3.6.9. I recently installed Ubuntu 22.04 on my laptop, but I had no success trying to install this python version. I tried to install with apt-get after adding the deadsnake repository, but this python version is not available. I tried installing from source by compiling, but it did not work. Running sudo make altinstall exited with this error: Segmentation fault (core dumped) make: *** [Makefile:1112: altinstall] Erro 139 

Answer: I have faced the same problems and could make it work by adding some additional flags when running ./configure Here are my steps: Step 1 – Prerequsities sudo apt-get install -y make build-essential libssl-dev zlib1g-dev \ libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \ libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev \ libgdbm-dev libnss3-dev libedit-dev libc6-dev  Step 2 – Download Python 3.6 wget https://www.python.org/ftp/python/3.6.15/Python-3.6.15.tgz tar -xzf Python-3.6.15.tgz  Step 3 – Compile Python Source cd Python-3.6.15 ./configure --enable-optimizations -with-lto --with-pydebug make -j 8 # adjust for number of your CPU cores sudo make altinstall  Step 4 – Check the Python Version python3.6 -V
Question: According to the docs:  allow_mutation whether or not models are faux-immutable, i.e. whether setattr is allowed (default: True)  Well I have a class : class MyModel(BaseModel): field1:int class Config: allow_mutation = True  If I try to add a field dynamically : model1 = MyModel(field1=1) model1.field2 = 2  And I get this error :  File "pydantic/main.py", line 347, in pydantic.main.BaseModel.__setattr__ ValueError: "MyModel" object has no field "field2"  Obviously, using setattr method will lead to the same error. setattr(model1, 'field2', 2)  Output:  File "pydantic/main.py", line 347, in pydantic.main.BaseModel.__setattr__ ValueError: "MyModel" object has no field "field2"  What did I miss here ?

Answer: You can use the Config object within the class and set the extra attribute to "allow" or use it as extra=Extra.allow kwargs when declaring the model Example from the docs : from pydantic import BaseModel, ValidationError, Extra class Model(BaseModel, extra=Extra.forbid): a: str try: Model(a='spam', b='oh no') except ValidationError as e: print(e) """ 1 validation error for Model b extra fields not permitted (type=value_error.extra) """
Question: Working in Python with Atom on a Mac. Code: from rubik.cube import Cube from rubik_solver import utils  Full error: Traceback (most recent call last): File "/Users/Audey/Desktop/solver.py", line 2, in <module> from rubik_solver import utils File "/Users/Audey/Library/Python/3.10/lib/python/site-packages/rubik_solver/utils.py", line 4, in <module> from past.builtins import basestring File "/Users/Audey/Library/Python/3.10/lib/python/site-packages/past/builtins/__init__.py", line 43, in <module> from past.builtins.noniterators import (filter, map, range, reduce, zip) File "/Users/Audey/Library/Python/3.10/lib/python/site-packages/past/builtins/noniterators.py", line 24, in <module> from past.types import basestring File "/Users/Audey/Library/Python/3.10/lib/python/site-packages/past/types/__init__.py", line 25, in <module> from .oldstr import oldstr File "/Users/Audey/Library/Python/3.10/lib/python/site-packages/past/types/oldstr.py", line 5, in <module> from collections import Iterable ImportError: cannot import name 'Iterable' from 'collections' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/collections/__init__.py)  The from rubik_solver import utils is what is causing the error as when I remove it the error does not appear. I am not sure what is causing the error and hove checked there code and found it on other sources so am sure that it should work. Any solves?

Answer: The Iterable abstract class was removed from collections in Python 3.10. See the deprecation note in the 3.9 collections docs. In the section Removed of the 3.10 docs, the item  Remove deprecated aliases to Collections Abstract Base Classes from the collections module. (Contributed by Victor Stinner in bpo-37324.)  is what results in your error. You can use Iterable from collections.abc instead, or use Python 3.9 if the problem is in a dependency that can't be updated.
Question: My organisation converts any Jupyter Notebooks (.ipynb files) it makes into python scripts (.py files) for easier management in our repos. I need to convert them back so I can run the notebooks but can't work out how. I believe they've been encoded using the nbconvert package but I couldn't find a way to convert the files back in the package docs. I've included the head of one of the .py files bellow in case it makes the encoding format more obvious. # --- # jupyter: # jupytext: # formats: ipynb,py:light # text_representation: # extension: .py # format_name: light # format_version: '1.4' # jupytext_version: 1.2.0 # kernelspec: # display_name: Python 3 # language: python # name: python3 # --- # # Title/filename here # ### Initialise import pandas as pd import numpy as np # etc # ### Title here # + do_stuff() 

Answer: Looks like the file was actually converted using program jupytext. To convert it back I found running jupytext --to ipynb my_file.py worked. More simply: opening the .py file in jupyter notebook allowed me to view the .py file as if it were a .ipynb file without converting.
Question: I am trying to launch the htop command in the Pytorch 1.10 - Python 3.8 CPU optimized AWS Sagemaker container. This works fine in other images I have used till now, but in this one, the command fails with a segfault: htop htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) Segmentation fault (core dumped)  More info : htop --version htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) htop: /opt/conda/lib/libncursesw.so.6: no version information available (required by htop) htop 2.2.0 - (C) 2004-2019 Hisham Muhammad Released under the GNU GPL. 

Answer: I fixed this with # Note: add sudo if needed: ln -fs /lib/x86_64-linux-gnu/libncursesw.so.6 /opt/conda/lib/libncursesw.so.6
Question: I'm calculating network communities for 4 networks using 2 methods:  'Leiden' method, which gives me 7 (a), 13 (b), 19 (c), 22 (d) communities.  'Stochastic block Model', also checking group membership of the nodes by inspecting levels of the hierarchy, like so:     state = gt.inference.minimize_nested_blockmodel_dl(g) state.print_summary() levels = state.get_levels() for s in levels: print(s) if s.get_N() == 1: break lstate = state.levels[0] b = lstate.get_blocks() print(b[10])   which prints: <BlockState object with 228 blocks (21 nonempty), degree-corrected, for graph <Graph object, undirected, with 228 vertices and 1370 edges, 1 internal vertex property, 1 internal edge property, at 0x7fbaff1c8d50>, at 0x7fba9fac1bd0> <BlockState object with 21 blocks (6 nonempty), for graph <Graph object, undirected, with 228 vertices and 96 edges, at 0x7fb9a3c51910>, at 0x7fb9a2dd1a10> <BlockState object with 6 blocks (1 nonempty), for graph <Graph object, undirected, with 21 vertices and 20 edges, at 0x7fb9a3c51590>, at 0x7fb9a3c51ed0> <BlockState object with 1 blocks (1 nonempty), for graph <Graph object, undirected, with 6 vertices and 1 edge, at 0x7fb9a6f034d0>, at 0x7fb9a3c51790> 190 <Graph object, undirected, with 3459 vertices and 134046 edges, 1 internal vertex property, 1 internal edge property, at 0x7fbb62e22790> l: 0, N: 3459, B: 294 l: 1, N: 294, B: 85 l: 2, N: 85, B: 34 l: 3, N: 34, B: 12 l: 4, N: 12, B: 4 l: 5, N: 4, B: 1 l: 6, N: 1, B: 1   and draws:  This looks like having WAY more communities than using Leiden, and I'm trying to wrap my head around why, as well as this SBM concept. Are these SBM graphs depicting adicional levels of hierarchy or is there something else going on here that justifies so many more communities?

Answer: The method of modularity maximization (of which Leiden is an implementation) has two important properties:  It only searches for assortative communities (i.e. groups with more internal connections than external ones). It is a statistically inconsistent method, that will both overfit and underfit, depending on the situation. A discussion on this matter can be found here: https://skewed.de/tiago/blog/modularity-harmful  The SBM inference method is different in both counts:  It finds groups with arbitrary mixing patterns, i.e. preferences of connections to other groups. Assortativity is a special case, but there are many others possible patterns. It achieves this in a statistically principled manner, avoiding both overfitting and underfitting. For a theoretical introduction, see: https://arxiv.org/abs/1705.10225. For a discussion on the differences between inferential and non-inferential methods see: https://arxiv.org/abs/2112.00183  Because of the above, one should not expect SBM inference and Leiden/Louvain to yield similar answers in general. Now, for whatever reason, you may be interested to find only assortative communities. You can also do that with the SBM, but using a more constrained parametrization. You can do this with graph-tool as explained here: https://graph-tool.skewed.de/static/doc/demos/inference/inference.html#assortative-community-structure
Question: I'm trying to validate WebApp data but the result is not what I wanted. Telegram documentation: data_check_string = ... secret_key = HMAC_SHA256(<bot_token>, "WebAppData") if (hex(HMAC_SHA256(data_check_string, secret_key)) == hash) { // data is from Telegram }  MyCode: BOT_TOKEN = '5139539316:AAGVhDje2A3mB9yA_7l8-TV8xikC7KcudNk' data_check_string = 'query_id=AAGcqlFKAAAAAJyqUUp6-Y62&user=%7B%22id%22%3A1246866076%2C%22first_name%22%3A%22Dante%22%2C%22last_name%22%3A%22%22%2C%22username%22%3A%22S_User%22%2C%22language_code%22%3A%22en%22%7D&auth_date=1651689536&hash=de7f6b26aadbd667a36d76d91969ecf6ffec70ffaa40b3e98d20555e2406bfbb' data_check_arr = data_check_string.split('&') needle = 'hash=' hash_item = '' telegram_hash = '' for item in data_check_arr: if item[0:len(needle)] == needle: telegram_hash = item[len(needle):] hash_item = item data_check_arr.remove(hash_item) data_check_arr.sort() data_check_string = "\n".join(data_check_arr) secret_key = hmac.new("WebAppData".encode(), BOT_TOKEN.encode(), hashlib.sha256).digest() calculated_hash = hmac.new(data_check_string.encode(), secret_key, hashlib.sha256).hexdigest() print(calculated_hash == telegram_hash) # print False  I'm trying to validate webapp data in python, but my code didn't give the intended result. the hash which my code gives me is different from the telegram's one. UPDATE: valid data added, and bot-token has been changed.

Answer: You need to unquote data_check_string from urllib.parse import unquote data_check_string = unquote('query_id=AAGcqlFKAAAAAJyqUUp6-Y62&user=%7B%22id%22%3A1246866076%2C%22first_name%22%3A%22Dante%22%2C%22last_name%22%3A%22%22%2C%22username%22%3A%22S_User%22%2C%22language_code%22%3A%22en%22%7D&auth_date=1651689536&hash=de7f6b26aadbd667a36d76d91969ecf6ffec70ffaa40b3e98d20555e2406bfbb')  And swap the arguments calculated_hash = hmac.new(secret_key, data_check_string.encode(), hashlib.sha256).hexdigest()
Question: I'm trying to deploy a simple model on the Triton Inference Server. It is loaded well but I'm having trouble formatting the input to do a proper inference request. My model has a config.pbtxt set up like this  max_batch_size: 1 input: [ { name: "examples" data_type: TYPE_STRING format: FORMAT_NONE dims: [ -1 ] is_shape_tensor: false allow_ragged_batch: false optional: false } ]  I've tried using a pretty straightforward python code to setup the input data like this (the outputs are not written but are setup correctly)  bytes_data = [input_data.encode('utf-8')] bytes_data = np.array(bytes_data, dtype=np.object_) bytes_data = bytes_data.reshape([-1, 1]) inputs = [ httpclient.InferInput('examples', bytes_data.shape, "BYTES"), ] inputs[0].set_data_from_numpy(bytes_data)  But I keep getting the same error message tritonclient.utils.InferenceServerException: Could not parse example input, value: '[my text input here]' [[{{node ParseExample/ParseExampleV2}}]]  I've tried multiple ways of encoding the input, as bytes or even as TFX serving used to ask like this { "instances": [{"b64": "CjEKLwoJdXR0ZXJhbmNlEiIKIAoecmVuZGV6LXZvdXMgYXZlYyB1biBjb25zZWlsbGVy"}]} I'm not exactly sure where the problems comes from if anyone knows?

Answer: If anyone gets this same problem, this solved it. I had to create a tf.train.Example() and set the data correctly example = tf.train.Example() example_bytes = str.encode(input_data) example.features.feature['utterance'].bytes_list.value.extend([example_bytes]) inputs = [ httpclient.InferInput('examples', [1], "BYTES"), ] inputs[0].set_data_from_numpy(np.asarray(example.SerializeToString()).reshape([1]), binary_data=False)
Question: I try to call an R object from Python inside a Quarto document: --- title: "pandas" format: html jupyter: python3 --- ```{r} data("penguins", package = "palmerpenguins") ``` ```{python} penguins=r.penguins penguins ```  When I execute the chunks one by one in RStudio, everything is okay: > data("penguins", package = "palmerpenguins") > reticulate::repl_python() # automatically executed by RStudio Python 3.10.4 (/Users/.../3.10.4/bin/python3.10) Reticulate 1.24 REPL -- A Python interpreter in R. Enter 'exit' or 'quit' to exit the REPL and return to R. >>> penguins=r.penguins >>> penguins species island bill_length_mm ... body_mass_g sex year 0 Adelie Torgersen 39.1 ... 3750 male 2007 1 Adelie Torgersen 39.5 ... 3800 female 2007 ...  However, when I try to render this document, it errors this: --------------------------------------------------------------------------- NameError Traceback (most recent call last) Input In [1], in <cell line: 2>() 1 # Python chunk ----> 2 penguins=r.penguins 3 penguins NameError: name 'r' is not defined  According to RMarkdown documentation, nothing else is required (so no e.g. rpy2). I try to add library(reticulate) or reticulate::repl_python() in the R chunk but it doesn't solve the issue. Note: I'm aware of an old unanswered similar question for RMarkdown. Thanks!

Answer: Quarto have two engines for render, knitr and jupyter.A related document is here. If we use: --- title: "pandas" format: html --- ```{r} data("penguins", package = "palmerpenguins") ``` ```{python} penguins=r.penguins penguins ```  The engine will be knitr. And while render, knitr will use reticulate(R Interface to Python) to run python code chunk. In this process, knitr will do some magic things to convert the form like r.penguins to the form of reticulate. So the document will be successfully rendered. In other words, knitr made some adaptations to let us could easily run python code chunk with reticulate, and if we don't use knitr engine, we can't use the form like r.penguins.  Quarto use r run all code chunks (auto use r package reticulate to run python chunks) when it uses knitr engine.   And Quarto use python to run all code chunks when it uses jupyter (jupyter: python3) engine. If we want run r code, we must use module (such as rpy2) in python chunk (not r chunk, codes in r chunk will not be run).   We can also use r to run all code chunks by setting jupyter: ir (if we have installed IRkernel).But codes in python chunk will not be run. We must use package (such as reticulate) in r chunk to run python code.  This is my understanding. My English is not good, so if some sentence made you puzzled, we could discuss further.
Question: I have a number of tables in different schemas, I used the pattern from the docs. Some of my tables require multi column constraints and it was unclear what the dictionary key would be for declaring that unique constraint as they mention in the section above In my model below, I'd like to create a unique constraint with name, key, org. I currently have to do this in sql... class Parent(Base): __tablename__ = 'parent' __table_args__ = {'schema': 'example'} id = Column(Integer, primary_key=True) name = Column(String(512)) key = Column(String()) org = Column(String(36)) 

Answer: I think I encountered that issue a while back. If I remember correctly, it was just a matter of moving the "schema dict" to inside a tuple which also contains your constraints. I can try to dig further if that does not work, but the documentation seems to agree that using declarative table configuration via __table_args__ can be a tuple containing positional arguments (like constraints) and as a final argument a dict with keyword arguments like schema is for Table. class Parent(Base): __tablename__ = 'parent' __table_args__ = ( UniqueConstraint('name', 'key', 'org'), {'schema': 'example'}, ) id = Column(Integer, primary_key=True) name = Column(String(512)) key = Column(String()) org = Column(String(36))
Question: How do I avoid initializing a class within a pyspark user-defined function? Here is an example. Creating a spark session and DataFrame representing four latitudes and longitudes. import pandas as pd from pyspark import SparkConf from pyspark.sql import SparkSession conf = SparkConf() conf.set('spark.sql.execution.arrow.pyspark.enabled', 'true') spark = SparkSession.builder.config(conf=conf).getOrCreate() sdf = spark.createDataFrame(pd.DataFrame({ 'lat': [37, 42, 35, -22], 'lng': [-113, -107, 127, 34]}))  Here is the Spark DataFrame +---+----+ |lat| lng| +---+----+ | 37|-113| | 42|-107| | 35| 127| |-22| 34| +---+----+  Enriching the DataFrame with a timezone string at each latitude / longitude via the timezonefinder package. Code below runs without errors from typing import Iterator from timezonefinder import TimezoneFinder def func(iterator: Iterator[pd.DataFrame]) -> Iterator[pd.DataFrame]: for dx in iterator: tzf = TimezoneFinder() dx['timezone'] = [tzf.timezone_at(lng=a, lat=b) for a, b in zip(dx['lng'], dx['lat'])] yield dx pdf = sdf.mapInPandas(func, schema='lat double, lng double, timezone string').toPandas()  The above code runs without errors and creates the pandas DataFrame below. The issue is the TimezoneFinder class is initialized within the user-defined function which creates a bottleneck In [4]: pdf Out[4]: lat lng timezone 0 37.0 -113.0 America/Phoenix 1 42.0 -107.0 America/Denver 2 35.0 127.0 Asia/Seoul 3 -22.0 34.0 Africa/Maputo  The question is how to get this code to run more like below, where the TimezoneFinder class is initialized once and outside of the user-defined function. As is, the code below generates this error PicklingError: Could not serialize object: TypeError: cannot pickle '_io.BufferedReader' object def func(iterator: Iterator[pd.DataFrame]) -> Iterator[pd.DataFrame]: for dx in iterator: dx['timezone'] = [tzf.timezone_at(lng=a, lat=b) for a, b in zip(dx['lng'], dx['lat'])] yield dx tzf = TimezoneFinder() pdf = sdf.mapInPandas(func, schema='lat double, lng double, timezone string').toPandas()  UPDATE - Also tried to use functools.partial and an outer function but still received same error. That is, this approach does not work: def outer(iterator, tzf): def func(iterator: Iterator[pd.DataFrame]) -> Iterator[pd.DataFrame]: for dx in iterator: dx['timezone'] = [tzf.timezone_at(lng=a, lat=b) for a, b in zip(dx['lng'], dx['lat'])] yield dx return func(iterator) tzf = TimezoneFinder() outer = partial(outer, tzf=tzf) pdf = sdf.mapInPandas(outer, schema='lat double, lng double, timezone string').toPandas() 

Answer: You will need a cached instance of the object on every worker. You could do that as follows instance = [None] def func(iterator: Iterator[pd.DataFrame]) -> Iterator[pd.DataFrame]: if instance[0] is None: instance[0] = TimezoneFinder() tzf = instance[0] for dx in iterator: dx['timezone'] = [tzf.timezone_at(lng=a, lat=b) for a, b in zip(dx['lng'], dx['lat'])] yield dx  Note that for this to work, your function would be defined within a module, to give the instance cache somewhere to live. Else you would have to hang it off some builtin module, e.g., os.instance = [].