eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: We are using pre-commit to format our Python code using black with the following configuration in .pre-commit-config.yaml: repos: - repo: https://github.com/ambv/black rev: 20.8b1 hooks: - id: black language_version: python3.7  As our packages are tested against and used in different Python versions (e.g. 3.7, 3.8, 3.9) I want to be able to use the pre-commit Hook on different Python versions. But when committing Code e.g. on Python 3.8, I get an error due to the language_version in my configuration (see above): C:\Users\FooBar\Documents\Programmierung\foo (dev -> origin) λ git commit -m "Black file with correct black version" [INFO] Initializing environment for https://github.com/ambv/black. [INFO] Installing environment for https://github.com/ambv/black. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('c:\\users\\FooBar\\anaconda\\python.exe', '-mvirtualenv', 'C:\\Users\\FooBar\\.cache\\pre-commit\\repobmlg3b_m\\py_env-python3.7', '-p', 'python3.7') return code: 1 expected return code: 0 stdout: RuntimeError: failed to find interpreter for Builtin discover of python_spec='python3.7' stderr: (none) Check the log at C:\Users\FooBar\.cache\pre-commit\pre-commit.log  How can I enable the pre-commit Hook on different Python-Versions e.g. only on Python 3? Thanks in advance!

Answer: one way would be to set language_version: python3 (this used to be the default for black) -- the actual language_version you use there doesn't matter all that much as black doesn't use it to pick the formatted language target (that's a separate option) generally though, you shouldn't need to set language_version as either (1) the hook itself will set a proper one or (2) it will default to your currently running python note also: you're using the twice-deprecated url for black -- it is now psf/black __ disclaimer: I created pre-commit and I'm a black contributor
Question: I'm trying to write two functions for converting Cartesian coordinates to spherical coordinates and vice-versa. Here are the equations that I've used for the conversions (also could be found on this Wikipedia page):  And  Here is my spherical_to_cartesian function: def spherical_to_cartesian(theta, phi): x = math.cos(phi) * math.sin(theta) y = math.sin(phi) * math.sin(theta) z = math.cos(theta) return x, y, z  Here is my cartesian_to_spherical function: def cartesian_to_spherical(x, y, z): theta = math.atan2(math.sqrt(x ** 2 + y ** 2), z) phi = math.atan2(y, x) if x >= 0 else math.atan2(y, x) + math.pi return theta, phi  And, here is the driver code: >>> t, p = 27.500, 7.500 >>> x, y, z = spherical_to_cartesian(t, p) >>> print(f"Cartesian coordinates:\tx={x}\ty={y}\tz={z}") Cartesian coordinates: x=0.24238129061573832 y=0.6558871334524494 z=-0.7148869687796651 >>> theta, phi = cartesian_to_spherical(x, y, z) >>> print(f"Spherical coordinates:\ttheta={theta}\tphi={phi}") Spherical coordinates: theta=2.367258771281654 phi=1.2168146928204135  I can't figure out why I'm getting different values for theta and phi than my initial values (the output values aren't even close to the input values). Did I make a mistake in my code which I can't see?

Answer: You seem to be giving your angles in degrees, while all trigonometric functions expect radians. Multiply degrees with math.pi/180 to get radians, and multiply radians with 180/math.pi to get degrees.
Question: I have a df with numbers: numbers = pd.DataFrame(columns=['number'], data=[ 50, 65, 75, 85, 90 ])  and a df with ranges (look up table): ranges = pd.DataFrame( columns=['range','range_min','range_max'], data=[ ['A',90,100], ['B',85,95], ['C',70,80] ] )  I want to determine what range (in second table) a value (in the first table) falls in. Please note ranges overlap, and limits are inclusive. Also please note the vanilla dataframe above has 3 ranges, however this dataframe gets generated dynamically. It could have from 2 to 7 ranges. Desired result: numbers = pd.DataFrame(columns=['number','detected_range'], data=[ [50,'out_of_range'], [65, 'out_of_range'], [75,'C'], [85,'B'], [90,'overlap'] * could be A or B * ])  I solved this with a for loop but this doesn't scale well to a big dataset I am using. Also code is too extensive and inelegant. See below: numbers['detected_range'] = nan for i, row1 in number.iterrows(): for j, row2 in ranges.iterrows(): if row1.number<row2.range_min and row1.number>row2.range_max: numbers.loc[i,'detected_range'] = row1.loc[j,'range'] else if (other cases...): ...and so on...  How could I do this?

Answer: You can use a bit of numpy vectorial operations to generate masks, and use them to select your labels: import numpy as np a = numbers['number'].values # numpy array of numbers r = ranges.set_index('range') # dataframe of min/max with labels as index m1 = (a>=r['range_min'].values[:,None]).T # is number above each min m2 = (a<r['range_max'].values[:,None]).T # is number below each max m3 = (m1&m2) # combine both conditions above # NB. the two operations could be done without the intermediate variables m1/m2 m4 = m3.sum(1) # how many matches? # 0 -> out_of_range # 2 -> overlap # 1 -> get column name # now we select the label according to the conditions numbers['detected_range'] = np.select([m4==0, m4==2], # out_of_range and overlap ['out_of_range', 'overlap'], # otherwise get column name default=np.take(r.index, m3.argmax(1)) )  output:  number detected_range 0 50 out_of_range 1 65 out_of_range 2 75 C 3 85 B 4 90 overlap  edit: It works with any number of intervals in ranges example output with extra['D',50,51]:  number detected_range 0 50 D 1 65 out_of_range 2 75 C 3 85 B 4 90 overlap
Question: ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'C:\Users\Sampath\anaconda3\Lib\site-packages\~5py\defs.cp38-win_amd64.pyd' Consider using the --user option or check the permissions. I tried pip install mediapipe

Answer: EnvironmentError: Access is denied errors usually stem from one of three reasons:  You do not have the proper permissions to install these files, and you should try running the same commands in an Administrator Command Prompt. 90% of the time, this should solve the problem. You don't have permission to install the package system-wide. You can instead install it for only your user by adding the --user flag:  pip install --user <PACKAGE>   If that doesn't work, then the problem is usually from an external program accessing a file, and you (or the installation script) are trying to delete that file (you cannot delete a file that is opened by another program). Try to restart your computer, so that whatever process is using that file will be shut down. Then, try the command again.
Question: What is the python way to define a (non-class) type like: typedef Dict[Union[int, str], Set[str]] RecordType 

Answer: This would simply do it? from typing import Dict, Union, Set RecordType = Dict[Union[int, str], Set[str]] def my_func(rec: RecordType): pass my_func({1: {'2'}}) my_func({1: {2}})  This code will generate a warning from your IDE on the second call to my_func, but not on the first. As @sahasrara62 indicated, more here https://docs.python.org/3/library/stdtypes.html#types-genericalias Since Python 3.9, the preferred syntax would be: from typing import Union RecordType = dict[Union[int, str], set[str]]  The built-in types can be used directly for type hints and the added imports are no longer required. Since Python 3.10, the preferred syntax is RecordType = dict[int | str, set[str]]  The | operator is a simpler way to create a union of types, and the import of Union is no longer required. Since Python 3.12, the preferred syntax is type RecordType = dict[int | str, set[str]]  The type keyword explicitly indicates that this is a type alias.
Question: is there a clean way of triggering a function call whenever I create/ instantiate a pydantic object? Currently I am "misusing" the root_validator for this: from pydantic import BaseModel class PydanticClass(BaseModel): name: str @root_validator() def on_create(cls, values): print("Put your logic here!") return values  So on PydanticClass(name="Test") executes my logic and simply returns the same object values. This works but I have two issues, which is why I would be interested in a cleaner solution:  I basically don't do a validation (return the same values). I think this function will also be executed once the object is changed, which I don't want.  So I am happy to learn about any better approaches/ solutions.

Answer: Your intentions are not fully clear. But I can suggest overriding the __init__ model method. In this case, your code will be executed once at object instantiation: from pydantic import BaseModel class PydanticClass(BaseModel): name: str def __init__(self, **data) -> None: super().__init__(**data) print("Put your logic here!")
Question: How can I create a pydantic object, without useing alias names? from pydantic import BaseModel, Field class Params(BaseModel): var_name: int = Field(alias='var_alias') Params(var_alias=5) # works Params(var_name=5) # does not work 

Answer: As of the pydantic 2.0 release, this behaviour has been updated to use model_config populate_by_name option which is False by default. from pydantic import BaseModel, Field, ConfigDict class Params(BaseModel): var_name: int = Field(alias='var_alias') model_config = ConfigDict( populate_by_name=True, ) Params(var_alias=5) # works Params(var_name=5) # works  For pydantic 1.x, you need to use allow_population_by_field_name model config option. from pydantic import BaseModel, Field class Params(BaseModel): var_name: int = Field(alias='var_alias') class Config: allow_population_by_field_name = True Params(var_alias=5) # works Params(var_name=5) # works
Question: I am trying to run my program which uses various dependencies, but since upgrading to Python 3.10 this does not work anymore. When I run "python3" in the terminal and from there import my dependencies I get an error: ImportError: cannot import name 'Mapping' from 'collections' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/collections/__init__.py)  This seems to be a general problem, but here is the traceback of my specific case: Traceback (most recent call last): File "/Users/mk/Flasktut/app.py", line 2, in <module> from flask import Flask, render_template File "/Users/mk/Flasktut/env/lib/python3.10/site-packages/flask/__init__.py", line 14, in <module> from jinja2 import escape File "/Users/mk/Flasktut/env/lib/python3.10/site-packages/jinja2/__init__.py", line 33, in <module> from jinja2.environment import Environment, Template File "/Users/mk/Flasktut/env/lib/python3.10/site-packages/jinja2/environment.py", line 16, in <module> from jinja2.defaults import BLOCK_START_STRING, \ File "/Users/mk/Flasktut/env/lib/python3.10/site-packages/jinja2/defaults.py", line 32, in <module> from jinja2.tests import TESTS as DEFAULT_TESTS File "/Users/mk/Flasktut/env/lib/python3.10/site-packages/jinja2/tests.py", line 13, in <module> from collections import Mapping ImportError: cannot import name 'Mapping' from 'collections' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/collections/__init__.py) 

Answer: Change: from collections import Mapping  to from collections.abc import Mapping
Question: My code looks something like this, which runs fine BDW without any errors from typing import Literal def verify(word: str) -> Literal['Hello XY']: a = 'Hello ' + word return a a = verify('XY')  Although, when I'm trying to do the type-checking using mypy, it throws an error error: Incompatible return value type (got "str", expected "Literal['Hello XY']") NOTE: To perform type-checking simply do mypy ./filename.py, after pip installing mypy. ALSO, When I do this, the type-checking works fine from typing import Literal def verify(word: str) -> Literal['Hello XY']: a = 'Hello ' + word return 'Hello XY' #changed here a = verify('XY')  What am I missing?

Answer: word can be any string, so this seems like a good thing that mypy complains because it cannot guess that you will always call it with the appropriate argument. In other words, for mypy, if you concatenate 'Hello ' with some str, it can give any str and not only 'Hello XY'. What you could do to check if the function is called appropriately, is instead to type word with a literal: from typing import Literal, cast hello_t = Literal['Hello there', 'Hello world'] def verify(word: Literal['there', 'world']) -> hello_t: a = cast(hello_t, 'Hello ' + word) return a a = verify('there') # mypy OK a = verify('world') # mypy OK a = verify('you') # mypy error  Note that a cast is still required because mypy cannot guess that the concatenation of 'Hello ' with a Literal['there', 'world'] is of type hello_t.
Question: I'm learning how to distribute python packages using setuptools and I have a problem. setuptools is setting the name of the folder containing a single python file as the name of my package. Below is the structure of my repository: gerador_endereco/ -- setup.py -- my_package/ -- __init__.py -- gerador_endereco.py  My setup.py is: setup( name='gerador_endereco', version='1.0.4', author='Michel Metran', description='API para criação ...', url='https://github.com/open-dsa/gerador_endereco', packages=find_packages(), install_requires=requirements, )  I understand that setuptools is related to the distribution of packages, composed of several modules. But I know that it is possible to distribute a package composed of a single module, but how can I import the package correctly, without the folder name appearing? # Install !pip install gerador-endereco # Import work using "my_package" directory: bad... from my_package.gerador_endereco import * # I'd like import like this!!! from gerador_endereco import * # Run listas = get_list_ceps_bairros(estado='sp', municipio='piracicaba')  The PyPi Package is in https://pypi.org/project/gerador-endereco/

Answer: setuptools is related to the distribution of packages, period. To install a module restructure you project: gerador_endereco/ -- setup.py -- gerador_endereco.py  and change setup.py; remove packages=find_packages(),  and add py_modules = ['gerador_endereco']  instead. See the docs at https://docs.python.org/3/distutils/setupscript.html#listing-individual-modules and https://packaging.python.org/guides/distributing-packages-using-setuptools/?#py-modules
Question: For example, if I provide a probability array of [0.5, 0.5], both functions will sample the index [0,1] with equal probability?

Answer: Yes:  [torch.distributions.categorical.Categorical()] is equivalent to the distribution that torch.multinomial() samples from.   https://pytorch.org/docs/stable/distributions.html#categorical
Question: i'm trying to join together 2 QuerySets. Right now, I'm using the | operator, but doing it this way won't function as an "append". My current code is: df = RegForm((querysetA.all() | querysetB.all()).distinct())  I need the elements from querysetA to be before querysetB. Is it even possible to accomplish while keeping them just queries?

Answer: This can be solved by using annotate to add a custom field for ordering on the querysets, and use that in a union like this: from django.db.models import Value a = querysetA.annotate(custom_order=Value(1)) b = querysetB.annotate(custom_order=Value(2)) a.union(b).order_by('custom_order')  Prior to django-3.2, you need to specify the output_field for Value: from django.db.models import IntegerField a = querysetA.annotate(custom_order=Value(1, IntegerField())) b = querysetB.annotate(custom_order=Value(2, IntegerField()))
Question: I'm trying to figure out how to send an entire address balance in a post EIP-1559 transaction (essentially emptying the wallet). Before the London fork, I could get the actual value as Total balance - (gasPrice * gas), but now it's impossible to know the exact remaining balance after the transaction fees because the base fee is not known beforehand. Is there an algorithm that would get me as close to the actual balance without going over? My end goal is to minimize the remaining Ether balance, which is essentially going to be wasted.

Answer: This can be done by setting the 'Max Fee' and the 'Max Priority Fee' to the same value. This will then use a deterministic amount of gas. Just be sure to set it high enough - comfortably well over and above the estimated 'Base Fee' to ensure it does not get stuck.
Question: Suggestions please, thanks :) pip list --outdated --format=freeze  Gives the following error: ERROR: Exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 223, in _main status = self.run(options, args) File "/usr/lib/python3/dist-packages/pip/_internal/commands/list.py", line 175, in run packages = self.get_outdated(packages, options) File "/usr/lib/python3/dist-packages/pip/_internal/commands/list.py", line 184, in get_outdated return [ File "/usr/lib/python3/dist-packages/pip/_internal/commands/list.py", line 184, in <listcomp> return [ File "/usr/lib/python3/dist-packages/pip/_internal/commands/list.py", line 237, in iter_packages_latest_infos for dist in map_multithread(latest_info, packages): File "/usr/lib/python3.9/multiprocessing/pool.py", line 870, in next raise value File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3/dist-packages/pip/_internal/commands/list.py", line 214, in latest_info all_candidates = finder.find_all_candidates(dist.key) File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 825, in find_all_candidates package_links = self.process_project_url( File "/usr/lib/python3/dist-packages/pip/_internal/index/package_finder.py", line 793, in process_project_url page_links = list(parse_links(html_page)) File "/usr/lib/python3/dist-packages/pip/_internal/index/collector.py", line 324, in wrapper_wrapper return list(fn(page)) File "/usr/lib/python3/dist-packages/pip/_internal/index/collector.py", line 335, in parse_links document = html5lib.parse( File "/usr/share/python-wheels/html5lib-1.1-py2.py3-none-any.whl/html5lib/html5parser.py", line 44, in parse tb = treebuilders.getTreeBuilder(treebuilder) File "/usr/share/python-wheels/html5lib-1.1-py2.py3-none-any.whl/html5lib/treebuilders/__init__.py", line 85, in getTreeBuilder return etree.getETreeModule(implementation, **kwargs).TreeBuilder AttributeError: module 'html5lib.treebuilders.etree' has no attribute 'getETreeModule' 

Answer: I solved this problem updating pip, i updated from pip 20.3.4 to 21.3 so just type: pip install pip -U  Seems like there is some bug in pip itself.
Question: I am having trouble finding a straightforward answer to the following question: If you compute the Cholesky decomposition of an nxn positive definite symmetric matrix A, i.e factor A=LL^T with L a lower triangular matrix, the complexity is O(n^3). For sparse matrices, there are apparently faster algorithms, but how much faster? What complexity can we achieve for such a matrix with say m<n^2 nonzero entries? Edit: my matrix is also approximately main diagonal (only the diagonal and a some adjacent diagonals below and above are nonzero). P.S I am eventually interested in implementations in either Julia or Python. Python has the sksparse.cholmod module (https://scikit-sparse.readthedocs.io/en/latest/cholmod.html) but it isn't clear to me what algorithm they are using and what its complexity is. Not sure about Julia, if anyone can tell me.

Answer: This can only be answered exactly for abitrary matrices if P=NP ... so it's not possible to answer in general. The time complexity depends on the fill-reducing ordering used, which is attempting to get an approximate solution to an NP hard problem. However, for the very special case of a matrix coming from a regular square 2D or 3D mesh, there is an answer. In this case, nested dissection gives an ordering that is asymptotically optimal. For a 2D s-by-s mesh, the matrix has dimension n = s^2 and I think about 5n entries. In this case, L has 31*(n log2(n)/8)+O(n) nonzeros, and the work is 829*(n^(3/2))/84+O(n log n). For a 3D s-by-s-by-s mesh with n = s^3, there are O(n^(4/3)) nonzeros in L and O(n^2) operations are required to compute L.
Question: In dask's LocalCluster, there is a parameter memory_limit. I can't find in the documentation (https://distributed.dask.org/en/latest/worker.html#memory-management) details about whether the limit is per worker, per thread, or for the whole cluster. This is probably at least in part because I have trouble following how keywords are passed around. For example, in this code: cluster = LocalCluster(n_workers=2, threads_per_worker=4, memory_target_fraction=0.95, memory_limit='32GB')  will that be 32 GB for each worker? For both workers together? Or for each thread? My question is motivated partly by running a LocalCluster with n_workers=1 and memory_limit=32GB, but it gets killed by the Slurm Out-Of-Memory killer for using too much memory.

Answer: The memory_limit keyword argument to LocalCluster sets the limit per worker. Related documentaion: https://github.com/dask/distributed/blob/7bf884b941363242c3884b598205c75373287190/distributed/deploy/local.py#L76-L78 Note, if the memory_limit given is greater than the available memory, the total available memory will be set for each worker. This behavior hasn't been documented yet, but a relevant issue is here: https://github.com/dask/dask/issues/8224 Screenshot of cluster with code: from dask.distributed import LocalCluster, Client cluster = LocalCluster(n_workers=2, threads_per_worker=4, memory_target_fraction=0.95, memory_limit='8GB') client = Client(cluster) client
Question: Python 3.10 is released and when I try to install NumPy it gives me this: NumPy 1.21.2 may not yet support Python 3.10.. what should I do?

Answer: If on Windows, numpy has not yet released a precompiled wheel for Python 3.10. However you can try the unofficial wheels available at https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy . Specifically look for  numpy‑1.21.2+mkl‑cp310‑cp310‑win_amd64.whl or numpy‑1.21.2+mkl‑cp310‑cp310‑win32.whl  depending on you system architecture. After downloading the file go to the download directory and run pip install "<filename>.whl".) (I have personally installed numpy‑1.21.2+mkl‑cp310‑cp310‑win_amd64.whl and it worked for me.)
Question: hi im using chrome driver but i cant fix this error mycode: options = Options() options.add_argument('--disable-gpu') options.add_argument('--disable-dev-shm-usage') self.site = webdriver.Chrome(executable_path="C:\chromedriver.exe",chrome_options=options) self.site.get("https://sgite.com/en/site/")  error: [23468:14696:1004/232130.459:ERROR:chrome_browser_main_extra_parts_metrics.cc(228)] crbug.com/1216328: Checking Bluetooth availability started. Please report if there is no report that this ends. [23468:14696:1004/232130.468:ERROR:chrome_browser_main_extra_parts_metrics.cc(231)] crbug.com/1216328: Checking Bluetooth availability ended. [23468:14696:1004/232130.514:ERROR:chrome_browser_main_extra_parts_metrics.cc(234)] crbug.com/1216328: Checking default browser status started. Please report if there is no report that this ends. [23468:14696:1004/232130.588:ERROR:chrome_browser_main_extra_parts_metrics.cc(238)] crbug.com/1216328: Checking default browser status ended.

Answer: If you are using Selenium with Python then add these extra options into your Selenium code- options = webdriver.ChromeOptions() options.add_experimental_option('excludeSwitches', ['enable-logging']) driver = webdriver.Chrome(options=options)