eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I've installed a handful of PyTorch versions (CUDA 11.7 nightly, CUDA 11.6 nightly, 11.3), but every time, torch.version.cuda returns 10.2. I'd like to run PyTorch on CUDA 11.7. My graphics card has CUDA capability sm_86. [me@legion imagen-test]$ sudo pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113 ... [me@legion imagen-test]$ python >>> import torch >>> print(torch.version.cuda) 10.2  When I actually try to use PyTorch, I get an error saying the PyTorch version I have installed doesn't support the newer version of CUDA my graphics card requires. >>> torch.Tensor([1,2,3]).cuda() ... NVIDIA GeForce RTX 3060 Laptop GPU with CUDA capability sm_86 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70. ... RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.  I'm completely stumped, and unsure where to go from here. I'd appreciate any help.

Answer: You've probably installed PyTorch with CUDA 10.2 among your different installed versions. This may be taking priority over the versions of PyTorch. To fix this, simply uninstall all versions of PyTorch with pip uninstall torch -y and reinstall PyTorch with CUDA 11.7. Source: https://discuss.pytorch.org/t/cuda-version-is-always-10-2/152876
Question: Not long ago my computer broke and I am stuck on an iPad. I installed iSH from the AppStore. Now I want to download Python and make sure pip works. I have tried apk add python, which lead to the pip issue, but pip installing is important for me. I have also found other ways using yam or apt(-get), but I do not know how to download either of them.

Answer: According information that you provided iSH using virtual environment with Alpine Linux x86 under the hood (I little bit simplify explanation, so it is not 100% correct. You can see details here). So if you want to install pip you have to search how to install pip in Alpine Linux. You will find many answers like that: apk add --update-cache python3 py3-pip  This information applicable to any other package that you will try to install. Not just pip.
Question: #25 3.990 × python setup.py egg_info did not run successfully. #25 3.990 │ exit code: 1 #25 3.990 ╰─> [1 lines of output] #25 3.990 error in anyjson setup command: use_2to3 is invalid. #25 3.990 [end of output]  This is a common error which the most common solution to is to downgrade setuptools to below version 58. This was not working for me. I tried installing python3-anyjson but this didn't work either. I'm at a complete loss.. any advice or help is much appreciated. If it matters: this application is legacy spaghetti and I am trying to polish it up for a migration. There's no documentation of any kind. The requirements.txt is as follows: cachetools>=2.0.0,<4 certifi==2018.10.15 Flask-Caching Flask-Compress Flask==2.0.3 cffi==1.2.1 diskcache earthengine-api==0.1.239 gevent==21.12.0 google-auth>=1.17.2 google-api-python-client==1.12.1 gunicorn==20.1.0 httplib2.system-ca-certs-locater httplib2==0.9.2 oauth2client==2.0.1 pyasn1-modules==0.2.1 redis requests==2.18.0 werkzeug==2.1.2 six==1.13.0 pyasn1==0.4.1 Jinja2==3.1.1 itsdangerous==2.0.1 Flask-Celery-Helper Flask-JWT==0.2.0 Flask-Limiter Flask-Mail Flask-Migrate Flask-Restless==0.16.0 Flask-SQLAlchemy Flask-Script Flask-Testing Flask==2.0.3 Pillow<=6.2.2 Shapely beautifulsoup4 boto celery==3.1.23 geopy gevent==21.12.0 numpy<1.17 oauth2client==2.0.1 passlib psycopg2 pyproj<2 python-dateutil==2.4.1 scipy 

Answer: Downgrading setuptools worked for me pip install "setuptools<58.0.0"  And then pip install django-celery
Question: I'm giving a shot to the pyproject.toml file, and I'm stuck on this simple task. Consider the following optional dependencies: [project.optional-dependencies] style = ["black", "codespell", "isort", "flake8"] test = ["pytest", "pytest-cov"] all = ["black", "codespell", "isort", "flake8", "pytest", "pytest-cov"]  Is there a way to avoid copy/pasting all the optional-dep in the all key? Is there a way to do all = style + test at least?

Answer: There is no such feature directly in the toml markup. However, there is a tricky way to do this in Python packaging by depending on yourself: [project.optional-dependencies] style = ["black", "codespell", "isort", "flake8"] test = ["pytest", "pytest-cov"] all = ["myproject[style]", "myproject[test]"]  Source:  Circular dependency is a feature that Python packaging is explicitly designed to allow, so it works and should continue to work.
Question: After a thorough search, I have not found a complete explanation and solution to this very common problem on the entire web. All scripts that need to encode with hashlib give me error: Python 3.10 import hashlib h = hashlib.new('ripemd160')  return: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.10/hashlib.py", line 166, in __hash_new return __get_builtin_constructor(name)(data) File "/usr/lib/python3.10/hashlib.py", line 123, in __get_builtin_constructor raise ValueError('unsupported hash type ' + name) ValueError: unsupported hash type ripemd160  I already tried to check if that hash exists in the library, and if I have it: print(hashlib.algorithms_available): {'md5', 'sm3', 'sha3_512', 'sha384', 'sha256', 'sha1', 'shake_128', 'sha224', 'sha512_224', 'sha512_256', 'blake2b', 'ripemd160', 'md5-sha1', 'sha512', 'sha3_256', 'shake_256', 'sha3_384', 'whirlpool', 'md4', 'blake2s', 'sha3_224'} I am having this problem in a vps with linux, but in my pc I use Windows and I don't have this problem. I sincerely appreciate any help or suggestion.

Answer: Hashlib uses OpenSSL for ripemd160 and apparently OpenSSL disabled some older crypto algos around version 3.0 in November 2021. All the functions are still there but require manual enabling. See issue 16994 of OpenSSL github project for details. To quickly enable it, find the directory that holds your OpenSSL config file or a symlink to it, by running the below command: openssl version -d  You can now go to the directory and edit the config file (it may be necessary to use sudo): nano openssl.cnf  Make sure that the config file contains following lines: openssl_conf = openssl_init [openssl_init] providers = provider_sect [provider_sect] default = default_sect legacy = legacy_sect [default_sect] activate = 1 [legacy_sect] activate = 1  Tested on: OpenSSL 3.0.2, Python 3.10.4, Linux Ubuntu 22.04 LTS aarch64, I have no access to other platforms at the moment.
Question: cx_Oracle was renamed to python-oracledb in the May 2022 release. It now comes with two modes, thin and thick. Thick mode uses the Oracle client libraries to connect to Oracle, while thin mode can connect directly. cx_Oracle previously always required using the Oracle client libraries. Is there any performance implications to using the thin mode instead of the thick mode?

Answer: Yes, there is, but it can vary depending on your workload. In our own tests we saw basic fetching and inserting performing with thin mode between 10% and 30% faster than thick mode. The main reason for the difference is the elimination of a copy/conversion step that is required in thick mode. Some more discussion can be found here: https://github.com/oracle/python-oracledb/discussions/5.
Question: Setuptools allows you to specify the minimum python version as such: from setuptools import setup [...] setup(name="my_package_name", python_requires='>3.5.2', [...]  However, how can you do this with the pyproject.toml? The following two things did NOT work: [project] ... # ERROR: invalid key python_requires = ">=3" # ERROR: no matching distribution found dependencies = ["python>=3"] 

Answer: According to PEP 621, the equivalent field in the [project] table is requires-python. More information about the list of valid configuration fields can be found in: https://packaging.python.org/en/latest/specifications/declaring-project-metadata/. The equivalent pyproject.toml of your example would be: [project] name = "my_package_name" requires-python = ">3.5.2" ...
Question: I am currently using Python 3.7.7, and I posed a coding challenge for myself. I would like to list all permutations of integers from 1 to N using a one-line code (perhaps a list comprehension). I cannot use itertools (or other packages which solve this with one function). For N <= 9, I found "cheaty" method: N = 3 print([list(str(i)) for i in range(10**N) if all([str(i).count(str(j)) == 1 for j in range(1,N+1)])])  Example: Out: [['1', '2', '3'], ['1', '3', '2'], ['2', '1', '3'], ['2', '3', '1'], ['3', '1', '2'], ['3', '2', '1']]  In the case of N = 3, this goes through all integers from 0 to 999 in order, and selects the ones that have exactly one 1, exactly one 2, and exactly one 3. (These are 123, 132, 213, 231, 312, 321; and from here, it's simple enough to convert them to a list.) However this obviously fails for N >= 10. I thought about converting the numbers to a higher base first, but that turned out to be even more difficult when restricting myself to only using list comprehension. Can anyone think of a way to do this for N >= 10?

Answer: A not-so-simple functional one-liner without any "outside" variable assignment except N. N = 3 (lambda n: (lambda f, n: f(f, n))(lambda f, n: [p[:i]+[n]+p[i:] for p in f(f, n-1) for i in range(len(p)+1)] if n > 1 else [[1]], n))(N)  Output [[3, 2, 1], [2, 3, 1], [2, 1, 3], [3, 1, 2], [1, 3, 2], [1, 2, 3]]
Question: I'm trying to run an Azure Function locally following the Microsoft guide: https://learn.microsoft.com/nl-nl/azure/azure-functions/create-first-function-cli-python?tabs=azure-cli%2Cbash%2Cbrowser#create-venv Whatever I try I get the same error over and over again when i try to start the function using "func start":  Found Python version 3.8.0 (py). Azure Functions Core Tools Core Tools Version: 4.0.4544 Commit hash: N/A (64-bit) Function Runtime Version: 4.3.2.18186 Could not load file or assembly 'Microsoft.Azure.WebJobs.Script.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3c5b9424214e8f8c'. The system cannot find the file specified.  My files are set up as follows Host.json { "version": "2.0", "logging": { "applicationInsights": { "samplingSettings": { "isEnabled": true, "excludedTypes": "Request" } } }, "extensionBundle": { "id": "Microsoft.Azure.Functions.ExtensionBundle", "version": "[2.*, 3.0.0)" } }  local.setting.json { "IsEncrypted": false, "Values": { "FUNCTIONS_WORKER_RUNTIME": "python", "AzureWebJobsStorage": "UseDevelopmentStorage=true" } }  requirements.txt azure-functions  function.json { "scriptFile": "__init__.py", "bindings": [ { "authLevel": "Anonymous", "type": "httpTrigger", "direction": "in", "name": "req", "methods": [ "get", "post" ] }, { "type": "http", "direction": "out", "name": "$return" } ] }  init.py import logging import azure.functions as func def main(req: func.HttpRequest) -> func.HttpResponse: logging.info('Python HTTP trigger function processed a request.') name = req.params.get('name') if not name: try: req_body = req.get_json() except ValueError: pass else: name = req_body.get('name') if name: return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.") else: return func.HttpResponse( "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.", status_code=200 )  Any help would be appreciated!

Answer: I came across this too. Not sure what caused it. But what fixed it for me was to download, uninstall, and reinstall the Azure Functions Core Tools (using 'repair' just returned a generic "Failed due to error", and I had to kill PowerToys)  Not sure if it's necessary, but I also cleared my nuget cache dotnet nuget locals all -c dotnet restore # in your sln folder  And reinstalled my vscode Azure plugins
Question: I have a 1d array of ids, for example: a = [1, 3, 4, 7, 9]  Then another 2d array: b = [[1, 4, 7, 9], [3, 7, 9, 1]]  I would like to have a third array with the same shape of b where each item is the index of the corresponding item from a, that is: c = [[0, 2, 3, 4], [1, 3, 4, 0]]  What's a vectorized way to do that using numpy?

Answer: Effectively, this solution is a one-liner. The only catch is that you need to reshape the array before you do the one-liner, and then reshape it back again: import numpy as np a = np.array([1, 3, 4, 7, 9]) b = np.array([[1, 4, 7, 9], [3, 7, 9, 1]]) original_shape = b.shape c = np.where(b.reshape(b.size, 1) == a)[1] c = c.reshape(original_shape)  This results with: [[0 2 3 4] [1 3 4 0]]
Question: I have a dataframe with some price indices across 5 years, from 2017 to 2021. It looks like this:     Country Industry Year Index     US Agriculture 2017 83   US Agriculture 2018 97.2   US Agriculture 2019 100   US Agriculture 2020 112   US Agriculture 2021 108   Japan Mining 2017 88   Japan Mining 2018 93   Japan Mining 2019 100   Japan Mining 2020 104   Japan Mining 2021 112     My base year is 2019, hence the Index for every row tagged with 2019 is 100. Everything else moves up or down. I want to generate another column called Percentage_Change showing the year on year change starting from 2019 as the base year. I tried using the pd.series.pct_change function, however, that calculates the year on year percentage change starting with 2017 and it generates an NaN value for all rows where the year is 2017, instead of 2019 which should be the base year. I want the output to look like this:     Country Industry Year Index Percentage_change     Japan Mining 2017 88 -5.37%   Japan Mining 2018 93 -7%   Japan Mining 2019 100 0   Japan Mining 2020 104 4%   Japan Mining 2021 112 7.69%     The percentage_change for Japan between 2021 and 2020 is (112-104)/104 = 7.69%, the difference between 2020 and 2019 is (104-100)/100 = 4%, the difference between 2018 and 2019 is (93-100)/100 = -7%, the difference between 2017 and 2018 is (88-93)/93 = -5.37% Is there any other way of calculating % change in pandas?

Answer: pct_change is computing a change relative to the previous value (which is why 2017 is NaN), and this doesn't seem to be what you want. If you want to compute a percentage change relative to 2019, as 2019 is already normalized to 100, simply subtract 100: df['Percentage_Change'] = df['Index'].sub(100)  output:  Country Industry Year Index Percentage_Change 0 US Agriculture 2017 83.0 -17.0 1 US Agriculture 2018 97.2 -2.8 2 US Agriculture 2019 100.0 0.0 3 US Agriculture 2020 112.0 12.0 4 US Agriculture 2021 108.0 8.0 5 Japan Mining 2017 88.0 -12.0 6 Japan Mining 2018 93.0 -7.0 7 Japan Mining 2019 100.0 0.0 8 Japan Mining 2020 104.0 4.0 9 Japan Mining 2021 112.0 12.0  bidirectional pct_change If you want a bidirectional pct_change "centered" on 100, you can use masks to compute the pct_change both ways: df['Percentage_Change'] = (df .assign(ref=df['Year'].eq(2019)) .groupby(['Country', 'Industry'], group_keys=False) .apply(lambda g: g['Index'].where(g['ref'].cummax()).pct_change() .fillna(g['Index'][::-1].pct_change().mask(g['ref'].cummax(), 0)) ) )  output:  Country Industry Year Index Percentage_Change 0 US Agriculture 2017 83.0 -0.146091 1 US Agriculture 2018 97.2 -0.028000 2 US Agriculture 2019 100.0 0.000000 3 US Agriculture 2020 112.0 0.120000 4 US Agriculture 2021 108.0 -0.035714 5 Japan Mining 2017 88.0 -0.053763 6 Japan Mining 2018 93.0 -0.070000 7 Japan Mining 2019 100.0 0.000000 8 Japan Mining 2020 104.0 0.040000 9 Japan Mining 2021 112.0 0.076923
Question: Here's what I did from pyrosm import OSM # Initialize the OSM parser object osm = OSM('/DATA/user/nabih/indonesia-latest.osm.pbf') # Read all drivable roads drive_net = osm.get_network(network_type="driving")  But it is memory error

Answer: https://osmcode.org/pyosmium/ provides a library to parse a osm.pbf. From what I remember, they keep the memory consumption to the minimum and provide different modes of parsing. Checkout their documentation for basic usage tutorial and references. The README of their GitHubprovides installation instructions.
Question: I have two dataframes, the first is the data I currently have in the database, the second would be a file that might have changed fields: name and/or cnpj and/or create_date Based on that, I need to create a third dataframe with only the rows that have undergone some kind of change, as in the example of the expected output. The key to making the comparisons needs to be: id_account Dataframe 1:     id_account name cnpj create_date     10 Agency Criss 10203040 2022-05-30   20 Agency Angel 11213141 2022-05-30   30 Supermarket Mario Bros 12223242 2022-05-30   40 Agency Mister M 13233343 2022-05-30   50 Supermarket Pokemon 14243454 2022-05-30   60 Supermarket of Dreams 15253580 2022-05-30     Dataframe 2:     id_account name cnpj create_date     10 Supermarket Carol 80502030 2022-05-30   20 Agency Angel 11213141 2022-05-30   30 Supermarket Mario Bros 12223242 2022-05-30   40 Supermarket Magical 60304050 2022-05-30   50 Supermarket Pokemon 14243454 2022-05-30   60 Supermarket of Dreams 90804050 2022-05-30     Expected output:     id_account name cnpj create_date     10 Supermarket Carol 80502030 2022-05-30   40 Supermarket Magical 60304050 2022-05-30   60 Supermarket of Dreams 90804050 2022-05-30     How can I do this? I've looked for a few ways, but I'm confused by the index.

Answer: If the data has same columns, but different number of rows, this is one possible solution: res = (pd.concat([df1,df2]) .drop_duplicates(keep=False) .drop_duplicates(subset='id_account', keep='last') )  Output:  id_account name cnpj create_date 0 10 Supermarket Carol 80502030 2022-05-30 3 40 Supermarket Magical 60304050 2022-05-30 5 60 Supermarket of Dreams 90804050 2022-05-30
Question: I am trying to unit test an inherited class for which its base class implements  __init_subclass__ method. Code is the following: quick_test.py import unittest from unittest.mock import create_autospec class Parent(): PROPERTY = NotImplemented def __init_subclass__(cls, **kwargs): if cls.PROPERTY is NotImplemented: raise NotImplementedError("Please implement the `PROPERTY`.") super().__init_subclass__(**kwargs) def __init__(self, connection_type="default"): self.connection_type = connection_type class Child(Parent): PROPERTY = "has value" class ChildNoProp(Parent): pass class TestClass(unittest.TestCase): def test_required_params(self): mock = create_autospec(Child) self.assertRaises(NotImplementedError, mock) if __name__ == '__main__': unittest.main()  The problem is I can't even reach the test case because ChildNoProp definition calls __init_subclass__ in base class and raises exception. Is there a way I can unit test this with current implementation, or should I scrap the error raising in __init_subclass__?

Answer: You can create the class inside the assertRaises block. with self.assertRaises(NotImplementedError): class ChildNoProp(Parent): pass  If the class declaration inside of a method makes you uncomfortable, you can use the type constructor directly. with self.assertRaises(NotImplementedError): type("ChildNoProp", (Parent,), {})
Question: I have a decorator that can be called either without or with arguments (all strings): @decorator def fct0(a: int, b: int) -> int: return a * b @decorator("foo", "bar") # any number of arguments def fct1(a: int, b: int) -> int: return a * b  I am having a hard time providing appropriate type hints so that type checkers will be able to properly validate the usage of the decorator, despite having read the related section of the doc of mypy. Here is what I have tried so far: from typing import overload, TypeVar, Any, Callable F = TypeVar("F", bound=Callable[..., Any]) @overload def decorator(arg: F) -> F: ... @overload def decorator(*args: str) -> Callable[[F], F]: ... def decorator(*args: Any) -> Any: # python code adapted from https://stackoverflow.com/q/653368 # @decorator -> shorthand for @decorator() if len(args) == 1 and callable(args[0]): return decorator()(args[0]) # @decorator(...) -> real implementation def wrapper(fct: F) -> F: # real code using `args` and `fct` here redacted for clarity return fct return wrapper  Which results in the following error from mypy: error: Overloaded function implementation does not accept all possible arguments of signature 1  I also have an error with pyright: error: Overloaded implementation is not consistent with signature of overload 1 Type "(*args: Any) -> Any" cannot be assigned to type "(arg: F@decorator) -> F@decorator" Keyword parameter "arg" is missing in source  I am using python 3.10.4, mypy 0.960, pyright 1.1.249.

Answer: The issue comes from the first overload (I should have read the pyright message twice!): @overload def decorator(arg: F) -> F: ...  This overload accepts a keyword parameter named arg, while the implementation does not! Of course this does not matter in the case of a decorator used with the @decorator notation, but could if it is called like so: fct2 = decorator(arg=fct). Python >= 3.8 The best way to solve the issue would be to change the first overload so that arg is a positional-only parameter (so cannot be used as a keyword argument): @overload def decorator(arg: F, /) -> F: ...  With support for Python < 3.8 Since positional-only parameters come with Python 3.8, we cannot change the first overload as desired. Instead, let's change the implementation to allow for a **kwargs parameter (an other possibility would be to add a keyword arg parameter). But now we need to handle it properly in the code implementation, for example: def decorator(*args: Any, **kwargs: Any) -> Any: if kwargs: raise TypeError("Unexpected keyword argument") # rest of the implementation here