eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to install pyautogui, but pip keeps throwing errors. How to fix it? I've tried installing libffi library. Here is some code: python3 -m pip install pyautogui Defaulting to user installation because normal site-packages is not writeable Collecting pyautogui Using cached PyAutoGUI-0.9.50.tar.gz (57 kB) ERROR: Command errored out with exit status 1: command: /usr/local/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sxm4ewnq/pyautogui/setup.py'"'"'; __file__='"'"'/tmp/pip-install-sxm4ewnq/pyautogui/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-85ugzov6 cwd: /tmp/pip-install-sxm4ewnq/pyautogui/ Complete output (11 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python3.8/site-packages/setuptools/__init__.py", line 19, in <module> from setuptools.dist import Distribution File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 34, in <module> from setuptools import windows_support File "/usr/local/lib/python3.8/site-packages/setuptools/windows_support.py", line 2, in <module> import ctypes File "/usr/local/lib/python3.8/ctypes/__init__.py", line 7, in <module> from _ctypes import Union, Structure, Array ModuleNotFoundError: No module named '_ctypes' ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.  That's from python REPL >>> sys.path ['', '/home/walenty/apps/Python-3.8.5/Modules/_ctypes', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/home/walenty/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/site-packages'] >>> import _ctypes Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named '_ctypes' 

Answer: Required Install foreign function interface headers sudo apt install libffi-dev Reinstall Python Substitute desired python version Ubuntu sudo add-apt-repository ppa:deadsnakes/ppa -y && sudo apt install --reinstall python3.9-distutils MacOS Use brew install python3.9 or port install python3.9 (I recommend port) Windows Use Microsoft Store Specify project python version Poetry poetry env use 3.9 Virtual envs virtualenv -p python3.9 myproject etc...
Question: The keras Conv2D layer does not come with an activation function itself. I am currently rebuilding the YOLOv1 model for practicing. In the YOLOv1 model, there are several Conv2D layers followed by activations using the leaky relu function. Is there a way to combine from keras.layers import Conv2D, LeakyReLU ... def model(input): ... X = Conv2D(filters, kernel_size)(X) X = LeakyReLU(X) ...  into a single line of code, like X = conv_with_leaky_relu(X)? I think it should be similar to def conv_with_leaky_relu(*args, **kwargs): X = Conv2D(*args, **kwargs)(X) X = LeakyReLU(X) return X  but this of course doesn't work because it is undefined what X ist. Any ideas?

Answer: You can just pass it as an activation: X = Conv2D(filters, kernel_size, activation=LeakyReLU())(X)
Question: Info: I am using OpenAI Gym to create RL environments but need multiple copies of an environment for something I am doing. I do not want to do anything like [gym.make(...) for i in range(2)] to make a new environment. Question: Given one gym env what is the best way to make a copy of it so that you have 2 duplicate but disconnected envs? Here is an example: import gym env = gym.make("CartPole-v0") new_env = # NEED COPY OF ENV HERE env.reset() # Should not alter new_env 

Answer: Astariul has an updated answer:: Their answer states: import copy env_2 = copy.deepcopy(env)  For more info about 'copy.deepcopy', and the copy library Link to copy library documentation
Question: How to filter the specific warning for specific module in python? MWE ERROR: cross_val_score(model, df_Xtrain,ytrain,cv=2,scoring='r2') /usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_ridge.py:148: LinAlgWarning: Ill-conditioned matrix (rcond=3.275e-20): result may not be accurate. overwrite_a=True).T  My attempt import warnings warnings.filterwarnings(action='ignore',module='sklearn') cross_val_score(model, df_Xtrain,ytrain,cv=2,scoring='r2') This works but I assume it ignores all the warnings such as deprecation warnings. I would like to exclude only LinAlgWarning  Required Like import warnings warnings.filterwarnings(action='ignore', category='LinAlgWarning', module='sklearn')  Is there a way like this filter specific warning?

Answer: You have to pass category as a WarningClass not in a String: from scipy.linalg import LinAlgWarning warnings.filterwarnings(action='ignore', category=LinAlgWarning, module='sklearn')
Question: I noticed that passing Python objects to native code with ctypes can break mutability expectations. For example, if I have a C function like: int print_and_mutate(char *str) { str[0] = 'X'; return printf("%s\n", str); }  and I call it like this: from ctypes import * lib = cdll.LoadLibrary("foo.so") s = b"asdf" lib.print_and_mutate(s)  The value of s changed, and is now b"Xsdf". The Python docs say "You should be careful, however, not to pass them to functions expecting pointers to mutable memory.". Is this only because it breaks expectations of which types are immutable, or can something else break as a result? In other words, if I go in with the clear understanding that my original bytes object will change, even though normally bytes are immutable, is that OK or will I get some kind of nasty surprise later if I don't use create_string_buffer like I'm supposed to?

Answer: Python makes assumptions about immutable objects, so mutating them will definitely break things. Here's a concrete example: >>> import ctypes as c >>> x = b'abc' # immutable string >>> d = {x:123} # Used as key in dictionary (keys must be hashable/immutable) >>> d {b'abc': 123}  Now build a ctypes mutable buffer to the immutable object. id(x) in CPython is the memory address of the Python object and sys.getsizeof() returns the size of that object. PyBytes objects have some overhead, but the end of the object has the bytes of the string. >>> sys.getsizeof(x) 36 >>> px=(c.c_char*36).from_address(id(x)) >>> px.raw b'\x02\x00\x00\x00\x00\x00\x00\x000\x8fq\x0b\xfc\x7f\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\xf0\x06\xe61\xeb\x00\x1b\xa9abc\x00' >>> px.raw[-4:] # last bytes of the object b'abc\x00' >>> px[-4] b'a' >>> px[-4] = b'y' # Mutate the ctypes buffer, mutating the "immutable" string >>> x # Now it has a modified value. b'ybc'  Now try to access the key in the dictionary. Keys are located in O(1) time using its hash, but the hash was on the original, "immutable" value so it is incorrect. The key can no longer be found by old or new value: >>> d # Note that dictionary key changed, too. {b'ybc': 123} >>> d[b'ybc'] # Try to access the key Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: b'ybc' >>> d[b'abc'] # Maybe original key will work? It hashes same as the original... Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: b'abc'
Question: I have a code that gives me a scatter plot of predicted vs actual values as a function of concentration. The data is pulled from an excel csv spreadsheet. This is the code: import matplotlib.pyplot as plt from numpy import loadtxt dataset = loadtxt("ColorPlot.csv", delimiter=',') x = dataset[:,0] y = dataset[:,1] z = dataset[:,2] scaled_z = (z - z.min()) / z.ptp() colors = plt.cm.viridis(scaled_z) sc=plt.scatter(x, y, c=colors) plt.clim(0, 100) plt.colorbar() plt.xlabel("Actual") plt.ylabel("Predicted") plt.show()  And with this I get a nice graph:  However if I change the color to something like colors = plt.cm.plasma(scaled_z)  I get the graph below but the colorbar remains unchanged.  I've tried lots of different things like cmap or edgecolors but I don't know how to change it. And I want to keep the code as simple as it currently is because I want to readily change the third variable of z based on my excel spreadsheet data. Is there also a way for the scale of the colorbar to pick up what the scale is from the excel spreadsheet without me manually specifying 0-100?

Answer: To get the right color bar, use the following code: colormap = plt.cm.get_cmap('plasma') # 'plasma' or 'viridis' colors = colormap(scaled_z) sc = plt.scatter(x, y, c=colors) sm = plt.cm.ScalarMappable(cmap=colormap) sm.set_clim(vmin=0, vmax=100) plt.colorbar(sm) plt.xlabel("Actual") plt.ylabel("Predicted") plt.show()  For my random generated data I got the following plot:  Now replace 'plasma' with 'viridis' and check the other variant.
Question: The Python Language Reference specifies object.__await__ as follows:  object.__await__(self) Must return an iterator. Should be used to implement awaitable objects. For instance, asyncio.Future implements this method to be compatible with the await expression.  That's it. I find this specification very vague and not very specific (ironically). Ok, it should return an iterator, but can it be an arbitrary iterator? Obviously not: import asyncio class Spam: def __await__(self): yield from range(10) async def main(): await Spam() asyncio.run(main())  RuntimeError: Task got bad yield: 0  I'm assuming the asyncio event loop expects a specific kind of object being yielded by the iterator. Then what exactly should it yield? (And why isn't this documented?)  Edit: as far as I can see, this isn't documented anywhere. But I've been investigating on my own, and I think that the key to understanding what objects the asyncio expects its coroutines to yield lies in task_step_impl in _asynciomodule.c.  Update: I've made a PR to the cpython repository with the aim of clarifying this: "Clarify the vague specification of object.__await__". The PR has now been merged and should be available in the docs for Python 3.10+.

Answer: The language doesn't care which iterator you return. The error comes from a library, asyncio, which has specific ideas about the kind of values that must be produced by the iterator. Asyncio requires __await__ to produce asyncio futures (including their subtypes such as tasks) or None. Other libraries, like curio and trio, will expect different kinds of values. Async libraries by and large don't document their expectations from __await__ because they consider it an implementation detail. As far as asyncio is concerned, you're supposed to be using higher-level constructs, such as futures and tasks, and await those, in addition to coroutines. There is rarely a need to implement __await__ manually, and even then you should use it to delegate the signals of another awaitable. Writing an __await__ that creates and yields a fresh suspend-value of its own requires it to be coupled with the event loop and have knowledge of its internals. You can think of __await__ as a tool to write a library similar to asyncio. If you are the author of such a library, the current specification is sufficient because you can yield whatever you like from the iterator, only the code in your event loop will observe the yielded values. If you're not in that position, you probably have no need to implement __await__.
Question: I have a simple FastAPI demo app which achieve a function: get different response json by calling a post api named changeResponse. The changeResponse api just changed a global variable, another api return different response through the same global variable. On local env, it works correctly, but the response always changes after i just call changeResponse once, when i build this on docker.The code is as follows: from typing import Optional from fastapi import FastAPI from util import read_json import enum app = FastAPI() type = "00" @app.post("/changeResponse") async def handle_change_download_response(param:Optional[str]): global type type = param print("type is "+type) return {"success":"true"} @app.post("/download") async def handle_download(param:Optional[str]): print("get download param: "+param) if legalDownload(param): print("type is "+type) return read_json.readDownloadSuccessRes(type) else: return read_json.readDownloadFailRes() def legalDownload(data:str)->bool: return True  the dockerfile is as follows: FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7 COPY ./app /app  what i except: call changeResponse param is 7, get response for 7, call changeResponse param is 8, get response for 8. what i get: call changeResponse param is 7, get reponse for 7, call changeReponse 8, sometime the response is 7, sometime is 8, impossible to predict

Answer: tiangolo/uvicorn-gunicorn-fastapi is based on uvicorn-gunicorn-docker image, which by defaults creates multiple workers. Excerpt from gunicorn_conf.py: default_web_concurrency = workers_per_core * cores Thus, the described situation arises because the request is processed by different workers (processes). Each of which has its own copy of the global variable Update: If you want to change the count of workers, use the following environment variables:  WORKERS_PER_CORE: It will set the number of workers to the number of CPU cores multiplied by this value. MAX_WORKERS: You can use it to let the image compute the number of workers automatically but making sure it's limited to a maximum. WEB_CONCURRENCY Override the automatic definition of number of workers.  You can set it like: docker run -d -p 80:80 -e WEB_CONCURRENCY="2" myimage  A more detailed description of these variables and examples here  If you want to share data between workers, pay attention to this topic.
Question: I've got multiple python programs that connect to Mt5 terminal using the following code. # Establish connection to the MetaTrader 5 terminal if not mt5.initialize("C:\\Program Files\\ICMarkets - MetaTrader 5 - 01\\terminal64.exe"): print("initialize() failed, error code =", mt5.last_error()) quit()  The python module for MT5 is the one here - https://www.mql5.com/en/docs/integration/python_metatrader5 The problem I have is that, when multiple programs connect to the same MT5 terminal.exe, the performance degrades & one or more python programs exit with errors. To overcome this, I have installed multiple copies of MT5 & have updated the python code such that different copies of python program use different installations of MT5. However, only the first installation of MT5 is the only one that can be invoked by all the python programs. Trying to use any other terminal.exe from other installation raises an exception & the connection fails. There isn't much on the internet either to troubleshoot this. If anyone has ideas to tackle this or has solved the problem, I'd be keen to hear from you please. The error as such is - initialize() failed, error code = (-10003, "IPC initialize failed, Process create failed 'C:\\Program Files\\ICMarkets - MetaTrader 5 - 02\terminal64.exe'")  This could be something to do with Windows's default pointing to the first installation or something like that that you wouldn't even think about. Just thinking aloud here.

Answer: From my experience, imho, the MT5 python API was not designed to handle multiple connections from the same machine simultaneously. I have overcome this by creating Virtual Machines and running everything through them. I used Oracle VM because it's free, I had past experience with it, but its not very good at sharing resources. If your machine is not very strong you might want to look into some other solution. I heard Docker is good at sharing the host resources.
Question: I've just upgraded from Fedora 32 to Fedora 33 (which comes with Python 3.9). Since then gcloud command stopped working: [guy@Gandalf32 ~]$ gcloud Error processing line 3 of /home/guy/.local/lib/python3.9/site-packages/XStatic-1.0.2-py3.9-nspkg.pth: Traceback (most recent call last): File "/usr/lib64/python3.9/site.py", line 169, in addpackage exec(line) File "<string>", line 1, in <module> File "<frozen importlib._bootstrap>", line 562, in module_from_spec AttributeError: 'NoneType' object has no attribute 'loader' Remainder of file ignored Traceback (most recent call last): File "/usr/lib64/google-cloud-sdk/lib/gcloud.py", line 104, in <module> main() File "/usr/lib64/google-cloud-sdk/lib/gcloud.py", line 62, in main from googlecloudsdk.core.util import encoding File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/__init__.py", line 23, in <module> from googlecloudsdk.core.util import importing File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/core/util/importing.py", line 23, in <module> import imp File "/usr/lib64/python3.9/imp.py", line 23, in <module> from importlib import util File "/usr/lib64/python3.9/importlib/util.py", line 2, in <module> from . import abc File "/usr/lib64/python3.9/importlib/abc.py", line 17, in <module> from typing import Protocol, runtime_checkable File "/usr/lib64/python3.9/typing.py", line 26, in <module> import re as stdlib_re # Avoid confusion with the re we export. File "/usr/lib64/python3.9/re.py", line 124, in <module> import enum File "/usr/lib64/google-cloud-sdk/lib/third_party/enum/__init__.py", line 26, in <module> spec = importlib.util.find_spec('enum') AttributeError: module 'importlib' has no attribute 'util' 

Answer: Update from GCP support GCP support mentioned that the new version 318.0.0 released on 2020.11.10 should support python 3.9 I updated my gcloud sdk to 318.0.0 and now looks like python 3.9.0 is supported. To fix this issue run gcloud components update Fedora 33 includes python 2.7 and to force GCloud SDK to use it please set this environment variable export CLOUDSDK_PYTHON=python2  You can add this export command to your ~/.bash_profile Python 3.9 is very new and is expected that Gcloud SDK does not support 3.9, it is written to be compatible with 2.7.x & 3.6 - 3.8 (3.8 can cause some compat issues I recommend to use 3.7) As a workaround, configure Python 3.8 or 3.7 (these versions work well for Gcloud and most of linux distros) as system wide interpreter and try to use gcloud commands.
Question: In Python 3.9 we can use type hinting in a lowercase built-in fashion (without having to import type signatures from the typing module) as described here: def greet_all(names: list[str]) -> None: for name in names: print("Hello", name)  I like very much this idea and I would like to know if it is possible to use this way of type hinting but in previous versions of python, such as Python 3.7, where we have write type hinting like this: from typing import List def greet_all(names: List[str]) -> None: for name in names: print("Hello", name) 

Answer: Simply, import annotations from __future__ and you should be good to go. from __future__ import annotations import sys !$sys.executable -V #this is valid in iPython/Jupyter Notebook def greet_all(names: list[str]) -> None: for name in names: print("Hello", name) greet_all(['Adam','Eve']) Python 3.7.6 Hello Adam Hello Eve
Question: I assume due to the fact Big Sur is sparkling new hotfixes for the new OS have not yet happen. When attempting to install modules that use clang for compilation, the following error is thrown: clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=11.0'  Currently running:  Mac OS Big Sur, 11.0 Beta Intel CPU (i386) Python 3.8.0 installed via pyenv  Multiple modules have clang dependencies, and so it seems this error is quite common. An example: pip install multidict  Installing older versions of Command Line Tools (e.g. 11.5) does not work as well.

Answer: Figure out the issue on my end. Previously I had installed XCode from the App Store (11.7) and set its SDKs as my default: sudo xcode-select --switch /Applications/Xcode.app/  However, it seems this come with an unsupported version of clang:  λ clang --version Apple clang version 11.0.3 (clang-1103.0.32.62) Target: x86_64-apple-darwin20.1.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin  Setting the xcode-select to the latest version via: sudo xcode-select --switch /Library/Developer/CommandLineTools   EDIT (11/15/2020) You might receive an error when attempting the above change: xcode-select: error: invalid developer directory '/Library/Developer/CommandLineTools' To fix this, you must install the latest Command Line Tools from the official Apple website here. At the time of writting this edit, I installed the Command Line Tools for Xcode 12.3 beta.  Changes clang to a working version:  λ clang --version Apple clang version 12.0.0 (clang-1200.0.32.2) Target: x86_64-apple-darwin20.1.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin  The built-in Big Sur SDK is version 10.15, which seems to work without an issue:  λ ls /Library/Developer/CommandLineTools/SDKs MacOSX.sdk MacOSX10.15.sdk  After the switch, multidict was installed successfully.  λ pip install multidict Collecting multidict Downloading multidict-4.7.6-cp38-cp38-macosx_10_14_x86_64.whl (48 kB) |████████████████████████████████| 48 kB 589 kB/s Installing collected packages: multidict Successfully installed multidict-4.7.6  Further investigation seems to indicate this is a design choice by Apple (source):  Therefore, ensuring your SDK is the default out-of-the-box as opposed to XCode's new SDK should be enough for the system to switch context when needed (and seems to work fine with pip+clang).
Question: I want to use a DecisionTreeRegressor for multi-output regression, but I want to use a different "importance" weight for each output (e.g. predicting y1 accurately is twice as important as predicting y2). Is there a way of including these weights directly in the DecisionTreeRegressor of sklearn? If not, how can I create a custom MSE criterion with different weights for each output in sklearn?

Answer: I am afraid you can only provide one weight-set when you fit https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html#sklearn.tree.DecisionTreeRegressor.fit And the more disappointing thing is that since only one weight-set is allowed, the algorithms in sklearn is all about one weight-set. As for custom criterion: There is a similar issue in scikit-learn https://github.com/scikit-learn/scikit-learn/issues/17436 Potential solution is to create a criterion class mimicking the existing one (e.g. MAE) in https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_criterion.pyx#L976 However, if you see the code in detail, you will find that all the variables about weights are "one weight-set", which is unspecific to the tasks. So to customize, you may need to hack a lot of code, including:  hacking the fit function to accept a 2D array of weights https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_classes.py#L142  Bypassing the checking (otherwise continue to hack...)  Modify tree builder to allow the weights https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_tree.pyx#L111 It is terrible, there are a lot of related variable, you should change double to double*  Modify Criterion class to accept a 2-D array of weights https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_criterion.pyx#L976  In init, reset and update, you have to keep attributions such as self.weighted_n_node_samples specific to outputs (tasks).   TBH, I think it is really difficult to implement. Maybe we need to raise an issue for scikit-learn group.