eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Consider this function that makes a simple GET request to an API endpoint: import httpx def check_status_without_session(url : str) -> int: response = httpx.get(url) return response.status_code  Running this function will open a new TCP connection every time the function check_status_without_session is called. Now, this section of HTTPX documentation recommends using the Client API while making multiple requests to the same URL. The following function does that: import httpx def check_status_with_session(url: str) -> int: with httpx.Client() as client: response = client.get(url) return response.status_code  According to the docs using Client will ensure that:  ... a Client instance uses HTTP connection pooling. This means that when you make several requests to the same host, the Client will reuse the underlying TCP connection, instead of recreating one for every single request.  My question is, in the second case, I have wrapped the Client context manager in a function. If I call check_status_with_session multiple times with the same URL, wouldn't that just create a new pool of connections each time the function is called? This implies it's not actually reusing the connections. As the function stack gets destroyed after the execution of the function, the Client object should be destroyed as well, right? Is there any advantage in doing it like this or is there a better way?

Answer:  Is there any advantage in doing it like this or is there a better way?  No, there is no advantage using httpx.Client in the way you've shown. In fact the httpx.<method> API, e.g. httpx.get, does exactly the same thing! The "pool" is a feature of the transport manager held by Client, which is HTTPTransport by default. The transport is created at Client initialization time and stored as the instance property self._transport. Creating a new Client instance means a new HTTPTransport instance, and transport instances have their own TCP connection pool. By creating a new Client instance each time and using it only once, you get no benefit over using e.g. httpx.get directly. And that might be OK! Connection pooling is an optimization over creating a new TCP connection for each request. Your application may not need that optimization, it may be performant enough already for your needs. If you are making many requests to the same endpoint in a tight loop, iterating within the context of the loop may give you some throughput gains, e.g. with httpx.Client(base_url="https://example.com") as client: results = [client.get(f"/api/resource/{idx}") for idx in range(100)]  For such I/O-heavy workloads you may do even better by executing results in parallel, e.g. using httpx.AsyncClient.
Question: What's wrong with this code: split_list = [3600, 3600, 3600, 3600, 3600, 3600, 3600, 3600, 45] split_list2 = [None, None, None, None, None, None, None, None, None, None, None, None] result = [3600, 3600, 3600, 3600, 3600, 3600, 3600, 3600, 45, None, None, None] for i in range(len(split_list)): split_list2[i] = split_list[i]  In PyCharm it issues a warning;  Unexpected type(s): (int, int) Possible type(s): (SupportsIndex, None) (slice, Iterable[None])  But the script runs just fine and this code works exactly as I expected. I don't like warnings in my IDE though, any quick fixes?

Answer: This warning is solved by updating to PyCharm to 2021.2.2. It seems to be a bug in earlier versions of the IDE's static type checker. One user reported in the comments that this bug regressed in the PyCharm 2021.2.3 release. I just tested it again using PyCharm 2022.1 Professional Edition and the bug has again been solved. Here's a screenshot:
Question: Since updating to Python 3.9.8, we get an error while using Black in our CI pipeline. black....................................................................Failed - hook id: black - exit code: 1 Traceback (most recent call last): File "../.cache/pre-commit/repol9drvp84/py_env-python3/bin/black", line 5, in <module> from black import patched_main File "../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/black/__init__.py", line 52, in <module> from typed_ast import ast3, ast27 File "../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/typed_ast/ast3.py", line 40, in <module> from typed_ast import _ast3 ImportError: ../.cache/pre-commit/repol9drvp84/py_env-python3/lib/python3.9/site-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape  The error can be easily reproduced with: % pip install typed_ast % python3 -c 'from typed_ast import ast3' Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: /usr/lib/python3/dist-packages/typed_ast/_ast3.cpython-39-x86_64-linux-gnu.so: undefined symbol: _PyUnicode_DecodeUnicodeEscape  Currently the only workaround is downgrading to Python 3.9.7. Is another fix available? See also Bug#998854: undefined symbol: _PyUnicode_DecodeUnicodeEscape

Answer: The initial error was a failing Python Black pipeline. Black failed because it was pinned to an older version which now fails with Python 3.9.8. Updating Black to the latest version, 21.10b0, fixed the error for me. See also typed_ast issue #169:  For others who may find this in a search, I ran into this problem via black because I had black pinned to an older version. The current version of black appears to no longer use typed-ast and thus won't encounter this issue.  Using the latest typed-ast version >=1.5.0 seems to work as well. E.g., pip install typed-ast --upgrade
Question: I want to install Python 3.10 on Ubuntu 18.04 (I'm currently on Python 3.8) from the deadsnakes repository with the following set of commands I found on the internet: sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.10  But I got the error sudo: add-apt-repository: command not found. More net research led me to this set of commands at "ModuleNotFoundError: No module named 'apt_pkg'" appears in various commands - Ask Ubuntu: sudo apt remove python3-apt sudo apt autoremove sudo apt autoclean sudo apt install python3-apt  Other web sources said the same thing, so I did that, but I still get the error message when I run sudo add-apt-repository ppa:deadsnakes/ppa. Then I found How to Fix 'add-apt-repository command not found' on Ubuntu & Debian - phoenixNAP, which advised this set of commands: sudo apt update sudo apt install software-properties-common sudo apt update  so I did that, but when I run sudo add-apt-repository ppa:deadsnakes/ppa I now get this error message: ~$ sudo add-apt-repository ppa:deadsnakes/ppa Traceback (most recent call last): File "/usr/bin/add-apt-repository", line 12, in <module> from softwareproperties.SoftwareProperties import SoftwareProperties, shortcut_handler File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 28, in <module> import apt_pkg ModuleNotFoundError: No module named 'apt_pkg'  I have found some web links that show a wide variety of solutions with earlier versions of Python. I'm currently on Python 3.8. Before I do anything more I want to ask what is the best way to solve the ModuleNotFoundError: No module named 'apt_pkg' error message when trying to install the deadsnakes repository to install Python 3.10, given the number of possible solutions I have seen. Thanks very much.

Answer: This worked for me: sudo apt-get install python3-apt --reinstall cd /usr/lib/python3/dist-packages sudo cp apt_pkg.cpython-38-x86_64-linux-gnu.so apt_pkg.so  The 38 in the filename above can be different for you.
Question: I am trying to implement a soap client for a server that uses NTLM authentication. The libraries that I use (requests-ntlm2 which relies on ntlm-auth) implement the MD4 algorithm that lies in the core of the NTLM protocol via the standard library's hashlib. Although hashlib seems to support MD4: >>> import hashlib >>> hashlib.algorithms_available {'md5-sha1', 'md4', 'shake_128', 'md5', 'blake2s', 'sha3_512', 'ripemd160', 'sha512', 'mdc2', 'blake2b', 'sha3_256', 'sha3_224', 'sha512_224', 'sha1', 'sha384', 'sha256', 'sha224', 'whirlpool', 'sha512_256', 'sha3_384', 'shake_256', 'sm3'} >>>  and so does the openssl library in my system: (victory) C:\code\python\services>openssl help: [...] Message Digest commands (see the `dgst' command for more details) blake2b512 blake2s256 md4 md5 mdc2 rmd160 sha1 sha224 sha256 sha3-224 sha3-256 sha3-384 sha3-512 sha384 sha512 sha512-224 sha512-256 shake128 shake256 sm3 [...]  when the authentication tries to run python produces an ValueError: unsupported hash type md4 error. Here is the relevant part of the traceback: C:\ProgramData\Miniconda3\envs\victory\lib\site-packages\ntlm_auth\compute_hash.py in _ntowfv1(password) 165 return nt_hash 166 --> 167 digest = hashlib.new('md4', password.encode('utf-16-le')).digest() 168 169 return digest C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __hash_new(name, data, **kwargs) 161 # This allows for SHA224/256 and SHA384/512 support even though 162 # the OpenSSL library prior to 0.9.8 doesn't provide them. --> 163 return __get_builtin_constructor(name)(data) 164 165 C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __get_builtin_constructor(name) 118 return constructor 119 --> 120 raise ValueError('unsupported hash type ' + name) 121 122 ValueError: unsupported hash type md4  Even when I try to merely call the MD4 from hashlib, I get the same result: >>> import hashlib >>> hashlib.new('md4') Traceback (most recent call last): File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 157, in __hash_new return _hashlib.new(name, data) ValueError: [digital envelope routines] initialization error During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 163, in __hash_new return __get_builtin_constructor(name)(data) File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 120, in __get_builtin_constructor raise ValueError('unsupported hash type ' + name) ValueError: unsupported hash type md4  Any insights about what's going on and/or any help would be immensely appreciated.

Answer: Well, it seems that there was something corrupted in my conda environment. I created a new identical one, and it's been working ever since without having to change anything else.
Question: According to the discussions on PyTorch forum :  What’s the difference between nn.ReLU() and nn.ReLU(inplace=True)? Guidelines for when and why one should set inplace = True?  The purpose of inplace=True is to modify the input in place, without allocating memory for additional tensor with the result of this operation. This allows to be more efficient in memory usage but prohibits the possibility to make a backward pass, at least if the operation decreases the amount of information. And the backpropagation algorithm requires to have intermediate activations saved in order to update the weights. Can one say, that this mode, should be turned on in layers only if the model is already trained, and one doesn't want to modify it anymore?

Answer: nn.ReLU(inplace=True) saves memory during both training and testing. However, there are some problems we may face when we use nn.ReLU(iplace=True) while calculating gradients. Sometimes, the original values are needed when calculating gradients. Because inplace destroys some of the original values, some usages may be problematic: def forward(self, x): skip = x x = self.relu(x) x += skip # inplace addition # Error!  The above two consecutive inplace operations will produce an error. However, it is fine to use first addition, then activation function with inplace=True: def forward(self, x): skip = x x += skip # inplace addition x = self.relu(x) # No error!
Question: Pandas operations usually create a copy of the original dataframe. As some answers on SO point out, even when using inplace=True, a lot of operations still create a copy to operate on. Now, I think I'd be called a madman if I told my colleagues that everytime I want to, for example, apply +2 to a list, I copy the whole thing before doing it. Yet, it's what Pandas does. Even simple operations such as append always reallocate the whole dataframe. Having to reallocate and copy everything on every operation seems like a very inefficient way to go about operating on any data. It also makes operating on particularly large dataframes impossible, even if they fit in your RAM. Furthermore, this does not seem to be a problem for Pandas developers or users, so much so that there's an open issue #16529 discussing the removal of the inplace parameter entirely, which has received mostly positive responses; some started getting deprecated since 1.0. It seems like I'm missing something. So, what am I missing? What are the advantages of always copying the dataframe on operations, instead of executing them in-place whenever possible? Note: I agree that method chaining is very neat, I use it all the time. However, I feel that "because we can method chain" is not the whole answer, since Pandas sometimes copies even in inplace=True methods, which are not meant to be chained. So, I'm looking some other answers for why this would be a reasonable default.

Answer: As evidenced here in the pandas documentation, "... In general we like to favor immutability where sensible." The Pandas project is in the camp of preferring immutable (stateless) objects over mutable (objects with state) to guide programmers into creating more scalable / parallelizable data processing code. They are guiding the users by making the 'inplace=False' behavior the default. In this software engineering stack exchange Peter Torok discusses the pros and cons between mutable and immutable object programming really nicely. https://softwareengineering.stackexchange.com/a/151735 In summary some software engineers feel that objects that are immutable (unchanging) lead to  less errors in the code - because object states are easy to lose track of and hard to track down increased scalability - it is easier to write multithreaded code, since one thread will not inadvertently modify the value contained by an object in another thread more concise code - since code is forced to be written in a functional programming and more mathematical style  I will agree that this does have it's inefficiencies since constantly making copies of the same objects for minor changes does not seem ideal. It has other benefits noted above.
Question: I'm trying to install sentence-transformers library. But when I import it, this error pops out: --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-1-c9f0b8c65221> in <module> ----> 1 import h5py ~\Anaconda3\envs\custom_env\lib\site-packages\h5py\__init__.py in <module> 32 raise 33 ---> 34 from . import version 35 36 if version.hdf5_version_tuple != version.hdf5_built_version_tuple: ~\Anaconda3\envs\custom_env\lib\site-packages\h5py\version.py in <module> 15 16 from collections import namedtuple ---> 17 from . import h5 as _h5 18 import sys 19 import numpy h5py\h5.pyx in init h5py.h5() ImportError: DLL load failed while importing defs: The specified procedure could not be found.  I have installed h5py library. What am I missing?

Answer: I had the same problem after a CUDA/CUdnn update that probably messed some DLLs. I did: pip uninstall h5py, which indicated a successful uninstall along with some error messages. And immediately after pip install h5py Sometimes simple solutions work.
Question: For the following line: print("{0: <24}".format("==> core=") + str(my_dict["core"]))  I am getting following warning message:  [consider-using-f-string] Formatting a regular string which could be a f-string [C0209]  Could I reformat it using f-string?

Answer: You could change the code to print(f"{'==> core=': <24}{my_dict['core']}"). The cast to string is implicit.
Question: ModuleNotFoundError Traceback (most recent call last) <ipython-input-14-2683ccd40dcb> in <module> 16 from omegaconf import OmegaConf 17 from PIL import Image ---> 18 from taming.models import cond_transformer, vqgan 19 import taming.modules 20 import torch ModuleNotFoundError: No module named 'taming'  I've tried everything can you please help me? I've tried putting it in the same folder and stuff so please help! 

Answer: Try the following command: pip install taming-transformers
Question: I have a set of data values for a scalar 3D function, arranged as inputs x,y,z in an array of shape (n,3) and the function values f(x,y,z) in an array of shape (n,). EDIT: For instance, consider the following simple function data = np.array([np.arange(n)]*3).T F = np.linalg.norm(data,axis=1)**2  I would like to convolve this function with a spherical kernel in order to perform a 3D smoothing. The easiest way I found to perform this is to map the function values in a 3D spatial grid and then apply a 3D convolution with the kernel I want. This works fine, however the part that maps the 3D function to the 3D grid is very slow, as I did not find a way to do it with NumPy only. The code below is my actual implementation, where data is the (n,3) array containing the 3D positions in which the function is evaluated, F is the (n,) array containing the corresponding values of the function and M is the (N,N,N) array that contains the 3D space grid. step = 0.1 # Create meshgrid xmin = data[:,0].min() xmax = data[:,0].max() ymin = data[:,1].min() ymax = data[:,1].max() zmin = data[:,2].min() zmax = data[:,2].max() x = np.linspace(xmin,xmax,int((xmax-xmin)/step)+1) y = np.linspace(ymin,ymax,int((ymax-ymin)/step)+1) z = np.linspace(zmin,zmax,int((zmax-zmin)/step)+1) # Build image M = np.zeros((len(x),len(y),len(z))) for l in range(len(data)): for i in range(len(x)-1): if x[i] < data[l,0] < x[i+1]: for j in range(len(y)-1): if y[j] < data[l,1] < y[j+1]: for k in range(len(z)-1): if z[k] < data[l,2] < z[k+1]: M[i,j,k] = F[l]  Is there a more efficient way to fill a 3D spatial grid with the values of a 3D function ?

Answer: For each item of data you're scanning pixels of cuboid to check if it's inside. There is an option to skip this scan. You could calculate corresponding indices of these pixels by yourself, for example: data = np.array([[1, 2, 3], #14 (corner1) [4, 5, 6], #77 (corner2) [2.5, 3.5, 4.5], #38.75 (duplicated pixel) [2.9, 3.9, 4.9], #47.63 (duplicated pixel) [1.5, 2, 3]]) #15.25 (one step up from [1, 2, 3]) step = 0.5 data_idx = ((data - data.min(axis=0))//step).astype(int) M = np.zeros(np.max(data_idx, axis=0) + 1) x, y, z = data_idx.T M[x, y, z] = F  Note that only one value of duplicated pixels is being mapped to M.
Question: Or if there are scrolling functionalities built in, to edit the scrolling settings? I tried this but it didn't work -- def test_exception(self, etype, value, tb, tb_offset=None): try: announce = Announce(etype, value) if announce.print: announce.title() announce.tips() announce.resources() announce.feedback() announce.scroll(self, etype, value, tb, tb_offset) #self.showtraceback((etype, value, tb), tb_offset=tb_offset) except: self.showtraceback((etype, value, tb), tb_offset=tb_offset) def scroll(self, etype, value, tb, tb_offset=None): b=widgets.HTML( value=self.showtraceback((etype, value, tb), tb_offset=tb_offset), placeholder='Some HTML', description='Some HTML', disabled=True ) a = HBox([b], layout=Layout(height='20px', overflow_y='10px')) display(a) 

Answer: Try going to cell > current outputs > toggle scrolling in the Jupyter UI to enable scrolling output for a cell.
Question: Hope you are doing well. I'm trying to send pdfs file from Nodejs to Flask using Axios. I read files from a directory (in the form of buffer array) and add them into formData (an npm package) and send an Axios request.  const existingFile = fs.readFileSync(path) console.log(existingFile) const formData = new nodeFormData() formData.append("file", existingFile) formData.append("fileName", documentData.docuName) try { const getFile = await axios.post("http://127.0.0.1:5000/pdf-slicer", formData, { headers: { ...formData.getHeaders() } }) console.log(getFile)} catch (e) {console.log(e, "getFileError")}  On flask side: I'm trying to get data from the request.  print(request.files) if (request.method == "POST"): file=request.form["file"] if file: print(file)  in request.file, I'm getting ImmutableMultiDict([]) but in request.form["file"], I'm getting data something like this:  how can I handle this type of file format or how can I convert this file format to python fileObject.

Answer: I solved this issue by updating my Nodejs code. We need to convert formData file into octet/stream format. so I did minor change in my formData code : before: formData.append("file", existingFile) after: formData.append("file", fs.createReadStream(existingFile)  Note: fs.createReadStream only accepts string or uint8array without null bytes. we cannot pass the buffer array.
Question: While importing torch (import torch) I'm facing the following error message: OSError: [WinError 127] The specified procedure could not be found. Error loading "C:\Users\myUserName\anaconda3\lib\site-packages\torch\lib\jitbackend_test.dll" or one of its dependencies. I tried the suggestion from this article but without success. Any ideas how to fix it? My environment:  NVIDIA GeForce GTX 1650 Windows 11 Cuda 11.5 Conda 4.10.3 Python 3.8.5 Torch 1.10 Microsoft Visual C++ Redistributable installed (https://aka.ms/vs/17/release/vc_redist.x64.exe) 

Answer: Fortunately, after extensive research, I found a solution. Someone suggested me to create a new conda environment. And that worked for me! Solution:  create new conda env by: conda create --name new-env install python: conda install python=3.8.5 run: conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch test cuda: import torch; print(torch.version.cuda); print(torch.cuda.is_available())
Question: I have a simple long-form dataset I would like to generate bar charts from. The dataframe looks like this: data = {'Year':[2019,2019,2019,2020,2020,2020,2021,2021,2021], 'Month_diff':[0,1,2,0,1,2,0,1,2], 'data': [12,10,13,16,12,18,19,45,34]} df = pd.DataFrame(data)  I would like to plot a bar chart that has 3 rows, each for 2019, 2020 and 2021. X axis being month_diff and data goes on Y axis. How do I do this? If the data was in different columns, then I could have just used this code: df.plot(x="X", y=["A", "B", "C"], kind="bar")  But my data is in a single column and ideally, I'd like to have different row for each year.

Answer: 1. seaborn.catplot The simplest option for a long-form dataframe is the seaborn.catplot wrapper, as Johan said: import seaborn as sns sns.catplot(data=df, x='Month_diff', y='data', row='Year', kind='bar', height=2, aspect=4)    2. pivot + DataFrame.plot Without seaborn:  pivot from long-form to wide-form (1 year per column) use DataFrame.plot with subplots=True to put each year into its own subplot (and optionally sharey=True)  (df.pivot(index='Month_diff', columns='Year', values='data') .plot.bar(subplots=True, sharey=True, legend=False)) plt.tight_layout()   Note that if you prefer a single grouped bar chart (which you alluded to at the end), you can just leave out the subplots param: df.pivot(index='Month_diff', columns='Year', values='data').plot.bar()    3. DataFrame.groupby + subplots You can also iterate the df.groupby('Year') object:  Create a subplots grid of axes based on the number of groups (years) Plot each group (year) onto its own subplot row  groups = df.groupby('Year') fig, axs = plt.subplots(nrows=len(groups), ncols=1, sharex=True, sharey=True) for (name, group), ax in zip(groups, axs): group.plot.bar(x='Month_diff', y='data', legend=False, ax=ax) ax.set_title(name) fig.supylabel('data') plt.tight_layout()