eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: It is my understanding that NumPy dropped support for using the Accelerate BLAS and LAPACK at version 1.20.0. According to the release notes for NumPy 1.21.1, these bugs have been resolved and building NumPy from source using the Accelerate framework on MacOS >= 11.3 is now possible again: https://numpy.org/doc/stable/release/1.21.0-notes.html, but I cannot find any documentation on how to do so. This seems like it would be an interesting thing to try and do because the Accelerate framework is supposed to be highly-optimized for M-series processors. I imagine the process is something like this:  Download numpy source code folder and navigate to this folder. Make a site.cfg file that looks something like:  [DEFAULT] library_dirs = /some/directory/ include_dirs = /some/other/directory/ [accelerate] libraries = Accelerate, vecLib   Run python setup.py build  The problem is I do not know 1. what the variables library_dirs and include_dirs should be so that NumPy knows to use Accelerate BLAS and LAPACK and 2. if there are any other additional steps that need to be taken. If anyone knows how to do this or can provide any insight, it would be greatly appreciated.

Answer: I actually attempted this earlier today and these are the steps I used:  In the site.cfg file, put  [accelerate] libraries = Accelerate, vecLib   Build with NPY_LAPACK_ORDER=accelerate python3 setup.py build  Install with python3 setup.py install   Afterwards, np.show_config() returned the following blas_mkl_info: NOT AVAILABLE blis_info: NOT AVAILABLE openblas_info: NOT AVAILABLE accelerate_info: extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers'] extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)] blas_opt_info: extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers'] extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)] lapack_mkl_info: NOT AVAILABLE openblas_lapack_info: NOT AVAILABLE openblas_clapack_info: NOT AVAILABLE flame_info: NOT AVAILABLE lapack_opt_info: extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers'] extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)] Supported SIMD extensions in this NumPy install: baseline = NEON,NEON_FP16,NEON_VFPV4,ASIMD found = ASIMDHP,ASIMDDP not found =  and my quick test suggest significant performance boost relative to OpenBlas.
Question: I have a rust and python project that I am building using Maturin(https://github.com/PyO3/maturin). It says that it requires a pyproject.toml file for the python dependencies. I have a dependency of uvloop, which is not supported on windows and arm devices. I have added the code that conditionally imports these packages. However, I do not know how to conditionally install these packages. Right now, these packages are getting installed by default on every OS. Here is the pyproject.toml file. [project] name = "robyn" dependencies = [ "watchdog>=2.1.3,<3", "uvloop>=0.16.0,<0.16.1", "multiprocess>=0.70.12.2,<0.70.12.3" ]  And the github link, jic anyone is interested: https://github.com/sansyrox/robyn/pull/94/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711R21

Answer: The syntax for environment markers is specified in PEP 508 – Dependency specification for Python Software Packages. I will show below how to exclude uvloop as a dependency on Windows platform with a marker for platform.system() which returns:  "Linux" on Linux "Darwin" on macOS "Windows" on Windows  Using pyproject.toml: [project] dependencies = [ 'uvloop ; platform_system != "Windows"', ]  Using setup.cfg: [options] install_requires = uvloop ; platform_system != "Windows"  Using setup.py: setup( install_requires=[ 'uvloop ; platform_system != "Windows"', ] )
Question: I want to be able to upload a file from Flask to Supabase Storage, but it only has documentation for the javascript api link to docs. Also, I can't find any examples or any open source project that does that. Here it is my function to upload: def upload_file(self): if 'file' not in request.files: flash('No file part') return redirect('/') file = request.files['file'] if file.filename == '': flash('No selected file') return redirect('/') filename = secure_filename(file.filename) # upload to supabase storage return file.path 

Answer: from storage3 import create_client url = "https://<your_supabase_id>.supabase.co/storage/v1" key = "<your api key>" headers = {"apiKey": key, "Authorization": f"Bearer {key}"} storage_client = create_client(url, headers, is_async=False) def upload_file(self): if 'file' not in request.files: flash('No file part') return redirect('/') file = request.files['file'] if file.filename == '': flash('No selected file') return redirect('/') filename = secure_filename(file.filename) buckets = storage_client.list_buckets() bucket = buckets[0] return bucket.upload(filename, file)  I didn't find official docs for python upload. Didn't test the code above so appreciate any feedback. I've based this on the github repo https://github.com/supabase-community/storage-py and the docs here https://supabase-community.github.io/storage-py/api/bucket.html
Question: I just started using Google colab for my projects and I tried to create and parse text files. But I don't quite understand how the file directory works here. Below are my questions:  On the left navigation pane (in the picture) that show the list of folders and files. Are they in my drive, if they are, where they are located? How can I find? How do I create/upload a file and store in google drive in order to parse it in my app? I have created one file in the folder "Contents" yesterday. But I woke up this morning, it's gone. Can anyone explain what happened?  Appreciate your time, thank you.  

Answer: The google colab folders are temporary and they will disappear after 8 hours I think. You need to save them to your mounted google drive location. The content folder is part of colab and will be deleted. You need to mount google drive to your Colab session. from google.colab import drive drive.mount('/content/gdrive')  Then you can write to google drive like with open('/content/gdrive/My Drive/file.txt', 'w') as f: f.write('content')  Or even save stuff like pandas files to csv there like df.to_csv('/content/gdrive/My Drive/file.csv')  You can also read files from there like this import pandas as pd df = pd.read_csv('/content/gdrive/My Drive/file.csv')  All of this will only work after you mount the drive of course.
Question: Getting an error for IndexError: invalid index to scalar variable on the yolo_layers line. network = cv2.dnn.readNetFromDarknet('yolov3.cfg', 'yolov3.weights') layers = network.getLayerNames() yolo_layers = [layers[i[0] - 1] for i in network.getUnconnectedOutLayers()]  This code won't work on my Jupyter notebook but will run fine on google collab. No idea why. Could be my python version?

Answer: It's may caused by the different versions of cv2. The version of cv2 module with CUDA support will give you a 2-D array when calling network.getUnconnectedOutLayers(). However, the version without CUDA support will give a 1-D array. You may try to take the brackets out which closing the index 0.
Question: I have a df with this structure: id a1_l1 a2_l1 a3_l1 a1_l2 a2_l2 a3_l2 1 1 5 3 1 2 3 2 1 5 3 1 2 3 3 2 5 3 5 5 3 4 5 5 3 5 5 3 5 5 5 2 6 5 5 2 7 5 5 2 8 2 5 2 9 3 5 1 10 3 5 1  I want to summarize in a table such that I get:  l1 l2 a1 0.4 0.5 a2 1 0.5 a3 0 0  In which what I'm doing is counting how may times 5 was present divided by the number of valid responses, so that for example: a1, l1 is equal to .4 as I have 4 values of 5 divided by 10. and a2, l1 equals .5 as I have 2 values of 5 divided by 4 valid responses per column. Thanks!

Answer: You can reshape to have a dataframe with MultiIndex, then perform a simple division of the (sum of the truthy values equal to 5) by not na. Finally, unstack: df2 = df.set_index('id') df2.columns = df2.columns.str.split('_', expand = True) df2 = (df2.eq(5).sum()/df2.notna().sum()).unstack()  output:  l1 l2 a1 0.4 0.5 a2 1.0 0.5 a3 0.0 0.0
Question: I want if the conditions are true if df[df["tg"] > 10 and df[df["tg"] < 32 then multiply by five otherwise divide by two. However, I get the following error  ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().  d = {'year': [2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001], 'day': [1, 2, 3, 4, 1, 2, 3, 4,], 'month': [1, 1, 1, 1, 2, 2, 2, 2], 'tg': [10, 11, 12, 13, 50, 21, -1, 23], 'rain': [1, 2, 3, 2, 4, 1, 2, 1]} df = pd.DataFrame(data=d) print(df) [OUT] year day month tg rain 0 2001 1 1 10 1 1 2001 2 1 11 2 2 2001 3 1 12 3 3 2001 4 1 13 2 4 2001 1 2 50 4 5 2001 2 2 21 1 6 2001 3 2 -1 2 7 2001 4 2 23 1 df["score"] = (df["tg"] * 5) if ((df[df["tg"] > 10]) and (df[df["tg"] < 32])) else (df["tg"] / 2) [OUT] ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().  What I want  year day month tg rain score 0 2001 1 1 10 1 5 1 2001 2 1 11 2 55 2 2001 3 1 12 3 60 3 2001 4 1 13 2 65 4 2001 1 2 50 4 25 5 2001 2 2 21 1 42 6 2001 3 2 -1 2 0.5 7 2001 4 2 23 1 46 

Answer: You can use where: df['score'] = (df['tg']*5).where(df['tg'].between(10, 32), df['tg']/5)
Question: I inherited some code that I need to rework since it is using deprecated imp module instead of importlib To test the functionality I have created a simple test script: # test.py def main(): print("main()") if __name__ == "__main__": print("__main__") main()  When I run that with the old code (below as a minimal example) # original imp based minimal example import imp import os import site script = r"./test.py" script_dir = os.path.dirname(script) script_name = os.path.splitext(os.path.basename(script))[0] site.addsitedir(script_dir) fp, pathname, description = imp.find_module(script_name, [script_dir]) imp.load_source('__main__', pathname, fp)  Would output the following: __main__ main()  Now I have some trouble mimicking this verbatim using the importlib module since I do not get the output as above with the following code: # my try so far using importlib import importlib import os import site script = os.path.abspath(r"./test.py") script_dir = os.path.dirname(script) script_name = os.path.splitext(os.path.basename(script))[0] site.addsitedir(script_dir) spec = importlib.util.spec_from_file_location(script_name, script) module = importlib.util.module_from_spec(spec) sys.modules[script_name] = module spec.loader.exec_module(module)  Instead it outputs nothing. :-( Any suggestions on how to mimic the old (imp) behavior using importlib?

Answer: Eventually I solved it like below based on https://github.com/epfl-scitas/spack/blob/a60ae07083a5744607064221a0cd48204a54394e/lib/spack/llnl/util/lang.py#L598-L625: if sys.version_info[0] == 3: if sys.version_info[1] >= 5: import importlib.util spec = importlib.util.spec_from_file_location(module_name, module_path) module = importlib.util.module_from_spec(spec) spec.loader.exec_module(module) elif sys.version_info[1] < 5: import importlib.machinery loader = importlib.machinery.SourceFileLoader(module_name, module_path) loader.load_module() elif sys.version_info[0] == 2: import imp imp.load_source(module_name, module_path)
Question: When Running Installation, pip install busio  Getting ERROR, ERROR: Could not find a version that satisfies the requirement busio (from versions: none) ERROR: No matching distribution found for busio  Python version is 3.7.3.

Answer: There is a module called "busio" (GitHub) by Adafruit which they describe as providing “hardware-driven interfaces for I2C, SPI, UART”. This can be installed via Adafruit’s Blinka package: pip3 install adafruit-blinka
Question: I have a WSL instance, Ubuntu 20.04 and I have created another Ubuntu 18.04 WSL instance. I installed Poetry on the 20.04 without issues. I am trying to install Poetry on the Ubuntu 18.04 instance, using the curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 - command. At the moment, my $HOME env var points to /home/fromzeroedu. However, after installation, Poetry is installed on my Windows user home: $ which poetry /mnt/c/Users/j/.poetry/bin/poetry  And if I try getting the version, I get: $ poetry --version /usr/bin/env: ‘python\r’: Permission denied  I even tried setting the POETRY_HOME prior to installation: export POETRY_HOME=/home/fromzeroedu/.poetry/bin/poetry  But Poetry still installs in the Windows user directory. Sometimes I love Poetry...

Answer: That was because bash didn't knew where to look for the bin so it found only the Windows executable (PATH is shared between wsl and windows) to solve it you needed to add the following to your ~/.bashrc (preferably on top) export PATH="$HOME/.poetry/bin:$PATH"  With the new installer (poetry 1.1.7 onward) the Bin path changed export PATH="$HOME/.local/bin:$PATH"  I faced this issue cause I'm using autocomplete plugin for oh-my-zsh and poetry needs to be added to PATH before the plugin is loaded. but the installation script append it at the end.
Question: I am trying to efficiently compute a summation of a summation in Python:  WolframAlpha is able to compute it too a high n value: sum of sum. I have two approaches: a for loop method and an np.sum method. I thought the np.sum approach would be faster. However, they are the same until a large n, after which the np.sum has overflow errors and gives the wrong result. I am trying to find the fastest way to compute this sum. import numpy as np import time def summation(start,end,func): sum=0 for i in range(start,end+1): sum+=func(i) return sum def x(y): return y def x2(y): return y**2 def mysum(y): return x2(y)*summation(0, y, x) n=100 # method #1 start=time.time() summation(0,n,mysum) print('Slow method:',time.time()-start) # method #2 start=time.time() w=np.arange(0,n+1) (w**2*np.cumsum(w)).sum() print('Fast method:',time.time()-start) 

Answer: (fastest methods, 3 and 4, are at the end) In a fast NumPy method you need to specify dtype=np.object so that NumPy does not convert Python int to its own dtypes (np.int64 or others). It will now give you correct results (checked it up to N=100000). # method #2 start=time.time() w=np.arange(0, n+1, dtype=np.object) result2 = (w**2*np.cumsum(w)).sum() print('Fast method:', time.time()-start)  Your fast solution is significantly faster than the slow one. Yes, for large N's, but already at N=100 it is like 8 times faster: start=time.time() for i in range(100): result1 = summation(0, n, mysum) print('Slow method:', time.time()-start) # method #2 start=time.time() for i in range(100): w=np.arange(0, n+1, dtype=np.object) result2 = (w**2*np.cumsum(w)).sum() print('Fast method:', time.time()-start)  Slow method: 0.06906533241271973 Fast method: 0.008007287979125977  EDIT: Even faster method (by KellyBundy, the Pumpkin) is by using pure python. Turns out NumPy has no advantage here, because it has no vectorized code for np.objects. # method #3 import itertools start=time.time() for i in range(100): result3 = sum(x*x * ysum for x, ysum in enumerate(itertools.accumulate(range(n+1)))) print('Faster, pure python:', (time.time()-start))  Faster, pure python: 0.0009944438934326172  EDIT2: Forss noticed that numpy fast method can be optimized by using x*x instead of x**2. For N > 200 it is faster than pure Python method. For N < 200 it is slower than pure Python method (the exact value of boundary may depend on machine, on mine it was 200, its best to check it yourself): # method #4 start=time.time() for i in range(100): w = np.arange(0, n+1, dtype=np.object) result2 = (w*w*np.cumsum(w)).sum() print('Fast method x*x:', time.time()-start)
Question: I have tried pip install scipy and everything appears fine, going through the path I opened the files and couldn't find any mention of the bootstrap library despite it being on their website: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bootstrap.html After looking on Github https://github.com/scipy/scipy/blob/master/scipy/stats/_bootstrap.py I can see there was an update 5 days ago although I last ran the code three days ago with no issues 

Answer: I had this issue and solved it by re-installing the scipy package with pip install -U scipy in order to upgrade to version 1.7
Question: I cannot access a web server on localhost port 5000 on macOS v12 (Monterey) (Flask or any other). E.g., use the built-in HTTP server, I cannot get onto port 5000: python3 -m http.server 5000 ... (stack trace) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socketserver.py", line 466, in server_bind self.socket.bind(self.server_address) OSError: [Errno 48] Address already in use  If you have Flask installed and you run the Flask web server, it does not fail on start. Let's take the minimum Flask example code: # Save as hello.py in the current working directory. from flask import Flask app = Flask(__name__) @app.route("/") def hello_world(): return "<p>Hello, World!</p>"  Then run it (provided you have Flask/Python 3 installed): export FLASK_APP=hello flask run  Output: * Running on http://127.0.0.1:5000/  However, if you try to access this server (from a browser or with anything else), it is denied: curl -I localhost:5000 HTTP/1.1 403 Forbidden Content-Length: 0 Server: AirTunes/595.13.1 

Answer: macOS Monterey introduced AirPlay Receiver running on port 5000. This prevents your web server from serving on port 5000. Receiver already has the port. You can either:  turn off AirPlay Receiver, or; run the server on a different port (normally best).  Turn off AirPlay Receiver Go to System Preferences → Sharing → Untick Airplay Receiver.  See more details You should be able to rerun the server now on port 5000 and get a response: python3 -m http.server 5000 Serving HTTP on :: port 5000 (http://[::]:5000/) ...  Run the server on a different port than 5000 It's probably a better idea to no longer use port 5000 as that's reserved for Airplay Receiver on macOS Monterey. Just to run the server on a different port. There isn't any need to turn off Airplay Receiver. python3 -m http.server 4999  or export FLASK_APP=hello flask run -p 4999
Question: I have a problem that started very recently. The Docker Alpine Python library is not installable any more: apk update && apk upgrade && apk add python  fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz v3.14.2-119-g9c4e1aa60c [https://dl-cdn.alpinelinux.org/alpine/v3.14/main] v3.14.2-120-g90167408c8 [https://dl-cdn.alpinelinux.org/alpine/v3.14/community] OK: 14943 distinct packages available (1/1) Upgrading alpine-keys (2.3-r1 -> 2.4-r0) 0% 2% # 5% ## 7% ### 10% #### 12% ##### 15% ###### 17% ####### 20% ######### 23% ########## 25% ########### 28% ############ 30% ############# 33% ############## 35% ############### 38% ################ 41% ################## 43% ################### 46% #################### 48% ##################### 51% ###################### 53% ####################### % ############################################OK: 7 MiB in 16 packages ERROR: unable to select packages: python (no such package): required by: world[python] Exited with code exit status 1 

Answer: You are trying to use the python (alias) library instead of python3. Try to use apk update && apk upgrade && apk add python3 instead.
Question: I'm very new to python/dash/plotly and I keep getting the same error:  ImportError: cannot import name 'dcc' from partially initialized module 'dash' (most likely due to a circular import)  Does anyone know how to fix this? I've imported the following: from dash import dcc from dash import html from dash.dependencies import Input, Output import plotly.io as pio 

Answer: "most likely due to a circular import": this is probably due to your file being named as a dash or as a module name. But I got the error message ImportError: cannot import name 'dcc' from 'dash' For me reinstalling dash fixed the issue.  pip3 uninstall dash pip3 install dash
Question: I am using Python to perform a Fast Fourier Transform on some data. I then need to extract the locations of the peaks in the transform in the form of the x-values. Right now I am using Scipy's fft tool to perform the transform, which seems to be working. However, when i use Scipy's find_peaks I only get the y-values, not the x-position that I need. I also get the warning: ComplexWarning: Casting complex values to real discards the imaginary part  Is there a better way for me to do this? Here is my code at the moment: import pandas as pd import matplotlib.pyplot as plt from scipy.fft import fft from scipy.signal import find_peaks headers = ["X","Y"] original_data = pd.read_csv("testdata.csv",names=headers) x = original_data["X"] y = original_data["Y"] a = fft(y) peaks = find_peaks(a) print(peaks) plt.plot(x,a) plt.title("Fast Fourier transform") plt.xlabel("Frequency") plt.ylabel("Amplitude") plt.show() 

Answer: There seem to be two points of confusion here:  What find_peaks is returning. How to interpret complex values that the FFT is returning.  I will answer them separately. Point #1 find_peaks returns the indices in "a" that correspond to peaks, so I believe they ARE values you seek, however you must plot them differently. You can see the first example from the documentation here. But basically "peaks" is the index, or x value, and a[peaks] will be the y value. So to plot all your frequencies, and mark the peaks you could do: plt.plot(a) plt.plot(peaks, a[peaks])  Point #2 As for the second point, you should probably read up more on the output of FFTs, this post is a short summary but you may need more background to understand it. But basically, an FFT will return an array of complex numbers, which contains both phase and magnitude information. What you are currently doing is implicitly only looking at the real part of the solution (hence the warning that the imaginary portion is being discarded), what you probably want instead to take the magnitude of your "a" array, but without more information on your application it is impossible to say.
Question: I have installed Python 3.10 from deadsnakes on my Ubuntu 20.04 machine. How to use it? python3 --version returns Python 3.8.10 and python3.10 -m venv venv returns error (I've installed python3-venv as well).

Answer: python3.10 --version will work. python3-venv is for 3.8, so install python3.10-venv. For reference: deadsnakes packages for 3.10 for Focal.