eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: In Python 3.12 we have type aliases like this: Python 3.12.4+ (heads/3.12:99bc8589f0, Jul 27 2024, 11:20:07) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> type S = str >>> S S  By this syntax I assumed that, from now, the type word is considered a keyword, but it's not: >>> type = 2 >>>  and also: >>> import keyword >>> keyword.iskeyword('type') False 

Answer: The PEG parser introduced in Python 3.9 is a lot more flexible than the old parser, so it's just capable of handling this kind of thing. Trying to make type a keyword would have broken too much existing code, so they just... didn't. match/case is a similar story - making those keywords would have broken way too much code, such as everything that uses re.match. async used to be treated similarly, although since it was introduced back in 3.5, they had to use a tokenizer hack to get it to work - the parser wasn't powerful enough to handle the problem on its own.
Question: I wanted Pyinstaller on my Debian machine, so I ran the following command: sudo pip3 install pyinstaller This returned the following error: error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification.  So then I ran this: sudo apt install python3-pyinstaller That returned this: Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package python3-pyinstaller  I don't know what else to try. pip has been doing this, but usually using apt, I can install Python packages pretty easily.

Answer: As the error message states, you would need to install through apt to install globally, if that's not available for whaterver reason, the next best thing is to install that package into a virtual environment of some sort (e.g. with virtualenv env and then source env/bin/activate to activate the virtual environment). In my case there are some packages I would definitely not install in any other way and that I want to be able to use without activating any virtual environment whatsoever (e.g. bpytop, ps_mem) so just as the error says just pass --break-system-packages as you run pip3 install pyinstaller. To clarify, the command should look like this: pip3 install pyinstaller --break-system-packages
Question: So I am learning Django at in advanced level. I already know how to include templates from BASE_DIR where manage.py is located. However I wanted to know how to locate templates in the specific app in Django. For example, I have a project named mysite and an app called polls. I then specify the templates directory in settings.py: DIRS=[os.path.join(BASE_DIR, "templates")]  However, I do not know how to set a templates directory specific to the polls app. This is the project structure: . ├── db.sqlite3 ├── manage.py ├── mysite │ ├── asgi.py │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── polls │ ├── admin.py │ ├── apps.py │ ├── __init__.py │ ├── migrations │ │ ├── __init__.py │ ├── models.py │ ├── static │ │ ├── images │ │ ├── scripts.js │ │ └── style.css │ ├── tests.py │ ├── urls.py │ └── views.py └── templates ├── polls │ └── index.html └── static ├── images ├── scripts.js └── style.css  I wanted it to look like this. . ├── db.sqlite3 ├── manage.py ├── mysite │ ├── asgi.py │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── polls │ ├── admin.py │ ├── apps.py │ ├── __init__.py │ ├── migrations │ │ ├── __init__.py │ ├── models.py │ ├── static │ │ ├── images │ │ ├── scripts.js │ │ └── style.css │ ├── templates <- New │ │ ├── polls │ | │ ├── index.html │ ├── tests.py │ ├── urls.py │ └── views.py └── templates ├── polls │ └── index.html └── static ├── images ├── scripts.js └── style.css 

Answer: If you have the app specific templates located at polls/templates/polls/, then you need to add that path to TEMPLATES in settings.py as shown below: TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [os.path.join(BASE_DIR, 'polls/templates')], # here the app template path is set 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, }, ]  And while rendering template the path should have a folder prefix like render(request, 'polls/index.html', context)  UPDATE: as in above settings APP_DIRS is set to True, django will automatically look into <yourapp>/<templates> directory for the templates even if the app template paths are not explicitly specified.
Question: I'm encountering an issue when trying to plot a simple graph using Matplotlib in Python while in debug mode in VSCode. The plot works perfectly in normal execution mode, but when I set a breakpoint and run the code in debug mode, the plot window becomes unresponsive. Example Code: import matplotlib.pyplot as plt # Sample data x = [1, 2, 3, 4, 5] y = [10, 20, 25, 30, 40] # Plotting the data plt.plot(x, y) plt.title('Sample Plot') plt.xlabel('x-axis') plt.ylabel('y-axis') # Show plot plt.show()  In normal execution mode, the plot window appears and is responsive. In debug mode with a breakpoint, the plot window appears but becomes unresponsive and eventually throws a "not responding" error.  I tried using different Matplotlib backends, but none of them resolved the issue. I expect to be able to plot graphs using Matplotlib in debug mode vscode without the plot window becoming unresponsive.

Answer: I also had this issue. I got it working again by running Python 3.11.9 instead of 3.12.*, and I also had to switch to matplotlib 3.8.3 instead of 3.9.*. Has been driving me crazy for a couple weeks. So, try setting up a virtual environment in VSCode with matplotlib 3.8.3 and switching your interpreter to 3.11.9 in that environment.
Question: I trying to run an Python file today and got this error below! Anyone know what's issue and the solution to fix it? Traceback (most recent call last): File "C:\Users\Al PC\PycharmProjects\Fe\report_auto-final-v2.7.py", line 60, in <module> driver = webdriver.Chrome(service=chrome_service, options=options) # ChromeDriverManager() File "C:\Users\Al PC\PycharmProjects\SocialMedia\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__ super().__init__( File "C:\Users\Al PC\PycharmProjects\SocialMedia\venv\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 53, in __init__ self.service.start() File "C:\Users\Al PC\PycharmProjects\SocialMedia\venv\lib\site-packages\selenium\webdriver\common\service.py", line 105, in start self._start_process(self._path) File "C:\Users\Al PC\PycharmProjects\SocialMedia\venv\lib\site-packages\selenium\webdriver\common\service.py", line 206, in _start_process self.process = subprocess.Popen( File "C:\Users\Al PC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 966, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\Al PC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1435, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, OSError: [WinError 193] %1 is not a valid Win32 application Process finished with exit code 1 

Answer: If you construct the service like this: service = ChromeService(ChromeDriverManager().install())  I noticed ChromeDriverManager().install() returns <path stuff>\chromedriver-win32\THIRD_PARTY_NOTICES.chromedriver instead of the chromedrive.exe. This works for me: chrome_install = ChromeDriverManager().install() folder = os.path.dirname(chrome_install) chromedriver_path = os.path.join(folder, "chromedriver.exe") service = ChromeService(chromedriver_path)
Question: undetected_chromedriver with webdriver_manager was working well few days ago for scraping websites but out of nowhere it started throwing the error: OSError: [Errno 8] Exec format error: '/Users/pd/.wdm/drivers/chromedriver/mac64/127.0.6533.72/chromedriver-mac-x64/THIRD_PARTY_NOTICES.chromedriver'  I am guessing it is related to recent update of webdriver_manager. This is the code: import undetected_chromedriver as uc from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.support import expected_conditions as EC def get_driver(): options = uc.ChromeOptions() # options.add_argument("--headless") options.add_argument("--no-sandbox") options.add_argument("--disable-dev-sim-usage") options.add_argument("--start-maximized") options.add_argument('--disable-popup-blocking') driver = uc.Chrome(driver_executable_path=ChromeDriverManager().install(), options=options, version_main=116) driver.maximize_window() return driver  It would be really great if someone can help me on this, Thanks.

Answer: The command ChromeDriverManager().install():  creates a new folder without the executable and it retrieves the wrong file.   First, you need to remove the .wdm folder and then reinstall webdriver-manager: Windows Location: r"C:\Users\{user}\.wdm" Linux Location: /home/{user}/.wdm Mac Location: /Users/{user}/.wdm rm -rf /home/user/.wdm pip uninstall webdriver-manager pip install webdriver-manager  Now, after executing ChromeDriverManager().install(), you should only see a single folder with the executable:  It check if there is really a chromedriver executable inside this folder. Second, it makes a correction to the file name: if 'THIRD_PARTY_NOTICES.chromedriver' in chromedriver_path: chromedriver_path = chromedriver_path.replace('THIRD_PARTY_NOTICES.chromedriver', 'chromedriver')
Question: I'm fairly new to programming as a whole and I'm trying to learn using Polars with the polars_talib library. However when I import polars_talib I get the following error: ModuleNotFoundError Traceback (most recent call last) Cell In[19], line 8 6 import mintalib as mt 7 import pandas_ta as pta ----> 8 from polars.utils.udfs import _get_shared_lib_location 9 import polars_talib as plt ModuleNotFoundError: No module named 'polars.utils'  And I can't for the life of me figure out why I'm getting this message. I expected the library to import without error, tried Googling for a fix but couldn't find anything useful. I've reinstalled Polars with all optional dependencies and polars ta_lib, still nothing.

Answer: This issue has been fixed in the latest version of the library. Please update to version 0.1.3, which now supports Polars v1, to resolve the problem.
Question: i am currently filtering some data and ran into trouble, when filtering smaller frequencies from a large trend.The Reverse FFTs seem to have large spikes at the beginning and the ending. Here is the Data before and after filtering smaller frequencies. I have looked into the mathematic phenomenon and it is called the Gibbs phenomenon. Is there a way around this to clear the data of some overlying frequencies without getting this effect. Or is there even a workaround to keep the spikes as small as possible. Here is the code BTW: fourier_transformation= np.fft.fft(Sensor_4) frequencies = np.fft.fftfreq(len(time), d=1/Values_per_second) fourier_transformation[np.abs(frequencies) > 0.18] = 0 Sensor_4 = np.fft.ifft(fourier_transformation) 

Answer: Following the suggestion of Martin Brown's comment, the following code subtracts a ramp before FFT and adds it back after IFFT (I needed to make up my own values for Sensor_4, Values_per_second, and time, as the corresponding variables were missing in the question, so you might need to tune the parameters to match your actual signal): import matplotlib.pyplot as plt import numpy as np # Create signal of Sensor_4 (similar to question) T = 600 Values_per_second = 100 # Just to have a value here time = np.arange(0, T, 1/Values_per_second) Sensor_4 = 2 * np.sin(.25 * (2 * np.pi * time / T)) - 2.25 Sensor_4 += .1 * np.sin(1.9 * (2 * np.pi * time / T)) Sensor_4 += .1 * np.sin(2 * np.pi * time * 0.18) frequencies = np.fft.fftfreq(len(time), d=1/Values_per_second) # Original version fourier_transformation = np.fft.fft(Sensor_4) fourier_transformation[np.abs(frequencies) > 0.18] = 0 filtered_no_ramp = np.real(np.fft.ifft(fourier_transformation)) # Using ramp, as suggested in # https://stackoverflow.com/questions/78788533#comment138912162_78788533 # (1) Create ramp, (2) subtract ramp, (3) FFT, (4) filter, (5) IFFT, (6) add ramp back ramp = np.linspace(Sensor_4[0], Sensor_4[-1], len(Sensor_4)) # (1) fourier_transformation = np.fft.fft(Sensor_4 - ramp) # (2, 3) fourier_transformation[np.abs(frequencies) > 0.18] = 0 # (4) filtered_ramp = np.real(np.fft.ifft(fourier_transformation)) + ramp # (5, 6) # Show result ax1 = plt.subplot(311, title="unfiltered") plt.plot(time, Sensor_4) plt.subplot(312, sharey=ax1, title="w/o ramp") plt.plot(time, filtered_no_ramp) plt.subplot(313, sharey=ax1, title="with ramp") plt.plot(time, filtered_ramp) plt.show()  Here is the result, before filtering (top), filtering without (center) and with (bottom) ramp subtraction:
Question: I'm developing a small website in Flask that relies on data from a CSV file to output data to a table on the frontend using JQuery. The user would select an ID from a drop-down on the front-end, then a function would run on the back-end where the ID would be used as a filter on the table to return data. The data returned would usually just be a single column from the dataframe as well. The usual approach, from my understanding, would be to load the CSV data into a SQLite DB on startup and query using SQL methods in python at runtime. However, in my case, the table is 15MB in size (214K rows) and will never grow past that point. All the data will be as is for the duration of the Apps lifecycle. As such, would it be easier and less hassle to just load the dataframe table into memory and just filter on a copy of it when requests come in? Is that scalable or am I just kicking a can down the road? Example: app = Flask(__name__) dir_path = os.path.abspath(os.path.dirname(__file__)) with app.app_context(): print("Writing DB on startup") query_df = pd.read_csv(dir_path+'/query_file.csv') @app.route('/getData', methods=["POST"]) def get_data(): id = request.get_json() print("Getting rows....") data_list = sorted(set(query_df[query_df['ID'] == id]['Name'].tolist())) return jsonify({'items': data_list, 'ID': id})  This may be a tad naive on my end but I could not find a straight answer for my particular use-case.

Answer: This line of code can be made much faster without adding any new dependencies, just by using the tools that Pandas gives you. data_list = sorted(set(query_df[query_df['ID'] == id]['Name'].tolist()))  The following optimizations can be made:  sorted() can be replaced by pre-sorting the dataframe. set() can be replaced by dropping duplicates with the same ID and Name. query_df[query_df['ID'] == id] requires searching the entire dataframe for matching ID values, and can be replaced with an index.  To prepare the dataframe, on the startup of your program, after reading the dataframe with read_csv(), you would do the following: name_lookup_series = query_df \ .sort_values(['ID', 'Name']) \ .drop_duplicates(['ID', 'Name']) \ .set_index('ID')['Name']  To look up any particular value, you would do the following: name_lookup_series.loc[[id_to_look_up]].tolist()  Benchmarking this, it is roughly 100x faster, using the following benchmark program: import pandas as pd import numpy as np np.random.seed(92034) N = 200000 df = pd.DataFrame({ 'ID': np.random.randint(0, N, size=N), 'Name': np.random.randint(0, N, size=N), }) df['ID'] = 'ID' + df['ID'].astype('str') df['Name'] = 'N' + df['Name'].astype('str') print("Test dataframe") print(df) id_to_look_up = np.random.choice(df['ID']) print("Looking up", id_to_look_up) print("Result, method 1", sorted(set(df[df['ID'] == id_to_look_up]['Name'].tolist()))) %timeit sorted(set(df[df['ID'] == id_to_look_up]['Name'].tolist())) name_lookup_series = df.copy() \ .sort_values(['ID', 'Name']) \ .drop_duplicates(['ID', 'Name']) \ .set_index('ID')['Name'] print("Result, method 2", name_lookup_series.loc[[id_to_look_up]].tolist()) %timeit name_lookup_series.loc[[id_to_look_up]].tolist()
Question: When I am trying to load a .pt file i am seeing the following error, str1='Dataset/ALL_feats_cgqa.pt' m = torch.load(str1)  the error is as follows,  File "/home/Storage1/pythonCodeArea/train.py", line 21, in load_embeddings m = torch.load(str1) File "/home/.local/lib/python3.10/site-packages/torch/serialization.py", line 1040, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/home/.local/lib/python3.10/site-packages/torch/serialization.py", line 1262, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.  I have no idea about this error. Any help will be highly appreciated. I have gone through these references without any solution, [1], [2], [3], [4].

Answer: My guess is: your .pt file is most likely broken/corrupted. Most of the references that you posted in your question hint at the same cause. One can reproduce the problem as follows (that isn't to say that your .pt file was produced this way, but rather is intended to show that a corrupted file can trigger exactly the message that you saw): import torch from pathlib import Path # TODO: Adjust location as suitable pt_file = Path("~/test.pt").expanduser() pt_file.write_text("v") # Write nothing but 'v' to .pt file torch.load(pt_file) # Raises "UnpicklingError: invalid load key, 'v'."  My suggestions for further diagnoses and steps would be:  Check the size of your .pt file: Is it reasonably large? For example, if it holds the weights of some relatively modern neural network, its size should probably be on the order of megabytes or gigabytes, rather than bytes or kilobytes.  Can you open the .pt file with an archive manager? Using torch.save() with recent PyTorch versions (≥1.6) saves .pt files as zip archives by default. This means if the file is not corrupted and if it was created this way, you should be able to open it with an archive manager (maybe you need to change the suffix to .zip first).  Look at the contents of your .pt file with a text editor or hex editor. What do you see?  Finally, try to get the .pt file again from its source (re-download, re-copy, etc.) or from a backup if this is possible; or else try to recreate it yourself (by re-training your model).
Question: There's a cloud function in Python that processes some data when a file is uploaded to firebase's bucket: @storage_fn.on_object_finalized(bucket = "my-bucket", timeout_sec = timeout_sec, memory = memory, cpu = cpu, region='us-central1') def validate_file_upload(event: storage_fn.CloudEvent[storage_fn.StorageObjectData]): process(event)  When it's deployed via firebase cli the function works properly firebase deploy --only functions:validate_file_upload However, when the same function is deployed via gcloud gcloud functions deploy validate_file_upload --gen2 --region=us-central1 ....... --entry-point=validate_file_upload --trigger-event-filters="type=google.cloud.storage.object.v1.finalized" --trigger-event-filters="bucket=my-bucket"  When function is triggered, it fails with  TypeError: validate_file_upload() takes 1 positional argument but 2 were given  The reason is that when function is deployed viafirebase, GCP sends 'Eventarc' object as single argument to cloud function, but if it's deployed via gcloud it sends two: (data, context) and naturally it causes exception Even in the documentation it states there should be only 1 argument:  A Cloud Storage trigger is implemented as a CloudEvent function, in which the Cloud Storage event data is passed to your function in the CloudEvents format  https://cloud.google.com/functions/docs/calling/storage How to make sure that gcloud deployment uses correct function prototype?

Answer: The answer is to use double-decorators: import functions_framework from cloudevents.http import CloudEvent from firebase_functions import storage_fn, options @functions_framework.cloud_event @storage_fn.on_object_finalized(bucket = "my-bucket", timeout_sec = timeout_sec, memory = memory, cpu = cpu, region='us-central1') def validate_file_upload(event: CloudEvent): process(event)  This way it'll work regardless of being deployed via gcloud or firebase. Also I've noticed that if the function was originally deployed with HTTP triggers as opposed to "bucket" trigger, it GCP it stays marked as HTTP untill you delete it and redeploy anew. Simple redeployment leaves this function marked as HTTP (altough in Triggers tab it'll show Event for buckets as expected)
Question: Ok, so I'm working with this ENDF data, see here. Sometimes in the files they have what is quite possibly the most annoying encoding of scientific notation floating point numbers I have ever seen1. There it is often used that instead of 1.234e-3 it would be something like 1.234-3 (omitting the "e"). Now I've seen a library that simply changes - into e- or + into e+ by a simple substitution. But that doesn't work when some of the numbers can be negative. You end up getting some nonsense like e-5.122e-5 when the input was -5.122-5. So, I guess I need to move onto regex? I'm open to another solution that's simpler but its the best I can think of right now. I am using the re python library. I can do a simple substitution where I look for [0-9]-[0-9] and replace that like this: import re str1='-5.634-5' x = re.sub('[0-9]-[0-9]','4e-5',str1) print(x)  But obviously this won't work generally because I need to get the numerals before and after the - to be what they were, not just something I made up... I've used capturing groups before but what would be the fastest way in this context to use a capturing group for the digits before and after the - and feed it back into the substitution using the Python regex library import re? 1 Yes, I know, fortran...80 characters...save space...punch cards...nobody cares anymore.

Answer: Probably wouldn't reach for regex for this, when some simple string ops should work: s.replace("-", "e-").replace("+", "e+").lstrip("e")
Question: I am trying to create a new column using apply in polars. For this, I tried the following operation: df = df.with_columns( pl.col("AH_PROC_REALIZADO") .apply(get_procedure_description) .alias("proced_descr") )  But I'm getting the error:  AttributeError: 'Expr' object has no attribute 'apply'  The function I am trying to apply looks as follows. def get_procedure_description(cod): proceds = { '41612': 'CIRURGIA ONCOLOGICA', '30401': 'RADIOTERAPIA', ... } for proced in proceds.keys(): if cod[: len(proced)] == proced: return proceds[proced] return None 

Answer: pl.Expr.apply was deprecated in favour of pl.Expr.map_elements in Polars release 0.19.0. Recently, pl.Expr.apply was removed in the release of Polars 1.0.0. You can adapt your code to the new version as follows. df.with_columns( pl.col("AH_PROC_REALIZADO") .map_elements(get_procedure_description, return_dtype=pl.String) .alias("proced_descr") )