eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a list of subcategories and a dataframe. I want to filter out the dataframe on the basis of each subcategory of the list. lst = [7774, 29409, 36611, 77553] import pandas as pd data = {'aucctlg_id': [143424, 143424, 143424, 143388, 143388, 143430], 'catalogversion_id': [1, 1, 1, 1, 1, 1.2], 'Key': [1434241, 1434241, 1434241, 1433881, 1433881, 14343012], 'item_id': [4501118, 4501130, 4501129, 4501128, 4501127, 4501126], 'catlog_description': ['M&BP PIG IRON FA', 'M&BP PIG IRON FA', 'M&BP PIG IRON FA', 'PIG IRON MIXED OG','PIG IRON MIXED OG', 'P.S JAM & PIG IRON FINES'], 'catlog_start_date': ['17-05-2024 11:00:00', '17-05-2024 11:00:00', '17-05-2024 11:00:00', '17-05-2024 11:00:00','17-05-2024 11:00:00', '17-05-2024 11:00:00'], 'subcategoryid': [29409, 29409, 29409, 7774, 7774, 36611], 'quantity': [200, 200, 200, 180, 180, 100], 'auctionable': ['Y', 'Y', 'Y', 'Y' ,'Y' ,'Y'] } df = pd.DataFrame(data) print(df)  I have tried using the following code but I want output as a dataframe here it generates a list and of a single subcategory: new=[] for i in range(0, len(lst)): mask1 = df['subcategoryid']==(lst[i]) df2 = df.loc[mask1] new.append(df2)  Required Output files, with the filtered data: df_7774, df_29409, df_36611 

Answer: You could pre-filter with isin, then use groupby: lst = [7774, 29409, 36611, 77553] out = dict(list(df[df['subcategoryid'].isin(lst)].groupby('subcategoryid')))  Which will create a dictionary of the desired dataframes: {7774: aucctlg_id catalogversion_id Key item_id catlog_description catlog_start_date subcategoryid quantity auctionable 3 143388 1.0 1433881 4501128 PIG IRON MIXED OG 17-05-2024 11:00:00 7774 180 Y 4 143388 1.0 1433881 4501127 PIG IRON MIXED OG 17-05-2024 11:00:00 7774 180 Y, 29409: aucctlg_id catalogversion_id Key item_id catlog_description catlog_start_date subcategoryid quantity auctionable 0 143424 1.0 1434241 4501118 M&BP PIG IRON FA 17-05-2024 11:00:00 29409 200 Y 1 143424 1.0 1434241 4501130 M&BP PIG IRON FA 17-05-2024 11:00:00 29409 200 Y 2 143424 1.0 1434241 4501129 M&BP PIG IRON FA 17-05-2024 11:00:00 29409 200 Y, 36611: aucctlg_id catalogversion_id Key item_id catlog_description catlog_start_date subcategoryid quantity auctionable 5 143430 1.2 14343012 4501126 P.S JAM & PIG IRON FINES 17-05-2024 11:00:00 36611 100 Y}  If you want to create files without intermediate: lst = [7774, 29409, 36611, 77553] for k, g in df[df['subcategoryid'].isin(lst)].groupby('subcategoryid'): g.to_excel(f'df_{k}.xlsx')
Question: So my music school runs a festival for all student bands to play together. Since there are some family members across diferent bands, I'm trying to create a solution to optimize the schedule of the bands - trying to minimize sum of time between bands that have relatives. Now I think I've found a solution. Don't know if it's the best but it looks like it's working. Now I'm trying to find a better way to present the solution. This "x" matrix is huge and maybe it would be better to extract only the "1"s and make a schedule table with the slots and what band is in a cronological order. But I don't know where to start. Hoping someone can point the direction. Relevant infos:  English is not my first language; I'm not a programmer, and everything I did at the code bellow was what I was able to learn about python last 2 days.  I'm using the code bellow. Feel free to criticize and make any suggestions. Thanks! from gekko import GEKKO import numpy as np m = GEKKO() #variables and constrains n = 20 #n of bands s = 37 #n of slots to play t = 25 #time in minutes one band spend on the stage x = m.Array(m.Var,(n,s),value=0,lb=0,ub=1,integer=True) #matrix of all bands(rows) x slots (columns) for j in range(s): m.Equation(m.sum([x[i,j] for i in range(n)])<=1) #since this is the decision i made it binary and sum=1 to 1 band only ocuppie one slot for i in range(n): m.Equation(m.sum([x[i,j] for j in range(s)])==1) #since this is the decision i made it binary and sum=1 to 1 band only ocuppie one slot z = [k for k in range(1,s+1)] #array with slot index to use to calc time w = z*x*t #time the band will perform ;; used in objetive function #objective #in this exemple the band (1,4) and the band (1,5) have a relationship, and so do (3,2) and (1,15), so the objetive function bellow will try to minimize the sum of the time between bands that have relationship y = m.abs2(w.item(1, 4)-w.item(1, 5))+ m.abs2(w.item(3, 2)-w.item(1, 15)) m.Minimize(y) #solver m.options.SOLVER = 1 m.solve() #print(w.item((1, 4))) print('y') print(y) print('x') print(x) 

Answer: You need to extract the band and slot numbers for each 1 in the matrix and sort the schedule by slot number: schedule = [(i+1, j+1) for i in range(n) for j in range(s) if x[i,j][0] == 1] schedule.sort(key=lambda x: x[1]) for band, slot in schedule: print(f"Band {band} is scheduled to play in slot {slot}")
Question: I have a code with a one-liner while and a try-except statement which behaves weirdly. This prints 'a' on Ctrl+C: try: while True: pass except KeyboardInterrupt: print("a")  and this too: try: i = 0 while True: pass except KeyboardInterrupt: print("a")  but this doesn't, and it throws a traceback: try: while True: pass except KeyboardInterrupt: print("a")  and neither does this code: try: while True: pass i = 0 except KeyboardInterrupt: print("a")  Addition some additional details. In 3.11, the instruction JUMP_BACKWARD was added and seems invloved with this issue see: Disassembler for Python bytecode In 3.12 when the code in the first and the 3rd blocks are disassembled the results are: Cannot be caught:  0 0 RESUME 0 2 2 NOP 3 >> 4 JUMP_BACKWARD 1 (to 4) >> 6 PUSH_EXC_INFO 4 8 LOAD_NAME 0 (KeyboardInterrupt) 10 CHECK_EXC_MATCH 12 POP_JUMP_IF_FALSE 11 (to 36) 14 POP_TOP 5 16 PUSH_NULL 18 LOAD_NAME 1 (print) 20 LOAD_CONST 1 ('a') 22 CALL 1 30 POP_TOP 32 POP_EXCEPT 34 RETURN_CONST 2 (None) 4 >> 36 RERAISE 0 >> 38 COPY 3 40 POP_EXCEPT 42 RERAISE 1 ExceptionTable: 4 to 4 -> 6 [0] 6 to 30 -> 38 [1] lasti 36 to 36 -> 38 [1] lasti None  Can be caught:  0 0 RESUME 0 2 2 NOP 3 4 NOP 4 >> 6 NOP 3 8 JUMP_BACKWARD 2 (to 6) >> 10 PUSH_EXC_INFO 5 12 LOAD_NAME 0 (KeyboardInterrupt) 14 CHECK_EXC_MATCH 16 POP_JUMP_IF_FALSE 11 (to 40) 18 POP_TOP 6 20 PUSH_NULL 22 LOAD_NAME 1 (print) 24 LOAD_CONST 1 ('a') 26 CALL 1 34 POP_TOP 36 POP_EXCEPT 38 RETURN_CONST 2 (None) 5 >> 40 RERAISE 0 >> 42 COPY 3 44 POP_EXCEPT 46 RERAISE 1 ExceptionTable: 4 to 8 -> 10 [0] 10 to 34 -> 42 [1] lasti 40 to 40 -> 42 [1] lasti None  The main differences that jump out are the two additional NOP and the different targets for JUMP_BACKWARD. Note: the exception really cannot be caught as this will also throw the exception in 3.12 try: try: while True: pass except KeyboardInterrupt: print("a") except Exception: print("b") 

Answer: Its a known CPython bug introduced in 3.11 and exists in 3.12. One of comments of the bug, mentioned that partial backport of this pull request looks to be the right direction to fix the bug. I built and tested following CPython versions from source using pyenv on Arch Linux with GCC 14.1.1 compiler:  3.11-dev: Python 3.11.9+ (heads/3.11:ba43157, May 20 2024, 04:40:02) 3.12-dev: Python 3.12.3+ (heads/3.12:30c687c, May 20 2024, 04:38:13) 3.13.0b1: Python 3.13.0b1 (main, May 20 2024, 04:14:35) 3.13-dev: Python 3.13.0b1+ (heads/3.13:27b61c1, May 20 2024, 04:24:49) 3.14-dev: Python 3.14.0a0 (heads/main:0abf997, May 20 2024, 08:25:05)  In 3.13.0b1, 3.13-dev and 3.14-dev the bug is fixed 😀👍 and exception handling works as expected. But 3.11-dev and 3.12-dev still have the bug. I hope it will be backport to existing stable 3.11 and 3.12 versions (in-time for inclusion in the next 3.11.10 and 3.12.4 bug-fix releases respectively). EDIT 1: 3.12.4 released in 2024-06-06: the bug didn't fixed. EDIT 2: 3.12.5 released in 2024-08-06: the bug didn't fixed.
Question: I would like to create a new column "Day2" which takes the second digit of column named "Days", so if we have Days equal to 35, we would take the number 5 to be in "Day2", I tried this but it's not working: DF["Day2"] = DF["Days"].where( DF["Days"] < 10, (DF["Days"] / 10 % 10).astype(int), )  It seems it's taking the first digit but never the second one, can someone help?

Answer: This is a very simple and pythonic solution: import pandas as pd data = [10,11,35,45,65] df = pd.DataFrame(data, columns=['Day1']) df['Day2'] = df['Day1'].mod(10) df  Result  Day1 Day2 0 10 0 1 11 1 2 35 5 3 45 5 4 65 5
Question: I don't find any free or open source libraries to calculate CMYK and spot color on pdf. I would be grateful if someone could guide me in the right direction as to what I should do to access color channels and calculate the percentage of color used ( C,M,Y,K and spot, Export each separately ) with Python. Point: Actually, I don't have a problem with extract C,M,Y,K because I can easily extract it from the image, but the problem is that when I add spot colors, it convert it into cmyk again. That's why I'm looking for it in PDF. Thanks

Answer: I hope this will be useful for those who have a similar problem in the future. Dependencies : Ghostscript - Pillow How does it work ? Ghostscript will separated colors ( C,M,Y,K, Spots ) and save each one as .tiff and Pillow calculate the percentage of color ( In fact, the file saved by Ghostscript is in grayscale mode and has only one color channel. 0 to 255 ) used on each file. Point: Before that, make sure you have installed Ghostscript from PIL import Image from django.http import HttpResponse import os , fnmatch def pdf_color_splitter(): # Where the photos of separated colors are placed => path = 'image_inputs/' if not os.path.exists(path): os.makedirs(path) # now we run ghostscript command for separated colors and save them as tiff files => os.system(f'gs -sDEVICE=tiffsep -o {path}c.tiff cmyk_calculate/2021.pdf') # get all .tiff FILES = fnmatch.filter(os.listdir(path), '*.tiff') # calculate colors coverage each separately splited_colors = [] for f in FILES: O_FILE = Image.open(path+f) image_sizew,image_sizeh = O_FILE.size # get width,height count=image_sizeh*image_sizew val=0 # Collects colored pixels for i in range(0, image_sizew): for j in range(1, image_sizeh): pixVal = O_FILE.getpixel((i, j)) if pixVal != 255 and type(pixVal) != tuple: # no white pixels val+= 100 - (pixVal//2.55) # Pay attention to the point below this code resp = {'name':f.split('.')[0].replace('c(','').replace(')',''),'coverage':val/count} split_colors.append(resp) os.remove(path+f) # remove .tiff file in the end return splited_colors  Look to this code val+= 100 - (pixVal//2.55)  So what was this for? We want a number in the range of 0 to 100 because we are working in CMYK mode and subtract the answer from 100 because the photo is in Grayscale mode (actually to get the correct color density).
Question: I'm trying to install Tensorflow on Cygwin but i'm getting these Errors : $ /usr/bin/python3 -m pip install tensorflow==2.12.0 ERROR: Could not find a version that satisfies the requirement tensorflow==2.12.0 (from versions: none) ERROR: No matching distribution found for tensorflow==2.12.0  $ /usr/bin/python3 -m pip install tensorflow ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none) ERROR: No matching distribution found for tensorflow  What im using:  Windows10  Cygwin last version  Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32  Python 3.7.12 (default, Nov 23 2021, 18:58:07) [GCC 11.2.0] on Cygwin   How to fix it ? i tried to upgrade PIP but nothing and got this (note: This error originates from a subprocess, and is likely not a problem with pip.)

Answer: It seems you're trying to install tensorflow 2.12.0 to python 3.7.0. According to release notes of tensorflow 2.12.0. It says that  Support for Python 3.7 has been removed. We are not releasing any more patches for Python 3.7.  I would suggest to upgrade python or downgrade tensorflow.
Question: I'm having trouble with the python keyring library. I want to extract passwords from an entry in the windows credential manager. After that I want to overwrite the password of that specific entry. Here is my Code. import keyring # The service name in the credentials manager is "User@Service", the username is "User" password = keyring.get_password("User@Service", "User") # Overwrite the password keyring.set_password("User@Service", "User", "NewPassword")  I would expect this to overwrite my old password, but it creates a new entry in the credentials manager with the service name "User@User@Service" I also tried to simpley do keyring.set_password("Service", "User", "NewPassword") but that creates a new entry with the service name "Service". I simply can't get it to overwrite an existing entry. Any help is appreciated. Thanks.

Answer: I ran into this problem today as well. Seems like the same thing mentioned here: https://github.com/jaraco/keyring/issues/545 I don't know if it was ever fixed or has been an issue since then. I can't get it to work, as it just creates a new credential unless both versions already exist (service and user@service). I am working around this by just deleting the credential and creating a new one. import keyring from keyring.errors import PasswordDeleteError try: keyring.delete_password("testapp", "username") except PasswordDeleteError: # In case the password doesn't exist yet pass keyring.set_password("testapp", "username", "pw")
Question: Since the slack file.upload is going to be depreciated, I was going to make new the methods work. The first one works as: curl -s -F files=@test.txt -F filename=test.txt -F token=xoxp-token -F length=50123 https://slack.com/api/files.getUploadURLExternal  and returns {"ok":true,"upload_url":"https:\/\/files.slack.com\/upload\/v1\/CwABAAAAXAoAAVnX....","file_id":"F07XXQ9XXXX"}  The second request is: curl -X POST \ -H "Authorization: Bearer xoxp-token" \ -H "Content-Type: application/json" \ -d '{ "files": [{"id":"F07XXXXXX", "title":"Slack API updates Testing"}], "channel_id": "C06EXXXXX" }' \ https://slack.com/api/files.completeUploadExternal  With the second request, I get 200 OK responses. {"ok":true,"files":[{"id":"XXXXXXX","created":XXXXXXXX,"timestamp":XXXXXXXX,"name":"test.txt","title":"Slack API updates Testing","mimetype":"","filetype":"","pretty_type":"","user":"XXXXXXXXX","user_team":"XXXXXXXXX","editable":false,"size":50123,"mode":"hosted","is_external":false,"external_type":"","is_public":false,"public_url_shared":false,"display_as_bot":false,"username":"","url_private":"https:\/\/files.slack.com\/files-pri\/XXXXXXXXX-XXXXXXX\/test.txt","url_private_download":"https:\/\/files.slack.com\/files-pri\/XXXXXXXXX-XXXXXXX\/download\/test.txt","media_display_type":"unknown","permalink":"https:\/\/XXXXXXXX.slack.com\/files\/XXXXXXXXX\/XXXXXXX\/test.txt","permalink_public":"https:\/\/slack-files.com\/XXXXXXXXX-XXXXXXX-XXXXXXXX","comments_count":0,"is_starred":false,"shares":{},"channels":[],"groups":[],"ims":[],"has_more_shares":false,"has_rich_preview":false,"file_access":"visible"}],"warning":"missing_charset","response_metadata":{"warnings":["missing_charset"]}}  Questions and problems: 1- How do I determine the length= in the first request (files.getUploadURLExternal ) 2- Even though I have provided the channel_id the file is still not getting uploaded on the Slack channel. Documentation: https://api.slack.com/methods/files.completeUploadExternal https://api.slack.com/methods/files.getUploadURLExternal

Answer: You skipped a step. You have to send a POST to the URL that is returned to you on the first step. In that post you are also doing the file upload. 2nd step should be: curl -F filename="@$filename" -H "Authorization: Bearer $SLACK_KEY" -v POST $upload_url then your 2nd step should be the 3rd step. As far as getting the byte size stat --printf="%s" file if you are using bash
Question: I am trying to install pandas on a Windows machine but get the following output: python -m pip install pandas Collecting pandas Using cached pandas-2.2.2.tar.gz (4.4 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error × Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [13 lines of output] + meson setup C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db\.mesonpy-6bapjes7\build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db\.mesonpy-6bapjes7\build\meson-python-native-file.ini The Meson build system Version: 1.2.1 Source dir: C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db Build dir: C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db\.mesonpy-6bapjes7\build Build type: native build Project name: pandas Project version: 2.2.2 Activating VS 15.9.60 ..\..\meson.build:2:0: ERROR: Value "c11" (of type "string") for combo option "C language standard to use" is not one of the choices. Possible choices are (as string): "none", "c89", "c99", "gnu89", "gnu90", "gnu9x", "gnu99". A full log can be found at C:\Users\serba\AppData\Local\Temp\pip-install-6cc_jg_i\pandas_90906587555b4b9d852626b7159094db\.mesonpy-6bapjes7\build\meson-logs\meson-log.txt [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.  I tried the following:  running the terminal as administrator updating pip installing older versions of pandas because I read there is some issue with the latest one looking inside the log file from the output, but it doesn't exist anymore after the process finishes  The output is still the same. I see it complaining about a C standard. Do I need a specific compiler to be installed or any other prerequisites? Thanks. Regards, Serban

Answer: So I got past this, it was a stupid reason this was happening. I had another pip binary somewhere in a MinGW installation folder and it was always calling that one. I made sure change the environment variables to point to a proper, standalone installation of pip (also python).
Question: I'm customizing an Odoo eCommerce website and I need to modify the add_qty parameter in the product details page dynamically via a URL parameter. I'm extending the WebsiteSale class to achieve this. Here's my code snippet: class WebsiteSaleCustom(WebsiteSale): def _prepare_product_values(self, product, category, search, **kwargs): values = super()._prepare_product_values(product, category, search, **kwargs) values['add_qty'] = int(kwargs.get('add_qty', 1)) return values  This solution works as expected upon the first load of the product details page. However, the issue arises when attempting to dynamically update the add_qty parameter. Suppose i hit this link http://localhost:8069/shop/black-shoe?add_qty=5 Quantity will set 5. For first time, it is setting 5 but if i change quantity it not changing in quantity field. It's remaining 5. Any insights or suggestions on how to achieve this would be greatly appreciated. Thank you for your help!

Answer: To address issues related to outdated or incorrect data being fetched by Odoo due to caching, you can use the method request.env.registry._clear_cache() before applying your solution. This method clears the cache, ensuring that Odoo fetches fresh data. from odoo.http import request from odoo.addons.website_sale.controllers.main import WebsiteSale class WebsiteSaleCustom(WebsiteSale): def _prepare_product_values(self, product, category, search, **kwargs): request.env.registry._clear_cache() values = super()._prepare_product_values(product, category, search, **kwargs) values['add_qty'] = int(kwargs.get('add_qty', 1)) return values
Question: I have a ton of documents that have been vectorized poorly / are using an old multimodal_field_combination. mappings={ 'combo_text_image': { "type": "multimodal_combination", "weights": { "name": 0.05, "description": 0.15, "image_url": 0.8 } } },  But I need to update to mappings={ 'combo_text_image': { "type": "multimodal_combination", "weights": { "name": 0.2, "image_url": 0.8 } } },  I've realized that to do this in the same index, I should delete the documents and then reindex but I haven't been able to find a mq.delete_all_documents() call. What's the best way to do this?

Answer: Here is a code snippet that would delete all documents from the index: def empty_index(index_name): index = mq.index(index_name) res = index.search(q = '', limit=400) while len(res['hits']) > 0: id_set = [] for hit in res['hits']: id_set.append(hit['_id']) index.delete_documents(id_set) res = index.search(q = '', limit=400)
Question: Given a 2D Numpy array, I'd like to be able to pad it on the left, right, top, bottom side, like the following pseudo-code. Is there anything like this already built into Numpy? import numpy as np a = np.arange(16).reshape((4, 4)) # [[ 0 1 2 3] # [ 4 5 6 7] # [ 8 9 10 11] # [12 13 14 15]] pad(a)[-4:2, -1:3] # or any syntax giving the same result #[[0 0 0 0] # [0 0 0 0] # [0 0 0 0] # [0 0 0 0] # [0 0 1 2] # [0 4 5 6]] pad(a)[-4:2, -1:6] #[[0 0 0 0 0 0 0] # [0 0 0 0 0 0 0] # [0 0 0 0 0 0 0] # [0 0 0 0 0 0 0] # [0 0 1 2 3 0 0] # [0 4 5 6 7 0 0]] 

Answer: Some trickery: import numpy as np class PaddedArray: def __init__(self, arr): self.arr = arr def __getitem__(self, idx): idx = tuple( slice(s.start or 0, s.stop or size) for size, s in zip(self.arr.shape, idx) ) slices = tuple( slice(max(0, s.start), min(size, s.stop)) for size, s in zip(self.arr.shape, idx) ) paddings = tuple( (max(0, -s.start), max(0, s.stop - size)) for size, s in zip(self.arr.shape, idx) ) return np.pad(self.arr[slices], paddings)  Testing: >>> PaddedArray(np.arange(16).reshape((4, 4)))[-4:2, -1:3] array([[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 1, 2], [0, 4, 5, 6]]) >>> PaddedArray(np.arange(16).reshape((4, 4)))[-4:2, -1:6] array([[0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 2, 3, 0, 0], [0, 4, 5, 6, 7, 0, 0]]) >>> PaddedArray(np.arange(32).reshape((4, 4, 2)))[-4:2, -1:3, :] array([[[ 0, 0], [ 0, 0], [ 0, 0], [ 0, 0]], [[ 0, 0], [ 0, 0], [ 0, 0], [ 0, 0]], [[ 0, 0], [ 0, 0], [ 0, 0], [ 0, 0]], [[ 0, 0], [ 0, 0], [ 0, 0], [ 0, 0]], [[ 0, 0], [ 0, 1], [ 2, 3], [ 4, 5]], [[ 0, 0], [ 8, 9], [10, 11], [12, 13]]])