eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to merge two RGBA images (with a shape of (h,w,4)), taking into account their alpha channels. Example :   What I've tried I tried to do this using opencv for that, but I getting some strange pixels on the output image. Images Used:  and  import cv2 import numpy as np import matplotlib.pyplot as plt image1 = cv2.imread("image1.png", cv2.IMREAD_UNCHANGED) image2 = cv2.imread("image2.png", cv2.IMREAD_UNCHANGED) mask1 = image1[:,:,3] mask2 = image2[:,:,3] mask2_inv = cv2.bitwise_not(mask2) mask2_bgra = cv2.cvtColor(mask2, cv2.COLOR_GRAY2BGRA) mask2_inv_bgra = cv2.cvtColor(mask2_inv, cv2.COLOR_GRAY2BGRA) # output = image2*mask2_bgra + image1 output = cv2.bitwise_or(cv2.bitwise_and(image2, mask2_bgra), cv2.bitwise_and(image1, mask2_inv_bgra)) output[:,:,3] = cv2.bitwise_or(mask1, mask2) plt.figure(figsize=(12,12)) plt.imshow(cv2.cvtColor(output, cv2.COLOR_BGRA2RGBA)) plt.axis('off')  Output :  So what I figured out is that I'm getting those weird pixels because I used cv2.bitwise_and function (Which btw works perfectly with binary alpha channels). I tried using different approaches  Question Is there an approach to do this (While keeping the output image as an 8bit image).

Answer: I was able to obtain the expected result in 2 stages. # Read both images preserving the alpha channel hh1 = cv2.imread(r'C:\Users\524316\Desktop\Stack\house.png', cv2.IMREAD_UNCHANGED) hh2 = cv2.imread(r'C:\Users\524316\Desktop\Stack\memo.png', cv2.IMREAD_UNCHANGED) # store the alpha channels only m1 = hh1[:,:,3] m2 = hh2[:,:,3] # invert the alpha channel and obtain 3-channel mask of float data type m1i = cv2.bitwise_not(m1) alpha1i = cv2.cvtColor(m1i, cv2.COLOR_GRAY2BGRA)/255.0 m2i = cv2.bitwise_not(m2) alpha2i = cv2.cvtColor(m2i, cv2.COLOR_GRAY2BGRA)/255.0 # Perform blending and limit pixel values to 0-255 (convert to 8-bit) b1i = cv2.convertScaleAbs(hh2*(1-alpha2i) + hh1*alpha2i)  Note: In the b=above the we are using only the inverse alpha channel of the memo image  But I guess this is not the expected result. So moving on .... # Finding common ground between both the inverted alpha channels mul = cv2.multiply(alpha1i,alpha2i) # converting to 8-bit mulint = cv2.normalize(mul, dst=None, alpha=0, beta=255,norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U) # again create 3-channel mask of float data type alpha = cv2.cvtColor(mulint[:,:,2], cv2.COLOR_GRAY2BGRA)/255.0 # perform blending using previous output and multiplied result final = cv2.convertScaleAbs(b1i*(1-alpha) + mulint*alpha)   Sorry for the weird variable names. I would request you to analyze the result in each line. I hope this is the expected output.
Question: Say I have a python script in this form import numpy as np x = [1, 2, 5, 3, 9, 6] x.sort() print(np.sum(x))  I want to extract all the functions that have been used in this script. For this example the outcome should be list.sort, np.sum, print. I can list all the builtin functions and then expand that list with dir(numpy). And then check whether each of those functions appear in the code. But what if there is a function like np.linalg.norm(x)? How will I find this function since norm will not be returned by dir(numpy). Note: I understand many other functions can be invoked which does not appear in the code. For this example list.__init__. But I only want those functions that appears in the code. Simply put, I want all the words/tokens that will be colored yellow by vscode in Dark+ theme. Is there any possible way to use rope for this problem?

Answer: You can use ast module to do something like that: import ast code = """ import numpy as np x = [1, 2, 5, 3, 9, 6] x.sort() print(np.sum(x)) """ module = ast.parse(code) for node in ast.walk(module): if isinstance(node, ast.Call): try: parent_name = node.func.value.id + '.' except AttributeError: parent_name = '' if isinstance(node.func, ast.Name): print(f'{parent_name}{node.func.id}') else: print(f'{parent_name}{node.func.attr}') # output: # x.sort # print # np.sum  Of course, there is x.sort instead of list.sort but I didn't find out how to get the actual parent type of the method.
Question: I am trying to run this Github project on docker on my machine. Here is my docker file that I am running after cloning the project on my local machine. FROM python:3 RUN mkdir -p /opt/cascade-server WORKDIR /opt/cascade-server COPY requirements.txt . RUN pip install -r requirements.txt COPY . . COPY docker_defaults.yml conf/defaults.yml CMD /bin/bash docker_start.sh  All my configurations are correct and working fine. When I run the project without docker containers, it runs correctly. When I build Docker containers to install it, it does not ends on python cascade.py -vv when running the below shell script from the last line of docker. #!/bin/bash if [ -f "conf/cascade.yml" ]; then echo "cascade.yml found. Using existing configuration." else echo "cascade.yml not found. Generating new config file from defaults" python cascade.py --setup_with_defaults fi python cascade.py -vv  All the logs are showing correct results till the last line executes from the shell script. Here is the error when the last line executes. <frozen importlib._bootstrap>:241: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 152 from C header, got 160 from PyObject <frozen importlib._bootstrap>:241: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 152 from C header, got 160 from PyObject <frozen importlib._bootstrap>:241: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 152 from C header, got 160 from PyObject <frozen importlib._bootstrap>:241: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 152 from C header, got 160 from PyObject <frozen importlib._bootstrap>:241: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 152 from C header, got 160 from PyObject /opt/cascade-server/app/async_wrapper.py:44: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/usr/local/lib/python3.10/site-packages/urllib3/util/__init__.py)', 'pymongo.ssl_context (/usr/local/lib/python3.10/site-packages/pymongo/ssl_context.py)', 'urllib3.util.ssl_ (/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py)']. gevent.monkey.patch_all() docker_start.sh: line 10: 18 Segmentation fault (core dumped) python cascade.py -vv  python version 

Answer: try to change the base image python:3 (which use Python 3.10) to python:3.7 base image like your local Python interpreter.
Question: I have a dict as below: dict1={ 'item1': {'result': [{'val': 228, 'no': 202}] }, 'item2': {'result': [{'value': 148, 'year': 201}] } }  How can we insert a new key 'category' to each item so that the output looks like below: output={ 'item1': {'category': {'result': [{'val': 228, 'no': 202}] } }, 'item2': {'category': {'result': [{'value': 148, 'year': 201}] } } }  Currently, i have key:value and im looking to insert newkey which takes same value, key:newkey:value I tried to do dict1['item1']['category1'] but this is adding a new key value pair.

Answer: Use to modify in-place the existing dictionary dict1: for key, value in dict1.items(): dict1[key] = { "category" : value } print(dict1)  Output {'item1': {'category': {'result': [{'val': 228, 'no': 202}]}}, 'item2': {'category': {'result': [{'value': 148, 'year': 201}]}}}  As an alternative use update: dict1.update((k, {"category": v}) for k, v in dict1.items())  Note that update receives both a dictionary or an iterable of key/value pairs, from the documentation:  update() accepts either another dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two).  Finally in Python 3.9+, you can use the merge operator (|=), as below: dict1 |= ((k, {"category": v}) for k, v in dict1.items())
Question: I'm trying to get the wireless debugging port of ADB in my Android, using the command here: & "D:\Tools\Nmap\nmap.exe" -T4 192.168.2.20 -p 37000-44000 | Where-Object {$_ -match "tcp open"} | ForEach-Object {$_.split("/")[0]} And I would like to make a Python script for further purposes: ip = '192.168.2.20' nmap_path = r'D:\Tools\Nmap\nmap.exe' def get_port(): port_result = subprocess.run( f'& "{nmap_path}" -T4 {ip} -p 37000-44000 | ' f'Where-Object {{$_ -match "tcp open"}} | ' f'ForEach-Object {{$_.split("/")[0]}}', shell=True ) port = port_result.stdout.decode('utf-8').strip() return port  But it gave the following error: & was unexpected at this time., which indicated that the command was run in CMD instead of PowerShell.  I do not want to use powershell -Command, nor saving the commands into a .ps1 file. Could I make subprocess.run to run specifically in PowerShell?

Answer: The documentation says subprocess() uses COMSPEC to determine which shell to run if you set shell=True. I don't have, want, or use Windows, but imagine you'd need something like: import os import subprocess # Change COMSPEC to point to Powershell os.putenv('COMSPEC',r'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe') subprocess.run(..., shell=True)  I'm assuming you can check that path.  I guess it would probably be good practice to save and restore the previous COMSPEC in case the change to Powershell upsets something else. So: # Save original COMSPEC savedCOMSPEC = os.getenv('COMSPEC') ... CODE FROM ABOVE ... # Restore previous COMSPEC os.putenv('COMSPEC', savedCOMSPEC)
Question: I'm trying to excute a code, but I keep geting this error when compiling this piece of code:  import tensorflow as tf from xba import XBA import torch torch.tensor([1, 2, 3, 4]).to(device="cuda:2")  torch.tensor([1, 2, 3, 4]).to(device="cuda:2") generates this error: " RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.  AnyIdea about the origine of the bug, it's just the first line of code!!

Answer: "cuda:2"  selects the third GPU in your system. If you don't have 3 GPUs (at least) in your system, you'll get this error. Assuming you have at least 1 properly installed and set up CUDA GPU available, try: "cuda:0"
Question: I am getting a new mac with m1 pro chip and want to install Python with Spyder IDE. I will be using conda to manage Python environments. I gather that as of now Spyder does not run natively on m1 chip while Python with different packages does, see for example: https://www.anaconda.com/blog/new-release-anaconda-distribution-now-supporting-m1 The question is, what is the right way to install Spyder. Suppose I'm using Miniconda, create a new environment and install Spyder: conda install spyder  will this Spyder work correctly or crash? do I need to run it using Rosetta2? (and how do I run only Spyder IDE under Rosetta2 while having Python run natively on m1?) sorry if I am getting some terminology wrong I am fairly new to this.

Answer: I personally have had no issue using Spyder through Anaconda, nonetheless it will be running on Rosetta (even if you download it directly). Python will be running using M1 inside the IDE. I haven't had any major issues. If you absolutely want to run python natively on M1 then python 3.9.1 is required and you should use the console.
Question: So i have the following sample df df = pd.DataFrame({'Id':[1,1,2,3],'Origin':['int','int','pot','pot'],'Origin2':['pot','int','int','int']}) Id Origin Origin2 0 1 int pot 1 1 int int 2 2 pot int 3 3 pot int  And i do the following replace command df.loc[df['Id'].eq(1)].apply(lambda x : x.replace('int':np.nan))  How could i update the original df, with both new columns using indexes. I tried df.update but after checking the documentation, i noticed it doesnt substitute 'non na' values by nan values? For better understanding, the columns in the index [0,1] ('id'= 1). Substitute the string 'int' by np.nan Wanted result: df = pd.DataFrame({'Id':[1,1,2,3],'Origin':[np.nan,np.nan,'pot','pot'],'Origin2':['pot',np.nan,'int','int']}) Id Origin Origin2 0 1 NaN pot 1 1 NaN NaN 2 2 pot int 3 3 pot int 

Answer: Use mixed boolean/label indexing: m = df['Id'].eq(1) df.loc[m, ['Origin', 'Origin2']] = df.loc[m, ['Origin', 'Origin2']].replace('int', np.nan)  Output:  Id Origin Origin2 0 1 NaN pot 1 1 NaN NaN 2 2 pot int 3 3 pot int
Question: I have a function in python 3.x def foo(): """Lorem ipsum for more info, see here""  I want to add a hyperlink to 'here' to point to a web site. How can I do that without installing external plugin?

Answer: Just add the link as a string into the docstring, like so: def foo(): """Lorem ipsum for more info, see here: www.myfancydocu.com""  The doctring is just a string, so there is no Hyperlink. But anyone that wants to look at the website can just copy the link. There are automatic documentation-builders that build a documentation out of your code and docstrings in e.g. html. Those can probably add hyperlinks to the documentation with a specific syntax, but that syntax then depends on which documentation-builder you use. If you only have your code, then just adding the url as a string is all you can do.
Question: I want to write a task that will only run once a day at 3:30 p.m. with Python FASTAPI. How can I do it? I tried this but it works all the time. schedule.every().day.at("15:30:00").do(job2) while True: schedule.run_all() 

Answer: Swap the schedule.run_all() for schedule.run_pending(). It should work! import schedule import time def job(): print("I'm working...") schedule.every().day.at("15:30:00").do(job) while True: schedule.run_pending()
Question: Imagine I have an order for specialists in some coding languages with multiple criterion in JSON format: request = {'languages_required': {'Python': 4, 'Java': 2}, 'other_requests': [] }  languages_required means that the candidate must have a skill in the language and the number is the minimum level of this language. The format of candidates dataframe is long: df = pd.DataFrame({'candidate': ['a', 'a', 'a', 'b', 'b', 'c', 'c', 'd', 'd', 'd'], 'language': ['Python', 'Java', 'Scala', 'Python', 'R', 'Python', 'Java', 'Python', 'Scala', 'Java'], 'skill': [5, 4, 4, 6, 8, 1, 3, 5, 2, 2]})  That gives:  candidate language skill 0 a Python 5 1 a Java 4 2 a Scala 4 3 b Python 6 4 b R 8 5 c Python 1 6 c Java 3 7 d Python 5 8 d Scala 2 9 d Java 2  What I need to do is to keep the candidates and their skills in required languages that meet the requirements from the request, that is:  Have skills in both mentioned languages Skills in these languages are equal or higher than values in the dictionary  So the desired output would be:  candidate language skill 0 a Python 5 1 a Java 4 7 d Python 5 9 d Java 2  I am able to filter the candidates with the languages based on keys() of the dictionary: lang_mask = df[df['language'].isin(request['languages_required'].keys())]\ .groupby('candidate')['language']\ .apply(lambda x: set(request['languages_required']).issubset(x))  But struggle with adding the 'is higher than' per language condition.

Answer: You need call first condition in one step and then second in another step: df = df[df['language'].map(request['languages_required']).le(df['skill'])] df = df[df.groupby('candidate')['language'].transform(lambda x: set(request['languages_required']).issubset(x))] print (df) candidate language skill 0 a Python 5 1 a Java 4 7 d Python 5 9 d Java 2  Or one row solution: df = (df[df['language'].map(request['languages_required']).le(df['skill'])] .pipe(lambda x: x[x.groupby('candidate')['language'].transform(lambda x: set(request['languages_required']).issubset(x))])) print (df) candidate language skill 0 a Python 5 1 a Java 4 7 d Python 5 9 d Java 2
Question: How to create a dummy variable if missing values are included? I have the following data and I want to create a Dummy variable based on several conditions. My problem is that it automatically converts my missing values to 0, but I want to keep them as missing values. import pandas as pd mydata = {'x' : [10, 50, np.nan, 32, 47, np.nan, 20, 5, 100, 62], 'y' : [10, 1, 5, np.nan, 47, np.nan, 8, 5, 100, 3]} df = pd.DataFrame(mydata) df["z"] = ((df["x"] >= 50) & (df["y"] <= 20)).astype(int) print(df) 

Answer: When creating your boolean-mask, you are comparing integers with nans. In your case, when comparing df['x']=np.nan with 50, your mask df['x'] >= 50 will always be False and will equal 0 if you convert it to an integer. You can just create a boolean-mask that equals True for all rows that contain any np.nan in the columns ['x', 'y'] and then assign np.nan to these rows. Code: import pandas as pd import numpy as np mydata = {'x' : [10, 50, np.nan, 32, 47, np.nan, 20, 5, 100, 62], 'y' : [10, 1, 5, np.nan, 47, np.nan, 8, 5, 100, 3]} df = pd.DataFrame(mydata) df["z"] = ((df["x"] >= 50) & (df["y"] <= 20)).astype("uint32") df.loc[df[["x", "y"]].isna().any(axis=1), "z"] = np.nan  Output:  x y z 0 10.0 10.0 0.0 1 50.0 1.0 1.0 2 NaN 5.0 NaN 3 32.0 NaN NaN 4 47.0 47.0 0.0 5 NaN NaN NaN 6 20.0 8.0 0.0 7 5.0 5.0 0.0 8 100.0 100.0 0.0 9 62.0 3.0 1.0  Alternatively, if you want an one-liner, you could use nested np.where statements: df["z"] = np.where( df.isnull().any(axis=1), np.nan, np.where((df["x"] >= 50) & (df["y"] <= 20), 1, 0) )
Question: my goal is to find if the following df has a 'circulation' given: df = pd.DataFrame({'From':['USA','UK','France','Italy','Russia','china','Japan','Australia','Russia','Italy'], 'to':['UK','France','Italy','Russia','china','Australia','New Zealand','Japan','USA','France']}) df   and if I graph it, it would look like this (eventually, note that the order on the df is different): USA-->UK-->France-->Italy-->Russia-->China-->Australia-->Japan-->Australia | | | | France USA  The point is this: You cannot go backward, so Italy cannot go to France and Russia cannot go to USA. Note: From can have multiple Tos How can I find it in pandas so the end result would look like this:  I can solve it without pandas (I get df.to_dict('records') and then iterate to find the circulation and then go back to pandas) but I wish to stay on pandas.

Answer: The logic is not fully clear, however you can approach your problem with a graph. Your graph is the following:  Let us consider circulating nodes, those that have more than one destination. You can obtain this with networkx: import networkx as nx G = nx.from_pandas_edgelist(df, source='From', target='to', create_using=nx.DiGraph) circulating = {n for n in G if len(list(G.successors(n)))>1} df['IS_CIRCULATING'] = df['From'].isin(circulating).astype(int)  output:  From to IS_CIRCULATING 0 USA UK 0 1 UK France 0 2 France Italy 0 3 Italy Russia 1 4 Russia china 1 5 china Australia 0 6 Japan New Zealand 0 7 Australia Japan 0 8 Russia USA 1 9 Italy France 1  With pure pandas: df['IS_CIRCULATING'] = df.groupby('From')['to'].transform('nunique').gt(1).astype(int)
Question: I have a GitHub Action workflow file that is doing multiple linting checks. flake8 is the first linting check and if it fails the entire workflow fails meaning the subsequent linting checks are name: lint on: push: pull_request: jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@main with: ref: ${{ github.head_ref }} - name: python uses: actions/setup-python@main with: # pulls latest version of python, alternatively specify exact version (i.e. 3.8 -> no quotes) python-version: '3.9' - name: install run: | pip install -r requirements.txt - name: Lint with flake8 run: | # fail if there are any flake8 errors flake8 . --count --max-complexity=15 --max-line-length=127 --statistics ## subsequent linting jobs 

Answer: You could use the continue-on-error in conjunction with the outcome of the step. From the doc:  steps.<step_id>.outcome string The result of a completed step before continue-on-error is applied. Possible values are success, failure, cancelled, or skipped. When a continue-on-error step fails, the outcome is failure, but the final conclusion is success.  As example:  - name: Lint with flake8 id: flake8 continue-on-error: true run: | # fail if there are any flake8 errors flake8 . --count --max-complexity=15 --max-line-length=127 --statistics - name: Check if 'Lint with flake8' step failed if: steps.flake8.outcome != 'success' run: | echo "flake8 fails" exit 1
Question: I have a single row data-frame like below Num TP1(USD) TP2(USD) TP3(USD) VReal1(USD) VReal2(USD) VReal3(USD) TiV1 (EUR) TiV2 (EUR) TiV3 (EUR) TR TR-Tag AA-24 0 700 2100 300 1159 2877 30 30 47 10 5  I want to get a dataframe like the one below ID Price Net Range 1 0 300 30 2 700 1159 30 3 2100 2877 47  The logic here is that a. there will be 3 columns names that contain TP/VR/TV. So in the ID, we have 1, 2 & 3 (these can be generated by extracting the value from the column names or just by using a range to fill) b. TP1 value goes into first row of column 'Price',TP2 value goes into second row of column 'Price' & so on c. Same for VR & TV. The values go into 'Net' & 'Range columns d. Columns 'Num', 'TR' & 'TR=Tag' are not relevant for the result. I tried df.filter(regex='TP').stack(). I get all the 'TP' column & I can access individual values be index ([0],[1],[2]). I could not get all of them into a column directly. I also wondered if there may be a easier way of doing this.

Answer: Assuming 'Num' is a unique identifier, you can use pandas.wide_to_long: pd.wide_to_long(df, stubnames=['TP', 'VR', 'TV'], i='Num', j='ID')  or, for an output closer to yours: out = (pd .wide_to_long(df, stubnames=['TP', 'VR', 'TV'], i='Num', j='ID') .reset_index('ID') .drop(columns=['TR', 'TR-Tag']) .rename(columns={'TP': 'Price', 'VR': 'Net', 'TV': 'Range'}) )  output:  ID Price Net Range Num AA-24 1 0 300 30 AA-24 2 700 1159 30 AA-24 3 2100 2877 47  updated answer out = (pd .wide_to_long(df.set_axis(df.columns.str.replace(r'\(USD\)$', '', regex=True), axis=1), stubnames=['TP', 'VReal', 'TiV'], i='Num', j='ID') .reset_index('ID') .drop(columns=['TR', 'TR-Tag']) .rename(columns={'TP': 'Price', 'VReal': 'Net', 'TiV': 'Range'}) )  output:  ID Price Net Range Num AA-24 1 0 300 30 AA-24 2 700 1159 30 AA-24 3 2100 2877 47
Question: Let's consider two pandas dataframes: import numpy as np import pandas as pd df = pd.DataFrame([1, 2, 3, 2, 5, 4, 3, 6, 7]) check_df = pd.DataFrame([3, 2, 5, 4, 3, 6, 4, 2, 1])  If want to do the following thing:  If df[1] > check_df[1] or df[2] > check_df[1] or df[3] > check_df[1] then we assign to df 1, and 0 otherwise If df[2] > check_df[2] or df[3] > check_df[2] or df[4] > check_df[2] then we assign to df 1, and 0 otherwise We apply the same algorithm to end of DataFrame  My primitive code is the following: df_copy = df.copy() for i in range(len(df) - 3): moving_df = df.iloc[i:i+3] if (moving_df >check_df.iloc[i]).any()[0]: df_copy.iloc[i] = 1 else: df_copy.iloc[i] = -1 df_copy 0 0 -1 1 1 2 -1 3 1 4 1 5 -1 6 3 7 6 8 7  Could you please give me a advice, if there is any possibility to do this without loop?

Answer: IIUC, this is easily done with a rolling.min: df['out'] = np.where(df[0].rolling(N, min_periods=1).max().shift(1-N).gt(check_df[0]), 1, -1)  output:  0 out 0 1 -1 1 2 1 2 3 -1 3 2 1 4 5 1 5 4 -1 6 3 1 7 6 -1 8 7 -1  to keep the last items as is: m = df[0].rolling(N).max().shift(1-N) df['out'] = np.where(m.gt(check_df[0]), 1, -1) df['out'] = df['out'].mask(m.isna(), df[0])  output:  0 out 0 1 -1 1 2 1 2 3 -1 3 2 1 4 5 1 5 4 -1 6 3 1 7 6 6 8 7 7