eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Consider the following image, stored as a numpy array: a = [[0,0,0,0,0,1,1,0,0,0], [0,0,0,0,1,1,1,1,0,0], [0,0,0,0,0,1,1,0,0,0], [0,0,0,0,0,0,0,0,0,0], [0,0,0,0,0,2,0,0,0,0], [0,0,0,0,0,2,2,0,0,0], [0,0,0,0,0,2,0,0,0,0], [0,0,0,0,3,3,3,0,0,0], [4,0,0,0,0,0,0,0,0,0], [4,4,0,0,0,0,0,0,0,0], [4,4,4,0,0,0,0,0,0,0]] a = np.array(a)  Zeros represent background pixels, 1,2,3 and 4 represent pixels that belong to objects. You can see that objects always form contiguous islands or regions in the image. I would like to know the distance between every pair of objects. As distance measure I'd like to have the shortest, staightline distance, between those pixels of the object, that are closest to each other. Example: Distance(2,3) = 1, because they are touching. Distance(1,2) = 2, because there is exactly one background pixel separating the two regions, or in other words, the closest pixels of the objects are two pixels apart. Can anybody tell me how one would approach this problem in Python? Or link me to some resources?

Answer: This is what you would need: from scipy.spatial.distance import cdist def Distance(a, m, n): return cdist(np.argwhere(a==m),np.argwhere(a==n),'minkowski',p=1.).min()  or similarly per @MaxPowers comment (claim: cityblock is faster):  return cdist(np.argwhere(a==m),np.argwhere(a==n),'cityblock').min()  Find the locations of islands and calculate pairwise distance of locations and get the minimum. I am not 100% sure of your desired distance, but I think you are looking for l1 norm. If not, you can change the cdist measure to your desired metric. output: Distance(a,2,3) 1.0 Distance(a,2,1) 2.0 Distance(a,3,1) 5.0 Distance(a,4,3) 5.0
Question: I am trying to use the AWS SAM CLI installed through Homebrew and I am seeing the following error when I try to use sam with any command: dyld: Library not loaded: @executable_path/../.Python Referenced from: /usr/local/Cellar/aws-sam-cli/0.53.0/libexec/bin/python3.7 Reason: image not found  Looking at the .Python file referenced in the error, it is symlinked to a python folder that doesn't actually exist: drwxr-xr-x 7 RCR staff 224 Jun 16 19:40 . drwxr-xr-x 9 RCR staff 288 Jul 8 14:55 .. lrwxr-xr-x 1 RCR staff 70 Jun 16 19:40 .Python -> ../../../../opt/python/Frameworks/Python.framework/Versions/3.7/Python drwxr-xr-x 39 RCR staff 1248 Jul 8 14:55 bin drwxr-xr-x 3 RCR staff 96 Jun 16 19:40 include drwxr-xr-x 3 RCR staff 96 Jun 16 19:40 lib -rw-r--r-- 1 RCR staff 61 Jun 16 19:40 pip-selfcheck.json  I do not have a 3.7 folder at that location, but I do have a 3.8 folder. That said, I am not sure what is the origin of this folder. My Python3 installation is from Homebrew and located in the Cellar as usual (../Cellar/python@3.8/3.8.3_1/bin/python3) and symlinked to /usr/local/bin/python3. Not sure if that is relevant but I figure more info can't hurt. I tried symlinking the .Python file to the 3.8 version I do have at that location but it only produced other errors. Any idea how I can get this CLI working?

Answer: Looks like 0.53.0 comes with python3.7 executables, there is a workaround until it is fixed: brew install --build-from-source aws-sam-cli  https://github.com/awslabs/aws-sam-cli/issues/2101 https://github.com/aws/homebrew-tap/issues/93
Question: I successfully installed ipython via pip. I wanted then to use it by launching it through windows 10 command prompt but am getting the following error 'ipython' is not recognized as an internal or external command, operable program or batch file.  I have gone through many questions on stackoverflow but cannot get a relevant solution. I tried pip install ipython to confirm the ipython is installed and following on the instruction on my tutorial, i typed ipython on cmd to launch the program and it has never worked. This is slowing down my learning, please help!

Answer: Search in your machine the ipython application (directory in which it is installled) and the add the path to PATH environment variables. For example in my case location was C:\Users\DELL\AppData\Local\Programs\Python\Python37\Scripts Add this path to PATH environment variable (see here) and your problem is solved.
Question: I have a Dataframe with around 80.000 observations taken every 15 min. The seasonal parameter m is assumed with 96, because every 24h the pattern repeats. When I insert these informations in my auto_arima algorithm, it takes a long time (some hours) until the following error message is given out: MemoryError: Unable to allocate 5.50 GiB for an array with shape (99, 99, 75361) and data type float64  The code that I am using: stepwise_fit = auto_arima(df['Hges'], seasonal=True, m=96, stepwise=True, stationary=True, trace=True) print(stepwise_fit.summary())  I tried it with resampling to hourly values, to reduce the amount of data and the m-factor to 24, but still my computer cannot calculate the result. How do find the weighting factors with auto_arima when you deal with large data ?

Answer: I don't recall the exact source where I read this, but neither auto.arima nor pmdarima are really optimized to scale, which might explain the issues you are facing. But there are some more important things to note about your question: With 80K data points at 15 minute intervals, ARIMA probably isn't the best type of model for your use case anyway:  With the frequency and density of your data, it is likely that there are multiple cycles/seasonal patterns, and ARIMA can handle only one seasonal component. So at the very least you should try a model that can handle multiple seasonalities like STS or Prophet (TBATS in R can also handle multiple seasonalities, but it is likely to suffer from the same issues as auto.arima, since it is in the same package). At 80K points and 15 minute measurement intervals, I assume you are most likely dealing with a "physical" time series that is the output of a sensor or some other metering/monitoring device (electrical load, network traffic, etc...). These types of time series are usually very good use cases for LSTM or other Deep Learning based models instead of ARIMA.
Question: I am aware of using axis=None to hide axis lines. But when you have actively used axis to modify the graph, is it possible to keep just the ticks, but hide the axis lines for both X and Y axis? For example, here is a graph I have where I'd like it to happen - import pandas as pd import altair as alt df = pd.DataFrame({'a': [1,2,3,4], 'b':[2000,4000,6000,8000]}) alt.Chart(df).mark_trail().encode( x=alt.X('a:Q', axis=alt.Axis(titleFontSize=12, title='Time →', labelColor='#999999', titleColor='#999999', titleAlign='right', titleAnchor='end', titleY=-30)), y=alt.Y('b:Q', axis=alt.Axis(format="$s", tickCount=3, titleFontSize=12, title='Cost →', labelColor='#999999', titleColor='#999999', titleAnchor='end')), size=alt.Size('b:Q', legend=None) ).configure_view(strokeWidth=0).configure_axis(grid=False)  The output should look like the ticks in this SO post. Note: The plot in that post has nothing to do with the demo provided here. its just for understanding purposes.

Answer: Vega-Lite calls the axis line the domain. You can hide it by passing domain=False to the axis configuration: import pandas as pd import altair as alt df = pd.DataFrame({'a': [1,2,3,4], 'b':[2000,4000,6000,8000]}) alt.Chart(df).mark_trail().encode( x=alt.X('a:Q', axis=alt.Axis(titleFontSize=12, title='Time →', labelColor='#999999', titleColor='#999999', titleAlign='right', titleAnchor='end', titleY=-30)), y=alt.Y('b:Q', axis=alt.Axis(format="$s", tickCount=3, titleFontSize=12, title='Cost →', labelColor='#999999', titleColor='#999999', titleAnchor='end')), size=alt.Size('b:Q', legend=None) ).configure_view(strokeWidth=0).configure_axis(grid=False, domain=False)
Question: I would like to create a type annotation T that describes a type that must be a subclass of both class A and class B. T = TypeVar('T', bound=A) only specifies that T must be a subclass of A. T = TypeVar('T', A, B) only specifies that T must be a subclass of A or a subclass of B but not necessarily both. I actually want something like T = TypeVar('T', bound=[A, B]) which would mean T must subclass both A and B. Is there a standard way of doing this?

Answer: What you are looking for is an intersection type. Strictly speaking, I do not believe Python's type annotations support this (at least not yet). However, you can get something similar with a Protocol: from typing import Protocol, TypeVar class A: def foo(self) -> int: return 42 class B: def bar(self) -> bool: return False class C(A, B): pass class D(A): pass class E(B): pass class ABProtocol(Protocol): def foo(self) -> int: ... def bar(self) -> bool: ... T = TypeVar('T', bound=ABProtocol) def frobnicate(obj: T) -> int: if obj.bar(): return obj.foo() return 0 frobnicate(C()) frobnicate(D()) frobnicate(E())  Mypy complains: test.py:26: error: Value of type variable "T" of "frobnicate" cannot be "D" test.py:27: error: Value of type variable "T" of "frobnicate" cannot be "E"  Of course, this requires you to explicitly annotate all the methods yourself, unfortunately, something like class ABProtocol(A, B, Protocol): pass isn't allowed
Question: I was builting a web app with streamlit, OpenCV and Torch on local machine. The whole project went well until I built a Docker file and was about to transport it to my Google Cloud Platform. Can anyone tell me what is really going wrong here?  Here is my Dockerfile: FROM pytorch/pytorch:latest RUN pip install virtualenv ENV VIRTUAL_ENV=/venv RUN virtualenv venv -p python3 ENV PATH="VIRTUAL_ENV/bin:$PATH" WORKDIR /app ADD . /app # Install dependencies RUN pip install -r requirements.txt # copying all files over COPY . /app # Expose port ENV PORT 8501 # cmd to launch app when container is run CMD streamlit run app.py # streamlit-specific commands for config ENV LC_ALL=C.UTF-8 ENV LANG=C.UTF-8 RUN mkdir -p /root/.streamlit RUN bash -c 'echo -e "\ [general]\n\ email = \"\"\n\ " > /root/.streamlit/credentials.toml' RUN bash -c 'echo -e "\ [server]\n\ enableCORS = false\n\ " > /root/.streamlit/config.toml'  And requirements.txt: albumentations==0.4.5 matplotlib==3.2.2 numpy==1.19.0 opencv-python==4.1.0.25 # opencv-python-headless==4.2.0.34 pandas==1.0.5 Pillow==7.1.2 scipy==1.5.0 streamlit==0.62.0 

Answer: Maybe, you should run following command before pip. apt update apt-get install -y libglib2.0-0 libsm6 libxrender1 libxext6
Question: I'm trying to read multiple CSV files from blob storage using python. The code that I'm using is: blob_service_client = BlobServiceClient.from_connection_string(connection_str) container_client = blob_service_client.get_container_client(container) blobs_list = container_client.list_blobs(folder_root) for blob in blobs_list: blob_client = blob_service_client.get_blob_client(container=container, blob="blob.name") stream = blob_client.download_blob().content_as_text()  I'm not sure what is the correct way to store the CSV files read in a pandas dataframe. I tried to use: df = df.append(pd.read_csv(StringIO(stream)))  But this shows me an error. Any idea how can I to do this?

Answer: Base on @sahaj-raj-malla answer: 2 snippets of code to load (or save) file from blob:  shorter load with pandas [necessary to pip install adlfs fsspec ]  import pandas as pd account_name = "my_account_stage_name" account_key = "loooooooooooooooooooooong_acccccooooooooount_keeeeeeeeeeeeeeeeey$$$$***$$$$$$$$$$$$$$22222222" connection_string = f"DefaultEndpointsProtocol=https;AccountName={account_name};AccountKey={account_key};EndpointSuffix=core.windows.net" pd.read_csv("abfs:///my_container_name/path/to/my/file/on/blob/file.csv", storage_options={"account_name": account_name, "connection_string": connection_string})   load with pandas & azure [necessary to pip install azure-storage-blob]  from azure.storage.blob import BlobServiceClient import pandas as pd account_name = "my_account_stage_name" account_key = "loooooooooooooooooooooong_acccccooooooooount_keeeeeeeeeeeeeeeeey$$$$***$$$$$$$$$$$$$$22222222" connection_string = f"DefaultEndpointsProtocol=https;AccountName={account_name};AccountKey={account_key};EndpointSuffix=core.windows.net" # load file from blob container_name = "my_container_name" blob_name = "path/to/my/file/on/blob/file.csv" blob_service_client = BlobServiceClient.from_connection_string(connection_string) container_client = blob_service_client.get_container_client(container_name) blob_client = container_client.get_blob_client(blob_name) # load to RAM, eg. jupyter notebook pd.read_csv(blob_client.download_blob()) # save file to ROM, eg. local file local_file_name = "path/to/my/file/on/disk/file.csv" with open(local_file_name, "wb") as my_blob_locally: download_stream = blob_client.download_blob() my_blob_locally.write(download_stream.readall())   How to get Connections string  goto Storage account -> Access Keys -> Show and copy Connection string
Question: Below is a python script that calculates the homography between two images and then map a desired point from one image to another import cv2 import numpy as np if __name__ == '__main__' : # Read source image. im_src = cv2.imread(r'C:/Users/kjbaili/.spyder-py3/webcam_calib/homography/khaledd 35.0 sec.jpg') # Five corners of the book in source image pts_src = np.array([[281, 238], [325, 297], [283, 330],[248, 325],[213, 321]]) # Read destination image. im_dst = cv2.imread(r'C:/Users/kjbaili/.spyder-py3/webcam_calib/homography/20.jpg') # Five corners of the book in destination image. pts_dst = np.array([[377, 251],[377, 322],[316, 315],[289, 284],[263,255]]) # Calculate Homography h, status = cv2.findHomography(pts_src, pts_dst) # provide a point i wish to map from image 1 to image 2 a = np.array([[260, 228]]) pointsOut = cv2.getPerspectiveTransform(a, h) # Display image cv2.imshow("treced_point_image", pointsOut) cv2.waitKey(0) cv2.destroyAllWindows()  However, when i display the image that contains the mapped point it returns the following error: error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\core\src\matmul.dispatch.cpp:531: error: (-215:Assertion failed) scn + 1 == m.cols in function 'cv::perspectiveTransform'  According to my knowledge this error means that parameter assigned to the function perspective transform is not correct or not being read. I checked the two images at the reading step and everything is fine. So anyone knows why this happens? Thanks in advance Khaled

Answer: You are passing wrong arguments to cv2.getPerspectiveTransform(). The function expects a set of four coordinates in the original image and the new coordinates in the transformed image. You can directly pass the pts_src and pts_dst to the function and you will get the transformation matrix. You can then get the transformed coordinates for point "a" by matrix multiplication like a_transformed = np.dot(matrix, a).
Question: I have a basic datatable in plotly/dash. My goal is to upload (or print for the sake of the example...) after I press the upload-button. The issue is, that I can't figure out how to get the n_clicks attribute of the button back to zero. So what happens is that after I clicked the button for the first time it prints continuously whenever something changes (row added or number changed/added), but what I want is for it to print only once whenever I click the button. This is the code: import dash from dash.dependencies import Input, Output, State import dash_table import dash_daq as daq import dash_core_components as dcc import dash_html_components as html import pandas as pd external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css'] df = pd.read_csv('.../dummy_csv.csv') app = dash.Dash(__name__) app.layout = html.Div([ html.Div(id='my-div'), dash_table.DataTable( id='adding-rows-table', style_data={ 'whiteSpace': 'normal', 'height': 'auto' }, style_table={ 'maxHeight': '800px' , 'overflowY': 'scroll' }, columns=[ {'name': i, 'id': i} for i in df.columns ], data=df.to_dict('records'), editable=True, row_deletable=True ), html.Button('+ Row', id='editing-rows-button', n_clicks=0), html.Button('Update', id='btn-nclicks-1', n_clicks=0 ), ]) @app.callback( Output(component_id='my-div', component_property='children'), [Input('btn-nclicks-1', 'n_clicks'), Input('adding-rows-table', 'data')] ) def update_output_div(n_clicks, data): if n_clicks > 0: print(data) # n_clicks = 0 # return n_clicks else: print("else loop") @app.callback( Output('adding-rows-table', 'data'), [Input('editing-rows-button', 'n_clicks')], [State('adding-rows-table', 'data'), State('adding-rows-table', 'columns')]) def add_row(n_clicks, rows, columns): if n_clicks > 0: rows.append({c['id']: '' for c in columns}) return rows if __name__ == '__main__': app.run_server(debug=True)  This the CSV: a,b,c,d 1,1,5,1 2,2,5,1 2,33,6,2 3,4,6,2  And this is the "faulty" output.

Answer: You could use the dash.callback_context property to trigger the callback only when the number of clicks has changed rather than after the first click. See the section on "Determining which Button Changed with callback_context" in the Dash documentation. The following is an example of how you could update your callback. @app.callback(Output(component_id='my-div', component_property='children'), [Input('btn-nclicks-1', 'n_clicks'), Input('adding-rows-table', 'data')]) def update_output_div(n_clicks, data): changed_id = [p['prop_id'] for p in dash.callback_context.triggered][0] if 'btn-nclicks-1' in changed_id: print(data) # n_clicks = 0 # return n_clicks else: print("else loop")
Question: Basically a distilled down version of this (as yet unanswered) question. I want to state that a variable should only take on values that are keys in a TypedDict. At present I'm defining a separate Literal type to represent the keys, for example: from typing import Literal, TypedDict class MyTD(TypedDict): a: int b: int mytd = MyTD(a=1, b=2) key = "a" mytd[key] # error: TypedDict key must be a string literal; expected one of ('a', 'b') MyTDKeyT = Literal["a", "b"] typed_key: MyTDKeyT = "b" mytd[typed_key] # no error  I would like to be able to replace the Literal definition for all the usual reasons of wanting to minimize duplicated code. Pseudo-code: key: Keys[MyTD] = "a" mytd[key] # would be no error not_key: Keys[MyTD] = "z" # error  Is there a way to achieve this? To clarify, given that mypy can tell me that the key type needs to be a literal of "a" or "b", I'm hoping there might be a less error prone way to annotate a variable to that type, rather than having to maintain two separate lists of keys side-by-side, once in the TypedDict definition, once in the Literal definition.

Answer: Using MyPy, I don't think this is possible. I ran this experiment: from typing import TypedDict class MyTD(TypedDict): a: str b: int d = MyTD(a='x', b=2) reveal_type(list(d))  The MyPy output was: Revealed type is "builtins.list[builtins.str]"  This indicates that internally it is not tracking the keys as literals. Otherwise, we would expect: Revealed type is "builtins.list[Literal['A', 'B']]"  Also, this errors out in MyPy, so __required_keys__ isn't even inspectable: reveal_type(MyTD.__required_keys__)
Question: I needed to create a relative path starting with the current directory as a "." dot For example, in windows ".\envs\.some.env" or "./envs/.some.env" elsewhere I wanted to do this using pathlib. A solution was found, but it has a kludgy replace statement. Is there a better way to do this using pathlib? The usage was django-environ, and the goal was to support multiple env files. The working folder contained an envs folder with the multiple env files within that folder. import environ from pathlib import Path import os domain_env = Path.cwd() dotdot = Path("../") some_env = dotdot / "envs" / ".some.env" envsome = environ.Env() envsome.read_env(envsome.str(str(domain_env), str(some_env).replace("..", "."))) print(str(some_env)) print(str(some_env).replace("..", ".")) dot = Path("./") # Path(".") gives the same result some_env = dot / "envs" / ".some.env" print(str(some_env))  On windows gives: ..\envs\.some.env .\envs\.some.env envs\.some.env 

Answer: Here's a multi-platform idea: import ntpath import os import posixpath from pathlib import Path, PurePosixPath, PureWindowsPath def dot_path(pth): """Return path str that may start with '.' if relative.""" if pth.is_absolute(): return os.fsdecode(pth) if isinstance(pth, PureWindowsPath): return ntpath.join(".", pth) elif isinstance(pth, PurePosixPath): return posixpath.join(".", pth) else: return os.path.join(".", pth) print(dot_path(PurePosixPath("file.txt"))) # ./file.txt print(dot_path(PureWindowsPath("file.txt"))) # .\file.txt print(dot_path(Path("file.txt"))) # one of the above, depending on host OS print(dot_path(Path("file.txt").resolve())) # (e.g.) /path/to/file.txt
Question: I try to predict 10 classes using this code #Predicting the Test set rules y_pred = model.predict(traindata) y_pred = np.argmax(y_pred, axis=1) y_true = np.argmax(testdata, axis=1) target_names = ["akLembut","akMundur","akTajam","caMenaik", "caMenurun", "coretanTengah", "garisAtas", "garisBawah", "garisBawahBanyak", "ttdCangkang"] print("\n"+ classification_report(y_true, y_pred, target_names=target_names))  But then I got an error message like this AxisError Traceback (most recent call last) <ipython-input-13-a2b02b251547> in <module>() 2 y_pred = model.predict(traindata) 3 y_pred = np.argmax(y_pred, axis=1) ----> 4 y_true = np.argmax(testdata, axis=1) 5 6 target_names = ["akLembut","akMundur","akTajam","caMenaik", "caMenurun", "coretanTengah", "garisAtas", "garisBawah", "garisBawahBanyak", "ttdCangkang"] <__array_function__ internals> in argmax(*args, **kwargs) 2 frames /usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py in _wrapit(obj, method, *args, **kwds) 45 except AttributeError: 46 wrap = None ---> 47 result = getattr(asarray(obj), method)(*args, **kwds) 48 if wrap: 49 if not isinstance(result, mu.ndarray): AxisError: axis 1 is out of bounds for array of dimension 1  I already train the data and I need to know each accuracy.

Answer: My guess is that your test_data array is only one-dimensional, so change to y_true = np.argmax(testdata, axis=0)
Question: I am trying out Poetry in an existing project. It used pyenv and virtual env originally so I have a requirements.txt file with the project's dependencies. I want to import the requirements.txt file using Poetry, so that I can load the dependencies for the first time. I've looked through poetry's documentation, but I haven't found a way to do this. Is there a way to do it? I know that I can add all packages manually, but I was hoping for a more automated process, because there are a lot of packages.

Answer: poetry doesn't support this directly. But if you have a handmade list of required packages (at best without any version numbers), that only contain the main dependencies and not the dependencies of a dependency you could do this: $ cat requirements.txt | xargs poetry add
Question: I have a very basic playbook that simply runs a script using the shell module on the target remote host. In the output it however fails stating python interpreter not found. Installing python on each target is not the solution I can pursue. Is it possible to use my Ansible automation to run the playbook and execute the script using shell module without having python dependency?

Answer: Any ansible operation requires python on the target node except the raw and script modules. Please note that these two modules are primarily meant to install ansible requirements (i.e. Python and its mandatory modules) on targets where they are missing. In other words, Python is definitely a requirement to run ansible following all best practices (e.g. using modules when they exists, creating idempotent tasks...). If installing Python on your targets is not an option, don't use ansible, choose an other tool. References:  Ansible managed node requirements raw module script module