eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question:  Hi Guys, I am trying to install spacy model == 2.3.5 but I am getting this error, please help me!

Answer: I had the similar error while executing pip install -r requirements.txt but for aiohttp module: socket.c -o build/temp.linux-armv8l-cpython-311/aiohttp/_websocket.o aiohttp/_websocket.c:198:12: fatal error: 'longintrepr.h' file not found #include "longintrepr.h" ^~~~~~~ 1 error generated. error: command '/data/data/com.termux/files/usr/bin/arm-linux-androideabi-clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for aiohttp Failed to build aiohttp ERROR: Could not build wheels for aiohttp, which is required to install pyproject.toml-based projects  Just in case I will leave here solution to my error. This error is specific to Python 3.11 version. On Python with 3.10.6 version installation went fine. To solve it I needed to update requirements.txt. Not working versions of modules with Python 3.11: aiohttp==3.8.1 yarl==1.4.2 frozenlist==1.3.0  Working versions: aiohttp==3.8.2 yarl==1.8.1 frozenlist==1.3.1  Links to the corresponding issues with fixes:  https://github.com/aio-libs/aiohttp/issues/6600 https://github.com/aio-libs/yarl/issues/706 https://github.com/aio-libs/frozenlist/issues/305
Question: What is the correct format for supplying a name to a Python package in a pyproject.toml? Here's the pyproject.toml file: [project] name = "foobar" version = "0.0.1" [build-system] requires = ["setuptools>=40.8.0", "wheel"] build-backend = "setuptools.build_meta"  A build called using python -m build results in the following error. running check warning: check: missing required meta-data: name, url warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied  Based on this reddit post question I had the same issue.

Answer: Update At the time the question was asked, setuptools did not have support for writing its configuration in a pyproject.toml file (PEP 621). So it was not possible to answer the question. Now and since its version 61.0.0, setuptools has support for PEP 621:  https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html   Original answer It seems that you are trying to write a PEP 621-style pyproject.toml with the setuptools build back-end. But, as of now, setuptools does not have support for PEP 621 yet. The work is ongoing:  https://discuss.python.org/t/help-testing-experimental-features-in-setuptools/13821 https://github.com/pypa/setuptools/tree/experimental/support-pyproject https://github.com/pypa/setuptools/search?q=621&type=issues  Until PEP 621 support arrives in setuptools, one can:  Use setup.cfg  https://setuptools.pypa.io/en/latest/userguide/declarative_config.html   Switch to a PEP 621-compatible build back-end instead of setuptools:  pdm flit trampolim enscons whey probably more...
Question: After updating Python package elasticsearch from 7.6.0 to 8.1.0, I started to receive an error at this line of code: count = es.count(index=my_index, body={'query': query['query']} )["count"]  receive following error message:  DeprecationWarning: The 'body' parameter is deprecated and will be removed in a future version. Instead use individual parameters. count = es.count(index=ums_index, body={'query': query['query']} )["count"]  I don't understand how to use the above-mentioned "individual parameters". Here is my query: query = { "bool": { "must": [ {"exists" : { "field" : 'device'}}, {"exists" : { "field" : 'app_version'}}, {"exists" : { "field" : 'updatecheck'}}, {"exists" : { "field" : 'updatecheck_status'}}, {"term" : { "updatecheck_status" : 'ok'}}, {"term" : { "updatecheck" : 1}}, { "range": { "@timestamp": { "gte": from_date, "lte": to_date, "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd" } } } ], "must_not": [ {"term" : { "device" : ""}}, {"term" : { "updatecheck" : ""}}, {"term" : { "updatecheck_status" : ""}}, { "terms" : { "app_version" : ['2.2.1.1', '2.2.1.2', '2.2.1.3', '2.2.1.4', '2.2.1.5', '2.2.1.6', '2.2.1.7', '2.1.2.9', '2.1.3.2', '0.0.0.0', ''] } } ] } }  In the official documentation, I can't find any chance to find examples of how to pass my query in new versions of Elasticsearch. Possibly someone has a solution for this case other than reverting to previous versions of Elasticsearch?

Answer: According to the documentation, this is now to be done as follows: # ✅ New usage: es.search(query={...}) # ❌ Deprecated usage: es.search(body={"query": {...}})  So the queries are done directly in the same line of code without "body", substituting the api you need to use, in your case "count" for "search". You can try the following: # ✅ New usage: es.count(query={...}) # ❌ Deprecated usage: es.count(body={"query": {...}}) enter code here  You can find out more by clicking on the following link: https://github.com/elastic/elasticsearch-py/issues/1698 For example, if the query would be: GET index-00001/_count { "query" : { "match_all": { } } }  Python client would be the next: my_index = "index-00001" query = { "match_all": { } } hits = en.count(index=my_index, query=query)  or hits = en.count(index=my_index, query={"match_all": {}})
Question: I am writing some Python code to create an order with the Binance API: from binance.client import Client client = Client(API_KEY, SECRET_KEY) client.create_order(symbol='BTCUSDT', recvWindow=59999, #The value can't be greater than 60K side='BUY', type='MARKET', quantity = 0.004)  Unfortunately I get the following error message: "BinanceAPIException: APIError(code=-1021): Timestamp for this request was 1000ms ahead of the server's time."  I already checked the difference (in miliseconds) between the Binance server time and my local time: import time import requests import json url = "https://api.binance.com/api/v1/time" t = time.time()*1000 r = requests.get(url) result = json.loads(r.content) print(int(t)-result["serverTime"]) OUTPUT: 6997  It seems that the recvWindow of 60000 is still not sufficient (but it may not exceed 60K). I still get the same error. Does anybody know how I can solve this issue? Many thanks in advance!

Answer: Probably the PC's time is out of sync. You can do it using Windows -> Setting-> Time & Language -> Date & Time -> 'Sync Now'. Screenshot:
Question: I installed Christoph Gohlke's prebuilt wheel Cartopy‑0.20.2‑cp39‑cp39‑win_amd64.whl using pip in an active virtual environment. The environment is using Python 3.9.5. When trying to import Cartopy I get the error message below. This used to work before and now it no longer works and I can't figure out why. Does anyone know what the issue could be or what I'm missing? --------------------------------------------------------------------------- ImportError Traceback (most recent call last) Input In [4], in <cell line: 1>() ----> 1 import cartopy 2 import cartopy.crs as ccrs 3 import cartopy.io.img_tiles as cimgt File ~\Downloads\GitHub\Project\venv\lib\site-packages\cartopy\__init__.py:110, in <module> 105 pass 108 # Commonly used sub-modules. Imported here to provide end-user 109 # convenience. --> 110 import cartopy.crs 111 import cartopy.feature File ~\Downloads\GitHub\Project\venv\lib\site-packages\cartopy\crs.py:27, in <module> 24 from pyproj.exceptions import ProjError 25 from shapely.prepared import prep ---> 27 import cartopy.trace 30 try: 31 # https://github.com/pyproj4/pyproj/pull/912 32 from pyproj.crs import CustomConstructorCRS as _CRS ImportError: DLL load failed while importing trace: The specified module could not be found. 

Answer: As mentioned by cgohlke in the comments, installing the wheels of shapely and pyproj from his website solves the issue. If the libraries are already installed, use --force-reinstall to overwrite the existing installations.
Question: I am trying to be sure that I understand it correctly: Is OAuth a bridge for only third party authenticator those so common like Facebook, Google? And using it improves user experience in secure way but not adding extra secure layer to Django login framework? Or only Authorization Code grant type is like that? Can I take it like this?

Answer: What is OAuth? According to RFC 6749:  The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.  Essentially, it is an authorization protocol used to share permissions between multiple applications. If you decide to implement OAuth, your application will be the one to allow other services to programmatically view your users' data and act on their behalf, if needed. Whenever an application requires access to another service that you use, it probably uses OAuth to perform those actions. (e.g. When games used to ask us to allow posting on Facebook on our behalf.) What OAuth is not? By looking at your question, I feel like there's a misunderstanding of OAuth. OAuth is not a bridge for third-party authentication methods. If you are looking for this type of authentication mechanism, you should take a look into Single Sign-On (SSO). For Django, you can use django-simple-sso. Does it enhance security? Depending on the use case, yes, it can enhance security. If your application needs to exchange information with other services, it is a good practice to limit what these third-party services are able to do in your app, feature and time-wise. Let's say, for example, that your user needs to give permission to another application to gather information from yours:  If you were to use the old-fashioned e-mail and password combination method, these credentials would be exposed in case of this third-party service had a data breach. Using OAuth on the other hand is much more secure, as the credentials stored in the server would not contain the user's password and have very specific roles, apart from being easily revoked.
Question: I have df with two columns A and B both of them are columns with string values. Example: df_1 = pd.DataFrame(data={ "A":['a','b','c'], "B":['a x d','z y w','q m c'] #string values not a list }) print(df_1) #output A B 0 a a x d 1 b z y w 2 c q m c  now what I'm trying to do is to preform loc in the df_1 to get all the row that col B cointain the string value in col A. In this example the output i want is the first and the third rows:  A B 0 a a x d # 'a x d' contain value 'a' 2 c q m c # 'q m c' contain value 'c'  I have tried different loc condition but got unhashable type: 'Series' error: df_1.loc[df_1["B"].str.contains(df_1["A"])] #TypeError: unhashable type: 'Series' df_1.loc[df_1["A"] in df_1["B"]] #TypeError: unhashable type: 'Series'  I really don't want to use a for/while loop because of the size of the df. Any idea how can I preform this?

Answer: There is no vectorial method, to map in using two columns. You need to loop here: mask = [a in b for a,b in zip(df_1['A'], df_1['B'])] df_1.loc[mask]  Output:  A B 0 a a x d 2 c q m c  comparison of speed (3000 rows) # operator.contains 518 µs ± 4.61 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) # list comprehension 554 µs ± 3.84 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) # numpy.apply_along_axis 7.32 ms ± 58.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # apply 20.7 ms ± 379 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
Question: Is it possible, using the docker SDK for Python, to launch a container in a remote machine? import docker client = docker.from_env() client.containers.run("bfirsh/reticulate-splines", detach=True) # I'd like to run this container ^^^ in a machine that I have ssh access to.  Going through the documentation it seems like this type of management is out of scope for said SDK, so searching online I got hints that the kubernetes client for Python could be of help, but don't know where to begin.

Answer: It's possible, simply do this: client = docker.DockerClient(base_url=your_remote_docker_url)  Here's the document I found related to this: https://docker-py.readthedocs.io/en/stable/client.html#client-reference  If you only have SSH access to it, there is an use_ssh_client option
Question: I have installed the psycopg2 with this command in my package folder : pip install --target ./package psycopg2 # Or pip install -t ./package psycopg2  now psycopg2 module is in my package and I have created the zip and upload it in AWS lambda. In my local sprint is working fine but on AWS lambda it was not working. It shows me error { "errorMessage": "Unable to import module 'lambda_function': No module named 'psycopg2._psycopg'", "errorType": "Runtime.ImportModuleError", "stackTrace": [] }  my lambda code is import psycopg2 def lambda_handler(): print('hello')  my all other modules are working fine

Answer: add this lib pip install aws-psycopg2
Question: I was recently using Jupyter lab and decided to update my pandas version from 1.2 to the latest (1.4). So I ran 'conda update pandas' which seemed to work fine. However when I then launched Jupyter lab in the usual way 'jupyter lab' and tried to open the workbook I had just been working on I got the below error:  Unreadable Notebook: C:\Users...\script.ipynb TypeError("init() got an unexpected keyword argument 'capture_validation_error'")  I am getting this same error when trying to open any of my .ipynb files that were previously working fine. I can also open them fine in jupyter notebook, but for some reason they don't work in Jupyter lab anymore. Any idea how I can fix this? Thanks

Answer: It turns out that a recent update to jupyter_server>=1.15.0 broke compatibility with nbformat<5.2.0, but did not update the conda recipe correctly per this Github pull request. It is possible that while updating pandas, you may have inadvertently also updated jupyterlab and/or jupyter_server. While we wait for the build with the merged PR to come downstream, we can fix this dependency issue by updating nbformat manually with conda install -c conda-forge nbformat  to get the newest version of nbformat with version >=5.2.
Question: Of all the answers I see on stackoverflow, such as 1, 2 and 3 are color-coded. In my case, I wouldn´t like it to be colored, especially since my dataset is largely imbalanced, minority classes are always shown in light color. I would instead, prefer it display the number of actual/predicted in each cell. Currently, I use: def plot_confusion_matrix(cm, classes, title, normalize=False, file='confusion_matrix', cmap=plt.cm.Blues): if normalize: cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis] cm_title = "Normalized confusion matrix" else: cm_title = title # print(cm) plt.imshow(cm, interpolation='nearest', cmap=cmap) plt.title(cm_title) plt.colorbar() tick_marks = np.arange(len(classes)) plt.xticks(tick_marks, classes, rotation=45) plt.yticks(tick_marks, classes) fmt = '.3f' if normalize else 'd' thresh = cm.max() / 2. for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])): plt.text(j, i, format(cm[i, j], fmt), horizontalalignment="center", color="white" if cm[i, j] > thresh else "black") plt.tight_layout() plt.ylabel('True class') plt.xlabel('Predicted class') plt.tight_layout() plt.savefig(file + '.png')  Output:  So I want the number shown only.

Answer: Use seaborn.heatmap with a grayscale colormap and set vmin=0, vmax=0: import seaborn as sns sns.heatmap(cm, fmt='d', annot=True, square=True, cmap='gray_r', vmin=0, vmax=0, # set all to white linewidths=0.5, linecolor='k', # draw black grid lines cbar=False) # disable colorbar # re-enable outer spines sns.despine(left=False, right=False, top=False, bottom=False)   Complete function: def plot_confusion_matrix(cm, classes, title, normalize=False, file='confusion_matrix', cmap='gray_r', linecolor='k'): if normalize: cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis] cm_title = 'Confusion matrix, with normalization' else: cm_title = title fmt = '.3f' if normalize else 'd' sns.heatmap(cm, fmt=fmt, annot=True, square=True, xticklabels=classes, yticklabels=classes, cmap=cmap, vmin=0, vmax=0, linewidths=0.5, linecolor=linecolor, cbar=False) sns.despine(left=False, right=False, top=False, bottom=False) plt.title(cm_title) plt.ylabel('True class') plt.xlabel('Predicted class') plt.tight_layout() plt.savefig(f'{file}.png')
Question: I want to how we can assign one column value to another column if it has null or 0 value I have a dataframe like this: id column1 column2 5263 5400 5400 4354 6567 Null 5656 5456 5456 5565 6768 3489 4500 3490 Null  The Expected Output is id column1 column2 5263 5400 5400 4354 6567 6567 5656 5456 5456 5565 6768 3489 4500 3490 3490  that is, if df['column2'] = Null/0 then it has take df['column1'] value. Can someone explain, how can I achieve my desired output?

Answer: Based on the answers to this similar question, you can do the following:  Using np.where: df['column2'] = np.where((df['column2'] == 'Null') | (df['column2'] == 0), df['column1'], df['column2'])   Instead, using only pandas and Python: df['column2'][(df['column2'] == 0) | (df['column2'] == 'Null')] = df['column1']
Question: Want to add MyPy checker to my html scraper. I manage to fix all errors except this one Expected type in class pattern. Source code: from bs4 import BeautifulSoup from bs4.element import Tag, NavigableString soup = BeautifulSoup(""" <!DOCTYPE html> <html> <body> EXTRA TEXT <p> first <b>paragraph</b> <br> <br> second paragraph </p> </body> </html> """, "lxml") tag = soup.select_one('body') for el in tag.children: match el: case NavigableString(): ... case Tag(name="p"): ... case Tag(): ...  mypy example.py Errors: example.py:24: error: Expected type in class pattern; found "Any" example.py:26: error: Expected type in class pattern; found "Any" example.py:28: error: Expected type in class pattern; found "Any" Found 3 errors in 1 file (checked 1 source file)  So, what does this error mean? And how can I fix it?

Answer: You can use TYPE_CHECKING to load classes that have the typing from typing import TYPE_CHECKING if TYPE_CHECKING: class NavigableString: ... class Tag: children: list[NavigableString | Tag] name: str class BeautifulSoup: def __init__(self, markup: str, features: str | None) -> None: ... def select_one(self, text: str) -> Tag: ... else: from bs4 import BeautifulSoup from bs4.element import Tag, NavigableString soup = BeautifulSoup( """ <!DOCTYPE html> <html> <body> EXTRA TEXT <p> first <b>paragraph</b> <br> <br> second paragraph </p> </body> </html> """, "lxml", ) tag = soup.select_one("body") for el in tag.children: match el: case NavigableString(): ... case Tag(name="p"): ... case Tag(): ...
Question: After updating to pandas 1.4, I now receive the following warning when using frame.append to append a dictionary to a Pandas DataFrame. FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.  Below is the code. This still works, though I would like to resolve the warning. report = report.append({ "period":period, "symbol":symbol, "start_date":start_date, "start_price":start_price, "start_market_cap":start_market_cap, "end_date":end_date, "end_price":end_price, "end_market_cap":end_market_cap, "return":return_ },ignore_index=True)  I have updated the code to the below, which kicks a different warning: report = pd.concat([report,{ "period":period, "symbol":symbol, "start_date":start_date, "start_price":start_price, "start_market_cap":start_market_cap, "end_date":end_date, "end_price":end_price, "end_market_cap":end_market_cap, "return":return_ }],ignore_index=True) TypeError: cannot concatenate object of type '<class 'dict'>'; only Series and DataFrame objs are valid  2 questions: Is the first warning wrong? What is the pandas 1.4 way to achieve this? Thanks.

Answer: Use loc to assign a single row value: report.loc[len(report)] = {"period":period, "symbol":symbol, "start_date":start_date, "start_price":start_price, "start_market_cap":start_market_cap, "end_date":end_date, "end_price":end_price, "end_market_cap":end_market_cap, "return":return_ }
Question: So I am rearranging a list based on an index pattern and would like to find a way to calculate the pattern I need to revert the list back to its original order. for my example I am using a list of 5 items as I can work out the pattern needed to revert the list back to its original state. However this isn't so easy when dealing with 100's of list items. def rearrange(pattern: list, L: list): new_list = [] for i in pattern: new_list.append(L[i-1]) return new_list print(rearrange([2,5,1,3,4], ['q','t','g','x','r'])) #['t', 'r', 'q', 'g', 'x']  and in order to set it back to the original pattern I would use print(rearrange([3,1,4,5,2],['t', 'r', 'q', 'g', 'x'])) #['q', 't', 'g', 'x', 'r']  What I am looking for is a way to calculate the pattern "[3,1,4,5,2]" regarding the above example. whist running the script so that I can set the list back to its original order. Using a larger example: print(rearrange([18,20,10,11,13,1,9,12,16,6,15,5,3,7,17,2,19,8,14,4],['e','p','b','i','s','r','q','h','m','f','c','g','d','k','l','t','a','n','j','o'])) #['n', 'o', 'f', 'c', 'd', 'e', 'm', 'g', 't', 'r', 'l', 's', 'b', 'q', 'a', 'p', 'j', 'h', 'k', 'i']  but I need to know the pattern to use with this new list in order to return it to its original state. print(rearrange([???],['n', 'o', 'f', 'c', 'd', 'e', 'm', 'g', 't', 'r', 'l', 's', 'b', 'q', 'a', 'p', 'j', 'h', 'k', 'i'])) #['e','p','b','i','s','r','q','h','m','f','c','g','d','k','l','t','a','n','j','o'] 

Answer: This is commonly called "argsort". But since you're using 1-based indexing, you're off-by-one. You can get it with numpy: >>> pattern [2, 5, 1, 3, 4] >>> import numpy as np >>> np.argsort(pattern) + 1 array([3, 1, 4, 5, 2])  Without numpy: >>> [1 + i for i in sorted(range(len(pattern)), key=pattern.__getitem__)] [3, 1, 4, 5, 2]