eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have created AWS elasticsearch domain https://search-xx-xx.us-east-1.es.amazonaws.com/ On click both elastic url and kibana below is the error i got {"Message":"User: anonymous is not authorized to perform: es:ESHttpGet"}  Below is code which is working fine import boto3 from requests_aws4auth import AWS4Auth from elasticsearch import Elasticsearch, RequestsHttpConnection session = boto3.session.Session() credentials = session.get_credentials() awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, session.region_name, 'es', session_token=credentials.token) es = Elasticsearch( ['https://search-testelastic-2276kyz2u4l3basec63onfq73a.us-east-1.es.amazonaws.com'], http_auth=awsauth, use_ssl=True, verify_certs=True, connection_class=RequestsHttpConnection ) def lambda_handler(event, context): es.cluster.health() es.indices.create(index='my-index', ignore=400) r = [{'Name': 'Dr. Christopher DeSimone', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Tajwar Aamir (Aamir)', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Bernard M. Aaron', 'Specialised and Location': 'Health'}, {'Name': 'Eliana M. Aaron', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Joseph J. Aaron', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Michael R. Aaron', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Darryl H. Aarons', 'Specialised and Location': 'Health'}, {'Name': 'Dr. William B. Aarons', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Sirike T. Aasmaa', 'Specialised and Location': 'Health'}, {'Name': 'Dr. Jacobo A. Abadi', 'Specialised and Location': 'Health'}] for e in enumerate(r): es.index(index="my-index", body=e[1])  Below is the access policy { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "es:*", "Resource": "arn:aws:es:us-east-1:xxxxxx:domain/xxxxx/*", "Condition": { "IpAddress": { "aws:SourceIp": "*" } } } ] } 

Answer: This error would indicate your ElasticSearch service does not support anonymous requests (those not signed with valid IAM credentials). Although your policy sees ok the official allow all policy looks like the below { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "es:*", "Resource": "arn:aws:es:us-east-1:xxxxxx:domain/xxxxx/*" } ] }
Question: For example: def foo(bar: int = None): pass  When I check a type/annotation of bar pycharm tells me that it is Optional[int]. bar: int = None looks much cleaner rather then bar: Optional[int] = None, especially when you have 10+ parameters. So can I simply omit Optional? Will tools like mypy or other linters highlight this case as en error? Looks like python itself doesn't like the idea: In [1]: from typing import Optional In [2]: from inspect import signature In [3]: def foo(a: int = None): pass In [4]: def bar(a: Optional[int] = None): pass In [5]: signature(foo).parameters['a'].annotation Out[5]: int In [6]: signature(bar).parameters['a'].annotation Out[6]: typing.Union[int, NoneType] 

Answer: No. Omitting Optional was previously allowed, but has since been removed.  A past version of this PEP allowed type checkers to assume an optional type when the default value is None [...] This is no longer the recommended behavior. Type checkers should move towards requiring the optional type to be made explicit.  Some tools may still provide the old behaviour for legacy support. Even if that is the case, do not rely on it being supported in the future.  In specific, mypy still supports implicit Optional by default, but explicitly notes this may change in the future:  Optional types and the None type (mypy v0.782) [...] You can use the --no-implicit-optional command-line option to stop treating arguments with a None default value as having an implicit Optional[...] type. It’s possible that this will become the default behavior in the future.  The deprecation of this behaviour is tracked in mypy/#9091
Question: I want to get the type hints for an object's attributes. I can only get the hints for the class and not an instance of it. I have tried using foo_instance.__class__ from here but that only shows the class variables. So in the example how do I get the type hint of bar? class foo: var: int = 42 def __init__(self): self.bar: int = 2 print(get_type_hints(foo)) # returns {'var': <class 'int'>} 

Answer: This information isn't evaluated and only exists in the source code. if you must get this information, you can use the ast module and extract the information from the source code yourself, if you have access to the source code. You should also ask yourself if you need this information because in most cases reevaluating the source code will be to much effort.
Question: I am building a python package and I am using Spacy library and Spacy model en_core_web_md. It can't be installed using pip. You can install it like this python -m spacy download en_core_web_md  I have place en_core_web_md folder in my Python package.  simple_eda  init.py simple_eda.py en_core_web_md   tests  setup.py  README.md  LICENSE   I can install package successfully but when I import, it gives me this error. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/shahid/anaconda3/envs/eda_test_6/lib/python3.5/site-packages/simple_nlp/__init__.py", line 1, in <module> from simple_nlp.simple_nlp import SimpleNLP File "/home/shahid/anaconda3/envs/eda_test_6/lib/python3.5/site-packages/simple_nlp/simple_nlp.py", line 22, in <module> nlp = spacy.load("en_core_web_md") File "/home/shahid/anaconda3/envs/eda_test_6/lib/python3.5/site-packages/spacy/__init__.py", line 30, in load return util.load_model(name, **overrides) File "/home/shahid/anaconda3/envs/eda_test_6/lib/python3.5/site-packages/spacy/util.py", line 175, in load_model raise IOError(Errors.E050.format(name=name)) OSError: [E050] Can't find model 'en_core_web_md'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.  Where should I place the folder, or should I add a link to that folder in setup.py file?

Answer: This solved my issue. try: nlp = spacy.load('en') except OSError: print('Downloading language model for the spaCy POS tagger\n' "(don't worry, this will only happen once)", file=stderr) from spacy.cli import download download('en') nlp = spacy.load('en')
Question: Is it possible to add some text on the same html file as my plotly graph? For example : This is the code that generates a graph : data = pd.read_csv('file.csv') data.columns = ['price', 'place', 'date'] fig = px.scatter(data, x = "place", y = "price", ) fig.write_html("done.html")  This graph will generate a pyplot graph in an html file and I want to add some simple text (such as a conclusion line explaning the graph) under the graph. This is an example of the output I would like: ly

Answer: You can use fig.update_layout(margin=dict()) to make room for an explanation, and then fig.add_annotation() to insert any text you'd like below the figure utself to get this:  Complete code: import plotly.graph_objects as go import numpy as np x = np.arange(-4,5) y=x**3 yticks=list(range(y.min(), y.max(), 14)) #yticks.append(y.max())9 # build figure fig = go.Figure(data=go.Scatter(x=x, y=y)) # make space for explanation / annotation fig.update_layout(margin=dict(l=20, r=20, t=20, b=60),paper_bgcolor="LightSteelBlue") # add annotation fig.add_annotation(dict(font=dict(color='yellow',size=15), x=0, y=-0.12, showarrow=False, text="A very clear explanation", textangle=0, xanchor='left', xref="paper", yref="paper")) fig.show()
Question: First, I tired this: @pytest.mark.django_db @pytest.fixture(scope='session') def created_user(django_db_blocker): with django_db_blocker.unblock(): return CustomUser.objects.create_user("User", "UserPassword") def test_api_create(created_user): user = created_user() assert user is not None  But I got an UndefinedTable error. So marking my fixture with @pytest.mark.django_db somehow didn’t actually register my Django DB. So next I tried pass the db object directly to the fixture: @pytest.fixture(scope='session') def created_user(db, django_db_blocker): with django_db_blocker.unblock(): return CustomUser.objects.create_user("User", "UserPassword") def test_api_create(created_user): user = created_user() assert user is not None  But then I got an error ScopeMismatch: You tried to access the 'function' scoped fixture 'db' with a 'session' scoped request object, involved factories  So finally, just to confirm everything was working, I tried: @pytest.fixture def created_user(db, django_db_blocker): with django_db_blocker.unblock(): return CustomUser.objects.create_user("User", "UserPassword") def test_api_create(created_user): user = created_user() assert user is not None  This works just fine, but now my create_user function is being called every single time my function is being setup or torn down. Whats the solution here?

Answer: @hoefling had the answer, I needed to pass django_db_setup instead. @pytest.fixture(scope='session') def created_user(django_db_setup, django_db_blocker): with django_db_blocker.unblock(): return CustomUser.objects.create_user("User", "UserPassword")
Question: Dear smart people of stackoverflow, I know this question has been asked a lot here but none of the posted solutions have worked for me as of yet. Any help here would be much appreciated: The Problem: Cannot connect to flask app server from other devices (PCs, mobiles) on the same network. (in other words: localhost works perfectly but I cannot connect from external device) What I've Tried: 1) Setting app.run(host='0.0.0.0', port=5000, debug=True, threaded=True) in the app.py so that the server will listen on all available network interfaces. 2) Enabling TCP traffic for port 5000 in local network in Windows Defender Firewall (with inbound and outbound rules added) 3) Using my host PC's IPv4 address in the URL bar of my external device's browser in the following format: http://<host_ipaddress>:<port>/ 4) Using my host PC's hostname in the URL bar of my external device's browser in the following format: http://<host_name>:<port>/ 5) Running the app.py file from Windows Powershell and Python (.py) Executor None of these solutions has worked so far, even after attempting to connect from a few different external devices. Thanks in advance for your help!

Answer: I solved the issue by changing my home network profile to private instead of public, which allows my PC to be discoverable. Completely overlooked that! Hope this helps someone!
Question: The scatterplot from seaborn produces dots with a small white boarder. This is helpful if there are a few overlapping dots, but it becomes visually noisy once there are many overlaying dots. How can the white borders be removed? import seaborn as sns; sns.set() import matplotlib.pyplot as plt tips = sns.load_dataset("tips") ax = sns.scatterplot(x="total_bill", y="tip", data=tips)  

Answer: Instead of edgecolors use linewidth = 0: import seaborn as sns; sns.set() import matplotlib.pyplot as plt tips = sns.load_dataset("tips") ax = sns.scatterplot(x="total_bill", y="tip", data=tips, linewidth=0)
Question: I have encountered a problem while trying to run my django project on a new Docker container. It is my first time using Docker and I can't seem to find a good way to run a django project on it. Having tried multiple tutorials, I always get the error about psycopg2 not being installed. requirements.txt: -i https://pypi.org/simple asgiref==3.2.7 django-cors-headers==3.3.0 django==3.0.7 djangorestframework==3.11.0 gunicorn==20.0.4 psycopg2-binary==2.8.5 pytz==2020.1 sqlparse==0.3.1  Dockerfile: # pull official base image FROM python:3.8.3-alpine # set work directory WORKDIR /usr/src/app # set environment variables ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 # install dependencies RUN pip install --upgrade pip COPY ./requirements.txt . RUN pip install -r requirements.txt # copy project COPY . . # set project environment variables # grab these via Python's os.environ # these are 100% optional here ENV PORT=8000 ENV SECRET_KEY_TWITTER = "***"  While running docker-compose build, I get the following error: Error: pg_config executable not found. pg_config is required to build psycopg2 from source. Please add the directory containing pg_config to the $PATH or specify the full executable path with the option: python setup.py build_ext --pg-config /path/to/pg_config build ... or with the pg_config option in 'setup.cfg'. If you prefer to avoid building psycopg2 from source, please install the PyPI 'psycopg2-binary' package instead.  I will gladly answer any questions that might lead to the solution. Also, maybe someone can recommend me a good tutorial on dockerizing django apps?

Answer: On Alpine Linux, you will need to compile all packages, even if a pre-compiled binary wheel is available on PyPI. On standard Linux-based images, you won't (https://pythonspeed.com/articles/alpine-docker-python/ - there are also other articles I've written there that might be helpful, e.g. on security). So change your base image to python:3.8.3-slim-buster or python:3.8-slim-buster and it should work.
Question: I am a beginner in Python. I started using PyCharm recently but I don't know what's the difference between Terminal and console. Some of the commands in Terminal do not work in console.

Answer: Before we can talk about the differences, we need to talk about what the two are in practice. The Terminal, essentially replaces your command-prompt/power-shell on windows and the terminal app on Mac, giving you a way to access them without leaving PyCharm.  The PyCharm console on the other hand, is a more advanced version of the "Python Console", which allows you to run bits of Python. It is also called the Python REPL or Read Eval Print Loop  You can invoke the Python Console from the terminal as well.
Question: I am using Selenium with Chrome driver to taking some website screenshots. I need the screenshots to be at very specific resolution (1024x768). I've noticed that although the browser is correctly set at this resolution, the screenshot on disk is saved at double resolution (2048x1536). I suspect this is due the retina resolution of the macbook where I am running the application (mid 2017 macbook pro). This is the code I am using: from selenium import webdriver from selenium.webdriver.chrome.options import Options width = 1024 height = 768 chrome_options = Options() chrome_options.add_argument('--disable-gpu') chrome_options.add_argument('--no-sandbox') chrome_options.add_argument('--lang=en') chrome_options.add_argument('--headless') chrome_options.add_argument(f'window-size={width}x{height}') driver = webdriver.Chrome(options=chrome_options) url = 'https://google.com' driver.get(url) print('Window size', driver.get_window_size()) # Window size {'width': 1024, 'height': 768} driver.save_screenshot('test.png') # Image is saved at 2048x1536  Is there a way to prevent the screenshot to be taken at double resolution on retina?

Answer: Found a possible solution: chrome_options.add_argument('--force-device-scale-factor=1')
Question: I use pytorch to train huggingface-transformers model, but every epoch, always output the warning: The current process just got forked. Disabling parallelism to avoid deadlocks... To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)  How to disable this warning?

Answer: Set the environment variable to the string "false" either by TOKENIZERS_PARALLELISM=false  in your shell or by: import os os.environ["TOKENIZERS_PARALLELISM"] = "false"  in the Python script
Question: I'm strugling to convert some MATLAB code into Julia. I have some 3D gridded data (temperature that varies bi-dimensionally and over time) and want to change from a (x,y,t) mesh to a more loose (xi,yi,ti) mesh. In MATLAB it would be a simple interp(x,y,t,T,xi,yi,ti). I tried using Interpolations, Dierckx, but both seemed to work only over 2D gridded data. Am I getting something wrong? I'm quite new in Julia programing... I'm already considering the possibility of solving the problem via PyCall with some NumPy/SciPy funtion. Thanks!

Answer: What led you to believe that Interpolations.jl works only for two-dimensional data? julia> a = rand(1:100, 10, 10, 10); julia> using Interpolations julia> itp = interpolate(a, BSpline(Linear())); julia> v = itp(1.4, 2.3, 3.7) 55.24
Question: I am trying to use one of the new capabilities of python 3.8 (currently using 3.8.3). Following the documentation I tried the example provided in the docs: from functools import singledispatchmethod class Negator: @singledispatchmethod @classmethod def neg(cls, arg): raise NotImplementedError("Cannot negate a") @neg.register @classmethod def _(cls, arg: int): return -arg @neg.register @classmethod def _(cls, arg: bool): return not arg Negator.neg(1)  This, however, yields the following error: ... TypeError: Invalid first argument to `register()`: <classmethod object at 0x7fb9d31b2460>. Use either `@register(some_class)` or plain `@register` on an annotated function.  How can I create a generic class method? Is there something I am missing in my example? Update: I have read Aashish A's answer and it seems lik an on-going issue. I have managed to solve my problem the following way. from functools import singledispatchmethod class Negator: @singledispatchmethod @staticmethod def neg(arg): raise NotImplementedError("Cannot negate a") @neg.register def _(arg: int): return -arg @neg.register def _(arg: bool): return not arg print(Negator.neg(False)) print(Negator.neg(-1))  This seems to work in version 3.8.1 and 3.8.3, however it seems it shouldn't as I am not using the staticmethod decorator on neither the undescore functions. This DOES work with classmethods, even tho the issue seems to indicate the opposite. Keep in mind if you are using an IDE that the linter won't be happy with this approach, throwing a lot of errors.

Answer: This seems to be a bug in the functools library documented in this issue.
Question: I am using Python 3.6 and Tensorflow 2.0, and have some Keras codes: import keras from keras.models import Sequential from keras.layers import Dense model = Sequential() model.add(Dense(1)) model.compile(optimizer='adam',loss='mean_squared_error',metrics=['accuracy'])  When I run this code, I got the following error:  Keras requires TensorFlow 2.2 or higher. Install TensorFlow via pip install tensorflow  I checked on https://keras.io/, it says Keras was built on Tensorflow 2.0. So I am confused. What exact version of Tensorflow does latest Keras support? and how to fix the above error? Thanks!

Answer: The problem is that the latest keras version (2.4.x) is just a wrapper on top of tf.keras, which I do not think is that you want, and this is why it requires specifically TensorFlow 2.2 or newer. What you can do is install Keras 2.3.1, which supports TensorFlow 2.x and 1.x, and is the latest real releases of Keras. You can also install Keras 2.2.4 which only supports TensorFlow 1.x. You can install specific versions like this: pip install --user keras==2.3.1
Question: I am making choropleth maps with geopandas. I want to draw maps with two layers of borders: thinner ones for national states (geopandas default), and thicker ones for various economic communities. Is this doable in geopandas? Here is an example: import geopandas as gpd import numpy as np import matplotlib.pyplot as plt world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres')) africa = world.query('continent == "Africa"') EAC = ["KEN", "RWA", "TZA", "UGA", "BDI"] africa["EAC"] = np.where(np.isin(africa["iso_a3"], EAC), 1, 0) africa.plot(column="pop_est") plt.show()  I have created a dummy variable for countries belonging to group EAC. I would like to draw a thicker border surrounding the countries in this group, while leaving the national borders in as well.  EDIT: I still don't know how to make this work in subplots. Here is an example: axs = ["ax1", "ax2"] vars = ["pop_est", "gdp_md_est"] fig, axs = plt.subplots(ncols=len(axs), figsize=(10, 10), sharex=True, sharey=True, constrained_layout=True) for ax, var in zip(axs, vars): africa.plot(ax=ax, column=var, edgecolor="black", missing_kwds={ "color": "lightgrey", "hatch": "///" }) ax.set_title(var) plt.show()   I wasn't able to apply Martin's solution directly.

Answer: Specify the background plot as an axis and use it within the second plot, plotting only EAC countries. To have only outlines, you need facecolor='none'. ax = africa.plot(column="pop_est") africa.loc[africa['EAC'] == 1].plot(ax=ax, facecolor='none', edgecolor='red', linewidth=2)   If you want a boundary only around those countries, you have to dissolve geometries before. africa.loc[africa['EAC'] == 1].dissolve('EAC').plot(ax=ax, facecolor='none', edgecolor='red', linewidth=2)
Question: I would like to make a paired histogram like the one shown here using the seaborn distplot. This kind of plot can also be referred to as the back-to-back histogram shown here, or a bihistogram inverted/mirrored along the x-axis as discussed here. Here is my code: import numpy as np import matplotlib.pyplot as plt import seaborn as sns green = np.random.normal(20,10,1000) blue = np.random.poisson(60,1000) fig, ax = plt.subplots(figsize=(8,6)) sns.distplot(blue, hist=True, kde=True, hist_kws={'edgecolor':'black'}, kde_kws={'linewidth':2}, bins=10, color='blue') sns.distplot(green, hist=True, kde=True, hist_kws={'edgecolor':'black'}, kde_kws={'linewidth':2}, bins=10, color='green') ax.set_xticks(np.arange(-20,121,20)) ax.set_yticks(np.arange(0.0,0.07,0.01)) ax.spines['top'].set_visible(False) ax.spines['right'].set_visible(False) plt.show()  Here is the output:  When I use the method discussed here (plt.barh), I get the bar plot shown just below, which is not what I am looking for.  Or maybe I haven't understood the workaround well enough... A simple/short implementation of python-seaborn-distplot similar to these kinds of plots would be perfect. I edited the figure of my first plot above to show the kind of plot I hope to achieve (though y-axis not upside down):  Any leads would be greatly appreciated.

Answer: Here is a possible approach using seaborn's displots. Seaborn doesn't return the created graphical elements, but the ax can be interrogated. To make sure the ax only contains the elements you want upside down, those elements can be drawn first. Then, all the patches (the rectangular bars) and the lines (the curve for the kde) can be given their height in negative. Optionally the x-axis can be set at y == 0 using ax.spines['bottom'].set_position('zero'). import numpy as np import matplotlib.pyplot as plt import seaborn as sns green = np.random.normal(20, 10, 1000) blue = np.random.poisson(60, 1000) fig, ax = plt.subplots(figsize=(8, 6)) sns.distplot(green, hist=True, kde=True, hist_kws={'edgecolor': 'black'}, kde_kws={'linewidth': 2}, bins=10, color='green') for p in ax.patches: # turn the histogram upside down p.set_height(-p.get_height()) for l in ax.lines: # turn the kde curve upside down l.set_ydata(-l.get_ydata()) sns.distplot(blue, hist=True, kde=True, hist_kws={'edgecolor': 'black'}, kde_kws={'linewidth': 2}, bins=10, color='blue') ax.set_xticks(np.arange(-20, 121, 20)) ax.set_yticks(np.arange(0.0, 0.07, 0.01)) ax.spines['top'].set_visible(False) ax.spines['right'].set_visible(False) pos_ticks = np.array([t for t in ax.get_yticks() if t > 0]) ticks = np.concatenate([-pos_ticks[::-1], [0], pos_ticks]) ax.set_yticks(ticks) ax.set_yticklabels([f'{abs(t):.2f}' for t in ticks]) ax.spines['bottom'].set_position('zero') plt.show()