eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I've spent half a day trying to figure it out on my own but now I've run out of ideas and googling requests. So basically what I want is to connect to our Snowflake database using snowflake-connector-python package. I was able to install the package just fine (together with all the related packages that were installed automatically) and my current pip3 list results in this: Package Version -------------------------- --------- asn1crypto 1.3.0 azure-common 1.1.25 azure-core 1.6.0 azure-storage-blob 12.3.2 boto3 1.13.26 botocore 1.16.26 certifi 2020.6.20 cffi 1.14.0 chardet 3.0.4 cryptography 2.9.2 docutils 0.15.2 gitdb 4.0.5 GitPython 3.1.3 idna 2.9 isodate 0.6.0 jmespath 0.10.0 msrest 0.6.17 oauthlib 3.1.0 oscrypto 1.2.0 pip 20.1.1 pyasn1 0.2.3 pyasn1-modules 0.0.9 pycparser 2.20 pycryptodomex 3.9.8 PyJWT 1.7.1 pyOpenSSL 19.1.0 python-dateutil 2.8.1 pytz 2020.1 requests 2.23.0 requests-oauthlib 1.3.0 s3transfer 0.3.3 setuptools 47.3.1 six 1.15.0 smmap 3.0.4 snowflake-connector-python 2.2.8 urllib3 1.25.9 wheel 0.34.2  Just to be clear, it's a clean python-venv although I've tried it on the main one, too. When running the following code in VScode: #!/usr/bin/env python import snowflake.connector # Gets the version ctx = snowflake.connector.connect( user='user', password='pass', account='acc')  I'm getting this error: AttributeError: module 'snowflake' has no attribute 'connector'  Does anyone have any idea what could be the issue here?

Answer:  AttributeError: module 'snowflake' has no attribute 'connector'  Your test code is likely in a file named snowflake.py which is causing a conflict in the import (it is ending up importing itself). Rename the file to some other name and it should allow you to import the right module and run the connector functions.
Question: According to NetworkX, draw_networkx(G, pos=None, arrows=True, with_labels=True, **kwds),  node_size can be scalar or array but font_size needs to be integer. How can I change the font size to be bigger if the nodes are big? In fact, is it possible to change font sizes according to node sizes?

Answer: There isn't really a way of passing an array of font sizes. Both nx.draw and draw_networkx_labels only accept integers as font sizes for all labels. You'll have to loop over the nodes and add the text via matplotlib specifying some size. Here's an example, scaling proportionally to the node degree: from matplotlib.pyplot import figure, text G=nx.Graph() e=[(1,2),(1,5),(2,3),(3,6),(5,6),(4,2),(4,3),(3,5),(1,3)] G.add_edges_from(e) pos = nx.spring_layout(G) figure(figsize=(10,6)) d = dict(G.degree) nx.draw(G, pos=pos,node_color='orange', with_labels=False, node_size=[d[k]*300 for k in d]) for node, (x, y) in pos.items(): text(x, y, node, fontsize=d[node]*5, ha='center', va='center')
Question: I have a program running on Google Colab in which I need to monitor GPU usage while it is running. I am aware that usually you would use nvidia-smi in a command line to display GPU usage, but since Colab only allows one cell to run at once at any one time, this isn't an option. Currently, I am using GPUtil and monitoring GPU and VRAM usage with GPUtil.getGPUs()[0].load and GPUtil.getGPUs()[0].memoryUsed but I can't find a way for those pieces of code to execute at the same time as the rest of my code, thus the usage numbers are much lower than they actually should be. Is there any way to print the GPU usage while other code is running?

Answer: Used wandb to log system metrics: !pip install wandb import wandb wandb.init()  Which outputs a URL in which you can view various graphs of different system metrics.
Question:  Hello, I have a simple dynamodb table here filled with placeholder values. How would i go about retrieving only sort_number, current_balance and side with a query/scan? I'm using python and boto3, however, just stating what to configure for each of the expressions and parameters is also enough.

Answer: Within the Boto3 SDK you can use:  get_item if you're trying to retrieve a specific value query, if you're trying to get values from a single partition (the hash key). scan if you're trying to retrieve values from across multiple parititions.  Each of these have a parameter named ProjectionExpression, using this parameter provides the following functionality  A string that identifies one or more attributes to retrieve from the specified table or index. These attributes can include scalars, sets, or elements of a JSON document. The attributes in the expression must be separated by commas.  ]You would specify the attributes that you want to retrieve comma separated, be aware that this does not reduce the cost of RCU that is applied for performing the interaction.
Question: I know most of python is implemented in C. I was wondering that how does python work under the hood(in terms of its implementation in C) when it comes to determining what type of variable it is in this case lets say x = 5 now if the check the type of x it will say class int but how is that implemented in C? What checks are made under the hood to determine that it belongs to Class int

Answer: This is a huge subject. The below document will give you more understanding. https://intopythoncom.files.wordpress.com/2017/04/internalsofcpython3-6-1.pdf Like you said a simple example for integer types To hold an integer type object, there is structure defined in C as said below typedef struct { PyObject_HEAD long ob_ival; } PyIntObject;  The object of C structure PyIntObject (section 7.5 of above said document) holds the objects which are integer type. If you are more interested, setup the environment and debug as said in same section 7.5 of the above document.  Objects/intobject.c and place a debug point on line number 89. Start debugging the application.  PyTypeObject is at higher level for the types to be represented. (look section 7.3 of above said document) As a programmer, it is curious aspect to know the internals. But do not spend too much time to understand unless you work at interpreter level.
Question: I'm attempting to map numpy dtypes to associated values using a dictionary lookup. I observe the following counterintuitive behavior: dtype = np.uint16 x = np.array([0, 1, 2], dtype=dtype) assert x.dtype == dtype d = {np.uint8: 8, np.uint16: 16, np.uint32: 32, np.float32: 32} print(dtype in d) # prints True print(x.dtype in d) # prints False  Using other dtypes produces similar results. So we have that np.uint16 == x.dtype, but the former is found in the dictionary's keys while the latter is not. Any explanation and/or simple workaround would be appreciated.

Answer: Dtypes don't work like they look at first glance. np.uint16 isn't a dtype object. It's just convertible to one. np.uint16 is a type object representing the type of array scalars of uint16 dtype. x.dtype is an actual dtype object, and dtype objects implement == in a weird way that's non-transitive and inconsistent with hash. dtype == other is basically implemented as dtype == np.dtype(other) when other isn't already a dtype. You can see the details in the source. Particularly, x.dtype compares equal to np.uint16, but it doesn't have the same hash, so the dict lookup doesn't find it.
Question: Because of some specific requirements I needed to compile a package (opencv with cuda support) from source. After successfull compilation my python-environment is able to import opencv without a problem: $ python Python 3.7.7 (default, Mar 10 2020, 15:16:38) [GCC 7.5.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import cv2 >>> cv2.__version__ '4.3.0' >>>  But if I try pip list opencv-python is not part of it: Package Version -------------------- -------- absl-py 0.9.0 astor 0.8.1 dlib 19.20.99 gast 0.3.3 google-pasta 0.2.0 grpcio 1.30.0 h5py 2.10.0 importlib-metadata 1.6.1 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.2 Markdown 3.2.2 numpy 1.19.0 pip 20.1.1 protobuf 3.12.2 setuptools 47.3.1 six 1.15.0 tensorboard 1.14.0 tensorflow-estimator 1.14.0 tensorflow-gpu 1.14.0 termcolor 1.1.0 Werkzeug 1.0.1 wheel 0.34.2 wrapt 1.12.1 zipp 3.1.0  The problem is that afterwards I need to install more packages via pip install -r requirements.txt and some of the packages listed in requirements.txt have opencv as a dependency. As pip is not aware of the opencv installation it now installes a different opencv version. Having two different versions installed along side each other does not sound like a clever solution for me... I could uninstall the pip install opencv later but that does not seem to be a good solution either... So how can I make pip aware of the other opencv installation before running the pip install?

Answer: After you compile opencv, you can install the package with pip or python setup.py install. I would recommend building a Python wheel for opencv+cuda, and then installing that wheel. Having a wheel will make installing easier if you ever need to reinstall or make a new environment. The general steps are:  Compile opencv Change into the opencv python directory (with setup.py) and run python setup.py bdist_wheel Run auditwheel repair my-python-wheel-1.5.2-cp35-cp35m-linux_x86_64.whl (change the wheel filename) (from https://stackoverflow.com/a/42106034/5666087)  You can get auditwheel with pip install auditwheel. Another useful reference is the build documentation for the opencv-python wheels, available at https://github.com/skvark/opencv-python#build-process. You can modify those steps to your build.
Question: For a numpy array, how can I change the value only if all elements along an axis are equal to another array? For example... array = np.array([[1, 0, 1], [0, 0, 1], [1, 1, 0], [0, 0, 0], [1, 0, 1]])  I want to replace all [1, 0, 1] with [1, 1, 1]... so that array becomes array([[1, 1, 1], [0, 0, 1], [1, 1, 0], [0, 0, 0], [1, 1, 1]])  When I use a boolean array, it checks each individual number. How can I compare the entire row at once instead?

Answer: Try with: array[(array == [1, 0, 1]).all(axis=1)] = [1, 1, 1]
Question: I have a list which contains many lists and in those there 4 tuples. my_list = [[(12, 1), (10, 3), (4, 0), (2, 0)], [(110, 1), (34, 2), (12, 1), (55, 3)]]  I want them in two separate lists like: my_list2 = [12,10,4,2,110,34,12,55] my_list3 = [1,3,0,0,1,2,1,3]  my attempt was using the map function for this. my_list2 , my_list3 = map(list, zip(*my_list))  but this is giving me an error: ValueError: too many values to unpack (expected 2) 

Answer: Your approach is quite close, but you need to flatten first: from itertools import chain my_list = [[(12, 1), (10, 3), (4, 0), (2, 0)], [(110, 1), (34, 2), (12, 1), (55, 3)]] my_list2 , my_list3 = map(list,zip(*chain.from_iterable(my_list))) my_list2 # [12, 10, 4, 2, 110, 34, 12, 55] my_list3 # [1, 3, 0, 0, 1, 2, 1, 3]
Question: I have the following list of strings: l1 = ['one','two','three']  I want to obtain a list that has, say, these same elements repeated n times. If n=3 I'd get: l2 = ['one','one','one','two','two','two','three','three','three']  What I am trying is this: l2 = [3*i for i in l1]  But what I obtain is this: l2 = ['oneoneone','twotwotwo','threethreethree']  If I try this: l2 = [3*(str(i)+",") for i in l1]  I obtain: l2 = ['one,one,one','two,two,two','three,three,three']  What am I missing?

Answer:  l2 = [j for i in l1 for j in 3*[i]]  This gives:  ['one', 'one', 'one', 'two', 'two', 'two', 'three', 'three', 'three']  This is equivalent to: l2 = [] for i in l1: for j in 3*[i]: l2.append(j)  Note that 3*[i] creates a list with 3 repeated elements (e.g. ['one', one', 'one'])
Question: Suppose you have a simple pandas dataframe with a MultiIndex: df = pd.DataFrame(1, index=pd.MultiIndex.from_tuples([('one', 'elem1'), ('one', 'elem2'), ('two', 'elem1'), ('two', 'elem2')]), columns=['col1', 'col2'])  Printed as a table:  col1 col2 one elem1 1 1 elem2 1 1 two elem1 1 1 elem2 1 1  Question: How do you add a "Total" row to that Dataframe? Expected output:  col1 col2 one elem1 1.0 1.0 elem2 1.0 1.0 two elem1 1.0 1.0 elem2 1.0 1.0 Total 4.0 4.0   First attempt: Naive implementation If I am just ignoring the MultiIndex and follow the standard way df.loc['Total'] = df.sum()  Output:  col1 col2 (one, elem1) 1 1 (one, elem2) 1 1 (two, elem1) 1 1 (two, elem2) 1 1 Total 4 4  It seems to be correct, but the MultiIndex is transformed to Index([('one', 'elem1'), ('one', 'elem2'), ('two', 'elem1'), ('two', 'elem2'), 'Total'], dtype='object')  Second attempt: Be explicit df.loc['Total', :] = df.sum()  or (being frustrated and changing the axis just out of spite) df.loc['Total', :] = df.sum(axis=1)  Output (the same for both calls):  col1 col2 one elem1 1.0 1.0 elem2 1.0 1.0 two elem1 1.0 1.0 elem2 1.0 1.0 Total NaN NaN  The MultiIndex is not transformed, but the Total is wrong (NaN != 4).

Answer: The solution You have to remove the index of df.sum() and just use the values: df.loc['Total', :] = df.sum().values  Output:  col1 col2 one elem1 1.0 1.0 elem2 1.0 1.0 two elem1 1.0 1.0 elem2 1.0 1.0 Total 4.0 4.0   Why was the second attempt wrong? The second attempt was almost correct. But df.sum() has the Index(['col1', 'col2'], dtype='object'). Consequently, pandas isn't able to match the index. The new index ('Total', '') is appended but without values. But why did df.loc['Total', :] = df.sum(axis=1) also fail? It has the correct Multiindex. Pandas does exactly what you told it, i.e. sum the columns. So, df.sum(axis=1) gives you the following dataframe: one elem1 2 elem2 2 two elem1 2 elem2 2  This dataframe can't be matched with the original df in any meaningful sense.
Question: I am trying to install packages in an offline manner. However, when I downloaded all packages and tried to install these packages on another computer, some error has emerged as shown in the following figure. This seems like it is failed to install the "dash-bootstrap-components" package. How can I solve it? By the way, the "dash-bootstrap-components" package is packaged as a "tar.gz" file, does this causes the failure? The following commands can reproduce the problem even on the same computer: pip download dash-bootstrap-components pip install --no-index --find-links ./ dash-bootstrap-components  My goal is to configure a python environment on a network-free computer. If this method is unavailable, is there any other methods that can work around it?  

Answer: I have solved this problem by manually download the "wheel" package from the internet and put it into the folder.
Question: I am not sure I quite understand what's happening in the below mini snippet (on Py v3.6.7). It would be great if someone can explain to me as to how can we mutate the list successfully even though there's an error thrown by Python. I know that we can mutate a list and update it, but what’s with the error? Like I was under the impression that if there's an error, then the x should remain the same. x = ([1, 2], ) x[0] += [3,4] # ------ (1)  The Traceback thrown at line (1) is > TypeError: 'tuple' object doesn't support item assignment..  I understand what the error means but I am unable to get the context of it. But now if I try to print the value of my variable x, Python says it's, print(x) # returns ([1, 2, 3, 4])  As far as I can understand, the exception has happened after Python allowed the mutation of the list to happen and then hopefully it tried re-assigning it back. It blew there I think as Tuples are immutable. Can someone explain what's happening under the hood? Edit - 1 Error From ipython console as an image; 

Answer: My gut feeling is that the line x[0] += [3, 4] first modifies the list itself so [1, 2] becomes [1, 2, 3, 4], then it tries to adjust the content of the tuple which throws a TypeError, but the tuple always points towards the same list so its content (in terms of pointers) is not modified while the object pointed at is modified. We can verify it that way: a_list = [1, 2, 3] a_tuple = (a_list,) print(a_tuple) >>> ([1, 2, 3],) a_list.append(4) print(a_tuple) >>> ([1, 2, 3, 4], )  This does not throw an error and does modify it in place, despite being stored in a "immutable" tuple.
Question: How do you round numbers for display in a plotly graph? I included an MRE below. Essentially I wanted rounded numbers to appear when the user hovers over the bar. import plotly.express as px import pandas as pd df = pd.DataFrame({'num': [1, 2, 3], 'sqrt': pd.Series([1, 2, 3]) ** 0.5}) fig = px.bar(df, x='num', y='sqrt', title='Square root') fig.show() 

Answer: You can do it two ways like this: METHOD-1: Using pd.series.round function. import plotly.express as px import pandas as pd df = pd.DataFrame({'num': [1, 2, 3], 'sqrt': (pd.Series([1, 2, 3]) ** 0.5).round(2)}) fig = px.bar(df, x='num', y='sqrt', title='Square root') fig.show()  METHOD-2: Using python builtin round function. For this to work I will have to use list comprehension instead of pandas series. As series doesn't support it directly. import plotly.express as px import pandas as pd df = pd.DataFrame({'num': [1, 2, 3], 'sqrt': [round(x**(1/2),2) for x in [1,2,3]]}) fig = px.bar(df, x='num', y='sqrt', title='Square root') fig.show()  EDIT: METHOD-3: As answered by @M. Forsythe below you can also do it using hover_data parameter of plotly. import plotly.express as px import pandas as pd df = pd.DataFrame({'num': [1, 2, 3], 'sqrt': pd.Series([1, 2, 3]) ** 0.5}) fig = px.bar(df, x='num', y='sqrt', title='Square root',hover_data={'sqrt':':.2f'}) fig.show()
Question: Plotly is not working in Jupyterlab. I assume that there is a conflict in required extensions but I'm not sure. On checking troubleshooting on Plotly https://plotly.com/python/troubleshooting/ , they advise to remove extensions and install them again. But I found that there is additional extension that came with Jupyterlab update called 'jupyterlab-plotly-extension' which is not mentioned by Plotly in their instructions to make it working in JupyterLab https://plotly.com/python/getting-started/#jupyterlab-support-python-35 My question is: which extensions should be installed to make Plotly working in JupyterLab?  jupyterlab-plotly as mentioned in Plotly support jupyterlab-plotly-extension that came with JupyterLab 

Answer: Enter 'jupyter labextension list' in a terminal or command to run the environment status. The example below shows my environment information with 'jupyter lab' running successfully. xxxxx-no-iMac:~ xxxxx$ jupyter labextension list JupyterLab v2.1.5 Known labextensions: app dir: /Library/Frameworks/Python.framework/Versions/3.6/share/jupyter/lab @jupyter-widgets/jupyterlab-manager v2.0.0 enabled OK @jupyterlab/git v0.20.0 enabled OK @lckr/jupyterlab_variableinspector v0.5.0 enabled OK jupyterlab-plotly v1.5.4 enabled OK nbdime-jupyterlab v2.0.0 enabled OK plotlywidget v1.5.4 enabled OK
Question: I have VSCode installed and python 3.6.8 I use tabs for my indentation. But when ever I save the file, all the tabs are being converted to spaces. This might be because of the formatter I use, i.e. Black. How do I prevent the formatter from doing that(Do all your formatting except inter-changing indents with spaces)? Thank you

Answer: Assuming you don't have VS Code set up to insert space into tabs, then Black will very likely replace them with spaces as that's the norm in the Python community and Black takes a very opinionated view on how to format Python code. You could try another formatter like yapf or autopep8 to see if they will leave physical tabs in. But do note that the vast majority of the Python community uses spaces, so you are potentially facing an uphill battle.
Question: I would like to have increasingly transparent violins in a seaborn.violinplot. I tried the following: import seaborn as sns tips = sns.load_dataset("tips") ax = sns.violinplot(x="day", y="total_bill", data=tips, color='r', alpha=[0.8, 0.6, 0.4, 0.2])  Which does not result in the desired output: 

Answer: Found this thread looking to change alpha values in general for violin plots, it seems you need to access matplotlib.PolyColections from your ax to even be able to set the alpha values, but since you need to access them anyways, you might as well set alpha values individually (at least in your case since you want individual alpha values). From my understanding, ax.collections contain both matplotlib.PolyCollections and matplotlib.PathCollections, you only need the PolyCollections, so I did the following and it seems to work: ax = sns.violinplot(x = 'day', y = 'total_bill', data = tips, color = 'r') for violin, alpha in zip(ax.collections[::2], [0.8,0.6,0.4,0.2]): violin.set_alpha(alpha)  ax.collections[::2] ignores PathCollections, as ax.collections comes in format of [PolyCollection1, PathCollection1, PolyCollection2, PathCollection2, ...] Output:
Question: I try to get a class diagram from Python source code in Client folder with pyreverse but it requires __init__.py (venv) C:\Users\User\Desktop\project> pyreverse Client parsing Client\__init__.py... Failed to import module Client\__init__.py with error: No module named Client\__init__.py.  I don't find any solution for this. Is there a way to get the diagram? Update: There are many files in Client folder: Client.py GUI.py script.py ...  This is a part of the Client.py code: import threading class Client: def __init__(self): self.socket = None self.listen_socket = None self.buff_dict = {} self.message_list_dict = {} self.lock = threading.Lock() self.target = None self.listen_flag = True  This is a part of the GUI.py code: import tkinter as tk class Window(object): def __init__(self, title, font, client): self.title = title self.font = font self.client = client self.root = tk.Tk() self.root.title(title) self.build_window() def build_window(self): pass class LoginWindow(Window): def __init__(self, client, font): super(LoginWindow, self).__init__('Login', font, client) self.build_window() 

Answer: Thanks to @Anwarvic and @bruno, I came up with the solution for this. Firstly, create empty __init__.py file inside Client folder: (venv) C:\Users\User\Desktop\project\Client> type NUL > __init__.py  Then go to the parent folder of the Client folder where I want to get the class diagram: (venv) C:\Users\User\Desktop\project> pyreverse Client -o png  But I got this error: The output format 'png' is currently not available. Please install 'Graphviz' to have other output formats than 'dot' or 'vcg'.  After some findings, I found this solution. Then I can run the pyreverse without any error. This is the class diagram I got using pyreverse:
Question: I am trying to install jaxlib on my windows 10 by the following command which I found on the documentation..  pip install jaxlib  It shows the following error Collecting jaxlib Could not find a version that satisfies the requirement jaxlib (from versions: None) No matching distribution found for jaxlib 

Answer: Jaxlib is not supported on windows you can see it here.. https://github.com/google/jax/issues/438
Question: a=[1,2,3,4] def func(x): return x**x b=sorted(a,func)  this line always gives a error->  TypeError: sorted expected 1 argument, got 2  in fact the syntax of sorted is sorted(iterable,key,reverse), in which key and reverse are optional, so according to this, second parameter i pass must go with key. and when i def my own func def func2(x,y=4,z=10): print(x,y,z) func2(100,200)--->output-->>100 200 10  here 200 automatically passed as y argument for func2. How does this work?

Answer: In addition to @user4815162342's answer, From the documentation, sorted(iterable, *, key=None, reverse=False)  Notice the * between iterable and key parameter. That is the python syntax for specifying that every parameter after * must be specified as keyword arguments. So your custom function should be defined as the following to apply the similar implementation: def func2(x, *, y=4, z=10): print(x, y, z) func2(100, 200)   TypeError: func2() takes 1 positional argument but 2 were given