eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I use the following to access bouth Google Drive and Google Sheets in my Google Colab Notebook: # Mount Google Drive from google.colab import drive drive.mount('/content/drive') # Google Sheets from google.colab import auth auth.authenticate_user() import gspread from google.auth import default creds, _ = default() gc = gspread.authorize(creds) from gspread_dataframe import get_as_dataframe, set_with_dataframe  My problem with this is that it asks for the user to confirm the authentication in a pop-up and I want to use colabctl (https://github.com/bitnom/colabctl) to automate the execution of some scripts, so I need to execute the authentication part without user interaction.

Answer: As a possible solution, I may suggest using Service Account.  Go to GCP Console  Select your current project (or create a new one):   Once the project is selected, go to IAM and Admin -> Service accounts -> + Create service account:   Next, you'll see a create service account prompt, which consists of 3 form steps. Fill in step 1 with any data you want. Skip steps 2 and 3.  You'll be redirected to the new service account page. Note an Email field there. You may share any Google Drive content for that email address (same as you do for users' personal email addresses).  Now you need to create a service account key, so you could use it to authenticate your script (select JSON key type):   After step 6, you should get your JSON key file downloaded. Put it into your Google Colab files. You may do something like that to initialize your google client (gc) :   import gspread gc = gspread.service_account(filename='/content/credentials.json')  Important: gspread version should be >=3.6.0 to support service_account() method. Note: See my example to list all shared Google Sheets using gspread:
Question: I upgraded my system from python 2 to python 3, and now when I run my code: from cryptography.hazmat.backends import default_backend  I am getting this error /usr/local/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.  How to resolve it?

Answer: I ran into this issue today and did some digging. Since the os is not provided I think you could consider one of the options below:  Upgrade your python version. This might be the best option since python 3.6 reached its EOL.  You might be using SSH in your code. Consider installing an older version of paramiko.  You can suppress the warning with these lines of code before importing Paramiko: import warnings warnings.filterwarnings(action='ignore',module='.*paramiko.*')    or If you want to be more selective about suppressing JUST that particular deprecation warning: import warnings from cryptography.utils import CryptographyDeprecationWarning warnings.filterwarnings("ignore", category=CryptographyDeprecationWarning)  source for option 3  Check if you installed it using pip when you needed to use conda.
Question: I am working on a python package with VS Code with the following layout of the opened workspace folder in VS Code workspace | + tests | | - test1.py | | + other_tests | | | - test2.py | + mymodule | | ...  What I want is to call in test1.py and test2.py the package mymodule with import mymodule  When I do this, I always get the error no module named 'mymodule'. I know i can load the path via inserting the following to all test files: import sys sys.path.append('<path_to>/workspace/')  Then mymodule is correctly loaded. (I know I can use pathlib.Path(__file__).parent[n] for some n to get <path_to>. But this process is really cumbersome if you have many test files in different folders... and it is just ugly. Is there any way, to tell VS Code to always include the workspace in the path? I tried a .env file with PYTHONPATH=<path_to>/workspace/  and I also tried to add the following to launch.json: "env": { "PYTHONPATH": "${workspaceFolder}" }  Both did not do the trick. (I am also not sure if PYTHONPATH is at all the right solution to my problem, i just tried it ^^). In short: I would like to tell VS Code, that it should always include workspace to the search path whenever a python function tries to load a module, regarding the location of the function, so I do not have to call sys.path.append('<path_to>/workspace/') in every single main file.

Answer: When the python interpreter is importing a package, it looks for the package in the following locations:  the directory containing the input script (or the current directory). PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH). the installation-dependent default.  So if your package and script file are in the same directory, it will be easily found. But obviously you chose a different directory. Then you have to use the methods mentioned in your article to specify the path to the interpreter, such as modifying the PYTHONPATH environment variable or using the sys.path.append() method. Here's a suggestion for another approach. Put your package where python can find it, say under the lib folder. This way you don't need to use the sys.path.append() method at the beginning of each file.  I am using a virtual environment UPDATE: You can use virtual environments. it won't make a difference. If you are worried about some factors and do not want to use it, there is no problem. Just find the lib folder in your current environment and put your package in it. for example ( As far as my machine is concerned ) :  The lib folder corresponding to the interpreter of this environment is in: C:\Users\Admin\AppData\Local\Programs\Python\Python310\Lib    The lib folder corresponding to the interpreter of this environment is in: C:\Users\Admin\anaconda3\Lib  PS : It can also be placed in the site-packages folder one level below the lib folder.
Question: I understand that hash of an immutable object is an integer representation of that object which is unique within the process's lifetime. Hash of an integer object is the same as the value held by the integer. For example, >>> int(1000).__hash__() 1000  But when the integer grows big enough, above principle breaks after a certain threshold as it seems. Its value seems to be rolled with in some limit. >>> int(10000000000000000).__hash__() 10000000000000000 >>> int(100000000000000000).__hash__() 100000000000000000 >>> int(1000000000000000000).__hash__() 1000000000000000000 >>> int(10000000000000000000).__hash__() 776627963145224196  Two questions:  What is the limit? What is the integer-space that hash table covers? How is the hash value calculated for an integer exceeding the above limit?  System information: Linux lap-0179 5.13.0-44-generic #49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux  Python interpreter: Python 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] on linux 

Answer: While this is machine and implementation dependent, for CPython on 64-bit machines, the hash() for a non-negative integer n is computed as n % k with k = (2 ** 61 - 1) (= 2305843009213693951) hence values between 0 and k - 1 are left as is. This is empirically evidenced here: k = 2 ** 61 - 1 for i in range(k - 2, k + 2): print(i, hash(i), i % k) # 2305843009213693949 2305843009213693949 # 2305843009213693950 2305843009213693950 # 2305843009213693951 0 # 2305843009213693952 1  For the complete ruleset, see the documentation.
Question: I am working with Paramiko on Linux, I would like to know if I can send a variable to shell. I want to enter to "enable mode" of a Cisco router. But I don't want to hard-code the password in the script. I am using getpass, but when I run the script it fails in the enable command. I get "bad secrets". I know that happens for wrong password. I know shell had to add \n in the end of the command. But how I can add to a variable? #!/bin/python3 import paramiko import time import getpass ssh_client = paramiko.SSHClient() ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) password = getpass.getpass('Enter password: ') router = {'hostname': '192.168.142.132', 'port': '22', 'username':'user', 'password': password} print(f"Connecting to {router['hostname']} ") ssh_client.connect(**router, look_for_keys=False, allow_agent=False) shell = ssh_client.invoke_shell() shell.send('enable\n') passwordE = getpass.getpass('Enter the enable password:') shell.send('passwordE') 

Answer: Use + operator. shell.send(passwordE + '\n')  Or you can simply call send twice: shell.send(passwordE) shell.send('\n')   Obligatory warning: Do not use AutoAddPolicy this way – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
Question: I have a graph that plots Bayesian mean point data in a line with credible intervals around the mean value.  I'm trying to fill between to two credibility lines with a translucent color so the mean line really pops through. I've tried the following: plt.fill_between(b.get_data(), c.data_get(), color='blue', alpha = .5)  I'm pulling this data out of an arviz inference set. Here is a toy dataset. import numpy as np import seaborn as sns import matplotlib.pyplot as plt mean = np.array([861.98525 , 705.23875 , 640.14575 , 658.727625, 728.23775 , 792.4645 , 803.045375, 763.425875, 721.785375, 713.182375, 740.543375, 781.466875]) confidence1 = np.array([788. , 607. , 493. , 443.975, 435.975, 412. , 366.975, 295. , 243. , 207. , 181. , 161. ]) confidence2 = np.array([ 938. , 811. , 815. , 935.025, 1150.025, 1391.05 , 1556.05 , 1624.05 , 1689.025, 1829. , 2078.125, 2390.025]) date = list(df_sat_test['t'].unique()) fig = plt.figure(figsize=(15,4)) a=sns.lineplot(x =date, y = mean, label = 'Posterior Predictive') b=sns.lineplot(x =date, y = confidence1, label = 'Confidence', color = 'r') c=sns.lineplot(x =date, y = confidence2, label = 'Confidence', color = 'r') # plt.fill_between(b.get_data(), c.data_get(), color='blue', alpha = .5) sns.scatterplot(x =df_sat_test['t'], y = np.array(test_ppc.observed_data.obs), label = 'True Value') plt.legend() 

Answer: If you get the last line graph, you will get data for three line graphs, which you can set to fill the first of the x-axis and y-axis and the second of the y-axis to fill to get the intended result. fig = plt.figure(figsize=(15,4)) a=sns.lineplot(x=date, y=mean, label = 'Posterior Predictive') b=sns.lineplot(x=date, y=confidence1, label='Confidence', color='r') c=sns.lineplot(x=date, y=confidence2, label='Confidence', color='r') line = c.get_lines() plt.fill_between(line[0].get_xdata(), line[1].get_ydata(), line[2].get_ydata(), color='blue', alpha=.5) #sns.scatterplot(x =df_sat_test['t'], y = np.array(test_ppc.observed_data.obs), label = 'True Value') plt.legend() plt.show()
Question: I'm using tkinter to make a python app and I need to let the user choose which key they will use to do some specific action. Then I want to make a button which when the user clicks it, the next key they press as well in keyboard as in mouse will be detected and then it will be bound it to that specific action. How can I get the key pressed by the user?

Answer: To expand on @darthmorf's answer in order to also detect mouse button events, you'll need to add a separate event binding for mouse buttons with either the '<Button>' event which will fire on any mouse button press, or '<Button-1>', (or 2 or 3) which will fire when that specific button is pressed (where '1' is the left mouse button, '2' is the right, and '3' is the middle...though I think on Mac the right and middle buttons are swapped). import tkinter as tk root = tk.Tk() def on_event(event): text = event.char if event.num == '??' else event.num label = tk.Label(root, text=text) label.place(x=50, y=50) root.bind('<Key>', on_event) root.bind('<Button>', on_event) root.mainloop()
Question: import random from Words import words import string def get_valid_word(words): word = random.choice(words) while '-' or ' ' in word: word = random.choice(words) return word.upper() def hangman() -> object: word = get_valid_word(words) word_letter = set(word) alphabet = set(string.ascii_uppercase) used_letter = set() while len(word) > 0: print("You have used these letter", ' '.join(used_letter)) word_list = [letter if letter in used_letter else '-' for letter in word] print("Current word: ", ' '.join(word_list)) user_letter = input("Guess a letter: ") if user_letter in alphabet - used_letter: used_letter.add(user_letter) if user_letter in word_letter: word_letter.remove(user_letter) elif user_letter in used_letter: print("You have already used this letter: ") else: print("Invalid input") if __name__ == '__main__': hangman()  I can't bring code to execute function hangman(). Terminal only shows the following output: /Users/nighttwinkle/PycharmProjects/pythonProject/venv/bin/python /Users/nighttwinkle/PycharmProjects/pythonProject/main.py 

Answer: The issue appears to be with the line while '-' or ' ' in word:  your syntax is slightly off meaning that this will always evaluate to True and thus the code enters an infinite loop at this point python interprets the above as while ('-') or (' ' in word):  an thus the string - is one half of the or statement and any non empty string will always evaluate to True leaving the while condition as: while True or ' ' in words:  try this intead: while '-' in word or ' ' in word:
Question: Let's say I have the following data frame  name age favorite_color grade 0 Willard Morris 20 blue 88 1 Al Jennings 19 blue 92 2 Omar Mullins 22 yellow 95 3 Spencer McDaniel 21 green 70  And I'm trying to get the grade for Omar which is "95" it can be easily obtained using ddf = df.loc[[2], ['grade']] print(ddf)  However, I want to use his name "Omar" instead of using the raw index "2". Is it possible? I tried the following syntax but it didn't work ddf = df.loc[['Omar Mullins'], ['grade']] 

Answer: Try this:  ddf = df[df['name'] == 'Omar Mullins']['grade']  to output the grade values. Instead: ddf = df[df['name'] == 'Omar Mullins']  will output the full row.
Question: I'm using two numpy arrays of floating point numbers Rs = np.linspace(*realBounds, realResolution, dtype=fdtype) Is = np.linspace(*imagBounds, imagResolution, dtype=fdtype)  to make a grid Zs of complex numbers with shape (realResolution, imagResolution). I'd like to only specify the datatype of Zs and use this to determine the float datatype fdtype. As I understand it from this page, complex datatypes are always represented as two floats with the same datatype, hence why Rs and Is both have dtype=fdtype. If the datatype of Zs is specified as np.csingle then I'd like fdtype to be np.single, if it's specified as np.dtype("complex128") then I'd like fdtype to be np.dtype("float64"), and so on. Is there a nice way to do this? Edit: I'm equally interested in the inverse function which sends, for example, np.single to np.csingle.

Answer: I don't think there is such a function in numpy but you can easily roll your own using num, which uniquely identifies each of the built-in types: def get_corresonding_dtype(dt): return {11: np.csingle, 12: np.cdouble, 13: np.clongdouble, 14: np.single, 15: np.double, 16: np.longdouble}[np.dtype(dt).num]  The dict was obtained from {np.dtype(dt).num: dt for dt in (np.single, np.double, np.longdouble, np.csingle, np.cdouble, np.clongdouble)} which returns {11: numpy.float32, 12: numpy.float64, 13: numpy.longdouble, 14: numpy.complex64, 15: numpy.complex128, 16: numpy.clongdouble}  on win-amd64 (sysconfig.get_platform()). The names may vary, e.g. on linux-x86_64 it shows numpy.complex256 instead of numpy.clongdouble, but thanks to num you don't need to pay attention to it. So for instance for all of np.double, np.float_, np.float64, float, 'float64' etc., get_corresonding_dtype will return numpy.complex128. If you like you can also add np.half as 23: np.csingle to the dict, as there's no complex half/float16 type in numpy.
Question: I am trying to merge two dataframes and create a new dataframe containing only the rows from the first dataframe that does not exist in the second one. For example: The dataframes that I have as input:  The dataframe that I want to have as output:  Do you know if there is a way to do that? If you could help me, I would be more than thankful!! Thanks, Eleni

Answer: Creating some data, we have two dataframes: import pandas as pd import numpy as np rng = np.random.default_rng(seed=5) df1 = pd.DataFrame(data=rng.integers(0, 5, size=(5, 2))) df2 = pd.DataFrame(data=rng.integers(0, 5, size=(5, 2)))  # df1 a b 0 3 4 1 0 4 2 2 2 3 3 1 4 4 0 # df2 a b 0 1 1 1 2 2 2 0 0 3 0 0 4 0 4  We can use pandas.merge to combine equal rows. And we can use its indicator=True feature to mark those rows that are only from the left (and right, when applicable). Since we only need those that are unique to left, we can merge using how="left" to be more efficient. dfm = pd.merge(df1, df2, on=list(df1.columns), how="left", indicator=True)  # dfm a b _merge 0 3 4 left_only 1 0 4 both 2 2 2 both 3 3 1 left_only 4 4 0 left_only  Great, so then the final result is using the merge but only keeping those that have an indicator of left_only: (dfm.loc[dfm._merge == 'left_only'] .drop(columns=['_merge']))   a b 0 3 4 3 3 1 4 4 0   If you'd want to deduplicate by a subset of the columns, that should be possible. In that case I would do the merge it like this, repeating the subset so that we don't get other columns in duplicate versions from the left and right side. pd.merge(df1, df2[subset], on=subset, how="left", indicator=True)
Question: I want to infinitely iterate through the list from 0 to the end, but in the next loop I want to start at 1 to the end plus 0, and the next loop would start at 2 to the end plus 0, 1, up to the last item where it would start again at 0 and go to the end. Here is my code: a = [ 0, 1, 2 ] offset = 0 rotate = 0 while True: print(a[rotate]) offset += 1 rotate += 1 if offset >= len(a): offset = 0 rotate += 1 if rotate >= len(a): rotate = 0  This is the solution I came up with so far. It's far from perfect. The result that I want is: 0, 1, 2 # first iteration 1, 2, 0 # second iteration 2, 0, 1 # third iteration 0, 1, 2 # fourth iteration  and so on.

Answer: You can use a deque which has a built-in and efficient rotate function (~O(1)): >>> d = deque([0,1,2]) >>> for _ in range(10): ... print(*d) ... d.rotate(-1) # negative -> rotate to the left ... 0 1 2 1 2 0 2 0 1 0 1 2 1 2 0 2 0 1 0 1 2 1 2 0 2 0 1 0 1 2
Question: class Publication(models.Model): title = models.CharField(max_length=30) class Article(models.Model): headline = models.CharField(max_length=100) publications = models.ManyToManyField(Publication) p1 = Publication.objects.create(title='The Python Journal') p2 = Publication.objects.create(title='Science News') p3 = Publication.objects.create(title='Science Weekly')  I want to filter articles, published in both p1 and p3. They might or might not publish in other publications but they must have to publish in p1 and p3. I have tried: Article.objects.filter(Q(publications=p1) & Q(publications=p3))  but it returns empty queryset which is not true

Answer: If you chain mutliple calls to filter() they should behave like being connected with an AND: articles = Article.objects.filter(publications=p1).filter(publications=p3).distinct()
Question: I have a problem, I want to count the unique words from a dataframe, but unfortunately it only counts the first sentences.  text 0 hello is a unique sentences 1 hello this is a test 2 does this works  import pandas as pd d = { "text": ["hello is a unique sentences", "hello this is a test", "does this works"], } df = pd.DataFrame(data=d) from collections import Counter # Count unique words def counter_word(text_col): print(len(text_col.values)) count = Counter() for i, text in enumerate(text_col.values): print(i) for word in text.split(): count[word] += 1 return count counter = counter_word(df['text']) len(counter) 

Answer: I think simplier is join values by space, then split for words and count: counter = Counter((' '.join(df['text'])).split()) print (counter) Counter({'hello': 2, 'is': 2, 'a': 2, 'this': 2, 'unique': 1, 'sentences': 1, 'test': 1, 'does': 1, 'works': 1})
Question: Here is the board I drew using pygame: https://i.sstatic.net/Hne6A.png I'm facing a bug with the thickness of the last two lines as I marked them on the image. I believe there is something wrong with the if statement in my code but I can't quite figure it out, it just won't take effect. here is the code that has drawn the board above: import pygame, sys pygame.init() width, height = 750, 750 rows, cols = 9, 9 BLACK = (0,0,0) screen = pygame.display.set_mode((width, height)) screen.fill((255, 255, 255, 255)) running = True while running: for event in pygame.event.get(): if event.type == pygame.QUIT: running = False sys.exit() surf = pygame.Surface((600, 600)) surf.fill((255,255,255)) padding = surf.get_width()/9 for i in range(rows+1): if i % 3 == 0: thick = 4 else: thick = 1 pygame.draw.line(surf, BLACK, (0, i*padding), (width, i*padding), width=thick) pygame.draw.line(surf, BLACK, (i*padding, 0), (i*padding, height), width=thick) surf_center = ( (width-surf.get_width())/2, (height-surf.get_height())/2 ) screen.blit(surf, surf_center) pygame.display.update() pygame.display.flip() 

Answer: To draw a thick line, half the thickness is applied to both sides of the line. The simplest solution is to make the target surface a little larger and add a small offset to the coordinates of the lines. For performance reasons, I also recommend creating the surface before the application loop and continuously blit it in the application loop: import pygame, sys pygame.init() width, height = 750, 750 rows, cols = 9, 9 BLACK = (0,0,0) screen = pygame.display.set_mode((width, height)) surf_size = 600 surf = pygame.Surface((surf_size+4, surf_size+4)) surf.fill((255,255,255)) padding = surf_size/9 for i in range(rows+1): thick = 4 if i % 3 == 0 else 1 offset = i * padding + 1 pygame.draw.line(surf, BLACK, (0, offset), (width, offset), width=thick) pygame.draw.line(surf, BLACK, (offset, 0), (offset, height), width=thick) running = True while running: for event in pygame.event.get(): if event.type == pygame.QUIT: running = False screen.fill((255, 255, 255, 255)) screen.blit(surf, surf.get_rect(center = screen.get_rect().center)) pygame.display.update() pygame.display.flip() pygame.quit() sys.exit()
Question: Considering this abstract class and a class implementing it: from abc import ABC class FooBase(ABC): foo: str bar: str baz: int def __init__(self): self.bar = "bar" self.baz = "baz" class Foo(FooBase): foo: str = "hello"  The idea here is that a Foo class that implements FooBase would be required to specify the value of the foo attribute, but the other attributes (bar and baz) would not need to be overwritten, as they're already handle by a method provided by the abstract class. From a MyPy type-checking perspective, is it possible to force Foo to declare the attribute foo and raise a type-checking error otherwise? EDIT: The rationale is that FooBase is part of a library, and the client code should be prevented from implementing it without specifying a value for foo. For bar and baz however, these are entirely managed by the library and the client doesn't care about them.

Answer: This is a partial answer. You can use class FooBase(ABC): @property @classmethod @abstractmethod def foo(cls) -> str: ... class Foo(FooBase): foo = "hi" def go(f: FooBase) -> str: return f.foo  It's only partial because you'll only get a mypy error if you try to instantiate Foo without an initialized foo, like class Foo(FooBase): ... Foo() # error: Cannot instantiate abstract class "Foo" with abstract attribute "foo"  This is the same behaviour as when you have a simple @abstractmethod. Only when instantiating it is the error raised. This is expected because Foo might not be intended as a concrete class, and may itself be subclassed. You can mitigate this somewhat by stating it is a concrete class with typing.final. The following will raise an error on the class itself. @final class Foo(FooBase): # error: Final class __main__.Foo has abstract attributes "foo" ...