eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I find the glimpse function very useful in R/dplyr. But as someone who is used to R and is working with Python now, I haven't found something as useful for Panda dataframes. In Python, I've tried things like .describe() and .info() and .head() but none of these give me the useful snapshot which R's glimpse() gives us. Nice features which I'm quite accustomed to having in glimpse() include:  All variables/column names as rows in the output All variable/column data types The first few observations of each column Total number of observations Total number of variables/columns  Here is some simple code you could work it with: R library(dplyr) test <- data.frame(column_one = c("A", "B", "C", "D"), column_two = c(1:4)) glimpse(test) # The output is as follows Rows: 4 Columns: 2 $ column_one <chr> "A", "B", "C", "D" $ column_two <int> 1, 2, 3, 4  Python import pandas as pd test = pd.DataFrame({'column_one':['A', 'B', 'C', 'D'], 'column_two':[1, 2, 3, 4]})  Is there a single function for Python which mirrors these capabilities closely (not multiple and not partly)? If not, how would you create a function that does the job precisely?

Answer: Here is one way to do it: def glimpse(df): print(f"Rows: {df.shape[0]}") print(f"Columns: {df.shape[1]}") for col in df.columns: print(f"$ {col} <{df[col].dtype}> {df[col].head().values}")  Then: import pandas as pd df = pd.DataFrame( {"column_one": ["A", "B", "C", "D"], "column_two": [1, 2, 3, 4]} ) glimpse(df) # Output Rows: 4 Columns: 2 $ column_one <object> ['A' 'B' 'C' 'D'] $ column_two <int64> [1 2 3 4]
Question: I have a set of U users and a set of S servers. I want to maximize the number of users allocated to a server while minimizing the number of servers used (this means that I have two objective functions). Each user has some requirements w and each server has a total capacity of C. The solver variables are the following: # x[i,j] = True if user u[j] is allocated to server s[i] # x[i,j] = False otherwise # y[i] = True if server s[i] is used to serve users # y[i] = False otherwise  As mentioned before, I want to maximize x[i,j] while minimizing y[i] The constraints are the following:  Capacity constraint: Since each server i has a certain capacity, the allocation of j users must not exceed that capacity Proximity constraint: Only users located within the range of the server can be allocated to it. A user can be located in the overlapping range of multiple servers Constraint family: Ensures every user is allocated to at most one server.  Using this library from ortools.sat.python import cp_model  So far I've done:  Create the solver variables (they are boolean) Create the constraints Maximize the x[i,j] variable Obtain the objective function  For instance, if I have 10 users and 4 servers all the 10 users are allocated among the 4 servers What I need but haven't been able to accomplish:  Maximize the x[i,j] variable AND Minimize the y[i] variable  For the same 10 users and the same 4 servers above, all the 10 users can be allocated among just 2 servers and not 4 I have tried the solution given in this post but it is not working since I got that the problem does not have an optimal solution

Answer: there are usually 2 approaches:  weighted sum: a * obj1 + b * obj2 lexicographic: optimize obj1, get optimal value, change objective to obj2, add constraint obj1 <= best_obj1_value (optional + slack). Then reoptimize. Bonus point when reusing the optimal solution with obj1 as a hint for the second solve.
Question: I would like to save python pandas DataFrame object as pickle. What's the diffrence in using pandas.to_pickle vs pickle.dumps? I've made some tests. Here's my test code : import pandas as pd import pickle df = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB'), index=['x', 'y']) # Save df.to_pickle('df1.pickle') with open('df2.pickle','wb') as f: pickle.dump(df, f) # Load df1 = pd.read_pickle('df1.pickle') with open('df2.pickle','rb') as f: df2 = pickle.load(f) # Is ok? if (df1 == df2).all().all(): print('Data is ok.') # Load opposite df3 = pd.read_pickle('df2.pickle') with open('df1.pickle','rb') as f: df4 = pickle.load(f) # Is ok? if (df3 == df4).all().all(): print('Data opposite is ok.')  Result : Data is ok. Data opposite is ok.  Is there any diffrence? I see some diffrence in output pickle file size. Pandas version file is bigger. -rw-rw-r-- 1 spasz spasz 694 lis 15 17:38 df1.pickle -rw-rw-r-- 1 spasz spasz 662 lis 15 17:38 df2.pickle  Tested on python 3.8.0, pandas 1.5.0.

Answer: After analyzing module code for pickle (python3.8) and pandas 1.5.0 here are my thoughts. Saving/dumping DataFrame to pickle Pickle code :  Default using pickle protocol DEFAULT_PROTOCOL if not specified. DEFAULT_PROTOCOL(==4) is not the HIGHEST_PROTOCOL(==5).  Pandas code :  Default using pickle protocol HIGHEST_PROTOCOL, Handles error GH#39002 when saving pickle with protocol==5 and compression bz2 or xz,  Reading DataFrame from pickle Pickle code :  No important diffrence to mention,  Pandas code :  Default use of direct call to pickle.load() Catches some exceptions to handle data written in previous pandas versions and python 2.7 unicode errors (see GH#28645 and GH#31988)  My final thoughts As @DeepSpace said, pandas pretty much calls pickle functions directly. If you're creating new data with current pandas versions, and not using pickle protocol==5 with bz2/xz compression, then you can safely use of pickle module functions.
Question: I am creating a website so I finshed register page in login page, every details are correct but that is showing please verify your details I mean the else part also I put in a print statement after username and password same details are printing when I typed but not access login. views.py def sign_in(request): if request.method == "POST": username=request.POST['username'] password=request.POST['password'] print(username,password) user = authenticate(username=username,password=password) if user is not None: login(request,user) messages.success(request,"you are logged successfully ") return redirect("front_page") else: messages.error(request,"please verify your details") return redirect("sign_in") return render(request,"login.html") def front_page(request): return render(request,"frontpage.html")  urls.py  path('log_in',views.sign_in,name="sign_in"), path('front_page',views.front_page,name="front_page"),  html  <div class="signup-form"> <form action="{% url 'sign_in' %}" method="POST"> {% csrf_token %} <h2 class="text-center">Login</h2> <p class="text-center">Please fill in this form to login your account!</p> <hr> <div class="form-group"> <div class="input-group"> <div class="input-group-prepend"> <span class="input-group-text"> <span class="fa fa-user"></span> </span> </div> <input type="text" class="form-control" name="username" placeholder="Username" required="required"> </div> </div> <div class="form-group"> <div class="input-group"> <div class="input-group-prepend"> <span class="input-group-text"> <i class="fa fa-lock"></i> </span> </div> <input type="password" class="form-control" name="password" placeholder="Password" required="required"> </div> </div> <div class="form-group d-flex justify-content-center"> <button type="submit" class="btn btn-primary btn-lg">Login</button> </div> <div class="text-center text-primary">Dont have a account? <a href="{% url 'register' %}">Create here </a></div> </form> </div>  I just want to login with username and password, but it shows please verify your details, but all the details are correct. Any ideas?

Answer: authenticate() also requires request as an argument so it should be: user = authenticate(request,username=username,password=password)  Your view for registering users should be like this: def register(request): if request.method == "POST": username=request.POST['username'] password=request.POST['password'] confirm_password=request.POST['confirm_password'] if password == confirm_password: if User.objects.filter(username=username).exists(): messages.info(request,"user name was taken please choose another one") return redirect('register') else: user1=User.objects.create_user(username=username,password=password) user1.is_active = False messages.success(request,"you account was created successfully thank you for choosing us ") return redirect("sign_in") else: messages.info(request,"please verify your passwords") return redirect('register') else: return render(request,'register.html')
Question: I am having some trouble working around what I can only assume is a bug in Tkinter. from tkinter import * def refocus(event, obj): obj.focus() root = Tk() options = ["Hello", "world", "How", "are", "you"] v1 = StringVar() v2 = StringVar() v3 = StringVar() o1 = OptionMenu(root, v1, *options) o1.configure(takefocus=1) o2 = OptionMenu(root, v2, *options) o2.configure(takefocus=1) o3 = OptionMenu(root, v3, *options) o3.configure(takefocus=1) o1.bind("<Configure>", lambda e=Event(), o=o1: refocus(e, o)) o2.bind("<Configure>", lambda e=Event(), o=o2: refocus(e, o)) o3.bind("<Configure>", lambda e=Event(), o=o3: refocus(e, o)) o1.pack(side=TOP) o2.pack(side=TOP) o3.pack(side=TOP) root.mainloop()  From the code there are 3 OptionMenus which I am trying to navigate using tab, space, arrows and Enter keys on the keyboard. The issue is as soon as a menu box pops up it seems to loose focus of the original Option menu. To fix this I have bound configure which re-focuses the option box that it was on, but this only works if the input has changed so I need a new method of doing this. Still there is an issue where if I select the wrong option, i have to cycle through all the other inputs (OptionMenus, Entrys, Checkboxes etc.) to get back to that option for it to open again. In cases where there is only 1 OptionMenu then it will not work until I click another input with a mouse I am looking for a way that I can focus back on the OptionMenu after the menu part has lost focus. I have also tried using o1['menu'].bind(... but this did not work at all. example process: o1 focused -> pressed space -> open menu -> arrows to move -> enter to select -> focus on o1 -> press space -> open menu -> arrows to move -> enter to select -> focus on o1 -> press tab -> o2 focused

Answer: Look at this: import tkinter as tk def open_option_menu(event): # Get the widget from the event that tkinter passed in obj = event.widget # Calculate the x/y position of the popup window x = obj.winfo_rootx() y = obj.winfo_rooty() + obj.winfo_height() # Show the popup window # obj["menu"] returns a `tk.Menu` object which has a `.post` method obj["menu"].post(x, y) # Stop tkinter from showing the popup window again. return "break" root = tk.Tk() options = ["Hello", "world", "How", "are", "you"] v1 = tk.StringVar(root) v2 = tk.StringVar(root) v3 = tk.StringVar(root) o1 = tk.OptionMenu(root, v1, *options) o2 = tk.OptionMenu(root, v2, *options) o3 = tk.OptionMenu(root, v3, *options) o1.configure(takefocus=True) o2.configure(takefocus=True) o3.configure(takefocus=True) o1.bind("<space>", open_option_menu) o2.bind("<space>", open_option_menu) o3.bind("<space>", open_option_menu) o1.pack() o2.pack() o3.pack() root.mainloop()  I think it's a bug in tkinter so I created a function open_option_menu that shows the popup window. If you don't like something about tkinter, you can rebuild it yourself with the functionality that you need. If you want to know more about the .post method, type help(tkinter.Menu.post) in python. Note, this problem only exists on Windows, so the solution only works with Windows. On Ubuntu, the popup isn't focused when its shown.
Question: I want to evaluate with accuracy, precision, recall, f1 like this code but it show same result. df = pd.read_csv(r'test.csv') X = df.iloc[:,:10] Y = df.iloc[:,10] X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2) clf = DecisionTreeClassifier() clf = clf.fit(X_train,y_train) predictions = clf.predict(X_test) accuracy = accuracy_score(y_test, predictions) precision = precision_score(y_test, predictions,average='micro') recall = recall_score(y_test, predictions,average='micro') f1 = f1_score(y_test, predictions,average='micro') print("Accuracy: ", accuracy) print("precision: ", precision) print("recall: ", recall) print("f1: ", f1)  It show output like this. Accuracy: 0.8058823529411765 precision: 0.8058823529411765 recall: 0.8058823529411765 f1: 0.8058823529411765  The output is same value. How to fix it?

Answer: According to sklearn's documentation, the behavior is expected when using micro as average and when dealing with a multiclass setting:  Note that if all labels are included, “micro”-averaging in a multiclass setting will produce precision, recall and F that are all identical to accuracy.  Here is a nice blog article describing why these scores can be equal (also with an intuitive example) TL;DR  F1 equals recall and precision if recall == precision In the case of micro averaging, the number of false positive always equals the number of false negative. Thus, recall == precision Finally, note that micro F1 always equals accuracy. See here
Question: What I'm trying to achieve is that when a row in col2 has a 1, it will copy that 1 onto all the other values in col2 as long as the rows in col1 have the same name. As an example, if the dataframe looks like this col1 col2 xx 1 xx 0 xx 0 xx 0 yy 0 yy 0 yy 0 zz 0 zz 0 zz 1  The output would be col1 col2 xx 1 xx 1 xx 1 xx 1 yy 0 yy 0 yy 0 zz 1 zz 1 zz 1 

Answer: Use groupby.transform('max'): df['col2'] = df.groupby('col1')['col2'].transform('max')  Output:  col1 col2 0 xx 1 1 xx 1 2 xx 1 3 xx 1 4 yy 0 5 yy 0 6 yy 0 7 zz 1 8 zz 1 9 zz 1
Question: I apologize if this is a duplicate question but I can't find an answer online. In Django Checklist Docs I see the following to keep secret key secure. with open('/etc/secret_key.txt') as f: SECRET_KEY = f.read().strip()  My project is deployed with AWS EBS. I've created a separate file called "secret_key.txt" which holds the key. How is this more secure than keeping the key in the settings.py config file? If someone can access my projects settings.py file to access the key, would they not be able to access the "secret_key.txt" file as well? How is creating a "secret_key.txt" file more secure? I've checked Google and Stack Overflow for reasoning but can't find an answer. Currently all sensitive information is protected using an .env file and including this file in .gitignore.

Answer: You usually add that file to the .gitignore, such that the file is not part of the (GitHub) repository. This means that you can add (other) settings in the project, and you load "sensitive" settings through environment variables, or files. This hackernoon post for example, discusses four ways to define sensitive variables such that these are not defined in files that you add to the subversioning system. Usually it is advisable to incude a settings.py in the project however, stripped from sensitive data. That way a peer can easily set up the project all the other (required) settings, and thus only has to define a limited number of sensitive variable to get the project running. I think however using an environment variable might be better, since it is probably easier to specify this, and thus to manage a number of processes that all might work with different values.
Question: I've got the win32 drivers from https://github.com/mozilla/geckodriver/releases and placed the exe under the python38 folder I'm running windows 11 OSError: [WinError 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher here you can find the full terminal output https://pastebin.com/k3Gvm2nU > `from selenium import webdriver > from selenium.webdriver.common.keys import Keys > from selenium.webdriver.common.by import By > > driver = webdriver.Firefox() > driver.get("http://www.python.org") > assert "Python" in driver.title > elem = driver.find_element(By.NAME, "q") > elem.clear() > elem.send_keys("l") > elem.send_keys(Keys.RETURN) > assert "No results found." not in driver.page_source`  this is the code, I was expecting it to open a firefox page but it doesn't, I think that geckodriver isn't running because its incompatible for some reasons?

Answer: You can use webdriver_manager to get rid of driver problems. You can use webdriver_manager for firefox as you can see in the link as follows for selenium 3 from selenium import webdriver from webdriver_manager.firefox import GeckoDriverManager driver = webdriver.Firefox(executable_path=GeckoDriverManager().install())  for selenium 4 from selenium import webdriver from selenium.webdriver.firefox.service import Service as FirefoxService from webdriver_manager.firefox import GeckoDriverManager driver = webdriver.Firefox(service=FirefoxService(GeckoDriverManager().install()))
Question: I want to create a docker image (docker version: 20.10.20)that contains python libraires from a requirement.txt file that contains 50 libraries. Without facing root user permissions how can proceed. Here is the file: From ubuntu:latest RUN apt update RUN apt install python3 -y WORKDIR /Destop/DS # COPY requirement.txt ./ # RUN pip install -r requirement.txt # it contains only pandas==1.5.1 COPY script2.py ./ CMD ["python3", "./script2.py"]  It failed at requiremnt.txt command *error it takes lot of time while creating image. because it ask for root permission.

Answer: For me the only problem in your Dockerfile is in the line RUN apt install python -y. This is erroring with Package 'python' has no installation candidate. It is expected since python refers to version 2.x of Python wich is deprecated and no longer present in the default Ubuntu repositories. Changing your Dockerfile to use Python version 3.x worked fine for me. FROM ubuntu:latest RUN apt update RUN apt install python3 python3-pip -y WORKDIR /Destop/DS COPY requirement.txt ./ RUN pip3 install -r requirement.txt COPY script2.py ./ CMD ["python3", "./script2.py"]  To test I used requirement.txt pandas==1.5.1  and script2.py import pandas as pd print(pd.__version__)  With this building the docker image and running a container from it executed succesfully. docker build -t myimage . docker run --rm myimage
Question: Minimum working example: #Python import matplotlib.pyplot as plt x = [0, 1, 2, 3, 4, 5] y = [0, 1, 2, 3, 4, 5] z = [0, 1, 2, 3, 4, 5] fig = plt.figure() ax = plt.axes(projection="3d") ax.scatter(x, y, z, c='g', s=20) plt.xlabel("X data") plt.ylabel("Y data") #plt.zlabel("Z data") DOES NOT WORK ax.view_init(60,35) plt.show()  Question: how to set up the label of the Z axis? For some reason plt has the xlabel and ylabel properties, but not the zlabel.

Answer: For 3D plots the labels need to be changed using the axes objects. Try something like this ax.set_xlabel('X Label') ax.set_ylabel('Y Label') ax.set_zlabel('Z Label')
Question: I'm trying to print a whole number (such as 39 for example) in the following format: 39. It must not be a str type object like '39.' for example, but a number e. g. n = 39.0 should be printed like 39. n = 39.0 #magic stuff with output 39.  I tried using :.nf methods (:.0f apparently -- didn't work), print(float(39.)) or just print(39.) In the first case, it looks like 39, in the second and third 39.0 I also tried float(str(39) + '.') and obviously it didn't work Sorry, if it's a stupid question, I've been trying to solve it for several hours already, still can't find any information.

Answer: From Format Specification Mini-Language (emphasis mine):  The '#' option causes the “alternate form” to be used for the conversion. The alternate form is defined differently for different types. This option is only valid for integer, float and complex types. For integers, when binary, octal, or hexadecimal output is used, this option adds the respective prefix '0b', '0o', '0x', or '0X' to the output value. For float and complex the alternate form causes the result of the conversion to always contain a decimal-point character, even if no digits follow it. Normally, a decimal-point character appears in the result of these conversions only if a digit follows it. In addition, for 'g' and 'G' conversions, trailing zeros are not removed from the result.  >>> n=39.0 >>> print(f'{n:#.0f}') 39.
Question: I'm just curious to hear other people's thoughts on why this specific piece of code might might run slower in Python 3.11 than in Python 3.10.6. Cross-posted from here. I'm new here - please kindly let me know if I'm doing something wrong. test.py script: import timeit from random import random def run(): for i in range(100): j = random() t = timeit.timeit(run, number=1000000) print(t)  Commands: (base) conda activate python_3_10_6 (python_3_10_6) python test.py 5.0430680999998 (python_3_10_6) conda activate python_3_11 (python_3_11) python test.py 5.801756700006081 

Answer: This looks like it's probably the PEP 659 optimizations not paying off for random.random. PEP 659 is an effort to JIT-optimize many common operations. (Not JIT compilation, but definitely JIT optimization.) It pays off for most Python code, but I think random.random isn't covered. random.random is a method (of a hidden random.Random instance) written in C, with no arguments other than self, so it should be using the METH_NOARGS calling convention. This calling convention has no specialized fast path. Both specialize_c_call and _Py_Specialize_Call just bail out instead of specializing the call. When PEP 659 doesn't pay off, the work that goes into supporting it is just overhead. I'm not sure what parts contribute how much overhead, but the bytecode is longer than before, due to generating PRECALL and CALL instructions (although I think there's some work going on to improve that), plus attempting specialization and tracking when to attempt specialization has its own overhead.
Question: I have data with timestamps. Users respond to questions and they also select day period (morning or evening). I want to drop rows where recorded timestamp and day period mismatch. So check, if timestamp is between 6am-12pm and discard if "daytime" is "evening", etc. df timestamps daytime 2020-04-10 11:40 Morning 2022-04-12 19:32 Morning *(discard)* 2022-04-12 20:53 Evening 2022-04-15 22:50 Morning *(discard)* 2022-04-16 09:31 Evening*(discard)*  The rule should be: if between 06:00-12:00 and 'daytime' is Evening ==> Remove row/ if between 18:00 - 00:00 and 'daytime' is Morning ==> Remove row I've tried: remove = df[ (6< df['timestamp'].dt.hour < 12 & df['period'] == 'Evening') | (18< df['timestamp'].dt.hour < 23 & df['period'] == 'Morning')] df.drop(remove , inplace=True) 

Answer: Instead of dropping you can use .query() to filter. df["timestamps"] = pd.to_datetime(df["timestamps"]) df = df.query( "timestamps.dt.hour.between(6, 12, inclusive='both') & daytime.eq('Morning') | " "timestamps.dt.hour.between(18, 23, inclusive='both') & daytime.eq('Evening')" ).reset_index(drop=True) print(df) timestamps daytime 0 2020-04-10 11:40:00 Morning 1 2022-04-12 20:53:00 Evening
Question: The gcloud auth print-identity-token command prints an identity token for the specified account. $(gcloud auth print-identity-token \ --audiences=https://example.com \ --impersonate-service-account my-sa@my-project.iam.gserviceaccount.com \ --include-email)  How do I do the same using Python?

Answer: Here a code sample (not so easy and well documented) import google.auth.transport.requests from google.auth.impersonated_credentials import IDTokenCredentials SCOPES = ['https://www.googleapis.com/auth/cloud-platform'] request = google.auth.transport.requests.Request() audience = 'my_audience' creds, _ = google.auth.default(scopes=SCOPES) icreds = google.auth.impersonated_credentials.Credentials( source_credentials=creds, target_principal="SA TO IMPERSONATE", target_scopes=SCOPES) id = IDTokenCredentials(icreds, target_audience=audience,include_email=True) id.refresh(request) print(id.token)