eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: i need a password validation in fastapi python, in this when user signup and create a password and passowrd are too sort not capital letter, special character etc. than fastapi give validation error i make a password validation code in python but i don't know how to use in fastapi def validate_password(s): l, u, p, d = 0, 0, 0, 0 capitalalphabets="ABCDEFGHIJKLMNOPQRSTUVWXYZ" smallalphabets="abcdefghijklmnopqrstuvwxyz" specialchar=""" ~`!@#$%^&*()_-+={[}]|\:;"'<,>.?/ """ digits="0123456789" if (len(s) >= 8): for i in s: # counting lowercase alphabets if (i in smallalphabets): l+=1 # counting uppercase alphabets if (i in capitalalphabets): u+=1 # counting digits if (i in digits): d+=1 # counting the mentioned special characters if(i in specialchar): p+=1 if (l>=1 and u>=1 and p>=1 and d>=1 and l+p+u+d==len(s)): print("Valid Password") else: print("Invalid Password") s = input("Enter the password: ") validate_password(s) 

Answer: You can import validator from Pydantic and fill it by your field name of your schema (in this case "password"). Usage in your schema file: from pydantic import BaseModel, validator class User(BaseModel): password: str @validator("password") def validate_password(cls, password, **kwargs): # Put your validations here return password  For this problem, a better solution is using regex for password validation and using regex in your Pydantic schema. Example of strong password regex validation: from pydantic import BaseModel, Field password_regex = "((?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[\W]).{8,64})" class User(BaseModel): password: str = Field(..., regex=password_regex)
Question: i am looking for a way to disaggregate data from a single row in a pandas df. My data looks like this edit: n stands for an unspecified number, e.g. in my working dataset I have 8 plots giving me 8 x 2 = 16 columns I would like to transform. data = { 'key':['k1', 'k2'], 'plot_name_1':['name', 'name'], 'plot_area_1':[1,2], 'plot_name_2':['name', 'name'], 'plot_area_2':[1,2], 'plot_name_n':['name', 'name'], 'plot_area_n':[1,2] } df = pd.DataFrame(data)   And I would like to end up here, adding an extra column to identify the plot number: data = { 'key':['k1','k1','k1', 'k2', 'k2', 'k2'], 'plot_number':['1', '2', 'n','1', '2', 'n'], 'plot_name':['name', 'name','name', 'name','name', 'name'], 'plot_area':[1,2,1,2,1,2], } df = pd.DataFrame(data)  

Answer: pd.wide_to_long can do this: In [160]: pd.wide_to_long(df, stubnames=["plot_name", "plot_area"], i="key", j="plot_number", sep="_", suffix=r"(?:\d+|n)").reset_index() Out[160]: key plot_number plot_name plot_area 0 k1 1 name 1 1 k2 1 name 2 2 k1 2 name 1 3 k2 2 name 2 4 k1 n name 1 5 k2 n name 2  where  "stubnames" are the common column prefix names to capture "sep" is the separator after those prefixes  "_" in your case   "suffix" is what's expected after the separator  one or more digits, or literal "n" in your case; if "n" was symbolic, you can have suffix=r"\d+" there   "i" argument is the "index" (i.e., the identifier variables) "j" signifies the name under which the suffixes are gathered.   As an aside, we need to paranthesise the regex when multiple suffixes are possible because of the way pandas uses the suffix under the hood when constructing the regex: regex = rf"^{re.escape(stub)}{re.escape(sep)}{suffix}$"  We see that suffix is interpolated directly, and an alternator (i.e., |) in it will see the left side as, e.g., not only \d+ but also what comes from stub & sep, too.
Question: I am creating an application with FastAPI and so far it goes like this:  But I'm having a problem with the endpoints. The /api/items/filter route has two query parameters: name and category. However, it gives me the impression that it is being taken as if it were api/items/{user_id}/filter, since when I do the validation in the documentation it throws me an error saying that I have not passed a value for user_id. (Also, previously it asked me to be authenticated (the only route that needed authentication was api/items/{user_id}. The problems are fixed when I define this endpoint first as shown below:  Why is this happening? Is there a concept that I am not clear?

Answer: Ordering your endpoints matters! Endpoints are matched in order they are declared in your FastAPI object. Let say you have only two endpoints, in this order:  api/items/{user_id} api/items/filter  In this order, when you request endpoint api/items/user_a, your request will be routed to (1) api/items/{user_id}. However, if you request api/items/filter, this also will be routed to (1) api/items/{user_id}! That is because filter is a match for {user_id}, and since this endpoint is evaluated before the second endpoint is evaluated for a match, the second endpoint is not evaluated at all. That is also why you are asked for authorization; you think you are requesting endpoint 2, but your request is actually routed to endpoint 1, with path parameter {user_id} = "filter". So, ordering your endpoints is important, and it is just where in your application you are defining them. See here in the docs.
Question: I'm trying to check if my pyspark dataframe is empty and I have tried different ways to do that, like: df.count() == 0 df.rdd.isEmpty() df.first().isEmpty()  But all this solutions are to slow, taking up to 2 minutes to run. How can I quicly check if my pyspark dataframe is empty or not? Do anyone have a solution for that? Thank you in advance!

Answer: The best way to check if your dataframe is empty or not after reading a table or at any point in time is by using limit(1) first which will reduce the number of rows to only 1 and will increase the speed of all the operation that you are going to do for dataframe checks. df.limit(1).count() == 0 df.limit(1).rdd.isEmpty() df.limit(1).take()  If you are just doing a data dependency check on a table and just want to know if table has data or not, it's always best to just apply limit 1 while reading from table itself for e.g. df = spark.sql("select * from <table> limit 1")  With that being said about the efficiency of checking dataframe is empty or not, now coming on to which is the fastest way of doing it is using .rdd.isEmpty() compared to count(), first() or take(1) Also if you see the backend implementation of first() and take(1) it's completely implemented on top of collect() which is mostly costly and should only be used when extremely necessary. The below-mentioned time is based on reading a parquet file with 2390491 records and having 138 columns. >>> df.count() 2390491 >>> len(df.columns) 138   Note: These are the time taken after applying .limit(1) to the dataframe for checking whether the dataframe is empty or not. Also lastly using, df.rdd.isEmpty() took the least amount of time 29ms after reducing the number of rows to 1.  Hope that helps..!! :) UPDATE: If you are using Spark >= 3.3, now you can directly use, df.isEmpty()  This is the fastest of all for checking an empty data frame in Spark >= 3.3
Question: I am using polars in place of pandas. I am quite amazed by the speed and lazy computation/evaluation. Right now, there are a lot of methods on lazy dataframe, but they can only drive me so far. So, I am wondering what is the best way to use polars in combination with other tools to achieve more complicated operations, such as regression/model fitting. To be more specific, I will give an example involving linear regression. Assume I have a polars dataframe with columns day, y, x1 and x2, and I want to generate a series, which is the residual of regressing y on x1 and x2 group by day. I have included the code example as follows and how it can be solved using pandas and statsmodels. How can I get the same result with the most efficiency using idiomatic polars? import pandas as pd import statsmodels.api as sm def regress_resid(df, yvar, xvars): result = sm.OLS(df[yvar], sm.add_constant(df[xvars])).fit() return result.resid df = pd.DataFrame( { "day": [1, 1, 1, 1, 1, 2, 2, 2, 2, 2], "y": [1, 6, 3, 2, 8, 4, 5, 2, 7, 3], "x1": [1, 8, 2, 3, 5, 2, 1, 2, 7, 3], "x2": [8, 5, 3, 6, 3, 7, 3, 2, 9, 1], } ) df.groupby("day").apply(regress_resid, "y", ["x1", "x2"]) # day # 1 0 0.772431 # 1 -0.689233 # 2 -1.167210 # 3 -0.827896 # 4 1.911909 # 2 5 -0.851691 # 6 1.719451 # 7 -1.167727 # 8 0.354871 # 9 -0.054905  Thanks for your help.

Answer: If you want to pass multiple columns to a function, you have to pack them into a Struct as polars expression always map from Series -> Series. Because polars does not use numpy memory which statsmodels does, you must convert the polars types to_numpy. This is often free in case of 1D structures. Finally, the function should not return a numpy array, but a polars Series instead, so we convert the result. import polars as pl from functools import partial import statsmodels.api as sm def regress_resid(s: pl.Series, yvar: str, xvars: list[str]) -> pl.Series: df = s.struct.unnest() yvar = df[yvar].to_numpy() xvars = df[xvars].to_numpy() result = sm.OLS(yvar, sm.add_constant(xvars)).fit() return pl.Series(result.resid) df = pl.DataFrame( { "day": [1, 1, 1, 1, 1, 2, 2, 2, 2, 2], "y": [1, 6, 3, 2, 8, 4, 5, 2, 7, 3], "x1": [1, 8, 2, 3, 5, 2, 1, 2, 7, 3], "x2": [8, 5, 3, 6, 3, 7, 3, 2, 9, 1], } ) (df.group_by("day") .agg( pl.struct("y", "x1", "x2").map_elements(partial(regress_resid, yvar="y", xvars=["x1", "x2"])) ) )
Question: In python, we can make an empty list easily by doing a = []. I want to do a similar thing but with Pytorch tensors. If you want to know why I need that, I want to get all of the data inside a given dataloader (to create another customer dataloader). Having an empty tensor can help me gather all of the data inside a tensor using a for-loop. This is a sudo code for it. all_data_tensor = # An empty tensor for data in dataloader: all_data_tensor = torch.cat((all_data_tensor, data), 0)  Is there any way to do this?

Answer: We can do this using torch.empty. But notice torch.empty needs dimensions and we should give 0 to the first dimension to have an empty tensor. The code will be like this: # suppose the data generated by the dataloader has the size of (batch, 25) all_data_tensor = torch.empty((0, 25), dtype=torch.float32) # first dimension should be zero. for data in dataloader: all_data_tensor = torch.cat((all_data_tensor, data), 0)
Question: I have a dataset with type dictionary which I converted to Dataset: ds = datasets.Dataset.from_dict(bio_dict) The shape now is: Dataset({ features: ['id', 'text', 'ner_tags', 'input_ids', 'attention_mask', 'label'], num_rows: 8805 })  When I use the train_test_split function of Datasets I receive the following error: train_testvalid = ds.train_test_split(test_size=0.5, shuffle=True, stratify_by_column="label")   ValueError: Stratifying by column is only supported for ClassLabel column, and column label is Sequence.  How can I change the type to ClassLabel so that stratify works?

Answer: You should apply the following class_encode_column function: ds = ds.class_encode_column("label")
Question: I have a pyproject.toml with [tool.poetry] name = "my-project" version = "0.1.0" [tool.commitizen] name = "cz_conventional_commits" version = "0.1.0"  I add a new feature and commit with commit message feat: add parameter for new feature  That's one commit. Then I call commitizen bump  Commitizen will recognize a minor version increase, update my pyproject.toml, and commit again with the updated pyproject.toml and a tag 0.2.0. That's a second commit. But now my pyproject.toml is "out of whack" (assuming I want my build version in sync with my git tags). [tool.poetry] name = "my-project" version = "0.1.0" [tool.commitizen] name = "cz_conventional_commits" version = "0.2.0"  I'm two commits in, one tagged, and things still aren't quite right. Is there workflow to keep everything aligned?

Answer: refer to support-for-pep621 and version_files you can add "pyproject.toml:^version" to pyproject.toml: [tool.commitizen] version_files = [ "pyproject.toml:^version" ]
Question: I have made separate functions for each page, but i want to change page to file upload when I click button on welcome_page.py. Found a switch_page function but I don't think I understand how it works. import streamlit as st from login import check_password from file_upload import file_upload from welcome import welcome_page from PIL import Image from numpy import asarray from streamlit_extras.switch_page_button import switch_page def file_upload(): datafile = st.file_uploader("Upload JPG",type=['jpg']) if datafile is not None: numpydata = asarray(datafile) print(type(numpydata)) st.image(datafile) return True return False if check_password(): if welcome_page(): switch_page('file_upload') fileDetails = file_upload() if fileDetails: print("file uploaded") 

Answer: Unfortunately, there isn't a built-in way to switch pages programmatically in a Streamlit multipage app. There is an open feature request here if you'd like to upvote it.
Question: I am getting the following error when trying to plot a lineplot with seaborn. TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''  Minimal example reproducing error: import matplotlib.pyplot as plt import seaborn as sns import pandas as pd dataset = { "x": [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3], "y": [1.0, 2.3, 4.5, 1.2, 3.4, 5.3, 1.1, 2.4, 3.6, 1.1, 3.3, 5.3], "id": ["a", "a", "a", "b", "b", "b", "a", "a", "a", "b", "b", "b"], "seed": [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1], } df = pd.DataFrame(data=dataset) print(df.dtypes) g = sns.lineplot(data=df, x="x", y="y", hue="id", errorbar="sd") plt.show() plt.close()  I have tried checking the datatypes of all inputs and Dataframe columns, and changing "id" to be an integer type (even though that is not my goal) and the error persists.

Answer: Numpy 1.24.0 has a bug that causes an exception within several seaborn functions. The bug has been fixed and numpy has released a new version to address it. The solution is to install numpy 1.24.1 the same way that you installed numpy 1.24.0.
Question: Context I have the following paragraph: text = """ בביהכנ"ס - בבית הכנסת דו"ח - דין וחשבון הת"ד - התיקוני דיקנא בגו"ר - בגשמיות ורוחניות ה"א - ה' אלוקיכם התמי' - התמיהה בהנ"ל - בהנזכר לעיל ה"א - ה' אלקיך ואח"כ - ואחר כך בהשי״ת - בהשם יתברך ה"ה - הרי הוא / הוא הדין ואת"ה - ואיגוד תלמידי """  this paragraph is combined with Hebrew words and their acronyms. A word contains quotation marks ("). So for example, some words would be: [ 'בביהכנ"ס', 'דו"ח', 'הת"ד' ]  Now, I'm able to match all the words with this regex: (\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b)   Question But how can I also match all the corresponding acronyms as a separate group? (the acronyms are what's not matched, so not the green in the picture). Example acronyms are: [ 'בבית הכנסת', 'דין וחשבון', 'התיקוני דיקנא' ]  Expected output The expected output should be a dictionary with the Words as keys and the Acronyms as values: { 'בביהכנס': 'בבית הכנסת', 'דו"ח': 'דין וחשבון', 'הת"ד': 'התיקוני דיקנא' }  My attempt What I tried was to match all the words (as above picture): (\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b)  and then match everything until the pattern appears again with .*\1, so the entire regex would be: (\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b).*\1  But as you can see, that doesn't work:   How can I match the words and acronyms to compose a dictionary with the words/acronyms?  Note When you print the output, it might be printed in Left-to-right order. But it should really be from Right to left. So if you want to print from right to left, see this answer: right-to-left languages in Python

Answer: You can do something like this: import re pattern = r'(\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b)\s*-\s*([^"]+)(\s|$)' text = """בביהכנ"ס - בבית הכנסת דו"ח - דין וחשבון הת"ד - התיקוני דיקנא""" for word, acronym, _ in re.findall(pattern, text): print(word + ' == ' + acronym)  which outputs בביהכנ"ס == בבית הכנסת דו"ח == דין וחשבון הת"ד == התיקוני דיקנא  Let's take a closer look how I built the regex pattern. Here's the pattern from your question that matches words: (\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b) This part will match the delimiter between a word and it's acronym: \s*-\s* (spaces then dash then spaces) This part will match anything except for double quote: ([^"]+) Finally, not to match the beginning of the next word let's match space/EOL in the end: (\s|$). Concatenate all the parts above and you'll get my pattern: (\b[\u05D0-\u05EA]*\"\b[\u05D0-\u05EA]*\b)\s*-\s*([^"]+)(\s|$) re.findall() will return a list of tuples, one tuple for one match. Each tuple will contain strings matching the groups (the stuff within parenthesis) in the same order that groups appear in the pattern. So we need group number 0 (word) and group number 1 (acronym) to build our dict. Group number 2 is not needed.
Question: I want the tray icon to change according to the value of p_out. Specifically depending on its value, I want it to get a different color. Here's the code import pystray import ping3 while True: p_out = ping3.ping("google.com", unit="ms") if p_out == 0: img = white elif p_out >= 999: img = red else: print(f'\n{p_out:4.0f}', end='') if p_out <= 50: img = green elif p_out <= 60: img = yellow elif p_out < 100: img = orange elif p_out >= 100: img = red icon = pystray.Icon(" ", img) icon.run()  I tried to 'reset' the pystray icon on every loop but it didn't work. The icon only changes when I stop and rerun the script.

Answer: As correctly stated in the comments, providing code that cannot run, does not help the community members to assist you. Τhe code references variables named white, red, green, yellow and orange, but these variables have not been defined or assigned values. Despite all this, the dynamic update of the tray icon is certainly something that can be useful to others. Therefore, below you may find your code with the necessary corrections applied. import ping3 import pystray import threading # Import the threading module for creating threads from PIL import Image # Import the Image module from PIL for creating images def update_icon(): while True: ping_output = ping3.ping("google.com", unit="ms") print(f'\n{ping_output:4.0f}', end='') if ping_output == 0: img = Image.new("RGB", (32, 32), (255, 255, 255)) # 32px32px, white elif 0 < ping_output <= 50: img = Image.new("RGB", (32, 32), (0, 255, 0)) # 32px32px, green elif 50 < ping_output <= 60: img = Image.new("RGB", (32, 32), (255, 255, 0)) # 32px32px, yellow elif 60 < ping_output < 100: img = Image.new("RGB", (32, 32), (255, 165, 0)) # 32px32px, orange elif ping_output >= 100: img = Image.new("RGB", (32, 32), (255, 0, 0)) # 32px32px, red icon.icon = img if __name__ == "__main__": icon = pystray.Icon("ping") icon.icon = Image.new("RGB", (32, 32), (255, 255, 255)) # 32px32px, white # Create a new thread to run the update_icon() function thread = threading.Thread(target=update_icon) # Start the thread thread.start() icon.run()  NOTES:  In order to use the Image module, you must first install Pillow, with a package manager like pip, using a command like this: pip install pillow. The purpose of creating a new thread -to run update_icon function- is to allow the tray icon to continue updating in the background, without blocking the main thread of execution. While 32x32px is a common size for icons, it is not necessarily the only size that can be used. Using a while True loop to run a continuous task can consume a large amount of system resources, such as CPU and memory, which can impact the overall performance of your program. However, I will not suggest anything about it as it is not relevant to the present question.
Question: I'm running Sublime Text Build 4143. Given a function with parameters that spill over the 80 character limit like so: def func(parameter_1, parameter_2, parameter_3, parameter_4, parameter_5, parameter_6): """ """ print("hello world") return  ST will show a PEP8 E501: line too long warning and highlight the line (which is fine):  But I can fold this function appropriately:  If I modify it to avoid the PEP8 warning:  I can no longer fold it:  This changed in the last update I think, because I used to be able to fold these functions without issues. How can I get around ths?

Answer: This is not a "fair" answer, but may be helpful. If you use black-like line folding (or anything similar, but with one important property: the closing parenthesis should be on its own line and be indented to the same level as def, see below), then folding works even better: def func( parameter_1, parameter_2, parameter_3, parameter_4, parameter_5, parameter_6, ): """ """ print("hello world") return  Now you have three arrows: the first folds function arguments, the second folds all function body and the third folds only the docstring. You can wrap parameters in any way you like, if closing parenthesis remains in place. I personally prefer this style, and it can be auto-formatted with black. Your solution is PEP8-compatible, but ST doesn't like it. It folds to the next line with the same level of indentation (so I'm very surprised that it worked before). This line-wrapping style is especially cute if you use type hinting: every argument appears on its own line together with type, and return type is written on the last line - still separate). This problem also arises in languages with goto construct and labels, which can be indented to the same level as function body - wrapping dies as well.
Question: In Python, a common way to filter a data frame is like this df.loc[(df['field 1'] == 'a') & (df['field 2'] == 'b'), 'field 3']...  When df name is long, or when there are more filter conditions (only two in the above), the above line will be long naturally. Moreover, it is a bit tedious to have type out df name for each condition. In R or SQL, we don't really need to do that. So, my question is if there is a way to shorten the above line in Python. For example, is there a way that I don't have to write down df name in each condition? Thanks.

Answer: you can use the query method as follows which is has an SQL like syntax df.query('field_1 == "a" and field_2 == "b" and field_3 > 3.2 ')  Here are the docs for it https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html