eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to package my project in order to upload it in PyPI. I have the following directory structure: . ├── docs ├── LICENSE ├── pyproject.toml ├── README.md ├── src │ ├── package_name │ │ ├── __init__.py │ │ ├── data.json │ │ ├── __main__.py │ │ ├── utils.py └── tests  My package is in under src named src/package_name. The pyproject.toml has the following lines: [build-system] requires = ["hatchling"] build-backend = "hatchling.build" [project] name = "project_name" version = "0.0.1" authors = [{name = "Foo Bar"}] license = {text = "GPL-3.0-only"} description = "A small example package" readme = "README.md" requires-python = ">=3.10" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: MIT License", "Operating System :: POSIX :: Linux", ] [project.urls] Homepage = "https://example.com"  I want a user to be able to perform the installation as: pip install project_name  but use the code as: >>> from package_name import x  Is there any way to achieve this with Hatch? I have read the Build-instructions but can't find how. Note I have tried the following: python3 -m build python3 -m twine upload --repository testpypi dist/* pip install --upgrade -i https://test.pypi.org/simple/ project_name  The problem is when I type: >>> import package_name Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'package_name'  I searched into path/to/site-packages but only path/to/site-packages/project_name-0.0.1.dist-info is there. Any ideas? Tried to solve my problem based on this Tutorial. The reason I used Hatch is because it was presented in the tutorial. Αn alternative solution (using also pyproject.toml with different backend) is also appreciated. Edit I have updated the directory structure as proposed in the comments.

Answer: I found the solution to my problem by simply switching to setuptools as backend and modifying the pyproject.toml as following: [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [tool.setuptools.packages.find] where = ["src"] [tool.setuptools.dynamic] version = {attr = "package_name.__version__"} [project] name = "package_name" dynamic = ["version"] authors = [{name = "Foo Bar"}] license = {text = "GPL-3.0-only"} description = "A small example package" readme = "README.md" requires-python = ">=3.10" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: POSIX :: Linux", ] [project.urls] Homepage = "https://example.com""  My solution is based on the setuptools Documentation.
Question: I would like to add a rate column base on row before current row, just like a diff() but need do some calculation just like below: import pandas as pd import numpy as np df = pd.DataFrame(np.random.random((5,2)),columns=['v1','v2']) print(df) rate = [] rate.append(0) prev = 0 for index, row in df.iterrows(): if index == 0: prev = row continue r = (prev['v2'] - row['v2'])/(row['v1'] - prev['v1']) prev = row rate.append(r) df['rate'] = rate print(df)  Current logic works what's more better way to do it in python?

Answer: You can use the pandas shift function. df['rate'] = (df['v2'] - df['v2'].shift(1)) / (df['v1'].shift(1) - df['v1']) df['rate'].iloc[0] = 0 df['rate'] = df['rate'].fillna(0)  The shift() method is used to shift the data by one row, which allows you to easily calculate differences between the current row and the previous row using vectorized operations. The .iloc[0] method is then used to set the rate for the first row to 0.
Question: I am trying to create a program for a class that will have a conversation. I want to force the user to input an int for the variables your_age and my_age. I've done a lot of research and this I've tried to code the program the way I believe I'm being told but it's not running if I put 'one' instead of '1'. If the user inputs 'one' I want them to get an error code and have the opportunity to re-enter the variable. Code: name = input('Salutations what is your name?') #user will input their name and it will be saved in the variable name print('That is a nice name ' + name + ' whats your favorite color') #genorate a sentance using the variable name color =input('') #this input is collecting the information from the above sentence print (color + ' is a great color')#make user feel like they picked a good color #your_age = int(input('How old are you?'))#user sets variable age while True: your_age = int(input('How old are you?')) try: your_age = int(your_age) break except ValueError: print('INVALID: you can round up or down but please enter your age using digits') my_age = int(input(' how old am I?')) combine_age = str(your_age + my_age) print ('Our combined age is ') print (combine_age)  I had the code running but I was stress testing and figured out that if I didn't put in a digit the program would crash. I researched on YouTube and in Automating the Boring Stuff and couldn't find a sufficient answer.

Answer: Just convert input into integer inside of Try block and implement error handling for both variables. your_age and my_age. name = input('Salutations what is your name?') #user will input their name and it will be saved in the variable name print('That is a nice name ' + name + ' whats your favorite color') #genorate a sentance using the variable name color =input('Your favorite color?') #this input is collecting the information from the above sentence print (color + ' is a great color')#make user feel like they picked a good color while True: try: your_age = int(input('How old are you?')) break except ValueError: print('INVALID: you can round up or down but please enter your age using digits only') while True: try: my_age = int(input('How old am I?')) break except ValueError: print('INVALID: you can round up or down but please enter your age using digits only') combine_age = str(your_age + my_age) print ('Our combined age is '+combine_age)
Question: I need to create multiple examples for the Response Body, to display it in API documentation http://127.0.0.1:8000/docs. I found an example for the Request Body in the documentation (there is drop-down list of: "A normal example", "An example with converted data", etc.), but I require the same approach for the Response Body. import uvicorn from fastapi import FastAPI from pydantic import BaseModel app = FastAPI() class Item(BaseModel): value: str model_config = { "json_schema_extra": { "examples": [ {"value": "A normal example"}, # working {"value": "An example with converted data"}, # NOT WORKING {"value": "Invalid data ..."}, # NOT WORKING ] } } @app.get(path="/") def get1() -> Item: return Item(value="a") if __name__ == "__main__": uvicorn.run(app) 

Answer: You can achieve this by adding the response parameter in the decorator method. import uvicorn from fastapi import FastAPI from pydantic import BaseModel app = FastAPI() class Item(BaseModel): message: str get1_responses = { 200: { "description": "Success", "content": { "application/json": { "examples": { "Normal": {"value": {"message": "A normal example"}}, "Converted": {"value": {"message": "An example with converted data"}}, } } }, }, 400 : { "description": "Bad request", "content": { "application/json": { "example": {"message": "Invalid data ..."} } }, } } @app.get(path="/", responses=get1_responses) def get1() -> Item: return Item(message="a") if __name__ == "__main__": uvicorn.run(app)  The swagger UI
Question: I'm creating a data frame and want to drop entries in it that are not relevant. I'm looking to drop the values that are not numbers. I have created the data frame using the following code (credit): import pandas as pd import os os.chdir('/pathdirectory/files') csv_files = [f for f in os.listdir() if f.endswith('.csv')] dfs = [] for csv in csv_files: df = pd.read_csv(csv, header=None) df = df.T df.columns = ['DC energy', 'AC energy', 'Capacity factor', 'Inverter Loss'] dfs.append(df) final_df = pd.concat(dfs, ignore_index=True) final_df  And it returns this data frame. Obviously I want to remove the wording from the data frame but I am struggling with doing this. Any help is greatly appreciated.

Answer: You should set the first columns of the CSVs as index: pd.read_csv(csv, header=None, index_col=0)  Alternatively: cols = ['DC energy', 'AC energy', 'Capacity factor', 'Inverter Loss'] final_df = pd.concat([pd.read_csv(csv, header=None, index_col=0) for csv in csv_files], axis=1, ignore_index=True).T.set_axis(cols)  Note that this assumes that all files have the same order of columns. You could also keep the default name: final_df = pd.concat([pd.read_csv(csv, header=None, index_col=0) for csv in csv_files], axis=1, ignore_index=True).T
Question: Currently strugging with nicely printing a array of "O"'s, where the columns and rows should be numbered. So I have a array which is just a nxn-matrix full of the string "O". Now I tried using the following method:  def __repr__(self) : Matrix = self.Spielstand #this is just the mentioned array of length n: Ausgabe = " " for j in range(len(Matrix[0])): Ausgabe += str(j + 1) + " " Ausgabe += "\n" for i in range(len(Matrix)): Ausgabe += str(i + 1) + " " for j in range(len(Matrix[i])): Ausgabe += str(Matrix[i][j]) Ausgabe += " " Ausgabe += "\n" return Ausgabe  Which works perfectly fine printing something like:  1 2 3 4 5 6 7 8 1 O O O O O O O O 2 O O O O O O O O 3 O O O O O O O O 4 O O O O O O O O 5 O O O O O O O O 6 O O O O O O O O 7 O O O O O O O O 8 O O O O O O O O  However when my n gets bigger than 9, things get kind of weird:  1 2 3 4 5 6 7 8 9 10 1 O O O O O O O O O O 2 O O O O O O O O O O 3 O O O O O O O O O O 4 O O O O O O O O O O 5 O O O O O O O O O O 6 O O O O O O O O O O 7 O O O O O O O O O O 8 O O O O O O O O O O 9 O O O O O O O O O O 10 O O O O O O O O O O  Is there a simple and nice way (preferably without using external libraries) to print the array nicely even if my n is double or even triple digit? I think using f-Strings and something going from center should work, but I have no clue how to implement it. Thanks for your help!

Answer: You can calculate margin size by counting the length of the number converted to string. Margin = len(str(len(Matrix)))  Martix -> (array of 12 "O") len(Martix) -> 12 str(12) -> "12" len("12") -> 2  I have created demo, with a solution to your problem: def generateMatrix(n): return [["O"] * n] * n def draw(n): Matrix = generateMatrix(n) Margin = len(str(len(Matrix))) Ausgabe = " " * (Margin + 1) for j in range(len(Matrix[0])): Ausgabe += str(j + 1) + " " * (Margin - len(str(j+1)) + 1) Ausgabe += "\n" for i in range(len(Matrix)): Ausgabe += str(i + 1) + " " * (Margin - len(str(i+1)) + 1) for j in range(len(Matrix[i])): Ausgabe += str(Matrix[i][j]) Ausgabe += " " * Margin Ausgabe += "\n" return Ausgabe print(draw(12))   1 2 3 4 5 6 7 8 9 10 11 12 1 O O O O O O O O O O O O 2 O O O O O O O O O O O O 3 O O O O O O O O O O O O 4 O O O O O O O O O O O O 5 O O O O O O O O O O O O 6 O O O O O O O O O O O O 7 O O O O O O O O O O O O 8 O O O O O O O O O O O O 9 O O O O O O O O O O O O 10 O O O O O O O O O O O O 11 O O O O O O O O O O O O 12 O O O O O O O O O O O O
Question: I've two dates in YYYYmm format start : 202307 end : 202612  want to split them in interval wise, based on provided interval for example split_months ('202307,'202405',5), will give me ((202307,202311), (202312,202404), (202405,202405))  tried with below code, bot stuck in the logic def split_months(start, end, intv): from datetime import datetime periodList = [] periodRangeSplitList = [] start = datetime.strptime(start ,"%Y%m") end = datetime.strptime(end ,"%Y%m") mthDiff = relativedelta.relativedelta(end, start) if mthDiff == 0: periodRangeSplitList.append((start,start)) return periodRangeSplitList diff = mthDiff / intv print(diff) for i in range(intv): periodList.append ((start + diff * i).strftime("%Y%m")) periodList.append(end.strftime("%Y%m")) print(periodList)  I tried, above code, but not working, can anyone share any suggestions ? Thanks

Answer: What about using a simple while loop? def split_months(start, end, intv): from datetime import datetime from dateutil import relativedelta periodList = [] start = datetime.strptime(start, '%Y%m') end = datetime.strptime(end, '%Y%m') step = relativedelta.relativedelta(months=intv-1) start_ = start while (end_:=start_+step) <= end: # add new period periodList.append((start_, end_)) # update next start start_ = end_ + relativedelta.relativedelta(months=1) # convert output to string return [(s.strftime('%Y%m'), e.strftime('%Y%m')) for s, e in periodList] split_months('202307', '202612', 5)  Output: [('202307', '202311'), ('202312', '202404'), ('202405', '202409'), ('202410', '202502'), ('202503', '202507'), ('202508', '202512'), ('202601', '202605'), ('202606', '202610')]  If you want to add the last interval up to the end, change the while condition and add an extra line (periodList.append((start_, end))): def split_months(start, end, intv): from datetime import datetime from dateutil import relativedelta periodList = [] start = datetime.strptime(start, '%Y%m') end = datetime.strptime(end ,'%Y%m') step=relativedelta.relativedelta(months=intv-1) start_ = start while (end_:=start_+step) < end: periodList.append((start_, end_)) start_ = end_ + relativedelta.relativedelta(months=1) periodList.append((start_, end)) return [(s.strftime('%Y%m'), e.strftime('%Y%m')) for s,e in periodList]  Example: # split_months('202307', '202612', 5) [('202307', '202311'), ('202312', '202404'), ('202405', '202409'), ('202410', '202502'), ('202503', '202507'), ('202508', '202512'), ('202601', '202605'), ('202606', '202610'), ('202611', '202612')] # split_months('202307', '202610', 5) [('202307', '202311'), ('202312', '202404'), ('202405', '202409'), ('202410', '202502'), ('202503', '202507'), ('202508', '202512'), ('202601', '202605'), ('202606', '202610')] # split_months('202307','202405',5) [('202307', '202311'), ('202312', '202404'), ('202405', '202405')]
Question: I have this character '³' in my dataset that I'm processing on top of. Generic Idea is to detect if a character is an integer, convert it into an integer and process on top of it. >>> x = '³' >>> x.isdigit() # Returns True True  Python detects this character as a digit. But raises the following error when I try to convert it >>> int(x) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid literal for int() with base 10: '³'  I would like it if such characters could also be converted to integer, to ease my further processing Not sure if this helps, but here is my locale info >>> import locale >>> locale.getdefaultlocale() ('en_US', 'UTF-8') 

Answer: You can use unicodedata and NFKC to convert it here is a detailed code with some error handling import unicodedata x = '³' try: regular_digit = unicodedata.normalize('NFKC', x) integer_value = int(regular_digit) print(integer_value) except ValueError: print(f"'{x}' is not a convertible superscript digit.")
Question: I want to groupby one column (for example, 'country'). Each row has an associated 'start_date' and 'end_date'. For every row in the groupby, I want to increment the counter if the 'start_date' in the current row occurs after the most recent 'end_date' in the prior rows (and not increment otherwise). I want the same logic to apply to the trailing mean. I have sorted by country and start_date. For example, I have a dataframe that can be generated with the following code: import pandas as pd # create df data = {'country': ['arg', 'arg', 'arg', 'arg', 'arg', 'usa', 'usa', 'usa'], 'start_date': ['2020-01-01', '2020-01-01', '2020-05-01', '2021-05-01', '2021-07-01', '2020-03-01', '2020-05-01', '2020-09-01'], 'end_date': ['2020-10-01', '2020-09-01', '2021-01-01', '2021-06-01', '2021-12-01', '2020-10-01', '2020-08-01', '2021-05-01'], 'value': [250, 300, 150, 170, 200, 150, 100, 120]} # Create DataFrame df = pd.DataFrame(data)  And the desired result (with the new columns trailing_mean and count) would be:     country start_date end_date value trailing_mean counter     arg 2020-01-01 2020-10-01 250 NA 0   arg 2020-01-01 2020-09-01 300 NA 0   arg 2020-05-01 2021-01-01 150 NA 0   arg 2021-05-01 2021-06-01 170 233.33 3   arg 2021-07-01 2021-12-01 200 217.5 4   usa 2020-03-01 2020-10-01 150 NA 0   usa 2020-05-01 2020-08-01 100 NA 0   usa 2020-09-01 2021-05-01 120 100 1     Notice how the trailing_mean is NA until there are records that have a start_date that occurs AFTER the end_date. On every record, the trailing mean only takes into account past records that have already completed (their end_date happens before the current record's start_date). This is the same logic for the counter. It is 0 and then it increments. It jumps from 0 to 3 because all three prior rows ended before that row has started I have tried to groupby country and iterate through the rows. But I am having trouble accounting for the differences in end_dates. You can't just look back at the prior row you have to look at all prior records because the end_dates are not sequential

Answer: IIUC, you can apply a custom function to generate your counts and trailing means for each group: def count_and_avg(df): mask = [df['end_date'] < start for start in df['start_date']] df = df.assign(count=[sum(m) for m in mask], trailing_mean=[df[m]['value'].sum() / sum(m) if sum(m) else 0 for m in mask] ) return df out = df.groupby('country').apply(count_and_avg).reset_index(drop=True)  Output for your sample data:  country start_date end_date value count trailing_mean 0 arg 2020-01-01 2020-10-01 250 0 0.000000 1 arg 2020-01-01 2020-09-01 300 0 0.000000 2 arg 2020-05-01 2021-01-01 150 0 0.000000 3 arg 2021-05-01 2021-06-01 170 3 233.333333 4 arg 2021-07-01 2021-12-01 200 4 217.500000 5 usa 2020-03-01 2020-10-01 150 0 0.000000 6 usa 2020-05-01 2020-08-01 100 0 0.000000 7 usa 2020-09-01 2021-05-01 120 1 100.000000
Question: How do you access the raw message data of an email when using imap-tools? Specifically so it can then be loaded into the email.message_from_bytes() function for forwarding? from imap_tools import MailBox, AND with MailBox('imap.gmail.com').login('asdf@gmail.com', '123456', 'INBOX') as mailbox: # get unseen emails from INBOX folder for msg in mailbox.fetch(AND(seen=False), mark_seen=False): pass # get the raw data from msg 

Answer: According to the source it looks like the msg.obj property contains the value after message_from_bytes has been run. class MailMessage: """The email message""" def __init__(self, fetch_data: list): raw_message_data, raw_uid_data, raw_flag_data = self._get_message_data_parts(fetch_data) self._raw_uid_data = raw_uid_data self._raw_flag_data = raw_flag_data self.obj = email.message_from_bytes(raw_message_data)  So msg.obj can be used directly.
Question: I have searched on stackoverflow for people facing similar issues and this topic Replicating MATLAB's `randperm` in NumPy is the most similar. However, although it is possible to recreate the behavior of randperm function from Matlab in Python using numpy random permutation, the numbers generated are not the same, even though I choose the same seed generator for both languages. I am a bit confused since my tests were relevant for other random functions between Matlab and Python. Here is what I have tried: Matlab rng(42); randperm(15)  which returns ans = 11 7 6 5 15 14 1 4 9 10 3 13 8 2 12  Python np.random.seed(42) print(np.random.permutation(range(1,16)))  which returns [10 12 1 14 6 9 3 2 15 5 8 11 13 4 7]  How can I change my Python code so it can reproduce the same order of random numbers than Matlab ?

Answer: It seems that Matlab and Numpy use the same random number generators by default, and the discrepancy is caused by the inner workings of randperm being different in the two languages. In old Matlab versions, randperm worked by generating a random array and outputting the indices that would make the array sorted (using the second output of sort). In more modern Matlab versions (I'm using R2017b), randperm is a built-in function, so the source code cannot be seen, but it seems to use the same method: >> rng('default') >> rng(42) >> randperm(15) ans = 11 7 6 5 15 14 1 4 9 10 3 13 8 2 12 >> rng(42) >> [~, ind] = sort(rand(1,15)) ind = 11 7 6 5 15 14 1 4 9 10 3 13 8 2 12  So, if the random number generators are actually the same in the two languages, which seems to be the case, you can replicate that behaviour in Numpy by defining your own version of randperm using argsort: >>> import numpy as np np.random.seed(42) ind = np.argsort(np.random.random((1,16)))+1 print(ind) [[11 7 6 5 15 16 14 1 4 9 10 3 13 8 2 12]]  Note, however, that relying on the random number generators being the same in the two languages is risky, and probably version-dependent.
Question: I have a dataframe grouped by issue_ids where i want to apply a custom function. The grouped dataframe looks as follows import pandas as pd import numpy as np sub_test=pd.DataFrame(columns=['issue_id','step','conversion_rate'],data=[['01-abc-234',0,0.45],['01-abc-234',1,0.35],['01-abc-234',2,0.15],['01-abc-234',3,1],['02-abc-234',0,0.05],['02-abc-234',1,0.15],['02-abc-234',2,0.65],['02-abc-234',3,1]]) sub_test.info()  I want to group by issue id and apply the following function for each grouped dataframe def calculate_conversion_step(row, df): if row == 0: return np.prod(df.loc[df['step'].isin([1, 2]), 'conversion_rate']) elif row == 1: return np.prod(df.loc[df['step'] == 2, 'conversion_rate']) else: return 1  Basically, what i am doing here is iterating through each dataframe for individual issue ids and applying the aforementioned function to each row of the filtered dataframe. I used .apply() but my dataframe is too large to function well with apply. final=pd.DataFrame() for issue_id in sub_test['issue_id'].unique(): int_df = sub_test[sub_test['issue_id'] == issue_id] # Apply the 'calculate_conversion_step' function to calculate 'conversion_step' for each issue int_df['conversion_step'] = int_df['step'].apply(lambda x: calculate_conversion_step(x, int_df)) # Concatenate the results for each issue final = pd.concat([final, int_df])  Is there anyway i can make it faster? this is my expected output

Answer: import numpy as np cond0, cond1, cond2 = sub_test['step'].eq(0), sub_test['step'].eq(1), sub_test['step'].eq(2) s1 = sub_test.groupby('issue_id')['conversion_rate'].transform(lambda x: x.where(cond1 | cond2).prod()) s2 = sub_test.groupby('issue_id')['conversion_rate'].transform(lambda x: x.where(cond2).sum()) sub_test['conversion_step'] = np.select([cond0, cond1], [s1, s2], 1)  output: issue_id step conversion_rate conversion_step 0 01-abc-234 0 0.45 0.0525 1 01-abc-234 1 0.35 0.1500 2 01-abc-234 2 0.15 1.0000 3 01-abc-234 3 1.00 1.0000 4 02-abc-234 0 0.05 0.0975 5 02-abc-234 1 0.15 0.6500 6 02-abc-234 2 0.65 1.0000 7 02-abc-234 3 1.00 1.0000