eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have two sub functions that feed into one main functions as defined below: Sub function 1: def func(x=1, y=2): z = x + y return z  Sub function 2: def func2(a=3, b=4): c = a - b return c  Main function: def finalFunc(lemons, input1, input2, input3, input4): result = func(input1, input2) + func2(input3, input4) + lemons return result  How do I call my main function but if the values for the sub functions aren't specified, they're treated as default? Similarly, if they are specified, then use them instead, e.g. >>> finalFunc(lemons=1) 3  or >>> finalFunc(lemons=1, input1=4, input4=6) 4  I don't want to specify the default values in my main function, as the sub functions are always changing. I want to keep the default values set at whatever the sub functions contain.

Answer: My recommendation: def func(x=None, y=None): if x is None: x = 1 if y is None: y = 2 z = x + y return z def func2(a=None, b=None): if a is None: a = 3 if b is None: b = 4 c = a-b return c def finalFunc(lemons, input1=None, input2=None, input3=None, input4=None): result = func(input1, input2) + func2(input3, input4) + lemons return result  It is low-tech, but it is DRY and creates no unnecessary coupling between the func, func2 and finalFunc. The simple if arg is None: idiom is familiar to most Python developers, and doesn't come with any weird surprises. I've attempted more "clever" approaches in the past, such as sharing the defaults in module level globals, or digging values out from func.__defaults__ to re-use, but the downsides always seem to outweigh the advantages.
Question: I am trying to create 2 new columns in output on checking condition but not sure how to do that. sample df: so_df = pl.DataFrame({"low_limit": [1, 3, 0], "high_limit": [3, 4, 2], "value": [0, 5, 1]})  low_limit high_limit value i64 i64 i64 1 3 0 3 4 5 0 2 1  Code for single column creation that works: so_df.with_columns(pl.when(pl.col('value') > pl.col('high_limit')) .then(pl.lit("High")) .when((pl.col('value') < pl.col('low_limit'))) .then(pl.lit("Low")) .otherwise(pl.lit("Within Range")).alias('Flag') )  output low_limit high_limit value Flag i64 i64 i64 str 1 3 0 "Low" 3 4 5 "High" 0 2 1 "Within Range"  Issue/Doubt: Creating 2 columns that doesn't work so_df.with_columns(pl.when(pl.col('value') > pl.col('high_limit')) .then(Flag = pl.lit("High"), Normality = pl.lit("Abnormal")) .when((pl.col('value') < pl.col('low_limit'))) .then(Flag = pl.lit("Low"), Normality = pl.lit("Abnormal")) .otherwise(Flag = pl.lit("Within Range"), Normality = pl.lit("Normal")) )  Desired output: low_limit high_limit value Flag Normality i64 i64 i64 str str 1 3 0 "Low" "Abnormal" 3 4 5 "High" "Abnormal" 0 2 1 "Within Range" "Normal"  I know I can do another with_Columns and using when-then again but that will take double the computation. So how can I create 2 new columns in 1 go ? something like: if (condition): Flag = '', Normality = '' 

Answer: You can select into a pl.struct and then extract multiple values out using .struct.field(...): df = so_df.with_columns( pl.when(pl.col("value") > pl.col("high_limit")) .then(pl.struct(Flag=pl.lit("High"), Normality=pl.lit("Abnormal"))) .when(pl.col("value") < pl.col("low_limit")) .then(pl.struct(Flag=pl.lit("Low"), Normality=pl.lit("Abnormal"))) .otherwise(pl.struct(Flag=pl.lit("Within Range"), Normality=pl.lit("Normal"))) .struct.field("Flag", "Normality") )  Output: shape: (3, 5) ┌───────────┬────────────┬───────┬──────────────┬───────────┐ │ low_limit ┆ high_limit ┆ value ┆ Flag ┆ Normality │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ str ┆ str │ ╞═══════════╪════════════╪═══════╪══════════════╪═══════════╡ │ 1 ┆ 3 ┆ 0 ┆ Low ┆ Abnormal │ │ 3 ┆ 4 ┆ 5 ┆ High ┆ Abnormal │ │ 0 ┆ 2 ┆ 1 ┆ Within Range ┆ Normal │ └───────────┴────────────┴───────┴──────────────┴───────────┘
Question: I want to read in a polars dataframe from a json string containing dates in the standard iso-format "yyyy-mm-dd". When I try to read the string in and set the dtype of the date column witheither schema or schema_override this results in only NULL values. MRE from datetime import datetime, timedelta from io import StringIO import polars as pl # Generate a list of dates start_date = datetime.today() dates = [start_date + timedelta(days=i) for i in range(100)] date_strings = [date.strftime("%Y-%m-%d") for date in dates] # Create a Polars DataFrame df = pl.DataFrame({"dates": date_strings}) df_reread = pl.read_json( StringIO(df.write_json()), schema_overrides={"dates": pl.Date}, )  output of print(df_reread) Error shape: (100, 1) ┌───────┐ │ dates │ │ --- │ │ date │ ╞═══════╡ │ null │ │ null │ │ null │ │ null │ │ null │ │ … │ │ null │ │ null │ │ null │ │ null │ │ null │ └───────┘  Question Is there anyway to correctly read in the Date dtype from a json string?

Answer: After having a bit of a play around, it looks like unfortunately dates being read from a JSON file have a bit of a quirk. It seems to me that currently they must be written in days since the unix epoch (which is how Polars internally represents dates) for things to work as you expect. I have raised this feature request on their github to hopefully get that improved. In the mean time, df = ( pl.DataFrame({"dates": "2024-01-01"}) # add this line below .select(pl.col("dates").cast(pl.Date).dt.epoch("d")) ) df_reread = pl.read_json( df.write_json().encode(), schema_overrides={"dates": pl.Date}, ) print(df_reread) # shape: (1, 1) # ┌────────────┐ # │ dates │ # │ --- │ # │ date │ # ╞════════════╡ # │ 2024-01-01 │ # └────────────┘  or do as you say with df_reread.with_columns(pl.col("dates").cast(pl.Date)
Question: I am working with a QTableView, and I'd like for the row to change its height to accommodate the content of the selected index. The function resizeRowToContents is not really what I am looking for. If I click on a cell [A] that doesn't need to change its height to display everything but a cell [B] in the row needs to, the row will increase its height to accommodate [B]. Using sizeHintForRow would result in the same behaviour as using resizeRowToContents The function sizeHintForIndex doesn't return the correct value for some reason. In the C++ source of Qt there is a private function heightHintForIndex that would be exactly what I need (because it's used by resizeRowToContents). Is there a function that I am missing, or does anyone know a way to do it? I am using my own QStyledItemDelegate for those cells.

Answer: Basically, I wrote the function heightHintForIndex in Python. def on_selection_changed(self, selected: QModelIndex , deselected: QModelIndex): self.setRowHeight(deselected.row(), 1) editor = self.indexWidget(selected) hint = 0 if (editor is not None) and (self.isPersistentEditorOpen(selected)): hint = max(hint, editor.sizeHint().height()) min_ = editor.minimumSize().height() max_ = editor.maximumSize().height() hint = max(min_, min(max_, hint)) option = QStyleOptionViewItem() self.initViewItemOption(option) option.rect.setY(self.rowViewportPosition(selected.row())) height = self.rowHeight(selected.row()) if height == 0: height = 1 option.rect.setHeight(self.rowHeight(selected.row())) option.rect.setX(self.columnViewportPosition(selected.column())) option.rect.setWidth(self.columnWidth(selected.column())) if self.showGrid(): option.rect.setWidth(option.rect.width() - 1) new_height = max(hint, self.itemDelegateForIndex(selected).sizeHint(option, selected).height()) self.setRowHeight(selected.row(), new_height)
Question: I was wondering whether following was possible with TypedDict and Unpack, inspired by PEP 692... Regular way of using TypedDict would be: class Config(TypedDict): a: str b: int def inference(name, **config: Unpack[Config]): c = config['a']+str(config["b"]) config: Config = {"a": "1", "b": 2} inference("example", **config) # or I could get argument hinting for each one from Config  However, I would be really interested if I could somehow unpack those keys to make it behave like they were directly introduced as variables into function signature: def inference(name, **?: Unpack[Config]): c = a+str(b)  so to mimic this explicit approach: def inference(name, a: str, b: int): c = a + str(b)  My motivation is I want to still leave code uncluttered with field or key access (verbose_name.another_verbose_name or verbose_name["another_verbose_name"], where latter has no type hinting in VSC). Also, this way I could define TypedDict for Config once, and when needing some new parameter for my inference, i could just straight add it to Config definition, without changing inference signature. Currently my workaround is explicit definition of arguments in inference signature and still invoking this with inference(name, **config). I still prefer this approach more than Dataclass, since I can skip asdict(dataclass_config) doing it so, which feels very unnecessary, especially when loading config from yaml or similar... Maybe my approach is all wrong and not suiting to the best practices... Also would love your input on this. Im just starting using more advanced Python topics... I can imagine that adding this functionality via PEP would require more than just typing work, so I would like to know if there is some common established solution.

Answer: No, there is no python syntax that will allow you to "unpack the dictionary to make it behave like they were directly introduced as variables into function signature". You could technically do it at runtime with something like this: def inference(name, **config: Unpack[Config]): vars().update(config) dosomething(a, b)  but that's very hacky, and the typechecker would not recognize subsequent uses of a or b (even though they will work at runtime). The typechecker will only recognize a variable if it is declared or passed as argument.  I understand you want to declutter your code, but as @Barmar's comment suggests, skipping variable declarations may not be the best way to do that if you are actually using those variables in your function. ** and Unpack are mostly useful if those arguments are passed down to another function
Question: There is a website like bonbast.com and I'm trying to get values but I'm just confused about how to do it. Values should be something like the output of "US Dollar" and "Euro". My code: import requests from bs4 import BeautifulSoup r = requests.get("https://bonbast.com/") soup = BeautifulSoup(r.content, "html.parser") usd1 = soup.find(id="usd1") print(usd1) same_val = soup.find_all(class_="same_val") print(same_val)  the output is blank and not showing the number of the currency. HTML page https://i.imgur.com/NrTaUY8.png

Answer: From the Reference of this Repo - https://github.com/drstreet/Currency-Exchange-Rates-Scraper/blob/master/scrapper.py Focusing only on the necessary parts to retrieve exchange rates USD and Euro. import requests from bs4 import BeautifulSoup import re def get_currency_data(): url = 'https://www.bonbast.com' session = requests.Session() headers = { 'User-Agent': 'Mozilla/5.0', 'Referer': url, 'Accept': 'application/json, text/javascript, */*; q=0.01', } response = session.get(url, headers=headers) soup = BeautifulSoup(response.text, 'html.parser') # Find the script containing the 'param' value scripts = soup.find_all('script') param = None for script in scripts: if '$.post' in script.text: match = re.search(r'param:\s*"([^"]+)"', script.text) if match: param = match.group(1) break if param: post_url = url + '/json' post_data = {'param': param} # Make POST request to fetch the JSON data response = session.post(post_url, headers=headers, data=post_data) data = response.json() # Extract relevant currency data (USD and Euro) currencies = {} for key, value in data.items(): if key == 'usd1': # USD sell price currencies['USD_sell'] = value elif key == 'usd2': # USD buy price currencies['USD_buy'] = value elif key == 'eur1': # Euro sell price currencies['EUR_sell'] = value elif key == 'eur2': # Euro buy price currencies['EUR_buy'] = value return currencies else: return {'error': 'Param not found'} # Execute the function and print the result result = get_currency_data() print(result)  output {'EUR_sell': '68500', 'EUR_buy': '68400', 'USD_sell': '62800', 'USD_buy': '62700'}
Question: I want to merge some 'n' tables in python. Each table has 2 columns in it. Currently, I'm trying with these 3 tables (table12, table13, table23).  Context: I have certain image files, each image has some objects inside it which are labelled as 'A', 'B', 'C', etc. If I'm comparing 2 images, I get an equivalence table which tells which object in image1 is similar to which object of image2. (for example. first row of table12 tells that object 'A' of image1 and object 'P' of image2 are same) There are chances that an object in one image doesn't have an equivalent object in the other image. I want to create a single table, making use of all these tables. Number of columns in the final merged table = number of images. Means, under each column, its objects will be mentioned and in the same row, we can find which objects are similar. I tried pandas merge (merging 2 tables, then merging third one on the merged of first and second) but it's giving result like: . If you want to test, here is the code: import pandas as pd import numpy as np table12 = pd.DataFrame({ 'img1': ['A', 'B', 'C', 'D', np.NaN, np.NaN], 'img2': ['P', np.NaN, 'Q', np.NaN, 'R', 'S'] }) table13 = pd.DataFrame({ 'img1': ['A', 'B', 'C', 'D', np.NaN, np.NaN], 'img3': [np.NaN, 'X', np.NaN, 'Y', 'Z', 'W'] }) table23 = pd.DataFrame({ 'img2': ['P', np.NaN, 'Q', np.NaN, 'R', np.NaN, 'S'], 'img3': [np.NaN, 'X', np.NaN, 'Y', 'Z', 'W', np.NaN] }) merged_12_13 = pd.merge(table12, table13, on='img1', how='outer') final_output = pd.merge(merged_12_13, table23, on=['img2', 'img3'], how='outer') 

Answer: Your logic in unclear, but assuming that the tables are to be kept in their original order and are top aligned, you should probably concat the individual columns: pd.concat([table12[['img1']], table23[['img2']], table13[['img3']]], axis=1)  Or, taking all columns as input and removing the duplicates: pd.concat([table12, table13, table23], axis=1).T.groupby(level=0).first().T  Output:  img1 img2 img3 0 A P NaN 1 B NaN X 2 C Q NaN 3 D NaN Y 4 NaN R Z 5 NaN S W 6 NaN NaN NaN
Question: import pandas df1 = pandas.DataFrame( { 'code': ['001', '001'], 'name': ['test1', 'test1'], 'date': ['2024-01-01', '2024-01-01'], 'value1': [1, 2], 'value2': [1, 2], 'sum': [2, 4] } ) df2 = pandas.DataFrame( { 'code': ['001', '001', '001', '002'], 'name': ['test1', 'test1', 'test1', 'test2'], 'date': ['2024-01-01', '2024-01-01', '2024-01-01', '2024-02-01'], 'value1': [1, 2, 3, 4], 'value2': [1, 2, 3, 4], 'sum': [2, 4, 6, 8] } ) result = pandas.merge(df1, df2, on=['code', 'name', 'date'], how='outer', indicator=True) print(result[['code', 'name', 'date', 'sum_x', 'sum_y', '_merge']])  When the above data is printed, it appears as shown below.  code name date sum_x sum_y _merge 0 001 test1 2024-01-01 2.0 2 both 1 001 test1 2024-01-01 2.0 4 both 2 001 test1 2024-01-01 2.0 6 both 3 001 test1 2024-01-01 4.0 2 both 4 001 test1 2024-01-01 4.0 4 both 5 001 test1 2024-01-01 4.0 6 both 6 002 test2 2024-02-01 NaN 8 right_only  Two data have been added to df2. I looked into it through many methods and searches, but the closest thing I tried was to use merge to create unique values ​​and compare them. But I couldn't create the output value I wanted. My goal is to compare 1:1 what was created and what was lost in df1 and df2. I kept trying to remove row values ​​that were unnecessarily created during merge, but it didn't work. The material I want to create is as below.  code name date sum_x sum_y _merge 0 001 test1 2024-01-01 2 2 both 4 001 test1 2024-01-01 4 4 both 5 001 test1 2024-01-01 NaN 6 right_only 6 002 test2 2024-02-01 NaN 8 right_only  I tried to delete unnecessary rows using deduplication and qurey. I also tried creating eigenvalues ​​and comparing them, but I couldn't create what I wanted. I tried looking up information using Stack Overflow and gpt.

Answer: It looks like you should include the sum columns as keys, optionally renaming them before the merge: result = pandas.merge(df1.rename(columns={'sum': 'sum_x'}) .drop(columns=['value1', 'value2']), df2.rename(columns={'sum': 'sum_y'}) .drop(columns=['value1', 'value2']), left_on=['code', 'name', 'date', 'sum_x'], right_on=['code', 'name', 'date', 'sum_y'], how='outer', indicator=True)  Output:  code name date sum_x sum_y _merge 0 001 test1 2024-01-01 2.0 2 both 1 001 test1 2024-01-01 4.0 4 both 2 001 test1 2024-01-01 NaN 6 right_only 3 002 test2 2024-02-01 NaN 8 right_only
Question: This is the problem: I'm trying to clean a text from all the special characters but want to keep the compound words like 'self-restraint' or 'e-mail' united as they are, with the middle dash. The problem is that this hyphen is recognized as a special character. I have python 3.10 I used several regex patterns to do it, but none worked fine. import re text = "This is -a -! -sample- ? e-mail text?classification? example- ! ?\} {]}[¿ with !self-restraint( - like @, #, and $." # cleaned_text = re.sub(r'(?<!\b[a-zA-Z]+)-|[^\w\s-]', ' ', text) cleaned_text = re.sub(r'\W+(?![a-zA-Z]+-[a-zA-Z]+)', ' ', text) # cleaned_text = re.sub(r'\W+[^\w]', ' ', text) # cleaned_text = re.sub(r'(?!\b[a-zA-Z]+-[a-zA-Z]+\b)', ' ', text) # cleaned_text = re.sub(r'(?!\b[a-zA-Z]+-[a-zA-Z]+\b)\W+', ' ', text) # cleaned_text = re.sub(r'\b(?![a-zA-Z]+-[a-zA-Z]+\b)\w*[^a-zA-Z\s-]+\w*\b', ' ', text) # cleaned_text = re.sub(r'\b[a-zA-Z]+-[a-zA-Z]+\b', ' ', text) # cleaned_text = re.sub(r'(?<!\w)-(?!\w)|[^\w\s-]|[^\w\s-]$', ' ', text) print(cleaned_text)  Actual output: This is a sample e mail text classification example with !self restraint like and Expected output: This is a sample e-mail text classification example with self-restraint like and

Answer: You could try something like this: import re text = "This is -a -! -sample- ? e-mail text?classification? example- ! ?\} {]}[¿ with !self-restraint( - like @, #, and $." cleaned_text = re.sub(r'\s+', ' ', re.sub(r'(?<!\S)[-]|[-](?!\S)|[^\w\s-]', ' ', text)).strip() print(cleaned_text)  Output This is a sample e-mail text classification example with self-restraint like and  Without more examples I can't definitively say whether or not it will succeed on other strings, although this should give you an idea of one possible way to approach this issue.
Question: How can I convert Python object - a list, into an ast.List object, so I can appent it to the main AST tree as a node  huge_list [ 1, "ABC", 4.5 ] object = ast.Assign([ast.Name(huge_list_name, ast.Store())], (ast.List(huge_list, ast.Load()))) object.lineno = None result = ast.unparse(object) print(result) tree.body.append(object)  but it fails while parsing each field from the sample list.

Answer: Assuming your list is made up of simple objects like strings and numbers, you can parse its representation back into an ast.Module, then dig the ast.List out of the module body: >>> huge_list = [1, "ABC", 4.5] >>> mod = ast.parse(repr(huge_list)) >>> [expr] = mod.body >>> expr.value <ast.List at 0x7fffed231190>
Question: I want to concat three list columns in a pl.LazyFrame. However the Lists often contain NULL values. Resulting in NULL for pl.concat_list MRE import polars as pl # Create the data with some NULLs data = { "a": [["apple", "banana"], None, ["cherry"]], "b": [None, ["dog", "elephant"], ["fish"]], "c": [["grape"], ["honeydew"], None], } # Create a LazyFrame lazy_df = pl.LazyFrame(data) list_cols = ["a", "b", "c"] print(lazy_df.with_columns(pl.concat_list(pl.col(list_cols)).alias("merge")).collect())  ┌─────────────────────┬─────────────────────┬──────────────┬───────────┐ │ a ┆ b ┆ c ┆ merge │ │ --- ┆ --- ┆ --- ┆ --- │ │ list[str] ┆ list[str] ┆ list[str] ┆ list[str] │ ╞═════════════════════╪═════════════════════╪══════════════╪═══════════╡ │ ["apple", "banana"] ┆ null ┆ ["grape"] ┆ null │ │ null ┆ ["dog", "elephant"] ┆ ["honeydew"] ┆ null │ │ ["cherry"] ┆ ["fish"] ┆ null ┆ null │ └─────────────────────┴─────────────────────┴──────────────┴───────────┘  Question How can I concat the lists even when some values are NULL? Tried solutions I've tried to fill the null values via expr.fill_null("") or expr.fill_null(pl.List("")) or expr.fill_null(pl.List([])) but could not get it to run through. How do I fill an empty list instead of NULL in cols of type pl.List[str]. And is there a better way to concat the three list columns?

Answer: You can use pl.Expr.fill_null() as follows: lazy_df.with_columns( pl.concat_list( pl.col(list_cols).fill_null([]) ).alias("merge") )
Question: I'm trying to import and use a module originally made as a standalone script. Preferably without altering it, to keep tool commonality with the authors. One function I'm using prompts for a y/n response, to which I want to always answer "y": input("\nContinue? (y/n): ").lower()  What's the best way to automate this answer?

Answer: Try replacing sys.stdin (the place where input reads it's data from) with something that always outputs y. import io import sys one_hunderd_yesses = io.StringIO('y\n'*100) original_stdin = sys.stdin sys.stdin = one_hunderd_yesses # Run the external module print(input("\nContinue? (y/n): ")) # Outputs "y" # When you are done, reset stdin sys.stdin = original_stdin  Try this on ATO
Question: I've been playing around with the start_date_time and the end_date_time like this: # Create GraphServiceClient graph_client = GraphServiceClient(credential, scopes) # Define date range for fetching events start_date = datetime.now(pytz.utc) end_date = start_date + timedelta(days=365) # Format dates correctly for the Graph API in ISO 8601 format start_date_str = start_date.strftime("%Y-%m-%dT%H:%M:%S%z") end_date_str = end_date.strftime("%Y-%m-%dT%H:%M:%S%z") # Insert colon in timezone offset start_date_str = start_date_str[:-2] + ":" + start_date_str[-2:] end_date_str = end_date_str[:-2] + ":" + end_date_str[-2:] print(f"Fetching events from {start_date_str} to {end_date_str}") # Set up the request configuration with query parameters as a dictionary query_params = CalendarViewRequestBuilder.CalendarViewRequestBuilderGetQueryParameters( start_date_time=start_date_str, end_date_time=end_date_str, select=['subject', 'start', 'end'], top=100, orderby=['start/dateTime ASC'], #filter="contains(subject, 'from HR Works')" ) headers = HeadersCollection() headers.add("Prefer", 'outlook.timezone="Europe/Berlin"') request_configuration = RequestConfiguration( query_parameters=query_params, headers=headers ) all_events = [] try: # Initial request events_page = await graph_client.users.by_user_id(user_id).calendars.by_calendar_id(calendar_id).events.get(request_configuration=request_configuration) ...  It says then basically Fetching events from 2024-10-09T12:27:39+00:00 to 2025-10-09T12:27:39+00:00 but checking on them I see lot's of events from 2023? for instance Absence from 2023-10-13T10:00:00.0000000 to 2023-10-13T11:00:00.0000000: Where is my issue? I tried tweaking the timestamps, but nothing worked. Also been diving into the official documentation which did not help: https://learn.microsoft.com/en-us/graph/sdks/create-requests?tabs=python#use-select-to-control-the-properties-returned EDIT: Using this query https://graph.microsoft.com/v1.0/me/calendarView?startDateTime=2023-06-14T00:00:00Z&endDateTime=2024-06-15T00:00:00Z in the GraphExplorer works, - meaning endDateTime and startDateTime parameters

Answer: Seems like you are calling wrong method. Checking your code events_page = await graph_client.users.by_user_id(user_id).calendars.by_calendar_id(calendar_id).events.get(request_configuration=request_configuration)  you are calling the endpoint v1.0/users/{user_id}/calendars/{calendar_id}/events For calendarView you need to call result = await graph_client.users.by_user_id(user_id).calendars.by_calendar_id(calendar_id).calendar_view.get(request_configuration = request_configuration)