eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following code snippet: from typing import TypedDict class Super(TypedDict): foo: int class SubA(Super): bar: int class SubB(Super): zap: int def print_props(inp: Super, key: str): print(inp[key])  When I call the method print_props with either an instance of SubA or SubB it would be valid as they are sub types of Super. But mypy will complain about inp[key] as the key must be literal "foo". Is it possible to give mypy hints so that it is capable of deciding which keys are valid? For example: "When print_props is called with an instance of SubB only "foo" and "zap" are valid." I took a look at generics; I think it is possible to declare a type variable that is restricted to sub types of Super, but is it possible to express the dependency between the concrete type of the type variable (SubA or SubB) and the literal values key should then be restricted to?

Answer: Overloads with Literal could well do it, though I do wonder if a different design would be better. I'm a bit concerned about the increasingly frequent usage of overload and Literal in SO answers. They both suggest a design smell to me @overload def printMyProps(input: SubA, key: Literal["foo", "bar"]) -> None: ... @overload def printMyProps(input: SubB, key: Literal["foo", "zap"]) -> None: ... def printMyProps(input: SubA | SubB, key: Literal["foo", "bar", "zap"]) -> None: print(input[key]) # type: ignore  I've used type: ignore because it's a short function and I can't use isinstance on TypedDict. TBH overload implementations often require type hacks. The API works as intended though
Question: I am trying to update my version of Poetry to 1.2.*, but when running poetry self update I get the error Could not find poetry-1.2.2-linux.sha256sum file... I can't figure out how to try and update Poetry to an earlier version for which hopefully the checksum exists.

Answer: You are trying to update a Poetry that was installed with the get-poetry.py installer. This installer is deprecated for more than a year now. Updating via poetry self update is not possible for these installation. Uninstall Poetry and reinstall with the recommended installer. More information are available at https://python-poetry.org/blog/announcing-poetry-1.2.0/
Question: I'm running Python3 (3.8.10) and am attempting a tutorial with the gym_super_mario_bros (7.3.0) and nes_py libraries. I followed various tutorials code and tried on multiple computers but get an error. I have tried to adjust some of the parameters like adding a 'truncated' variable to the list of values to return. As this is a tutorial level example I'm curious what is wrong. It looks like something with env.step(). Below is the code: from nes_py.wrappers import JoypadSpace from gym_super_mario_bros.actions import SIMPLE_MOVEMENT env = gym_super_mario_bros.make('SuperMarioBros-v0') env = JoypadSpace(env, SIMPLE_MOVEMENT) done = True for step in range(1000): if done: env.reset() state, reward, done, info = env.step(env.action_space.sample()) env.render() env.close()  The error I get is below: /home/d/.local/lib/python3.8/site-packages/gym/envs/registration.py:555: UserWarning: WARN: The environment SuperMarioBros-v0 is out of date. You should consider upgrading to version `v3`. logger.warn( /home/d/.local/lib/python3.8/site-packages/gym/utils/passive_env_checker.py:195: UserWarning: WARN: The result returned by `env.reset()` was not a tuple of the form `(obs, info)`, where `obs` is a observation and `info` is a dictionary containing additional information. Actual type: `<class 'numpy.ndarray'>` logger.warn( /home/d/.local/lib/python3.8/site-packages/gym/utils/passive_env_checker.py:219: DeprecationWarning: WARN: Core environment is written in old step API which returns one bool instead of two. It is recommended to rewrite the environment with new step API. logger.deprecation( Traceback (most recent call last): File "mario.py", line 12, in <module> state, reward, done, info = env.step(env.action_space.sample()) File "/home/d/.local/lib/python3.8/site-packages/nes_py/wrappers/joypad_space.py", line 74, in step return self.env.step(self._action_map[action]) File "/home/d/.local/lib/python3.8/site-packages/gym/wrappers/time_limit.py", line 50, in step observation, reward, terminated, truncated, info = self.env.step(action) ValueError: not enough values to unpack (expected 5, got 4)  Any guidance is appreciated, thank you!

Answer: Move to "...../python3.8/site-packages/gym/wrappers/time_limit.py".And delete all "truncated"
Question: I work in my app with a database. This database stores data with a randomly generated ID of the type UUID v4. Now I'm wondering, how common is it for a big (let's be optimistic :D ) app with many users to have a duplicate ID? The ID is the primary key in the SQL database so it could only crash one API call. Is it a clean practice to check if the UUID exists (and thus catch a possible crash in the backend) or is it redundant as it's very unlikely to happen? Especially considering:  random in python is not that random there are 2¹²² combinations  EDIT: Based on the comments it seems unnecessary to check for duplicates. Thanks!

Answer: This shouldn't be a problem in practice due to the very low probability of collisions. See the Wikipedia article on UUID4 collision  For example, the number of random version-4 UUIDs which need to be generated in order to have a 50% probability of at least one collision is 2.71 quintillion This number is equivalent to generating 1 billion UUIDs per second for about 85 years. A file containing this many UUIDs, at 16 bytes per UUID, would be about 45 exabytes.  NB. UUID1 and UUID2 have a time component which make it impossible to have collisions if the UUIDs are generated at a reasonable enough frequency.
Question: I have created a heatmap using the seaborn and matplotlib package in python, and while it is perfectly suited for my current needs, I really would prefer to have the labels on the x-axis of the heatmap to be placed at the top of the plot, rather than at the bottom (which seems to be its default). So an abridged form of my data looks like this: NP NP1 NP2 NP3 NP4 NP5 identifier A1BG~P04217 -0.094045 0.012229 0.102279 1.319618 0.002383 A2M~P01023 -0.805089 -0.477339 -0.351341 0.089735 -0.473815 AARS1~P49588 0.081827 -0.099849 -0.287426 0.101588 0.136366 ABCB6~Q9NP58 0.109911 0.458039 -0.039325 -0.484872 1.905586 ABCC1~I3L4X2 -0.560155 0.580285 0.012868 0.291303 -0.407900 ABCC4~O15439 0.055264 0.138630 -0.204665 0.191241 0.304999 ABCE1~P61221 -0.510108 -0.059724 -0.233365 0.078956 -0.651327 ABCF1~Q8NE71 -0.348526 -0.135414 -0.390021 -0.190644 -0.276303 ABHD10~Q9NUJ1 0.237959 -2.060834 0.325901 -0.778036 -4.046345 ABHD11~Q8NFV4 0.294587 1.193258 -0.797294 -0.148064 -1.153391  And when I use the following code: import seaborn as sns import matplotlib as plt fig, ax = plt.subplots(figsize=(10,30)) ax = sns.heatmap(df_example, annot=True, xticklabels=True)  I get this kind of plot: https://imgpile.com/i/T3zPH1 I should note that the this plot was made from the abridged dataframe above, the actual dataframe has thousands of identifiers, making it very long. But as you can see, the labels on the x axis only appear at the bottom. I have been trying to get them to appear on the top, but seaborn doesn't seem to allow this kind of formatting. So I have also tried using plotly express, but while I solve the issue of placing my x-axis labels on top, I have been completely unable to format the heat map as I had before using seaborn. The following code: import plotly.express as px fig = px.imshow(df_example, width= 500, height=6000) fig.update_xaxes(side="top") fig.show()  yields this kind of plot: https://imgpile.com/i/T3zF42. I have tried many times to reformat it using the documentation from plotly (https://plotly.com/python/heatmaps/), but I can't seem to get it to work. When one thing is fixed, another problem arises. I really just want to keep using the seaborn based code as above, and just fix the x-axis labels. I'm also happy to have the x-axis label at both the top and bottom of the plot, but I can't get that work presently. Can someone advise me on what to do here?

Answer: Ok, so I did a bit more research, and it turns out you can add the follow code with the seaborn approach: plt.tick_params(axis='both', which='major', labelsize=10, labelbottom = False, bottom=False, top = False, labeltop=True)
Question: {'Functions': {0: {'Function-1': {0: {'Function': 'dd', 'Function2': 'd3'}}}}}  From the above json i would like to remove the {0: } item and add a list in that place so that the value is enclosed in a list like shown in Desired Output. Please note the above json is an put of a jsondiff. Desired output {"Functions":[{"Function-1":[{"Function":"dd","Function2":"d3"}]}]}  The below is my current code :  from jsondiff import diff json1 = json.loads(""" { "Name": "Temperature \u0026 Pressure Measurement", "Id": "0x0102", "Channels": [ { "Data": [ { "Channel0": [ { "Enable": 0, "Unit": "Celsius" } ], "Channel1": [ { "Enable": 0, "Unit": "Celsius" } ], "Channel2": [ { "Enable": 0, "Unit": "Celsius" } ] } ] } ], "Events": [ { "event1": 0, "event2": 0 } ], "Diagnostics": [ { "diag1": 0, "diag2": 0 } ], "Functions": [ { "Function-1": [ { "Function": "2d" } ] } ] } """) json2 = json.loads(""" { "Name": "Temperature \u0026 Pressure Measurement", "Id": "0x0102", "Channels": [ { "Data": [ { "Channel0": [ { "Enable": 0, "Unit": "Celsius" } ], "Channel1": [ { "Enable": 0, "Unit": "Celsius" } ], "Channel2": [ { "Enable": 0, "Unit": "Celsius" } ] } ] } ], "Events": [ { "event1": 0, "event2": 0 } ], "Diagnostics": [ { "diag1": 0, "diag2": 0 } ], "Functions": [ { "Function-1": [ { "Function": "dd", "Function2":"d3" } ] } ] } """) # This gives the difference between the json and this is what we want to operate on ! the 'res' may vary based on the changes made to json2 res = str(diff(json1, json2)) print('----------------------') print('------- DIFF -------') print('----------------------') print(f'{res}') print('----------------------') print('----------------------') print('') print('----------------------') print('---Expected Output---') print('----------------------') print('{"Functions":[{"Function-1":[{"Function":"dd","Function2":"d3"}]}]}') print('----------------------') print('----------------------')  EDIT:: To be more clear the res variable will change always. So i think it cannot always be achieved by using string replace because number of bracket may change based on the difference from json1 and json2

Answer: Code: from ndicts.ndicts import NestedDict  STEP 1// Convert nested dict to flat dict fd = list(pd.json_normalize(Mydict).T.to_dict().values())[0] #Output {'Functions.0.Function-1.0.Function': 'dd', 'Functions.0.Function-1.0.Function2': 'd3'}  STEP 2// Convert flat dictionary to nested by split/removing the 0 nd = NestedDict() for key, value in fd.items(): n_key = tuple(key.split(".0.")) nd[n_key] = value dic = nd.to_dict() dic #Output {'Functions': {'Function-1': {'Function': 'dd', 'Function2': 'd3'}}}  STEP 3// To add [], you can convert to dict to json and replace { with [{. json.loads(json.dumps(dic).replace('{','[{').replace('}','}]'))[0]  Desire Outout {'Functions': [{'Function-1': [{'Function': 'dd', 'Function2': 'd3'}]}]}  Package ref question ref
Question: Is there any elegant way of splitting a text by a word and keep the word as well. Although there are some works around split with re package and pattern like (Python RE library String Split but keep the delimiters/separators as part of the next string), but none of them works for this scenario when the delimiter is repeated multiple times. For example:  s = "I want to split text here, and also keep here, and return all as list items"  Using partition:  s.partition("here") >> ('I want to split text ', 'here', ', and also keep here, and return all as list items')  Using re.split(): re.split("here",s) >> ['I want to split text ', ', and also keep ', ', and return all as list items']  The desired output should be something to the following list: ['I want to split text', 'here', ' , and also keep ', 'here', ' , and return all as list items'] 

Answer: Yes. What you're looking for is a feature of the re.split() method. If you use a capture group in the expression, it will return the matched terms as well: import re s = "I want to split text here, and also keep here, and return all as list items" r = re.split('(here)', s) print(r)  Result: ['I want to split text ', 'here', ', and also keep ', 'here', ', and return all as list items']  If you define multiple capture groups, it will return each of them individually. So you can return just a part of the delimiter, or multiple parts that each get returned. I've done some fairly crazy things with this feature in the past. It can replace an appreciable amount of code that would otherwise be necessary.
Question: I have 40 Python unit tests and each of them open a Selenium driver as they are separate files and cannot share the same driver. from selenium import webdriver webdriver.Firefox()  The above commands will take the focus to the new opened window. For example, if I am on my editor and typing something, in the middle of my work, suddenly a selenium browser is opening and Linux switch to that window. I am not sure if Windows or Mac have a similar problem or not. This means that every time I run a unit, I cannot use my computer as it keeps switching away from the application that I am currently using. How can I tell Selenium not to switch to the opened window?

Answer: Here is an example of running Selenium/Firefox on linux, in headless mode. You can see various imports as well - gonna leave them there. Browser will start in headless mode, will go to ddg page and print out the page source, then quit. from selenium.common.exceptions import NoSuchElementException, TimeoutException from selenium import webdriver from selenium.webdriver.firefox.service import Service from selenium.webdriver.common.keys import Keys from selenium.webdriver.firefox.options import Options as Firefox_Options from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support.ui import Select from selenium.webdriver.support import expected_conditions as EC firefox_options = Firefox_Options() firefox_options.add_argument("--width=1280") firefox_options.add_argument("--height=720") firefox_options.headless = True driverService = Service('chromedriver/geckodriver') ## path where geckodriver is browser = webdriver.Firefox(service=driverService, options=firefox_options) wait = WebDriverWait(browser, 20) browser.get('https://duckduckgo.com') print(browser.page_source) browser.quit()  Browser can also take screenshots while in headless mode, if any unit test requires it. A reasonably well structured documentation for Selenium can be found at https://www.selenium.dev/documentation/ (with some gaps, of course, but generally decent).
Question: I am trying to split the string below on a number of delimiters including \n, comma(,), and colon(:) except when the colon is part of a time value. Below is my string: values = 'City:hell\nCountry:rome\nUpdate date: 2022-09-26 00:00:00'  I have tried: result = re.split(':|,|\n', values)  However, this ends up splitting the time resulting in ` ['City','hell','Country','rome','Update date',' 2022-09-26 00','00','00']  Whereas the expected outcome is ['City','hell','Country','rome','Update date', '2022-09-26 00:00:00']  Any help/assistance will be appreciated

Answer: You could use look-behind to ensure that what is before : is not a pair of digits re.split('(?<![0-9]{2}):\s*|,|\n', values)  It separates by  colons with optional spaces when they are not preceded by digits , \n  So : is a separator (when not preceded by a pair of digits). But so is :  or :  (still, when they are not preceded by a pair of digits). Consequence is that if, as it is the case if your string, there is a space after a colon, then that space is not included in the next field (since it is part of the separator, not of a field) Or, you could also keep the first version of my answer (without \s*) and just .strip() the fields.
Question: I have code which writes bytes to a serial port (pySerial 3.5) and then receives bytes. The code should work for micropython too, which uses UART instead of pySerial. With UART, I have to add a small delay before reading. The user should not have to pass an additional flag whether to add that delay, because the serial_port object is already platform specific, for example the UART implementation provides a .any() method which the pySerial implementation does not have. So my first attempt is to check for this method, and only delay when it exists. def __init__(self, serial_port): self.serial_port.write(my_bytes) # When checking for any on serial_port, I receive no bytes. if "any" in self.serial_port: print("UART specific delay will happen here") # When instead checking with getattr(self.serial_port, "any", None), bytes come in raw_config = self.serial_port.read(128)  As soon as I add this "any" in self.serial_port check, the read() method returns an empty byte array. When I remove the check, I get bytes again. When I replace the check by getattr(self.serial_port, "any", None), I get bytes too. When I just run time.sleep() or anything else, I get bytes. When I add the in check, bytes are gone. Why on earth? Isn't an in check supposed to be side effect free? (All runs were executed with pySerial ports only.)

Answer: As mentioned in a comment by deceze, in just calls the __contains__() method. Depending on how this is implemented, it can have side effects. A simple example to demonstrate this: class Example: def __contains__(self, x): print('Side effect!') x = Example() if 'something' in x: print('Found') else: print('Not found')  Output: Side effect! Not found  In the case of your serial_port example, I guess the __contains__() method is implemented in such a way that it reads bytes until "any" is found, or until there are no more bytes. Consequently, all bytes are already consumed and your function returns an empty array. NB: According to the documentation:  For objects that don’t define __contains__(), the membership test first tries iteration via __iter__(), then the old sequence iteration protocol via __getitem__().
Question: I have a dataframes as follows: df1 = col_1 val_1 0 4.0 0.89 1 4.0 0.56 2 49.0 0.7 3 49.0 1.23 4 52.0 0.8 5 52.0 0.12 6 32.0 0.5  I want to find the index value when the value in col_1 changes and put in a list I tried the following: n_change = (np.where(~df1.col_1.diff(+1).isin([0, np.nan])))  But it returns a tuple of array and it is difficult to itterate through it. I want a solution as follows n_change = [2,4,6] or n_change = array(2,4,6)  Is there a better way to do this?

Answer: You can use: df.index[df['col_1'].ne(df['col_1'].shift().bfill())] # or with diff # df.index[df['col_1'].diff().fillna(0).ne(0)]  output: Int64Index([2, 4, 6], dtype='int64') As list: df.index[df['col_1'].ne(df['col_1'].shift().bfill())].tolist()  output: [2, 4, 6] With your solution: np.where(~df.col_1.diff().isin([0, np.nan]))[0].tolist()  output: [2, 4, 6]
Question: I have a pandas dataframe as folllows, import pandas as pd df = pd.DataFrame({'text': ['set an alarm for [time : two hours from now]','wake me up at [time : nine am] on [date : friday]','check email from [person : john]']}) print(df)  original dataframe  text 0 set an alarm for [time : two hours from now] 1 wake me up at [time : nine am] on [date : friday] 2 check email from [person : john]  I would like to repeat the list and the labels (date, time, and person) for all the values inside the lists if the value inside the list is more than one. so the desired output is, desired output:  new_text 0 set an alarm for [time : two] [time : hours] [time : from] [time : now] 1 wake me up at [time : nine] [time : am] on [date : friday] 2 check email from [person : john]  I have so far tried to separate the lists from the original column, but do not know how to continue. df['separated_list'] = df.text.str.split(r"\s(?![^[]*])|[|]").apply(lambda x: [y for y in x if '[' in y]) 

Answer: You can use a regex with a custom function as replacement: df['new_text'] = df.text.str.replace( r"\[([^\[\]]*?)\s*:\s*([^\[\]]*)\]", lambda m: ' '.join([f'[{m.group(1)} : {x}]' for x in m.group(2).split()]), # new chunk for each word regex=True)  output:  text new_text 0 set an alarm for [time : two hours from now] set an alarm for [time : two] [time : hours] [time : from] [time : now] 1 wake me up at [time : nine am] on [date : friday] wake me up at [time : nine] [time : am] on [date : friday] 2 check email from [person : john] check email from [person : john]  regex demo
Question: I'd like to have a context manager that is opened (entered?) for the lifetime of the application. It should be opened when a module loads and be closed when the module is destroyed. Wrapping the whole module code won't work, since then the conetext manager is closed when the module is finished loading and is no longer open when the methods are actually called. import aiohttp with context as c: def f1(): c.do_something('one') def f2(): c.do_something('two')  Specifically, I want to create a aiohttp.ClientSession for the module which is reused for multiple different requests and cleanly closed when the application stops. Is it even a good idea to use a context manager for this or is there a better way?

Answer: A context manager is not suitable for managing the lifecycle of an object across modules. Instead, you can register the close method of your aiohttp.ClientSession instance with the atexit module so that it would be called when the app stops for any reason. Since in this case the close method is a coroutine, register the asyncio.run function with the close coroutine as an argument: import aiohttp import asyncio import atexit c = aiohttp.ClientSession(some_url) atexit.register(asyncio.run, c.close())
Question: I am trying to match the pattern 5X000Y000 of 9 digit number. What I have tried I have written the below regex B_5 = 530004000 B_5_pattern = re.sub(r'^5(\d(000))(\d(000))', "Bronze", str(B_5)) print(B_5_pattern)  What I want to achieve I want to update my regex to add a condition that X000 can not be the same as Y000. (X!=Y). So the regex will match 530004000 but will not match 530003000

Answer: You could use: B_5 = "530004000" if re.search(r'^5(\d)0{3}(?!\1)\d0{3}$', B_5): print("MATCH")
Question: I'm trying to change a datetime column to a string column using polars library. I only want the dates on the new column: import polars as pl df = pl.from_repr(""" ┌─────────────────────┐ │ date_time │ │ --- │ │ datetime[ns] │ ╞═════════════════════╡ │ 2007-04-19 00:00:00 │ │ 2007-05-02 00:00:00 │ │ 2007-05-03 00:00:00 │ │ 2007-05-03 00:00:00 │ └─────────────────────┘ """)  The solution below is including the time, I just need the date. df.with_columns(pl.col('date_time').cast(pl.String))  shape: (4, 1) ┌───────────────────────────────┐ │ date_time │ │ --- │ │ str │ ╞═══════════════════════════════╡ │ 2007-04-19 00:00:00.000000000 │ │ 2007-05-02 00:00:00.000000000 │ │ 2007-05-03 00:00:00.000000000 │ │ 2007-05-03 00:00:00.000000000 │ └───────────────────────────────┘ 

Answer: You should try this: # Polars df = df.with_columns(pl.col('date_time').dt.strftime('%Y-%m-%d')) # Pandas df['date_time'] = df['date_time'].dt.strftime('%Y-%m-%d')  Edit: added Polars