eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Is there a way in Python to initialise a variable so that it will work with += "a string" or with += 100? For example: a = <what to put here> a += 100 # a = 100 b = <same what to put here from above> b += "a string" # b = "a string"  If I use: a = "" a += "a string" # this will work fine b = "" b += 0 # -> TypeError  Once the type of the variable has been defined, it is okay to get a TypeError afterwards if I try to += a different type. For example: a = <what to put here> a += "a string" # a = "a string" a += 200 # -> okay to get a TypeError here  I’m only interested in making this work for ints and strings.

Answer: a = type('', (), {'__add__': lambda _, x: x})() a += 100 b = type('', (), {'__add__': lambda _, x: x})() b += "a string" print([a, b])  Output (Attempt This Online!): [100, 'a string']
Question: I want to add new items to the original csv file. The original file's ID increases by 1 each time an item is added, as shown below.     Id Name     0 Alpha   1 Beta   2 Gamma   3 Delta     I want to add the following array items = ["Epsilon", "Beta", "Zeta"] to the original csv file and eliminate duplicates, which would finally look like this:     Id Name     0 Alpha   1 Beta   2 Gamma   3 Delta   4 Epsilon   5 Zeta     I tried it with pandas, but the id column becomes "nan" for some reason. import pandas as pd items = ["Epsilon", "Beta", "Zeta"] df = pd.read_csv('original.csv', index_col='Id') for i in range(len(items)): df=df.append({'Id': len(df), 'Name': items[i]}, ignore_index=True) df = df.drop_duplicates(['Name'], ignore_index=True) df  I would appreciate it if you could help me with this problem.

Answer: Try: items = ["Epsilon", "Beta", "Zeta"] df = pd.concat([df, pd.DataFrame({"Name": items})]).drop_duplicates(subset="Name") df["Id"] = range(len(df)) print(df) # df.to_csv('out.csv')  Prints:  Id Name 0 0 Alpha 1 1 Beta 2 2 Gamma 3 3 Delta 0 4 Epsilon 2 5 Zeta
Question: I have following test.yaml: Camera.ColourOrder: "RGB" Camera.ColourDepth: "UCHAR_8" Camera.Spectrum: "Visible_NearIR" IMU.T_b_c1: !!opencv-matrix rows: 4 cols: 4 dt: f data: [0.999903, -0.0138036, -0.00208099, -0.0202141, 0.0137985, 0.999902, -0.00243498, 0.00505961, 0.0021144, 0.00240603, 0.999995, 0.0114047, 0.0, 0.0, 0.0, 1.0]  I am trying to read it using python opencv cv2 module (because it contains opencv related objects like opencv_matrix embedded in yaml, which will give error if tried reading using python's inbuilt yaml module). But it is giving me error as shown below: >>> import os >>> 'test.yaml' in os.listdir() True >>> import cv2 >>> fs = cv2.FileStorage("test.yaml", cv2.FILE_STORAGE_READ) cv2.error: OpenCV(4.8.1) /io/opencv/modules/core/src/persistence.cpp:699: error: (-5:Bad argument) Input file is invalid in function 'open' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> SystemError: <class 'cv2.FileStorage'> returned a result with an error set  I tried removing IMU.T_b_c1 and keeping just Camera.xyz values, still the same error. What I am missing here? PS: is there any other way / module to read such yaml file with embedded opencv objects? I am ok with not using cv2 and read some other package / module.

Answer: Judging by the error message and the implementation, OpenCV expects YAML data to be preceded by a YAML directive (%YAML). The directive is also shown in the documentation's sample. Try the following (notice first line) %YAML:1.0 Camera.ColourOrder: "RGB" Camera.ColourDepth: "UCHAR_8" Camera.Spectrum: "Visible_NearIR" IMU.T_b_c1: !!opencv-matrix rows: 4 cols: 4 dt: f data: [0.999903, -0.0138036, -0.00208099, -0.0202141, 0.0137985, 0.999902, -0.00243498, 0.00505961, 0.0021144, 0.00240603, 0.999995, 0.0114047, 0.0, 0.0, 0.0, 1.0]
Question: I have a dataframe like this: data = { 'name': ['101 blueberry 2023', '102 big cat 2023', '103 small white dog 2023'], 'number': [116, 118, 119]} df = pd.DataFrame(data) df  output:  name number 0 101 blueberry 2023 116 1 102 big cat 2023 118 2 103 small white dog 2023 119  I would like to change the first and last numbers in the name column. For example, the first number in name to the number in the number column, and the last number in name to '2024'. So finally it would look like:  name number 0 116 blueberry 2024 116 1 118 big cat 2024 118 2 119 small white dog 2024 119  I have tried splitting name into a list and changing the first and last elements of the list. df['name_pieces'] = df['name'].split(' ') df  output:  name number name_pieces 0 101 blueberry 2023 116 [101, blueberry, 2023] 1 102 big cat 2023 118 [102, big, cat, 2023] 2 103 small white dog 2023 119 [103, small, white, dog, 2023]  I can access the first item of the lists using .str, but I cannot change the item. df['name_pieces'].str[0]  output: 0 101 1 102 2 103  but trying to assign the first value of the list gives an error df['name_pieces'].str[0] = df['number']  output: TypeError: 'StringMethods' object does not support item assignment  How can I replace the first and last value of name inside this dataframe?

Answer: Don't bother with the lists. You can just extract the part of the strings you want and join the other parts. df.assign(name= df['number'].astype(str) + df['name'].str.extract(r'( .* )', expand=False) + '2024' )   name number 0 116 blueberry 2024 116 1 118 big cat 2024 118 2 119 small white dog 2024 119  This regex gets the longest part of the string surrounded by spaces, i.e the part between the first space and last space.  Here's a variation if you'd rather think about name primarily: df.assign(name= df['name'].str.extract(r'( .* )', expand=False) .radd(df['number'].astype(str)) .add('2024') )
Question: The CPython Tutorial defines a custom initializer for a custom type which has the following lines : if (first) { tmp = self->first; Py_INCREF(first); self->first = first; Py_XDECREF(tmp); }  but the tutorial advises against this simpler but more nefarious version : if (first) { Py_XDECREF(self->first); Py_INCREF(first); self->first = first; }  For the following reason :  But this would be risky. Our type doesn’t restrict the type of the first member, so it could be any kind of object. It could have a destructor that causes code to be executed that tries to access the first member; or that destructor could release the Global interpreter Lock and let arbitrary code run in other threads that accesses and modifies our object.  I understand the reason why a multithreaded programming can break the simple version. There can be a race condition where the new self->first is set by one thread and freed by another thread (by way of Py_XDECREF). This is avoided in the correct version because both threads will have had to set self->first before freeing it. The part I don't understand is this section :  It could have a destructor that causes code to be executed that tries to access the first member  If first is an object that is getting garbage collected, its destructor would have to be called and it should have access to itself through the duration of the destructor. Why can this be dangerous? Thank you!

Answer: Let's suppose we have: custom = Custom(...) # global variable class SomePyClass: def __del__(self): # access global variable custom.__init__(1, 2, 3)  Let's suppose that custom.first is an instance of SomePyClass and that this instance gets destructed at the line Py_XDECREF(self->first);. The arbitrary code in the destructor is going to cause custom.__init__ to be called again. This will cause Py_XDECREF to be called again on an object that really should have a refcount of 0 and which is already in the process of getting destroyed. That alone represents a reference counting error since it's ref-count will end up below 0 (note this may not be exactly what happens since it temporarily gets given a ref-count of 1 for the duration of the destructor, but it absolutely is a ref-counting error). Note also the new value of first that is being assigned to in __init__ is also mis-reference-counted: while our instance is decrefed twice, the value that is about to be assigned is replaced without decrefing it.
Question: I have a GitHub environment secret {{ secrets.GITHUBAPP_KEY } that holds a .pem key, in a workflow step, I'm trying to pass the secret to an env variable GITHUBAPP_KEY  - name: Do Certain Step run: Insert fancy command here env: GITHUBAPP_KEY: "${{ secrets.GITHUBAPP_KEY }}"  Here is the error the GitHub actio workflow gets when I run it: error: error parsing STDIN: invalid Yaml document separator: --END RSA PRIVATE KEY-----" The key is the correct format, I have also wrapped double quotes around the secrets context, yet, the file still does not get parsed correctly. How can I solve this issue?

Answer: As the SSH private key is spanned over multiple lines, you can assign it to an env var as a multiline string by using pipe | like this: env: GITHUBAPP_KEY: | ${{ secrets.GITHUBAPP_KEY }}  See https://yaml-multiline.info/ for interactive examples.
Question: Here is a Venn Diagram I drew in Python: !pip install matplotlib_venn from matplotlib_venn import venn2 import matplotlib.pyplot as plt from matplotlib_venn import venn2 import matplotlib.pyplot as plt set1_size = 20 set2_size = 25 intersection_size = 5 venn2(subsets=(set1_size, set2_size, intersection_size)) plt.show()  This makes a brown part, a green part and a red part. I want to extract the green part separately, the brown part separately and the red part separately. Is there something that can be done to do this? I tried looking but found nothing. Is it even possible? Or is it possible to just have the venn diagram as blank color, and then just individually highlight certain parts? Ex: center as yellow, (left circle without center) as purple, etc?

Answer: Possible arguments for .get_patch_by_id() include '10', '11', and '01'. See also this link. See the patch documentation for other patch parameters (e.g. color) that you can set. from matplotlib_venn import venn2 import matplotlib.pyplot as plt set1_size = 20 set2_size = 25 intersection_size = 5 fig, ax = plt.subplots() vd = venn2(subsets=(set1_size, set2_size, intersection_size)) vd.get_patch_by_id('10').set(visible=False, label=None) vd.get_label_by_id('10').set_text("") vd.get_patch_by_id('11').set(visible=False, label=None) vd.get_label_by_id('11').set_text("") plt.show()
Question: I have a csv file in which the first line reads something like the following: Pyscip_V1.11 Ref: #001=XYZ_0[1234] #50=M3_0[112] #51=M3_1[154] #52=M3_2[254]...  and so on. What I'd like to do is create filtered value such that the first column is Ref and it takes all the values after the # sign like 001,50,51,52... The second column name is ID and it takes the value after = like XYZ_0,M3_0,M3_1,M3_2,M3_3... And finally make a third column which takes all the values present in the square brackets like 1234,112,154,254,... header_pattern = r'Pyscip_V(\d+\.\d+) Ref:' version_match = re.search(header_pattern, first_line.iloc[0, 0]) version_number = version_match.group(1) if version_match else '' matches = re.findall(r'#(\d+)=(\w+_\d)\[([\d]+)\]', first_line.iloc[0, 0]) parsed_df = [] for match in matches: row_dict = { 'Ref': match[0] if match[0] else '', 'ID': match[1] if match[1] else '', 'Ser_No': match[2] if match[2] else '' } parsed_df.append(row_dict) new_df = pd.DataFrame(parsed_df)  However, I only get enpty dataframe. What seems to be the problem here? Edit: the data from 3rd row looks like the following: ID Date XYZ_0 M3_0 M3_1 M3_2 1 22.12.2023 12.6 0.5 1.2 2.3  The expected outcome is Ref ID Num 001 XYZ_0 1234 50 M3_0 112 51 M3_1 154 

Answer: I would open the csv file, extract the first line and process it, only after read the rest of the CSV with pandas. For that, your original approach and regex are fine. import re import pandas as pd with open('my_csv.csv') as f: first_line = next(f) header_df = pd.DataFrame(re.findall(r'#(\d+)=(\w+_\d)\[([\d]+)\]', first_line), columns=['Ref', 'ID', 'Num']) data_df = pd.read_csv(f, sep=r'\s+') print(header_df) print(data_df)  Output: # header_df Ref ID Num 0 001 XYZ_0 1234 1 50 M3_0 112 2 51 M3_1 154 3 52 M3_2 254 # data_df ID Date XYZ_0 M3_0 M3_1 M3_2 0 1 22.12.2023 12.6 0.5 1.2 2.3
Question: This is mostly a question about python: How is x in [a, b, c] evaluated differently than x in {a, b, c}. The context I'm struggling with is like this: import polars as pl s = pl.Series(["a", "b"], dtype=pl.Categorical) s.dtype in [pl.Categorical, pl.Enum] # True s.dtype in {pl.Categorical, pl.Enum} # False s.dtype in {pl.Categorical: 1, pl.Enum: 2} # False  I want to understand python better. I also wonder whether polars can do anything to make the second case work, since it currently seems like a footgun.

Answer: EDIT: polars dtypes do not follow the usual equals-contract in multiple ways, and this is considered by design: https://github.com/pola-rs/polars/issues/9564 (thanks @jqurious), specifically, they violate transitivity and hashcode-consistency. In polars, a specific datatype is considered equal to a more general datatype even when different specific datatypes are not equal to each other. eg "pl.List == pl.List(str) => True pl.List(int) == pl.List(str) => False".  Original answer, where I assumed hashcode-inconsistency is all that is happening here: This is a broken equality-implementation. See this: import polars as pl s = pl.Series(["a", "b"], dtype=pl.Categorical) print(s.dtype is pl.Categorical) # False print(s.dtype == pl.Categorical) # True print(hash(s.dtype) == hash(pl.Categorical) ) # False  s.dtype is a new object that is considered equal to pl.Categorical. However, the __hash__-function is not overridden correctly: Two objects that are considered equal must have the same hashcode (or ban usage of the hash-function). Algorithms may rely on this and produce weird results if this is not true for a class! If you override __eq__ on a class, read this: https://docs.python.org/3/reference/datamodel.html#object.__hash__ What is happening here is that obj in list iterates the list and tests equality of the elements with obj, while obj in set uses the hashcode of obj to locate the object in the set, and if no object with that hashcode exists in the set the conclusion is that it is not in the set.
Question: In pydantic v2, the following code: from __future__ import annotations import pydantic from pprint import pprint class System(pydantic.BaseModel): id: int name: str subsystems: list[System] | None = None @pydantic.computed_field() @property def computed(self) -> str: return self.name.upper() systems = System( id=1, name="Main system", subsystems=[ System(id=2, name="Subsystem A"), System(id=3, name="Subsystem B"), ], ) pprint(systems.model_dump(), indent=2)  prints: { 'computed': 'MAIN SYSTEM', 'id': 1, 'name': 'Main system', 'subsystems': [ { 'computed': 'SUBSYSTEM A', 'id': 2, 'name': 'Subsystem A', 'subsystems': None}, { 'computed': 'SUBSYSTEM B', 'id': 3, 'name': 'Subsystem B', 'subsystems': None}]}  I want to exclude the computed field computed. pprint(systems.model_dump(exclude={'computed': True}), indent=2)  prints only root element without computed: { 'id': 1, 'name': 'Main system', 'subsystems': [ { 'computed': 'SUBSYSTEM A', 'id': 2, 'name': 'Subsystem A', 'subsystems': None}, { 'computed': 'SUBSYSTEM B', 'id': 3, 'name': 'Subsystem B', 'subsystems': None}]}  How to exclude computed field (e.g. computed) from dump? I want to serialize via .model_dump() because:  I have many models and models those include System are serialized via .model_dump() method as well. .model_dump() has some useful parameters like exclude_defaults, by_alias and more. I use them as well. 

Answer: According to the documentation on computed_field:  computed_field Decorator to include property and cached_property when serializing models or dataclasses. This is useful for fields that are computed from other fields, or for fields that are expensive to compute and should be cached.  In other words, if don't want to include (= exclude) a field we shouldn't use computed_field decorator: from __future__ import annotations import pydantic from pprint import pprint class System(pydantic.BaseModel): id: int name: str subsystems: list[System] | None = None # comment or remove it # @pydantic.computed_field() @property def computed(self) -> str: return self.name.upper() systems = System( id=1, name="Main system", subsystems=[ System(id=2, name="Subsystem A"), System(id=3, name="Subsystem B"), ], ) pprint(systems.model_dump(), indent=2) print(systems.computed) # but it is still there  Output is: { 'id': 1, 'name': 'Main system', 'subsystems': [ {'id': 2, 'name': 'Subsystem A', 'subsystems': None}, {'id': 3, 'name': 'Subsystem B', 'subsystems': None}]} MAIN SYSTEM
Question: When I run the following command: mamba create --name eco-tech-h2gam-venv regionmask cartopy  I get this error: Looking for: ['regionmask', 'cartopy'] error libmamba Could not open lockfile 'C:\ProgramData\anaconda3\pkgs\cache\cache.lock'  Any idea how to overcome it? Additional Details  OS: Windows 11 Anaconda 3 base distribution 

Answer: Lock files can be cleared using the mamba clean --locks command. See in the documentation (abridged): $ mamba clean -h # usage: mamba clean [-h] [-a] [-i] [-p] [-t] [-f] [-c [TEMPFILES ...]] [-l] [--json] [-v] # [-q] [-d] [-y] [--locks] # # Removal Targets: # --locks Remove lock files.
Question: I have this boruta code, and I want to generate the results in pandas with columns included model = RandomForestRegressor(n_estimators=100, max_depth=5, random_state=42) # let's initialize Boruta feat_selector = BorutaPy( verbose=2, estimator=model, n_estimators='auto', max_iter=10, # numero di iterazioni da fare random_state=42, ) # train Boruta # N.B.: X and y must be numpy arrays feat_selector.fit(np.array(X), np.array(y)) # print support and ranking for each feature print("\n------Support and Ranking for each feature------\n") for i in range(len(feat_selector.support_)): if feat_selector.support_[i]: print("Passes the test: ", X.columns[i], " - Ranking: ", feat_selector.ranking_[i], "✔️") else: print("Doesn't pass the test: ", X.columns[i], " - Ranking: ", feat_selector.ranking_[i], "❌") # features selected by Boruta X_filtered = feat_selector.transform(np.array(X))  My selected result is this: X.columns[feat_selector.support_] Index(['J80', 'J100', 'J160', 'J200', 'J250'], dtype='object') X_filtered array([[12.73363 , 8.518314 , 5.2625847 , ..., 0.06733382]])  How do I generate the result in Pandas dataframe with the headers? Now I have up to 25 headers.

Answer: Since support_ is a boolean mask, you can index the columns and create a new dataframe. X_filtered = pd.DataFrame( feat_selector.transform(X.values), columns=X.columns[feat_selector.support_] )  Then again, with the latest master version, you can pass a dataframe to transform() and flag return_df=True. So that would look like: X_filtered = feat_selector.transform(X, return_df=True)
Question: It seems that checking if np.nan is in a list after pulling the list from a pandas dataframe does not correctly return True as expected. I have an example below to demonstrate: from numpy import nan import pandas as pd basic_list = [0.0, nan, 1.0, 2.0] nan_in_list = (nan in basic_list) print(f"Is nan in {basic_list}? {nan_in_list}") df = pd.DataFrame({'test_list': basic_list}) pandas_list = df['test_list'].to_list() nan_in_pandas_list = (nan in pandas_list) print(f"Is nan in {pandas_list}? {nan_in_pandas_list}")  I would expect the output of this program to be: Is nan in [0.0, nan, 1.0, 2.0]? True Is nan in [0.0, nan, 1.0, 2.0]? True  But instead it is Is nan in [0.0, nan, 1.0, 2.0]? True Is nan in [0.0, nan, 1.0, 2.0]? False  What is the cause of this odd behavior or am I missing something? Edit: Adding on to this, if I run the code: for item in pandas_list: print(type(item)) print(item)  it has the exact same output as if I were to swap pandas_list with basic_list. However pandas_list == basic_list evaluates to False.

Answer: TL;DR pandas is using different nan object than np.nan and in operator for list checks if the object is the same.  The in operator invokes __contains__ magic method of list, here is source code: static int list_contains(PyListObject *a, PyObject *el) { PyObject *item; Py_ssize_t i; int cmp; for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) { item = PyList_GET_ITEM(a, i); Py_INCREF(item); cmp = PyObject_RichCompareBool(item, el, Py_EQ); Py_DECREF(item); } return cmp; }  You see there is PyObject_RichCompareBool called which states:  If o1 and o2 are the same object, PyObject_RichCompareBool() will always return 1 for Py_EQ and 0 for Py_NE.  So: basic_list = [0.0, nan, 1.0, 2.0] for v in basic_list: print(v == nan, v is nan) print(nan in basic_list)  Prints: False False False True False False False False True  And: df = pd.DataFrame({"test_list": basic_list}) pandas_list = df["test_list"].to_list() for v in pandas_list: print(v == nan, v is nan) print(nan in pandas_list)  Prints: False False False False False False False False False  Evidently, pandas is using different nan object.
Question: I'm very new to polars and i'm trying to translate some pandas statements. The pandas line is as follows: df.loc[df['col_x'].isna(), 'col_y'] = df['col_z']  that is to say: replace the values of col_y corresponding to null values of col_x with values of col_z. In polars i'm trying with select, but to no avail.

Answer: Here you have the solution for Polars and Pandas: Polars: df = ( df .with_columns(pl.when(pl.col('col_x').is_nan()).then(pl.col('col_z')) .otherwise(pl.col('col_y')).alias('col_y')) )  Pandas: df["col_y"] = np.where(pd.isnull(df['col_x']), df['col_z'], df['col_y'])  I hope this helps!
Question: I have move from Selenium/SeleniumBase to DrissionPage for it's Undetectable Ability. Everything was easy to understand. Except the last part of it. That is Closing the Browser. The Docs says that page.quit() would close the browser. But When I run it in my local machine I get this error: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\threading.py", line 973, in _bootstrap_inner self.run() File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\threading.py", line 910, in run self._target(*self._args, **self._kwargs) File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\site-packages\DrissionPage\chromium_driver.py", line 123, in _recv_loop message = loads(message_json) File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\PC\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)  Here's The least of code: from DrissionPage import ChromiumPage from time import sleep page = ChromiumPage() page.get('https://www.google.com/') sleep(5) page.quit()  My specs are: { "OS": "Windows 10 64 Bit", "OS_Version": "10.0 Build (10240)", "Browser": "Google Chrome", "Browser Version": "120.0.6099.130 (Official Build) (64-bit)" }  How to fix this error and close it? Or, is there any alternative way to close it?

Answer: I am the author of DrissionPage, thank you for liking my work. Version 4.0 is fine. Now in beta, the latest is 4.0.0b26. pip install DrissionPage==4.0.0b26