eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I ran into this problem when working with defaultdict. Here's a program that demonstrates it: from collections import defaultdict d1 = defaultdict(default_factory=dict) d2 = defaultdict(dict) print("d1's default_factory:", d1.default_factory) print("d2's default_factory:", d2.default_factory) try: d1['key'].update({'a': 'b'}) except KeyError: print("d1 caused an exception") try: d2['key'].update({'a': 'b'}) except KeyError: print("d2 caused an exception")  The above outputs: d1's default_factory: None d2's default_factory: <class 'dict'> d1 caused an exception  Should this happen?

Answer: default_factory is a positional-only argument. It cannot be passed by keyword. d1 = defaultdict(default_factory=dict) creates a defaultdict with no default factory and a key 'default_factory' with value dict. It's as if you did d1 = defaultdict() d1['default_factory'] = dict
Question: Its the sample input. i wanted to group according to the Year column,and wanted to use value counts on month column, then to sort the 'month' column according to the month order.    Year month     2000 Oct   2002 Jan   2002 Mar   2000 Oct   2002 Mar   2000 Jan    I did this: df.groupby(['Year'])['month'].value_counts() i got the following output:    year month     2000 Oct 2    Jan 1   2002 Mar 2    Jan 1    now i need to sort the month in the original month order.what can i do? i want the following output:    year month     2000 Jan 1    Oct 2   2002 Jan 1    Mar 2   

Answer: You can use groupby() and sort_values(by=['Year', 'month']): import pandas as pd def _sort_month(df): month_order = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'] df['month'] = pd.Categorical(df['month'], categories=month_order, ordered=True) GB = df.groupby(['Year'])['month'].value_counts() G = GB.reset_index(name='count') res = G.sort_values(by=['Year', 'month']) res.set_index(['Year', 'month'], inplace=True) return res[res['count'] > 0] df = pd.DataFrame({'Year': [2000, 2002, 2002, 2000, 2002, 2000], 'month': ['Oct', 'Jan', 'Mar', 'Oct', 'Mar', 'Jan']}) print(_sort_month(df))  Prints  count Year month 2000 Jan 1 Oct 2 2002 Jan 1 Mar 2
Question: I am using Python 2.7. (Switching to Python 3 for this particular code is not an option, please don't suggest it.) I am writing unit tests for some code. Here is the relevant piece of code: class SingleLineGrooveTable: VEFMT = '.3f' @classmethod def formatve(cls, value, error=None): er = 0 if error is not None: er = error v = value elif len(value) > 1: v, er = value else: v = value return format(v, cls.VEFMT), format(er, cls.VEFMT)  and my test is: import unittest class TestSingleLineGrooveTable(unittest.TestCase): def test_formatve_no_error(self): e_v = '3.142' e_er = '0.000' r_v, r_er = SingleLineGrooveTable.formatve([3.1423]) self.assertEqual(e_v, r_v) self.assertEqual(e_er, r_er)  (Yes, I know it's funny I'm getting an error on the test with "no_error" in the name...) When I run the test, it throws ValueError: Unknown format code 'f' for object of type 'str' on the return statement for the function. But I can't figure out where it's getting a str from. Possibly relevant, this code and the code I have that uses it were copied pretty much wholesale from someone else's code (who I can no longer contact), so maybe I'm calling it in the wrong way, but still, that's a list, not a string! What is going on here? How do I fix this?

Answer: On Python 2, object.__format__ effectively delegates to format(str(self), format_spec). You can see the implementation here. Since list inherits object.__format__, your first format call is effectively calling format(str([3.1423]), '.3f'). That's why you get the error message you do. This would still produce an error on Python 3. It'd just be a different error.
Question: I am trying to add a Compound to a List to make a Compound List: import nbtlib from nbtlib.tag import * CpdList = List(Compound()) Cpd = Compound() Cpd['name'] = String("Name") Cpd['test'] = Byte(0) CpdList.append(Cpd)  This returns the following error: >>> nbtlib.tag.IncompatibleItemType: Compound({'name': String('Name'), 'test': Byte(0)}) should be a End tag I don't understand what an End tag is, and how to solve this issue. I tried adding an End tag to the Compound, but without success: Cpd['End'] = End 

Answer:  You just need to properly use nbtlib.tag.List with the Compound type and properly make a call:  import nbtlib from nbtlib.tag import Compound, List, String, Byte CpdList = List[Compound]() Cpd = Compound() Cpd['name'] = String("Name") Cpd['test'] = Byte(0) CpdList.append(Cpd) print(CpdList)  Note: Note that the nbtlib.tag.List expects the same type of elements, and the Compound type is not directly allowed in a List tag. The error mentions the need for an End tag, which is a terminator for Compound tags.  In case you need a dynamic type list, you can use python native list(): import nbtlib from nbtlib.tag import Compound, String, Byte CpdList = list[Compound]() Cpd = Compound() Cpd['name'] = String("Name") Cpd['test'] = Byte(0) CpdList += [Cpd, 0, 'alice', {1: 2}] print(CpdList)  Prints [Compound({'name': String('Name'), 'test': Byte(0)}), 0, 'alice', {1: 2}]   Which is same as:  import nbtlib from nbtlib.tag import Compound, String, Byte CpdList = list(Compound()) Cpd = Compound() Cpd['name'] = String("Name") Cpd['test'] = Byte(0) CpdList += [Cpd, 0, 'alice', {1: 2}] print(CpdList)
Question: I have a dataframe like below.  A B C 0 3 329430734 998 1 3 329430742 258 2 3 329430776 126 3 3 329430778 998 4 3 329430784 33 5 3 329430851 21 6 3 329430897 998 7 3 329430917 998 8 3 329430943 998 9 3 329430945 998  and I am trying to transform the data in below way to store it in a new column. df["x"]= np.where((df['C'] == 998) | (df['C'] == 999), max(0, df['A']-1), df['C'] )  However i am getting below while trying transform the column values. ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().  I tried to use apply function also using the way below but getting the same error. def maximum(x, y): if ((y == 998) | (y == 999)): return (max(0, x-1)) else: return (y)  Unable to find out the issue behind it. Can anyone help me to resolve this issue?

Answer: You can use np.maximum: df["x"] = np.where(df["C"].isin([998, 999]), np.maximum(0, df["A"] - 1), df["C"])   A B C x 0 3 329430734 998 2 1 3 329430742 258 258 2 3 329430776 126 126 3 3 329430778 998 2 4 3 329430784 33 33 5 3 329430851 21 21 6 3 329430897 998 2 7 3 329430917 998 2 8 3 329430943 998 2 9 3 329430945 998 2
Question: I have this code: >>> d = np.dtype([("pos", np.float64, 3)]) >>> d[0].kind 'V'  Why does it return 'V' instead of 'f'? In the full code, I need to know if the field corresponds to an integer, float, string...

Answer: d[0].kind does not return 'f' because it is not a floating-point dtype: it is a structured dtype with a floating point base. There are several other attributes of structured dtypes that you might be able to inspect depending on your particular goal. For example: >>> d[0].base dtype('float64') >>> d[0].subdtype[0].kind 'f'
Question: i'm trying to display an home image at top of the page but it doesn't work. I have the same image in an other page and it works. also if i hold mouse on image source link on the ide it says me that the path is correct.  . : Here is the line code (path in the images): <p><img src="../media/images/home.png" width="30px" alt="home image">HOME</p>

Answer: Relative paths are probably the culprit: if you visit a page that has more slashes, then the .. thus points to a different directory. Use absolute paths, so: /media/… instead of ../media/….
Question: This is my DataFrame: import pandas as pd import numpy as np df = pd.DataFrame( { 'x': [1, np.nan, 3, np.nan, 5], 'y': [np.nan, 7, 8, 9, np.nan], 'x_a': [1, 2, 3, 4, 5], 'y_a': [6, 7, 8, 9, 10] } )  Expected output is fill_na columns x and y:  x y x_a y_a 0 1.0 6.0 1 6 1 2.0 7.0 2 7 2 3.0 8.0 3 8 3 4.0 9.0 4 9 4 5.0 10.0 5 10  Basically I want to fillna x with x_a and y with y_a. In other words each column should be paired with another column that has the suffix _a and the column name. I can get this output by using this code: for col in ['x', 'y']: df[col] = df[col].fillna(df[f'{col}_a'])  But I wonder if it is the best/most efficient way? Suppose I got hundreds of columns like these

Answer: What about using an Index to select all columns at once and set_axis to realign the DataFrame: cols = pd.Index(['x', 'y']) df[cols] = df[cols].fillna(df[cols+'_a'].set_axis(cols, axis=1))  NB. this is assuming all columns in cols and all '_a' columns exist. If you're not sure you could be safe and use intersection and reindex: cols = pd.Index(['x', 'y']).intersection(df.columns) df[cols] = df[cols].fillna(df.reindex(columns=cols+'_a').set_axis(cols, axis=1))  Or for an approach that is fully independent of explicitly passing input columns and just relying on the suffix (_a): suffix = '_a' # find columns "xyz" that have a "xyz_a" counterpart c1 = df.columns.intersection(df.columns+suffix) c2 = c1.str.removesuffix(suffix) # select, fillna, update df[c2] = df[c2].fillna(df[c1].set_axis(c2, axis=1))  Output:  x y x_a y_a 0 1.0 6.0 1 6 1 2.0 7.0 2 7 2 3.0 8.0 3 8 3 4.0 9.0 4 9 4 5.0 10.0 5 10  Example for which the second approach would be needed: df = pd.DataFrame( { 'x': [1, np.nan, 3, np.nan, 5], 'z': [np.nan, 7, 8, 9, np.nan], 'p_a': [1, 2, 3, 4, 5], 'y_a': [6, 7, 8, 9, 10] } )
Question: Here's a naive solution of what I want to do using map_elements. How can I do this with only Polars functions? import polars as pl # Create a DataFrame with a column containing lists of strings df = pl.DataFrame({ "list_of_strings": [["a", "b", "c"], ["d", "e", "f"], ["g", "h", "i"]] }) # Define a function to concatenate lists of strings into a single string def concatenate_list_of_strings(lst): return "".join(lst) # Apply the function to the DataFrame df = df.with_column( pl.col("list_of_strings").map_elements(concatenate_list_of_strings, return_dtype=pl.String).alias("concatenated_string") ) print(df) 

Answer: As already mentioned in the comments, there is pl.Expr.list.join in polars' native expression API to join all string items in a sublist with a separator between them. df.with_columns( pl.col("list_of_strings").list.join("") )  shape: (3, 1) ┌─────────────────┐ │ list_of_strings │ │ --- │ │ str │ ╞═════════════════╡ │ abc │ │ def │ │ ghi │ └─────────────────┘
Question: I want to convert a duration to integer number of hour (or minute or seconds). I though the .dt namespace would work the same as for the datetimes, but I get an error instead. This example from datetime import datetime import polars as pl pl.__version__ dx = pl.DataFrame({'dt1': datetime(2024, 1, 12, 13, 45)}) dx.select((pl.col('dt1') - pl.col('dt1').dt.date()).dt.hour())  gives the error: --------------------------------------------------------------------------- InvalidOperationError Traceback (most recent call last) Cell In[5], line 2 1 dx = pl.DataFrame({'dt1': datetime(2024, 1, 12, 13, 45)}) ----> 2 dx.select((pl.col('dt1') - pl.col('dt1').dt.date()).dt.hour()) File ~/src/poste/sda-poste-logistics/venv/lib/python3.11/site-packages/polars/dataframe/frame.py:8461, in DataFrame.select(self, *exprs, **named_exprs) 8361 def select( 8362 self, *exprs: IntoExpr | Iterable[IntoExpr], **named_exprs: IntoExpr 8363 ) -> DataFrame: 8364 """ 8365 Select columns from this DataFrame. 8366 (...) 8459 └───────────┘ 8460 """ -> 8461 return self.lazy().select(*exprs, **named_exprs).collect(_eager=True) File ~/src/poste/sda-poste-logistics/venv/lib/python3.11/site-packages/polars/lazyframe/frame.py:1967, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, no_optimization, streaming, background, _eager, **_kwargs) 1964 # Only for testing purposes atm. 1965 callback = _kwargs.get("post_opt_callback") -> 1967 return wrap_df(ldf.collect(callback)) InvalidOperationError: `hour` operation not supported for dtype `duration[μs]`  on both polars 0.20.31 and 1.0.0-a1. Is this a bug or am I doing something wrong?

Answer: After the subtraction, you don't have a date anymore but a timedelta, you should use dt.total_seconds: dx.select((pl.col('dt1') - pl.col('dt1').dt.date()).dt.total_seconds())  Output: ┌───────┐ │ dt1 │ │ --- │ │ i64 │ ╞═══════╡ │ 49500 │ └───────┘  Or, as total_hours / fractional hours: dx.select((pl.col('dt1') - pl.col('dt1').dt.date()).dt.total_hours()) ┌─────┐ │ dt1 │ │ --- │ │ i64 │ ╞═════╡ │ 13 │ └─────┘ dx.select((pl.col('dt1') - pl.col('dt1').dt.date()).dt.total_seconds()/3600) ┌───────┐ │ dt1 │ │ --- │ │ f64 │ ╞═══════╡ │ 13.75 │ └───────┘
Question: PyCharm suddenly changed the way it shows the stack trace on the run tab and does not let me click on the exception (or anywhere else) and go to the specific point of the error file anymore. How can I fix this? Running on OSX Sonoma, PyCharm 2022.2.3 (Community Edition) 

Answer: The short answer: Eliminate all (direct or indirect) imports of the pretty_errors package. The medium answer If you use the torchmetrics package in your project, which tries to use pretty_errors, one of the following should work: (1) just uninstall the pretty_errors package from your Python environment, (2) update torchmetrics to a version that no longer uses pretty_errors (update: such as v1.5). If you don't use torchmetrics, you might want to identify any other package that uses the pretty_errors package and eliminate the corresponding imports of pretty_errors. The long answer I had the same problem with exactly the same look and behavior of console outputs. In my case, I could trace it down to using the package torchmetrics in a project, which, in turn, imported the package pretty_errors. The pretty_errors package changes stack traces in the way that you experienced (compare screenshots on the project's site). And here is the corresponding code in torchmetrics/__init__.py where pretty_errors was imported (lines 17–18 in the source code): if package_available("pretty_errors"): import pretty_errors # noqa: F401  Once I commented out those lines locally, stack traces looked and behaved as usual again. Above I wrote "was imported", because the current source code of torchmetrics does not use pretty_errors any longer – exactly for the reason that it makes stack traces arguably unusable – see corresponding pull request and commit on GitHub. So, at some point, probably the best solution is updating torchmetrics in your project. Update: By now, this should be possible with torchmetrics v1.5. For older versions, just uninstalling pretty_errors from your environment should do the trick. In particular, it won't break torchmetrics, since the import of pretty_errors is conditional there, anyway (see code snippet above). If you don't use torchmetrics in your project, of course, then it must be another package that relies on pretty_errors, or you directly imported pretty_errors yourself. Identifying the culprit(s) shouldn't be that hard, and the corresponding solution should always be eliminating the use of pretty_errors.
Question: I am using the hypothesis library and I would like to annotate my code with type hints. The docs are mentioning the hypothesis.strategies.SearchStrategy as the type for all search strategies. Take this example: @composite def int_strategy(draw: DrawFn) -> hypothesis.strategies.SearchStrategy[int]: ... # some computation here resulting in ``x`` being an ``int`` return x  Running mypy will (rightly so) result in an error along those lines: error: Returning Any from function declared to return "SearchStrategy[Any]" [no-any-return] I mean, I am actually returning an int, not a SearchStrategy. How am I supposed to type annotate my hypothesis strategies?

Answer: Functions decorated with @composite should be type hinted as normal: @composite def int_strategy(draw: DrawFn) -> int: ...  @composite will then automatically transform this to something like: # As if it doesn't have the `draw` parameter and that it returns a `SearchStrategy` def int_strategy() -> SearchStrategy[int]: ...  Don't believe me? Ask Mypy: # At call site reveal_type(int_strategy) # () -> SearchStrategy[int] reveal_type(int_strategy()) # SearchStrategy[int]  This is the same with other decorators: The eventual type of a function is determined by its original type hints and all of its @decorators'. In composite()'s case, this is how it is defined (at least at type-checking time): # Takes a function whose first argument is of type `DrawFn` # and returns a function without that argument, # returning a `SearchStrategy` that will output # values of the same type as the original's return type. def composite( f: Callable[Concatenate[DrawFn, P], Ex] ) -> Callable[P, SearchStrategy[Ex]]: ...  In fact, using SearchStrategy[] as the return type is such a common mistake that the maintainers made it so that you would get a runtime warning: @composite def int_strategy() -> SearchStrategy[int]: ...  tests/test_foo.py:6 /project/tests/test_foo.py:6: HypothesisWarning: Return-type annotation is `st.SearchStrategy[int]`, but the decorated function should return a value (not a strategy)
Question: mypy throws an error when I have an editable installation (pip install -e .) of my library. It works fine with the non-editable installation (pip install .). I was able to reproduce it with a toy example, so here are the files: . ├── src │ └── my_ns │ └── mylib │ ├── __init__.py │ ├── main.py │ ├── py.typed │ └── second.py ├── mypy.ini └── pyproject.toml  main.py def something() -> None: print("I am something")  second.py from my_ns.mylib.main import something def something_else() -> None: something() print("I am something else")  pyproject.toml [build-system] requires = ["setuptools", "setuptools-scm"] build-backend = "setuptools.build_meta" [project] name = "mylib" requires-python = ">=3.10" version = "0.1.0" [tool.setuptools.packages.find] where = ["src"] [tool.setuptools.package-data] "*" = ["*py.typed"]  mypy.ini [mypy] namespace_packages = True explicit_package_bases = True exclude = (?x)( ^tests/ # ignore everything in tests directory | ^test/ # ignore everything in test directory | ^setup\.py$ # ignore root's setup.py )  my_ns is a namespace package, so it does by intention not include a __init__.py (and must remain a namespace). This is the result when running mypy 1.10.0: $ mypy --config-file mypy.ini . src/my_ns/mylib/main.py: error: Source file found twice under different module names: "src.my_ns.mylib.main" and "my_ns.mylib.main" Found 1 error in 1 file (errors prevented further checking)  How can I make mypy work with an editable install and support namespace packages?

Answer: Add the following to your mypy.ini: mypy_path = src  (Credit goes to Mario Ishac, who got this from a GitHub issue comment.)
Question: I'm trying to debug some Django library code with the default VS Code Python and Django debugging settings (and "justMyCode" = False). I've set a breakpoint in one of the library functions:  I call this from some user code, eg. formset.save(). When I debug and hit the breakpoint, VS Code jumps to this user code instead of the library code as I'd have expected:  Pressing "Step Into" seems to progress the library code, but keeps jumping back to the user code which is calling it. The call stack seems to know about the library code, although it is greyed out:  If I click on save then the currently executing statement is highlighted:  I can go through the tedious process of stepping through, clicking the top stack frame, and then doing usual things like inspecting locals. The issue just seems to be about where VS Code is jumping to every time the debugger breaks (or maybe where pdb is telling VS Code to jump -- I'm not sure). I want the library code which is currently being executed to be jumped to, and that's what I would have expected to happen.

Answer: I see the same thing. Looks like a recent bug in VSCode. I rolled back to VSCode version 1.89 and it worked as expected. This was fixed in version 1.90.1 (list of addressed issues).
Question: Suppose I have a numpy array [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16], How do I take 4 elements every 8 elements). Here is the expected result: a -> [1,2,3,4, 9,10,11,12] b -> [5,6,7,8, 13,14,15,16]  My array has hundreds of elements. I went through the numpy array documentation but I never succeeded to perform this computation other then a loop which is very slow. EDIT: The array can have up to 3 interleave sub-array of 4 elements 4 elt sample0, 4 elt sample 1, 4 elt sample2, 4 elt sample0, 4 elt sample 1, 4 elt sample2, 4 elt sample0, 4 elt sample 1, 4 elt sample2 ...  My array has 499875840 elements !

Answer: For a generic and pure numpy approach, you could argsort then split: N = 4 # number of consecutive elements M = 2 # number of output arrays idx = np.argsort(np.arange(len(arr))%(N*M)//N, kind='stable') # array([ 0, 1, 2, 3, 8, 9, 10, 11, 4, 5, 6, 7, 12, 13, 14, 15]) a, b = np.split(arr[idx], M)  As a one liner: out = np.split(arr[np.argsort(np.arange(len(arr))%(N*M)//N, kind='stable')], M)  Output: # a / out[0] array([ 1, 2, 3, 4, 9, 10, 11, 12]) # b / out[1] array([ 5, 6, 7, 8, 13, 14, 15, 16])  Output with arr = np.arange(32) as input: # a array([ 0, 1, 2, 3, 8, 9, 10, 11, 16, 17, 18, 19, 24, 25, 26, 27]) # b array([ 4, 5, 6, 7, 12, 13, 14, 15, 20, 21, 22, 23, 28, 29, 30, 31])  Output with arr = np.arange(32), N = 4, M = 4: (array([ 0, 1, 2, 3, 16, 17, 18, 19]), array([ 4, 5, 6, 7, 20, 21, 22, 23]), array([ 8, 9, 10, 11, 24, 25, 26, 27]), array([12, 13, 14, 15, 28, 29, 30, 31])  timings Paul's approach is faster than mine (but limited to 2 arrays as output).  generalization A reshaping approach, as proposed by @hpaulj, can be generalized using: N = 4 # number of consecutive elements M = 3 # number of samples/output arrays out = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M) # or to map to individual variables a,b,c = arr.reshape(-1, M, N).transpose(1, 0, 2).reshape(-1, arr.size//M)  @U13-Forward's approach only works when M = 2, if can however be generalized using a list comprehension: N = 4 # number of consecutive elements M = 3 # number of samples/output arrays reshaped = arr.reshape(-1, N*M) out = [reshaped[:, n*N:n*N+N].ravel() for n in range(M)]