eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Enum datatypes are good abstractions for enumerable datatype like days of week, months etc. Nevertheless the simplest tests show that we pay 2.5 slower performance for such datatypes. Do we have any explanation for such behavior? Consider two simple enums in Python import enum import timeit class IntDow(enum.Enum): MONDAY=enum.auto() TUESDAY=enum.auto() WEDNESDAY=enum.auto() THURSDAY=enum.auto() FRIDAY=enum.auto() SATURDAY=enum.auto() SUNDAY=enum.auto() class StrDow(str, enum.Enum): MONDAY="MONDAY" TUESDAY="TUESDAY" WEDNESDAY="WEDNESDAY" THURSDAY="THURSDAY" FRIDAY="FRIDAY" SATURDAY="SATURDAY" SUNDAY="SUNDAY" int_mon = IntDow["MONDAY"] int_tue = IntDow["TUESDAY"] str_mon = StrDow["MONDAY"] str_tue = StrDow["TUESDAY"] raw_mon = "MONDAY" raw_tue = "TUESDAY"  and carry out the simplest equality tests >>> timeit.timeit(lambda: int_mon == IntDow.MONDAY, number=100000) 0.017555099999299273 >>> timeit.timeit(lambda: int_mon == int_tue, number=100000) 0.00824300000022049 >>> timeit.timeit(lambda: int_mon == IntDow.TUESDAY, number=100000) 0.018771999999444233 >>> timeit.timeit(lambda: str_mon == StrDow.MONDAY, number=100000) 0.01836639999964973 >>> timeit.timeit(lambda: str_mon == StrDow.TUESDAY, number=100000) 0.01744440000038594 >>> timeit.timeit(lambda: str_mon == str_tue, number=100000) 0.007430400000885129 >>> timeit.timeit(lambda: raw_mon == "MONDAY", number=100000) 0.007222599999295198 >>> timeit.timeit(lambda: raw_mon == "TUESDAY", number=100000) 0.00726819999908912 >>> timeit.timeit(lambda: raw_mon == raw_tue, number=100000) 0.007780600000842242  We can see that variable to constant comparison for enum is approximately 2.5 times slower than for str. Do we have any explanation for such behavior?

Answer: The bulk of the extra time is not spent in the equality test, but in looking up the member from the enum (i.e. IntDow.MONDAY). In those cases where performance is critical, export the members from the enum first: MONDAY, TUESDAY, ... = IntDow   Disclosure: I am the author of the Python stdlib Enum, the enum34 backport, and the Advanced Enumeration (aenum) library.
Question: there have been explanations about the different between miniforge and miniconda  miniforge is the community (conda-forge) driven minimalistic conda installer. Subsequent package installations come thus from conda-forge channel. miniconda is the Anaconda (company) driven minimalistic conda installer. Subsequent package installations come from the anaconda channels (default or otherwise).  as for mambaforge, mambaforge-pypy3, miniforge, miniforge-pypy3, how do we choose which package to install?

Answer: mamba* use the c/c++ implementation of the conda protocol "mamba" instead of the python implementation which is called conda. The *pypy3 variants ship with PyPy as the python implementation in the base environment instead of CPython.
Question: I have the follwing df: df = pd.DataFrame({ 'col1': [1, np.nan, np.nan, np.nan, 1, np.nan, np.nan, np.nan], 'col2': [np.nan, 2, np.nan, np.nan, np.nan, 2, np.nan, np.nan], 'col3': [np.nan, np.nan, 3, np.nan, np.nan, np.nan, 3, np.nan], 'col4': [np.nan, np.nan, np.nan, 4, np.nan, np.nan, np.nan, 4] })  It has the following display:  col1 col2 col3 col4 0 1.0 NaN NaN NaN 1 NaN 2.0 NaN NaN 2 NaN NaN 3.0 NaN 3 NaN NaN NaN 4.0 4 5.0 NaN NaN NaN 5 NaN 6.0 NaN NaN 6 NaN NaN 7.0 NaN 7 NaN NaN NaN 8.0  My goal is to keep all rows begining with float (not NaN value) and join to them the remaining ones. The new_df I want to get is:  col1 col2 col3 col4 0 1 2 3 4 4 5 6 7 8  Any help form your side will be highly appreciated (I upvote all answers). Thank you!

Answer: If need join first values per groups defined by non missing values in df['col1'] use: df = (df.reset_index() .groupby(df['col1'].notna().cumsum()) .first() .set_index('index'))
Question: Considering the following problem import openmdao.api as om class Sys(om.Group): def setup(self): self.add_subsystem('sys1', om.ExecComp('v1 = a + b'), promotes=['*']) self.add_subsystem('sys2', om.ExecComp('v2 = v1 + c'), promotes=['*']) if __name__ == '__main__': prob = om.Problem() model = prob.model comp = model.add_subsystem('comp', Sys(), promotes=['*']) prob.setup() prob.run_model() comp.list_inputs()  the list_inputs command gives the following 4 Input(s) in 'comp' varname val ------- ---- sys1 a [1.] b [1.] sys2 c [1.] v1 [2.]  However we can clearly see that v1 in an 'internal' input to the system. If we were to attach this to an IndepVarComp or another system, we would not have to provide v1 since it is already internally connected. Is there a function that can list the inputs that are unconnected or that must be provided to a Group?

Answer: Generally, I prefer to rely on the visual tools such as the N2. However, here is a scriptable solution that I use on occation. Fair warning, it requires the use of one non-public attribute of system... but this is how I do it: import openmdao.api as om class Sys(om.Group): def setup(self): self.add_subsystem('sub_sys1', om.ExecComp('v1 = a + b'), promotes=['*']) self.add_subsystem('sub_sys2', om.ExecComp('v2 = v1 + c'), promotes=['*']) if __name__ == '__main__': prob = om.Problem() model = prob.model comp = model.add_subsystem('comp', Sys(), promotes=['*']) prob.setup() prob.run_model() comp_inputs = prob.model.list_inputs(out_stream=None, prom_name=True) # filter the inputs into connected and unconnected sets connect_dict = comp._conn_global_abs_in2out unconnected_inputs = set() connected_inputs = set() for abs_name, in_data in comp_inputs: if abs_name in connect_dict and (not 'auto_ivc' in connect_dict[abs_name]): connected_inputs.add(in_data['prom_name']) else: unconnected_inputs.add(in_data['prom_name']) print(connected_inputs) print(unconnected_inputs)
Question: I am basically trying to do something like this but without the for-loop... I tried with np.put_along_axis but it requires times to be of dimension 10 (same as last index of src).  import numpy as np src = np.zeros((5,5,10), dtype=np.float64) ix = np.array([4, 0, 0]) iy = np.array([1, 3, 4]) times = np.array([1 ,2, 4]) values = np.array([25., 10., -65.]) for i, time in enumerate(times): src[ix, iy, time] += values[i] 

Answer: One approach is to use np.add.at, preparing the indices first (as below): r = len(values) indices = (np.tile(ix, r), np.tile(iy, r), np.repeat(times, r)) np.add.at(src, indices, np.repeat(values, r)) print(src)  Output [[[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 25. 10. 0. -65. 0. 0. 0. 0. 0.] [ 0. 25. 10. 0. -65. 0. 0. 0. 0. 0.]] [[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] [[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] [[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] [[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 25. 10. 0. -65. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]]
Question: As I understand it, when a user interacts with an interactive matplotlib plot (i.e. by clicking, pressing a key, etc.), an Event is triggered, which can be linked to an arbitrary callback function, if desired. Interactive matplotlib plots often come with a navigation toolbar that includes certain features like zooming and rubberband selection. My question is, is there a way to watch for these things from the backend and react when a user performs one of these actions using the nav bar/mouse? I have gone through the list of event names on the event handling page of matplotlib's documentation, as well as looked over the API reference for the NavigationToolbar2 class, but I haven't been able to find any connection between the two. Is an event even the thing to be looking for, or is there some other way to detect these kinds of interactions?

Answer: Resolved on my own. In addition to the event types and the fig.canvas.mpl_connect() syntax shown on the "event handling" documentation page, you can also associate a callback function with an Axes instance directly, and this way has some different kinds of events that can be used as triggers. The API reference for the Axes class has this to say:  The Axes instance supports callbacks through a callbacks attribute which is a CallbackRegistry instance. The events you can connect to are 'xlim_changed' and 'ylim_changed' and the callback will be called with func(ax) where ax is the Axes instance.  ...and then the syntax to connect these axis events to a user-defined callback func on an existing axis instance ax might look something like this: def func(axes): print("New axis y-limits are", axes.get_ylim()) cb_registry = ax.callbacks cid = cb_registry.connect('ylim_changed', func)  The same approach could be used to watch for x-axis changes as well.
Question: So, essentially, in Python 3.7 (as far as I know) if you try to do this, import asyncio async def sleep(): asyncio.sleep(1) async def main(): tasks = (sleep() for _ in range(5)) for task in asyncio.as_completed(tasks): result = await task if __name__ == "__main__": asyncio.run(main())  It crashes with TypeError: expect a list of futures, not generator  But the type hints clearly specify that it accepts an Iterable, which a Generator is. If you turn tasks into a list, it works, of course, but... what am I missing? And why would it be subjected to lists? I don't see why it should not allow generators.

Answer: You are right. The documentation here is not consistent with the actual behavior. The official documentation refers to the first argument as an "iterable". And typeshed as of today also annotates the first argument with Iterable[...]. However, in the CPython code for as_completed the first argument is passed to coroutines.iscoroutine, which checks, if it is an instance of types.GeneratorType. Obviously, that is what it is, which has it return True and cause the TypeError. And of course a generator is also an iterable. Which means the function does not in fact accept an iterable as the docs claim, but only a non-generator iterable. Maybe someone else here can shine additional light on the background or thought process here. In any case, I would argue this is worth opening an issue over, if one addressing this does not exist yet. EDIT: Apparently (and unsurprisingly) we were not the first ones to notice this. Thanks to @KellyBundy for pointing it out.
Question: I want to convert DataFrame using pandas. I would like to convert it into dictionary format like {'Plant Delivering ID': [Ship-To ID]} there are multiple 'Ship-To ID' for single 'Plant Delivering ID' My Original Data Format:  I would like to convert it to:  How do I convert it?

Answer: Use pandas.DataFrame.groupby and then get the result of groupby with apply(list) at the end convert the result to dict with pandas.Series.to_dict. df.groupby('Plant Delivering ID')['Ship-To-ID'].apply(list).to_dict()
Question:  A static method can be called either on the class (such as C.f()) or on an instance (such as C().f()). Moreover, they can be called as regular functions (such as f()).  Could someone elaborate on the bold part of the extract from the documentation for Python static methods? Reading this description one would expect to be able to do something like this: class C: @staticmethod def f(): print('f') def g(self): f() print('g') C().g()  But this generates: NameError: name 'f' is not defined  My question is not about the use-cases where the static method call is name-qualified either with an instance or a class name. My question is about the correct interpretation of the bold part of the documentation.

Answer: The subsection 4.2.2. Resolution of names of the CPython documentation specifies the rules of visibility for variables defined in different type of nested code blocks. In particular it specifies the working of the visibility of the local class scope's variables inside its methods:  The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods
Question: I am one of the devs of a (fairly large) C++ simulation tool. Disclaimer : I'm more of a physicist than a dev. I wrote Python bindings for that project using pybind11. I managed to get the Python module to compile with cmake. I then managed to write a setup.py file using skbuild that does compile the Python module : python3 setup.py sdist bdist_wheel  In _skbuild/linux-x86_64-3.9/cmake-build/lib/ (and in the tar archive dist/cytosim-0.0.0.tar.gz) there is indeed a compiled library : cytosim.cpython-39-x86_64-linux-gnu.so. However, when I want to install the module : pip3 install dist  I get an error : gcc: error: src/py3/dist.c: No such file or directory  I am very confused because I do not have an directory called py3 in src. Any pointer ? Anything I'm doing wrong ? Thanks !

Answer: The command pip3 install dist  tries (and fails) to install the dist package from the pypi repository. Maybe try pip3 install dist/cytosim-0.0.0.tar.gz  instead.
Question: First of all let's assume the following:  I am building a python package mypackage and want to make it available broadly My package has the following python dependencies: "A","B","C" and "D" and we assume further that each dependency covers an independent use-case of the package (i.e. A is needed for users wanting to do A-type stuff, B is needed for B-type stuff, etc.) A, B, C and D are all pretty heavy and take each tons of times to install. The majority of the package's users are not developers and actually do not even know which type of stuff they will be interested in (whether it is one letter-stuff or any multiple letters simultaneously) and do not know how to do install with options Some users are power-users and know from the get-go that they will only use C and D-stuff so they will only need C or D as dependencies. In fact some of the non developer users might actually turn into power users given enough time to practice  From reading all hypotheses above then it makes perfect sense to have a default install installing A,B, C and D and having options available for power-users to install only C or D (or any package combination such as A and D). Aka:  pip install mypackage => installs A, B, C and D pip install mypackage[C, D] => installs C and D but not A and B  This exact same problem is stated in this other question under the name negative extra_requires. Because indeed the desired behavior is that extra_requires should install fewer packages than the default install. It is also connected to discussions and issues in several places. I wanted to know 1. has the situation changed or is it planned ? 2. what would be a way to circumvent this issue/go about this if not?

Answer:   has the situation changed…?   No.  or is it planned?  Nobody knows. Most probably no.   what would be a way to circumvent this issue…?   Do not install dependencies with plain pip install mypackage. Declare separate extras A, B, C and D. Declare a combined extra all that includes all dependencies: extras_require={ 'A': ['A'], 'B': ['B'], 'C': ['C'], 'D': ['D'], 'all': ['A', 'B', 'C', 'D'], }  Document and teach your users to install the package using pip install "mypackage[all]"  Teach your powerful users to install the package using pip install "mypackage[A, C]"  or in any combination of dependencies they need.
Question: belows is in builtins.pyi  def max(__arg1: _T, __arg2: _T, *_args: _T, key: Callable[[_T], SupportsLessThan]) -> _T:  I do know what the name mangling means and know that the name mangling will influence every "__xxx" identifier as long as in class definition field. So I have three questions:  why there paramters "__arg1" "__arg2" used the "__xxx" pattern and parameter "key" not; what is the function and purpose of "__xxx" pattern specially used in the function's parameter or in this case; I feel awkward to find the implementation of name mangling, and could you tell me where it is in cpython source code.  Many thanks. further work: def wat1(__a): def wat2(__b): return __b, __a return __a print(wat1.__code__.co_varnames) print(wat1.__code__.co_cellvars) dis.dis(wat1)  output(linux python==3.10.8): ('__a', 'wat2') ('__a',) 21 0 LOAD_FAST 0 (__a) 2 RETURN_VALUE 22 0 LOAD_FAST 0 (__b) 2 LOAD_DEREF 0 (__a) 4 BUILD_TUPLE 2 6 RETURN_VALUE 

Answer: Name mangling applies only to names used in a class definition, not function parameters. In this case the leading underscores are only a naming convention to indicate that parameters with such names are not to be passed with keyword arguments, but rather only positional ones. This is to say that you should call max with: max(1, 2)  rather than: max(__arg1=1, __arg2=2)  The key argument on the other hand, are named without an underscore prefix, indicating that it is meant to be passed with a keyword argument: max(1, -2, key=abs)
Question: I wanted to run external programs using python but I receive an error saying I don't have the file the code I wrote: import subprocess subprocess.run(["ls", "-l"])  Output: Traceback (most recent call last): File "C:\Users\hahan\desktop\Pythonp\main.py", line 3, in <module> subprocess.run(["ls", "-l"]) File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 501, in run with Popen(*popenargs, **kwargs) as process: File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 969, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\hahan\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1438, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified  I expected it to return the files in that directory

Answer: The stack trace suggests you're using Windows as the operating system. ls not something that you will typically find on a Windows machine unless using something like CygWin. Instead, try one of these options: # use python's standard library function instead of invoking a subprocess import os os.listdir()  # invoke cmd and call the `dir` command import subprocess subprocess.run(["cmd", "/c", "dir"])  # invoke PowerShell and call the `ls` command, which is actually an alias for `Get-ChildItem` import subprocess subprocess.run(["powershell", "-c", "ls"])
Question: I took the following steps to setup an IPython backend in Google Colab notebook: !pip install ipympl from google.colab import output output.enable_custom_widget_manager()  Then I log the (x,y) location of the user's click on a figure: %matplotlib ipympl import matplotlib import matplotlib.pyplot as plt fig, ax = plt.subplots() def onclick(event): ix, iy = event.xdata, event.ydata print(ix, iy) cid = fig.canvas.mpl_connect('button_press_event', onclick)  I need the code to wait here until the user selects at least one data point. However, the code will run if I have another command in the same notebook, for example: print ('done!')  will run without waiting for the user to pick a data point. I tried using this before: plt.waitforbuttonpress() print('done!')  However, the compiler gets stuck at plt.waitforbuttonpress() and it doesn't let the user click the figure. Thanks for your help

Answer: You could, instead of putting the code you want to run after a button_press_event, after the event listener, you could instead put it in the onclick function. Something like this: %matplotlib ipympl import matplotlib import matplotlib.pyplot as plt fig, ax = plt.subplots() def onclick(event): ix, iy = event.xdata, event.ydata print(ix, iy) print('done!') cid = fig.canvas.mpl_connect('button_press_event', onclick)  Or: %matplotlib ipympl import matplotlib import matplotlib.pyplot as plt fig, ax = plt.subplots() def after_onclick(): print('done!') def onclick(event): ix, iy = event.xdata, event.ydata print(ix, iy) after_onclick() cid = fig.canvas.mpl_connect('button_press_event', onclick)  The reason for this is because mpl_connect works rather oddly (see this question). Instead of waiting for the event to be registered and pausing the code execution, it will run the entire file but keep the event listener open. The problem with this way is that it will run the code every time the event is registered. If that is not what you want and you only want it to run once try this: %matplotlib ipympl import matplotlib import matplotlib.pyplot as plt fig, ax = plt.subplots() def after_onclick(): print('done') fig.canvas.mpl_disconnect(cid) def onclick(event): ix, iy = event.xdata, event.ydata print(ix, iy) after_onclick() cid = fig.canvas.mpl_connect('button_press_event', onclick)  This version adds fig.canvas.mpl_disconnect(cid) so that the code will only run once.
Question: A long time ago I read about the closest true value to zero, like zero = 0.000000001, something like that. In the article they mentioned about this value in Python and how to achieve it. Does anyone knows about this? I have look up here in SO but all the answers are about the closest value to zero of an array and that's not my point.

Answer: The minimum positive denormalized value in Python3.9 and up is given by math.ulp(0.0) which returns 5e-324, or 4.940656e-324 when printed with format(math.ulp(0.0), '.7').
Question: I'm making a program that downloads PDFs from the internet. Here's a example of the code: import httpx # <-- This also happens with the requests module URL = "http://62.182.86.140/main/0/aee7239ffcf7871e1d6687ced1215e22/Markus%20Nix%20-%20Exploring%20Python-Entwickler%20%282005%29.djvu" r = httpx.get(URL, timeout=20.0).content.decode("ascii") with open(f"./example.pdf", "w") as f: f.write(str(content))  But when I write to a file, none of my pdf viewers (tried okular and zathura) can read them. But when I download it using a program like wget, there's no problems. Then when I compare the two files (one downloaded with python, and the other with wget), everything is encoded, and I can't figure out how to decode it (.decode() doesn't work).

Answer: import httpx def main(url): r = httpx.get(url, timeout=20) with open('file.djvu', 'wb') as f: f.write(r.content) main('http://62.182.86.140/main/0/aee7239ffcf7871e1d6687ced1215e22/Markus%20Nix%20-%20Exploring%20Python-Entwickler%20%282005%29.djvu')
Question: I am looking for the correct permutation matrix that would take matrix a and turn it into matrix b given a = np.array([[1,4,7,-2],[3,0,-2,-1],[-4,2,1,0],[-8,-3,-1,2]]) b = np.array([[-4,2,1,0],[3,0,-2,-1],[-8,-3,-1,2],[1,4,7,-2]])  I tried x = np.linalg.solve(a,b)  However, I know this is incorrect and it should be np.array([[0,0,1,0],[0,1,0,0],[0,0,0,1],[1,0,0,0]])  What numpy code would deliver this matrix from the other two?

Answer: Generally, if you have some PA = B and you want P then you need to solve the equation for P. Matrix multiplication is not commutative, so you have to right multiply both sides by the inverse of A. With numpy, the function to get the inverse of a matrix is np.linalg.inv(). Using the matrix multiplication operator @, you can right-multiply with b to get P, taking note that you will end up with floating point precision errors: In [4]: b @ np.linalg.inv(a) Out[4]: array([[-1.38777878e-16, -3.05311332e-16, 1.00000000e+00, -1.31838984e-16], [ 0.00000000e+00, 1.00000000e+00, 6.93889390e-17, 0.00000000e+00], [ 0.00000000e+00, 0.00000000e+00, -1.11022302e-16, 1.00000000e+00], [ 1.00000000e+00, -4.44089210e-16, 5.55111512e-17, 0.00000000e+00]])  As @Mad Physicist points out, you can compare this matrix with > 0.5 to convert it to a boolean matrix: In [7]: bool_P = (b @ np.linalg.inv(a)) > 0.5 In [8]: bool_P @ a Out[8]: array([[-4, 2, 1, 0], [ 3, 0, -2, -1], [-8, -3, -1, 2], [ 1, 4, 7, -2]]) In [9]: bool_P @ a == b # our original equation, PA = B Out[9]: array([[ True, True, True, True], [ True, True, True, True], [ True, True, True, True], [ True, True, True, True]])