eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: This is my dataframe: df = pd.DataFrame({'a': [20, 21, 333, 444], 'b': [20, 20, 20, 20]})  I want to create column c by using this mask: mask = (df.a >= df.b)  And I want to get the last row that meets this condition and create column c. The output that I want looks like this:  a b c 0 20 20 NaN 1 21 20 NaN 2 333 20 NaN 3 444 20 x  I tried the code below but it didn't work: df.loc[mask.cumsum().gt(1) & mask, 'c'] = 'x' 

Answer: For a mask to flag the last value satisfying a condition, use duplicated() by keeping last. We know that mask consists of at most 2 values (True/False). If we can create another mask that flags the last occurrences these values as True, then we can chain it with mask itself for the desired mask. This is accomplished by ~mask.duplicated(keep='last') because mask.duplicated(keep='last') flags duplicates as True except for the last occurrence, so its negation gives us what we want. df = pd.DataFrame({'a': [20, 21, 333, 444], 'b': [20, 20, 20, 20]}) mask = (df.a >= df.b) df['c'] = pd.Series('x', df.index).where(mask & ~mask.duplicated(keep='last'))  If you want to slice/assign, then you can use this chained mask as well. df.loc[mask & ~mask.duplicated(keep='last'), 'c'] = 'x'   A shorter version of @mandy8055's answer is to call idxmax() to get the index of the highest cum sum (although this is showing a FutureWarning on pandas 2.1.0). As pointed out by @mozway, this works as long as there's at least one True value in mask. df.loc[mask.cumsum().idxmax(), 'c'] = 'x'
Question: I'm trying to have a boolean variable flip (True becomes False, False becomes True), and this variable is inside of a function. However, I have an issue where I either have to assign the variable inside the function (thus having the variable reset to what I assigned it to inside the function), or don't do that, which causes an UnboundLocalError. Any help on this would be great. I was creating a function, click(), which flips a boolean every iteration of a loop def click(): alternate = True #not doing this will cause an error alternate = not alternate #flipping the variable  and then having a loop run using the function while True: click() print(alternate) #constantly prints False  while what I wanted was to print True, False, True, etc.

Answer: python does not have pass by reference. it has mutable and immutable variables. so if your variable is mutable you need to return the value each time and reset it like this: def click(alternate): alternate = not alternate #flipping the variable return alternate  and: alternate = True while True: alternate=click(alternate ) print(alternate) #constantly prints False
Question: Whilst computing SVD I encountered strange behaviour in relation to np.reciprocal. Under certain conditions (i.e. additional steps on an unrelated variable or printing a variable) the output changes for some reason. The following is a simplified version of the code. import numpy as np from numpy.linalg import eig from math import sqrt def ComputeSVD(A): m, n = A.shape eigenvalues, eigenvectors = eig(A.T @ A) Sigma = np.zeros((m,n)) np.fill_diagonal(Sigma, np.sqrt(eigenvalues)) print(Sigma) U = np.zeros((m,m)) for i in range(len(eigenvalues)): U[:,i] = A @ (eigenvectors[:,i]) return Sigma def PseudoInverse(A): S = ComputeSVD(A) S_inv = np.reciprocal(S, where= (S != 0)) return S_inv A = np.array([[1.0, 1.0, 1.0], [1.0, 2.0, 3.0], [1.0, 4.0, 9.0]]) print(PseudoInverse(A))  The output is incorrect for the non-diagonal entries: [[10.64956309 0. 0. ] [ 0. 1.2507034 0. ] [ 0. 0. 0.15015641]] [[ 0.09390057 -0.74904542 0.64830637] [-0.34457359 0.79955008 -0.73997748] [-0.92878374 0.32438956 6.65972239]]  However when removing the parts computing related to U import numpy as np from numpy.linalg import eig from math import sqrt def ComputeSVD(A): m, n = A.shape eigenvalues, eigenvectors = eig(A.T @ A) Sigma = np.zeros((m,n)) np.fill_diagonal(Sigma, np.sqrt(eigenvalues)) print(Sigma) return (Sigma) def PseudoInverse(A): #same as before A = np.array([[1.0, 1.0, 1.0], [1.0, 2.0, 3.0], [1.0, 4.0, 9.0]]) print(PseudoInverse(A))  Now the Output is correct: [[10.64956309 0. 0. ] [ 0. 1.2507034 0. ] [ 0. 0. 0.15015641]] [[0.09390057 0. 0. ] [0. 0.79955008 0. ] [0. 0. 6.65972239]]  Even stranger is that when we leave the parts relating to U but, before we invert the non-zero entries we, print(S) the issue is again rectified. import numpy as np from numpy.linalg import eig from math import sqrt def ComputeSVD(A): #Same as first time def PseudoInverse(A): S = ComputeSVD(A) print(S) S_inv = np.reciprocal(S, where= (S != 0)) return S_inv A = np.array([[1.0, 1.0, 1.0], [1.0, 2.0, 3.0], [1.0, 4.0, 9.0]]) print(PseudoInverse(A))  Now the correct output is produced again. (see the second output)

Answer: The line  S_inv = np.reciprocal(S, where=(S != 0))  creates an uninitialized array and fills the entries where the condition S != 0 is True with reciprocals of the corresponding elements of S. All other elements are left unchanged, so they may be zeroes, but may also have some random values. In order to fix this, use the out argument to initialize an array where the result will be stored:  S_inv = np.reciprocal(S, where=(S != 0), out=np.zeros_like(S))
Question: I have a pandas data frame with a column where a condition is met based on other elements in the data frame (not shown). Additionally, I have a column that extends the validness one row further with the following rule: If a valid row is followed directly by ExtendsValid, that row is also valid, even if the underlying valid condition doesnt apply. Continue filling valid forward as long as ExtendsValid is 1 I have illustrated the result in column "FinalValid" (desired result. Doesnt have to be a new column, can also fill Valid forward). Note that rows 8 and 9 in the example also become valid. Also note that row 13 does NOT result in FinalValid, because you need a preceding valid row. Preceding valid row can be Valid or an extended valid row. So far if I had a problem like that I did a cumbersome multi-step process:  Create a new column for when "Valid" or "ExtendsValid" is true Create a new column marking the start point for each "sub-series" (a consecutive set of ones) Number each sub-series fillna using "group by" for each sub series  I can provide sample code but I am really looking for a totally different, more efficient approach, which of course must be non-iterating as well. Any ideas would be welcome.     # Valid ExtendsValid FinalValid     1 0 0 0   2 1 0 1   3 0 0 0   4 0 0 0   5 1 0 1   6 0 0 0   7 1 0 1   8 0 1 1   9 0 1 1   10 0 0 0   11 1 0 1   12 0 0 0   13 0 1 0   14 0 0 0    

Answer: IIUC, you want to ffill the 1s only if there is an uninterrupted series of 1s starting on Valid and eventually continuing on ExtendsValid. For this you can use a groupby.cummin: df['FinalValid'] = ( (df['Valid']|df['ExtendsValid']) .groupby(df['Valid'].cumsum()) .cummin() )  Output: NB. I slightly modified the input on row 3 to better illustrate the logic.  # Valid ExtendsValid FinalValid 0 1 0 0 0 1 2 1 0 1 2 3 0 0 0 3 4 0 1 0 4 5 1 0 1 5 6 0 0 0 6 7 1 0 1 7 8 0 1 1 8 9 0 1 1 9 10 0 0 0 10 11 1 0 1 11 12 0 0 0
Question: I have asked about finding sum of higher elements in the row/column and got really good answer. However this approach does not allow me to manipulate current element. My input dataframe is something like this: array([[-1, 7, -2, 1, 4], [ 6, 3, -3, 5, 1]])  Basically, I would like to have a output matrix which shows me for each element how many values are higher in the given row and column, like this: array([[3, 0, 4, 2, 1], [0, 2, 4, 1, 3]], dtype=int64)  scipy ranked function really works well here. (Thanks to @Tom) the tricky part is here since this matrix is correlation matrix and scores are between -1 and 1, I would like to add one middle step (normalization factor) before counting higher values: If the element is negative, add +3 to that element and then count how many values are higher If the element is positive, subtract -3 from that element and then count how many values are higher in the row. e.g.: first element of row is negative we add +3 and then row would be 2 7 -2 1 4 -> sum of the higher values from that element is 2 second element of row is positive we subtract -3 and then row would be -1 4 -2 1 4 -> sum of the higher values from that element is 0 ... so we do this normalization for each row and row-wise desired output would be: 2 0 2 3 1 1 3 4 2 3  I don't want to use loop for that because since the matrix is 11kx12k, it takes so much time. If I use ranked with lamda, than instead of doing for each element, It adds and subtracts in the same time to the all row values, which It is not what I want. corr = np.array([[-1, 7, -2, 1, 4], [ 6, 3, -3, 5, 1]]) def element_wise_bigger_than(x, axis): return x.shape[axis] - rankdata(x, method='max', axis=axis) ld = lambda t: t + 3 if t<0 else t-3 f = np.vectorize(ld) element_wise_bigger_than(f(corr), 1) 

Answer: A possible solution, based on numba and numba prange to parallelize the for loop: from numba import jit, prange, njit, set_num_threads import numpy as np @njit(parallel=True) def get_horizontal(a): z = np.zeros((a.shape[0], a.shape[1]), dtype=np.int32) for i in prange(a.shape[0]): for j in range(a.shape[1]): aux = a[i, j] if a[i, j] < 0: a[i, j] += 3 elif a[i, j] > 0: a[i, j] -= 3 else: pass z[i, j] = (a[i, j] < a[i, :]).sum() a[i, j] = aux return z a = np.array([[-1, 7, -2, 1, 4], [ 6, 3, -3, 5, 1]]) set_num_threads(6) # to use only 6 threads get_horizontal(a)   Runtime: By using the following array, a = np.random.randint(-10, 10, size=(11000, 12000))  the runtime, on my computer, is less than 1 minute.  Output: array([[2, 0, 2, 3, 1], [1, 3, 4, 2, 3]], dtype=int32)
Question: I am a bit confused about the difference in behavior between __doc__ and other methods: # python 3.10.13 class Parent: def __init__(self, doc): self._doc = doc def __doc__(self): return self._doc def __other__(self): return self._doc class Child(Parent): pass >>> print(Parent("test").__doc__()) test >>> print(Child("test").__doc__) None # TypeError if call __doc__() since it is not defined >>> print(Child("test").__other__()) test  But it does not work as property either: # python 3.10.13 class Parent: def __init__(self, doc): self._doc = doc @property def __doc__(self): return self._doc @property def __other__(self): return self._doc class Child(Parent): pass >>> print(Parent("test").__doc__) test >>> print(Child("test").__doc__) None >>> print(Child("test").__other__) test  This is mentioned and somewhat discussed here @22750354, but I need help finding out why the Child class does not inherit the method/property of __doc__. And are there other methods like __doc__ in terms of inheritance? Thank you.

Answer: As a first approximation1, when you write obj.a  Python attempts to lookup a by  checking whether obj has an attributed named a, then checking whether obj's class (obj.__class__) has an attribute a, then recursively checking each parent class (obj.__class__.__mro__) for an attribute a.  If at any point, an attribute named a is found, that then that object gets returned as the result. This is what is inheritance means in Python. Child classes don't copy their parents, they just delegate attribute lookups to them. Now, whenever you define a class as class Child(Parent): ...  that class object always gets assigned a __doc__ attribute: the docstring if one was provided, or None otherwise. >>> Child.__doc__ is None True  Because of this, any lookups for the attribute __doc__ will only ever propagate up to the class level before a matching object is found. Attribute lookup will never reach the stage of checking the parent classes. 1The complete attribute lookup process is much more complicated. See 3.3.2. Customizing attribute access from the Python Data Model docs for the bigger picture.
Question: I am using PyQt5. I want to know how to make my QComboBox open with ±10 items instead of the full screen. This only happens with the fusion style applied. Is there any way I can make this behave with a small drop down instead? I have tried to use setMaxVisibleItems(5), but it didn't make a difference. Here is what it is looking like now: 

Answer: As pointed out in QTBUG-89037, there's an undocumented stylesheet property that can be used to change the behaviour of the popup: setStyleSheet('QComboBox {combobox-popup: 0}')  A value of 0 will show the normal scrollable list-view with a maximum number of visible items, whilst a value of 1 shows the humungous menu. However, in some circumstances, setting a stylesheet may have undesirable side-effects, since it could potentially interfere with the default behaviour of QStyle. So an alternative solution might be to use a proxy-style, and reimplement its styleHint method so that it returns either 0 or 1 for QStyle.SH_ComboBox_Popup. Both of these solutions work with both Qt5 and Qt6, and will also work for styles such as "cleanlooks" as well as "fusion". Here's a simple demo that implements both solutions:  from PyQt5 import QtWidgets # from PyQt6 import QtWidgets class Window(QtWidgets.QWidget): def __init__(self): super().__init__() self.combo = QtWidgets.QComboBox() self.combo.addItems(f'Item-{i:02}' for i in range(1, 50)) layout = QtWidgets.QVBoxLayout(self) layout.addWidget(self.combo) class ProxyStyle(QtWidgets.QProxyStyle): def styleHint(self, hint, option, widget, data): if hint == QtWidgets.QStyle.StyleHint.SH_ComboBox_Popup: return 0 return super().styleHint(hint, option, widget, data) if __name__ == '__main__': app = QtWidgets.QApplication(['Combo Test']) app.setStyle('fusion') # app.setStyle('cleanlooks') # style = ProxyStyle(app.style()) # app.setStyle(style) app.setStyleSheet('QComboBox {combobox-popup: 0}') window = Window() window.setGeometry(600, 100, 200, 75) window.show() app.exec()
Question: I have this code repo I created manual UML which look like this:  I am trying to auto generate the UML via pyreverse: pyreverse -o png -p ShoppingCart ./mainService.py  Format png is not supported natively. Pyreverse will try to generate it using Graphviz...  Unfortunately, it gives me blank diagram. What can I do to get the project's classes in the diagram? This is the file structure: . ├── Entity │ ├── Apple.py │ ├── Buy1Get1FreeApple.py │ ├── Buy3OnPriceOf2Orange.py │ ├── Offer.py │ ├── Orange.py │ ├── Product.py │ └── ShoppingCart.py ├── Enum │ └── ProductType.py └── mainService.py 

Answer: In short Assuming the installation of pyreverse and graphviz is correct, all you need to do is to package your project adding some emplty __init__py files in each folder. Alternatively, you'd hjhave to add all the modules manually in the command line. More details - step by step About the error message Assuming everything is installed correctly, your command line should give you the warning message, which is absolutely normal:  Format png is not supported natively. Pyreverse will try to generate it using Graphviz...  What's wrong? The diagram will stay empty because you tell pyreverse to analyse a single file and there is no class defined in that file. If you'd add manually the different modules to be analysed: pyreverse -o png -p ShoppingCart mainService.py Entity\Apple.py Enum\ProductType.py Entity\Orange.py Entity\ShoppingCart.py  you would then very well obtain a rudimentary diagram:  If you would add the options -AS upfront you'd get all ancestors in the project and all associated classes recursively:  How to package your project? This is cumbersome. Fortunately, there's little missing to package your project. For a full reference, you may look here. But in short, it is sufficient to add an empty __init__.py file in your project folder, and each subfolder where you store the modules: . ├── Entity │ ├── __init__.py <<===== add this empty file │ ├── Apple.py │ ├── Buy1Get1FreeApple.py │ ├── Buy3OnPriceOf2Orange.py │ ├── Offer.py │ ├── Orange.py │ ├── Product.py │ └── ShoppingCart.py ├── Enum │ ├── __init__.py <<===== add this empty file │ └── ProductType.py ├── __init__.py <<===== add this empty file └── mainService.py  You will then be able to run the simpler command line: pyreverse -AS -o png -p ShoppingCart .  and obtain this magnificient diagram:  The packaging helps python and pyreverse to understand that these are not files to be analysed in isolation, but in the context of a package made of subpackages, etc.
Question: I'm try to filter rows in some method so I need the output model to be of the same type as input model to the sqlAlchemy query. I followed this answer https://stackoverflow.com/a/38160409/1374078 . However would it be possible to get the original model, so that I can access the model's methods by name? e.g. row.foo_field, otherwise getting generic row type > type(row) <class 'sqlalchemy.engine.row.Row'> 

Answer: Assuming that your code looks something like this*: with orm.Session(engine) as s: row_number_column = ( sa.func.row_number() .over(partition_by=User.name, order_by=sa.desc(User.id)) .label('row_number') ) q = sa.select(User) q = q.add_columns(row_number_column) for row in s.execute(q): print(row)  Then the results look like this: (<__main__.User object at 0x7fb6e2fe5580>, 1) (<__main__.User object at 0x7fb6e2fe5670>, 1) (<__main__.User object at 0x7fb6e2fe56a0>, 1) (<__main__.User object at 0x7fb6e2fe56d0>, 1) (<__main__.User object at 0x7fb6e2fe5700>, 1)  As you can see, each result row is a tuple of (user_instance, row_number) so you can access the user instance as row[0], or unpack during iteration:  for user, row in s.execute(q): print(user, row_number)   * This is a SQLAlchemy 2.0-style query, but the legacy Query style query in the Q&A linked in the question produces the same result.
Question: I have a two dimensional numpy array where some rows may have nans. I want to select the occurrence or absence of nans in rows of these arrays as per the following prescription:  If a row does not start with a nan, then the result for that array will be -1. If a row starts with a nan, then the result will be the index of the last nan in the continuous unbroken sequence of nans which started at the beginning of that row.  What is the most optimal way of doing this? In my actual work, I will be dealing with numpy arrays with millions of rows. As an example lets consider the below array: import numpy as np arr = np.array([[1,11,np.nan,111,1111], [np.nan, np.nan, np.nan, 2, 22], [np.nan, np.nan, 3, 33, np.nan], [4, np.nan, np.nan, 44, 444], [np.nan, 5, 55, np.nan, 555], [np.nan, np.nan, np.nan, np.nan, np.nan]])  Here the expected result will be result = [-1, 2, 1, -1, 0, 4]. Below is a successful code that I have tried. But, I would like a more optimal solution. result = [] for i in range(arr.shape[0]): if np.isnan(arr[i])[0] == False: result += [-1] elif np.all(np.isnan(arr[i])): result += [arr.shape[1]-1] else: result += [np.where(np.isnan(arr[i]) == False)[0][0] - 1] 

Answer: You can add a column of non-nan with hstack, check which values are nan with isnan and get the position of the first non-nan with argmin: out = np.isnan(np.hstack([arr, np.ones((arr.shape[0], 1))])).argmin(axis=1)-1  Or without concatenation and using where to fix the case in which all values are nan: tmp = np.isnan(arr) out = np.where(tmp.all(axis=1), arr.shape[1], tmp.argmin(axis=1))-1  Output: out = array([-1, 2, 1, -1, 0, 4])
Question: I have been trying to add key values of a list into a dict whereas the key is the amount of times X is repeated in the list, and the value is X itself. my_list = ["apple", "cherry", "apple", "potato", "tomato", "apple"] my_grocery = {} while True: try: prompt = input().upper().strip() my_list.append(prompt) except EOFError: my_list_unique = sorted(list(set(my_list))) for _ in my_list_unique: my_grocery[my_list.count(_)] = _ #print(f'{my_list.count(_)} {_}') print(my_grocery) break  The expected output was: {3: APPLE, 1: CHERRY, 1: POTATO, 1: TOMATO}  The the actual output received was: {3: 'APPLE', 1: 'TOMATO'}  Does anyone have any idea why is that

Answer: you couldn't have duplicated keys in dict, in your case it's "1", you can use Counter for vise versa key-value savings occurrences of each product type from collections import Counter my_list = [] while True: try: prompt = input().strip() if not prompt: break my_list.append(prompt) except EOFError: break item_counts = Counter(my_list) print(item_counts)   Counter({'tomato': 2, 'apple': 2, 'mango': 1})
Question: I have the following input vector. ['a','b','c']  I want to list all possible combinations. There are three restrictions:  The values have to be inserted into an output vector of six positions. One given value from the input vector can only occur once in the output vector. The order of the values has to be the same in the output vector as in the input vector.  Two permissions:  Positions can be left empty. (Not shown here:) The input vector can have more than one of any given value (such as ['a','b','a','c'])  Given this input vector above, the only valid outputs are the following (I might have missed one or two but you get the idea): [' ',' ',' ','a','b','c'], [' ',' ','a',' ','b','c'], [' ','a',' ',' ','b','c'], ['a',' ',' ',' ','b','c'], [' ',' ','a','b',' ','c'], [' ','a','b',' ',' ','c'], ['a','b',' ',' ',' ','c'], [' ','a',' ','b',' ','c'], ['a',' ','b',' ',' ','c'], ['a',' ',' ','b',' ','c'], [' ',' ','a','b','c',' '], [' ','a','b','c',' ',' '], ['a','b','c',' ',' ',' '], [' ','a',' ','b','c',' '], ['a',' ','b','c',' ',' '], ['a',' ','b',' ','c',' ']  My first idea was to generate circa 6! vectors where the vectors are random combinations of the possible values, including the empty value: [abc ]|[abc ]...[abc ] and then remove all vectors where a/b/c occur more than they do in the input vector and the order of abc is not the same as in the input vector. But this brute force measure would take ages. I'll have to do this a lot, and for input vectors and output vectors of varying sizes.

Answer: Choosing 3 out of the 6 indices and placing the elements there: from itertools import combinations v = ['a','b','c'] n = 6 for I in combinations(range(n), len(v)): out = [' '] * n for i, out[i] in zip(I, v): pass print(out)
Question: I'm having some problems with mypy. I have an abstract class and a class that inherits from it: from __future__ import annotations from abc import abstractmethod, ABC from typing import Union class Base(ABC): @abstractmethod def the_method(self, a_class: Union[Base, float, int]) -> None: ... @abstractmethod def other_method(self) -> None: ... class MyClass(Base): def __init__(self, something: str = "Hello") -> None: self.something = something def the_method(self, a_class: Union[MyClass, float, int]) -> None: print(a_class) def other_method(self) -> None: print(self.something)  I am aware of the Liskov substitution principle. However MyClass is a type of Base since it inherits from it. But mypy still raises an error: main.py:21: error: Argument 1 of "the_method" is incompatible with supertype "Base"; supertype defines the argument type as "Base | float | int" [override] main.py:21: note: This violates the Liskov substitution principle main.py:21: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides Found 1 error in 1 file (checked 1 source file)  What am I doing wrong?

Answer: Base.the_method accepts Base, so subclasses need to accept at least Base too. If I have a class Foo(Base), that should be accepted. At the moment, it's not accepted by MyClass.the_method. Liskov substitution works in the opposite direction for method parameters and return types - superclasses are OK for parameters, subclasses are OK for return types. You have lots of options. For example, you could make MyClass.the_method accept Base class MyClass(Base): def the_method(self, a_class: Union[Base, float, int]) -> None: ...  or (and there's a discussion in the comments about whether this next example also violates LSP ... either way, mypy is overly permissive with this) change Base to accept instances of the enclosing class from typing import Self class Base(ABC): @abstractmethod def the_method(self, a_class: Union[Self, float, int]) -> None: ...