eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I don't understand how it works. I don't know if I understood the purpose of this function wrong. I tried to search what posx=event.pos[0] means but all I found was that if you want to take x, write the code of posx,posy=pygame.mouse.get_pos() and then take posx. But I still can't understand the method he followed in the example I saw.

Answer: See pygame.event module. The MOUSEMOTION, MOUSEBUTTONUP and MOUSEBUTTONDOWN events provide a position property pos with the position of the mouse cursor. pos is a tuple with 2 components, the x and y coordinate. e.g.: for event in pygame.event.get(): if event.type == pygame.MOUSEBUTTONDOWN: print("mouse cursor x", event.pos[0]) print("mouse cursor y", event.pos[1])  pygame.mouse.get_pos() returns a Tuple and event.pos is a Tuple. Both give you the position of the mouse pointer as a tuple with 2 components: ex, ey = event.pos mx, my = pygame.mouse.get_pos()  pygame.mouse.getpos() returns the current position of the mouse. The pos attribute stores the position of the mouse when the event occurred. Note that you can call pygame.event.get() much later than the event occurred. If you want to know the position of the mouse at the time of the event, you can call it up using the pos attribute.
Question: I am trying to fill in a gap in my understanding of how Python objects and classes work.  A bare object instance does not support attribute assignment in any way: > object().a = 5 # or > setattr(object(), 'a', 5) AttributeError: 'object' object has no attribute 'a'  I assume that this is because a bare object instance does not possess an __dict__ attribute: > '__dict__' in dir(object()) False  However a bare object instance does have a defined __setattr__ attribute, so this is a bit confusing to me: > '__setattr__' in dir(object()) True   An instance of a regular empty class on the other hand has full ability of attribute assignment: class B(object): pass > B().a = 5 > setattr(B(), 'a', 5)   My question is: what inherent difference between an object instance and a class B(object) instance allows the latter to have assignable attributes, if B inherits directly from object?

Answer: The object() class is like a fundamental particle of the python universe, and is the base class (or building block) for all objects (read everything) in Python. As such, the stated behavior is logical, for not all objects can (or should) have arbitrary attributes set. For example, it wouldn't make sense if a NoneType object could have attributes set, and, just like object(), a None object also does not have a __dict__ attribute. In fact, the only difference in the two is that a None object has a __bool__ attribute. For example: n = None o = object() type(n) >>> <class 'NoneType'> set(dir(n)) - set(dir(o)) >>> {'__bool__'} isinstance(n, object) >>> True bool(n) >>> False  Inheritance from object is automatic, and, just like any other means of inheriting, one can add their own class methods and attributes to the child. Python automatically adds the __dict__ attribute for custom data types as you already showed. In short, it is much easier to add an object's __dict__ attribute than to take it away for objects that do not have custom writable attributes (i.e. the NoneType). Update based on comment: Original comment:  Would it then be safe to assume that it is __setattr__ that checks for the existence of __dict__ and raises an exception accordingly? – bool3max  In CPython, the logic behind object.__setattr__(self, name, value) is implemented by Objects/object.c _PyObject_GenericSetAttrWithDict (see CPython source code). Specifically, it looks like if the name argument is a string, then the object is checked for a __dict__ object in one of its slots and makes sure it is "ready" (see this line). The readiness state of the object is determined by the PyType_Ready(), briefly described and quoted from here:  Defining a Python type in C involves populating the fields of a PyTypeObject struct with the values you care about. We call each of those fields a “slot”. On[c]e the definition is ready, we pass it into the PyType_Ready() function, which does a number of things, inclulding exposing most of the type definition to Python’s attribute lookup mechanism.
Question: just a small problem. I'm using around 6 threads, all of which are printing something every couple of seconds. Occasionally they print on the same line like this: OUTPUT OUTPUT OUTPUTOUTPUT OUTPUT OUTPUT  This leaves an empty line and a double print as you can see. Is there a way that I can make sure this doesn't happen. I saw something saying: print("OUTPUT", end="\n")  This didn't work so I've come back to stack overflow!

Answer: One way to manage this is to have a wrapper function for print and utilise a threading.Lock. Here's an example: from threading import Thread, Lock import time LOCK = Lock() def doprint(msg): with LOCK: print(msg) def athread(): doprint('Thread is starting') time.sleep(0.1) doprint('Thread is ending') threads = [] for _ in range(10): t = Thread(target=athread) t.start() threads.append(t) for t in threads: t.join()  In this way, print() can only ever be invoked by a thread that has successfully acquired the lock
Question: I have a large piece of text that is missing spaces after some of the periods. However the text also contains decimal numbers. Here's what I have so far to fix the problem using regex (I'm using python): re.sub(r"(?!\d\.\d)(?!\. )\.", '. ', my_string) But the first escape group doesn't seem to work. It still matches periods in decimal numbers. Here is sample text to make sure any potential solution works: this is a.match this should also match.1234 and this should 123.match this should NOT match. Has space after period this also should NOT match 1.23 

Answer: You can use re.sub(r'\.(?!(?<=\d\.)\d) ?', '. ', text)  See the regex demo. The trailing space is matched optionally, so if it is there, it will be removed and put back. Details  \. - a dot (?!(?<=\d\.)\d) - do not match any further if the dot before was a dot between two digit  ? - an optional space.  See a Python demo: import re text = "this is a.match\nthis should also match.1234\nand this should 123.match\n\nthis should NOT match. Has space after period\nthis also should NOT match 1.23" print(re.sub(r'\.(?!(?<=\d\.)\d) ?', '. ', text))  Output: this is a. match this should also match. 1234 and this should 123. match this should NOT match. Has space after period this also should NOT match 1.23  Alternatively, use a (?! ) lookahead as in your attempt: re.sub(r'\.(?!(?<=\d\.)\d)(?! )', '. ', text)  See the regex demo and the Python demo.
Question: Let's assume I have the number 100 which I need to divide into N parts each of which shouldn't exceed 30 initially. So the initial grouping would be (30,30,30). The remainder (which is 10) is to be distributed among these three groups by adding 2 to each group in succession, thus ensuring that each group is a multiple of 2. The desired output should therefore look like (34,34,32). Note: The original number is always even. I tried solving this in Python and this is what I came up with. Clearly it's not working in the way I thought it would. It distributes the remainder by adding 1 (and not 2, as desired) iteratively to each group. num = 100 parts = num//30 #Number of parts into which 'num' is to be divided def split(a, b): result = ([a//b + 1] * (a%b) + [a//b] * (b - a%b)) return(result) print(split(num, parts))  Output: [34, 33, 33]  Desired output: [34, 34, 32] 

Answer: Simplified problem: forget about multiples of 2 First, let's simplify your problem for a second. Forget about the multiples of 2. Imagine you want to split a non-necessarily-even number n into k non-necessarily-even parts. Obviously the most balanced solution is to have some parts be n // k, and some parts be n // k + 1. How many of which? Let's call r the number of parts with n // k + 1. Then there are k - r parts with n // k, and all the parts sum up to:  (n // k) * (k - r) + (n // k + 1) * r == (n // k) * (k - r) + (n // k) * r + r == (n // k) * (k - r + r) + r == (n // k) * k + r  But the parts should sum up to n, so we need to find r such that: n == (n // k) * k + r  Happily, you might recognise Euclidean division here, with n // k being the quotient and r being the remainder. This gives us our split function: def split(n, k): d,r = divmod(n, k) return [d+1]*r + [d]*(k-r)  Testing: print( split(50, 3) ) # [17, 17, 16]  Splitting into multiples of 2 Now back to your split_even problem. Now that we have the generic function split, a simple way to solve split_even is to use split: def split_even(n, k): return [2 * x for x in split(n // 2, k)]  Testing: print( split_even(100, 3) ) # [34, 34, 32]  Generalisation: multiples of m It's trivial to do the same thing with multiples of a number m other than 2: def split_multiples(n, k, m=2): return [m * x for x in split(n // m, k)]  Testing: print( split_multiples(102, 4, 3) ) # [27, 27, 24, 24]
Question: Suppose I have the following function: def print_twice(x): for i in x: print(i) for i in x: print(i)  When I run: print_twice([1,2,3])  or: print_twice((1,2,3))  I get the expected result: the numbers 1,2,3 are printed twice. But when I run: print_twice(zip([1,2,3],[4,5,6]))  the pairs (1,4),(2,5),(3,6) are printed only once. Probably, this is because the zip returns a generator that terminates after one pass. How can I modify the function print_twice such that it will correctly handle all inputs? I could insert a line at the beginning of the function: x = list(x). But this might be inefficient in case x is already a list, a tuple, a range, or any other iterator that can be iterated more than once. Is there a more efficient solution?

Answer:  I could insert a line at the beginning of the function: x = list(x). But this might be inefficient in case x is already a list, a tuple, a range, or any other iterator that can be iterated more than once. Is there a more efficient solution?  Copying single-use iterables to a list is perfectly adequate, and reasonably efficient even for multi-use iterables. The list (and to some extend tuple) type is one of the most optimised data structures in Python. Common operations such as copying a list or tuple to a list are internally optimised;1 even for iterables that are not special-cased, copying them to a list is significantly faster than any realistic work done by two (or more) loops. def print_twice(x): x = list(x) for i in x: print(i) for i in x: print(i)  Copying indiscriminately can also be advantageous in the context of concurrency, when the iterable may be modified while the function is running. Common cases are threading and weakref collections.  In case one wants to avoid needless copies, checking whether the iterable is a Collection is a reasonable guard. from collections.abc import Collection x = list(x) if not isinstance(x, Collection) else x  Alternatively, one can check whether the iterable is in fact an iterator, since this implies statefulness and thus single-use. from collections.abc import Iterator x = list(x) if isinstance(x, Iterator) else x x = list(x) if iter(x) is x else x  Notably, the builtins zip, filter, map, ... and generators all are iterators.  1Copying a list of 128 items is roughly as fast as checking whether it is a Collection.
Question: I have a list of 'Id's' that I wish to associate with a property from another list, their 'rows'. I have found a way to do it by making smaller dictionaries and concatenating them together which works, but I wondered if there was a more pythonic way to do it? Code row1 = list(range(1, 6, 1)) row2 = list(range(6, 11, 1)) row3 = list(range(11, 16, 1)) row4 = list(range(16, 21, 1)) row1_dict = {} row2_dict = {} row3_dict = {} row4_dict = {} for n in row1: row1_dict[n] = 1 for n in row2: row2_dict[n] = 2 for n in row3: row3_dict[n] = 3 for n in row4: row4_dict[n] = 4 id_to_row_dict = {} id_to_row_dict = {**row1_dict, **row2_dict, **row3_dict, **row4_dict} print('\n') for k, v in id_to_row_dict.items(): print(k, " : ", v)  Output of dictionary which I want to replicate more pythonically 1 : 1 2 : 1 3 : 1 4 : 1 5 : 1 6 : 2 7 : 2 8 : 2 9 : 2 10 : 2 11 : 3 12 : 3 13 : 3 14 : 3 15 : 3 16 : 4 17 : 4 18 : 4 19 : 4 20 : 4  Desired output Same as my output above, I just want to see if there is a better way to do it?

Answer: This dict-comprehension should do it: rows = [row1, row2, row3, row4] {k: v for v, row in enumerate(rows, 1) for k in row}
Question: I have this dataframe Python 3.9.0 (v3.9.0:9cf6752276, Oct 5 2020, 11:29:23) [Clang 6.0 (clang-600.0.57)] on darwin >>> import pandas as pd >>> import datetime as datetime >>> pd.__version__ '1.3.5' >>> dates = [datetime.datetime(2012, 2, 3) , datetime.datetime(2012, 2, 4)] >>> x = pd.DataFrame({'Time': dates, 'Selected': [0, 0], 'Nr': [123.4, 25.2]}) >>> x.set_index('Time', inplace=True) >>> x Selected Nr Time 2012-02-03 0 123.4 2012-02-04 0 25.2  An integer value from an integer column is converted to a float in the example but I do not see the reason for this conversion. In both cases I assume I pick the value from the 'Selected' column from the first row. What is going on? >>> x['Selected'].iloc[0] 0 >>> x.iloc[0]['Selected'] 0.0 >>> x['Selected'].dtype dtype('int64') 

Answer: x.iloc[0] selects a single "row". A new series object is actually created. When it decides on the dtype of that row, a pd.Series, it uses a floating point type, since that would not lose information in the "Nr" column. On the other hand, x['Selected'].iloc[0] first selects a column, which will always preserve the dtype. pandas is fundamentally "column oriented". You can think of a dataframe as a dictionary of columns (it isn't, although I believe it used to essentially have that under the hood, but now it uses a more complex "block manager" approach, but these are internal implementation details)
Question: Here's an example of what I mean: def foo(): foo = 5 print(foo + 5) foo() # => 10  The code doesn't produce any errors and runs perfectly. This contradicts the idea that variables and functions shouldn't have the same name unless you overwrite them. Why does it work? And when applied to real code, should I use different function/local variable names, or is this perfectly fine?

Answer: foo = 5 creates a local variable inside your function. def foo creates a global variable. That's why they can both have the same name. If you refer to foo inside your foo() function, you're referring to the local variable. If you refer to foo outside that function, you're referring to the global variable. Since it evidently causes confusion for people trying to follow the code, you probably shouldn't do this.
Question: i have a df which I want to replace values that are not the max value for each column to 0. code: data = { "A": [1, 2, 3], "B": [3, 5, 1], "C": [9, 0, 1] } df = pd.DataFrame(data)  sample df:  A B C 0 1 3 9 1 2 5 0 2 3 1 1  result trying to get:  A B C 0 0 0 9 1 0 5 0 2 3 0 0  kindly advise. many thanks

Answer: Try: df[df!=df.max()] = 0  Output:  A B C 0 0 0 9 1 0 5 0 2 3 0 0
Question: I have a column in pandas data frame like the one shown below; LGA Alpine (S) Ararat (RC) Ballarat (C) Banyule (C) Bass Coast (S) Baw Baw (S) Bayside (C) Benalla (RC) Boroondara (C)  What I want to do, is to remove all the special characters from the ending of each row. ie. (S), (RC). Desired output should be; LGA Alpine Ararat Ballarat Banyule Bass Coast Baw Baw Bayside Benalla Boroondara  I am not quite sure how to get desired output mentioned above. Any help would be appreciated. Thanks

Answer: I have different approach using regex. It will delete anything between brackets: import re import pandas as pd df = {'LGA': ['Alpine (S)', 'Ararat (RC)', 'Bass Coast (S)'] } df = pd.DataFrame(df) df['LGA'] = [re.sub("[\(\[].*?[\)\]]", "", x).strip() for x in df['LGA']] # delete anything between brackets
Question: So I am trying to scrape this website: https://www.auto24.ee I was able to scrape data from it without any problems, but today it gives me "Response 403". I tried using proxies, passing more information to headers, but unfortunately nothing seems to work. I could not find any solution on the internet, I tried different methods. The code that worked before without any problems: import requests headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36', } page = requests.get("https://www.auto24.ee/", headers=headers) print(page) 

Answer: The code here import requests headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36'} page = requests.get("https://www.auto24.ee/", headers=headers) print(page.text)  Always will get something as the following  <div class="cf-section cf-wrapper"> <div class="cf-columns two"> <div class="cf-column"> <h2 data-translate="why_captcha_headline">Why do I have to complete a CAPTCHA?</h2> <p data-translate="why_captcha_detail">Completing the CAPTCHA proves you are a human and gives you temporary access to the web property.</p> </div> <div class="cf-column"> <h2 data-translate="resolve_captcha_headline">What can I do to prevent this in the future?</h2> <p data-translate="resolve_captcha_antivirus">If you are on a personal connection, like at home, you can run an anti-virus scan on your device to make sure it is not infected with malware.</p>  The website is protected by CloudFlare. By standard means, there is minimal chance of being able to access the WebSite through automation such as requests or selenium. You are seeing 403 since your client is detected as a robot. There may be some arbitrary methods to bypass CloudFlare that could be found elsewhere, but the WebSite is working as intended. There must be a ton of data submitted through headers and cookies that show your request is valid, and since you are simply submitting only a user agent, CloudFlare is triggered. Simply spoofing another user-agent is not even close to enough to not trigger a captcha, CloudFlare checks for MANY things. I suggest you look at selenium here since it simulates a real browser, or research guides to (possibly?) bypass Cloudflare with requests. Update Found 2 python libraries cloudscraper and cfscrape. Both are not usable for this site since it uses cloudflare v2 unless you pay for a premium version.
Question: Please note that a similar question was asked a while back but never answered (see Winsorizing does not change the max value). I am trying to winsorize a column in a dataframe using winsorize from scipy.stats.mstats. If there are no NaN values in the column then the process works correctly. However, NaN values seem to prevent the process from working on the top (but not the bottom) of the distribution. Regardless of what value I set for nan_policy, the NaN values are set to the maximum value in the distribution. I feel like a must be setting the option incorrectly some how. Below is an example that can be used to reproduce both correct winsorizing when there are no NaN values and the problem behavior I am experiencing when there NaN values are present. Any help on sorting this out would be appreciated. #Import import pandas as pd import numpy as np from scipy.stats.mstats import winsorize # initialise data of lists. data = {'Name':['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T'], 'Age':[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0]} # Create 2 DataFrames df = pd.DataFrame(data) df2 = pd.DataFrame(data) # Replace two values in 2nd DataFrame with np.nan df2.loc[5,'Age'] = np.nan df2.loc[8,'Age'] = np.nan # Winsorize Age in both DataFrames winsorize(df['Age'], limits=[0.1, 0.1], inplace = True, nan_policy='omit') winsorize(df2['Age'], limits=[0.1, 0.1], inplace = True, nan_policy='omit') # Check min and max values of Age in both DataFrames print('Max/min value of Age from dataframe without NaN values') print(df['Age'].max()) print(df['Age'].min()) print() print('Max/min value of Age from dataframe with NaN values') print(df2['Age'].max()) print(df2['Age'].min()) 

Answer: It looks like the nan_policy is being ignored. But winsorization is just clipping, so you can handle this with pandas. def winsorize_with_pandas(s, limits): """ s : pd.Series Series to winsorize limits : tuple of float Tuple of the percentages to cut on each side of the array, with respect to the number of unmasked data, as floats between 0. and 1 """ return s.clip(lower=s.quantile(limits[0], interpolation='lower'), upper=s.quantile(1-limits[1], interpolation='higher')) winsorize_with_pandas(df['Age'], limits=(0.1, 0.1)) 0 3.0 1 3.0 2 3.0 3 4.0 4 5.0 5 6.0 6 7.0 7 8.0 8 9.0 9 10.0 10 11.0 11 12.0 12 13.0 13 14.0 14 15.0 15 16.0 16 17.0 17 18.0 18 18.0 19 18.0 Name: Age, dtype: float64 winsorize_with_pandas(df2['Age'], limits=(0.1, 0.1)) 0 2.0 1 2.0 2 3.0 3 4.0 4 5.0 5 NaN 6 7.0 7 8.0 8 NaN 9 10.0 10 11.0 11 12.0 12 13.0 13 14.0 14 15.0 15 16.0 16 17.0 17 18.0 18 19.0 19 19.0 Name: Age, dtype: float64
Question: Assuming i have the following data frame       date flag user num     0 2019-01-01 1 a 10   1 2019-01-02 0 a 20   2 2019-01-03 1 b 30   3 2019-03-04 1 b 40     I want to create a cumulative sum of the nums grouped by user only if flag == 1 so i will get this:      date flag user num cumsum     0 2019-01-01 1 a 10 10   1 2019-01-02 0 a 20 10   2 2019-01-03 1 b 30 30   3 2019-03-04 1 b 40 70     So far i was able to cumsum by flag, disregarding the group by user df['cumsum'] = df[df['flag'] == 1 ]['num'].transform(pd.Series.cumsum)  or cumsum by user disregarding the flag df['cumsum'] = df.groupby('user')['num'].transform(pd.Series.cumsum)  I need help making them work together.

Answer: You could multiply num by flag to make num = 0 where flag = 0, group by user, and cumsum: df['cumsum'] = df['num'].mul(df['flag']).groupby(df['user']).cumsum()  Output: >>> df date flag user num cumsum 0 2019-01-01 1 a 10 10 1 2019-01-02 0 a 20 10 2 2019-01-03 1 b 30 30 3 2019-03-04 1 b 40 70
Question: I have two very simple questions regarding updating conda. I.e. when updating one of my environments with conda update --all, I get a warning ==> WARNING: A newer version of conda exists. <== current version: xyz1 latest version: xyz2 Please update conda by running $ conda update -n base conda  My setup comprises a base environment and two actual work environments, say, (env1) and (env2). The latter two environments are kept up to date with conda update --all, issued within each of those environments. The base environment was only generated in the installation process of Anaconda. Question 1: Should one run conda update -n base conda on the command line of the OS (linux) prior to activating any environment? Or should one activate a particular environment? Or is the environment out of which this command is issued irrelevant? Question 2: After running conda update -n base conda from out of whatever environment, as determined by the answer to question 1, would a subsequent conda update --all issued within one of my two work environments (env1,2) install or update any additional stuff, only as a consequence of the previous conda update -n base conda? (PS.: I find many questions on stackoverflow regarding conda update conda, but they don't seem to cover this one.)

Answer: Q1: -n explicitly specifies environment - this command will run in any environment and even if you have no environment active. Q2: In all but very few cases updating conda will not affect the packages ought to be installed in other environments. conda plays the role as a package manager. The packages are pulled from an index that is independent of conda's version.