eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a class variable as shown below: class MyClass: def __init__(self): self.value: MyOtherClass | None = None self.initialize_value() def initialize_value(self): self.value = MyOtherClass() def use_value(self): return self.value.used  self.value is guaranteed to be initialized as an instance of MyOtherClass before being used in self.use_value(); however, it must remain as None in the constructor. The dilemma here is how to properly typehint this situation.  If I choose to typehint it as shown above, I get an error: Item "None" of "Optional[MyOtherClass]" has no attribute "used" [union-attr] If I instead remove the variable definition from the constructor and move the typehint to self.initialize_value(), I get an Instance attribute value defined outside __init__ error from Pylint  What is the best way to go about this? Thanks a lot!

Answer: To avoid repetition, when such an attribute is used throughout the class in multiple methods, a common pattern for me is protecting it and defining a property that raises an error, if the attribute behind it is not set: class MyClass: def __init__(self): self._value: MyOtherClass | None = None self.initialize_value() @property def value(self) -> MyOtherClass: if self._value is None: raise AttributeError("...") return self._value def initialize_value(self): self._value = MyOtherClass() def use_value(self): return self.value.used  This also removes ambiguity for the type checker since the return type of the property is always MyOtherClass.
Question: Given the following code and tests: # some module from other.module import a_func # returns False by default def do_stuff(): return "banana" if a_func() else "pear" ############################# # tests in a different module @pytest.fixture def my_fixture(): with mock.patch("other.module.a_func", lambda: True): yield class TestMyStuff: def test_something(self): assert do_stuff() == "pear" def test_something_else(self, my_fixture): assert do_stuff() == "banana"  The last test fails if I run all my tests at once, but succeeds if I only run it. I know that this might be due to the fact that by the time test_something_else runs, other.module.a_func has already been imported and loaded, and so the patch isn't taking effect. How do I make it work, or how should I restructure my code so that I can mock the result of a_func to test different outcomes of do_stuff?

Answer: You should do mock.patch("some.module.a_func")  instead of mock.patch("other.module.a_func")  from other.module import a_func means that a_func becomes part of the some.module, so patching the origin of function definition has no effect - instead, patching should be done where the function is used. Read Where to patch for more info.
Question: I want to open a new folder in different tabs on the same window of Windows File Explorer in Windows 11, instead of opening a new window every time. I tried to use  Start.exe C:\ Explorer.exe C: \ -W 0 Explorer.exe C: \ --windows 0 in terminal and Python. import os import sys gpus = sys.argv[1] path = os.path.realpath(gpus) os.startfile(path)  but it always opens in a new window.

Answer: I did some research on your problem and concluded that it is not allowed to open Tabs in the file explorer. In this link we can see that as of 11/09/2022 there is no response by MSFT https://techcommunity.microsoft.com/t5/windows-11/22h2-explorer-tabs-save-configuration/m-p/3672808
Question: I am using a Kaggle sample data. As shown bellow, 40% of the location is in CA and 47% of the category includes FOODS. What I am trying to achieve is to randomly select data from this data frame, while more or less preserve the same distribution for the values of the these two columns. Does python/Pandas have such a capability? >>> df = pd.read_parquet("~/dimension.parquet") >>> df.groupby('location')['location'].count().transform(lambda x: x/x.sum()) location CA 0.4 TX 0.3 WI 0.3 >>> df.groupby('category')['category'].count().transform(lambda x: x/x.sum()) category FOODS 0.471302 HOBBIES 0.185307 HOUSEHOLD 0.343391 

Answer: Your can select a fraction of each group with groupby.sample: # selecting 10% of each group df.groupby(['location', 'category']).sample(frac=0.1)  But if your data is large and you select a decent number of rows, this should naturally maintain a representativity of the proportions: df.sample(n=1000)  Example, let's pick 500 (0.05%) or 5000 (0.5%) rows out of 1M rows with defined frequencies: np.random.seed(0) n = 1_000_000 df = pd.DataFrame({'location': np.random.choice(['CA', 'TX', 'WI'], p=[0.4, 0.3, 0.3], size=n), 'category': np.random.choice(['A', 'B', 'C'], p=[0.85, 0.1, 0.05], size=n)}) out = df.sample(n=500) out['location'].value_counts(normalize=True) CA 0.388 TX 0.312 WI 0.300 Name: location, dtype: float64 out['category'].value_counts(normalize=True) A 0.822 B 0.126 C 0.052 Name: category, dtype: float64  With df.sample(n=5000): CA 0.3984 TX 0.3064 WI 0.2952 Name: location, dtype: float64 A 0.8468 B 0.1042 C 0.0490 Name: category, dtype: float64  Frequencies of the original population: CA 0.399295 WI 0.300520 TX 0.300185 Name: location, dtype: float64 A 0.850125 B 0.099679 C 0.050196 Name: category, dtype: float64  We observe that both samples are fairly representative of the original population, with nevertheless some loss of precision with the smaller sampling. In contrast, the groupby.sample maintains a close to original proportion even with very small samples (here with ~200 rows (0.02%)): out2 = df.groupby(['location', 'category']).sample(frac=0.0002) print(out2['location'].value_counts(normalize=True)) print(out2['category'].value_counts(normalize=True)) len(out2) CA 0.4 TX 0.3 WI 0.3 Name: location, dtype: float64 A 0.85 B 0.10 C 0.05 Name: category, dtype: float64 200
Question: I have a dataclass and I want to iterate over in in a loop to spit out each of the values. I'm able to write a very short __iter__() within it easy enough, but is that what I should be doing? I don't see anything in the documentation about an 'iterable' parameter or anything, but I just feel like there ought to be... Here is what I have which, again, works fine. from dataclasses import dataclass @dataclass class MyDataClass: a: float b: float c: float def __iter__(self): for value in self.__dict__.values(): yield value thing = MyDataclass(1,2,3) for i in thing: print(i) # outputs 1,2,3 on separate lines, as expected  Is this the best / most direct way to do this?

Answer: The simplest approach is probably to make a iteratively extract the fields following the guidance in the dataclasses.astuple function for creating a shallow copy, just omitting the call to tuple (to leave it a generator expression, which is a legal iterator for __iter__ to return: def __iter__(self): return (getattr(self, field.name) for field in dataclasses.fields(self)) # Or writing it directly as a generator itself instead of returning a genexpr: def __iter__(self): for field in dataclasses.fields(self): yield getattr(self, field.name)  Unfortunately, astuple itself is not suitable (as it recurses, unpacking nested dataclasses and structures), while asdict (followed by a .values() call on the result), while suitable, involves eagerly constructing a temporary dict and recursively copying the contents, which is relatively heavyweight (memory-wise and CPU-wise); better to avoid unnecessary O(n) eager work. asdict would be suitable if you want/need to avoid using live views (if later attributes of the instance are replaced/modified midway through iterating, asdict wouldn't change, since it actually guarantees they're deep copied up-front, while the genexpr would reflect the newer values when you reached them). The implementation using asdict is even simpler (if slower, due to the eager pre-deep copy): def __iter__(self): yield from dataclasses.asdict(self).values() # or avoiding a generator function: def __iter__(self): return iter(dataclasses.asdict(self).values())  There is a third option, which is to ditch dataclasses entirely. If you're okay with making your class behave like an immutable sequence, then you get iterability for free by making it a typing.NamedTuple (or the older, less flexible collections.namedtuple) instead, e.g.: from typing import NamedTuple class MyNotADataClass(NamedTuple): a: float b: float c: float thing = MyNotADataClass(1,2,3) for i in thing: print(i) # outputs 1,2,3 on separate lines, as expected  and that is iterable automatically (you can also call len on it, index it, or slice it, because it's an actual subclass of tuple with all the tuple behaviors, it just also exposes its contents via named properties as well).
Question: I've been looking for some more detailed information regarding the Variable subclasses in tkinter, namely BooleanVar, DoubleVar, IntVar, and StringVar. I'm hoping someone with broader knowledge can point me in the right direction. Given the constructor: tkinter.Variable(master=None, value=None, name=None) I'm curious what the utility of the master parameter is for these classes. I understand that it's equivalent to the master parameter for other tkinter widgets, but I'm not sure I understand how it affects these variable classes specifically; it's a bit more intuitive when the widget is the child of a given class. I typically ignore this value, as I understand that (assuming this is in my root class that inherits from Tk) self.var = tk.StringVar() is equivalent to self.var = tk.StringVar(self), or self.var = tk.StringVar(None). Should I be including this? Is it providing some functionality I would otherwise be missing? I'm not necessarily looking for "What is the best practice here", but rather an explanation of the intended use. Any info is much appreciated! Here's a link to what little information I've been able to find, if anyone else is curious TkDocs - Variable

Answer: When you create an instance of Tk, you are doing more than just creating a widget. For each instance, you are also creating an embedded Tcl interpreter. This tcl interpreter is where all of the widgets and variables and image objects exist. The objects within this interpreter are only available to that interpreter and cannot be shared with other interpreters. If you create multiple instances of Tk, the master parameter lets you tell tkinter which interpreter each variable belongs to. Without it, the variables and widgets will be created in the interpreter of the first instance of Tk.
Question: Python interpreter version used in the code base I am working on has recently been updated from Python 3.7 to 3.9. A few new warnings similar to one in the title have started showing up when some of the tools written in Python are executed. I've searched the net extensively, read the What's New in 3.10 but haven't found an answer about what it exactly means, and what possible actions I can take to address it. I have an option to grep the source code of CPython of course, but I'd rather avoid it if possible. The warning seems to predict change in the visibility of class members. The code in question was not written by me. The original author is (of course) no longer available. Personally, I never use underscored members in an attempt to affect their visibility. Here is how the code around the warning looks like: class Cmd(Enum): ... @classmethod def __call_set(cls, # << Here the warning ...): ... 

Answer: writing an attribute as so: _attr makes it a private attribute, and __attr makes it a protected attribute. This deprecation warning seems to indicate the attributes in question will be made not private and not protected in 3.10. TL;DR They won't have the underscore in 3.10, and they will be completely visible.
Question: I want to create a class attribute, that are dependent to another class attribute (and I tell class attribute, not instance attribute). When this class attribute is a string, as in this topic, the proposed solution class A: foo = "foo" bar = foo[::-1] print(A.bar)  works fine. But when the class attribute is a list or a tuple, as in my case, the following code dose not work... x=tuple('nice cup of tea') class A: remove = ('c','o','a',' ') remains = tuple(c for c in x if not c in remove) print(A.remains)  raise Traceback (most recent call last): File "[***.py]", line 3, in <module> class A: File "[***.py]", line 5, in A remains = tuple(c for c in x if not c in remove) File "[***.py]", line 5, in <genexpr> remains = tuple(c for c in x if not c in remove) NameError: name 'remove' is not defined  Why this kind of methods works if my class attributes is less complex (as simple string, as in the mentioned previous topic) but not for tuple? After investigations, I found this way: x=tuple('nice cup of tea') def sub(a,b): return tuple(c for c in a if not c in b) class A: remove = ('c','o','a',' ') remains = sub(x, remove) print(A.remains)  that works, but does not suit me, for these reasons:  I don't understand why this one, via an intermediary function, works and not without. I don't want to add a function for just a single line with an elementary operation. 

Answer: Python is trying to look up remove in the global scope, but it doesn't exist there. x, on the other hand, is looked up in the enclosing (class) scope. See the documentation: Resolution of names  Class definition blocks and arguments to exec() and eval() are special in the context of name resolution. A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace. The namespace of the class definition becomes the attribute dictionary of the class. The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods -- this includes comprehensions and generator expressions since they are implemented using a function scope. This means that the following will fail: class A: a = 42 b = list(a + i for i in range(10))   Displays for lists, sets and dictionaries  The iterable expression in the leftmost for clause is evaluated directly in the enclosing scope and then passed as an argument to the implicitly nested scope. Subsequent for clauses and any filter condition in the leftmost for clause cannot be evaluated in the enclosing scope as they may depend on the values obtained from the leftmost iterable.
Question: I am trying to create a neat legend in Pyplot. So far I have this:   fig = plt.figure() ax = plt.gca() marker_size = [20.0, 40.0, 60.0, 100.0, 150.0] marker_color = ['black', 'red', 'pink', 'white', 'yellow'] ranges = [0.0, 1.5, 20.0, 60.0, 500.0] marker_edge_thickness = 1.2 s = [(m ** 2) / 100.0 for m in marker_size] scatter_kwargs = {'edgecolors' : 'k', 'linewidths' : marker_edge_thickness} for i in range(len(marker_size)): if i == (len(marker_size) - 1): label_str = '{:>5.1f} $\leq$ H$_2$'.format(ranges[i]) else: label_str = '{:>5.1f} $\leq$ H$_2$ < {:>5.1f}'.format(ranges[i], ranges[i + 1]) ax.scatter([], [], s = s[i], c = marker_color[i], label = label_str, **scatter_kwargs) #ax.legend(prop={'family': 'monospace'}) ax.legend() plt.show()  It is ok but the symbols don't align properly between the rows. I would like to align the rows at multiple points, with alignment on the decimal points, the less-than and greater-than symbols, and the H2. I could use a monotype font (as per this answer: Adding internal spaces in pyplot legend), but this is ugly and seems to be incompatible with the subscript 2 in H2. This would be possible in LaTeX (e.g. using the alignat environment); is it possible in Pyplot?

Answer: You could replace the spaces by '\u2007', a space that is as wide as a digit. In most fonts, a space character is much narrower than a digit. Except for monospaced fonts, which don't look as nice, each letter has its own width. The character width can even be different depending on which letter goes before and after (E.g. with 'VA', the 'A' can be a bit under the 'V', this is called "kerning"). Special characters are introduced in UTF-8 for spaces with a specific width. import matplotlib.pyplot as plt fig = plt.figure() ax = plt.gca() marker_size = [20.0, 40.0, 60.0, 100.0, 150.0] marker_color = ['black', 'red', 'pink', 'white', 'yellow'] ranges = [0.0, 1.5, 20.0, 60.0, 500.0] marker_edge_thickness = 1.2 s = [(m ** 2) / 100.0 for m in marker_size] scatter_kwargs = {'edgecolors': 'k', 'linewidths': marker_edge_thickness} for i in range(len(marker_size)): label_str = f"{ranges[i]:>5.1f}$\leq$H$_2$" if i < (len(marker_size) - 1): label_str += f"$<${ranges[i + 1]:>5.1f}" label_str = label_str.replace(' ', '\u2007') ax.scatter([], [], s=s[i], c=marker_color[i], label=label_str, **scatter_kwargs) ax.legend() plt.tight_layout() plt.show()
Question: I used shapely.ops.unary_union on a number of 6-sided shapely.geometry.Polygons, and obtained the following shape A:  Note how there are two "cracks" in the upper part. These are not intended, and are presumably caused by some floating-point edge cases. If you construct another shape B that sits inside of A, and if A happens to intersect one of these "cracks", then A.covers(B) will be False! In my particular case, this leads to a test suite failure, because A.covers(B) is supposed to be an invariant. Therefore I need to deal with this somehow. Is there some algorithm I can use to "seal" these cracks? In practice, these "cracks" will not affect the functioning of the application, because we only care about the outer border of A covering B. Thus I am open to solutions that adjust the individual hexagons in order to make the cracks go away by introducing overlaps. However, I cannot accept the outer border of this shape changing, because that would actually no longer be testing the application as it is intended to be used. To summarize, I want the outcome to look like this (my hand-edited version): 

Answer: You can fix this by buffering and un-buffering the shape. Here's an example of a polygon with a small crack: from shapely.geometry import Polygon bad_polygon = Polygon([[0, 0], [1, 0], [1, 0.4999], [0.5, 0.5], [1, 0.5001], [1, 1], [0, 1], [0, 0]])   To fix it, expand the shape slightly, and contract it the same amount, using the buffer() method. tol = 1e-4 bad_polygon.buffer(tol).buffer(-tol)  The value tol must be at least as large as half the distance across the crack at the crack's widest point.
Question: It seems that several asyncio functions, like those showed here, for synchronization primitives are not thread safe... By being not thread safe, considering for example asyncio.Lock, I assume that this lock won't lock the global variable, when we're running multiple threads in our computer, so race conditions are problem. So, what's the point of having this Lock that doesn't lock? (not a criticism, but an honest doubt) What are the case uses for these unsafe primitives?

Answer: For use-cases, look at what's Python asyncio.Lock() for? As for why it is not thread-safe. It is mostly performance. Asyncio is not made for multithreaded work like old servers used to do, the asyncio eventloop itself is not thread-safe and only runs coroutines in a single thread, hence a lock that is only running in a single thread doesn't need to be threadsafe, and doesn't need to suffer from the extra overhead from using a thread-safe primitives. Interacting with asyncio.lock suspends the current coroutine and instructs the eventloop to resume another coroutine (remember the eventloop is not thread-safe), those primitives don't need interaction with the OS unlike thead-safe ones which need to interact with the OS, which is the entire reason why asyncio is faster than old multithreaded servers.  You can use loop.run_in_executor to do tasks in another thread, those tasks can hold a threading.Lock while doing their work. This is preferred over trying to acquire the lock in the asyncio eventloop thread. To use a threading.Lock in an async function you could wait for it indirectly using the answer in this question How to use threading.Lock in async function while object can be accessed from multiple thread as commented by @dano.  It is not that those primitives can't be made threadsafe. They can be made to work similar to loop.call_soon_threadsafe or the above method which is more expensive than a normal threading.lock. Thread-safe locks are expensive and thread-safe interactions with the eventloops are even more expensive, because you need to send a message to the eventloop to interact with it.
Question: I am trying to load the dataset using Torch Dataset and DataLoader, but I got the following error: AttributeError: '_MultiProcessingDataLoaderIter' object has no attribute 'next'  the code I use is: class WineDataset(Dataset): def __init__(self): # Initialize data, download, etc. # read with numpy or pandas xy = np.loadtxt('./data/wine.csv', delimiter=',', dtype=np.float32, skiprows=1) self.n_samples = xy.shape[0] # here the first column is the class label, the rest are the features self.x_data = torch.from_numpy(xy[:, 1:]) # size [n_samples, n_features] self.y_data = torch.from_numpy(xy[:, [0]]) # size [n_samples, 1] # support indexing such that dataset[i] can be used to get i-th sample def __getitem__(self, index): return self.x_data[index], self.y_data[index] # we can call len(dataset) to return the size def __len__(self): return self.n_samples dataset = WineDataset() train_loader = DataLoader(dataset=dataset, batch_size=4, shuffle=True, num_workers=2)  I tried to make the num_workers=0, still have the same error. Python version 3.8.9 PyTorch version 1.13.0 

Answer: I too faced the same issue, when i tried to call the next() method as follows dataiter = iter(dataloader) data = dataiter.next()  You need to use the following instead and it works perfectly: dataiter = iter(dataloader) data = next(dataiter)  Finally your code should look like follows: class WineDataset(Dataset): def __init__(self): # Initialize data, download, etc. # read with numpy or pandas xy = np.loadtxt('./data/wine.csv', delimiter=',', dtype=np.float32, skiprows=1) self.n_samples = xy.shape[0] # here the first column is the class label, the rest are the features self.x_data = torch.from_numpy(xy[:, 1:]) # size [n_samples, n_features] self.y_data = torch.from_numpy(xy[:, [0]]) # size [n_samples, 1] # support indexing such that dataset[i] can be used to get i-th sample def __getitem__(self, index): return self.x_data[index], self.y_data[index] # we can call len(dataset) to return the size def __len__(self): return self.n_samples dataset = WineDataset() dataloader = DataLoader(dataset=dataset, batch_size=4, shuffle=True, num_workers=2) dataiter = iter(dataloader) data = next(dataiter)
Question: It's useful to be able to create frozen dataclasses. I'm wondering if there is a way to do something similar for regular python classes (ones with an __init__ function with complex logic possibly). It would be good to prevent modification after construction in some kind of elegant way, like frozen dataclasses.

Answer: yes. All attribute access in Python is highly customizable, and this is just a feature dataclasses make use of. The easiest way to control attribute setting is to create a custom __setattr__ method in your class - if you want to be able to create attributes during __init__ one of the ways is to have an specific parameter to control whether each instance is frozen already, and freeze it at the end of __init__: class MyFrozen: _frozen = False def __init__(self, ...): ... self._frozen = True def __setattr__(self, attr, value): if getattr(self, "_frozen", None): raise AttributeError("Trying to set attribute on a frozen instance") return super().__setattr__(attr, value)