eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: On the same lines as the solution provided in this link, I am trying to get all leaf words of one stem word. I am using the community-contributed (@Divyanshu Srivastava) package get_word_forms Imagine I have a shorter sample word list as follows: my_list = [' jail', ' belief',' board',' target', ' challenge', ' command']  If I work it manually, I do the following (which is go word-by-word, which is very time-consuming if I have a list of 200 words): get_word_forms("command")  and get the following output: {'n': {'command', 'commandant', 'commandants', 'commander', 'commanders', 'commandership', 'commanderships', 'commandment', 'commandments', 'commands'}, 'a': set(), 'v': {'command', 'commanded', 'commanding', 'commands'}, 'r': set()}  'n' is noun, 'a' is adjective, 'v' is verb, and 'r' is adverb. If I try to reverse-stem the entire list in one go: [get_word_forms(word) for word in sample]  I fail at getting any output: [{'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}, {'n': set(), 'a': set(), 'v': set(), 'r': set()}]  I think I am failing at saving the output to the dictionary. Eventually, I would like my output to be a list without breaking it down into noun, adjective, adverb, or verb: something like: ['command','commandant','commandants', 'commander', 'commanders', 'commandership', 'commanderships','commandment', 'commandments', 'commands','commanded', 'commanding', 'commands', 'jail', 'jailer', 'jailers', 'jailor', 'jailors', 'jails', 'jailed', 'jailing'.....] .. and so on. 

Answer: One solution using nested list comprehensions after stripping forgotten spaces: all_words = [setx for word in my_list for setx in get_word_forms(word.strip()).values() if len(setx)] # Flatten the list of sets all_words = [word for setx in all_words for word in setx] # Remove the repetitions and sort the set all_words = sorted(set(all_words)) print(all_words) ['belief', 'beliefs', 'believabilities', 'believability', 'believable', 'believably', 'believe', 'believed', 'believer', 'believers', 'believes', 'believing', 'board', 'boarded', 'boarder', 'boarders', 'boarding', 'boards', 'challenge', 'challengeable', 'challenged', 'challenger', 'challengers', 'challenges', 'challenging', 'command', 'commandant', 'commandants', 'commanded', 'commander', 'commanders', 'commandership', 'commanderships', 'commanding', 'commandment', 'commandments', 'commands', 'jail', 'jailed', 'jailer', 'jailers', 'jailing', 'jailor', 'jailors', 'jails', 'target', 'targeted', 'targeting', 'targets']
Question: Can types be associated with enums, so that Pyright can infer the type from an equality check? (Without cast() or isinstance().) from dataclasses import dataclass from enum import Enum, auto class Type(Enum): FOO = auto() BAR = auto() @dataclass class Foo: type: Type @dataclass class Bar: type: Type item = next(i for i in (Foo(Type.FOO), Bar(Type.BAR)) if i.type == Type.BAR) reveal_type(item) # How to have this be `Bar` instead of `Foo | Bar`? 

Answer: You want a discriminated union (also known as tagged union). In a discriminated union, there exists a discriminator (also known as a tag field) which can be used to differentiate the members. You currently have an union of Foo and Bar, and you want to discriminate them using the .type attribute. However, this field cannot be the discriminator since it isn't different for each member of the union. (playgrounds: Pyright, Mypy) for i in (Foo(Type.FOO), Bar(Type.BAR)): reveal_type(i) # Foo | Bar  mischievous_foo = Foo(Type.BAR) # This is valid naughty_bar = Bar(Type.FOO) # This too for i in (mischievous_foo, naughty_bar): if i.type == Type.FOO: reveal_type(i) # Runtime: Bar, not Foo  If Foo.type can only ever be Type.FOO and Bar.Type be Type.BAR, then it is important that you reflect this in the types: (Making type a dataclass field no longer makes sense at this point, but I'm assuming they are only dataclasses for the purpose of this question.) @dataclass class Foo: type: Literal[Type.FOO] @dataclass class Bar: type: Literal[Type.BAR]  As Literal[Type.FOO] and Literal[Type.BAR] are disjoint types, i will then be narrowable by checking for the type of .type: (playgrounds: Pyright, Mypy) for i in (Foo(Type.FOO), Bar(Type.BAR)): if i.type == Type.FOO: reveal_type(i) # Foo  Foo(Type.BAR) # error Bar(Type.FOO) # error  ...even in a generator, yes: item = next(i for i in (Foo(Type.FOO), Bar(Type.BAR)) if i.type == Type.BAR) reveal_type(item) # Bar
Question: As you can see in the first screenshot, /products/new isn't showing up as a valid URL although I followed the coding tutorial from YouTube exactly. For some reason there's a blank character before "new" but no blank space in the current path I'm trying to request. I don't know if that's normal or not. I'm using django version 2.1 if that matters The URL does work for products/salt/. What's weird is the URL used to be products/trending/ but I got the same error as with products/new so I randomly changed the URL to products/salt and it started working for me.  [Page not found (404) Request Method: GET Request URL: http://127.0.0.1:8000/products/new/ Using the URLconf defined in pyshop.urls, Django tried these URL patterns, in this order:   admin/ products/ products/ salt products/ new The current path, products/new/, didn't match any of these.]1  from django.http import HttpResponse from django.shortcuts import render def index(request): return HttpResponse('Hello World') def trending(request): return HttpResponse('Trending Products') def new(request): return HttpResponse('New Products')[2] from django.urls import path from . import views urlpatterns = [ path('', views.index), path('salt', views.trending), path('new', views.new)[3] 

Answer: Add a trailing slash / to your URLpatterns to resolve this issue i.e. new/ and trending/. Also as mentioned in my comment, I would suggest you upgrade to a secure version of Django to access newer features.
Question: I'm wondering why pandas Series subtract a pandas dataframe produce such a strange result. df = pd.DataFrame(np.arange(10).reshape(2, 5), columns='a-b-c-d-e'.split('-')) df.max(axis=1) - df[['b']]  What are the steps for pandas to produce the result?  b 0 1 0 NaN NaN NaN 1 NaN NaN NaN 

Answer: By default an operation between a DataFrame and a Series is broadcasted on the DataFrame by column, over the rows. This makes it easy to perform operations combining a DataFrame and aggregation per column: # let's subtract the DataFrame to its max per column df.max(axis=0) - df[['b']] a b c d e b NaN 5 NaN NaN NaN 1 NaN 0 NaN NaN NaN  Here, since you're aggregating per row, this is no longer possible. You should use rsub with the parameter axis=0: df[['b']].rsub(df.max(axis=1), axis=0)  Output:  b 0 3 1 3  Note that using two Series would also align the values: df.max(axis=1) - df['b']  Output: 0 3 1 3 dtype: int64  Why 3 columns with df.max(axis=1) - df[['b']]? First, let's have a look at each operand: # df.max(axis=1) 0 4 1 9 dtype: int64 # df[['b']] b 0 1 1 6  Since df[['b']] is 2D (DataFrame), and df.max(axis=1) is 1D (Series), df.max(axis=1) will be used as if it was a "wide" DataFrame: # df.max(axis=1).to_frame().T 0 1 0 4 9  There are no columns in common, thus the output is only NaNs with the union of column names ({'b'}|{0, 1} -> {'b', 0, 1}). If you replace the NaNs that are used in the operation by 0 this makes it obvious how the values are used: df[['b']].rsub(df.max(axis=1).to_frame().T, fill_value=0) b 0 1 0 -1.0 4.0 9.0 1 -6.0 NaN NaN  Now let's check a different example in which one of the row indices has the same name as one of the selected columns: df = pd.DataFrame(np.arange(10).reshape(2, 5), columns=['a', 'b', 'c', 'd', 'e'], index=['b', 0] ) df.max(axis=1) - df[['b']]  Now the output only has 2 columns, b the common indice and 1 the second index in the Series ({'b', 1}|{'b'} -> {'b', 1}):  1 b b NaN 3 1 NaN -2
Question: I'm wondering how to use the capitalize function when another column has a specific value. For example, I want to change the first letter of students with Master's degree. # importing pandas as pd import pandas as pd # creating a dataframe df = pd.DataFrame({ 'A': ['john', 'bODAY', 'minA', 'peter', 'nicky'], 'B': ['Masters', 'Graduate', 'Graduate', 'Masters', 'Graduate'], 'C': [27, 23, 21, 23, 24] }) # Expected result # A B C #0 John Masters 27 #1 bODAY Graduate 23 #2 minA Graduate 21 #3 Peter Masters 23 #4 nicky Graduate 24  I tried it like this, but it didn't apply well. df[df['B']=='Masters']['A'].str = df[df['B']=='Masters']['A'].str.capitalize() 

Answer: Here is the complete code: import pandas as pd # Creating the DataFrame df = pd.DataFrame({ 'A': ['john', 'bODAY', 'minA', 'peter', 'nicky'], 'B': ['Masters', 'Graduate', 'Graduate', 'Masters', 'Graduate'], 'C': [27, 23, 21, 23, 24] }) # Capitalize column A conditionally based on B df['A'] = df.apply(lambda row: row['A'].capitalize() if row['B'] == 'Masters' else row['A'], axis=1) # Display the updated DataFrame print(df)  Output:  A B C 0 John Masters 27 1 bODAY Graduate 23 2 minA Graduate 21 3 Peter Masters 23 4 nicky Graduate 24
Question: I've got some data I'm reading into Python using Pandas and want to keep track of units with the Pint package. The values have a range of scales, so have mixed units, e.g. lengths are mostly meters but some are centimeters. For example the data: what,length foo,5.3 m bar,72 cm  and I'd like to end up with the length column in some form that Pint understands. Pint's Pandas integration suggests that it only supports the whole column having the same datatype, which seems reasonable. I'm happy with some arbitrary unit being picked (e.g. the first, most common, or just SI base unit) and everything expressed in terms of that. I was expecting some nice way of getting from the data I have to what's expected, but I don't see anything. import pandas as pd import pint_pandas length = pd.Series(['5.3 m', "72 cm"], dtype='pint[m]')  Doesn't do the correct thing at all, for example: length * 2  outputs 0 5.3 m5.3 m 1 72 cm72 cm dtype: pint[meter]  so it's just leaving things as strings. Calling length.pint.convert_object_dtype() doesn't help and everything stays as strings.

Answer: Going through the examples, it looks like pint_pandas is expecting numbers rather than strings. You can use apply to do the conversion: from pint import UnitRegistry ureg = UnitRegistry() df["length"].apply(lambda i: ureg(i)).astype("pint[m]")  However, why keep the column as Quantity objects instead of just plain float numbers?
Question: I have a DataFrame with a species column and four arbitrary data columns. I want to group it by species and sum up the four data columns for each one. I've tried to do this in two ways: once by grouping by df.columns[0] and once by grouping by df.iloc[:, 0]. data = { 'species': ['a', 'b', 'c', 'd', 'e', 'rt', 'gh', 'ed', 'e', 'd', 'd', 'q', 'ws', 'f', 'fg', 'a', 'a', 'a', 'a', 'a'], 's1': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], 's2': [9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9], 's3': [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21], 's4': [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10] } df = pd.DataFrame(data) grouped_df1 = df.groupby(df.columns[0], as_index=False).sum() grouped_df2 = df.groupby(df.iloc[:, 0], as_index=False).sum()  Both methods correctly sum the data in the four rightmost columns. But for some reason, the second method also sums up the names of the species, concatenating them into one long, repeating string. Here's the result from the first method, which is what I'm looking for: print(grouped_df1) species s1 s2 s3 s4 0 a 91 54 97 60 1 b 2 9 3 10 2 c 3 9 4 10 3 d 25 27 28 30 4 e 14 18 16 20 5 ed 8 9 9 10 6 f 14 9 15 10 7 fg 15 9 16 10 8 gh 7 9 8 10 9 q 12 9 13 10 10 rt 6 9 7 10 11 ws 13 9 14 10  And here's the result from the df.iloc method, which incorrectly sums up the species data: print(grouped_df2) species s1 s2 s3 s4 0 aaaaaa 91 54 97 60 1 b 2 9 3 10 2 c 3 9 4 10 3 ddd 25 27 28 30 4 ee 14 18 16 20 5 ed 8 9 9 10 6 f 14 9 15 10 7 fg 15 9 16 10 8 gh 7 9 8 10 9 q 12 9 13 10 10 rt 6 9 7 10 11 ws 13 9 14 10  Why is the second method summing up the species names as well as the numerical data?

Answer: In groupby - column name is treated as an intrinsic grouping key, while a Series is treated as an external key. Reference - https://pandas.pydata.org/docs/reference/groupby.html When using df.iloc[:, 0]: Pandas considers the string values in the species column as a separate grouping key independent of the DataFrame structure. When using df.columns[0]: Pandas directly uses the column 'species' within the DataFrame as the grouping key. This allows Pandas to manage the grouping and summation correctly. Code COrrection You should always reference the column name explicitly grouped_df1 = df.groupby('species', as_index=False).sum()  Or this also works grouped_df1 = df.groupby(df[df.columns[0]], as_index=False).sum()
Question: I'm performing some calculations to generate chaotic solutions to a mathematical function. I have an infinite loop that looks something like this: f = open('solutions.csv', 'a') while True: x = generate_random_parameters() # x is a list of floats success = test_parameters(x) if success: print(','.join(map(str, x)), file=f, flush=True)  The implementation of generate_random_parameters() and test_parameters() is not very important here. When I want to stop generating solutions I want to ^C, but I want to ensure that solutions.csv keeps its integrity/doesn't get corrupted/etc, in case I happen to interrupt when the file is being written to. So far I haven't observed this happening, but I'd like to remove any possibility that this could occur. Additionally, since the program will never terminate on its own I don't have a corresponding f.close() -- this should be fine, correct? Appreciate any clarification.

Answer: One simple approach to ensuring that the current call to print finishes before the program exits from a keyboard interrupt is to use a signal handler to unset a flag on which the while loop runs. Set the signal handler only when you're about to call print and reset the signal handler to the original when print returns, so that the preceding code in the loop can be interrupted normally: import signal def interrupt_handler(signum, frame): global running running = False text = 'a' * 99999 running = True with open('solutions.csv', 'a') as f: while running: ... # your calculations original_handler = signal.signal(signal.SIGINT, interrupt_handler) print(text, file=f, flush=True) # your output signal.signal(signal.SIGINT, original_handler)  Also note that it is more idiomatic to use open as a context manager to handle the closure of an open file when exiting the block for any reason.
Question: I am using OpenCV LineDetector class in order to parse tables. However, I face an issue when I try to detect lines inside the table. for the following image:  I use img = cv2.imread(TABLE_PATH) gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) lsd = cv2.createLineSegmentDetector(cv2.LSD_REFINE_ADV, sigma_scale=0.6) dlines = lsd.detect(gray) lines = (Line(x0, y0, x1, y1) for x0, y0, x1, y1 in dlines[0][:, 0])  in order to detect line segments. However, the results are lousy. these are the lines it detects:  How can I make sure that words are not detected as lines. I cannot use hardcoded thresholds since they would work for one example but not for the other. Solutions in python or java would be appreciated

Answer: You got some lines detected, but that set contained some undesirable ones. You could just filter the set of lines for line length. If you do that, you can easily exclude the very short lines coming from the text in that picture. Implementation: that's a list comprehension, only including lines that are long enough. Write a predicate function that gives you the length of one line, then you can use that in the list comprehension. That is independent of how you scraped lines out of the picture. the LSD is one, but there are routines based on the Hough transform too, which might fare better or worse than what you have. You probably also noticed that your approach didn't find some lines that it should have. You might want to tweak the parameters you pass to your line detector. Or try another line detection approach.
Question: I have the following hashable pydantic model: class TafReport(BaseModel, frozen=True): download_date: dt icao: str issue_time: dt validity_time_start: dt validity_time_stop: dt raw_report: str  Now I don't want these reports to be considered different just because their download date is different (I insert that with the datetime.now()). How can i exclude download_date from being considered in the __hash__ and __eq__ functions so that I can do stunts like: tafs = list(set(tafs))  and have a unique set of tafs even though two might have differing download date? I'm looking for a solution where I don't have to overwrite the __hash__ and __eq__ methods... I checked out this topic but it only answers how to exclude a field from the model in general (so it doesn't show up in the json dumps), but I do want it to show up in the json dump.

Answer: Unfortunately there is no built-in option at the moment, but there are two options that you can consider: Changing from BaseModel to a Pydantic dataclass: from dataclasses import field from datetime import datetime as dt from pydantic import TypeAdapter from pydantic.dataclasses import dataclass @dataclass(frozen=True) class TafReport: download_date: dt = field(compare=False) icao: str issue_time: dt validity_time_start: dt validity_time_stop: dt raw_report: str TafReportAdapter = TypeAdapter(TafReport) SameTime = dt.now() TafReport1 = TafReport(download_date=dt.now(), icao='icao', issue_time=SameTime, validity_time_start=SameTime, validity_time_stop=SameTime, raw_report='raw_report') TafReport2 = TafReport(download_date=dt.now(), icao='icao', issue_time=SameTime, validity_time_start=SameTime, validity_time_stop=SameTime, raw_report='raw_report') print(TafReportAdapter.dump_json(TafReport1), hash(TafReport1)) print(TafReportAdapter.dump_json(TafReport2), hash(TafReport2))  This will give the same hash while the download_date is different. Exclude the download_date from the model and allow extra fields: from datetime import datetime as dt from pydantic import BaseModel class TafReport(BaseModel, frozen=True, extra='allow'): icao: str issue_time: dt validity_time_start: dt validity_time_stop: dt raw_report: str SameTime = dt.now() TafReport1 = TafReport(icao='icao', issue_time=SameTime, validity_time_start=SameTime, validity_time_stop=SameTime, raw_report='raw_report', download_date=dt.now()) TafReport2 = TafReport(icao='icao', issue_time=SameTime, validity_time_start=SameTime, validity_time_stop=SameTime, raw_report='raw_report', download_date=dt.now()) print(TafReport1.model_dump(), hash(TafReport1)) print(TafReport2.model_dump(), hash(TafReport2))  In this case the hash function is build based on the fields provided in the model. But allowing extra fields without defining them in the model gives you the ability to add the download_date without affecting the hash function build in the model.
Question: I'm working on a project that uses pyfakefs to mock my filesystem to test folder creation and missing folders in a previously defined tree structure. I'm using Python 3.13 on Windows and get this output from the terminal after running my test: Terminal output: (Does anyone have a tip for formatting terminal output without getting automatic syntax highlighting?) E ====================================================================== ERROR: test_top_folders_exist (file_checker.tests.file_checker_tests.TestFolderCheck.test_top_folders_exist) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Users\juank\dev\projects\python\gamedev_eco\file_checker\tests\file_checker_tests.py", line 20, in test_top_folders_exist self.fs.create_dir(Path.cwd() / "gdd") ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "C:\Users\juank\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyfakefs\fake_filesystem.py", line 2191, in create_dir dir_path = self.absnormpath(dir_path) File "C:\Users\juank\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyfakefs\fake_filesystem.py", line 1133, in absnormpath path = self.replace_windows_root(path) File "C:\Users\juank\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyfakefs\fake_filesystem.py", line 1418, in replace_windows_root if path and self.is_windows_fs and self.root_dir: ^^^^^^^^^^^^^ File "C:\Users\juank\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyfakefs\fake_filesystem.py", line 357, in root_dir return self._mount_point_dir_for_cwd() ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "C:\Users\juank\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyfakefs\fake_filesystem.py", line 631, in _mount_point_dir_for_cwd if path.startswith(str_root_path) and len(str_root_path) > len(mount_path): ^^^^^^^^^^^^^^^ AttributeError: 'WindowsPath' object has no attribute 'startswith' ---------------------------------------------------------------------- Ran 1 test in 0.011s FAILED (errors=1)  Test: from pyfakefs.fake_filesystem_unittest import TestCase class TestFolderCheck(TestCase): """Test top folders = gdd marketing business""" @classmethod def setUp(cls): cls.setUpClassPyfakefs() cls.fake_fs().create_dir(Path.cwd() / "gamedev_eco") cls.fake_fs().cwd = Path.cwd() / "gamedev_eco" def test_top_folders_exist(self): self.fs.create_dir(Path.cwd() / "gdd")  What is confusing for me is that the Setup class method can create a folder and change cwd to that new folder but I'm not able to create a folder inside a test. Does anyone have experience working with pyfakefs? Can anyone lend me a hand with this issue please?

Answer: The issue has been acknowledged, fixed, and the fix has been included in the 5.7.4 release of pyfakefs. No workaround should thus be necessary, any longer.