eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to convert whipser-diarization to poetry. It goes well until I add nemo_toolkit[asr]==1.20.0, which depends on youtokentome (that name is well thought of, btw)  File "/tmp/tmpexmdke23/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 507, in run_setup super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script) File "/tmp/tmpexmdke23/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in run_setup exec(code, locals()) File "<string>", line 5, in <module> ModuleNotFoundError: No module named 'Cython'  So I tried adding cython to the dependencies. It works fine if I poetry shell and execute cython, so it is avaible. My pyproject so far: ... [tool.poetry.dependencies] python = "^3.10" faster-whisper = "0.9.0" wget = "^3.2" transformers = ">=4.26.1" whisperx = {git = "https://github.com/m-bain/whisperX.git", rev = "49e0130e4e0c0d99d60715d76e65a71826a97109"} deepmultilingualpunctuation = "^1.0.1" cython = "^3.0.5" [build-system] requires = ["poetry-core", "cython"] build-backend = "poetry.core.masonry.api"  I added cython to the requires section, but that doesn't resolve the error.

Answer: Building this package is done in an isolated environment, so it doesn't matter if Cython is installed in your current environment. youtokentome has to define its build requirements according to PEP 518. There seems to be a Pull Request for it for a long time: https://github.com/VKCOM/YouTokenToMe/pull/108
Question: Having this array with code and collection, where X is a mask that can be "any number": input_array = [{"code": "XXXX10", "collection": "one"}, {"code": "XXX610", "collection": "two"}, {"code": "XXXX20", "collection": "three"}]  I want a function that given any 6 digit code, for example 000710 returns the value that matches the best code mask (for the example would be one). This is my try: def get_collection_from_code(analysis_code): for collection in input_array: actual_code = collection["code"] mask_to_re = actual_code.replace("X", "[\d\D]") pattern = re.compile("^" + mask_to_re + "$") if pattern.match(analysis_code): print("Found collection '" + str(collection["collection"]) + "' for code: " + str(analysis_code)) return collection["collection"] res = get_collection_from_code("010610") print(res)  The problem here is that if I inpuit the code 010610 (and I want to return two), it returns one as also matches the pattern XXXX10 first. For better understanding, if I input there values, I would like to have those ouputs: 010610 > two 010010 > one 123420 > three 

Answer: You could iterate the entire collection, saving the length of the "X" part of any match, and then return the shortest: input_array = [{"code": "XXXX10", "collection": "one"}, {"code": "XXX610", "collection": "two"}, {"code": "XXXX20", "collection": "three"}] def get_collection_from_code(analysis_code): results = {} for collection in input: actual_code = collection["code"] mask_to_re = actual_code.replace("X", "[\d\D]") pattern = re.compile("^" + mask_to_re + "$") if pattern.match(analysis_code): results[collection["collection"]] = actual_code.count('X') if len(results): best = sorted(results.items(), key=lambda i:i[1])[0] print("Found collection '" + str(best[0]) + "' for code: " + str(analysis_code)) return best[0] res = get_collection_from_code("010610") # Found collection 'two' for code: 010610  Note I've saved all the matches in case you want to process them in any way. Otherwise you could just check for the "best" match in each iteration and update that instead.
Question: In Python, when collecting a sequence of Optional values into an Iterable or List while filtering out None values, how do you express to the typechecker that the element type of the resulting Iterable is no longer Optional and the list does not contain Nones? Take the following example: def a(i: InputType) -> Optional[ResultType]: ... def b(is: List[InputType]) -> Iterable[ResultType]: return filter(partial(is_not, None), (a(x) for x in is))  The type checker (Pyright in my case) complains that I can't put the potential None value in the Iterable[ScanFileResult] return type. But since I'm filtering it out, I need some way to assert to prove to the typechecker that the element type is no longer Optional[ResultType] but just ResultType. What's the cleanest way of doing this?

Answer: Following the suggestion of @MateenUlhaq in the comments the problem you are facing is probably due to the use of filter(partial(is_not, None)) that makes it hard for the language server to infer the type of ots return value. You can make it easier for it by filtering out the Nones at the beginning. The following is an example: def a(x: int) -> int | None: if x % 2 == 0: return x return None def b(xs: list[int]): print(sum(xs)) xs = [ax for x in range(10) if (ax:=a(x)) is not None] b(xs)  With this pyright does not complain since it can infer the type of xs to be list[int] Applying this to your specific case you would get something like: def b(inputs: List[InputType]) -> Iterable[ResultType]: return (ax for x in inputs if (ax:=a(x)) is not None)  This works on python version 3.10+ (as it uses the := operator).
Question: this code creates me a table in an excel file and a pie plot, the label fields, taken from a list of values, are way too long.  plt.pie(df.Confronto.value_counts(), labels=textwrap.shorten(df.Confronto.value_counts().index.tolist(), width=10, placeholder="...")) plt.title("Confronto {} - {}".format(Mese,Mese2,)) plt.show()  So I tried to shorten it using text.shorten, but it gives me this error: labels=textwrap.shorten(df.Confronto.value_counts().index.tolist(), width=10, placeholder="...")) AttributeError: 'list' object has no attribute 'strip'  I tried to cast but it gave me an error about the indeterminate length of the label labels=textwrap.shorten(str(df.Confronto.value_counts().index.tolist()), width=10, placeholder="...") raise ValueError("'label' must be of length 'x'") ValueError: 'label' must be of length 'x'  The image below shows the result of the code in its current state: the labels are too long and at the time the image is created nothing is read, I wish I could put a maximum on the length of the labels. 

Answer: I am not sure as to what you are trying to achieve with this code, so I'll just explain why the exception is raised. That happens because pandas.Index.tolist returns a list, and textwrap.shorten is used on strings. So, cannot strip a string if it's a list. I assume df.Confronto.value_counts().index.tolist() returns a list of int, like [1,2,3]. You may try joining the list first with label_string = ''.join(map(str,[1,2,3])) Out: '123'  It first maps str() over your list of int to produce a list of str, then joins the list in a single string with a '' symbol. However, labels= takes the list of str, so you might want to restore it back to list with label_list = [ch for ch in label_string]  or even slice the original df.Confronto.value_counts().index.tolist() to reduce the number of elements, and then map str() over it. After clarification in comments I suggest a list comprehension to iterate textwrap on given list: label_list = [textwrap.shorten(string, width=10, placeholder="...") for string in df.Confronto.value_counts().index.tolist()]
Question: Maximum subarray sum with at most K elements : Given an array of integers and a positive integer k, find the maximum sum of a subarray of size less than or equal to k. A subarray is a contiguous part of the array. For example, if the array is [5, -3, 5, 5, -3, 5] and k is 3, then the maximum subarray sum with at most k elements is 10, which is obtained by the subarray [5, 5] My initial thought was to use the Kadane's algorithm with a sliding window of K. Below is the code: maxi = nums[0] max_so_far = 0 prev = 0 for i in range(len(nums)): max_so_far += nums[i] if (i - prev) >= k: max_so_far -= nums[prev] prev += 1 maxi = max(maxi, max_so_far) if max_so_far < 0: max_so_far = 0 prev = i + 1 return maxi  but this approach won't work for the test case - nums = [5, -3, 5, 5, -3, 5] k = 3  Edit: I found the solution - Prefix Sum + Monotonic Deque - O(n) Time complexity def maxSubarraySum(self, nums, k) -> int: prefix_sum = [0] * len(nums) prefix_sum[0] = nums[0] for i in range(1, len(nums)): prefix_sum[i] = prefix_sum[i-1] + nums[i] q = deque() for i in range(k): while len(q) > 0 and prefix_sum[i] >= prefix_sum[q[-1]]: q.pop() q.append(i) maxi = max(prefix_sum[:k]) for i in range(1, len(nums)): if q[0] < i: q.popleft() if i + k - 1 < len(nums): while len(q) > 0 and prefix_sum[i + k - 1] >= prefix_sum[q[-1]]: q.pop() q.append(i + k - 1) maxi = max(maxi, prefix_sum[q[0]] - prefix_sum[i-1]) return maxi 

Answer: There is an O(n) solution at https://cs.stackexchange.com/a/151327/10147 We can have O(n log n) with divide and conquer. Consider left and right halves of the array: the solution is either in (1) the left half exclusively, (2) the right half exclusively, or (3) a suffix of the left combined with a prefix of the right. To solve (3) in O(n), iterate from the middle to the left, recording for each index the higher of the highest seen or the total sum. Then iterate to the right and add a similar record for prefix length l with the recorded value for index k - l (or the longest possible if k-l is out of bounds) in the first iteration. For the example, [5, -3, 5, 5, -3, 5] k = 3, we have: [5, -3, 5, 5, -3, 5] 7 5 5 <--- ---> 5 5 7 ^-----^ best = 5 + 5 = 10
Question: I am trying to reproduce some C++ code in Python that involves random number generations. The C++ code uses the MT19937 generator as follows: #include <random> #include <iostream> int main() { std::mt19937 generator(1234); std::uniform_real_distribution<double> distribution(0.0, 1.0); for (int i = 0; i < 10; ++i) { std::cout << distribution(generator) << std::endl; } return 0; }  The Python version is (with NumPy 1.23.3) import numpy as np rng = np.random.Generator(np.random.MT19937(1234)) for _ in range(10): print(rng.random())  In both cases, the random seed is set to 1234. But the two produce different outputs on my machine (macOS 14.0 ARM). The C++ code outputs 0.497664 0.817838 0.612112 0.77136 0.86067 0.150637 0.198519 0.815163 0.158815 0.116138  while the Python code outputs 0.12038356302504949 0.4037014194964441 0.8777026256367374 0.9565788014497463 0.42646002242298486 0.28304326113156464 0.9009410688498408 0.830833142531224 0.6752899264264728 0.3977176012599666  Why do the two MT19937 generators produce different sequences despite the same seed? How (if possible) can I make them the same?

Answer: The Mersenne Twister generator has a defined sequence for any seed that you give it. There are also test values that you can use to verify that the generator you use is conforming. The distributions are on the other hand not standardized and may produce different values in different implementations. Remove the distribution to compare the generators.  Note also that std::mt19937 is a 32 bit generator and it's not obvious (to me) if the numpy version is a 32 or 64 bit generator. You may want to compare std::mt19937_64 with the numpy implementation - still without involving a distribution of course.
Question: This is my dataframe: import pandas as pd df = pd.DataFrame( { 'a': ['a', 'a', 'a', 'c', 'b', 'a', 'a', 'b', 'c'], 'b': [20, 20, 20,-70, 70, -10, -10, -1, -1], } )  And this is the output that I want. I want to create a dataframe with two rows:  direction length sum 0 long 3 60 1 short 2 -20  I want to get the largest streak of positive (long) and negative(short) numbers in b. And then get the length and sum of b values in that streak and create a new dataframe. Note that if for example the largest streak is two and there are more than one streak with that size, I want the streak that its sum of b is more that the rest. In my df, the largest long streak is the first three values. And the largest short streak is two. Since there are more than one streak with that size, I want the one that its sum is more. So I want rows 5 and 6. This is what I have tried but I don't know how to follow up: df['streak'] = df['b'].ne(df['b'].shift()).cumsum() df['size'] = df.groupby('streak')['b'].transform('size') a b streak size 0 a 20 1 3 1 a 20 1 3 2 a 20 1 3 3 c -70 2 1 4 b 70 3 1 5 a -10 4 2 6 a -10 4 2 7 b -1 5 2 8 c -1 5 2 

Answer: You need to use several groupby operations, one to find the streaks, one to aggregate the length and sum, and one to filter the output with idxmax on the largest length: group = df['b'].ne(df['b'].shift()).cumsum() out = (df .assign(direction=np.sign(df['b'])) .replace({'direction': {1: 'long', -1: 'short'}}) .groupby(['a', 'direction', group], as_index=False) .agg(length=('b', 'count'), sum=('b', 'sum')) .sort_values(by='sum', key=abs, ascending=False) .loc[lambda d: d.groupby('direction')['length'].idxmax(), ['direction', 'length', 'sum']] )  Output:  direction length sum 0 long 3 60 1 short 2 -20  Intermediate before the final loc:  a direction length sum 0 a long 3 60 1 a short 2 -20 2 b long 1 70 3 b short 1 -1 4 c short 1 -70 5 c short 1 -1
Question: How can i avoid extra spaces in the tabulate grid? rows = [ ["A1", "B2"], ["C3", "D4"], ["E5", "E6"], ] print(tabulate(rows, headers="firstrow", tablefmt='rounded_outline'))  gives me 2 extra spaces in every cell ╭──────┬──────╮ │ A1 │ B2 │ ├──────┼──────┤ │ C3 │ D4 │ │ E5 │ E6 │ ╰──────┴──────╯  how can i solve it to get ╭────┬────╮ │ A1 │ B2 │ ├────┼────┤ │ C3 │ D4 │ │ E5 │ E6 │ ╰────┴────╯ 

Answer: You can leverage the MIN_PADDING constant. import tabulate rows = [ ["A1", "B2"], ["C3", "D4"], ["E5", "E6"], ] tabulate.MIN_PADDING = 0 print(tabulate.tabulate(rows, headers="firstrow", tablefmt="rounded_outline"))  Outputs: ╭────┬────╮ │ A1 │ B2 │ ├────┼────┤ │ C3 │ D4 │ │ E5 │ E6 │ ╰────┴────╯
Question: Running python pre-commit with black latest version 23.11.0 leads to a wired InvalidManifestError. snippet from .pre-commit-config.yaml repos: - repo: https://github.com/psf/black rev: 23.11.0 hooks: - id: black types: [] files: ^.*.pyi?$ # format .py and .pyi files`  output message: │ │ stdout = 'An error has occurred: InvalidManifestError: \n==> File │ │ │ │ /Users/robot/.cache/pre-c'+329 │ │ │ │ stdout_list = [ │ │ │ │ │ 'An error has occurred: InvalidManifestError: \n', │ │ │ │ │ '==> File │ │ │ │ /Users/robot/.cache/pre-commit/repoxhmwyits/.pre-commit-hooks.yaml\n', │ │ │ │ │ "==> At Hook(id='black')\n", │ │ │ │ │ '==> At key: stages\n', │ │ │ │ │ '==> At index 0\n', │ │ │ │ │ '=====> Expected one of commit, commit-msg, manual, merge-commit, │ │ │ │ post-checkout, '+86, │ │ │ │ │ 'Check the log at /Users/robot/.cache/pre-commit/pre-commit.log\n' │ │ │ │ ] 

Answer: you're using an outdated version of pre-commit. you need to be using at least version 3.2.0 which introduced the renamed stages  disclaimer: I wrote pre-commit
Question: I need to rewrite a lot of files, but I need them to keep their modification time as I need them indexed by time. I am aware that I could use os, but as I like pathlib's object oriented way, I would prefer to use pathlib if that is an option. Pathlib-docs mention getting the st_mtime, but not setting it. I only found this description at tutorialspoint.com (last example) which uses an assignment method that seems not to work: filepath.Path.stat().st_ctime = timestamp  Is it possible with pathlib or must I use os?

Answer: No, it's not. pathlib is for manipulating paths, not files. It's useful to think of it as a convenience wrapper around path string. Path.stat() is just a convenience method, doing nothing but calling os.stat() on path. So yes, you have to use os.utime().
Question: The C code: // my_module.c #include <stdio.h> __declspec(dllexport) void hello() { printf("Hello World!\n"); } __declspec(dllexport) int add_numbers(int a, int b) { return a + b; } // entry point int main() { return 0; }  The build script: # build.py from setuptools._distutils.ccompiler import new_compiler compiler = new_compiler() compiler.compile(["my_module.c"]) compiler.link_shared_lib(["my_module.obj"], "my_module")  The main script: # main.py import ctypes my_module = ctypes.CDLL("./my_module.dll") my_module.add_numbers.argtypes = ctypes.c_int, ctypes.c_int my_module.add_numbers.restype = ctypes.c_int my_module.hello.argtypes = () my_module.hello.restype = None result = my_module.add_numbers(3, 4) print(type(result), result) my_module.hello()  After running python build.py , the dll is created without issues. However, when running python main.py , the "add_numbers" function works, but calling the "hello" function results in "OSError: exception: access violation writing 0x0000000000002C44". Am I missing something? Do I somehow need to tell the compiler to include the "stdio.h" header?

Answer: it seems distutils is linking the msvc CRT incorrectly. you shouldn't import anything with an underscore such as _distutils, as it is not a part of the public API and you shouldn't use it. since this is a simple windows dll you could call cl.exe directly and compile it. (make sure you open x64 Native Tools Command Prompt for VS 2022 command prompt before you do) cl.exe /LD my_module.c  which will work, but if you have more files then you should probably create a cmake project for it and use it to build your C dll from python. a quick look at the dependencies of the one generated from distutils.  vs the one from just cl.exe directly.  copying all extra dependencies from the windows sdk to the dll folder should get it to work, but this is not the correct approach.
Question: I have a MultiIndex object with 2 levels: import pandas as pd mux = pd.MultiIndex.from_tuples([(1,1), (2,3)]) >>> MultiIndex([(1, 1), (2, 3)], )  I want to multiply it by a list l=[4,5], I tried pd.MultiIndex.from_product([mux.values, [4,5]]) >>> MultiIndex([((1, 1), 4), ((1, 1), 5), ((2, 3), 4), ((2, 3), 5)], )  I managed to get my expected result with from itertools import product pd.MultiIndex.from_tuples([(*a, b) for a, b in product(mux, [4,5])]) >>> MultiIndex([(1, 1, 4), (1, 1, 5), (2, 3, 4), (2, 3, 5)], )  Is there a better way to do this operation ?

Answer: I think your approach is quite reasonable. Another option using a cross-merge with an intermediate DataFrame format: pd.MultiIndex.from_frame(mux.to_frame().merge(pd.Series([4, 5], name=2), how='cross'))  Output: MultiIndex([(1, 1, 4), (1, 1, 5), (2, 3, 4), (2, 3, 5)], names=[0, 1, 2])
Question: This is my example data data = {'pro1': [1, 1, 1, 0], 'pro2': [0, 1, 1, 1], 'pro3': [0, 1, 0, 1], 'pro4': [0.2, 0.5, 0.3, 0.1]} df = pd.DataFrame(data)  I want to make striplot in seaborn like this (but actually wrong when running): sns.stripplot(x=['pro1', 'pro2', 'pro3'], y='pro4', data=df)  This is my alternative code: # Create a figure with two subplots that share the y-axis fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4), sharey=True) # List of column names l = ['pro1', 'pro2', 'pro3', 'pro4'] # Subplot 1: Positive values df1 = df.copy(deep=True) for i in l: # Set values to pro4 only when the corresponding pro1, pro2, pro3 is 1 df1[i] = df1.apply(lambda row: row['pro4'] if row[i] == 1 else None, axis=1) df1.drop(['pro4'], axis=1, inplace=True) sns.stripplot(data=df1, ax=ax1) ax1.set_title('Positive Values') # Add a title to the subplot # Subplot 2: Zero values df1 = df.copy(deep=True) for i in l: # Set values to pro4 only when the corresponding pro1, pro2, pro3 is 0 df1[i] = df1.apply(lambda row: row['pro4'] if row[i] == 0 else None, axis=1) df1.drop(['pro4'], axis=1, inplace=True) sns.stripplot(data=df1, ax=ax2) ax2.set_title('Zero Values') # Add a title to the subplot # Show the plots plt.show()  Result:  My questions are: "is there any more simple way to do for the same result like below?" sns.stripplot(x=['pro1', 'pro2', 'pro3'], y='pro4', hue = [0,1], data=df) 

Answer: In my opinion, the easiest is to melt and catplot: import seaborn as sns sns.catplot(df.melt('pro4'), x='variable', y='pro4', hue='variable', col='value', kind='strip')  Output:
Question: Example Data: Here is a simplified representation of my dataframes: My first dataframe(df1) is like:     col1 col2 col3 col4 col5     1 2 a 3 4   11 22 aa 33 44   111 222 aaa       And my second dataframe(df2) is something like:     col3 col4 col5     a 3 4   aa 332 442   aaa 333 444     And i want my merged dataframe (result_df) to look something like:     col1 col2 col3 col4 col5     1 2 a 3 4   11 22 aa 33 44   11 22 aa 332 442   111 222 aaa 333 444     I attempted to use the pd.merge function with a left join: result_df = pd.merge(df1, df2, on='col3', how='left') result_df looks like:     col1 col2 col3 col4_x col5_x col4_y col5_y     1 2 a 3 4 3 4   11 22 aa 33 44 332 442   111 222 aaa   333 444     I'm trying to understand why the resulting dataframe has additional columns and how to achieve the desired output. Any help or insights are greatly appreciated.

Answer: A proposition with merge/lreshape : mg = pd.merge(df1, df2, on="col3", how="left") grps = {c: [f"{c}_{s}" for s in ["x", "y"]] for c in df1.columns.intersection(df2.columns).drop("col3")} out = pd.lreshape(mg, grps).drop_duplicates().convert_dtypes()  NB: The loop is really optional here and can be replaced with a hardcoded mapping of the common columns (except the one to join on, i.e, col3) between both DataFrames : grps = {'col4': ['col4_x', 'col4_y'], 'col5': ['col5_x', 'col5_y']}  Output : print(out) col1 col2 col3 col4 col5 0 1 2 a 3 4 1 11 22 aa 33 44 3 11 22 aa 332 442 4 111 222 aaa 333 444 [4 rows x 5 columns]
Question: I want to autogenerate an ID field for my Pydantic model and I don't want to allow callers to provide their own ID value. I've tried a variety of approaches using the Field function, but the ID field is still optional in the initializer. class MyModel(BaseModel): item_id: str = Field(default_factory=id_generator, init_var=False, frozen=True)  I've also tried using PrivateAttr instead of Field, but then the ID field doesn't show up when I call model_dump. This seems like a pretty common and simple use case, but I can't find anything in the docs for how to accomplish this.

Answer: Use a combination of a PrivateAttr field and a computed_field property: from uuid import uuid4 from pydantic import BaseModel, PrivateAttr, computed_field class MyModel(BaseModel): _id: str = PrivateAttr(default_factory=lambda: str(uuid4())) @computed_field @property def item_id(self) -> str: return self._id print(MyModel().model_dump())
Question: I am trying to bin a date column. I was able to do that fairly easy using Pandas library. But I am getting the following error with Polars: Traceback (most recent call last): File "\workspace\polars.py", line 4, in <module> pl.col("ts").cut( File "\venvs\polars\Lib\site-packages\polars\utils\deprecation.py", line 192, in wrapper return function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "\venvs\polars\Lib\site-packages\polars\expr\expr.py", line 3716, in cut self._pyexpr.cut(breaks, labels, left_closed, include_breaks) TypeError: argument 'breaks': must be real number, not datetime.datetime  My Code is: >>> df ┌────────────┐ │ ts │ │ --- │ │ date │ ╞════════════╡ │ 2021-10-01 │ │ 2021-11-01 │ │ 2022-03-01 │ └────────────┘ >>> start_year = df.select("ts").min().item() >>> end_year = df.select("ts").max().item() >>> df = df.with_columns( pl.col("ts").cut( pl.datetime_range(start_year, end_year, interval="3mo", eager=True), ).alias("period_bins") ) 

Answer: This looks like a bug - do you want to report it to https://github.com/pola-rs/polars/issues ? For now, as a workaround, you could cast to int: breaks = pl.date_range(start_year, end_year, interval="3mo", eager=True) df.with_columns( pl.col("ts").cast(pl.Int64).cut( breaks.cast(pl.Int64), ).alias("period_bins") )