eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a sorted linked list class SortedLinkedList: # ... def insert(self, value: int): # ... if node.value > value: self.add_before(node, value) # ...  I would like to generalize the type of values that a Node can hold from only ints to any object that overloads the > operator by implementing the __gt__() magic method. In other languages I would achieve this by using an Interface, but Python apparently has no analog. I've seen suggestions to fake interfaces by using abstract classes like class Sortable(ABC): @abstractmethod def __gt__(self, other) -> bool: pass class SortedLinkedList: # ... def insert(self, value: Sortable, node: Node): # ...  The problem is this approach requires extending and using subclasses from Sortable, which means types that already have > functionality like integers cannot be used linkedlist.insert(5) # Pylance red squiggles  Argument of type "Literal[5]" cannot be assigned to parameter "value" of type "Sortable" in function "insert" "Literal[5]" is incompatible with "Sortable" Pylance(reportGeneralTypeIssues)  I understand that Interfaces are not necessary pre-runtime given Python's dynamic duck typing and implicit style. I am not a fan, and am opting to use available tooling like typing and Pylance to achieve a strictly typed developer experience. I am also not looking to use runtime checks like .hasattr(value, '__gt__'). I'm wanting this to register on the type system/language server/IDE level, as expressivity, readability, and IDE intellisense are the main benefits of strict typing. Is there any way to achieve this?

Answer: What you're looking for is typing.Protocol. class Sortable(Protocol): def __gt__(self, other) -> bool: ...  With this, any class that defines __gt__ will be detected as an implicit subtype of Sortable. Note that in Pythons >= 3.8, you may need to to make other positional: class Sortable(Protocol): def __gt__(self, other, /) -> bool: ...  This is because type checkers might not handle dunder-methods like __gt__ any different from a normal method and thus expects that the arguments might need to be callable by keyword. Adding the / tells the checkers that __gt__'s arguments won't ever need to be passed by keyword. Conversely, if an argument will only be passed by keyword, mark the protocol's method accordingly: class Something(Protocol): def method(self, *, name) -> str: ...
Question: I have a simple data generation question. I would request for any kind of help with the code in R or Python. I am pasting the table first.     Total Num1_betw_1_to_4 Num2_betw_1_to_3 Num3_betw_1_to_3     9 3 3 3   7 1 3 3   9 4 3 2   9 3 3 3   5 2 2 1   7 3 2 2   9 3 3 3   7 2 3 2   5      6      2      4      9        In the above table, first column values are given. Now I want to generate 3 values in column 2, 3 and 4 which sum up to value in column 1 for each row. But each of the column 2, 3 and 4 have some predefined data ranges like: column 2 value must lie between 1 and 4, column 3 value must lie between 1 and 3, and, column 4 value must lie between 1 and 3. I have printed first 8 rows for your understanding. In real case, only "Total" column values will be given and remaining 3 columns will be blank for which values have to be generated. Any help would be appreciated with the code.

Answer: This is straightforward in R. First make a data frame of all possible allowed values of each column: df <- expand.grid(Num1_1_to_4 = 1:4, Num2_1_to_3 = 1:3, Num3_1_to_3 = 1:3)  Now throw away any rows that don't sum to 7: df <- df[rowSums(df) == 7,]  Finally, sample this data frame: df[sample(nrow(df), 1),] #> Num1_1_to_4 Num2_1_to_3 Num3_1_to_3 #> 19 3 2 2
Question: Given the following scenario: import attrs @attrs.define(kw_only=True) class A: values: list[float] = attrs.field(converter=float) A(values=["1.1", "2.2", "3.3"])  which results in *** TypeError: float() argument must be a string or a real number, not 'list'  Obviously it's due to providing the whole list to float, but is there a way to get attrs do the conversion on each element, without providing a custom converter function?

Answer: As far as I know, attrs doesn't have a built-in option to switch conversion or validation to "element-wise", the way Pydantic's validators have the each_item parameter. I know you specifically did not ask for a converter function, but I don't really see much of an issue in defining one that you can reuse as often as you need to. Here is one way to implement a converter for your specific case: from attrs import define, field from collections.abc import Iterable from typing import Any def float_list(iterable: Iterable[Any]) -> list[float]: return [float(item) for item in iterable] @define class A: values: list[float] = field(converter=float_list) if __name__ == '__main__': a = A(values=["1.1", "2.2", "3.3"]) print(a)  It is not much of a difference to your example using converter=float. The output is of course A(values=[1.1, 2.2, 3.3]).  You could even have your own generic converter factory for arbitrary convertible item types: from attrs import define, field from collections.abc import Callable, Iterable from typing import Any, TypeAlias, TypeVar T = TypeVar("T") ItemConv: TypeAlias = Callable[[Any], T] ListConv: TypeAlias = Callable[[Iterable[Any]], list[T]] def list_of(item_type: ItemConv[T]) -> ListConv[T]: def converter(iterable: Iterable[Any]) -> list[T]: return [item_type(item) for item in iterable] return converter @define class B: foo: list[float] = field(converter=list_of(float)) bar: list[int] = field(converter=list_of(int)) baz: list[bool] = field(converter=list_of(bool)) if __name__ == '__main__': b = B( foo=range(0, 10, 2), bar=["1", "2", 3.], baz=(-1, 0, 100), ) print(b)  Output: B(foo=[0.0, 2.0, 4.0, 6.0, 8.0], bar=[1, 2, 3], baz=[True, False, True]) The only downside to that approach is that the mypy plugin for attrs (for some reason) can not handle this type of converter function and will complain, unless you add # type: ignore[misc] to the field definition in question.
Question: I want to write a calculator program. My goal is to replace multiple list items(with datatype str) by one(int). I've tried the .insert() method, but it creates a new inner list and places it at the end of the main list. Something like this: input_list = ['4','5','6','+','8','7','4'] #expected result output_list = [456, '+', 874] #actual result input_list = ['4','5','6','+','8','7','4',['4','5','6']]  I also tried extend method and also without success. My code: num = "" start = "" for x in range(len(list)): if list[x].isdigit() == True: if start == "": start = x num += list[x] continue else: num += list[x] continue else: num = int(num) list.insert(num,list[start:x]) num = "" start = "" continue 

Answer: You can use itertools.groupby and pass str.isdigit to its key. It will group the numbers together. from itertools import groupby input_list = ["4", "5", "6", "+", "8", "7", "4"] result = [] for k, g in groupby(input_list, key=str.isdigit): string = "".join(g) # It's True for numbers, False for operators. if k: number = int(string) result.append(number) else: result.append(string) print(result)  output: [456, '+', 874]  It's also possible with list-comprehension: from itertools import groupby input_list = ["4", "5", "6", "+", "8", "7", "4"] result = [ int("".join(g)) if k else "".join(g) for k, g in groupby(input_list, key=str.isdigit) ] print(result)   Note: I intentionally wrote two "".join(g) inside the list-comprehension. Because I had to. Don't try using walrus because you won't get your expected result: result = [ int((s := "".join(g))) if k else s for k, g in groupby(input_list, key=str.isdigit) ]  the (s := "".join(g)) part is only evaluated when the k is True, not always. That means if k is False, you'll get the previous value of s which is evaluated in the previous iteration which is '456'.
Question: My Pandas data frame contains the following data reading from a csv file: id,values 1001-MAC, 10 1034-WIN, 20 2001-WIN, 15 3001-MAC, 45 4001-LINUX, 12 4001-MAC, 67 df = pd.read_csv('example.csv') df.set_index('id', inplace=True)  I have to sort this data frame based on the id column order by given suffix list = ["WIN", "MAC", "LINUX"]. Thus, I would like to get the following output:  id,values 1034-WIN, 20 2001-WIN, 15 1001-MAC, 10 3001-MAC, 45 4001-MAC, 67 4001-LINUX, 12  How can I do that?

Answer: Here is one way to do that: import pandas as pd df = pd.read_csv('example.csv') idx = df.id.str.split('-').str[1].sort_values(ascending=False).index df = df.loc[idx] df.set_index('id', inplace=True) print(df)
Question: What is the proper way to document keys inside a TypedDict? I don't see much information inside PEP 589 – TypedDict. I can think of a few solutions:  Document keys inside a docstring (is there a standard field to use here?): class Foo(TypedDict): """ I am a documented schema. Keys: key: This key has some implications needing documentation. """ key: int   Document keys using inline comments inside the TypedDict class Foo(TypedDict): """I am a documented schema.""" # This key has some implications needing documentation key: int    Is there an official approach to this? Anything specific that Sphinx may look for?

Answer: If we look at the class typing.TypedDict(dict) signature in the documentation we see it's a class, as the official docs start by saying:  TypedDict declares a dictionary type that expects all of its instances to have a certain set of keys  So documenting the TypedDict is straightforward because it also uses Class Definition Syntax each of the variables (meant to express key name and value type pairs) act as Class and Instance Variables. When it's said in the question:  I can think of a few solutions:  Document keys inside a docstring Document keys using inline comments inside the TypedDict   You can't use type comments with the usual # type: comment syntax in a TypedDict definition. Otherwise the usual two alternatives of putting a docstring above each variable declaration, or documenting the members in the class docstring are valid.  PEP 589 - Class-based Syntax  The class body should only contain lines with item definitions of the form key: value_type, optionally preceded by a docstring. The syntax for item definitions is identical to attribute annotations, but there must be no initializer, and the key name actually refers to the string value of the key instead of an attribute name.  Type comments cannot be used with the class-based syntax, for consistency with the class-based NamedTuple syntax. (Note that it would not be sufficient to support type comments for backwards compatibility with Python 2.7, since the class definition may have a total keyword argument, as discussed below, and this isn’t valid syntax in Python 2.7.) Instead, this PEP provides an alternative, assignment-based syntax for backwards compatibility, discussed in Alternative Syntax.    Last but not least:   (...) (is there a standard field to use here?)   Using Google style docstrings with the napoleon extension the docstring section of choice would be Attributes:. But if you want to be more precise you can create a custom section using napoleon_custom_sections (here's an example post). I tried looking around but I couldn't find a TypedDict documented in any public API. Since there's isn't an established convention you can use what you prefer. (If your documentation explicitly shows class inheritance it's implicit to the type what the Attributes: section will mean.)
Question: I need modify a csv file with pandas. I have the following table: Interface Description 1 Used 2 Used 3 Used 4 Used 6 Used 8 Used 12 Used 17 Used  I need to match the "Interface" column with a range of 1, 20, complete the table with the missing numbers and place the word "free" in the "Description" column and order it like this: Interface Description 1 Used 2 Used 3 Used 4 Used 5 free 6 Used 7 free 8 Used 9 free 10 free 11 free 12 Used 13 free 14 free 15 free 16 free 17 Used 18 free 19 free 20 free 

Answer: Use merge in combination with fillna df = pd.DataFrame({ 'Interface': [1, 2, 3, 4, 6, 8, 12, 17], 'Description': 'Used'}) df2 = pd.DataFrame({'Interface': range(1, 21)}).merge(df, how="left").fillna("free")
Question: I am trying to load the parquet file with row size group = 10 into duckdb table in chunks. I am not finding any documents to support this. This is my work so on: see code import duckdb import pandas as pd import gc import numpy as np # connect to an in-memory database con = duckdb.connect(database='database.duckdb', read_only=False) df1 = pd.read_parquet("file1.parquet") df2 = pd.read_parquet("file2.parquet") # create the table "my_table" from the DataFrame "df1" con.execute("CREATE TABLE table1 AS SELECT * FROM df1") # create the table "my_table" from the DataFrame "df2" con.execute("CREATE TABLE table2 AS SELECT * FROM df2") con.close() gc.collect()  Please help me load both the tables with parquet files with row group size or chunks. ALso, load the data to duckdb as chunks

Answer: df1 = pd.read_parquet("file1.parquet")  This statement will read the entire parquet file into memory. Instead, I assume you want to read in chunks (i.e one row group after another or in batches) and then write the data frame into DuckDB. This is not possible as of now using pandas. You can use something like pyarrow (or fast parquet) to do this. Here is an example from pyarrow docs. iter_batches can be used to read streaming batches from a Parquet file. This can be used to read in batches, read certain row groups or even certain columns. import pyarrow.parquet as pq parquet_file = pq.ParquetFile('example.parquet') for i in parquet_file.iter_batches(batch_size=10): print("RecordBatch") print(i.to_pandas())  Above example simply reads 10 records at a time. You can further limit this to certain row groups or even certain columns like below. for i in parquet_file.iter_batches(batch_size=10, columns=['user_address'], row_groups=[0,2,3]):  Hope this helps!
Question: I am using python's wonderful plotnine package. I would like to make a plot with dual y-axis, let's say Celsius on the left axis and Fahrenheit on the right. I have installed the latest version of plotnine, v0.10.1. This says the feature was added in v0.10.0. I tried to follow the syntax on how one might do this in R's ggplot (replacing 'dot' notation with underscores) as follows: import pandas as pd from plotnine import * df = pd.DataFrame({ 'month':('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'), 'temperature':(26.0,25.8,23.9,20.3,16.7,14.1,13.5,15.0,17.3,19.7,22.0,24.2), }) df['month'] = pd.Categorical(df['month'], categories=('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'), ordered=True) p = (ggplot(df, aes(x='month', y='temperature')) + theme_light() + geom_line(group=1) + scale_y_continuous( name='Celsius', sec_axis=sec_axis(trans=~.*1.8+32, name='Fahrenheit') ) ) p  This didn't like the specification of the transformation, so I tried a few different options. Removing this altogether produces the error: NameError: name 'sec_axis' is not defined  The documentation does not contain a reference for sec_axis, and searching for 'secondary axis' doesn't help either. How do you implement a secondary axis in plotnine?

Answer: This github issue thread that was mentioned in the question does not say in any way that the secondary axis feature has been implemented. It was added to v0.10.0 milestones list before it was released. Here, milestones list means a todo list of what was planned to be implemented before the version releases. However, upon the actual release, the changelog does not mention the secondary axis feature, which means that it was only planned to be implemented and was not actually implemented. Long story short, the planned feature didn't make it into development and release. So, I'm sorry to say that currently as of v0.10.0 and now v0.10.1 it seems that this feature isn't there yet in plotnine.
Question: Here is my 2d array: [['P1', 27], ['P2', 44], ['P3', 65], ['P4', 51], ['P5', 31], ['P6', 18], ['P7', 33]]  I have created a loop to attach each appropriate label to its corresponding value within this 2d array. At the end of the loop I would like to subtract every value by a different number. For example: if I wanted to subtract the values by 5 [['P1', 22], ['P2', 39], ['P3', 60], ['P4', 46], ['P5', 26], ['P6', 13], ['P7', 28]]  How do I do this while the array has both a str type and int?

Answer: This is a simple job for a list comprehension: >>> L = [['P1', 27], ['P2', 44], ['P3', 65], ['P4', 51], ['P5', 31], ['P6', 18], ['P7', 33]] >>> [[s, n - 5] for s, n in L] [['P1', 22], ['P2', 39], ['P3', 60], ['P4', 46], ['P5', 26], ['P6', 13], ['P7', 28]]  Note that using a comprehension creates a new list and the original data will remain unmodified. If you want to modify the original in-place, it will be preferable to use a for-loop instead: >>> for sublist in L: ... sublist[1] -= 5 ... >>> L [['P1', 22], ['P2', 39], ['P3', 60], ['P4', 46], ['P5', 26], ['P6', 13], ['P7', 28]]
Question: I'm intending to use yt-dlp to download a video and then cut the video down afterward using ffmpeg. But to be able to use ffmpeg I am going to have to know the name of the file that yt-dlp produces. I have read through their documentation but I can't seem to find a way of getting the file name back into my program.

Answer: here are the doc examples the numbers you mentioned (like .f399) I believe are temp only and are eventually removed when the final file is merged. if you want to get the filename: import subprocess someFilename = subprocess.getoutput('yt-dlp --print filename https://www.youtube.com/something') # then pass someFilename to FFmpeg  to use your own filename: subprocess.run('yt-dlp -o thisIsMyName https://www.youtube.com/something') # this will likely download a file named thisIsMyName.webm  but if you are not sure of the file type/extension beforehand and just want to get that: someFileType = subprocess.getoutput('yt-dlp --print filename -o "%(ext)s" https://www.youtube.com/something') print(someFileType)  it's not very efficient but to help explain it: import subprocess someFileType = subprocess.getoutput('yt-dlp --print filename -o "%(ext)s" https://www.youtube.com/something') subprocess.run('yt-dlp -o "myFileName.%(ext)s" https://www.youtube.com/something') subprocess.run(f'ffmpeg -i "myFileName.{someFileType}" outputFile.mp4')
Question: I would like to customize the labels on the geopandas plot legend. fig, ax = plt.subplots(figsize = (8,5)) gdf.plot(column = "WF_CEREAL", ax = ax, legend=True, categorical=True, cmap='YlOrBr',legend_kwds = {"loc":"lower right"}, figsize =(10,6))   Adding "labels" in legend_kwds does not help. I tried to add labels with legend_kwds in the following ways, but it didn't work- legend_kwds = {"loc":"lower right", "labels":["low", "mid", "high", "strong", "severe"] legend_labels:["low", "mid", "high", "strong", "severe"] legend_labels=["low", "mid", "high", "strong", "severe"]

Answer: Since the question does not have reproducible code and data to work on. I will use the best possible approach to give a demo code that the general readers can follow and some of it may answer the question. The code I provide below can run without the need of external data. Comments are inserted in various places to explain at important steps. # Part 1 # Classifying the data of choice import pandas as pd import geopandas as gpd import matplotlib.pyplot as plt world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres')) world['gdp_per_cap'] = world.gdp_md_est / world.pop_est num_classes = 4 #quartile scheme has 4 classes # You can use values derived from your preferred classification scheme here num_qtiles = [0, .25, .5, .75, 1.] #class boundaries for quartiles # Here is the categorical data to append to the dataframe # They are also used as legend's label texts qlabels = ["1st quartile","2nd quartile","3rd quartile","4th quartile"] #matching categorical data/labels # Conditions # len(num_qtiles)-1 == num_classes # len(qlabels) == num_classes # Create a new column for the categorical data mentioned above world['gdp_quartile'] = pd.qcut(world['gdp_per_cap'], num_qtiles, labels=qlabels) # Plotting the categorical data for checking ax1 = world['gdp_quartile'].value_counts().plot(figsize=(5,4), kind='bar', xlabel='Quartile_Classes', ylabel='Countries', rot=45, legend=True)  The output of part1:-  # Part 2 # Plot world map using the categorical data fig, ax = plt.subplots(figsize=(9,4)) # num_classes = 4 # already defined #color_steps = plt.colormaps['Reds']._resample(num_classes) #For older version color_steps = plt.colormaps['Reds'].resampled(num_classes) #Current version of matplotlib # This plots choropleth map using categorical data as the theme world.plot(column='gdp_quartile', cmap = color_steps, legend=True, legend_kwds={'loc':'lower left', 'bbox_to_anchor':(0, .2), 'markerscale':1.29, 'title_fontsize':'medium', 'fontsize':'small'}, ax=ax) leg1 = ax.get_legend() leg1.set_title("GDP per capita") ax.title.set_text("World Map: GDP per Capita") plt.show()  Output of part2:-  Edit Additional code, use it to replace the line plt.show() above. This answers the question posted in the comment below. # Part 3 # New categorical texts to use with legend new_legtxt = ["low","mid","high","v.high"] for ix,eb in enumerate(leg1.get_texts()): print(eb.get_text(), "-->", new_legtxt[ix]) eb.set_text(new_legtxt[ix]) plt.show()
Question: I have written a R script where I use some python lines through the reticulate package. I need to share it with some colleagues who don't know about programming and I've created a batch file so I can run it without them even opening R. However, I tried using the install_miniconda() function to silently install python to run the code without them knowing (I guess people are reluctant to installing a couple of programs) but R throws an error: > reticulate::install_miniconda(path="C:/") # * Installing Miniconda -- please wait a moment ... # * Downloading "https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe" ... # trying URL 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe' # Content type 'application/octet-stream' length 74687656 bytes (71.2 MB) # downloaded 71.2 MB # Error: Miniconda installation failed [unknown reason]  I tried without passing any path but my computer user has spaces on it so I cannot use it, that's why I resorted to supplying it with the root path "C:/" Can someone help me understand what is happening? Note: I am using R 4.2.1 on Windows 11 (also tried on Windows 10 with the same result)

Answer: Try installing rminiconda from the github like this: remotes::install_github("hafen/rminiconda") rminiconda::install_miniconda(name='your_name')  After that you can specify the installation using reticulate like this: py <- rminiconda::find_miniconda_python("your_name") reticulate::use_python(py, required = TRUE)