eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Let's say I have a list of 15 different points with x and y coordinates. How can I find the 5 most collinear points? They don't need to be perfectly collinear but they should reliably be the most collinear points of all possible combinations. My current approach:  Get all possible combinations using itertools.combinations Iterate over every combination of 5 points Calculate the slope from first point to every other point Calculate the absolute differences between the first slope and all other slopes Sum up all differences Compare the sums of all combinations and return the combination with the smallest sum  This works sometimes but sometimes also returns garbage. Here is an example: The algorithm works with the points_0deg list. If I take the same points and rotate them 80 degrees (points_80deg) the red points are found to be the most collinear instead of the green points.  import itertools points_0deg = [[818.5, 395.5], [688.5, 586.5], [556.5, 448.5], [819.5, 779.5], [657.5, 892.5], [558.5, 727.5], [658.5, 278.5], [453.5, 279.5], [426.5, 588.5], [877.5, 589.5], [458.5, 893.5], [301.5, 403.5], [296.5, 774.5], [241.5, 583.5], [558.5, 585.5]] points_80deg = [[498.5, 397.5], [665.5, 558.5], [505.5, 665.5], [356.5, 533.5], [781.5, 711.5], [876.5, 463.5], [958.5, 641.5], [619.5, 816.5], [700.5, 373.5], [925.5, 838.5], [414.5, 906.5], [322.5, 737.5], [580.5, 998.5], [776.5, 975.5], [640.5, 687.5]] def check_collinear(points): x1, y1 = points[0] slope = [] for point in points[1:]: x, y = point slope.append((y - y1) / (x - x1) if (x - x1) != 0 else float('inf')) diff = [] for val in slope[1:]: diff.append(abs(slope[0] - val)) return sum(diff) def find_most_collinear(points): combinations = list(itertools.combinations(points, 5)) collinear_points = None min_sum = float('inf') for comb in combinations: s = check_collinear(comb) if s <= min_sum: min_sum = s collinear_points = comb return collinear_points if __name__ == '__main__': print(find_most_collinear(points_0deg)) print(find_most_collinear(points_80deg)) 

Answer: A simple algorithm is to find the two extreme points of the set (find the average M, then the point A most distant from M and then the point B most distant from A). Then as score use the sum of squared distances from the line passing through A and B. In code: # gg is the candidate group avg = ((sum(x for x, y in gg)/5), (sum(y for x, y in gg)/5)) ax, ay = max(gg, key=lambda p:(p[0]-avg[0])**2 + (p[1]-avg[1])**2) bx, by = max(gg, key=lambda p:(p[0]-ax)**2 + (p[1]-ay)**2) abx, aby = bx - ax, by - ay ab2 = abx**2 + aby**2 def dist2(p): t = ((p[0] - ax)*abx + (p[1] - ay)*aby) / ab2 xx = ax + t*abx yy = ay + t*aby return (p[0]-xx)**2 + (p[1]-yy)**2 score = sum(dist2(p) for p in gg) if best is None or score < best[0]: best = score, gg # lower wins  Instead of just the line passing from extremes the correct thing would be to find the line with minimum score but that's a bit annoying to compute exactly and this very simple method may be is enough for your use cases
Question: How can I apply a function over last two dimensions? E.g. I generated an array below of (2,3,3) dimensions, the resulting array should have the same dimenstions where the function is apply to a[0,:,:] and a[1,:,:]. I understand I can go with a for loop, but might there be an in-built function specially for these type of operations? a = np.arange(18).reshape(2,3,3) c = [] for i in range(2): c.append(np.linalg.pinv(a[i,:,:])) result = np.array(c)  Here I used np.linalg.pinv but assume a generic function f: (n,k)-> (n,k), e.g f = lambda x: x**3 +61

Answer: Assuming a 3D array, you can directly map your function on the array, this will loop over the first dimension and apply the function on the remaining dimensions: out = np.array(list(map(np.linalg.pinv, a)))  NB. this is not vectorized. Output: array([[[-5.55555556e-01, -1.66666667e-01, 2.22222222e-01], [-5.55555556e-02, 1.98795266e-16, 5.55555556e-02], [ 4.44444444e-01, 1.66666667e-01, -1.11111111e-01]], [[-1.30555556e+00, -1.66666667e-01, 9.72222222e-01], [-5.55555556e-02, 1.31569730e-15, 5.55555556e-02], [ 1.19444444e+00, 1.66666667e-01, -8.61111111e-01]]])
Question: I published a package on PyPI and then realised I should change a few details in the attached README.md. Is it possible to change the readme without uploading a new version of the whole package? And if not, what is the most correct way to update the README on PyPI? I am using poetry to manage the package, and when I change README and run poetry publish --build, there is an error, because PyPI does not allow re-uploading a file with an already used name. It seems like only changing the documentation should be a possibility, as, e.g., writing some extra help or changing a contact should not require a whole round of changing version and testing that everything still works?

Answer: Unfortunately, no it isn't, because when you go to publish, the versions are immutable. This is by design. My suggestion would be to increment a minor version z on x.y.z in order to publish the latest version with updates to the README. So something like: 0.0.9 -> 0.0.10 or 2.1.0 -> 2.1.1
Question: When a python dataclass has a simple attribute that only needs a default value, it can be defined either of these ways. from dataclasses import dataclass, field @dataclass class ExampleClass: x: int = 5 @dataclass class AnotherClass: x: int = field(default=5)  I don't see any advantage of one or the other in terms of functionality, and so would go with the less verbose version. Of course field offers other bells and whistles, but I don't need them yet and could easily refactor to use field later. Is there any advantage to using field for a simple default over just a type hint?

Answer: No, if all you need is a field with a default value and no other special behavior, assigning the value directly to the class variable is equivalent to a field with only a default parameter. x: int = field(default=5) x: int = 5  In fact, Python goes way out of its way to make sure the two behave equivalently. From PEP 557,  If the default value of a field is specified by a call to field(), then the class attribute for this field will be replaced by the specified default value. If no default is provided, then the class attribute will be deleted. The intent is that after the dataclass decorator runs, the class attributes will all contain the default values for the fields, just as if the default value itself were specified.  So whether or not you assign a field, at runtime, the value of both of the x's above on the class (as opposed to an instance) will be the number 5. For the sake of completeness, the reasons you would want to call field rather than writing simply a type annotation include:  Providing a default_factory, which is a callable that runs every time an instance is constructed (i.e. x: list = [] will be the same list every time, whereas x: list = field(default_factory=list) will be a new list for each instance) Removing the field from the parameters from consideration in some or all of the dataclass-generated functionality. You can remove the field from __init__ params, from the printed __repr__, from the comparison method, and from the generated __hash__. Adding third party metadata. This doesn't affect dataclasses but can be used to share arbitrary information about a dataclass field with a third party library.  If none of the above apply to you, stick to the simple syntax and don't call field.
Question: I've got a very long list of keys, and I am calling a REST API with each key to GET some metadata about it. The API can only accept one key at a time, but I wondered if there was a way I could batch or multi-thread the calls from my side?

Answer: The other reply to this looks like ChatGPT so it should be ignored. I did, however, use its code as a base to write a function that does what I want. import requests from concurrent.futures import ThreadPoolExecutor API_ENDPOINT = 'https://api.example.com/metadata' def get_metadata_for_key(key): url = f"{API_ENDPOINT}/{key}" response = requests.get(url) if response.status_code == 200: return response.json() else: return None def get_save_metadata(keys, workers): results = {} batches = [keys[i : i + workers] for i in range(0, len(keys), workers)] with ThreadPoolExecutor(max_workers=workers) as executor: for batch in tqdm(batches): #tqdm shows a progress bar futures = {key: executor.submit(get_metadata_for_key, key) for key in batch} futures_clean = {k: v.result() for k, v in futures.items() if v is not None} results.update({k: xmltodict.parse(v) for k, v in futures_clean.items()}) return results
Question: I am having a dataframe df: columnA columnB columnC columnD columnE A B 10 C C A B 10 D A B C 20 A A B A 20 D A B A 20 D C  I want to drop the duplicates if there are duplicates entries for columnA, columnB, columnC in my case the duplicates are: columnA columnB columnC columnD columnE A B 10 C C A B 10 D A B A 20 D A B A 20 D C  How can I keep the one of the duplicate rows, where columnE is equal to C ? So that the output for the full dataframe is: columnA columnB columnC columnD columnE A B 10 C C B C 20 A A B A 20 D C 

Answer: You can use DataFrame.sort_values for prefer C values first with DataFrame.drop_duplicates and or original order add DataFrame.sort_index: out = (df.sort_values('columnE', key=lambda x: x.ne('C')) .drop_duplicates(['columnA','columnB','columnC']) .sort_index()) print (out) columnA columnB columnC columnD columnE 0 A B 10 C C 2 B C 20 A A 4 B A 20 D C  Or use DataFrameGroupBy.idxmax for indices with prefer C with DataFrame.loc for select rows and Series.sort_values for original ordering: idx = df['columnE'].eq('C').groupby([df['columnA'],df['columnB'],df['columnC']]).idxmax() out = df.loc[idx.sort_values()] print (out) columnA columnB columnC columnD columnE 0 A B 10 C C 2 B C 20 A A 4 B A 20 D C
Question: I am trying to learn how to use the .startswith() method and also the filter() function. For some reason I am not getting the result I expected. I don't know if I am misunderstanding 'startswith' or 'filter'. here is the code i used names = ['aaron','anthony','tom','henry','barry'] def start_a(names): for name in names: if name.startswith('a'): return True print(list(filter(start_a, names)))  i was expecting to get ['aaron', 'anthony'] however i got ['aaron', 'anthony', 'barry'] does anyone know where i went wrong? thanks

Answer: start_a() is looping over the characters in the name, because it just receives one list element as its parameter. So it's actually checking whether the name contains a, not whether it starts with a. filter() does the looping over the list for you, you don't need another loop in the function. def start_a(name): return name.startswith('a')
Question: Let's say I want to install pybox2d (but this applies to other packages as well), and I can see on the Anaconda website that this package obviously exists, but it cannot be found when trying to install it on my new Macbook (one of the ones with the new Apple M1 or M2 CPUs). What should I do? conda search pybox2d -c conda-forge Loading channels: done No match found for: pybox2d. Search: *pybox2d* PackagesNotFoundError: The following packages are not available from current channels: - pybox2d Current channels: - https://conda.anaconda.org/conda-forge/osx-arm64 - https://conda.anaconda.org/conda-forge/noarch - https://repo.anaconda.com/pkgs/main/osx-arm64 - https://repo.anaconda.com/pkgs/main/noarch - https://repo.anaconda.com/pkgs/r/osx-arm64 - https://repo.anaconda.com/pkgs/r/noarch To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org and use the search bar at the top of the page.  Note: There are other Stack Overflow questions which relate to this but don't quite answer the question as I have asked it here, so I wanted to make it more direct.  How to specify the architecture or platform for a new conda environment? (Apple Silicon) How to set up a conda osx-64 environment on ARM mac? Cannot install Python 3.7 on osx-arm64 

Answer: If you look at the linked webpage above (at the time of writing), you can see that osx-arm64 is not listed under the "Installers". However, in the output above we can see that we are only searching in osx-arm64, and not in osx-64. The explanation for this is that this package is not built for our new Apple architecture (the new M1 and M2 chips are ARM-based, not Intel-based). However, it is possible to run programs which were compiled for Apple Intel using some kind of translation called Rosetta. We can have some python environments which are built off of the x86 instruction set. To do this, just add CONDA_SUBDIR=osx-64 to your env creation. You will also want to configure that environment so that it always installs this type of package. This way, your other environments can still be osx-arm64 by default, and you can just configure an environment as osx-64 only when needed (only when you need an old package that hasn't been cross-compiled to the new architecture). $ CONDA_SUBDIR=osx-64 conda create -n my_intel_env --file environment.yml $ conda activate my_intel_env $ conda config --env --set subdir osx-64  The downside of using the old architecture, as far as I know, is increased startup costs for the program, when the program needs to be translated from one instruction set to another. I'm not sure if this happens only once or more often. See Also  If you need to install Tensorflow, see this blog. If you are using PyTorch, you may also need to downgrade mkl for now. How to specify the architecture or platform for a new conda environment? (Apple Silicon) How to set up a conda osx-64 environment on ARM mac?
Question: Python3.11 introduced StrEnum and IntEnum which inherit str or int respectively, and also inherit ReprEnum, which in turn inherits Enum. ReprEnum's implementation is actually empty. >>> print(inspect.getsource(ReprEnum)) class ReprEnum(Enum): """ Only changes the repr(), leaving str() and format() to the mixed-in type. """  If I create a StrEnum and check the MRO, I can see that str comes first. class Strings(StrEnum): A = "a"  >>> Strings.__mro__ (<enum 'Strings'>, <enum 'StrEnum'>, <class 'str'>, <enum 'ReprEnum'>, <enum 'Enum'>, <class 'object'>)  Both str and Enum define a __str__ and a __repr__ >>> str.__repr__ <slot wrapper '__repr__' of 'str' objects> >>> str.__str__ <slot wrapper '__str__' of 'str' objects> >>> Enum.__repr__ <function Enum.__repr__ at 0x7ffff69f72e0> >>> Enum.__str__ <function Enum.__str__ at 0x7ffff69f7380>  How then does __repr__ get inherited from Enum and __str__ get inherited from str?

Answer: The __repr__ method comes the normal way, inherited from Enum (via StrEnum) >>> Strings.__repr__ is StrEnum.__repr__ is Enum.__repr__ True  For the __str__ method, the metaclass EnumType checks for the presence of ReprEnum and "hoists up" the str and format handling of the mixed-in data type into the class namespace at class definition time here: class EnumType(type): ... def __new__(metacls, cls, bases, classdict, *, boundary=None, _simple=False, **kwds): ... # Also, special handling for ReprEnum if ReprEnum is not None and ReprEnum in bases: if member_type is object: raise TypeError( 'ReprEnum subclasses must be mixed with a data type (i.e.' ' int, str, float, etc.)' ) if '__format__' not in classdict: enum_class.__format__ = member_type.__format__ classdict['__format__'] = enum_class.__format__ if '__str__' not in classdict: method = member_type.__str__ if method is object.__str__: # if member_type does not define __str__, object.__str__ will use # its __repr__ instead, so we'll also use its __repr__ method = member_type.__repr__ enum_class.__str__ = method classdict['__str__'] = enum_class.__str__ ...  Now that a Strings.__str__ method may be found directly in the class namespace, the MRO needn't be traversed.
Question: I am new to using regex but I feel my pattern may be too complex. I am looking for a pattern of a minimum number of brackets with a maximum number of dots interspersed. I can't see a way for regex to count the numbers of dots in the overall pattern instead of sequentially. For example: ...((((((((.(((..((..((((.(((((((.(..(((((.(((.(((...))).))).)))))..)..))))))).))))..))..))).))))))))(((.((.(((((...((........))))))))))))............  If I want to identify a run of at least 25 (s with a maximum of 15 .s interspersed from the first ( to the last:  ...((((((((.(((..((..((((.(((((((.(..(((((.(((.(((...))).))).)))))..)..))))))).))))..))..))).))))))))(((.((.(((((...((........))))))))))))............  My regex is currently searching for a a sequence with a maximum of 15 consecutive .s instead. Is this possible? If not should I be using an alternative (i.e. pyparsing) This is what I have so far: (\.{0,15}\(){25,} 

Answer: I think you can use a combination of regex and string manipulation in python like this for example : import re #sample data text = "...((((((((.(((..((..((((.(((((((.(..(((((.(((.(((...))).))).)))))..)..))))))).))))..))..))).))))))))(((.((.(((((...((........))))))))))))............" #to match everything between the outer brackets pattern = r"\([^()]*\)" #find all matches of the pattern matches = re.findall(pattern, text) #iterate through the matches for match in matches: dot_count = match.count(".") if dot_count <= 15: print("Pattern matched!") break else: print("Pattern not matched.")  Output: Pattern matched!
Question: My Dataframe looks like this.     id age Gender snapshot_1 performance_13 snapshot_5 performance_17 snapshot_7 performance_19     1 34 M   80 30 40 30   2 42 F 65 55 60 15 25 45     ALL Id's data need to be grouped for snapshot/performance with ID repetition like below. For Snapshot and its corresponding performance window could be fetched from the number after underscore from 1st dataframe.     ID Age Gender Snapshot_Window Performance_window Snapshot_Value Performance_Value     1 34 M 1 13     2 42 F 1 13 65 55   1 34 M 5 17 80 30   2 42 F 5 17 60 15   1 34 M 7 19 40 30   2 42 F 7 19 25 45    

Answer: Using a MultiIndex to reshape, to handle an arbitrary number of categories:  tmp = df.set_index(['id', 'age', 'Gender']) idx = (tmp.columns.to_series().str.split('_', n=1, expand=True) .assign(n=lambda x: x.groupby(0).cumcount()) ) out = (tmp .set_axis(pd.MultiIndex.from_frame(idx[[0, 'n']]), axis=1) .stack(dropna=False).add_suffix('_value') .reset_index(tmp.index.names) .join(idx.pivot(index='n', columns=0, values=1) .add_suffix('_window') ) .rename_axis(index=None, columns=None) )  Output:  id age Gender performance_value snapshot_value performance_window snapshot_window 0 1 34 M NaN NaN 13 1 0 2 42 F 55.0 65.0 13 1 1 1 34 M 30.0 80.0 17 5 1 2 42 F 15.0 60.0 17 5 2 1 34 M 30.0 40.0 19 7 2 2 42 F 45.0 25.0 19 7
Question: I have successfully installed some python code on a Win10 machine using Anaconda and conda environments and would like to install exactly the same environment on another computer, also Win10. This page  indicates that you can save a file that contains the environment info on computer 1, then recall it on computer 2, using:  computer 1: conda list --explicit > spec-file.txt copy the file from the working directory of computer 1 to the working directory of computer 2 computer 2: conda create --name myenv --file spec-file.txt  Step 1 works fine for me but step 3 fails with a ResolvePackageNotFound error, listing basically all the packages that are in the text file. Am I missing something? Is there a way to semi-automatically install the packages from that text file instead? Edit: summary from the answers and comments (thank you!):  if conda install was used to install packages in computer 1, then what is written above is the best way to do an exact replication. if pip install was used instead for packages installation (as was the case for me), the chosen solution below is the most appropriate one (conda env export and conda env create) 

Answer: try using export -- Computer 1 - conda env export > spec-file.yml Computer 2 - conda env create -f spec-file.yml
Question: In python polars, I was wondering if it will be possible to use .eval() to perform an operation between an element and a column. For example, given the following dataframe: import polars as pl df = pl.DataFrame({"list": [[2, 2, 2], [3, 3, 3]], "scalar": [1, 2]})  Is it possible to subtract each element of the list column by the value of scalar column? i.e. from this shape: (2, 2) ┌───────────┬────────┐ │ list ┆ scalar │ │ --- ┆ --- │ │ list[i64] ┆ i64 │ ╞═══════════╪════════╡ │ [2, 2, 2] ┆ 1 │ │ [3, 3, 3] ┆ 2 │ └───────────┴────────┘  to this shape: (2, 3) ┌───────────┬────────┬───────────┐ │ list ┆ scalar ┆ diff │ │ --- ┆ --- ┆ --- │ │ list[i64] ┆ i64 ┆ list[i64] │ ╞═══════════╪════════╪═══════════╡ │ [2, 2, 2] ┆ 1 ┆ [1, 1, 1] │ │ [3, 3, 3] ┆ 2 ┆ [1, 1, 1] │ └───────────┴────────┴───────────┘ 

Answer: I think that native functionality for this is on the roadmap (see this github issue https://github.com/pola-rs/polars/issues/8006) but you can do this as follows: df.with_row_index().pipe( lambda df: df.join( df.explode("list") .with_columns(sub=pl.col("list") - pl.col("scalar")) .group_by("index") .agg(pl.col("sub")), on="index", ) )  Basically, I add an index column to have a unique ID for each row. Then I pipe so I can use this index column in further operations. I do a join to add the arithmetic column. In the join I explode the list column to get it as rows, do the arithmetic then do a group_by to gather things back into a list for each row and join this new column back to the df. I'm sure there are other ways to do it but this should get you going
Question: I'm working with DuckDB in Python (in a Jupyter Notebook). How can I force DuckDB to print all rows in the output rather than truncating rows? I've already increased output limits in the Jupyter Notebook. This would be the equivalent of setting .maxrows in the CLI, but I can't find how to do this in Python.

Answer: You can use show: duckdb.sql("select * from range(100)").show(max_rows=100)
Question: I'd like to find the number of months between today and a date field in a python polars dataframe. How do you do that? (datetime.today() - pl.col('mydate')).dt.days()  returns it only in days.

Answer: I figured out a way to do it using the pl.date_ranges() function. This mimics the Excel DATEDIF formula. Essentially, you are creating a list of unique months between the start date and the end dates, then calculating the length of the list to find the months. df.with_columns(diffInMonths=(pl.date_ranges(pl.col('Start Date'),pl.col('End Date'),'1mo',closed='right')).list.len())  This was the smoothest way for me to calculate the difference between two dates in months. Be aware: pl.date_range() and pl.date_ranges() do different things, so be sure to use the plural version.
Question: I'm trying to document instance variables in my Python classes so they show up in VS Code when I hover over them. I've found that this works: class TsCity: def __init__(self) -> None: self.name: str = "" """The city name."""  But this is pretty ugly. I would ideally like to use the Google-style docstring instead: self.name: str = "" #: Doc comment *inline* with attribute  But this doesn't show up in VS Code properly. Is there a way to make this type of docstring work in VS Code?

Answer: This (#: lorem ipsum) is the Sphinx/Google style of attribute documentation (as opposed to the pep-0224 style (""" lorem ipsum """)) I asked the maintainers of the Python extension about whether there's an existing feature request issue ticket asking for Sphinx-style attribute documentation at https://github.com/microsoft/pylance-release/issues/1576#issuecomment-1668718385. One of them (Rich Chiodo) replied:  I don't believe so. But if it's not part of standard doc strings, it would probably take a lot of up votes for us to add it.  I take that to mean that- no, there is no way to get hover info for this style of attribute doc comments at the time of this writing- at least not with Microsoft's Python extension (the most popular Python extension at the time of this writing). Perhaps there's an extension that adds support for this, but I don't know of it. Consider making a feature-request issue ticket to the Microsoft Python repo for this feature.