BrandonFors/pandas-github-issues huggingface dataset:
Update is_platform_arm() to detect 32-bit arm and other variants
State: closed

- When running an arm32 ("linux32'd") chroot on an arm64 machine,
  Python's platform.machine() will return "armv8l".

- In other cases, on "real" arm32, it'll return whatever uname says
  (just like in the first case) which might be e.g. armv7a.

Keeping the other options ("aarch64", "arm64") given that Windows
or other kernels might choose to return different values and these
were added for a reason, but at least this fixes detection on Linux.

This allows tests like test_subtype_integer_errors to be skipped
as intended on arm.

Bug: https://bugs.gentoo.org/818964
Signed-off-by: Sam James <sam@gentoo.org>

- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: ARM
description: aarch64 architecture

Comments:
https://app.circleci.com/pipelines/github/pandas-dev/pandas/1865/workflows/32e3f6d0-8386-40e9-9fe1-1a540ae30de5/jobs/32707 looks like 1 test fails on this
> [app.circleci.com/pipelines/github/pandas-dev/pandas/1865/workflows/32e3f6d0-8386-40e9-9fe1-1a540ae30de5/jobs/32707](https://app.circleci.com/pipelines/github/pandas-dev/pandas/1865/workflows/32e3f6d0-8386-40e9-9fe1-1a540ae30de5/jobs/32707) looks like 1 test fails on this

Thanks! I wonder why, actually. That test isn't being run conditionally anyway and this PR is _supposed_ to reduce the number of tests we run on ARM, but that doesn't mean it's doing that. Will poke if it fails again after rebasing on master.
The test from before now passes, so I think it must either be flaky or was fixed on master since (thank goodness!).

The other failing CI just timed out.
thanks @thesamesam 
Thanks a lot @jreback! By the way, is there a procedure for requesting a backport for this? But no problem if we can't, it is only a small patch for us to carry downstream for a bit.

(This function is only used for checking if we should skip tests)

Cheers!
we only backport regressions 
No problem at all, thanks again!
REF: extract params used in Series.__repr__
State: closed

This PR gives flexibility if we want the series string output to be like the Series repr, but with some adjustments.

So with this PR we we can now do:

```python
>>> params = pd.io.formats.format.get_series_repr_params(ser)
>>> params[my_param] = new_value
>>> ser.to_string(**params)
```

Followup to #43987.


Labels:
name: Refactor
description: Internal refactoring of code
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
thanks @topper-123
BUG: Index/Series.to_frame not respecting explicit name=None
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Looks like this broke a doctest, will fix and turn that into a real test
hmm can you rebase again. oddly a number of failures.
REF: clarify missing.interpolate_2d is inplace
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Refactor
description: Internal refactoring of code
name: Performance
description: Memory or execution speed performance

Comments:
does this change any tests? i guess not

any perf diffs?
> does this change any tests? any perf diffs?

Doesn't change any behavior.  I haven't measured perf, but something truly weird would have to happen for perf to be affeced.
REF: Remove ArrowStringArray.fillna
State: closed

Equivalent to base class method.

Labels:
name: Strings
description: String extension data type and string data
name: Clean

Comments:
any performance benchmarks on this? did it change?
> any performance benchmarks on this? did it change?

didn't run any benchmarks, but the only difference between the base class method and the one here is this uses `self.to_numpy("object")` where the base class uses `self.astype(object)`
CLN: remove unused algos_common_helper functions
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
pls make sure that these are not exposed in any public api (i don't think so) but check `pd.api.types`
not exposed
TST: enable 2D tests for Categorical
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
can you rebase
rebased + green
CI: Python Dev build
State: closed

xref #44173

Ref: https://github.com/pandas-dev/pandas/issues/44173#issuecomment-952289166

Means we will have 3 more jobs (hopefully can reduce this back..) - but all can run in parallel.
https://github.com/pandas-dev/pandas/runs/4026818833?check_suite_focus=true

Labels:
name: CI
description: Continuous Integration

Comments:
thanks @alimcmaster1 very awesome!
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DOCS: Update pyarrow version requirement in "What's new in 1.4.0"
State: closed

Updated the pyarrow version requirement from 0.17 to 1.0.1 in the "Multithreaded CSV reading" section, to make it in line with #44064.

Docs only, no functional changes.


Labels:
name: Docs
name: Dependencies
description: Required and optional dependencies

Comments:
thanks @EwoutH
TST: added test_join_multiindex_dates
State: closed

- [ ] closes #33692
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

Added tests to make sure that  joining dataframes with multiindices containing dates doesn't break.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
closing as stale, pls ping if you want to continue working.
Fix series with none equals float series
State: closed

- [x] closes #44190
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
@github-actions pre-commit
Good catch!  Could use a whatsnew note
Hello @CloseChoice! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-10-28 03:51:53 UTC
As I've mentioned previously, for this specific case `left.equals(right)` wasn't symmetric. These cases can be tested well by hypothesis tests, what do you think about an issue for adding these tests to make sure that the function is symmetric (and maybe find some bugs)?
> As I've mentioned previously, for this specific case `left.equals(right)` wasn't symmetric. These cases can be tested well by hypothesis tests, what do you think about an issue for adding these tests to make sure that the function is symmetric (and maybe find some bugs)?

happy to have tests for symmetry. its ok to use hypothesis to find cases (but separate PR)
@jreback Alright, is there anything else to do here?
thanks @CloseChoice 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DEPR: warn on checks retained for fastparquet
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

cc @mdurant

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: IO Parquet
description: parquet, feather

Comments:
cc @martindurant 
can you rebase
PERF: read_csv GH#44106
State: closed

- [x] closes #44106
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Using the example in #44106 I time 188.4s on master and 3.9s on this PR.

Labels:
name: Performance
description: Memory or execution speed performance
name: IO CSV
description: read_csv, to_csv

Comments:
can you add a release note for 1.3.5
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
should have asked if we have an asv that covers this?
> should have asked if we have an asv that covers this?

not that i know of.  @haghshenas want to have a go at implementing a benchmark for this?
PERF: improve efficiency of `BaseMaskedArray.__setitem__`
State: closed

This somewhat deals with #44172, though that won't be fully resolved until 2D `ExtensionArray`s are supported (per the comments there).

CC @jbrockmendel

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance

Comments:
@alexreg status here? pls merge master as well
@jreback Sorry, haven't had much time lately. It would probably take me a while to properly wrap my head around the testing framework and write good tests. Ideally someone else could step in with that, but if not, maybe I'll get to it in a month or two.

I rebased on current master, however, and pushed that. (Nice and easy.)
> Ideally someone else could step in with that

None of us are going to write tests for you, but we will [help](https://github.com/pandas-dev/pandas/pull/44186#discussion_r740343732) you through getting the hang of writing them.

```
def test_descriptive_name():
    obj = pd.whatever()
    with pytest.raises(something, match=expected_exception_message):
         obj[0] = some_invalid_value_that_doesnt_raise_now_but_does_with_your_bugfix
```
ENH: added regex argument to Series.str.split
State: closed

- [X] closes #43563 
- [X] closes #32835
- [X] closes #25549
- [X] xref #37963 
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] whatsnew entry

I've preserved current behavior, in which regex = None. Currently, it handles the pattern as a regex if the length of pattern is not 1. I believe that in the future, this may be worth considering deprecating. 


Labels:
name: Enhancement
name: Strings
description: String extension data type and string data

Comments:
Hello @saehuihwang! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:







There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-11-02 23:23:40 UTC
cc @attack68 if any comments
lgtm
Before merging I think it would be worth just revisiting if we want to get rid of the pattern length 1 logic - its pretty ropey, and there is a good opportunity here.
> Before merging I think it would be worth just revisiting if we want to get rid of the pattern length 1 logic - its pretty ropey, and there is a good opportunity here.

Yes, once we have a `regex` kwarg, we could add a deprecation since we would have a path for users to change their code, but I don't think it needs to be done in this PR.

Some issues refer to consistency with the `replace` method, but mainly considering `split` here, I would probably like to see...

Firstly, `Series.str.split` to be compatible with Python `str.split`

So that would require that the default is to always treat `sep` as a literal and rename the `n` arg `maxsplit`

Secondly, add the functionality of re.split to the pandas method without affecting the above.

There are two ways to use the split capability of the Python `re` module, `re.split(pattern, string, maxsplit=0, flags=0)` and `Pattern.split(string, maxsplit=0)`

So I am happy that a `regex` kwarg is added (could perhaps be kwarg only) and also that the `sep` argument can now accept a compiled regex.

By accepting a compiled regex we are now implicitly adding the `case` and `flags` capability in this PR. 

Adding `flags` kwarg (in a follow-up) would make pandas `split` with `regex=True` and a string passed to `sep` consistent with Python `re.split(pattern, string, maxsplit=0, flags=0)`

And also adding a `case` kwarg as a special case of `flags` would make this consistent with pandas `replace`

I think these points are covered by https://github.com/pandas-dev/pandas/pull/44185/files#r739415484 and can be done as a follow-up.

@attack68 anything else preventing merging this

---

Thanks @saehuihwang for the PR.
no ,  think thats a great plan and lgtm
thanks @saehuihwang very nice.

would like to refactor the inner code if possible to have all of the regex handling in one place if you are interested.
BUG: np.timedelta64 + Period
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Period
description: Period data type
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
This can fail for 0D arrays

```python
import pandas as pd
import numpy as np
result = np.array(0) == pd.Period("2000", freq="B")
```
Now raises `TypeError: iteration over a 0-d array` while it returned `False`  before (where `type(result) == numpy.bool_`).
Good catch.  PR to address this will be welcome.
PERF: Improve performance in rolling.mean(engine=numba)
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

This also starts to add a shared aggregation function (sum) that can shared between rolling/groupby/DataFrame when using the numba engine.

```
df = pd.DataFrame(np.ones((10000, 1000)))
roll = df.rolling(10)
roll.sum(engine="numba", engine_kwargs={"nopython": True, "nogil": True, "parallel": True})
%timeit roll.sum(engine="numba", engine_kwargs={"nopython": True, "nogil": True, "parallel": True})

211 ms ± 12.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) <- PR
424 ms ± 9.23 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) <- master

```

Labels:
name: Performance
description: Memory or execution speed performance
name: Window
description: rolling, ewma, expanding

Comments:
great, can you make sure that we have sufficient benchmarks

e.g. 
- cython vs numba parallel=True & False
- small number of cols vs large number
TST: tests using invalid_scalar fixture
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
this could be a new user facing message right, can you add a whatsnew note
> this could be a new user facing message right, can you add a whatsnew note

not exactly; pretty hard to reach from Series/DataFrame methods
can you rebase
https://github.com/pandas-dev/pandas/runs/4051113097?check_suite_focus=true looks to be an actual issue
WEB: Update institutional sponsors list
State: closed

Makes the lists consistent with the governance docs and adds d-fine

cc @MarcoGorelli Would you like to add gousto too?

Labels:
name: Docs

Comments:
> Makes the lists consistent with the governance docs and adds d-fine
> 
> cc @MarcoGorelli Would you like to add gousto too?

Thanks for the tag, will do when I get a chance!

EDIT: admittedly I've been struggling to make use of time to contribute, but hope to be able to do so regularly in the future
thanks @phofl
Added two examples to documentation of DataFrame.std 
State: closed

- [x] closes part of  #44162

Added two examples to the documentation of DataFrame.std function so that users can understand how to use it with different delta degrees of freedom.

Labels:
name: Docs

Comments:
Thanks for all the hints, I'll be sure to run pre-commit next time.
I opened another pull request #44226 where I used pre-commit before committing changes. This should resolve the problems with this pull request.
in the future just push to the same branch to update the existing PR
BUG: metadata propagation in .loc, .iloc and Series.to_frame()
State: closed

 - [x] ~~closes~~ Incremental progress on #28283
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Hello @EmilianoJordan! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





















There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-11-13 06:11:39 UTC
Failures look to be happening on windows Python 3.9.

The test is: 
- pandas/tests/io/test_gcs.py  
- test_to_csv_compression_encoding_gcs[zip-utf-8] 

I'm not sure what the cause of these failures are and would appreciate some direction with this. 

perf impact?
I messed up the git management on this one.
TST:  Adding tests for checking Boolean series and df as indexes for series
State: closed

- [ ] closes #18579
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] After discussion in #18579 the current tests were suggested


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @bdrum
TST: Move some consistency rolling tests to misc rolling functions
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

Since some tests are not running on windows/moments (#37535), moving the "consistency" rolling tests to the relevant files.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Window
description: rolling, ewma, expanding

Comments:
actually pls rebase this one.
TST: Move some consistency expanding tests to test_expanding
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

Since some tests are not running on windows/moments (#37535), moving the "consistency" expanding tests to `test_expanding.py`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Window
description: rolling, ewma, expanding

Comments:
i guess the 3.10-dev are unrelated?
IO ExcelFile New Method
State: closed

…frame access

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew Added method to ExcelFile class to read all the sheets. This allows the users to access a dictionary of all pd.DataFrames using the .sheets() method and access them more easily. Before, you would need to then call a pd.read_excel(ExcelFile, sheet_name) to access the data in that sheet. 


Labels:

Comments:
Thanks for the contribution, but it appears your diff includes many files which doesn't seem intended.

We recommend following this guide for contributing: https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#working-with-the-code

Also, is this feature associated with a reported issue in the tracker? We require an issue and discussion on feature requests before proceeding, so closing this for now.

Let us know if you need help
Definitely some insights would be helpful! First time trying to do something like this so not surprised it didn't pan out. 


I added a method to the pd.ExcelFile class that reads all the sheets. I have found when I use this class, having this method helps my code be more succinct on the front end. I figured I would try and take a stab at making an update. 

It sounds like I need to:

Start an issue and have a discussion and then if there is support, submit my change according to more stringent guidelines. 

One quick question I hope can be answered: should I just fork the io.excel._base.py file or the entire package when making a small update?

Any help would be appreciated!


"""
def sheets(self):
        return {sheet: read_excel(self._io, sheet_name = sheet) for sheet in self._reader.sheet_names}
"""
> One quick question I hope can be answered: should I just fork the io.excel._base.py file or the entire package when making a small update?

Yes please clone the entire repository as documentation and tests, which are in different locations, would need updating if this feature is accepted.
CLN: remove unused fields.pyx code/asvs
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Clean

Comments:
LGTM :)
CI: Fix mypy failures
State: closed

- [x] closes #44143


Labels:
name: CI
description: Continuous Integration
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> Thanks @alimcmaster1 merge on green

ty - will do! Thanks for the prompt review!
Shouldn't the type (of the input and output) be changed to np.ndarray as the dtype might change?
LGTM
/azp run
<samp>
Azure Pipelines successfully started running 1 pipeline(s).<br>

</samp>