BrandonFors/pandas-github-issues huggingface dataset:
STYLE: moving unwanted pattern check to precommit
State: closed

- [x] closes #42123 
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them




Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
cool, thanks - can you remove "patterns" from ci/code_checks as well? (e.g. in line 20)
@MarcoGorelli done. 
Please let me know if this looks okay.
Thanks @debnathshoham
DOC: More subtotals / margins in pivot_table
State: closed

- [ ] closes #4817 
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Added an example in the documentation for DataFrame.pivot_table to show how to display a pivot table as having a multi-level index, simulating Excel's pivot table with MI and partial group aggregates.

Labels:
name: Docs
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @jxb4892
BUG: read_fwf raise if len colspec doesnt match len names
State: closed

- [x] closes #40830
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO Fixed Width
description: read_fwf

Comments:
@mzeitlin11 how's this look?
> @mzeitlin11 how's this look?

Sorry for delay, this LGTM, caveat that not too familiar with read_fwf, but with all the awesome added test coverage all these paths are exercised
thanks @debnathshoham
CI: Add ASV Bot
State: closed

- [ ] closes #41157
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Triggers with command ``@github-actions benchmark -b ^regexgoeshere``(doesn't work rn, needs to be merged to work, this is an example result though, https://github.com/lithomas1/pandas/pull/6#issuecomment-893920379, which is me testing it on the io.csv benchmarks). Our benchmarks are super slow right now, they take six hours to do, so having this bot might help for people with weaker computers. Unfortunately, this also can clog up our CI pipelines, so I added a concurrency limit, which restricts runs to 1 per person.

cc @jbrockmendel @simonjayhawkins 




Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
can u require a regex to target only matching benchmarks? would make it way faster
most PRs would be OK with `-b \(\?\!libs\)`
@lithomas1 happy to merge this and can PR again to update, lmk. (ideally would like the selective benchmarks though)
Looking to get some more changes in. This may take a while though.
Got the regexes working. e.g. https://github.com/lithomas1/pandas/pull/7#issuecomment-912154191

Unfortunately doesn't work with -b \(\?\!libs\) right now. I guess it doesn't like the backslash.

I think I'll try to get this to work in a follow up. This PR is already getting large.
thanks @lithomas1
TST: reorganize misplaced month / business month tests (#27085)
State: closed

- [ ] xref #27085 (chips away at, yes)
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Continuation of:
* https://github.com/pandas-dev/pandas/pull/42284 
* https://github.com/pandas-dev/pandas/pull/42896 

This change moves business month tests to own file and moves some displaced month tests into `test_month.py`.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Frequency
description: DateOffsets

Comments:
thanks @felixDulys
Backport PR #42304: DEPS: update setuptools min version
State: closed

Backport PR #42304

Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
needs #42903 merged first, see https://github.com/pandas-dev/pandas/pull/42720#issuecomment-893311648
@simonjayhawkins can rebase to make sure passes or just merge (after merged precursor)
TST: move custom business day tests to own file (#27085)
State: closed

- [ ] xref #27085 (chips away at, yes)
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Continuation of https://github.com/pandas-dev/pandas/pull/42284
Moving CustomBusinessDay / CDay tests into own file.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Frequency
description: DateOffsets

Comments:
thanks @felixDulys failures unrelated
TST: add read_json test with date values GH33787
State: closed

- [x ] closes #33787
- [x ] tests added / passed
- [x ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

First attempt at contributing to pandas, or any open source projects! I think all the linting tests passed, since I installed and used the pre-commit package described in the documentation and everything was either passed or skipped.

This change added a short test for GH33787, which was to make sure that read_json was able to convert lists with date values.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
thanks @horaceklai
API: make concat_compat match CategoricalIndex._concat
State: closed

- [x] closes #41626

This will probably need to be done with a deprecation, this is just to to clarify which tested behaviors would change.

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Categorical
description: Categorical Data Type
name: Mothballed
description: Temporarily-closed PR the author plans to return to

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
mothballing to clear the queue
TST: DataFrame initialization with an empty DataFrame with dtype
State: closed

- [x] closes #20654
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Hello @gcaria! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-08-23 13:49:31 UTC
Thanks for the help @jreback. 

However, it seems that pandas classes get capitalized in the test names (at least in this file), see the test just above (`test_constructor_DataFrame`) and the various `test_constructor_Series_...`.
> Thanks for the help @jreback.
> 
> However, it seems that pandas classes get capitalized in the test names (at least in this file), see the test just above (`test_constructor_DataFrame`) and the various `test_constructor_Series_...`.

Thanks for the PR - should be lower case as Jeff suggest and per PEP8. Can you double check why CI is failing/merge master? This looks good to me otherwise :) 
Thanks @alimcmaster1.Rebased to master and now all tests have passed
thanks @gcaria
CI: Revive Banklist Tests
State: closed

- [x] closes #38988
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
Hello @lithomas1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-09-01 04:50:03 UTC
can you rebase
This is un-mergeable in its current state because the lxml engine is inconsistent with the other engines which do read this in properly. It has to do with the ``attrs`` keyword, which when unspecified makes lxml return the correct result.

I do not have much time to patch the lxml engine right now because school has started again, and I am mostly focusing my efforts on other PRs.

Because the doc update just does a plain ``read_html``, I can update the PR to do a doc update only, if you want.
i see
we can just remove those 2
tests entirely as well
@github-actions pre-commit.
I removed the attrs checking component, and commented it. An incomplete test is better than no test I guess...
thanks looks fine. @alimcmaster1 if any comments.
thanks @lithomas1 - LGTM
BUG: Raise proper TypeError for scalar indexer in Index.take
State: closed

- [x ] closes https://github.com/pandas-dev/pandas/issues/42875#issuecomment-892736707
- [ x] tests added / passed
- [ x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x ] whatsnew entry

Fixes: https://github.com/pandas-dev/pandas/issues/42875#issuecomment-892736707
TypeError is being raised on passing scalar value in DataFrame.take

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Hello @radioactive11! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



* In the file [`pandas/tests/indexes/test_indexing.py`](https://github.com/pandas-dev/pandas/blob/e250e0ba74c484df1f21403f0d339d3d7834f6cd/pandas/tests/indexes/test_indexing.py):

> [Line 74:89](https://github.com/pandas-dev/pandas/blob/e250e0ba74c484df1f21403f0d339d3d7834f6cd/pandas/tests/indexes/test_indexing.py#L74): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (100 > 88 characters)



##### Comment last updated at 2021-09-25 12:57:16 UTC
you will have to add a test for this
@jreback From what I understand, I should add a test to check whether this exception is being raised on passing a scalar value, right? 
@jorisvandenbossche I have made the following changes as per your suggestions
- Checking for dimensions: if not a 1-dimensional array, then raise a TypeError
- Added a test to verify whether TypeError is being raised on passing a scalar/non 1 dimension array
@jreback I have made changes as per your suggestions. I also added `match` parameter in the test as @debnathshoham suggested
@debnathshoham I have made the changes as per your suggestions and the tests are passing as well. Please have a look and do suggest if further changes need to be made.
@jreback I changed it to `indices.ndim != 1` as per your suggestions. Also added a test case with indices being two-dimensional
there are a number of failing tests
@jreback I removed a blank line which was causing a flake8 test to fail. However I can still see a number of cancelled tests. When I run a subset of the tests which deal with the indexer (`pytest pandas/tests/indexes/test_indexing.py` to be specific), all of them pass. 
> @jreback I removed a blank line which was causing a flake8 test to fail. However I can still see a number of cancelled tests. When I run a subset of the tests which deal with the indexer (`pytest pandas/tests/indexes/test_indexing.py` to be specific), all of them pass.

your change as small as it is, is possibly affecting other tests, you will need to investigate

Appears this PR has been dormant for a while and the CI is still failing so closing. If you're interested in continuing please merge master, ensure the tests are passing and we can reopen
PERF: Groupby.shift dont re-call libgroupby.group_shift_indexer
State: closed

ATM we're calling libgroupby.group_shift_indexer for every column, but we only need to call it once.

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
wow is there a relevant asv that improves here?
> wow is there a relevant asv that improves here?

There are a few that reach here, but they are all single-column so won't see a boost.  #42841 adds multi-column cases to some of those
BUG: dataframe.rolling along rows drops float16
State: closed

- [x] closes #41779
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Window
description: rolling, ewma, expanding

Comments:
thanks @debnathshoham failures unrelated
TST: remove chained assignment outside indexing tests
State: closed

Small piece broken off from https://github.com/pandas-dev/pandas/pull/41878

This updates a  bunch of tests that are unnecessarily using chained assignment. Outside tests specifically about indexing, I think it's better/cleaner to use the more idiomatic single .loc call (and also makes it easier to change the behaviour as I am doing in https://github.com/pandas-dev/pandas/pull/41878).

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
> Outside tests specifically about indexing, I think it's better/cleaner to use the more idiomatic single .loc call

+1
DOC: add cookbook link about hidden sheets
State: closed

- [x] closes #19842 
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Just a tiny addition to the cookbook section of the User Guide.


Labels:
name: Docs
name: IO Excel
description: read_excel, to_excel

Comments:
thanks @ahawryluk
TST: Replace deprecated pyarrow.parquet.ParquetDataset
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Ref: [ARROW-13074](https://issues.apache.org/jira/browse/ARROW-13074?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=17391457#comment-17391457)


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
thanks @rhshadrach
BUG: GH42866 DatetimeIndex de-serializing fails in PYTHONOPTIMIZE mode
State: closed

- [x] closes #42866
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
Thanks @GorAlexander 

@jbrockmendel does pandas have any way of testing code run in `-O` mode?
> does pandas have any way of testing code run in -O mode?

take a look at test_oo_optimizable
Thanks - @GorAlexander do you want to have a look there and add a test?
Hello @GorAlexander! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-08-04 21:56:34 UTC
looks fine, can you add a whatsnew note. 1.4 bug fix, Datetimelike section. ping on green.
@github-actions pre-commit
Hi @jreback , I have fixed code-styles and added whatsnew note.
Is it possible to apply this fix to version 1.3?
would be ok for 1.3.x if u can move the note 
no we merge to master and then backport

just change the note
@jreback , thanks! I've moved the note to v1.3.2, so it must be ready.
Hi @jreback , the whatsnew note was corrected, however GH still highlights about 'changes requested'. Is there smth else to do to get the PR approved?
thanks @GorAlexander 
@meeseeksdev backport 1.3.x
Something went wrong ... Please have  a look at my logs.
DEPR: Index inferring numeric dtype from ndarray[object]
State: closed

- [x] closes #40489
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
> any perf implications?

haven't run any asvs, but i expect we'll pick up a bit in the constructor in the affected corner cases
Didn't check how it's related, but it seems it caused a 4x slowdown specifically in `is_all_dates` for MultiIndex and IntervalIndex:

https://pandas.pydata.org/speed/pandas/#index_cached_properties.IndexCache.time_is_all_dates?python=3.8&Cython=0.29.21&p-index_type='IntervalIndex'&p-index_type='MultiIndex'&commits=126a19d0
Add warning about NA merges
State: closed

This is in reference to #32306.

While the discussion on how to fix this is still ongoing, a warning will help users to at least know about this and take the appropriate precautions. 


Labels:
name: Docs
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
@pandas-dev/pandas-core if any comments.
thanks @kasuteru
REGR: `styler.highlight_min/max` did not ignore `pd.NA` and caused error
State: closed

- [x] closes #42750 
- [x] tests added / passed
- [x] whatsnew entry


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Styler
description: conditional formatting using DataFrame.style

Comments:
thanks @attack68 
@meeseeksdev backport 1.3.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.3.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 b9edc9a2a3e9cb3251b55e42a87cff2ded2ae6cb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #42861: REGR: `styler.highlight_min/max` did not ignore `pd.NA` and caused error'
```

4. Push to a named branch :

```
git push YOURFORK 1.3.x:auto-backport-of-pr-42861-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #42861 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.3.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 b9edc9a2a3e9cb3251b55e42a87cff2ded2ae6cb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #42861: REGR: `styler.highlight_min/max` did not ignore `pd.NA` and caused error'
```

4. Push to a named branch :

```
git push YOURFORK 1.3.x:auto-backport-of-pr-42861-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #42861 on branch 1.3.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
linking #42879
DEPR: unused kind arg in Index methods
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
great
I think this broke the stack level detection: https://github.com/pandas-dev/pandas/pull/42636/checks?check_run_id=3249879283
should be fixed in #42897
BUG :  Allow axis = 'columns' with limit and no method to work on fillna. 
State: closed

- [x] closes #40989
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

Added an if-condition for when axis = 1 so as to transpose the result in pandas/tests/frame/methods/test_fillna.py, and created a test for this change. The test also tests for `inplace =True`. This solves issue #40989 

Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
lgtm. @mzeitlin11 or @phofl if you'd take a quick look here.
thanks @GYvan
ENH: Provide a method to convert DataFrames to structured arrays.
State: closed

Structured arrays can be significantly faster than record arrays:
```
In [1]: s = np.random.rand(1000, 2).astype([("a", float), ("b", float)]); r = s.view(np.recarray)

In [2]: %timeit s["a"]
69.4 ns ± 0.0733 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [3]: %timeit r["a"]
1.2 µs ± 10 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [4]: %timeit r.a
1.86 µs ± 7.42 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
```
Therefore it can be desirable to convert DataFrames to
structured arrays rather than record arrays (the recipe
for the conversion directly comes from the numpy docs at
https://numpy.org/doc/stable/user/basics.rec.html#record-arrays,
ignoring the handling of non-record inputs).

Currently the implementation simply reuses the to_record implementation.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:

Comments:
Feel free to close if you're not going to accept this, I don't mind keeping this as a helper in my own utils.py...
Agreed with Jeff.  A note in the docstring would be OK
I'm not sure what specifically you suggest adding to the docstring?
Backport PR #42787: BUG: 1D slices over extension types turn into N-dimensional slices over ExtensionArrays
State: closed

Backport PR #42787

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Thanks for taking care of this, @simonjayhawkins!