BrandonFors/pandas-github-issues huggingface dataset:
REF: Enhance consistency on Styler methods
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- Added an entry in the latest `doc/source/whatsnew/v1.4.1.rst` file if fixing a bug or adding a new feature. NOT NEEDED

pandas 1.4.0 introduces deprecation of `hide_columns()` and `hide_index()` in favor to `hide()`. Problem is the messages docstrings, parameters and warning messages pass parameters in different ways making it confusing and non-readable. This PR was created to address that and make all those consistent.


Labels:
name: Styler
description: conditional formatting using DataFrame.style

Comments:
This effectively just changes the quotation symbol from single to double quotes in a warning. No need for a whatsnew entry.

Not really convinced that the error is unreadable with single quotes instead of double:

![Screen Shot 2022-02-02 at 19 08 20](https://user-images.githubusercontent.com/24256554/152212304-d171f664-c502-4fb3-9c69-0c334b95b08b.png)


> This effectively just changes the quotation symbol from single to double quotes in a warning. No need for a whatsnew entry.
> 
> Not really convinced that the error is unreadable with single quotes instead of double:
> 
> ![Screen Shot 2022-02-02 at 19 08 20](https://user-images.githubusercontent.com/24256554/152212304-d171f664-c502-4fb3-9c69-0c334b95b08b.png)

can you check this for consistency with other deprecations that include quotes. a) we are probably not consistent b) we could check forth this (maybe)
@attack68 @jreback I checked on whole code, warning is consistent with the quotes inside the columns, also the command is also consistent with black if people want to copy-paste the code.
@staticdev status of this?
@jreback from my side it is ready to be merged.
@attack68 fine, I commited your suggestion.
Thanks @staticdev and @attack68 for merging in main one more time.
Lose the index before 'na_value' assignment in 'to_numpy'
State: closed

- [X] closes #45774 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This Pull Request addresses the issue where casting a multi-indexed Series `to_numpy` fails if an `na_value` is supplied. See the issue #45774 for more information.

The choice to use `asanyarray` over other solutions to lose the index is one based on performance and simplicity. The following simple benchmark was used to make this determination:

```python
import time
import pandas as pd
import numpy as np

t0 = time.time()

series = pd.Series(range(100000000))

#res = series.isna() # Orignal buggy: Elapsed time:  0.19312691688537598 sec
#res = tuple(series.isna()) # Elapsed time:  5.303609132766724 sec
#res = series.isna().reset_index(drop=True) # Elapsed time:  0.2203369140625 sec
#res = series.isna().to_numpy()  # Elapsed time:  0.19327998161315918 sec
#res = np.array(series.isna())  # Elapsed time:  0.25583529472351074 sec
res = np.asanyarray(series.isna())  # Elapsed time:  0.19619202613830566 sec

t1 = time.time()

#print(res)
print("Elapsed time: ", t1 - t0)
```

This demonstrates that there is no significant overhead to using the `to_numpy` operation.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
thanks @DamianBarabonkovQC
Test series replace for different numpy int
State: closed

- [✔ ] closes #45311
- [ ✔] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [✔ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ X ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: replace
description: replace method

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-02-02 07:19:47 UTC
TST: Skip pyarrow hanging in MacOS CI
State: closed

xref https://github.com/pandas-dev/pandas/pull/45507

Saw this test hang a process while working on https://github.com/pandas-dev/pandas/pull/45478

```

2022-02-01T23:22:15.3331120Z pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[c_high] 
2022-02-01T23:22:15.3377730Z [gw0] PASSED pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[c_high] 
2022-02-01T23:22:15.4241640Z pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[c_low] 
2022-02-01T23:22:15.6520590Z [gw0] PASSED pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[c_low] 
2022-02-01T23:22:15.6527280Z pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[python] 
2022-02-01T23:22:15.6529130Z [gw0] PASSED pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[python] 
2022-02-01T23:22:16.0842680Z pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalid_usecols[pyarrow] 
...
2022-02-02T00:20:32.2973430Z ##[error]The operation was canceled.
2022-02-02T00:20:32.3006150Z ##[section]Finishing: Test
```




Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
TST: add test for compatibility with numpy datetime64
State: closed

- [x] Closes #41617 
- [x] test added
- [ ] whatsnew entry 

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @ChrisAlbertsen! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-02-10 06:51:57 UTC
@github-actions pre-commit
Thanks @ChrisAlbertsen
Group by aggregate functions test case
State: closed

- [✔] closes #37275
- [✔] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [✔] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @weikhor! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-02-02 04:50:11 UTC
thanks @weikhor
DOC: minor tidy of 1.4.1 release notes
State: closed


Labels:
name: Docs

Comments:
Doc Build and Upload / Doc Build and Upload (pull_request) green so merging.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 da0156a302f3f7c25bfee0b410bb23865b76f3dd
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45758: DOC: minor tidy of 1.4.1 release notes'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45758-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45758 on branch 1.4.x (DOC: minor tidy of 1.4.1 release notes)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
urgh. should have waited till #45759 is merged to 1.4.x
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 da0156a302f3f7c25bfee0b410bb23865b76f3dd
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45758: DOC: minor tidy of 1.4.1 release notes'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45758-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45758 on branch 1.4.x (DOC: minor tidy of 1.4.1 release notes)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: move pydata-sphinx-theme to the conda packages section
State: closed

xref https://github.com/pandas-dev/pandas/pull/45503#issuecomment-1021163224

Labels:
name: Docs

Comments:
should probably backport for consistency of branches
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
See also https://github.com/pandas-dev/pandas/pull/45641
Backport PR #45581 on branch 1.4.x (CI: Fix actions-310 testing 3.9 instead of 3.10)
State: closed

Backport PR #45581

Labels:
name: CI
description: Continuous Integration

Comments:
@jreback there was failures here that we are not seeing on master. currently investigating locally. could revert this or maybe follow-up with removing the xfails. It maybe only that the only change needed is to ci/deps/actions-310.yaml
the xfails were removed from master https://github.com/pandas-dev/pandas/pull/45651#issuecomment-1022868410

will see if #45651 is suitable for backporting https://github.com/pandas-dev/pandas/pull/45651#issuecomment-1024446836
BUG: Series.mask with small int dtypes raising
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
can you add a whatsnew here
> can you add a whatsnew here

whatsnew added + green
DOC: fix incorrect pie chart documentation
State: closed

in pandas-1.2 we used to get a semicircle. However since pandas-1.3 and later the values get rescaled so that they sum up to 1.
This pull request is updating the documentation to reflect this. It looks like matplotlib still draws a partial circle, so this is a change that was made in pandas.

https://pandas.pydata.org/pandas-docs/version/1.2/user_guide/visualization.html#pie-plot
![image](https://user-images.githubusercontent.com/23830955/151903457-57e39718-b549-4513-bc2c-edc7d5600e8d.png)

https://pandas.pydata.org/docs/dev/user_guide/visualization.html#visualization-pie
![image](https://user-images.githubusercontent.com/23830955/151903662-2bb1c12a-2e09-4da8-b3e8-283c48333de5.png)


Labels:
name: Docs
name: Visualization
description: plotting

Comments:
Could you merge the main branch one more time?
> Could you merge the main branch one more time?

I am not sure how to do it.
Thanks @partev
BUG: update caught exception
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

two things got merged in quick succession

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
cool merge on green
TST: use correct input in test
State: closed

This PR uses the correct input to ensure that the bug #44011 was correctly fixed.

This change was requested here: https://github.com/pandas-dev/pandas/issues/44011#issuecomment-973278817

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Styler
description: conditional formatting using DataFrame.style

Comments:
thanks @rendner
TST: Add @td.skip_if_no_scipy to failing test
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 2cdc03f80e720f720d2a1a2072c839b7276b6b8d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45731: TST: Add @td.skip_if_no_scipy to failing test'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45731-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45731 on branch 1.4.x (TST: Add @td.skip_if_no_scipy to failing test)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 2cdc03f80e720f720d2a1a2072c839b7276b6b8d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45731: TST: Add @td.skip_if_no_scipy to failing test'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45731-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45731 on branch 1.4.x (TST: Add @td.skip_if_no_scipy to failing test)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@mroeschke can you push a manual backport here
> @mroeschke can you push a manual backport here

https://github.com/pandas-dev/pandas/pull/45739/files
0-indexing consistency
State: closed

- [ ] closes #45608 (Replace 45608 with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `https://pandas.pydata.org/pandas-docs/stable/user_guide/advanced.html` file if fixing a bug or adding a new feature.


Labels:

Comments:
Hello @pizza654321! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-31 02:40:05 UTC
closing as stale if you want to update with tests pls ping.
TST: Allow tm.network to catch urllib.error.URLError
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Addresses these flaky network errors due to external servers: https://github.com/pandas-dev/pandas/runs/4992270675?check_suite_focus=true

```
    def http_error_default(self, req, fp, code, msg, hdrs):
>       raise HTTPError(req.full_url, code, msg, hdrs, fp)
E       urllib.error.HTTPError: HTTP Error 500: Internal Server Error
```

Also modifies a condition that just because the test can connect to a "check url" (google) doesn't necessarily mean the test can connect to the target server.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Network
description: Local or Cloud (AWS, GCS, etc.) IO Issues

Comments:
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
BUG: fix __finalize__ xfails
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
pls add a whatsnew for this in a followup
BUG: DataFrameGroupby.transform("size") fails
State: closed

- [x] closes #27469
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

axis=1 produces incorrect results for both aggregations and transforms. Opened #45715 and will be tackling separately.

Labels:
name: Bug
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
is this a regression in 1.4.x? or just not tested previously
@jreback

> is this a regression in 1.4.x? or just not tested previously

Not sure if it's a regression - if it is it's from before July 2019.
thanks @rhshadrach
PERF: groupby.transform broadcast perf
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


`groupby.transform` broadcasts user-defined functions by passing a repeated array to np.concatenate. Passing the array once to np.tile and letting numpy repeat the array is quite a bit faster. 


```
import numpy as np
import pandas as pd

N = 1_000_000

df = pd.DataFrame(
    data=np.random.rand(N, 3),
    index=np.random.randint(0, 5, N)
)

%timeit df.groupby(level=0).transform(lambda x: np.max(x, axis=0))

302 ms ± 8.18 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)     <- main
118 ms ± 2.02 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)   <- PR
```

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
@phofl - there are not many groupby transform asvs that hit this code path. I just added one for this case.
> We like to target ASVs to be around 80ms, is that right? Just want to make sure, looks appropriate as-is.

yeah generally < 100ms is good, but we have a lot that are not so can cleanup separately.
thanks @lukemanley
TST/CI: Fix numexpr test failing in downstream build
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.