BrandonFors/pandas-github-issues huggingface dataset:
WEB: Fix redirect of blog posts
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

@datapythonista 

I think https://github.com/pandas-dev/pandas/pull/50897 broke the blog-redirects. This works locally.

Labels:
name: Web
description: pandas website

Comments:
Ups, not sure how I missed this.

It's a bit more complex. We've got two different kinds of posts, the ones in our website, and the ones in other blogs. The ones in our own blog are currently working, the broken ones are the ones in other blogs (like your blog or mine). Your changes makes the external posts work, but it breaks the ones we write in our own blog/website. I opened #51649 with the fix that works for both (I thought it was faster/easier to open a new PR than comment here, I hope you don't mind).
Ah makes sense. Did not think about our own blog. Thx for noticing.

Yeah no worries, closing here then
PERF: ArrowExtensionArray.fillna when array does not contains any nulls
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

```
import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randn(1_000_000, 10), dtype="float64[pyarrow]")

%timeit df.fillna(0.0)
```

```
25 ms ± 5.42 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)        -> main
76.4 µs ± 4.43 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)  -> PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
Could you test this on #51411 as well? This should address the problem for CoW
> Could you test this on #51411 as well? This should address the problem for CoW

I tested locally on top of your branch in #51411 and it seems to work. What "problem" are you referring to?
Performance, we are avoiding the copy on the block level there 
@phofl - hmmm... clearly this doesn't play well with https://github.com/pandas-dev/pandas/pull/51411 

Not sure what I did yesterday to convince myself it did... sorry
actually, what is the expected behavior for COW wrt pyarrow? Since arrow arrays are immutable, `ArrowExtensionArray.copy` doesn't actually copy the underlying pyarrow array. I think tests like this (which are failing) are expected to fail:

```
import pandas as pd
import numpy as np

arr = pd.array([1, 2, 3], dtype="int64[pyarrow]")
arr2 = arr.copy()

assert not np.shares_memory(arr, arr2)  # -> raises
```
CoW handles the copies globally on the block level, hence we can remove the check above.

You can adjust the tests, eas made to many defensive copies without CoW, you got rid of one of them, which is a good thing
Ok will do. Is the comment you added still relevant? (`# TODO(CoW): Not necessary anymore when CoW is the default`)

I would think this fastpath can still apply after CoW is the default (e.g. for whatever reason CoW is not being used)
We are not planning on having a non CoW mode when CoW is the default 
Merge when ready @phofl 
Could you rebase @lukemanley 
Thanks @lukemanley
BUG: Series.any and Series.all for empty or all-null pyarrow-backed dtypes
State: closed

- [x] closes #51624
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley 
back porting as well, because we need it for the other pr
@meeseeksdev backport 2.0.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 4700a6101e6338fec437ed1222b9f827b4f66cd5
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #51631: BUG: Series.any and Series.all for empty or all-null pyarrow-backed dtypes'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-51631-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #51631 on branch 2.0.x (BUG: Series.any and Series.all for empty or all-null pyarrow-backed dtypes)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
PERF: ArrowExtensionArray.isna when zero nulls or all nulls
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

Perf improvement in `ArrowExtensionArray.isna` when array has zero nulls or is all nulls. 

ASV added for `DataFrame.isna`:

```
       before           after         ratio
     [f9438649]       [3cb3cf47]
     <main>           <perf-arrow-isna>
-      32.1±0.5ms      7.95±0.08ms     0.25  frame_methods.Isna.time_isna('float64[pyarrow]')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
thx @lukemanley
BUG: Change default of group_keys to True
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think this was missed because `no_default` is truthy.

Labels:
name: Bug
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
thx @rhshadrach
DOC: Improvements to groupby.rst
State: closed

- [x] closes #46415 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Doing a general cleanup of the groupby user guide. The main new thing being introduce here is language to steer users way from groupby.apply.

Labels:
name: Docs
name: Groupby
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
thx @rhshadrach
REF: remove iterate_slices
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Made possible by #51398 cc @rhshadrach 


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @jbrockmendel
STYLE: replace unwanted-patterns-strings-to-concatenate with ruff #51406 #51591
State: closed

![issue](https://user-images.githubusercontent.com/73106537/221244703-837ad7da-02a7-4c38-85e4-bc8e48518f51.JPG)


Labels:

Comments:
can u help me with this error please
![Capture](https://user-images.githubusercontent.com/73106537/221245720-0749b050-f586-48c9-a6c5-6c8ae2d65f2a.JPG)

sorry, there's already an open PR for this, from someone who had already been assigned

if you're interested in working on an issue, please comment `take` so it's assigned to you (as per the contributing guide)
take

sorry i wasn't able to solve the issue in time. can u guide me to do next
DOC: fix EX02 errors in docstrings
State: closed

Some work towards issue https://github.com/pandas-dev/pandas/issues/51236

Makes docstrings pass EX02 error for:
```
pandas.plotting.radviz
pandas.tseries.frequencies.to_offset
```

Labels:
name: Docs

Comments:
doctests and doctrings had a conflict on `pandas.plotting.radviz` example return.
doctests got `<AxesSubplot: xlabel='y(t)', ylabel='y(t + 1)'>`
doctrings got `<AxesSubplot: >`
you're right - I'll take a look at why that might be
> for the plot one, I think we should just remove the output and skip them, like is done here
> 
> https://github.com/pandas-dev/pandas/blob/3f2b18aed1468d3643bf638f7859cacf3141c8f4/pandas/plotting/_core.py#L333-L339
> 
> It's not really a meaningful doctest anyway, `<AxesSubplot: >` could include anything 😄

Thank you. Will do.

Just out of curiosity, do you happen to know that why doctests got <AxesSubplot: xlabel='y(t)', ylabel='y(t + 1)'>? I ran the plot on python console and got <AxesSubplot: > too. 
I think it's because one of the doctests (hist_plot if I remember right) from _core.py doesn't have the doctest+skip, and so was setting some option
BUG: Weighted rolling aggregations min_periods=0
State: closed

- [x] closes #51449
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Bug
name: Window
description: rolling, ewma, expanding

Comments:
> Looks pretty good. Could you add a whatsnew note in `v2.1.0.rst`?

Sure thing. Done.
Looks like there's a merge conflict. Can you merge main once more?
Thanks @Savlik
CI: Debug segfaults on npdev
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
Got the backtrace, going to post on #51589.
Have we tried asking @seberg for help?
Didn't ask yet, was going to try to get this down to a MRE, but got sidetracked.
Hmmmmpf, some other PRs still seem to fail in much the same way?  Seems my fix may have not been complete.
> Hmmmmpf, some other PRs still seem to fail in much the same way? Seems my fix may have not been complete.

We cache our dependencies, I'll just cleared the cache, and now we're hitting compile failures because of the latest Cython 3 beta on our numpydev build.

I'll let you know if the fix works.
@seberg Looks like the segfaults have disappeared. We do have some other failures, but those seem Cython related.

Thanks for the quick fix.
REGR: MultiIndex.isin with empty iterable raising
State: closed

- [ ] closes #51599 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: MultiIndex

Comments:
Thanks @phofl
BUG: stack doesn't preserve lower precision floating dtypes
State: closed

- [ ] closes #51059 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
This looked okay. Could you merge in main once more and add a whatsnew?
Going to mothball for now but feel free to reopen
DOC: fix EX02 docstring errors for three plotting docstrings
State: closed

Some work towards issue https://github.com/pandas-dev/pandas/issues/51236
The issue is being worked on by several people and has more docstrings remaining to do.

Makes docstrings pass for:
`pandas.plotting.andrews_curves`
`pandas.plotting.autocorrelation_plot`
`pandas.plotting.lag_plot`

Labels:
name: Docs
name: Visualization
description: plotting

Comments:
@MarcoGorelli This is currently a draft because I'm unsure about something.

I got these docstrings to pass by changing instances of `AxesSubplot` in the `Expected` outputs to what the `Got` outputs said in a few places, I don't know if this is correct though. The other options would be that there are mistakes in the docstrings in the ways the methods are called or in the parameters passed in or something, or that there are mistakes in the methods themselves. 

I don't know what I would be searching for in the alternative possibilities that I mentioned and maybe what I did was correct anyway and nothing else is necessary, so I thought I would make a draft and ask.
Hey - looks like these are failing in CI when run as doctests
```
https://github.com/pandas-dev/pandas/actions/runs/4256504265/jobs/7405473685
```
But then they do pass when run as part of the docstring validation job...I don't know why that might be, I'll take a look
@MarcoGorelli Oh, maybe it's because `infer_dtype`  is not passing `./ci/code_checks.sh docstrings`. I think it was accidentally broken in another pull request on this issue, or maybe broken for some other reason, or maybe it was broken by this pull request. I was pretty sure before that that mistake wasn't from this pull request, but I'll check soon here. I figured that we would clean it up later but didn't think about that it would be necessary for the validation job to clear.

Or maybe that's not it? In the failures, it's giving the same errors that were there before my changes, which were the reasons that these ended up on the `--ignore_functions` list to begin with. It's also listing `pandas.plotting.parallel_coordinates` and `pandas.plotting.radviz` as failures, even though those haven't been worked on yet and are still on the `--ignore_functions` list.
@MarcoGorelli uh oh, I'm not very good at Git yet and was having a lot of trouble fetching from the main upstream repository and then pushing my changes without conflicts, and I deleted the commits hoping to start fresh not realizing that I would close the issue by doing that. Obviously I would have preferred to have kept the pull request here, but it closed. I'm working on opening a new pull request now, sorry for making a mess of this.
DOC Fix EX01 issues in docstrings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards https://github.com/pandas-dev/pandas/issues/37875
Attaching resulting pages:
[pandas.DataFrame.mean - doc.pdf](https://github.com/pandas-dev/pandas/files/10816682/pandas.DataFrame.mean.-.doc.pdf)
[pandas.DataFrame.median - doc.pdf](https://github.com/pandas-dev/pandas/files/10816683/pandas.DataFrame.median.-.doc.pdf)


Labels:
name: Docs

Comments:
Nice as always @DeaMariaLeon 
Thank you @mroeschke!
PERF: maybe_promote
State: closed

Test runtime for pandas/tests/indexing/multiindex/test_indexing_slow.py decreases from 30s to 14.5s.

See discussion around https://github.com/pandas-dev/pandas/pull/39692#issuecomment-1441051740

Labels:
name: Performance
description: Memory or execution speed performance
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
our asv site appears to be done, but locally a lot of the regressions im seeing since 1.5.0 are for `groupby.GroupByMethods.time_dtype_as_field('datetime', ...)` and profiling points at maybe_promote so im guessing this will address a bunch of those

cc @phofl 
thx @jbrockmendel
STYLE: replace unwanted-patterns-strings-to-concatenate with ruff #51406
State: closed


Labels:
name: Docs

Comments:
hey, i need help regarding this please.
![git test](https://user-images.githubusercontent.com/73106537/220987225-7e1e0dcb-ce99-47df-b966-93fbbe35d918.JPG)

See here https://beta.ruff.rs/docs/rules/explicit-string-concatenation/
thanks for your PR

closing in favour of https://github.com/pandas-dev/pandas/pull/51613 (who, in fairness, had already said that they were working on this and to whom the issue had been assigned)
sir i have mostly fixed the errors. one last can u help me with that. please
![issue](https://user-images.githubusercontent.com/73106537/221244225-08254ec4-fb59-4228-b59e-a14fd0dca7a8.JPG)
CI: Try numpydev skip test
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
Here's what I got from debugging on my repo https://github.com/mroeschke/pandas/pull/16

```
2023-02-23T18:26:51.3325944Z [gw0] node down: Not properly terminated
2023-02-23T18:26:51.3816201Z [gw0] FAILED pandas/tests/test_sorting.py::TestSorting::test_nargsort[False-last-exp6-<lambda>] 
2023-02-23T18:26:51.3816794Z 
2023-02-23T18:26:51.3817054Z replacing crashed worker gw0
```

```
2023-02-23T18:33:56.3583342Z [gw2] node down: Not properly terminated
2023-02-23T18:33:56.4068472Z [gw2] FAILED pandas/tests/extension/test_interval.py::TestMethods::test_sort_values[<lambda>-False] 
2023-02-23T18:33:56.4068954Z 
2023-02-23T18:33:56.4069119Z replacing crashed worker gw2
```
There are different tests failing, all sorting related as it seems
could be related to the avx512 sorting thing numpy implemented last week?
Yeah, this is not going to be easy to debug. I think pytest exists when all workers crash, so you'd only be able to flush out segfaulting tests 2 at a time.

I will try to get a backtrace with catchsegv, but in the meantime it's probably better to pin numpy to a specific nightly version.