BrandonFors/pandas-github-issues huggingface dataset:
WEB: Update feed to Tom's posts
State: closed

The link to Tom's blog is broken, and it used to fetch everything (not only pandas related things).

Updated the link to the rss new location for the pandas tag.

CC: @TomAugspurger 

Labels:
name: Web
description: pandas website

Comments:
@TomAugspurger looks like your blog has invalid dates for some posts. Can you have a look at your side please? I don't think we want to add any hack for this in the pandas codebase.
Maybe fixed on my side.
Cool, rerunning the CI, let's see if it works now.
Seems like at least one of the posts still have a published date with year set to zero: https://github.com/pandas-dev/pandas/actions/runs/4248115161/jobs/7386912564#step:5:21
I think I have this fixed locally. https://github.com/pandas-dev/pandas/actions/runs/4248115161 is the re-triggered job.
/preview
No preview found for PR #51438. Did the docs build complete?
Excellent. Seems like the web could be generated without problems this time. :) I just merged master, so we can easily preview the blog and see if your posts are there (this PR was opened before the previewing system, and the preview wasn't triggered at the end of the docs build).
Oh, nice, you've got a recent article, it was easy to check if this was working: https://pandas.pydata.org/preview/51438/community/blog/

Feel free to merge this when you want. I'm going to read it. ;)
Ups, seems like it's an article from 2014 that appears in the rss as being from this week. Not sure if it's still relevant the article, but I guess better to fix the date so it doesn't appear as recent in the pandas blog.
@TomAugspurger could you have a look at the problem above? If it's not easy to fix, I may remove the feed to your blog for now, so it's not broken (I'd like to fail the CI is a blog feed is broken in a separate PR, so we realize of problems earlier). Then you can add it back whenever the dates in your blog are fixed.

Happy to wait few more days, but if it's going to take much longer, maybe let's do this?
Closing. Feel free to reopen or open a new PR once the blog is fixed.
BUG : Add Deprecation FutureWarning for parse function call in read_excel
State: closed

- [ ] closes #50953 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel
name: Deprecate
description: Functionality to remove in pandas
name: Stale

Comments:
@rhshadrach @simonjayhawkins  - Closed this by mistake was merging the latest main to my branch. Will resolve mentioned comments and will create a new PR.
Why not just reopen?
> Why not just reopen?

Okay sorry wasn't aware about it. Reopening now.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
I'd like to make sure there isn't a better way of deprecating specifying arguments by position as recommended in https://github.com/pandas-dev/pandas/pull/51437#pullrequestreview-1302390009. cc @mroeschke @phofl 
> I'd like to make sure there isn't a better way of deprecating specifying arguments by position as recommended in [#51437 (review)](https://github.com/pandas-dev/pandas/pull/51437#pullrequestreview-1302390009). cc @mroeschke @phofl

I think you can use in `deprecate_nonkeyword_arguments` in pandas/util/_decorators.py
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
STYLE ruff: enable PIE
State: closed

- [x] closes #51407
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
@MarcoGorelli 
There is a [conflict](https://github.com/pandas-dev/pandas/pull/51434/conflicts) here. How should it be resolved?
> @MarcoGorelli There is a [conflict](https://github.com/pandas-dev/pandas/pull/51434/conflicts) here. How should it be resolved?

hey - I'd suggest looking at https://opensource.com/article/20/4/git-merge-conflict for how to resolve conflicts
looks like you managed, nice!

just need to revert changes to doc/source/user_guide/style.ipynb, if you do something like
```
git fetch upstream
git checkout upstream/main -- doc/source/user_guide/style.ipynb
git commit -a -m 'revert changes to doc/source/user_guide/style.ipynb'
git push origin HEAD
```
it should work
@MarcoGorelli 
I think I am done with the requested changes barring any changes.
> Nice work, looks good to me pending green

@MarcoGorelli 
Does that mean I could close the pull request? The reason for my asking is that I am also seeing a lot of errors popping up in the checks that are being carried out. 
By the way, this would be my first open-source contribution, when completed. Thanks for the opportunity.
> @MarcoGorelli Does that mean I could close the pull request? The reason for my asking is that I am also seeing a lot of errors popping up in the checks that are being carried out. 

when it's ready, we will merge it - but first, the errors need fixing up
@MarcoGorelli  Sorry, I have been away for a while. I have re-run and I still see the errors. Looking through to see where I might have gone wrong.

> @MarcoGorelli Sorry, I have been away for a while. I have re-run and I still see the errors. Looking through to see where I might have gone wrong.

no worries :) I've suggested a couple of fixes, the rest looks good!
@MarcoGorelli All tests passed. I guess merging can now be done.
yup, should be good, I'll just wait for a few more jobs to finish
REF: Refactor reference handling to the BlockManager level for iterrows
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

let's see if this passes everything

Labels:
name: Copy / view semantics

Comments:
Merging, this was only a removed comment
BUG: transpose not respecting CoW
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is the same solution as in iterrows, I work on a solution on the BlockManager level now

Labels:
name: Copy / view semantics

Comments:
Updated for new block manager method
@phofl there are some relevant failures here with the usage of `add_reference` in the CoW build and a failure in the ArrayManager build
Is it possible to get views when splitting the block and converting the dtype? We got the errors when converting object to a proper dtype which splitted the block
That sounds impossible, unless one of the split blocks kept its object dtype?
Ok then my train of though was correct. No there is no reason to split and keep object 
Merging, I'll tackle the transpose case separately
"Backport PR #51393 on branch 1.5.x (CI: Pin matplotlib to < 3.7.0)"
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
thx
TST: add CoW test for update()
State: closed

xref https://github.com/pandas-dev/pandas/issues/49473

Under the hood, DataFrame.update() is using `.loc` setitem, and Series.update() uses `_mgr.putmask(..)`, and both already handle CoW. This is therefore just adding an explicit test for the `update()` method.

Labels:
name: Copy / view semantics

Comments:
thx @jorisvandenbossche
BUG: Fix GroupBy.idxmin/idxmax retrun wrong type on empty
State: closed

- [x] closes #50993 
- [x] some tests already exist
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby

Comments:
Thanks @luke396
BUG: Series.corr/cov raising with masked dtype
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
thx @lukemanley
PERF: DataFrame.corr avoid copy
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
@jbrockmendel looks like a merge went wrong?
Yes it does.  locally im looking at moving the fix to a lower level, not ready to push yet.
fix moved to a lower-level, merge problem fixed, green
Thanks @jbrockmendel
ENH: Add dict support for read_csv date_format
State: closed

- [x] closes #51240 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
Thanks @phofl 
thx @MarcoGorelli for helping out
So it appears the tests added here can raise a UserWarning 

```
============================================================================= warnings summary =============================================================================
pandas/tests/io/parser/test_parse_dates.py::test_parse_dates_dict_format_two_columns[c_high-foo-parse_dates1]
pandas/tests/io/parser/test_parse_dates.py::test_parse_dates_dict_format_two_columns[c_low-foo-parse_dates1]
pandas/tests/io/parser/test_parse_dates.py::test_parse_dates_dict_format_two_columns[python-foo-parse_dates1]
  /.../opt/miniconda3/envs/pandas-dev/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.parsing._maybe_warn_about_dayfirst'

  Traceback (most recent call last):
    File ".../pandas/core/tools/datetimes.py", line 137, in _guess_datetime_format_for_array
      guessed_format = guess_datetime_format(
  UserWarning: Parsing dates in %d- %m-%Y format when dayfirst=False (the default) was specified. Pass `dayfirst=True` or specify a format to silence this warning.

    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
```

I would expect no warning since an explicit format is passed?
That's a good question - so, `parse_dates` is `{'foo': [0, 1]}`, and `date_format` specifies column `'foo'`, but in 

https://github.com/pandas-dev/pandas/blob/7598f71993f442c6420456e1e2c3c8d3bc65c80b/pandas/io/parsers/base_parser.py#L1117-L1119

, `col` is `'a_b'`, and so it doesn't find the date format in the dict and so `date_fmt` is `None`, so the format has to be guessed. It's reassuring that the correct format was guessed 😄 

Thanks for spotting this

@phofl should the user have passed `date_format={'a_b': '%d- %m-%Y'}`, or does this need fixing in the code?
code I'd say, good catch btw.


#51479
CI/STYLE: run black before ruff in pre-commit
State: closed

Something I have been noticing recently: when commit / running pre-commit, I would often get some warnings from ruff, but for things that black will fix afterwards anyway. While there is no harm done here, it can still be confusing, thinking you also need to fix those warnings. So it might be better to first run black and then ruff. 

This commit does that, moving black first. Because it is a "local" hook, we now have two "local" sections, but that _seems_ to work fine. 


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
The right order is actually ruff- black - ruff (optional) (see https://github.com/charliermarsh/ruff/issues/3143) 

Although it seems we're just days/weeks away from `ruff` being able to blacken the code for you also (https://github.com/charliermarsh/ruff/issues/1904)
I don't think we already let ruff autofix things, in which case that shouldn't be needed to run black again afterward?

But yes, good to know (we probably also want to start using autofix at some point)
the issue with autofixing is that due to pandas semantics it doesn't work particularly well with E711

https://github.com/pandas-dev/pandas/blob/d8a1a0d329205d825582a56eaad87c8702f99c87/pandas/tests/groupby/test_groupby_dropna.py#L653-L654

Maybe we can just turn E711 off and let it autofix though
> The right order is actually black - ruff - black (see [astral-sh/ruff#3143](https://github.com/astral-sh/ruff/issues/3143))

Just wanted to correct this real quick for anybody else who ends up here through a search engine: the right order is actually **Ruff -> Black -> Ruff (optional)**.

As stated in the linked issue:

              (We test in CI that running Ruff, then Black, then Ruff again is a stable operation.)

_Originally posted by @charliermarsh in https://github.com/astral-sh/ruff/issues/3143#issuecomment-1440853844_
            
And also as exemplified in their pre-commit config: https://github.com/astral-sh/ruff/blob/3ccd1d580d2d680ad6d4bb7faec16bca39af53f7/.pre-commit-config.yaml#L45-L61
Thanks for correcting that @cosmojg ! I've also updated my original comment (for the same reason -- in case someone ends up here via a search engine)
As I mentioned before, as far as I understand, running ruff before black is only important if you let ruff autofix things? (or are there other reasons to run it before black?)

And we don't do that (yet) in pandas. In that case (not needing the initial ruff to autofix), the "ruff - black - ruff" sequence becomes "black - ruff", which is what we do.
@jorisvandenbossche Correct, running Ruff before Black is only important if you let Ruff autofix things.

To clarify, I was only trying to correct @timvink's comment as this thread appears second on the results page when searching for "pre-commit black before ruff or ruff before black" on Kagi. Sorry about the notification! And thank you for all your hard work on Pandas!
@jorisvandenbossche No problem, thanks for the clarification!
BUG: GroupBy.ohlc with as_index=False
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby

Comments:
Thanks @jbrockmendel
BUG: DataFrame.to_html(na_rep="-") with non-scalar data
State: closed

- [x] closes #47103
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
Thanks @topper-123
ENH: Optimize CoW for fillna with ea dtypes
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Merging, happy to address potential comments in a follow up
BUG: Arithmetic inplace ops not respecting CoW
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We should add tests for all op-possibilities in a follow up

Labels:
name: Bug
name: Copy / view semantics

Comments:
greenish
TST: clip inplace doesn't modify views with CoW
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Copy / view semantics

Comments:
thx @lithomas1
ENH: Add support for Index.min/max with arrow string
State: closed

- [x] closes #51397 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Index
description: Related to the Index class or subclasses
name: Arrow
description: pyarrow functionality

Comments:
Thanks @phofl