BrandonFors/pandas-github-issues huggingface dataset:
BUG: fixes weekday for dates before 1752
State: closed

- [x] closes #53738 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
Approach was to switch to implementing [Gauss's algorithm for weekday](http://berndt-schwerdtfeger.de/wp-content/uploads/pdf/cal.pdf).  Ended up being quite interesting. 
nice work!

just got a couple of requests:

- on the unit test side, could you add a couple of negative dates please?
- could we also add a parametric test using hypothesis please, in which we compare the output against the standard library output? feel free to reach out if you're not sure how to do this
Thanks @MarcoGorelli! 

Haven't used Hypothesis before by tried to follow the patterns elsewhere in the codebase. 
Also by "standard library" I assumed you were referring to `datetime`. Let me know if I missed the mark any what you were looking for. 
yup, that's right! one way to do it could be to make a dict with
```python
{0: 'Mon', 1: 'Tue', ...}
```
and then generate a random datetime `datetime`, and compare `Timestamp(datetime).weekday` with `datetime.strftime('%a')`

something like that
Sorry @MarcoGorelli did you see this push https://github.com/pandas-dev/pandas/pull/53795/commits/ca826d8b30838da53794e8b0c7108844c678d130 ? 

I had already added the Hypothesis driven test before my pervious note, sorry for the confusion.  
CI failure is unrelated, but probably safer to wait til that's addressed before merging
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 20530b1f45045bfb95b0a79cda70fa7b804c9a6a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53795: BUG: fixes weekday for dates before 1752'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53795-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53795 on branch 2.0.x (BUG: fixes weekday for dates before 1752)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BUG: combine_first ignoring others columns if other is empty
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @phofl
REF: remove unused merge args
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Clean

Comments:
Thanks @jbrockmendel
DOC: fix Sphinx parallel build error caused for `doc/source/getting_started/intro_tutorials/04_plotting.rst`
State: closed

- [x] closes #53777
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR appears to correct the specific issue. Need to do a full build again to verify that no other similar issues exist.

Labels:
name: Docs

Comments:
Looks good to me: When I ran into #53777, I came to the same fix as you did. Additionally, `doc/source/user_guide/10min.rst` needs an `:okwarning:` too where on line 750 (before `plt.show()`).

That should fix this type of error for the docs.

What I'm wondering is this: Why is there no failure for the GitHub actions that build the docs? I tried to replicate the environment that GitHub actions use locally, but I still run into the necessity of having `:okwarning:`.

Does someone have an idea?
Thanks @tpaxman
BUG: DataFrame.stack sorts columns
State: closed

- [x] closes #53786 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Looks pretty good, some of the doctests need adjusting. 
Thanks @rhshadrach
DEPR: Remove literal string input for read_html 
State: closed

- [X] closes #53767 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
Just a heads up, it would probably be easier to review if one PR is opened for one `read_*` function deprecation
@mroeschke I was kind of wondering that too...I just went with what was stated in the GH Issue haha 😄 

The deprecation logic for `read_html` should be good to go. I'll re-word the PR and mark it as ready for approval.
@mroeschke PR ready for inspection
Improve frame histogram
State: closed

- [x] closes #53281 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TYP: reshape.merge
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel
CoW: Add warning for chained assignment with fillna
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Warnings
description: Warnings that appear or should be added to pandas
name: Copy / view semantics

Comments:
@jorisvandenbossche I'd appreciate if you can take a look locally, I can't reproduce these failures on Mac
i get similar-looking failures on mac
Arm or intel? Purged environment and still can't reproduce...
intel
Hm shit. Thx for checking
This is Python version specific unfortunately... 3.10 has a different ref count compared to 3.11
BUG: handle none values in `json.normalize()`
State: closed

This PR fixes the error which is thrown when a field value in `json` is `null` while `json_normalize` it to a `DataFrame`. Implemented tests to validate the change

- [x] closes #53719 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
PERF: concat in no-reindexing case
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @jbrockmendel
PERF: avoid unnecessary casting in merge
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thx @jbrockmendel
DOC: operator wrapper names do not match math operators
State: closed

- [x] closes #53724 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
@MarcoGorelli done.
BUG: bad display for complex series with nan
State: closed

- [ ] Closes #53762
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [ ] *Not sure if this needs a changelog entry, because previously series/dataframes with complex nans will raise.*

Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Complex
description: Complex Numbers

Comments:
Nice thanks @Charlie-XIAO
BUG: to_json not serializing non-nanosecond numpy dt64 correctly
State: closed

- [ ] closes #53686 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
@lithomas1 is there a viable way to do this using values_for_json, i.e. keep this complexity out of the C code?
> @lithomas1 is there a viable way to do this using values_for_json, i.e. keep this complexity out of the C code?

Long term, we could move to using ``to_native_types`` like CSV does, and just directly feed strings to the JSON serializer (for datetime/EA types).

I don't think this PR adds too much complexity (the core changes are just ~3 lines), so maybe we can merge this for now, and then try to pull the logic for anything other than native types (e.g. ints, floats) out of the JSON C code?
The challenge with values_for_json is also that to_json has keyword arguments that control the JSON output, e.g. date_format and date_unit. Those are both handled in the serializer currently; values_for_json would have to handle those arguments generically across EAs
> values_for_json would have to handle those arguments generically across EAs

that sounds like a better long-term solution than doing more here (assuming perf is OK).  I'm happy to defer to you two on this PR in the interim.
gonna put this in then.

Thanks for the reviews all.
POC: DataFrame.stack not including NA rows
State: closed

Demonstration for #53515

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
@jorisvandenbossche - there are some cases where DataFrame.stack sorts the result; I believe these are bugs. Other than this, the only tricky situation I've encountered is handling duplicates in the column that is being stacked. In this POC, I raise a ValueError instead. See #53761 
Closing in favor of #53921. @jorisvandenbossche - happy to continue any discussion here, or move over to #53921.
TYP: reshape.merge
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel 
Hah! Guess I was too slow..
BUG / CoW: Series.transform not respecting CoW
State: closed

- [x] xref #49473 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Could think about back porting, but I wouldn't do it at this point

Labels:
name: Copy / view semantics

Comments:
No objection here - mainly because the fix is so simple. But in general we don't support UDFs that mutate the object:

https://pandas.pydata.org/pandas-docs/dev/user_guide/gotchas.html#mutating-with-user-defined-function-udf-methods

If we want to revisit this then I'm certainly open, but otherwise I don't think we should go out of our way to support this anti-pattern.
ENH / CoW: Add lazy copy to eval
State: closed

- [x] xref #49473 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
The typing error is related? (don't see it on main, and it happens in the eval.py file, although I don't see how it actually can be related ..)
The ignore didn't seem to have worked ..
TST / CoW: Add test for mask
State: closed

- [x] xref #49473 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
FYI there are actual related test failures in the Copy-on-Write build
Thx, looks like I had a copy-paste error
Ready to merge now
BUG: df constructor not copying ea backed series
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Follow-up on https://github.com/pandas-dev/pandas/pull/52947#discussion_r1187276744


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Copy / view semantics

Comments:
Thanks!