BrandonFors/pandas-github-issues huggingface dataset:
BUG: read_csv raising for arrow engine and parse_dates
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Not sure if this is an actual regression, but ties a bit into the dtype backend and would be nice if this works, since this raises for every parse_date case with numpy dtype backend (and I want to advertise the engine in my pyarrow blog...)

Labels:
name: IO CSV
description: read_csv, to_csv
name: Arrow
description: pyarrow functionality

Comments:
Can you share a MRE of the issue you're having?

I'm wondering if #50056 fixes/causes the issue.
```
data = """a,b
2000-01-01 00:00:00,1
2000-01-01 00:00:01,1"""

result = pd.read_csv(StringIO(data), parse_dates=["a"], engine="pyarrow")
```

Nope, still raises after your PR is in

Edit: The main issue is that arrow infers as datetime and we try to infer again, which raises.
Thanks @phofl 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 aaf503784a7cbf6425d681551d1e1e686ce14815
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53295: BUG: read_csv raising for arrow engine and parse_dates'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53295-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53295 on branch 2.0.x (BUG: read_csv raising for arrow engine and parse_dates)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Will backport later today or tomorrow 
Might want to consider backporting #50056 and #52087 too.
The first one is a bit less common, can think about the second one if you like
That'd be great, I think there were a couple issues opened about nans in string columns not being read properly related to the second.
BUG `date_range` with end, periods, and freq failing
State: closed

- [x] Closes #53293
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst`

I think this does not need a changelog entry (though I have added one for now), since this bug seems to be introduced just two weeks ago and does not exist in any prior versions of pandas. Please let me know if I should remove the changelog entry.


Labels:

Comments:
Thanks but given that this wasn't originally failing in the CI for our Windows builds I don't think we should make this change to avoid unintended side effects. Closing
@mroeschke I'm not sure why CI is not failing. I have opened #53604 with a reproducible example and my installed versions. If anyone can reproduce with that example, I think we should reopen this.
BUG: allow DataFrame[mask]=value with mixed non-numeric dtypes
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Looks like this check was implemented to avoid some inconsistency, but I can no longer find what that inconsistency was.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Could also just call this a bugfix since we're going from raising to not raising
Yeah I would be okay calling this a bug fix
Changed from deprecation to bugfix
Thanks @jbrockmendel
FIX `RangeIndex` rsub by constant
State: closed

- [ ] Closes #53255
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests)
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Index
description: Related to the Index class or subclasses

Comments:
Nice. Thanks @Charlie-XIAO.
BUG: Missing date formatting parameters
State: closed

Missing parameters from __init__() in super().init() causes excelwriter using openpyxl as engine to ignore date formatting.

Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel
name: Stale

Comments:
Thanks for the PR. Is there an open issue regarding this? Also, can you add tests.
The relevant issue is #44284 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the PR, but it looks like this is stale. @arthurpeixotomelo - if you like to continue just comment here and we can reopen!
BUG: Change default of bool_only to False
State: closed

- [x] closes #53258 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: API - Consistency
description: Internal Consistency of API/Behavior
name: Reduction Operations
description: sum, mean, min, max, etc.
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
@phofl users shouldn't experience any difference with `bool_only=None` vs `bool_only=False`, so I think this is okay to change for 2.1
Agreed
Thanks @rhshadrach
ENH: Add sort keyword to stack
State: closed

- [x] xref #15105 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
if the default were False (mentioned in the #15105), would it be straightforward to reproduce the current behavior with something like `df.stack(...).some_other_method()`?  If so, longer-term ideally we'd have users do that and not have a keyword
Yeah that's a likely future behavior where we default to False and the deprecate this keyword, but we need to set it to true for backward compat
Any other comments here @jbrockmendel?
DOC: add missing parameters to offsets classes: BYearBegin, YearEnd, YearBegin
State: closed

xref  #52431
Updated documentation for offsets classes. Added missing parameters, See Also section, and examples to `BYearBegin`, `YearEnd`, and `YearBegin`. Corrected the default value for `month` in the class `BYearEnd`.


Labels:
name: Docs
name: Frequency
description: DateOffsets

Comments:
@MarcoGorelli, could you please take a look at this pr?
BUG: Resampler.ohlc with empty data
State: closed

- [x] closes #42902 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Resample
description: resample method

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Just a whatsnew conflict.
Thanks @jbrockmendel
DOC: Add to_sql example of conflict with `method` parameter
State: closed

- [ ] closes #15988 (Replace xxxx with the GitHub issue number)
- [ ] closes #14553 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Appears there's no sqlalchemy agnostic way to implement this, but this appears possible with the current `method` parameter so added a docstring demonstrating it.

Labels:
name: Docs
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Looks like the tests are passing here. Happy to follow up if there are any comments
Hello, here I want to add the value of the original field to the incoming value to get the latest value. How to do this?
BUG: Add AM/PM token support on guess_datetime_format
State: closed

- [ ] closes #53147
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
@MarcoGorelli I tried it from the top, it was a dummy bug, but I almost couldn't figure it out until I setup my local env.
Thanks @juli4nb4dillo 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 f2de598d9002801e2056b4f8e2f587fd8bb07bc1
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53257: BUG: Add AM/PM token support on guess_datetime_format'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53257-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53257 on branch 2.0.x (BUG: Add AM/PM token support on guess_datetime_format)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
CLN: Fix typo in parameter name, standardize a bit
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Clean

Comments:
thanks @deepyaman
BUG: Use operator.xor() for dataframe boolean dtypes
State: closed

- [ ] closes #52579 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
@jbrockmendel Could you review my modifications?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
CI: Remove 32 bit testing
State: closed

- [ ] closes #28842 (Replace xxxx with the GitHub issue number)
- [ ] closes #44453 (Replace xxxx with the GitHub issue number)
- [ ] closes #46360 (Replace xxxx with the GitHub issue number)

With the meson build PR, we dropped windows 32 bit wheels. Additionally, other pydata libraries are also starting to remove 32 bit wheel support https://discuss.python.org/t/dropping-32-bit-packages/5476/9. Since we don't ship these wheels anymore, I am not sure if we should continue to test this specifically. 



Labels:
name: CI
description: Continuous Integration
name: 32bit
description: 32-bit systems

Comments:
This was also previously discussed in e.g. https://github.com/pandas-dev/pandas/issues/15889 and the consensus seemed to be that it's worth supporting 32-bit builds in source, even though wheels would not be published. Especially:
> Moreover, my very general and superficial analysis of issues like https://github.com/pandas-dev/pandas/issues/24613 (or - now speaking out of experience - https://github.com/pandas-dev/pandas/issues/19439 ) is that they actually help us fix mistakes in the code, regardless of the use of 32 bits.

Seems like the challenge of *windows* 32-bit *release wheels* shouldn't reflect on the challenge of running CI on linux.
Ah okay. I suppose we can keep this around and support those who install from source on 32 bit platforms. Closing
In https://github.com/pandas-dev/pandas/issues/44453, I also [say](https://github.com/pandas-dev/pandas/issues/44453#issuecomment-968468140):
>Apologies if I'm misunderstanding here, but it looks like Numpy isn't dropping 32-bit support https://github.com/numpy/numpy/issues/20277#issuecomment-957529088. It looks like they're only dropping wheel generation because of infra constraints but will continue to test.
>
>I'm interested because in Gentoo, we still support a number of 32-bit architectures (arm, x86 being the two main ones), and Pandas ends up in the depgraph often unavoidably if we want to run test suites on programs.
>
>Is the plan for Pandas to drop support entirely for 32-bit or to just stop producing wheels?

Numpy continues to have this position, per https://discuss.python.org/t/dropping-32-bit-packages/5476/16.
> Ah okay. I suppose we can keep this around and support those who install from source on 32 bit platforms. Closing

Thank you! It is really appreciated, I promise.
BUG: Add AM/PM token support on guess_datetime_format
State: closed

- [ ] closes #53147
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Is this PR or https://github.com/pandas-dev/pandas/pull/53257 supposed to fix this? 
@mroeschke It's duplicate, sorry for that, I wanted to test it from the top so I made another branch.
This one is wrong anyway.
TST: Adds test coverage for pd.read_json index type
State: closed

- [x] closes #28558 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

## Description: 
@mroeschke mentioned that the merged fix for issue [#28558](https://github.com/pandas-dev/pandas/issues/28558) could use a [test](https://github.com/pandas-dev/pandas/issues/28558#issuecomment-1528187186). This PR adds the missing test for index type equality between the `DataFrame` and `read_json` functions.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Updated with your suggestions @mroeschke 
Updated with your latest suggestion @mroeschke. Let me know if there is anything else. 
Thanks @mcgeestocks
DOC: Fixing EX01 - added examples
State: closed


- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

If I'm not mistaken, the parameter `downcast` for the `bfill` method should be `None`, `False`, or `infer` (per #53103 after pandas v2.1.0). So I did not include an example`dict` for it. 

`DataFrame.pipe` had examples already. 
`DataFrame.iterrows`also had examples, but needed the word "Examples" to pass the test. 


Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon 
Thank you @mroeschke
BUG `DataFrameGroupBy.agg` with list not respecting `as_index=False`
State: closed

- [x] Closes #52849(
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints)
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file

Dealing with non-categorical data is fairly easy, but dealing with categorical data is kind of tricky. I'm currently taking a pretty dirty approach, so if maintainers have better solutions please let me know and I'll make changes correspondingly.

Labels:
name: Bug
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
@rhshadrach Thanks for your review! New test added and I've made your suggested modification.
Thanks @Charlie-XIAO
BUG: preserve dtype for right/outer merge of datetime with different resolutions
State: closed

Follow-up on https://github.com/pandas-dev/pandas/pull/53213, fixing one more corner case for the right/outer joins making the key column object dtype.

Expanded the test added in that PR to test the different join types. 
The whatsnew added in https://github.com/pandas-dev/pandas/pull/53213 should also already cover this.

Labels:
name: Bug
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jorisvandenbossche