BrandonFors/pandas-github-issues huggingface dataset:
BLD: Bump our C standard to 11
State: closed

We're getting Cython compilation errors with Cython < 3 and C99, on the wheel builders.

NOTE: Cython 3 seems to handle this OK.

Let's see if C11 fixes it since we can't go directly to Cython 3 yet.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
FWIW I wouldn't mind even jumping to C17. I don't think people compile against pandas so I don't think we need to worry about supporting old standards. 

I'd really love to jump to C23 as soon as all compilers support it
> FWIW I wouldn't mind even jumping to C17. I don't think people compile against pandas so I don't think we need to worry about supporting old standards.
> 
> I'd really love to jump to C23 as soon as all compilers support it

Is there anything we need from C17?

IIUC, the C standard doesn't really change that much anymore (at least not a lot compared to C++), and I'd prefer to only bump if we need something from a newer C standard.
C17 doesn't have any new features. Just bug fixes for C11
@WillAyd OK to merge here?

I'm a little worried about requiring recent compilers for C17 and above, so maybe we can discuss that in a separate issue?
Ok sure merge away. C17 is just bug fixes so I'm not sure which compilers would support one not the other today. But in either case this is an upgrade over C99
Thanks @lithomas1 good to get this updated
provide right index name if the index is not unique
State: closed

- [x] closes #55815 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
PERF: array_strptime
State: closed

array_strptime is conceptually similar to array_to_datetime but stores local values instead of UTC values, then we pass the result through a localization pass that goes through object dtype.  The upside of doing it the current way is that array_strptime does not need a fallback path for when mixed tzs/awareness is detected.  The downside is everything else.

This changes array_strptime to store UTC values and follow the same pattern used by array_to_datetime (hopefully we can move toward sharing more code, though some of that will have to wait until deprecations are enforced in 3.0).  It introduces a fallback _array_strptime_object_fallback that mirrors _array_to_datetime_object that we'll be able to get rid of once the mixed-tz deprecation is enforced in 3.0.

```
vals = np.array(["2016-01-01 02:03:04.567"] * 10**4, dtype=object)
vals2 = np.array([pd.Timestamp(x) for x in vals], dtype=object)
vals3 = np.array([x + "-01:00" for x in vals], dtype=object)

fmt = "%Y-%m-%d %H:%M:%S.%f"

%timeit pd.to_datetime(vals, format=fmt, cache=False)
2.39 ms ± 28.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
2.11 ms ± 45.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR

%timeit pd.to_datetime(vals2, format=fmt, cache=False)
23.9 ms ± 680 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)   # <- main
10.7 ms ± 113 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR


%timeit pd.to_datetime(vals3, format=fmt+"%z", cache=False)
99.7 ms ± 1.43 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- main
63.1 ms ± 1.65 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
```

Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance

Comments:
> but as far as I understand, _array_strptime_object_fallback can be removed in 3.0 anyway, right?

correct.

Thanks @jbrockmendel
Backport PR #55427 on branch 2.1.x (DOC: Remove outdated docs about NumPy's broadcasting)
State: closed

Backport PR #55427: DOC: Remove outdated docs about NumPy's broadcasting

Labels:
name: Docs

Comments:
This also needs the change included in https://github.com/pandas-dev/pandas/pull/55885 to have the doc build passing, but so will already merge this
DEPR: kind keyword in resample
State: closed

- [x] closes #55946 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Resample
description: resample method
name: Deprecate
description: Functionality to remove in pandas

Comments:
Was there an existing issue discussing this too?
no
Thanks @jbrockmendel
Parquet/Feather IO: disable PyExtensionType autoload
State: closed

Similar to https://github.com/apache/arrow/pull/38608, but doing it on our side as well for people that use latest pandas with older pyarrow. 
(using a similar approach as https://github.com/pitrou/pyarrow-hotfix, without adding a required dependency on that package)

Labels:
name: IO Parquet
description: parquet, feather

Comments:
Can you add a whatsnew?
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.1.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 851fea0ea38985cd7d5e0a3b7a7a7539b5883307
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #55894: Parquet/Feather IO: disable PyExtensionType autoload'
```

4. Push to a named branch:

```
git push YOURFORK 2.1.x:auto-backport-of-pr-55894-on-2.1.x
```

5. Create a PR against branch 2.1.x, I would have named this PR:

> "Backport PR #55894 on branch 2.1.x (Parquet/Feather IO: disable PyExtensionType autoload)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Thanks @jorisvandenbossche 
Manual backport -> https://github.com/pandas-dev/pandas/pull/55900
Backport PR #55764 on branch 2.1.x (REGR: fix return class in _constructor_from_mgr for simple subclasses)
State: closed

Backport PR #55764: REGR: fix return class in _constructor_from_mgr for simple subclasses

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Subclassing
description: Subclassing pandas objects

Comments:
I think we might need the fix in #55885 for this.
I think it is https://github.com/pandas-dev/pandas/pull/55427 that needs to be backported
Or both ;)
Removed my temp commits again
ENH: warn when silently ignoring log keywords in PiePlot
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Thanks @jbrockmendel
REF: Mess with data less outside of MPLPlot.__init__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Thanks @jbrockmendel
REF: Ensure MPLPlot.data is a DataFrame after __init__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Looks like typing issues cropped up
Thanks @jbrockmendel
REF: make plotting less stateful (6)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Just throwing on your radar while on this topic: It would be really nice to also avoid state on matplotlib axes themselves https://github.com/pandas-dev/pandas/issues/54485
yah ive got a branch thats chipping away at that but its really slow-going.
Thanks @jbrockmendel
TYP: plotting, make weights kwd explicit
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Visualization
description: plotting
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel
TST: Add dedicated test for plt.savefig
State: closed

I'm observing that testing `plt.savefig` seems to appear redundant over certain plotting types and not necessarily comprehensive over all the supported plotting types. Additionally, by isolating `plt.savefig` testing, I'm seeing a 20 second time save when testing `pandas/tests/plotting`

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting

Comments:
Looks like tests are passing here so going to merge. Happy to follow up if needed
REF: make plotting less stateful (4)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Thanks @jbrockmendel
REF: misplaced tests, parametrize, simplify
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @jbrockmendel
Added str in index_col attribute in read_excel function description #55843
State: closed

### Added str in index_col attribute of read_excel function...





Labels:

Comments:
Thanks for the PR but this has been fixed by https://github.com/pandas-dev/pandas/pull/55857 so closing. Happy to have your help on other issues
I will try to resolve some other issue @mroeschke and this was my first pr so I am open to feedback.
Backport PR #55227 on branch 2.1.x (BUG: Interchange object data buffer has the wrong dtype / from_dataframe incorrect )
State: closed

Backport PR #55227: BUG: Interchange object data buffer has the wrong dtype / from_dataframe incorrect 

Labels:
name: Interchange
description: Dataframe Interchange Protocol

Comments:
probably good to wait for https://github.com/pandas-dev/pandas/pull/55855 to check the tests still pass
TST: avoid usage of unnecessary xfail in test_datetime64
State: closed

I noticed those xfails when running the tests in this file locally, because those are for some reason very slow.

First, what we mark as "xfail" is not actually an expected test failure like not-yet-implemented or not-yet-fixed-bug (i.e. something that could be fixed at some point). It's simply that DataFrame broadcasting works differently, so if we want to test this, we should be asserting that it raises a ValueError, which is the expected behaviour. 
But for this test, we can actually use a fixture that excludes DataFrame and keep the test working the same for the rest.

And for some reason, this speeds up running the tests in this file a lot (for me locally, this goes from about 105s to 5s, saving almost two minutes). I don't understand why, though ;)

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @jorisvandenbossche
DOC: fix typos newly found by codespell
State: closed

Mostly in code comments and some in docs.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
looks like this was already done in https://github.com/pandas-dev/pandas/pull/55846 - closing then but thanks for your pr
Added str to index_col attribute in the read_excel function description
State: closed

- [ ] closes #55843
- [ ] All [code checks passed]
- [ ] Added [type annotations] to new arguments/methods/functions.


Labels:
name: IO Excel
description: read_excel, to_excel
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
@shyamala-venkat - thanks for the PR, I edited your OP above so that the issue reads `#55843`. By including the `#`, GitHub links to the issue and it will automatically be closed when this PR is merged.

Thanks @shyamala-venkat 
Thank you @rhshadrach for approving the pull request.
DEPS: Use ipython run_cell instead of run_code; remove pytest-asyncio
State: closed

pytest-asyncio is used for a unit test that uses an ipython API `run_code` that is async. I think the equivalent functionality is tested via `run_cell` which is not async and therefore allows us to remove the dependency.

Additionally, it appears this plugin is causing our CI failures

https://github.com/pytest-dev/pytest-asyncio/issues/655 
https://github.com/pandas-dev/pandas/actions/runs/6775657564/job/18415436948 

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dependencies
description: Required and optional dependencies

Comments:
This gets us closer to a green CI and `pytest-asyncio` IMO wasn't an important dependency so going to merge
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.1.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 31fde44d6616a7e3c0f184d47ef74be8e3ca72cd
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #55853: DEPS: Use ipython run_cell instead of run_code; remove pytest-asyncio'
```

4. Push to a named branch:

```
git push YOURFORK 2.1.x:auto-backport-of-pr-55853-on-2.1.x
```

5. Create a PR against branch 2.1.x, I would have named this PR:

> "Backport PR #55853 on branch 2.1.x (DEPS: Use ipython run_cell instead of run_code; remove pytest-asyncio)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
REF: make plotting less stateful (2)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting

Comments:
Thanks @jbrockmendel
Add classifier for 3.12
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Link https://github.com/pandas-dev/pandas/issues/53665
Thanks @jack-mcivor
ENH: Raise TypeError for offsets which are not supported as period frequency
State: closed

- [x] closes #55785
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Added to constructor of `Period` check if frequency is supported as period frequency, if not a ValueError is raised.

The reason: so far we get
`AttributeError: 'pandas._libs.tslibs.offsets.QuarterBegin' object has no attribute '_period_dtype_code'`

Labels:
name: Frequency
description: DateOffsets
name: Period
description: Period data type

Comments:
Thanks @natmokval 
@natmokval, does this PR also close: https://github.com/pandas-dev/pandas/issues/5091 ? 
> @natmokval, does this PR also close: #5091 ?

@cancan101, I think, this PR doesn't close #5091. We still have the same behavior. For example, frequency  changes from 'ME' to 'MS':
```
>>> ts = pd.date_range('1/1/2012', periods=4, freq=pd.offsets.MonthEnd())
>>> per = ts.to_period()
>>> ts_per = per.to_timestamp()
>>> ts
DatetimeIndex(['2012-01-31', '2012-02-29', '2012-03-31', '2012-04-30'], dtype='datetime64[ns]', freq='ME')
>>> per
PeriodIndex(['2012-01', '2012-02', '2012-03', '2012-04'], dtype='period[M]')
>>> ts_per
DatetimeIndex(['2012-01-01', '2012-02-01', '2012-03-01', '2012-04-01'], dtype='datetime64[ns]', freq='MS')
```
BUG: Fix rounding of percentile 99.999% to 100%
State: closed

- [x] closes #55765
- [x] [Tests added and passed]
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
> Could you add a whatsnew note to `v2.2.0.rst`?

Sure. Added
Thanks @parthi-siva