BrandonFors/pandas-github-issues huggingface dataset:
BUG: Interchange object data buffer has the wrong dtype / from_dataframe incorrect 
State: closed

- [ ] closes #54781 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I haven't included a whatsnew note as this isn't user-facing

Labels:
name: Interchange
description: Dataframe Interchange Protocol

Comments:
The tricky thing will be to test this ... Given that we don't yet have implementations to test with (pandas itself doesn't yet return the correct dtype), should we add some very specific unit tests explicitly targetting the helper function (where we can pass both types of dtypes). Although I assume that's still difficult, as those helpers take a Column object, and then we would have to mock a Column object just for those tests ..
good point, marking as draft til I get back to that
I've tried this out with:
- pandas: this branch
- polars: https://github.com/pola-rs/polars/pull/10787

and now
```python
df = pl.DataFrame({'a': ['foo', 'bar']})
print(pd.api.interchange.from_dataframe(df))
```
runs fine (whereas using pandas main and that branch from polars would've raised 
```python
>       assert protocol_data_dtype[2] in (
            ArrowCTypes.STRING,
            ArrowCTypes.LARGE_STRING,
        )  # format_str == utf-8
E       AssertionError
```
)

@stinodego fancy taking a look?
thanks for your review 🙏 ! I've tried addressing your comments 
> thanks for your review 🙏 ! I've tried addressing your comments

Yeah I think this should do it!
Any chance this can get merged and be part of the next release? 🙏 

This is blocking improvements to the protocol across all dataframe libraries.
@jorisvandenbossche fancy taking a look please?

I'd suggest backporting this, it has no user facing impact anyway
@MarcoGorelli 

Should this block the release?
> @MarcoGorelli
> 
> Should this block the release?

not at all! if it can't be backported to 1.1.2, no issue, definitely not a blocker
merging then, thanks all!
BUG: manage raw ods file without cell cache
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


```python
import ezodf
ods = ezodf.newdoc("ods", "test_unempty_cells.ods")
sheet = ezodf.Sheet("base", size=(6, 1))
ods.sheets += sheet

sheet[0, 0].set_value("Column 1")
for i in range(1, 6, 2):
    sheet[i, 0].set_value(i)
    #sheet[i, 0].append(ezodf.Paragraph(str(i)))
ods.save()
```

should generate

|    |   Column 1 |
|---:|-----------:|
|  0 |          1 |
|  1 |        nan |
|  2 |          3 |
|  3 |        nan |
|  4 |          5 |


but it generates (an empty DataFrame)

| Column 1   |
|------------|


A hacky fix in building the ODS file is commented out in the snippet `sheet[i, 0].append(ezodf.Paragraph(str(i)))`.
There is a cache in ODS file and data is correctly read from the origin but the logic to determine if a full row is empty mistakenly relies on the cache.

This PR removes that logic et fixes the bug.

Labels:
name: IO Excel
description: read_excel, to_excel

Comments:
Nice thanks @guillett
aca va tu mensaje de commit
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Por que?
sorry, I was just checking pandas documentation, didnt know that it's gonna pull a request in the main repository, just thought it was just working in my forked repository
DOC: Add whatsnew for 2.1.2
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
pre-commit.ci autofix
BUILD: Fix duplicate files warning
State: closed

- [ ] closes #54888  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
Is there a way for meson to automatically pick these up instead of manually having to specify? For sure doesn't change often, but would be nice to avoid hard coding
> Is there a way for meson to automatically pick these up instead of manually having to specify? For sure doesn't change often, but would be nice to avoid hard coding

Sadly, I think it has to be hardcoded, since the meson devs discourage using things like wildcards that would let us include all pyi files at once.
(Luckily, this is only for _libs though, since we have a mix of C/Cython/Python sources in there.)
Will self merge then so this gets in for the release.
One thing you could do is offload the file list itself to a separate plaintext file containing one filename per line, and use it from meson as:
```meson
sources_to_install = fs_mod.read('sources.list').strip().split()
```

It's then much easier to statically verify through any means you like that the list of files is updated. You could have a CI script verify it as a test, for example, or auto refresh the file via a git-pre-commit hook.

This fulfills meson's requirement -- that any changes to the build plan are detected and force a reconfigure, so that running `meson install` will not result in outdated installdata that doesn't install all the files it was supposed to. Use of `fs_mod.read()` causes the file in question to become a regenerate dependency.
BUG: RecursionError in loc.setitem
State: closed

- [x] closes #52927 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Thanks @jbrockmendel
Fix NameError in 'Evaluation order matters'
State: closed

This code in 'Indexing and selecting data' of doc wouldn't execute

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks. I think there's another occurrence of this usage in this file. Could you fix that one as well? 
@mroeschke done
Thanks @sergei3000
TYP: Allow None in Period.strftime
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Fixes the ASV failures in the first two lines of: https://github.com/pandas-dev/pandas/pull/55179#issue-1899772995

While not familiar with this code, I believe this is the correct resolution because (a) of the ASV `tslibs.period.PeriodUnaryMethods.time_strftime_default` explicitly passes `None` and (b) the `fmt` argument is passed to `period_format` which has the signature `object fmt=None`.

Labels:
name: Docs
name: Period
description: Period data type
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
It might be nice to mention in the doc-string what `None` does.
@twoertwein - done.
Thanks @rhshadrach
Revert "DEPR: Deprecate returning a DataFrame in SeriesApply.apply_standard
State: closed

Reasons for reverting:

- This was done for performance reasons, but this kills the use case to unpack nested data (lists for example)
- The deprecation is poorly done (axis 1 does not emit a warning)
- There is no documentation what users should do, this is necessary if we deprecate something for performance reasons

#52116

closes #55134

Labels:
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
I'm okay with reverting this for now. Could you link any related open user issues (if available)?

Also might be worth adding a unit test for unpacking nested data with apply.

cc @topper-123 
Added a test and linked the issue in the top post
Thanks @phofl. This deprecation is probably highly relevant in the PDEP 13 discussion 
yep agree
CI: Fix no BLAS error in 32 bit build
State: closed


Labels:
name: CI
description: Continuous Integration
name: 32bit
description: 32-bit systems

Comments:
Guessing there is nothing we can install to get openBLAS on this host?
I think we might, but not sure how to approach it. From https://github.com/numpy/numpy/issues/24703, my understanding is that this workaround (or installing openblas) shouldn't be necessary soon
DOC: correct an example in v0.14.0.rst to fix "NameError"
State: closed

while building whatsnew/v0.14.0.rst the error occurs:
```
----> 1 g[['B']].head(1)

NameError: name 'g' is not defined
```
corrected the example to fix the error.

Labels:
name: Docs
name: Groupby

Comments:
@rhshadrach, could you please take a look at this PR?
Thanks @natmokval 
Thanks @rhshadrach for reviewing this PR.
DOC: added deprecation message in docs for DataFrame.rolling
State: closed

- [ ] closes #55133 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Docs
name: Window
description: rolling, ewma, expanding

Comments:
@lithomas1 can you please help me understand the errors?
Thanks @rsm-23
DOC: Update read_csv docs to reflect behaviour with `parse_dates`
State: closed

- [x] closes #55158
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
Thanks @jfadia
DEPS: Bump to 1.26 proper for Python 3.12
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dependencies
description: Required and optional dependencies
name: Python 3.12

Comments:
Thanks @lithomas1
BUG: .rolling() returns incorrect values when ts index is not nano seconds
State: closed

- [X] closes #55106
- [X] closes #55026
- [X] [Tests added and passed]

The bug is referred to and fixed inside the code. It needs to be generalised. @jorisvandenbossche we need to have the first feedback if we can generalise the data type with some utils or more generally. Done during the in-person sprint check.

Labels:
name: Bug
name: Window
description: rolling, ewma, expanding
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution
name: Sprints
description: Sprint Pull Requests

Comments:
The test was parametrised, the rolling unit generalised and the version commit reverted :)
@jorisvandenbossche please let me know in case of any other changes
Bumping off of 2.1.1.
@lithomas1 This fix is also related to other subfunctions like `max`. More at https://github.com/pandas-dev/pandas/issues/55026. The PR with the related tests is available.
Can you add a whatsnew note for 2.1.2?
> Can you add a whatsnew note for 2.1.2?

Gladly added :)
@jorisvandenbossche I have modified the validation function based on your suggestion. Tests are merged and the previous conversion has been removed. Please check it out and let me know 
@jorisvandenbossche I was about to ask how is the merge on this fix going on? Anything that I can help with?
Tests are failing

```
FAILED pandas/tests/window/test_rolling.py::tests_empty_df_rolling[1s] - TypeError: cannot get datetime metadata from non-datetime type
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_invalid_window_nonfixed[2MS] - AssertionError: Regex pattern did not match.
 Regex: '\\<2 \\* MonthBegins\\> is a non-fixed frequency'
 Input: 'Value must be Timedelta, string, integer, float, timedelta or convertible, not MonthBegin'
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_invalid_window_nonfixed[freq1] - AssertionError: Regex pattern did not match.
 Regex: '\\<2 \\* MonthBegins\\> is a non-fixed frequency'
 Input: 'Value must be Timedelta, string, integer, float, timedelta or convertible, not MonthBegin'
FAILED pandas/tests/window/test_timeseries_window.py::TestRollingTS::test_rolling_on_empty - TypeError: cannot get datetime metadata from non-datetime type
```
Thanks @hkhojasteh and @jorisvandenbossche
DOC: complete parameter descriptions for DataFrame.insert's docstring
State: closed

issue: https://github.com/jorisvandenbossche/pydata-amsterdam-pandas-sprint/issues/1
- bullet point item #5: "complete the parameter descriptions"

- [x] closes https://github.com/jorisvandenbossche/pydata-amsterdam-pandas-sprint/issues/1, bullet point 5
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

@jorisvandenbossche 


Labels:
name: Docs
name: Sprints
description: Sprint Pull Requests

Comments:
Thanks @mcnoat
DOC: Added alternative examples for deprecated DataFrame.last()
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Datetime
description: Datetime data dtype
name: Stale
name: Sprints
description: Sprint Pull Requests

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: fix see also links in pandas.DataFrame.agg docstring
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Sprints
description: Sprint Pull Requests

Comments:
Thanks @jorisvandenbossche for the code review and inputs!
DOCS: temporary pin pydata-sphinx-theme to 0.13
State: closed


The pydata-sphinx-theme has a new release, and that causes some issues with the top bar:

![image](https://github.com/pandas-dev/pandas/assets/1020496/c0f2a1ff-c2a4-46e8-9ed9-6ab43316c20c)

So we can maybe temporarily pin the theme until that is fixed

Labels:
name: Docs

Comments:
(labeling as 2.1.1, because we should ensure to use the previous theme in the release branch in case we don't fix this by the time of the 2.1.1 release)
thanks @jorisvandenbossche
Update __init__.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks but please tackle issue labeled `good first issues` if interested in contributing!
CLN: de-duplicate code in _matplotlib.converter
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Visualization
description: plotting

Comments:
Thanks @jbrockmendel
Replace cpplint with clang-format
State: closed

cpplint isn't seemingly well maintained and focuses on C++, which we aren't using. I think clang-format is a much better tool for our formatting

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Since we are adding a hook to pre-commit I'm guessing pre-commit.ci will have to be updated somehow to account for it. Does @MarcoGorelli or @lithomas1 know how that works?
> Since we are adding a hook to pre-commit I'm guessing pre-commit.ci will have to be updated somehow to account for it. Does @MarcoGorelli or @lithomas1 know how that works?

I think Marco's out on vacation right now, so I'll do my best to help.

Can we try not using pocc's wrapper around clang-format? I think that one doesn't install clang-format for you, so it won't work on pre-commit CI. 

You can try adding something manual like 
https://github.com/pandas-dev/pandas/blob/cc58350c92a9e24bef586cd1b898c01640b33abc/.pre-commit-config.yaml#L131-L388

(see the pyright one. 
We can try adding a new entry something like
```
id: clang-format
name: clang-format
entry: << clang-format command goes here >>
language: conda
additional_dependencies:
- clang-format
```
(I'm just typing this off of my head. Idea is to get pre-commit to pull clang-format off of conda. Don't expect it to work out of the box, though)
Ah OK thanks. I'll give that a shot - didn't realize pocc was missing something compared to the other repo to auto install
Any concerns on this one?
Thanks @WillAyd
docs: add explanation about `freq` units in `Timedelta.round()`
State: closed

- ~closes #xxxx (Replace xxxx with the GitHub issue number)~
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type

Comments:
Thanks @Borda