BrandonFors/pandas-github-issues huggingface dataset:
STYLE enable ruff PLW2901
State: closed

- [ ] closes #51708
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added relative entries in "to be gradually enabled" inside pyproject.toml.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Hi @MarcoGorelli, I just pushed an update of the PR. I renamed all the variables by setting a more intuitive variable name. Please let me know if it's fine for you. In that case I will continue like this also for the other sections.
> thanks for working on this
> 
> to be honest, I'm no longer sold on enabling this code, having seen the changes it entails it probably makes things less clear more often than not. this is my fault for having suggested the issue, sorry about that
> 
> the `line_stripped` change one is nice though, that's probably an improvement
> 
> I'd suggest we:
> 
>     * keep the `line_stripped` change
> 
>     * revert the rest
>       and then we can get this in

Oh no problem, no need to say sorry. To be fair I should've thought a bit more on the variable names. Will wait for a feedback on the last push before continuing.
looks like most tests are failing, could you check what it might be please?
> looks like most tests are failing, could you check what it might be please?

The line "pandas/core/*" was repeated twice in the pyproject.toml file. I didn't see it in the merge conflict. Now should be fine.

The error seems to be connected to my changes in the `scripts/no_bool_in_generic.py` file. I will investigate and fix it today.
Hi @MarcoGorelli, I think I completed all the requested changes. Please let me know if I missed something. Just updated the branch with main and fixed the bug I introduced. 
@MarcoGorelli Sorry I have been away in the last few days. I just pushed the last merge with the main branch. Ideally there should be no more errors. If that is the case do you need any further update of the branch?
PERF: PeriodArray._format_native_types
State: closed

- [x] closes #23979 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

#23979 suggests implementing PeriodArray._format_native_types with self.to_timestamp()._format_native_types.  That _is_ faster than this, but this avoids copies, is robust to weekly freqs, and should further improve with #51459.

```
pi = pd.period_range("2016-01-01", periods=10**5, freq='H')
pa = pi._data
pa[4] = pd.NaT

%timeit pa._format_native_types()
246 ms ± 5.61 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- PR
411 ms ± 13.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- main

%timeit pa._format_native_types(date_format="%Y-%m-%d")
274 ms ± 8.41 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- PR
420 ms ± 13.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- main

In [18]: %timeit pa.to_timestamp()._format_native_types()
164 ms ± 3.43 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Period
description: Period data type

Comments:
Thanks @jbrockmendel
BUG: PeriodDtype constructor allowing invalid construction, incorrect exception class
State: closed

- [x] closes #27388 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Period
description: Period data type

Comments:
Thanks @jbrockmendel
ENH: Let to_feather accept index and non string column names
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

this is no longer necessary

Labels:
name: IO Parquet
description: parquet, feather

Comments:
Thanks @phofl
CI: Fix np dev build failure
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
thanks @phofl.
CLN: assorted
State: closed

- [x] closes #34381 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

closes #34381 by editing to_offset docstring to clarify returned object is BaseOffset subclass

Labels:
name: Clean

Comments:
Thanks @jbrockmendel
WEB: Add robots.txt to disallow docs preview
State: closed

With the new doc previewer system, we have multiple copies of our web and docs in the `/preview/` path of our domain. I initially added a `robots.txt` file in that directory to be excluded by crawlers, but I double-checked, and seems like crawlers just check for the file in the root. So, it needs to be added here.

Labels:
name: Web
description: pandas website

Comments:
Thanks @datapythonista
PERF: fastpaths in tzconversion
State: closed

Should address some asv regressions that happened between 1.4.0 and 1.5.0

Labels:
name: Performance
description: Memory or execution speed performance
name: Timezones
description: Timezone data dtype

Comments:
Which ones in particular improved?
i dont have them on hand, but my recollection is it was tslibs-specific asvs and it was utc cases and empty cases, both of which had gone from super-fast to just-fast
Thanks @jbrockmendel
STYLE enable TCH on pandas/core/io
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
so this one can just be removed? cool, thanks @potap75
ENH: Implement DataFrame.from_pyarrow and DataFrame.to_pyarrow
State: closed

- [X] closes #51760

A mention in the docs is missing, and I guess we can add `pyarrow.Table` to the type annotations of what `DataFrame` accepts, but this should be mostly done otherwise.

Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks all for the feedback. Changed the approach here, to use `DataFrame.from_pyarrow` and `DataFrame.to_pyarrow`.

There are couple of things that could still be added:

- Showing those methods in the PyArrow user guide
- Exposing the parameters in `pyarrow.Table.from_pandas` to `DataFrame.to_pyarrow`

Also, there is a test failure, but seems like it's an error in `assert_frame_equal` (or at least the error message doesn't make any sense). But would be great to know that there aren't major objections to this approach before moving forward with this PR.
Thanks all for the feedback, good points. I'll address the comments if there is consensus to move forward with this. @phofl [thinks](https://github.com/pandas-dev/pandas/pull/51762#issuecomment-1454750287) we could maybe just let people use PyArrow methods instead. I opened #51799 to discuss a new API that would allow PyArrow itself to implement the pandas methods instead.
To clarify, I am not sure about this. Just don’t know if these functions add much value or are more likely to cause confusion?

one benefit is that we can select a good default value for types_mapper, but can’t see much apart from that 
yeah would be likely to accept your pdep; this is a bit ad hoc
Cool. Let's close this in favor of https://github.com/pandas-dev/pandas/pull/51799, which seems a better approach.
CLN: Use type_mapper instead of manual conversion
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: Arrow
description: pyarrow functionality

Comments:
pandas/io/parquet.py:253: error: Incompatible types in assignment (expression has type "Type[ArrowDtype]", target has type overloaded function)  [assignment]

thx fixed
Good call. I think there are a few more places where we can use type_mapper


```
./pandas/io/orc.py:                        pa_table.column_names, pa_table.itercolumns()
./pandas/io/feather_format.py:                        pa_table.column_names, pa_table.itercolumns()
./pandas/io/parsers/arrow_parser_wrapper.py:                    for col_name, pa_col in zip(table.column_names, table.itercolumns())
./pandas/io/parquet.py:                            pa_table.column_names, pa_table.itercolumns()
./pandas/io/json/_json.py:                                    pa_table.column_names, pa_table.itercolumns()
./pandas/tests/io/test_parquet.py:                    pa_table.column_names, pa_table.itercolumns()
```
Updated 
Merging this into 2.0 would have the benefit that we get index support as soon as pyarrow 12 is out
> Merging this into 2.0 would have the benefit that we get index support as soon as pyarrow 12 is out

SGTM
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 3e0c1dab6bcbe95ea1349f144e6f4bc606471ae0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #51766: CLN: Use type_mapper instead of manual conversion'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-51766-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #51766 on branch 2.0.x (CLN: Use type_mapper instead of manual conversion)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: Add explanation how to convert arrow table to pandas df
State: closed

- [ ] closes #51760 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Arrow
description: pyarrow functionality

Comments:
Not totally sure, maybe it's better for now to expected users to be explicit when converting? I think we can expected users that have arrow tables to be aware of the to_pandas method?
Thanks @phofl, this is useful
TST: Test to verify behavior of groupby.std with no numeric columns and numeric_only=True
State: closed

- [x] closes #51080
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby

Comments:
Thanks @NumanIjaz!
DEPR: added deprecation warning for NDFrame.bool
State: closed

- [x] closes #51749
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
You'll need to update the docs to remove/suppress deprecated usages

> You'll need to update the docs to remove/suppress deprecated usages

Sorry, but I coudn't understand can please explain it again or are you talking about adding a whatsnew entry
@ramvikrams - the method you are deprecating is used in our documentation. Building the docs now emits warnings; the documentation needs to be adjusted so that these warnings no longer appear:

https://github.com/pandas-dev/pandas/actions/runs/4323640786/jobs/7547478298#step:6:67
> @ramvikrams - the method you are deprecating is used in our documentation. Building the docs now emits warnings; the documentation needs to be adjusted so that these warnings no longer appear:
> 
> https://github.com/pandas-dev/pandas/actions/runs/4323640786/jobs/7547478298#step:6:67


I actually looked over the net and revious pr's but could not find out how to adjust the documentation , can you please sujjest a walkthrough or any pr that had the same issue
> can you please sujjest a walkthrough or any pr that had the same issue

#51395
So should I remove the examples from docstring of bool and other bool instances in the doc
> So should I remove the examples from docstring of bool and other bool instances in the doc

I think you should leave the examples in. You can add them to the filtered warnings here:

https://github.com/pandas-dev/pandas/blob/27f7365e3ea3d8827b7ccd210f2f9a39aa3cf601/pandas/conftest.py#L133
@rhshadrach actually now I did the required changes but still `Doc Build and Upload` fails could you please check
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/51756/
@rhshadrach is the PR mergeable now
Thanks @ramvikrams!
> Thanks @ramvikrams!

thanks sir
STYLE enable ruff TCH on pandas/core/arrays
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
I finished the task STYLE enable ruff TCH #51740 


thanks @clarkwiththew - can you remove the folder for which you've enabled this (pandas/core/arrays) from `pyproject.toml` please?
I removed the folder from pyproject.toml

thanks - can you also run `pre-commit run --all-files` please?
something went wrong when merging, there's lots of unrelated changes
I'm fixing it right now
I've made some changes, hopefully the issues should be removed
not quite, sorry
I think the pre-commit changed a lot of other files, I'm going to try again on a new branch
I ran the pre-commit and only the edited files should be changed now. I made a new PR here:

https://github.com/pandas-dev/pandas/pull/51786
STYLE enable ruff PLR5501 #51709
State: closed

- [x] closes #51709
- [x] Added "elif instead of else if"

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
well done! looks like there's a merge conflict, could you resolve it please?
hey - please check https://www.atlassian.com/git/tutorials/using-branches/merge-conflicts
Hey, I wanted to ask, what these errors are about I can't quite get it?
Hey, this new error is because of numpy i think. it is not being import for some reason.
yup, that one's unrelated - this should be good to go, I'll check again later, thanks!
Hey @MarcoGorelli , did u find anything?
Hey, 
Hey,
Thanks for advice and helping solve these issue checks cleared. what should I do next?
nothing, this looks ready, feel free to take on other issues if interested - I'll merge when I've had a chance to double-check everything
DOC: Fix item docstring
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Hey @phofl `Series.item()` shares the docstring with `Index.item()`. So, with this change the word `Series` is shown in `index.item()` too:
https://pandas.pydata.org/docs/dev/reference/api/pandas.Index.item.html#pandas.Index.item

Also, it should be "If the data is not length = 1" instead of "-1",  shouldn't it be??
Revert "CLN: Remove read_orc dtype checking (#51604)"
State: closed

This reverts commit b69bd0755d654c0dba42042df7b2251918274327.

- [x] closes #51733 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Looks like timeouts started after this commit. Checking if this would fix it before looking closer

Labels:
name: CI
description: Continuous Integration
name: Arrow
description: pyarrow functionality

Comments:
Was able to reproduce locally, we get segfaults with pyarrow 7.0, 8.0 works as expected
LGTM
thanks @phofl
BUG: indexing empty pyarrow backed object returning corrupt object
State: closed

- [x] closes #51734 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

looks like an empty chunked array creates problems later on.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Arrow
description: pyarrow functionality

Comments:
I can confirm that it solves the original issue.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 a07cb65459f24446e82354854ff6658f29414c0a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #51741: BUG: indexing empty pyarrow backed object returning corrupt object'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-51741-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #51741 on branch 2.0.x (BUG: indexing empty pyarrow backed object returning corrupt object)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).