BrandonFors/pandas-github-issues huggingface dataset:
DEPR: positional indexing on Series __getitem__/__setitem__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref #50617, possibly closes

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
PERF: Performance regression in Groupby.apply with group_keys=True
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I came accross this bottleneck through the changed default of ``group_keys``. Technically this is not a regression, but it's visible to all users that don't set ``group_keys`` explicitly. This change provides around a 20-30% speedup in groupby apply when you have many groups

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Nice find. Thanks @phofl 
> Technically this is not a regression, but it's visible to all users that don't set `group_keys` explicitly.

Just want to make sure I understand this - `group_keys=False` was much faster than `group_keys=True`, and this PR improves the `group_keys=True` case. Is that right?
Yep
use is_integer in df.insert
State: closed

- [ ] closes #53193 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Which section of the whatsnew note should I add it to, Conversion?
> Which section of the whatsnew note should I add it to, Conversion?

Sure
done

> > Which section of the whatsnew note should I add it to, Conversion?
> 
> Sure


Thanks, @auderson.
CI/BLD: Build Linux ARM wheels on GHA
State: closed

Builds on top of https://github.com/pandas-dev/pandas/pull/53087

Uses QEMU on GHA to build/test these wheels instead of using CircleCI separately

Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
Thanks for trying this out, but I don't think we should move forward with this. 
3/4 hours is just way too long, compared to the 30-40ish minutes that CircleCI takes.
Yeah I can't find a way to speed this up so going to close. Happy to have your review on https://github.com/pandas-dev/pandas/pull/53087
BUG/CoW: Series.rename not making a lazy copy when passed a scalar
State: closed

- [ ] closes #52450 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I put this in for 2.0.2 since we claim that the lazy-copy was implemented for ``rename`` in 2.0


Labels:
name: Copy / view semantics

Comments:
thx @lithomas1
DOC: Fix version switcher
State: closed

- [X] closes #53180

Labels:
name: Docs

Comments:
This is the same error I was getting in my earlier branch

```
2023-05-11T18:16:18.6054339Z WARNING: The version switcher "https://pandas.pydata.org/versions.json" file cannot be read due to the following error:
2023-05-11T18:16:18.6055458Z HTTPError('404 Client Error: Not Found for url: https://pandas.pydata.org/versions.json')
```
I think the error is because the CI docs build is looking for the production versions file, which it doesn't exist right now, but it'll exist after merging. So I'm going to merge this, and I think the next docs build should work.
Ok, it's a bit trickier, if I merge the next build will also fail before creating the file. I created the file manually in production and I'm rerunning. If it works, I'll merge, and when the docs build syncs with prod, it'll start rewriting the file.

If something is merge before this, it'll delete the file again, but I'll just recreate it until this works.
Thanks for investigating @datapythonista . Feel free to merge when ready
CI is green, and the file is still there, I think everything should be fine once the CI of the merge completes.
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/53188/
DOC: Fix rst formatting in dev environment docs
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This isn't formatted correctly in the [currently deployed version of the docs](https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas).

Labels:
name: Docs

Comments:
Thanks @ngoldbaum
CI: Clean caches of closed PRs
State: closed

Looks like we regularly go over the 10GB CI cache limit. One step that should help is removing CI caches from closed PRs

https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries

Labels:
name: CI
description: Continuous Integration

Comments:
Can we test this somehow without merging?
I guess we could change the `on` condition to make this PR clean the caches, but I think this PR is pretty innocuous. I'll merge for now and see how things go on main. I can revert if it causes any problems
DOC Fixing EX01 - added examples
State: closed

Towards #37875

I couldn't locally build the html files for `Series.SeriesGroupBy.is_monotonic_increasing` and `Series.SeriesGroupBy.is_monotonic_decreasing`. Followed https://github.com/noatamir/pyladies-berlin-sprints/issues/13 to split the docstrings from Series - but I'm not sure if adding #GH13 to the docstring is correct

Couldn't use a variable like {klass} for `item()` to avoid hardcoding `Series` or `Index`.

Labels:
name: Docs

Comments:
I can't tell (for sure) if failing tests are related to my changes 
Thanks @MarcoGorelli! - love the cat
Switch to setup-micromamba
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

`mamba-org/provision-with-micromamba` is now replaced by `mamba-org/setup-micromamba`.

Labels:

Comments:
Thanks, but looks like this is already being worked on in https://github.com/pandas-dev/pandas/pull/53168 so closing this one in favor of that
Add asv benchmarks for Block.setitem()
State: closed

- [x] closes #25756 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I believe that this solves the issue , however let me know if i need to change anything as i am not familliar with asv benchmarks

Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
Thank you for your response @mroeschke  . I solved the issues as you told me , if there is anything else i can do let me know.
Looks like the new benchmark is failing, please run it locally to see if it's working correctly https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#running-the-performance-test-suite
I was able to resolve the issue on my local system. Thank you for your feedback @mroeschke 
@mroeschke moved the import at the start of the file as you told me 
@mroeschke Looks like that the tests have passed now and i also moved the imports as you told me. If there is anything else i can do, please let me know.
Thanks @steliospetrakis02
Add `np.intc` to `_factorizers` in `pd.merge` 
State: closed

- [x] closes #52451
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

continuation of https://github.com/pandas-dev/pandas/pull/52478

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Windows
description: Windows OS

Comments:
One comment otherwise looks good
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 64e88e3d45a4cebe110a20634ae280297e4951a8
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53175: Add `np.intc` to `_factorizers` in `pd.merge` '
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53175-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53175 on branch 2.0.x (Add `np.intc` to `_factorizers` in `pd.merge` )"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Thanks @Hoxbro
Add test for groupby with TimeGrouper
State: closed

- [x] closes #17202
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
The added test doesn't look related to the linked issue?
Thank you for your  response @mroeschke .The test I am referring to is test  #17202 . I changed it now
Thanks @steliospetrakis02
BUG/REF: ArrowExtensionArray non-nanosecond units
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

Fixes a few bugs related to non-nanosecond units for pyarrow duration and timestamp types.

The underlying issue is upstream (https://github.com/apache/arrow/issues/33321) where pyarrow does not handle pandas `Timedelta` and `Timestamp` non-nano units correctly.

Example (1 seconds -> 0 seconds):

```
In [1]: import pandas as pd

In [2]: import pyarrow as pa

In [3]: td = pd.Timedelta(1, unit="s").as_unit("s")

In [4]: pa.scalar(td, type=pa.duration("s"))
Out[4]: <pyarrow.DurationScalar: datetime.timedelta(0)>

In [5]: pa.array([td], type=pa.duration("s"))
Out[5]: 
<pyarrow.lib.DurationArray object at 0x10751d120>
[
  0
]
```


Labels:
name: Bug
name: Upstream issue
description: Issue related to pandas dependency
name: Arrow
description: pyarrow functionality
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Is there some amount of this we can hope to rip out once pyarrow is updated?
> Is there some amount of this we can hope to rip out once pyarrow is updated?

I think it can all be removed eventually. It may take some time as it needs to be fixed upstream first and then the pyarrow min version needs to catch up.
Thanks @lukemanley
Switch to setup-micromamba
State: closed

We are deprecating `provision-with-micromamba` and replacing it with https://github.com/mamba-org/setup-micromamba.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
Thanks looks pretty good. Could you merge in main once more and take out of draft mode?
@pavelzw sorry for not synchronizing on this. I’ll merge your changes into this. Anything else I should include?
No worries, I didn't see anything else that was not in my PR on first glance.
Should be good when green
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 2cce8385a3d8822a2058fdd4a8228fe45baf0cd1
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53168: Switch to setup-micromamba'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53168-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53168 on branch 2.0.x (Switch to setup-micromamba)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Sweet, thanks @jonashaag 
Noting based on the conflicts, I don't think we should backport this. 
@jonashaag looks like `extra-specs` is not a valid argument in `setup-micromamba` what is the equivalent way to install environment packages ad hoc?

https://github.com/pandas-dev/pandas/actions/runs/4961454335/jobs/8878246555
```yml
extra-specs: |
  a
  b
  c
```

should be changed to
```yml
create-args: >-
  a
  b
  c
```
Thanks! I will create a PR to update this
Whoops thanks
ENH/DEPR: infer date objects to date[pyarrow] dtype
State: closed

Sits on top of #53025.

A few issues this brings up that we should decide on before merging

- [ ] constructing a Categorical from an object-dtype Index does type inference on the categories; is that always wanted?
- [ ] infer_dtype will see a mixed sequence of dates-and-datetimes and return "date"; sure we want that?  AFAICT the main place it is relevant is in Index._maybe_promote
- [ ] Desired behavior of `DatetimeIndex.get_indexer(index_of_dates)`?  ATM the index_of_dates would have object dtype and get cast to DTI in Index._maybe_promote.
- [ ] date32 vs date64?
- [ ] extend the relevant tests with the future behavior
- [x] handle construction from scalar

Labels:
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: datetime.date
description: stdlib datetime.date support
name: Arrow
description: pyarrow functionality

Comments:
mothballing until after (at least) #53025
BUG: from_dataframe with empty objects
State: closed

- [ ] closes #53155 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality
name: Interchange
description: Dataframe Interchange Protocol

Comments:
Going to merge, but happy to address any followups.
BUG: SparseDtype requires numpy dtype
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Sparse
description: Sparse Data Type

Comments:
Seems like a user was relying on passing a sparse categorical type, does it mean it needs deprecating? https://github.com/pandas-dev/pandas/issues/49987
> Seems like a user was relying on passing a sparse categorical type, does it mean it needs deprecating? https://github.com/pandas-dev/pandas/issues/49987

They were, but then it would silently give them an unwanted cast to Sparse[object]

```
df = pd.DataFrame([["a", 0],["b", 1], ["b", 2]], columns=["A","B"])
>>> df["A"].astype(SparseDtype("category"))
Sparse[object, nan]
```
Thanks @jbrockmendel
TYP: remove mypy ignore[assignment] from pandas/core/methods/to_dict.py
State: closed

Related to #37715
mypy ignore[assignment] was removed from pandas/core/methods/to_dict.py

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hi @jbrockmendel, could you please take a look at this pr?
theres nothing wrong here, but @simonjayhawkins has made a pretty good argument that we don't want to alter code to satisfy mypy.  maybe he can comment on an alternative
> theres nothing wrong here, but @simonjayhawkins has made a pretty good argument that we don't want to alter code to satisfy mypy. maybe he can comment on an alternative

Thank you, @jbrockmendel, it sounds reasonable.
DOC: #53035 clarify the behavior of sep=None.
State: closed

Clarified that only the first valid line is used to detect the separator if sep=None in read_csv.

- [ ] closes #53035  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
Thanks @Toroi0610