BrandonFors/pandas-github-issues huggingface dataset:
DOC: start v1.6.0 release notes
State: closed

xref #45198

Labels:
name: Docs
name: Release

Comments:
Guess these need updating 

```
2022-08-23T18:04:09.3434488Z /home/runner/work/pandas/pandas/doc/source/whatsnew/index.rst:30: WARNING: toctree contains reference to nonexisting document 'whatsnew/v1.6.0'
2022-08-23T18:04:09.3435191Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v.1.6.0.rst:225: WARNING: Explicit markup ends without a blank line; unexpected unindent.
2022-08-23T18:04:09.3435768Z looking for now-outdated files... none found
2022-08-23T18:04:09.5629983Z pickling environment... done
2022-08-23T18:04:09.5644924Z /home/runner/work/pandas/pandas/doc/source/whatsnew/v.1.6.0.rst: WARNING: document isn't included in any toctree
```
> Guess these need updating

same problem in https://github.com/pandas-dev/pandas/pull/45198, needs a blank line, see commits in #45198
WEB: Use mambaforge for the getting started installation instructions?
State: closed

- [x] closes #46793 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Also doing some other assorted minor doc changes whilst here

Labels:
name: Docs

Comments:
@datapythonista seeing as you'd opened the issue - any objections here?
Thanks @MarcoGorelli. Happy to have the followups in another PR 
@meeseeksdev backport 1.5.x
backporting as this doesn't show up in the docs
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 3a3ed661c0a62b97f6fffc05bbf1fe6769b908cc
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #48220:  WEB: Use mambaforge for the getting started installation instructions?'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48220-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48220 on branch 1.5.x ( WEB: Use mambaforge for the getting started installation instructions?)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
TST: dt.total_seconds() stores float with appending 0.00000000000001 fix
State: closed

- [x] closes #34290 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Timedelta
description: Timedelta data type

Comments:
Thanks @Th3nn3ss
REGR: properly update DataFrame cache in Series.__setitem__
State: closed

Potential fix for https://github.com/pandas-dev/pandas/issues/47172


https://github.com/pandas-dev/pandas/pull/43406 added an `inplace` keyword to `_maybe_update_cacher`, but made the default `False`. While before that PR, the effective behaviour would rather match `True` I think (that PR was meant to make some cases _not_ inplace). 
However, if we are in the case of `Series.__setitem__` needing a cache update, so meaning that this series is a column of a DataFrame and we might need to update that DataFrame, I think we always should try update that DataFrame inplace (we are not in a `DataFrame.__setitem__` case, but rather setting into a column we got with `DataFrame.__getitem__`, so chained assignment, which currently still is expected to work in this case).

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
I am not fully sure how to exactly test https://github.com/pandas-dev/pandas/issues/47172 (but I verified manually that it fixes the memory increase), but I did add a test that fails on main and 1.4 (and worked on pandas 1.3) and passes with this fix.
> Potential fix for #47172
> 
> #43406 added an `inplace` keyword to `_maybe_update_cacher`, but made the default `False`. While before that PR, the effective behaviour would rather match `True` I think (that PR was meant to make some cases _not_ inplace). However, if we are in the case of `Series.__setitem__` needing a cache update, so meaning that this series is a column of a DataFrame and we might need to update that DataFrame, I think we always should try update that DataFrame inplace (we are not in a `DataFrame.__setitem__` case, but rather setting into a column we got with `DataFrame.__getitem__`, so chained assignment, which currently still is expected to work in this case).

Thanks for the explanation. makes sense.

> I am not fully sure how to exactly test #47172 (but I verified manually that it fixes the memory increase), but I did add a test that fails on main and 1.4 (and worked on pandas 1.3) and passes with this fix.

looks fine.
@meeseeksdev backport 1.5.x
WEB: Removing links to pdf version of the docs from web and docs
State: closed

Remove the links to the docs pdf in preparation to not generate it anymore in future releases.

Labels:
name: Docs
name: Web
description: pandas website

Comments:
> LGTM. Related, do you think we should keep the commands to generate the PDF (in `make.py` I believe)?

Good point. I'd leave it for now. We may still get feedback of users who do care about the pdf, or maybe people want to be able to build the pdf locally even if we don't publish them anymore. I'd leave it for now, as it's just one function in that make.py, so almost no maintenance or complexity cost.
Thanks @datapythonista
PERF: #48212 Cache the results of get_block_type (1.1.x)
State: closed

This gave a ~5% performance boost. It looks like things started to slow down in this area after 0.19.2 with more costly type checking.

As mentioned on #48212, I'm upgrading a large codebase and 1.1.x is the limit I can get to without significant changes (upgrading of codebase, libraries, and environments/machines), so this is a great stepping stone. I'd appreciate if this would get accepted as a patch release for it, I'm happy to go through the other 1.x branches to apply the change once a pattern is confirmed.

It looks like the automated CI run doesn't like trying to run on/for 1.1.x, so I'll have a run when I swap machines, but imagine I'll have to make a branch based on master sooner rather than later to get confidence from CI.

- [ ] ~closes #48212 (Replace xxxx with the Github issue number)~ - not closing as targeting a 1.1.x
- [ ] ~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Hi, thanks for looking into this.

We do not backport anything to our old branches except the newest released branch. This is currently 1.4.x

Please always submit the prs to main. This is necessary to see that everything passes before we backport some changes. That said, in general we do not backport performance improvements if there is not a good reason for it (large performance decrease that can be isolated really well)
CI: Automatically upload docs on release
State: closed

This should make the publishing of the docs automatic one creating and pushing a new tag. My understanding is that versioneer should take care of showing the version correctly in the docs.

I'd merge this before the 1.5 release in few hours, and if there is any problem, we do the publishing manually once more, and we fix the action for the next release. But I don't think it should fail.

The pdf is missing, but I think we're happy to remove it (I'll open a separate PR for it).

If I'm not wrong, for patch releases we upload the new docs, but delete the previous docs of that minor (so, for 1.4.4 we would delete 1.4.3 docs), and we create a symlink for the previous release. If that's the case, this would still be useful, but we should add in a follow up the deleting and the symlinks.

CC: @simonjayhawkins @mroeschke @lithomas1 @jorisvandenbossche 


Labels:
name: Docs
name: CI
description: Continuous Integration
name: Release

Comments:
> If I'm not wrong, for patch releases we upload the new docs, but delete the previous docs of that minor (so, for 1.4.4 we would delete 1.4.3 docs), and we create a symlink for the previous release. If that's the case, this would still be useful, but we should add in a follow up the deleting and the symlinks.

No we keep the all the patch release docs. AFAIK the only time we overwrite docs is for the release candidate docs (as they all point to, say, 1.5.0)
> No we keep the all the patch release docs.

but we do change the symlink.

so at the moment https://pandas.pydata.org/pandas-docs/version/1.4/ points to https://pandas.pydata.org/pandas-docs/version/1.4.3/, but the 1.4.3 docs will not be removed on the 1.4.4 release.
Ah, that makes sense. I was checking the server. We do have a `stable` symlink, and then a symlink for each minor version pointing to the last patch release of that version (e.g. `1.4 -> 1.4.3`). Those are the ones that would be nice to update automatically here too. The release candidates seem to have a normal docs build in the server, as the rest of versions.

What this is missing is the zipped docs for the prod docs.
TST: Filter/catch pyarrow PerformanceWarnings
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Generally same approach as https://github.com/pandas-dev/pandas/pull/48093


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Arrow
description: pyarrow functionality

Comments:
thx @mroeschke
TST: Wrote test for Float64 conversion #40729
State: closed

- [ ] closes #40729 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Thanks @srotondo
BUG: Series casting large ints to floats in some cases
State: closed

- [x] closes #25880 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


I am not sure that we should fix this at all, but if we want to, we have to keep the ints somehow when converting to an array. Object is one option here

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Stale

Comments:
Does this cause any performance regression?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Still working on this @phofl? 
Is on the agenda, but not enough time right now, closing for now
WEB: standardized footer GH48012 
State: closed

- [ ] closes #48012 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Web
description: pandas website

Comments:
Ups, I ended up updating the PR before I intended.  I wanted to merge the commits into a single commit with a better name :/ Sorry for that
> Ups, I ended up updating the PR before I intended. I wanted to merge the commits into a single commit with a better name :/ Sorry for that

Don't worry about it, we squash all the commits into one when we merge PRs, so it doesn't add much value to squash them before. And in some cases it's useful to have separate commits, so we don't need to review everything again, and can check just what changed.

Checking your changes, I'm wondering if it'd make sense to have the footer content for the website in the configuration yaml. I think we the whole config to the context, so not much change would be needed, except for the year. For it, a simple content preprocessor that replaces a string like `{year}` by the current year could be used. I think this could be a bit better than the javascript. But happy to use the javascript if you have a strong preference for it.

For Sphinx, is Sphinx adding the word `Copyright` automatically? That's annoying, quite a poor design decision in my opinion. If you want to have a quick look and see if there is an easy way to overwrite that, it would be great. But I wouldn't spend too much time on it, Sphinx code is a huge mess.
I opened #48543 to update the footer for the docs, as we want to have it merged before the release on Monday. I'll let you continue here with the update for the website, which is independent from the release.
@datapythonista hi, sorry just wanted to let you know that I started working today and have a lot to do now. If it is something with a deadline please go ahead. Otherwise, I might take a while (like a couple of weeks) :)
No problem, thanks for letting us know. If I have the time to look at this before you, I'll let you know and close this PR. Thanks for the work on this!
Superseded by #48557

Thanks @FrnRmn for the work on this.
Revert #47934
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Thanks @jbrockmendel
CLN: Remove unreachable code in timedeltas.pyx
State: closed

- [x] closes #48196 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~
~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~
~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~


Labels:
name: Timedelta
description: Timedelta data type
name: Clean

Comments:
Thanks @kianelbo
BUG: fixed json_normalize not working with list records
State: closed

When v is a list like object, numpy tries to create a 2D array if v isn't ragged despite the 'dtype=object'. Solution: add an empty list to make the array ragged and then remove.

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Hello @zubin3333! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-08-22 15:55:31 UTC
Thanks for the pull request, but I believe this issue is already close to solved by another user in this pull request https://github.com/pandas-dev/pandas/pull/47708. Closing, but if that pull request does not make it through we can reopen this one. In the meantime, if you'd still like to contribute, we recommend addressing issues labeled `good first issue` and ensure no pull request is actively addressing the issue.
TYP: Autotyping
State: closed

Running autotyping on pandas https://github.com/JelleZijlstra/autotyping

Would be great to add it to pre-commit but before that many abstractmethods need return type annotations (autotyping let's them return `None` or wait for https://github.com/JelleZijlstra/autotyping/issues/33).


closes #46337
xref #47497

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
This should be ready from my perspective @mroeschke Can work on integrating autotyping into the CI after its next release.
Thanks @twoertwein!
BUG: Reindex casting to object in certain situation
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Stumbled across this when doing #48185 
Longterm we could support a mask in take_2d_multi

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
cc @mroeschke gentle ping

Ci is green, forgot to remove 1.5 whatsnew
ENH: consistency of input args for boundaries for resample
State: closed

- [ ] closes #40245
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
ok I close this PR Thank
BUG: reindex using wrong fill value when indexing cols and index for uint dtypes
State: closed

- [x] closes #48184 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Any reason why we are not compiling the cython algo for uint dtypes?


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
cc @mroeschke gentle ping
LGTM. Feel free to merge on greenish
CI: Switch to large for circleci
State: closed

- [ ] ref #48179 (Replace xxxx with the Github issue number)
Just checking what this would change


Labels:
name: CI
description: Continuous Integration

Comments:
Looks like some of the CircleCI builds are completing okay now. Do you think this is still needed?
Some of them is the important part :)

Not all of them were failing before either, but we are right at the 60 minute limit (57, 58, ...). It would definitely improve stability.
Okay maybe good to rebase a few time to confirm this helps the consistency of the CircleCI build
Makes sense, will do that
Large costs 20 credits per minute while medium costs 10, ci minutes are 35 to close to 60, so it is not much more in total, thanks merging for now. Can revert if we run out