BrandonFors/pandas-github-issues huggingface dataset:
BUG: Fix window aggregations to skip over unused elements (GH-45647)
State: closed

- [x] closes #45647
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Window
description: rolling, ewma, expanding

Comments:
add a whatsnew to 1.4.1 in the bug fix section https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v1.4.1.rst
small comment @mroeschke good here?
thanks @rtpsw
Revert "PERF: nancorr pearson (#42761)"
State: closed

- [x] closes #45640
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
thanks @phofl 

I guess we should re-open the *original* issue though that was a bit off point.
I think it was not closer back then, just referenced
BUG: frame.shift(axis=1) with ArrayManager
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: DataFrame
description: DataFrame data structure

Comments:
> Just needs a whatsnew note

should only affect ArrayManager, which isn't public/documented.
still don't reallly like this code path but ok. ping on green.
ping
Switch deps to Conda
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Code Style
description: Code style, linting, code_checks
name: Dependencies
description: Required and optional dependencies

Comments:
is there any discussion on this proposal and wider implications?
It’s a follow up to https://github.com/pandas-dev/pandas/pull/44817
Thanks @jonashaag 
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
TST: Added test for setitem loc using datetime-like str
State: closed

Test to ensure index type conversion when updating existing dataframe
which has datetime as index with datetime-like str using loc

- [x] closes #28249
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Datetime
description: Datetime data dtype

Comments:
Hello @parthi-siva! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-26 11:20:51 UTC
thanks @parthi-siva
BUG: SeriesGroupBy.value_counts - index name missing in categorical columns
State: closed

- [X] closes  #44324
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] whatsnew entry


Labels:
name: Bug
name: Groupby
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Categorical
description: Categorical Data Type
name: Series
description: Series data structure

Comments:
Thanks @NumberPiOso
BUG: NumericArray * td64_array
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
can you rebase
Rebased + greenish
BUG: is_bool_dtype raises AttributeError when checking categorical Series
State: closed

- [X] closes #45615
- [X] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [X] whatsnew entry


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Could you merge master?
also pls merge master
thanks @Robbie-Palmer 
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
DOC: Improve reshaping.rst
State: closed

* Add more sphinx references
* Links function docs back to `reshaping.rst`
* Convert `code-block` to `ipython`


Labels:
name: Docs

Comments:
thanks @mroeschke 

@simonjayhawkins has been backporting these doc changes so ok in the future
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
CI: use pipx and build to simplify and remove a workaround (2)
State: closed

- CI: use pipx and build to simplify and remove a workaround
- ci: drop setup-python, show build version

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

This cleans up the SDist build slightly. First, it uses PyPA's "build" instead of directly calling `setup.py`, which will respect PEP 518 requirements and avoids the workaround of installing NumPy to be able to run setup.py. It also uses pipx to avoid needless setup; PyPA's pipx is provides as a first-class package manager on GitHub Actions (and Azure, they share images).

Repushing #43157. No changes (except rebasing). I think the next step was to make sure the local SDist builds were brought in sync with this.


Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
> I think the next step was to make sure the local SDist builds were brought in sync with this.

probably should do that before making the changes to the ci here in pandas which is just to catch problems early.
Thanks @henryiii for the PR. will close this to clear the queue.

As mentioned above and https://github.com/pandas-dev/pandas/pull/43157#issuecomment-1046224160, the sdist build on ci should replicate the release process to catch issues early so any changes should probably be made to https://github.com/pandas-dev/pandas-release in the first instance.

Given that there is active discussion on using cibuildwheel and pandas could soon be migrating  https://github.com/pandas-dev/pandas/issues/44027#issuecomment-1068625999, I think I am happier not to make any changes to the release process for now.
CLN: Remove Python 2 compat in np_datetime.c
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
nice.  might be worth upstreaming this to numpy too
thanks @lithomas1
CI: Fix failing CI for test_alignment_deprecation_many_inputs
State: closed

xref https://github.com/pandas-dev/pandas/issues/45182

https://github.com/pandas-dev/pandas/pull/45179 was a little too strict since numpy just had a point release.

This should (hopefully) get us to green @jbrockmendel 


Labels:
name: CI
description: Continuous Integration
name: numba
description: numba-accelerated operations

Comments:
LGTM pending greenish
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
DOC: don't suggest melt instead of lookup
State: closed

- [X] closes #45589
- [ ] tests added / passed
- [X] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @adi611! The failures were unrelated 
Thanks for giving me this opportunity.
DOC/WEB: update versions.json to include 1.4 stable docs url
State: closed

Small follow-up on https://github.com/pandas-dev/pandas/pull/45555

Labels:
name: Docs
name: Web
description: pandas website

Comments:
Yeah, including it means we need to update it every time (but that's a small thing to do, I think).

Personally, I find it important that the main url we generally promote (the stable /docs) is accessible from the dropdown.
CI: Fix actions-310 testing 3.9 instead of 3.10
State: closed

- [x] tests added / passed



Labels:
name: CI
description: Continuous Integration

Comments:
test_write_ignoring_index?
LGTM cc @jreback should get us back to green?
> should get us back to green?

Not necessarily. Just an error in our CI configs.
Azure failures are the flaky timeouts
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 573805782558885b04efc59a49ff3090ca783ef5
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45581: CI: Fix actions-310 testing 3.9 instead of 3.10'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45581-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45581 on branch 1.4.x (CI: Fix actions-310 testing 3.9 instead of 3.10)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 573805782558885b04efc59a49ff3090ca783ef5
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #45581: CI: Fix actions-310 testing 3.9 instead of 3.10'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-45581-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #45581 on branch 1.4.x (CI: Fix actions-310 testing 3.9 instead of 3.10)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
REF: Use more context managers to close files
State: closed

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Failure unrelated.
TST: Change test_groupby_copy to apply_mutate.py
State: closed

Pending tasks from #45509
In line with issue #44803

Change test_groupby_copy from `pandas/tests/groupby/test_groupby.py` to `pandas/tests/groupby/test_apply_mutate.py`

- [X] Already closed #44803 via #45509

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Could you merge in main one more time? Otherwise LGTM
Thanks for the follow up @NumberPiOso
CI: Use xsel instead of xclip for numpydev
State: closed

Hopefully fixes this flaky clipboard test

```
_________________________ test_raw_roundtrip[abcd...] __________________________
[gw1] linux -- Python 3.10.0 /usr/share/miniconda/envs/pandas-dev/bin/python

data = 'abcd...'

    @pytest.mark.single
    @pytest.mark.clipboard
    @pytest.mark.parametrize("data", ["\U0001f44d...", "Ωœ∑´...", "abcd..."])
    def test_raw_roundtrip(data):
        # PR #25040 wide unicode wasn't copied correctly on PY3 on windows
        clipboard_set(data)
>       assert data == clipboard_get()
E       AssertionError: assert 'abcd...' == 'Ωœ∑´...'
E         - Ωœ∑´...
E         + abcd...

pandas/tests/io/test_clipboard.py:313: AssertionError
```


Labels:
name: CI
description: Continuous Integration

Comments:
green!! yay
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
DEPR: datetimelike.astype(int_other_than_i8) return requested dtype
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

xref #45034 this should be just non-controversial parts of that discussion cc @jorisvandenbossche 

Labels:
name: Datetime
description: Datetime data dtype
name: Deprecate
description: Functionality to remove in pandas

Comments:
dont' we have an issue about this? e.g. can you add the xref to the top
Thanks!
DEPR: treating float data as wall-times in DTA._from_sequence
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Oops looks like this one slipped through the cracks. Care to merge main one more time?
green
Thanks @jbrockmendel
PERF: TimedeltaArray.astype(object)
State: closed

```
In [2]: tdi = pd.timedelta_range("1 second", periods=10**5, freq="s")

In [5]: %timeit tdi.astype(object)
388 ms ± 43.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- PR
588 ms ± 63.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   # <- main
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Timedelta
description: Timedelta data type

Comments:
worth a note in a followup
Fixes mypy attribute issue in io/parquet by adding a hasattr check
State: closed

- [x] closes #45480, #44914
- [x] Passes tests in pytest
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them



Labels:
name: IO Parquet
description: parquet, feather
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Instead of checking for `io.BufferedWritter`, it might be better to use `io.FileIO` since it has `.name`:
https://github.com/python/typeshed/blob/3d28f0981f470a3e94d819705f8a95b77eaee56d/stdlib/io.pyi#L67

Probably would also make sense to change the if condition to `if isinstance(path_or_handle, io.FileIO) and isintance(path_or_handle.name, (str, bytes))` as `.name` could apparently also be an `int`.
@Anirudhsekar96 great ping on green
great ping on green
The checks have completed running
Thank you @Anirudhsekar96
TST: Update error msg for Python 3.10
State: closed

- [ ] closes #41940
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
thanks @lithomas1
REGR: check_flags not respected in assert_frame_equal
State: closed

- [x] closes #45554
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Also, we should maybe consider #45092 a bug-fix for the flags propagation to the Series object and add some tests for that? I guess this is tested somewhere if the propagation of the flags to the Series when indexing a DataFrame is the right thing to do.
Hm in general I am fine with #45092, since the flags are now propagated because of the added finalize call. Which is what we should expect I think.

I don't think that we have many tests where we check for the flags (might be wrong here), because this is related to the finalize issue, where we check for metadata. If the metadata are propagated, the flag is propagated too.
> Hm in general I am fine with #45092, since the flags are now propagated because of the added finalize call. Which is what we should expect I think.
> 
> I don't think that we have many tests where we check for the flags (might be wrong here), because this is related to the finalize issue, where we check for metadata. If the metadata are propagated, the flag is propagated too.

agree,  yeah we should actcually turn this to true as a default if possible.
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
BUG: align not adding levels only on one side if intersection is equal
State: closed

- [x] closes #45224
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

This is now consistent with the case when both sides do not have the same intersection of indexes

Labels:
name: MultiIndex

Comments:
thanks @phofl
BUG: Segfault in to_json with tzaware datetime
State: closed

- [ ] closes #42130
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Basically, timedelta.total_seconds returns a float. PyInt_AsLong(actually PyLong_AsLong) wants an int, but would cast using __int__ for Pythons < 3.10. This stopped happening in python 3.10. (https://docs.python.org/3/c-api/long.html#c.PyLong_AsLong). 

IMO, it probably is fine to cast the result from float to int, as I don't think that any timezone has an offset from UTC that contains fractions of a second. But, I could be wrong given that I don't use timezones and are not too familiar with them.

Labels:
name: Bug
name: IO JSON
description: read_json, to_json, json_normalize
name: Segfault
description: Non-Recoverable Error

Comments:
Thanks @lithomas1 awesome work
ENH: Experimental Higher Order Methods API
State: closed

Start of #41112, second attempt at #43736.

- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them

Built docs can be viewed in #43736 (wording slightly changed from there, but results are the same). If we go forward with this approach, needs to run on the CI with failures allowed.

cc @jbrockmendel 

Labels:
name: Enhancement
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Might wait until Monday to take a look.  Troubleshooting azure left me cranky.
@jreback @jbrockmendel friendly ping.
WEB: update web/pandas/versions.json for docs dropdown menu
State: closed

I don't think the milestone is relevant for web updates so backport not needed?

Labels:
name: Web
description: pandas website

Comments:
merge when ready 
> merge when ready

does ci check the web build or just upload it?
ahh yes. ci does build something so should wait for that to complete.
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
prob don't need to backport but what-the-heck
> I don't think the milestone is relevant for web updates so backport not needed?

Indeed, web things only are served from master, so never need to be backported.
CI: Azure timeout debug
State: closed

Parallel troubleshooting with https://github.com/pandas-dev/pandas/pull/45507#issuecomment-1018536106


Labels:
name: CI
description: Continuous Integration

Comments:
Possibly a candidate for the hanging behavior: https://github.com/pandas-dev/pandas/pull/45389

Maybe the `os.unlink(fname)` call?
Yeah from the logs maybe the teardown step is hanging?

```
2022-01-22T17:25:12.8017367Z [gw0] PASSED pandas/tests/io/parser/test_unsupported.py::test_close_file_handle_on_invalide_usecols[python] 
2022-01-22T18:34:32.0699589Z ##[error]The operation was canceled.
```
> Yeah from the logs maybe the teardown step is hanging?

yah in the other PR i short-circuited the pyarrow case and it got past this OK, but now looks like its hanging on a totally unrelated offsets test.
@mroeschke still needed?
REGR: changed behavior in series[period].mask(foo, bar, inplace=True)
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

Behavior change in 1.4rc was undocumented.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Period
description: Period data type

Comments:
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
Backport PR #45489 on branch 1.4.x (Bug tojson memleak)
State: closed

Backport PR #45489: Bug tojson memleak

Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
cancel workflow not responding. i'll merge since will be waiting for ci to complete on 1.4.x and performing other checks before starting release.
Backport PR #45519 on branch 1.4.x (EA interface: rename ExtensionArray._hasnans to ._hasna)
State: closed

Backport PR #45519

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
The conflict was from #45362. `_hasnans` was not used in the code removed before `common_dtype_categorical_compat` was added.
Bug: Resample ignoring closed=right for TimedeltaIndex
State: closed

- [x] closes #45414
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Resample
description: resample method

Comments:
Thanks @phofl