BrandonFors/pandas-github-issues huggingface dataset:
CI: Merge datamanager job into posix
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature

Avoids an essentially duplicate config


Labels:
name: CI
description: Continuous Integration

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 d9147ebedea3905430471b2a368e4110f196dbd0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46143: CI: Merge datamanager job into posix'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46143-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46143 on branch 1.4.x (CI: Merge datamanager job into posix)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 1.4.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.4.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 d9147ebedea3905430471b2a368e4110f196dbd0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #46143: CI: Merge datamanager job into posix'
```

4. Push to a named branch:

```
git push YOURFORK 1.4.x:auto-backport-of-pr-46143-on-1.4.x
```

5. Create a PR against branch 1.4.x, I would have named this PR:

> "Backport PR #46143 on branch 1.4.x (CI: Merge datamanager job into posix)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@mroeschke if you can put up a manual backport
Sure. I'll wait until all the backport PRs are in 1.4.x just to have a cleaner PR
Bug fix - extension array with 2d datetime64
State: closed

- [x] closes #38390
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
> does this change user visible formatting or just with the internal formatters?
@jreback Hi code changes with the internal formatters

Hi, I am still stuck in the PR. Need more time. 
@jbrockmendel ok here?

@weikhor if you can merge master
will take another look today or tomorrow
Hi It is difficult for me to answer questions provided. I need more time. 
I close this since I stuck this for long time. I get simpler task. Thank for review.
Inconsistent handling of duplicate axes + mixed dtypes in DataFrame.where #25399
State: closed

- [x] closes #25399
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks @mitlabence
CLN: Remove Categorical.itemsize
State: closed

The method doesn't even work in main, it is useless and removal keeps the same error code as before, so I think it's OK to just remove and not do a deprecation cycle here:

```python
>>> cat = pd.Categorical([9, 6])
>>> cat.itemsize
AttributeError: 'Int64Index' object has no attribute 'itemsize'  # main
AttributeError: 'Categorical' object has no attribute 'itemsize'  # new
```



Labels:
name: Clean

Comments:
Looks good after the merge conflict is resolved
Rebased.
Thanks @topper-123
TST: misplaced array tests
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @jbrockmendel
TYP: groupby, sorting
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
can you rebase
Rebased + green
DOC: Fixed issue 46125
State: closed

https://github.com/pandas-dev/pandas/issues/46125

- [x] closes 46125 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Awesome thanks @gunghub!
Welcome! 😄 @mroeschke
Remove some dead code in core/arrays/interval.py
State: closed

The typing work both @twoertwein and I did in #46080 and #44922 found some code that I conjecture is dead code as it doesn't make sense from a typing perspective.

So this PR removes the dead code, which will then allow either of those PR's to get rid of some `#type : ignore` lines.

Not clear to me how to add tests for dead code removal.  In theory, all tests should pass when dead code is removed.



Labels:
name: Interval
description: Interval data type
name: Clean

Comments:
closing per comment above.
Could use a test
DOC: Improve options.rst
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

* Removed some images that now render in ipython in sphinx
* Referenced user guide section in option function docstrings
* Added more sphinx links to pandas functions and objects
* Clarified & simplified language.


Labels:
name: Docs

Comments:
The docstring changes look good.  I dont know enough about the .rst rendering to comment on the unicode files.
The hosted docs have these rendering correctly (and actually looks like the screenshots are incorrect?)

https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html#unicode-formatting
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
DOC: added "How to read these guides" section to User Guide page (#39130)
State: closed

Based on changes by rhuille <raphael.huille@gmail.com> (#39207)

- [ ] closes #39130
- [x] ~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~
- [x] ~All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).~
- [x] ~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~


Labels:
name: Docs

Comments:
Thanks @ehallam!
REF: isinstance(x, int) -> is_integer(x)
State: closed


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Is there user-facing in a way that could represent a bugfix?
> Is there user-facing in a way that could represent a bugfix?

I can't remember why this came up, but there was a reason. It might have been when formatting new Int64 with thousands separator, but the Int64 didn't get detected by `int` so didn't show the separator. This fix was recommended to me in another PR recently.
you could actually test this (e.g. pass in np.int64 intead of ints for some args), but nbd.
@meeseeksdev backport 1.4.x
CI: Align MacOS dependencies with other builds
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature

xref https://github.com/pandas-dev/pandas/issues/29685

MacOS/Windows/Linux will use the same dependency file after this change 


Labels:
name: CI
description: Continuous Integration

Comments:
are the CI failures here real?
> are the CI failures here real?

The MacOS ones, yes (pyarrow is tricky apparently). I have no idea why the GHA Linux ones are failing though since the changes are totally unrelated.  
@meeseeksdev backport 1.4.x
Something went wrong ... Please have a look at my logs.
DOC: Add note for tail/head when n is larger then length of DataFrame
State: closed

- [ ] closes #46079 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Stale

Comments:
Hello @benjamrio! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/generic.py`](https://github.com/pandas-dev/pandas/blob/7a105ce74f4a07096abe83a654a2b262ef2d7208/pandas/core/generic.py):

> [Line 5165:80](https://github.com/pandas-dev/pandas/blob/7a105ce74f4a07096abe83a654a2b262ef2d7208/pandas/core/generic.py#L5165): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 5241:80](https://github.com/pandas-dev/pandas/blob/7a105ce74f4a07096abe83a654a2b262ef2d7208/pandas/core/generic.py#L5241): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace


This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but this was addressed by https://github.com/pandas-dev/pandas/pull/46466. Closing.
Update generic.py
State: closed

Added documentation on head and tail methods. Parameter n can be larger than the object size.

- [x] closes #46073


Labels:

Comments:
Hello @benjamrio! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/generic.py`](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py):

> [Line 5164:1](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5164): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 5165:89](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5165): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (97 > 88 characters)
> [Line 5170:38](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5170): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 5240:89](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5240): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (97 > 88 characters)
> [Line 5241:1](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5241): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace
> [Line 5245:38](https://github.com/pandas-dev/pandas/blob/f31746d4726174cee9a7362f6a76c93b40c6e0ed/pandas/core/generic.py#L5245): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
BUG: PandasArray._quantile when empty
State: closed

Not user-facing, but still good to have fixed.

Gets rid of our last non-idiomatic usage of _from_factorized, xref #33276

Labels:
name: Bug
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Yes
Not seeing any tests for quantile in arrays, am I missing it, or might they be somewhere else?
Tested indirectly in the dataframe/series tests
gentle ping; a couple of follow-ups in the works
nice!
PERF: avoid unnecessary copies factorize
State: closed

Reached via yak-shaving on #33276.

The core.algorithms change should only affect non-64bit cases.  The Categorical change could help across the board.

```
import numpy as np
import pandas as pd

arr = np.arange(10**5, dtype="uint32")

%timeit pd.factorize(arr)
2.27 ms ± 67.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
1.08 ms ± 28.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR

cat = pd.Categorical(np.arange(10**5))
%timeit cat.factorize()
2.67 ms ± 115 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- main
1.06 ms ± 37.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR

cat2 = pd.Categorical(["A", "B", "C"] * 1000)
%timeit cat2.factorize()
108 µs ± 11.3 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <- main
30.9 µs ± 919 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)  # <- PR
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
> Might be good to follow up with a whatsnew note

sure.  will add that to the next "assorted" branch
TST: Parameterize tests
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @mroeschke
PERF: support mask in group_last
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref #37493

We have existing tests with MaskedArrays for this.

Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
```
ser = pd.Series(range(10**5), dtype="Int64")
df = pd.DataFrame({"A": 1, "B": ser})
gb = df.groupby("A")

%timeit gb.last()
726 µs ± 26 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- main
527 µs ± 3.88 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <- PR
```

> Also the xref link doesn't seem to exist.

Fixed.
sure assume first is an easy followup
REF/TST: misplaced hashtable tests, rename HashTable.map map_keys_to_values
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Not sure if these errors are related to this PR or not, but seeing several CI build failures on the test below:

Examples that have failed this test:
#46163
#45575

```
def test_vector_resize(
        self, writable, htable, uniques, dtype, safely_resizes, nvals
    ):
        # Test for memory errors after internal vector
        # reallocations (GH 71[57](https://github.com/pandas-dev/pandas/runs/5348598022?check_suite_focus=true#step:11:57))
        vals = np.array(np.random.randn(1000), dtype=dtype)
    
        # GH 21[68](https://github.com/pandas-dev/pandas/runs/5348598022?check_suite_focus=true#step:11:68)8 ensures we can deal with read-only memory views
        vals.setflags(write=writable)
    
        # initialise instances; cannot initialise in parametrization,
        # as otherwise external views would be held on the array (which is
        # one of the things this test is checking)
        htable = htable()
        uniques = uniques()
    
        # get_labels may append to uniques
        htable.get_labels(vals[:nvals], uniques, 0, -1)
        # to_array() sets an external_view_exists flag on uniques.
        tmp = uniques.to_array()
        oldshape = tmp.shape
    
        # subsequent get_labels() calls can no longer append to it
        # (except for StringHashTables + ObjectVector)
        if safely_resizes:
            htable.get_labels(vals, uniques, 0, -1)
        else:
            with pytest.raises(ValueError, match="external reference.*"):
>               htable.get_labels(vals, uniques, 0, -1)
E               Failed: DID NOT RAISE <class 'ValueError'>
```
@lukemanley possible. have merge quite a lot of things. happy to take a patch if this is still happening on master.
ENH: allow concat of Styler objects
State: closed

  - [x] closes #43875
  - [x] closes #43894 


![Screenshot 2022-02-23 at 21 12 16](https://user-images.githubusercontent.com/24256554/155400528-fd512767-c303-4724-8c59-ab6fb6169cdc.png)
![Screenshot 2022-02-23 at 21 12 23](https://user-images.githubusercontent.com/24256554/155400535-c980193d-78a7-4147-92b9-1ac2cf403f08.png)



Labels:
name: Enhancement
name: Styler
description: conditional formatting using DataFrame.style

Comments:
Not sure if this assessment is correct, but this would this be a generic version of `pd.pivot_table` + styling? 
https://pandas.pydata.org/docs/reference/api/pandas.pivot_table.html
> only a very brief look but the same problem as the other PR this is commingling calculation and display

yes, ok i now see your point. how about I change tack and require a user to pass an already calculated dataframe or Styler object which will be appended to the footer? This would also get around any need to format or style the footer since it would inherit from the passed Styler object?
@jreback I completely reworked this and managed to get the entire implementation under 20 lines execution code. All calculations are done by the user, they just pass a second styler object.
DOC: Remove old note about default unit
State: closed

`to_datetime` docstring indicates that the default unit is 'ms', but it is 'ns'. This PR simply removes that comment so that there is no need to keep it in sync with the implementation.

- [ ] ~closes #xxxx (Replace xxxx with the Github issue number)~ NA
- [ ] ~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~ NA
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~ NA


Labels:
name: Docs
name: Datetime
description: Datetime data dtype

Comments:
thanks @gsganden
Typeinterval part2
State: closed

Is on top of https://github.com/pandas-dev/pandas/pull/46080

I believe the typing for `Interval` here is correct as it passes the tests created here:
https://github.com/microsoft/python-type-stubs/pull/167/

Leaving this as draft status as it requires #46080 to be approved.

Or a decision to go with #44922, in which case we kill #46080 and this one.



Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Looks pretty good @Dr-Irv! I'll look at part 1 tomorrow.
> Looks pretty good @Dr-Irv! I'll look at part 1 tomorrow.

Thanks!
other PR merged so pls rebase
> other PR merged so pls rebase

all set and all green
thanks @Dr-Irv keep em coming!
Bug/45740 stack_multi_columns extension array downcast
State: closed

- [x] closes #45740 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Stale

Comments:
Hello @ladidadadum! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-02-26 14:11:40 UTC
can you merge master and update to comments
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge the main branch, address the review comments, and we can reopen.