BrandonFors/pandas-github-issues huggingface dataset:
[WIP] ENH: Groupby mode
State: closed

- [x] closes #19254 
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Proof of concept for #19254 

Is relatively fast compared to @rhshadrach method
```
In [4]: %timeit df.groupby("a").mode()
2.52 ms ± 415 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [5]: %timeit df.groupby("a").mode()
1.99 ms ± 60.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [6]: %timeit gb_mode(df, ['a'], 'b')
3.9 ms ± 310 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [7]: %timeit gb_mode(df, ['a'], 'b')
3.8 ms ± 270 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [8]: %timeit gb_mode(df, ['a'], 'b')
3.89 ms ± 433 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
```

API for what to do when multimodal needs discussion though. 

Labels:
name: Enhancement
name: Groupby
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @lithomas1! Can you include the frame used for the time test? If the values don't have high cardinality (> 500 distinct values?), can you also test with one that does? My expectation is that the cython implementation performs much better the larger the distinct values are.
Dont have time for this.
CLN: assorted cleanups
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
any other issues here?
PERF: is_list_like
State: closed

```
In [3]: arr = np.array(0)
In [4]: arr2 = np.array([0])

In [5]: %timeit is_list_like([])
118 ns ± 2.21 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)   # <-- PR
328 ns ± 3.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)   # <-- master

In [6]: %timeit is_list_like(0)
271 ns ± 5.65 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)   # <-- PR
348 ns ± 4.17 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)   # <-- master

In [7]: %timeit is_list_like(arr)
111 ns ± 1.71 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)   # <-- PR
369 ns ± 24.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)  # <-- master

In [8]: %timeit is_list_like(arr2)
109 ns ± 3.39 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)   # <-- PR
340 ns ± 1.91 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)  # <-- master

```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
nice! ping on green-ish.
ping
nice!
DOC: Add list of UDF methods to mutation gotcha
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Ref: https://github.com/pandas-dev/pandas/pull/39762#discussion_r575832330

Labels:
name: Docs
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
thanks @rhshadrach
solved raise an error on redundant definition (#39823)
State: closed

- [ ] closes #39823
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO CSV
description: read_csv, to_csv

Comments:
Hello @Bhard27! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-02-22 06:08:48 UTC
Sure, I'll work on it. Please excuse the review request, I accidentally clicked on it.
Backport PR #39795 on branch 1.2.x (fix benchmark failure with numpy 1.20+)
State: closed

Backport PR #39795: fix benchmark failure with numpy 1.20+

Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
ImportError: cannot import name 'np_version_under1p20' from 'pandas.compat.numpy'

will fix up shortly
The changes that caused the failure were added in #38379 which is on master, milestoned 1.3 and not on 1.2.x

this backport is not needed, so closing
[ArrayManager] Implement concat with axis=1 (merge/join)
State: closed

The (hopefully) "easy" part of https://github.com/pandas-dev/pandas/pull/39612 for which there seems less discussion: this separates the pieces from https://github.com/pandas-dev/pandas/pull/39612 that are needed to get `concat` with `axis=1` working (`concat_axis=0`), which at least enables the join / merge usecases.

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
small comments, generally looks good
OT should we create a Internal - ArrayManager  label?
The windows failure is also happening on master, and the linux failure is a flaky resource warning, so both unrelated. 

Thanks for the reviews! Going to merge now, and will do the `verify_integrity` rename in a follow-up.
TST: JSON with tz roundtrip
State: closed

test was ignored previously cause `tz` not supported. was fixed by PR #35973

might as well include the test now.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: Timezones
description: Timezone data dtype

Comments:
This passes everything except on the `Linux py38_np_dev` build. Strange error.
thanks @attack68
CI: remove stale assignments
State: closed

Use a third-party GitHub Action to handle the claiming of issues. [See it in action.](https://github.com/afeld/pandas/issues/2#issuecomment-779635007)

It also handles automatically unassigning them after a period of inactivity, which is the more interesting part. The motivation here is that I bet there are a lot of issues where someone has used `take`, but the issue remains open without movement. [By default](https://github.com/JasonEtco/slash-assign-action#options), this Action will remove the assignment after 21 days. Open to suggestions about whether that should be longer or shorter.

- [ ] ~~closes #xxxx~~
- [ ] ~~tests added / passed~~
- [ ] ~~Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them~~
- [ ] ~~whatsnew entry~~


Labels:
name: CI
description: Continuous Integration
name: Stale

Comments:
can we *also* accept `/take`? 
Nice, thanks!

`take` is pretty commonly engrained in contributors' memory - would it be possible to delete the old workflow and replace it with this new one but make it trigger on 'take' instead of '/assign', so that nothing changes from the contributors' side but underneath this new workflow is triggered?
From a pass through the source code of the Action, it _seems_ like `/action` is hard-coded.

https://github.com/JasonEtco/slash-assign-action/blob/842404647dfd10109c8c4feb3e776807eb5e2071/src/lib/comment-handler.ts#L8

Could submit a patch to make the slash command configurable, but based on the framework they are using, removing the leading slash would be harder.

If we feel strongly about sticking with `take`, could have the current Action do the assignment and use the same label as this new Action, and just rely on the new one for the cleanup. Thoughts?
> could have the current Action do the assignment and use the same label as this new Action, and just rely on the new one for the cleanup

As in, if a user comments `take` then the issue will be assigned to them (by the existing workflow), and then the assignment will be removed after 21 days (by the new workflow)? If that's possible, sounds good.
Else, switching to `/assign` is also fine with me
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
@afeld is this ready?
Not sure about this. The project seems like a [one-off](https://github.com/JasonEtco/slash-assign-action/graphs/contributors) development, I wouldn't expect it to be maintained.

And feels like implementing an action to unassign actions should be as easy as our assign action. Not against this, but I'd rather maintain couple of curl calls than someone else's typescript code for this.
Appears like this PR has gone stale and ideally we'd want to just roll out own method for removing stale assignments. Going to close for now but let us know if you want to continue.
TST/REF: collect tests by method
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@WillAyd do you have time to look at the npdev json failure (i think its affecting all PRs ATM)
Can’t dive too deep at the moment but could have some time tonight. If I had to guess the origin it may come from the PyArg_ParseTupleAndKeywords in objToJSON.c since the format string specifies int for the doublePrecision and indent arguments. Haven’t really debugged though

> On Feb 16, 2021, at 11:39 AM, jbrockmendel <notifications@github.com> wrote:
> 
> 
> @WillAyd <https://github.com/WillAyd> do you have time to look at the npdev json failure (i think its affecting all PRs ATM)
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub <https://github.com/pandas-dev/pandas/pull/39832#issuecomment-780071483>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEU4UJBNZHQ32377727YGDS7LCWBANCNFSM4XVXMQCQ>.
> 


@seberg any ideas what in npdev could be causing a `DeprecationWarning: an integer is required (got type float).  Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python` to be issued?  I looked at recently-merged PRs and didn't see anything obvious.

Only occurs with datetime64tz columns
There is one stranger change in the recent dev version that might be related, would have to dig in

If you had a `string + number` or maybe also `string + datetime64` promotion in an array creation, like `np.array(["string", 1])`, that might give you an object array, which in turn might run into this Deprecation warning?  That is a bit around the corner though :). (https://github.com/numpy/numpy/issues/18425 although that one focuses on masked arrays)

I also recently modified that `float + timedelta64` always raises an error during a promotion. That could lead to a similar "to object" behaviour when using `np.array([...])`.
@jbrockmendel have you been able to reproduce locally? I tried with 3.8.5 and the DEV numpy but couldn't get the test to fail
> have you been able to reproduce locally? I tried with 3.8.5 and the DEV numpy but couldn't get the test to fail

are you passing `-W error`?  im getting the warning with np 1.19.5, which actually suggests its something on our end
FYI opened https://github.com/pandas-dev/pandas/issues/39854 for the failure
@jbrockmendel same number tests before / after?
well your after will be higher (as you split some).
> same number tests before / after?

nothing removed, just moved or split
TST/REF: share Series setitem tests
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
presume that this has the same number / coverage before & after?
> presume that this has the same number / coverage before & after?

correct
BUG: PandasArray._from_sequence with list-of-tuples
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
thanks @jbrockmendel 

this didn't look user facing so no whatsnew note?
> this didn't look user facing so no whatsnew note?

thats my thought too
PERF: use arr.size instead of np.prod(arr.shape) in _can_use_numexpr
State: closed

xref https://github.com/pandas-dev/pandas/pull/39772

Using `np.prod` gives quite some overhead, and I *think* it's always equivalent to `.size` ?

```
In [1]: arr = np.random.randn(3, 2)

In [2]: %timeit arr.size
42.8 ns ± 0.628 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [3]: %timeit np.prod(arr.shape)
6.63 µs ± 344 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
> and I think it's always equivalent to .size

AFAIK this is correct

Could grep for this pattern elsewhere?

LGTM
Changed a few more (there are other cases, eg in nanops, but where we only have the shape, and not the original array, so that would require more code changes, and thus out of scope for this PR)
thanks @jorisvandenbossche
REF: remove unnecessary Block attrs
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
can you rebase
rebased + green
REF: implement HybridBlock
State: closed

Methods that would work with any NDArrayBackedExtensionArray, not just DTA/TDA.  Written to facilitate experimenting with Blocks backed by 2D EAs.

Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
pls rebase.

can you make plain this is a NDArrayBackedExtensionArray, maybe (even though more verbose) name the block similar to this.
rebased + renamed + green
TYP: typing refactor mypy fix
State: closed

`pandas/io/formats/style.py:168: error: Incompatible types in assignment (expression has type "Union[DataFrame, Series]", variable has type "DataFrame")  [assignment]`

refactor to remove the `assert` statement otherwise needed to solve the above `mypy` error.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking
name: Styler
description: conditional formatting using DataFrame.style

Comments:
LGTM cc @simonjayhawkins 
thanks @attack68
BUG: add `col<k>` class identifier to blank index name row cells
State: closed

- [x] closes #39716
- [x] tests passed
- [x] linting and typing checks

Labels:
name: Bug
name: Styler
description: conditional formatting using DataFrame.style

Comments:
practically unless a user is setting table styles this doesn't change the display right?
@jreback no this doesn't change user display. 
*Unless* the user has set a table style with some version of `.col` as css selector. In this case that selector will now apply to all cells in the columns *instead of* all cells in the column except one of multiple header rows.
thanks @attack68
DOC: skip evaluation of code in v0.8.0 release notes
State: closed

As noted in https://github.com/pandas-dev/pandas/pull/39364#issuecomment-778678278. A little unclear to me why it's not failing in other pull requests, as I can reproduce the issue locally.

- [ ] ~~closes #xxxx~~
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] ~~whatsnew entry~~


Labels:
name: Bug
name: Docs

Comments:
thanks @afeld
Regression in to_excel when setting duplicate column names
State: closed

- [x] closes #39695
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: IO Excel
description: read_excel, to_excel

Comments:
Changes LGTM.  Restarted CI on failed build since it looks unrelated.
thanks @phofl 
@meeseeksdev backport 1.2.x
CLN: Styler simplify existing builtin methods
State: closed

Now that `Styler.apply` can accept `ndarray` in all cases we can reorganise the module and wrap all the UI facing functions into one:

- highlight null
- highlight max
- highlight min

Labels:
name: Refactor
description: Internal refactoring of code
name: Visualization
description: plotting
name: Styler
description: conditional formatting using DataFrame.style

Comments:
thanks @attack68
DEP: Remove xlrd as being the default reader for xlsx
State: closed

- [x] closes #38884
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

One thing I noticed while working on this is that xlrd is in requirements-dev but not environment.yml; not sure if it should be removed from requirements-dev.

cc @jreback @jorisvandenbossche 

Labels:
name: IO Excel
description: read_excel, to_excel
name: Dependencies
description: Required and optional dependencies

Comments:
dont' we have *any* tests that needed to be changed? (also pls merge master)
> One thing I noticed while working on this is that xlrd is in requirements-dev but not environment.yml; not sure if it should be removed from requirements-dev.

requirements-dev is autogenerated from environment.yml so this would be strange (but it appears to be in both); maybe we should sort `requirements-dev.txt` ?
Can we add a test that openpyxl is actually being dispatched to as the default?
@jreback

> requirements-dev is autogenerated from environment.yml so this would be strange (but it appears to be in both)

Whoops - not sure what I was looking at then :)

@jreback, @WillAyd - tests added. For ExcelFile we can get the engine directly, for pd.read_excel I monkey patched ExcelFile.parser.
very nice @rhshadrach
fix benchmark failure with numpy 1.20+
State: closed

xref https://github.com/pandas-dev/pandas/pull/38379#pullrequestreview-580260176

This will be blocking using numpy 1.20 on Github actions Checks, xref https://github.com/pandas-dev/pandas/pull/36092#issuecomment-771112784

Labels:
name: CI
description: Continuous Integration

Comments:
thanks @simonjayhawkins 
@meeseeksdev backport 1.2.x
changed milestone, backport not needed. see https://github.com/pandas-dev/pandas/pull/39842#issuecomment-779915656
CLN: Better method of determining read-only status of openpyxl worksheet
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

Labels:
name: IO Excel
description: read_excel, to_excel
name: Clean

Comments:
Thanks for following up with openpyxl!

(failure is unrelated resource warning)
Updated description of cut and qcut
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] A colleague of mine complained about the quality of the `cut` description. I took a chance to update the description of `cut` and `qcut`. If there are some suggestions to improve the description - please let me know.


Labels:
name: Docs
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Hello @Ceay1! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2021-02-18 00:02:27 UTC
I believe I have addressed all the suggestions. Please let me know if something else needs to be done on my end.
>  A colleague of mine complained about the quality of the cut description. I took a chance to update the description of cut and qcut. If there are some suggestions to improve the description - please let me know.

would more examples help aside from the updated descriptions? 
can you add a link to wikipedia entries? https://en.wikipedia.org/wiki/Discretization_of_continuous_features or similar might be useful to point to.
@jreback , thank you for the feedback. I added a reference to wiki and dropped series. 

Agree that adding examples would help. There are a couple of open issues on qcut. I would like to first update high-level functions description, then (hopefully) look at some of the open issues, and after that suggest appropriate examples. Hope this plan of starting with the description works with you.
sounds good will look soon
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.
> This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

This pull request is limited in scope to updated descriptions of qcut and cut. All comments on the description of the cut and qcut have been addressed. Further comments (eg wrt additional examples) will be addressed in subsequent pull requests.


> sounds good will look soon

ping @jreback
@Ceay1 can you please open a new PR that  targets master, instead of the branch `1.2.x`, so we can merge? Closing this one.
CLN: Remove "how" return from agg
State: closed

- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them


Labels:
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
thanks! yeah don't even remember the case where this was originally needed.
DOC: Add reference to Text Extensions for Pandas project
State: closed

This PR adds a reference to our Text Extensions for Pandas library, which adds extension types for representing natural language processing data in Pandas DataFrames, to the "Pandas ecosystem" section of the documentation.

Main web page for our project: https://ibm.biz/text-extensions-for-pandas
Source code: https://github.com/CODAIT/text-extensions-for-pandas
API docs: https://text-extensions-for-pandas.readthedocs.io/


Labels:
name: Docs

Comments:
Pushed a second commit to remove some whitespace that was stopping a pre-commit hook.
thanks @frreiss that looks pretty cool. If you folks had any difficulties / suggestions for implementing EA extensions please open an issue.
CLN: remove redundant openpyxl type conversions
State: closed

openpyxl already converts cell values to Python types, preferring
integers where possible. For backwards-compatibility, we have to convert
integers back to float if not `convert_float`

- [ ] closes #xxxx
- [x] tests passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry


Labels:
name: IO Excel
description: read_excel, to_excel
name: Clean

Comments:
thanks @ahawryluk