BrandonFors/pandas-github-issues huggingface dataset:
BUG: qcut can fail for highly discontinuous data distributions
State: closed

Fixes #15069. Needs refactoring

- [x] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: cut
description: cut, qcut

Comments:
Hello @puneet29! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-04-02 21:13:47 UTC
Thanks @puneet29 

However, before this PR can be accepted, you need to write tests / make sure you don't break existing ones - see [contributing to the code base](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#contributing-to-the-code-base):

> pandas is serious about testing and strongly encourages contributors to embrace test-driven development (TDD). This development process “relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test.” So, before actually writing any code, you should write your tests. Often the test can be taken from the original GitHub issue. However, it is always worth considering additional use cases and writing corresponding tests.
> 
> Adding tests is one of the most common requests after code is pushed to pandas. Therefore, it is worth getting in the habit of writing tests ahead of time so this is never an issue.

Hey @MarcoGorelli ! Kindly guide me on what to do with the duplicates parameter, because this PR deals with duplicate values in bins. Can you review the code? Thanks a lot.
@puneet29 this needs tests so we know that the bug is fixed.  it will probably go in pandas.tests.reshape.test_qcut
I think this is stale so closing for now, but ping @puneet29 if you'd like to pick back up
Hi, I'm sorry. I will take it up again. I was quite busy before. 😅
No worries - thanks!
Hi @jbrockmendel @MarcoGorelli 
I was having trouble with what output do we expect from discontinuous data. Can you please check and verify if I have implemented it correctly? Thanks! :)
> Kindly guide me on what to do with the duplicates parameter

From what I've understood reading through the original issue, the output should not change if `duplicates` is 'drop' - it's in the case when the `duplicates` keyword hasn't been set that the expected output should change.

Anyway, if you write a test case, then it'll be easier for anyone reviewing to tell how the output has changed.
closing as stale if you want to continue, please open a new PR.
CLN: inline indexing 1-liners
State: closed

The only 2-liners here are _convert_scalar_indexer and _convert_slice_indexer, which are effectively `axis = min(axis, self.ndim-1)`, but that is unnecessary since all places where these are called have axis < self.ndim.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Clean

Comments:
also needs rebase
rebased+green
thanks
TST: add regression test for apply case from GH-31605
State: closed

@jorisvandenbossche you wanted to put more tests in for this issue. I put the user reported code example into a test. Did you have anything else in mind? I didn't merge it with the above one since the above acted on an empty Dataframe.

- [x] closes #31605
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
thanks @fjetter
Backport PR #31607: DOC: combine regressions in section in v1.0.1 whatsnew
State: closed

Backport https://github.com/pandas-dev/pandas/pull/31607

Labels:

Comments:
https://github.com/pandas-dev/pandas/pull/31618 and https://github.com/pandas-dev/pandas/pull/31619 should be merged before this one.
REGR: fix non-reduction apply with tz-aware objects
State: closed

Closes https://github.com/pandas-dev/pandas/issues/31505


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
LGTM pending green
Seems all green (pending travis), thanks for the suggestion @jbrockmendel !
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 881d0b7f88b328f30a57b88a2b0f0f65478d2dc1
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #31614: REGR: fix non-reduction apply with tz-aware objects'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-31614-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #31614 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Hmm, apparently my latest merge of master messed up the whatsnew. Will fix that in a PR to master, and directly in the backport of this PR

EDIT -> https://github.com/pandas-dev/pandas/pull/31686
BUG: Ensure same index is returned for slow and fast path in groupby.apply
State: closed

This _fixes_ the internal check to be consistent with the slow apply path

- [x] closes #31612
- [x] closes #14927
closes #13056 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby

Comments:
Hello @fjetter! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-05-27 12:48:36 UTC
are any of the other issues you referenced closed by this? (aside from the ones you listed)
is this a regression from 0.25.x?
> are any of the other issues you referenced closed by this? (aside from the ones you listed)

The issues I referenced in the issue are already closed. I primarily referenced them to highlight this as a hotspot.

> is this a regression from 0.25.x

I believe this is not a regression and has been around for a while now. I'll check anyhow to confirm.

- [x] check if it a regression
If this is not a regression, let's keep it for 1.1 then, I would say
Agreed.
also pls merge master
I've found that if I disable fast_apply, then there is a test failure in tests.groupby.test_whitelist for `tshift`.  any idea why this doesnt affect that?
can you merge master and ping on green and we can look
sorry, was a bit swamped. Will rebase and address the comments
@jreback I rebased but it seems there is a (probably unrelated) test failure on the azure pipelines with `test_tab_complete_*` tests. Is this a known issue?
> there is a (probably unrelated) test failure on the azure pipelines with test_tab_complete_* tests. Is this a known issue?

It is unrelated, but not yet trouble-shot.
> > there is a (probably unrelated) test failure on the azure pipelines with test_tab_complete_* tests. Is this a known issue?
> 
> It is unrelated, but not yet trouble-shot.

@fjetter #33567 has been fixed. can you merge upstream/master.
I rebased on master and kept the changelog for 1.1.0
ping on green.
Trying to rewrite the changelog. pretty difficult to actually list all tickets affected by this since most users don't go into depths that far.
There is also an umbrella issue which tracks these kind of things which we may be able to close now #13056 another opinion appreciated)
> Trying to rewrite the changelog. pretty difficult to actually list all tickets affected by this since most users don't go into depths that far.
> There is also an umbrella issue which tracks these kind of things which we may be able to close now #13056 another opinion appreciated)

what you wrote is ok, the idea is really to just list the affected tickets so that users can click thru if desired. when we close an issue we want it referenced in the whatsnew and a test to confirm :->
thanks @fjetter very nice!
DOC: combine regressions in section in v1.0.1 whatsnew
State: closed

Similarly how we did it for 0.24.x, gathering all regressions in the first section of the whatsnew (as those are the most important part of 1.0.1)

cc @TomAugspurger 

Labels:
name: Docs

Comments:
Given that there is not much remaining in the "Bug fixes" section, I can probably remove the subtitles there?
> I can probably remove the subtitles there?

Agreed.
This will cause merge conflicts with outstanding PRs. What order do you want to merge these in?
I can fixup conflicts later here, or also help fixing conflicts in the other PRs. Doesn't matter really to me
Let's just merge this since it's already passing.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 a9d24502f50045456fe94abb91b867318fcaf54a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #31607: DOC: combine regressions in section in v1.0.1 whatsnew'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-31607-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #31607 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: add back google analytics with the new doc theme
State: closed

WIth the new theme, we didn't have google analytics anymore (how we had it before https://github.com/pandas-dev/pandas/pull/27662), so did a quick PR to add that to the theme with an option: https://github.com/pandas-dev/pydata-bootstrap-sphinx-theme/pull/84

Labels:
name: Docs

Comments:
cc @TomAugspurger it's a pitty that I forgot this for the 1.0.0 final release, but so can include it for 1.0.1
Thanks!
REF: simplify PeriodIndex.get_loc
State: closed

This makes PeriodEngine follow patterns in DatetimeEngine, so soon we can have it subclass DatetimeEngine instead of Int64Engine, which will lead to more simplifications.

This also puts us within striking distance of sharing get_loc code

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Period
description: Period data type

Comments:
can you rebase
rebased+green
REF: implement tests/indexes/objects/
State: closed

Many of the tests in test_base are about `Index.__new__` cases that return a subclass.  This is intended specifically for Index-only tests.

Labels:
name: Refactor
description: Internal refactoring of code
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Does the "objects" mean "object dtype" or Index "objects" ?
> Does the "objects" mean "object dtype" or Index "objects" ?

Good question, im open to a better name.  The idea here is for tests specific to the base Index class
Maybe something like `base_index` ?
great; test_base is huge!
BUG: read_csv used in file like object RawIOBase is not recognize encoding option
State: closed

- [x] closes #31575
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Is this going in 1.1 or 1.0.1? @jorisvandenbossche added the issue to 1.0.1, and if it's fixing a regression it should go there.

If so, you'll need to move the release note @paihu.
@paihu make sure to merge master you have a conflict
I think this issue is regression bug. so I insert `Fix regressions` section. 
We don't want to wrap text(`TextIOBase`). We need to wrap only binary(`BufferedIOBase` and `RawIOBase`).
Because in that case (eg something like a StringIO object), it is assumed to be already encoded UTF string, so encoding shouldn't be applied then?
Thanks @paihu!
Thanks @paihu!
TST: troubleshoot npdev build
State: closed


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
gentle ping @jreback this fixes the CI
you have an included profile file :(

remove

ping when pushed can just merge 
thanks!
Remember that in general, CI fixes should be backported
@MeeseeksDev backport to 1.0.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 27f365d4d46880613ce1f125090f09e1be69a36a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #31594: TST: troubleshoot npdev build'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-31594-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #31594 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
REF: Move Loc-only methods to Loc
State: closed


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Clean

Comments:
looks good, straight moves?
> straight moves?

in a couple of places i got rid of `o = self.obj` and substituted in `self.obj`
rebased+green
gentle ping; merge conflicts are going to pile up
yep, can you rebase; merge on green.
BUG: Handle object arrays with NaN in cut
State: closed

- [x] closes #31586
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:

Comments:
Hello @dsaxton! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-02-03 17:43:48 UTC
REF: define _convert_to_indexer in Loc
State: closed

_convert_to_indexer is overriden by iloc, so the version currently defined in LocationIndexer is only used in Loc.  This moves it directly to Loc.

Also, since ix has been removed, we have an unnecessary level of the class hierarchy, so this merges NDFrameIndexer and LocationIndexer.

Follow-ups will move base class methods that are actually Loc-specific to Loc. 

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Clean

Comments:
thanks
Dispatch IntervalIndex.argsort to IntervalArray
State: closed

cc @jschendel thoughts on if we can do better in the non-ascending/"quicksort" case?

Labels:
name: Interval
description: Interval data type

Comments:
any timings on the various cases?
lgtm. perf timings?
havent done any timings, am hoping to get guidance from @jschendel on the remaining cases
Closing to clear the queue, will revisit, still hoping @jschendel can weigh in on how best to handle the remaining cases.
CLN: Fix ReadMe Badges
State: closed

Noticed when discussing https://github.com/pandas-dev/pandas/issues/31560 our badges are written in HTML + take up a lot of vertical space on our Readme.

My branch (a lot more compact vs master):
https://github.com/alimcmaster1/pandas/tree/mcmali-readme#pandas-powerful-python-data-analysis-toolkit

I've also added a NumFocus one :) 


Labels:
name: Docs

Comments:
all look good locally? (these are rendered by github so no testing)
> all look good locally? (these are rendered by github so no testing)

Yep I’ve checked everything as expected. You can see how it will render here 

https://github.com/alimcmaster1/pandas/tree/mcmali-readme#pandas-powerful-python-data-analysis-toolkit
thanks!
TYP: Arraylike
State: closed

reopening #31518 

cc @jbrockmendel @WillAyd 

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
makes sense from the part i understand (i.e. not Protocol)

Does it matter that MultiIndex doesnt have `._data`?  (that would be inaccurate in the status quo, not unique to this PR)
> Does it matter that MultiIndex doesnt have `._data`? (that would be inaccurate in the status quo, not unique to this PR)

something we need to watch out for.  
There are many occurrences of isinstance(..., MultiIndex) that would presumably be unnecessary if MutliIndex obeyed Liskov but that's another issue.
At least some of the test failures look related to the exception mesages
> At least some of the test failures look related to the exception mesages

don't really want to be changing these.

from https://docs.python.org/3/library/typing.html

> Changed in version 3.7: Generic no longer has a custom metaclass.

when using Generic, we get test failures on the output of `type()` when using assert, e.g. from `assert isinstance(values, PeriodArray), type(values)`

The test failures are only 3.6, so assume it is due to the metaclass.

from https://www.python.org/dev/peps/pep-0484/#instantiating-generic-classes-and-type-erasure

> Outside the class definition body, a class attribute cannot be assigned, and can only be looked up by accessing it through a class instance that does not have an instance attribute with the same name:

the errors we are getting are presumably from the way assert gets the repr from a class attribute in python 3.6
CI: Remove warning raising after new matplotlib release
State: closed

- [ ] xref #31562   
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

seems CI is broken after the new release of matplotlib

Labels:
name: CI
description: Continuous Integration

Comments:
xref #30588 
Hello @charlesdong1991! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-02-02 16:12:22 UTC
this looks better simpler, i didn't notice this `skip_if_no` decorator, thanks for the tip! @jreback 

since it uses `>=` in `safe_import`, so setting the `min_version` to `3.1.3`, it works locally for me
ping @jreback 
thanks for pro-actively fixing this @charlesdong1991 !
DOC fix *_option() docstring
State: closed

- [x] closes #28780
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Is there any way to see the artefacts of the documentation built?
Do we want a non-regression test for this?
> Is there any way to see the artefacts of the documentation built?

Yes you can build the documentation locally. See https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#building-the-documentation
> Is there any way to see the artefacts of the documentation built?

Yes probably if not too much work
> Yes you can build the documentation locally. See https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#building-the-documentation

I was more referring to the artefacts stored by the CIs. I wanted to not build the documentation locally :)
> Yes probably if not too much work

I assume that you are referring to the unit test. I mentioning this because supposedly, you already have the numpydoc validation which is running (but not catching this one). It might be better to contribute upstream this case than just adding a single brittle test, isn't it?
> > Yes you can build the documentation locally. See https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#building-the-documentation
> 
> I was more referring to the artefacts stored by the CIs. I wanted to not build the documentation locally :)

Gotcha yeah sadly we don't (to my knowledge) create any docs artefacts in azure for PRs.
We easily could though: https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/pipeline-artifacts?view=azure-devops&tabs=yaml. I usually just build locally and show the output in my PR :)
I build locally for the `pd.*_option` and it solves the issue (kudos for being able to build a single function :))
BUG accept and deprecate negative integer for max_colwidth
State: closed

- [x] closes #31532
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Deprecate
description: Functionality to remove in pandas

Comments:
Do you have an entry point in the documentation to add the removal of support for version 1.2, to ease the maintanance on your side?
So from the tests failing it seems `is_nonnegative_int` will handle `None` while `is_int` will not.
It seems a bit surprising at first. @jorisvandenbossche would it make sense that they behave the same? Probably this just an internal function anyway.
ping @jreback it's green :)
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 8dfb54967a935562e3379f999a696fdd1b4ba545
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #31569: BUG accept and deprecate negative integer for max_colwidth'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-31569-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #31569 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
thanks @glemaitre 
@glemaitre if u want to do the manual back port above would be great
I'll do that.
Description of Plotting Backends added in the end.
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
@mehersaipreetam is this still active? Can you fix up merge conflicts and remove the added file?