BrandonFors/pandas-github-issues huggingface dataset:
CLN: assorted cleanups
State: closed


Labels:
name: Clean

Comments:
@datapythonista is there an easy way to run the docstring validation on non-public/non-API functions?  here im thinking of tslibs.ccalendar

(side-note, does the docstring validation check that type annotations match docstring parameters and return types?)
The script runs on all public functions/methods when called without parameters, but it can run on private functions if you pass it as parameter (e.g. `./scripts/validate_docstrings.py pandas.Series._private_method`).

We don't check type annotations at the moment, but I guess shouldn't be difficult to implement it. The script now lives in numpydoc, I'm already working on replacing our version by the one in numpydoc.
@WillAyd @mroeschke this is pretty spread out.  LMK if there are pieces I can split out that you'd be comfortable merging.
looks fine, happy to merge now and follow up with those 2 issues or repush, lmk.
thanks @jbrockmendel
Backports for 0.25.3
State: closed

This includes pieces of #27826 except for the whatsnew note, which appeared in 0.25.1. Not sure if we want to move that here or keep as is, but test and implementation were included here

Labels:
name: Docs
name: Groupby

Comments:
@pandas-dev/pandas-core any outstanding thoughts on this before moving forward with release?
lgtm.
CLN: Consistent pandas.util.testing imports in remaining test suite
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Fyi: When the remaining PRs of this are merged, I'll add another one to add the linting rule - then we can close the issue

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
thanks @SaturnFromTitan 

when the last of the imports are done, can you prepare a change to fail the CI on incorrect imports. this go in the ci/code_checks.sh script, see how we exclude numpy.testing (but maybe need to be a bit more sophistacted here).
TST: Test DataFrame rename with ChainMap
State: closed

Closes https://github.com/pandas-dev/pandas/issues/23859

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: DataFrame
description: DataFrame data structure

Comments:
Thanks!
CLN: tighten Exception catching in indexing.py
State: closed

get rid of is_lexsorted_for_tuple while we're at it

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Clean

Comments:
Are exceptions simply being handled more specifically now?

Or are they be handled higher up in the stack now?
Echoing @gfyoung's question, it would be helpful if you add some explanation why you are removing / changing things in your PRs.
lgtm. 
> Are exceptions simply being handled more specifically now?  Or are they be handled higher up in the stack now?

They are no longer reached.  AFAICT this is leftover from Panel indexing
can you add a whatsnew note, technically this *was* public api; ping on green.
ping
Fix SS06 formatting errors
State: closed

Fixing more SS06 issues, mentioned in #29254 

Fixed docstrings for the following:
- pandas.DataFrame
- pandas.DataFrame.transform
- pandas.DataFrame.corrwith
- pandas.DataFrame.describe
- pandas.DataFrame.kurt
- pandas.DataFrame.kurtosis
- pandas.DataFrame.skew
- pandas.DataFrame.align
- pandas.DataFrame.drop_duplicates
- pandas.DataFrame.duplicated

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Hello @deepandas11! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-31 07:37:34 UTC
Repeating this comment in this PR, since this cause conflicts if one gets merged first?

> Hi @deepandas11, looks like we edited the same issues in two different pull requests, but it looks like yours passed checks. Moving forward maybe we can have people claim the issues so we can reduce redundant work?

https://github.com/pandas-dev/pandas/issues/29254#issuecomment-548193511
I think this was already covered by the other PR
CLN: Consistent pandas.util.testing imports in pandas/tests/frame - part 4
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks!
CLN: Consistent pandas.util.testing imports in pandas/tests/frame - part 3
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
thanks @SaturnFromTitan
CLN: Consistent pandas.util.testing imports in pandas/tests/frame - part 2
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:

Comments:
Thanks!
CLN: Consistent pandas.util.testing imports in pandas/tests/frame - part1
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:

Comments:
Thanks!
CLN: Consistent pandas.util.testing imports in pandas/tests/io
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @SaturnFromTitan !
0.25.3 whatsnew
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
did we push the rolling patch to the 0.25.x branch?
No I was thinking merge this to master than I can backport this with the fix, though if you prefer another way lmk
no that’s fine

didn’t know if u did the backport yet
You need the v0.25.3 tag for that to work. Using HEAD is the best we can do
till the release.

On Wed, Oct 30, 2019 at 2:41 PM William Ayd <notifications@github.com>
wrote:

> *@WillAyd* commented on this pull request.
> ------------------------------
>
> In doc/source/whatsnew/v0.25.3.rst
> <https://github.com/pandas-dev/pandas/pull/29296#discussion_r340821184>:
>
> > +including other versions of pandas.
> +
> +.. _whatsnew_0253.bug_fixes:
> +
> +Bug fixes
> +~~~~~~~~~
> +
> +Groupby/resample/rolling
> +^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +- Bug in :meth:`DataFrameGroupBy.quantile` where NA values in the grouping could cause segfaults or incorrect results (:issue:`28882`)
> +
> +Contributors
> +~~~~~~~~~~~~
> +
> +.. contributors:: v0.25.2..v0.25.3
>
> @jgehrcke <https://github.com/jgehrcke> @TomAugspurger
> <https://github.com/TomAugspurger> would either of you happen to know how
> to get this to work? Do I need to pin this to HEAD before making the
> release then update the whatsnew after tagging 0.25.3?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/29296?email_source=notifications&email_token=AAKAOIRL3D52PP7ZTRDU6C3QRHPM5A5CNFSM4JG6SLL2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJZCZBY#pullrequestreview-309472391>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKAOIXE3CZAH7F3BWRRWKLQRHPM5ANCNFSM4JG6SLLQ>
> .
>
Fix SS06 formatting errors 
State: closed

More SS06 (Summary should fit in a single line) errors fixed (#29254):

The following docstrings were examined and fixed:
```
pandas.DataFrame
pandas.DataFrame.transform
pandas.DataFrame.corrwith
pandas.DataFrame.describe
pandas.DataFrame.kurt
pandas.DataFrame.kurtosis
pandas.DataFrame.skew
pandas.DataFrame.align
pandas.DataFrame.drop_duplicates
pandas.DataFrame.duplicated

```

Validated with python scripts/validate_docstrings.py

Also referencing issue: #27977

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Hello @profwacko! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-30 16:57:01 UTC
Thanks @profwacko
CLN: Consistent pandas.util.testing imports in pandas/tests/indexing
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
@SaturnFromTitan can you merge master and push again on this and your other PRs? CI failure unrelated and should have been fixed on master
@WillAyd Thanks for letting me know. The PRs are rebased now and all checks passed ✅ 
Thanks @SaturnFromTitan !
CLN: Consistent pandas.util.testing imports in pandas/tests/indexes
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @SaturnFromTitan !
CLN: Consistent pandas.util.testing imports in pandas/tests/groupby
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Little bit on the larger side, but went through, and all looks good.

Thanks @SaturnFromTitan !
CLN: Consistent pandas.util.testing imports in pandas/tests/resample
State: closed

Part of #29272

Fyi: the changes were created via a script posted in the issue

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @SaturnFromTitan !
CLN: Consistent pandas.util.testing imports in pandas/tests/reshape
State: closed

Part of #29272

Fyi: the changes were created via a script posted in the issue

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @SaturnFromTitan !
DOC: Fix SS06 formatting errors in Styler docstrings
State: closed

Errors fixed (from the list in #29254 ):
pandas.io.formats.style.Styler.from_custom_template
pandas.io.formats.style.Styler.apply
pandas.io.formats.style.Styler.applymap
pandas.io.formats.style.Styler.set_properties
pandas.io.formats.style.Styler.clear
pandas.io.formats.style.Styler.background_gradient
pandas.io.formats.style.Styler.use
pandas.io.formats.style.Styler.where

Validated the fixes with ```python scripts/validate_docstrings.py```
Refrencing issue: #29254 

Labels:
name: Docs

Comments:
@WillAyd Ty so much for the corrections. Not taken for granted the fact you literally marked the problems, Instead of just saying that there's grammer errors.
CLN: Consistent pandas.util.testing imports in pandas/tests/util
State: closed

Part of #29272 

Fyi: All changes are generated from a script posted in the issue.

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks
CLN: Consistent pandas.util.testing imports in pandas/tests/series/indexing
State: closed

Part of #29272

Fyi: the changes were created via a script posted in the issue

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks @SaturnFromTitan 

these are good size for these pieces.
CLN: update imports in tests
State: closed

Medium-term goal is to separate out tslibs-only parts of these tests to make sure we have sufficient coverage in isolation.

Labels:
name: Clean

Comments:
this is fine
> This is getting into personal development preferences but as a counter argument I sometimes find these imports very annoying. When debugging the pd namespace is almost always available, yet when copy pasting blocks of code that use this import machinery you then have to go up and see what is imported from top of the module.

+1 on what @WillAyd said here. 
My personal preference is also to use `pd.` in tests when we are testing public things.
thanks @jbrockmendel 
> I think we as a team have some disagreements over import styles but OK to move forward with this for now and leave that resolution to a larger discussion

Yep, but maybe not do *more* of this PRs before having such discussion then?
CLN: Consistent pandas.util.testing imports in pandas/tests/series
State: closed

Part of #29272

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
lgtm
Merge conflicts are going to pile up if we dont merge this promptly, merging.
Thanks @SaturnFromTitan
Add simple test for GH 28448
State: closed

- [x] closes #28448
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
Won't have a chance to look at this anytime soon, so don't wait for me.

On Wed, Oct 30, 2019 at 12:12 PM William Ayd <notifications@github.com>
wrote:

> *@WillAyd* approved this pull request.
>
> lgtm @TomAugspurger <https://github.com/TomAugspurger> or @jbrockmendel
> <https://github.com/jbrockmendel> feel free to merge if it looks good to
> you as well
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/29269?email_source=notifications&email_token=AAKAOISNDNBFE2PODAJQIXLQRG6AJA5CNFSM4JGLPMHKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJYLSNY#pullrequestreview-309377335>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAKAOIWIA6W6KI2GKWEN4WDQRG6AJANCNFSM4JGLPMHA>
> .
>

small comment, not a blocker.  lgtm
thanks @MarcoGorelli
BLD: ABCSeries mapper maps unmapped categories to "" instead of NaN
State: closed

- [x] closes #29162
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry



Labels:
name: Categorical
description: Categorical Data Type

Comments:
I think this is going to break a lot of things and maybe not the direction we want. Did you see @jorisvandenbossche comment in the original post?
Yeah, just saw it. I'll close this pr and think about another approach to resolve the issue
Fix SS06 formatting errors
State: closed

Errors resolved (from list in #29254):
pandas.DataFrame.filter
pandas.DataFrame.first
pandas.DataFrame.idxmax
pandas.DataFrame.idxmin
pandas.DataFrame.last
pandas.DataFrame.reindex
pandas.DataFrame.pivot_table
pandas.DataFrame.reorder_levels
pandas.DataFrame.unstack
pandas.DataFrame.melt

Validated with `python scripts/validate_docstrings.py`

Referencing issue: #27977


Labels:
name: Docs

Comments:
Hello @mcalingo! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-29 09:34:47 UTC
Looks like `filter` has some lint complaints
Thanks @mcalingo
TST: Test named aggregations with functions
State: closed

Closes https://github.com/pandas-dev/pandas/issues/28467


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Hello @gfyoung! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-29 06:55:20 UTC
thanks @gfyoung
REF: Pre-empt TypeError in groupby
State: closed

cc @WillAyd @jreback, should be orthogonal to #29228.

Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Was this `TypeError` ever tested?  Is it possible to cover in the tests?
> Was this TypeError ever tested? Is it possible to cover in the tests?

The `pass` on L225 is currently reached in the tests.  The `raise` on L227 is not.
k looks fine (hopefully you have followons!). rebase and ping on green.
ping
Thanks @jbrockmendel
Multiindex recurse error fix
State: closed

test and 3 line fix for a small bug mentioned in 2 issues (1 closed). test might be expanded, I only checked for the error disappearing

- [x] closes #25760 (and #28956 )
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: MultiIndex

Comments:
Hmm it doesn't appear a consensus was ever made on this, but the suggestion was to raise. Why do you think this is better than raising?
there are join operations that modify the order of MultiIndex names, like

```python
x=pd.DataFrame(np.arange(4),pd.MultiIndex.from_product([[1,2],[3,4]],names=['a','b']))
x2=pd.DataFrame(np.arange(8),pd.MultiIndex.from_product([[1,2],[5,6],[3,4]],names=['a','c','b']))
x2.join(x, lsuffix='x1')
```

so I thought it is not such a definitive feature and I couldn't think of a use case where it might be a problem

also, this change doesn't break any of the tests, so there seems to be no agreement on the other side either

anyway, raising a recursion error is definitely not an intended effect, changing it to a more informative error can also be a way to go. I just feel that it is more in line with the general way of python (and pandas) for the api to be more practical, convenient and flexible than restrictive 

can you merge master
thanks @endremborza
DOC: use new logo in the sphinx theme as well
State: closed

With this change (https://github.com/pandas-dev/pandas-sphinx-theme/commit/84134e0804325538918324246f449e694466929b) in the theme, that should work.

However, locally, I am running into the problem that sphinx/html doesn't seem to like the svg logo's .. For local testing I added a converted png which works, but just replacing `.png` with `.svg` makes that the html does not include the image (at least in firefox).

cc @datapythonista 


Labels:
name: Docs

Comments:
OK, that issue could be fixed with css in the theme (apparently need to explicitly set the height for the img tag in the navbar to force the scaling)
Are we going to keep the docs independent from the web? My preference is to use the template of the web to build the docs, and I guess this is not needed if we do that.
Well, we have https://github.com/pandas-dev/pandas/issues/28580 for that discussion. In my last comment there, I put some examples of other doc sites / websites. Interested to hear your thoughts about that. 
Let's continue that discussion there.

> I guess this is not needed if we do that.

As long as we use the sphinx theme, this is needed (so since such integration won't happen today, going to merge this to have the new logo on the dev docs short term). But this relative path at least ensures to not duplicate the actual logo source. 
But even if we re-use the layouts of the web, I am not fully sure that sphinx can work with layouts/images outside of its directory (eg with this PR, sphinx copies the image to `_static` in the build directory)
Timedelta index 29236
State: closed

- [ ] closes #29236 

only remaining docstring errors are for `TimedeltaIndex.mean`

    Parameters {*args, **kwargs} not documented
    Unknown parameters {skipna}




Labels:
name: Docs

Comments:
Hello @HughKelley! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-10-28 22:18:00 UTC
@HughKelley can you address linting issues
@WillAyd what is the 1.0 milestone used for?
> @WillAyd what is the 1.0 milestone used for?

We use this for tracking PRs making their way into our next release
Thanks @HughKelley !
Fix SS06 formatting errors
State: closed

SS06 (Summary should fit in a single line) errors fixed (#29254) for the following:

```
 pandas.DataFrame.slice_shift
 pandas.DataFrame.to_period
 pandas.DataFrame.plot
 pandas.DataFrame.sparse.density
 pandas.DataFrame.style
 pandas.Series.as_blocks
 pandas.Series.ix
 pandas.Series.ptp
 pandas.DataFrame.as_blocks
 pandas.DataFrame.ix
```
Validated with `python scripts/validate_docstrings.py`

Also referencing issue: #27977

Labels:
name: Docs

Comments:
Hi @WillAyd, added the changes you suggested.
Great thanks @profwacko !