BrandonFors/pandas-github-issues huggingface dataset:
DOC: add release note about revert for 1.0.5
State: closed

Whatsnew for https://github.com/pandas-dev/pandas/pull/34632

cc @simonjayhawkins @alimcmaster1 

Labels:
name: Docs

Comments:
LGTM - thanks for doing this @jorisvandenbossche - I haven’t had a chance to!
@alimcmaster1 yes, that's what the whatsnew is supposed to say. But my wording might be confusing?
> @alimcmaster1 yes, that's what the whatsnew is supposed to say. But my wording might be confusing? 

Apols I misread - makes sense!
REF: avoid using DTA/PA methods in Week.apply_index
State: closed

We avoid a couple of array allocations in the process.

```
In [2]: dti = pd.date_range("2016-01-01", periods=10000, freq="S")              
In [3]: off = pd.offsets.Week(1, False, 3)                                      
In [4]: %timeit dti + off                                                       
3.22 ms ± 30.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <-- master
344 µs ± 4.97 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR
```

Labels:
name: Frequency
description: DateOffsets

Comments:
sure
REF: de-duplicate code in liboffsets
State: closed

```
In [2]: dti = pd.date_range("2016-01-01", periods=10000, freq="S")              
In [3]: off = pd.offsets.BMonthEnd(2)                                           
In [4]: %timeit dti + off                                                       
3.36 ms ± 14 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <-- master
998 µs ± 12.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR
```

Labels:
name: Frequency
description: DateOffsets
name: Clean

Comments:
ahh guess merge something else, can you rebase again
rebased+green.  i think just one de-duplication branch after this
BUG: Groupby with as_index=False raises error when type is Category
State: closed

- [x] closes #32599
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This regression was a symptom of `DataFrameGroupBy.size` ignoring `as_index=False`. Fixing this now makes the result either a frame or series, so where the internals use `size()` they now use `grouper.size()` which always returns a series.

I also implemented @dsaxton's suggestion to make the column named "size" when `as_index=False` (https://github.com/pandas-dev/pandas/issues/5755#issuecomment-522321259).

Labels:
name: Groupby
name: Categorical
description: Categorical Data Type

Comments:
thanks @rhshadrach
REF: remove roll_check, use roll_convention
State: closed

Avoid bespoke logic for these two cases.  This will make it feasible to collapse shift_months down to a single case (following #34762 and the one after that that makes get_day_of_month nogil)

Labels:
name: Refactor
description: Internal refactoring of code
name: Datetime
description: Datetime data dtype
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
needs a rebase
rebased+green
typo: pivot_table -> pivot
State: closed

I'm new to pandas and reading the docs for the first time.

I believe that the reference in the *pivot* section should be to `pivot`, not `pivot_table`, to match the code in the example.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Thanks @hartzell - good to have fresh eyes on this!
typo: rows -> columns
State: closed

I'm new to pandas and reading the docs for the first time.  By my reading of the first part of this paragraph, there's a dictionary of lists and the lists are `["Braund...", "Allen...", ...]`, `[22, 35, 58]`, and `["male", "male", "female"]`.

The original sentence says: "... and the values in each list as rows of the DataFrame", but the values in the first list (`["Baund...]`) are the values in the first **column** of the illustration.  Likewise the list `[22,...]` is the second column and `["male",...]` is the third.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
thanks
BUG: DataFrameGroupBy.quantile raises for non-numeric dtypes rather than dropping columns
State: closed

- [x] closes #27892
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Unlike what is mentioned in #27892, this will raise if there are no columns to aggregate. Both mean and median raise with "No numeric types to aggregate" in such a case, so I was thinking perhaps we should be consistent with them. Any thoughts @WillAyd and @TomAugspurger?



Labels:
name: Bug
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
looks like you are including the changes in #34372 here as well
@rhshadrach merging master should fix the CI failure.
@jreback - the response never updated but Travis passed. Responses to your questions are above.
Accidentally deleted PR branch
@jreback Friendly ping. Responses to your questions are above.
thanks @rhshadrach
TST: Period with Timestamp overflow
State: closed

- [x] closes #13346
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

The first part of the test checks that the period inside boundaries does not raise exception when calling `start_time` and `end_time` methods. The second part checks that methods raise exceptions when outside of boundaries.
The parameters are `Timestamp.min` and `Timestamp.max`, so the tests will follow any change of the boundaries.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Period
description: Period data type
name: Stale

Comments:
Hi @OlivierLuG - is this still active? If so, can you address Matt's comments?
I was sure to made answers... I've made a new PR taking into account @mroeschke review.
can u merge master and will look 
@OlivierLuG is this still active?
> @OlivierLuG is this still active?

I'm still here. I'll take a look to this PR on sunday.
can you merge master, i think a lot has changed since this was last rebase (but it still should pass); ping on green.
OK, done and ping on green
Thanks @OlivierLuG
REF: refactor NDFrame.interpolate to avoid dispatching to fillna
State: closed

xref https://github.com/pandas-dev/pandas/pull/31048#issuecomment-643658604, #33959

Labels:
name: Refactor
description: Internal refactoring of code
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
thanks @simonjayhawkins
Bump up minimum numpy version in windows37 job
State: closed

- [x] closes #34724 
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration

Comments:
Can you update the name of the job to reflect the numpy version?

https://github.com/pandas-dev/pandas/blob/a29cee3e0b4f1bf5f91ed7e27ebb3c6d01f47c5e/ci/azure/windows.yml#L16
thanks @Bharat123rox 
> Can you update the name of the job to reflect the numpy version?

we no longer have Travis since this change.
> > Can you update the name of the job to reflect the numpy version?
> 
> we no longer have Travis since this change.

it's back. must be unrelated.
this is an azure job
DOC: updated strings.py for SS06 errors
State: closed


Labels:
name: Docs

Comments:
Hello @willpeppo! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-13 15:13:23 UTC
thanks
CLN: make Info and DataFrameInfo subclasses
State: closed

precursor to #31796

Makes Info class and DataFrameInfo subclass so there's no need for all the
```python
if isinstance(data, ABCDataFrame)
```

Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Clean

Comments:
looks good can you merge master and ping on green.
> looks good can you merge master and ping on green.

Thanks @jreback - green now
Friendly ping :smile: (sorry to do this, but I can submit a PR for `Series.info` if/when this is merged)
thanks @MarcoGorelli very nice
DOC: updated files for SS06 errors
State: closed


Labels:
name: Docs

Comments:
@willpeppo can you fix merge conflict and address comments?
@willpeppo still active?
Yes sorry.  Will get back going on documentation shortly.   I’d like to be an ongoing contributor.  Sorry! 
@willpeppo Thanks for the PR. closing as stale for now. LMK when you are ready to continue and i'll reopen.
BUG: pd.NA in format strings with formatting parameters
State: closed

``pd.NA`` raises if passed to a format string and format parameters are supplied. This is different behaviour than ``np.nan`` and makes converting arrays containing ``pd.NA`` to strings very brittle and annoying.

Examples:

```python
>>> format(pd.NA)
'<NA>'  # master and PR, ok
>>> format(pd.NA, ".1f")
TypeError  # master
'<NA>'  # this PR
>>> format(pd.NA, ">5")
TypeError  # master
' <NA>'  # this PR, tries to behave like a string, then falls back to '<NA>', like np.na
```

The new behaviour mirrors the behaviour of ``np.nan``.

Labels:

Comments:
Some problem, closing temporarily.
PERF: avoid copy in replace
State: closed

Closes #34136.

Hopefully this preserves the right behavior. I could imagine breaking something if a caller was relying on `putmask(., inplace=False)` returning a copy.

```
import pandas as pd
import numpy as np

df = pd.DataFrame({"A": 0, "B": 0}, index=range(4 * 10 ** 7))

# the 1 can be held in self._df.blocks[0], while the inf and -inf cant
%timeit df.replace([np.inf, -np.inf, 1], np.nan, inplace=False)



# 1.0.3
483 ms ± 10.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# master
900 ms ± 18.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

# PR
490 ms ± 8.64 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
thanks
BUG: Fixed regression in PeriodIndex loc
State: closed

Closes https://github.com/pandas-dev/pandas/issues/33964.

This is the smallest change I could get. I'm not sure what a more comprehensive fix would look like.

Regression only on master, so no need for a whatsnew.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Period
description: Period data type

Comments:
thanks @TomAugspurger
PERF: Fixed perf regression in TimedeltaIndex.get_loc
State: closed

Closes https://github.com/pandas-dev/pandas/issues/34510

1.0.3

```python
In [10]: index = pd.timedelta_range(start="1985", periods=1000, freq="D")
    ...: timedelta = index[500]
    ...:
    ...: %timeit index.get_loc(timedelta)

3.7 µs ± 275 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

```

This PR

```
3.25 µs ± 67.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
LGTM
lgtm
Thanks!
BUG: Fixed Series.replace for EA with casting
State: closed

Closes https://github.com/pandas-dev/pandas/issues/34530

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: replace
description: replace method

Comments:
@TomAugspurger 

in the OP

> The code does not work on Pandas Version 1.0.4. It throws an exception. However, it works on Pandas Version 1.0.1, and also with dtype="int64.

is this PR a candidate for backport?
Yeah, I suppose so. Let me update the whatsnew.
OK should be ready once CI passes @simonjayhawkins. I've added to the 1.0.5 milestone.
thanks @TomAugspurger 
@MeeseeksDev backport to 1.0.x
Changed the way a few sentences were written
State: closed

Added comma before 'and'

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
Hey! This is my first PR. The Travis CI failed to build. I don't know what to do now.
> Hey! This is my first PR. The Travis CI failed to build. I don't know what to do now.

this is a known issue, xref #34721
Thanks @ObliviousParadigm 
Thank you! Also, I don't know how else to contribute. Could I please get some pointers on what I can do next? 
A good place to start is probably the good first issues on GitHub:

https://github.com/pandas-dev/pandas/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22

If you see one that interests you (even without that label) typically a good way to start
PERF: interpolate_1d returns function to apply columnwise
State: closed

This doesn't provide a significant improvement for the existing asv due to the bulk of the time creating python sets which is in the function applied columnwise. see #34727

even without #34727 this could provide perf improvements for other index types ( and for unsorted indexes with creating a column-wise function for the numpy call.

will look at adding asvs for these cases.

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
I'll close this to clear the queue for now. Some deeper changes now planned so will reopen once complete.
PERF: remove use of Python sets for interpolate
State: closed

have been investigating avoiding internals for interpolation. xref #34628


This PR address the issue of using Python sets, which is responsible for the bulk of the time in our current asv.

There are other improvements (will raise other PRs), but would need new benchmarks to show a benefit such as different index types and unsorted indexes. so this PR is targeting our current benchmark first.

prelim results (will post asv results if tests pass)
```
N = 10000
# this is the worst case, where every column has NaNs.
df = pd.DataFrame(np.random.randn(N, 100))
df.values[::2] = np.nan
df
%timeit df.interpolate()
# 189 ms ± 24.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  <-- master
# 65.1 ms ± 635 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  <-- PR
```


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
some environments are failing with `TypeError: diff() got an unexpected keyword argument 'prepend'`
looks fine.
do we have sufficient asv's for this?
will get back to this after 1.0.5 we don't have many asvs so could probably add more The current asv is a single float block with a sorted RangeIndex and no limit (which can be fastpathed) so not representative of potentially much slower cases
I'll close this for now. still don't have access to desktop and running asvs on laptop is painful.
DOC: changed 'providing' to 'that provides'
State: closed

Changed this as it sounds more clear.

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
Hey! This is my first commit. The Travis CI failed to build. What do I do?
CLN: remove verbose and private_key reference #34640
State: closed

- [ ] closes #34640 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
Thanks very much for the help, however, there is already an open PR #34654 targeting this issue.

Feel free to look for other issues to fix!
BUG #34621 added nanosecond support to class Period
State: closed

Closes #34621
Closes #17053
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

The class Period do not support nanosecond. I've made a quick and dirty code to support nanoseconds. I have struggled to find an alternative to dateutil.parser, but I didn't found an alternative.

The PR may be a performance issue as I've add a Timestamp constructor to the dateutil.parser. There is for sure a better solution. Please let me know !

Labels:
name: Period
description: Period data type

Comments:
See https://github.com/pandas-dev/pandas/issues/34621#issuecomment-642976243. I don't think this is a bug.

Edit: Misunderstood, confirmed a bug
Hello @OlivierLuG! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-16 18:43:06 UTC
Closes #34621
Closes #17053
this looked ok, pls merge master and address comments, ping on green
Closing in favor of #38175
Removed __div__ impls from Cython
State: closed

Alternate to #34711 which may close #34704

Labels:
name: Build
description: Library building on various platforms

Comments:
I believe failure is unrelated. Looking into it but @jbrockmendel might have some ideas as well given his comment here ref #33858:

https://github.com/pandas-dev/pandas/blob/08febd2b9f77fd8f943de92939788e8bc53ee2d2/pandas/tests/extension/base/dtype.py#L78
Nevermind had the wrong PR. Looks like it _might_ trace back to #22699
hmm what is this needed for?
These throw unused function warnings / errors in newer Cython versions during compilation 

Sent from my iPhone

> On Jun 14, 2020, at 7:36 AM, Jeff Reback <notifications@github.com> wrote:
> 
> ﻿
> hmm what is this needed for?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

thanks
@MeeseeksDev backport to 1.0.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 7240ed459e60c91e706940a0db72228c3c70beeb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #34718: Removed __div__ impls from Cython'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-34718-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #34718 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
xref #34802
DOC: updated pandas/core/series.py for SS06 errors
State: closed


Labels:
name: Docs

Comments:
Hello @willpeppo! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-11 20:13:30 UTC
Thanks @willpeppo
DOC: updated core/indexes/base.py for SS06 errors
State: closed


Labels:
name: Docs

Comments:
Hello @willpeppo! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-06-11 17:12:29 UTC
thanks
no problem @WillAyd ... out of curiousity it said one of the checks failed but I couldn't figure out why... do you know why it's not a problem ?   Thank you.
Make Cython in numpydev config Consistent
State: closed

Follow up to #34711 and just matching the other config files. Not sure generally if we should be specifying here and in higher level configs as may be ambiguous as to what Cython version gets installed, but going with the quick patch for now

Labels:

Comments:
I guess another alternate is to hard code only this to Cython 0.29.19 and leave the higher level config alone (essentially reverting #34711) @TomAugspurger 
Does the CI failure make sense to you?

```
____________________ TestDtype.test_check_dtype[Int64Dtype] ____________________

[gw1] linux -- Python 3.9.0 /home/travis/virtualenv/python3.9-dev/bin/python

self = <pandas.tests.extension.test_integer.TestDtype object at 0x7f7438777400>

data = <IntegerArray>

[   1,    2,    3,    4,    5,    6,    7,    8, <NA>,   10,   11,   12,   13,

   14,   15,   16,   17,...5,   86,   87,   88,   89,   90,   91,

   92,   93,   94,   95,   96,   97, <NA>,   99,  100]

Length: 100, dtype: Int64

    def test_check_dtype(self, data):

        dtype = data.dtype

    

        # check equivalency for using .dtypes

        df = pd.DataFrame(

            {"A": pd.Series(data, dtype=dtype), "B": data, "C": "foo", "D": 1}

        )

    

        # np.dtype('int64') == 'Int64' == 'int64'

        # so can't distinguish

        if dtype.name == "Int64":

            expected = pd.Series([True, True, False, True], index=list("ABCD"))

        else:

            expected = pd.Series([True, True, False, False], index=list("ABCD"))

    

        # FIXME: This should probably be *fixed* not ignored.

        # See libops.scalar_compare

        with warnings.catch_warnings():

            warnings.simplefilter("ignore", DeprecationWarning)

            result = df.dtypes == str(dtype)

    

>       self.assert_series_equal(result, expected)

pandas/tests/extension/base/dtype.py:84: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

pandas/tests/extension/base/base.py:13: in assert_series_equal

    return tm.assert_series_equal(left, right, *args, **kwargs)

pandas/_libs/testing.pyx:68: in pandas._libs.testing.assert_almost_equal

    cpdef assert_almost_equal(a, b,

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   raise_assert_detail(obj, msg, lobj, robj, index_values=index_values)

E   AssertionError: Series are different

E   

E   Series values are different (25.0 %)

E   [index]: [A, B, C, D]

E   [left]:  [True, True, False, False]

E   [right]: [True, True, False, True]
```

I think my preference is to just pin in the environment files and work to fixing the needs for the pins before 1.1 is released.
Surprising to see that at first glance. Just restarted let's see
Looks like that failure is showing up on a few other PRs
#34718 might be better than this for now. Let's see how that looks first