BrandonFors/pandas-github-issues huggingface dataset:
DEPR: indexing Series with single-entry list
State: closed

- [x] closes #31299 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Could use a better warning message, suggestions welcome.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Deprecate
description: Functionality to remove in pandas

Comments:
> Will users see this warning in addition to the warnings in #31299 (comment)?

No, this will change the list to tuple before calling `ndarray.__getitem__`

> Will need a whatsnew note.

updated
thanks, can you add to the deprecation issue (do this for anything we are deprecating).
BUG: Increased support for subclassed types.
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@EmilianoJordan still active? Can you fix up and get green?
thanks @EmilianoJordan
Ensure conversion to "native" types for integer EA
State: closed

- [x] closes #29738
- [x] tests added / passed
- [x] passes `black pandas`


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Stale

Comments:
One test is failing. see, https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=26857&view=logs&j=bef1c175-2c1b-51ae-044a-2437c76fc339&t=770e7bb1-09f5-5ebf-b63b-578d2906aac9&l=169

I think it is because the series is being converted into int8 dtype somehow. Any thoughts?
Hey @rushabh-v - sorry for only getting round to this now.

> One test is failing. see, https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=26857&view=logs&j=bef1c175-2c1b-51ae-044a-2437c76fc339&t=770e7bb1-09f5-5ebf-b63b-578d2906aac9&l=169

The build no longer exists. I'll just make another couple of suggestions, if you then ping me when you've done them I'll take a look at the failing tests
Also, a whatsnew entry will be required (v1.1.0, I believe)
@MarcoGorelli I have made all the changes. Can you please take a look at the logs now? see https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=28135&view=logs&j=bef1c175-2c1b-51ae-044a-2437c76fc339&t=770e7bb1-09f5-5ebf-b63b-578d2906aac9&l=125
Sure, I'll look at this later this week.

(note to self: to reproduce the error:
```
pytest pandas/tests/extension/test_integer.py
```
)
I think there might be another underlying problem here, which I've raised in #31899. 
So can you review and merge this PR now or we should wait for #31899 to be resolved?
Yes, the tests should be passing before we can merge. I'll look into 31899 today
Current tests fail because of this:
```
>>> import pandas as pd
>>> s1 = pd.Series([100], dtype='Int8')                                                                     
>>> s2 = pd.Series([100], dtype='Int8')
```
So, _without_ the current fix, we have
```
>>> [a+b for (a, b) in zip(s1.values, s2.values)]
[-56]
>>> s1.combine(s2, lambda x, y: x+y)                                                                        
0    -56
dtype: Int64
```

_With_ the current fix:
```
>>> [a+b for (a, b) in zip(s1.values, s2.values)]
[200]
>>> s1.combine(s2, lambda x, y: x+y)                                                                        
0    -56
dtype: Int64
```

cc @jorisvandenbossche 
@rushabh-v is this still active? Can you merge master and try to get green?
still fails
Any updates?
Not from me - I removed the 'good first issue' tag from the original issue as I think there's some underlying issues that need to be solved here first, and they aren't so easy
@rushabh-v can you merge master
closing in favor of #37377
CI: Fix jedi upgrades causes deprecation warning
State: closed

- [ ] closes #31324 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
Assume the warning is raised in the Jedi code? Is there an issue open upstream - so hopefully this gets fixed?
emm, will take a look at jedi and see if a new issue is needed there
@alimcmaster1 

```
- Big **Script API Changes**:
    - The line and column parameters of ``jedi.Script`` are now deprecated
    - ``completions`` deprecated, use ``complete`` instead
    - ``goto_assignments`` deprecated, use ``goto`` instead
    - ``goto_definitions`` deprecated, use ``infer`` instead
    - ``call_signatures`` deprecated, use ``get_signatures`` instead
    - ``usages`` deprecated, use ``get_references`` instead
    - ``jedi.names`` deprecated, use ``jedi.Script(...).get_names()``
- ``BaseDefinition.goto_assignments`` renamed to ``BaseDefinition.goto``
```

Update:
i opened an issue there, and see if can get a quick response and see where issue should be solved, maybe for now we could leave a TODO first and merge this to make CI green
https://github.com/davidhalter/jedi/issues/1481

Update2:
based on explanation from jedi, seems `ipython` uses something which is deprecated in jedi and raises warning in the latest version, so we see a warning here.
thanks @charlesdong1991 

can you create an issue to see what we should do once a new jedi is released (e.g. revert this patch), but maybe we do nothing.
based on jedi explanation, I think `Ipython` needs to react on the deprecation, and I will create an issue here and also in `ipython` later tonight.

Once ipython changes and no longer raises warning, i think we could revert the change then. @jreback
Update testing.py
State: closed

Comment added

Labels:

Comments:
Hello @spchandgude! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/testing.py`](https://github.com/pandas-dev/pandas/blob/c5b3b183b4a8880539ef752097289ae7c373ce69/pandas/testing.py):

> [Line 4:1](https://github.com/pandas-dev/pandas/blob/c5b3b183b4a8880539ef752097289ae7c373ce69/pandas/testing.py#L4): [E265](https://duckduckgo.com/?q=pep8%20E265) block comment should start with '# '


Thanks for your interest in contributing.

I don't think this comment adds much value to this file in particular. If you're interest in contributing, we recommend looking at the issues labeled `good first issue`: https://github.com/pandas-dev/pandas/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
BUG Series rolling count ignores min periods
State: closed

This PR targets 1.0.x branch and is a manual backport for #30923
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:

Comments:
you need to clone the 1.0.x branch and cherry-pick this.
got it, will close this PR
k you could have actually pushed to this branch after you rebased but ok
REF: make PeriodIndex.get_value wrap PeriodIndex.get_loc
State: closed

AFAICT the only behavior this changes is in how datetime objects are treated.  After this they are more consistent.

```
dti = pd.date_range("2016-01-01", periods=3, freq="MS")
pi = dti.to_period("H")
ser = pd.Series(range(7, 10), index=pi)

key = dti[0]
```

master
```
>>> pi.get_loc(key)
0
>>> pi.get_value(ser, key)
KeyError: Timestamp('2016-01-01 00:00:00', freq='MS')
>>> ser.loc[key]
7
>>> ser[key]
KeyError: Timestamp('2016-01-01 00:00:00', freq='MS')
```

Under this PR, the get_value and `__getitem__` calls both return 7.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Period
description: Period data type

Comments:
sgtm.
@jreback gentle ping, plenty more steps to go
yep thanks
@Meeseeksdev backport to 1.0.x

Backporting this, in an attempt to close https://github.com/pandas-dev/pandas/issues/32108.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 5402ea57a6f0fe606a3de3731dcb903186b1f4c2
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #31318: REF: make PeriodIndex.get_value wrap PeriodIndex.get_loc'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-31318-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #31318 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Seems like backporting this will depend on too many other changes (at least PeriodIndex._get_values_for_loc). I won't pursue this any further.
Move DataFrame.info() to live with similar functions
State: closed

#31233

- [x ] closes #31233
(Not sure these are relevant for a doc change?)
- [ ] tests added / passed
- [ ] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Docs

Comments:
lgtm. ping on green.
thanks @jpeacock29
REF: DatetimeIndex.get_value wrap DTI.get_loc
State: closed

After this, DTI.get_value is identical to TDI.get_value.  Separate PR to get PeriodIndex to match too, then de-duplicate.

Labels:
name: Datetime
description: Datetime data dtype
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
great!
Add test for multiindex json
State: closed

- [x] related to #31028 and adds test for #27618 
- [x] tests added / passed
- [x] passes `black pandas`


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: MultiIndex

Comments:
does this close #31028 ?
That is still being discussed there. But @WillAyd mentioned [here](https://github.com/pandas-dev/pandas/issues/31028#issuecomment-574744068) that this test is to be added.
thanks @rushabh-v; this leaves that issue open for discussion then.
POC: use typing.final
State: closed

When working on index/indexing code I frequently find myself checking whether a method is overriden in subclasses.  `@typing.final` marks a method as not-overriden in a way that mypy can check (py38+).  This imports typing.final in py38+ and makes a dummy otherwise.

We'd need to make sure the CI runs mypy in py38.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
cc @simonjayhawkins for opinion on mypy usage, @datapythonista on CI config
> I think regardless of this PR we should use the latest version of python in environment.yml. I think it was pinned to 3.7 when it was the latest, and there is no reason to use an old version.

so change python=3.7 to python=3.8 in environment.yml?
> so change python=3.7 to python=3.8 in environment.yml?

Or probably even better `3.*`, so we simply use the latest available python. 
So its now complaining:

`pandas/_typing.py:28: error: All conditional function variants must have identical signatures`, but looking at the the 3.8 version of final it doesnt have any annotation
Closing to clear the queue
REF: define _get_slice_axis in correct classes
State: closed

Since ix has been removed, a bunch of _NDFrameIndexer methods are defined in now-weird places.  This fixes the one that confused me today.

Labels:
name: Refactor
description: Internal refactoring of code
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
thanks, followons appreciated
CLN: remove _set_subtyp
State: closed

its not used anywhere, apparently legacy leftover

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Clean

Comments:
https://github.com/pandas-dev/pandas/search?q=_subtyp&unscoped_q=_subtyp

@jbrockmendel : Do you intend to remove any remaining ones after this (as a follow-up)?  Or could we tackle all of them in this PR?
sure
> Do you intend to remove any remaining ones after this

You mean legacy methods?  Nothing actively planned, just as I come across them.  Or are you suggesting _subtyp should be removed from _internal_names?
> > Do you intend to remove any remaining ones after this
> 
> You mean legacy methods? Nothing actively planned, just as I come across them. Or are you suggesting _subtyp should be removed from _internal_names?

this actually migtht be doable
PERF: avoid copies if possible in fill_binop
State: closed

This has a small perf bump, but more importantly is necessary for the blockwise frame-with-frame PR that is coming up.

```
import pandas as pd
from pandas.core.ops import *

arr = np.arange(10**6)
df = pd.DataFrame({"A": arr})
ser = df["A"]

%timeit result = df.add(df, fill_value=4)
7.77 ms ± 20.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <-- master
5.46 ms ± 36.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <-- PR

%timeit result = ser.add(ser, fill_value=1)
6.79 ms ± 56.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <-- master
4.65 ms ± 82.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <-- PR
```


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
> Not being super familiar here, is there a way to avoid copy without having to special case for the mask presence?

Not sure I understand the question.  Can you give an example?
> Not sure I understand the question.

@jbrockmendel : I believe @WillAyd is trying to determine if there is a faster way to figure out if we need to `.copy` or not.  I don't think it's a blocker, but if we could, then all the better.

`.any` is still `O(n)`, so the run-time is theoretically unchanged with this PR.
> determine if there is a faster way to figure out if we need to .copy or not

Well, we could pre-check for dtypes that can't hold NAs and short-circuit in those cases.

Other than that, we might be able to do 1 O(n) pass instead of 2-3 if we did it all in cython, but i dont think thats worthwhile at this point 
this is fine. pls add asvs when you can.
REF: pass str_rep through arithmetic ops more consistently
State: closed

This doesn't get all of the places where we fail to pass str_rep, still working out a couple of places where passing it breaks tests.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: expressions
description: pd.eval, query

Comments:
Hello @jbrockmendel! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-02-22 15:50:32 UTC
closing to clear the queue
re-opening, this is a step on the path to doing frame-with-frame ops blockwise
lgtm. rebase ping on green.
rebased+green
thanks
PERF: do DataFrame.op(series, axis=0) blockwise
State: closed

Also fixes the same bug as #31271 (with the same test ported), so if this is accepted that will be closeable.

~2000x speedups for very-wide mixed-dtype cases.

```
arr = np.arange(10**6).reshape(100, -1)    
df = pd.DataFrame(arr)
df["C"] = 1.0
ser = df[0]

In [11]: %timeit df.eq(ser, axis=0)
1.58 s ± 20 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <-- master
595 µs ± 79.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  <-- PR

In [12]: %timeit df.add(ser, axis=0)
2.06 s ± 52.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <-- master                                                                                                                   
1.04 ms ± 5.07 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  # <-- PR

```

Labels:
name: Performance
description: Memory or execution speed performance
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
add a whatsnew in perf section
> add a whatsnew in perf section

done
rebased+green
is the github author thing fixed?
> is the github author thing fixed?

not that im aware of
GH thing is fixed now i think
thanks
CI: Updated version of macos image
State: closed

- [x] closes #31281
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: CI
description: Continuous Integration

Comments:
Will #30990 work after this change?

does ```gcc``` support macos 10.15?
Hmm though looks like the macOS build is stuck in queued
> Hmm though looks like the macOS build is stuck in queued

@WillAyd Not anymore, I guess it was downloading the image or refreshing the cache.
seems reasonable
REF: combine all alignment into _align_method_FRAME
State: closed

This puts us within striking distance of de-duplicating _flex_comp_method_FRAME and _comp_method_FRAME, with _arith_method_FRAME not far behind

Labels:
name: Clean

Comments:
Hello @jbrockmendel! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:









There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-01-24 19:59:03 UTC
thanks
added shuffle method to DataFrame
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

I have added a shuffle method to pd.DataFrame, as writing df.sample(frac=1) is not intuitive. The method just calls df.sample with 1 line of code.
To test it, I have added a test, here is the code:
```# flake8: noqa

"""
Public testing utility functions.
"""
import pandas as pd
from pandas.util.testing import (
    assert_frame_equal, assert_index_equal, assert_series_equal)


df = pd.DataFrame([[1, 2], [3, 4], [5, 6]], columns=['a', 'b'])
shuffled = df.shuffle(random_state=1)
expected = pd.DataFrame([[1, 2], [5, 6], [3, 4]], columns=['a', 'b'])
expected.set_index(pd.Series([0, 2, 1]), inplace=True)
assert_frame_equal(shuffled, expected)
```
This PR doesn't resolve any issue, I just thought a shuffle method would make a nice addition to pandas.
Also, I did not understand what you mean by 'whatsnew entry', could you please clarify?

Labels:
name: Enhancement
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
Thanks for the PR! So our general direction has actually been to reduce our API instead of expanding, so I'm not sure we want to add this but let's see what others think
I could go either way, but right now I prefer better documentation in random_sample.

I'm particularly concerned since both `random.shuffle` and `numpy.random.shuffle` are inplace operations, while this can't be (since the index is immutable and shuffled too)
As to reducing the API size, I don't think this should be a problem. The actual code itself is a one liner, and this method could be really useful for ml, especially for beginners.
@mrinalTheCoder : First off, thanks for your contribution, and welcome to `pandas`! 🚀 

Just so you know, for the future, it would be preferable to open an issue first instead of a PR from the get-go so that we can discuss the relative merits of your proposed enhancement.

In this particular case, I do think that @TomAugspurger brings up a really good point regarding the confusion this could cause.  I would also lean towards improving the docs vs. adding another method.
I agree with the above comments and am -1. This is virtually the same as ``.sample``.

options:
- revert the change and add documentation (an example) in .sample
- close
Ok, got it. Thanks for all you input!
TYP: Type annotations in pandas/io/sas/sas_constants.py
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Oh, so only keep the annotations of ```Dict, List, Tuple```?
That's right
Oh I see.

Closing this then, (sorry for the waste of time)
DOC: move convert_dtypes whatsnew to 1.0
State: closed

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry
   - moved whatsnew entry from 1.1 to 1.0

Per comment here from @jreback: https://github.com/pandas-dev/pandas/pull/30929#issuecomment-577989924


Labels:

Comments:
@Dr-Irv Can you also update the versionadded directive in the docstring?
> @Dr-Irv Can you also update the versionadded directive in the docstring?

done
Thanks!
BUG: 27453 right merge order
State: closed

Resurrection of #27762

I fixed the NameError in the test, and also changed the example given in the whatsnew entry (I couldn't see a difference between the outputs of 0.25.x and 1.0.0 in the original)

- [x] closes #27453
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
lgtm. not sure of the 1.0 milestone at this point but let's see what @TomAugspurger and @jreback think
Is this fixing a regression in 0.25? Otherwise would prefer to keep it for 1.0.1

> On Jan 24, 2020, at 10:51, William Ayd <notifications@github.com> wrote:
> 
> ﻿
> lgtm. not sure of the 1.0 milestone at this point but let's see what @TomAugspurger and @jreback think
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.

@TomAugspurger as in, is it something that was working in 0.25 but isn't working on master? If so, then no, it's not a regression in 0.25.

Should I make a v1.0.1.rst whatsnew file then?
> @TomAugspurger as in, is it something that was working in 0.25 but isn't working on master? If so, then no, it's not a regression in 0.25.
> 
> Should I make a v1.0.1.rst whatsnew file then?

no there is already a PR for that
> Otherwise would prefer to keep it for 1.0.1

@TomAugspurger sure, whatsnew entry added in v1.0.1
I think this should be 1.1 at this point? AFAIK not a regression
> I think this should be 1.1 at this point? AFAIK not a regression

@WillAyd Sure, have moved the whatsnew entry to v1.1.0
ping :)
@jreback thanks - done
thanks @MarcoGorelli for taking this over the line!
BUG: DataFrame.floordiv(ser, axis=0) not matching column-wise bheavior
State: closed

We're pretty close to being able to combine _combine_frame and _combine_match_index (and hopefully get rid of them both before long)

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Clean

Comments:
Is this an outstanding bug or just a test coverage gap? If former probably worth a whatsnew
> Is this an outstanding bug or just a test coverage gap? If former probably worth a whatsnew

Bug, will add whatsnew
great thanks