BrandonFors/pandas-github-issues huggingface dataset:
BLD: bump numpy min version to 1.15.4
State: closed

- [x] closes #33718
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
Mothballing to clear the queue
@WillAyd the MacOSPy36 failure is at build-time, any thoughts?

@TomAugspurger the LinuxPY36 failure looks like its a conda-dependency issue, any thoughts?

One of the other two failures should be fixed by #33898 leaving one more that i need to troubleshoot.
Seems to be an issue between NumPy 1.15.4 and python=3.6.1

```
 ~  conda create -n ttttt -c defaults -c conda-forge numpy=1.15.4 python=3.6.1                                                                                                                                                                                                                                                           10.9s  Thu Apr 30 16:20:32 2020
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
python=3.6.1
numpy=1.15.4 -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
numpy=1.15.4 -> mkl_fft[version='>=1.0.6,<2.0a0'] -> python[version='2.7.*|3.5.*|3.6.*|>=3.5,<3.6.0a0|>=3.8,<3.9.0a0']
Note that strict channel priority may have removed packages required for satisfiability.
```

Python 3.6.2 seems fine. I'd recommend removing the `defaults` channel from that build.
Haven't debugged macOS failure but one thing worth updating first perhaps is the dependency file for it. I think it still has numpy 1.14 pinned

https://github.com/pandas-dev/pandas/blob/1940fcea056325cbe878c72078c51f80ea8482f4/ci/deps/azure-macos-36.yaml#L22
> Python 3.6.2 seems fine. I'd recommend removing the defaults channel from that build.

first couple tries didnt work, any other suggestions?  no rush
If you're able to reproduce locally, then binary search on the env file is often best (comment out half the file, `conda env create -f ...`, see if it breaks).

Turns out scipy=0.19.0 was the (or at least a) problem. It's not compatible with NumPy 1.15.4.

I'd recommend bumping scipy to 1.2.0, which is the first release after NumPy 1.15.4.
Can you bump the versions in `pandas/compat/_optional.py` for scipy and any other optional deps that are increased?
> I'd recommend bumping scipy to 1.2.0, which is the first release after NumPy 1.15.4.

that did the trick, thanks
great!
TST: pd.concat on two (or more) series produces all-NaN dataframe
State: closed

- [ ] closes #11058
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
thanks
REF: avoid passing SingleBlockManager to Series
State: closed

cc @jorisvandenbossche IIRC you didnt want to disallow SingleBlockManager because geopandas passes it.  In those cases, does it also pass `fastpath=True`?  If so, we can consider deprecating allowing SingleBlockManager in the non-fastpath case

Labels:
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
I am not fully sure anymore about the context where I would have said that (do you remember the issue/PR?) 
In any case, geopandas handles the case where the `data` passed to the constructor is a SingleBlockManager. And it processes it like kind of a fastpath. But I think geopandas did that rather for compatibility with pandas passing such objects to the constructor, rather than geopandas wanting to pass block managers ourselves (need to check though).

But can you explain a bit in more detail what you propose to change or what you want to achieve? (I don't see much in the diff. You didn't yet change cases that are passing block managers to the constructor?)

> I am not fully sure anymore about the context where I would have said that (do you remember the issue/PR?)

I think it was in the PR where originally I wanted to make a `_from_mgr` fastpath, which ended up getting folded into `__init__`.

> But can you explain a bit in more detail what you propose to change or what you want to achieve?

After this PR, the check `if isinstance(data, SingleBlockManager)` on L289 is never hit in the tests, so we could disallow it and simplify the constructor.
> After this PR, the check if isinstance(data, SingleBlockManager) on L289 is never hit in the tests, so we could disallow it and simplify the constructor.

But that's because the SingleBlockManager is also already handled at L206. 
So you are only speaking about the case of passing SingleBlockManager + other keywords like index/dtype ? I think that is a good idea to disallow/deprecate that.
BUG: DTA/TDA/PA setitem incorrectly allowing i8
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
I supppose this is worth a whatsnew note as well
Yes, all good
thanks @jbrockmendel 

if you can add a whatsnew in a followup
BUG: DTI/TDI/PI.where accepting incorrectly-typed NaTs
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

In addition to the bug in the title, this makes DTI accept datetime/dt64, TDI accept timedelta/td64, and PI accept Period.  This brings these into closer alignment with the other methods, which we ultimately want to make as consistent as possible.



Labels:
name: Bug
name: Index
description: Related to the Index class or subclasses

Comments:
If/when this goes through, these tests need some serious cleanup
can you merge master
rebased+green
DOC: Improve to_parquet documentation
State: closed

fixes #29476

- [x] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry  / NA


Labels:
name: Docs

Comments:
Thanks for the PR @larroy

Looks like this doc string actually lives in `pandas/core/frame.py` so its rendered here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_parquet.html

Otherwise looks good to me!
LGTM - minor comment
@pandas-dev/pandas-core to review/merge
Hello @larroy! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-05-28 17:40:39 UTC
Cc @WillAyd this looks good to me.
Thanks @larroy 
Thank you all for the review and help.
TYP: remove #type:ignore from core.arrays.datetimelike
State: closed

pandas\core\arrays\datetimelike.py:1535: error: Signatures of "__iadd__" and "__add__" are incompatible
pandas\core\arrays\datetimelike.py:1544: error: Signatures of "__isub__" and "__sub__" are incompatible

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Nice, LGTM pending green
Thanks @simonjayhawkins
BUG: DTI/TDI.insert doing invalid casting
State: closed

- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Index
description: Related to the Index class or subclasses

Comments:
> does this closes any issues?

Not AFAICT
looks good, can you merge master and ping on green.
rebased+green
fix bug of overflow validaion in 'reshape'
State: closed

- [x] closes #26314
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Hello @chjinche! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`pandas/core/reshape/reshape.py`](https://github.com/pandas-dev/pandas/blob/80332719980b8a28b7995e26eabe2fa59d80921e/pandas/core/reshape/reshape.py):

> [Line 119:89](https://github.com/pandas-dev/pandas/blob/80332719980b8a28b7995e26eabe2fa59d80921e/pandas/core/reshape/reshape.py#L119): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (93 > 88 characters)


Thanks @chjinche for the PR

can you add a release note and a test (for the test I _think_ can use https://github.com/pandas-dev/pandas/issues/26314#issuecomment-524915368)

@chjinche closing as stale. ping to reopen if you want to continue.
CLN: Split dtype inference tests
State: closed

Breaking up some very large tests in dtypes/test_inference.py

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Hello @dsaxton! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-04-21 02:34:24 UTC
small comment request, otherwise LGTM nice cleanup
thanks @dsaxton 

would take a followup to use the `nulls_fixture` rather than hard-coding the nan values
REF: collect validator methods for DTA/TDA/PA
State: closed

Zero logic changes in this PR.

xref #33685

We have a _lot_ of methods for converting arguments to DTA/TDA/PA/DTI/TDI/PI methods to/from i8 representations.  ATM these have slightly different behavior, which we should try to bring into sync (and then de-duplicate the code).

This PR just collects all those methods in one place to make the upcoming steps easier.

Two more methods that I think/hope can eventually be added to this grouping are the comparisons and get_loc, but those are more complicated, but those are more complicated, so holding off on those for now.

Labels:
name: Refactor
description: Internal refactoring of code
name: Index
description: Related to the Index class or subclasses

Comments:
sure
Grammatically updated HTML  Footer
State: closed

Grammatically updated HTML  Footer, Adding a period in the 89th line.

- [x] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Web
description: pandas website

Comments:
thanks
CI/DEP: Use numba.extending.is_jitted for numba version > 0.49.0
State: closed

- [x] xref #33686
- [ ] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
`azure-36-minimum_versions.yaml` is testing numba = 0.46.0, so we should be covered 
@Meeseeksdev backport to 1.0.x


Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.0.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 e0d4622cb7189adba8fc65571ff5a608564f8c67
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #33687: CI/DEP: Use numba.extending.is_jitted for numba version > 0.49.0'
```

4. Push to a named branch :

```
git push YOURFORK 1.0.x:auto-backport-of-pr-33687-on-1.0.x
```

5. Create a PR against branch 1.0.x, I would have named this PR:

> "Backport PR #33687 on branch 1.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BLD: Set max version of numba to 0.48.0
State: closed

This should fix the errors in the CI (Travis) builds, e.g.

```
E       AttributeError: module 'numba' has no attribute 'targets'
```

Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
makes sense, cc @mroeschke 
Ah. This had slipped my mind.

The reason this is failing is due to https://github.com/numba/numba/issues/5256#issuecomment-586899566

I'm working on the patch right now that would be preferable to pinning the numba version, but if this is blocking anything urgent, happy to merge
https://github.com/pandas-dev/pandas/pull/33687 should equivalently fix the failing builds due to numba v0.50.0
Sounds good, closing in favor of #33687 
@mgmarino thanks for the catch. Merged in https://github.com/pandas-dev/pandas/pull/33687 so things should be working now
BUG: arg validation in DTA/TDA/PA.take, DTI/TDI/PI.where
State: closed

We have a _lot_ of really similar validation/casting methods in these classes.  This is a step in smoothing out the differences between them so we can share them.

Labels:
name: Bug
name: Refactor
description: Internal refactoring of code

Comments:
looks fine, pls rebase
rebased+green
TST: Remove some old xfails
State: closed

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @mroeschke! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:













There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2020-04-20 18:52:09 UTC
nice, LGTM pending green
thanks
CLN,TYP: Use subsitutions in info docstrings
State: closed

Another precursor to #31796, which should make it easier to review/merge

----

screenshots of docs

![image](https://user-images.githubusercontent.com/33491632/79745028-4f7bb480-82ff-11ea-90d5-8c111a449126.png)
![image](https://user-images.githubusercontent.com/33491632/79744668-aaf97280-82fe-11ea-95e6-7cc34eafbeac.png)
![image](https://user-images.githubusercontent.com/33491632/79744688-b8aef800-82fe-11ea-970f-053825465ef3.png)
![image](https://user-images.githubusercontent.com/33491632/79744720-c5335080-82fe-11ea-88c1-4728488f6221.png)


Labels:
name: Docs

Comments:
Lgtm - @jbrockmendel merge if happy
Thanks @MarcoGorelli
Solve missing interpolation method (cubicspline)
State: closed

By commit 8bb2cc1 scipy.interpolate.CubicSpline method is
referenced in the pandas documentation (see pandas/core/generic.py)
but it is not wrapped by any interpolation method. This PR solves
this by adding the corresponding wrapper. SciPy's CubicSpline
is a cubic spline data interpolator that allows explicit control
of the boundary conditions for the interval.

Changes to be committed:
	modified:   pandas/core/missing.py
	modified:   pandas/tests/series/methods/test_interpolate.py

- [x] closes #xxxx (no issue opened but PR #20270 commented instead)
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Enhancement
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
thanks @khyox very nice
Many thanks @jreback and @mroeschke for your guidance on my 1st PR to the awesome pandas package!
CLN: make DTA/TDA check clearer
State: closed


Labels:
name: Clean

Comments:
@jbrockmendel @jreback Could you explain why this change is needed, please?
I think this change is part of the cause of https://github.com/pandas-dev/pandas/issues/23735 and  https://github.com/pandas-dev/pandas/issues/34531
> Could you explain why this change is needed, please?

an isinstance check is much clearer than a hasattr check.  It isnt clear to me how this would affect performance anywhere unless we were doing this hasattr check somewhere else to get to a fastpath
REF: use array_algos shift for Categorical.shift
State: closed

We have duplicate code in Categorical.shift and Categorical.take that can be refactored out into Categorical._validate_fill_value (which is the name that DTA/TDA/PA use for the same method, which I intend to share eventually, xref #33660).  From there the rest of Categorical.shift can dispatch to array_algos.transforms.shift.

The logic changed here is a TypeError becoming a ValueError (which matches what DTA/TDA/PA do)

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
lgtm. comment (can merge and followon or do here).
> (can merge and followon or do here).

Will do follow-up to make the copy consistent (and occur in exactly one place)
> > (can merge and followon or do here).
> 
> Will do follow-up to make the copy consistent (and occur in exactly one place)

gr8
TST: Dtype constants into pandas._testing to avoid direct imports from conftest
State: closed

part of #30914

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

---------------------------------------------------------------------------------------

Took a naive approach to move these constants into `pandas._testing`. Could also move them to a separate module instead. Let me know what you think is best.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
looks pretty good
LGTM pending green
@jbrockmendel @jreback Thanks for the review. CI is green now.
thanks @SaturnFromTitan
BUG: Fix StringArray use_inf_as_na bug
State: closed

- [x] closes #33655
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
you also have #33629  for this same issue?
> you also have #33629 for this same issue?

We were thinking we may want to treat them separately since they're slightly different issues: https://github.com/pandas-dev/pandas/pull/33629#discussion_r410871916. Should both bugs be in the same PR?
Can you also add a test somewhere for an object dtype Series that has pd.NA in it? (that won't be covered by the string dtype tests)

This shouldn't be "draft" anymore?
> This shouldn't be "draft" anymore?

I was hoping to finish up the PR linked above first because it overlaps somewhat with this one
Does this address #33209?
> Does this address #33209?

Yes I think it should
> Can you also add a base extension test for this? (there should already be `isna()` tests, so you can add a similar one there but which is using `use_inf_as_na`

Added some base extension use_inf_as_na tests, however I'm getting failures from test_sparse.py (dtype-related, I believe), test_json.py and test_decimal.py (the latter two I think because of improper handling of null values; e.g., {} and Decimal("NaN") under the use_inf_as_na context). Should I xfail for these for now and try to address in a follow-up?
@dsaxton can you merge master once more?

And what is left for this PR? I think only the comment of @jbrockmendel to not do the expensive `libmissing.isnaobj_old(values.to_numpy())` check for certain dtypes where this will never be needed (like Periods) ? 

> And what is left for this PR?

@jorisvandenbossche There was also this issue that I still need to figure out (I think it could potentially be fixed with the short circuiting mentioned above): https://github.com/pandas-dev/pandas/pull/33656#issuecomment-621947813
@jreback ping
thanks @dsaxton
DOC: Fix heading capitalization in doc/source/whatsnew - part6 (#32550)
State: closed

- [ ] Modify files v0.22.0.rst, v0.23.1.rst, v0.19.0.rst, v0.24.0.rst, v0.24.2.rst

-[ ] Add exceptions in 'validate_rst_title_capitalization.py'

Labels:
name: Docs

Comments:
I added 'Expanding' exception keyword in validate_rst_title_capitalization.py, and now it is causing troubles in doc/source/reference/window.rst line 42. How would you like to treat this word? I figured it should be capitalized in doc/source/whatsnew/v0.23.0.rst line 300.

Also keyword exception 'Index' in validate_rst_title_capitalization now causes trouble in doc/source/whatsnew/v0.24.0.rst line 848 as word 'index' there should be lower case.
Thanks for the PR! Can you merge master? 
I really don't understand what's happening with azure. What is the problem? I tried to update the PR but there are still problems with azure, what's happening?
> I really don't understand what's happening with azure. What is the problem? I tried to update the PR but there are still problems with azure, what's happening?

Likely just a flakey test - I would merge master and push again. Also mind linking the issue in the PR description?
BUG: Create empty dataframe with string dtype fails
State: closed

- [ ] closes #33623, closes #27953
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Thanks @kotamatsuoka for working on this. can you add a whatsnew.
> can you add a whatsnew.

@simonjayhawkins 
I added a whatnew. Is it collect? This is first time.
thanks @kotamatsuoka
TST: add messages to pytest.raises
State: closed

- [x] xref #30999
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @proost
BUG/API: getitem behavior with list match ndarray/index/series
State: closed

- [x] closes #33642
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
updated, thanks
thanks
IO: Fix S3 Error Handling
State: closed

closes #27679 
closes #32486 

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv
name: IO Parquet
description: parquet, feather

Comments:
Seems pretty reasonable to me if green. Can you add a whatsnew?
Thanks for the review @TomAugspurger 
green @jreback 
thanks @alimcmaster1