BrandonFors/pandas-github-issues huggingface dataset:
ENH: bool sparse now supports logical op
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

implemented bool (logical and / or) op for bool `SparseArray`. Currently it raises if `sp_index` are different.

```
s1 = pd.SparseArray([True, True, True, True, True], dtype=np.bool, fill_value=False)
s2 = pd.SparseArray([False, True, False, True, True], dtype=np.bool, fill_value=False)
s1 & s2
# ValueError: operands could not be broadcast together with shapes (5,) (3,) 
```


Labels:
name: Enhancement
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/14000?src=pr) is 85.28% (diff: 100%)

> Merging [#14000](https://codecov.io/gh/pydata/pandas/pull/14000?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #14000   diff @@
==========================================
  Files           139        139          
  Lines         50214      50224    +10   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42822      42832    +10   
  Misses         7392       7392          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a0d05db...6db3096](https://codecov.io/gh/pydata/pandas/compare/a0d05dbf7477df863486d82a0cdd4e3e93023864...6db309637fb81e168be98b2d0094cf6951ee8bc7?src=pr)

thanks!
API: SparseSeries comparison now returns sparse
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Because #13985 has been fixed, `SparseSeries` comparison op now can return `SparseSeries` (it returns normal `Series` on current master). 

also fixed a bug when `SparseArray` created from `SparseSeries` may not inherit `dtype`.


Labels:
name: API Design
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13999?src=pr) is 85.25% (diff: 100%)

> Merging [#13999](https://codecov.io/gh/pydata/pandas/pull/13999?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13999   diff @@
==========================================
  Files           139        139          
  Lines         50380      50384     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42953      42955     +2   
- Misses         7427       7429     +2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [453bc26...eafc94c](https://codecov.io/gh/pydata/pandas/compare/453bc26e07ebbfea33dda9ff2f4efab683718506...eafc94c980eea9644621bc66bc3b0e2b7d6fc1ad?src=pr)

thanks!
Fixed to_string with line_width and without index
State: closed

When calling to_string on a DataFrame with line_width, this calls
_join_multiline internally which only sets the idx variable when
index is True, otherwise idx is never set resulting in the error:

  UnboundLocalError: local variable 'idx' referenced before assignment


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13998?src=pr) is 85.27% (diff: 100%)

> Merging [#13998](https://codecov.io/gh/pydata/pandas/pull/13998?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13998   diff @@
==========================================
  Files           139        139          
  Lines         50211      50212     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42819      42820     +1   
  Misses         7392       7392          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [5f49638...e0b1840](https://codecov.io/gh/pydata/pandas/compare/5f496380beb81d3986ca48cf41b893d95c227895...e0b1840d733d1c16f2307b8e03de9f610b5ced7c?src=pr)

@cr3 Thanks! 
Can you add a small whatsnew notice (in docs/source/whatsnew/v0.19.0.txt, in the bug fixes section)

Also, I noticed `line_width` is not mentioned in the `to_string` docstring. Would you like to add an explanation for that as well?

@jorisvandenbossche, thanks for the suggestion, I rebased my branch with both your suggestions. To be pedantic, I'm not sure if the description of `line_width` in the docstring should start with an upper or lower case character; the first descriptions in the docstring used lower case and the last ones used upper case, so I decided to be consistent with the last ones.

> To be pedantic, I'm not sure if the description of line_width in the docstring should start with an upper or lower case character

No problem, I like that :-) Well, I think it should start with upper case, but we indeed do not follow this rigourously in the code base.

@cr3 Thanks!
CLN: move _na_value to DatetimeIndexOpsMixin
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

Because `PeriodIndex` now shares the same `NaT` repr (#13609), can use the same definition on `DatetimeIndexOpsMixin`


Labels:
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13997?src=pr) is 85.28% (diff: 100%)

> Merging [#13997](https://codecov.io/gh/pydata/pandas/pull/13997?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13997   diff @@
==========================================
  Files           139        139          
  Lines         50214      50211     -3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42822      42820     -2   
+ Misses         7392       7391     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a0d05db...41268c3](https://codecov.io/gh/pydata/pandas/compare/a0d05dbf7477df863486d82a0cdd4e3e93023864...41268c38985ff8d084d3c4096022d328779eff16?src=pr)

@sinhrks Looks good. Can be merged?

lgtm. go ahead @jorisvandenbossche
TST: Fix unused tests in tseries/test_base.py
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- enabled unused patterns left as comments
- made `tz` related tests more comprehensive


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13996?src=pr) is 85.27% (diff: 100%)

> Merging [#13996](https://codecov.io/gh/pydata/pandas/pull/13996?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13996   diff @@
==========================================
  Files           139        139          
  Lines         50206      50215     +9   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42817      42822     +5   
- Misses         7389       7393     +4   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6f88416...869ef66](https://codecov.io/gh/pydata/pandas/compare/6f8841648d068cc04c7bfe0625f131b7c7d6469e...869ef6698041b439188f8a5f5114c5c7aa5b8795?src=pr)

thanks!
CLN: Move PeriodIndex.repeat to DatetimeIndexOpsMixin
State: closed

- [x] related to #6469
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`


Labels:
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13995?src=pr) is 85.28% (diff: 100%)

> Merging [#13995](https://codecov.io/gh/pydata/pandas/pull/13995?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13995   diff @@
==========================================
  Files           139        139          
  Lines         50206      50210     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42817      42821     +4   
  Misses         7389       7389          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6f88416...c627549](https://codecov.io/gh/pydata/pandas/compare/6f8841648d068cc04c7bfe0625f131b7c7d6469e...c6275491cbe6fc972e16cb922a3cb7e199f43cec?src=pr)

thxs!
BUG: groupby cumsum with axis=1 computes cumprod
State: closed

- [x] closes #13993
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby

Comments:
lgtm. ping on green.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13994?src=pr) is 85.28% (diff: 100%)

> Merging [#13994](https://codecov.io/gh/pydata/pandas/pull/13994?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13994   diff @@
==========================================
  Files           139        139          
  Lines         50206      50206          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42817      42820     +3   
+ Misses         7389       7386     -3   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [6f88416...56dc1c7](https://codecov.io/gh/pydata/pandas/compare/6f8841648d068cc04c7bfe0625f131b7c7d6469e...56dc1c759c4ffc2f14543991e9ac0e7896241150?src=pr)

@jreback ~~All green.~~ I swear, it was.

green means travis is green (its yellow atm).

@jreback OK, _now_ it's all green.

thanks!
Use DeprecationWarning for core.common deprecations (GH13634)
State: closed

Related to second question in #13634 (whether to use FutureWarning or DeprecationWarning in deprecating the public pandas.core.common functions).

As those functions are mostly used in library code, and less directly by users in their own code, I think a DeprecationWarning is more appropriate in this case. 
For example, in our own docs, we started to get warnings due to an example with a statsmodels regression that uses patsy using one of those functions. Note that recent IPython also shows DeprecationWarnings when using a deprecated function interactively.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Clean

Comments:
lgtm

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13990?src=pr) is 85.28% (diff: 100%)

> Merging [#13990](https://codecov.io/gh/pydata/pandas/pull/13990?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13990   diff @@
==========================================
  Files           139        139          
  Lines         50206      50206          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42818      42818          
  Misses         7388       7388          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [29d9e24...2de5d48](https://codecov.io/gh/pydata/pandas/compare/29d9e24f4c778b0c9ebe9288bfc217808d2c6edb...2de5d481cba8d7041de96f6f133261855937a666?src=pr)

thanks @jorisvandenbossche 

I made a slight doc update.

Thanks!
DOC: suppress warnings for Panel4D deprecation
State: closed

Other options is to just remove those sections in the latest docs ... (they are always available in the older ones).

The example in the HDF5 docs with a legacy file will have to be updated to not include Panel4D, or the reading of such files will have to be adapted in the future to handle those when Panel4D is actually removed (@jreback)


Labels:
name: Docs

Comments:
hmm I think removing them and pointing the old docs is better?

though the precedent is to leave them iirc (eg remote data)

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13989?src=pr) is 85.28% (diff: 100%)

> Merging [#13989](https://codecov.io/gh/pydata/pandas/pull/13989?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13989   diff @@
==========================================
  Files           139        139          
  Lines         50206      50206          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42818      42817     -1   
- Misses         7388       7389     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [29d9e24...7d067aa](https://codecov.io/gh/pydata/pandas/compare/29d9e24f4c778b0c9ebe9288bfc217808d2c6edb...7d067aace9dda76944c9a60ca8e9ca4ef0588562?src=pr)

yep, will leave them in for now
API: PeriodIndex.values now return array of Period objects
State: closed

- [x] split from #13941 (comment)
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

This change itself needs more tests / perf testing, split from the original PR.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: API Design
name: Period
description: Period data type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13988?src=pr) is 85.26% (diff: 93.25%)

> Merging [#13988](https://codecov.io/gh/pydata/pandas/pull/13988?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13988   diff @@
==========================================
  Files           139        139          
  Lines         50446      50475    +29   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43018      43040    +22   
- Misses         7428       7435     +7   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [df2d9ab...d7637c9](https://codecov.io/gh/pydata/pandas/compare/df2d9ab917a1ef4732cfb6c3317432f60c071074...d7637c9f3a5da88e0cb7b2a1cca8b476f8878ddc?src=pr)

this looks fine. pls add a small section for showing the changes in whatsnew (though if this should be merged after #13941 can just add on there (and vice-versa)

OK, let #13941 be first, because this PR includes some workarounds which can be removed once we have `period` dtype.

@sinhrks pls rebase. This will be a welcome change!

Needs a new rebase ;) (for your fixed of the tests)

Can you also add a whatsnew notice?

Added whatsnew, now ready for review.

thanks!
CLN: move PeriodIndex.__getitem__ to DatetimeIndexOpsMixin
State: closed

- [x] related to #6469
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

merge `PeriodIndex.__getitem__` to `DatetimeIndexOpsMixin`


Labels:
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13987?src=pr) is 85.27% (diff: 100%)

> Merging [#13987](https://codecov.io/gh/pydata/pandas/pull/13987?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13987   diff @@
==========================================
  Files           139        139          
  Lines         50214      50208     -6   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42822      42816     -6   
  Misses         7392       7392          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a0d05db...48d187d](https://codecov.io/gh/pydata/pandas/compare/a0d05dbf7477df863486d82a0cdd4e3e93023864...48d187d9d84cfb445f526dc20943aaf1f1ff4609?src=pr)

thanks!
BUG/CLN: datetimelike Index.equals may return True with non-Index
State: closed

- [x] closes #13107
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13986?src=pr) is 85.26% (diff: 87.50%)

> Merging [#13986](https://codecov.io/gh/pydata/pandas/pull/13986?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13986   diff @@
==========================================
  Files           139        139          
  Lines         50562      50557     -5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43116      43108     -8   
- Misses         7446       7449     +3   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [59524af...580151a](https://codecov.io/gh/pydata/pandas/compare/59524af1e90d3dda2d885e711f7258704a020b6d...580151a16a4586728ef4315f45a64b2ad1a2dfd0?src=pr)

Fixed impl and tests (added tests are not properly executed because it was overriden.

I'm still +1 to `.equals` should be `False` for non-`Index`, but there seems to be some tests which accepts `list` and returns `True`.

 So now I think we should keep current behavior for a while. Needs to decide whether to change it in future.

lgtm. very minor comments. ping on green.

@sinhrks Can you add some tests that explicitly test the behaviour for indexes with other attributes? (see my comment here https://github.com/pydata/pandas/pull/13986#discussion_r74693133) If it is not there yet of course.

I would also clarify this behaviour in the documentation.

Further, I would say that changing Index.equals to return False with non-Index objects is an API change, and not a bug fix, so would add a note about that in the whatsnew.

@sinhrks can you rebase

thanks @sinhrks
BUG: Sparse indexing with bool sparse may be incorrect
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

indexing with `SparseArray` with `bool` dtype may return incorrect result because of some internal conversions.

```
arr = pd.SparseArray([1, 2, 3])
indexer = pd.SparseArray([True, False, True], fill_value=False, dtype=bool)

# NG
arr[indexer]
# [2.0, 2.0]
# Fill: nan
# IntIndex
# Indices: array([0, 1], dtype=int32)

# Expected (after this PR)
arr[indexer]
# [1.0, 3.0]
# Fill: nan
# IntIndex
# Indices: array([0, 1], dtype=int32)
```


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13985?src=pr) is 85.28% (diff: 83.33%)

> Merging [#13985](https://codecov.io/gh/pydata/pandas/pull/13985?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13985   diff @@
==========================================
  Files           139        139          
  Lines         50206      50211     +5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42818      42822     +4   
- Misses         7388       7389     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [29d9e24...0909fa8](https://codecov.io/gh/pydata/pandas/compare/29d9e24f4c778b0c9ebe9288bfc217808d2c6edb...0909fa89f54f81d2cdd3a61d73dbd13c948d20af?src=pr)

thanks!
CLN: Datetimelike._can_hold_na
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] no whatsnew, as it is internal fix

datetimelike `Index._can_hold_na` is all `False` even though these can store `NaT`.

```
idx = pd.DatetimeIndex(['2011-01-01', 'NaT'])
idx.hasnans
# True

idx._can_hold_na
# False

idx._nan_idxs
# array([], dtype=int64)
```

Because of `test_base.py` condition below, datetime-like Indexes had skipped some tests until now.
- https://github.com/pydata/pandas/blob/master/pandas/tests/test_base.py#L219


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13983?src=pr) is 85.28% (diff: 100%)

> Merging [#13983](https://codecov.io/gh/pydata/pandas/pull/13983?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13983   diff @@
==========================================
  Files           139        139          
  Lines         50543      50547     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43101      43107     +6   
+ Misses         7442       7440     -2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8fdfa51...5435247](https://codecov.io/gh/pydata/pandas/compare/8fdfa518105cde9ae16449578b3d75c13c8c19a5...543524700440ff9ed7e17c6d93fc5937499db6e5?src=pr)

Updated not to rely on #13979. `_get_unique_index` can be removed after #13984.

@jreback @jorisvandenbossche can u check? This is needed to make some `Index` dtype related fixes.

This is looking good to me. @jreback OK?

needs #13979 first

@sinhrks updated to not rely on that one

@jreback Now this PR doesn't rely on #13979. 

@sinhrks This needs a rebase now I merged the other

lgtm. ping on green.

fixed lint error, and now green.

Thanks!