BrandonFors/pandas-github-issues huggingface dataset:
BUG: Right result for DatetimeIndex + TimeDelta when timezone is set(…
State: closed

- [x] closes #13905 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

…#13905)


Labels:
name: Bug
name: Timedelta
description: Timedelta data type
name: Timezones
description: Timezone data dtype

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13935?src=pr) is 85.30% (diff: 100%)

> Merging [#13935](https://codecov.io/gh/pydata/pandas/pull/13935?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13935   diff @@
==========================================
  Files           139        139          
  Lines         50157      50161     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42785      42790     +5   
+ Misses         7372       7371     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [b7abef4...b17da31](https://codecov.io/gh/pydata/pandas/compare/b7abef4949fb1ba7fd1004feba4f47ace7004282...b17da314e42ea0db4fbb60695e8c8f43334efb9c?src=pr)

Pls add a test for `Index` constructor covering possible patterns.

acutally need to localize if freq < 1D, otherwise can simply add it.

E.g. if we are crossing a dst boundary, then need to convert to UTC, operate, then re-convert.

So I think we should continue to always convert / operate / re-convert for simplicity.

@jreback you're right, but `DatetimeIndex` itself handles this, so not needed in `ops.py`?
- https://github.com/pydata/pandas/blob/master/pandas/tseries/index.py#L771

can you rebase / update?

can you rebase / update

can you rebase / update
closing, but if you want to update, pls comment
BUG: freqstr may be parsed incorrectly
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Invalid freq like below doesn'traise `ValueError` properly. The PR makes all the below raise.

```
pd.tseries.frequencies.to_offset('U1')
# <Micro>
pd.tseries.frequencies.to_offset('3U1')
# <3 * Micros>
pd.tseries.frequencies.to_offset('-33-22U')
# <-22 * Micros>
```


Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
might fix / xref to https://github.com/pydata/pandas/issues/8419

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13930?src=pr) is 85.30% (diff: 100%)

> Merging [#13930](https://codecov.io/gh/pydata/pandas/pull/13930?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13930   diff @@
==========================================
  Files           139        139          
  Lines         50157      50162     +5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42785      42791     +6   
+ Misses         7372       7371     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [b7abef4...e797a2d](https://codecov.io/gh/pydata/pandas/compare/b7abef4949fb1ba7fd1004feba4f47ace7004282...e797a2d93333ac6b48b56c3a63f0e667ad97e7e2?src=pr)

Now `pd.tseries.frequencies.to_offset('.5S')` explicitly raises. I'll comment on #8419.

@sinhrks merged #13874 so this needs a rebase

ok, rebased.

minor change. lgtm. ping when green.

thanks!
BUG: DatetimeTz shift raises AmbiguousTimeError near DST
State: closed

- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

As the same as #13650, caused by `.tz_localize(None)` which can be removed.

```
import pandas as pd
s = pd.Series(pd.date_range('2016-11-06', freq='H', periods=10, tz='US/Eastern'))
s.shift(1)
# pytz.exceptions.AmbiguousTimeError: Cannot infer dst time from Timestamp('2016-11-06 01:00:00'), try using the 'ambiguous' argument
```


Labels:
name: Bug
name: Timezones
description: Timezone data dtype

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13926?src=pr) is 85.30% (diff: 100%)

> Merging [#13926](https://codecov.io/gh/pydata/pandas/pull/13926?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13926   diff @@
==========================================
  Files           139        139          
  Lines         50138      50141     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42769      42773     +4   
+ Misses         7369       7368     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3186fef...c079ee3](https://codecov.io/gh/pydata/pandas/compare/3186fef545b55ca7b4a3c79c800f32b1d586545e...c079ee3a5cdf6d77e53d647c4122132faf942ee7?src=pr)

ty sir!
BLD: split join.pyx from algos.pyx
State: closed

closes #13921


Labels:
name: Build
description: Library building on various platforms
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13925?src=pr) is 85.30% (diff: 100%)

> Merging [#13925](https://codecov.io/gh/pydata/pandas/pull/13925?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13925   diff @@
==========================================
  Files           139        139          
  Lines         50138      50141     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42769      42771     +2   
- Misses         7369       7370     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3186fef...15f763b](https://codecov.io/gh/pydata/pandas/compare/3186fef545b55ca7b4a3c79c800f32b1d586545e...15f763bdde445a9a7b5c67299e9a8503a5a12841?src=pr)
BUG: na_rep given precedence in to_html
State: closed

- [x] closes #13828
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

~~I know new tests are appreciated from the get-go, but it's the end of the day for me. I'll add new tests and the appropriate whatsnew entry tomorrow.~~


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/13911?src=pr) is 84.64% (diff: 90.00%)
> Merging [#13911](https://codecov.io/gh/pandas-dev/pandas/pull/13911?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #13911   diff @@
==========================================
  Files           144        144          
  Lines         51016      51024     +8   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43184      43191     +7   
- Misses         7832       7833     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ccb501...685c6d2](https://codecov.io/gh/pandas-dev/pandas/compare/3ccb50131b698bccee21780c660a70bed87396d0...685c6d2f786c6851c4d59c07db7cfc10e63f6e2f?src=pr)
@jreback I think I understood what you meant by iterating over non-null values. Please have a look.

@jreback

can you rebase / update?

Rebased and all green.

lgtm. @jorisvandenbossche 

i think this got lost in the shuffle, can you update.
Rebased, moved whatsnew to 0.20.0, need to look into the code duplication @jreback mentioned.
can you rebase / update
closing as stale. please ping if you'd like to continue.
BUG: pd.to_datetime doesn't raises AttributeError with specific input…
State: closed

- [x] closes #12424
- [x] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

…s when errors='ignore'(#12424)


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13909?src=pr) is 85.30% (diff: 100%)

> Merging [#13909](https://codecov.io/gh/pydata/pandas/pull/13909?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13909   diff @@
==========================================
  Files           139        139          
  Lines         50138      50138          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42769      42769          
  Misses         7369       7369          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e5ee5d2...58d3d19](https://codecov.io/gh/pydata/pandas/compare/e5ee5d2e034c9a4be795596633c199d0ba23970b...58d3d196c31b10cb546bc792106bfa79cb71f88b?src=pr)

pls add a release note as well

thanks!
ENH: Accept list as level for groupby in non-MultiIndexed objects
State: closed

- [x] closes #13901
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13907?src=pr) is 85.25% (diff: 100%)

> Merging [#13907](https://codecov.io/gh/pydata/pandas/pull/13907?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13907   diff @@
==========================================
  Files           140        140          
  Lines         50568      50575     +7   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43112      43119     +7   
  Misses         7456       7456          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [a7469cf...fcbb724](https://codecov.io/gh/pydata/pandas/compare/a7469cf98275a183ad2e4bfafa9706a1ef8d035e...fcbb72402f0cf4da1d2ccfdda39ac62a1175fa35?src=pr)

@sinhrks Made all changes and additions.

@jreback @sinhrks Check out the latest amendment. How does it look to you?

@jreback There is a test that fails, but it has nothing to do with `groupby` and I cannot reproduce it on my machine. Thoughts?

@jreback

can you rebase / update?

Rebased and all green.

ok lgtm. @jorisvandenbossche 

thanks!
ENH: sparse astype now supports int64 and bool
State: closed

- [x] split from #13849
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

`SparseXXX` now support `astype` other than `float64` dtype. It raises `ValueError` when its `sp_values` or `fill_value` cannot be converted to specified dtype.

```
s = pd.SparseSeries([1., 0., 2., 0.], fill_value=0)
s.astype(np.int64)
#0    1
#1    0
#2    2
#3    0
# dtype: int64
# BlockIndex
# Block locations: array([0, 2], dtype=int32)
# Block lengths: array([1, 1], dtype=int32)

pd.SparseSeries([1, np.nan, 2, np.nan]).astype(np.int64)
# ValueError: unable to coerce current fill_value nan to int64 dtype
```

Also, `fill_value` setter now checks input value dtype and raises `ValueError` if value cannot be stored to current dtype.

```
s = pd.SparseSeries([1, 0, 2, 0], fill_value=0, dtype=np.int64)
s.fill_value = 2
s
#0    1
#1    2
#2    2
#3    2
# dtype: int64
# BlockIndex
# Block locations: array([0, 2], dtype=int32)
# Block lengths: array([1, 1], dtype=int32)

s.fill_value = np.nan
# ValueError: unable to set fill_value nan to int64 dtype
```


Labels:
name: Enhancement
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13900?src=pr) is 85.29% (diff: 100%)

> Merging [#13900](https://codecov.io/gh/pydata/pandas/pull/13900?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13900   diff @@
==========================================
  Files           139        139          
  Lines         50057      50077    +20   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42692      42712    +20   
  Misses         7365       7365          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [45d54d0...1c669ad](https://codecov.io/gh/pydata/pandas/compare/45d54d067ed9a0dfdb456f253fb0bede9969187c...1c669adb43f46831047507d56aff4b447a041e35?src=pr)

lgtm, is this dependent on anything or standalone?

thx for the review. it's independent.

thanks!
BUG: union_categorical fastpath sort
State: closed

Follow up to a problem with #13846.   This path was tested, but the test just happened to be constructed in such a way it worked with the buggy implementation.


Labels:
name: Bug
name: Categorical
description: Categorical Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13899?src=pr) is 85.28% (diff: 100%)

> Merging [#13899](https://codecov.io/gh/pydata/pandas/pull/13899?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13899   diff @@
==========================================
  Files           139        139          
  Lines         50057      50057          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          42692      42692          
  Misses         7365       7365          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [45d54d0...b07a7d3](https://codecov.io/gh/pydata/pandas/compare/45d54d067ed9a0dfdb456f253fb0bede9969187c...b07a7d392f2f7a82846061db71ef34963dd81ecd?src=pr)

ty sir!
BUG: allow describe() for DataFrames with only boolean columns
State: closed

- [x] closes #13891
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Closes #13891. Existing tests pass. ~~No new tests added yet.~~


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
Thx for the PR! We appreciate including the tests from first to see the detail.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13898?src=pr) is 85.30% (diff: 100%)

> Merging [#13898](https://codecov.io/gh/pydata/pandas/pull/13898?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13898   diff @@
==========================================
  Files           139        139          
  Lines         50143      50143          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42777      42776     -1   
- Misses         7366       7367     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [7e15923...26201aa](https://codecov.io/gh/pydata/pandas/compare/7e15923d908a42008fb7fecde58be626277fd82e...26201aa0a9e4d7f48410bf8f244ead9d682d200a?src=pr)

@sinhrks Changes made.

lgtm. ping on green.

> ping on green.

?

when all of the checks have passed it will turn grin. ping then.

@jreback

thanks @agraboso
BUG: fix precision to specify decimal places
State: closed

- [x] closes #13257 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

Use 'f' as a string conversion type to specify the number of decimals to
represent. The 'g' format will instead fall back to the 'e' format
causing the unexpected behavior seen in GHI13257.

See: [old-string-formatting](https://docs.python.org/3.5/library/stdtypes.html#old-string-formatting)

Replicated code from the issue, with output:
<img width="702" alt="example output with precision" src="https://cloud.githubusercontent.com/assets/7526790/17386623/77eb83f4-59ba-11e6-8193-fde0e839a184.png">


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Also, add tests with `Float64Index` like:

```
df = pd.DataFrame(data = np.random.randn(4,4),
                  index=[11.1, 22.2, 33.3, 44.4]
                  columns=[1.1111, 2.2222, 3.3333, 4.4444])
```

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13897?src=pr) is 85.28% (diff: 100%)

> Merging [#13897](https://codecov.io/gh/pydata/pandas/pull/13897?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13897   diff @@
==========================================
  Files           139        139          
  Lines         50057      50057          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42692      42691     -1   
- Misses         7365       7366     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [45d54d0...0731bcb](https://codecov.io/gh/pydata/pandas/compare/45d54d067ed9a0dfdb456f253fb0bede9969187c...0731bcbc8dc1042a263ad99452443278911d6bfc?src=pr)

@TomAugspurger ?

@drewrey pls add a whatsnew note (bug fix section)

Changing it to `g` was intentional to avoid things like

``` python
In [1]: "{:f}".format(1)
Out[1]: '1.000000'
```

and instead have that formatted as `1`. I glanced quickly and didn't see any tests for that :/

Could you add a test with an integer and a float column like `1.0`, use the default precision / formatter, and verify that the display value is `1`, and then make sure your change passes that test?

Looking further,  it seems like we have a bug in master.

``` python
m = pd.DataFrame({"a": [1], "b": [1.]})
```

`m.style._translate()['body'][0][1]['display_value']`  # should be '1', is correct

`m.style._translate()['body'][0][2]['display_value']`  # should be '1.0', currently is '1', is incorrect

Ideally `df.style` would have the same output as `df._repr_html()` by default.

can you rebase / update?

can you add a whatnew note in 0.19.2

@TomAugspurger ?

closing, but if you'd like to update, pls do so.
API/BUG: Fix Series ops inconsistencies
State: closed

- [x] closes #1134, closes #4581, closes #13538
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

This includes the fix only for #1134 and not for other inconsistencies (like #13637), as it needs further discussions.

Changes:
- series comparison operator to check whether labels are identical (currently: ignores labels)
- series boolean operator to align with labels (currently: only keeps left index)


Labels:
name: API Design
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13894?src=pr) is 85.25% (diff: 100%)

> Merging [#13894](https://codecov.io/gh/pydata/pandas/pull/13894?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13894   diff @@
==========================================
  Files           139        139          
  Lines         50386      50394     +8   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42959      42965     +6   
- Misses         7427       7429     +2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [51b20de...fe322be](https://codecov.io/gh/pydata/pandas/compare/51b20de44fdcb998f25392558e029621e10c451c...fe322be77d77811bb27eadb6abb29dfe66c7bd63?src=pr)

will have a look.

@jorisvandenbossche 
@wesm 

For dataframes, the operator (eg `==`) is strict regarding indices, but the equivalent method (eg `df1.eq(df2)`) is flexible as it aligns. 
For series, both options raise now (current PR) or are broken (master). I think we should also make this distinction and make `s1.eq(s2)` flexible (=align and not raise) as discussed in https://github.com/pydata/pandas/issues/1134?

This seems OK to me. I'm sure this will cause some API breakage but it would be best to encourage folks to use the flex comparison methods if they want auto-alignment

In light of Wes' comment, I think the whatsnew notice can be a bit more explicit in that regard. So say that if you did `s1 == s2` before (which now raises), that you have to do `s1.values == s2.values` if you want to keep the same non-aligning / index-ignoring behaviour, or `s1.eq(d2)` to get auto-alignment.

Modulo the doc comments, this is good to go for me!

LGTM 
@jreback ?

@sinhrks Thanks a lot!