BrandonFors/pandas-github-issues huggingface dataset:
BUG: Fix bug with symmetric difference of two equal MultiIndexes GH12490
State: closed

- [x] closes #13490
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixes a bug where the symmetric difference of two equal MultiIndexes would raise a TypeError. MultiIndex used to use the `Index.symmetric_difference`. With this PR it it's own implementation that is more like `MultiIndex.difference`.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: MultiIndex

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13504?src=pr) is **84.34%**

> Merging [#13504](https://codecov.io/gh/pydata/pandas/pull/13504?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13504   diff @@
==========================================
  Files           138        138          
  Lines         51107      51122    +15   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43103      43117    +14   
- Misses         8004       8005     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [bf4786a...8cb445f](https://codecov.io/gh/pydata/pandas/compare/bf4786a15f51d14ccf1318e44ae080868d8595ee...8cb445fec5f51eb5f3de65ea319f9c4643554172)

If Travis passes again I think I got all your comments fixed
- [x] Move whatsnew
- [x] Move tests to test_multi.py
- [x] Constructor for empty MultiIndex (`MultiIndex._create_as_empty(nlevels)`)

can you rebase / update?

closing, but pls reopen / comment if you can continue
TST: Fix MMapWrapper init test for Windows
State: closed

Turns out Windows errors differently when an invalid `fileno` is passed into the `mmap` constructor, so there's no need to skip the test (xref: 9670b31).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Windows
description: Windows OS

Comments:
lgtm, ping when green.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13494?src=pr) is **84.33%**

> Merging [#13494](https://codecov.io/gh/pydata/pandas/pull/13494?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13494   diff @@
==========================================
  Files           138        138          
  Lines         51106      51106          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43102      43102          
  Misses         8004       8004          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [1a12ead...db7c6b1](https://codecov.io/gh/pydata/pandas/compare/1a12eadaf92eb1ee05446d24e5d7a12cde971ce3...db7c6b1190f3e173dda5e0097bc5e65a835c0d90)

@jreback : Travis is happy, so ready to merge if there are no concerns.
DOC/WIP: travis notebook doc build
State: closed

Just want to see the output log from the pull request.


Labels:
name: Docs
name: CI
description: Continuous Integration

Comments:
https://travis-ci.org/pydata/pandas/jobs/139080123

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13493?src=pr) is **84.33%**

> Merging [#13493](https://codecov.io/gh/pydata/pandas/pull/13493?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13493   diff @@
==========================================
  Files           138        138          
  Lines         51106      51106          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43102      43102          
  Misses         8004       8004          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [30d710f...df6939a](https://codecov.io/gh/pydata/pandas/compare/30d710f4c8a07cb7ea3bc91f6eb05c4bbdfa2f24...df6939a747ceeaf822faf2a83510175dbe0ef519)

@jorisvandenbossche so my last commit succeeded https://travis-ci.org/pydata/pandas/jobs/139093566#L1569

I'm not sure why the `python -m ipykernel install` was necessary, I've never had to do that on my own system. I want to look into the correct way to do this, but won't have a chance till tonight or tomorrow.

Hmm, I am not sure this is due to the installing of the kernel, as the logs say:

```
python -m ipykernel install
[Errno 13] Permission denied: '/usr/local/share/jupyter'
```

So I think it is just adding the `ipykernel` as dependency which was needed?

Ahh, I missed that. Let me retry just using `notebook` to get everything like Jeff suggested.

In principle, notebook is not needed (nbconvert and ipykernel should suffice as well), although doesn't hurt of course :-)

Apart from the clean-up, I think this can be merged?

@jorisvandenbossche fixed

@jorisvandenbossche http://pandas-docs.github.io/pandas-docs-travis/style.html 🎉 

A few minor things, like dependencies on ipywidgets and seaborn didn't work. I can do another followup.

Yes, we're coming closer :-)

Also a unicode thing (just needs a `u""` I think)
DOC: find kernelspec
State: closed

xref https://github.com/pydata/pandas/pull/13487

Problem was the notebook was written with a python3 kernel, but the doc build is python2. 
There are potentially problems here if you have non-default kernel names in your environment...


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13491?src=pr) is **84.33%**

> Merging [#13491](https://codecov.io/gh/pydata/pandas/pull/13491?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13491   diff @@
==========================================
  Files           138        138          
  Lines         51083      51083          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43083      43083          
  Misses         8000       8000          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [b01e07e...cb967e1](https://codecov.io/gh/pydata/pandas/compare/b01e07e3395f7cee5a9922af979ea41aa40ff892...cb967e14b67fdc16932f7f6de1a2755f50032648)

Isn't there a way to say "use the same kernel as the one that is calling me"? 
Because this will only work for "python2"/"python3" kernel names, which is OK for travis, but eg locally I build the docs mainly with my 'dev' environment for which I have a kernel named 'dev'

I couldn't find anything like that immediately. Will look tonight.

No matter, if I build the docs from that environment, the kernel will be named python2/3 anyhow, it's only when I select in the notebook the kernel itself (starting the notebook from another environment), that the name of this kernel will be different.

There is a difference between kernel name and display name in the `kernelspec`

Let's give it another try! :-) 

https://travis-ci.org/pydata/pandas/jobs/138920776 is the build.

Hmm, still not working: "Failed to convert source/html-styling.ipynb" in the logs

Just saw that 😞, I'll look into it more later.
DOC: add nbformat for notebook conversion
State: closed

The build [here](https://travis-ci.org/pydata/pandas/jobs/138633677#L1866) didn't succeed. Had `nbconvert` in the debs, but not `nbformat`.


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13487?src=pr) is **84.33%**

> Merging [#13487](https://codecov.io/gh/pydata/pandas/pull/13487?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13487   diff @@
==========================================
  Files           138        138          
  Lines         51083      51083          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43083      43083          
  Misses         8000       8000          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [20dd17a...9c74612](https://codecov.io/gh/pydata/pandas/compare/20dd17a70063387ccca5f7aca3dbefc904d99457...9c74612a9b31884fde33c2e11f2e8c53e79026a6)

https://travis-ci.org/pydata/pandas/jobs/138720940

something still not working, http://pandas-docs.github.io/pandas-docs-travis/style.html

style is blank :<
ENH:read_html() handles tables with multiple header rows #13434
State: closed

- [ ] closes #13434 
- [ ] read_html() handles tables with multiple header rows 

Now it produces the following output for the issue
  Unnamed: 0  Age Party
0    Hillary   68     D
1     Bernie   74     D
2     Donald   69     R


Labels:
name: Enhancement
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap

Comments:
Could you add tests for the new behavior, a release note, and a sentence (and maybe example) to the documentation? And the travis failure was for a style check. `git diff upstream/master | flake8 --diff`.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13485?src=pr) is 84.33% (diff: 33.33%)

> Merging [#13485](https://codecov.io/gh/pydata/pandas/pull/13485?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **0.90%**

``` diff
@@             master     #13485   diff @@
==========================================
  Files           140        138     -2   
  Lines         50563      51084   +521   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43103      43083    -20   
- Misses         7460       8001   +541   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [1e61aed...8629537](https://codecov.io/gh/pydata/pandas/compare/1e61aed165385a9e302ff1afea7eb01bd8bd7ecf...8629537f2c753133ba1436411fc5aae1253b6528?src=pr)

I think I first overlooked the issue. In the second commit I made a change such that we select the first row of the table that contains  th tags in thead of the table. And this should make previous test cases to pass. I will add some new test cases to test the modified behaviour and also i will change the documentation

I see that there are no tests added for beautifulsoup parser. is there any reason for that ?. And I think so that is the reason for the decrease in code coverage 

your test should be called by each parser unless you put it in a specific class (which only exercises specific parsers)

can you rebase / update?

needs tests.

please reopen / comment if you are continuing.
BUG: is_normalized returned False for local tz
State: closed

- [x] closes #13459 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

is_normalized returned False for normalized date_range
in case of local timezone 'Asia/Kolkata'
Fixes: #13459


Labels:
name: Bug
name: Timezones
description: Timezone data dtype

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13484?src=pr) is **84.33%**

> Merging [#13484](https://codecov.io/gh/pydata/pandas/pull/13484?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13484   diff @@
==========================================
  Files           138        138          
  Lines         51083      51106    +23   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43083      43102    +19   
- Misses         8000       8004     +4   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [3b75e03...4d48367](https://codecov.io/gh/pydata/pandas/compare/3b75e03e1bf9798c56f1d20cbc5a5c6c0d560979...4d48367d489b75a248e9e1758cafbfcea96176de)

@jreback Please review.

looks good. just a few doc comments. ping when green.

```
((pandas3.5)) C:\Users\conda\Documents\pandas3.5>nosetests pandas\tseries\tests\test_timezones.py
............................................................S.......................................................................E.....

======================================================================
ERROR: test_normalize_tz_local (pandas.tseries.tests.test_timezones.TestTimeZones)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\conda\Documents\pandas3.5\pandas\tseries\tests\test_timezones.py", line 1409, in test_normalize_tz_local
    with set_timezone(timezone):
  File "C:\Miniconda2\envs\pandas3.5\lib\contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "C:\Users\conda\Documents\pandas3.5\pandas\util\testing.py", line 2688, in set_timezone
    setTZ(tz)
  File "C:\Users\conda\Documents\pandas3.5\pandas\util\testing.py", line 2685, in setTZ
    time.tzset()
AttributeError: module 'time' has no attribute 'tzset'

----------------------------------------------------------------------
Ran 140 tests in 1.374s

FAILED (SKIP=1, errors=1)
```

so appears `time.tzset()` doesn't exist on windows (or maybe is called something else). If you can figure this out would be great.

If you can then we need to do a platform check inside the context manager and raise a Skiptest (though maybe just have the context manager raise something helpful and let the test writer worry about the skip)

@jreback Its green. How does it look now?

thanks!
BUG: Series/Index results in datetime/timedelta incorrectly if inputs are all nan/nat like
State: closed

- [x] closes #13467  
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13477?src=pr) is **84.33%**

> Merging [#13477](https://codecov.io/gh/pydata/pandas/pull/13477?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will not change coverage

``` diff
@@             master     #13477   diff @@
==========================================
  Files           138        138          
  Lines         51092      51092          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43087      43087          
  Misses         8005       8005          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [c2cc68d...a017a8c](https://codecov.io/gh/pydata/pandas/compare/c2cc68d6eb233bc74c6bd032650704030c4b9a9d...a017a8c2f84ecfdd8243e2a4a3215a6ca7c43af3?src=pr)

@sinhrks puzzled why this fixed things as all you did was move something

ATM, it doesn't affect to the behavior. However, I found some inconsistency when I adding the test. Updated in #13467.

ok as I commented on the other issue, I think all -> NaT/nan should be M8.

@sinhrks ready to merge? or is there still an open question on this

@jreback Updated based on #13467, and found that there is a test which expect `np.timedelta('nat')` is regarded as `timedelta` dtype (I think it's natural)
- https://github.com/pydata/pandas/blob/master/pandas/tests/series/test_quantile.py#L42

How about skipping only `np.nan` and `pd.NaT` in `infer_dtype`? I mean:
- `[pd.NaT, np.datetime64('nat')]` -> datetime
- `[pd.NaT, np.timedelta64('nat')]` -> timedelta
- `[np.datetime64('nat'), np.timedelta64('nat')]` -> mixed (object)

yeah that looks right

the problem is we can't know definitively the inferred type if we only have pd.NaT (or just np.nan for that matter)

so I suppose object is correct in those cases (though might break things)

Yeah. `pd.NaT` should be ragarded as `datetime` in all nan-like case, if not specified otherwise (no `np.datetime` or `np.timedelta`).
- `[pd.NaT, np.nan]` -> datetime (it results in `timedelta` on current master, NG)
- `[np.nan, pd.NaT]` -> datetime (OK on current master)
- `[pd.NaT, pd.NaT]` -> datetime

Updated based on the above discussion.

@sinhrks looks really good. esp all of the added tests! just a couple of very minor questions. ping when ready.

Now all green.

IIRC this was green. but rebase and ping.

@jreback @jorisvandenbossche rebased, and all green.

@sinhrks Thanks!
BUG: fix to_datetime to handle int16 and int8
State: closed

- [x] closes #13451 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixes #13451


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
tests!

@jreback Hi, I am a bit confused about which file should the tests be written in? There is no separate file that contains tests for pandas.tseries.tools .

https://github.com/pydata/pandas/blob/master/pandas/tseries/tests/test_timeseries.py#L2453 or you can make another function in the same class

lgtm (minor comment), pls add a whatsnew entry. ping when green.

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13464?src=pr) is **84.32%**

> Merging [#13464](https://codecov.io/gh/pydata/pandas/pull/13464?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13464   diff @@
==========================================
  Files           138        138          
  Lines         51069      51072     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43066      43069     +3   
  Misses         8003       8003          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [013c2ce...dc4944d](https://codecov.io/gh/pydata/pandas/compare/013c2ce760083e5454552ad44bba8d2f44da2caa...dc4944d120755bc61050cd53e22c9b3d8a672a2e)

thanks!
BUG: Fix a bug when using DataFrame.to_records with unicode column names
State: closed

 Fix a bug when using DataFrame.to_records with unicode column names in python 2
- [x] closes #11879
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
tests please

can you rebase / update?

## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/13462?src=pr) is 84.75% (diff: 100%)
> Merging [#13462](https://codecov.io/gh/pandas-dev/pandas/pull/13462?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #13462   diff @@
==========================================
  Files           145        145          
  Lines         51139      51139          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43344      43343     -1   
- Misses         7795       7796     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [136a6fb...39e8226](https://codecov.io/gh/pandas-dev/pandas/compare/136a6fbc3c1b3d57c42577a032752fe644044985...39e82260c1f676bc456bbd9b553a720b6c29be9e?src=pr)
I've resynced my branch but it still fails. Not sure whether it is specifically due to my code though.

can you rebase / update according to comments
closing as stale. if you want to update, pls comment.
I changed the test as requested. Note that i corrected another problem due to the fact that numpy does not allow to specify dtype with unicode field names as list of tuples, but allows it using dictionnaries
pls add a whatsnew for 0.20.0. lgtm. otherwise
can you rebase / update
thanks @AlexisMignon
BUG: Fix for .str.replace with invalid input
State: closed

- [x] closes #13438
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

.str.replace now raises TypeError when replacement is invalid


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Strings
description: String extension data type and string data

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13460?src=pr) is **84.28%**

> Merging [#13460](https://codecov.io/gh/pydata/pandas/pull/13460?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13460   diff @@
==========================================
  Files           138        138          
  Lines         50937      50939     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42934      42936     +2   
  Misses         8003       8003          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [d814f43...d5c3f1b](https://codecov.io/gh/pydata/pandas/compare/d814f433940031029f1ddf0d9abdecdf4ad31dac...d5c3f1b077dd28bfe6f59498610603998f3c796d)

thanks @priyankjain
BLD: use inline macro
State: closed

closes #13448

This built locally (on Windows) for py27 and py35.


Labels:
name: Build
description: Library building on various platforms

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13456?src=pr) is **84.28%**

> Merging [#13456](https://codecov.io/gh/pydata/pandas/pull/13456?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13456   diff @@
==========================================
  Files           138        138          
  Lines         50929      50929          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42925      42926     +1   
+ Misses         8004       8003     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [f752886...22a4781](https://codecov.io/gh/pydata/pandas/compare/f7528866e07048f051f46f621ca76947730c7d32...22a478101457db70e20f8b4ee44067fff203ada8)

thanks @chris-b1 this seems to fix.