BrandonFors/pandas-github-issues huggingface dataset:
CLN: Resubmit of GH14700.  Fixes GH14554.  Errors other than Indexing…
State: closed

…Error and KeyError now bubble up appropriately.

 - [x] closes #14554
 - [ ] tests added / passed (Not required per GH14700 discussion)
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14912?src=pr) is 84.57% (diff: 0.00%)
> Merging [#14912](https://codecov.io/gh/pandas-dev/pandas/pull/14912?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14912   diff @@
==========================================
  Files           144        144          
  Lines         51057      51057          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43180      43180          
  Misses         7877       7877          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f1cfe5b...458c0cc](https://codecov.io/gh/pandas-dev/pandas/compare/f1cfe5b4eb614f9b4e201e764e9bd25f2389b355...458c0cc0c56f8c50af9cbeb823b9d6e3433a396b?src=pr)
so this is segfaulting (with the OP issue) on py3. any thoughts on this
thanks!

note as per my comment above, this (and master and 0.19.1) segfault in python 3. not sure what if anything can do about that (they segfault because of the recursion). any ideas?
BUG: select_dtypes now allows 'datetimetz' for generically selecting datetimes with timezones
State: closed

was missing a generic way to use select_dtypes to select *any* datetime with tz.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14910?src=pr) is 84.57% (diff: 100%)
> Merging [#14910](https://codecov.io/gh/pandas-dev/pandas/pull/14910?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14910   diff @@
==========================================
  Files           144        144          
  Lines         51057      51060     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43180      43183     +3   
  Misses         7877       7877          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [f1cfe5b...2e1b58d](https://codecov.io/gh/pandas-dev/pandas/compare/f1cfe5b4eb614f9b4e201e764e9bd25f2389b355...2e1b58d0a6eb9b76f68e12bfc125fea9f2fca221?src=pr)
TST: Groupby/transform with grouped NaN (#9941)
State: closed

 - [x] closes #9941
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

The original issue works on 0.19.1 as expected (resulting value is `nan`). Doesn't look like this was fixed in a PR for 0.19.2 or 0.20

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14907?src=pr) is 84.56% (diff: 100%)
> Merging [#14907](https://codecov.io/gh/pandas-dev/pandas/pull/14907?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14907   diff @@
==========================================
  Files           144        144          
  Lines         51058      51058          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43177      43176     -1   
- Misses         7881       7882     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e503d40...30c1c70](https://codecov.io/gh/pandas-dev/pandas/compare/e503d40ace473556990e5453ed5b4c9aa96e24ff...30c1c70d5e1e266db7e8a125c1ac4d3cd89d6da8?src=pr)
thanks!
TST: Test timedelta arithmetic (#9396)
State: closed

 - [x] closes #9396
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Original issue was with using `div` with `datetime.timedelta`.  Doesn't appear this was fixed recently.  

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Timedelta
description: Timedelta data type

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14906?src=pr) is 84.56% (diff: 100%)
> Merging [#14906](https://codecov.io/gh/pandas-dev/pandas/pull/14906?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14906   diff @@
==========================================
  Files           144        144          
  Lines         51058      51058          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43177      43176     -1   
- Misses         7881       7882     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e503d40...56c5ba3](https://codecov.io/gh/pandas-dev/pandas/compare/e503d40ace473556990e5453ed5b4c9aa96e24ff...56c5ba37c270c5e2f02932f7c21a1edfe1f4d461?src=pr)
thanks!
Catch warning introduced by GH14432 in test case
State: closed

Follow on to #14432 to catch the newly introduced `FutureWarning` in the `test_groupby_multi_categorical_as_index` test case.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby

Comments:
thanks!

of course I just merged a pr where we moved the tests :)

can you move 
Haha, yeah, I was a bit confused for a minute there. Changes should be moved to new location now.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14902?src=pr) is 84.55% (diff: 100%)
> Merging [#14902](https://codecov.io/gh/pandas-dev/pandas/pull/14902?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14902   diff @@
==========================================
  Files           144        144          
  Lines         51043      51043          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43160      43159     -1   
- Misses         7883       7884     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [73bc6cf...c30fa2b](https://codecov.io/gh/pandas-dev/pandas/compare/73bc6cfb35a886b18955b1f0a07f7647f1f1d409...c30fa2bad158ca6735ea1002b420cc6e9b7aca69?src=pr)
thanks!
Clean up construction of Series with dictionary and datetime index
State: closed

 - [X] closes #14894
 - [X] tests added / passed
 - [X] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry

Fix usage of fast_multiget with index which was always throwing an exception that was then caught; add ASV that show slight improvement

Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance

Comments:
Thx for the PR! can u paste the vbench result?
lgtm. pls add a whatsnew in 0.19.2 (perf section).
asv continuous -f 1.1 upstream/master HEAD -b series_methods.series_constructor_dict_data_datetime_index
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
·· Installing into conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit hash 5f05fdcf:
[  0.00%] ·· Building for conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[  0.00%] ·· Benchmarking conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running ...x.time_series_constructor_no_data_datetime_index      3.21s
[ 50.00%] · For pandas commit hash 3ba2cff9:
[ 50.00%] ·· Building for conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
[ 50.00%] ·· Benchmarking conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· Running ...x.time_series_constructor_no_data_datetime_index      3.78s    before     after       ratio
  [3ba2cff9] [5f05fdcf]
-     3.78s      3.21s      0.85  series_methods.series_constructor_dict_data_datetime_index.time_series_constructor_no_data_datetime_index

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14895?src=pr) is 85.31% (diff: 100%)
> Merging [#14895](https://codecov.io/gh/pandas-dev/pandas/pull/14895?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14895   diff @@
==========================================
  Files           144        144          
  Lines         51016      51015     -1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43522      43522          
+ Misses         7494       7493     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ba2cff...56be091](https://codecov.io/gh/pandas-dev/pandas/compare/3ba2cff9c55cd16b172f9feb09da551990753f3b...56be091356c9df8e42cb25d313b1f28e2c01f1dc?src=pr)
thanks! (I adjusted the asv slightly)
TST: to_json keeps column info with empty dataframe (#7445)
State: closed

 - [x] closes #7445
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Closed by #9805


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
the original issue has sufficient tests AFICT. what does this add?
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14893?src=pr) is 85.31% (diff: 100%)


> No coverage report found for **master** at e7df751.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [e7df751...fe2b110](https://codecov.io/gh/pandas-dev/pandas/compare/e7df7516ff44185ded46f89a463169c997ae0662...fe2b110939c0ca40f7889491ad61ad4abbab483a?src=pr)
Ah, yes, that is maybe true. The original PR has roundtrip tests for empty frames, while this one adds an explicit test of the output format in such a case. So that's maybe slightly useful to add, but in any case, if we keep it, would be good to have them at the same place
Agreed, makes sense to move it to `test_frame_empty`. Doesn't look like this specific case is tested there. I'll make the change later tonight. 
thanks!
DOC: fixed typo
State: closed

 - [ ] closes #xxxx
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry



Labels:
name: Docs

Comments:
@smsaladi thanks

@sinhrks BTW, when you such a PR and label it, you are certainly welcome to just merge directly as well (eg for this one with only small chang in the tutorial docs, you don't necessarily have to wait until travis passes)
ok, i've waited for tavis:)
Add new optional "separator" argument to json_normalize
State: closed

 - [x] closes #14883
 - [x] tests added / passed (added test `test_shallow_nested_with_separator`)
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry (v0.20.0)


Labels:
name: API Design
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
I appreciate all of these comments, and implemented them. And then I found my code doesn't even work. I am doing something clearly wrong here—I thought it was to simply substitute

```python
    meta_keys = [sep.join(val) for val in meta]
```

for

```python
    meta_keys = ['.'.join(val) for val in meta]
```

It's not. That doesn't work. Happy to take any suggestions.
> And then I found my code doesn't even work.

I am not sure that is the reason, but in your last version, it is still `separator.join` instead of `sep.join`
Thanks for the comments, which I've incorporated into a new pull request https://github.com/pandas-dev/pandas/pull/14950 (edit). However, despite y'all pointing out some issues with my code, I am baffled why the simple replace-`.`-with-`sep` doesn't work, happy to take suggestions.

(Feel free to close this pull request whenever it's appropriate to do so; it's supplanted by the new one.)
DOC: Add documentation about cpplint
State: closed

1) Minor spelling and command errors.
2) Add that we also use AppVeyor for testing.
3) With the introduction of `cpplint`, document it and how to use.

Follow-up to #14814.


Labels:
name: Docs

Comments:
@jreback , @jorisvandenbossche : There's a massive backlog on Travis with a good number of old builds (many of which are mine).  Do you think you could cancel them before they start?
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14890?src=pr) is 85.31% (diff: 100%)
> Merging [#14890](https://codecov.io/gh/pandas-dev/pandas/pull/14890?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14890   diff @@
==========================================
  Files           144        144          
  Lines         51006      51016    +10   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43512      43523    +11   
+ Misses         7494       7493     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [033d345...db09105](https://codecov.io/gh/pandas-dev/pandas/compare/033d34596f5327472007f0f15029b86050ee2592...db091058aa4d8f9a150e78153c0738d5c113c1fd?src=pr)
thanks!
BUG: regression in DataFrame.combine_first with integer columns (GH14687)
State: closed

closes #14687


Labels:
name: Bug
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
minor change, lgtm otherwise.
@sinhrks see https://github.com/pandas-dev/pandas/issues/14687. The code I now removed was introduced in https://github.com/pandas-dev/pandas/pull/13970 (see https://github.com/pandas-dev/pandas/pull/13970/files#diff-1e79abbbdd150d4771b91ea60a4e1cc7L3703). 
Can you still remember the reason for the `notnull(series).all()`?
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14886?src=pr) is 85.30% (diff: 100%)
> Merging [#14886](https://codecov.io/gh/pandas-dev/pandas/pull/14886?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14886   diff @@
==========================================
  Files           144        144          
  Lines         51004      51005     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43511      43512     +1   
  Misses         7493       7493          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [5f889a2...7f325cd](https://codecov.io/gh/pandas-dev/pandas/compare/5f889a2106f6584583458e01dbd0f3b9b696fab2...7f325cdef23a7c1ef06a5a6ef0f0f62fbf5b5273?src=pr)
@jorisvandenbossche Thx for the fix.

I think it is to keep caller's dtype if it doesn't have missing. If #14687 example can keep ``int`` dtype, the logic should be unnecessary.
> I think it is to keep caller's dtype if it doesn't have missing.

OK, because there is a ``_possibly_downcast_to_dtype(arr, this_dtype)`` later on, it will be cast to int again if possible, and in this way also preserves the int, AFAIU
TST: Test datetime array assignment with different units (#7492)
State: closed

 - [x] closes #7492
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

The example in this issue currently works in master. Added a test to confirm. Based on a quick skim of the recent PRs, doesn't look like it was fixed recently. 


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Datetime
description: Datetime data dtype

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14884?src=pr) is 85.30% (diff: 100%)


> No coverage report found for **master** at 3ba2cff.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [3ba2cff...56fdc1f](https://codecov.io/gh/pandas-dev/pandas/compare/3ba2cff9c55cd16b172f9feb09da551990753f3b...56fdc1f4b954da0573d4aa3bc5335971a12e603d?src=pr)
Changes made and all green. 
@mroeschke Thanks!
BUG: Respect value of raise_on_error
State: closed

#### Code Sample, a copy-pastable example if possible

Pasteable snippet.
```python
import numpy as np
import pandas as pd
x = pd.DataFrame(data=[[1,2],[3,4]], columns=['A','B'])
x.astype({'C': np.int32}, raise_on_error=False)
```

Output
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tomdr/VirtualEnvs/work-py3/lib/python3.5/site-packages/pandas/core/generic.py", line 3041, in astype
    raise KeyError('Only a column name can be used for the '
KeyError: 'Only a column name can be used for the key in a dtype mappings argument.'
```
#### Problem description

The method argument `raise_on_error` of `DataFrame.astype` suggests there is the option to ignore errors caused by mismatch in column names. Currently, an exception is raised regardless of the value of this argument. This is fixed in this pull request.

Before: https://github.com/pandas-dev/pandas/blob/v0.19.1/pandas/core/generic.py#L3041 

#### Expected Output

An exception should not be raised. 

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.19.1
nose: 1.3.7
pip: 9.0.1
setuptools: 21.0.0
Cython: 0.24.1
numpy: 1.11.2
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.0.0
sphinx: 1.4.8
patsy: None
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.5.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.14
pymysql: 0.7.9.None
psycopg2: 2.6.2 (dt dec pq3 ext lo64)
jinja2: 2.8
boto: None
pandas_datareader: None
</details>


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
I understand the keyword is to ignore dtype conversion error, not input validation. So I think the documentation should be fixed to clarify this. 

If we're ignoring all kinds of errors, you should fix ``KeyError`` raised from ``Series``.
I agree with @sinhrks that the current meaning of the `raise_on_error` keyword is to raise/ignore conversion errors, not errors for column names.

The keyword is renamed in https://github.com/pandas-dev/pandas/pull/14967, so let's making this clearer in the docs there.

@tomderuijter The use case for ignoring keys in the dict that are not present in the column names is also certainly a valid use case. So we could discuss whether we want to support this in a certain way (similar requests came up for `drop` to drop columns I think). Welcome to open a new issue about this to discuss!
DOC: Update to docstring of pd.DataFrame(dtype)
State: closed

 - [ ] closes #14764 
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry

All suggestions to improve this are welcome.

Labels:
name: Docs
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14868?src=pr) is 85.26% (diff: 100%)
> Merging [#14868](https://codecov.io/gh/pandas-dev/pandas/pull/14868?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14868   diff @@
==========================================
  Files           144        144          
  Lines         50980      50980          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43470      43470          
  Misses         7510       7510          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [14e4815...6ee14e7](https://codecov.io/gh/pandas-dev/pandas/compare/14e4815391dcd8c9fe91479fed629410bf63ca33...6ee14e781b3058d36a25dd6d9ca2c29039a2d6c6?src=pr)
Personally, I am not convinced that these examples are needed in the DataFrame docstring

- "Leaving dtype=None in constructor will infer a wider type than necessary" -> these are just the default dtypes, so I would phrase it in any case differently
- "A list/dict/Series/array-like is not allowed." -> We should just clarify in the explanation of the parameter above that "only a single dtype is allowed". Or maybe rephrase the example to just show that you can specify a single dtype.
- "Fix up dtypes after declaration" -> Also, I don't think this is needed here. Could maybe go somewhere in the tutorial docs.

@patniharshit Can you update?
yeah sure
can you rebase / update
I am extremely sorry to say that I won't be able to complete this now. Either someone take this up or feel free to close/delete this.
thanks @patniharshit np problem.
DOC: fix groupby.rst building errors #14861
State: closed

Rename _resample()_ `df` to `df_re` under **New syntax to window and resample operations** section.


Labels:
name: Docs
name: Groupby

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14863?src=pr) is 85.26% (diff: 100%)
> Merging [#14863](https://codecov.io/gh/pandas-dev/pandas/pull/14863?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14863   diff @@
==========================================
  Files           144        144          
  Lines         50980      50980          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43469      43469          
  Misses         7511       7511          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [0c82abe...7282490](https://codecov.io/gh/pandas-dev/pandas/compare/0c82abef8bc8fc7419a3b8823229cafd195595ac...728249016b33f753e04fbcbecda112495b33f897?src=pr)
thanks!