BrandonFors/pandas-github-issues huggingface dataset:
ERR: MultiIndex searchsorted
State: closed

 - [ ] closes #14833 
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: MultiIndex

Comments:
@ShaharBental on the issue I indicated that this should simply raise a ``NotImplementedError`` (or maybe a ``ValueError``). I don't think this is actually useful as the indexing methods on a ``MultiIndex`` already do this (they use ``searchsorted`` internally in certain cases).
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14990?src=pr) is 84.66% (diff: 100%)
> Merging [#14990](https://codecov.io/gh/pandas-dev/pandas/pull/14990?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14990   diff @@
==========================================
  Files           144        144          
  Lines         51056      51072    +16   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43225      43241    +16   
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [aba7d25...d22e816](https://codecov.io/gh/pandas-dev/pandas/compare/aba7d255a165bbc221be106839c59a876655c0c2...d22e816034416a1e3ba314b9629c40027a4d0905?src=pr)
@jreback Sorry, changed it to simply raise a NotImplementedError
I think it should be NotImplementedError and not ValueError, as the problem lies with the type of self and not any specific value.
pls add a note in other api changes in 0.20.0
closing as stale. please ping if you'd like to continue.
MAINT: Abstract index helpers in pxi.in file
State: closed

Title is self-explanatory.  xref #14937.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14989?src=pr) is 84.66% (diff: 100%)
> Merging [#14989](https://codecov.io/gh/pandas-dev/pandas/pull/14989?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14989   diff @@
==========================================
  Files           144        144          
  Lines         51056      51056          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43225      43225          
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [aba7d25...6711b97](https://codecov.io/gh/pandas-dev/pandas/compare/aba7d255a165bbc221be106839c59a876655c0c2...6711b9771a48419689a8fc77ba0b47aaecc67430?src=pr)
thanks!
CLN: used np.__version__ and removed instantiation
State: closed

@rkern and others [recommend](http://stackoverflow.com/questions/1520234/how-to-check-which-version-of-numpy-im-using) using `np.__version__` instead of `np.version`. However I realize this is pandas "internals" so there may be some reason for using `np.version` over `np.__version__`. I could not find any such reason. I also removed  a duplicate `LooseVersion(_np_version)` and reused the existing `_nlv` instance.

 - [x] doesn't close any issue, that I'm aware of
 - [x] I ran `nosetests .\pandas\tests\test_compat.py` and 5 tests passed
 - [x] couldn't get the `git` command to work, but `flake8 .\compat\numpy\__init__.py` passes fine
 - [x] whatsnew entry should be covered by `several new features, enhancements, and performance improvements` in the v0.20.0.txt file


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Clean

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14988?src=pr) is 84.66% (diff: 100%)
> Merging [#14988](https://codecov.io/gh/pandas-dev/pandas/pull/14988?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will not change coverage

```diff
@@             master     #14988   diff @@
==========================================
  Files           144        144          
  Lines         51056      51056          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
  Hits          43225      43225          
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [aba7d25...f56a7ff](https://codecov.io/gh/pandas-dev/pandas/compare/aba7d255a165bbc221be106839c59a876655c0c2...f56a7ff4099ea4acfec7c3cb61b7bb11dd0ffe1d?src=pr)
@john-a-m that's a really old link, but in any event, ``__version__`` is usually the recommended way. so thank you.
ENH: Added 'sum' to the set of statistics returned by df.field.describe()
State: closed

I've added ``sum`` to the set of statistics returned by the ``describe`` method.

I believe it should be added because in my experience, the sum is just as commonly needed as the other values already returned by ``describe``. I have often found myself writing an additional line of code after ``describe`` to calculate the sum of the same series. Adding sum to the mix would remove that requirement and make life easier for myself and others.

I believe I am not alone. For instance, ``sum`` is already included in the descriptive statistics methods of other programs, like Microsoft Excel.

 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: Enhancement
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
@palewire See discussion in https://github.com/pandas-dev/pandas/issues/14483
Thanks for the link. 

The discussion I see there seems like an argument for completely deprecating describe. 

I can respect that, but if you are going to keep it in the project, why not expand it to include common aggregations like sum, which is prominently featured in the examples for the agg expansion? Having both seems like a road to confusion for users.
There are no plans to deprecate `describe`. And I also do not see the potential confusion. The addition of `agg` should be looked at like this (apart from other reasons to have `agg`):

- `describe` gives you a default set of statistics
- if you want a custom set of statistics, the `agg` method let's you rather easily create a function to do this

But of course, there can still be discussion about what the default set of statistics should be, and whether this should include a sum or not. But, what I just want to say is that, for me, this discussion is separate from the introduction of `agg`. 

There are some reasons listed in #14483 to not include sum in `describe`. If you have additional arguments to include it (eg the other programs you mention, although R is an example that does not include sum in its summary), please add it to that issues (to keep the discussion in a central place)
@palewire 

closing this, but if you would like to raise an issue for discussion (xref the previous issues). with a convincing argument that would be fine.
BUG: Avoid flaky usecols set in C engine
State: closed

Explanation of the bug can be found <a href="https://github.com/pandas-dev/pandas/issues/14792#issuecomment-269115098">here</a>.

Closes #14792.


Labels:
name: Bug
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14984?src=pr) is 84.78% (diff: 100%)
> Merging [#14984](https://codecov.io/gh/pandas-dev/pandas/pull/14984?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14984   diff @@
==========================================
  Files           145        145          
  Lines         51090      51094     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43315      43319     +4   
  Misses         7775       7775          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [7f0eefc...82cf55b](https://codecov.io/gh/pandas-dev/pandas/compare/7f0eefc2f81d6b34f252a5da38d31e515c295106...82cf55b1bd1ba8ec4f29a36458bd75aa361810e3?src=pr)
@jreback , @jorisvandenbossche : Addressed comments, and everything is green.
ok thanks @gfyoung
Fixed power operation on indicies
State: closed

Fixed power operation on indicies to allow for power on rows. See #14973 

 - [ ] closes #14973
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14981?src=pr) is 84.66% (diff: 100%)
> Merging [#14981](https://codecov.io/gh/pandas-dev/pandas/pull/14981?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14981   diff @@
==========================================
  Files           144        144          
  Lines         51056      51057     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43225      43226     +1   
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [aba7d25...6c793ed](https://codecov.io/gh/pandas-dev/pandas/compare/aba7d255a165bbc221be106839c59a876655c0c2...6c793ed8147e230eb3859828e44b31d6feeb444c?src=pr)
this needs a test & a whatsnew note in 0.20.0
Will work on it!
Closed this pull request and updated it with #14996
there is no reason to open another PR for the same issue. you just push to the same one. fine that you already did this.
DOC: Adding repository pandas_exercises to tutorials.rst
State: closed


Labels:
name: Docs

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14980?src=pr) is 84.66% (diff: 100%)
> Merging [#14980](https://codecov.io/gh/pandas-dev/pandas/pull/14980?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14980   diff @@
==========================================
  Files           144        144          
  Lines         51047      51056     +9   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43216      43226    +10   
+ Misses         7831       7830     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [d7e8f31...6985079](https://codecov.io/gh/pandas-dev/pandas/compare/d7e8f316dc7155252fa47ea23a3074d7d0197e54...6985079f3c083e860e7cc938c07d00001a7e3fae?src=pr)
@guipsamora Seems like a nice repo of exercises! 
BTW, if you are interested in a scriptable way to automatically create the versions with and without solutions, you are welcome to check out https://github.com/jorisvandenbossche/nbtutor
BUG: applymap on empty DataFrame returns Series (#8222)
State: closed

 - [x] closes #8222
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry for 0.20.0

Within `applymap`, the data is converted into a numpy array with `asobject` [here](https://github.com/pandas-dev/pandas/blob/master/pandas/core/frame.py#L4291) and evaluated with the `apply` method `_apply_empty_result` since this frame is empty. The numpy array is compared to an `Series` [here](https://github.com/pandas-dev/pandas/blob/master/pandas/core/frame.py#L4112) which proceeds to get returned as a `Series` [here](https://github.com/pandas-dev/pandas/blob/master/pandas/core/frame.py#L4118). 

Found that passing in the data without using `asobject` allows the logic to return a copy of the empty dataframe [here](https://github.com/pandas-dev/pandas/blob/master/pandas/core/frame.py#L4120).


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: API Design

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14977?src=pr) is 84.65% (diff: 100%)
> Merging [#14977](https://codecov.io/gh/pandas-dev/pandas/pull/14977?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14977   diff @@
==========================================
  Files           144        144          
  Lines         51047      51049     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43216      43218     +2   
  Misses         7831       7831          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [d7e8f31...255db8e](https://codecov.io/gh/pandas-dev/pandas/compare/d7e8f316dc7155252fa47ea23a3074d7d0197e54...255db8e8e65ccb51497379dea12f318fa503da89?src=pr)
Changes made and all green. Thanks!
thanks!
[Depr] raise_on_error kwarg with errors kwarg in astype#14878
State: closed

 - [x] closes #14878 
 - [x] tests passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry

Please check that the entry in `whatsnew/v0.20.0.txt`. Unsure that the update was in the 
`_whatsnew_0200.deprecations` or `_whatsnew_0200.prior_deprecations` so put it in the
former.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14967?src=pr) is 84.75% (diff: 100%)
> Merging [#14967](https://codecov.io/gh/pandas-dev/pandas/pull/14967?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **0.10%**

```diff
@@             master     #14967   diff @@
==========================================
  Files           144        145     +1   
  Lines         51030      51144   +114   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43198      43348   +150   
+ Misses         7832       7796    -36   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8b497e4...0ae8550](https://codecov.io/gh/pandas-dev/pandas/compare/8b497e49387ad167acb1c8ae31d67676e30add5e...0ae855067cd8d5d42282ff0956fb774f86f80c8f?src=pr)
Update after review
@m-charlton Can you also add a test confirming that the old keyword still works but gives a deprecation warning?

You can do this with:

```
        with tm.assert_produces_warning(FutureWarning):
            res = df.astype(.., raise_on_error=False)
        exp = df.astype(.., errors='ignore')
```
xref https://github.com/pandas-dev/pandas/pull/14877 we should try to clarify that the `errors` keyword only is meant for errors during the type conversion. Maybe we can change the "invalid input" to something else? Eg "invalid data for the provided dtype"?
Back from Christmas/New Year break. Will makes changes today and push for review
minor test change. ping on green.
@m-charlton Thanks a lot!
ERR: raise on missing values in pd.pivot_table
State: closed

 - [x] closes #14938
 - [x] tests added - for #14938
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14965?src=pr) is 84.66% (diff: 66.66%)
> Merging [#14965](https://codecov.io/gh/pandas-dev/pandas/pull/14965?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14965   diff @@
==========================================
  Files           144        144          
  Lines         51043      51046     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43214      43216     +2   
- Misses         7829       7830     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [45910ae...ff4df5a](https://codecov.io/gh/pandas-dev/pandas/compare/45910ae646eba417120dd2bd4ada68a018c97b32...ff4df5af143f2c8b4449ec9168789b639cf10bc4?src=pr)
very minor comments. ping on green.
@jreback Changes made as you requested and all green.
thanks!
DOC :Cheatsheet update. 
State: closed

Update to cheatsheet, from suggestions from a colleague. 

1. Fix bug in example for outer join
2. Add head() and tail() and use space for nsmallest()
3. Formatting of logic table.


Labels:
name: Docs

Comments:
@jreback fixed a few issues in the CheatSheet, so this version (the pptx and pdf) can just be copied over.
@Dr-Irv thanks. I merged. In the future do this.


```
git checkout master
git pull
git checkout -b <newbranch>
```
then you will have a new branch that *starts* with master
BUG: Series.ffill() with mixed dtypes containing tz-aware datetimes f…
State: closed

…ails. (GH14956)

Seems to work with all the datetime classes usually encountered, although 'tz' seems to be the idiom in the codebase (not sure why?). If both need to be supported I can replace `getattr(val, 'tzinfo', None)` with `getattr(val, 'tz', None) or getattr(val, 'tzinfo', None)`, thus also giving precedence to the former (if available).

(breaking commit was 4de83d25d751d8ca102867b2d46a5547c01d7248)

 - [ x] closes #14956 
 - [x] tests added / passed
 - [x ] passes ``git diff upstream/master | flake8 --diff``
 - [x ] whatsnew entry


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14960?src=pr) is 84.65% (diff: 100%)
> Merging [#14960](https://codecov.io/gh/pandas-dev/pandas/pull/14960?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14960   diff @@
==========================================
  Files           144        144          
  Lines         51043      51043          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43214      43213     -1   
- Misses         7829       7830     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [45910ae...8550ed1](https://codecov.io/gh/pandas-dev/pandas/compare/45910ae646eba417120dd2bd4ada68a018c97b32...8550ed10dc1ef45ed1b120e8ef87df69744d75cf?src=pr)
pls add a test in pandas.types.tests.test_inference for the inference check
and a test in series.tests.test_missing (e.g. the same exact test of the issue you had)

otherwise lgtm.
Added some simple tests and corrected whitespace issue. Please let me know if you'd like some more extensive/different testing.
@grutts lgtm. ping on green.
@jreback all green. Had to make another commit to pass PEP8
thanks!

note I had to add an option to the test_inference test to make it *fail* w/o the change. The importance of testing first before the fix.
TST: matplotlib 2.0 fix in log limits for barplot (GH14808)
State: closed

Closes #14808

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting

Comments:
👍  when the tests pass
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14957?src=pr) is 84.66% (diff: 100%)
> Merging [#14957](https://codecov.io/gh/pandas-dev/pandas/pull/14957?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will increase coverage by **<.01%**

```diff
@@             master     #14957   diff @@
==========================================
  Files           144        144          
  Lines         51030      51047    +17   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43198      43217    +19   
+ Misses         7832       7830     -2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [8b497e4...1f1a80d](https://codecov.io/gh/pandas-dev/pandas/compare/8b497e49387ad167acb1c8ae31d67676e30add5e...1f1a80d61cc1516356e72e950045ca0d55408307?src=pr)
@TomAugspurger had to change it a bit further (the build in allowed failures for matplotlib 2.0cr2 was stll failing). Merged now, but I am wondering if we should actually test those 'implementation' details of matplotlib, and not just that the plot works.
I agree that the test isn't all that useful.
TST: Groupby.groups of datetimeindex (#11442)
State: closed

 - [x] closes #11442
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``

Doesn't look a PR in 0.19.2 and 0.20.0 addressed this, but it works on master 0.19.1

Labels:
name: Bug
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby

Comments:
## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14952?src=pr) is 84.65% (diff: 100%)
> Merging [#14952](https://codecov.io/gh/pandas-dev/pandas/pull/14952?src=pr) into [master](https://codecov.io/gh/pandas-dev/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

```diff
@@             master     #14952   diff @@
==========================================
  Files           144        144          
  Lines         51030      51030          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43199      43198     -1   
- Misses         7831       7832     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [74de478...6c16211](https://codecov.io/gh/pandas-dev/pandas/compare/74de478392e09cf938d244f5990da4e001afc84c...6c1621168af47d32fad745daa1a3d85dbf6988ef?src=pr)
thanks!