BrandonFors/pandas-github-issues huggingface dataset:
BUG: #12624 where Panel.fillna() ignores inplace=True
State: closed

- [x] closes #12624 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixes #12624

Author: Pedro M Duarte pmd323@gmail.com


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
xref #12633 

duped in #12633 

thanks!

Thanks for the feedback, Jeff! I wanted to get home to fix it but seems like it has been taken care of.  Will look for another easy one I can try to help with. 

@PedroMDuarte np. this actually happens pretty rarely when 2 people are working on the same issue. In any event lots of issues to try for!
BUG: .rename* not treating Series as mapping
State: closed

Closes https://github.com/pydata/pandas/issues/12623

I added com.is_dict_like in https://github.com/pydata/pandas/pull/11980
and failed to use it for the `rename` method. Using that now and did
some refactoring while I was in there. Added more tests for rename.


Labels:
name: Bug

Comments:
I might remove or rename `core._is_scalar_or_list_like`. It doesn't seem like a well-defined or clear check...

@TomAugspurger btw, make sure you are rebased on current master. `dateutil=2.5.0` flowed thru and broke some tests and I just fixed in (maybe  2 hrs ago)

Ok, the tests for the different possible arguments are [here](https://github.com/pydata/pandas/pull/12626/files#diff-f042fb8fba5656b3854ad92563ed6480R93). I simplified the stuff with `is_scalar_or_mapping` to just do the check in the code. Should be clearer now, but a bit less DRY.

thanks @TomAugspurger
TST: Holiday tests/docs
State: closed

```
>>> from pandas.tseries.holiday import next_monday_or_tuesday
>>> from pandas.tseries.holiday import AbstractHolidayCalendar
>>>from pandas.tseries.holiday import Holiday
>>>class WrongBoxingDay(AbstractHolidayCalendar):
           rules = [
               Holiday('xmas', month=12, day=25, observance=next_monday_or_tuesday),
               Holiday('boxing day', month=12, day=26, observance=next_monday_or_tuesday),
               ]
>>>import pandas as pd
>>>cal = WrongBoxingDay()
>>>cal.holidays(pd.datetime(2016,12,1), pd.datetime(2016,12,31))
DatetimeIndex(['2016-12-27', '2016-12-27'], dtype='datetime64[ns]', freq=None, tz=None)
```

The next_monday_or_tuesday function in holiday module isn't doing what it is supposed to do.
In the example above, xmas should move to Dec 27 while boxing day should stay on Dec 26, but it was also moved to Dec 27.

This bug still exits in the latest version.

How my commit works is explained in the docstring of the function.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Docs
name: Datetime
description: Datetime data dtype

Comments:
As described in the doc, `next_monday_or_tuesday` moves both Sunday (Dec 25) and Monday (Dec 26) to Tuesday. The correct `Calendar` definition should be:

```
class WrongBoxingDay(AbstractHolidayCalendar):
    rules = [
               Holiday('xmas', month=12, day=25, observance=next_monday),
               Holiday('boxing day', month=12, day=26, observance=next_monday_or_tuesday),
               ]
cal.holidays(pd.datetime(2016,12,1), pd.datetime(2016,12,31))
# DatetimeIndex(['2016-12-26', '2016-12-27'], dtype='datetime64[ns]', freq=None)
```

Appreciated if you improve the doc.

Thanks @sinhrks. I see your definition doesn't cause both xmas and boxing day moving to tuesday. Maybe my commit just a bit less confusing (maybe only to myself :) ) as when you see two adjacent holidays you just need to assign next_monday_or_tuesday to both holidays.

@enfeizhan the `Holiday` class is missing lots of tests for things like this. So going to reprupose this PR for tests and/or doc updates. Behavior should remain unchanged.

pls rebase if you'd like to rebase / update and add some tests
BUG: Mixed period cannot be displayed with ValueError
State: closed

- [x] closes #xxxx (not reported)
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

```
# we can create Series contains Periods with mixed freq
s = pd.Series([pd.Period('2011-01', freq='M'), pd.Period('2011-02-01', freq='D')])

# print the created Series raises ValueError
print(s)
# ValueError: Input has different freq=D from PeriodIndex(freq=M)
```


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Period
description: Period data type

Comments:
I would change the exception that `PeriodIndex` raises to maybe: `IncompatibileFrequency(ValueError)` and catch that. Its a more informative error I think. (its in `src/periods.pyx/extract_ordinals`)

## [Current coverage](https://codecov.io/github/pydata/pandas?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2) is `88.64%`

> Merging **#12615** into **master** will not affect coverage as of [`b12fd75`](https://codecov.io/github/pydata/pandas/commit/b12fd75bfd2b583116e754970f4d6c6b2f325ad2)

``` diff
@@            master   #12615   diff @@
=======================================
  Files          275      275       
  Stmts       133688   133716    +28
  Branches         0        0       
  Methods          0        0       
=======================================
+ Hit         118511   118539    +28
  Partial          0        0       
  Missed       15177    15177       
```

> Review entire [Coverage Diff](https://codecov.io/github/pydata/pandas/compare/f71537ab2561ab5727008095e9685966619fa7b9...b12fd75bfd2b583116e754970f4d6c6b2f325ad2) as of [`b12fd75`](https://codecov.io/github/pydata/pandas/commit/b12fd75bfd2b583116e754970f4d6c6b2f325ad2)

---

### [Uncovered Suggestions](https://codecov.io/github/pydata/pandas/features/suggestions?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2)
1. `+0.07%` via [...s/io/sas/sas7bdat.py#30...121](https://codecov.io/github/pydata/pandas/pandas/io/sas/sas7bdat.py?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2#30) 
2. `+0.03%` via [...s/io/sas/sas7bdat.py#531...569](https://codecov.io/github/pydata/pandas/pandas/io/sas/sas7bdat.py?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2#531) 
3. `+0.03%` via [...as/tools/plotting.py#633...664](https://codecov.io/github/pydata/pandas/pandas/tools/plotting.py?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2#633) 
4. _[See 7 more...](https://codecov.io/github/pydata/pandas/features/suggestions?ref=b12fd75bfd2b583116e754970f4d6c6b2f325ad2)_

> Powered by [Codecov](https://codecov.io). Updated on successful CI builds.

OK, changed to raise `IncompatibleFrequency` and fixed corresponding tests.

lgtm. let me know if you want to change the import now or later.

lgtm. ping on green

thanks @sinhrks
COMPAT: dateutil 2.5.0 bug fix
State: closed

COMPAT: test with latest dateutil via pip
COMPAT: dateutil 2.5.0 bugfix causes some tests to fail, skip for now

xref #12611


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
cc @neirbowj 

maybe we should just raise on this version, this is a very subtle bug which they should have tested (and which used to work). If they put out a fix before next release of pandas then I think we will.
BUG: ensure Series.name is hashable, #12610 
State: closed

- [x] closes #12610
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`

pandas >= 0.18.0 will no longer support compatibility with Python version 2.6 and since Python 2.6 we can use the abstract base class collections.Hashable.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
we already have an pandas.core.common.is_hashable just use this

but this needs to be checked more generally (higher up in the constructor)

prob need a few more tests 

ok.

no reason to close
just update push again

actually I realize that the ref issue #9193 was not merged.

can you rebase/update this. Then add the `.name` attribute like was one there?

(this covers setting as well).

further for the actual tests

lets test both the constructor and setting with:
- a bunch of invalid things (you did some already: dict, np.array, list)
- a bunch of valid things (int, float, string, datetime, tuple)

Okay, I will try to do this soon.
Additional test - not problem.

add `.name` atribute - you mean about:

```
@property
def name(self):
    return self._name

@name.setter
def name(self, value):
    if is_hashable(value):
```

P.S. Not a topic of this conversation: yesterday I add to wiki translation about Pandas on Ukrainian https://uk.wikipedia.org/wiki/Pandas -  think it's useful.

Yes, if you take pretty that other PR and incorporate would be great. You can prob remove the explicit check in the constructor as it will use `.name` to set the name as well (where all of the checking should actually happen).

Thanks for the wiki update! cool!

I think you might need to change: https://github.com/pydata/pandas/blob/master/pandas/core/series.py#L235 as well (and look and see if anywhere else is doing something similar)

thanks @chinskiy keep em coming!
added pd as namespace
State: closed

- [x ] closes https://github.com/pydata/pandas/pull/12268#discussion_r55924987

This is the msg:
more consistent with code in the rest of the document

added more namespace

Generally, also numpy's randn should used as
np.random.randn

isolated namespace changes

Note:
I hope I got the rebase & merge right.


Labels:
name: Docs

Comments:
@dacoex no, @sinhrks is asking if you can make that change. should be straightforward.

Ah, OK. will do.

pls build the docs. This makes most things actually fail as there are lots of additional imports that are needed (or `pd.`) pre-pended

@dacoex can you update

Sorry, I'd been busy with some other items off the coding domain.
What shall I update?
Reference:
https://github.com/pydata/pandas/pull/12608#issuecomment-204411671

can you rebase and update

can you rebase / update?

pls reopen when you can update
Chooses eval engine automatically depending on whether numexpr is installed
State: closed

Fixes #12008


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
can you add some tests for this (e.g. the example from the issue). 
The 2.7_LOCALE build (the 2nd one down on the travis builds), does NOT have numexpr installed (its the only one). So if you mark this tests as slow it will get run there (and should pass with the python parser) and on the other builds (don't mark as slow) with the numexpr engine.

So the tests should assert the state of `_NUMEXPR_INSTALLED` (IOW whichever you are actually testsing), then run the query. I don't think there is an easy way to figure out which engine is actually used (we may have to do some gymnastics here to assert that)

this needs a whatsnew note, put in API changes (even though it is more of a bug-fix)

Which whatsnew file should it be put in? 0.18.1?

Also, I'm a little confused about the tests: surely the test is either marked as slow or not marked as slow? I assume the tests should go in `test_query_eval.py`, but having looked at the tests in there I would have thought they would have caught this bug previously...

I'm happy to have a go at writing a test, with a bit of help, but I may not be the best person to do this - and someone else may be able to do it in a tiny fraction of the time.

0.18.1

you need 2 tests the same 1 slow and 1 not-slow (don't mark it)
they will be run in separate builds

replaced by https://github.com/pydata/pandas/commit/504ad4641e5b3a87b655da5dd31840de12495e97

but #12867 is still relevant (e.g. we may want to show a `PerformanceWarning` when `numexpr` is not installed
CLN GH10559 Rename sheetname variable to sheet_name for consistency
State: closed

- [x] closes #10559
- [x] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: IO Excel
description: read_excel, to_excel

Comments:
reason you closed this?

Hi jreback, I did not mean to close the pull request. This is my first attempt at contributing and I'm going through some growing pains. I realized as soon as I submitted the request that I had accidentally changed the keyword arguments instead of keeping them as sheetname. I will make another attempt at fixing the issue if that's okay. 

contributing docs are [here](http://pandas.pydata.org/pandas-docs/stable/contributing.html)

pls reopen if you like to rebase / update

Hi @jreback, sorry for going M.I.A. on this. I want to fix this issue, but I have a few questions. When you say sheetname=None, do you mean you want to change sheetname=0 to sheetname=None? If this is the case, my plan is to change these arguments in read_excel() and _parse_excel() while also adding a deprecation warning. Please let me know your thoughts.

so you rename the arg as you did, then add the deprecated kwarg at the end, e.g. `sheetname=None`, so then you can pick it up both if its passed positionally (e.g. `sheet_name` will be set), then you can issue a warning if the `sheetname` kwarg is set you can issue a warning 

@jreback Hi cloned the fork and created a dev environment by using python setup.py develop. When I made the necessary changes, I noticed that all of the tests still passed when they shouldn't have. I read the docs again to see if I missed anything, and asked on IRC, but we couldn't come up with a solution. Is there something I need to be doing with nosetests in order to run a similar test suite to the one on Travis?

Thanks,
Mandeep
BUG: Matched Round Signature with NumPy's
State: closed

Closes #12600 by re-accepting the `out` parameter as a `**kwargs` argument.  Also removes the `out` parameter from the signature for `DataFrame.round` and replaces it with `**kwargs` like its `Series` counterpart.  Furthermore, since `out` is a parameter that is not supported in `pandas`, checks that `out` is `None` and throws an error if it isn't.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
@gfyoung when we deal with this issue. can you do a search on current numpy as see what functions `_wrap_it` is called for? then let's setup a test in pandas for the same. (could be another issue/PR)

Yep, sounds good to me as a follow-up PR.

@jreback: Any updates on this PR?  I've been running tests and diff-ing locally to make sure that Travis should be happy once I remove the `[ci skip]` tag in my commit message.

i c you have removed dependene on #12413 great

Added the validator for `out`, and Travis is passing.  Should be good to merge then.

ok, looks good.

ping when green (on your travis run is fine).

@jreback : Travis has finally given the green light.  Ready to merge if there's nothing else to change.

ok, looks good. will merge in a bit.

thanks @gfyoung for the patience!

@jreback : No worries.  Better to get it right the first time. :smile:
Include index names in column names of HTML output of DataFrame Style…
State: closed

Include index names in column names of HTML output of DataFrame Styler, if the index names are available (previous behaviour is not to include index column names and instead leave blank spaces)


Labels:
name: Duplicate Report
description: Duplicate issue or pull request
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks! That was fixed in https://github.com/pydata/pandas/pull/12090

Still quite a few other issues around `.style` if this is an area that interests you

https://github.com/pydata/pandas/issues/11610
https://github.com/pydata/pandas/issues?q=is%3Aopen+is%3Aissue+label%3AStyle

Sorry, I hadn't seen the fix - I must be working off an old version...  Cheers!
DOC: clean up some doc-build warnings
State: closed


Labels:
name: Docs

Comments:
only big ones left are these

```
/Users/jreback/pandas/doc/source/generated/pandas.Series.str.partition.rst:6: WARNING: don't know which module to import for autodocumenting u'Series.str.partition' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)
/Users/jreback/pandas/doc/source/generated/pandas.Series.str.repeat.rst:6: WARNING: don't know which module to import for autodocumenting u'Series.str.repeat' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)
```

and one on `groupby.nth` I can't seem to figure out...

On the `groupby.nth`, I also already looked at before, but didn't spot the error either

merged via https://github.com/pydata/pandas/commit/fcab956d41ce74a927f30b3723f0fdfde7df75b6
BUG: value_count(normalize=True, dropna=True) counting missing in denominator, #12558 
State: closed

Closes #12558 


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
#12566 fixes this too, will need to fix groupby. Lemme know if you wanna work on this and I'll close mine. 

@kawochen Oops! Sorry, did not mean to double up. If you have this covered I can close -- not sure what you mean about groupby. 

`groupby` has a version of `value_counts`.  I fixed it for `numpy` > 1.8 and have an idea for older versions.  Lemme play around with it tonight (EST).  If it doesn't work I'll close it and let you have fun :)

@kawochen Sounds good! Just let me know.
ENH: allow categoricals in msgpack
State: closed

Supercedes #12191.

I've made a best-effort to rebase this against master.

It seems to work here.


Labels:
name: Enhancement
name: Categorical
description: Categorical Data Type

Comments:
- add a note in whatsnew/0.18.1

> add a note in whatsnew/0.18.1

Done. Please let me know if it is to taste.

ok #12574  has been merged. pls rebase/update.

Rebased.

Wow, I think I managed to clear all of your comments. Everything suddenly made sense in the end. Please let me know if I missed something and if I should squash or reword my commits.

I broke the NDFrame tests. Investigating.

@pwaller can you rebase/update

this looks pretty good code wise. can you squash everything. 

cc @kawochen 

Squashed.

Rebased and comments addressed.

I'm getting a test failure I'm not sure how to address and I don't have much time to look into it:

```
======================================================================
ERROR: test_basic_frame (pandas.io.tests.test_packers.TestNDFrame)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pydata/pandas/pandas/io/tests/test_packers.py", line 410, in test_basic_frame
    i_rec = self.encode_decode(i)
  File "/home/travis/build/pydata/pandas/pandas/io/tests/test_packers.py", line 75, in encode_decode
    return read_msgpack(p, **kwargs)
  File "/home/travis/build/pydata/pandas/pandas/io/packers.py", line 203, in read_msgpack
    return read(fh)
  File "/home/travis/build/pydata/pandas/pandas/io/packers.py", line 188, in read
    l = list(unpack(fh, encoding=encoding, **kwargs))
  File "_unpacker.pyx", line 459, in pandas.msgpack._unpacker.Unpacker.__next__ (pandas/msgpack/_unpacker.cpp:4709)
  File "_unpacker.pyx", line 390, in pandas.msgpack._unpacker.Unpacker._unpack (pandas/msgpack/_unpacker.cpp:3843)
  File "/home/travis/build/pydata/pandas/pandas/io/packers.py", line 639, in decode
    blocks = [create_block(b) for b in obj[u'blocks']]
  File "/home/travis/build/pydata/pandas/pandas/io/packers.py", line 637, in create_block
    dtype=b[u'dtype'])
  File "/home/travis/build/pydata/pandas/pandas/core/internals.py", line 2518, in make_block
    return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
  File "/home/travis/build/pydata/pandas/pandas/core/internals.py", line 1903, in __init__
    placement=placement, **kwargs)
  File "/home/travis/build/pydata/pandas/pandas/core/internals.py", line 1314, in __init__
    raise TypeError("values must be {0}".format(self._holder.__name__))
TypeError: values must be Categorical
```

Updated again. Thanks for your patience.

To fix that error

```
diff --git a/pandas/io/packers.py b/pandas/io/packers.py
index ba38c8e..f009793 100644
--- a/pandas/io/packers.py
+++ b/pandas/io/packers.py
@@ -266,7 +266,7 @@ def convert(values):

     # convert object
     if is_categorical_dtype(values):
-        return v
+        return values

     if dtype == np.object_:
         return v.tolist()
```

Forced again.

ok looks good. ping when green.

thanks!
CI: add osx to travis, xref #8630
State: closed

CI: remove numpy_dev 2.7 build
CI: remove extraneous requirements


Labels:
name: CI
description: Continuous Integration

Comments:
@jorisvandenbossche note that this fixed the problem of having `statsmodels` removed after the build (and `xarray` as well), which was a problem in the doc-build

[this](https://github.com/jreback/pandas/commit/b355c774f503edcf6c11d5584e84f18e40cd43b9) fixed the cython issues (wasn't building) in enhancedperf.rst.

note that technically we have a minimum requirement on ipython/cython for doc-building (e.g. whenever `Cython` replaced `cythongmagic`)
BUG: GH12558 where nulls contributed to normalized value_counts
State: closed

- [x] closes #12558
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
@nickeubank tests passed. Does this look OK?

done

Looks great!

thanks @kawochen @nickeubank
Update asv config + fix some broken benchmarks
State: closed

- Enable platform-dependent config in asv (needs asv git version for it to do something)
- Enable wheel cache in asv (in asv git version)
- Fix a few easily fixed broken benchmarks


Labels:
name: Performance
description: Memory or execution speed performance

Comments:
can you rebase this. we can prob throw this in 0.18.0; it doesn't affect user code.

Can you run vs 0.17.1 and post anything interesting?

Rebased.
Don't have time to run the full bench suite right now, but can be done via
`asv continuous 6c30cbecf8 master`

benchmarks: https://gist.github.com/pv/8ff97c3f6faa62fd6a5e --- not sure if all of those are real, or fluctuations (laptop under load)

I think we need to update the perf docs in contributing?

Not sure --- they already instruct to install the asv git version.

Added explanation on how to choose between conda / virtualenv / existing environment to contributer docs.

thanks @pv 

pls check the docs when built (prob a few hours) and if necessary issue a follow-up

should these be ignored as well?

```
[jreback-~/pandas] git st
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        asv_bench/.nojekyll
        asv_bench/results.orig/
        asv_bench/run_versions.sh

nothing added to commit but untracked files present (use "git add" to track)
```

sorry....these look like mine. nvm!