BrandonFors/pandas-github-issues huggingface dataset:
WRN: let + * and - pass thru on boolean with a warning
State: closed

closes #7210


Labels:
name: API Design
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
`not supported` also sounds a bit strange (as this is not clear that your code will actually work). In what regard is it not supported? Or is it just not recommended as it can have performance consequences?

ah right ... that reminds me i actually need to test this with numexpr which was the whole point! numexpr doesn't support boolean arithmetic, in this case that's what I mean when I say not supported

I thought u were going to do a warning, then just translate the symbols and do it anyhow

(e.g. '*' -> '&') etc

ah, I thought you were going to detect this and then just use python evaluation and not numexpr (in that sense my comment above).

What I mean is: `not supported` <-> `but my code will still run` is not clear from the message now.

i see @jreback i misunderstood what you meant by translation...that's actually probably the simplest method

yes I agree with @jorisvandenbossche (that's even better), but I would still do the warning; you can just bail out of the evaluation and 'fallback' to python eval

ok so translate or no? very simple and would keep the performance for those ops (not that they are really used that much)

@cpcloud I think just fallback as @jorisvandenbossche suggests (with a warning)

ok

how about 'operator "BLAH" not supported by numexpr for bool dtype, use "BLARG" instead'?

still not totally clear that it works tho

@jorisvandenbossche ok with the latest message?

ok!

Looks good to me

need to adjust the release notes / v0.14.0.txt to reflect this (and add this issue number too).
go ahead do that and squash and can merge when ready.

```
In [2]: df = DataFrame(dict(A=np.random.randn(50000))).isnull()

In [3]: df*df
pandas/computation/expressions.py:184: UserWarning: evaluating in Python space because the '*' operator is not supported by numexpr for the bool dtype, use '&' instead
  unsupported[op_str]))
Out[3]: 
           A
0      False
1      False
2      False
3      False
4      False
5      False
6      False
7      False
8      False
9      False
10     False
11     False
12     False
13     False
14     False
15     False
16     False
17     False
18     False
19     False
20     False
21     False
22     False
23     False
24     False
```

@jreback good on the rls notes?

yep
BUG: string methods on empty series (GH7241)
State: closed

closes #7241


Labels:
name: Bug
name: Strings
description: String extension data type and string data

Comments:
this will need a release note (though 0.14.1 section is not there yet)

can u squash to a single commit

ok

This is good.

I wonder if for consistency perhaps these should return empty Series/DataFRames with object dtype?

```
In [8]: s.str.findall('a')
Out[8]: Series([], dtype: object)
```

agreed @wabu can u put in tests for all string methods in empty series (their maybe some already existing) and see what breaks if u enforce object dtype

It's not necessarily object for all string methods, I guess it should match the dtype of a non-empty example... for example str.count and str.len returns int64 (actually these could be the only examples).

right those are counting methods so they should be int64 in empty

sounds like the right thing to do, I'll have a look.
I realized some time ago that other methods behave strange with empty series/dfs, but did not have time to look at this (perhaps it's already fixed).

I changed _na_map. as noted in the release notes, now `str_extract` returns series with only nan values as `dtype=object` instead of `float`. I think it's strange if you have object for empty, but float for not-matching strings.

old:

```
>>> pd.Series(['foo']).str.extract('(bar)')
0 NaN
dtype: float
```

new:

```
>>> pd.Series(['foo']).str.extract('(bar)')
0 NaN
dtype: object
```

thanks for putting the empty tests together!

You have a merge conflict (probably in release notes) so please rebase off master.

closed via https://github.com/pydata/pandas/commit/f24f2e81e702ea6ec338cf179be3273f67cb5bbb

(there was another merge conflict on the release notes!)

Thanks!
BUG: grouped hist raises AttributeError with single group
State: closed

Includes 3 minor fixes for `hist`.
- `DataFrame.hist` raises `AttributeError` when the target column specifiedwith `by` kw only contains a single value.

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import pandas.util.testing as tm

n=100
gender = tm.choice(['Male', 'Female'], size=n)
classroom = tm.choice(['A', 'B', 'C'], size=n)
single = tm.choice(['S'], size=n)

df = pd.DataFrame({'gender': gender,
                   'classroom': classroom,
                   'single': single, 
                   'height': np.random.normal(66, 4, size=n),
                   'weight': np.random.normal(161, 32, size=n),
                   'category': np.random.randint(4, size=n)})

df.hist(by='single')
# AttributeError: 'AxesSubplot' object has no attribute 'ravel'
```
- `hist` can accept `rot` kw only when `by` is specified, but `rot` actually does nothing. I understand `rot` value should be used for `xticklabels` rotation if `xrot` is not specified?

```
df.hist(rot=45)
# AttributeError: Unknown property rot
# -> This is OK because we can use xrot kw

df.hist(by='classroom', rot=45)
# It raises no error, but ticks are NOT rotated.
```
- `hist` always displays `xticklabels` on all axes even if specifying `sharex=True`. Other plots only draws `xticklabels` on bottom axes if `sharex` is True.


Labels:
name: Bug
name: Groupby
name: Visualization
description: plotting

Comments:
@TomAugspurger ?

For me, `df.hist(by='classroom', rot=45)` _does_ rotate the xlabels.

I'd say `rot` should be a synonym for `xrot in hist. If you one of them, use that for the`xticklabels`. If you see both raise a`ValueError`. Make sure it's consistent for`hist_frame`,`hist_series`, and`grouped_hist`.

Strange. The latest master seems to output following result, and both `grouped_hist` and `_grouped_plot` doesn't have `rot` logic. I'm not sure on what part xticks can be rotated in current code.

```
axes = df.hist(by='classroom', rot=30)
```

![figure_1](https://cloud.githubusercontent.com/assets/1696302/3137995/521a2034-e867-11e3-9a6c-8d1aec257f9e.png)

Whatever, how about remove `rot` fix from this PR? I only cared to get `grouped_hist` worked assuming it is a special case of `grouped_hist`. To make plot functions consistent, it is better to use same logic as the`plot` and `boxplot` (not only `hist`). I'll open separate issue once organized.

@sinhrks pls rebase

this looks ok, @TomAugspurger ?
BUG: multi-index output formatting is buggy (GH7174)
State: closed

Closes #7174

Adjusts the truncation in the notebook for multiindex dfs. Both rows and columns are affected. 

http://nbviewer.ipython.org/gist/bjonen/492fea9559fd73edf579


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: MultiIndex

Comments:
can u show the results for the test cases
(df and df.T) with mi on an axis

What do you mean by mi on an "axis"? I added mi on both index and columns in the notebook. 

that's what I meant (has at least a multi index) on 1 axis 

@cpcloud @jorisvandenbossche @TomAugspurger ?

Seems like multiindex levels > 2 wasnt as bad as I thought (see updated notebook). I'll add some more tests to the notebook to make sure this is working. 

looking good

bit o' merge conflicts there @bjonen let us know if u need help

looks like on the last 3 level MI there's a row missing in the last column

Thx @cpcloud , fixed it. 

nice. looking pretty good. just squash down and should be good to go. maybe put up a final notebook with all the fixes that we can use as a reference for the issue.

Ok will merge and squash. 

You mean add the notebook from above to the repo? If so, where? 

no i just meant update the notebook, which it looks like you did, so nevermind. :+1: 

ok great

Added some more cases to the notebook and squashed. If you spot something strange, let me know, otherwise this is ok from my side. 

Worth it to add a MultiIndex example In whatsnew? and/or docs/FAQ (where the display options are)

For me it's fine like this. I think most users will just assume central truncation works for the different types of indices and backends (terminal, notebook). 

But if you think it's better to showcase the way truncation works more, I'm happy to add an example.
BUG: bar plot can now handle bottom and left kw properly
State: closed

Actually not a bug, but allows `bar` and `barh` plot to accept `bottom` and `left` kw respectively to specify the starting point.

Currently, passing `bottom` or `left` keyword results in `TypeError`

```
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(5, 5))
df.plot(kind='bar', bottom=1, ax=axes[0])
# TypeError: bar() got multiple values for keyword argument 'bottom'
```
### After fix

```
import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 2)

df = pd.DataFrame(np.random.randn(5, 5))
df.plot(kind='bar', bottom=1, ax=axes[0])

df2 = pd.DataFrame(np.random.rand(5, 5))
df2.plot(kind='barh', left=[1, 2, 3, 4, 5], stacked=True, ax=axes[1])
```

![figure_1](https://cloud.githubusercontent.com/assets/1696302/3074746/24f8bebe-e351-11e3-9506-ac7de60db7fd.png)


Labels:
name: Enhancement
name: Visualization
description: plotting

Comments:
@TomAugspurger pls review when you can

I'm fine labeling this as a bug. `**kwargs` is supposed to pass along all the args of the original matplotlib function/method. `left` and `bottom` are args for `bar`.

Merged via https://github.com/pydata/pandas/commit/4adc6fa62895496a3fbbf36aca876ec91ca0d47b

Thanks @sinhrks
BUG: boxplot returns incorrect dict
State: closed

#7096 returns incorrect `dict` when number of columns and subplots are different.

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import pandas.util.testing as tm


n=100
gender = tm.choice(['Male', 'Female'], size=n)
classroom = tm.choice(['A', 'B', 'C'], size=n)

df = pd.DataFrame({'gender': gender,
                          'classroom': classroom,
                          'height': np.random.normal(66, 4, size=n),
                          'weight': np.random.normal(161, 32, size=n),
                          'category': np.random.randint(4, size=n)})

df.boxplot(by='classroom', return_type='axes')
# {'category': array([<matplotlib.axes.AxesSubplot object at 0x104700e90>,
#        <matplotlib.axes.AxesSubplot object at 0x10671ca90>], dtype=object),
#  'height': array([<matplotlib.axes.AxesSubplot object at 0x106744dd0>,
#        <matplotlib.axes.AxesSubplot object at 0x106766b50>], dtype=object)}

# This must be a dict with 3 keys ('category', 'height' and 'weight') which value is AxesSubplot (not numpy.array).
```

Also, `boxplot` sometimes return `OrderedDict` and sometimes `dict` inconsistently.

```
df.boxplot(by='classroom', return_type='dict')
# OrderedDict([('category', {'medians': [<matplotlib.lines.Line2D object at 0x106fe8c10>,
# ...
```

The fix makes `boxplot` to always return `OrderedDict` which has correct mapping of keys and values


Labels:
name: Bug
name: Visualization
description: plotting

Comments:
@TomAugspurger 

Just that one comment. Returning a dict vs. an OrderedDict doesn't really matter in that case, and I don't have a strong preference either way.  Everything else looks fine.

@TomAugspurger is this for 0.14.0?

I see, ok for 0.14.0
DOC: fix datetime-like Series ops in docs + some extra api docs
State: closed

Closes #7213.

\+ added some entries in api.rst


Labels:
name: Docs

Comments:
hah! you are faster....go ahead and merge and i'll rebase I had to fix something else with the attributes anyhow

ah, sorry, I was adding some things in the api.rst, and just thought to quickly fix that to
TST/DOC: Test for and document incompatibility with openpyxl 2.0.0 and later
State: closed

Let users down easy if v0.14.0 remains incompatible with openpyxl 2.0.0 and later 

closes #7169 .


Labels:
name: IO Excel
description: read_excel, to_excel
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
I [read](http://stackoverflow.com/a/11887885/1816890) that `LooseVersion` was to be deprecated by [PEP-386](http://legacy.python.org/dev/peps/pep-0386/#roadmap), but I guess that hasn't happened yet.

@jorisvandenbossche @cpcloud ?

@neirbowj pls squash down to 1 commit 

@neirbowj 

this is failing the 3.4 build (which has 2.03 of openpyxl).

https://travis-ci.org/pydata/pandas/jobs/25839724

I think you might need to leave openpyxl in the engine map by just have its value of `None`

then you can skip tests when the writer is `None`

something like that

@jreback So "Allowed Failures" aren't really? ;-)

The problem is not the writer registration, it looks like it's the version detection. I suspect that the too-old `lxml` is interfering. For some reason, `pandas.compat.openpyxl_compat.is_compat()` thinks that `'2.0.3'` is an incompatible version. I'll investigate.

they are just so the testing doesn't take too long

they r testing 3.4 and numpy 1.9

the 3.4 tests current versions of a lot of the deps  (all the other deps are fixed)

so shouldn't fail

So... `lxml` has nothing to do with it. I proposed I would write a test that fails when an incompatible version of `openpyxl` is installed and I succeeded in doing so. If the tests as configured need to pass under 3.4, then that was the wrong spec. I should back out that test, and update `pandas.io.tests.test_excel._skip_if_no_openpyxl` to treat an incompatible version as "no openpyxl."

ok this all looks fine

should the warning be a UserWaarning?

pls squash to a single commit and can merge this

also pls open an issue for the actual modifications needed to support > 2.0 
which can be worked on for 0.14.1

maybe an ImportWarning ?

`openpyxl.xml` itself uses the default (`UserWarning`) when it encounters the too-old `lxml`. I'm thinking we should follow that example.

```
def lxml_available():
    try:
        from lxml.etree import LXML_VERSION
        LXML = LXML_VERSION >= (3, 3, 1, 0)
        if not LXML:
            import warnings
            warnings.warn("The installed version of lxml is too old to be used with openpyxl")
            return False  # we have it, but too old
        else:
            return True  # we have it, and recent enough
    except ImportError:
        return False  # we don't even have it
```

ahh ok that's fine then

looking good

@neirbowj thanks!

@neirbowj 

This just failed, seems odd, right? https://travis-ci.org/jreback/pandas/jobs/25910838

Yes, that does seem odd. I'm looking into it now.

 I restarted the test and it DID recur

though it is NOT present in master atm; maybe had trouble installing the library....weird

The commit in this PR is not an ancestor of your `categorical` branch. Those two branches diverged just before I committed the workaround. Rebase and you'll be in good shape.
API: remove datetime-like index ops from Series
State: closed

as per discussed in #7146. may provide a `to_index()` and/or an accessor in future versions


Labels:
name: API Design

Comments:
@cpcloud @jorisvandenbossche ?

Just added a note about the new whatsnew entry, but for the rest am OK on disabling this for now.

@jorisvandenbossche fair comments

the issue is that it can become ambiguous, so don't want to add something w/o dealing with that.

Series.weekday wasn't documented at all so I don't think a big deal to remove.

@jreback OK for the weekday

What do you mean with ambiguous? What can become ambiguous? `min/max/unique/nunique/value_counts` will not always act on the series values?

not the usage of `Series.weekday` _could_ be ambiguous (and applies to all of the properties), IF the series contains a datetimeindex as well (or periodindex)

```
In [1]: s      = Series(pd.date_range('20130101',periods=3),index=pd.date_range('20130104',periods=3))

In [2]: s
Out[2]: 
2013-01-04   2013-01-01
2013-01-05   2013-01-02
2013-01-06   2013-01-03
Freq: D, dtype: datetime64[ns]

In [3]: s.index.weekday
Out[3]: array([4, 5, 6])

In [4]: pd.DatetimeIndex(s).weekday
Out[4]: array([1, 2, 3])
```

ah yes OK, `weekday` is indeed ambiguous like the other datetime accessors. And so they are disabled again in this PR. That's all OK (but I thought you were speaking about the others)

no, the others were added (not in this PR necessarily, though max/min were), but don't think had any notes on them

What was actually added? (I just realize I don't fully understand this)

For example `max()` already is an index method in 0.13, so I don't fully get what `support 'max()/... on Index types'` in the release notes is supposed to mean?

all of these were added as mixin methods that is they were moved from index/series directly to the ops mixin class (so max.min) were not added to index

but the other were added to index

There still some occurences in the docs (that now error):

http://pandas-docs.github.io/pandas-docs-travis/timeseries.html#custom-business-days-experimental 
http://pandas-docs.github.io/pandas-docs-travis/whatsnew.html#experimental-features
BUG: Bug in expressions evaluation with reversed ops, showing in series-dataframe ops (GH7198)
State: closed

closes #7198
closes #7192


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
@cpcloud this is fine in 0.13.1 and broke in 0.14.0...any clue?

I think this is the same issues as in #7192 (as the subtraction is not commutative)

I think this is my fault. I added the op() calls for the rops. This particular issue was untested. 

fessing up! :)

np....this tests this explicity 

took me a bit to figure out that sub/div are not commutative while add/mul ARE!

Might be better to just remove the rops op($ calls than to add the reversed check

Unless that isn't the issue. But I think that's the only thing other than the raise on bool  ops that has changed between 0.13.1 and 14

python somehow seems to handle them correctly (e.g. it in-line reverses)

```
__rtruediv__(x,y)
__truediv__(x,y)
```

do the right thing!

whereas in numexpr the 2nd is backwards

I don't have any idea WHEN python decides to call the reversed ops, but no biggie, just added a flag and it should work

It's done at the c level in pyobject_richcompare I followed it once for some Timestamp compare stuff. Good times 

haha....ok...no worries
BUG: CustomBusinessDay apply raises NameError when np.datetime64 is pass...
State: closed

`CustomBusinessDay` raises `NameError` when `np.datetime64` is passed. It works if `normalize=True` is specified.

```
d = pd.offsets.CustomBusinessDay().apply(np.datetime64('2011-01-01 09:00:00'))
# NameError: global name 'np_day_incr' is not defined

d = pd.offsets.CustomBusinessDay(normalize=True).apply(np.datetime64('2011-01-01 09:00:00'))
#2011-01-03 00:00:00
```


Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
pls rebase

thank you
BUG: Easter works incorrectly in negative offsets
State: closed

When an offset is negative, Easter skips the date just before and returns one before.

```
(- pd.offsets.Easter()).apply(pd.Timestamp('2011-01-01 09:00:00'))
#2009-04-12 00:00:00

(- pd.offsets.Easter()).apply(pd.Timestamp('2012-01-01 09:00:00'))
# returns 2010-04-04 00:00:00

(- pd.offsets.Easter()).apply(pd.Timestamp('2010-04-05 09:00:00'))
#2009-04-12 00:00:00
```


Labels:
name: Bug
name: Frequency
description: DateOffsets

Comments:
this just Easter or a more general holiday apply issue?

Other offset _looks_ handle negative offsets correctly. I'll check other offsets during #7156.

thanks
BUG: fix isnull for 0d object arrays
State: closed

If this looks good, it would be nice to get it in 0.14, too.


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
pls test for several dtypes (float64/int64) as well

and use assertTrue/False rather than bare assert

need a release note (ref the pr number)

@jreback New commit up.

looks good
can u squash to 1 commit pls

Squashed

thanks!
CLN: Deprecation of assert_
State: closed

#7131: Run s/self.assert_(/self.assertTrue( on remaining instances I can find. These were the multi-line expressions, and I tried to retain the formatting of the expressions


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
here's an example of running with `PANDAS_TESTING_MODE='deprecate'` on 3.4: https://travis-ci.org/jreback/pandas/jobs/25480011

Thanks, @jreback, I will look through that in more detail later today.
CLN: Deprecation of assertEquals
State: closed

#7131: s/self.assertEquals(/self.assertEqual(


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Note: This was a straight search-and-replace.

FYI: if you want to see what's remaining on 3.4, copy this line: https://github.com/pydata/pandas/blob/master/.travis.yml#L79

to both of the 3.4 entries in `.travis.yml` (it STILL may be too much output but maybe ok)

thanks!
CLN: Deprecation of assert_
State: closed

#7131: Convert more instances of self.assert_(not b) to self.assertFalse(b) and self.assert_(b) to self.assertTrue(b)


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
There are still a few instances of assert_ left to be converted: if an assertion has multiple lines, which may require reformatting, I did not yet touch it.

after this prob won't be any assert_ left!

thanks!
CLN: Python 3.4 deprecation of assert_
State: closed

#7131: replace some instances of assert_(not b) with assertFalse(b) and some instances of assert_(b) with assertTrue(b). Work-in-progress


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
hah wondering if u would see that issue!

gr!

FYI you can set the env variable PANDAS_TESTING_MODE  to deprecate to have it show all of these kind of earnings (it's turned off in the 3.4 build because it makes the output really long)

ping in green

It's like a Batsignal went on.

Good to know about the environment variable; thanks for pointing it out.

thanks!

I have to add the env variable usage to the wiki
BUG: PeriodIndex slices with out of bounds results incorrect
State: closed

Closes #5407.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Period
description: Period data type

Comments:
side issue does DatetimeIndex handle this properly?

thanks!

Yes it does. Test case includes `DatetimeIndex` to check `PeriodIndex` behaves identically as `DatetimeIndex`.
TST: Add layout tests for boxplot
State: closed

Added `layout` test cases related to #7035, changing `_check_axes_shape` logic to meet with #7096.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Visualization
description: plotting

Comments:
@TomAugspurger pls have a look
marking for 0.14.1 unless urgent; lmk what u think

@TomAugspurger ?

Fine by me. Thanks @sinhrks.