BrandonFors/pandas-github-issues huggingface dataset:
ENH/CLN: add HistPlot class inheriting MPLPlot
State: closed

Because `hist` and `boxplot` are separated from normal `plot`, there are some inconsistencies with these functions. Looks better to include them to `MPLPlot` framework.

Maybe `scatter` and `hist` can be deprecated in 0.15 if `MPLPlot` can offer better `GroupBy` plot (plan to do in separate PR).
### Example

This allows to use `kind='hist` in `DataFrame.plot` and `Series.plot`. (No changes for `DataFrame.hist`)

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

df = pd.DataFrame(np.random.randn(1000, 5))
df.plot(kind='hist', subplots=True)
```

![figure_1](https://cloud.githubusercontent.com/assets/1696302/3712412/16aa1896-1513-11e4-863d-e0b6d631c12f.png)

```
df.plot(kind='hist', stacked=True)
```

![figure_2](https://cloud.githubusercontent.com/assets/1696302/3712413/1bded702-1513-11e4-9ff3-3947783d5213.png)
### Remaining Items
- [x] Add a release note in API section detailing this change/enhancement
- [x] Modify doc
- [x] Add tests (both histogram and kde) 
  - [x] Add support for `rot` and `fontsize` (depending on `orientation` kw) (rely on #7844)
  - [x] Add tests for xticklabels and yticklabels
  - [x] Add tests for colors
  - [x] Handling nan
- [x] Check `stacked=True` can be supported (`DataFrame.hist` doesn't support it though..).
  - [x] Add tests for stacking


Labels:
name: Enhancement
name: Visualization
description: plotting
name: API Design
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
does this change anything? or is just an internal refactor?

what is more consistent?

this independent of #7717 ?

It looks like the main API change is that as of this PR you can make a histogram with `df.plot(kind='hist')`, as well as `df.hist()`. Seems like a win for consistency!

@sinhrks also let's add a release note in API section detailing this change/enhancement

Will do. Added example and remaining items on top.

is `DataFrame.hist` now just a call to `df.plot(kind='hist')`?

change the title on this as well (the PR), no longer a WIP, rigth?

Going to leave `hist` as it is (maybe `hist` can be deprecate in future). There are some items should be done / considered. Once finished, I'll change the title.

I think this can be reviewed (though `rot` and `fontsize` processing relies on #7844)

so #7844  should be merged first? or does this include as well?

Yes, #7844 should be first.

this looked ok, @sinhrks can you rebase

@TomAugspurger have a look?

OK, rebased.

I just gave it a quick look (didn't test, computer was stolen!) and I think it looks good.

Only comment was maybe setting a random seed in the doc example, very minor.

Nice work! 

> On Aug 10, 2014, at 16:21, Sinhrks notifications@github.com wrote:
> 
> OK, rebased.
> 
> —
> Reply to this email directly or view it on GitHub.

@sinhrks hmm, needs rebasing again, then ok to merge

ping when pushed

@jreback Rebased and got green.

thanks!
BUG: is_superperiod and is_subperiod cannot handle higher freq than S
State: closed

Closes #7772, Closes #7760.


Labels:
name: Bug
name: Visualization
description: plotting
name: Period
description: Period data type

Comments:
any idea why this was this way? just maybe not implemented?

I think so. The functions are only used in `resample` and `tseries.plotting` which looks no need to exclude these frequencies.

ok, gr8 thanks!
BUG: reset_index with MultiIndex contains PeriodIndex raises ValueError
State: closed

Closes #7746, Closes #7793.

Sorry, caused by #7533. The `ValueError` is being raised when  `PeriodIndex` or `DatetimeIndex` with tz 's unique values are less than `MultiIndex` length.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Period
description: Period data type
name: MultiIndex

Comments:
@sinhrks nice! thanks

your fixes are always nicely tested and put together :)
BUG: timeseries subplots may display unnecessary minor ticklabels
State: closed

Related to #7457. The fix was incomplete because it only hides major ticklabels, not minor ticklabels. This causes incorrect result in time-series plot.

(And another problem is `rot` default is not applied to minor ticks. I'll check this separatelly)

```
df = pd.DataFrame(np.random.randn(10, 4), index=pd.date_range(start='2014-07-01', freq='M', periods=10))
df.plot(subplots=True)
```
### Current result:

Minor ticklabels of top 3 axes are not hidden.
![figure_ng](https://cloud.githubusercontent.com/assets/1696302/3634314/36ebc68c-0f22-11e4-9f9c-da803dd27d5e.png)
### Result after fix:

![figure_ok](https://cloud.githubusercontent.com/assets/1696302/3634865/57a28f1a-0f4e-11e4-847f-fff225740439.png)


Labels:
name: Bug
name: Visualization
description: plotting

Comments:
Ah, this is not good as vertical grids are hidden. Leave it a while...

@sinhrks this is a bug-fix or  enhancement?

Fixed to display grid lines, and only hides major and minor ticklabels (hiding minor tick labels is little tricky than I imagined). Updated image of result after fix.

@jreback I think it should be classified as a bug as the same as #7457 (#5897).

Modified to use `try...except` to avoid any issue caused by `mpl.ticker`

@sinhrks ping your unmerged issues that you think are ready

@jreback I think this one is ready.

thanks!
BUG: fix reading pre-0.14.1 pickles of containers with one block and dup items
State: closed

Series, frames and panels that contain only one block can be unpickled
under the assumption that block items correspond to manager items 1-to-1
(as pointed out in #7329).

I still don't have any of darwin/win/32bit platforms at hand, so I
cannot post those pickles.  Also, my linux has a fresher python than
indicated by filenames of existing pickle test data.  I did generate
some data to test the fix, the question is do you want me to upload it?


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
just put up some legacy files whatever your python / platform (if u want to multiple versions of oython that's ok)

it doesn't matter much that we have for every platform for very version

cross compat is pretty much there

Ok, I've added pickles for 0.13.0, 0.14.0 (just in case) and 0.14.1.

perfect! ping when green

you have to add the directory of 0.14.0 to `setup.py`, see here: https://github.com/pydata/pandas/blob/master/setup.py#L578

(actually prob is a better way to do by adding all directories under data), any idea?

@immerrr pls rebase another commit added the 0.14.0 pickle dir already

rebased and green

@immerrr thanks for the fix! preserving pickle compat is never easy!
BUG: read_column did not preserve UTC tzinfo
State: closed

BUG: Fixes #7777, HDFStore.read_column did not preserve timezone information
when fetching a DatetimeIndex column with tz=UTC


Labels:
name: Bug
name: IO HDF5
description: read_hdf, HDFStore
name: Timezones
description: Timezone data dtype

Comments:
Hmm, I'm not quite sure that will work.  `self.values` probably needs to remain an Index, and when a Series is constructed from an index, it calls `_to_embed` which removes UTC tzinfo.  That's why I workaround `_to_embed` by constructing the series from a list instead of an index. 

that's not what I mean, re-factor it out as a function which you can _call_ from both places, rather than duplicated the code

You mean something like the following that can be called in convert and when returning the series (with preserve_UTC = True)?

``` python

def _set_tz(values, tz, preserve_UTC=False):
    if tz is not None and isinstance(values, Index):
        tz = _ensure_decoded(tz)
        if values.tz is None:
            values = values.tz_localize('UTC').tz_convert(tz)
        if preserve_UTC:
            if tz == pytz.utc:
                values = list(values)

    return values
```

yep

though I _think_ you can simply do `tz == 'UTC'` (which works for dateutil and pytz)

cc @dbew

You can't do `tz == 'UTC'`, but you can do `tslib.get_timezone(tz) == 'UTC'`.  

@alorenzo175 ok, gr8! 

hmm maybe should document that somewhere .....

That looks good to me and the tests are sensible. 

@alorenzo175 this look good.  just need  release note in v0.15.0 (reference original issue in bug fix section). and I think good to go

@jreback should be good to go now

perfect, now just squash to a snigle commit and then I can merge, see here: https://github.com/pydata/pandas/wiki/Using-Git

squashed

@alorenzo175 thanks for the fix!
Raise exception on non-unique column index in to_hdf for fixed format.
State: closed

Fixes #7761.


Labels:
name: Bug
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO HDF5
description: read_hdf, HDFStore

Comments:
hmm, maybe make this a `NotImplementedError` (as its not invalid, just not supported). Also maybe a note in the io.rst (in the Fixed section)

Well, with the current fixed format it is in fact invalid. There is no way to get this right without changing `block_items` to point on columns using integers, i.e. their position in the zeroth axis.

it _could_ be implemented (it just isn't). You are preventing the writing (which invalidates the reading) either way. The exception will be more informative (`ValueError` is saying something about the input is not valid. It IS valid, just not in writing :).

Actually you might want to have a nice exception message saying that you CAN store this in 'table' format.

It could not be implemented because `block_items` contains only strings. You'd have to change the fixed format to implement this, for all other cases (i.e. append and compression) a `ValueError` is raised.

If this is the only thing keeping you from merging I'll change it to `NotImplementedError` ;)

@filmor thanks for this!

this is technically for a non-unique info_axis (e.g. columns for df, but 0 axis for Series/Panel). Can you add some tests for this? And update the note in v0.15.0 (I moved it a bit)
BUG: Prevent config paths to contain python keywords
State: closed


Labels:
name: API Design

Comments:
can u add a test for this pls

this looks fine, can you add a note in v0.15.0 (use this PR number as the issue number); in Bug Fixes

I added a note, but since v0.15.0 had been modified since I started this PR I had to rebase. Or I thought I had. This is my first rebase ever so I didn't mess up. Did I?

https://github.com/pydata/pandas/wiki/Using-Git

yep

```
git rebase -i origin/master
```

should only have your commits left

then you can squash (use s)

then 

```
git push myfork thisbranch -f
```

you can rebase as I showed above

pls rebase and squash

apologies for not taking care of this earlier, and thanks a lot for the help! Hopefully I did the rebase correctly this time.

this is fine
ping when green 

thanks!

Thank you for the guidance! next time will read the docs more carefully

@germangh no thank you! always appreciate contributions! make next time easier :)
BUG: unwanted conversions of timedelta dtypes when in a mixed datetimelike frame (GH7778)
State: closed

closes #7778

TST: tests for internals/as_matrix() for all dtypes (including categoricals)


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Timedelta
description: Timedelta data type

Comments:
cc @JanSchulz

I fixed up doing something like

`df.values` when you have a categorical (or 2) as a dtype.

@cpcloud 

should 

```
df = DataFrame({'bool' : [True, False], 'int' : [1, 2] })
df.values
```

be `object` or `int`? (I have it as object now).

hm `int` more useful, but `object` more consistent with the fact that numpy separates `bool` from `int`, i think `int`. is this an api breaking change?

I think it was object before
let me see

@cpcloud only took the usual 5x iterations for make the 1.6.1 stuff work :<

:rage: on 1.6
FIX: Documentation for for 0.14.1 change log
State: closed

Change log is missing some key words on changes to StataReader and StataWriter.


Labels:
name: Docs

Comments:
A small omission to the 0.14.1 docs where I left off the class names.

hmm, these are not normally regenerated. when @jorisvandenbossche is back he can do if necessary

Just saw the 0.15-dev docs today and noticed the omissions.  Figured I put the option out there.

np. thanks.

In any case, merging this so it is fixed in future docs. I will put in on my todo list to also update the current stable docs (but will be something for next week at the earliest)
Closes #7758 - astype(unicode) returning unicode.
State: closed

I didn't have to rely on `infer_dtype` as suggested by @jreback in #7758
as I delegated to numpy to do all the dirty job :
Just calls numpy.unicode on all the values.

Please have a critical look at this pull request before merging :
as I am unfamiliar with both pandas and cython; and may have 
misunderstood the way pandas is working.

Unit tests are attached and seem to work alright on `python 2.7.2` and `python 3.3.0`

closes #7758 


Labels:
name: Bug
name: Enhancement
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Unicode
description: Unicode strings

Comments:
looks pretty good!
pls add note in v0.15.0.txt (with the issue number) - bug fix section is ok

Done.

merged via https://github.com/pydata/pandas/commit/a797b28c87d90a439dfa2c12b4a11e62bf0d6db2

thanks! nice fix
Update common.py
State: closed

Force violate array to be of type int. When no nan values are found, the array defaults to an empty array of type float, so you get an error: **\* IndexError: arrays used as indices must be of integer (or boolean) type


Labels:

Comments:
need a test for this
does this fix sn open issue?
can u show what exactly the problem is?

Sorry for not adding a test. I didn't report an issue or looked for an open one. Just fixed it straight away. See a simple testcase below:

``` python
from pandas import DataFrame
from numpy import NaN
df = DataFrame({'A': [1,2,3,5]})

# interpolation with 'limit' argument gives error when there are no nan values in the series:
df.interpolate(limit=1)

# after adding a NaN value, no error
df.loc[1] = NaN
df.interpolate(limit=1)
```

@tdamsma your example above would need to be incorporated in this PR as a test, see here: https://github.com/pydata/pandas/wiki/Testing

@tdamsma your example looks correct to me. closing this. pls create an issue showing what the bug actually is.
ENH/BUG: DatetimeIndex and PeriodIndex in-place ops behaves incorrectly
State: closed

Fixes 2 issues related to `DetetimeIndex` and `PeriodIndex` ops.
- Addition / subtraction between `PeriodIndex` raise `TypeError` (Closes #3367).

```
pidx + pidx
# TypeError: unsupported operand type(s) for +: 'PeriodIndex' and 'PeriodIndex'
```
- In-place addition / subtraction doesn't return the same result as normal addition / subtraction.  Specifically, `PeriodIndex` in-place operation results in `Int64Index` (Closes #6527)

```
didx = pd.date_range('2011-01-01', freq='D', periods=5)

# This results shift (expected)
didx + 1
# <class 'pandas.tseries.index.DatetimeIndex'>
# [2011-01-02, ..., 2011-01-06]
# Length: 5, Freq: D, Timezone: None

# This result adding 1 unit (nano second) (NG)
didx += 1
didx 
# <class 'pandas.tseries.index.DatetimeIndex'>
# [2011-01-01 00:00:00.000000001, ..., 2011-01-05 00:00:00.000000001]
# Length: 5, Freq: None, Timezone: None
```


Labels:
name: API Design
name: Period
description: Period data type

Comments:
This can solve #3367. Test cases are added.

look good..thanks
BUG: _flex_binary_moment() doesn't preserve column order or handle multiple columns with the same label
State: closed

Closes https://github.com/pydata/pandas/issues/7542.


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
I left the following unchanged for two distinct DataFrames with pairwise=False, as it's not obvious to me what the desired behavior is in this case.

```
            if pairwise is False:
                ...
                res_columns = arg1.columns.union(arg2.columns)
```

I updated the code to properly deal with multiple columns with the same label (except for the case of two distinct DataFrames and pairwise=False, where it's not obvious to me what the desired behavior would be).

need a release note for v0.15.0 in bug fixes

can't use a context manager under. 2.6 like that

iirc just create a function
thrn call
self.assertRaises(ValueError, f)

How do I test the message in the ValueError? The following works for me on 3.4, but per https://docs.python.org/2.6/search.html?q=assertRaisesRegex&check_keywords=yes&area=default I suspect it doesn't work on 2.6.

```
                    self.assertRaisesRegex(ValueError, "'arg1' columns are not unique", f, df, df2)
                    self.assertRaisesRegex(ValueError, "'arg2' columns are not unique", f, df2, df)
```

their are lots of examples in the code, e.g. : https://github.com/pydata/pandas/blob/master/pandas/tests/test_index.py#L52 (and you can use a context manger). don't use the `self.assertRaisesRegex`, rather the `tm.assertRaisesRegex` (which is fixed for 2.6)....

hmm...maybe should fix that

OK, updated to use `tm.assertRaisesRegexp`. Thanks for the help...

@seth-p I added them to the `TestCase` anyhow (so when you do `self.assertRaisesRegexp` will work (for the future)

... `tm.assertRaisesRegexp` worked.

ok, looks good....ping when green

One of the Travis builds failed for no discernible-to-me reason: https://travis-ci.org/pydata/pandas/jobs/30798802.

> On Jul 24, 2014, at 7:11 PM, jreback notifications@github.com wrote:
> 
> ok, looks good....ping when green
> 
> —
> Reply to this email directly or view it on GitHub.

sometimes this happens np (Travis is flaky )

Anything else I need to do, then?

thanks!
ENH: Use left._constructor on pd.merge
State: closed

Use the _constructor property when creating the merge result to
preserve the output type.

If a [GeoPandas](http://github.com/geopandas/geopandas) `GeoDataFrame` is merged with a `DataFrame`, the result is hard-coded to always be `DataFrame` [GeoPandas Issue #118](https://github.com/geopandas/geopandas/issues/118). We'd like it to return `GeoDataFrame` in these cases

```
>>> import geopandas as gpd
>>> import pandas as pd

>>> gdf = gpd.GeoDataFrame(...)
>>> df = pd.DataFrame(...)

>>> merged = pd.merge(gdf, df, on='column')
>>> type(merged)
GeoDataFrame
```

This PR uses `left._constructor` to generate the result type for merge operations.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
sure

can u add a test pls (create an object that inherits from DataFrame and use it to confirm that this is correct)

@jreback Done.

It would also be nice to mention this in the docs (i.e. that the output type is taken from left, so that `pd.merge(df, gdf, on='column')` still returns a DataFrame...)

@jwass this is fine, can you update the docs (and do-string). squash and looks ready.

@JanSchulz Missed your original comment from a few weeks ago - sorry for the delay here.

@jreback squashed and rebased. Let me know if the docs should change somewhere else or should be re-worded.

can you add a note in v0.15.0.txt in API section (and reference this PR number)

@jreback Yes - added the `__finalize__` call in `ordered_merge`. While I was at it, added the test/doc for `ordered_merge` to reflect the new behavior here too.
Added the note for v0.15.0. Squashed, rebased.

merged via https://github.com/pydata/pandas/commit/123a55579007457f0a901404600a6ea470daf16f

thanks!
BUG: DTI.value_counts doesnt preserve tz
State: closed

Found 2 problems related to `value_counts`.
- `DatetimeIndex.value_counts` loses tz.

```
didx = pd.date_range('2011-01-01 09:00', freq='H', periods=3, tz='Asia/Tokyo')
print(didx.value_counts())
#2011-01-01 00:00:00    1
#2011-01-01 01:00:00    1
#2011-01-01 02:00:00    1
# dtype: int64
```
- `PeriodIndex.value_counts` results in `Int64Index`, and unable to drop `NaT`.

```
pidx = pd.PeriodIndex(['2011-01-01 09:00', '2011-01-01 10:00', pd.NaT], freq='H')
print(pidx.value_counts())
#  359410                 1
#  359409                 1
# -9223372036854775808    1
# dtype: int64
```


Labels:
name: Bug
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Timezones
description: Timezone data dtype

Comments:
better

rebase this

@sinhrks pls rebase

@jreback Rebased.

can you run a perf test on this? 

Attached. `replace` looks irrelevant for this.

```
reshape_stack_simple                         |   4.2387 |   3.6786 |   1.1522 |
append_frame_single_mixed                    |   3.7584 |   3.1943 |   1.1766 |
replace_replacena                            |   3.1474 |   2.6023 |   1.2094 |
replace_fillna                               |   4.2147 |   3.1803 |   1.3252 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [01a9e18] : BUG: DTI.value_counts doesnt preserve tz
Base   [bfa9bc5] : Merge pull request #7733 from sinhrks/tsplot_bug
```

yep, agreed. ping when green

@jreback Now green

perfect, ping when green
BUG: Repeated timeseries plot may result in incorrect kind
State: closed

Must be revisited after #7717.

Repeated line and area plot may result incorrect if it requires resampling.

```
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

fig, axes = plt.subplots(2, 2, figsize=(7, 5))
np.random.seed(1)

df1 = pd.DataFrame(np.random.rand(5, 2), pd.date_range('2011-01-01', periods=5, freq='D'))
df2 = pd.DataFrame(np.random.rand(2, 2), pd.date_range('2011-01-01', periods=2, freq='M'))

df1.plot(kind='line', ax=axes[0][0], legend=False)
df2.plot(kind='area', ax=axes[0][0], legend=False)

df1.plot(kind='area', ax=axes[1][0], legend=False)
df2.plot(kind='line', ax=axes[1][0], legend=False)

df2.plot(kind='line', ax=axes[0][1], legend=False)
df1.plot(kind='area', ax=axes[0][1], legend=False)
# ValueError: Argument dimensions are incompatible

df2.plot(kind='area', ax=axes[1][1], legend=False)
df1.plot(kind='line', ax=axes[1][1], legend=False)
```
### Result using current master
- line with low  freq -> area with high freq results in `ValueError` (top-right axes)
- area with low  freq -> line with high freq results in all lines, not area (bottom-right axes)
  ![figure_ng](https://cloud.githubusercontent.com/assets/1696302/3560065/dd93e976-0958-11e4-83e1-acd1dbffa1df.png)
### Result after fix

![figure_ok](https://cloud.githubusercontent.com/assets/1696302/3560079/622f2eb6-0959-11e4-90ed-5e7eee5dea52.png)


Labels:
name: Bug
name: Visualization
description: plotting

Comments:
Rebased. I think this can be reviewed.

thanks!
BUG: allow get default value upon IndexError, GH #7725
State: closed

Fixes #7725 by adding IndexError to the tuple of caught exceptions.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
nice - release note for 0.15.0 pls

Do you mean in the bug fixes section of v0.15.0.txt?

yep 

@dsm054 can  you look at addl comment on #7725. pls rebase as well.

though we had already merged this, thanks @dsm054
ENH: Add uint and bool support in to_stata
State: closed

Added support for uint (uint8, uint16 and uint32, but not uint64) and bool
datatypes in to_stata.  
closes #7097 
closes #7365


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO Stata
description: read_stata, to_stata

Comments:
we'll just call you mr. stata from now on :)

can you move the release note to 0.15.0.txt (0.14.1 coming tomorrow), so too late for that 

I think this is ready.

@jreback I added some docs.

minor edits, otherwise looks good. 

thanks @bashtage continued awesome stata support is great!

merged via https://github.com/pydata/pandas/commit/7745d726b0625c8a4ab017629be0c4eb6241a81a