Repository: pandas-dev/pandas

Issue:
ENH: Add Series.histogram()
Should I write a `Series.histogram` that calls `numpy.histogram` and returns a new Series of counts indexed by bins?

```
In [4]: s = Series(randn(100))

In [5]: counts, bins = np.histogram(s)

In [6]: Series(counts, index=bins[:-1])
Out[6]: 
-2.968575     1
-2.355032     4
-1.741488     5
-1.127944    26
-0.514401    23
 0.099143    23
 0.712686    12
 1.326230     5
 1.939773     0
 2.553317     1
dtype: int32
```

...would be accomplished by...

```
Series.histogram()
```

which would accept at the arguments that `np.histogram` accepts (e.g., choosing bins manually, specifying number or range of bins, etc.).

from http://stackoverflow.com/a/17150734/1221924

The proposed method doesn't save all that much typing, but it would help users discover this convenient way of storing a histogram. Proceed?



Discussion:
What you are proposing is basically this
(I am not sure what is a better result in any event)

```
In [9]: s.groupby(pd.cut(s,10)).count()
Out[9]: 
(-2.955, -2.419]     1
(-2.419, -1.889]     2
(-1.889, -1.359]     2
(-1.359, -0.829]     8
(-0.829, -0.299]    24
(-0.299, 0.231]     21
(0.231, 0.761]      15
(0.761, 1.291]      14
(1.291, 1.821]      10
(1.821, 2.351]       3
dtype: int64
```

That's new to me. Not sure which output is more valuable for whatever users would do next.

yours more friendly for plotting 
cut is NaN friendly (not sure if np.histogram is)

@jreback cut returns string row labels, right? 

it returns a Categorical which has both labels and levels

would this be more useful as a top level function instead of a method?

`np.histogram` is not `nan` friendly

I think an instance method on series is about right (but maybe using groupy and cut as above)

I don't have strong feelings about this. It's certainly easy to work around, but I really hate working with Categoricals.

problem is np.histogram doesn't like nan
maybe just drop em anyhow

or u can just get the bins from cut

``` python
cat, bins = cut(s, 10, retbins=True)
s = Series(cat.values, index=bins[:-1], name=cat.name)
```

not sure if `name` should be there...

confusing having different hist and histogram methods... perhaps this could be an argument (bins) to value_counts ?

Good idea.
On Aug 6, 2013 6:52 PM, "Andy Hayden" notifications@github.com wrote:

> confusing having different hist and histogram methods... perhaps this
> could be an argument (bins) to value_counts ?
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/3945#issuecomment-22217991
> .

This is pretty easy, pr on the way, will also add more functionality to Series.value_counts which was never updated from pd.value_counts.


Resolving patch:
diff --git a/pandas/core/algorithms.py b/pandas/core/algorithms.py
--- a/pandas/core/algorithms.py
+++ b/pandas/core/algorithms.py
@@ -148,7 +148,7 @@ def factorize(values, sort=False, order=None, na_sentinel=-1):
     return labels, uniques
 
 
-def value_counts(values, sort=True, ascending=False, normalize=False):
+def value_counts(values, sort=True, ascending=False, normalize=False, bins=None):
     """
     Compute a histogram of the counts of non-null values
 
@@ -161,26 +161,39 @@ def value_counts(values, sort=True, ascending=False, normalize=False):
         Sort in ascending order
     normalize: boolean, default False
         If True then compute a relative histogram
+    bins : integer, optional
+        Rather than count values, group them into half-open bins,
+        convenience for pd.cut, only works with numeric data
 
     Returns
     -------
     value_counts : Series
+
     """
     from pandas.core.series import Series
+    from pandas.tools.tile import cut
+
+    values = Series(values).values
 
-    values = np.asarray(values)
+    if bins is not None:
+        try:
+            cat, bins = cut(values, bins, retbins=True)
+        except TypeError:
+            raise TypeError("bins argument only works with numeric data.")
+        values = cat.labels
 
     if com.is_integer_dtype(values.dtype):
         values = com._ensure_int64(values)
         keys, counts = htable.value_count_int64(values)
-    elif issubclass(values.dtype.type, (np.datetime64,np.timedelta64)):
 
+    elif issubclass(values.dtype.type, (np.datetime64,np.timedelta64)):
         dtype = values.dtype
         values = values.view(np.int64)
         keys, counts = htable.value_count_int64(values)
 
         # convert the keys back to the dtype we came in
-        keys = Series(keys,dtype=dtype)
+        keys = Series(keys, dtype=dtype)
+
     else:
         mask = com.isnull(values)
         values = com._ensure_object(values)
@@ -188,6 +201,11 @@ def value_counts(values, sort=True, ascending=False, normalize=False):
 
     result = Series(counts, index=com._values_from_object(keys))
 
+    if bins is not None:
+        # TODO: This next line should be more efficient
+        result = result.reindex(np.arange(len(cat.levels)), fill_value=0)
+        result.index = bins[:-1]
+
     if sort:
         result.sort()
         if not ascending:
diff --git a/pandas/core/categorical.py b/pandas/core/categorical.py
--- a/pandas/core/categorical.py
+++ b/pandas/core/categorical.py
@@ -6,7 +6,6 @@
 from pandas.core.base import PandasObject
 from pandas.core.index import Index
 import pandas.core.common as com
-from pandas.core.frame import DataFrame
 
 
 def _cat_compare_op(op):
@@ -182,6 +181,7 @@ def describe(self):
         Returns a dataframe with frequency and counts by level.
         """
         #Hack?
+        from pandas.core.frame import DataFrame
         grouped = DataFrame(self.labels).groupby(0)
         counts = grouped.count().values.squeeze()
         freqs = counts/float(counts.sum())
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -27,6 +27,7 @@
     _is_index_slice, _maybe_convert_indices)
 from pandas.core import generic
 from pandas.core.internals import SingleBlockManager
+from pandas.core.categorical import Categorical
 import pandas.core.expressions as expressions
 from pandas.tseries.index import DatetimeIndex
 from pandas.tseries.period import PeriodIndex, Period
@@ -579,6 +580,10 @@ def __init__(self, data=None, index=None, dtype=None, name=None,
                     index = data.index
                 else:
                     data = data.reindex(index, copy=copy)
+            elif isinstance(data, Categorical):
+                if name is None:
+                    name = data.name
+                data = np.asarray(data)
             elif isinstance(data, types.GeneratorType):
                 data = list(data)
             elif isinstance(data, (set, frozenset)):
@@ -1525,7 +1530,7 @@ def count(self, level=None):
 
         return notnull(_values_from_object(self)).sum()
 
-    def value_counts(self, normalize=False):
+    def value_counts(self, normalize=False, sort=True, ascending=False, bins=None):
         """
         Returns Series containing counts of unique values. The resulting Series
         will be in descending order so that the first element is the most
@@ -1536,14 +1541,21 @@ def value_counts(self, normalize=False):
         normalize: boolean, default False
             If True then the Series returned will contain the relative
             frequencies of the unique values.
+        sort : boolean, default True
+            Sort by values
+        ascending : boolean, default False
+            Sort in ascending order
+        bins : integer, optional
+            Rather than count values, group them into half-open bins,
+            a convenience for pd.cut, only works with numeric data
 
         Returns
         -------
         counts : Series
         """
         from pandas.core.algorithms import value_counts
-        return value_counts(self.values, sort=True, ascending=False,
-                            normalize=normalize)
+        return value_counts(self.values, sort=sort, ascending=ascending,
+                            normalize=normalize, bins=bins)
 
     def unique(self):
         """


Test patch:
diff --git a/pandas/tests/test_algos.py b/pandas/tests/test_algos.py
--- a/pandas/tests/test_algos.py
+++ b/pandas/tests/test_algos.py
@@ -3,7 +3,7 @@
 
 import numpy as np
 
-from pandas.core.api import Series
+from pandas.core.api import Series, Categorical
 import pandas as pd
 
 import pandas.core.algorithms as algos
@@ -63,6 +63,44 @@ def test_on_index_object(self):
 
         tm.assert_almost_equal(result, expected)
 
+class TestValueCounts(unittest.TestCase):
+    _multiprocess_can_split_ = True
+
+    def test_value_counts(self):
+        from pandas.tools.tile import cut
+
+        arr = np.random.randn(4)
+        factor = cut(arr, 4)
+
+        tm.assert_isinstance(factor, Categorical)
+
+        result = algos.value_counts(factor)
+        expected = algos.value_counts(np.asarray(factor))
+        tm.assert_series_equal(result, expected)
+
+    def test_value_counts_bins(self):
+        s = [1, 2, 3, 4]
+        result = algos.value_counts(s, bins=1)
+        self.assertEqual(result.tolist(), [4])
+        self.assertEqual(result.index[0], 0.997)
+
+        result = algos.value_counts(s, bins=2, sort=False)
+        self.assertEqual(result.tolist(), [2, 2])
+        self.assertEqual(result.index[0], 0.997)        
+        self.assertEqual(result.index[1], 2.5)
+
+    def test_value_counts_dtypes(self):
+        result = algos.value_counts([1, 1.])
+        self.assertEqual(len(result), 1)
+
+        result = algos.value_counts([1, 1.], bins=1)
+        self.assertEqual(len(result), 1)
+
+        result = algos.value_counts(Series([1, 1., '1']))  # object
+        self.assertEqual(len(result), 2)
+
+        self.assertRaises(TypeError, lambda s: algos.value_counts(s, bins=1), ['1', 1])
+
 
 def test_quantile():
     s = Series(np.random.randn(100))
diff --git a/pandas/tests/test_categorical.py b/pandas/tests/test_categorical.py
--- a/pandas/tests/test_categorical.py
+++ b/pandas/tests/test_categorical.py
@@ -7,7 +7,6 @@
 
 import numpy as np
 
-from pandas.core.api import value_counts
 from pandas.core.categorical import Categorical
 from pandas.core.index import Index, Int64Index, MultiIndex
 from pandas.core.frame import DataFrame
@@ -89,18 +88,6 @@ def test_comparisons(self):
         expected = np.repeat(False, len(self.factor))
         self.assert_(np.array_equal(result, expected))
 
-    def test_value_counts(self):
-        from pandas.tools.tile import cut
-
-        arr = np.random.randn(4)
-        factor = cut(arr, 4)
-
-        tm.assert_isinstance(factor, Categorical)
-
-        result = value_counts(factor)
-        expected = value_counts(np.asarray(factor))
-        tm.assert_series_equal(result, expected)
-
     def test_na_flags_int_levels(self):
         # #1457
 
diff --git a/pandas/tests/test_series.py b/pandas/tests/test_series.py
--- a/pandas/tests/test_series.py
+++ b/pandas/tests/test_series.py
@@ -371,6 +371,16 @@ def test_constructor_generator(self):
         exp.index = lrange(10, 20)
         assert_series_equal(result, exp)
 
+    def test_constructor_categorical(self):
+        cat = pd.Categorical([0, 1, 2, 0, 1, 2], ['a', 'b', 'c'])
+        res = Series(cat)
+        exp = Series({0: 'a', 1: 'b', 2: 'c', 3: 'a', 4: 'b', 5: 'c'})
+        assert_series_equal(res, exp)
+
+        cat.name = 'foo'
+        res = Series(cat)
+        self.assertEqual(res.name, cat.name)
+
     def test_constructor_maskedarray(self):
         data = ma.masked_all((3,), dtype=float)
         result = Series(data)
@@ -2966,6 +2976,18 @@ def test_value_counts_nunique(self):
         expected = Series([4, 3, 2, 1], index=['b', 'a', 'd', 'c'])
         assert_series_equal(hist, expected)
 
+        # don't sort, have to sort after the fact as not sorting is platform-dep
+        hist = s.value_counts(sort=False)
+        hist.sort()
+        expected = Series([3, 1, 4, 2], index=list('acbd'))
+        expected.sort()
+        assert_series_equal(hist, expected)
+
+        # sort ascending
+        hist = s.value_counts(ascending=True)
+        expected = Series([1, 2, 3, 4], index=list('cdab'))
+        assert_series_equal(hist, expected)
+
         # relative histogram.
         hist = s.value_counts(normalize=True)
         expected = Series([.4, .3, .2, .1], index=['b', 'a', 'd', 'c'])
@@ -2973,6 +2995,24 @@ def test_value_counts_nunique(self):
 
         self.assertEquals(s.nunique(), 4)
 
+        # bins
+        self.assertRaises(TypeError, lambda bins: s.value_counts(bins=bins), 1)
+
+        s1 = Series([1, 1, 2, 3])
+        res1 = s1.value_counts(bins=1)
+        exp1 = Series({0.998: 4})
+        assert_series_equal(res1, exp1)
+        res1n = s1.value_counts(bins=1, normalize=True)
+        exp1n = Series({0.998: 1.0})
+        assert_series_equal(res1n, exp1n)
+
+        res4 = s1.value_counts(bins=4)
+        exp4 = Series({0.998: 2, 1.5: 1, 2.0: 0, 2.5: 1}, index=[0.998, 2.5, 1.5, 2.0])
+        assert_series_equal(res4, exp4)
+        res4n = s1.value_counts(bins=4, normalize=True)
+        exp4n = Series({0.998: 0.5, 1.5: 0.25, 2.0: 0.0, 2.5: 0.25}, index=[0.998, 2.5, 1.5, 2.0])
+        assert_series_equal(res4n, exp4n)
+
         # handle NA's properly
         s[5:7] = np.nan
         hist = s.value_counts()