Make .str/.dt available for Series of type category with string/datetime
State: closed

If a series is a type category and the underlying Categorical has categories of type string or datetime, then make it possible to use the `.str`/`.dt` assessor on such a series.

The string/dt methods work on the categories (and therefore fast if we have
only a few categories), but return a Series with a dtype other than
category (integer, boolean, string,...), so that it is no different if we use
`.str` / `.dt` on a series of type string or of type category.

The main reason for that is that I think things like `s.str.slice(...) + s.str.slice(...)` should work.

Closes: https://github.com/pydata/pandas/issues/10661


Labels:
name: Enhancement
name: API Design
name: Categorical
description: Categorical Data Type

Comments:
I've no idea why `.cat` is not in `dir(series_of_type_category)`?!?

@JanSchulz 
https://github.com/pydata/pandas/blob/master/pandas/core/series.py#L2675

@kawochen Thanks! 7a20c36

Yay, it's green :-)

can you update the doc as well (maybe a reference / example in the Categorical section to one in the TextMethods section)

Added docs in categorical.rst

Ok, @jreback, you were right, it needed more tests...

Rebased + new tests + fixes... Lets wait what travis says...

Ok, two failures: https://travis-ci.org/pydata/pandas/builds/91286698, but I can't reproduce both errors (in py3.5 and py2.7, and it seems I'm too stupid to get a proper py3.4 environment which can run `python setup.py build_ext --inplace` :-( )`.

On py3.3/3.4:
`nosetests pandas.io.tests.test_data`

```
=====================================================================
ERROR: test_get_underlying_price (pandas.io.tests.test_data.TestYahooOptions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pydata/pandas/pandas/util/testing.py", line 1774, in wrapper
    return t(*args, **kwargs)
  File "/home/travis/build/pydata/pandas/pandas/io/tests/test_data.py", line 400, in test_get_underlying_price
    quote_price = options_object._underlying_price_from_root(root)
  File "/home/travis/build/pydata/pandas/pandas/io/data.py", line 737, in _underlying_price_from_root
    underlying_price = root.xpath('.//*[@class="time_rtq_ticker Fz-30 Fw-b"]')[0]\
IndexError: list index out of range
```

Py27, https://travis-ci.org/pydata/pandas/jobs/91286713
`nosetests pandas.tseries.tests.test_timeseries`

```
======================================================================
FAIL: test_datetime64_with_DateOffset (pandas.tseries.tests.test_timeseries.TestDatetimeIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pydata/pandas/pandas/tseries/tests/test_timeseries.py", line 2668, in test_datetime64_with_DateOffset
    assert_func(klass([op + x for x in s]), op + s)
  File "/opt/python/2.7.9/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/travis/build/pydata/pandas/pandas/util/testing.py", line 2042, in assert_produces_warning
    % expected_warning.__name__)
AssertionError: Did not see expected warning of class 'PerformanceWarning'.
```

ignore the first one, 2nd I am fixing now.

Ok, remodeled the tests and rebased. You could cancel the following travis run, the difference to the latest is just some reshuffeling in the commits: https://travis-ci.org/pydata/pandas/builds/91400953

Yay, https://travis-ci.org/pydata/pandas/builds/91400953 is green so hopefull this one (no code change in the PR, only moved some changes to a different commit) will also be green :-)

@jreback green... :-) Thanks for fixing whatever needed fixing :-)

@jreback Hopefully addressed your comments. Thanks for the review!

@JanSchulz Nice! I just added a few minor doc comments

@jorisvandenbossche Addressed your comments. Thanks for the review!

couple comments. pls squash when finished.

@jreback: please no squash: I really put some work into making each commit stand on its own, the unit of change is each commit (4 of them...) and not the complete PR.

Ok, addressed the comments, please review the doc change if that is clearer now.

ok, lgtm. ping on green.

on the squash; ok for this one :) normally we just squashem as a matter of course.

@JanSchulz ok, ping when green (after any more doc changes you have)

Ok, added something to text.rst. Please review. It would also good if someone with access to travis would cancel the other builds  so that we have a few hours less to wait for this :-)

@jreback ping :-)

merged via https://github.com/pydata/pandas/commit/6eefe756a9cdd8d8b9efc95920f8434245bba8cf

thanks!

as usual, pls review built docs for accuracy

@jreback Thanks for all the reviews and for your patience! The PR got much better thanks to you and @jorisvandenbossche!

Will add a PR forthis:

```
In [124]: date_s = pd.Series(date_range('1/1/2015', periods=5))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-124-f7c79f210480> in <module>()
----> 1 date_s = pd.Series(date_range('1/1/2015', periods=5))

NameError: name 'date_range' is not defined
```

@JanSchulz no thank you!

great PR as always from you.

I got the doc fix, was making some corrections anyhow.