ENH: add to_offset method to Timedelta #9064
State: closed

Closes #9064


Labels:
name: Enhancement
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Timedelta
description: Timedelta data type
name: Frequency
description: DateOffsets

Comments:
Another option is to ensure it is first a `pd.Timedelta` and then use `total_seconds` (this always has that, also on python 2.6).

Some other things:
- can you update the docstring of `to_offset`
- needs some more tests I think. Certainly test for both `datetime.timedelta/pandas.Timedelta` input
- I was wondering if it would be possible that eg `to_offset(pd.Timedelta('1 day'))` would return a 1 Day offset instead of 86400 Seconds offset.

cc @bjonen @cancan101

your method will not handle < 1 sec if its not divisible. better to start go thru the time hierarchy and add offsets. e.g. `pd.Timedelta('1 day 1 sec') == Day(1) + Second(1)`, you can use the `.components` property to get these.

@jorisvandenbossche working on these
- add handling of pd.Timedelta objects. 
- update the docstring of to_offset. 
- Looking into keeping the resolution same e.g., Day(1) to Day(1) instead of Second(86400). I guess this would be taken care of if I could traverse the heirarchy and add each offset as it is (suggestion by @jreback)

@jreback my initial understanding was that converting everything into total_seconds would serve the purpose. However, going through the heirarchy and adding the offsets certainly sounds like a better way. Looking into this.

Thanks,

@jorisvandenbossche I will add more tests.

A question. @jreback when you say .components, are you referring to something similar to what I found in timedeltas.rst? -> "You can use the .components property to access a reduced form of the timedelta. This returns a DataFrame indexed similarly to the Series" or did you mean directly use the attributes days,hours,minutes,seconds,milliseconds,microseconds,nanoseconds? Thanks.

.components of a TimeDelta is a named tuple with the data u need

you can better use the `.components`, as the others we are probably going to change.

Ok thanks. I am seeing an inconsistency though. pd.Timedelta doesn't allow nanoseconds in constructor but its components list till nanoseconds. Here is what I can reproduce.

<pre>
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pandas.tslib import Timedelta
>>> td = Timedelta(nanoseconds=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pandas/tslib.pyx", line 1723, in pandas.tslib.Timedelta.__new__ (pandas/tslib.c:29743)
raise ValueError("cannot construct a TimeDelta from the passed arguments, allowed keywords 
are " [days, seconds, microseconds, milliseconds, minutes, hours, weeks]
>>> td=Timedelta(seconds=1)
>>> td.components._fields
('days', 'hours', 'minutes', 'seconds', 'milliseconds', 'microseconds', 'nanoseconds')
</pre>

hmm
nanoseconds is not allowed as an argument to datetime.timedelta but I t should be allowed here

u can add that if u want (and a separate test)

Also, if I try to add Nano() DateOffset to any other pd.DateOffset on my machine, I get this error: TypeError: ufunc add cannot use operands with types dtype('O') and dtype('<m8[ns]'). Something I need to fix in my build etc.?

Example:

<pre>
>>> from pandas import offsets
>>> d1=offsets.Second(1)
>>> d2=offsets.Milli(1)
>>> d3=offsets.Micro(1)
>>> d4=offsets.Nano(1)
>>> 5*d1+4*d2+3*d3+2*d4
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/tseries/offsets.py", line 2027, in __add__
    return _delta_to_tick(self.delta + other.delta)
TypeError: ufunc add cannot use operands with types dtype('O') and dtype('&lt;m8[ns]')
>>> 5*d1+4*d2+3*d3
&lt;5004003 * Micros&gt;
>>> 5*d1+4*d2+0*d3
&lt;5004 * Millis&gt;
>>> 5*d1+4*d2+1*d3
&lt;5004001 * Micros&gt;
</pre>

Thanks,

Thanks @jreback I will club adding nanoseconds in constructor to this change.

hmm adding a Nano should work
maybe not converted properly (and clearly not tested)

Ok thanks, I will need to look into these more and see if I spot a fix. I guess these need to be fixed first to properly traverse hierarchy and add offsets otherwise things seem to be working fine till microseconds.

@jorisvandenbossche, @jreback I have updated the PR with the following changes:
- to_offset can take either datetime.timedelta or pd.Timedelta as input
- changed doc string of the function accordingly
- fixed output time resolution to be the same as input
- added more tests

PLMK if there are better ways to get DateOffset objects from td.components._fields. I have created a local name to offset object map in the method. Will attend to review comments as they come by. In the meantime, I will look into: a) why adding Nano to DateOffset is failing and b) adding support of Nano in pd.Timedelta constructor.

From what I see, support for nano is broken in Timedelta and I am able to reproduce that in various ways. However, it seems like this ENH should should not need any further changes related to the nano fixes. Would it be a good idea to treat these two separately? I can go ahead and create a separate issue for the nano fix and continue the fix related discussions there, while this ENH can be merged independently. 

couple of minor changes, pls rebase and squash.

Incorporated review comments and squashed. Created a new issue to track nano fix here. https://github.com/pydata/pandas/issues/9273. Thanks.

ok, looks good. Pls add a release note referencing the original issue. You can put it in the enhancements section.

@jreback done adding note

minor import correction, otherwise looks good. ping after you push and its green.

Ok, changed imports, docstring and release note. Thanks.

@tvyomkesh thanks!