Repository: pandas-dev/pandas

Issue:
date_range treats '.5s' as '5s'
I'm new to pandas, so maybe I'm missing something, but I'd expected freq='.5s' to give me 500ms intervals, not 5s intervals:

> > > pd.date_range(0, periods=5, freq='.5s')[1]
> > > Timestamp('1970-01-01 00:00:05', offset='5S')
> > > pd.date_range(0, periods=5, freq='500ms')[1]
> > > Timestamp('1970-01-01 00:00:00.500000', offset='500L')
> > > pd.**version**
> > > '0.14.1'
> > > import sys; sys.version
> > > '3.4.1 (default, Jul  6 2014, 20:01:46) \n[GCC 4.9.0]'

Behaviour is the same under Python 2 and 3.

date_range treats '.5s' as '5s'
I'm new to pandas, so maybe I'm missing something, but I'd expected freq='.5s' to give me 500ms intervals, not 5s intervals:

> > > pd.date_range(0, periods=5, freq='.5s')[1]
> > > Timestamp('1970-01-01 00:00:05', offset='5S')
> > > pd.date_range(0, periods=5, freq='500ms')[1]
> > > Timestamp('1970-01-01 00:00:00.500000', offset='500L')
> > > pd.**version**
> > > '0.14.1'
> > > import sys; sys.version
> > > '3.4.1 (default, Jul  6 2014, 20:01:46) \n[GCC 4.9.0]'

Behaviour is the same under Python 2 and 3.



Discussion:
these don't take fractional units ATM (I don't think was ever implemented). I don't think very difficult to fix. welcome to have a pull-request.

https://github.com/pydata/pandas/blob/master/pandas/tseries/frequencies.py#L97

I think this may have to do with the integer division when assigned values to a freq group. For example, if you also try .05s, it gets cut to 5S. I believe this is because of the (freq // 1000) operation used in grouping.

Someone else might be able to correct me or give additional insights.

Edit: jreback beat me to it. I might look into this tonight as it seems like it would be interesting, but simple. No promises though.

@jbradish sounds great!

basically you need to patch `pandas.tseries.frequencies.to_offset()` to handle.

I think fix should be 2 parts:

#### 1. `offsets.Tick` support fraction

Currently, `Tick` and `DateOffset` coerces input to `int` implicitly. I think `Tick` can return smaller tick to based on fraction, and `DateOffset` should explicitly raise.

```
pd.tseries.offsets.Hour(0.5)
# <0 * Hours>

pd.tseries.offsets.MonthEnd(0.5)
# <0 * MonthEnds>
```

#### 2. `to_offset` support fraction

After `Tick` support fraction, it can be done with a slight change of regex and impl (related to #13930).

these don't take fractional units ATM (I don't think was ever implemented). I don't think very difficult to fix. welcome to have a pull-request.

https://github.com/pydata/pandas/blob/master/pandas/tseries/frequencies.py#L97

I think this may have to do with the integer division when assigned values to a freq group. For example, if you also try .05s, it gets cut to 5S. I believe this is because of the (freq // 1000) operation used in grouping.

Someone else might be able to correct me or give additional insights.

Edit: jreback beat me to it. I might look into this tonight as it seems like it would be interesting, but simple. No promises though.

@jbradish sounds great!

basically you need to patch `pandas.tseries.frequencies.to_offset()` to handle.

I think fix should be 2 parts:

#### 1. `offsets.Tick` support fraction

Currently, `Tick` and `DateOffset` coerces input to `int` implicitly. I think `Tick` can return smaller tick to based on fraction, and `DateOffset` should explicitly raise.

```
pd.tseries.offsets.Hour(0.5)
# <0 * Hours>

pd.tseries.offsets.MonthEnd(0.5)
# <0 * MonthEnds>
```

#### 2. `to_offset` support fraction

After `Tick` support fraction, it can be done with a slight change of regex and impl (related to #13930).


Resolving patch:
diff --git a/pandas/tseries/frequencies.py b/pandas/tseries/frequencies.py
--- a/pandas/tseries/frequencies.py
+++ b/pandas/tseries/frequencies.py
@@ -38,32 +38,55 @@ class FreqGroup(object):
     FR_NS = 12000
 
 
-US_RESO = 0
-MS_RESO = 1
-S_RESO = 2
-T_RESO = 3
-H_RESO = 4
-D_RESO = 5
+RESO_NS = 0
+RESO_US = 1
+RESO_MS = 2
+RESO_SEC = 3
+RESO_MIN = 4
+RESO_HR = 5
+RESO_DAY = 6
 
 
 class Resolution(object):
 
-    # defined in period.pyx
-    # note that these are different from freq codes
-    RESO_US = US_RESO
-    RESO_MS = MS_RESO
-    RESO_SEC = S_RESO
-    RESO_MIN = T_RESO
-    RESO_HR = H_RESO
-    RESO_DAY = D_RESO
+    RESO_US = RESO_US
+    RESO_MS = RESO_MS
+    RESO_SEC = RESO_SEC
+    RESO_MIN = RESO_MIN
+    RESO_HR = RESO_HR
+    RESO_DAY = RESO_DAY
 
     _reso_str_map = {
+        RESO_NS: 'nanosecond',
         RESO_US: 'microsecond',
         RESO_MS: 'millisecond',
         RESO_SEC: 'second',
         RESO_MIN: 'minute',
         RESO_HR: 'hour',
-        RESO_DAY: 'day'}
+        RESO_DAY: 'day'
+    }
+
+    # factor to multiply a value by to convert it to the next finer grained
+    # resolution
+    _reso_mult_map = {
+        RESO_NS: None,
+        RESO_US: 1000,
+        RESO_MS: 1000,
+        RESO_SEC: 1000,
+        RESO_MIN: 60,
+        RESO_HR: 60,
+        RESO_DAY: 24
+    }
+
+    _reso_str_bump_map = {
+        'D': 'H',
+        'H': 'T',
+        'T': 'S',
+        'S': 'L',
+        'L': 'U',
+        'U': 'N',
+        'N': None
+    }
 
     _str_reso_map = dict([(v, k) for k, v in compat.iteritems(_reso_str_map)])
 
@@ -160,6 +183,47 @@ def get_reso_from_freq(cls, freq):
         """
         return cls.get_reso(cls.get_str_from_freq(freq))
 
+    @classmethod
+    def get_stride_from_decimal(cls, value, freq):
+        """
+        Convert freq with decimal stride into a higher freq with integer stride
+
+        Parameters
+        ----------
+        value : integer or float
+        freq : string
+            Frequency string
+
+        Raises
+        ------
+        ValueError
+            If the float cannot be converted to an integer at any resolution.
+
+        Example
+        -------
+        >>> Resolution.get_stride_from_decimal(1.5, 'T')
+        (90, 'S')
+
+        >>> Resolution.get_stride_from_decimal(1.04, 'H')
+        (3744, 'S')
+
+        >>> Resolution.get_stride_from_decimal(1, 'D')
+        (1, 'D')
+        """
+
+        if np.isclose(value % 1, 0):
+            return int(value), freq
+        else:
+            start_reso = cls.get_reso_from_freq(freq)
+            if start_reso == 0:
+                raise ValueError(
+                    "Could not convert to integer offset at any resolution"
+                )
+
+            next_value = cls._reso_mult_map[start_reso] * value
+            next_name = cls._reso_str_bump_map[freq]
+            return cls.get_stride_from_decimal(next_value, next_name)
+
 
 def get_to_timestamp_base(base):
     """
@@ -472,12 +536,17 @@ def to_offset(freq):
                                          splitted[2::4]):
                 if sep != '' and not sep.isspace():
                     raise ValueError('separator must be spaces')
-                offset = get_offset(name)
+                prefix = _lite_rule_alias.get(name) or name
                 if stride_sign is None:
                     stride_sign = -1 if stride.startswith('-') else 1
                 if not stride:
                     stride = 1
+                if prefix in Resolution._reso_str_bump_map.keys():
+                    stride, name = Resolution.get_stride_from_decimal(
+                        float(stride), prefix
+                    )
                 stride = int(stride)
+                offset = get_offset(name)
                 offset = offset * int(np.fabs(stride) * stride_sign)
                 if delta is None:
                     delta = offset
@@ -493,7 +562,9 @@ def to_offset(freq):
 
 
 # hack to handle WOM-1MON
-opattern = re.compile(r'([\-]?\d*)\s*([A-Za-z]+([\-][\dA-Za-z\-]+)?)')
+opattern = re.compile(
+    r'([\-]?\d*|[\-]?\d*\.\d*)\s*([A-Za-z]+([\-][\dA-Za-z\-]+)?)'
+)
 
 
 def _base_and_stride(freqstr):


Test patch:
diff --git a/pandas/tseries/tests/test_frequencies.py b/pandas/tseries/tests/test_frequencies.py
--- a/pandas/tseries/tests/test_frequencies.py
+++ b/pandas/tseries/tests/test_frequencies.py
@@ -39,6 +39,21 @@ def test_to_offset_multiple(self):
         expected = offsets.Hour(3)
         assert (result == expected)
 
+        freqstr = '2h 20.5min'
+        result = frequencies.to_offset(freqstr)
+        expected = offsets.Second(8430)
+        assert (result == expected)
+
+        freqstr = '1.5min'
+        result = frequencies.to_offset(freqstr)
+        expected = offsets.Second(90)
+        assert (result == expected)
+
+        freqstr = '0.5S'
+        result = frequencies.to_offset(freqstr)
+        expected = offsets.Milli(500)
+        assert (result == expected)
+
         freqstr = '15l500u'
         result = frequencies.to_offset(freqstr)
         expected = offsets.Micro(15500)
@@ -49,6 +64,16 @@ def test_to_offset_multiple(self):
         expected = offsets.Milli(10075)
         assert (result == expected)
 
+        freqstr = '1s0.25ms'
+        result = frequencies.to_offset(freqstr)
+        expected = offsets.Micro(1000250)
+        assert (result == expected)
+
+        freqstr = '1s0.25L'
+        result = frequencies.to_offset(freqstr)
+        expected = offsets.Micro(1000250)
+        assert (result == expected)
+
         freqstr = '2800N'
         result = frequencies.to_offset(freqstr)
         expected = offsets.Nano(2800)
@@ -107,10 +132,8 @@ def test_to_offset_invalid(self):
             frequencies.to_offset('-2-3U')
         with tm.assertRaisesRegexp(ValueError, 'Invalid frequency: -2D:3H'):
             frequencies.to_offset('-2D:3H')
-
-        # ToDo: Must be fixed in #8419
-        with tm.assertRaisesRegexp(ValueError, 'Invalid frequency: .5S'):
-            frequencies.to_offset('.5S')
+        with tm.assertRaisesRegexp(ValueError, 'Invalid frequency: 1.5.0S'):
+            frequencies.to_offset('1.5.0S')
 
         # split offsets with spaces are valid
         assert frequencies.to_offset('2D 3H') == offsets.Hour(51)
@@ -379,6 +402,26 @@ def test_freq_to_reso(self):
             result = Reso.get_freq(Reso.get_str(Reso.get_reso_from_freq(freq)))
             self.assertEqual(freq, result)
 
+    def test_resolution_bumping(self):
+        # GH 14378
+        Reso = frequencies.Resolution
+
+        self.assertEqual(Reso.get_stride_from_decimal(1.5, 'T'), (90, 'S'))
+        self.assertEqual(Reso.get_stride_from_decimal(62.4, 'T'), (3744, 'S'))
+        self.assertEqual(Reso.get_stride_from_decimal(1.04, 'H'), (3744, 'S'))
+        self.assertEqual(Reso.get_stride_from_decimal(1, 'D'), (1, 'D'))
+        self.assertEqual(Reso.get_stride_from_decimal(0.342931, 'H'),
+                         (1234551600, 'U'))
+        self.assertEqual(Reso.get_stride_from_decimal(1.2345, 'D'),
+                         (106660800, 'L'))
+
+        with self.assertRaises(ValueError):
+            Reso.get_stride_from_decimal(0.5, 'N')
+
+        # too much precision in the input can prevent
+        with self.assertRaises(ValueError):
+            Reso.get_stride_from_decimal(0.3429324798798269273987982, 'H')
+
     def test_get_freq_code(self):
         # freqstr
         self.assertEqual(frequencies.get_freq_code('A'),
diff --git a/pandas/tseries/tests/test_tslib.py b/pandas/tseries/tests/test_tslib.py
--- a/pandas/tseries/tests/test_tslib.py
+++ b/pandas/tseries/tests/test_tslib.py
@@ -14,7 +14,7 @@
 from pandas.tseries.index import date_range, DatetimeIndex
 from pandas.tseries.frequencies import (
     get_freq,
-    US_RESO, MS_RESO, S_RESO, H_RESO, D_RESO, T_RESO
+    RESO_US, RESO_MS, RESO_SEC, RESO_HR, RESO_DAY, RESO_MIN
 )
 import pandas.tseries.tools as tools
 import pandas.tseries.offsets as offsets
@@ -1528,11 +1528,11 @@ def test_resolution(self):
 
         for freq, expected in zip(['A', 'Q', 'M', 'D', 'H', 'T',
                                    'S', 'L', 'U'],
-                                  [D_RESO, D_RESO,
-                                   D_RESO, D_RESO,
-                                   H_RESO, T_RESO,
-                                   S_RESO, MS_RESO,
-                                   US_RESO]):
+                                  [RESO_DAY, RESO_DAY,
+                                   RESO_DAY, RESO_DAY,
+                                   RESO_HR, RESO_MIN,
+                                   RESO_SEC, RESO_MS,
+                                   RESO_US]):
             for tz in [None, 'Asia/Tokyo', 'US/Eastern',
                        'dateutil/US/Eastern']:
                 idx = date_range(start='2013-04-01', periods=30, freq=freq,