Repository: pandas-dev/pandas

Issue:
Openpyxl Timedelta Behavior Inconsistent in Excel Roundtrip
When round-tripping excel files in tandem with Timedelta data, the values returned by `openpyxl` differ from those returned by `xlwt` and `xlsxwriter`. To illustrate:

```python
In []: df = pd.DataFrame([pd.to_timedelta(1, 'D')])

In []: df.to_excel('some_file.xls') # xlwt
    .: pd.read_excel('some_file.xls')
Out[]: 
   0
0  1

In []: pd.set_option('io.excel.xlsx.writer', 'xlsxwriter')
In []: df.to_excel('some_file.xlsx')
    .: pd.read_excel('some_file.xlsx')
Out[]: 
   0
0  1

In []: pd.set_option('io.excel.xlsx.writer', 'openpyxl')
In []: df.to_excel('some_file.xlsx')
    .: pd.read_excel('some_file.xlsx')
Out[]: 
           0
0 1900-01-01
```

Only the `xlwt` option was being tested previously, but I uncovered this nuance while refactoring tests in #19829.

While the `openpyxl` result maintains date-like metadata I question if it's worth being different from the other engines, especially since that metadata misrepresents the object as a datetime and not necessarily a Timedelta (assuming the latter doesn't exist in Excel). Curious to hear others' thoughts

<details>

INSTALLED VERSIONS
------------------
commit: 92dbc78af9f34af83691d86347fc04d2c543b94c
python: 3.6.4.final.0
python-bits: 64
OS: Darwin
OS-release: 17.4.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.0.dev0+383.g92dbc78af
pytest: 3.4.1
pip: 9.0.1
setuptools: 38.5.1
Cython: 0.27.3
numpy: 1.14.1
scipy: 1.0.0
pyarrow: 0.8.0
xarray: 0.10.0
IPython: 6.2.1
sphinx: 1.7.0
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: 0.4.0
matplotlib: 2.1.2
openpyxl: 2.5.0
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.2.1
pymysql: 0.8.0
psycopg2: None
jinja2: 2.10
s3fs: 0.1.3
fastparquet: 0.1.4
pandas_gbq: None
pandas_datareader: None

</details>


Xlsxwriter, Py27 and Linux Failure on to_excel with Timedelta
See [this comment](https://github.com/pandas-dev/pandas/pull/19829#pullrequestreview-99172569) in #19829. The test to write Timedelta data to an excel file is failing only with the combination of xlsxwriter as the engine using Py27 and on Linux. All other engines (save openpyxl which has an issue referenced in #19900), Python versions and OSes appear to work, but this particular combination kept failing on Travis. 

Need help from someone running Linux to debug further. Right now the test cases are imperatively skipping this combination:

https://github.com/pandas-dev/pandas/blob/1e4c50a56f7e953ab84308f000dff6fc1ac71171/pandas/tests/io/test_excel.py#L1378

[Here's the Travis build that was failing](https://travis-ci.org/pandas-dev/pandas/jobs/346081657) and I've provided the traceback of the test below for reference

<details>

```python
=================================== FAILURES ===================================
[31m[1m________ TestExcelWriter.test_to_excel_timedelta[xlsxwriter-.xlsx-True] ________[0m
[gw0] linux2 -- Python 2.7.14 /home/travis/miniconda3/envs/pandas/bin/python

self = <pandas.tests.io.test_excel.TestExcelWriter object at 0x7f44821a0b50>
merge_cells = True, engine = 'xlsxwriter', ext = '.xlsx'

[1m    def test_to_excel_timedelta(self, merge_cells, engine, ext):[0m
[1m        # GH 19242, GH9155 - test writing timedelta to xls[0m
[1m        if engine == 'openpyxl':[0m
[1m            pytest.xfail('Timedelta roundtrip broken with openpyxl')[0m
[1m        frame = DataFrame(np.random.randint(-10, 10, size=(20, 1)),[0m
[1m                          columns=['A'],[0m
[1m                          dtype=np.int64[0m
[1m                          )[0m
[1m        expected = frame.copy()[0m
[1m        frame['new'] = frame['A'].apply(lambda x: timedelta(seconds=x))[0m
[1m        expected['new'] = expected['A'].apply([0m
[1m            lambda x: timedelta(seconds=x).total_seconds() / float(86400))[0m
[1m>       frame.to_excel(self.path, 'test1')[0m

[31m[1mpandas/tests/io/test_excel.py[0m:1384: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[31m[1mpandas/core/frame.py[0m:1599: in to_excel
[1m    engine=engine)[0m
[31m[1mpandas/io/formats/excel.py[0m:652: in write
[1m    freeze_panes=freeze_panes)[0m
[31m[1mpandas/io/excel.py[0m:1772: in write_cells
[1m    val, style)[0m
[31m[1m../../../miniconda3/envs/pandas/lib/python2.7/site-packages/xlsxwriter/worksheet.py[0m:57: in cell_wrapper
[1m    return method(self, *args, **kwargs)[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <xlsxwriter.worksheet.Worksheet object at 0x7f44821a04d0>, row = 1
col = 2, args = (Timedelta('-1 days +23:59:52'), None)
token = Timedelta('-1 days +23:59:52')
num_types = (<type 'float'>, <type 'int'>, <type 'long'>, <class 'decimal.Decimal'>, <class 'fractions.Fraction'>)
str_types = <type 'basestring'>

[1m    @convert_cell_args[0m
[1m    def write(self, row, col, *args):[0m
[1m        """[0m
[1m            Write data to a worksheet cell by calling the appropriate write_*()[0m
[1m            method based on the type of data being passed.[0m
[1m    [0m
[1m            Args:[0m
[1m                row:   The cell row (zero indexed).[0m
[1m                col:   The cell column (zero indexed).[0m
[1m                *args: Args to pass to sub functions.[0m
[1m    [0m
[1m            Returns:[0m
[1m                 0:    Success.[0m
[1m                -1:    Row or column is out of worksheet bounds.[0m
[1m                other: Return value of called method.[0m
[1m    [0m
[1m            """[0m
[1m        # Check the number of args passed.[0m
[1m        if not len(args):[0m
[1m            raise TypeError("write() takes at least 4 arguments (3 given)")[0m
[1m    [0m
[1m        # The first arg should be the token for all write calls.[0m
[1m        token = args[0][0m
[1m    [0m
[1m        # Types to check in Python 2/3.[0m
[1m        if sys.version_info[0] == 2:[0m
[1m            num_types = (float, int, long, Decimal, Fraction)[0m
[1m            str_types = basestring[0m
[1m            date_types = (datetime.datetime, datetime.date, datetime.time)[0m
[1m        else:[0m
[1m            num_types = (float, int, Decimal, Fraction)[0m
[1m            str_types = str[0m
[1m            date_types = (datetime.datetime, datetime.date, datetime.time)[0m
[1m    [0m
[1m        # Write None as a blank cell.[0m
[1m        if token is None:[0m
[1m            return self.write_blank(row, col, *args)[0m
[1m    [0m
[1m        # Write boolean types.[0m
[1m        if isinstance(token, bool):[0m
[1m            return self.write_boolean(row, col, *args)[0m
[1m    [0m
[1m        # Write datetime objects.[0m
[1m        if isinstance(token, date_types):[0m
[1m            return self.write_datetime(row, col, *args)[0m
[1m    [0m
[1m        # Write number types.[0m
[1m        if isinstance(token, num_types):[0m
[1m            return self.write_number(row, col, *args)[0m
[1m    [0m
[1m        # Write string types.[0m
[1m        if isinstance(token, str_types):[0m
[1m            # Map the data to the appropriate write_*() method.[0m
[1m            if token == '':[0m
[1m                return self.write_blank(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_formulas and token.startswith('='):[0m
[1m                return self.write_formula(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('(ftp|http)s?://', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('mailto:', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('(in|ex)ternal:', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_numbers:[0m
[1m                try:[0m
[1m                    f = float(token)[0m
[1m                    if not self._isnan(f) and not self._isinf(f):[0m
[1m                        return self.write_number(row, col, f, *args[1:])[0m
[1m                except ValueError:[0m
[1m                    # Not a number, write as a string.[0m
[1m                    pass[0m
[1m    [0m
[1m                return self.write_string(row, col, *args)[0m
[1m    [0m
[1m            else:[0m
[1m                # We have a plain string.[0m
[1m                return self.write_string(row, col, *args)[0m
[1m    [0m
[1m        # We haven't matched a supported type. Try float.[0m
[1m        try:[0m
[1m>           f = float(token)[0m
[31m[1mE           TypeError: float() argument must be a string or a number[0m

[31m[1m../../../miniconda3/envs/pandas/lib/python2.7/site-packages/xlsxwriter/worksheet.py[0m:417: TypeError
[31m[1m_______ TestExcelWriter.test_to_excel_timedelta[xlsxwriter-.xlsx-False] ________[0m
[gw0] linux2 -- Python 2.7.14 /home/travis/miniconda3/envs/pandas/bin/python

self = <pandas.tests.io.test_excel.TestExcelWriter object at 0x7f4481b83750>
merge_cells = False, engine = 'xlsxwriter', ext = '.xlsx'

[1m    def test_to_excel_timedelta(self, merge_cells, engine, ext):[0m
[1m        # GH 19242, GH9155 - test writing timedelta to xls[0m
[1m        if engine == 'openpyxl':[0m
[1m            pytest.xfail('Timedelta roundtrip broken with openpyxl')[0m
[1m        frame = DataFrame(np.random.randint(-10, 10, size=(20, 1)),[0m
[1m                          columns=['A'],[0m
[1m                          dtype=np.int64[0m
[1m                          )[0m
[1m        expected = frame.copy()[0m
[1m        frame['new'] = frame['A'].apply(lambda x: timedelta(seconds=x))[0m
[1m        expected['new'] = expected['A'].apply([0m
[1m            lambda x: timedelta(seconds=x).total_seconds() / float(86400))[0m
[1m>       frame.to_excel(self.path, 'test1')[0m

[31m[1mpandas/tests/io/test_excel.py[0m:1384: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[31m[1mpandas/core/frame.py[0m:1599: in to_excel
[1m    engine=engine)[0m
[31m[1mpandas/io/formats/excel.py[0m:652: in write
[1m    freeze_panes=freeze_panes)[0m
[31m[1mpandas/io/excel.py[0m:1772: in write_cells
[1m    val, style)[0m
[31m[1m../../../miniconda3/envs/pandas/lib/python2.7/site-packages/xlsxwriter/worksheet.py[0m:57: in cell_wrapper
[1m    return method(self, *args, **kwargs)[0m
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <xlsxwriter.worksheet.Worksheet object at 0x7f4481a754d0>, row = 1
col = 2, args = (Timedelta('-1 days +23:59:54'), None)
token = Timedelta('-1 days +23:59:54')
num_types = (<type 'float'>, <type 'int'>, <type 'long'>, <class 'decimal.Decimal'>, <class 'fractions.Fraction'>)
str_types = <type 'basestring'>

[1m    @convert_cell_args[0m
[1m    def write(self, row, col, *args):[0m
[1m        """[0m
[1m            Write data to a worksheet cell by calling the appropriate write_*()[0m
[1m            method based on the type of data being passed.[0m
[1m    [0m
[1m            Args:[0m
[1m                row:   The cell row (zero indexed).[0m
[1m                col:   The cell column (zero indexed).[0m
[1m                *args: Args to pass to sub functions.[0m
[1m    [0m
[1m            Returns:[0m
[1m                 0:    Success.[0m
[1m                -1:    Row or column is out of worksheet bounds.[0m
[1m                other: Return value of called method.[0m
[1m    [0m
[1m            """[0m
[1m        # Check the number of args passed.[0m
[1m        if not len(args):[0m
[1m            raise TypeError("write() takes at least 4 arguments (3 given)")[0m
[1m    [0m
[1m        # The first arg should be the token for all write calls.[0m
[1m        token = args[0][0m
[1m    [0m
[1m        # Types to check in Python 2/3.[0m
[1m        if sys.version_info[0] == 2:[0m
[1m            num_types = (float, int, long, Decimal, Fraction)[0m
[1m            str_types = basestring[0m
[1m            date_types = (datetime.datetime, datetime.date, datetime.time)[0m
[1m        else:[0m
[1m            num_types = (float, int, Decimal, Fraction)[0m
[1m            str_types = str[0m
[1m            date_types = (datetime.datetime, datetime.date, datetime.time)[0m
[1m    [0m
[1m        # Write None as a blank cell.[0m
[1m        if token is None:[0m
[1m            return self.write_blank(row, col, *args)[0m
[1m    [0m
[1m        # Write boolean types.[0m
[1m        if isinstance(token, bool):[0m
[1m            return self.write_boolean(row, col, *args)[0m
[1m    [0m
[1m        # Write datetime objects.[0m
[1m        if isinstance(token, date_types):[0m
[1m            return self.write_datetime(row, col, *args)[0m
[1m    [0m
[1m        # Write number types.[0m
[1m        if isinstance(token, num_types):[0m
[1m            return self.write_number(row, col, *args)[0m
[1m    [0m
[1m        # Write string types.[0m
[1m        if isinstance(token, str_types):[0m
[1m            # Map the data to the appropriate write_*() method.[0m
[1m            if token == '':[0m
[1m                return self.write_blank(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_formulas and token.startswith('='):[0m
[1m                return self.write_formula(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('(ftp|http)s?://', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('mailto:', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_urls and re.match('(in|ex)ternal:', token):[0m
[1m                return self.write_url(row, col, *args)[0m
[1m    [0m
[1m            elif self.strings_to_numbers:[0m
[1m                try:[0m
[1m                    f = float(token)[0m
[1m                    if not self._isnan(f) and not self._isinf(f):[0m
[1m                        return self.write_number(row, col, f, *args[1:])[0m
[1m                except ValueError:[0m
[1m                    # Not a number, write as a string.[0m
[1m                    pass[0m
[1m    [0m
[1m                return self.write_string(row, col, *args)[0m
[1m    [0m
[1m            else:[0m
[1m                # We have a plain string.[0m
[1m                return self.write_string(row, col, *args)[0m
[1m    [0m
[1m        # We haven't matched a supported type. Try float.[0m
[1m        try:[0m
[1m>           f = float(token)[0m
[31m[1mE           TypeError: float() argument must be a string or a number[0m

[31m[1m../../../miniconda3/envs/pandas/lib/python2.7/site-packages/xlsxwriter/worksheet.py[0m:417: TypeError
```

</details>


Discussion:

Resolving patch:
diff --git a/pandas/io/excel.py b/pandas/io/excel.py
--- a/pandas/io/excel.py
+++ b/pandas/io/excel.py
@@ -775,35 +775,6 @@ def _pop_header_name(row, index_col):
         return none_fill(row[i]), row[:i] + [''] + row[i + 1:]
 
 
-def _conv_value(val):
-    """ Convert numpy types to Python types for the Excel writers.
-
-        Parameters
-        ----------
-        val : object
-            Value to be written into cells
-
-        Returns
-        -------
-        If val is a numpy int, float, or bool, then the equivalent Python
-        types are returned. :obj:`datetime`, :obj:`date`, and :obj:`timedelta`
-        are passed and formatting must be handled in the writer. :obj:`str`
-        representation is returned for all other types.
-    """
-    if is_integer(val):
-        val = int(val)
-    elif is_float(val):
-        val = float(val)
-    elif is_bool(val):
-        val = bool(val)
-    elif isinstance(val, (datetime, date, timedelta)):
-        pass
-    else:
-        val = compat.to_str(val)
-
-    return val
-
-
 @add_metaclass(abc.ABCMeta)
 class ExcelWriter(object):
     """
@@ -949,6 +920,39 @@ def _get_sheet_name(self, sheet_name):
                              'cur_sheet property')
         return sheet_name
 
+    def _value_with_fmt(self, val):
+        """Convert numpy types to Python types for the Excel writers.
+
+        Parameters
+        ----------
+        val : object
+            Value to be written into cells
+
+        Returns
+        -------
+        Tuple with the first element being the converted value and the second
+            being an optional format
+        """
+        fmt = None
+
+        if is_integer(val):
+            val = int(val)
+        elif is_float(val):
+            val = float(val)
+        elif is_bool(val):
+            val = bool(val)
+        elif isinstance(val, datetime):
+            fmt = self.datetime_format
+        elif isinstance(val, date):
+            fmt = self.date_format
+        elif isinstance(val, timedelta):
+            val = val.total_seconds() / float(86400)
+            fmt = '0'
+        else:
+            val = compat.to_str(val)
+
+        return val, fmt
+
     @classmethod
     def check_extension(cls, ext):
         """checks that path's extension against the Writer's supported
@@ -1378,7 +1382,9 @@ def write_cells(self, cells, sheet_name=None, startrow=0, startcol=0,
                 row=startrow + cell.row + 1,
                 column=startcol + cell.col + 1
             )
-            xcell.value = _conv_value(cell.val)
+            xcell.value, fmt = self._value_with_fmt(cell.val)
+            if fmt:
+                xcell.number_format = fmt
 
             style_kwargs = {}
             if cell.style:
@@ -1465,25 +1471,16 @@ def write_cells(self, cells, sheet_name=None, startrow=0, startcol=0,
         style_dict = {}
 
         for cell in cells:
-            val = _conv_value(cell.val)
-
-            num_format_str = None
-            if isinstance(cell.val, datetime):
-                num_format_str = self.datetime_format
-            elif isinstance(cell.val, date):
-                num_format_str = self.date_format
-            elif isinstance(cell.val, timedelta):
-                delta = cell.val
-                val = delta.total_seconds() / float(86400)
+            val, fmt = self._value_with_fmt(cell.val)
 
             stylekey = json.dumps(cell.style)
-            if num_format_str:
-                stylekey += num_format_str
+            if fmt:
+                stylekey += fmt
 
             if stylekey in style_dict:
                 style = style_dict[stylekey]
             else:
-                style = self._convert_to_style(cell.style, num_format_str)
+                style = self._convert_to_style(cell.style, fmt)
                 style_dict[stylekey] = style
 
             if cell.mergestart is not None and cell.mergeend is not None:
@@ -1741,23 +1738,17 @@ def write_cells(self, cells, sheet_name=None, startrow=0, startcol=0,
             wks.freeze_panes(*(freeze_panes))
 
         for cell in cells:
-            val = _conv_value(cell.val)
-
-            num_format_str = None
-            if isinstance(cell.val, datetime):
-                num_format_str = self.datetime_format
-            elif isinstance(cell.val, date):
-                num_format_str = self.date_format
+            val, fmt = self._value_with_fmt(cell.val)
 
             stylekey = json.dumps(cell.style)
-            if num_format_str:
-                stylekey += num_format_str
+            if fmt:
+                stylekey += fmt
 
             if stylekey in style_dict:
                 style = style_dict[stylekey]
             else:
                 style = self.book.add_format(
-                    _XlsxStyler.convert(cell.style, num_format_str))
+                    _XlsxStyler.convert(cell.style, fmt))
                 style_dict[stylekey] = style
 
             if cell.mergestart is not None and cell.mergeend is not None:


Test patch:
diff --git a/pandas/tests/io/test_excel.py b/pandas/tests/io/test_excel.py
--- a/pandas/tests/io/test_excel.py
+++ b/pandas/tests/io/test_excel.py
@@ -1373,11 +1373,6 @@ def test_to_excel_interval_labels(self, merge_cells, engine, ext):
 
     def test_to_excel_timedelta(self, merge_cells, engine, ext):
         # GH 19242, GH9155 - test writing timedelta to xls
-        if engine == 'openpyxl':
-            pytest.xfail('Timedelta roundtrip broken with openpyxl')
-        if engine == 'xlsxwriter' and (sys.version_info[0] == 2 and
-                                       sys.platform.startswith('linux')):
-            pytest.xfail('Not working on linux with Py2 and xlsxwriter')
         frame = DataFrame(np.random.randint(-10, 10, size=(20, 1)),
                           columns=['A'],
                           dtype=np.int64