Repository: pandas-dev/pandas

Issue:
ENH: Excel - allow for multiple rows to be treated as hierarchical columns
related #4468

Add keyword argument to ExcelFile parser to take an integer / list for the rows to interpret as header rows. If more than one, interpret as hierarchical columns / MultiIndex.

Presumably this would also allow you to round-trip  Data Frame with hierarchical columns.

Basically, if you have something spanning two columns, just converts to two cells with the data in the original cell, ending up just like what you need for csv reader.



Discussion:
http://pandas.pydata.org/pandas-docs/dev/io.html#reading-columns-with-a-multiindex

(but for csv), and ths _might/probably_ needs special handling for excel

only special handling would be converting merged cells into repeated entries like csv, so this is relatively minor.

I.e. 

```
____________________
| bar    |   baz    |
| A | B  | C | D | E|
|___________________|
```

just needs to change to something like

``` python
[['bar', 'bar', 'baz', 'baz', 'baz'], ['A', 'B', 'C', 'D', 'E']]
```

under the hood

so, really, a function that takes in merged cell and splits it into individual cells all with the same value would be sufficient to take advantage of csv's existing behavior.

@jtratner I think that is right, in your example `header=[0,1]` if it produces the output you put then should parse to a mi

related is the reverse (in `to_excel`); again exists in `to_csv`, but would need porting (then could round-trip)

@cancan101 interested in implementing this? just a minor modification of your `get_effective_cell` function.

I can take a look at this. I am equally interested in solving this for HTML files, for example: http://www.sec.gov/Archives/edgar/data/47217/000104746913006802/a2215416z10-q.htm#CCSCI

Yeah, that's basically the same thing, you just want to end up with the following arrays.

``` python
>>> data = [
    ['Three months ended April 30','Three months ended April 30',
     'Six months ended April 30', 'Six months ended April 30'],
    ['2013', '2012', '2013', '2012']
]
>>> MultiIndex.from_arrays(data)
MultiIndex
[(u'Three months ended April 30', u'2013'), (u'Three months ended April 30', u'2012'), (u'Six months ended April 30', u'2013'), (u'Six months ended April 30', u'2012')]
```

So if you have something like:

```
<td colspan=2>Span2</td>
```

You want to convert that into 2 cells with text 'Span2'

Exactly. I am going to create a similar issues to this one for HTML. FWIW, It would be great to merge the IO backends  so that functionality like this can be shared. See: #4682. Shoot closing the other issue.

@cancan101 well, I believe they mostly are, they just pass to a TextReader which does the majority of the work. (so, for example, the ExcelFile reader has to do some magic to convert all the values to a list of lists that can be passed to text reader).  I think you could do both of these in the same issue and then refactor the multiindex creation methods from read_csv out for something they can all use - check out code around here for how it works under the hood (I think):

https://github.com/pydata/pandas/blob/master/pandas/io/parsers.py#L703

I believe that I looked at the parsers backends and that some, but not all, of the parsers use `TextReader`. I believe that HTML parser does not use TextReader 

Now comes the other reason for improving the ExcelParser and/or the HTML parser: parsing hierarchical row indexes. A good example of this would be (different link from above): http://www.sec.gov/Archives/edgar/data/47217/000104746913006802/a2215416z10-q.htm#CCSE. 

This table has 4 major sections (they can be identified as lines with no other data):
1. Net revenue:
2. Costs and expenses:
3. Net earnings per share:
4. Weighted-average shares used to compute net earnings per share:

Within the first of those sections are a number of lines items and a section total.

A good feature of the parser would be to extract this structure from the table. Obviously this is non-trivial.

@cancan101 hierarchical row indexes are almost the same thing as hierarchical columns - the goal is to turn the input into something similar to what [csv takes for hierarchical indexes](http://pandas.pydata.org/pandas-docs/dev/io.html#reading-columns-with-a-multiindex). You just need to convert rowspan/horizontally-merged cells into repeated horizontal cells.

@jtratner That is not actually the issue here. In this case there is no explicit indication in the table itself that the rows should be grouped in a certain manner. For example there is no rowspan / merged cell. In this case the user would need to supply some additional information: for example to treat the heading in an empty row as a new level in the hierarchy,

@cancan101 that's probably outside the scope of what the pandas parser could do [if it's that complicated, user probably should handle the cleaning after that] - I'd suggest going for a very simple implementation to start out with (i.e., take colspan or merge length --> convert it to individual cells --> use read_csv's existing functionality to get a multiindex going) then after you have that working, you can consider what else makes sense to add.

@jtratner The above is unrelated to the hierarchical columns. That would be another issues. The simplest option might be split the table on those empty lines. For example have some options for the above link to split out 3 tables rather than one. Alternatively the user could attempt to detect this himself this given that that row will have a lot of nans for values.

@cancan101 can you try passing a list to header and see if it converts to mi? it may work already.

@jratner I will take a look. A pre-requisite of dealing with merged cells in Excel is my PR #4439. It likely needs cleaning up.

@cancan101 `read_html` _does_ parse `header=[a list of integers]` into a `MultiIndex`:

```
In [16]: url = 'http://www.fdic.gov/bank/individual/failed/banklist.html'

In [17]: df = read_html(url,header=[0,1])[0]

In [18]: type(df.columns)
Out[18]: pandas.core.index.MultiIndex

In [21]: df
Out[21]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 510 entries, 2 to 511
Data columns (total 7 columns):
(Sunrise Bank of Arizona, Community South Bank)                 510  non-null values
(Phoenix, Parsons)                                              510  non-null values
(AZ, TN)                                                        510  non-null values
(34707, 19849)                                                  510  non-null values
(First Fidelity Bank, National Association, CB&S Bank, Inc.)    510  non-null values
(2013-08-23 00:00:00, 2013-08-23 00:00:00)                      510  non-null values
(2013-08-27 00:00:00, 2013-08-27 00:00:00)                      510  non-null values
dtypes: datetime64[ns](2), int64(1), object(4)
```

If there's a more specific issue that you've come across, please create a new one just for that. I (and I think the other devs) would prefer to keep the IO backends' issues separate from each other.

Here are the issues:
1. Refactor `read_html` to use `TextParser`
2. Excel merging cells etc.
3. Support the use of `colspan` to create a `MultiIndex` for `read_html`.

(1 should be done before 3)

I'll create 1, 3 is #4683, and I think you should create 2 since you've got the clearest idea of what the issue is :smile:.

@cpcloud That page is an interesting example. Why is this not the first row of the header:

```
Bank Name   City    ST  CERT    Acquiring Institution   Closing Date    Updated Date
```

@cancan101 That _is_ the first row of the header if you don't pass in the `header` argument. The header is not counted as part of the data, currently. This is a bug, or at least an inconsistency. This will be fixed in the refactor of `read_html` to use `TextParser`:

```
In [9]: url = 'http://www.fdic.gov/bank/individual/failed/banklist.html'

In [10]: df = read_html(url, header=[0, 1])[0]

In [11]: df
Out[11]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 510 entries, 0 to 509
Data columns (total 7 columns):
(Bank Name, Sunrise Bank of Arizona)                                  510  non-null values
(City, Phoenix)                                                       510  non-null values
(ST, AZ)                                                              510  non-null values
(CERT, 34707)                                                         510  non-null values
(Acquiring Institution, First Fidelity Bank, National Association)    510  non-null values
(Closing Date, August 23, 2013)                                       510  non-null values
(Updated Date, August 27, 2013)                                       510  non-null values
dtypes: datetime64[ns](2), int64(1), object(4)

In [12]: df = read_html(url)[0]

In [13]: df
Out[13]:
<class 'pandas.core.frame.DataFrame'>
Int64Index: 512 entries, 0 to 511
Data columns (total 7 columns):
Bank Name                512  non-null values
City                     512  non-null values
ST                       512  non-null values
CERT                     512  non-null values
Acquiring Institution    512  non-null values
Closing Date             512  non-null values
Updated Date             512  non-null values
dtypes: datetime64[ns](2), int64(1), object(4)
```

@cpcloud I am not sure if this feature make sense: auto detecting the number of header rows based upon the table's use of `hr` or `thead`. For example passing: `header="auto"`. While not all tables use these tags, it would be helpful to use them when they exist.

@cancan101 Right. There's a couple of things you should know, just so that you understand why/how the header parsing is happening. There's no dependence on `thead` elements. The parse is structured so that a `tuple` of `list`s of elements is returned from all the HTML parsers, i.e.,

``` python
header, body, footer = data
```

Since I have control over where I place the `header` `list`, it can be the first row of the `DataFrame` instance, which conveniently is the default `header` keyword argument to `TextParser` and I believe all of the other IO functions with a `header` parameter.

You're right that not all tables use the `thead` element, however I've found that many do; much more so than, say, the `caption` element, and especially in cases where tables are used as data tables (as opposed to being used to style the page).

So, it's not that the _number_ of header rows is detected. One thing that there _isn't_ a test case for is multiple header rows, which is partially what some of these issues are about.

@cpcloud I also believe that some tables use `th` without using `thead`. I am not sure if the converse is true: using `td`s withing a `thead`.

@cancan101 Yes, that case is covered. `th` and `thead` are both parsed as headers cells. `td` within a `thead`/`th` should be okay. If it isn't please open an issue and I'll take a look.

@cpcloud Should this ticket serve as item 2 above? Or should I open a new ticket?

Neither. The title of this issue indicates that you think Excel file reading should be able to handle a list passed to `header` and create a `MultiIndex` based on that `list`. That's different than the merging cells issue.

@cpcloud I should probably provide an [example Excel file](http://services.corporate-ir.net/SEC/Document.Service?id=P3VybD1hSFIwY0RvdkwyRndhUzUwWlc1cmQybDZZWEprTG1OdmJTOWtiM2R1Ykc5aFpDNXdhSEEvWVdOMGFXOXVQVmhNVXlacGNHRm5aVDA0T1RVeE16STBKbk4xWW5OcFpEMDFOdz09JnR5cGU9MSZmbj1TYWxlc2ZvcmNlLnhscw==). See the sheet labeled "Condensed Consolidated Statemen"
At this point I am trying to figure out what feature are missing from the current `ExcelParser` to allow parsing of that table.

@cancan101 Can you save that in an Excel format that is compatible with Google Drive? I don't have Excel.

@cpcloud What is the best way to share the xls file?

Can you save it as `the_file.xlsx` and then post a public dropbox link or google drive link?

https://docs.google.com/spreadsheet/ccc?key=0AsjbCMQSMZS8dEhBRFZiODgxR1RvSC1LZ0FfcndCM0E

@cancan101 For tables like this that are so small and contain so little data, IMHO it would be overkill to use `read_excel` here. Simply reading in the raw text, searching and constructing the `DataFrame` instance from whatever you found would suffice. Do you have another example where that method would be too slow/clunky and `read_excel` would be needed?

What do you mean by "reading in the raw text, searching and constructing the DataFrame instance from whatever you found"?

To double check, you are looking at the table with the title "Condensed Consolidated Statements of Operations"?

Oh! Silly me. I didn't see that there are a ton of sheets.

So does the table look worthy of read_excel?
On Aug 28, 2013 2:32 PM, "Phillip Cloud" notifications@github.com wrote:

> Oh! Silly me. I didn't see that there are a ton of sheets.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/4679#issuecomment-23436902
> .

@cancan101 It would be useful to be able to merge cells and to create multiindex columns by passing `header=[...]` (two separate issues)

The reason I could see them being the same issue is inferring what cells to
merge from the header row.
On Aug 28, 2013 4:22 PM, "Phillip Cloud" notifications@github.com wrote:

> @cancan101 https://github.com/cancan101 It would be useful to be able
> to merge cells and to create multiindex columns by passing header=[...](two separate issues)
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/4679#issuecomment-23444385
> .

@cancan101 I think cpcloud's comment was before he realized there were more sheets :) .  To expand on @cpcloud 's suggestion, I think two really great things you could add to read_excel would be (in two separate PRs):
1. A `split_merged` keyword argument to read_excel that takes merged cells and splits them into individual cells each with the same value as the original merged cell. [default for `split_merged` should be `None`] - This would be just a variation on what you had for `get_effective_cell`
2. Support for passing a list for header/index_col, so you can create hierarchical columns and indexes. If you pass a list for header or index col and `split_merged` is `None`, then `split_merged` is forced to `True`.

In terms of more complicated examples like the SEC docs you cited, it might make more sense for you to create a separate wrapper/plugin/add-on to pandas that handles specific cases like your example. I'd be hesitant to add too much magic to `read_excel`.

In terms of `split_merged`, I was thinking you could modify the worksheet _before_ you pass it into `_excel_parser`, so you don't have to modify `_excel_parser` at all.  Plus, with the assumption that merged cells would mostly occur in column headers or in indexes, generally the number of merged cells would be much less than the number of total cells in the workbook, so it makes sense to special case it, especially because it looks to me like you can iterate over `merged_cells` relatively easily.

@cancan101 right but merging cells is one thing

ability to use multiindex columns is another (which should be done first, so you can then incorporate that into merging)

they could be closed by the same PR since they _are_ intertwined, but when coding it i think it wll be helpful if you just think about one at a time so that you don't get overwhelmed and write a giant `if elif` suite to deal with the complexity

@cpcloud good suggestion. for multiindex columns, I'd think an excel file that looked something like this would be a good test case:

```
a     b c c
q r s t u v
1 2 3 4 5 6
```

Which should end up with a set of columns that looks like this:

```
   a        b  c
   q  r  s  t  u  v
0  1  2  3  4  5  6
```

You can represent the columns you expect with the following `MultiIndex`

``` python
labels = [[0, 0, 0, 1, 2, 2], [0, 1, 2, 3, 4, 5]]
levels = [['a', 'b', 'c'], ['q', 'r', 's', 't', 'u', 'v']]
mi = MultiIndex(labels=labels, levels=levels)
```

If you want to generate that directly with `read_csv`, you'd do the following:

``` python
In [1]: import pandas
In [2]: text = """
a        b  c  c
q r s t u v
1 2 3 4 5 6
"""
In [4]: from StringIO import StringIO
In [7]: pandas.read_csv(StringIO(text.strip()), sep='\s+', header=[0, 1], tupleize_cols=False)
```

So following the same convention for `read_excel` would probably be good.

@cancan101 my example was slightly wrong. Columns need to be empty, not repeated.

@jtratner What do you mean by "modify the worksheet before you pass it into _excel_parser"?

As for making a plugin for the SEC docs, I don't necessarily expect `read_excel` to have enough parameters to parse the worksheet off the bat. I am fine having the `read_excel` return a raw table with enough information that I can perform operations on the resulting DataFrame to get the data useable. My only concern would be that `read_excel` returns data that is rich enough (i.e. doesn't drop information contained in the Excel file) for me to finish cleaning the data.

I believe I have been added to this thread in error.

On Wed, Aug 28, 2013 at 5:53 PM, Alex Rothberg notifications@github.comwrote:

> @jtratner https://github.com/jtratner What do you mean by "modify the
> worksheet before you pass it into _excel_parser"?
> As for making a plugin for the SEC docs, I don't necessarily expect
> read_excel to have enough parameters to parse the worksheet off the bat.
> I am fine having the read_excel return a raw table with enough
> information that I can perform operations on the resulting DataFrame to get
> the data useable. My only concern would be that read_excel returns data
> that is rich enough (i.e. doesn't drop information contained in the Excel
> file) for me to finish cleaning the data.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/4679#issuecomment-23455110
> .

As I am looking at the ExcelParser, does anyone know about this: #4563 ?

Another issue I have to deal with is the split between constructing an `ExcelFile` and calling `parse` on the object. In the case of `split_merged=True`, I presumably have to pass `formatting_info=True` to the `xlrd.open_workbook` call. Unfortunately the `open_workbook` is made from the constructor whereas I assume `split_merged=True` will be passed to the `parse` method.
 Thoughts? This is not as much of an issue for calls to `read_excel`.

@cancan101 btw - I'm totally wrong - there's no need for a split merged function to handle hierarchical columns - sorry about that!!

so I'd just not bother doing anything with merged cells (since they show up as leftmost column with value and rest blank), so it should be very simple to allow for multiindex columns with merged cells (don't even need to think about it)

@jtratner What do you mean? I am a little lost here.

@cancan101 if you read an excel file withmerged columns, it's read in with the leftmost/topmost column containing the value, which is what you want for MultiIndex anyways. If you copy over the header logic from read_csv it should probably just work after being read in. (obviously after that you'd probably want to separate it out for both).

Again, I apologize for the confusion...I just wasn't thinking clearly there.

@jtratner I will have to take a look at the code. I'm not 100% clear on "it's read in with the leftmost/topmost column containing the value" and if this is exactly how the merged cell should be read.

@cancan101 try this excel file - just load it in with read_excel and you can see exactly what's going on with it:

https://docs.google.com/file/d/0B7SeTh4dLc3mUnpnLWRTandCa0E/edit?usp=sharing

@jtratner, am I missing something or does the CSV reader require me to duplicate the column (i.e. it does not just read to the left)?

CSV:

```
a,,b,,c,
a1,a2,b1,b2,c1,c2
1,1,1,1,1,1
2,2,2,2,2,2
```

and then:

```
In [66]: pd.read_csv("test2.csv",header=[0,1],tupleize_cols=False)
Out[66]: 
    a  Unnamed: 1_level_0   b  Unnamed: 3_level_0   c  Unnamed: 5_level_0
   a1                  a2  b1                  b2  c1                  c2
0   2                   2   2                   2   2                   2
```

This came up here: http://stackoverflow.com/q/23703638/1240268 (with a strange html table example)


Resolving patch:
diff --git a/pandas/core/format.py b/pandas/core/format.py
--- a/pandas/core/format.py
+++ b/pandas/core/format.py
@@ -4,7 +4,6 @@
 # pylint: disable=W0141
 
 import sys
-import warnings
 
 from pandas.core.base import PandasObject
 from pandas.core.common import adjoin, notnull
@@ -1641,14 +1640,11 @@ class ExcelFormatter(object):
     inf_rep : string, default `'inf'`
         representation for np.inf values (which aren't representable in Excel)
         A `'-'` sign will be added in front of -inf.
-    verbose: boolean, default True
-        If True, warn user that the resulting output file may not be
-        re-read or parsed directly by pandas.
     """
 
     def __init__(self, df, na_rep='', float_format=None, cols=None,
                  header=True, index=True, index_label=None, merge_cells=False,
-                 inf_rep='inf', verbose=True):
+                 inf_rep='inf'):
         self.df = df
         self.rowcounter = 0
         self.na_rep = na_rep
@@ -1661,7 +1657,6 @@ def __init__(self, df, na_rep='', float_format=None, cols=None,
         self.header = header
         self.merge_cells = merge_cells
         self.inf_rep = inf_rep
-        self.verbose = verbose
 
     def _format_value(self, val):
         if lib.checknull(val):
@@ -1682,10 +1677,6 @@ def _format_header_mi(self):
                 raise NotImplementedError("Writing to Excel with MultiIndex"
                                           " columns and no index ('index'=False) "
                                           "is not yet implemented.")
-            elif self.index and self.verbose:
-                warnings.warn("Writing to Excel with MultiIndex columns is a"
-                              " one way serializable operation. You will not"
-                              " be able to re-read or parse the output file.")
 
         has_aliases = isinstance(self.header, (tuple, list, np.ndarray, Index))
         if not(has_aliases or self.header):
@@ -1796,18 +1787,14 @@ def _format_regular_rows(self):
             else:
                 index_label = self.df.index.names[0]
 
+            if isinstance(self.columns, MultiIndex):
+                self.rowcounter += 1
+
             if index_label and self.header is not False:
-                if self.merge_cells:
-                    yield ExcelCell(self.rowcounter,
-                                    0,
-                                    index_label,
-                                    header_style)
-                    self.rowcounter += 1
-                else:
-                    yield ExcelCell(self.rowcounter - 1,
-                                    0,
-                                    index_label,
-                                    header_style)
+                yield ExcelCell(self.rowcounter - 1,
+                                0,
+                                index_label,
+                                header_style)
 
             # write index_values
             index_values = self.df.index
@@ -1841,19 +1828,21 @@ def _format_hierarchical_rows(self):
                                                (list, tuple, np.ndarray, Index)):
                 index_labels = self.index_label
 
+            # MultiIndex columns require an extra row
+            # with index names (blank if None) for
+            # unambigous round-trip
+            if isinstance(self.columns, MultiIndex):
+                self.rowcounter += 1
+
             # if index labels are not empty go ahead and dump
             if (any(x is not None for x in index_labels)
                     and self.header is not False):
 
-                if not self.merge_cells:
-                    self.rowcounter -= 1
-
                 for cidx, name in enumerate(index_labels):
-                    yield ExcelCell(self.rowcounter,
+                    yield ExcelCell(self.rowcounter - 1,
                                     cidx,
                                     name,
                                     header_style)
-                self.rowcounter += 1
 
             if self.merge_cells:
                 # Format hierarchical rows as merged cells.
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -1336,9 +1336,6 @@ def to_excel(self, excel_writer, sheet_name='Sheet1', na_rep='',
         inf_rep : string, default 'inf'
             Representation for infinity (there is no native representation for
             infinity in Excel)
-        verbose: boolean, default True
-             If True, warn user that the resulting output file may not be
-             re-read or parsed directly by pandas.
 
         Notes
         -----
@@ -1371,7 +1368,7 @@ def to_excel(self, excel_writer, sheet_name='Sheet1', na_rep='',
                                        index=index,
                                        index_label=index_label,
                                        merge_cells=merge_cells,
-                                       inf_rep=inf_rep, verbose=verbose)
+                                       inf_rep=inf_rep)
         formatted_cells = formatter.get_formatted_cells()
         excel_writer.write_cells(formatted_cells, sheet_name,
                                  startrow=startrow, startcol=startcol)
diff --git a/pandas/io/excel.py b/pandas/io/excel.py
--- a/pandas/io/excel.py
+++ b/pandas/io/excel.py
@@ -18,6 +18,7 @@
                            BytesIO, string_types)
 from pandas.core import config
 from pandas.core.common import pprint_thing
+from pandas.util.decorators import Appender
 import pandas.compat as compat
 import pandas.compat.openpyxl_compat as openpyxl_compat
 import pandas.core.common as com
@@ -68,15 +69,11 @@ def get_writer(engine_name):
         raise ValueError("No Excel writer '%s'" % engine_name)
 
 
-def read_excel(io, sheetname=0, **kwds):
-    """Read an Excel table into a pandas DataFrame
+excel_doc_common = """
+    Read an Excel table into a pandas DataFrame
 
     Parameters
-    ----------
-    io : string, file-like object, or xlrd workbook.
-        The string could be a URL. Valid URL schemes include http, ftp, s3,
-        and file. For file URLs, a host is expected. For instance, a local
-        file could be file://localhost/path/to/workbook.xlsx
+    ----------%(io)s
     sheetname : string, int, mixed list of strings/ints, or None, default 0
 
         Strings are used for sheet names, Integers are used in zero-indexed sheet
@@ -97,20 +94,23 @@ def read_excel(io, sheetname=0, **kwds):
         * [0,1,"Sheet5"] -> 1st, 2nd & 5th sheet as a dictionary of DataFrames
         * None -> All sheets as a dictionary of DataFrames
 
-    header : int, default 0
-        Row to use for the column labels of the parsed DataFrame
+    header : int, list of ints, default 0
+        Row (0-indexed) to use for the column labels of the parsed
+        DataFrame. If a list of integers is passed those row positions will
+        be combined into a ``MultiIndex``
     skiprows : list-like
         Rows to skip at the beginning (0-indexed)
     skip_footer : int, default 0
         Rows at the end to skip (0-indexed)
+    index_col : int, list of ints, default None
+        Column (0-indexed) to use as the row labels of the DataFrame.
+        Pass None if there is no such column.  If a list is passed,
+        those columns will be combined into a ``MultiIndex``
     converters : dict, default None
         Dict of functions for converting values in certain columns. Keys can
         either be integers or column labels, values are functions that take one
         input argument, the Excel cell content, and return the transformed
         content.
-    index_col : int, default None
-        Column to use as the row labels of the DataFrame. Pass None if
-        there is no such column
     parse_cols : int or list, default None
         * If None then parse all columns,
         * If int then indicates last column to be parsed
@@ -119,22 +119,21 @@ def read_excel(io, sheetname=0, **kwds):
           column ranges (e.g. "A:E" or "A,C,E:F")
     na_values : list-like, default None
         List of additional strings to recognize as NA/NaN
+    thousands : str, default None
+        Thousands separator
     keep_default_na : bool, default True
         If na_values are specified and keep_default_na is False the default NaN
         values are overridden, otherwise they're appended to
     verbose : boolean, default False
-        Indicate number of NA values placed in non-numeric columns
-    engine: string, default None
-        If io is not a buffer or path, this must be set to identify io.
-        Acceptable values are None or xlrd
+        Indicate number of NA values placed in non-numeric columns%(eng)s
     convert_float : boolean, default True
         convert integral floats to int (i.e., 1.0 --> 1). If False, all numeric
         data will be read in as floats: Excel stores all numbers as floats
         internally
-    has_index_names : boolean, default False
-        True if the cols defined in index_col have an index name and are
-        not in the header. Index name will be placed on a separate line below
-        the header.
+    has_index_names : boolean, default None
+        DEPCRECATED: for version 0.17+ index names will be automatically inferred
+        based on index_col.  To read Excel output from 0.16.2 and prior that
+        had saved index names, use True.
 
     Returns
     -------
@@ -143,6 +142,19 @@ def read_excel(io, sheetname=0, **kwds):
         for more information on when a Dict of Dataframes is returned.
 
     """
+read_excel_kwargs = dict()
+read_excel_kwargs['io'] = """
+    io : string, file-like object, or xlrd workbook.
+        The string could be a URL. Valid URL schemes include http, ftp, s3,
+        and file. For file URLs, a host is expected. For instance, a local
+        file could be file://localhost/path/to/workbook.xlsx"""
+read_excel_kwargs['eng'] = """
+    engine: string, default None
+        If io is not a buffer or path, this must be set to identify io.
+        Acceptable values are None or xlrd"""
+
+@Appender(excel_doc_common % read_excel_kwargs)
+def read_excel(io, sheetname=0, **kwds):
     engine = kwds.pop('engine', None)
 
     return ExcelFile(io, engine=engine).parse(sheetname=sheetname, **kwds)
@@ -193,83 +205,23 @@ def __init__(self, io, **kwds):
             raise ValueError('Must explicitly set engine if not passing in'
                              ' buffer or path for io.')
 
+    @Appender(excel_doc_common % dict(io='', eng=''))
     def parse(self, sheetname=0, header=0, skiprows=None, skip_footer=0,
               index_col=None, parse_cols=None, parse_dates=False,
               date_parser=None, na_values=None, thousands=None, chunksize=None,
-              convert_float=True, has_index_names=False, converters=None, **kwds):
-        """Read an Excel table into DataFrame
+              convert_float=True, has_index_names=None, converters=None, **kwds):
 
-        Parameters
-        ----------
-        sheetname : string, int, mixed list of strings/ints, or None, default 0
-
-            Strings are used for sheet names, Integers are used in zero-indexed sheet
-            positions.
-
-            Lists of strings/integers are used to request multiple sheets.
-
-            Specify None to get all sheets.
-
-            str|int -> DataFrame is returned.
-            list|None -> Dict of DataFrames is returned, with keys representing sheets.
-
-            Available Cases
-
-            * Defaults to 0 -> 1st sheet as a DataFrame
-            * 1 -> 2nd sheet as a DataFrame
-            * "Sheet1" -> 1st sheet as a DataFrame
-            * [0,1,"Sheet5"] -> 1st, 2nd & 5th sheet as a dictionary of DataFrames
-            * None -> All sheets as a dictionary of DataFrames
-        header : int, default 0
-            Row to use for the column labels of the parsed DataFrame
-        skiprows : list-like
-            Rows to skip at the beginning (0-indexed)
-        skip_footer : int, default 0
-            Rows at the end to skip (0-indexed)
-        converters : dict, default None
-            Dict of functions for converting values in certain columns. Keys can
-            either be integers or column labels
-        index_col : int, default None
-            Column to use as the row labels of the DataFrame. Pass None if
-            there is no such column
-        parse_cols : int or list, default None
-            * If None then parse all columns
-            * If int then indicates last column to be parsed
-            * If list of ints then indicates list of column numbers to be
-              parsed
-            * If string then indicates comma separated list of column names and
-              column ranges (e.g. "A:E" or "A,C,E:F")
-        parse_dates : boolean, default False
-            Parse date Excel values,
-        date_parser : function default None
-            Date parsing function
-        na_values : list-like, default None
-            List of additional strings to recognize as NA/NaN
-        thousands : str, default None
-            Thousands separator
-        chunksize : int, default None
-            Size of file chunk to read for lazy evaluation.
-        convert_float : boolean, default True
-            convert integral floats to int (i.e., 1.0 --> 1). If False, all
-            numeric data will be read in as floats: Excel stores all numbers as
-            floats internally.
-        has_index_names : boolean, default False
-            True if the cols defined in index_col have an index name and are
-            not in the header
-        verbose : boolean, default False
-            Set to True to print a single statement when reading each
-            excel sheet.
-
-        Returns
-        -------
-        parsed : DataFrame or Dict of DataFrames
-            DataFrame from the passed in Excel file.  See notes in sheetname argument
-            for more information on when a Dict of Dataframes is returned.
-        """
         skipfooter = kwds.pop('skipfooter', None)
         if skipfooter is not None:
             skip_footer = skipfooter
 
+        if has_index_names is not None:
+            warn("\nThe has_index_names argument is deprecated; index names "
+                 "will be automatically inferred based on index_col.\n"
+                 "This argmument is still necessary if reading Excel output "
+                 "from 0.16.2 or prior with index names.", FutureWarning,
+                 stacklevel=3)
+
         return self._parse_excel(sheetname=sheetname, header=header,
                                  skiprows=skiprows,
                                  index_col=index_col,
@@ -418,8 +370,40 @@ def _parse_cell(cell_contents,cell_typ):
             if sheet.nrows == 0:
                 return DataFrame()
 
+            if com.is_list_like(header) and len(header) == 1:
+                header = header[0]
+
+            # forward fill and pull out names for MultiIndex column
+            header_names = None
             if header is not None:
-                data[header] = _trim_excel_header(data[header])
+                if com.is_list_like(header):
+                    header_names = []
+                    for row in header:
+                        if com.is_integer(skiprows):
+                            row += skiprows
+                        data[row] = _fill_mi_header(data[row])
+                        header_name, data[row] = _pop_header_name(data[row], index_col)
+                        header_names.append(header_name)
+                else:
+                    data[header] = _trim_excel_header(data[header])
+
+            if com.is_list_like(index_col):
+                # forward fill values for MultiIndex index
+                if not com.is_list_like(header):
+                    offset = 1 + header
+                else:
+                    offset = 1 + max(header)
+
+                for col in index_col:
+                    last = data[offset][col]
+                    for row in range(offset + 1, len(data)):
+                        if data[row][col] == '' or data[row][col] is None:
+                            data[row][col] = last
+                        else:
+                            last = data[row][col]
+
+            if com.is_list_like(header) and len(header) > 1:
+                has_index_names = True
 
             parser = TextParser(data, header=header, index_col=index_col,
                                 has_index_names=has_index_names,
@@ -433,6 +417,7 @@ def _parse_cell(cell_contents,cell_typ):
                                 **kwds)
 
             output[asheetname] = parser.read()
+            output[asheetname].columns = output[asheetname].columns.set_names(header_names)
 
         if ret_dict:
             return output
@@ -463,6 +448,29 @@ def _trim_excel_header(row):
         row = row[1:]
     return row
 
+def _fill_mi_header(row):
+    # forward fill blanks entries
+    # from headers if parsing as MultiIndex
+    last = row[0]
+    for i in range(1, len(row)):
+        if row[i] == '' or row[i] is None:
+            row[i] = last
+        else:
+            last = row[i]
+    return row
+
+# fill blank if index_col not None
+def _pop_header_name(row, index_col):
+    """ (header, new_data) for header rows in MultiIndex parsing"""
+    none_fill = lambda x: None if x == '' else x
+
+    if index_col is None:
+        # no index col specified, trim data for inference path
+        return none_fill(row[0]), row[1:]
+    else:
+        # pop out header name and fill w/ blank
+        i = index_col if not com.is_list_like(index_col) else max(index_col)
+        return none_fill(row[i]), row[:i] + [''] + row[i+1:]
 
 def _conv_value(val):
     # Convert numpy types to Python types for the Excel writers.


Test patch:
diff --git a/pandas/io/tests/data/test_index_name_pre17.xls b/pandas/io/tests/data/test_index_name_pre17.xls
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/test_index_name_pre17.xls differ
diff --git a/pandas/io/tests/data/test_index_name_pre17.xlsm b/pandas/io/tests/data/test_index_name_pre17.xlsm
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/test_index_name_pre17.xlsm differ
diff --git a/pandas/io/tests/data/test_index_name_pre17.xlsx b/pandas/io/tests/data/test_index_name_pre17.xlsx
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/test_index_name_pre17.xlsx differ
diff --git a/pandas/io/tests/data/testmultiindex.xls b/pandas/io/tests/data/testmultiindex.xls
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/testmultiindex.xls differ
diff --git a/pandas/io/tests/data/testmultiindex.xlsm b/pandas/io/tests/data/testmultiindex.xlsm
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/testmultiindex.xlsm differ
diff --git a/pandas/io/tests/data/testmultiindex.xlsx b/pandas/io/tests/data/testmultiindex.xlsx
new file mode 100644
Binary files /dev/null and b/pandas/io/tests/data/testmultiindex.xlsx differ
diff --git a/pandas/io/tests/test_excel.py b/pandas/io/tests/test_excel.py
--- a/pandas/io/tests/test_excel.py
+++ b/pandas/io/tests/test_excel.py
@@ -14,6 +14,7 @@
 import numpy as np
 from numpy.testing.decorators import slow
 
+import pandas as pd
 from pandas import DataFrame, Index, MultiIndex
 from pandas.io.parsers import read_csv
 from pandas.io.excel import (
@@ -21,7 +22,7 @@
     _Openpyxl2Writer, register_writer, _XlsxWriter
 )
 from pandas.io.common import URLError
-from pandas.util.testing import ensure_clean
+from pandas.util.testing import ensure_clean, makeCustomDataframe as mkdf
 from pandas.core.config import set_option, get_option
 import pandas.util.testing as tm
 
@@ -415,11 +416,8 @@ def test_read_xlrd_Book(self):
 
     @tm.network
     def test_read_from_http_url(self):
-        # TODO: remove this when merging into master
-        url = ('https://raw.github.com/davidovitch/pandas/master/'
+        url = ('https://raw.github.com/pydata/pandas/master/'
                'pandas/io/tests/data/test1' + self.ext)
-#        url = ('https://raw.github.com/pydata/pandas/master/'
-#               'pandas/io/tests/data/test' + self.ext)
         url_table = read_excel(url)
         local_table = self.get_exceldf('test1')
         tm.assert_frame_equal(url_table, local_table)
@@ -518,6 +516,132 @@ def test_reader_seconds(self):
         actual = self.get_exceldf('times_1904', 'Sheet1')
         tm.assert_frame_equal(actual, expected)
 
+    def test_read_excel_multiindex(self):
+        #GH 4679
+        mi = MultiIndex.from_product([['foo','bar'],['a','b']])
+        mi_file = os.path.join(self.dirpath, 'testmultiindex' + self.ext)
+
+        expected = DataFrame([[1, 2.5, pd.Timestamp('2015-01-01'), True],
+                            [2, 3.5, pd.Timestamp('2015-01-02'), False],
+                            [3, 4.5, pd.Timestamp('2015-01-03'), False],
+                            [4, 5.5, pd.Timestamp('2015-01-04'), True]],
+                            columns = mi)
+
+        actual = read_excel(mi_file, 'mi_column', header=[0,1])
+        tm.assert_frame_equal(actual, expected)
+        actual = read_excel(mi_file, 'mi_column', header=[0,1], index_col=0)
+        tm.assert_frame_equal(actual, expected)
+
+        expected.columns = ['a', 'b', 'c', 'd']
+        expected.index = mi
+        actual = read_excel(mi_file, 'mi_index', index_col=[0,1])
+        tm.assert_frame_equal(actual, expected, check_names=False)
+
+        expected.columns = mi
+        actual = read_excel(mi_file, 'both', index_col=[0,1], header=[0,1])
+        tm.assert_frame_equal(actual, expected, check_names=False)
+
+        expected.index = mi.set_names(['ilvl1', 'ilvl2'])
+        expected.columns = ['a', 'b', 'c', 'd']
+        actual = read_excel(mi_file, 'mi_index_name', index_col=[0,1])
+        tm.assert_frame_equal(actual, expected)
+
+        expected.index = list(range(4))
+        expected.columns = mi.set_names(['c1', 'c2'])
+        actual = read_excel(mi_file, 'mi_column_name', header=[0,1], index_col=0)
+        tm.assert_frame_equal(actual, expected)
+
+        expected.index = mi.set_names(['ilvl1', 'ilvl2'])
+        actual = read_excel(mi_file, 'both_name', index_col=[0,1], header=[0,1])
+        tm.assert_frame_equal(actual, expected)
+
+        actual = read_excel(mi_file, 'both_name', index_col=[0,1], header=[0,1])
+        tm.assert_frame_equal(actual, expected)
+
+        actual = read_excel(mi_file, 'both_name_skiprows', index_col=[0,1],
+                            header=[0,1],  skiprows=2)
+        tm.assert_frame_equal(actual, expected)
+
+
+    def test_excel_multindex_roundtrip(self):
+        #GH 4679
+        _skip_if_no_xlsxwriter()
+        with ensure_clean('.xlsx') as pth:
+            for c_idx_names in [True, False]:
+                for r_idx_names in [True, False]:
+                    for c_idx_levels in [1, 3]:
+                        for r_idx_levels in [1, 3]:
+                            # column index name can't be serialized unless MultiIndex
+                            if (c_idx_levels == 1 and c_idx_names):
+                                continue
+
+                            # empty name case current read in as unamed levels, not Nones
+                            check_names = True
+                            if not r_idx_names and r_idx_levels > 1:
+                                check_names = False
+
+                            df = mkdf(5, 5, c_idx_names,
+                                        r_idx_names, c_idx_levels,
+                                        r_idx_levels)
+                            df.to_excel(pth)
+                            act = pd.read_excel(pth, index_col=list(range(r_idx_levels)),
+                                                header=list(range(c_idx_levels)))
+                            tm.assert_frame_equal(df, act, check_names=check_names)
+
+                            df.iloc[0, :] = np.nan
+                            df.to_excel(pth)
+                            act = pd.read_excel(pth, index_col=list(range(r_idx_levels)),
+                                                header=list(range(c_idx_levels)))
+                            tm.assert_frame_equal(df, act, check_names=check_names)
+
+                            df.iloc[-1, :] = np.nan
+                            df.to_excel(pth)
+                            act = pd.read_excel(pth, index_col=list(range(r_idx_levels)),
+                                                header=list(range(c_idx_levels)))
+                            tm.assert_frame_equal(df, act, check_names=check_names)
+
+    def test_excel_oldindex_format(self):
+        #GH 4679
+        data = np.array([['R0C0', 'R0C1', 'R0C2', 'R0C3', 'R0C4'],
+                         ['R1C0', 'R1C1', 'R1C2', 'R1C3', 'R1C4'],
+                         ['R2C0', 'R2C1', 'R2C2', 'R2C3', 'R2C4'],
+                         ['R3C0', 'R3C1', 'R3C2', 'R3C3', 'R3C4'],
+                         ['R4C0', 'R4C1', 'R4C2', 'R4C3', 'R4C4']])
+        columns = ['C_l0_g0', 'C_l0_g1', 'C_l0_g2', 'C_l0_g3', 'C_l0_g4']
+        mi = MultiIndex(levels=[['R_l0_g0', 'R_l0_g1', 'R_l0_g2', 'R_l0_g3', 'R_l0_g4'],
+                                ['R_l1_g0', 'R_l1_g1', 'R_l1_g2', 'R_l1_g3', 'R_l1_g4']],
+                        labels=[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4]],
+                        names=['R0', 'R1'])
+        si = Index(['R_l0_g0', 'R_l0_g1', 'R_l0_g2', 'R_l0_g3', 'R_l0_g4'],  name='R0')
+
+        in_file = os.path.join(self.dirpath, 'test_index_name_pre17' + self.ext)
+
+        expected = pd.DataFrame(data, index=si, columns=columns)
+        with tm.assert_produces_warning(FutureWarning):
+            actual = pd.read_excel(in_file, 'single_names', has_index_names=True)
+        tm.assert_frame_equal(actual, expected)
+
+        expected.index.name = None
+        actual = pd.read_excel(in_file, 'single_no_names')
+        tm.assert_frame_equal(actual, expected)
+        with tm.assert_produces_warning(FutureWarning):
+            actual = pd.read_excel(in_file, 'single_no_names', has_index_names=False)
+        tm.assert_frame_equal(actual, expected)
+
+        expected.index = mi
+        with tm.assert_produces_warning(FutureWarning):
+            actual = pd.read_excel(in_file, 'multi_names', has_index_names=True)
+        tm.assert_frame_equal(actual, expected)
+
+        expected.index.names = [None, None]
+        actual = pd.read_excel(in_file, 'multi_no_names', index_col=[0,1])
+        tm.assert_frame_equal(actual, expected, check_names=False)
+        with tm.assert_produces_warning(FutureWarning):
+            actual = pd.read_excel(in_file, 'multi_no_names', index_col=[0,1],
+                                   has_index_names=False)
+        tm.assert_frame_equal(actual, expected, check_names=False)
+
+
 
 class XlsReaderTests(XlrdTests, tm.TestCase):
     ext = '.xls'
@@ -537,6 +661,8 @@ class XlsmReaderTests(XlrdTests, tm.TestCase):
     check_skip = staticmethod(_skip_if_no_xlrd)
 
 
+
+
 class ExcelWriterBase(SharedItems):
     # Base class for test cases to run with different Excel writers.
     # To add a writer test, define the following:
@@ -781,7 +907,6 @@ def test_roundtrip_indexlabels(self):
             reader = ExcelFile(path)
             recons = reader.parse('test1',
                                   index_col=0,
-                                  has_index_names=self.merge_cells
                                   ).astype(np.int64)
             frame.index.names = ['test']
             self.assertEqual(frame.index.names, recons.index.names)
@@ -794,7 +919,6 @@ def test_roundtrip_indexlabels(self):
             reader = ExcelFile(path)
             recons = reader.parse('test1',
                                   index_col=0,
-                                  has_index_names=self.merge_cells
                                   ).astype(np.int64)
             frame.index.names = ['test']
             self.assertEqual(frame.index.names, recons.index.names)
@@ -807,7 +931,6 @@ def test_roundtrip_indexlabels(self):
             reader = ExcelFile(path)
             recons = reader.parse('test1',
                                   index_col=0,
-                                  has_index_names=self.merge_cells
                                   ).astype(np.int64)
             frame.index.names = ['test']
             tm.assert_frame_equal(frame, recons.astype(bool))
@@ -837,8 +960,7 @@ def test_excel_roundtrip_indexname(self):
 
             xf = ExcelFile(path)
             result = xf.parse(xf.sheet_names[0],
-                              index_col=0,
-                              has_index_names=self.merge_cells)
+                              index_col=0)
 
             tm.assert_frame_equal(result, df)
             self.assertEqual(result.index.name, 'foo')
@@ -925,8 +1047,7 @@ def test_to_excel_multiindex(self):
             frame.to_excel(path, 'test1', merge_cells=self.merge_cells)
             reader = ExcelFile(path)
             df = reader.parse('test1', index_col=[0, 1],
-                              parse_dates=False,
-                              has_index_names=self.merge_cells)
+                              parse_dates=False)
             tm.assert_frame_equal(frame, df)
             self.assertEqual(frame.index.names, df.index.names)
 
@@ -943,8 +1064,7 @@ def test_to_excel_multiindex_dates(self):
             tsframe.to_excel(path, 'test1', merge_cells=self.merge_cells)
             reader = ExcelFile(path)
             recons = reader.parse('test1',
-                                  index_col=[0, 1],
-                                  has_index_names=self.merge_cells)
+                                  index_col=[0, 1])
 
             tm.assert_frame_equal(tsframe, recons)
             self.assertEqual(recons.index.names, ('time', 'foo'))
@@ -1475,15 +1595,14 @@ def test_excel_raise_error_on_multiindex_columns_and_no_index(self):
             with ensure_clean(self.ext) as path:
                 df.to_excel(path, index=False)
 
-    def test_excel_warns_verbosely_on_multiindex_columns_and_index_true(self):
+    def test_excel_multiindex_columns_and_index_true(self):
         _skip_if_no_xlwt()
         cols = MultiIndex.from_tuples([('site', ''),
                                           ('2014', 'height'),
                                           ('2014', 'weight')])
-        df = DataFrame(np.random.randn(10, 3), columns=cols)
-        with tm.assert_produces_warning(UserWarning):
-            with ensure_clean(self.ext) as path:
-                df.to_excel(path, index=True)
+        df = pd.DataFrame(np.random.randn(10, 3), columns=cols)
+        with ensure_clean(self.ext) as path:
+            df.to_excel(path, index=True)
 
     def test_excel_multiindex_index(self):
         _skip_if_no_xlwt()