Repository: pandas-dev/pandas

Issue:
ENH: handle multiple tbody in read_html()
#### Code Sample
```
url = 'https://www.wunderground.com/history/airport/KORD/2018/3/21/CustomHistory.html?dayend=10&monthend=4&yearend=2018'
df = pandas.read_html(url)[1]
```
#### Problem description

Expected: table of weather information at page bottom. 

Actual:
 0.21.1 -  first two rows
 0.22.0 - exception
 current master - first two rows

For reference, Google Sheets IMPORTHTML loads this table correctly.,



Discussion:
It looks like the second table has multiple `tbody` tags but the parser only looks at the first:

https://github.com/pandas-dev/pandas/blob/ad5affb6417c26bc7f3bc10668f0cf63fd867940/pandas/io/html.py#L394

PRs welcome
Thanks. I'll probably have to fix this.
We could always enhance `read_html` to behave like `read_excel` where we can read in multiple `tbody` elements.  That doesn't seem too unreasonable for the moment.
How is read_excel similar to this? There’s no tbody in that case.
@gfyoung are you just referring to the ability to return multiple tables? `read_html` technically already does that as it returns a list instead of just a DataFrame, but chime in if I misunderstand.

@jstray one design consideration to think about - should multiple `tbody` elements return a MultiIndexed DataFrame? I suppose having those multiple `tbody` tags in the first place is indicative of different groupings within the table, so maybe it makes sense for each of them to be a unique value within the first level of a MultiIndex?

Not saying that needs to happen at the outset as certainly being able to just parse them would be an improvement over what we have today. Just throwing that out there as food for thought as you try a patch
@WillAyd : Sorry, misspoke there.  Please ignore that comment. 😄 

Resolving patch:
diff --git a/pandas/io/html.py b/pandas/io/html.py
--- a/pandas/io/html.py
+++ b/pandas/io/html.py
@@ -324,7 +324,7 @@ def _parse_thead(self, table):
         raise com.AbstractMethodError(self)
 
     def _parse_tbody(self, table):
-        """Return the body of the table.
+        """Return the list of tbody elements from the parsed table element.
 
         Parameters
         ----------
@@ -333,8 +333,8 @@ def _parse_tbody(self, table):
 
         Returns
         -------
-        tbody : node-like
-            A <tbody>...</tbody> element.
+        tbodys : list of node-like
+            A list of <tbody>...</tbody> elements
         """
         raise com.AbstractMethodError(self)
 
@@ -388,13 +388,17 @@ def _parse_raw_tfoot(self, table):
             np.array(res).squeeze()) if res and len(res) == 1 else res
 
     def _parse_raw_tbody(self, table):
-        tbody = self._parse_tbody(table)
+        tbodies = self._parse_tbody(table)
 
-        try:
-            res = self._parse_tr(tbody[0])
-        except IndexError:
-            res = self._parse_tr(table)
-        return self._parse_raw_data(res)
+        raw_data = []
+
+        if tbodies:
+            for tbody in tbodies:
+                raw_data.extend(self._parse_tr(tbody))
+        else:
+            raw_data.extend(self._parse_tr(table))
+
+        return self._parse_raw_data(raw_data)
 
     def _handle_hidden_tables(self, tbl_list, attr_name):
         """Returns list of tables, potentially removing hidden elements


Test patch:
diff --git a/pandas/tests/io/test_html.py b/pandas/tests/io/test_html.py
--- a/pandas/tests/io/test_html.py
+++ b/pandas/tests/io/test_html.py
@@ -396,6 +396,33 @@ def test_empty_tables(self):
         res2 = self.read_html(StringIO(data2))
         assert_framelist_equal(res1, res2)
 
+    def test_multiple_tbody(self):
+        # GH-20690
+        # Read all tbody tags within a single table.
+        data = '''<table>
+            <thead>
+                <tr>
+                    <th>A</th>
+                    <th>B</th>
+                </tr>
+            </thead>
+            <tbody>
+                <tr>
+                    <td>1</td>
+                    <td>2</td>
+                </tr>
+            </tbody>
+            <tbody>
+                <tr>
+                    <td>3</td>
+                    <td>4</td>
+                </tr>
+            </tbody>
+        </table>'''
+        expected = DataFrame({'A': [1, 3], 'B': [2, 4]})
+        result = self.read_html(StringIO(data))[0]
+        tm.assert_frame_equal(result, expected)
+
     def test_header_and_one_column(self):
         """
         Don't fail with bs4 when there is a header and only one column