Repository: pandas-dev/pandas

Issue:
BUG: concat with dataframes heterogeneous in length and tuple keys
#### A small, complete example of the issue

``` python
import numpy as np
import pandas as pd
df1 = pd.DataFrame(np.random.rand(4).reshape((2, 2)), columns=list('AB'))
df2 = pd.DataFrame(np.random.rand(6).reshape((3, 2)), columns=list('AB'))
print(pd.concat([df1, df2], keys=[('bee', 'bah'), ('bee', 'bouh')], names=['zig', 'zag', None]))
```
#### Expected Output (works w/ 0.18.1)

```
                   A         B
zig zag                       
bee bah  0  0.689411  0.364650
         1  0.143786  0.415858
    bouh 0  0.557942  0.646434
         1  0.551938  0.695175
         2  0.991975  0.880451
```
#### 0.19 raises

```
/path/to/lib/python3.4/site-packages/pandas/tools/merge.py in _make_concat_multiindex(indexes, keys, levels, names)
   1673         else:
   1674             codes, categories = _factorize_from_iterable(concat_index)
-> 1675             levels.append(categories)
   1676             label_list.append(codes)
   1677 

AttributeError: 'tuple' object has no attribute 'append'
```
#### Output of `pd.show_versions()`

<details>
## INSTALLED VERSIONS

commit: None
python: 3.4.5.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-74-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: 
LOCALE: None.None

pandas: 0.19.0
nose: None
pip: 8.1.2
setuptools: 27.2.0
Cython: None
numpy: 1.11.2
scipy: 0.17.1
statsmodels: 0.6.1
xarray: None
IPython: 4.2.0
sphinx: 1.4.1
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.7
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None
</details>

BUG: concat with dataframes heterogeneous in length and tuple keys
#### A small, complete example of the issue

``` python
import numpy as np
import pandas as pd
df1 = pd.DataFrame(np.random.rand(4).reshape((2, 2)), columns=list('AB'))
df2 = pd.DataFrame(np.random.rand(6).reshape((3, 2)), columns=list('AB'))
print(pd.concat([df1, df2], keys=[('bee', 'bah'), ('bee', 'bouh')], names=['zig', 'zag', None]))
```
#### Expected Output (works w/ 0.18.1)

```
                   A         B
zig zag                       
bee bah  0  0.689411  0.364650
         1  0.143786  0.415858
    bouh 0  0.557942  0.646434
         1  0.551938  0.695175
         2  0.991975  0.880451
```
#### 0.19 raises

```
/path/to/lib/python3.4/site-packages/pandas/tools/merge.py in _make_concat_multiindex(indexes, keys, levels, names)
   1673         else:
   1674             codes, categories = _factorize_from_iterable(concat_index)
-> 1675             levels.append(categories)
   1676             label_list.append(codes)
   1677 

AttributeError: 'tuple' object has no attribute 'append'
```
#### Output of `pd.show_versions()`

<details>
## INSTALLED VERSIONS

commit: None
python: 3.4.5.final.0
python-bits: 64
OS: Linux
OS-release: 3.13.0-74-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: 
LOCALE: None.None

pandas: 0.19.0
nose: None
pip: 8.1.2
setuptools: 27.2.0
Cython: None
numpy: 1.11.2
scipy: 0.17.1
statsmodels: 0.6.1
xarray: None
IPython: 4.2.0
sphinx: 1.4.1
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.7
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None
</details>



Discussion:
Thanks for the report, PRs welcome!

Thanks for the report, PRs welcome!


Resolving patch:
diff --git a/pandas/core/categorical.py b/pandas/core/categorical.py
--- a/pandas/core/categorical.py
+++ b/pandas/core/categorical.py
@@ -2055,14 +2055,14 @@ def _factorize_from_iterables(iterables):
 
     Returns
     -------
-    codes_tuple : tuple of ndarrays
-    categories_tuple : tuple of Indexes
+    codes_list : list of ndarrays
+    categories_list : list of Indexes
 
     Notes
     -----
     See `_factorize_from_iterable` for more info.
     """
     if len(iterables) == 0:
-        # For consistency, it should return a list of 2 tuples.
-        return [(), ()]
-    return lzip(*[_factorize_from_iterable(it) for it in iterables])
+        # For consistency, it should return a list of 2 lists.
+        return [[], []]
+    return map(list, lzip(*[_factorize_from_iterable(it) for it in iterables]))


Test patch:
diff --git a/pandas/tests/frame/test_combine_concat.py b/pandas/tests/frame/test_combine_concat.py
--- a/pandas/tests/frame/test_combine_concat.py
+++ b/pandas/tests/frame/test_combine_concat.py
@@ -57,6 +57,24 @@ def test_concat_multiple_tzs(self):
         expected = DataFrame(dict(time=[ts2, ts3]))
         assert_frame_equal(results, expected)
 
+    def test_concat_tuple_keys(self):
+        # GH 14438
+        df1 = pd.DataFrame(np.ones((2, 2)), columns=list('AB'))
+        df2 = pd.DataFrame(np.ones((3, 2)) * 2, columns=list('AB'))
+        results = pd.concat((df1, df2), keys=[('bee', 'bah'), ('bee', 'boo')])
+        expected = pd.DataFrame(
+            {'A': {('bee', 'bah', 0): 1.0,
+                   ('bee', 'bah', 1): 1.0,
+                   ('bee', 'boo', 0): 2.0,
+                   ('bee', 'boo', 1): 2.0,
+                   ('bee', 'boo', 2): 2.0},
+             'B': {('bee', 'bah', 0): 1.0,
+                   ('bee', 'bah', 1): 1.0,
+                   ('bee', 'boo', 0): 2.0,
+                   ('bee', 'boo', 1): 2.0,
+                   ('bee', 'boo', 2): 2.0}})
+        assert_frame_equal(results, expected)
+
     def test_append_series_dict(self):
         df = DataFrame(np.random.randn(5, 4),
                        columns=['foo', 'bar', 'baz', 'qux'])