BUG: Fixed handling of non-list value_vars in melt
State: closed

The `value_vars` argument of `melt` is now cast to `list` like the `id_vars` argument. This works for tuples, lists, ndarrays, and scalars.

 - [x] closes #15348 
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15351?src=pr&el=h1) Report
> Merging [#15351](https://codecov.io/gh/pandas-dev/pandas/pull/15351?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/e8840725447859531ddcc4b878266f2043fb6465?src=pr&el=desc) will **increase** coverage by `3.04%`.


```diff
@@            Coverage Diff             @@
##           master   #15351      +/-   ##
==========================================
+ Coverage    88.7%   91.75%   +3.04%     
==========================================
  Files         134      135       +1     
  Lines       49489    49439      -50     
==========================================
+ Hits        43901    45364    +1463     
+ Misses       5588     4075    -1513
```


| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15351?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/reshape.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS5weQ==) | `99.25% <100%> (ø)` | :white_check_mark: |
| [pandas/__init__.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL19faW5pdF9fLnB5) | `91.66% <ø> (-0.65%)` | :x: |
| [pandas/tseries/index.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvaW5kZXgucHk=) | `95.32% <ø> (-0.1%)` | :x: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.15% <ø> (-0.04%)` | :x: |
| [pandas/util/nosetester.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvbm9zZXRlc3Rlci5weQ==) | | |
| [pandas/conftest.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2NvbmZ0ZXN0LnB5) | `75% <ø> (ø)` | |
| [pandas/util/_tester.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX3Rlc3Rlci5weQ==) | `35.29% <ø> (ø)` | |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.39% <ø> (+0.05%)` | :white_check_mark: |
| [pandas/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2Zvcm1hdHMvZm9ybWF0LnB5) | `95.38% <ø> (+0.06%)` | :white_check_mark: |
| [pandas/io/excel.py](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...3038f646fb949b60073a73df3cf10e52c46905fc?src=pr&el=tree#diff-cGFuZGFzL2lvL2V4Y2VsLnB5) | `79.54% <ø> (+0.15%)` | :white_check_mark: |
| ... and [16 more](https://codecov.io/gh/pandas-dev/pandas/pull/15351/changes?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15351?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15351?src=pr&el=footer). Last update [e884072...a2f2510](https://codecov.io/gh/pandas-dev/pandas/compare/e8840725447859531ddcc4b878266f2043fb6465...a2f2510734b08c81018b320c46e3c777583713ea?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
I've addressed your requests.

One comment: in `reshape.py`, the type check is now
```python
if not is_list_like(id_vars):
    id_vars = [id_vars]
else:
    id_vars = list(id_vars)
```
(and the same for `value_vars`). This seems a bit redundant to me since, unless I'm mistaken, `list` would work for both. But I thought I'd ask in case I'm missing something.
@jbradt no, this is quite common to do. We need a list result here even if a scalar is passed in, but you can't simply put ``list()`` around everything, e.g. things like datetimes or strings would be misinterpreted.
@jreback Ah, got it. I hadn't thought of strings, etc. Thanks for clearing that up!
@jbradt np. that why we have this special ``is_list_like`` to get this kind of consistency w/o having to repeat things all over.
just a couple of trvial edits. ping when green (might be a while as travis is doing lots of builds)
Didn't look into the code, but: shouldn't tuples be reserved for specifying multi-indexed columns? 
Not sure if that's very useful in the case of melt, though, as you get tuples in the columns of the resulting frame.
@jreback Changes made. Everything's green.
So from the doc-string examples
```
In [6]: df = pd.DataFrame({'A': {0: 'a', 1: 'b', 2: 'c'},
   ...: ...                    'B': {0: 1, 1: 3, 2: 5},
   ...: ...                    'C': {0: 2, 1: 4, 2: 6}})
   ...: 

In [7]: df
Out[7]: 
   A  B  C
0  a  1  2
1  b  3  4
2  c  5  6

In [9]: df.columns = [list('ABC'), list('DEF')]
   ...: 

In [10]: df
Out[10]: 
   A  B  C
   D  E  F
0  a  1  2
1  b  3  4
2  c  5  6

In [11]: pd.melt?

In [12]: pd.melt(df, id_vars=[('A', 'D')], value_vars=[('B', 'E')])
Out[12]: 
  (A, D) variable_0 variable_1  value
0      a          B          E      1
1      b          B          E      3
2      c          B          E      5
```

So I think this is fine, a tuple is a-list like and NOT a scalar that selects multiple columns.

Actually you get a terrible error message now. If you pass a tuple with a MI.
```
In [13]: pd.melt(df, id_vars=[('A', 'D')], value_vars=('B', 'E'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-db2c189272b2> in <module>()
----> 1 pd.melt(df, id_vars=[('A', 'D')], value_vars=('B', 'E'))

/Users/jreback/pandas/pandas/core/reshape.py in melt(frame, id_vars, value_vars, var_name, value_name, col_level)
    772         if not isinstance(value_vars, (tuple, list, np.ndarray)):
    773             value_vars = [value_vars]
--> 774         frame = frame.loc[:, id_vars + value_vars]
    775     else:
    776         frame = frame.copy()

TypeError: can only concatenate list (not "tuple") to list
```

@jbradt can you add this example. I think we need a better message here.
@jreback Sure, but I'm not totally sure I understand what you mean. Do you want an example like you wrote above that says that you **must** pass a list of tuples if you have a `MultiIndex`? Or do you want me to check in `melt` if there's a `MultiIndex` and raise a better error message if so?

Incidentally, the exception I get after my changes is much more heinous than what you showed:

```pycon
In [8]: pd.melt(df, id_vars=[('A', 'D')], value_vars=('B', 'E'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-db2c189272b2> in <module>()
----> 1 pd.melt(df, id_vars=[('A', 'D')], value_vars=('B', 'E'))

/Users/josh/Documents/Code/pandas/pandas/core/reshape.py in melt(frame, id_vars, value_vars, var_name, value_name, col_level)
    774         else:
    775             value_vars = list(value_vars)
--> 776         frame = frame.loc[:, id_vars + value_vars]
    777     else:
    778         frame = frame.copy()

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in __getitem__(self, key)
   1335             except (KeyError, IndexError):
   1336                 pass
-> 1337             return self._getitem_tuple(key)
   1338         else:
   1339             key = com._apply_if_callable(key, self.obj)

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in _getitem_tuple(self, tup)
    803     def _getitem_tuple(self, tup):
    804         try:
--> 805             return self._getitem_lowerdim(tup)
    806         except IndexingError:
    807             pass

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in _getitem_lowerdim(self, tup)
    915         # we may have a nested tuples indexer here
    916         if self._is_nested_tuple_indexer(tup):
--> 917             return self._getitem_nested_tuple(tup)
    918
    919         # we maybe be using a tuple to represent multiple dimensions here

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in _getitem_nested_tuple(self, tup)
    990
    991             current_ndim = obj.ndim
--> 992             obj = getattr(obj, self.name)._getitem_axis(key, axis=axis)
    993             axis += 1
    994

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1538                     raise ValueError('Cannot index with multidimensional key')
   1539
-> 1540                 return self._getitem_iterable(key, axis=axis)
   1541
   1542             # nested tuple slicing

/Users/josh/Documents/Code/pandas/pandas/core/indexing.py in _getitem_iterable(self, key, axis)
   1084                 try:
   1085                     result = self.obj.reindex_axis(keyarr, axis=axis,
-> 1086                                                    level=level)
   1087
   1088                     # this is an error as we are trying to find

/Users/josh/Documents/Code/pandas/pandas/core/frame.py in reindex_axis(self, labels, axis, method, level, copy, limit, fill_value)
   2796                      self).reindex_axis(labels=labels, axis=axis,
   2797                                         method=method, level=level, copy=copy,
-> 2798                                         limit=limit, fill_value=fill_value)
   2799
   2800     @Appender(_shared_docs['rename'] % _shared_doc_kwargs)

/Users/josh/Documents/Code/pandas/pandas/core/generic.py in reindex_axis(self, labels, axis, method, level, copy, limit, fill_value)
   2413         method = missing.clean_reindex_fill_method(method)
   2414         new_index, indexer = axis_values.reindex(labels, method, level,
-> 2415                                                  limit=limit)
   2416         return self._reindex_with_indexers({axis: [new_index, indexer]},
   2417                                            fill_value=fill_value, copy=copy)

/Users/josh/Documents/Code/pandas/pandas/indexes/multi.py in reindex(self, target, method, level, limit, tolerance)
   1563             else:
   1564                 # hopefully?
-> 1565                 target = MultiIndex.from_tuples(target)
   1566
   1567         if (preserve_names and target.nlevels == self.nlevels and

/Users/josh/Documents/Code/pandas/pandas/indexes/multi.py in from_tuples(cls, tuples, sortorder, names)
   1012                 tuples = tuples._values
   1013
-> 1014             arrays = list(lib.tuples_to_object_array(tuples).T)
   1015         elif isinstance(tuples, list):
   1016             arrays = list(lib.to_object_array_tuples(tuples).T)

/Users/josh/Documents/Code/pandas/pandas/src/inference.pyx in pandas.lib.tuples_to_object_array (pandas/lib.c:68717)()
   1457     result = np.empty((n, k), dtype=object)
   1458     for i in range(n):
-> 1459         tup = tuples[i]
   1460         for j in range(k):
   1461             result[i, j] = tup[j]

TypeError: Expected tuple, got str
```

That also happens if `value_vars=(('A', 'B'))` since `list` flattens the tuple.
@jbradt right, I think we should explicity check & raise an better message if a tuple is passed (when you have a MI). It should be a *list* of tuples, so this is an easy condition to catch (and give a better message).
lgtm. ping on green.
@jbradt there is some noise here because of our switchover to pytest and numpy getting upgraded.

https://travis-ci.org/pandas-dev/pandas/jobs/200158415

but I think you have some linting errors. and use ``tm.assertRaisesRegex``


@jreback I think that's an old build since I already fixed the linting error, but I just pushed a change to use `tm.assertRaisesRegexp`. Sometimes I forget about Python 2...
@jbradt sure no problem.
thanks @jbradt