Repository: pandas-dev/pandas

Issue:
DEPR: list of lists in Series.str.cat
The `.str.cat`-method is the only one in the `str`-accessor that takes another Series as an argument, and as such, is a bit of a special case (e.g. it had no index alignment until v0.23).

It makes sense to support lists of objects which get concatenated sequentially, and list of lists have been supported since at least v0.17, see https://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.Series.str.cat.html

When I wrote #20347, I tried very hard to keep signature backwards-compatible, and the example from the v0.17-22 docs working:
```
>>> Series(['a', 'b']).str.cat([['x', 'y'], ['1', '2']], sep=',')
0    a,x,1
1    b,y,2
dtype: object
```
However, this added lots of complexity, and I think that this should be simplified, especially in light of @TomAugspurger's comment in #21894
> As a reminder, the plan is to have no new deprecations in 0.25.x and 1.0.0. So this [v0.24] is the last round of deprecations before 1.0.

My suggestion is to modify the allowed combinations (as of v0.23) as follows:
```
Type of "others"                        |  action  |  comment
---------------------------------------------------------------------
list-like of strings                    |   keep   |  as before; mimics behavior elsewhere,
                                                      cf.: pd.Series(range(3)) + [2,4,6]
Series                                  |   keep   |
np.ndarray (1-dim)                      |   keep   |
DataFrame                               |   keep   |  sequential concatenation
np.ndarray (2-dim)                      |   keep   |  sequential concatenation
list-like of
    Series/Index/np.ndarray (1-dim)     |   keep   |  sequential concatenation
list-like containing list-likes (1-dim)
    other than Series/Index/np.ndarray  |   DEPR   |  sequential concatenation
```

In other words, if the user wants sequential concatenation, there are many possibilities available, and list-of-lists does not have to be one of them, IMO. This would substantially simplify (post-deprecation) the code for `str.cat._get_series_list`, which is currently a bit complicated. https://github.com/pandas-dev/pandas/blob/v0.23.3/pandas/core/strings.py#L2089

Finally, for completeness, the example from the v0.17-22 docs has been removed for v0.23, but there are two examples in https://pandas.pydata.org/pandas-docs/stable/text.html#concatenating-a-series-and-many-objects-into-a-series that would fall under the deprecation I'm suggesting.


Discussion:
To check for myself, and as a little "advert" for this proposal, I wanted to see what `_get_series_list` would look like with the above deprecation (and removal of the `FutureWarning`, once `str.cat` aligns by default). This is to be compared with https://github.com/pandas-dev/pandas/blob/v0.23.3/pandas/core/strings.py#L2089

```
def _get_series_list(self, others):
    """
    Auxiliary function for :meth:`str.cat`. Turn potentially mixed input
    into a list of Series (elements without an index must match the length
    of the calling Series/Index).

    Parameters
    ----------
    others : Series, DataFrame, np.ndarray, list-like or list-like of
        objects that are either Series, Index or np.ndarray (1-dim)

    Returns
    -------
    list : others transformed into list of Series
    """
    from pandas import Index, Series, DataFrame

    # self._orig is either Series or Index
    idx = self._orig if isinstance(self._orig, Index) else self._orig.index

    # Generally speaking, all objects without an index inherit the index
    # `idx` of the calling Series/Index - i.e. must have matching length.
    # Objects with an index (i.e. Series/Index/DataFrame) keep their own.
    if isinstance(others, Series):
        return [others]
    elif isinstance(others, Index):
        return [Series(others.values, index=others)]
    elif isinstance(others, DataFrame):
        return [others[x] for x in others]
    elif isinstance(others, np.ndarray) and others.ndim == 2:
        others = DataFrame(others, index=idx)
        return [others[x] for x in others]
    elif is_list_like(others):
        others = list(others)  # ensure iterators do not get read twice etc

        # in case of list-like `others`, all elements must be
        # either Series/Index/np.ndarray (1-dim)...
        if all(isinstance(x, (Series, Index))
               or (isinstance(x, np.ndarray) and x.ndim == 1) for x in others):
            los = []
            while others:  # iterate through list and append each element
                los = los + self._get_series_list(others.pop(0))
            return los
        # ... or just strings
        elif all(not is_list_like(x) for x in others):
            return [Series(others, index=idx)]
    raise TypeError('others must be Series, Index, DataFrame, np.ndarrary or '
                    'list-like (either containing only strings or containing '
                    'only objects of type Series/Index/np.ndarray[1-dim])')
```

Resolving patch:
diff --git a/pandas/core/strings.py b/pandas/core/strings.py
--- a/pandas/core/strings.py
+++ b/pandas/core/strings.py
@@ -1930,8 +1930,8 @@ def _get_series_list(self, others, ignore_index=False):
 
         Parameters
         ----------
-        others : Series, DataFrame, np.ndarray, list-like or list-like of
-            objects that are either Series, np.ndarray (1-dim) or list-like
+        others : Series, Index, DataFrame, np.ndarray, list-like or list-like
+            of objects that are Series, Index or np.ndarray (1-dim)
         ignore_index : boolean, default False
             Determines whether to forcefully align others with index of caller
 
@@ -1941,7 +1941,7 @@ def _get_series_list(self, others, ignore_index=False):
                  boolean whether FutureWarning should be raised)
         """
 
-        # once str.cat defaults to alignment, this function can be simplified;
+        # Once str.cat defaults to alignment, this function can be simplified;
         # will not need `ignore_index` and the second boolean output anymore
 
         from pandas import Index, Series, DataFrame
@@ -1986,11 +1986,20 @@ def _get_series_list(self, others, ignore_index=False):
             # either one-dimensional list-likes or scalars
             if all(is_list_like(x) for x in others):
                 los = []
-                warn = False
+                join_warn = False
+                depr_warn = False
                 # iterate through list and append list of series for each
                 # element (which we check to be one-dimensional and non-nested)
                 while others:
                     nxt = others.pop(0)  # nxt is guaranteed list-like by above
+
+                    # GH 21950 - DeprecationWarning
+                    # only allowing Series/Index/np.ndarray[1-dim] will greatly
+                    # simply this function post-deprecation.
+                    if not (isinstance(nxt, (Series, Index)) or
+                            (isinstance(nxt, np.ndarray) and nxt.ndim == 1)):
+                        depr_warn = True
+
                     if not isinstance(nxt, (DataFrame, Series,
                                             Index, np.ndarray)):
                         # safety for non-persistent list-likes (e.g. iterators)
@@ -2013,8 +2022,14 @@ def _get_series_list(self, others, ignore_index=False):
                     nxt, wnx = self._get_series_list(nxt,
                                                      ignore_index=ignore_index)
                     los = los + nxt
-                    warn = warn or wnx
-                return (los, warn)
+                    join_warn = join_warn or wnx
+
+                if depr_warn:
+                    warnings.warn('list-likes other than Series, Index, or '
+                                  'np.ndarray WITHIN another list-like are '
+                                  'deprecated and will be removed in a future '
+                                  'version.', FutureWarning, stacklevel=3)
+                return (los, join_warn)
             elif all(not is_list_like(x) for x in others):
                 return ([Series(others, index=idx)], False)
         raise TypeError(err_msg)
@@ -2037,8 +2052,8 @@ def cat(self, others=None, sep=None, na_rep=None, join=None):
             Series/Index/DataFrame) if `join` is not None.
 
             If others is a list-like that contains a combination of Series,
-            np.ndarray (1-dim) or list-like, then all elements will be unpacked
-            and must satisfy the above criteria individually.
+            Index or np.ndarray (1-dim), then all elements will be unpacked and
+            must satisfy the above criteria individually.
 
             If others is None, the method returns the concatenation of all
             strings in the calling Series/Index.


Test patch:
diff --git a/pandas/tests/test_strings.py b/pandas/tests/test_strings.py
--- a/pandas/tests/test_strings.py
+++ b/pandas/tests/test_strings.py
@@ -313,7 +313,9 @@ def test_str_cat_mixed_inputs(self, series_or_index):
             assert_series_or_index_equal(s.str.cat([tt, s]), exp)
 
         # Series/Index with list of list-likes
-        assert_series_or_index_equal(s.str.cat([t.values, list(s)]), exp)
+        with tm.assert_produces_warning(expected_warning=FutureWarning):
+            # nested lists will be deprecated
+            assert_series_or_index_equal(s.str.cat([t.values, list(s)]), exp)
 
         # Series/Index with mixed list of Series/list-like
         # s as Series has same index as t -> no warning
@@ -327,7 +329,10 @@ def test_str_cat_mixed_inputs(self, series_or_index):
             assert_series_or_index_equal(s.str.cat([tt, s.values]), exp)
 
         # Series/Index with iterator of list-likes
-        assert_series_or_index_equal(s.str.cat(iter([t.values, list(s)])), exp)
+        with tm.assert_produces_warning(expected_warning=FutureWarning):
+            # nested list-likes will be deprecated
+            assert_series_or_index_equal(s.str.cat(iter([t.values, list(s)])),
+                                         exp)
 
         # errors for incorrect lengths
         rgx = 'All arrays must be same length, except.*'
@@ -348,11 +353,11 @@ def test_str_cat_mixed_inputs(self, series_or_index):
 
         # list of list-likes
         with tm.assert_raises_regex(ValueError, rgx):
-            s.str.cat([z.values, list(s)])
+            s.str.cat([z.values, s.values])
 
         # mixed list of Series/list-like
         with tm.assert_raises_regex(ValueError, rgx):
-            s.str.cat([z, list(s)])
+            s.str.cat([z, s.values])
 
         # errors for incorrect arguments in list-like
         rgx = 'others must be Series, Index, DataFrame,.*'
@@ -423,11 +428,15 @@ def test_str_cat_align_mixed_inputs(self, join):
         e = concat([t, s], axis=1, join=(join if join == 'inner' else 'outer'))
         sa, ea = s.align(e, join=join)
         exp = exp_outer.loc[ea.index]
-        tm.assert_series_equal(s.str.cat([t, u], join=join, na_rep='-'), exp)
+
+        with tm.assert_produces_warning(expected_warning=FutureWarning):
+            # nested lists will be deprecated
+            tm.assert_series_equal(s.str.cat([t, u], join=join, na_rep='-'),
+                                   exp)
 
         # errors for incorrect lengths
         rgx = 'If `others` contains arrays or lists.*'
-        z = ['1', '2', '3']
+        z = Series(['1', '2', '3']).values
 
         # unindexed object of wrong length
         with tm.assert_raises_regex(ValueError, rgx):
@@ -442,8 +451,8 @@ def test_str_cat_special_cases(self):
         t = Series(['d', 'a', 'e', 'b'], index=[3, 0, 4, 1])
 
         # iterator of elements with different types
-        exp = Series(['aaA', 'bbB', 'c-C', 'ddD', '-e-'])
-        tm.assert_series_equal(s.str.cat(iter([t, ['A', 'B', 'C', 'D']]),
+        exp = Series(['aaa', 'bbb', 'c-c', 'ddd', '-e-'])
+        tm.assert_series_equal(s.str.cat(iter([t, s.values]),
                                          join='outer', na_rep='-'), exp)
 
         # right-align with different indexes in others