BUG: fix DataFrame constructor w named Series
State: closed

closes #7893

Closes https://github.com/pydata/pandas/issues/9232

Problem was passing Series w/ a name to DataFrame w/ the `columns` kwarg.

Before:

``` python
In [55]: x = pd.Series(range(5), name=1)

In [56]: y = pd.Series(range(5), name=0)

In [57]: pd.DataFrame(x, columns=[0])
Out[57]:
Empty DataFrame
Columns: [0]
Index: []

In [58]: pd.DataFrame(x, columns=[1])
Out[58]:
   1
0  0
1  1
2  2
3  3
4  4

In [59]: pd.DataFrame(y, columns=[0])
Out[59]:
   0
0  0
1  1
2  2
3  3
4  4

In [60]: pd.DataFrame(y, columns=[1])
Out[60]:
   1
0  0
1  1
2  2
3  3
4  4
```

after

``` python
In [1]: x = pd.Series(range(5), name=1)

In [2]: y = pd.Series(range(5), name=0)

In [4]: pd.DataFrame(x, columns=[0])
Out[4]: 
   0
0  0
1  1
2  2
3  3
4  4

In [5]: pd.DataFrame(y, columns=[1])
Out[5]: 
   1
0  0
1  1
2  2
3  3
4  4
```

There were two intertwined problems
1. we checked `if getattr(data, 'name', None):`, which returned False when data.name was `False`ish (like 0). I now compare it directly against None.
2. If `data` has a name and the columns kwarg is specified, the constructor returned an Empty DataFrame w/ the column specified in columns. Now, we do what's [documented](http://pandas.pydata.org/pandas-docs/version/0.15.2/dsintro.html#from-a-series):

> The result will be a DataFrame with the same index as the input Series, and with one column whose name is the original name of the Series (only if no other column name provided).


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: API Design

Comments:
this is an api change

i will have to look at this

Yep, I wasn't sure where to document it in whatsnew. I can expand it.

I think the docs are pretty clear that this was the _intended_ behavior..

How I always understood it, it is the `y` case (`pd.DataFrame(y, columns=[1])` _not_ giving an empty dataframe) that was buggy and not the `x` case (`pd.DataFrame(x, columns=[0])` giving an empty frame). It was sometimes odd or surpising behaviour, but giving a series/frame to `DataFrame` is regarded as a reindex.

I wanted to say that this is in line with how a DataFrame itself is treated when given to `DataFrame`, but it appears this is still somewhat different (not giving an empty frame, but one with NaNs):

```
In [42]: x = pd.Series(range(5), name=1)

In [43]: pd.DataFrame(x.to_frame(), columns=[0])
Out[43]: 
    0
0 NaN
1 NaN
2 NaN
3 NaN
4 NaN
```

see more discusion on #7893 

this AFAIK was a longstanding behavior which I think I broke in 0.12-13 . I agree with @jorisvandenbossche here

This should return a column of `NaN` if the passed name in the columns does not match, e.g. its a reindex. (the bug is its returning an empty frame). In fact the undelrying code should do exactly `to_frame()` as this handles the edge cases.

I think we should have a mini section in basics.rst to show conversions of series -> dataframe (and obviously in the 0.16.0 whatsnew)

OK, so I'll repurpose this PR to make to_frame() and the constructor consistent (both return a column of NaNs) and rewrite the docs.

Just for the docs, if you want to give a series as input to `DataFrame`, but _do_ want to overwrite the series name (which _is_ a valid use case), what is the recommended way to do? As first changing the series name, or afterwards renaming the dataframe is somewhat cumbersome. 
I suppose giving it as a dict:

```
In [59]: s = pd.Series([1,2,3], index=[10,11,12], name='test')

In [60]: pd.DataFrame(s)
Out[60]: 
    test
10     1
11     2
12     3

In [61]: pd.DataFrame(s, columns=['col'])
Out[61]: 
Empty DataFrame
Columns: [col]
Index: []

In [62]: pd.DataFrame({'col': s})
Out[62]: 
    col
10    1
11    2
12    3
```

OK, but of course for just this, using `to_frame` is the simplest:

```
In [63]: s.to_frame(name='col')
Out[63]: 
    col
10    1
11    2
12    3
```

@TomAugspurger can you update the top section. I don't think this is correct. The columns argument will cause a reindex of the input (after the column name is set by _name_). So if they different you would get a column of `nans`. What I think is here is inconsistent with that.

@TomAugspurger can you rebase this. and confirm (just change the top section) of the existing and new behavior (may also need an example in the api breaking section of the same).

@TomAugspurger ?

@jreback so the expected behavior is

``` python
In [24]: y = pd.Series(range(5), name=0)

In [25]: pd.DataFrame(y, columns=[1])
Out[25]: 
   1
0  NaN
1  NaN
2  NaN
3  NaN
4  NaN
```

?

IMO, it should be identical to what you get from a dict

``` python
In [29]: pd.DataFrame({y.name: y.values}, columns=[1])  # y.name is 0
Out[29]: 
Empty DataFrame
Columns: [1]
Index: []
```

So that should also be NaNs? Or no since I didn't provide an index?

@TomAugspurger 

I think [24/25] is the correct return. (0.15.2 does what you have for 24/25). [29] is just wrong (same in master).

The canonical way to do this is: `y.to_frame(name=1)` which in effect is: `DataFrame({ name : y })`, where name is y.name or the new_name if provided. 

This has to be a _reindex_, e.g. construct the dict using the current name THEN reindex. (its done like this for a dictionary construction, but it is not entirely correct; the [1] column in your example should still exist, eg. [24/25] and NOT [29])

Imagine this scenario:

```
In [1]: DataFrame({'y1' : Series(range(5),name='foo'), 'y2' : Series(range(5),name='bar') }, columns=['y2'])
Out[1]: 
   y2
0   0
1   1
2   2
3   3
4   4
```

So the columns are selecting (e.g. reindexing) on the passed dictionary keys, if you JUST pass a single Series, then it must be `{ s.name : s }` so the same thing should happen

TL;DR; but makes sense?

Yep, it definitely is a reindex. We'll go with [24]/[25].

And I'm comfortable saying that

``` python
In [16]: pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]}, columns=["C"])
```

Returning an Empty DataFrame instead of NaNs was a bug (so this isn't an API change)

@TomAugspurger agree with your [16]. thxs

This is breaking some tests unfortunately. When constructing from a dict of `{col: Series}` with differing indicies and specifying `columns`. Here's what I do now

``` python
In [2]: s1 = pd.Series([1, 2, 3], name='a')

In [3]: s2 = pd.Series([4, 5, 6], name='b', index=[2, 3, 4])

In [5]: pd.DataFrame({'a': s1, 'b': s2}, columns=['a'])
Out[5]: 
    a
0   1
1   2
2   3
3 NaN
4 NaN
```

Before it was 

``` python
In [5 master]: pd.DataFrame({'a': s1, 'b': s2}, columns=['a'])
Out[5 master]: 
    a
0   1
1   2
2   3
```

Since we filtered out `b` before determining what the index was. I think it's good that we clarifying all these constructors and make them consistent, but it's feeling pretty disrupting.

@TomAugspurger hmm, I see what you mean. I don't think this should change though. We are only reindexing on the columns (as opposed to creating the frame first from the ENTIRE dict, then reindexing)

Hmm I guess that is why the [29] is the way it is.

Ok, why don't you investigate and see what makes the most sense.

Is it cool to push this all off to 0.17? I don't have much time today and I need to squash a weird bug in the plot accessors. Is there anything that is obviously broken and should be fixed today? I'd say that this is obviously wrong.

``` python
In [1]: y = pd.Series(range(5), name=0)

In [2]: z = pd.Series(range(5), name=1)

In [3]: pd.DataFrame(y, columns=[2])
Out[3]: 
   2
0  0
1  1
2  2
3  3
4  4

In [4]: pd.DataFrame(z, columns=[2])
Out[4]: 
Empty DataFrame
Columns: [2]
Index: []
```

I jsut don't want to make anything harder down the road.

sure let's push thx!

@TomAugspurger ought to re-examine this for 0.17.0. pls rebase when you have a chance

@TomAugspurger what's status?

Just reread through. I'll have something by the end of next week.

@TomAugspurger pushing, but if you get to it next week or 2 can put in.

@TomAugspurger status?

Won't really have time to work on this in the short term. Better to close or leave open?

let's close for now.