Repository: pandas-dev/pandas

Issue:
Allow setting a column with a scalar and no index (revert #16823)
Extract of discussion from #16823

#### Code Sample, a copy-pastable example if possible

```python
In [2]: df = pd.DataFrame()

In [3]: df['dummy'] = 1
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-3122daad6dab> in <module>()
----> 1 df['dummy'] = 1

/home/nobackup/repo/pandas/pandas/core/frame.py in __setitem__(self, key, value)
   2515         else:
   2516             # set column
-> 2517             self._set_item(key, value)
   2518 
   2519     def _setitem_slice(self, key, value):

/home/nobackup/repo/pandas/pandas/core/frame.py in _set_item(self, key, value)
   2584         """
   2585 
-> 2586         self._ensure_valid_index(value)
   2587         value = self._sanitize_column(key, value)
   2588         NDFrame._set_item(self, key, value)

/home/nobackup/repo/pandas/pandas/core/frame.py in _ensure_valid_index(self, value)
   2561             if not is_list_like(value):
   2562                 # GH16823, Raise an error due to loss of information
-> 2563                 raise ValueError('If using all scalar values, you must pass'
   2564                                  ' an index')
   2565             try:

ValueError: If using all scalar values, you must pass an index
```
#### Problem description

Previously, the above would just add a new (obviously empty) column.

@jreback [objects](https://github.com/pandas-dev/pandas/issues/16823#issuecomment-336867308) that if this is allowed, then we should also allow initialization with only scalars (as in ``pd.Dataframe({'a' : 1, 'b' : 2})``

I'm not 100% sure of what @jorisvandenbossche [suggests](https://github.com/pandas-dev/pandas/issues/16823#issuecomment-336825391), but he agrees with me that the current state is inconsistent.

[My view](https://github.com/pandas-dev/pandas/issues/16823#issuecomment-336878719) is that previously things were just (almost) fine:
- at initialization, a ``DataFrame`` _needs_ to have an index. You can avoid providing one expliclty only if it can be automatically built for the values you pass (i.e. 1-dimensional objects of the same length, or a single 2-dimensional block of data). Scalars clearly do not satisfy this requirement, so the constructor will raise if passed only scalars (but ``pd.DataFrame({'A'  : range(3), 'B' : 23})`` works, which is cool).
- at assignment, there is already an index, and in particular, when assigning a(n entire) column you know you'll _never_ alter the index. More specifically, when you assign a scalar to a column, you know it will alter all _existing_ rows, which means "none" if the index is empty. And if the column does not exist, it will just be added, clearly empty as well.

In both cases, scalars/empty indexes represent no exception to the general behavior.

For consistency, we might want to fix the following too:

``` python
In [2]: pd.DataFrame().loc[1] = 0
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-20bba0809d0b> in <module>()
----> 1 pd.DataFrame().loc[1] = 0

/home/nobackup/repo/pandas/pandas/core/indexing.py in __setitem__(self, key, value)
    192             key = com._apply_if_callable(key, self.obj)
    193         indexer = self._get_setitem_indexer(key)
--> 194         self._setitem_with_indexer(indexer, value)
    195 
    196     def _has_valid_type(self, k, axis):

/home/nobackup/repo/pandas/pandas/core/indexing.py in _setitem_with_indexer(self, indexer, value)
    421                     # no columns and scalar
    422                     if not len(self.obj.columns):
--> 423                         raise ValueError("cannot set a frame with no defined "
    424                                          "columns")
    425 

ValueError: cannot set a frame with no defined columns

```

but I will detail this in a separate issue.

#### Expected Output

None, but a new column ``"dummy"`` is added to ``df``.

#### Output of ``pd.show_versions()``

<details>

INSTALLED VERSIONS
------------------
commit: 5bf7f9a4ff6968fd444fa7098f8dc95586591994
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-3-amd64
machine: x86_64
processor: 
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: it_IT.UTF-8

pandas: 0.21.0rc1+18.g5bf7f9a4f
pytest: 3.0.6
pip: 9.0.1
setuptools: None
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 5.1.0.dev
sphinx: 1.5.6
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.3.0
numexpr: 2.6.1
feather: 0.3.1
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.6
lxml: None
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.2.1


</details>



Discussion:

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -2557,17 +2557,13 @@ def _ensure_valid_index(self, value):
         passed value
         """
         # GH5632, make sure that we are a Series convertible
-        if not len(self.index):
-            if not is_list_like(value):
-                # GH16823, Raise an error due to loss of information
-                raise ValueError('If using all scalar values, you must pass'
-                                 ' an index')
+        if not len(self.index) and is_list_like(value):
             try:
                 value = Series(value)
             except:
-                raise ValueError('Cannot set a frame with no defined'
-                                 'index and a value that cannot be '
-                                 'converted to a Series')
+                raise ValueError('Cannot set a frame with no defined index '
+                                 'and a value that cannot be converted to a '
+                                 'Series')
 
             self._data = self._data.reindex_axis(value.index.copy(), axis=1,
                                                  fill_value=np.nan)
diff --git a/pandas/core/reshape/pivot.py b/pandas/core/reshape/pivot.py
--- a/pandas/core/reshape/pivot.py
+++ b/pandas/core/reshape/pivot.py
@@ -454,9 +454,6 @@ def crosstab(index, columns, values=None, rownames=None, colnames=None,
 
     from pandas import DataFrame
     df = DataFrame(data, index=common_idx)
-    if not len(df):
-        return DataFrame(index=common_idx)
-
     if values is None:
         df['__dummy__'] = 0
         kwargs = {'aggfunc': len, 'fill_value': 0}


Test patch:
diff --git a/pandas/tests/frame/test_indexing.py b/pandas/tests/frame/test_indexing.py
--- a/pandas/tests/frame/test_indexing.py
+++ b/pandas/tests/frame/test_indexing.py
@@ -722,9 +722,11 @@ def test_setitem_empty_frame_with_boolean(self):
                 assert_frame_equal(df, df2)
 
     def test_setitem_scalars_no_index(self):
-        # GH16823
+        # GH16823 / 17894
         df = DataFrame()
-        pytest.raises(ValueError, df.__setitem__, 'foo', 1)
+        df['foo'] = 1
+        expected = DataFrame(columns=['foo']).astype(np.int64)
+        assert_frame_equal(df, expected)
 
     def test_getitem_empty_frame_with_boolean(self):
         # Test for issue #11859
diff --git a/pandas/tests/indexing/test_loc.py b/pandas/tests/indexing/test_loc.py
--- a/pandas/tests/indexing/test_loc.py
+++ b/pandas/tests/indexing/test_loc.py
@@ -423,14 +423,15 @@ def test_loc_setitem_consistency(self):
 
     def test_loc_setitem_consistency_empty(self):
         # empty (essentially noops)
-        # GH16823
+        expected = DataFrame(columns=['x', 'y'])
+        expected['x'] = expected['x'].astype(np.int64)
         df = DataFrame(columns=['x', 'y'])
-        with tm.assert_raises_regex(ValueError, 'If using all scalar values'):
-            df.loc[:, 'x'] = 1
+        df.loc[:, 'x'] = 1
+        tm.assert_frame_equal(df, expected)
 
         df = DataFrame(columns=['x', 'y'])
-        with tm.assert_raises_regex(ValueError, 'If using all scalar values'):
-            df['x'] = 1
+        df['x'] = 1
+        tm.assert_frame_equal(df, expected)
 
     def test_loc_setitem_consistency_slice_column_len(self):
         # .loc[:,column] setting with slice == len of the column
diff --git a/pandas/tests/indexing/test_partial.py b/pandas/tests/indexing/test_partial.py
--- a/pandas/tests/indexing/test_partial.py
+++ b/pandas/tests/indexing/test_partial.py
@@ -575,16 +575,24 @@ def f():
     def test_partial_set_empty_frame_row(self):
         # GH5720, GH5744
         # don't create rows when empty
+        expected = DataFrame(columns=['A', 'B', 'New'],
+                             index=pd.Index([], dtype='int64'))
+        expected['A'] = expected['A'].astype('int64')
+        expected['B'] = expected['B'].astype('float64')
+        expected['New'] = expected['New'].astype('float64')
+
         df = DataFrame({"A": [1, 2, 3], "B": [1.2, 4.2, 5.2]})
         y = df[df.A > 5]
-        # GH16823
-        # Setting a column with a scalar and no index should raise
-        with tm.assert_raises_regex(ValueError, 'If using all scalar values'):
-            y['New'] = np.nan
+        y['New'] = np.nan
+        tm.assert_frame_equal(y, expected)
+        # tm.assert_frame_equal(y,expected)
 
+        expected = DataFrame(columns=['a', 'b', 'c c', 'd'])
+        expected['d'] = expected['d'].astype('int64')
         df = DataFrame(columns=['a', 'b', 'c c'])
-        with tm.assert_raises_regex(ValueError, 'If using all scalar values'):
-            df['d'] = 3
+        df['d'] = 3
+        tm.assert_frame_equal(df, expected)
+        tm.assert_series_equal(df['c c'], Series(name='c c', dtype=object))
 
         # reindex columns is ok
         df = DataFrame({"A": [1, 2, 3], "B": [1.2, 4.2, 5.2]})
diff --git a/pandas/tests/reshape/test_pivot.py b/pandas/tests/reshape/test_pivot.py
--- a/pandas/tests/reshape/test_pivot.py
+++ b/pandas/tests/reshape/test_pivot.py
@@ -1228,8 +1228,7 @@ def test_crosstab_no_overlap(self):
         s2 = pd.Series([4, 5, 6], index=[4, 5, 6])
 
         actual = crosstab(s1, s2)
-        expected = pd.DataFrame(
-            index=pd.Index([], dtype='int64')).astype('int64')
+        expected = pd.DataFrame()
 
         tm.assert_frame_equal(actual, expected)