Repository: pandas-dev/pandas

Issue:
API: allow dependent assignment?
Since kwarg order will be [guaranteed](https://docs.python.org/3.6/whatsnew/3.6.html#pep-468-preserving-keyword-argument-order) in python 3.6, we could allow this...though would only want to try with a 3.6+ version check, otherwise code could more or less randomly work/not work, depending on dict order.

``` python
In [44]: df = pd.DataFrame({'a': [1,2,3]})

In [45]: df.assign(b=1, c=lambda x: x['b'] * 2)
```



Discussion:
xref: https://github.com/pydata/pandas/issues/9777#issuecomment-246043778

This would be a great, but I want to think a bit about exposing a feature that would so easily lead to subtle differences between 3.5 and 3.6. Not sure how best to handle it :/

Either way, we could probably deprecate the sorting of keys (documentation only; can't print a warning each time someone uses `.assign`).

FYI, here's the current docs (in a warning block):

> Since the function signature of assign is **kwargs, a dictionary, the order of the new columns in the resulting DataFrame cannot be guaranteed to match the order you pass in. To make things predictable, items are inserted alphabetically (by key) at the end of the DataFrame.
> All expressions are computed first, and then assigned. So you can't refer to another column being assigned in the same call to assign.

Yeah, the back-compat is awkward, maybe this could be opt-in from an option, so it would at least raise on older pythons, although that's ugly too.
`pd.set_option('ordered_assign', True)  # raises on py < 3.6`

@mrocklin thoughts on this? IIRC you wanted this initially to make things easier for dask? How much would break for you / dask users if we just removed the sorting of keys?
I think that Pandas should make whatever decisions make sense for Pandas.  Dask.dataframe will adapt as necessary.  I suspect that this won't cause a problem.  Dask.dataframe will just try the `assign` operation on an empty dataframe to determine the column ordering.  Whatever Pandas does we'll probably do automatically.
Ultimately, I think this we should switch to this behavior and stop sorting.

But I don't see much of a rush. It might be better to wait until Python 3.6 is more widely used, even until pandas 2.0.
I would also be comfortable just changing this for Python 3.6+. My guess is that there aren't that many folks using Python 3.6 in production yet, so this will probably cause minimal issues.
I think we should sort this out for 0.21. I propose

- No change to 2.7 - 3.5; order is sorted by key
- For 3.6+, we use the original order the user passed in
- We don't allow dependent assignment yet

Maybe in the future, when there's less 3.5 and lower code around, we can allow for dependent assignment.
 +1 on your summary
sgtm on @TomAugspurger summary as well.
HI All,  Tom said this was a good one for new contributors so I'm going after it. 

Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -82,6 +82,7 @@
 from pandas.compat import (range, map, zip, lrange, lmap, lzip, StringIO, u,
                            OrderedDict, raise_with_traceback)
 from pandas import compat
+from pandas.compat import PY36
 from pandas.compat.numpy import function as nv
 from pandas.util._decorators import Appender, Substitution
 from pandas.util._validators import validate_bool_kwarg
@@ -2575,12 +2576,12 @@ def assign(self, **kwargs):
 
         Notes
         -----
-        Since ``kwargs`` is a dictionary, the order of your
-        arguments may not be preserved. To make things predicatable,
-        the columns are inserted in alphabetical order, at the end of
-        your DataFrame. Assigning multiple columns within the same
-        ``assign`` is possible, but you cannot reference other columns
-        created within the same ``assign`` call.
+        For python 3.6 and above, the columns are inserted in the order of
+        **kwargs. For python 3.5 and earlier, since **kwargs is unordered,
+        the columns are inserted in alphabetical order at the end of your
+        DataFrame.  Assigning multiple columns within the same ``assign``
+        is possible, but you cannot reference other columns created within
+        the same ``assign`` call.
 
         Examples
         --------
@@ -2620,14 +2621,18 @@ def assign(self, **kwargs):
         data = self.copy()
 
         # do all calculations first...
-        results = {}
+        results = OrderedDict()
         for k, v in kwargs.items():
             results[k] = com._apply_if_callable(v, data)
 
+        # preserve order for 3.6 and later, but sort by key for 3.5 and earlier
+        if PY36:
+            results = results.items()
+        else:
+            results = sorted(results.items())
         # ... and then assign
-        for k, v in sorted(results.items()):
+        for k, v in results:
             data[k] = v
-
         return data
 
     def _sanitize_column(self, key, value, broadcast=True):


Test patch:
diff --git a/pandas/tests/frame/test_mutate_columns.py b/pandas/tests/frame/test_mutate_columns.py
--- a/pandas/tests/frame/test_mutate_columns.py
+++ b/pandas/tests/frame/test_mutate_columns.py
@@ -4,6 +4,7 @@
 import pytest
 from pandas.compat import range, lrange
 import numpy as np
+from pandas.compat import PY36
 
 from pandas import DataFrame, Series, Index, MultiIndex
 
@@ -61,14 +62,23 @@ def test_assign_multiple(self):
                               [3, 6, 9, 3, 6]], columns=list('ABCDE'))
         assert_frame_equal(result, expected)
 
-    def test_assign_alphabetical(self):
+    def test_assign_order(self):
         # GH 9818
         df = DataFrame([[1, 2], [3, 4]], columns=['A', 'B'])
         result = df.assign(D=df.A + df.B, C=df.A - df.B)
-        expected = DataFrame([[1, 2, -1, 3], [3, 4, -1, 7]],
-                             columns=list('ABCD'))
+
+        if PY36:
+            expected = DataFrame([[1, 2, 3, -1], [3, 4, 7, -1]],
+                                 columns=list('ABDC'))
+        else:
+            expected = DataFrame([[1, 2, -1, 3], [3, 4, -1, 7]],
+                                 columns=list('ABCD'))
         assert_frame_equal(result, expected)
         result = df.assign(C=df.A - df.B, D=df.A + df.B)
+
+        expected = DataFrame([[1, 2, -1, 3], [3, 4, -1, 7]],
+                             columns=list('ABCD'))
+
         assert_frame_equal(result, expected)
 
     def test_assign_bad(self):