Repository: dask/dask

Issue:
Implement Series `__iter__` protocol
This would solve this stack overflow question: 

https://stackoverflow.com/questions/56416326/iterate-through-a-dask-series-getting-unique-values-from-dask-series-to-list/56418050#56418050

Probably this is just a call to one of the `iter` methods already defined.  I don't know exactly what Pandas semantics are though, so this might take a little investigating to see what they do.


Discussion:
@TomAugspurger, any thoughts on this?
The only complex thing `Series.__iter__` does is convert NumPy types to the
Python scalar equivalent.

I imagine that we can chain together a bunch of pandas.Series.__iter__ and
we'll be fine.

On Mon, Jun 3, 2019 at 9:57 AM jakirkham <notifications@github.com> wrote:

> @TomAugspurger <https://github.com/TomAugspurger>, any thoughts on this?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/dask/dask/issues/4871?email_source=notifications&email_token=AAKAOIWT7SG6IPYHVAOK3FLPYUWLZA5CNFSM4HSDI7R2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWZVM4A#issuecomment-498292336>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIREWRRZOYVRUVSH66TPYUWLZANCNFSM4HSDI7RQ>
> .
>

@mrocklin @TomAugspurger:  can this be implemented the same as [iterrows()](https://github.com/dask/dask/blob/master/dask/dataframe/core.py#L3801)?:
```
@derived_from(pd.DataFrame)
def __iter__(self):
    for i in range(self.npartitions):
        df = self.get_partition(i).compute()
        for row in df.__iter__():
            yield row
```
That seems sensible.

On Wed, Jul 3, 2019 at 10:33 PM Blane <notifications@github.com> wrote:

> @mrocklin <https://github.com/mrocklin> @TomAugspurger
> <https://github.com/TomAugspurger>: can this be implemented the same as
> iterrows()
> <https://github.com/dask/dask/blob/master/dask/dataframe/core.py#L3801>?:
>
> @derived_from(pd.DataFrame)
> def __iter__(self):
>     for i in range(self.npartitions):
>         df = self.get_partition(i).compute()
>         for row in df.__iter__():
>             yield row
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/dask/dask/issues/4871?email_source=notifications&email_token=AAKAOITR3HVXLPIF7OLT5ELP5VVPXA5CNFSM4HSDI7R2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZGG3KY#issuecomment-508325291>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIRIAFIMXQQDRB7FYCTP5VVPXANCNFSM4HSDI7RQ>
> .
>


Resolving patch:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -2602,9 +2602,14 @@ def rename(self, index=None, inplace=False, sorted_index=False):
         --------
         pandas.Series.rename
         """
-        from pandas.api.types import is_scalar, is_list_like, is_dict_like
+        from pandas.api.types import is_scalar, is_dict_like, is_list_like
+        import dask.dataframe as dd
 
-        if is_scalar(index) or (is_list_like(index) and not is_dict_like(index)):
+        if is_scalar(index) or (
+            is_list_like(index)
+            and not is_dict_like(index)
+            and not isinstance(index, dd.Series)
+        ):
             res = self if inplace else self.copy()
             res.name = index
         else:
@@ -2683,6 +2688,13 @@ def iteritems(self):
             for item in s.iteritems():
                 yield item
 
+    @derived_from(pd.Series)
+    def __iter__(self):
+        for i in range(self.npartitions):
+            s = self.get_partition(i).compute()
+            for row in s:
+                yield row
+
     @classmethod
     def _validate_axis(cls, axis=0):
         if axis not in (0, "index", None):


Test patch:
diff --git a/dask/dataframe/tests/test_dataframe.py b/dask/dataframe/tests/test_dataframe.py
--- a/dask/dataframe/tests/test_dataframe.py
+++ b/dask/dataframe/tests/test_dataframe.py
@@ -2873,6 +2873,13 @@ def test_series_iteritems():
         assert a == b
 
 
+def test_series_iter():
+    s = pd.DataFrame({"x": [1, 2, 3, 4]})
+    ds = dd.from_pandas(s, npartitions=2)
+    for (a, b) in zip(s["x"], ds["x"]):
+        assert a == b
+
+
 def test_dataframe_iterrows():
     df = pd.DataFrame({"x": [1, 2, 3, 4], "y": [10, 20, 30, 40]})
     ddf = dd.from_pandas(df, npartitions=2)