Repository: dask/dask

Issue:
Series metadata inference fails on comparison with dask Scalar
``` python
import dask.dataframe as dd
import pandas as pd

df = pd.DataFrame({"x": range(10)})
a = dd.from_pandas(df, npartitions=2)

a.x > a.x.quantile(.5)
```

raises

``` pytb
$ python cmp.py
Traceback (most recent call last):
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/utils.py", line 146, in raise_on_meta_error
    yield
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/core.py", line 2616, in _emulate
    return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/pandas/core/ops.py", line 812, in wrapper
    raise ValueError(msg)
ValueError: Can only compare identically-labeled Series objects

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "cmp.py", line 7, in <module>
    a.x > a.x.quantile(.5)
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/core.py", line 1030, in <lambda>
    return lambda self, other: elemwise(op, self, other)
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/core.py", line 2459, in elemwise
    meta = _emulate(op, *args, **kwargs)
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/core.py", line 2616, in _emulate
    return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
  File "/usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/utils.py", line 159, in raise_on_meta_error
    raise ValueError(msg)
ValueError: Metadata inference failed in `gt`.

Original error is below:
------------------------
ValueError('Can only compare identically-labeled Series objects',)

Traceback:
---------
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/utils.py", line 146, in raise_on_meta_error
    yield
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/dask/dask/dataframe/core.py", line 2616, in _emulate
    return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
  File "/Users/tom.augspurger/Envs/dask/lib/python3.5/site-packages/pandas/core/ops.py", line 812, in wrapper
    raise ValueError(msg)
```

That's with latest dask and pandas==0.19.0; With pandas=0.18.1, I get a similar message, the difference being

``` pytb
Original error is below:
------------------------
ValueError('Series lengths must match to compare',)
```

Haven't had a chance to dig into where the inference is failing :/
Just enough to see that scalar need to be a dask `Scalar`; calling `.quantile(.5).compute()` would cause that line to work.



Discussion:
Thanks for the bug report. This is an issue in the metadata inference for `quantile` - the `_meta` attribute is a series instead of a scalar as it should be. Fixing. Note that comparison with scalars does work:

``` python
In [7]: a.x < a.x.sum()
Out[7]: dd.Series<lt-b3d9..., npartitions=2, divisions=(0, 5, 9)>
```


Resolving patch:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -102,7 +102,11 @@ def __init__(self, dsk, name, meta, divisions=None):
         # objects.
         self.dask = dsk
         self._name = name
-        self._meta = make_meta(meta)
+        meta = make_meta(meta)
+        if isinstance(meta, (pd.DataFrame, pd.Series, pd.Index)):
+            raise ValueError("Expected meta to specify scalar, got "
+                             "{0}".format(type(meta).__name__))
+        self._meta = meta
 
     @property
     def _meta_nonempty(self):
@@ -2760,7 +2764,12 @@ def quantile(df, q):
     from dask.array.percentile import _percentile, merge_percentiles
 
     # currently, only Series has quantile method
-    if isinstance(q, (list, tuple, np.ndarray)):
+    if isinstance(df, Index):
+        meta = pd.Series(df._meta_nonempty).quantile(q)
+    else:
+        meta = df._meta_nonempty.quantile(q)
+
+    if isinstance(meta, pd.Series):
         # Index.quantile(list-like) must be pd.Series, not pd.Index
         df_name = df.name
         finalize_tsk = lambda tsk: (pd.Series, tsk, q, None, df_name)
@@ -2770,11 +2779,6 @@ def quantile(df, q):
         return_type = Scalar
         q = [q]
 
-    if isinstance(df, Index):
-        meta = pd.Series(df._meta_nonempty).quantile(q)
-    else:
-        meta = df._meta_nonempty.quantile(q)
-
     # pandas uses quantile in [0, 1]
     # numpy / everyone else uses [0, 100]
     qs = np.asarray(q) * 100


Test patch: