Repository: dask/dask-ml

Issue:
GridsearchCV with prescattered data
During a recent debug session with @TomAugspurger we thought that dask-ml could take advantage of data which was pre-scattered:

> client.scatter(data, broadcast=True)

Current understanding is that dask-ml will not take advantage of pre-scatter data as the call to `KFold` will chop up data and not see the cached data on the workers.

Discussion:
TomAugspurger: Thanks. I think you're missing a `.utils` in your test.

I think what you have (directly testing to_keys against client.scatter) is about as good as we could do. I never came up with an easy way to test it using GridSearchCV itself.

w.r.t. using `type(x).__name__`, that might be a bit better. We *might* be able to get a pandas DataFrame here.
lesteve: Done. I also test pandas Dataframe now.

Patch:
diff --git a/dask_ml/model_selection/utils.py b/dask_ml/model_selection/utils.py
index eab9c80f..4972cffd 100644
--- a/dask_ml/model_selection/utils.py
+++ b/dask_ml/model_selection/utils.py
@@ -85,7 +85,7 @@ def to_keys(dsk, *args):
             yield x.key
         else:
             assert not is_dask_collection(x)
-            key = "array-" + tokenize(x)
+            key = type(x).__name__ + "-" + tokenize(x)
             dsk[key] = x
             yield key
 


Test patch:
diff --git a/tests/test_model_selection.py b/tests/test_model_selection.py
index 09a5da87..4afecad4 100644
--- a/tests/test_model_selection.py
+++ b/tests/test_model_selection.py
@@ -1,6 +1,10 @@
+import numpy as np
 from scipy import stats
+import pandas as pd
 from sklearn.svm import SVC
 
+from dask.distributed import Client
+
 import dask_ml.model_selection as dms
 
 
@@ -14,3 +18,14 @@ def test_search_basic(xy_classification):
     param_dist = {"C": stats.uniform}
     b = dms.RandomizedSearchCV(SVC(kernel="rbf", gamma=0.1), param_dist)
     b.fit(X, y)
+
+
+def test_to_keys_numpy_array():
+    rng = np.random.RandomState(0)
+    arr = rng.randn(20, 30)
+    df = pd.DataFrame(data=arr)
+    dsk = {}
+    grid_search_keys = list(dms.utils.to_keys(dsk, arr, df))
+    with Client() as client:
+        data_futures = client.scatter([arr, df])
+    assert grid_search_keys == [f.key for f in data_futures]