Repository: dask/dask

Issue:
Dataframe: to_csv doesn't allow "get"-keyword to set the scheduler, but respects global settings
What scheduler is used by to_csv cannot be infulenced with the get-keyword. Accordingly, the API does not mention the get keyword either. However, to_csv seems to respect the global scheduler settings, which can become confusing in the following usecase:

```
#generate test dataframe, containing a variety of data
rowsPerPartition = 5000
npartitions = 500

def pdfGenerator(nrows):
    '''generates pandas dataframe containing dtypes float64, int64 and object (random text)'''
    randomTextGenerator = lambda length: "".join( [random.choice(string.letters + string.digits) for i in xrange(length)] )
    textColumn1 = map(randomTextGenerator, [50]*nrows)
    floatColumn1 = np.random.uniform(-100, 100, nrows)
    intColumn1 = np.random.randint(-100, 100, nrows)
    return pd.DataFrame({'t': textColumn1, 'f': floatColumn1, 'i': intColumn1})

d = dask.delayed(pdfGenerator(rowsPerPartition))
ddf = dd.from_delayed([d]*npartitions)

#save ddf to csv (uses only one core):
path = tempfile.mkdtemp()
%time ddf.to_csv(os.path.join(path, 'a*.csv'))
shutil.rmtree(path)
print('')

#set distributed scheduler as default:
c = distributed.Client()

#same code is much faster now, since it uses the distributed scheduler:
path = tempfile.mkdtemp()
%time ddf.to_csv(os.path.join(path, 'a*.csv'))
shutil.rmtree(path)
print('')

#if I try to use the multiprocessing scheduler in the typical dask way by providing get, 
#it still uses the distributed scheduler:
path = tempfile.mkdtemp()
%time ddf.to_csv(os.path.join(path, 'a*.csv'), get = dask.multiprocessing.get)
shutil.rmtree(path)
print('')
```

So I thought, the get-keyword is simply ignored. But apparently, it is not, since the following throws an error:

```
#providing the distributed scheduler to the get keyword causes an error:
path = tempfile.mkdtemp()
%time ddf.to_csv(os.path.join(path, 'a*.csv'), get = c.get)
shutil.rmtree(path)
```

I might therefore make sense to change the to_csv function in a way, that it respects the get-keyword. Alternatively, it could throw an informative error, whenever the get-keyword is provided. Furthermore, it would be great, if to_csv worked in parallel with the multiprocessing scheduler.



Discussion:

Resolving patch:
diff --git a/dask/dataframe/io/csv.py b/dask/dataframe/io/csv.py
--- a/dask/dataframe/io/csv.py
+++ b/dask/dataframe/io/csv.py
@@ -293,13 +293,13 @@ def _to_csv_chunk(df, **kwargs):
 
 
 def to_csv(df, filename, name_function=None, compression=None, compute=True,
-           **kwargs):
+           get=None, **kwargs):
     values = [_to_csv_chunk(d, **kwargs) for d in df.to_delayed()]
     values = write_bytes(values, filename, name_function, compression,
                          encoding=None)
 
     if compute:
         from dask import compute
-        compute(*values)
+        compute(*values, get=get)
     else:
         return values


Test patch:
diff --git a/dask/dataframe/io/tests/test_csv.py b/dask/dataframe/io/tests/test_csv.py
--- a/dask/dataframe/io/tests/test_csv.py
+++ b/dask/dataframe/io/tests/test_csv.py
@@ -769,3 +769,22 @@ def test_to_csv_series():
         result = dd.read_csv(os.path.join(dir, '*'), header=None,
                              names=['x']).compute()
     assert (result.x == df0).all()
+
+
+def test_to_csv_with_get():
+    from dask.multiprocessing import get as mp_get
+    flag = [False]
+
+    def my_get(*args, **kwargs):
+        flag[0] = True
+        return mp_get(*args, **kwargs)
+
+    df = pd.DataFrame({'x': ['a', 'b', 'c', 'd'],
+                       'y': [1, 2, 3, 4]})
+    ddf = dd.from_pandas(df, npartitions=2)
+
+    with tmpdir() as dn:
+        ddf.to_csv(dn, index=False, get=my_get)
+        assert flag[0]
+        result = dd.read_csv(os.path.join(dn, '*')).compute().reset_index(drop=True)
+        assert_eq(result, df)