Repository: dask/dask

Issue:
dd.merge with npartitions='auto' generates TypeError
When I was trying to figure out how to resolve ``MemoryError`` issue #5159 caused by uneven partitions after merging two dataframes, ``dd.merge( npartitions='auto')`` came on my radar. 

I just have found that when I try to merge with ``npartitons='auto'`` it generates an TypeError message. Here's a reproducible example.

```
>>> quotes
                    time0 time ticker     bid     ask
0 2016-05-25 13:30:00.023   GOOG  720.50  720.93
1 2016-05-25 13:30:00.023   MSFT   51.95   51.96
2 2016-05-25 13:30:00.030   MSFT   51.97   51.98
3 2016-05-25 13:30:00.041   MSFT   51.99   52.00
4 2016-05-25 13:30:00.048   GOOG  720.50  720.93
5 2016-05-25 13:30:00.049   AAPL   97.99   98.01
6 2016-05-25 13:30:00.072   GOOG  720.50  720.88
7 2016-05-25 13:30:00.075   MSFT   52.01   52.03

quotes = pd.read_clipboard()
quotes['time'] = pd.to_datetime(quotes.time0 +' ' + quotes.time)
quotes.drop(columns='time0',inplace=True)

>>> trades
                    time0 time ticker   price  quantity
0 2016-05-25 13:30:00.023   MSFT   51.95        75
1 2016-05-25 13:30:00.038   MSFT   51.95       155
2 2016-05-25 13:30:00.048   GOOG  720.77       100
3 2016-05-25 13:30:00.048   GOOG  720.92       100
4 2016-05-25 13:30:00.048   AAPL   98.00       100

trades = pd.read_clipboard()
trades['time'] = pd.to_datetime(trades.time0 + ' ' + trades.time)
trades.drop(columns='time0', inplace=True)



dd_quotes = dd.from_pandas(quotes,npartitions=4)
dd_trades = dd.from_pandas(trades,npartitions=2)


dd.merge(dd_quotes,dd_trades,on='ticker',npartitions='auto')

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\gunsu.son\AppData\Local\Continuum\miniconda3\lib\site-packages\dask\dataframe\multi.py", line 563, in merge
    indicator=indicator,
  File "C:\Users\gunsu.son\AppData\Local\Continuum\miniconda3\lib\site-packages\dask\dataframe\multi.py", line 279, in hash_join
    lhs2 = shuffle_func(lhs, left_on, npartitions=npartitions, shuffle=shuffle)
  File "C:\Users\gunsu.son\AppData\Local\Continuum\miniconda3\lib\site-packages\dask\dataframe\shuffle.py", line 250, in shuffle
    compute=compute,
  File "C:\Users\gunsu.son\AppData\Local\Continuum\miniconda3\lib\site-packages\dask\dataframe\shuffle.py", line 281, in rearrange_by_column
    return rearrange_by_column_disk(df, col, npartitions, compute=compute)
  File "C:\Users\gunsu.son\AppData\Local\Continuum\miniconda3\lib\site-packages\dask\dataframe\shuffle.py", line 359, in rearrange_by_column_disk
    (name, i): (collect, p, i, df._meta, barrier_token) for i in range(npartitions)
TypeError: 'str' object cannot be interpreted as an integer
```


Discussion:
Can you provide an example that doesn't rely on read_clipboard? It'll be easier to turn into a unit test.

I wonder the docstring for `npartitions` allowing 'auto' was updated without updating the implementation.

I'm not sure where best the `npartitions='auto'` should be resolved to an integer. Are you interested in exploring @Matthew-gunsuSon? 

Resolving patch:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -3735,10 +3735,11 @@ def merge(
             whose merge key only appears in `left` DataFrame, "right_only" for
             observations whose merge key only appears in `right` DataFrame,
             and "both" if the observation’s merge key is found in both.
-        npartitions: int, None, or 'auto'
+        npartitions: int or None, optional
             The ideal number of output partitions. This is only utilised when
-            performing a hash_join (merging on columns only). If `None`
-            npartitions = max(lhs.npartitions, rhs.npartitions)
+            performing a hash_join (merging on columns only). If ``None`` then
+            ``npartitions = max(lhs.npartitions, rhs.npartitions)``.
+            Default is ``None``.
         shuffle: {'disk', 'tasks'}, optional
             Either ``'disk'`` for single-node operation or ``'tasks'`` for
             distributed operation.  Will be inferred by your current scheduler.


Test patch: