Repository: dask/dask

Issue:
reset_index resets the index to the default per partition
Reading the documentation, I was under the impression that `reset_index` will reset the index of the dataframe, but the following example shows that's not the case. I believe that the index gets reset per partition.

```python
import dask.dataframe as dd
import pandas as pd

pdf = pd.DataFrame({'name': ['a', 'a', 'a', 'b', 'b', 'b', 'c'], 'value': [1, 3, 2, 6, 8, 7, 9]})
ddf = dd.from_pandas(pdf, npartitions=3)

ddf[ddf['value'] <= 7].reset_index(drop=True).index.compute()
```

Outputs:
``` Int64Index([0, 1, 2, 0, 1], dtype='int64')```

Also, why is the number of partitions 2 in the above example? 
```
In [2]: ddf
Out[2]: dd.DataFrame<from_pa..., npartitions=2, divisions=(0, 3, 6)>
```




Discussion:
That's correct.  We don't know how many rows are in the dask.dataframe without reading through all of your data, so it is difficult to construct an accurate range index.

The logic in `from_pandas(npartitions=...)` doesn't guarantee an exact number of partitions.  However in your case I don't see why it shouldn't perform as expected.  If you felt like looking into the source code here and submitting a PR that would be most welcome.  The logic in `dask/dataframe/io/io.py::from_pandas` should be accessible.

Thank you for the fast reply! That makes sense.

As for the number of partitions, it looks like the number of partitions is one less in cases when the divisions would include the same value twice. Adding it again works, but it looks weird. 
`dd.DataFrame<from_pa..., npartitions=3, divisions=(0, 3, 6, 6)>`
Going by the documentation:

> Divisions includes the minimum value of every partition’s index and the maximum value of the last partition’s index.

it looks like that would would be expected. However, if there is no guarantee on the exact number of partitions, it might be better to just leave it as is since it's less confusing.


Resolving patch:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -300,8 +300,26 @@ def index(self):
         return Index(merge(dsk, self.dask), name,
                      self._meta.index, self.divisions)
 
-    @derived_from(pd.DataFrame)
     def reset_index(self, drop=False):
+        """Reset the index to the default index.
+
+        Note that unlike in ``pandas``, the reset ``dask.dataframe`` index will
+        not be monotonically increasing from 0. Instead, it will restart at 0
+        for each partition (e.g. ``index1 = [0, ..., 10], index2 = [0, ...]``).
+        This is due to the inability to statically know the full length of the
+        index.
+
+        For DataFrame with multi-level index, returns a new DataFrame with
+        labeling information in the columns under the index names, defaulting
+        to 'level_0', 'level_1', etc. if any are None. For a standard index,
+        the index name will be used (if set), otherwise a default 'index' or
+        'level_0' (if 'index' is already taken) will be used.
+
+        Parameters
+        ----------
+        drop : boolean, default False
+            Do not try to insert index into dataframe columns.
+        """
         return self.map_partitions(M.reset_index, drop=drop).clear_divisions()
 
     @property
@@ -1745,10 +1763,6 @@ def dt(self):
         from .accessor import DatetimeAccessor
         return DatetimeAccessor(self)
 
-    @derived_from(pd.Series)
-    def reset_index(self, drop=False):
-        return super(Series, self).reset_index(drop=drop)
-
     @cache_readonly
     def cat(self):
         from .accessor import CategoricalAccessor
diff --git a/dask/dataframe/io/io.py b/dask/dataframe/io/io.py
--- a/dask/dataframe/io/io.py
+++ b/dask/dataframe/io/io.py
@@ -114,7 +114,9 @@ def from_pandas(data, npartitions=None, chunksize=None, sort=True, name=None):
     df : pandas.DataFrame or pandas.Series
         The DataFrame/Series with which to construct a dask DataFrame/Series
     npartitions : int, optional
-        The number of partitions of the index to create.
+        The number of partitions of the index to create. Note that depending on
+        the size and index of the dataframe, the output may have fewer
+        partitions than requested.
     chunksize : int, optional
         The size of the partitions of the index.
     sort: bool


Test patch: