Repository: dask/dask

Issue:
Documentation for 'keep' argument to 'drop_duplicates' on dask dataframe and series
The documentation for `drop_duplicates` [for dask dataframes](https://docs.dask.org/en/stable/generated/dask.dataframe.DataFrame.drop_duplicates.html) and [for dask series](https://docs.dask.org/en/latest/generated/dask.dataframe.Series.drop_duplicates.html) says "(Not supported in Dask)" for the `keep` argument, but actually it seems to be implemented and seems to work fine when I've used it. It looks like it is just `keep=False` that is not impemented; `keep='first'` and `keep='last'` are working.

It would be good if either the docs reflected this or, if you're still not supposed to use `keep` because it might be buggy, then a warning/error when you do would be good.


Documentation for 'keep' argument to 'drop_duplicates' on dask dataframe and series
The documentation for `drop_duplicates` [for dask dataframes](https://docs.dask.org/en/stable/generated/dask.dataframe.DataFrame.drop_duplicates.html) and [for dask series](https://docs.dask.org/en/latest/generated/dask.dataframe.Series.drop_duplicates.html) says "(Not supported in Dask)" for the `keep` argument, but actually it seems to be implemented and seems to work fine when I've used it. It looks like it is just `keep=False` that is not impemented; `keep='first'` and `keep='last'` are working.

It would be good if either the docs reflected this or, if you're still not supposed to use `keep` because it might be buggy, then a warning/error when you do would be good.




Discussion:
@billiejoe-bw Thanks for reporting!

I agree we can improve the docs, `@derived_from`'s new `inconsistencies` keyword argument can be useful here. :)

The docstrings for many Dask DataFrame/Array functions are generated from pandas/numpy docs. The arguments that are in the pandas function docstring but not in the Dask DataFrame function's signature, are automatically marked as "Not supported in Dask".
However, there are many Dask DataFrame functions that can still pass these as keyword arguments to pandas. It might be worth updating the utility to not show the disclaimer if the function has `**kwargs` in its signature. 
@billiejoe-bw Thanks for reporting!

I agree we can improve the docs, `@derived_from`'s new `inconsistencies` keyword argument can be useful here. :)

The docstrings for many Dask DataFrame/Array functions are generated from pandas/numpy docs. The arguments that are in the pandas function docstring but not in the Dask DataFrame function's signature, are automatically marked as "Not supported in Dask".
However, there are many Dask DataFrame functions that can still pass these as keyword arguments to pandas. It might be worth updating the utility to not show the disclaimer if the function has `**kwargs` in its signature. 

Resolving patch:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -684,7 +684,10 @@ def get_partition(self, n):
             msg = f"n must be 0 <= n < {self.npartitions}"
             raise ValueError(msg)
 
-    @derived_from(pd.DataFrame)
+    @derived_from(
+        pd.DataFrame,
+        inconsistencies="keep=False will raise a ``NotImplementedError``",
+    )
     def drop_duplicates(
         self, subset=None, split_every=None, split_out=1, ignore_index=False, **kwargs
     ):


Test patch: