DOC Set sort default to no_default, note that it may differ in other libraries
State: closed

This comes out of  Consortium for Python Data API Standards working group meetings - the idea is that there are other DataFrame libraries with a similar API to pandas, and it'd be good to standardise where possible.

For methods which use hash-based methods (e.g. that call `factorize`), then to sort or not to sort can vary across libraries. For example, for `DataFrame.groupby`, in vaex the default is `sort=False`, whilst in pandas it's `sort=True`.

From the minutes of the 31 March 2022 meeting:

> 
> Ashwin: so the change would be to change to the sort argument to None in pandas.
> 
> Jeff: in pandas, we don't allow None as a default value. If not passed, we default to boolean.
> 
> Ashwin: yeah, my proposal is making None the default. Should not change default behavior.
> 
> Jeff: sounds reasonable to me.

Also, from the [pandas standardisation document](https://docs.google.com/document/d/1PIFF0JhPEn9SLK1dagS11lTzPg1DCcuvam7GYE8jK88):

> Decision: make sort=None with a note not to rely on ordering unless sort=True is specified by the user.
> We changed our minds on this; pandas may stay with sort=True, others may recommend sort=False.

It's hard to tell exactly what's required here, so I figured I'd just open a PR with what I think is being asked and we can drive the conversation forwards from here. The docs already mention "Get better performance by turning this off."

Labels:
name: Docs
name: API Design
name: Stale

Comments:
cc @jorisvandenbossche @jreback @shwina @kkraus14  , as this came out of the Data APIs Consortium (though I've not yet attended and so have just been attempting to parse bits together here)
There are two options we could choose from here:

1. Pandas keeps `sort=True`, but the DataFrame API standard defines `sort=None` (or `no_default`). This would technically still make Pandas "compliant" with the API standard. But a _consumer_ of the standard, e.g., an end-user or downstream library, would still specify `sort=True | False` when invoking `groupby` to support different DataFrame API implementations. 

2. Pandas changes its default to `no_default`. This forces more users to think about what behaviour they want, and brings more visibility to the API standard.



I don't think that we want to change the default behavior. We would have to deprecate this which would be really noisy
IIUC, I think changing the default to `sort=None/no_default` but falling back under the hood to `True` is mainly to align cosmetically with the Data API standard.

I think changing the default `sort` behavior (especially for `groupby`) from `True` to `False` is a "hard" behavior change but has interest from Dask for a better default performance behavior cc @ian-r-rose  @rjzamora  
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
not seeing much uptake on this one, closing for now and we can rediscuss later

the default can change in the API-standard-compliant namespace, let's keep the default in pandas reflective of what pandas actually does