DEPR: Deprecate sort=None for union and implement sort=True
State: closed

- [ ] progress towards #24471 
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This is a WIP pull request because I wanted to make sure I'm going in the right direction before continuing.

In #24959 it was agreed that the default for some of the set operations should be ``sort=None`` for backwards compatibility in 0.24.1. ``sort=None`` generally tries to sort in all cases _except_ when the two indices are equal or one of the indices is empty. This meant that in a future release we could introduce a proper ``sort=True`` which would try to sort in all cases and change the default to ``sort=False`` in all cases.

Are there any potential problems with this plan? 

In this PR I have started to deprecate ``sort=None`` for ``Index.union`` and to introduce ``sort=True``. ``sort=None`` is currently the default for this set operation so a ``FutureWarning`` will be given by default. I will make similar changes to the ``union`` method in all the index classes if this is the approach we want to take.

Labels:
name: API Design

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=h1) Report
> Merging [#25980](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0610a60d06c190cdd597480962f98b97de7b4286?src=pr&el=desc) will **decrease** coverage by `49.91%`.
> The diff coverage is `61.53%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25980/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #25980       +/-   ##
===========================================
- Coverage   91.82%   41.91%   -49.92%     
===========================================
  Files         175      175               
  Lines       52540    52562       +22     
===========================================
- Hits        48247    22030    -26217     
- Misses       4293    30532    +26239
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `41.91% <61.53%> (+1.04%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `55.75% <61.53%> (-41.05%)` | :arrow_down: |
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/groupby/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9jYXRlZ29yaWNhbC5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `0% <0%> (-99.37%)` | :arrow_down: |
| [pandas/io/sas/sas7bdat.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXM3YmRhdC5weQ==) | `0% <0%> (-91.16%)` | :arrow_down: |
| [pandas/io/sas/sas\_xport.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfeHBvcnQucHk=) | `0% <0%> (-90.1%)` | :arrow_down: |
| [pandas/core/tools/numeric.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvbnVtZXJpYy5weQ==) | `10.44% <0%> (-89.56%)` | :arrow_down: |
| ... and [134 more](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=footer). Last update [0610a60...923b4d2](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=h1) Report
> Merging [#25980](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/b115a6bf5553445fdf29824623ea2b7c3a424660?src=pr&el=desc) will **decrease** coverage by `50.06%`.
> The diff coverage is `61.53%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25980/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #25980       +/-   ##
===========================================
- Coverage   91.87%   41.81%   -50.07%     
===========================================
  Files         180      174        -6     
  Lines       50859    50711      -148     
===========================================
- Hits        46726    21204    -25522     
- Misses       4133    29507    +25374
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `41.81% <61.53%> (-0.18%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `55.65% <61.53%> (-41.6%)` | :arrow_down: |
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/gcs.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2djcy5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/groupby/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9jYXRlZ29yaWNhbC5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/s3.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3MzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `0% <0%> (-99.37%)` | :arrow_down: |
| [pandas/io/sas/sas7bdat.py](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXM3YmRhdC5weQ==) | `0% <0%> (-91.16%)` | :arrow_down: |
| ... and [152 more](https://codecov.io/gh/pandas-dev/pandas/pull/25980/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=footer). Last update [b115a6b...2286cd1](https://codecov.io/gh/pandas-dev/pandas/pull/25980?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thanks for the PR @reidy-p ! Looks like this has gone stale though, so closing to clean up our queue. Please ping if you'd like to pick it back up