Repository: pandas-dev/pandas (Python)

Issue:
BUG: Inconsistent implicit sorting when doing pd.concat([pd.Series(1, Index(dtype='object'), ...], axis=1)
### Pandas version checks

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [ ] I have confirmed this bug exists on the [main branch](https://pandas.pydata.org/docs/dev/getting_started/install.html#installing-the-development-version-of-pandas) of pandas.


### Reproducible Example

```python
# some methods to transform the example DatetimeIndex objects into other index types
def identity(index: pd.DatetimeIndex) -> pd.DatetimeIndex:
    return index
def to_int64_index(index: pd.DatetimeIndex) -> pd.Int64Index:
    return index.astype(int)
def to_date_object(index: pd.DatetimeIndex) -> pd.Index:
    return pd.Index(index.date, dtype=object)
def to_int_object(index: pd.DatetimeIndex) -> pd.Index:
    return pd.Index(index.astype(int), dtype=object)
def to_timestamp_object(index: pd.DatetimeIndex) -> pd.Index:
    i = pd.Index(index, dtype=object)
    assert i.dtype.str == '|O'
    return i

@pytest.mark.parametrize('transform1,transform2', [
    (to_date_object, to_date_object), # an example of the issue, so test case will fail
    (to_int_object, to_int_object), # an example of the issue, so test case will fail
    (to_date_object, identity),
    (identity, to_date_object),
    (to_timestamp_object, to_timestamp_object),
    (identity, identity),  # i.e. a DatetimeIndex will be used
    (to_int64_index, to_int64_index),
])
def test_issue(transform1, transform2):
    """Concatenating objects with object indices has inconsistent default sorting behaviour when compared to other
    index types.

    An example inconsistent/non-monotonic result is below which happens when using and object index:
                      0  1
        2000-01-01  0.0  0
        2001-01-01  1.0  1
        1998-01-01  NaN -2
        1999-01-01  NaN -1
    """
    # index1 is a subset
    index1 = transform1(pd.to_datetime(['2000', '2001']))
    index2 = transform2(pd.to_datetime(['1998', '1999', '2000', '2001']))
    series1 = pd.Series([0, 1], index1)
    series2 = pd.Series([-2, -1, 0, 1], index2)
    result = pd.concat([series1, series2], axis=1)
    assert result.index.is_monotonic_increasing
```


### Issue Description

`pd.concat([series1, series2], axis=1)` does not implicitly sort the index when the indices do not match when the index is `pd.Index(dtype='object')`, which is a behavioral change that came in somewhere between 0.19.2 and 1.1.5, and is inconsistent with other index types.

This was an unpleasant surprise found in code when concatenating two series with `datetime.date` indices (like in the first failing test case) after upgrading a codebase.

A simpler way to reproduce:
```python
print(
    pd.concat([
        pd.Series([0, 1], pd.Index(pd.to_datetime(['2000', '2001']).date, dtype=object)),
        pd.Series([-2, -1, 0, 1], pd.Index(pd.to_datetime(['1998', '1999', '2000', '2001']).date, dtype=object)),
    ], axis=1)
)
```
```
              0  1
2000-01-01  0.0  0
2001-01-01  1.0  1
1998-01-01  NaN -2
1999-01-01  NaN -1
```

### Expected Behavior

I was expecting the frame resulting frame to be sorted, as is the case for other index types, and as was the case in at least 0.19.2 but has not been the case since at least 1.1.5.

If I had to guess a rationale for the change between versions, it might be due to assuming that objects aren't necessarily sortable, and the code doesn't even attempt it.

### Installed Versions

<details>

INSTALLED VERSIONS
------------------
commit           : 2e218d10984e9919f0296931d92ea851c6a6faf5
python           : 3.8.16.final.0
python-bits      : 64
OS               : Linux
OS-release       : 4.15.0-202-generic
Version          : #213-Ubuntu SMP Thu Jan 5 19:19:12 UTC 2023
machine          : x86_64
processor        :
byteorder        : little
LC_ALL           : None
LANG             : C.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.5.3
numpy            : 1.24.2
pytz             : 2022.7.1
dateutil         : 2.8.2
setuptools       : 57.5.0
pip              : 22.0.4
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
brotli           : None
fastparquet      : None
fsspec           : None
gcsfs            : None
matplotlib       : None
numba            : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pyreadstat       : None
pyxlsb           : None
s3fs             : None
scipy            : None
snappy           : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
zstandard        : None
tzdata           : None

</details>

BUG: concat(axis=1) sorts by default for DatetimeIndex
### Pandas version checks

- [X] I have checked that this issue has not already been reported.

- [X] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas.

- [X] I have confirmed this bug exists on the [main branch](https://pandas.pydata.org/docs/dev/getting_started/install.html#installing-the-development-version-of-pandas) of pandas.


### Reproducible Example

```python
import pandas as pd

a = pd.Series(1, index=pd.to_datetime(['2000-01-01', '2000-01-03']))
b = pd.Series(2, index=pd.to_datetime(['2000-01-02', '2000-01-04']))

pd.concat([a, b], axis=1)
```



### Issue Description

`concat` documentation states concat will not sort by default. I believe the default sort behavior was deprecated some time ago. However, there is still a `sort=True` behavior for `DatetimeIndex`:

```
              0    1
2000-01-01  1.0  NaN
2000-01-02  NaN  2.0
2000-01-03  1.0  NaN
2000-01-04  NaN  2.0
```

The source of this behavior is a hard-coded special-casing in `pandas.core.indexes.api.union_indexes`. Making this not sort by default seems to break at least one test (`test_concat_datetime_timezone`) and possibly more. 


### Expected Behavior

Per the docs, I would expect no sort by default. But that might surprise users if changed. I suppose this either needs to be documented (if intentional) or deprecated (if not intentional).

### Installed Versions

.


Discussion:
Hi, thanks for your report. Any reason you are not setting sort=True? This seems to work.

Edit: I'd rather expect it to never sort. The docs also say that this was changed in 1.0.0
looks like sorting for dtis was added intentionally in a6eb92ba1f43ec8bfa8922593ddee85830cc42b3

cc @jbrockmendel I think we can keep the passed value for sort in the 
```
if len(dtis) == len(indexes):
```
case. This breaks one tests but reordering the result fixes it

Hi I am a first time contributor would like to know if I shall update the docs or shall make changes to sort the bug

> Hi I am a first time contributor would like to know if I shall update the docs or shall make changes to sort the bug
> 
> 

Hey @kshitiz305, I think this needs a bit more discussion before determining which path to take.
@phofl - just noticed you had opened https://github.com/pandas-dev/pandas/pull/51683 for this issue. Any particular reason for closing it?
That kind of fell off my radar back then. You can continue if you are interested, the fix itself should be good



Patch:
diff --git a/doc/source/whatsnew/v2.2.0.rst b/doc/source/whatsnew/v2.2.0.rst
index a795514aa31f8..609f99e26cf3b 100644
--- a/doc/source/whatsnew/v2.2.0.rst
+++ b/doc/source/whatsnew/v2.2.0.rst
@@ -247,8 +247,8 @@ Groupby/resample/rolling
 
 Reshaping
 ^^^^^^^^^
+- Bug in :func:`concat` ignoring ``sort`` parameter when passed :class:`DatetimeIndex` indexes (:issue:`54769`)
 - Bug in :func:`merge` returning columns in incorrect order when left and/or right is empty (:issue:`51929`)
--
 
 Sparse
 ^^^^^^
diff --git a/pandas/core/indexes/api.py b/pandas/core/indexes/api.py
index a8ef0e034ba9b..6a36021d9e7c5 100644
--- a/pandas/core/indexes/api.py
+++ b/pandas/core/indexes/api.py
@@ -288,7 +288,6 @@ def _find_common_index_dtype(inds):
             raise TypeError("Cannot join tz-naive with tz-aware DatetimeIndex")
 
         if len(dtis) == len(indexes):
-            sort = True
             result = indexes[0]
 
         elif len(dtis) > 1:


Test patch:
diff --git a/pandas/tests/reshape/concat/test_datetimes.py b/pandas/tests/reshape/concat/test_datetimes.py
index 2f50a19189987..12d28c388d508 100644
--- a/pandas/tests/reshape/concat/test_datetimes.py
+++ b/pandas/tests/reshape/concat/test_datetimes.py
@@ -77,23 +77,23 @@ def test_concat_datetime_timezone(self):
 
         exp_idx = DatetimeIndex(
             [
-                "2010-12-31 15:00:00+00:00",
-                "2010-12-31 16:00:00+00:00",
-                "2010-12-31 17:00:00+00:00",
                 "2010-12-31 23:00:00+00:00",
                 "2011-01-01 00:00:00+00:00",
                 "2011-01-01 01:00:00+00:00",
+                "2010-12-31 15:00:00+00:00",
+                "2010-12-31 16:00:00+00:00",
+                "2010-12-31 17:00:00+00:00",
             ]
         )
 
         expected = DataFrame(
             [
-                [np.nan, 1],
-                [np.nan, 2],
-                [np.nan, 3],
                 [1, np.nan],
                 [2, np.nan],
                 [3, np.nan],
+                [np.nan, 1],
+                [np.nan, 2],
+                [np.nan, 3],
             ],
             index=exp_idx,
             columns=["a", "b"],