BUG: creating Categorical from pandas Index/Series with "object" dtype infers string
State: open

- [x] closes #61778
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Preserve [object](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) dtype for categories when constructing [Categorical](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) from pandas objects

This PR fixes an inconsistency in how pandas infers the dtype of categories when constructing a [Categorical](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) from different input types:

When constructing a [Categorical](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) from a pandas [Series](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) or [Index](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) with [dtype="object"](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1), the categories' dtype is now preserved as [object](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1).
When constructing from a NumPy array with [dtype="object"](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) or a raw Python sequence, pandas continues to infer the most specific dtype for the categories (e.g., [str](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) if all elements are strings).
This change brings the behavior of [Categorical](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) in line with how [Series](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) and [Index](https://github.com/pandas-dev/pandas/compare/main...niruta25:pandas:niruta_issue61778?expand=1) handle dtype preservation, making the API more consistent and predictable.

Example

```
pd.options.future.infer_string = True

ser = pd.Series(["foo", "bar", "baz"], dtype="object")
idx = pd.Index(["foo", "bar", "baz"], dtype="object")
arr = np.array(["foo", "bar", "baz"], dtype="object")
pylist = ["foo", "bar", "baz"]

cat_from_ser = pd.Categorical(ser)
cat_from_idx = pd.Categorical(idx)
cat_from_arr = pd.Categorical(arr)
cat_from_list = pd.Categorical(pylist)

# Series/Index with object dtype: preserve object dtype
assert cat_from_ser.categories.dtype == "object"
assert cat_from_idx.categories.dtype == "object"

# Numpy array or list: infer string dtype
assert cat_from_arr.categories.dtype == "str"
assert cat_from_list.categories.dtype == "str"
```

Documentation and release notes have been updated.
Closes: https://github.com/pandas-dev/pandas/issues/61778

Labels:

Comments:
@jbrockmendel Regarding this [bug](https://github.com/pandas-dev/pandas/issues/61778), the change to always preserve object dtype for categories when constructing a Categorical from a pandas Series or Index with dtype="object" is a behavioral change that affects a wide range of pandas internals and user-facing APIs. Hence I am seeing a lot of failures.

I see two ways to resolve without changing overall behavior. 
1. Only Preserve object Dtype When All Elements Are Not Strings

- If the input is a pandas Series/Index with dtype="object", only preserve object dtype for categories if not all elements are strings.
- If all elements are strings, allow inference to str (the current behavior).

2. Add a Keyword Argument to Categorical (e.g., preserve_object_dtype=False)

- Add an explicit option to the Categorical constructor to preserve the object dtype for categories.
- Default to the current behavior, but allow users to opt in to preservation.

Let me know your thoughts.