DEPR: deprecate Index.is_categorical
State: closed

- [ ] progress towards https://github.com/pandas-dev/pandas/issues/50042
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Index
description: Related to the Index class or subclasses

Comments:
@topper-123 Please have a look. Although I am not sure why so many of the checks are failing. Could you tell me if you know?
You have to either 
1. look through the test logs to see what errors the are reporting
2. run the test suite yourself

The second option is preferable, but is a bit more demanding, e.g. you will have to learn how to compile the pandas code and more. However I recommend doing that, as there is valuable knowledge in knowing that. see [here](https://pandas.pydata.org/pandas-docs/stable/development/contributing_codebase.html)

If you fix this by looking through the logs, press "details" on the right of a test and go through the log text to see if you find the issue. I typically do that by pressing "view raw logs" and searching for the text string "FAILED". You can try that. In this case here are some log texts from your PR, see if this helps:

```bash
2022-12-13T09:59:39.0966248Z FAILED pandas/tests/indexes/test_base.py::TestIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0966954Z FAILED pandas/tests/indexes/test_base.py::TestMixedIntIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0967709Z FAILED pandas/tests/indexes/categorical/test_category.py::TestCategoricalIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0968475Z FAILED pandas/tests/indexes/datetimes/test_datetimelike.py::TestDatetimeIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0969201Z FAILED pandas/tests/indexes/interval/test_base.py::TestBase::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0969959Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestFloatNumericIndex::test_is_categorical_is_deprecated[float64] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0970741Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestFloatNumericIndex::test_is_categorical_is_deprecated[float32] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0971498Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestIntNumericIndex::test_is_categorical_is_deprecated[int64] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0972243Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestIntNumericIndex::test_is_categorical_is_deprecated[int32] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0972975Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestIntNumericIndex::test_is_categorical_is_deprecated[int16] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0973710Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestIntNumericIndex::test_is_categorical_is_deprecated[int8] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0974462Z FAILED pandas/tests/indexes/numeric/test_numeric.py::TestUIntNumericIndex::test_is_categorical_is_deprecated[uint64] - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0975245Z FAILED pandas/tests/indexes/period/test_period.py::TestPeriodIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0975963Z FAILED pandas/tests/indexes/ranges/test_range.py::TestRangeIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
2022-12-13T09:59:39.0976709Z FAILED pandas/tests/indexes/timedeltas/test_timedelta.py::TestTimedeltaIndex::test_is_categorical_is_deprecated - AssertionError: Did not see expected warning of class 'FutureWarning'
```

Great.

I do see you have a few unrelated commit among yours. You should get rid of those.
> Great.
> 
> I do see you have a few unrelated commit among yours. You should get rid of those.

@topper-123 Is there a clean method of deleting unrelated commits?

Also, I think all checks are passed but one
> Is there a clean method of deleting unrelated commits?

it looks like they’re gone now, so that’s good.
All the renaming under the “Returns” sections seem unrelated and not needed for this PR, is that correct? If yes, can you get tid of them.
> All the renaming under the “Returns” sections seem unrelated and not needed for this PR, is that correct? If yes, can you get tid of them.

Return section?
In the doc strings, where it starts with

```
Returns
-------
```
> 

I am a bit confused which file/method are you referring to. Did I make any changes in that section as well?
All the instances where you chaange the text below the "Returns" section.

for example in `pandas/core/indexes/base.py` you change:

```
Returns
-------
Index
```

to


```
Returns
-------
union : Index
```

Can you roll back all changes like this.
> All the instances where you chaange the text below the "Returns" section.
> 
> for example in `pandas/core/indexes/base.py` you change:
> 
> ```
> Returns
> -------
> Index
> ```
> 
> to
> 
> ```
> Returns
> -------
> union : Index
> ```
> 
> Can you roll back all changes like this.

Yes, I just found that out through git history, I am on to it
@topper-123 Now there is a test failure

`File "/home/runner/micromamba/envs/test/lib/python3.8/site-packages/requests/models.py", line 10[21](https://github.com/pandas-dev/pandas/actions/runs/3693694966/jobs/6253988365#step:5:22), in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/users/sinhrks`
that is unrelated to your PR, but has to do with the CI. I've restarted it, should hopefully work now.

I see there is still issues of the type I mentioned above, e.g. `new_index : %(klass)s` has replaced `%(klass)s` and many others.
> that is unrelated to your PR, but has to do with the CI. I've restarted it, should hopefully work now.
> 
> I see there is still issues of the type I mentioned above, e.g. `new_index : %(klass)s` has replaced `%(klass)s` and many others.

let me look into it
> that is unrelated to your PR, but has to do with the CI. I've restarted it, should hopefully work now.
> 
> I see there is still issues of the type I mentioned above, e.g. `new_index : %(klass)s` has replaced `%(klass)s` and many others.

Is it fine now?
> Is it fine now?

Can you have a look at https://github.com/pandas-dev/pandas/pull/50225/files please? There's many unrelated changes which shouldn't be there
Like @MarcoGorelli says, there are still many unrelated changes. Those need to be removed.

Beside looking through https://github.com/pandas-dev/pandas/pull/50225/files, you can locally also use git to see the difference between this and the main branch. Are you using git on the terminal or in a GUI?
I think the simplest thing at this point might be:
```
git fetch upstream
git reset upstream/main
```
Then do
```
git add -p 
```
and type `y` for any change you want to keep, and `n` for any change you want to discard

Finally
```
git checkout -- .  # unstage the changes you discarded
git commit -m '<your commit message goes here>'
git push origin HEAD -f
```
> Like @MarcoGorelli says, there are still many unrelated changes. Those need to be removed.
> 
> Beside looking through https://github.com/pandas-dev/pandas/pull/50225/files, you can locally also use git to see the difference between this and the main branch. Are you using git on the terminal or in a GUI?

I am using pycharm. Most of these changes happened because I ran git rebase main, I should not have done that. What do you suggest that I should do?
> I think the simplest thing at this point might be:
> ```
> git fetch upstream
> git reset upstream/main
> ```
> Then do
> ```
> git add -p 
> ```
> and type `y` for any change you want to keep, and `n` for any change you want to discard
> 
> Finally
> ```
> git checkout -- .  # unstage the changes you discarded
> git commit -m '<your commit message goes here>'
> git push origin HEAD -f
> ```

Ok I will do that. I apologize for the inconvenience. If only I had been more careful and didn't perform 'rebase', I could have avoided this mess. I will be careful in the future. Hope I didn't bother you too much.  
All right, looks very close now. You’ve deleted a text in the whatsnew, that should ger back in, but else, it getting very close.
> All right, looks very close now. You’ve deleted a text in the whatsnew, that should ger back in, but else, it getting very close.

Are you referring to this text:
`- Deprecated argument ``infer_datetime_format`` in :func:`to_datetime` and :func:`read_csv`, as a strict version of it is now the default (:issue:`48621`)`
Yes.
> Yes.

I think all the issues should be resolved by now.

 @topper-123  and @MarcoGorelli thank you for all the help and guidance. I leant a lot during this pull request
> **Code Checks / Docstring validation, typing, and other manual pre-commit hooks (pull_request) ** Failing after 48m

This check is failing now

`Code Checks / Docstring validation, typing, and other manual pre-commit hooks (pull_request) Failing after 48m
`

These are the failure logs:
```
=================================== FAILURES ===================================
________________ [doctest] pandas.plotting._misc.andrews_curves ________________
304     -------
305     class:`matplotlip.axis.Axes`
306 
307     Examples
308     --------
309 
310     .. plot::
311         :context: close-figs
312 
313         >>> df = pd.read_csv(
UNEXPECTED EXCEPTION: <HTTPError 503: "hostname doesn't match against certificate">
Traceback (most recent call last):
  File "/home/runner/micromamba/envs/test/lib/python3.8/doctest.py", line [133](https://github.com/pandas-dev/pandas/actions/runs/3706934092/jobs/6282749278#step:7:134)6, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pandas.plotting._misc.andrews_curves[0]>", line 1, in <module>
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 907, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 579, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 1371, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 1625, in _make_engine
    self.handles = get_handle(
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 712, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 364, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 266, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: hostname doesn't match against certificate
/home/runner/work/pandas/pandas/pandas/plotting/_misc.py:313: UnexpectedException
_____________ [doctest] pandas.plotting._misc.parallel_coordinates _____________
436     -------
437     matplotlib.axis.Axes
438 
439     Examples
440     --------
441 
442     .. plot::
443         :context: close-figs
444 
445         >>> df = pd.read_csv(
UNEXPECTED EXCEPTION: <HTTPError 503: "hostname doesn't match against certificate">
Traceback (most recent call last):
  File "/home/runner/micromamba/envs/test/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest pandas.plotting._misc.parallel_coordinates[0]>", line 1, in <module>
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 907, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 579, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line [137](https://github.com/pandas-dev/pandas/actions/runs/3706934092/jobs/6282749278#step:7:138)1, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/runner/work/pandas/pandas/pandas/io/parsers/readers.py", line 1625, in _make_engine
    self.handles = get_handle(
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 712, in get_handle
    ioargs = _get_filepath_or_buffer(
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 364, in _get_filepath_or_buffer
    with urlopen(req_info) as req:
  File "/home/runner/work/pandas/pandas/pandas/io/common.py", line 266, in urlopen
    return urllib.request.urlopen(*args, **kwargs)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/home/runner/micromamba/envs/test/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 503: hostname doesn't match against certificate
```

Also, I have not made any changes in this file
That looks unrelated to your PR. The CI is a complex thing and sometimes things just fail and need to be fixed again. Ive restarted that test. We`ll see if it`s flakey or something needs to be fixed in the CI.
> Let's wait until after 2.0.0 is out to merge though, to not end up with too many FutureWarnings in the codebase where it's not clear if they're a deprecation to be enforced or a new one?

But isn't the next release 2.0, so all FutureWarnings will be new ones (because the old ones will have been removed when 2.0 is released?
yeah sorry what I meant was - let's first make sure we've removed all the old futurewarnings before adding new ones
> let's first make sure we've removed all the old futurewarnings before adding new ones

Ok. Talking about deprecations, could you review #50195? It's a blocker for #49560, which is the big deprecation that I'm working on.
I think this looks quite good now. Please note that some if the CI may fail independently of your PR, so it’s not necessarily anything wrong if some fail. You can read the test logs to get an indication if the failures are not related to your PR and just ask if in doubt.