API: Make Categorical.searchsorted returns a scalar when supplied a scalar
State: closed

- [x] closes #21019
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

``Categorical.searchsorted`` returns the wrong shape for scalar input. Numpy arrays and all other array types return a scalar if the input is a scalar, but ``Categorical`` does not.

For example:

```python
>>> import numpy as np
>>> np.array([1, 2, 3]).searchsorted(1)
0
>>> np.array([1, 2, 3]).searchsorted([1])
array([0])
>>> import pandas as pd
>>> d = pd.date_range('2018', periods=4)
>>> d.searchsorted(d[0])
0
>>> d.searchsorted(d[:1])
array([0])

>>> n = 100_000
>>> c = pd.Categorical(list('a' * n + 'b' * n + 'c' * n), ordered=True)
>>> c.searchsorted('b')
array([100000], dtype=int32)  # master
100000  # this PR. Scalar input should lead to scalar output
>>> c.searchsorted(['b'])
array([100000], dtype=int32)  # master and this PR
```

This new implementation is BTW quite a bit faster than the old implementation, because we avoid recoding the codes when doing the ``self.codes.searchsorted(code, ...)`` bit:

```python
>>> %timeit c.searchsorted('b')
237 µs  # master
6.12 µs  # this PR
```

A concequence of the new implementation is that KeyError is now raised when a key isn't found. Previously a ValueError was raised.


Labels:
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
Hello @topper-123! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/arrays/categorical.py`](https://github.com/topper-123/pandas/blob/cbccb8732bdc9f36cfd185bd9552cc126850f928/pandas/core/arrays/categorical.py) !

 - There are no PEP8 issues in the file [`pandas/core/indexes/category.py`](https://github.com/topper-123/pandas/blob/cbccb8732bdc9f36cfd185bd9552cc126850f928/pandas/core/indexes/category.py) !

 - There are no PEP8 issues in the file [`pandas/tests/arrays/categorical/test_analytics.py`](https://github.com/topper-123/pandas/blob/cbccb8732bdc9f36cfd185bd9552cc126850f928/pandas/tests/arrays/categorical/test_analytics.py) !


looks fine, ping on green.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=h1) Report
> Merging [#23466](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/91d1c50327c209e03ffc32c5fd64f9666de81e20?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23466/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23466      +/-   ##
==========================================
+ Coverage   92.24%   92.24%   +<.01%     
==========================================
  Files         161      161              
  Lines       51433    51434       +1     
==========================================
+ Hits        47446    47447       +1     
  Misses       3987     3987
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.64% <100%> (ø)` | :arrow_up: |
| #single | `42.28% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/23466/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.35% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=footer). Last update [91d1c50...d7b6873](https://codecov.io/gh/pandas-dev/pandas/pull/23466?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Green.
It turns out that making ``get_indexer`` fast in not easy. The issue is that the method needs an Index as its argument, or converts its input to an Index. Converting to Index is a very slow process, and probably it's best to make ``get_indexer`` use arrays/ExtensionArrays (lower overhead when creating, presumably), but that's a completely different issue.

So I've reverted to make minimal changes in ``searchsorted``, and only do the changes in the API (scalar input leads to scalar output).

I'll take a look at making ``get_indexer`` faster in a seperate PR and then - if I succeed - make ``searchsorted`` faster using ``get_indexer``.
thanks @topper-123