OrdinalEncoder fails inverse_transform with np.nan as

# Describe the bug

I want to use inverse_transform on the OrdinalEncoder from and to np.nan. 

Steps/Code to Reproduce
```python
from sklearn.preprocessing import OrdinalEncoder
import numpy as np

enc = OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value=np.nan, encoded_missing_value=np.nan)

X_train = [["a", "1"]]
enc.fit(X_train)

X_test = [["c", "1"]]
X_test_encoded= enc.transform(X_test)
X_test_decoded = enc.inverse_transform(X_test_encoded)
```

# Actual result
```
Traceback (most recent call last):
  File "/home/ME/.pyenv/versions/3.10.4/lib/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/home/ME/.local/share/virtualenvs/python-XEcZKjNR/lib/python3.10/site-packages/sklearn/preprocessing/_encoders.py", line 1420, in inverse_transform
    X_tr[:, i] = self.categories_[i][np.where(unknown_labels, 0, labels)]
IndexError: index -9223372036854775808 is out of bounds for axis 0 with size 1
```

# Expected result

I expected that np.nan inverse transform is the same as `unknown_value`, so also np.nan in this case. The expected array would be `[np.nan, 0]`. As this might be a dtype problem (nan as float, rest of array might be string), None would also be an option

Versions

```
python==3.10.4
scikit-learn==1.1.1
numpy==1.22.4
```

COMMENT:
> I am not able to reproduce this issue locally with the versions you provided

I am able to reproduce it locally. The index error can be traced to the fact that inside the main loop of `inverse_transform`, we have `labels = X_test_encoded[:, i].astype("int64", copy=False)` where `X_test_encoded[:, i]=[nan]`

I'm actually surprised that:
1. It worked for you locally
2. `np.array([np.nan]).astype(np.int64) == np.array([-9223372036854775808])` while `np.array([np.nan], dtype=np.int64)` errors. I suppose this is an upstream bug as well.

For reference: 
<details>
<summary>System Info</summary>

```
System:
    python: 3.9.12 (main, Jun  1 2022, 11:38:51)  [GCC 7.5.0]
executable: /home/micky774/anaconda3/envs/scikit-dev/bin/python
   machine: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.dev0
          pip: 21.2.4
   setuptools: 61.2.0
        numpy: 1.22.3
        scipy: 1.7.3
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: mkl
         prefix: libmkl_rt
       filepath: /home/micky774/anaconda3/envs/scikit-dev/lib/libmkl_rt.so.1
        version: 2021.4-Product
threading_layer: intel
    num_threads: 6

       user_api: openmp
   internal_api: openmp
         prefix: libiomp
       filepath: /home/micky774/anaconda3/envs/scikit-dev/lib/libiomp5.so
        version: None
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/micky774/anaconda3/envs/scikit-dev/lib/libgomp.so.1.0.0
        version: None
    num_threads: 12
```
</details>

Labels: Bug, module:preprocessing