LocalOutlierFactor might not work with duplicated samples

This an investigation from the discussion in https://github.com/scikit-learn/scikit-learn/discussions/27838

`LocalFactorOutlier` might be difficult to use when there are duplicate values larger then `n_neighbors`. In this case, the distance for these neighbors is `0`, meaning that the local reachibility density is therefore infinite (or in the algorithm `1 / 1e-10`). The issue starts for sample next to those local peaky density: they might use the `1 / 1e-10` as measure, meaning that they will have a really negative `negative_local_outlier` while the value of the sample could be really close to the one of the plateau. I will now provide a minimum sythetic example to show the issue:

```python
import numpy as np
from sklearn.neighbors import LocalOutlierFactor

rng = np.random.default_rng(0)
x = rng.permutation(np.hstack([
    [0.1] * 10,  # constant values
    np.linspace(0.1, 0.2, num=30),
    rng.random(5) * 100  # the clear outliers
]))
X = x.reshape(-1, 1)

lof = LocalOutlierFactor(n_neighbors=5, contamination=0.1)
outliers = lof.fit_predict(X)

indices = np.where(outliers == -1)
# check that shows that outliers can be found from the linspace
print(X[indices])

print(lof.negative_outlier_factor_[indices])
```

```shell
array([[ 0.10344828],
       [26.97867138],
       [81.32702392],
       [63.69616873],
       [ 0.10689655]])

array([-3.31034492e+07, -1.22583005e+03, -1.10083976e+03, -9.37195958e+02,
       -4.13793114e+07])
```

In the results above, we see that the first and last values should not be considered as outliers but because they have their neighbors coming from the constant part (i.e. plateau at 0.1), the local reachibility density is 1e10 and thus the negative outlier factor is set to -1e7.

Running the same code without the constant part will give:

```python
import numpy as np
from sklearn.neighbors import LocalOutlierFactor

rng = np.random.default_rng(0)
x = rng.permutation(np.hstack([
    np.linspace(0.1, 0.2, num=30),
    rng.random(5) * 100  # the clear outliers
]))
X = x.reshape(-1, 1)

lof = LocalOutlierFactor(n_neighbors=5, contamination=0.1)
outliers = lof.fit_predict(X)

indices = np.where(outliers == -1)
# check that shows that outliers can be found from the linspace
print(X[indices])

print(lof.negative_outlier_factor_[indices])
```

```shell
[[81.32702392]
 [63.69616873]
 [ 4.09735239]
 [26.97867138]]
[-1100.83976485  -937.19595768  -237.40975167 -1225.83005019]
```

which is more what one would expect.

Now, my question would be if we can have a strategy to limit such a corner case that is ill-defined currently algorithmically? A potential solution is to find such extreme value and raise a warning and mentioning that there are duplicate and increasing `n_neighbors` could alleviate the problem?

ping @ngoix @albertcthomas @agramfort if you have any input.

COMMENT:
Another potential approach: 

Adding a toggle parameter that allows programmers to manually handle duplicate samples. This parameter can be used to indicate whether the algorithm should automatically identify and handle duplicate samples, or allow programmers to intervene and handle them manually, exploring other strategies as well.

In terms of implementation, for example, introduce a parameter named `handle_duplicates` (or similar) with a default value set to automatically handle duplicate samples. When developers encounter the issue described above, they can set this parameter to `False`, thereby preventing the algorithm from automatically handling duplicate samples and instead allowing developers to handle these cases themselves.

This design can provide more flexible control, allowing developers to choose how to handle duplicate samples as needed while retaining the automation features of the algorithm. This toggle parameter can serve as an additional option to address the issue described in the text and offer more customization options.

Labels: Bug