Inconsistent `median`/`quantile` behaviour now `_weighted_percentile` ignores NaNs

As of https://github.com/scikit-learn/scikit-learn/pull/29034, `_weighted_percentile` handles NaNs by ignoring them when calculating `percentile`.
`np.median` and `np.percentile` on the other hand, will return NaN if a NaN is present in the input (`np.nanmedian` and `np.nanpercentile` will ignore nans).

There are many cases in the codebase where, if `sample_weight` is `None`, a `np` function is used (NaN returned), if `sample_weight` is given, `_weighted_percentile` used and NaNs ignored.

Summary of affected cases:

* `DummyRegressor.fit`
* `AbsoluteError`/`PinballLoss`/`HuberLoss`  - `fit_intercept_only` method
* `median_absolute_error`
* `d2_pinball_score`
* `SplineTransformer._get_base_knot_positions` - I think this was the original reason for https://github.com/scikit-learn/scikit-learn/pull/29034

Maybe we could assess on a case by case basis whether it makes sense to return NaN if present in the input? @ogrisel suggested that we may want to raise a warning in some cases as well.

cc @StefanieSenger

COMMENT:
Hi all,

Thanks for the detailed issue and discussion around the inconsistent NaN handling in _weighted_percentile and its downstream effects. I've taken a closer look at the code paths mentioned in the issue, and I can confirm the described inconsistencies.

Here's a summary of what I found:

**Core Inconsistency:**

- When sample_weight is None, functions like DummyRegressor.fit (with strategy="median" or "quantile"), AbsoluteError.fit_intercept_only, PinballLoss.fit_intercept_only, and median_absolute_error typically rely on np.median() or np.percentile(). These NumPy functions propagate NaNs (i.e., if there's a NaN in the input, the result is NaN).

- When sample_weight is provided, these functions (or the functions they call, like _weighted_percentile) generally switch to a behavior where NaNs in the input array (y_true or absolute errors) are ignored during the calculation (as _weighted_percentile itself does).

- HuberLoss.fit_intercept_only is an interesting case, as it consistently ignores NaNs by calling _weighted_percentile with an array of ones for weights even when sample_weight is None.
- The d2_pinball_score function also exhibits this inconsistency, as its internal calculation of y_quantile (the baseline prediction) uses np.percentile (propagates NaNs) when sample_weight is None, and _weighted_percentile (ignores NaNs) when sample_weight is present. This, combined with how mean_pinball_loss handles (or doesn't explicitly handle) NaNs in its direct inputs, leads to different outcomes.

**Affected Components Investigated:**

- DummyRegressor(strategy="median").fit
- DummyRegressor(strategy="quantile").fit
- AbsoluteError().fit_intercept_only
- PinballLoss().fit_intercept_only
- median_absolute_error()
- d2_pinball_score()

This inconsistent behavior depending on the mere presence of the sample_weight argument can indeed be confusing and lead to unexpected results.
**Moving Forward -(Unifying Behavior):**
The key question seems to be: What should the desired consistent behavior be regarding NaNs for these functions?

1. Option A: Always ignore NaNs (similar to how _weighted_percentile currently behaves, and how np.nanmedian / np.nanpercentile work). If this is the preferred direction (which PR #29034 also seemed to lean towards for _weighted_percentile itself):

- 
- The fix would involve modifying the code paths where sample_weight is None to use np.nanmedian(), np.nanpercentile(), or to call _weighted_percentile() with a dummy array of ones as weights.

2. Option B: Always propagate NaNs (current np.median/np.percentile behavior).

- This would require _weighted_percentile to be modified to propagate NaNs if present in the input array (perhaps via a new parameter, though this might be a more significant change to its current design).

Given the existing behavior of _weighted_percentile and its use in contexts where ignoring NaNs with weights is useful, Option A (always ignore NaNs) seems like a potentially more straightforward path to consistency for these higher-level functions.
I'd appreciate any thoughts from the maintainers on the preferred uniform behavior. Once a direction is decided, I'd be happy to try and help with a PR.
Thanks!

Labels: module:utils