Halving searches crash when using a PredefinedSplit as cv

### Describe the bug

In some cases, it might be necessary to use a predefined split with an explicit training and testing set instead of cross-validation (for example if the data has a specific distribution of properties that should be the same in a training and testing set).
However, when attempting to use a halving search (`HalvingRandomSearchCV ` or `HalvingGridSearchCV`) with a PredefinedSplit as cv, the search crashes with the error
> sklearn.utils._param_validation.InvalidParameterError: The 'n_samples' parameter of resample must be an int in the range [1, inf) or None. Got 0 instead.

(after a long stack of internal calls).

However, [as I understand it](https://scikit-learn.org/stable/modules/grid_search.html#successive-halving-user-guide), the basic idea of increasing a resource (like the number of samples taken from the (predefined) split) while reducing the amount of candidates should not depend on the specific type of split; therefore, using a predefined split should work with a halving search as it would with a non-halving search.

### Steps/Code to Reproduce

```python
from sklearn.ensemble import RandomForestRegressor
from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import PredefinedSplit, HalvingRandomSearchCV

import numpy as np

# 10 input features
# 4 output features
# 80 training data points
# 20 testing data points
train_input_values = np.random.rand(80, 10)
train_output_values = np.random.rand(80, 4)
test_input_values = np.random.rand(20, 10)
test_output_values = np.random.rand(20, 4)

# Define the search parameters
model = RandomForestRegressor()
hyperparameter_grid = {"n_estimators": [10, 100]}

# Define the train/test split
total_input_values = np.concat((train_input_values, test_input_values))
total_output_values = np.concat((train_output_values, test_output_values))
cv = PredefinedSplit([-1] * len(train_input_values) + [0] * len(test_input_values))

# Perform the search
random_search = HalvingRandomSearchCV(model, hyperparameter_grid, cv=cv)
random_search.fit(total_input_values, total_output_values)
```

### Expected Results

The halving random search should work with a predefined split just as it would with a cross-validation split, increasing the resources while reducing the amount of candidates.

### Actual Results

```
Traceback (most recent call last):
  File "<python-input-0>", line 27, in <module>
    random_search.fit(total_input_values, total_output_values)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.13/site-packages/sklearn/base.py", line 1389, in wrapper
    return fit_method(estimator, *args, **kwargs)
  File "venv/lib/python3.13/site-packages/sklearn/model_selection/_search_successive_halving.py", line 253, in fit
    super().fit(X, y=y, **params)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.13/site-packages/sklearn/base.py", line 1389, in wrapper
    return fit_method(estimator, *args, **kwargs)
  File "venv/lib/python3.13/site-packages/sklearn/model_selection/_search.py", line 1024, in fit
    self._run_search(evaluate_candidates)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "venv/lib/python3.13/site-packages/sklearn/model_selection/_search_successive_halving.py", line 357, in _run_search
    results = evaluate_candidates(
        candidate_params, cv, more_results=more_results
    )
  File "venv/lib/python3.13/site-packages/sklearn/model_selection/_search.py", line 982, in evaluate_candidates
    for (cand_idx, parameters), (split_idx, (train, test)) in product(
                                                              ~~~~~~~^
        enumerate(candidate_params),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        enumerate(cv.split(X, y, **routed_params.splitter.split)),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "venv/lib/python3.13/site-packages/sklearn/model_selection/_search_successive_halving.py", line 41, in split
    test_idx = resample(
        test_idx,
    ...<2 lines>...
        n_samples=int(self.fraction * len(test_idx)),
    )
  File "venv/lib/python3.13/site-packages/sklearn/utils/_param_validation.py", line 206, in wrapper
    validate_parameter_constraints(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        parameter_constraints, params, caller_name=func.__qualname__
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "venv/lib/python3.13/site-packages/sklearn/utils/_param_validation.py", line 98, in validate_parameter_constraints
    raise InvalidParameterError(
    ...<2 lines>...
    )
sklearn.utils._param_validation.InvalidParameterError: The 'n_samples' parameter of resample must be an int in the range [1, inf) or None. Got 0 instead.
```

### Versions

```shell
System:
    python: 3.13.2 (main, Feb  4 2025, 14:51:09) [Clang 16.0.0 (clang-1600.0.26.6)]
executable: venv/bin/python
   machine: macOS-14.3-arm64-arm-64bit-Mach-O

Python dependencies:
      sklearn: 1.6.1
          pip: 25.0
   setuptools: 75.8.2
        numpy: 2.2.3
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: None
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: venv/lib/python3.13/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
Hey,
first of all: Thank you so much for the quick reply and work on this. I really appreciate this!

> you expect the test and train sets to remain constant, but that goes against the whole purpose of the HalvingRandomSearchCV

I… don't really get this tbh. Maybe I'm misunderstanding how the halving search works, but I don't see why this would go against its purpose.
The [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.HalvingRandomSearchCV.html#sklearn.model_selection.HalvingRandomSearchCV) even states that:
> Due to implementation details, the folds produced by cv must be the same across multiple calls to cv.split().

So from my understanding, this is even required for the sets.

----

The way the [user guide](https://scikit-learn.org/stable/modules/grid_search.html#successive-halving-user-guide) explains the halving search, it states that the core principle is to increase the amount of resources in each step and reduce the amount of candidates.
If the resources are e.g. `n_estimators`, why would the sets being constant contradict this? You could still use the constant train and test sets to first train 100 candidates with them with 10 trees, then 50 candidates with them with 20 trees, etc.
And if the resources are the number of training samples (which they are per default), the search could / should e.g. first train 100 candidates with 10 samples out of the constant training set, then train 50 candidates with 20 samples out of the training set, etc.

I don't see why using an explicit train / test split instead of cross-validation would go against this basic principle.

Labels: Bug