Add gradient clipping to SGDRegressor for stability and differential privacy

### Describe the workflow you want to enable

Add gradient clipping to SGDRegressor to:
1. Improve training stability when dealing with outliers or ill-conditioned data
2. Enable differentially private regression by bounding the influence of any single observation

This addition would enable users to:
- Train more stable models on real-world data with outliers
- Implement differentially private regression with minimal additional code
- Control the maximum influence of any single observation on the model

### Describe your proposed solution

Add a `clip_value` parameter to SGDRegressor that bounds the L2 norm of gradients during training:

```python
# Example usage
model = SGDRegressor(clip_value=1.0)
```

Implementation would involve:
1. Add `clip_value` parameter (default: None for no clipping)
2. Add gradient clipping in the update step:
```python
def _clip_gradient(self, grad):
    if self.clip_value is None:
        return grad
    norm = np.linalg.norm(grad)
    if norm > self.clip_value:
        return grad * (self.clip_value / norm)
    return grad
```

### Research basis and precedent

Gradient clipping is well-established:
- Introduced in ["On the difficulty of training Recurrent Neural Networks"](https://arxiv.org/abs/1211.5063) (2013, [7000+ citations](https://scholar.google.com/scholar?cites=3353056030101542547&as_sdt=20005&sciodt=0,9&hl=en))
- Standard practice in deep learning frameworks (PyTorch, TensorFlow)
- Core component of differentially private SGD as formalized in ["Deep Learning with Differential Privacy"](https://arxiv.org/abs/1607.00133) (2016, [6000+ citations](https://scholar.google.com/scholar?cites=11431158613977668861&as_sdt=20005&sciodt=0,9&hl=en))

Implemented in:
- PyTorch: `torch.nn.utils.clip_grad_norm_`
- TensorFlow: `tf.clip_by_norm`
- JAX: `jax.example_libraries.optimizers.clip_grads`
- Scikit-learn's neural networks

### Benefits

1. **Stability**: Prevents exploding gradients and improves convergence on ill-conditioned problems
2. **Privacy**: Essential building block for differentially private regression
3. **Simple implementation**: Minimal code changes required
4. **No new hyperparameters** when using default None (maintains backward compatibility)

### Benchmarks

Would be happy to provide benchmarks showing:
1. Improved stability on datasets with outliers
2. Convergence comparisons with/without clipping
3. Privacy-utility tradeoffs when used for differential privacy

Let me know if you'd like me to provide implementation details or benchmarks to support this addition.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I have two sets of questions when thinking about this:
1. @lorentzenchr said we already have gradient clipping. Is this clipping used in the `SGDRegressor`? I think yes, but wanted to double check. If yes, how is the clipping proposed here different? Is it the same but we make it user configurable? Is it different? Do we want to make this configurable? (Ignoring all motivation for doing this related to differential privacy for a moment)
2. [do we want to venture into the field of differential privacy](https://github.com/scikit-learn/scikit-learn/issues/30114#issuecomment-2454439564)? I am not an expert on differential privacy but I suspect that just adding user configurable gradient clipping and #30114 will only be the start if people start using scikit-learn for differential privacy applications. So I think we should have a bigger picture discussion/decision here. In particular I think we should avoid the case where we add a bit of differential privacy features (like this one) but then are negative on others. It reminds me of the MLP estimator which we added but now really wish people didn't use it and prefer not to add new features to it.

To make a decision on (1) I'd like to understand if this gradient clipping is the same/has the same use-case as the one we already have or not. If it is the same, what is to be gained from making it configurable.

For (2) I think we'd need someone who is more into differential privacy who could give us a broad overview style estimate of how much other features are missing in scikit-learn for serious differential privacy applications.

A kinda related question is: could the differential privacy use-case be solved by inheriting from `SGDRegressor` or otherwise using its (public) building blocks to make a differential privacy enbabled `SGDRegressor`? In this case there is the option of going the scikit-learn-contrib route

Labels: New Feature, Needs Decision