scikit-learn/scikit-learn — pull request #568 by mblondel (2012-01-18 11:07:44+00:00)
Title: More convenient warm start

Here are a few tiny changes to SGD and ElasticNet / Lasso which should make using warm-start much more convenient on a regularization path. Now you can do the following:

```python
clf = SGDClassifier(warm_start=True)
best_alpha = None
best_score = -np.inf
# from more regularized to less regularized
for alpha in (1.0, 0.1, 0.01):
clf.set_params(alpha=alpha)
clf.fit(X, y)
score = clf.score(X_val, y_val)
if score > best_score:
best_score = score
best_alpha = alpha
```

I think this way of doing is intuitive and, importantly, it doesn't rely on the internal parameters of the classifiers. You can clone the best estimator if you need to, too.

On the other hand, if you set `warm_start=False`, which is the default, the subsequent call to fit(X, y) erase the previous model.

SGDClassifier has now a partial_fit(X, y) method too but it has the following differences:

- X can be a subset of the entire dataset.
- It makes only one epoch over X.
- It saves the current state of the learning rate and start from there, while with warm-start, fit starts with a fresh learning rate.

Mathieu