scikit-learn/scikit-learn — pull request #849 by pprett (2012-05-11 07:53:22+00:00)
Title: WIP: Multinomial Logistic Regression via SGD

This is an early PR for multinomial logistic regression via stochastic gradient descent. ``SGDClassifier`` now has a new parameter ``multi_class`` (similar to ``LinearSVC``). If ``loss=='log'`` and ``multi_class=='multinomial'`` SGDClassifier trains a multinomial LR model instead ``n_classes`` binary LR models.

Refactorings:
* sgd_fast.pyx: LossFunctions now have a new method ``weight_update`` (see MultinomialLogLoss for the new weight update); I've also added a multi-class Perceptron.
* New learning rate schedule: Exponential decay - according to LingPipe's Bob Carpenter this works fine for Multinomial LR, Buttou'-style learning rate calibration is a bit tricky and I've had mixed results in the past (comments/remarks much appreciated).