scikit-learn/scikit-learn — pull request #2962 by larsmans (2014-03-12 21:47:50+00:00)
Title: [MRG] use threads instead of multiprocessing in SGD

New take on the idea of running SGD workers in threads instead of processes. Despite the locking needed to do shuffling in between iterations, this actually speeds up SGD on 20 newsgroups (all categories, fixed random states). Current master:

```
================================================================================
Perceptron
________________________________________________________________________________
Training:
Perceptron(alpha=0.0001, class_weight=None, eta0=1.0, fit_intercept=True,
n_iter=50, n_jobs=-1, penalty=None, random_state=42, shuffle=False,
verbose=0, warm_start=False)
train time: 10.516s
test time: 0.138s
f1-score: 0.796
dimensionality: 129792
density: 0.098179

================================================================================
Passive-Aggressive
________________________________________________________________________________
Training:
PassiveAggressiveClassifier(C=1.0, fit_intercept=True, loss='hinge',
n_iter=50, n_jobs=-1, random_state=42, shuffle=False,
verbose=0, warm_start=False)
train time: 11.422s
test time: 0.143s
f1-score: 0.854
dimensionality: 129792
density: 0.456828

================================================================================
Elastic-Net penalty
________________________________________________________________________________
Training:
SGDClassifier(alpha=0.0001, class_weight=None, epsilon=0.1, eta0=0.0,
fit_intercept=True, l1_ratio=0.15, learning_rate='optimal',
loss='hinge', n_iter=50, n_jobs=-1, penalty='elasticnet',
power_t=0.5, random_state=42, shuffle=False, verbose=0,
warm_start=False)
train time: 29.376s
test time: 0.140s
f1-score: 0.849
dimensionality: 129792
density: 0.032481
```

This PR:

```
================================================================================
Perceptron
________________________________________________________________________________
Training:
Perceptron(alpha=0.0001, class_weight=None, eta0=1.0, fit_intercept=True,
n_iter=50, n_jobs=-1, penalty=None, random_state=42, shuffle=False,
verbose=0, warm_start=False)
train time: 5.702s
test time: 0.144s
f1-score: 0.796
dimensionality: 129792
density: 0.098179

================================================================================
Passive-Aggressive
________________________________________________________________________________
Training:
PassiveAggressiveClassifier(C=1.0, fit_intercept=True, loss='hinge',
n_iter=50, n_jobs=-1, random_state=42, shuffle=False,
verbose=0, warm_start=False)
train time: 7.008s
test time: 0.141s
f1-score: 0.854
dimensionality: 129792
density: 0.456828

================================================================================
Elastic-Net penalty
________________________________________________________________________________
Training:
SGDClassifier(alpha=0.0001, class_weight=None, epsilon=0.1, eta0=0.0,
fit_intercept=True, l1_ratio=0.15, learning_rate='optimal',
loss='hinge', n_iter=50, n_jobs=-1, penalty='elasticnet',
power_t=0.5, random_state=42, shuffle=False, verbose=0,
warm_start=False)
train time: 24.980s
test time: 0.140s
f1-score: 0.849
dimensionality: 129792
density: 0.032481
```