scikit-learn/scikit-learn — pull request #2295 by larsmans (2013-07-28 14:49:32+00:00)
Title: [MRG] speed up logistic_sigmoid (using less code)

I tried to optimize the RBMs, which didn't work, but I did optimize the `logistic_sigmoid` in the process. As I suspected, with the right shifting and scaling, logistic sigmoid can be reduced to tanh to within 16 decimals precision (there is a test for this).

I did not yet optimize `log_logistic_sigmoid`, which is suffering from strides checking in a double `for` loop and (probably) branch prediction failures in the `>0` test. Ping @vene, @amueller, @fabianp.

Timings, before:

```
>>> x = np.linspace(-1000, 1000, 10000)
>>> %timeit logistic_sigmoid(x)
1000 loops, best of 3: 570 us per loop
```

After:

```
>>> %timeit logistic_sigmoid(x)
1000 loops, best of 3: 286 us per loop
```

For comparison:

```
>>> %timeit np.tanh(x)
10000 loops, best of 3: 117 us per loop
>>> %timeit .5 * (1 + np.tanh(x/2.))
1000 loops, best of 3: 301 us per loop
```