slinderman/nb_spiketrains — pull request #3 by mattjj (2014-11-25 18:44:02+00:00)
Title: minor speedups with some new numerical code

I added `logdet_low_rank2` and `quad_form_diag_plus_lr2` which give some minor speed improvements (but remain the bottleneck).

Here is a profile snippet using the code before this PR:

```
260 160000 1051426 6.6 1.1 from utils.utils import invert_low_rank, det_low_rank, logdet_low_rank, quad_form_diag_plus_lr
261 # Ainv = 1.0/np.asscalar(self.sigma) * np.eye(N)
262 160000 2381164 14.9 2.5 d = np.asscalar(self.sigma) * np.ones((N,))
263 160000 1004804 6.3 1.1 Ainv = 1.0/d
264
265 # Sig_marg_inv = invert_low_rank(Ainv, X, self.Sigma_A, X.T, diag=True)
266 160000 962476 6.0 1.0 yy = y-mu_marg
267 160000 47260305 295.4 49.5 tmp = -1./2. * quad_form_diag_plus_lr(yy, d, X, self.Sigma_w, X.T)
268 out = tmp \
269 - N/2*np.log(2*np.pi) \
270 160000 37701745 235.6 39.5 - 0.5 * logdet_low_rank(Ainv, X, self.Sigma_w, X.T, diag=True)
```

Here's a profile snippet of the same section (using the same seed) after this PR:

```
260 160000 921334 5.8 1.6 from utils.utils import invert_low_rank, det_low_rank, logdet_low_rank2, quad_form_diag_plus_lr2
261 # Ainv = 1.0/np.asscalar(self.sigma) * np.eye(N)
262 160000 2072269 13.0 3.6 d = np.asscalar(self.sigma) * np.ones((N,))
263 160000 904181 5.7 1.6 Ainv = 1.0/d
264
265 # Sig_marg_inv = invert_low_rank(Ainv, X, self.Sigma_A, X.T, diag=True)
266 160000 878196 5.5 1.5 yy = y-mu_marg
267 160000 25023198 156.4 43.6 tmp = -1./2. * quad_form_diag_plus_lr2(yy, d, X, self.Sigma_w, X.T)
268 out = tmp \
269 - N/2*np.log(2*np.pi) \
270 160000 22805308 142.5 39.7 - 0.5 * logdet_low_rank2(Ainv, X, self.Sigma_w, X.T, diag=True)
```

So the total times on those two lines go from 47260305 to 25023198 and from 37701745 to 22805308, respectively. On my laptop, the overall iteration time for the HMM demo using some made-up synthetic data parameters (N=40, Nstates=20) improved about 33%.

The function `resample_As_and_regression_models` remains the bottleneck.