scikit-learn/scikit-learn — pull request #424 by kcarnold (2011-11-03 19:35:08+00:00)
Title: Reorthogonalize

We were discussing reorthogonalization on the mailing list. Since I'd tried it out, here's a trivial patch (on top of my other pull request, https://github.com/scikit-learn/scikit-learn/pull/423, so there's some noise in the commits). I'm submitting this to contribute to the discussion; I don't think this patch should be accepted as-is.

My observations:

How much the power iteration helps apparently depends on how quickly
the singular spectrum decays. The matrices I was playing with had
slowly decaying spectra; I found q=10 got acceptable results (judging
by whether the singular values matched the non-randomized results). I
didn't find that the orthonormalization step helped much for my case,
and the QR did slow things down significantly, so I didn't pursue this
patch further. Perhaps orthonormalizing every few iterations would be
a good balance.