ctn-waterloo/nengo — pull request #168 by jaberg (2013-10-10 21:46:22+00:00)
Title: Backprop for better encoder-initialization

Presently, encoders are (by default) chosen randomly from a unit hypersphere. By default we also make a rate mode assumption for calculating decoders. It's possible to get better encoders by viewing the whole transformation from input signal -> non-linearities -> various decoded signals as a 1-hidden-layer machine-learning-style neural network. Essentially you pay some computation time up front for a more compact / accurate model.

This PR is working code, but not merge-ready. The idea is to provide functionality that iterates over a `Model`'s ensembles, and fills some/all of them in with optimized encoders.

So far, I find that on the oscillator example, even with 100 neurons, the two ensembles (A and T) that are supposed to match are a lot closer when their encoders are initialized this way.

Outstanding questions include:

* Is this a feature that anyone would want?
* How should this functionality be exposed? Where should the code go?
* How problematic is this dependency on scipy.optimize?
* What sort of experiments & metrics would people like to see on this technique?
* Any thoughts on propagating domains & ranges through the connections so that `eval_points` can be drawn from a more representative set?