Toolkit for multilayer backprop. General Nonlinearity and propagation of signal function created.

I believe the functions work correctly, although backpropagation is not behaving as expected I expect the mistake is minor, so saving what I have so far.
Updated Multilayer

Fixed the bug (In previous version I forgot to recalculate the unit values before computing updates to W)
Began noisy version of backprop averaging over T 3 times before each update.

Wrote a function to propagate the noisy system for T timesteps, and began computing the appropriate quantities to compute noisy backprop
BP works for single layer

Got noisy BP working (checked in linear case) and next will extend to multiple layers
Extended and fixed local noisy BP

Extended for noisy BP over all layers and fixed the problem that I was averaging over the wrong noise and not updating
Converted the noisy backprop to a function

Just Converted the noisy backprop to a function, need to convert other files still
converted all 3 algorithms to functions

All 3 functions are descending reasonably and now need to write a script to compare them
basic comparison between different methods

Makes basic plots to compare error descent as a function of interation. Need to make more sophisticated with thresholds, etc.
Basic comparisons plots & annealing

Added a function that anneals the noise level in the network, might in future add one that increases the averaging time between updates (because presumably we want higher variance at this point)
added online version of backprop

Also added 1/sqrt(M) (to get good initialization values of W and thus the nonlinearities) in bpOnline. Things seem to scale ok with M now, also need to Write an online version of noisy sim. bp. I think having online learning may be more realistic anyway biologically.
reorganized

before writing online localNoisy
added local online function
I tried comparing the different versions of backprop by initializing them to the same values, playing with number of examples and non-linearities. The performance seems to get drastically worse with more examples, which may be a programming error, because with long time averages, the results should converge to backprop

Next step is noisy non-local backprop
I am commiting before making major changes to simplify comparison by seeding random number generators

Will soon be comparing computed derivatives with exact derivatives
Simplified version

Uses random number gen seeding
Fixed 2 bugs (one in regular backprop and one in noisy). Now the both seem to converge correctly in all limits (except deep linear, which is probably fine)

Next, need to run more tests, think about optimizing parameters appropriately and how to pick an appropriate noise level and time averaging
Latest streamlined version of backprop

Next step speeding up adaptive parameter? Momentum term? Anisotropy in updates
added global version of local bp
ran comparisons of the old and new method of noisy bp
Tested that dW scaled as I expected