I tried comparing the different versions of backprop by initializing them to the same values, playing with number of examples and non-linearities. The performance seems to get drastically worse with more examples, which may be a programming error, because with long time averages, the results should converge to backprop

Next step is noisy non-local backprop
I am commiting before making major changes to simplify comparison by seeding random number generators

Will soon be comparing computed derivatives with exact derivatives
Simplified version

Uses random number gen seeding
Fixed 2 bugs (one in regular backprop and one in noisy). Now the both seem to converge correctly in all limits (except deep linear, which is probably fine)

Next, need to run more tests, think about optimizing parameters appropriately and how to pick an appropriate noise level and time averaging