Weights are updating (still need a mean of zero) and forward propagation
is working for at least a single layer.  I haven't tested it to see if
it works for all layers yet.