Validation and monitoring of neg-log-likelihoods averaged over examples and words. Recording of regularization variables.
memory saving in validation and random tests even when validation fails
Experimental adadelta gradient prescaling
weight decay in sgd_ada_delta
no weight shrinkage after nan in gradient in sgd_ada_delta
fixes to weight decay
revert to weight shrinking after a nan in minibatch
SGD adadelta with cutoff adaptation
weight decay moved to model class
gradient cutoff can now shrink the gradient to something else than the cutoff level and better adaptive cutoff constant initialization
layers have ndim property, SGD_adadelta will not pront grad stats when cutoff adaptation is disabled
actual fix
With weight matrix column norm clipping
validation supports dropout, we either sum or divide by batch size to work wiuth nonuniformily sized batches