weight decay in sgd_ada_delta
no weight shrinkage after nan in gradient in sgd_ada_delta