lisa-lab/pylearn2 — pull request #1200 by yelite (2014-11-02 04:20:27+00:00)
Title: Add AdaGrad to learning_rule

Implements the AdaGrad learning rule as described in: "Adaptive subgradient methods for online learning and stochastic optimization", Duchi J, Hazan E, Singer Y.

And I noticed there are some redundant code in learning_rule, like:
```
updates = OrderedDict()

for (param, grad) in grads.iteritems():
```
Should we move these to the base class?