Added save_records to push monitor function and updated implementation of RNNs

Added save_records to push monitor function:
Initialize {batch, val, example, epoch, time} records of new monitor channels with those of old monitor channels in order to preserve and pass the training information, when re-training from the checkpoint. Now, it can be much easier to plot the whole curves. A user can use this functionality by setting save_records flag of push_monitor function into 1. 

Updated implementation of RNNs:
1. One could choose to use or not use the masks.
2. LSTM implementation has changed
3. included add_noise function which adds Gaussian noise to parameters
4. Recurrent weights are always orthogonally initialized using scipy.linalg.orth()


Yes I thought adding copy_history as a flag will require a lot of
modifications.


--Junyoung

On Fri, Nov 21, 2014 at 1:03 PM, Pascal Lamblin <notifications@github.com>
wrote:

> In pylearn2/monitor.py:
>
> > @@ -697,11 +697,12 @@ def add_channel(self, name, ipt, val, dataset=None, prereqs=None,
> >                                    "monitor's datasets"))
> >
> >          if name in self.channels:
> > -            raise ValueError("Tried to create the same channel twice (%s)" %
> > -                             name)
> > -
> > -        self.channels[name] = MonitorChannel(ipt, val, name, data_specs,
> > -                                             dataset, prereqs)
> > +            self.channels[name] = MonitorChannel(ipt, val, name, data_specs,
> > +                                                 dataset, prereqs,
> > +                                                 self.channels[name])
>
> Actually, add_channel does not know whether we are reloading or setting
> the channels for the first time.
> I think we could make add a member of Monitor instead, for instance
> Monitor.on_channel_conflict, that would default to "error", but could be
> "copy_history" or "overwrite". Then, push_monitor_ would change the value
> of that member to "copy_history".
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/lisa-lab/pylearn2/pull/1259/files#r20730405>.
>