Allowing model targets to be in a different space from model outputs

In response to: https://github.com/lisa-lab/pylearn2/issues/1004

When returning data_specs, DefaultDatasetMixin and get_monitoring_data_specs now call get_target_space instead of get_output_space. The base model class's get_target_space defaults to get_output_space unless the model has specified the _target_space attribute.

Softmax now has 2 new inputs: binary_targets and target_dim which allow the user to have a softmax with IndexSpace targets. This saves a lot of computation time not transfering and multiplying one-hot vectors.
I think we should find why garbage is returned. Can you run in debugmode?


On Wed, Jul 16, 2014 at 12:44 PM, Mehdi Mirza <notifications@github.com>
wrote:

> In pylearn2/models/mlp.py:
>
> > @@ -1457,33 +1476,43 @@ def cost(self, Y, Y_hat):
> >          z = z - z.max(axis=1).dimshuffle(0, 'x')
> >          log_prob = z - T.log(T.exp(z).sum(axis=1).dimshuffle(0, 'x'))
> >          # we use sum and not mean because this is really one variable per row
> > -        log_prob_of = (Y * log_prob).sum(axis=1)
> > +
> > +        if self._has_binary_target:
> > +            # The following code is the equivalent of accessing log_prob by the
> > +            # indices in Y, but it is written such that the computation can
> > +            # happen on the GPU rather than CPU.
> > +
> > +            flat_Y = Y.flatten()
> > +            flat_log_prob = log_prob.flatten()
> > +            flat_indices = flat_Y + T.arange(Y.shape[0])*self.n_classes
>
> So apparently theano optimization in some cases results T.arange to return
> garbage. And adding dtype to T.arange will fix it.
> I don't like the idea of hardcoding int32 or int64 in the code. Perhaps we
> can add a config.intX to pylearn2 like theano's floatX to fix this issue.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/lisa-lab/pylearn2/pull/1009/files#r15011416>.
>