Allowing model targets to be in a different space from model outputs

In response to: https://github.com/lisa-lab/pylearn2/issues/1004

When returning data_specs, DefaultDatasetMixin and get_monitoring_data_specs now call get_target_space instead of get_output_space. The base model class's get_target_space defaults to get_output_space unless the model has specified the _target_space attribute.

Softmax now has 2 new inputs: binary_targets and target_dim which allow the user to have a softmax with IndexSpace targets. This saves a lot of computation time not transfering and multiplying one-hot vectors.
I added Tile related optimization yesterday that Arnaud merged in Theano.
So if you updated Theano, this could explain some of the changes.

Here is a PR that make Tile copy the data when NumPy doesn't to respect the
view_map:

https://github.com/Theano/Theano/pull/1971

Can you test with it, without the dtype parameter?


On Wed, Jul 16, 2014 at 3:01 PM, Rahul <notifications@github.com> wrote:

> In pylearn2/models/mlp.py:
>
> > @@ -1457,33 +1476,43 @@ def cost(self, Y, Y_hat):
> >          z = z - z.max(axis=1).dimshuffle(0, 'x')
> >          log_prob = z - T.log(T.exp(z).sum(axis=1).dimshuffle(0, 'x'))
> >          # we use sum and not mean because this is really one variable per row
> > -        log_prob_of = (Y * log_prob).sum(axis=1)
> > +
> > +        if self._has_binary_target:
> > +            # The following code is the equivalent of accessing log_prob by the
> > +            # indices in Y, but it is written such that the computation can
> > +            # happen on the GPU rather than CPU.
> > +
> > +            flat_Y = Y.flatten()
> > +            flat_log_prob = log_prob.flatten()
> > +            flat_indices = flat_Y + T.arange(Y.shape[0])*self.n_classes
>
> @nouiz <https://github.com/nouiz> It is int64, the same dtype as
> Y.shape[0].
> In debug mode, theano raised this. I'm not sure what this means and if
> this is relevant.
>
> Traceback (most recent call last):
>   File "/u/huilgolr/pylearn2/pylearn2/scripts/train.py", line 252, in <module>
>     args.verbose_logging, args.debug)
>   File "/u/huilgolr/pylearn2/pylearn2/scripts/train.py", line 242, in train
>     train_obj.main_loop(time_budget=time_budget)
>   File "/u/huilgolr/pylearn2/pylearn2/train.py", line 207, in main_loop
>     rval = self.algorithm.train(dataset=self.dataset)
>   File "/u/huilgolr/pylearn2/pylearn2/training_algorithms/sgd.py", line 450, in train
>     self.sgd_update(*batch)
>   File "/u/huilgolr/Theano/theano/compile/function_module.py", line 579, in __call__
>     outputs = self.fn()
>   File "/u/huilgolr/Theano/theano/compile/debugmode.py", line 2070, in deco
>     return f()
>   File "/u/huilgolr/Theano/theano/compile/debugmode.py", line 1833, in f
>     _check_viewmap(node, storage_map)
>   File "/u/huilgolr/Theano/theano/compile/debugmode.py", line 786, in _check_viewmap
>     raise BadViewMap(node, oi, outstorage, bad_alias.values())
> theano.compile.debugmode.BadViewMap:   node: <theano.tensor.basic.Tile object at 0x5e0b610>(ARange.0, TensorConstant{(1,) of 1})
>   node.inputs: [('ARange.0', 185176272), ('TensorConstant{(1,) of 1}', 185176592)]
>   node.outputs: [('<theano.tensor.basic.Tile object at 0x5e0b610>.0', 185176656)]
>   view_map: {}
>   destroy_map: {}
>   aliased output: 0
>   aliased output storage: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
>  25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
>  50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
>  75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
>   aliased to inputs: [0]
>
> And unfortunately (for the purpose of debugging this issue), I can't seem
> to reproduce it anymore. The error I showed you was coming since yesterday
> evening for me as well as @memimo <https://github.com/memimo>. I doubt if
> the disappearance of the error is because of clearing theano-cache. I did
> that a couple of hours ago.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/lisa-lab/pylearn2/pull/1009/files#r15020065>.
>