Add GpuSqrSumAx0 to lower the memory usage on the GPU.

Don't merge yet. I think I'll try to reuse GpuCAReduce.
If we do this, we your need to time if this slow down computation of not.
Don't forget, some reduction are not optimized. For example, if we reduce
on all axies, we use only 1 block. So for case where the pre_scalar_op is
compute intensive, doing this would result in slow down.

If we fix all those corner case, maybe we can enable it for all cases.


On Mon, Jun 9, 2014 at 10:48 PM, abergeron <notifications@github.com> wrote:

> In theano/sandbox/cuda/opt.py:
>
> > @@ -685,6 +685,24 @@ def local_gpu_careduce(node):
> >      return False
> >
> >
> > +@register_opt("low_memory")
> > +@local_optimizer([GpuCAReduce])
> > +def local_gpu_elemwise_careduce(node):
> > +    if (isinstance(node.op, GpuCAReduce) and
> > +        node.op.pre_scalar_op is None and
> > +        node.inputs[0].owner and
> > +        isinstance(node.inputs[0].owner.op, GpuElemwise) and
> > +        # The Op support all scalar with 1 inputs.  We don't
> > +        # automatically add more case, as some like trigonometic
> > +        # operation with some reduction pattern will probably result
> > +        # to slow down.
> > +        isinstance(node.inputs[0].owner.op.scalar_op, scal.basic.Sqr)
>
> And for the second question. Why don't we want to do this for any scalar
> with one input and one output (including Composite)?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/Theano/Theano/pull/1888/files#r13575593>.
>