Repository: Theano/Theano

Issue:
Optimization: 0/x -> 0
Discussed in https://github.com/Theano/Theano/issues/4086#issuecomment-186286921, and required for #3758. I'm adding this as an issue so it's not forgotten.

**Arguments against this optimization:**
It will hide NaNs (e.g., 0/0 or 0/nan would become 0).

**Arguments for this optimization:**
There are other optimizations that hide NaNs as well (e.g., for 0 \* nan or x / x), and the equivalent 0 \* inv(x) is also optimized to 0 already.

**Implementation:**
The optimization of 0*x is part of `theano.tensor.opt.Canonizer`, in its only incarnation `theano.tensor.opt.local_mul_canonizer`. Or maybe it's done by `theano.tensor.opt.local_mul_specialize`, after canonicalization? The latter clearly does it, the former has a lot of code that I don't have the time to read through.

So I guess we'd add a `local_div_of_zero` that looks similar to one of the cases of `local_mul_specialize` (https://github.com/Theano/Theano/blob/5f4eab9/theano/tensor/opt.py#L5382)? If there's consensus, I can do that.



Discussion:
"0 \* inv(x)" is optimized by the opt "0 \* x". It don't have a specific optimization.

I don't think local_mul_canonizer do that optimization. But we should make sure. If it do or don't add this in its docstring. If it don't why local_mul_specialize is only registered in specialize? I would register it in canonizer and stabilize.

This is always tricky to add such an optimization. We should test it well before merging. Make sure it don't break DLT, and other repo on top of Theano. But we should start by seing if Theano tests pass:)

is rprop this kind of optimization

On Fri, Mar 11, 2016 at 12:17 PM, Frédéric Bastien <notifications@github.com

> wrote:
> 
> "0 \* inv(x)" is optimized by the opt "0 \* x". It don't have a specific
> optimization.
> 
> I don't think local_mul_canonizer do that optimization. But we should make
> sure. If it do or don't add this in its docstring. If it don't why
> local_mul_specialize is only registered in specialize? I would register it
> in canonizer and stabilize.
> 
> This is always tricky to add such an optimization. We should test it well
> before merging. Make sure it don't break DLT, and other repo on top of
> Theano. But we should start by seing if Theano tests pass:)
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/Theano/Theano/issues/4213#issuecomment-195462105.

This is optimization done on Theano graph. This isn't machine learning optimization technique. In Theano repo, we don't talk much about machine learning optimization, so most of time, we refert to graph optimization/speed optimization.


Resolving patch:
diff --git a/theano/tensor/opt.py b/theano/tensor/opt.py
--- a/theano/tensor/opt.py
+++ b/theano/tensor/opt.py
@@ -5271,6 +5271,17 @@ def local_intdiv_by_one(node):
             return [node.inputs[0].astype(node.outputs[0].dtype)]
 
 
+@register_canonicalize
+@gof.local_optimizer([T.int_div, T.true_div])
+def local_zero_div(node):
+    """0 / x -> 0
+    """
+    if isinstance(node.op, T.Elemwise) and isinstance(
+            node.op.scalar_op, (theano.scalar.IntDiv, theano.scalar.TrueDiv)):
+        if local_mul_canonizer.get_constant(node.inputs[0]) == 0:
+            return [broadcast_like(0, node.outputs[0], node.fgraph)]
+
+
 @gof.local_optimizer([T.pow])
 def local_pow_specialize(node):
     # here, we are past the point of canonicalization, so we don't want


Test patch:
diff --git a/theano/tensor/tests/test_opt.py b/theano/tensor/tests/test_opt.py
--- a/theano/tensor/tests/test_opt.py
+++ b/theano/tensor/tests/test_opt.py
@@ -6123,6 +6123,27 @@ def test3(self):
         assert len(divs) == 0
 
 
+def test_local_zero_div():
+    """Tests 0/x -> 0"""
+    mode = theano.compile.mode.get_default_mode().including("local_zero_div")
+    for t in (T.scalar, T.ivector, T.ftensor4):
+        x = t('x')
+        for op in (T.int_div, T.true_div):
+            y = op(0, x)
+            g = optimize(FunctionGraph([x], [y]))
+            # the division should be gone
+            divs = [node for node in g.toposort()
+                    if isinstance(node.op, T.elemwise.Elemwise) and
+                    isinstance(node.op.scalar_op, type(op.scalar_op))]
+            assert len(divs) == 0
+            # the output type should match the unoptimized one
+            output = g.outputs[0]
+            assert output.ndim == y.ndim
+            assert output.type == y.type
+            # and the output should be zero
+            assert theano.tensor.get_scalar_constant_value(output) == 0
+
+
 def test_local_expm1():
     x = matrix('x')
     u = T.scalar('u')