Repository: kymatio/kymatio

Issue:
TST `backward` for complex modulus is not tested
Specifically, codecov reports that [lines 90-105](https://github.com/kymatio/kymatio/blob/master/kymatio/scattering1d/backend/backend_torch.py#L90) of `backend_torch.py` are [never executed](https://codecov.io/gh/kymatio/kymatio/src/master/kymatio/scattering1d/backend/backend_torch.py#L90). This is odd because we specifically test the `backward` function in the [corresponding test](https://github.com/kymatio/kymatio/blob/master/kymatio/scattering1d/tests/test_utils.py#L87). Somehow, torch must be bypassing our code.




Discussion:
Summary of discussion from #355:

It turns out `pytest` does run the code. If you put an `assert(False)` in there, it crashes. For some reason, `pytest-cov` isn't registering these lines.
Here's a minimal example:
```
import torch

class MyReLU(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input):
        ctx.save_for_backward(input)
        return input.clamp(min=0)

    @staticmethod
    def backward(ctx, grad_output):
        input, = ctx.saved_tensors
        grad_input = grad_output.clone()
        grad_input[input < 0] = 0
        return grad_input

def test_relu():
    relu = MyReLU.apply

    x = torch.randn((1, ), requires_grad=True)

    y = relu(x)

    y.backward()
```

If you run `pytest --cov --cov-report=term-missing` on this, it'll report the lines in `backward` as misses, even though they are executed.
So it's not a matter of `pytest-cov` plugin to `pytest`, but in the underlying `coverage.py` package. Running `coverage run pytest` followed by `coverage report` gives the same result.

What could be happening is that the `y.backward()` call eventually disappears into a C extension (see [here](https://github.com/pytorch/pytorch/blob/80927fc068548bd3fd4621c79b9475439d751f3e/torch/autograd/__init__.py#L91)), which `coverage` may have trouble following.
Looks like `coverage` uses [`sys.settrace`](https://docs.python.org/3/library/sys.html#sys.settrace) internally (see [here](https://coverage.readthedocs.io/en/latest/howitworks.html)). Seems like `sys.settrace` doesn't see what happens after we go into the C extension. Indeed, if we have the following code
```
import sys
import torch

class MyReLU(torch.autograd.Function):
    @staticmethod
    def forward(ctx, input):
        ctx.save_for_backward(input)
        return input.clamp(min=0)

    @staticmethod
    def backward(ctx, grad_output):
        input, = ctx.saved_tensors
        grad_input = grad_output.clone()
        grad_input[input < 0] = 0
        return grad_input

def tracefunc(frame, event, arg):
    print('%s, %s, %s' % (frame, event, arg))
    return tracefunc

if __name__ == '__main__':
    relu = MyReLU.apply

    x = torch.randn((1, ), requires_grad=True)
    y = relu(x)

    sys.settrace(tracefunc)
    y.backward()
    sys.settrace(None)
```
and just run it (no `pytest`, no `coverage`), we get printouts for every line inside PyTorch until the C extension is invoked, then nothing.
alright, is it a known issue? if yes, we can't do much about it..
> alright, is it a known issue? if yes, we can't do much about it..

I would think so, but would have to dig a little more/check with the `coverage` people.

One workaround is to "simulate" the PyTorch autograd algorithm by calling `forward` and `backward` ourselves. The other way is to add some pragmas to those lines to have `coverage` ignore them.
let's call them explicitely, it's the safer, as the error could be still catched
Agreed. Will look into this.

Resolving patch:
diff --git a/kymatio/scattering1d/backend/backend_torch.py b/kymatio/scattering1d/backend/backend_torch.py
--- a/kymatio/scattering1d/backend/backend_torch.py
+++ b/kymatio/scattering1d/backend/backend_torch.py
@@ -95,17 +95,12 @@ def backward(ctx, grad_output):
             grad_output = grad_output.unsqueeze(ctx.dim)
             output = output.unsqueeze(ctx.dim)
 
-        if ctx.p == 2:
-            grad_input = x.mul(grad_output).div(output)
-        else:
-            input_pow = x.abs().pow(ctx.p - 2)
-            output_pow = output.pow(ctx.p - 1)
-            grad_input = x.mul(input_pow).mul(grad_output).div(output_pow)
+        grad_input = x.mul(grad_output).div(output)
 
         # Special case at 0 where we return a subgradient containing 0
         grad_input.masked_fill_(output == 0, 0)
 
-        return grad_input, None, None, None
+        return grad_input
 
 # shortcut for ModulusStable.apply
 modulus = ModulusStable.apply


Test patch:
diff --git a/kymatio/scattering1d/tests/test_scattering1d.py b/kymatio/scattering1d/tests/test_scattering1d.py
--- a/kymatio/scattering1d/tests/test_scattering1d.py
+++ b/kymatio/scattering1d/tests/test_scattering1d.py
@@ -303,10 +303,6 @@ def test_differentiability_scattering(random_state=42):
     scattering = Scattering1D(J, T, Q)
     x = torch.randn(128, T, requires_grad=True)
 
-    if force_gpu:
-        scattering.cuda()
-        x = x.cuda()
-
     s = scattering.forward(x)
     loss = torch.sum(torch.abs(s))
     loss.backward()
diff --git a/kymatio/scattering1d/tests/test_utils.py b/kymatio/scattering1d/tests/test_utils.py
--- a/kymatio/scattering1d/tests/test_utils.py
+++ b/kymatio/scattering1d/tests/test_utils.py
@@ -105,10 +105,21 @@ def test_modulus(random_state=42):
     diff = x.grad - x_grad
     assert torch.max(torch.abs(diff)) <= 1e-7
 
+    # Manually check `forward`/`backward` using fake context object. This
+    # ensures that `backward` is included in code coverage since going through
+    # the PyTorch C extension seems to not play well with `coverage.py`.
+    class FakeContext:
+        def save_for_backward(self, *args):
+            self.saved_tensors = args
+
+    ctx = FakeContext()
+    y = backend.ModulusStable.forward(ctx, x)
+    y_grad = torch.ones_like(y)
+    x_grad_manual = backend.ModulusStable.backward(ctx, y_grad)
+    assert (x_grad_manual - x_grad).abs().max() < 1e-7
+
     # Test the differentiation with a vector made of zeros
     x0 = torch.zeros(100, 4, 128, 2, requires_grad=True)
-    if force_gpu:
-        x0 = x0.cuda()
     x_abs0 = modulus_complex(x0)
     loss0 = torch.sum(x_abs0)
     loss0.backward()