Repository: kymatio/kymatio

Issue:
3D crashes on CPU
Getting the following error when calling `forward`:
```
  File "/home/janden/projects/kymatio/scattering/scattering3d/scattering3d.py", line 326, in forward
    conv_modulus, method, method_args, j_1))
  File "/home/janden/projects/kymatio/scattering/scattering3d/scattering3d.py", line 175, in _compute_scattering_coefs
    return self._compute_standard_scattering_coefs(input_array)
  File "/home/janden/projects/kymatio/scattering/scattering3d/scattering3d.py", line 105, in _compute_standard_scattering_coefs
    convolved_input = self._low_pass_filter(input_array, self.J)
  File "/home/janden/projects/kymatio/scattering/scattering3d/scattering3d.py", line 85, in _low_pass_filter
    cuda = isinstance(input_array, torch.cuda.FloatTensor)
RuntimeError: cuda runtime error (30) : unknown error at /opt/conda/conda-bld/pytorch_1532584813488/work/aten/src/THC/THCGeneral.cpp:74
```
From what I understand, the code is set up to detect whether the input is GPU and apply the appropriate operators (there's not `cuda()` or `cpu()` functions to convert the transformer like in 1D and 2D). Looks like when I run this without CUDA enabled, it crashes when trying to determine the type of the input tensor. It might be my personal config, but I should mention that `torch.cuda.is_available()` works without crashing.

More bigger picture, do we want to modify this implementation to more closely align with the 1D/2D calling sequence? May make things easier for users (and also prevent the above type of errors).


Discussion:
Hmm so I think that this `isinstance(input_array, torch.cuda.FloatTensor)` call is not a good idea for how to find out whether a tensor is on GPU and should be fixed. Is that the call that is causing the error? (can't tell for sure)

I think the calling sequence should be as similar as possible. That said, the code does something smart, quite implicitly, which is to copy filters back and forth from GPU. This is very useful in the 3D setting (as opposed to having all the filters in GPU memory all the time), because 3D images of any reasonable size together with the extra multiples of memory required to process them end up taking up a lot of space on the device.
So yes, we should make the things as similar as possible, but there might be some limits. (That was also one reason for having separate code bases and merging what is mergeable later on rather than trying to unify everything from the beginning)
Yes that's the call that causes the crash.

Right. That makes sense. Maybe that's an argument for having the same behavior for 1D/2D? The reason I'm bringing this up now is that it would be better to commit to a similar API from the beginning, as opposed to having things break down the road. Of course, we should only do this if we're relatively certain this is the way we want to go.
There is also this `is_cuda` method that Tensors and Variables have. I think this is a standard pytorch thing, but haven't pushed to adding it because I wasn't sure which version it was added, etc.

There is a big chance that this would solve the issue (on top of making the code more readable). One could also just somehow log somewhere that "hey we are currently doing CPU".

> Maybe that's an argument for having the same behavior for 1D/2D?

Actually I'd like to make sure I understand precisely what behavior you are referring to here
Ok. What I meant is that it may make sense to have the 1D/2D implementations detect whether the input is CPU/GPU and move their filters to/from the GPU depending, just like the 3D implementation. That is, we wouldn't have any `cuda()` or `cpu()` functions for `Scattering1D` and `Scattering2D`. The calling sequence would then be the same for 1D/2D/3D.
I'd say let's not do that:
1. The pytorch convention of saying `.cuda` and `.cpu` to networks and them transferring reliably to that device seems like a useful thing to hold on to (and aspire to), because that is the behavior people will be used to.
2. After release we can think about this magic `check_array` function which could do the converse of what you suggest (if set to do so): Move the data to where the model is (as opposed to moving the filters to where the data is). (This does not preclude 3D from storing most of its filters on CPU for space reasons - if it is declare to be on GPU it will perform its computations on GPU and that is what matters)

Ok. You're right, we don't want to do anything unexpected. But that's all the more reason to have the 3D behave more like 1D and 2D (with `cuda()` and `cpu()` methods, that is), no? Like you say, these methods do not have to explicitly move all the filters on/off the GPU, but may just set a flag telling the object whether it's allowed to use the GPU or not.
Yeah, I 100% agree with that.

(Question is whether this should be before release or pushed to later. Though if we can find somebody to do it before release that would be great of course)
Ok good. Again, for the sake of stabilizing the API, before release is better. I would take a look but 1) I'm still behind on the 1D doc and 2) I'm not very familiar with any of the 3D stuff. I may take a stab at it, though.
@louity can you add this feature?
This needs to be fixed before release.
Yeah this is more important than having a skcuda backend for 3D
Interestingly, this doesn't crash if `method='integral'` is specified. All tests in the 3D suite have this set, which is why Travis doesn't complain.
Looks like the culprit is `Scattering3D._low_pass_filter` which checks CUDAness of the input using `isinstance(x, torch.cuda.FloatTensor)`. If `method='integral'`, this is never called, so no crash. The only other function checking CUDAness in this instance is `Scattering3D._rotation_covariant_convolution_and_modulus`, which uses `x.is_cuda`.

Resolving patch:
diff --git a/kymatio/scattering3d/scattering3d.py b/kymatio/scattering3d/scattering3d.py
--- a/kymatio/scattering3d/scattering3d.py
+++ b/kymatio/scattering3d/scattering3d.py
@@ -86,7 +86,7 @@ def _low_pass_filter(self, input_array, j):
         output: the result of input_array :math:`\\star phi_J`
 
         """
-        cuda = isinstance(input_array, torch.cuda.FloatTensor)
+        cuda = input_array.is_cuda
         low_pass = self.gaussian_filters[j]
         if cuda:
             low_pass = low_pass.cuda()
@@ -240,7 +240,7 @@ def _convolution_and_modulus(self, input_array, l, j, m=0):
                 .. math:: \\text{input}_\\text{array} \\star \\psi_{j,l,m})
 
         """
-        cuda = isinstance(input_array, torch.cuda.FloatTensor)
+        cuda = input_array.is_cuda
         filters_l_m_j = self.filters[l][j][m]
         if cuda:
             filters_l_m_j = filters_l_m_j.cuda()


Test patch:
diff --git a/kymatio/scattering3d/tests/test_scattering3d.py b/kymatio/scattering3d/tests/test_scattering3d.py
--- a/kymatio/scattering3d/tests/test_scattering3d.py
+++ b/kymatio/scattering3d/tests/test_scattering3d.py
@@ -152,3 +152,26 @@ def test_larger_scales():
     for J in range(3, 4+1):
         scattering = Scattering3D(J=J, shape=shape, L=L, sigma_0=sigma_0)
         Sx = scattering(x, method='integral')
+
+def test_scattering_methods():
+    if backend.NAME == "skcuda":
+        warnings.warn(("The skcuda backend is not yet implemented for 3D "
+            "scattering, but that's ok (for now)."), RuntimeWarning,
+            stacklevel=2)
+        return
+
+    shape = (32, 32, 32)
+    J = 4
+    L = 3
+    sigma_0 = 1
+    x = torch.randn((1,) + shape)
+
+    scattering = Scattering3D(J=J, shape=shape, L=L, sigma_0=sigma_0)
+    Sx = scattering(x, method='standard')
+    Sx = scattering(x, method='standard', rotation_covariant=False)
+
+    points = torch.zeros(1, 1, 3)
+    points[0,0,:] = torch.tensor(shape)/2
+
+    Sx = scattering(x, method='local', points=points)
+    Sx = scattering(x, method='local', points=points, rotation_covariant=False)