Repository: Project-MONAI/MONAI

Issue:
789 cuda resampling part2
related to #789 

### Description
this is an experimental feature, mainly to provide basic code structure and build/test CI environments for the resampling model

- adds cuda/cpp impl. of higher-order interpolation layer (in both 'pull' and 'push' directions -- differentiable)
- by default no cpp/cuda extension installations (unless user enables the flag `BUILD_MONAI=1 python setup.py develop`)
- pypi install won't have the precompiled with this PR
- CI is enabled to test building with a few versions of pytorch+cuda/cpu+linux/macos

the resampling code is adapted from https://github.com/balbasty/nitorch
cc @balbasty @brudfors

### Status
**ready**

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not applicable items -->
- [x] Non-breaking change (fix or new feature that would not break existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing functionality to change).
- [x] New tests added to cover the changes.
- [x] Integration tests passed locally by running `./runtests.sh --codeformat --coverage`.
- [x] In-line docstrings updated.



Discussion:
Nice to see the great progress here! In addition to unit tests, I think it's important to also have some performance benchmarks. As shown in a [quick and dirty 2D test for linear interpolation](https://colab.research.google.com/drive/17pI6xPGLTGcpGBulBrWZeNnBfLAwzqZT?usp=sharing) differences of 2 orders of magnitudes in resampling times are not uncommon.

Resolving patch:
diff --git a/monai/networks/layers/spatial_transforms.py b/monai/networks/layers/spatial_transforms.py
--- a/monai/networks/layers/spatial_transforms.py
+++ b/monai/networks/layers/spatial_transforms.py
@@ -41,8 +41,10 @@ def backward(ctx, grad):
         grads = _C.grid_pull_backward(grad, *var, *opt)
         if ctx.needs_input_grad[0]:
             grad_input = grads[0]
-        if ctx.needs_input_grad[1]:
-            grad_grid = grads[1]
+            if ctx.needs_input_grad[1]:
+                grad_grid = grads[1]
+        elif ctx.needs_input_grad[1]:
+            grad_grid = grads[0]
         return grad_input, grad_grid, None, None, None
 
 
@@ -133,8 +135,10 @@ def backward(ctx, grad):
         grads = _C.grid_push_backward(grad, *var, *opt)
         if ctx.needs_input_grad[0]:
             grad_input = grads[0]
-        if ctx.needs_input_grad[1]:
-            grad_grid = grads[1]
+            if ctx.needs_input_grad[1]:
+                grad_grid = grads[1]
+        elif ctx.needs_input_grad[1]:
+            grad_grid = grads[0]
         return grad_input, grad_grid, None, None, None, None
 
 
@@ -328,8 +332,10 @@ def backward(ctx, grad):
             grads = _C.grid_grad_backward(grad, *var, *opt)
             if ctx.needs_input_grad[0]:
                 grad_input = grads[0]
-            if ctx.needs_input_grad[1]:
-                grad_grid = grads[1]
+                if ctx.needs_input_grad[1]:
+                    grad_grid = grads[1]
+            elif ctx.needs_input_grad[1]:
+                grad_grid = grads[0]
         return grad_input, grad_grid, None, None, None
 
 


Test patch: