Package: elasticdeform
Elastic deformations for N-dimensional images (Python, SciPy, NumPy, TensorFlow, PyTorch)This library implements elastic grid-based deformations for N-dimensional images.The elastic deformation approach is described inRonneberger, Fischer, and Brox, "U-Net: Convolutional Networks for Biomedical
Image Segmentation" (https://arxiv.org/abs/1505.04597)Çiçek et al., "3D U-Net: Learning Dense Volumetric
Segmentation from Sparse Annotation" (https://arxiv.org/abs/1606.06650)The procedure generates a coarse displacement grid with a random displacement
for each grid point. This grid is then interpolated to compute a displacement for
each pixel in the input image. The input image is then deformed using the
displacement vectors and a spline interpolation.In addition to the normal, forward deformation, this package also provides a
function that can backpropagate the gradient through the deformation. This makes
it possible to use the deformation as a layer in a convolutional neural network.
For convenience, TensorFlow and PyTorch wrappers are provided inelasticdeform.tfandelasticdeform.torch.Installationpip install elasticdeform
or
pip install git+https://github.com/gvtulder/elasticdeformThis library requires Python 3 and NumPy development headers.On Windows, try to install the precompiled binaries directly usingpip install elasticdeform.
If that does not work,these precompiled packagesmight be an alternative option.ExamplesThis basic example deforms an image with a random 3 x 3 deformation grid:importnumpy,imageio,elasticdeformX=numpy.zeros((200,300))X[::10,::10]=1# apply deformation with a random 3 x 3 gridX_deformed=elasticdeform.deform_random_grid(X,sigma=25,points=3)imageio.imsave('test_X.png',X)imageio.imsave('test_X_deformed.png',X_deformed)Multiple inputsIf you have multiple images, e.g., an image and a segmentation image, you can
deform both simultaneously by providing a list of inputs. You can specify
a different spline order for each input.# apply deformation to inputs X and Y[X_deformed,Y_deformed]=elasticdeform.deform_random_grid([X,Y])# apply deformation to inputs X and Y,# with a different interpolation for each input[X_deformed,Y_deformed]=elasticdeform.deform_random_grid([X,Y],order=[3,0])Multi-channel imagesBy default, a deformation will be applied to every dimension of the input. If you
have multi-channel images, you can use theaxisparameter to specify which axes
should be deformed. The same deformation will be applied for each channel.For example, to deform an RGB image across the first two dimensions, run:X_deformed=elasticdeform.deform_random_grid(X,axis=(0,1))When deforming multiple inputs, you can provide a tuple of axes for each input:X=numpy.random.rand(3,200,300)Y=numpy.random.rand(200,300)[X_deformed,Y_deformed]=elasticdeform.deform_random_grid([X,Y],axis=[(1,2),(0,1)])CroppingIf you intend to crop a small subpatch from the deformed image, you can provide
the crop dimensions to the deform function. It will then compute only the cropped
output pixels, while still computing the deformation grid based on the full image
dimensions. This saves computation time.X=numpy.random.rand(200,300)# define a crop regioncrop=(slice(50,150),slice(0,100))# generate a deformation griddisplacement=numpy.random.randn(2,3,3)*25# deform full imageX_deformed=elasticdeform.deform_grid(X,displacement)# compute only the cropped regionX_deformed_crop=elasticdeform.deform_grid(X,displacement,crop=crop)# the deformation is the samenumpy.testing.assert_equal(X_deformed[crop],X_deformed_crop)Rotate and zoomThe deformation functions acceptrotateandzoomparameters, which allows you
to combine the elastic deformation with rotation and scaling. This can be useful
as data augmentation step. The rotation and zoom are applied to the output
coordinates, using the center pixel of the output patch as the origin.# apply deformation with a random 3 x 3 grid,# rotate by 30 degrees and rescale with a factor 1.5X_deformed=elasticdeform.deform_random_grid(X,sigma=25,points=3,rotate=30,zoom=1.5)Note that the output shape remains the same. The mapping of the input to the
output is rotated within the given output frame.Rotate and zoom can be combined with thecropargument. In that case, the
scaling and rotation is performed relative to the center of the cropped output.For more advanced transformations, it is also possible to provide an affine
transformation matrix directly.GradientThedeform_grid_gradientfunction can be used to backpropagate the gradient of
the output with respect to the input. Calldeform_grid_gradientwith the
parameters that were used for the forward step.X=numpy.random.rand(200,300)# generate a deformation griddisplacement=numpy.random.randn(2,3,3)*25# perform forward deformationX_deformed=elasticdeform.deform_grid(X,displacement)# obtain the gradient w.r.t. X_deformed (e.g., with backpropagation)dX_deformed=numpy.random.randn(*X_deformed.shape)# compute the gradient w.r.t. XdX=elasticdeform.deform_grid_gradient(dX_deformed,displacement)Note: The gradient function will assume that the input has the same size as the
output. If you used thecropparameter in the forward phase, it is necessary to
provide the gradient function with the original, uncropped input shape in theX_shapeparameter.TensorFlow wrapperTheelasticdeform.tfmodule provides a wrapper fordeform_gridin TensorFlow.
The function uses TensorFlow Tensors as input and output, but otherwise uses
the same parameters.importnumpyimportelasticdeform.tfasetfdisplacement_val=numpy.random.randn(2,3,3)*5X_val=numpy.random.rand(200,300)dY_val=numpy.random.rand(200,300)# construct TensorFlow input and top gradientdisplacement=tf.Variable(displacement_val)X=tf.Variable(X_val)dY=tf.Variable(dY_val)# the deform_grid function is similar to the plain Python equivalent,# but it accepts and returns TensorFlow TensorsX_deformed=etf.deform_grid(X,displacement,order=3)# the gradient w.r.t. X can be computed in the normal TensorFlow manner[dX]=tf.gradients(X_deformed,X,dY)PyTorch wrapperTheelasticdeform.torchmodule provides a wrapper fordeform_gridin PyTorch.
The function uses PyTorch Tensors as input and output, but otherwise uses
the same parameters.importnumpyimportelasticdeform.torchasetorchdisplacement_val=numpy.random.randn(2,3,3)*5X_val=numpy.random.rand(200,300)dY_val=numpy.random.rand(200,300)# construct PyTorch input and top gradientdisplacement=torch.tensor(displacement_val)X=torch.tensor(X_val,requires_grad=True)dY=torch.tensor(dY_val)# the deform_grid function is similar to the plain Python equivalent,# but it accepts and returns PyTorch TensorsX_deformed=etorch.deform_grid(X,displacement,order=3)# the gradient w.r.t. X can be computed in the normal PyTorch mannerX_deformed.backward(dY)print(X.grad)License informationThis library was written byGijs van Tulderat theBiomedical Imaging Group Rotterdam,
Erasmus MC, Rotterdam, the NetherlandsIt is inspired by a similar, Python-based implementation byFlorian Calvet.
This C-based implementation gives the same results, but is faster and has
a gradient implementation.This C implementation includes a modified version of theNI_GeometricTransformfromSciPy's ndimage library.This code is made available under the BSD license. SeeLICENSE.txtfor details.If you want to cite this library, please see.https://github.com/gvtulder/elasticdeformhttps://elasticdeform.readthedocs.io/