Package: pypatchify
PyPatchifyFast and easy image and n-dimensional volume patchificationInstall and RequirementsThe package can be installed fromPyPI:pipinstallpypatchifyIt supports both numpy arrays and pytorch tensors. However pytorch is not strictly required. The only necessary dependency is:numpy >= 1.21.5To install all dependencies (including pytorch and pytest) run the followingpython-mpipinstallrequirements.txtHello WorldThe library is designed to be easy to use while keeping the computational overhead as low as possible. The following simple example shows how to patchify and unpatchify a batch of rgb images:importpypatchifyimportnumpyasnp# create 16 random rgb imagesimgs=np.random.uniform(0,1,size=(16,3,256,256))# patchify into non-overlapping blocks of size 64x64patched_imgs=pypatchify.patchify(imgs,(64,64))# re-create the original images from the patchesimgs=pypatchify.unpatchify(patched_imgs,(256,256))In case the created patches are further processed, for example by passing them through a neural network, it might make sense to collapse the different patches into a batch size as follows:imgs=np.random.uniform(0,1,size=(16,3,256,256))# patchify into non-overlapping blocks of size 64x64patched_imgs=pypatchify.patchify_to_batches(imgs,(64,64),batch_dim=0)# re-create the original images from the patchesimgs=pypatchify.unpatchify_from_batches(patched_imgs,(256,256),batch_dim=0)Note that the implementations are not restricted to 2d images only but can patchify and unpatchify any multi-dimensional volume:vols=np.random.uniform(0,1,size=(16,32,32,64,64))# patchify into non-overlapping blocks of size 61x8x32x16patched_vols=pypatchify.patchify_to_batches(vols,(16,8,32,16),batch_dim=0)# re-create the original images from the patchesvols=pypatchify.unpatchify_from_batches(patched_vols,(32,32,64,64),batch_dim=0)GPU-Acceleration and DifferentiableAlso when working with neural networks its probably more convenient to directly work with pytorch tensors. This can be done by simply passing the torch tensors to the function at hand. Note that all implementations allow gpu-tensors which drastically decrease the runtime of any of the patchification functions. Also there is no need to move memory between cpu and gpu.importtorchimportpypatchify# create a random img tensor and move to cudaimgs=torch.rand(16,3,256,256).cuda()# patchify into non-overlapping blocks of size 64x64patched_imgs=pypatchify.patchify_to_batches(imgs,(64,64),batch_dim=0)# re-create the original images from the patchesimgs=pypatchify.unpatchify_from_batches(patched_imgs,(256,256),batch_dim=0)Furthermore all the functions are completly differentiable allowing for gradient propagation through patchification and un-patchification.# let f and g be differentiable functions# possibly neural networksf=torch.tanh# processes the imagesg=torch.sigmoid# processes the patched images# create a random img tensor and move to cudaimgs=torch.rand(16,3,256,256,requires_grad=True)# apply functions and patchifypatched_imgs=pypatchify.patchify_to_batches(f(imgs),(64,64),batch_dim=0)unpatched_imgs=pypatchify.unpatchify_from_batches(g(patched_imgs),(256,256),batch_dim=0)# compute some kind of loss and backpropagateloss=unpatched_imgs.sum()# dummy lossloss.backward()# check gradients in input imgsgrads=imgs.grad# should be all onesOther FrameworksThe library makes it very easy to support other frameworks besides numpy and pytorch. All work that needs to be done is to implement the following few functions:shape: get the shape of a given tensorstrides: get the strides of the underlying memoryreshape: reshape a given tensor to a given shapetranspose: permute the dimensions of a given tensor to a given permuationas_strided: apply a given shape and strides to the memory of a given tensorNote that most frameworks already support these functions. To now integrate the framework just inherit from thepypatchify.patch.Patchifyclass and enter the functions:classNewFramework(Patchify[NewTensorType]):# get shape and strides from tensor objectshape:Callablestrides:Callable# tensor operationsreshape:Callabletranspose:Callableas_strided:CallableThe class now holds static member functions for all the patchification functionality includingpatchify,unpatchify, etc.