Localized Persistent Homologies for more Effective Deep Learning

1 INTRODUCTION . In many image segmentation tasks , the topology of the resulting mask is as important as , if not more than , its pixel-wise accuracy . For example , a model of an aortic valve that does not form a ring is biologically implausible . Similarly , networks of curvilinear structures—-be they roads in aerial images , blood vessels in Computer Tomography ( CT ) scans , or dendrites and axons in Light Microscopy ( LM ) image stacks—should not feature breaks that disrupt connectivity or false connections between disjoint structures . Unfortunately , deep networks trained by minimizing pixelwise loss functions , such as the cross-entropy or the mean square error , are subject to such mistakes . This is in part because it often takes very few mislabeled pixels to alter the topology significantly with little impact on the pixel-wise accuracy . In other words , it is possible for a network trained in this manner to deliver both a good pixel classification accuracy and an incorrect topology . Specialized solutions to this problem have been proposed in the form of loss functions that compare the topology of the prediction to that of the annotation . They are effective for specific applications but do not naturally generalize . For example , the perceptual loss of ( Mosińska et al. , 2018 ) penalizes topological differences between the prediction and the ground truth , but can not be guaranteed to detect them all . Similarly , minimizing the MALIS loss for segmenting electron microscopy scans ( Briggman et al. , 2009 ; Funke et al. , 2018 ) yields better region boundaries but does not penalize interruptions in loopy linear structures . This has been addressed by ( Oner et al. , 2021 ) for delineation of 2D road networks but the proposed solution is not applicable to 3D image stacks . Persistent Homology ( PH ) ( Edelsbrunner & Harer , 2008 ) is an elegant approach to describing and comparing topological structure of data , which is well-established in the field of topological data analysis ( TDA ) . It offers the promise to address the connectivity problem in a generic way , both for 2D and 3D images . Homology is the study of topological features in an object , such as its connected components ( 0-homology classes ) , loops ( 1-homology classes ) and closed surfaces ( 2-homology classes ) . Persistent homology detects homology classes in objects filtered at different scales . A homology class which appears at a particular scale and disappear at a larger one is represented by a scale interval called a persistence interval . The set of persistence intervals for all the homology classes characterizes the overall topology of the structure . It can be represented by a persistence diagram . The similarity of these diagrams across two different structures can then be used to quantify their topological similarity . This has been successfully exploited to train deep networks for delineation ( Hu et al. , 2019 ) , image segmentation ( Hu et al. , 2019 ; Clough et al. , 2019 ; 2020 ) and crowd counting ( Abousamra et al. , 2021 ) . However , existing techniques do not unleash the full power of persistent homology because the persistent diagrams are global image descriptors that ignore the location of the topological features , which reduces their descriptive power . As shown in Fig . 1 , this can result in networks that still fail to enforce the proper topology . This is because , when training a deep network , a persistence-based loss can be low even if the network predicts a structure that is quite different from the ground-truth . To remedy this , we introduce a new approach to computing persistence diagrams that takes location into account and increases their descriptive power , as shown in Fig . 3 . Our main contribution is a novel filtration technique that combines two filtrations commonly used in TDA : thresholding based filtrations and height functions . It results in a loss that applies both to 2D and 3D images and significantly improves performance compared to state-of-the-art topological methods . 2 RELATED WORK . Training a deep network that produces topologically correct segmentations has typically been done by designing loss functions that , when minimized , favor plausible topology . In this section , we briefly review first those that do not rely on Persistent Homology , and then those that do . Losses designed to enforce topological correctness Several such losses have been proposed already to go beyond pixel-wise classification accuracy by encoding more global properties . In ( Li et al. , 2020 ) , the connectivity between neighboring pixel pairs is used as an additional source of supervision . This approach has been shown to improve connectivity , but since disconnections or false connections are not penalized explicitly , there is no guarantee it captures all such errors . The perceptual loss of ( Mosińska et al. , 2018 ) is based on the assumption that a pre-trained neural network can capture differences of connectivity between the prediction and the ground-truth . However , even though it has been shown experimentally to improve the topology of masks produced by a deep net , there is no guarantee that this assumption holds in general . Making the Rand index of segmentations produced by the network similar to that of ground truth ones ( Briggman et al. , 2009 ; Funke et al. , 2018 ) helps when modeling tree-like structures , both in 2D and in 3D , but can not prevent disconnections in loopy structures . This shortcoming has been addressed by ( Oner et al. , 2021 ) by detecting disconnections of 2D loopy structures as interconnections of background regions , but the proposed solution does not generalize to 3D . Losses that rely on Persistent Homology Persistent Homology ( Edelsbrunner et al. , 2000 ; Zomorodian & Carlsson , 2004 ) is an established topological data descriptor . Among its numerous applications is comparing topological structures of binary images . It has been used to enforce the correct Betti number on binary masks resulting from inference in Markov Random Fields ( Chen et al. , 2011 ) . Recently , it has been demonstrated that persistence diagrams can be computed also for greyscale images and differentiated with respect to the pixel values ( Hu et al. , 2019 ; Clough et al. , 2019 ; Gabrielsson et al. , 2020 ; Leygonie et al. , 2021 ; Carriere et al. , 2021 ) . Hence , they can be used as loss or prior terms for training deep networks . In this vein , ( Clough et al. , 2019 ) proposed a loss term that enforces a sequence of desired Betti numbers on the predicted segmentation . This approach was further extended to a loss function that tends to equalize the Betti number of the prediction and the ground truth ( Clough et al. , 2020 ) . ( Hu et al. , 2019 ) proposed instead to construct a loss term around a difference of persistence diagrams of the prediction and the ground truth . Their persistence diagrams are obtained by thresholding the prediction and the ground truth . As we show in the next section , for binary ground truth this results in trivial persistence diagrams , that encode no more information than the ground truth Betti number . In consequence , both approaches can be interpreted as equalizing the Betti numbers of the prediction and the ground truth . ( Wang et al. , 2020 ) improved upon this technique by applying it to predicted and ground truth distance maps instead of binary annotations and class affinity maps . As discussed in more detail in section 3 , this makes the loss function more effective at detecting and penalizing topological errors . Unfortunately , even the improved technique remains susceptible to errors provoked by incorrectly matching the persistence diagrams of the prediction and the ground truth . By making the diagrams depend on location of the topological features in the image , our method makes them more diverse and minimizes potential for erroneous matches . It has also been proposed to detect disconnections in predicted 2D and 3D structures using Discrete Morse Theory ( Hu et al. , 2021 ) . Topological features that are inconsistent with the ground truth are then penalized in the loss function . However , when the annotations lack spatial precision , which is often the case for neurite and road centerline annotation like the ones studied here , ground-truth inaccuracies may confuse the network . By contrast , our technique allows for considerable misalignment between the prediction and the ground truth . 3 METHOD . We first introduce Persistent Homology and its application to characterizing two-dimensional images and three-dimensional image stacks . As PH provides global descriptors that ignore location of topological features , we then introduce our approach to accounting for it . 3.1 PERSISTENT HOMOLOGY . In the interest of simplicity , we introduce PH for binary images and image stacks , where homology classes are limited to connected components , loops , and closed surfaces . We refer the interested reader to the review ( Edelsbrunner et al. , 2000 ) for a more general treatment , applicable to nonimage and higher-dimensional data . At the heart of persistent homology is detecting homology classes ( connected components , loops , closed surfaces ) at many different scales . The ones that exist over a wide scale range are called persistent and deemed more likely to represent true features , as opposed to sampling artifacts or noise . Here , scale has a very specific meaning . It refers to the parameter of a filtration function F that is applied to an image X to produce topological objects called cubical complex . Cubical complexes resulting from filtering images and their properties are described for instance by ( Garin et al. , 2020 ) and by ( Bleile et al. , 2021 ) . A reader not familiar with algebraic topology can think of them as binary masks . The masks obtained for different scales form a sequence of inclusions , that is , for a pair of scale parameters s1 < s2 , the mask F ( X , s1 ) is entirely contained within the mask F ( X , s2 ) . The simplest example of a function for filtering grayscale images is thresholding , where the threshold acts as the scale , as shown in Fig . 2 . As the scale changes , homology classes in the filtered cubical complex emerge and disappear . To capture this , the scale range is sampled from small to large , the image is filtered at the selected scale values , homology classes in the resulting binary masks are detected algebraically ( Edelsbrunner et al. , 2000 ) , and correspondence is established between the homology classes found at consecutive scales . For each class , this yields a pair ( b , d ) , where b is the scale at which the homology class appears and d the scale at which it disappears . We will refer to them as birth and death times and to the interval [ b , d ] as the persistence interval of the homology class . The set PX = { ( bh , dh ) } h∈HX , where HX is the set of all homology classes found in the filtered image X , is called the persistence diagram of X , and was first introduced by ( Barannikov , 1994 ) . In practice , we use the Gudhi library ( Maria et al. , 2014 ) to compute persistence diagrams from images . Fig . 2 depicts the birth and death of a specific homology class . To compare images X1 and X2 , one-to-one matching is performed between their persistence diagrams , PX1 and PX2 , with the cost of matching a homology g ∈ HX1 to a homology h ∈ HX2 set to cg , h = ( bh − bg ) 2 + ( dh − dg ) 2 and the cost of leaving an interval [ b , d ) unmatched is set to the distance between the point ( b , d ) and the diagonal in R2 . The optimal matching can be found using the Hungarian algorithm . Its cost that we denote as C ( X1 , X2 ) quantifies the topological discrepancy between X1 and X2 by penalizing differences between corresponding homology classes and ones that only appear in either X1 or X2 .

Summary: In this article, the authors propose a new topological loss for training neural networks on images representing curvilinear structures. In addition to the usual MSE or cross-entropy losses, they suggest to add a regularization term based on persistent homology, in order to preserve the topological properties in the images predicted by the network. However, the common filtrations proposed in the literature (pixel values, distance maps) for doing so have their own flaws and are not always suited for such images, so the authors propose a new filtration that combines those common filtration with filtrations based on heights, i.e., with the persistent homology transform. Using this new filtration, they then provide a few experiments on which their loss compares favorably to other topological and non-topological losses (used on the same fixed neural net architecture).