Localized Persistent Homologies for more Effective Deep Learning

1 INTRODUCTION . In many image segmentation tasks , the topology of the resulting mask is as important as , if not more than , its pixel-wise accuracy . For example , a model of an aortic valve that does not form a ring is biologically implausible . Similarly , networks of curvilinear structures—-be they roads in aerial images , blood vessels in Computer Tomography ( CT ) scans , or dendrites and axons in Light Microscopy ( LM ) image stacks—should not feature breaks that disrupt connectivity or false connections between disjoint structures . Unfortunately , deep networks trained by minimizing pixelwise loss functions , such as the cross-entropy or the mean square error , are subject to such mistakes . This is in part because it often takes very few mislabeled pixels to alter the topology significantly with little impact on the pixel-wise accuracy . In other words , it is possible for a network trained in this manner to deliver both a good pixel classification accuracy and an incorrect topology . Specialized solutions to this problem have been proposed in the form of loss functions that compare the topology of the prediction to that of the annotation . They are effective for specific applications but do not naturally generalize . For example , the perceptual loss of ( Mosińska et al. , 2018 ) penalizes topological differences between the prediction and the ground truth , but can not be guaranteed to detect them all . Similarly , minimizing the MALIS loss for segmenting electron microscopy scans ( Briggman et al. , 2009 ; Funke et al. , 2018 ) yields better region boundaries but does not penalize interruptions in loopy linear structures . This has been addressed by ( Oner et al. , 2021 ) for delineation of 2D road networks but the proposed solution is not applicable to 3D image stacks . Persistent Homology ( PH ) ( Edelsbrunner & Harer , 2008 ) is an elegant approach to describing and comparing topological structure of data , which is well-established in the field of topological data analysis ( TDA ) . It offers the promise to address the connectivity problem in a generic way , both for 2D and 3D images . Homology is the study of topological features in an object , such as its connected components ( 0-homology classes ) , loops ( 1-homology classes ) and closed surfaces ( 2-homology classes ) . Persistent homology detects homology classes in objects filtered at different scales . A homology class which appears at a particular scale and disappear at a larger one is represented by a scale interval called a persistence interval . The set of persistence intervals for all the homology classes characterizes the overall topology of the structure . It can be represented by a persistence diagram . The similarity of these diagrams across two different structures can then be used to quantify their topological similarity . This has been successfully exploited to train deep networks for delineation ( Hu et al. , 2019 ) , image segmentation ( Hu et al. , 2019 ; Clough et al. , 2019 ; 2020 ) and crowd counting ( Abousamra et al. , 2021 ) . However , existing techniques do not unleash the full power of persistent homology because the persistent diagrams are global image descriptors that ignore the location of the topological features , which reduces their descriptive power . As shown in Fig . 1 , this can result in networks that still fail to enforce the proper topology . This is because , when training a deep network , a persistence-based loss can be low even if the network predicts a structure that is quite different from the ground-truth . To remedy this , we introduce a new approach to computing persistence diagrams that takes location into account and increases their descriptive power , as shown in Fig . 3 . Our main contribution is a novel filtration technique that combines two filtrations commonly used in TDA : thresholding based filtrations and height functions . It results in a loss that applies both to 2D and 3D images and significantly improves performance compared to state-of-the-art topological methods . 2 RELATED WORK . Training a deep network that produces topologically correct segmentations has typically been done by designing loss functions that , when minimized , favor plausible topology . In this section , we briefly review first those that do not rely on Persistent Homology , and then those that do . Losses designed to enforce topological correctness Several such losses have been proposed already to go beyond pixel-wise classification accuracy by encoding more global properties . In ( Li et al. , 2020 ) , the connectivity between neighboring pixel pairs is used as an additional source of supervision . This approach has been shown to improve connectivity , but since disconnections or false connections are not penalized explicitly , there is no guarantee it captures all such errors . The perceptual loss of ( Mosińska et al. , 2018 ) is based on the assumption that a pre-trained neural network can capture differences of connectivity between the prediction and the ground-truth . However , even though it has been shown experimentally to improve the topology of masks produced by a deep net , there is no guarantee that this assumption holds in general . Making the Rand index of segmentations produced by the network similar to that of ground truth ones ( Briggman et al. , 2009 ; Funke et al. , 2018 ) helps when modeling tree-like structures , both in 2D and in 3D , but can not prevent disconnections in loopy structures . This shortcoming has been addressed by ( Oner et al. , 2021 ) by detecting disconnections of 2D loopy structures as interconnections of background regions , but the proposed solution does not generalize to 3D . Losses that rely on Persistent Homology Persistent Homology ( Edelsbrunner et al. , 2000 ; Zomorodian & Carlsson , 2004 ) is an established topological data descriptor . Among its numerous applications is comparing topological structures of binary images . It has been used to enforce the correct Betti number on binary masks resulting from inference in Markov Random Fields ( Chen et al. , 2011 ) . Recently , it has been demonstrated that persistence diagrams can be computed also for greyscale images and differentiated with respect to the pixel values ( Hu et al. , 2019 ; Clough et al. , 2019 ; Gabrielsson et al. , 2020 ; Leygonie et al. , 2021 ; Carriere et al. , 2021 ) . Hence , they can be used as loss or prior terms for training deep networks . In this vein , ( Clough et al. , 2019 ) proposed a loss term that enforces a sequence of desired Betti numbers on the predicted segmentation . This approach was further extended to a loss function that tends to equalize the Betti number of the prediction and the ground truth ( Clough et al. , 2020 ) . ( Hu et al. , 2019 ) proposed instead to construct a loss term around a difference of persistence diagrams of the prediction and the ground truth . Their persistence diagrams are obtained by thresholding the prediction and the ground truth . As we show in the next section , for binary ground truth this results in trivial persistence diagrams , that encode no more information than the ground truth Betti number . In consequence , both approaches can be interpreted as equalizing the Betti numbers of the prediction and the ground truth . ( Wang et al. , 2020 ) improved upon this technique by applying it to predicted and ground truth distance maps instead of binary annotations and class affinity maps . As discussed in more detail in section 3 , this makes the loss function more effective at detecting and penalizing topological errors . Unfortunately , even the improved technique remains susceptible to errors provoked by incorrectly matching the persistence diagrams of the prediction and the ground truth . By making the diagrams depend on location of the topological features in the image , our method makes them more diverse and minimizes potential for erroneous matches . It has also been proposed to detect disconnections in predicted 2D and 3D structures using Discrete Morse Theory ( Hu et al. , 2021 ) . Topological features that are inconsistent with the ground truth are then penalized in the loss function . However , when the annotations lack spatial precision , which is often the case for neurite and road centerline annotation like the ones studied here , ground-truth inaccuracies may confuse the network . By contrast , our technique allows for considerable misalignment between the prediction and the ground truth . 3 METHOD . We first introduce Persistent Homology and its application to characterizing two-dimensional images and three-dimensional image stacks . As PH provides global descriptors that ignore location of topological features , we then introduce our approach to accounting for it . 3.1 PERSISTENT HOMOLOGY . In the interest of simplicity , we introduce PH for binary images and image stacks , where homology classes are limited to connected components , loops , and closed surfaces . We refer the interested reader to the review ( Edelsbrunner et al. , 2000 ) for a more general treatment , applicable to nonimage and higher-dimensional data . At the heart of persistent homology is detecting homology classes ( connected components , loops , closed surfaces ) at many different scales . The ones that exist over a wide scale range are called persistent and deemed more likely to represent true features , as opposed to sampling artifacts or noise . Here , scale has a very specific meaning . It refers to the parameter of a filtration function F that is applied to an image X to produce topological objects called cubical complex . Cubical complexes resulting from filtering images and their properties are described for instance by ( Garin et al. , 2020 ) and by ( Bleile et al. , 2021 ) . A reader not familiar with algebraic topology can think of them as binary masks . The masks obtained for different scales form a sequence of inclusions , that is , for a pair of scale parameters s1 < s2 , the mask F ( X , s1 ) is entirely contained within the mask F ( X , s2 ) . The simplest example of a function for filtering grayscale images is thresholding , where the threshold acts as the scale , as shown in Fig . 2 . As the scale changes , homology classes in the filtered cubical complex emerge and disappear . To capture this , the scale range is sampled from small to large , the image is filtered at the selected scale values , homology classes in the resulting binary masks are detected algebraically ( Edelsbrunner et al. , 2000 ) , and correspondence is established between the homology classes found at consecutive scales . For each class , this yields a pair ( b , d ) , where b is the scale at which the homology class appears and d the scale at which it disappears . We will refer to them as birth and death times and to the interval [ b , d ] as the persistence interval of the homology class . The set PX = { ( bh , dh ) } h∈HX , where HX is the set of all homology classes found in the filtered image X , is called the persistence diagram of X , and was first introduced by ( Barannikov , 1994 ) . In practice , we use the Gudhi library ( Maria et al. , 2014 ) to compute persistence diagrams from images . Fig . 2 depicts the birth and death of a specific homology class . To compare images X1 and X2 , one-to-one matching is performed between their persistence diagrams , PX1 and PX2 , with the cost of matching a homology g ∈ HX1 to a homology h ∈ HX2 set to cg , h = ( bh − bg ) 2 + ( dh − dg ) 2 and the cost of leaving an interval [ b , d ) unmatched is set to the distance between the point ( b , d ) and the diagonal in R2 . The optimal matching can be found using the Hungarian algorithm . Its cost that we denote as C ( X1 , X2 ) quantifies the topological discrepancy between X1 and X2 by penalizing differences between corresponding homology classes and ones that only appear in either X1 or X2 .

Summary: This paper presents a new approach for detecting topological structures in 2D and 3D imaging data. Based on the computational topology framework persistent homology, the paper develops a type of loss term that can be used to assess to what extent topology is being preserved when solving segmentation or structural extraction tasks. A suite of experiments is used to assess the quality of the extracted structures.