# A Novel Evolution Strategy of Level Set Method for the Segmentation of Overlapping Cervical Cells
###### Abstract  
Development of an accurate and automated algorithm to completely segment cervical cells in Pap images is still one of the most challenging tasks. The main reasons are the presence of overlapping cells and the lack of guiding mechanism for the convergence of ill-defined contours to the actual cytoplasm boundaries. In this paper, we propose a novel method to address these problems based on level set method (LSM). Firstly, we proposed a morphological scaling-based topology filter (MSTF) and derived a new mathematical toolbox about vector calculus for evolution of level set function (LSF). Secondly, we combine MSTF and the mathematical toolbox into a multifunctional filtering algorithm 2D codimension two-object level set method (DCTLSM) to split touching cells. The DCTLSM can morphologically scale up and down the contour while keeping part of the contour points fixed. Thirdly, we design a contour scanning strategy as the evolution method of LSF to segment overlapping cells. In this strategy, a cutting line can be detected by morphologically scaling the union LSF of the pairs of cells. Then, we used this cutting line to construct a velocity field with an effective guiding mechanism for attracting and repelling LSF. The performance of the proposed algorithm was evaluated quantitatively and qualitatively on the ISBI-2014 dataset. The experimental results demonstrated that the proposed method is capable of fully segmenting cervical cells with superior segmentation accuracy compared with recent peer works. cervical cell; image segmentation; level set method; ISBI; topology filtering 1234455 process is very difficult and time consuming, and cytotechnologists are susceptible to make erroneous decisions [6] mainly caused by intra- and interobserver variability. Therefore, as a preparatory work of automated diagnosis, automated slide analysis has been a research focus. As the most essential task in automated slide analysis, the automated segmentation of cervical cells in the extended depth-of-field (EDF) images captured in different fields of view (FOVs) [7] is an ongoing concern [8; 9; 10]. Although some work has been done for cervical cell segmentation, such as segmentation of isolated cells (i.e., free-lying cells) and nuclei, it is still a challenge to develop an accurate and complete segmentation algorithm for overlapping cells in Pap images [10]. Due to the influence of poor contrast, overlapping, folding, polymorphic nonconvex shapes and the impurity introduced from the cell deposition process, accurate and complete segmentation of cytoplasm from cellular clumps in Pap images is the biggest obstacle for this challenge. Therefore, it is essential to develop a new system that can commendably carry out automated segmentation of overlapping cytoplasm for cervical cells in the Pap images. The literature reviews about related studies on segmentation of cervical cells are in the following. Prior to the first attempt [11] to segment the overlapping cervical cells completely, the systems could be divided into three groups, including (1) segmentation of the nuclei [12; 13; 14; 15; 16; 17; 18] of cells, (2) segmentation of the isolated cells [19; 20; 21] and (3) segmentation of clusters of cells crowded together (i.e., cellular clumps) [22]. Then, a few complete segmentation systems for overlapping cervical cells emerged in recent years. In general, the main algorithms used in these systems can be roughly classified into the following algorithm groups: (1) graph-based methods [22; 23; 24; 25; 26], (2) morphological processing-based methods [27; 28; 29] and (3) level set method (LSM)-based approaches [30; 31; 32; 33]. Graph-based methods, such as Voronoi diagram method [23], provide a new idea for cervical cells segmentation; however, the segmentations look unnatural. Morphological processing-based methods are simply to be used but rely on the parameter configuration.