Patent Description:
A number of approaches have been reported to address spine localization and segmentation. For example, in Glocker et al. [Reference <NUM>], given sparse annotation of vertebra center, vertebra localization is achieved by random forest and mean shift clustering. Kelm et al. [Reference <NUM>] also describes an iterative marginal spacing learning algorithm for spine detection. Boykov et al. [Reference <NUM>] describes a graph-cut method for spine segmentation.

Kelm et al. [Reference <NUM>] and Asem et al. [Reference <NUM>] describe a graph-cut method combined with shape information for spine segmentation.

Aslan et al. [Reference <NUM>] describes shape prior constraints fused into a graph-cut framework. Asem et al. [Reference <NUM>] also describes a graph-cut method to incorporate both appearance models and shape constraints approaches.

Shape statistic analysis has also been applied in spine segmentation. In Ma and Lu [Reference <NUM>], a shape deformable model is studied with learning based boundary detection. A registration-based method is applied with statistical multi-vertebrae anatomical shape with a pose model in Rasoulian et al. [Reference <NUM>]. Additionally, spine canal segmentation is conducted by random walk algorithm in Wang et al. [Reference <NUM>].

<CIT> discloses a method and apparatus for automatic spinal canal segmentation in medical image data. An initial set of spinal canal voxels is detected in the 3D medical image using a trained classifier. A spinal canal topology defined by a current set of spinal canal voxels is refined based on an estimated medial line of the spinal canal. Seed points are sampled based on the refined spinal canal topology. An updated set of spinal canal voxels is detected in the 3D medical image using random walks segmentation based on the sampled seed points. The spinal canal topology refinement, seed points sampling, and random walks segmentation are repeated in order to provide cascaded random walks segmentation to generate a final spinal canal segmentation result.

<CIT> discloses a framework for localizing anatomical structures. In accordance with one aspect, the framework receives a learned regressor and image data of a subject. The learned regressor may be invoked to predict a first spatial metric from a seed voxel to a target anatomical structure in the image data. The learned regressor may further be invoked to predict second spatial metrics from candidate voxels to the target anatomical structure. The candidate voxels may be located around a search region defined by the first spatial metric. The candidate voxel associated with the smallest second spatial metric may then be output as a localized voxel.

<CIT> discloses a method for segmenting vertebrae in digitized images includes providing a plurality of digitized whole-body images, detecting and segmenting a spinal cord using 3D polynomial spinal model in each of the plurality of images, finding a height of each vertebrae in each image from intensity projections along the spinal cord, and building a parametric model of a vertebrae from the plurality of images. The method further includes providing a new digitized whole-body image including a spinal cord, fitting an ellipse to each vertebrae of the spinal cord to find the major and minor axes, and applying constraints to the major and minor axes in the new image based on the parametric model to segment the vertebrae.

<NPL> discloses a deformable model for automatic segmentation of prostates from three-dimensional ultrasound images, by statistical matching of both shape and texture.

The above described references have a number of drawbacks some of which include not efficiently detecting anatomical features, searching too large a search space, requiring complicated false positive removal strategies, or requiring extensive manual annotation to build the shape information.

Accordingly, there exists a need for a method and system to segment the spine in 3D CT with increased speed and accuracy.

Aspects of the present invention provide a method for identifying an anatomic structure of a patient as set out in claim <NUM> and a system for segmenting an anatomic structure of a patient as set out in claim <NUM>.

Further preferred and optional features of the invention are set out in the dependent claims.

In embodiments, the anatomic structure is a plurality of interconnected components. Examples of interconnected components include, for example, the digits of the spine such as L1, L2,. or in embodiments the ribs.

In embodiments, the centerline is the rib centerline. The centerline may be detected based on machine leaning.

In embodiments, the identifying is performed by a computer and based on an annotated exemplary anatomical structure.

In embodiments, input is CT data, and preferably 3D CT image data.

In embodiments, the output spine segmentation is sent to a display.

In embodiments, a method for identifying an anatomic structure of a patient comprises computing a constrained region encompassing the anatomic structure from a 3D image data set of the patient; automatically segmenting the anatomic structure based on the above computing step; and identifying the anatomic structure.

In embodiments, the computing step is based on a bounding box algorithm.

In embodiments, the computing step is based on detecting a centerline of the anatomic structure from the 3D image data set of the patient.

In embodiments, the centerline is a spine centerline, and the detecting also comprises detecting the spinal canal of the patient.

In embodiments, the computing step comprises computing a cylindrical shaped constrained region that encompasses the spinal canal and spine centerline.

In embodiments, the method further comprises localizing a disc center and vertebra center based on the computing step.

In embodiments, the method comprises constructing foreground and background constraints for a spine digit based on the localizing step.

In embodiments, the anatomical structure is a rib. And the method comprises constructing hard constraints for each rib.

In embodiments, the method comprises constructing foreground and background constraints for the rib based on the rib centerline.

Benefits of the subject invention include: increasing the speed, reducing the search space, and boosting the performance of detection.

The description, objects and advantages of the present invention will become apparent from the detailed description to follow, together with the accompanying drawings.

Before the present invention is described in detail, it is to be understood that this invention is not limited to particular variations set forth herein as various changes or modifications may be made to the invention described and equivalents may be substituted without departing from the scope of the invention. As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope of the present invention. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process act(s) or step(s) to the objective(s), scope of the present invention. All such modifications are intended to be within the scope of the claims made herein.

Methods recited herein may be carried out in any order of the recited events which is logically possible, as well as the recited order of events. Furthermore, where a range of values is provided, it is understood that every intervening value, between the upper and lower limit of that range and any other stated or intervening value in that stated range is encompassed within the invention.

All existing subject matter mentioned herein (e.g., publications, patents, patent applications and hardware) is incorporated by reference herein in its entirety except insofar as the subject matter may conflict with that of the present invention (in which case what is present herein shall prevail).

More specifically, as used herein and in the appended claims, the singular forms "a," "an," "said" and "the" include plural referents unless the context clearly dictates otherwise. Last, it is to be appreciated that unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

<FIG> illustrates an overview of a spine segmentation framework <NUM>.

Block <NUM> shows input CT data such as 3D CT image data of a patient.

Block <NUM> states to detect the spine centerline. Additionally, in embodiments, an augmented spine constrained region <NUM> is estimated based on the spine centerline result.

Block <NUM> states to detect the spine canal centerline.

Block <NUM> states to localize the vertebra center and intervertebral disc center. As discussed further herein, the localization step <NUM> may be estimated by probabilistic inference.

Block <NUM> states to construct or build foreground and background constraints. As discussed further herein, in embodiments, this step may include constructing case-specific foreground and background constraints for each vertebra digit in the segmentation framework. Each vertebra digit may be segmented based on the constraints.

Block <NUM> is the spine segmentation result. The spine segmentation <NUM> includes, as discussed further herein, an accurate segmentation of a combination or plurality of vertebrae. In embodiments, the entire spine is segmented.

<FIG> illustrates a spine segmentation system <NUM>. The system <NUM> shown in <FIG> includes a processor <NUM> operable to segment the spine based on various data and information as will be described in more detail below.

System <NUM> is shown having a memory device <NUM> which receives, holds or stores various information including, for example, 3D CT imaging data. In embodiments, 3D CT image data is in DICOM format.

The system <NUM> shown in <FIG> includes a user input device <NUM> such as, for example, a keyboard, joystick, or mouse. The user input device allows a user such as the physician to add or input data and information as well as modify the result and to make notes in the files and records.

In embodiments, the physician may make annotations to the data such as, as described further herein, identify anatomical features and components of the spine model.

The system <NUM> shown in <FIG> also includes a display <NUM> which may present reports, data, images, results and models in various formats including without limitation 3D visualizations and standard medical imaging views.

It is to be understood, however, that although the system in <FIG> is shown with a memory <NUM> for receiving and storing various information the invention is not so limited. In an alternative embodiment the system may be configured to merely access a memory device such as a USB stick, a CD, drive, or other media storage device.

In another embodiment the processor is connectable to a memory device <NUM> through the internet or through another communication line to access a network. For example, patient data CT scans may be stored on a server of a hospital and the processor of the instant application is adapted to access such data via a communication line and process the data.

Displays may be incorporated with the processor in an integrated system or the displays may cooperate with the processor from a remote location. A processor may be adapted to send or deliver data across a network to one or more displays or portable computer devices, tablets, or smart phones such as the iphone® manufactured by Apple, Inc. Cupertino, CA, United States. Indeed, although the computer system shown in <FIG> includes a number of various components incorporated into a system, the invention is not so limited. The invention is intended to be limited only as defined in the appended claims.

Now with reference to <FIG>, an overview of a method for segmenting the spine is described. Spine segmentation method <NUM> includes: detection of anatomical features <NUM> (e.g., detecting and extracting the spine centerline and spine canal centerline); localization <NUM> (e.g., localization of the vertebra and disc centers); and segmentation <NUM> (e.g., segmentation of each spine digit based on the spine vertebra localization results). Input <NUM> includes volumetric image data of the patient such as 3D CT data.

In embodiments, the step of detection includes extracting the spine centerline and spine canal centerline.

Spine centerline extraction uses machine learning algorithms to compute a probabilistic map. In embodiments, as described in more detail herein, a voxel-wise learned classifier is applied. Physicians or experts annotate the spine centerline of samples.

Next, and given the annotation of the spine centerline by the physicians or expert, all the voxels on the spine centerline are treated as positive (spine centerline) samples. Other voxels are negative samples in the training. For an unknown volume I, the learned classifier assigns each voxel x a prediction value p(c(x)) which represents the likelihood of voxel x being on centerline. A prediction map may be denoted as Pc.

<FIG> show the prediction results of an unknown volume. In particular, <FIG> show results of spine centerline detection by a machine learning algorithm in the form of a probability map or a prediction map. <FIG> show the prediction results in a sagittal view and a 3D visualization view, respectively. Voxels with prediction value (p(c(x)) > <NUM>) are highlighted as brighter (white). In other embodiments, voxels with prediction values greater than a user defined threshold value (e.g., > <NUM>, <NUM>, etc.) may be highlighted as brighter (white).

An aim of the spine centerline extraction is to find the path which has the maximum prediction values. By reversing the prediction result, the problem may be solved using generalized shortest path. The reversed probability map is written as <MAT>.

To compute the shortest path, the length of a discrete path is defined:
<MAT>.

where Γ is an arbitrary discrete path with n voxels given by a set of voxels {x<NUM>,. The shortest distance between voxels xa and xb is then written as: <MAT>
where Pa,b represents the set of all discrete paths between these two voxels.

The key ingredient of the generalized shortest path is the distance dg(xp,xq) between voxels xp and xq: <MAT>.

<NUM>, the first term <MAT> is a finite difference approximation of the gradient on P^c between the voxels (xp,xq); <MAT> (xp,xq) is the average responses on <MAT> between (xp,xq); and de(xp,xq) denotes the Euclidean distance between these two voxels. Parameters (α,β,γ) are applied to balance these three terms.

To find the generalized shortest path on <MAT>, a source voxel and a sink voxel is specified. In embodiments, a strategy automatically computes these terminals. The strategy takes all the voxels S at z=-<NUM> plane as seed points. Then, a multiple source shortest path algorithm is conducted from S along z direction. The multiple source shortest path algorithm stops when a visiting voxel x*'s neighbor is out of the image. x* is treated as sink xt. Finally, the source xs is indentified by tracing back from x*. An example of a shortest path algorithm is described Skiena [Reference <NUM>].

The spine canal centerline may be extracted similar to the spine centerline extraction.

<FIG> illustrate components of a spine vertebra model. The model comprises a spine centerline <NUM>, vertebra centers <NUM>, intervertebral disc centers <NUM>, and spine canal centerline <NUM>. The spine vertebra localization described herein finds each component in the spine vertebra model. For each component, and given the manual annotations by the physician mentioned above, a voxel-wise classifier is learned by a supervised machine learning algorithm. For example the following algorithms may be used: Boosting algorithm (as described in Reference <NUM>, <NUM>), Random forest (as described <NPL>)), and support vector machine (SVM) (as described in<NPL>).

In embodiments of the subject invention, the spine centerline is used to assist in determining the vertebra center localization. In embodiments, a constraint is that the vertebra center is on the spine centerline and vertebra digit center detection is a probabilistic inference problem.

Mathematically, given an unknown 3D volumetric CT image I, our goal is to localize the vertebra centers. Let v(x) be one vertebra center, where x =(x,y,z) is the 3D coordinate in image I. The localization task is to find a set of vertebra centers V = {vk(x)|k = <NUM>. Nv} which are present in the image I. The spine centerline is denoted as C = {ci(x)|i = <NUM>. Nc}, where ci(x) is one spine centerline point located at x in image I.

The probability of voxel x being vertebra center is defined by: <MAT>
where p(c(x)) is the probability of voxel x being on centerline and p(v(x)|c(x)) is the conditional probability of being vertebra center given p(c(x)).

The vertebra center is determined by an iterative hierarchical searching. Specifically, the spine centerline is first extracted as described above. An augmented spine constrained space is created. The constrained space is used for searching for the vertebra centers.

An example of the constrained searching space is the augmented spine constrained region <NUM> shown in <FIG>. Vertebra centers are localized in the constrained search region <NUM>. The proposed probabilistic inference algorithm reduces the searching space and also boosts the performance of vertebra center localization.

With reference to Eqn. <NUM>, the vertebra center localization is formalized in a probability inference framework. An augmented spine constrained region Rcst is created near spine centerline. A voxelwise vertebra center classifier is also trained from manual annotations. The classifier predicts each voxel x ∈ Rcst a probability p(v(x)) it being vertebra center.

The vertebra centers may be estimated using a mean shift algorithm such as that described in, for example, [References <NUM> and <NUM>]. Seed points are limited to the spine centerline. In embodiments, for a seed point x, the vertebra center is estimated by: <MAT>
where Gaussian kernel K parameterized with σv and xi ∈ S are local neighbors of x.

A number of benefits and advantages arise from the subject invention. The seed points of the subject invention are more reliable than that described in Glocker et al [Reference <NUM>] where the seed points are randomly initialized. Additionally, the use of the probabilistic inference boosts the localization performance. Yet another benefit is that a complex false positive removal strategy is not required. Instead, in embodiments of the subject invention, use of a threshold removes the false positives. The threshold may be based on average response of m(x) within σv.

Localization of the intervertebral disc center <NUM> is carried out in a similar manner to the algorithm used to localize the vertebrae center <NUM>.

In other embodiments, localization of the anatomic structure is carried out by other techniques including, for example, bounding box localization. An example of bounding box localization is described in Criminisi et al. [Reference <NUM>].

In embodiments, the step of spine segmentation comprises constructing case-specific foreground and background constraints based on the results of spine vertebra localization. By "case-specific", it is meant that the foreground/background constraints are constructed for each individual spine digit.

Constraints include spine centerline and spine canal centerline. The location of the candidate vertebra is limited to a tube-like region. The tube-like region also roughly limits the size of vertebra. The detected vertebra center and intervertebral disc centers also serve as foreground and background hard constraints for spine segmentation.

<FIG> provides an example of the above mentioned hard constraints for one digit. For each vertebra digit, a tube-like region <NUM> is constructed by the spine centerline and canal centerline. Foreground (<NUM>) are voxels near the estimated vertebra center. Two cone-like background regions (320a,b) are created from its adjacent vertebra centers and intervertebral disc centers. Other voxels within the tube-like region are unknown.

<FIG> is the corresponding segmentation of the vertebra digit <NUM> shown in <FIG>, shown in the sagittal view.

Note that compared to [Reference <NUM>], where seed points are randomly initialized, the seed points of the subject invention are more reliable. Also, the probabilistic inference can boost the localization performance. Another benefit is that the subject methods do not require complex false positive removal strategy. A simple threshold is sufficient to remove the false positives. The threshold is based on average response of m(x) within σv.

The proposed hard constraints can be applied to segmentation framework using a wide variety of techniques including, e.g., graph-cut and level-set as described in references <NUM> and <NUM>, respectively. In an example described herein, a graph-cut algorithm is applied to demonstrate the effectiveness of the proposed foreground and background constraints.

The set of voxel-labeling is denoted by f = {f<NUM>,. Each label fi of foreground or background for voxel xi is written as fi ∈ {<NUM>,<NUM>}. By having sufficient hard constraints, the Graph-cut is simplified to only rely on pair-wise interaction. The energy function of the segmentation can then expressed by:
<MAT>
where V(fp, fq) represents the penalty for the discontinuity of labels between voxels xp and xq. de(xp,xq) denotes the Euclidean distance between these two voxels. δ(·) is an indicator function.

Unlike in Kelm et al. [Reference <NUM>], the hard constraints and simplified energy minimization of the subject invention aid in avoiding over-segmentation of the spine canal and adjacent vertebra.

The method described herein was tested on a dataset that included <NUM> volumetric CT data for lung cancer procedure.

The CT size is about <NUM>× <NUM> × <NUM> voxels. The resolution of the data is around <NUM> mm × <NUM> mm×<NUM> mm.

For each data, experts manually annotated the spine centerline, canal centerline, vertebra centers and intervertebral disc centers.

A <NUM>-fold cross-validation was applied in the experiments. In each cross validation, <NUM> data volumes were used for training and the other <NUM> data volumes were used for testing.

A voxel-wise classifier was trained for each component of the spine vertebra model. We used a machine learning (e.g., boosting trees algorithm) and a box-like feature (e.g., 3D Haar feature) to capture local contrast. In this example, we used a boosting tree (namely, Adaboost) as described in References <NUM> and <NUM>.

In the experiment, given the spine centerline C = {ci(x)|i = <NUM>. Nc}, the spine centerline classifier was trained by positive training samples xp and negative training samples xn: <MAT>
where xc ∈ C be an annotated location of spine centerline. Parameters τp = <NUM> and τn = <NUM> (in mm) are used to control the training samples generation.

Training samples of vertebra centers and intervertebral disc centers were extracted from constrained space Rcst.

The localization error (in mm) of vertebra center and intervertebral disc center were used to evaluate the method. The statistical results are listed in Table <NUM>, shown below. The average localization error for both vertebra center and intervertebral disc center are about <NUM>.

Segmentation was performed with a <NUM>-neighbor for the pair-wise energy in Eqn. <NUM>, shown above. σ = <NUM> is set for all the volumes. In the experiment, segmentation was achieved for each of the annotated vertebra.

The segmentation results are shown in <FIG> are sagittal views of various spine segmentation results. <FIG> are 3D visualizations of the spine segmentation results shown in <FIG>, respectively. The experimental results demonstrate the above described method is effective for a complete spine localization and segmentation.

Described herein is a novel system and method for spine segmentation. Spine segmentation has a wide range of applications including without limitation screening and diagnosis for suspect tissues, treatment planning, and therapies such as excision or ablation of tumors.

Claim 1:
A method (<NUM>, <NUM>) for segmenting an anatomic structure of a patient, wherein the anatomic structure is the spine and comprises a plurality of stacked interconnected components (<NUM>, <NUM>) being vertebrae, a spine centreline (<NUM>), and a spinal canal centreline (<NUM>), the method comprising:
receiving a 3D image data set (<NUM>, <NUM>) of the patient;
detecting (<NUM>, <NUM>, <NUM>) the spine centreline (<NUM>) and the spinal canal centreline (<NUM>) of the anatomic structure from the 3D image data set of the patient wherein the detecting (<NUM>, <NUM>) is based on a prediction map, wherein the spine centerline extends through the center of each of the plurality of interconnected components (<NUM>, <NUM>);
localizing (<NUM>, <NUM>) a center of each of the plurality of interconnected components; and
automatically segmenting (<NUM>, <NUM>) the anatomic structure based on the detecting and localizing steps (<NUM>, <NUM>, <NUM>).