Patent Description:
Prostate cancer malignancy grading depends of the accurate identification and classification of the glandular components in the prostate tissue. In the grading system according to Gleason, malignant glands are classified into benign, PIN, and grades <NUM>-<NUM>, although the grades <NUM> and <NUM> are no longer in use. In the new ISUP grade group system [<NUM>] [<NUM>], there are five grade groups, ranging from discrete well-formed glands to tissue that lacks gland formation with or without poorly formed/fused/cribriform glands. Regardless of the grading system, it is necessary that the pathologist identify the morphological variations in the glandular components as they are closely linked to the aggressiveness of the cancer.

Digital pathology is an emerging field, where glass slides are scanned and stored as digital images for improved workflow, computer-aided analysis, and storage and management of the data. Digital pathology facilitates remote consultation of experts across the world and may alleviate the pathologist deficit that is anticipated in most countries from population growth and increase in disease incidence. Once tissue slides are digitized, it is possible to enhance the resulting images digitally and also extract information to support the pathologists' decision process. This has the potential to reduce the intra-and inter-observer diagnostic variation and improve the prognostication, thereby improve patients' quality of life and reduce the healthcare burden from needless treatment. But computer-aided analysis of tissue data requires high-quality image data, where the tissue components are clearly delineated and where the stain variations and noise are kept to a minimum.

Pathologists rely on multiple, contrasting stains to analyze tissue samples, but histological stains are developed for visual analysis and are not always ideal for automatic analysis.

Earlier work described a methodology to compare different histological stains for classification of components in histological tissue. This methodology was used to evaluate stains for both supervised and unsupervised classification which showed that certain stains consistently outperform others according to objective error criteria [<NUM>].

Earlier work also describes an automatic method (the Blind Color Decomposition, BCD, method) for highly accurate blind color decomposition of histological images into density maps, one for each stained tissue type [<NUM>]. The method decouples intensity from color information and bases the decomposition only on the tissue absorption characteristics of each stain. The method also models biochemical noise, as well as noise from the CCD (charge-coupled device) array in the microscope. Careful selection of the tissue stain combined with BCD color decomposition lays the foundation for accurate computer-aided processing of tissue data [<NUM>]. These techniques form the basis for gland segmentation of histological tissue.

<NPL> discloses an automated gland segmentation and classification method used for automated Gleason grading of a prostatic carcinoma tissue image. In comparison to the other segmentation-based methods, the method combines the similarity of morphological patterns associated with a grade with the domain knowledge such as the appearance of nuclei and blue mucin for the grading task.

<NPL> discloses the advantages of Sirius-hematoxylin (Sir-Htx) stain over Hematoxylin-Eosin (H&E) for blind color decomposition. When compared to ground truth defined by an experienced pathologist, the relative root-mean-square errors of the color decomposition mixing matrices for Sir-Htx are better than those for H&E by a factor of two, and the Pearson correlation coefficients of the density maps resulting from the decomposition of Sir-Htx-stained tissue gives a <NUM>% correlation with the ground truth.

The method according to the invention identifies individual glands in prostate tissue image data for the purpose of classifying these glands into categories, including benign, prostatic intraepithelial neoplasia (PIN), and grading patterns as defined by Gleason or by the ISUP new grade group system. A microscope captures histological tissue image data from a tissue sample that is stained with at least one stain, said stain being light absorbent and stains the stroma so that it is well differentiated from other tissue in the sample. The method transforms the histological tissue image data into at least one density map, said density map corresponding to the stroma in the histological tissue image, preferably according to <CIT>[<NUM>]. From the stromal density data, the method according to the invention identifies the glands by utilizing morphological operations to find a mask, said mask corresponding to non-stromal regions, and to find one seed in each disconnected or weakly connected region in said mask. Furthermore, the method according to the invention grows the seeds until said seeds meet said mask and identifies at least one gland by applying said grown seeds to the histological tissue image. The method according to the invention may also utilize a second stain, said stain being light absorbent, and said stain being absorbed primarily by the epithelium, and transform the histological tissue image data to an epithelial density map to remove small objects that are not part of a glandular structure. To improve the segmentation, this second density map may also be used in combination with the stromal density map to refine the non-stromal regions mask.

The invention defines a method for identifying and classifying glands in histological tissue image data according to claim <NUM>.

The image data comprises a set of pixels. The method comprises the steps of:.

The method according to the invention may also utilize a second stain, said stain being light absorbent, and said stain being absorbed primarily by the epithelium, and a second density map is derived representing the epithelium in said histological tissue image data. Both the first and second density maps are preferably derived using the BCD method, although other methods are possible to use. According to one embodiment of the invention, the mask covers low-density regions in a combination of the two density maps, wherein said combination is the pixel-by-pixel subtraction of the epithelial density map from the stromal density map.

According to one embodiment of the invention, the mask identification further comprises the steps of:.

The morphological opening may use adaptive techniques, preferably employing a tensor-based elliptical structuring element. The thresholding may preferably employ gradient maximization techniques, but other thresholding techniques may also be used.

According to one embodiment of the invention the finding of said seed further comprises the steps of:.

The morphological erosion may preferably employ adaptive techniques. The adaptive erosion may preferably employ a tensor-based elliptical structuring element. The thresholding may preferably employ gradient maximization techniques, but other thresholding techniques may also be used. According to one embodiment of the invention the region-growing preferably employs watershed techniques.

The image capture and analysis apparatus according to claim <NUM> comprises:.

Thanks to the present invention it is possible to provide an automated and reliable segmentation of glandular structures in histological tissue images, which is a prerequisite for a computer-aided tool for the classification of prostate cancer glands into their categories. In contrast to methods in the literature, the method according to this invention identifies glandular structures of all malignancy grades.

One advantage of the method according to the invention is that it can be adapted easily to different stains and staining methods, provided the stain allows a good differentiation of the glandular structures from the surrounding stroma.

A further advantage is that this method generalizes to the segmentation and classification of other types of histological tissue which has a distinct glandular structure.

Preferred embodiments of the invention are described with reference to the accompanying figures, wherein.

In the following, the focus is on prostate cancer tissue, but the method of the invention may be applied to other histological tissue data.

Malignancy grading of the prostate relies heavily on changes in the glandular architecture. A healthy prostate comprises branched ducts and glands, with two layers of cells (<FIG>). Malignant glands of low grade are regular in size with a central lumen, surrounded by one layer of epithelial cells with the nuclei located basally (<FIG>). These individual, discrete well-formed glands belong to grade <NUM> on the Gleason scale [<NUM>]. When the cancer progresses in degree of malignancy, the glands lose uniformity in size and shape and the inter-glandular distances becomes more variable. These glands are referred to as poorly-formed or fine caliber Gleason grade <NUM> (<FIG>). Other types of grade <NUM> glands form cribriform structures with multiple luminae (<FIG>), or fuse into irregular structures (<FIG>). Intra-ductal carcinoma, which can form cribriform structures, is also considered to be of grade <NUM> in the Gleason system. Finally, in Gleason grade <NUM> we see a lack of gland formation, with individual cells or files of epithelial cells (<FIG>). Cribriform structures with necrosis are also considered of highest grade. In summary, all glands have an epithelium with at least one epithelial nucleus, surrounded by stroma.

Prostate gland segmentation is a key component in prostate tissue classification. From the glandular structures it is possible to extract glandular features that are known to be linked to malignancy, such as number of luminae, nuclear crowding, and roundness of the glands and their luminae, and are used by pathologists in routine practice for grading.

There are many examples in the literature of prostate gland segmentation as part of automatic malignancy grading systems. Naik et al. [<NUM>] find the lumen using color information and use the lumen boundary to initialize level set curves which evolve until they reach the epithelial nuclei. The final glandular structure includes only the lumen and the epithelium without the nuclei. Nguyen et al. [<NUM>] also start with the lumen and grow that structure to include the epithelial nuclei. These methods work from the lumen out to a layer of epithelial nuclei, and can thus successfully find only benign glands, glands of Gleason grade <NUM>, and some poorly formed glands of grade <NUM>, but cannot identify cribriform structures and grade <NUM>. Vidal et al. use level sets and mean filtering to extract regions of interest in prostate tissue, but do not accurately segment individual glands [<NUM>]. Peng et al. employ principal component analysis, K-means clustering, followed by region growing to segment prostatic glands [<NUM>]. The authors state that finding high-grade cancer is difficult and also not necessary for finding cancerous foci. This is however not always true, since in more aggressive cases, fine caliber <NUM> and grade <NUM> may appear without surrounding lower grade cancer. There are many recent attempts to apply deep learning to tissue segmentation, as for example done by Xu et al. Tabesh et al. use a different approach identifying small objects in the prostate tissue with similar characteristics which are used directly for classification of cancerous and non-cancerous tissue, without identification of the underlying glandular structure [<NUM>]. In summary, without the glandular structures it is impossible to identify all the Gleason grades shown in <FIG>.

It is clear that to automatically identify all glandular patterns shown in <FIG>, an algorithm must work from the stromal border and in, not from the lumen out. However, traditionally prostatic tissue is stained with Hematoxylin-Eosin [<NUM>], which gives poor differentiation between epithelium and stroma, as both are stained in shades of red/pink by eosin. But, an accurate prostate gland segmentation algorithm that works for all types of prostate glands requires a stain with good differentiation between glandular epithelium and stroma.

<FIG> illustrates the prostate cancer grades according to Gleason. The histological tissue image in <FIG> shows a benign gland, FigurelB shows well-formed glands (Gleason grade <NUM>), <FIG> shows poorly formed glands (Gleason grade <NUM>), <FIG> shows a cribriform gland (Gleason grade <NUM>), <FIG> show small fused glands (Gleason grade <NUM>), <FIG> shows large fused glands (Gleason grade <NUM>), <FIG> shows intraductal carcinoma (Gleason grade <NUM>), and <FIG> shows poorly formed glands and single cells (Gleason grades <NUM> and <NUM>).

Referring to <FIG>, according to the present invention, the histological image data is segmented into its glandular components, in accordance with the method described in <FIG>. The histological tissue in <FIG>, which has been captured from a histological tissue sample stained with Picro-Sirius red Hematoxylin [<NUM>], is decomposed into two density maps corresponding to the epithelial tissue (<FIG>) and the stromal tissue (<FIG>), according to known methods of producing density maps, preferably according to the BCD method. It should be noted that the stain combination used herein is one of several stain combinations suitable for the method according to the invention; other stain combinations that provides a good differentiation between the stroma and neighboring tissue may also be used, such as Mallory's trichrome. In accordance with the present invention, and with reference to <FIG>, <FIG> shows the stromal density map after morphological opening, and <FIG> the non-stromal mask. In accordance with the present invention, and illustrated in <FIG>, <FIG> shows the result of the removal of non-epithelial objects. <FIG> shows the result of morphological erosion of the stromal density map; <FIG> the resulting seeds; <FIG> individual seeds superimposed on the non-stromal mask; and <FIG> shows the result obtained by applying a watershed with seeds in <FIG> and mask in <FIG> on the image in <FIG>.

Referring to <FIG>, according to the method of the present invention, histological tissue image data, which has been acquired with a microscope from a tissue sample, classifies prostate glands into categories. The method for classification of a gland into a category comprises steps to.

Referring to <FIG>, according to the method of the present invention, histological tissue image data which has been acquired with a microscope from a tissue sample, is segmented into at least one gland. The method for segmentation of the histological tissue image into glands comprises steps to.

It should be noted that the steps <NUM> and <NUM> may be performed in any order.

<FIG> illustrates schematically an image capture and analysis apparatus suitable for carrying out the invention. This schematic is for illustration purposes only and those skilled in the art will appreciate that many other system configurations are possible. The image capture and analysis apparatus comprises:.

In one embodiment of the invention, the image capture system apparatus is adapted to capturing histological tissue image data from a tissue sample that has been stained with at least one stain, the said stains being light absorbent and is absorbed by stroma.

In one embodiment of the invention, the computer system is adapted to execute the steps of the method herein.

In steps <NUM> and <NUM>, the above described image capture system apparatus is used to record the histological tissue image data from a tissue sample stained with one or more stains.

In step <NUM>, the method derives a stromal density map from the tissue image data, preferably using the Blind Color Decomposition (BCD) method, but other methods, such as non-negative matrix factorization, may also be used.

In step <NUM>, the method finds the boundary of at least one gland using the stromal density map, preferably using morphological operations, but other methods may also be used.

In step <NUM>, the method utilizes said boundary in the density map to find the corresponding gland in the histological tissue data.

In step <NUM>, the glands are classified into categories based on the glands' associated features. The classification of a gland into a category may be determined based on its morphology, said morphology defined by features, including, but not limited to number of luminae, nuclear crowding, and roundness of the glands and their luminae. Also, the classification of a gland into a category is determined by the content of said gland.

In step <NUM>, the method derives a stromal density map and optionally an epithelial density map from the histological tissue image data, preferably using the Blind Color Decomposition (BCD) method, but other methods, such as non-negative matrix factorization may also be used.

In step <NUM>, the method identifies a mask, said mask covering the low-density regions in said stromal density map, that is said mask covering non-stromal regions. To find said mask, the method preferably applies an adaptive morphological opening, preferably with tensor-based elliptical structuring elements [<NUM>], to said stromal density map, with reference to <FIG>. The local structure tensor adapts elliptical structuring elements to lines in regions of strong single-directional features and to disks where the stroma has no prevalent direction. An adaptive filter which varies depending on the local image structure ensures correct separation of distinct glands without removal of small glands.

The method further utilizes the contrast between stromal tissue and non-stromal tissue in the stromal density map to ensure a good separation between said stromal and non-stromal regions. To accomplish said separation, the morphological opening applied to the stromal density map is followed preferably by the use of gradient maximization thresholding to arrive at a binary representation of the non-stromal mask, with reference to <FIG>. The gradient maximization thresholding technique finds the threshold that best separates the stromal and non-stromal component by maximizing the mean of the Sobel operator along the boundaries of the binary representation of the mask.

To improve the identification of the non-stromal tissue, the epithelial density map may be combined with the stromal density map by subtracting the epithelial density map from the stromal density map, pixel-by-pixel. By identifying the mask from the combined density maps, the glandular boundaries become more accurate.

The method further removes objects without epithelial content, by referring to said epithelial density map corresponding to said stromal density map, with reference to <FIG>.

The binary regions in the non-stromal mask are either disconnected, or weakly connected that is connected by only a few pixels. In step <NUM>, the method finds one seed for each disconnected or weakly connected region in said mask, said seeds being contained in said regions. The seed is obtained by eroding said stromal density map using the adaptive filter with reference to step <NUM> above, and with reference to <FIG>.

The method further utilizes the contrast between stromal tissue and non-stromal tissue in said stromal density map after erosion to ensure a good separation between said stromal and non-stromal components preferably by the use of a thresholding method to arrive at a binary representation of the seeds, with reference to <FIG>. This thresholding method may utilize the gradient maximization technique with reference to step <NUM> above, but other techniques may be used. The erosion will separate glands that are weakly connected, but will not remove small glands. <FIG> shows the seed (one level of grey for each seed) overlaid on the non-stromal mask with reference to step <NUM> above.

In step <NUM>, the method grows the seeds until said seeds meet said mask. The method preferably utilizes the watershed method [<NUM>] for growing said seeds towards said non-stromal mask, but other region growing techniques may be employed. The final segmentation mask for the individual glands, with reference to <FIG>, is obtained by applying a watershed algorithm initialized with the seeds, with reference to <FIG>, and said non-stromal mask as boundary, also with reference to <FIG>.

Claim 1:
A method for identifying and classifying glands in histological tissue image data from a tissue sample, wherein said image data comprising a set of pixels, said method comprising the step of:
- capturing (<NUM>/<NUM>) of histological tissue image data from a tissue sample that has been stained with at least one stain, wherein said at least one stain being light absorbent, and said at least one stain being absorbed primarily by the stroma;
- segmenting (<NUM>) into at least one gland, wherein a gland is surrounded by stromal tissue;
- classifying (<NUM>) said at least one gland into a category;
the method is characterized by:
- deriving (<NUM>/<NUM>) at least one stromal density map, wherein said at least one stromal density map corresponding to the portion of the histological tissue image data that represents the stroma and wherein said at least one stromal density map is derived using a blind color decomposition method;
- wherein segmenting is done for the at least one stromal density map by:
- identifying (<NUM>) a mask, wherein said mask covering low-density regions in said at least one stromal density map, that is said mask covering non-stromal tissue, utilizing said at least one stromal density map and a contrast between the stromal tissue and non-stromal tissue in said stromal density map;
- finding (<NUM>) one seed for each disconnected or weakly connected region in said mask, said seeds being contained in said regions;
- growing (<NUM>) said seeds until said seeds meet said mask;
- utilizing (<NUM>) the boundaries of said grown seeds to identify at least one gland in the histological tissue image data.