BACKGROUND
quality assurance in diagnostic histopathology plays a critical role in development of a treatment plan for patients with prostate cancer  <cit> .
methods to integrate quality development, maintenance, and improvement of diagnostic accuracy are, hence, critical to cancer management in any setting.
in diagnostic prostate pathology, gleason grading  <cit>  is the most commonly used grading system that is based upon the structural patterns of the tumor.
the gleason grade is a primary determinant in treatment planning  <cit> .
however, it is well known that the grading of prostate tissues suffers from intra- and inter-pathologist variability ; for example, the exact intra-pathologist agreement was achieved in 43–78 % of the instances, and 36–81 % of the exact inter-pathologist agreement was reported.
it is also known that the variability of the grading can be reduced with focused retraining.
there could be many ways to educate pathologists such as meetings, courses, online tutorials, and etc  <cit> , but these are not time- and cost-effective and rarely implemented.
therefore, building an automated, fast, and objective method to aid pathologists in evaluating prostate can improve prostate cancer diagnosis.
when a pathologist evaluates a tissue sample, he/she looks at a stained tissue and mentally compares it against a fund of knowledge and experience and may consult publications when needed.
in essence, the pathologist is matching structural patterns with samples they have seen earlier and mentally recalling the diagnosis made such that they can make the same diagnosis in the specific test case.
despite training, intra- and inter-observer variation and controversial areas still exist  <cit> .
to aid and improve the diagnostic process, there have been several research efforts to develop automated systems for the detection and grading of prostate cancer.
the majority of the previous methods have used morphological features  to characterize and classify tissue samples into correct classes, and others have also used fourier transform  <cit> , wavelet transform  <cit> , and fractal analysis  <cit>  to extract texture features.
though these methods claim to be accurate, the information that pathologists will obtain by using such methods may be limited since these only provide the predicted grade in general.
the prediction also relies on the conditions of the training and testing datasets such as acquisition settings  <cit>  and staining  <cit> .
alternatively, content-based image retrieval  systems  have been proposed to aid cancer pathology.
the main objective is to effectively and efficiently manage an enormous amount of image data and to provide similar cases to a new test case that is examined.
in addition to clinical usage, cbir systems can help medical research, education, and training  <cit> .
the similar cases can be determined as owning the same grade  or sub-structures  <cit> ; for instance, single lumen glands, multi-lumen glands blood vessels, and lymphocytes in prostate  <cit> .
the basic premise of such systems in diagnostic histopathology is that tissue samples that have the same grade or similar characteristics and patterns with the sample of interest will afford useful information to pathologists and improve the decision-making process.
similar to cancer detection and grading systems, tissue is represented as several quantitative features such as morphology  <cit> , histogram  <cit> , color  <cit> , and texture .
the similar samples can be retrieved by computing distance metrics or similarity scores between a new case and the previously diagnosed or examined cases.
in order to improve tissue representation and retrieval, features are often post-transformed and/or their weights are adjusted in an implicit or explicit manner; for example, kernel function  <cit> , simplex method  <cit> , manifold learning  <cit> , boosting  <cit> , and self-organizing map   <cit> .
previous retrieval systems have been measured against a gold standard of diagnostic category and grade of tumor, defined by a pathologist.
prostate cancer is, in particular, a multifactorial disease and a mixture of heterogeneous growth patterns  <cit> , and hence tissues belonging to the same gleason grade may possess different cellular, nuclear, or glandular sub-patterns.
a number of histological variants, in fact, exist in prostate carcinoma and some of the variants cannot be addressed by the gleason grading system  <cit> .
moreover, the gleason grading system results in a tumor grade that correlates with overall outcomes , but fails to provide information on risk of metastasis, and correlates poorly with the clinical decision making process.
further, the gleason grading system has gone through several refinements over time  and may undergo further changes  <cit> .
these changes result variations among pathologists in practice  <cit>  and disrupt developing robust automated grading and retrieval systems.
here, we report developing a new computer information and management and decision-support system that consists of a database of pre-defined prostate tissues and a retrieval process .
the database retains tissue images, clinical information, and one or more measurements of the structure of tissue.
the retrieval process utilizes the structural/morphological features of the tissue sample image and provides tissue samples similar to the sample under consideration from the database.
in assessing tissue morphology, we utilize infrared  chemical imaging for providing cell type information in tissue  <cit> .
we previously reported its utility in stabilizing and improving the automated cancer detection  <cit> .
as a retrieval function, we adopt a machine learning ranking approach, called ranking-support vector machine   <cit>  in conjunction with a two-stage “feature selection” step  <cit> .
ranking-svm learns a ranking function of high generalization due to maximum-margin property  <cit> .
feature selection step finds the most informative subsets while preserving the essential characteristics of the data.
moreover, we propose to determine the ground truth tissue similarity based on various structural properties of tissue, not solely on gleason grade or a single structural component.
here, the structural properties are examined by pathologists.
combining different structural components of tissue ensures better characterization of tissue structure, and thus more accurate measurement of tissue similarity can be made.
thereby, the automated and computerized analysis and human experts’ assessment of tissue morphology are correlated through a machine learning approach.fig.
<dig> overview of system.
as a query is given, the closest matches and their clinical information are retrieved from the database .
provided with the matching cases, pathologists make a diagnosis , and updating may or may not be conducted .
q, d, ranking, f, and s denote a query, database, retrieval process, single feature, and subset of features, respectively
the rest of the paper is organized as follows.
in methods section, we begin with a description of the dataset and data preparation process.
in the following subsections, we describe the three key components of our new system – tissue similarity measure, tissue morphological feature extraction, and tissue retrieval function.
then, feature selection and balanced training are described.
in results section, the experimental results, including tissue similarity measure and tissue retrieval, via cross-validation are demonstrated.
in discussion section, the implications and limitations of our study are discussed.
finally, we conclude in conclusions section.
methods
samples and data preparation
this study and protocols were approved by the university of illinois institutional review board  and was conducted as per the permission of the irb in accordance with relevant guidelines and regulations.
we have obtained  <dig> prostate cancer tissue samples , composed of  <dig> ,  <dig> ,  <dig> ,  <dig> , and  <dig>  samples.
both hematoxylin and eosin  stained and ft-ir images are available for the samples.
tissue samples were first sectioned to ~5um thick sections, with a section being placed on a standard glass slide and a serial section on ir transparent baf <dig> slide.
stained with h&e, tissue images were acquired on a standard optical microscope at 40x magnification, and the size of a pixel is  <dig> um ×  <dig> um.
on ir transparent baf <dig> slides, ft-ir images were acquired at a spatial pixel size of  <dig> um ×  <dig> um and a spectral resolution of 4 cm- <dig> at an undersampling ratio of  <dig> using perkin-elmer spotlight imaging system.
the spectral profile of a pixel was truncated to a spectral range of 4000-720 cm- <dig>  detailed description of sample preparation and data acquisition for ft-ir imaging are available in fernandez et al.
<cit> .
clinical information  of the samples were prepared by pathologic review, and  <dig> morphological features were also extracted.
the database we build here, therefore, contains  <dig> tissue images  and their clinical information and  <dig> morphological features.
morphologic criteria and tissue similarity measure
we define  <dig> criteria to describe the architectural properties of tissue: 1) gland crowding, 2) gland roundness, 3) stromal reaction, 4) nuclear grade, 5) clefts 6) lumen/gland ratio, 7) gland continuity, 8) cell separation, and 9) gleason score.
the details of the criteria are listed in table  <dig>  some of the properties are the criteria used in the gleason grading system, and others were adopted from the literature.
although some criteria are overlapped with the gleason grading system, their importance and interpretation in our system may vary.
the gleason grading system may be also able to describe certain properties of tissues that cannot be characterized by the overlapping criteria.
in the gleason grading system, gland arrangement , variations in size and shape of gland , sheets of cells , and single cells  are examined.
nuclear morphometry  , reactive stroma  , and retraction clefting   <cit>  have been reported to be useful for prostate diagnosis and prognosis.
in the digital and computerized tissue analyses, structural features describing gland arrangement  <cit>  and shape  <cit>  and the size of gland and lumen   <cit>  have been adopted to characterize tissue.
individual cells also showed a moderate correlation with patient outcomes  <cit> .table  <dig> description of  <dig> morphologic criteria
agleason score is the sum of predominant and secondary scores.
in our set, it ranges from  <dig> to 10
for each of the criteria, a pathologist examines a tissue sample  and assigns a score ranging from  <dig> to  <dig>  or  <dig> to  <dig>  except gleason score which in our set of tissues ranges from  <dig> to  <dig>  the score range from  <dig> to  <dig> may be interpreted as none, low, and high, and the range from  <dig> to  <dig> may be considered as none, low, mid, and high.
due to its qualitative nature, it is difficult to highly stratify, and the impact and measurability of each criterion varies.
restricting the score range to none, low, mid, and high , in general, the scores are intended to be specific to differing morphologic patterns as well as to be reproducible by other pathologists.
using the scores of the  <dig> morphologic criteria, tissue morphologic similarity  between tissue samples is measured.
although well-defined and measured, the importance or relevance of each criterion differs.
for example, the significance of gland crowding score  <dig> may differ from that of gland roundness score  <dig>  and the difference between two samples having gland crowding score  <dig> and  <dig> may not be identical to the difference between two samples owning stromal reaction score  <dig> and  <dig>  in these cases, the absolute values of the scores and the difference of the scores are identical.
recognizing the intrinsic relationship between scores and criteria, we utilize the distribution of each criterion score over the samples in the database.
regardless of the absolute value of a score, two samples away from each other in the distribution of the scores of a criterion are likely dissimilar in terms of the criterion, and vice versa.
in other words, tissue similarity between two samples with respect to a morphologic criterion is related to the number of samples between them as ordered by the score for the criterion.
accordingly, let tms be the tissue morphologic similarity between two tissue samples d <dig> and d <dig> and computed as follows: tmsd1d2=∑i=19tmsid1d <dig> where tmsi is the morphologic tissue similarity for ith criteria.
tmsi is calculated as follows: tmsid1d2=1−∑s=sd1i+1sd2i−1his+12hisd1i+hisd2iz where sdi is the ith morphologic criterion score of a tissue sample d, hi is the number of samples having ith morphologic criterion score s, and z is a normalization factor.
due to normalization, tmsi ranges from  <dig> to  <dig>  1 ≤ i ≤  <dig>  thereby tms ranges from  <dig> to  <dig>  in this study, tms scores represent the true similarity between tissue samples and serve as the gold standard of tissue retrieval.
morphological feature extraction
in prostate cancer, epithelial cells  <cit> , which line ducts and acini in intact tissue in three-dimensional structures, are of great interest.
as cancer grows, epithelial cells grow  in and out of the glands in an uncontrolled way, and thus the structure of tissue, especially the local glandular structure, is distorted.
we also note that the role of stroma cells, connective cells supporting epithelial cells, in cancer tissue has been recently recognized  <cit> .
to quantify the alterations in tissue morphology, we focus here on the nuclear and cellular morphology of epithelial and stromal cells and lumens .
in order to quantify the nuclear and cellular morphology of epithelial and stromal cells and lumens , we first segment epithelium and stroma in tissue by adopting fourier transform infrared  spectroscopy imaging due to its accuracy and robustness  <cit> .
ft-ir has been extensively validated in classifying histologic cell types in tissue  <cit>  and provides a color coded cell type image of tissue.
cell type segmentation in h&e images is challenging due to limited information, color variations, etc.
rigid-body image registration overlays the epithelium and stroma segmentation from ft-ir imaging with the corresponding h&e image by using outer shape and empty space  in tissues  <cit> .
second, lumens and nuclei are identified from h&e images by considering their color intensities and geometric properties  <cit> .
using the segmented nuclei and lumens, we finally define a number of quantities measuring the morphologic changes in tissue, and the quantities include the size, number, distance, spatial distribution, and shape of epithelial nuclei and lumens .
detailed description of the quantities is available in supplementary information.
in total, we defined  <dig> quantities, of which  <dig> quantities were previously shown to be effective in detecting prostate cancer tissue with high accuracy  <cit> .
computing average, standard deviation, sum total, minimum, and maximum of all or some of these quantities,  <dig> morphological features are extracted from a tissue sample.
the details of tissue segmentation and feature extraction process are described elsewhere in kwak et al.
<cit> .fig.
<dig> morphologic feature extraction and morphologic criteria.
a cell type segmentation from ft-ir imaging is overlaid with a tissue image .
lumen  and nuclei  are segmented using tresholding and applying shape, size, and intensity constraints.
b using the segmentation results, a number of morphological features are computed.
c a pathologist examines and scores tissue images  for the  <dig> morphologic criteria.
the segmented tissue images are also shown for comparison
tissue retrieval
given a query , its morphological features are extracted and used to search for similar pre-examined samples from the database.
to retrieve the most similar samples, we adopt ranking-svm  <cit> , which learns a function mapping onto a ranking in a pair-wise fashion .
that is, ranking-svm provides a complete ranking of the entire samples in the database for the query.
since tms score serves as the gold standard of the tissue similarity , ranking-svm attempts to learn and reproduce the human experts’ interpretation of the tissue samples.
the feature vector difference between a query and the samples in the database is used for retrieval.
we note that if a sample in the database is highly ranked to the query, then the query should be highly ranked for the sample .
ranking-svm is an asymmetric measure, i.e., the ranking of a sample to the query would not be equal to the ranking of the query to the sample.
combining the two rankings, we seek to attain the more symmetric rankings between the query and the samples and to achieve the more accurate and specific retrieval .
we define the ranking of a sample to the query as rankingqdid=ranking−svmqdid+ranking−svmdi,q;d\di∪q,i= <dig> …,m where ranking ‐ svm denotes the ranking of the sample di in the database d to the query q and ranking − svm is the ranking of the query q to the sample di in the database d when the query q is switched with the sample di.
based on the ranking, top-t samples are retrieved.
since it is the sum of two rankings, it is likely that several rankings are tied.
in such cases, the final ranking is determined by the ranking of the sample to the query, i.e., ranking − svm, which is intuitive because the retrieval is done for the query.
feature selection
feature selection is the step where the retrieval algorithm examines all available features  with respect to the training samples, and selects a subset to use on test data.
this selection is generally based on the criterion of high accuracy on training data, but also strives to ensure generalizability beyond the training data.
we adopt a two-stage feature selection approach here.
in the first stage, we order the features by their individual retrieval performance and sequentially measure the retrieval performance of a feature set by adding a new feature one at a time according to the order.
in the second stage, feature selection continues with the feature set resulting the best retrieval performance in the first stage as the starting point, following the sequential floating forward selection  method  <cit> .
this method sequentially adds new features followed by conditional deletion of already selected features.
throughout the feature selection procedure, the retrieval capability of a feature set is measured by normalized discounted cumulative gain   <cit> , which is a popular measure to evaluate ranking algorithms with multiple levels of relevance.
ndcg utilizes the relevance  and ranking of the retrieved samples and is based on two assumptions: 1) highly relevant samples are more valuable when they are retrieved earlier 2) highly relevant samples are more valuable than marginally relevant samples to the query.
given a database d and tms scores, the performance of the retrieval function f for a query q at rank position t is computed as follows: ndcgqfdtms=dcgidcg  dcgqfdtms=∑t=1t2tmsqrt−1log21+t where rt indicates the tth closest sample to the query q, retrieved by the retrieval function f, from the database d, and idcg denotes a normalization factor that is computed with the ideal  rank of the retrieved samples, scaling the optimal retrieval to  <dig>
balanced training
ranking-svm tries to learn an overall ranking of the training dataset.
when trained on biased or unbalanced training dataset, ranking-svm may be biased towards dominant dataset, and thus its retrieval capability may be limited.
to prevent this, we sought to take roughly balanced sub-samples of the training dataset and trained ranking-svm on the sub-samples.
to obtain the roughly balanced training dataset, we first divide the total tms score range into p equal-width partitions.
then, np number of pairs of samples from each partition was randomly selected.
we set np to the smallest number of pairs of samples in a partition.
RESULTS
tissue morphologic similarity measure
for  <dig> prostate cancer samples, we asked a pathologist  to score them according to the  <dig> morphologic criteria.
the pathologist was not involved in preparing the tissue samples and kept blind to the previous diagnosis and clinical information of the samples.
provided with the scores for the  <dig> morphologic criteria, tissue morphologic similarity  was measured for all possible pairs of  <dig> tissue samples  and used as the gold standard for training and validating our approach.
we noted that tms score, ranging from  <dig> to  <dig>  is not evenly distributed, and mid-range score  is mostly dominant.
notably, only small number of pairs of samples gained a high tms score, e.g., ~2 % of pairs of samples scores ≥ <dig> .fig.
<dig> tissue morphologic similarity scores.
a tissue morphologic similarity scores are computed and drawn for all possible pairs of tissue samples.
b the frequency and cumulative density of similarity scores are plotted.
mid-range scores  are mostly dominant, and high scoring  samples are very rare
tissue retrieval system provides good matching cases
to evaluate the tissue retrieval system, we performed k-fold cross-validation .
the entire dataset was divided into k roughly equal-sized partitions, one partition was left out as “test data” , the union of the remaining k –  <dig> partitions  was used to build the database where top-t similar samples are retrieved for each query .
this was repeated k times with different choices of the left-out partition.
in each repetition, the 2-stage feature selection was carried out on the training data via a cross-validation .
the average ndcg at rank position t of the tissue retrievals for the queries, across all k repetitions, was computed to measure the performance of the retrieval.
to handle the imbalance of tms scores in the dataset, a roughly balanced training dataset was formed by dividing the entire score range into p equal-width partitions  and randomly taking equal number of samples from each partition.
the method was implemented in idl  on  <dig>  <dig> ghz intel core duo machine running windows  <dig> with 2gb memory and c++  on a  <dig> ghz intel core  <dig> duo machine running redhat linux  <dig> with 2gb memory.
the average processing time for tissue segmentation and morphological feature extraction is ~8 min per sample, and the tissue retrieval time is ~1 s. the ranking-svm training and the feature selection took ~3 s and ~90 min, respectively.
although we have computed tms scores and used them to train and test the retrieval process, it is unclear what similarity score is sufficient to provide useful information with pathologists when evaluating unknown samples.
setting a threshold tms too high score is unrealistic because there are not enough samples available; as mentioned above, only ~2 % of the training samples have similarity score ≥ <dig> for a query .
setting the tms threshold lower is not beneficial to pathologists.
we therefore adopted a new data management approach: in order to examine the retrieval performance in a broad sense, we changed a threshold similarity score ths from  <dig> to  <dig>  and designated a sample as a good match  to a query if their similarity score is ≥ ths.
then, we counted the number of good matches  among the retrieved samples for each query and plotted the fraction of the queries retrieving ≥ ng.
ng among the retrieved samples is equivalent to the fraction of the retrieved samples that are relevant to the query .
that is, fig.
4a shows the fraction of the queries achieving a precision level equal to or higher than  <dig> ,  <dig> ,  <dig> ,  <dig> , and  <dig>  it is noticeable that ~80 % and ~60 % of the queries retrieving ≥ <dig> and ≥ <dig> good matching cases  as setting ths to  <dig> and  <dig>  respectively.
compared to the random chance of retrieving ≥ <dig> and ≥ <dig> good matches, both were increased by two-fold, and the retrievals were statistically significant  by a binomial test .
as shown in fig.
4b, it was obvious that tms scores of pairs of the query and its top-t matching samples are higher than those of pairs of the query and all the samples in the database, especially tms scores are  <dig> or greater.fig.
<dig> tissue retrieval performance.
the number of queries retrieving at least n
g number of good matches by our system , out of t retrieved samples, is computed , and compared to a the random chance  and c knn retrieval  obtaining that number of good matching cases.
the frequency and cumulative density of similarity scores are plotted for b the entire training samples and t matching samples by our system, respectively.
d the frequency and cumulative density are also plotted for knn retrieval.
a good matching case is defined as a pair of samples whose similarity score is ≥ th
s, th
s =  <dig>  …,  <dig>  random chance of retrieving ≥ n
g good matching cases is computed as prx≥ng=∑x≥ngnssxns−nsst−xmt where n
s and n
ss denote the number of samples in the database and the number of samples whose tms with the query ≥ th
s, respectively
th
s
n
g
the number of queries  retrieving at least n
g number of good matching cases as a good match is defined as tms ≥ th
s and its statistical significance.
assuming the number of good matches follows a binomial distribution, p-value is computed as prx≥nq=∑x≥nqmxpx1−pm−x where p is a random chance of retrieving ≥ n
g good matches .
the number of queries and  its statistical significance
moreover, we performed the tissue retrieval by using the k-nearest neighbor  algorithm , instead of ranking-svm.
examining the number of good matches, ranking-svm consistently outperformed knn; for instance, setting ths to  <dig> and  <dig>  ranking-svm demonstrated a  <dig> -fold increase in the fraction of the queries retrieving ≥ <dig> and ≥ <dig> good matches, respectively .
we investigated the distribution of tms scores of pairs of the query and top-t matching samples by ranking-svm and knn .
ranking-svm showed higher tms scores than knn .
further, the retrieval results were evaluated by using ndcg .
considering top-t matching samples, ranking-svm achieved the average ndcg of  <dig> , and  <dig>  ndcg was obtained by knn on average.
ndcg was computed for the ranking of the entire samples in the database; ranking-svm and knn showed the average ndcg of  <dig>  and  <dig>  ndcg, respectively.table  <dig> tissue retrieval performance
data represent average ± standard deviation of ndcg
tms score reveals the complicated relationship between tissues
we examined the utility of tms scores in retrieving similar tissue samples by a visual comparison between tissue h&e images.
the relationship between tms score and gleason sum score was also investigated since gleason sum score is the only definite information available in prostate pathology today.
in fig.
<dig>  the examples of queries and their matching cases are presented.
a pair of samples belonging to the same grade tends to have a  high tms score, for example, in the second row of fig.
<dig>  three retrieved samples with gleason sum score  <dig> have > <dig>  tms score for the query whose gleason sum score is  <dig>  other two samples have different gleason sum score as well as lower tms scores .
however, high tms scoring sample pairs are not necessarily to be the same grade.
in the last row of fig.
<dig>  none of the retrieved samples are diagnosed with the same gleason sum score with the query, but their tms scores are generally high.
four of them have > <dig>  tms score, of which each has identical scores with the query for at least  <dig> morphologic criteria except gleason score, demonstrating the capability of tms scoring system in examining the relationship between tissues beyond the gleason grading system.
these types of relationships between tissue samples can never be retrieved or assessed if an automated system is built solely on the gleason grading system.
thus, tms scoring system may help to analyze the complicated and complex tissue morphology and to broaden our understanding.fig.
<dig> examples of queries and their matching cases.
for each query ,  <dig> closest matches are retrieved.
the least similar sample is also vshown .
tms denotes tissue morphologic similarity score for a pair of samples.
gs indicates a gleason sum score which is a sum of predominant and secondary gleason scores
discussion
herein, a tissue retrieval system has been developed and tested for prostate cancer.
this approach is particularly well suited for cancer and other diagnostic situations where there are multiple parameters applied to defining a grade.
in the system, a database allows pathologists to easily manage and maintain the previous cases and outcomes, and immediate access to them is available due to efficient retrieval algorithm.
accordingly, the performance of tissue retrieval is reliant on both a database and a retrieval process.
hence, further study on matching algorithm, performance measure, and data handling, e.g., data normalization, would be necessary, and a large-scale validation study should be conducted to optimize and stabilize the system for various queries, tasks and users’ demands.
the size of the database may substantially affect the performance of the retrieval system.
in tissue retrieval, it is assumed that the database contains enough number of similar samples to any kind of query.
that is, the retrieval system will benefit from the large-scale database, including a variety of patterns of tissue samples from multiple institutions.
the retrieval system with the large-scale database will not only serve for various queries and tasks but also improve and stabilize tms scores.
the similarity score for a criterion between two samples is dependent on the number of samples between them according to the criterion.
the distribution of the samples will become more realistic, leading to the more accurate and reliable similarity measure.
moreover, scoring tissue samples by multiple pathologists will further aid in improving tms scores.
however, with the limited size of the database, the distribution of tms score for one query differs from another .
some may have many high scoring sample pairs, but some may have few of them.
in the latter cases, the retrieval system may return the most similar samples, i.e., the retrieval is valid and useful, but it is a seemingly bad retrieval due to relatively lower tms score.
the overall distribution of tms score also affects the retrieval.
in our study, a limited number of tissue sample pairs show a high or low tms score , i.e., it is likely that the system retrieves tissue samples owning mid-range tms scores.
in fact, as we trained ranking-svm on the entire training dataset, i.e., without balanced training, less number of samples owning higher tms scores was retrieved for the query , for example, tms score ≥ <dig>  accordingly, taking a roughly balanced subset of the training dataset is a valid decision and helps to provide a more effective and robust retrieval process.
gleason grades in the dataset are not evenly distributed.
a lack of a sufficient number of samples per grade may result in a loss of information of certain patterns in prostate cancer.
however, the imbalance of the distribution in this study is not likely to have a significant impact on the retrieval system.
the system is still able to retrieve matching cases from the database.
a high tms score does not indicate that a sample pair has the same grade.
the effect of each grade on the retrieval system may be further studied to improve and stabilize the retrieval system.
we only retrieved the  <dig> closest samples to a query.
the more samples we retrieve, the higher probability the system provides well matched cases with pathologists.
however, retrieving many samples  will be burden to pathologists due to additional time and effort to decide what samples are relevant and useful.
hence, providing the most similar samples would be more helpful and effective.
it necessitates little time and work from pathologists to judge on the retrieved samples, however deliver good matches.
we note that if a pathologist would like to retrieve more or fewer samples from the database, then the retrieval system  should be re-trained by adjusting the number of retrievals.
if more samples are added to the database, then the whole system should be re-trained  by computing tms scores and morphological features and constructing a new ranking-svm.
moreover, as one or more morphological properties are of interest to a pathologist, the similarity score can be re-computed and used to train the retrieval system.
the pathologist may indicate that certain matches were better than others, resulting in an updating of the database  and matching algorithms as needed.
the updating may be conducted in real-time.
therefore, the system is potentially adaptable to users’ demand and purpose.
the  <dig> morphological criteria were manually scored by a pathologist and used to measure tms score.
like gleason grading, it is still a qualitative measure.
based on the qualitative measure, the pathologist categorizes  tissue samples per criterion.
it is well known that such qualitative measure is subject to inter- and intra-observer variability, i.e., likely mis-score  tissue samples, in particular for the borderline cases.
poor scoring , in our study, will disrupt the similarity measure.
however, the impact of mis-scoring on the retrieval system may not be as significant as that of gleason grading.
mis-scoring in gleason grading may give rise to a totally different pattern and outcome prediction.
unlikely, tms score is a combined measure of the  <dig> different properties and varies in a continuous fashion.
some mis-scorings of the  <dig> criteria clearly affect the similarity measure but may not cause a complete change in the tissue similarity.
nevertheless, a follow-up study is desirable to examine the influence of mis-scorings among the  <dig> criteria on the similarity measure and the tissue retrieval performance.
CONCLUSIONS
we have presented an efficient and effective tissue management and decision-support system.
tms score offers an alternate means of assessing tissue characteristics and similarities as well as developing and testing computerized methods.
next steps in development would be the validation and application of this system with additional users.
the system can be applied to a diversity of diagnostic entities in histopathology.
the approach is adaptable in scale, including reference dataset, scoring metrics and matches presented to the pathologist.
we anticipate that this approach will open a new direction for the development of automated methods for cancer pathology.
additional file
additional file 1: supplementary material.
acknowledgements
this work was supported by national institutes of health – national cancer institute via grant r01ca <dig>
availability of data and material
the source code, datasets, and supplementary information are available through the following link: chemimage.illinois.edu.
authors’ contributions
conception and design: jk, smh, ss, rb.
development of methodology: jk, ak-b, ss, rb.
acquisition of data: jk, ak-b, smh.
analysis and interpretation of data: jk, akb, ss, rb.
writing, review, and/or revision of the manuscript: jk, ak-b, smh, ss, rb.
study supervision: ss, rb.
all authors read and approved the final manuscript.
competing interests
the authors declare that they have no competing interests.
ethics statement
this study was performed on diagnostic specimens with information that neither identified the subjects directly nor indirectly through identifiers linked to the subjects.
it was approved by and performed in accordance with the university of illinois at urbana-champaign institutional review board.
the approved project is entitled “optical spectroscopy and imaging of archival fixed tissue,” case number  <dig>  and consisted only of secondary analysis performed on anonymized archival tissue and, as such, according to the university of illinois at urbana-champaign irb policy, is exempt from written, informed consent.

 quality assurance in diagnostic histopathology plays a critical role in development of a treatment plan for patients with prostate cancer. methods to integrate quality development, maintenance, and improvement of diagnostic accuracy are, hence, critical to cancer management in any setting. in diagnostic prostate pathology, gleason grading is the most commonly used grading system that is based upon the structural patterns of the tumor. the gleason grade is a primary determinant in treatment planning. however, it is well known that the grading of prostate tissues suffers from intra- and inter-pathologist variability; for example, the exact intra-pathologist agreement was achieved in 43–78 % of the instances, and 36–81 % of the exact inter-pathologist agreement was reported. it is also known that the variability of the grading can be reduced with focused retraining. there could be many ways to educate pathologists such as meetings, courses, online tutorials, and etc, but these are not time- and cost-effective and rarely implemented. therefore, building an automated, fast, and objective method to aid pathologists in evaluating prostate can improve prostate cancer diagnosis. when a pathologist evaluates a tissue sample, he/she looks at a stained tissue and mentally compares it against a fund of knowledge and experience and may consult publications when needed. in essence, the pathologist is matching structural patterns with samples they have seen earlier and mentally recalling the diagnosis made such that they can make the same diagnosis in the specific test case. despite training, intra- and inter-observer variation and controversial areas still exist. to aid and improve the diagnostic process, there have been several research efforts to develop automated systems for the detection and grading of prostate cancer. the majority of the previous methods have used morphological features to characterize and classify tissue samples into correct classes, and others have also used fourier transform, wavelet transform, and fractal analysis to extract texture features. though these methods claim to be accurate, the information that pathologists will obtain by using such methods may be limited since these only provide the predicted grade in general. the prediction also relies on the conditions of the training and testing datasets such as acquisition settings and staining. alternatively, content-based image retrieval systems have been proposed to aid cancer pathology. the main objective is to effectively and efficiently manage an enormous amount of image data and to provide similar cases to a new test case that is examined. in addition to clinical usage, cbir systems can help medical research, education, and training. the similar cases can be determined as owning the same grade or sub-structures; for instance, single lumen glands, multi-lumen glands blood vessels, and lymphocytes in prostate. the basic premise of such systems in diagnostic histopathology is that tissue samples that have the same grade or similar characteristics and patterns with the sample of interest will afford useful information to pathologists and improve the decision-making process. similar to cancer detection and grading systems, tissue is represented as several quantitative features such as morphology, histogram, color, and texture. the similar samples can be retrieved by computing distance metrics or similarity scores between a new case and the previously diagnosed or examined cases. in order to improve tissue representation and retrieval, features are often post-transformed and/or their weights are adjusted in an implicit or explicit manner; for example, kernel function, simplex method, manifold learning, boosting, and self-organizing map. previous retrieval systems have been measured against a gold standard of diagnostic category and grade of tumor, defined by a pathologist. prostate cancer is, in particular, a multifactorial disease and a mixture of heterogeneous growth patterns, and hence tissues belonging to the same gleason grade may possess different cellular, nuclear, or glandular sub-patterns. a number of histological variants, in fact, exist in prostate carcinoma and some of the variants cannot be addressed by the gleason grading system. moreover, the gleason grading system results in a tumor grade that correlates with overall outcomes, but fails to provide information on risk of metastasis, and correlates poorly with the clinical decision making process. further, the gleason grading system has gone through several refinements over time and may undergo further changes. these changes result variations among pathologists in practice and disrupt developing robust automated grading and retrieval systems. here, we report developing a new computer information and management and decision-support system that consists of a database of pre-defined prostate tissues and a retrieval process. the database retains tissue images, clinical information, and one or more measurements of the structure of tissue. the retrieval process utilizes the structural/morphological features of the tissue sample image and provides tissue samples similar to the sample under consideration from the database. in assessing tissue morphology, we utilize infrared chemical imaging for providing cell type information in tissue. we previously reported its utility in stabilizing and improving the automated cancer detection. as a retrieval function, we adopt a machine learning ranking approach, called ranking-support vector machine in conjunction with a two-stage “feature selection” step. ranking-svm learns a ranking function of high generalization due to maximum-margin property. feature selection step finds the most informative subsets while preserving the essential characteristics of the data. moreover, we propose to determine the ground truth tissue similarity based on various structural properties of tissue, not solely on gleason grade or a single structural component. here, the structural properties are examined by pathologists. combining different structural components of tissue ensures better characterization of tissue structure, and thus more accurate measurement of tissue similarity can be made. thereby, the automated and computerized analysis and human experts’ assessment of tissue morphology are correlated through a machine learning approach.fig. overview of system. as a query is given, the closest matches and their clinical information are retrieved from the database. provided with the matching cases, pathologists make a diagnosis, and updating may or may not be conducted. q, d, ranking, f, and s denote a query, database, retrieval process, single feature, and subset of features, respectively the rest of the paper is organized as follows. in methods section, we begin with a description of the dataset and data preparation process. in the following subsections, we describe the three key components of our new system – tissue similarity measure, tissue morphological feature extraction, and tissue retrieval function. then, feature selection and balanced training are described. in results section, the experimental results, including tissue similarity measure and tissue retrieval, via cross-validation are demonstrated. in discussion section, the implications and limitations of our study are discussed. finally, we conclude in conclusions section. methods samples and data preparation this study and protocols were approved by the university of illinois institutional review board and was conducted as per the permission of the irb in accordance with relevant guidelines and regulations. we have obtained prostate cancer tissue samples, composed of,,,, and samples. both hematoxylin and eosin stained and ft-ir images are available for the samples. tissue samples were first sectioned to ~5um thick sections, with a section being placed on a standard glass slide and a serial section on ir transparent baf slide. stained with h&e, tissue images were acquired on a standard optical microscope at 40x magnification, and the size of a pixel is um × um. on ir transparent baf slides, ft-ir images were acquired at a spatial pixel size of um × um and a spectral resolution of 4 cm- at an undersampling ratio of using perkin-elmer spotlight imaging system. the spectral profile of a pixel was truncated to a spectral range of 4000-720 cm- detailed description of sample preparation and data acquisition for ft-ir imaging are available in fernandez et al.. clinical information of the samples were prepared by pathologic review, and morphological features were also extracted. the database we build here, therefore, contains tissue images and their clinical information and morphological features. morphologic criteria and tissue similarity measure we define criteria to describe the architectural properties of tissue: 1) gland crowding, 2) gland roundness, 3) stromal reaction, 4) nuclear grade, 5) clefts 6) lumen/gland ratio, 7) gland continuity, 8) cell separation, and 9) gleason score. the details of the criteria are listed in table  some of the properties are the criteria used in the gleason grading system, and others were adopted from the literature. although some criteria are overlapped with the gleason grading system, their importance and interpretation in our system may vary. the gleason grading system may be also able to describe certain properties of tissues that cannot be characterized by the overlapping criteria. in the gleason grading system, gland arrangement, variations in size and shape of gland, sheets of cells, and single cells are examined. nuclear morphometry, reactive stroma, and retraction clefting have been reported to be useful for prostate diagnosis and prognosis. in the digital and computerized tissue analyses, structural features describing gland arrangement and shape and the size of gland and lumen have been adopted to characterize tissue. individual cells also showed a moderate correlation with patient outcomes.table description of morphologic criteria agleason score is the sum of predominant and secondary scores. in our set, it ranges from to 10 for each of the criteria, a pathologist examines a tissue sample and assigns a score ranging from to or to except gleason score which in our set of tissues ranges from to the score range from to may be interpreted as none, low, and high, and the range from to may be considered as none, low, mid, and high. due to its qualitative nature, it is difficult to highly stratify, and the impact and measurability of each criterion varies. restricting the score range to none, low, mid, and high, in general, the scores are intended to be specific to differing morphologic patterns as well as to be reproducible by other pathologists. using the scores of the morphologic criteria, tissue morphologic similarity between tissue samples is measured. although well-defined and measured, the importance or relevance of each criterion differs. for example, the significance of gland crowding score may differ from that of gland roundness score and the difference between two samples having gland crowding score and may not be identical to the difference between two samples owning stromal reaction score and in these cases, the absolute values of the scores and the difference of the scores are identical. recognizing the intrinsic relationship between scores and criteria, we utilize the distribution of each criterion score over the samples in the database. regardless of the absolute value of a score, two samples away from each other in the distribution of the scores of a criterion are likely dissimilar in terms of the criterion, and vice versa. in other words, tissue similarity between two samples with respect to a morphologic criterion is related to the number of samples between them as ordered by the score for the criterion. accordingly, let tms be the tissue morphologic similarity between two tissue samples d and d and computed as follows: tmsd1d2=∑i=19tmsid1d where tmsi is the morphologic tissue similarity for ith criteria. tmsi is calculated as follows: tmsid1d2=1−∑s=sd1i+1sd2i−1his+12hisd1i+hisd2iz where sdi is the ith morphologic criterion score of a tissue sample d, hi is the number of samples having ith morphologic criterion score s, and z is a normalization factor. due to normalization, tmsi ranges from to 1 ≤ i ≤  thereby tms ranges from to in this study, tms scores represent the true similarity between tissue samples and serve as the gold standard of tissue retrieval. morphological feature extraction in prostate cancer, epithelial cells, which line ducts and acini in intact tissue in three-dimensional structures, are of great interest. as cancer grows, epithelial cells grow in and out of the glands in an uncontrolled way, and thus the structure of tissue, especially the local glandular structure, is distorted. we also note that the role of stroma cells, connective cells supporting epithelial cells, in cancer tissue has been recently recognized. to quantify the alterations in tissue morphology, we focus here on the nuclear and cellular morphology of epithelial and stromal cells and lumens. in order to quantify the nuclear and cellular morphology of epithelial and stromal cells and lumens, we first segment epithelium and stroma in tissue by adopting fourier transform infrared spectroscopy imaging due to its accuracy and robustness. ft-ir has been extensively validated in classifying histologic cell types in tissue and provides a color coded cell type image of tissue. cell type segmentation in h&e images is challenging due to limited information, color variations, etc. rigid-body image registration overlays the epithelium and stroma segmentation from ft-ir imaging with the corresponding h&e image by using outer shape and empty space in tissues. second, lumens and nuclei are identified from h&e images by considering their color intensities and geometric properties. using the segmented nuclei and lumens, we finally define a number of quantities measuring the morphologic changes in tissue, and the quantities include the size, number, distance, spatial distribution, and shape of epithelial nuclei and lumens. detailed description of the quantities is available in supplementary information. in total, we defined quantities, of which quantities were previously shown to be effective in detecting prostate cancer tissue with high accuracy. computing average, standard deviation, sum total, minimum, and maximum of all or some of these quantities, morphological features are extracted from a tissue sample. the details of tissue segmentation and feature extraction process are described elsewhere in kwak et al..fig. morphologic feature extraction and morphologic criteria. a cell type segmentation from ft-ir imaging is overlaid with a tissue image. lumen and nuclei are segmented using tresholding and applying shape, size, and intensity constraints. b using the segmentation results, a number of morphological features are computed. c a pathologist examines and scores tissue images for the morphologic criteria. the segmented tissue images are also shown for comparison tissue retrieval given a query, its morphological features are extracted and used to search for similar pre-examined samples from the database. to retrieve the most similar samples, we adopt ranking-svm, which learns a function mapping onto a ranking in a pair-wise fashion. that is, ranking-svm provides a complete ranking of the entire samples in the database for the query. since tms score serves as the gold standard of the tissue similarity, ranking-svm attempts to learn and reproduce the human experts’ interpretation of the tissue samples. the feature vector difference between a query and the samples in the database is used for retrieval. we note that if a sample in the database is highly ranked to the query, then the query should be highly ranked for the sample. ranking-svm is an asymmetric measure, i.e., the ranking of a sample to the query would not be equal to the ranking of the query to the sample. combining the two rankings, we seek to attain the more symmetric rankings between the query and the samples and to achieve the more accurate and specific retrieval. we define the ranking of a sample to the query as rankingqdid=ranking−svmqdid+ranking−svmdi,q;d\di∪q,i= …,m where ranking ‐ svm denotes the ranking of the sample di in the database d to the query q and ranking − svm is the ranking of the query q to the sample di in the database d when the query q is switched with the sample di. based on the ranking, top-t samples are retrieved. since it is the sum of two rankings, it is likely that several rankings are tied. in such cases, the final ranking is determined by the ranking of the sample to the query, i.e., ranking − svm, which is intuitive because the retrieval is done for the query. feature selection feature selection is the step where the retrieval algorithm examines all available features with respect to the training samples, and selects a subset to use on test data. this selection is generally based on the criterion of high accuracy on training data, but also strives to ensure generalizability beyond the training data. we adopt a two-stage feature selection approach here. in the first stage, we order the features by their individual retrieval performance and sequentially measure the retrieval performance of a feature set by adding a new feature one at a time according to the order. in the second stage, feature selection continues with the feature set resulting the best retrieval performance in the first stage as the starting point, following the sequential floating forward selection method. this method sequentially adds new features followed by conditional deletion of already selected features. throughout the feature selection procedure, the retrieval capability of a feature set is measured by normalized discounted cumulative gain, which is a popular measure to evaluate ranking algorithms with multiple levels of relevance. ndcg utilizes the relevance and ranking of the retrieved samples and is based on two assumptions: 1) highly relevant samples are more valuable when they are retrieved earlier 2) highly relevant samples are more valuable than marginally relevant samples to the query. given a database d and tms scores, the performance of the retrieval function f for a query q at rank position t is computed as follows: ndcgqfdtms=dcgidcg dcgqfdtms=∑t=1t2tmsqrt−1log21+t where rt indicates the tth closest sample to the query q, retrieved by the retrieval function f, from the database d, and idcg denotes a normalization factor that is computed with the ideal rank of the retrieved samples, scaling the optimal retrieval to balanced training ranking-svm tries to learn an overall ranking of the training dataset. when trained on biased or unbalanced training dataset, ranking-svm may be biased towards dominant dataset, and thus its retrieval capability may be limited. to prevent this, we sought to take roughly balanced sub-samples of the training dataset and trained ranking-svm on the sub-samples. to obtain the roughly balanced training dataset, we first divide the total tms score range into p equal-width partitions. then, np number of pairs of samples from each partition was randomly selected. we set np to the smallest number of pairs of samples in a partition.  tissue morphologic similarity measure for prostate cancer samples, we asked a pathologist to score them according to the morphologic criteria. the pathologist was not involved in preparing the tissue samples and kept blind to the previous diagnosis and clinical information of the samples. provided with the scores for the morphologic criteria, tissue morphologic similarity was measured for all possible pairs of tissue samples and used as the gold standard for training and validating our approach. we noted that tms score, ranging from to is not evenly distributed, and mid-range score is mostly dominant. notably, only small number of pairs of samples gained a high tms score, e.g., ~2 % of pairs of samples scores ≥.fig. tissue morphologic similarity scores. a tissue morphologic similarity scores are computed and drawn for all possible pairs of tissue samples. b the frequency and cumulative density of similarity scores are plotted. mid-range scores are mostly dominant, and high scoring samples are very rare tissue retrieval system provides good matching cases to evaluate the tissue retrieval system, we performed k-fold cross-validation. the entire dataset was divided into k roughly equal-sized partitions, one partition was left out as “test data”, the union of the remaining k – partitions was used to build the database where top-t similar samples are retrieved for each query. this was repeated k times with different choices of the left-out partition. in each repetition, the 2-stage feature selection was carried out on the training data via a cross-validation. the average ndcg at rank position t of the tissue retrievals for the queries, across all k repetitions, was computed to measure the performance of the retrieval. to handle the imbalance of tms scores in the dataset, a roughly balanced training dataset was formed by dividing the entire score range into p equal-width partitions and randomly taking equal number of samples from each partition. the method was implemented in idl on ghz intel core duo machine running windows with 2gb memory and c++ on a ghz intel core duo machine running redhat linux with 2gb memory. the average processing time for tissue segmentation and morphological feature extraction is ~8 min per sample, and the tissue retrieval time is ~1 s. the ranking-svm training and the feature selection took ~3 s and ~90 min, respectively. although we have computed tms scores and used them to train and test the retrieval process, it is unclear what similarity score is sufficient to provide useful information with pathologists when evaluating unknown samples. setting a threshold tms too high score is unrealistic because there are not enough samples available; as mentioned above, only ~2 % of the training samples have similarity score ≥ for a query. setting the tms threshold lower is not beneficial to pathologists. we therefore adopted a new data management approach: in order to examine the retrieval performance in a broad sense, we changed a threshold similarity score ths from to and designated a sample as a good match to a query if their similarity score is ≥ ths. then, we counted the number of good matches among the retrieved samples for each query and plotted the fraction of the queries retrieving ≥ ng. ng among the retrieved samples is equivalent to the fraction of the retrieved samples that are relevant to the query. that is, fig. 4a shows the fraction of the queries achieving a precision level equal to or higher than,,,, and it is noticeable that ~80 % and ~60 % of the queries retrieving ≥ and ≥ good matching cases as setting ths to and respectively. compared to the random chance of retrieving ≥ and ≥ good matches, both were increased by two-fold, and the retrievals were statistically significant by a binomial test. as shown in fig. 4b, it was obvious that tms scores of pairs of the query and its top-t matching samples are higher than those of pairs of the query and all the samples in the database, especially tms scores are or greater.fig. tissue retrieval performance. the number of queries retrieving at least n g number of good matches by our system, out of t retrieved samples, is computed, and compared to a the random chance and c knn retrieval obtaining that number of good matching cases. the frequency and cumulative density of similarity scores are plotted for b the entire training samples and t matching samples by our system, respectively. d the frequency and cumulative density are also plotted for knn retrieval. a good matching case is defined as a pair of samples whose similarity score is ≥ th s, th s =   …,  random chance of retrieving ≥ n g good matching cases is computed as prx≥ng=∑x≥ngnssxns−nsst−xmt where n s and n ss denote the number of samples in the database and the number of samples whose tms with the query ≥ th s, respectively th s n g the number of queries retrieving at least n g number of good matching cases as a good match is defined as tms ≥ th s and its statistical significance. assuming the number of good matches follows a binomial distribution, p-value is computed as prx≥nq=∑x≥nqmxpx1−pm−x where p is a random chance of retrieving ≥ n g good matches. the number of queries and its statistical significance moreover, we performed the tissue retrieval by using the k-nearest neighbor algorithm, instead of ranking-svm. examining the number of good matches, ranking-svm consistently outperformed knn; for instance, setting ths to and ranking-svm demonstrated a -fold increase in the fraction of the queries retrieving ≥ and ≥ good matches, respectively. we investigated the distribution of tms scores of pairs of the query and top-t matching samples by ranking-svm and knn. ranking-svm showed higher tms scores than knn. further, the retrieval results were evaluated by using ndcg. considering top-t matching samples, ranking-svm achieved the average ndcg of, and ndcg was obtained by knn on average. ndcg was computed for the ranking of the entire samples in the database; ranking-svm and knn showed the average ndcg of and ndcg, respectively.table tissue retrieval performance data represent average ± standard deviation of ndcg tms score reveals the complicated relationship between tissues we examined the utility of tms scores in retrieving similar tissue samples by a visual comparison between tissue h&e images. the relationship between tms score and gleason sum score was also investigated since gleason sum score is the only definite information available in prostate pathology today. in fig.  the examples of queries and their matching cases are presented. a pair of samples belonging to the same grade tends to have a high tms score, for example, in the second row of fig.  three retrieved samples with gleason sum score have > tms score for the query whose gleason sum score is other two samples have different gleason sum score as well as lower tms scores. however, high tms scoring sample pairs are not necessarily to be the same grade. in the last row of fig.  none of the retrieved samples are diagnosed with the same gleason sum score with the query, but their tms scores are generally high. four of them have > tms score, of which each has identical scores with the query for at least morphologic criteria except gleason score, demonstrating the capability of tms scoring system in examining the relationship between tissues beyond the gleason grading system. these types of relationships between tissue samples can never be retrieved or assessed if an automated system is built solely on the gleason grading system. thus, tms scoring system may help to analyze the complicated and complex tissue morphology and to broaden our understanding.fig. examples of queries and their matching cases. for each query, closest matches are retrieved. the least similar sample is also vshown. tms denotes tissue morphologic similarity score for a pair of samples. gs indicates a gleason sum score which is a sum of predominant and secondary gleason scores discussion herein, a tissue retrieval system has been developed and tested for prostate cancer. this approach is particularly well suited for cancer and other diagnostic situations where there are multiple parameters applied to defining a grade. in the system, a database allows pathologists to easily manage and maintain the previous cases and outcomes, and immediate access to them is available due to efficient retrieval algorithm. accordingly, the performance of tissue retrieval is reliant on both a database and a retrieval process. hence, further study on matching algorithm, performance measure, and data handling, e.g., data normalization, would be necessary, and a large-scale validation study should be conducted to optimize and stabilize the system for various queries, tasks and users’ demands. the size of the database may substantially affect the performance of the retrieval system. in tissue retrieval, it is assumed that the database contains enough number of similar samples to any kind of query. that is, the retrieval system will benefit from the large-scale database, including a variety of patterns of tissue samples from multiple institutions. the retrieval system with the large-scale database will not only serve for various queries and tasks but also improve and stabilize tms scores. the similarity score for a criterion between two samples is dependent on the number of samples between them according to the criterion. the distribution of the samples will become more realistic, leading to the more accurate and reliable similarity measure. moreover, scoring tissue samples by multiple pathologists will further aid in improving tms scores. however, with the limited size of the database, the distribution of tms score for one query differs from another. some may have many high scoring sample pairs, but some may have few of them. in the latter cases, the retrieval system may return the most similar samples, i.e., the retrieval is valid and useful, but it is a seemingly bad retrieval due to relatively lower tms score. the overall distribution of tms score also affects the retrieval. in our study, a limited number of tissue sample pairs show a high or low tms score, i.e., it is likely that the system retrieves tissue samples owning mid-range tms scores. in fact, as we trained ranking-svm on the entire training dataset, i.e., without balanced training, less number of samples owning higher tms scores was retrieved for the query, for example, tms score ≥ accordingly, taking a roughly balanced subset of the training dataset is a valid decision and helps to provide a more effective and robust retrieval process. gleason grades in the dataset are not evenly distributed. a lack of a sufficient number of samples per grade may result in a loss of information of certain patterns in prostate cancer. however, the imbalance of the distribution in this study is not likely to have a significant impact on the retrieval system. the system is still able to retrieve matching cases from the database. a high tms score does not indicate that a sample pair has the same grade. the effect of each grade on the retrieval system may be further studied to improve and stabilize the retrieval system. we only retrieved the closest samples to a query. the more samples we retrieve, the higher probability the system provides well matched cases with pathologists. however, retrieving many samples will be burden to pathologists due to additional time and effort to decide what samples are relevant and useful. hence, providing the most similar samples would be more helpful and effective. it necessitates little time and work from pathologists to judge on the retrieved samples, however deliver good matches. we note that if a pathologist would like to retrieve more or fewer samples from the database, then the retrieval system should be re-trained by adjusting the number of retrievals. if more samples are added to the database, then the whole system should be re-trained by computing tms scores and morphological features and constructing a new ranking-svm. moreover, as one or more morphological properties are of interest to a pathologist, the similarity score can be re-computed and used to train the retrieval system. the pathologist may indicate that certain matches were better than others, resulting in an updating of the database and matching algorithms as needed. the updating may be conducted in real-time. therefore, the system is potentially adaptable to users’ demand and purpose. the morphological criteria were manually scored by a pathologist and used to measure tms score. like gleason grading, it is still a qualitative measure. based on the qualitative measure, the pathologist categorizes tissue samples per criterion. it is well known that such qualitative measure is subject to inter- and intra-observer variability, i.e., likely mis-score tissue samples, in particular for the borderline cases. poor scoring, in our study, will disrupt the similarity measure. however, the impact of mis-scoring on the retrieval system may not be as significant as that of gleason grading. mis-scoring in gleason grading may give rise to a totally different pattern and outcome prediction. unlikely, tms score is a combined measure of the different properties and varies in a continuous fashion. some mis-scorings of the criteria clearly affect the similarity measure but may not cause a complete change in the tissue similarity. nevertheless, a follow-up study is desirable to examine the influence of mis-scorings among the criteria on the similarity measure and the tissue retrieval performance.  we have presented an efficient and effective tissue management and decision-support system. tms score offers an alternate means of assessing tissue characteristics and similarities as well as developing and testing computerized methods. next steps in development would be the validation and application of this system with additional users. the system can be applied to a diversity of diagnostic entities in histopathology. the approach is adaptable in scale, including reference dataset, scoring metrics and matches presented to the pathologist. we anticipate that this approach will open a new direction for the development of automated methods for cancer pathology. additional file additional file 1: supplementary material. acknowledgements this work was supported by national institutes of health – national cancer institute via grant r01ca availability of data and material the source code, datasets, and supplementary information are available through the following link: chemimage.illinois.edu. authors’ contributions conception and design: jk, smh, ss, rb. development of methodology: jk, ak-b, ss, rb. acquisition of data: jk, ak-b, smh. analysis and interpretation of data: jk, akb, ss, rb. writing, review, and/or revision of the manuscript: jk, ak-b, smh, ss, rb. study supervision: ss, rb. all authors read and approved the final manuscript. competing interests the authors declare that they have no competing interests. ethics statement this study was performed on diagnostic specimens with information that neither identified the subjects directly nor indirectly through identifiers linked to the subjects. it was approved by and performed in accordance with the university of illinois at urbana-champaign institutional review board. the approved project is entitled “optical spectroscopy and imaging of archival fixed tissue,” case number and consisted only of secondary analysis performed on anonymized archival tissue and, as such, according to the university of illinois at urbana-champaign irb policy, is exempt from written, informed consent.
we also develop a tissue similarity measure scheme to broaden our understanding of tissue characteristics. the system includes a database of previously evaluated prostate tissue images, clinical information and a tissue retrieval process. the retrieval process seeks to find the closest matching cases with the tissue of interest. thus, we expect the system to facilitate quality maintenance and quality improvement of cancer pathology. providing the closest matching cases and their clinical information with pathologists will help to conduct consistent and reliable diagnoses. based on the criteria, true tissue similarity is determined and serves as the gold standard of tissue retrieval. prostate cancerdatabasetissue morphologytissue retrievalinfrared imagingdecision supporthttp://dx.doi.org/ /100000002national institutes of healthr01ca138882bhargava rohit http://dx.doi.org/ /100000054national cancer instituter01ca197516bhargava rohit issue-copyright-statement© the author 2016 here, we found a minimum of and matching cases, out of for ~80 % and ~60 % of the queries when a match was defined as the tissue similarity score ≥ and ≥ respectively. we were also able to examine the relationship between tissues beyond the gleason grading system due to the tissue similarity scoring system.
 the current practice of histopathology review is limited in speed and accuracy. the current diagnostic paradigm does not fully describe the complex and complicated patterns of cancer. to address these needs, we develop an automated and objective system that facilitates a comprehensive and easy information management and decision-making. we also develop a tissue similarity measure scheme to broaden our understanding of tissue characteristics.  the system includes a database of previously evaluated prostate tissue images, clinical information and a tissue retrieval process. in the system, a tissue is characterized by its morphology. the retrieval process seeks to find the closest matching cases with the tissue of interest. moreover, we define morphologic criteria by which a pathologist arrives at a histomorphologic diagnosis. based on the criteria, true tissue similarity is determined and serves as the gold standard of tissue retrieval. here, we found a minimum of and matching cases, out of for ~80 % and ~60 % of the queries when a match was defined as the tissue similarity score ≥ and ≥ respectively. we were also able to examine the relationship between tissues beyond the gleason grading system due to the tissue similarity scoring system.  providing the closest matching cases and their clinical information with pathologists will help to conduct consistent and reliable diagnoses. thus, we expect the system to facilitate quality maintenance and quality improvement of cancer pathology. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords prostate cancerdatabasetissue morphologytissue retrievalinfrared imagingdecision supporthttp://dx.doi.org/ /100000002national institutes of healthr01ca138882bhargava rohit http://dx.doi.org/ /100000054national cancer instituter01ca197516bhargava rohit issue-copyright-statement© the author 2016 