BACKGROUND
over the past few years gene expression connectivity mapping has gained much popularity among biomedical researchers because of its promising applications as demonstrated by an increasing number of studies in different research areas: drug discovery , drug re-positioning , predictive toxicology  <cit> , and chemical carcinogenicity assessment  <cit> , to name a few.
the connectivity map  concept was first introduced by lamb et al.
<cit>  with the idea of using gene expression profiles/signatures to represent different biological states, and then to establish the connections among these states based on their gene expression patterns.
the establishment of a connection between two biological states may have different implications.
if the connection was made because the key set of genes were similarly up- or down-regulated in the two states, it may indicate that the two states have the same activated biological processes or pathways, such as the autophagy process and an autophagy enhancing compound fasudil identified in  <cit> , and consequently such a connection would be very useful for gaining insights into the underlying mechanism of one state based on the knowledge of the other, and vice versa.
on the other hand if the connection was made because the key set of genes were oppositely regulated, it may indicate that the two states negate each other.
if one is an undesirable state such as a disease, and the other is a chemical compound-induced state, the compound may be useful to treat that particular disease  <cit> , as in the case of application to inflammatory bowel disease  <cit> , and human skeletal muscle atrophy  <cit> .
from a user point of view, one important question often asked is: what is the best way of making a gene expression signature that can represent the biological state of interest accurately and be able to return meaningful biological connections?
within the gene expression connectivity mapping framework, there are three key components: reference gene expression profiles, query gene signatures, and expression pattern matching algorithms.
to date, much of the methodological development has been focused on the first and the third components above .
one important aspect related to the development has been the efforts to reduce the high number of possible confounding factors and consequent batch effects within the heterogeneous cmap datasets, eg., by utilising filtering and normalisation steps to improve the signal to noise ratio  <cit> , or by explicitly considering the variability in the transcriptional response and capturing the “consensus” response to a compound across multiple cell lines and dosages  <cit> .
in this paper, we are mainly concerned with the construction of query gene signatures, which was previously left to users without much guidance.
for discussing the rationale of the “gene signature progression” method to be introduced, we first outline the differences between reference profiles and gene signatures in the context of connectivity mapping.
the idea of using some form of molecular profile to characterize a biological state is not completely new.
in essence this idea was the basis of many clinical practices.
for example, various biochemical tests measure the molecular profile of a patient, which can then be used to infer or learn about the biological state of the patient .
the main assumption behind the connectivity mapping concept is the generalization of molecular profile  measurement.
in this case the bio-marker molecules are the messenger rnas of genes in an organism.
because of the sheer number of different mrnas measured, they collectively provide a much richer and complex description of the biological state than a few marker molecules could achieve.
theoretically if we have identified all the different molecules and their abundance in a cell, we will have an exact specification of the cell’s biological state.
the main assumption underlying the connectivity mapping concept was that the specification of the mrna profiles as measured by microarrays can provide an adequate description of the biological state.
with this premise, we discuss here the roles played by reference profiles and gene signatures in the connectivity mapping framework.
as described previously  <cit> , a reference gene expression profile contains the full list of genes that were measured by the microarrays, and specifies their abundance in one condition relative to another.
because a reference profile contains information about each gene, it is intended to provide a comprehensive description of the biological state it represents.
a query gene signature, on the other hand, is a short list of genes that capture the most important features of a biological state.
its difference to a reference profile is that a gene signature is not meant to provide the full description of how each gene has changed its expression, but rather to provide the most prominent features in that biological state.
obviously any biological condition  will have many features/aspects, but they are not of equal importance.
for establishing the connection between two biological states, one needs to decide what features to include in the gene signature.
the aim of gene signature construction is to compile a list of genes that collectively capture the most prominent features of the biological state it is intended to represent.
here in this paper we propose a two-stage process to achieve that goal.
the initial input is assumed to be from microarray gene expression data that have been properly processed through some common steps, such as quality control, background correction, normalization, scaling or transformations, where appropriate.
these pre-processing steps are very important in microarray data analysis and some recent research focused on these aspects, eg  <cit> , but they are not in the scope of the present work.
our starting point is that we have obtained the gene expression dataset, from which the expression levels of the genes can be quantified in the available biological samples.
given these data, we propose the following two-stage process for the construction of gene expression signature in connectivity mapping:  differential gene expression analysis to provide an ordered list of significant genes.
gene signature progression analysis to systematically determine the minimum length of the gene signatures able to return significant connections to reference drugs with an acceptable false discovery rate.
detailed description of the approach is presented in the following “methods: a two-stage process” section.
methods: a two-stage process
a schematic workflow of the two-stage process is depicted in fig.
<dig>  the aim of the first stage, differential expression analysis, is to derive a ranked list of degs  together with their regulation status , which can serve as an input to the general connectivity mapping process.
the second stage, gene signature progression analysis, involves creating multiple gene signatures consisting of an increasing number of top ranked genes, and then querying the sscmap iteratively with gene signatures of increasing length until a number of significant drugs are returned with a pre-set acceptable false discovery rate.
fig.
<dig> flow chart of the two stage process.
the output from stage  <dig> is a ranked list of significant genes with their regulation status .
this is used as input to stage  <dig>  gene signature progression.
starting from k= <dig>  the top k genes are fed to sscmap as a query signature to pull out significant drugs.
this process is run iteratively with increasing k until a pre-set target fdr is achieved for the returned significant drugs
the sscmap connectivity mapping framework was developed previously to introduce a more principled statistical test in connectivity mapping  <cit> .
it was bundled with  <dig> compound-induced reference gene expression profiles as its core database.
when a user-supplied query gene signature is presented to it, sscmap calculates a connection score between the query signature and each set of reference profiles in the core database, then performs computationally intensive permutation tests, to assess the statistical significance of each observed connection score.
a number of drugs with significant connection to the query signature are then returned as the results of this process.
differential gene expression analysis
in general, differential gene expression analysis involves two or more biological conditions.
for gene signature construction in connectivity mapping, we are mainly concerned with cases where they are two conditions.
one of them is a control condition which serves as a reference point.
the other condition is the state of our interest, for example, a disease state or a state as a result of some form of biological, chemical, or genomic perturbation experiment.
this is similar to the construction of reference gene expression profiles, where a vehicle control condition and a drug treated condition are required.
an important issue in the differential gene expression analysis is the multiple testing correction that must be considered when conducting a large number of statistical tests at the same time.
when tens of thousands of genes are being tested in the same analysis, the conventional statistical significance level of  <dig> , which was developed for single statistical test, is no longer adequate.
purely by chance,  <dig> % of the genes tested will turn out to have a p-value less than  <dig>  even if there is no difference at all between the two biological conditions being compared.
to safeguard the findings of differential gene expression against too many false positives creeping in, the false discovery rate  has become a commonly accepted measure to control in situations where multiple hypothesis testing is involved  <cit> .
this measure allows users to estimate the number of false positives given the total number of positive results obtained.
an equivalent way of achieving this goal is to control the expected number of false positives directly, and then calculate the empirical false discovery rate.
for example, to control the expected number of false positives to be efp, one can set the threshold p-value as efp/n, where n is the total number of hypotheses being simultaneously tested, and in the case of microarray differential gene expression analysis, n is the total number of genes  measured by the microarrays.
in this paper we set efp= <dig> to apply a tight control on the expected number of false positives in differential expression analysis.
this is very important because it ensures we can have the confidence in the differentially expressed genes declared.
with differential expression analysis providing a list of significant genes, here comes a key guiding principle of gene signature construction: only genes that are differentially expressed with stringent statistical significance should be passed on to the next stage.
one immediate question is: what if the number of significant genes in the differential expression analysis is so small that no significant connections could be established in the subsequent connectivity mapping step.
in such a situation, the guiding principle above should still prevail.
a more frequently encountered situation nowadays is that the microarray experiments included a large number samples and the studies were adequately powered, and a long list of genes have passed the significance criteria in the differential expression analysis.
but should we include all of them in the subsequent connectivity mapping step?
is it necessarily the better if we include more genes in the query signature?
to address these issues, the second step “gene signature progression” comes into play.
gene signature progression
given a long list of differentially expressed genes  obtained rigorously, how should we proceed to gene expression connectivity mapping using these genes as input?
before describing the detailed procedure of gene signature progression, it is useful to outline the rationales and motivation behind the idea.
the gene signature progression procedure was conceived based on the following argument: although a long list of significant genes have been identified with high statistical rigor, these genes are not equally important in characterizing the biological state of interest, and hence they should have different weights.
we need a way to quantify the importance and contribution of these genes.
here we adopt similar guiding principles used in the construction of reference gene expression profiles for sscmap.
briefly,  up-regulated genes and down-regulated genes should be treated on a equal-footing basis.
genes that are differentially expressed to a greater extent or those with greater significance should have more weight and thus receive higher ranks.
note that in reference  <cit>  genes were sorted by the absolute value of their log expression ratio .
in the present case of gene signature construction we have the option of using statistical significance p value to sort and rank the genes.
here we describe two ways of sorting the genes and ranking them.
m1: sorting the genes by their p values
the first natural solution to ranking the genes is to sort them by their p values in ascending order, with the smallest p value ranked the highest.
the rationale behind this method is that the smaller the p value, the more significant the differential gene expression.
m2: sorting the genes by their absolute log expression ratios
the argument supporting this method is that all these genes have passed the statistical significance criteria in the differential expression analysis step, and they are all genuine differentially expressed genes.
now it is a matter of ranking them based on their biological effects.
in this case, the absolute value of log expression ratio is chosen as the closest proxy to quantify their biological effect.
it should be noted that the criteria in m <dig> and m <dig> are not independent of each other, but they are strongly correlated.
in achieving a small p value in the differential gene expression analysis, the magnitude of differential expression  must be reasonably large as compared to its variance.
large log ratio or small variance alone would not necessarily lead to statistical significance.
one could argue that method m <dig> is already well balanced because the p value already contains information on the magnitude of differential expression and the consistency of the gene’s behavior across replicates.
in the two case studies of this paper, we applied m <dig> in sorting and ranking the genes.
after the genes have received their absolute ranks using one of the methods above, they are given signed ranks which are just the absolute ranks multiplied by + <dig> or − <dig> depending on the direction of their differential expression, in a similar way as in the construction of reference gene expression profiles  <cit> .
once signed ranks have been assigned to all genes in the deg list, we use this gene list as an input to subsequent connectivity mapping analysis, assuming that the genes with the highest  ranks  represent the most important features of the biological state.
in order to return the most relevant connections to our biological state of interest, we devised a “gene signature progression” procedure to systematically determine the minimal  length of the gene signature that can accurately represent the biological state.
the gene signature progression method proceeds as follows:
forming a gene signature
given a ranked list of genes, we make a gene signature consisting of the top k ranked genes, denoted by sk, where k is an integer from  <dig> to n, and n is the total number of genes in the initial deg list passed on from the differential gene expression analysis.
querying sscmap
we use the signature sk as an input to query the sscmap, which returns the connection scores of sk to all the reference profile sets  and their corresponding p-values.
calculating the empirical fdr of the returned connections
we set a threshold p-value to determine if any returned connections are statistically significant.
in our analysis the threshold p-value is set as nfp/nsets, where nfp is the number of false connections we are prepared to tolerate and expected to have in the results, and nsets is the total number of reference sets we are querying.
under this setting if we observe np as the total number of significant connections, we use nfp/np to estimate the empirical false discovery rate   <cit> .
in this way for each run of connectivity mapping using a gene signature sk, we get an efdr, which is a function of the signature length k.
progressively looping over the gene signatures sk’s
starting from k= <dig> with step increment until k=n, to obtain the efdr for each sk; break out from the loop when the first efdr ≤α is observed, where α is a pre-set fdr threshold.
target fdr achieved
take the break-out k from the previous step as the optimal length of the gene signature.
it is possible that after looping over all the progression signatures, none has returned an efdr ≤α.
in such cases, the gene signature with the smallest efdr could be taken as the optimal one.
this is not ideal in terms of the efdr achievable, but nevertheless the result is just a reflection of the underlying biology.
from a user’s point of view, one has then to decide, given the achieved false discovery rate, whether one wants to follow up the findings or not.
RESULTS
a “simulated experiment”
first, we carried out a “simulated experiment” to see if the gene signature progression approach indeed allows the capture of the most prominent feature of the biological state for which we know the truth.
for this purpose, we created a gene signature for histone deacetylase  inhibitor based on the internal reference profiles of cmap database.
among the over  <dig> drugs profiled in the camp reference dataset, the following drugs are known to be hdac inhibitors : vorinostat, trichostatin a, valproic acid, hc toxin, sodium phenylbutyrate, scriptaid, and ms- <dig>  the number of individual reference profiles for these drugs are listed in table  <dig>  to obtain an adequate sample size and also to avoid possible dominance of results by the drugs with large number of replicates, eg, trichostatin a and valproic acid, we used the following four drugs: hc toxin, sodium phenylbutyrate, scriptaid, and ms- <dig>  which have relatively small numbers of replicates in the dataset, to create the hdac inhibitor gene signature.
we combined their data to obtain a single list of ranked significant genes that are associated with these hdac inhibitors .
the genes were then sorted by the absolute values of their sum rank scores, and finally we obtained an ordered list of genes with signed ranks.
as this was the result of combing four hdac inhibitors specified above, this gene list was expected to represent the biological state as induced by these hdac inhibitors.
using this list of ranked genes as input, we performed the gene signature progression analysis, k= <dig> was determined to be the minimum length of gene signature that returned a number of significant drugs connection with an overall fdr no greater than  <dig> .
the results of sscmap connectivity mapping analysis using this minimum length gene signature are presented in table  <dig>  as can been seen from this table, not only the four input drugs, hc toxin, sodium phenylbutyrate, scriptaid, and ms- <dig>  were identified as significantly connected to the gene signature , the other major hdac inhibitors, vorinostat, trichostatin a, and valproic acid were also pulled out as significantly connected to the hdaci signature.
it should be emphasized that the reference data for these  <dig> drugs, were not involved at all in creating the hdaci gene signature.
so these results are reassuring because the gene expression connectivity mapping framework and the gene signature progression approach has worked very well in this “simulated experiment”.
from this experiment, we see the added value of the gene signature progression approach to connectivity mapping.
as designed, it helped to capture and highlight the most important feature of the biological state being studied.
in this case, it was hdac inhibition.
without the gene signature progression step, an arbitrarily chosen signature length would be often used, for example, k= <dig> or k= <dig>  using signatures of these lengths to perform gene expression connectivity mapping, these hdac inhibitors would still be returned as significant connections, but they were returned together with many other drugs .
due to the co-existence of many other seemingly significant drugs, the main biological theme  could potentially be weakened or even masked.
therefore, this example demonstrates that the signature progression approach does what it was designed for: to help users capture the most important biological theme with sharp focus and high precision.
all  <dig> known hdac inhibitors, including those not used in signature generation, were pulled out.
sm  = <dig> indicates that the connection p value is less than the preset threshold e
fp/n
sets=1/1309≈ <dig> ×10−4; ps  is also shown here
a case study with lung cancer
as an example application of the gene signature progression method introduced here, we analyzed a public dataset from an independent study  <cit> , where the tumor and paired normal tissues from  <dig> non-smoker female lung cancer patients were profiled using affymetrix human genome u <dig> plus  <dig>   microarrays.
we downloaded the  <dig> raw affeymetrix cel files from the gene expression omnibus  dataset gse <dig>  for connectivity mapping analysis we wanted to compile a gene signature describing the deviation of tumors from normals, so that we could have an accurate representation of the cancerous state.
raw microarray data were pre-processed using the affymetrix mas <dig> algorithm with the scaling parameter sc =  <dig> as implemented in the r package affy  <cit> .
we then worked on the log <dig> scale by transforming mas <dig> values to log2mas <dig>  as the reference profiles in the connectivity mapping database were generated using affymetrix human genome u133a  microarrays, the common probesets  between the hg-u133_plus_ <dig> and the hg-u11a arrays were extracted.
a paired-sample t test was applied to each of the common probesets to identify which is differentially expressed between the lung cancer condition and the normal condition.
with a sample size of  <dig> the study was well powered for identifying differential gene expression.
as a result,  <dig> probesets had a p-value less than the threshold discussed above efp/n=1/22277≈ <dig> ×10− <dig>  this gave an estimated efdr of 1/6313≈ <dig> ×10− <dig>  so we were very confident that these differentially expressed genes all reflected some aspects of the real biology.
but feeding all these genes as part of the gene signature to connectivity mapping would be difficult as the gene signature is too long and it would overwhelm the connectivity mapping algorithm.
it is also very difficult for human comprehension to include such a large number of “features” if one wants to get a better grasp of the biology under study.
this example demonstrates the needs for some form of filtering and selection procedure so that the major theme of the biology be captured and the big picture is not buried under many minor details.
to this end we set to filter out the probesets that were less prominent.
we took the following steps to shorten the list of selected genes:  only genes that met the statistical significance criteria were passed on to the next step; those regarded as statistically non-significant were not considered further.
in this case we set to tolerate an expected number of false positives efp= <dig> as outlined above.
following the previous step, we required genes to have a minimum mean expression level across the two conditions.
specifically, genes with mean log2mas <dig> values less than  <dig> in both conditions were excluded from further analysis.
this minimum value  <dig> for log2mas <dig>  although somewhat arbitrary, was based on our extensive experience dealing with microarray data.
the rationale of this filtering was that for genes with low expression levels in both conditions, we were less confident about their differential expression status, and also because of their low expression levels, their biological significance was considered less important than those with higher expression.
following the previous step, we required genes to have a minimum expression difference between the two conditions.
specifically, genes with an absolute expression difference less than  <dig> were excluded from further analysis.
as we work on log <dig> scale data, an absolute difference of  <dig>  is equivalent to a fold change  of  <dig> in mas <dig> expression values.
so essentially we required genes to have a minimum fold change of  <dig>  either up or down.
after the three filtering steps described, we still had  <dig> probesets to work with.
these served as our input deg list to the gene signature progression analysis with connectivity mapping.
these  <dig> probesets were sorted in ascending order of their p-values, the ones on top of the sorted list were regarded as most important.
we were interested in compounds with potential inhibitory effects on the tumorous state, therefore we looked for the connections with negative connection scores and used a one-sided test to calculate the p-value associated with the signature-to-drug connection.
briefly, for an observed connection score between the query signature and a reference expression profile, we generated  <dig> random gene signatures of the same length, and calculated the connections scores between these random signatures and the reference profile.
each random gene signature is just a list of genes randomly sampled  from all genes profiled in the reference dataset, with their up- or down- regulation status randomly set, as detailed in  <cit> .
the proportion of random scores that were less than the observed connection score gave an estimate of one-sided  p-value, which was then used to determine whether the observed score was statistically significant or not.
in our connectivity mapping analysis, the query gene signature was compared to each set of reference profiles derived from the same compound; a connection score between the query signature and each reference set was calculated and the corresponding left-sided p-value was estimated as described above.
the results returned from sscmap with a given query signature included the list of compounds examined, their connection scores to this query gene signature, and the associated p-values.
we followed the gene signature progression procedure by generating progression signatures of increasing length k consisting of the top k probesets from the sorted deg list obtained.
our target false discovery rate was set moderately as  <dig> .
figure  <dig> shows a few snapshots in the gene signature progression process.
panel  of this figure, for example, shows the results of sscmap using the gene signature s <dig> as the query input, consisting of the top  <dig> ranked genes from the differential expression analysis.
as can be seen in this case,  <dig> compounds were above the horizontal line, the position corresponding to the threshold p-value.
as we expected  <dig> false connections among these significant compounds, the empirical fdr was calculated as 1/7≈ <dig> , which was higher than the pre-set fdr target  <dig> .
therefore we needed to progress the gene signature further after this k= <dig> sscmap run.
by following the gene signature progression procedure, we found that k= <dig> was the minimum signature length that returned significant connections with an efdr ≤ <dig> .
this list of gene was taken as our optimal gene signature for the lung cancer-vs-normal comparison  and was then fed to sscmap to identify compounds that could inhibit the lung tumorous phenotype, or more precisely, to identify compounds with potential to push the tumorous state closer to the normal state.
gene-signature perturbation method was employed in the subsequent connectivity mapping analysis.
the details of the gene signature perturbation process, its motivation and rationale can be found in our previous work  <cit> .
in table  <dig> we show the results of connectivity mapping using this k= <dig> signature, with a perturbation stability score obtained for each drug.
fig.
<dig> snapshots of the gene signature progression process in the lung cancer case study.
each panel shows results of sscmap with the top k ranked genes as the query signature, with k= <dig> , <dig>  and  <dig>  respectively for –.
this figure exemplifies the intermediate results of individual steps in the gene signature progression process.
the blue horizontal line indicates the position corresponding to the threshold p-value.
any points above this line are considered statistically significant.
the number of compounds significantly connected to the query gene signature  are  <dig> , <dig>  and  <dig> respectively for panels –.
k= <dig> is the optimal signature length at which the preset fdr target ≤ <dig>  was first achieved
sm  = <dig> indicates that the connection p value is less than the preset threshold e
fp/n
sets=1/1309≈ <dig> ×10−4; ps  is also shown here
breast cancer case study
as the second example of applying the gene signature progression method, we analyzed the geo dataset gse <dig> from an independent study on human breast cancers from malaysian patients of different ethnicity   <cit> .
in total  <dig> pairs of tumor and normal tissues were profiled using affymetrix hg-u133a microarrays.
similar to the lung cancer case study, the microarray experiment raw cel files were downloaded from geo, and processed using the mas <dig> algorithm to obtain the gene expression values.
log <dig> transformation was applied to the mas <dig> expression values, and subsequent analyses were carried out using log2mas <dig> values.
this dataset was based on the same microarray platform as the reference profiles in sscmap, namely the affymetrix hg-u133a microarray platform with  <dig> probestes.
paired sample t test was applied to each of the  <dig> probesets individually and a two-sided p-value obtained for each probeset.
the threshold p-value was set as 1/22283≈ <dig> ×10− <dig>  any probeset with a p-value less than this threshold was considered as statistically significant.
for this case study,  <dig> probesets met this criterion and were passed on to subsequent filtering.
after filtering out probesets whose absolute differential expression is lower than  <dig>  and those with mean expression levels lower than  <dig>  in both conditions, finally  <dig> probesets were left to form the query gene signature.
the filtering process described above ensured that the genes  included in the query gene signature met the criteria of both statistical significance and biological significance.
these  <dig> probesets were finally sorted by their p-values in ascending order.
the complete list of these probesets and their corresponding gene symbols can be found in additional file  <dig>
following the gene signature progression process we identified that k= <dig> was the minimum length for the gene signature to return significant drug connections with efdr ≤ <dig> , hence this was taken as the optimal length of gene signature in our connectivity mapping analysis.
gene signature perturbation was then applied to this k= <dig> gene signature, and the perturbation stabilities calculated for each connection obtained.
in table  <dig> we show the results of sscmap using this k= <dig> gene signature and with gene signature perturbation procedure applied.
table  <dig> summarizes the key figures of the two case studies described in this paper.
discussion
it is worth noting that there are a couple common drugs from the two case studies presented here.
these are: trichostatin a  and exemestane.
trichostatin a is primarily an antifungal antibiotic and also a potent inhibitor of histone deacetylase  activity.
its antitumor activity against breast cancer in vitro and in vivo has already been demonstrated  <cit> .
several studies showed that tsa can induce apoptosis in human breast cancer cells, for instance, via targeting the mitochondrial respiratory chain and increasing mitochondrial reactive oxygen species  <cit> , or through the involvement of 15-lox- <dig>  <cit> .
while the use of exemestane in breast cancer has been long documented  <cit> , there is evidence that this compound also has antiproliferative effect in lung cancer cells  <cit> , and it was even suggested as a new treatment option for mesothelioma patients  <cit> .
among the candidate compounds returned from the lung cancer case study, rofecoxib was shown effective inhibiting the growth of small tumors in xenograft models of non-small cell lung cancer, suggesting its potential value as adjuvant therapy after surgery  <cit> .
perhaps, the most interesting result is that gefitinib is among the significant drug candidates.
it is interesting to note that our gene signature in the lung cancer case study was based on expression data from taiwan female non-smoker lung cancer patients, while gefitinib has been reported in an independent study to be highly effective in treating women, non-smoker or former light smoker, advanced non-small cell lung cancer patients of asian origin  <cit> .
this result is indeed “spot-on” in obtaining pertinent candidate drugs.
in the two case studies presented here, we have used p-values to sort and rank the genes .
but this is not the only way that genes can be sorted and ranked.
the other method  could be employed as well.
as both ranking methods  and  described in the “methods: a two-stage process” section have their merits, sometimes it can be difficult to argue for one against another.
the third way of ranking the genes can be a weighted combination of  and .
for example, we can rank the genes using method  and method  separately, so that each gene receives two ranks, rp and re, based on p value  and on effect , respectively.
then a weighted average of the two ranks can be calculated by r=wprp+were, where wp+we= <dig>  the weighted average ranks eventually determine the final ranking of genes.
finally it is also possible to incorporate external biological knowledge to adjust the ranking achieved above.
researchers will have to draw upon their expertise knowledge in the subject area and to make a decision on whether a given genes’s rank should be boosted or lowered.
CONCLUSIONS
in this paper we introduced a gene signature progression method into connectivity mapping, which enables a standardized procedure for constructing high quality gene signatures.
the essence is to determine the minimum length of the gene signature that allows significant connections returned with a target false discovery rate.
this progression method is particularly useful when the number of differentially expressed genes identified is large, and when there is a need to rank and prioritize them to be included in the final query signature.
on the other hand, if the number of differentially expressed genes is small, the two-stage process proposed in this paper stipulates that only the degs, but not others, should be included in the final gene signature.
this means that, if in the end, after including all the degs into the query signature, the false discovery rate of returned drugs is too high, one should then be cautious in following up the returned drugs, because too high a fdr would mean a low rate of success in any experimental validation attempt.
so it is important that the statistical rigor set out in the proposed two-stage process should be adhered to.
the results from the two case studies demonstrate that the approach developed here is capable of obtaining pertinent candidate drugs with high precision.
future development in this area is likely to involve incorporating existing biological knowledge stored in various biological databases in a systematic and automated way, to augment the current approach based on expression data.
ethics approval and consent to participate
not applicable.
consent for publication
not applicable.
availability of data and materials
the datasets supporting the conclusions of this article are included within the article and its additional files.
additional files
additional file  <dig> significant genes associated with hdac inhibitors.
the full list of  <dig> genes  and their rank order, obtained in the “simulated experiment” with  <dig> hdac inhibitors of the cmap reference profiles.
additional file  <dig> significant drugs returned with the k= <dig> hdaci gene signature.
the list of significant drugs returned from sscmap using the k= <dig> hdaci gene signature from the “simulated experiment”.
all  <dig> known hdac inhibitors were pulled out, but together with many other drugs, which may or may not related to hdac inhibition.
the main biological theme  could potentially be weakned or even masked due to the co-existence of many other drugs.
additional file  <dig> lung cancer gene signature.
the full list of  <dig> genes  and their ranks for the optimal gene signature obtained from the lung cancer dataset gse <dig>
additional file  <dig> breast cancer gene signature.
the full list of  <dig> genes  and their ranks for the optimal gene signature obtained from the breast cancer dataset gse <dig>
abbreviations
cmapconnectivity map
degdifferentially expressed gene
efdrempirical false discovery rate
fdrfalse discovery rate
geogene expression omnibus
hdachistone deacetylase
hdacihistone deacetylase inhibitor
tsatrichostatin a
competing interests
the authors declare that they have no competing interests.
authors’ contributions
qw, pwh and sdz designed the study.
qw collected the data, developed algorithms, and drafted the manuscript.
qw and csk analysed the data.
all authors contributed to writing the manuscript and provided technical input.
all authors read and approved the final manuscript.
funding
this work was supported by an epsrc doctoral training grant and by the uk bbsrc/mrc/epsrc co-funded grant bb/i009051/ <dig>  the research leading to the results has received funding from the people programme  of the european union’s seventh framework programme fp7/2007-2013/ under rea grant agreement no  <cit> .

 over the past few years gene expression connectivity mapping has gained much popularity among biomedical researchers because of its promising applications as demonstrated by an increasing number of studies in different research areas: drug discovery, drug re-positioning, predictive toxicology, and chemical carcinogenicity assessment, to name a few. the connectivity map concept was first introduced by lamb et al. with the idea of using gene expression profiles/signatures to represent different biological states, and then to establish the connections among these states based on their gene expression patterns. the establishment of a connection between two biological states may have different implications. if the connection was made because the key set of genes were similarly up- or down-regulated in the two states, it may indicate that the two states have the same activated biological processes or pathways, such as the autophagy process and an autophagy enhancing compound fasudil identified in, and consequently such a connection would be very useful for gaining insights into the underlying mechanism of one state based on the knowledge of the other, and vice versa. on the other hand if the connection was made because the key set of genes were oppositely regulated, it may indicate that the two states negate each other. if one is an undesirable state such as a disease, and the other is a chemical compound-induced state, the compound may be useful to treat that particular disease, as in the case of application to inflammatory bowel disease, and human skeletal muscle atrophy. from a user point of view, one important question often asked is: what is the best way of making a gene expression signature that can represent the biological state of interest accurately and be able to return meaningful biological connections? within the gene expression connectivity mapping framework, there are three key components: reference gene expression profiles, query gene signatures, and expression pattern matching algorithms. to date, much of the methodological development has been focused on the first and the third components above. one important aspect related to the development has been the efforts to reduce the high number of possible confounding factors and consequent batch effects within the heterogeneous cmap datasets, eg., by utilising filtering and normalisation steps to improve the signal to noise ratio, or by explicitly considering the variability in the transcriptional response and capturing the “consensus” response to a compound across multiple cell lines and dosages. in this paper, we are mainly concerned with the construction of query gene signatures, which was previously left to users without much guidance. for discussing the rationale of the “gene signature progression” method to be introduced, we first outline the differences between reference profiles and gene signatures in the context of connectivity mapping. the idea of using some form of molecular profile to characterize a biological state is not completely new. in essence this idea was the basis of many clinical practices. for example, various biochemical tests measure the molecular profile of a patient, which can then be used to infer or learn about the biological state of the patient. the main assumption behind the connectivity mapping concept is the generalization of molecular profile measurement. in this case the bio-marker molecules are the messenger rnas of genes in an organism. because of the sheer number of different mrnas measured, they collectively provide a much richer and complex description of the biological state than a few marker molecules could achieve. theoretically if we have identified all the different molecules and their abundance in a cell, we will have an exact specification of the cell’s biological state. the main assumption underlying the connectivity mapping concept was that the specification of the mrna profiles as measured by microarrays can provide an adequate description of the biological state. with this premise, we discuss here the roles played by reference profiles and gene signatures in the connectivity mapping framework. as described previously, a reference gene expression profile contains the full list of genes that were measured by the microarrays, and specifies their abundance in one condition relative to another. because a reference profile contains information about each gene, it is intended to provide a comprehensive description of the biological state it represents. a query gene signature, on the other hand, is a short list of genes that capture the most important features of a biological state. its difference to a reference profile is that a gene signature is not meant to provide the full description of how each gene has changed its expression, but rather to provide the most prominent features in that biological state. obviously any biological condition will have many features/aspects, but they are not of equal importance. for establishing the connection between two biological states, one needs to decide what features to include in the gene signature. the aim of gene signature construction is to compile a list of genes that collectively capture the most prominent features of the biological state it is intended to represent. here in this paper we propose a two-stage process to achieve that goal. the initial input is assumed to be from microarray gene expression data that have been properly processed through some common steps, such as quality control, background correction, normalization, scaling or transformations, where appropriate. these pre-processing steps are very important in microarray data analysis and some recent research focused on these aspects, eg, but they are not in the scope of the present work. our starting point is that we have obtained the gene expression dataset, from which the expression levels of the genes can be quantified in the available biological samples. given these data, we propose the following two-stage process for the construction of gene expression signature in connectivity mapping: differential gene expression analysis to provide an ordered list of significant genes. gene signature progression analysis to systematically determine the minimum length of the gene signatures able to return significant connections to reference drugs with an acceptable false discovery rate. detailed description of the approach is presented in the following “methods: a two-stage process” section. methods: a two-stage process a schematic workflow of the two-stage process is depicted in fig. the aim of the first stage, differential expression analysis, is to derive a ranked list of degs together with their regulation status, which can serve as an input to the general connectivity mapping process. the second stage, gene signature progression analysis, involves creating multiple gene signatures consisting of an increasing number of top ranked genes, and then querying the sscmap iteratively with gene signatures of increasing length until a number of significant drugs are returned with a pre-set acceptable false discovery rate. fig. flow chart of the two stage process. the output from stage is a ranked list of significant genes with their regulation status. this is used as input to stage gene signature progression. starting from k= the top k genes are fed to sscmap as a query signature to pull out significant drugs. this process is run iteratively with increasing k until a pre-set target fdr is achieved for the returned significant drugs the sscmap connectivity mapping framework was developed previously to introduce a more principled statistical test in connectivity mapping. it was bundled with compound-induced reference gene expression profiles as its core database. when a user-supplied query gene signature is presented to it, sscmap calculates a connection score between the query signature and each set of reference profiles in the core database, then performs computationally intensive permutation tests, to assess the statistical significance of each observed connection score. a number of drugs with significant connection to the query signature are then returned as the results of this process. differential gene expression analysis in general, differential gene expression analysis involves two or more biological conditions. for gene signature construction in connectivity mapping, we are mainly concerned with cases where they are two conditions. one of them is a control condition which serves as a reference point. the other condition is the state of our interest, for example, a disease state or a state as a result of some form of biological, chemical, or genomic perturbation experiment. this is similar to the construction of reference gene expression profiles, where a vehicle control condition and a drug treated condition are required. an important issue in the differential gene expression analysis is the multiple testing correction that must be considered when conducting a large number of statistical tests at the same time. when tens of thousands of genes are being tested in the same analysis, the conventional statistical significance level of, which was developed for single statistical test, is no longer adequate. purely by chance, % of the genes tested will turn out to have a p-value less than even if there is no difference at all between the two biological conditions being compared. to safeguard the findings of differential gene expression against too many false positives creeping in, the false discovery rate has become a commonly accepted measure to control in situations where multiple hypothesis testing is involved. this measure allows users to estimate the number of false positives given the total number of positive results obtained. an equivalent way of achieving this goal is to control the expected number of false positives directly, and then calculate the empirical false discovery rate. for example, to control the expected number of false positives to be efp, one can set the threshold p-value as efp/n, where n is the total number of hypotheses being simultaneously tested, and in the case of microarray differential gene expression analysis, n is the total number of genes measured by the microarrays. in this paper we set efp= to apply a tight control on the expected number of false positives in differential expression analysis. this is very important because it ensures we can have the confidence in the differentially expressed genes declared. with differential expression analysis providing a list of significant genes, here comes a key guiding principle of gene signature construction: only genes that are differentially expressed with stringent statistical significance should be passed on to the next stage. one immediate question is: what if the number of significant genes in the differential expression analysis is so small that no significant connections could be established in the subsequent connectivity mapping step. in such a situation, the guiding principle above should still prevail. a more frequently encountered situation nowadays is that the microarray experiments included a large number samples and the studies were adequately powered, and a long list of genes have passed the significance criteria in the differential expression analysis. but should we include all of them in the subsequent connectivity mapping step? is it necessarily the better if we include more genes in the query signature? to address these issues, the second step “gene signature progression” comes into play. gene signature progression given a long list of differentially expressed genes obtained rigorously, how should we proceed to gene expression connectivity mapping using these genes as input? before describing the detailed procedure of gene signature progression, it is useful to outline the rationales and motivation behind the idea. the gene signature progression procedure was conceived based on the following argument: although a long list of significant genes have been identified with high statistical rigor, these genes are not equally important in characterizing the biological state of interest, and hence they should have different weights. we need a way to quantify the importance and contribution of these genes. here we adopt similar guiding principles used in the construction of reference gene expression profiles for sscmap. briefly, up-regulated genes and down-regulated genes should be treated on a equal-footing basis. genes that are differentially expressed to a greater extent or those with greater significance should have more weight and thus receive higher ranks. note that in reference genes were sorted by the absolute value of their log expression ratio. in the present case of gene signature construction we have the option of using statistical significance p value to sort and rank the genes. here we describe two ways of sorting the genes and ranking them. m1: sorting the genes by their p values the first natural solution to ranking the genes is to sort them by their p values in ascending order, with the smallest p value ranked the highest. the rationale behind this method is that the smaller the p value, the more significant the differential gene expression. m2: sorting the genes by their absolute log expression ratios the argument supporting this method is that all these genes have passed the statistical significance criteria in the differential expression analysis step, and they are all genuine differentially expressed genes. now it is a matter of ranking them based on their biological effects. in this case, the absolute value of log expression ratio is chosen as the closest proxy to quantify their biological effect. it should be noted that the criteria in m and m are not independent of each other, but they are strongly correlated. in achieving a small p value in the differential gene expression analysis, the magnitude of differential expression must be reasonably large as compared to its variance. large log ratio or small variance alone would not necessarily lead to statistical significance. one could argue that method m is already well balanced because the p value already contains information on the magnitude of differential expression and the consistency of the gene’s behavior across replicates. in the two case studies of this paper, we applied m in sorting and ranking the genes. after the genes have received their absolute ranks using one of the methods above, they are given signed ranks which are just the absolute ranks multiplied by + or − depending on the direction of their differential expression, in a similar way as in the construction of reference gene expression profiles. once signed ranks have been assigned to all genes in the deg list, we use this gene list as an input to subsequent connectivity mapping analysis, assuming that the genes with the highest ranks represent the most important features of the biological state. in order to return the most relevant connections to our biological state of interest, we devised a “gene signature progression” procedure to systematically determine the minimal length of the gene signature that can accurately represent the biological state. the gene signature progression method proceeds as follows: forming a gene signature given a ranked list of genes, we make a gene signature consisting of the top k ranked genes, denoted by sk, where k is an integer from to n, and n is the total number of genes in the initial deg list passed on from the differential gene expression analysis. querying sscmap we use the signature sk as an input to query the sscmap, which returns the connection scores of sk to all the reference profile sets and their corresponding p-values. calculating the empirical fdr of the returned connections we set a threshold p-value to determine if any returned connections are statistically significant. in our analysis the threshold p-value is set as nfp/nsets, where nfp is the number of false connections we are prepared to tolerate and expected to have in the results, and nsets is the total number of reference sets we are querying. under this setting if we observe np as the total number of significant connections, we use nfp/np to estimate the empirical false discovery rate. in this way for each run of connectivity mapping using a gene signature sk, we get an efdr, which is a function of the signature length k. progressively looping over the gene signatures sk’s starting from k= with step increment until k=n, to obtain the efdr for each sk; break out from the loop when the first efdr ≤α is observed, where α is a pre-set fdr threshold. target fdr achieved take the break-out k from the previous step as the optimal length of the gene signature. it is possible that after looping over all the progression signatures, none has returned an efdr ≤α. in such cases, the gene signature with the smallest efdr could be taken as the optimal one. this is not ideal in terms of the efdr achievable, but nevertheless the result is just a reflection of the underlying biology. from a user’s point of view, one has then to decide, given the achieved false discovery rate, whether one wants to follow up the findings or not.  a “simulated experiment” first, we carried out a “simulated experiment” to see if the gene signature progression approach indeed allows the capture of the most prominent feature of the biological state for which we know the truth. for this purpose, we created a gene signature for histone deacetylase inhibitor based on the internal reference profiles of cmap database. among the over drugs profiled in the camp reference dataset, the following drugs are known to be hdac inhibitors: vorinostat, trichostatin a, valproic acid, hc toxin, sodium phenylbutyrate, scriptaid, and ms- the number of individual reference profiles for these drugs are listed in table to obtain an adequate sample size and also to avoid possible dominance of results by the drugs with large number of replicates, eg, trichostatin a and valproic acid, we used the following four drugs: hc toxin, sodium phenylbutyrate, scriptaid, and ms- which have relatively small numbers of replicates in the dataset, to create the hdac inhibitor gene signature. we combined their data to obtain a single list of ranked significant genes that are associated with these hdac inhibitors. the genes were then sorted by the absolute values of their sum rank scores, and finally we obtained an ordered list of genes with signed ranks. as this was the result of combing four hdac inhibitors specified above, this gene list was expected to represent the biological state as induced by these hdac inhibitors. using this list of ranked genes as input, we performed the gene signature progression analysis, k= was determined to be the minimum length of gene signature that returned a number of significant drugs connection with an overall fdr no greater than. the results of sscmap connectivity mapping analysis using this minimum length gene signature are presented in table as can been seen from this table, not only the four input drugs, hc toxin, sodium phenylbutyrate, scriptaid, and ms- were identified as significantly connected to the gene signature, the other major hdac inhibitors, vorinostat, trichostatin a, and valproic acid were also pulled out as significantly connected to the hdaci signature. it should be emphasized that the reference data for these drugs, were not involved at all in creating the hdaci gene signature. so these results are reassuring because the gene expression connectivity mapping framework and the gene signature progression approach has worked very well in this “simulated experiment”. from this experiment, we see the added value of the gene signature progression approach to connectivity mapping. as designed, it helped to capture and highlight the most important feature of the biological state being studied. in this case, it was hdac inhibition. without the gene signature progression step, an arbitrarily chosen signature length would be often used, for example, k= or k= using signatures of these lengths to perform gene expression connectivity mapping, these hdac inhibitors would still be returned as significant connections, but they were returned together with many other drugs. due to the co-existence of many other seemingly significant drugs, the main biological theme could potentially be weakened or even masked. therefore, this example demonstrates that the signature progression approach does what it was designed for: to help users capture the most important biological theme with sharp focus and high precision. all known hdac inhibitors, including those not used in signature generation, were pulled out. sm = indicates that the connection p value is less than the preset threshold e fp/n sets=1/1309≈ ×10−4; ps is also shown here a case study with lung cancer as an example application of the gene signature progression method introduced here, we analyzed a public dataset from an independent study, where the tumor and paired normal tissues from non-smoker female lung cancer patients were profiled using affymetrix human genome u plus microarrays. we downloaded the raw affeymetrix cel files from the gene expression omnibus dataset gse for connectivity mapping analysis we wanted to compile a gene signature describing the deviation of tumors from normals, so that we could have an accurate representation of the cancerous state. raw microarray data were pre-processed using the affymetrix mas algorithm with the scaling parameter sc = as implemented in the r package affy. we then worked on the log scale by transforming mas values to log2mas as the reference profiles in the connectivity mapping database were generated using affymetrix human genome u133a microarrays, the common probesets between the hg-u133_plus_ and the hg-u11a arrays were extracted. a paired-sample t test was applied to each of the common probesets to identify which is differentially expressed between the lung cancer condition and the normal condition. with a sample size of the study was well powered for identifying differential gene expression. as a result, probesets had a p-value less than the threshold discussed above efp/n=1/22277≈ ×10− this gave an estimated efdr of 1/6313≈ ×10− so we were very confident that these differentially expressed genes all reflected some aspects of the real biology. but feeding all these genes as part of the gene signature to connectivity mapping would be difficult as the gene signature is too long and it would overwhelm the connectivity mapping algorithm. it is also very difficult for human comprehension to include such a large number of “features” if one wants to get a better grasp of the biology under study. this example demonstrates the needs for some form of filtering and selection procedure so that the major theme of the biology be captured and the big picture is not buried under many minor details. to this end we set to filter out the probesets that were less prominent. we took the following steps to shorten the list of selected genes: only genes that met the statistical significance criteria were passed on to the next step; those regarded as statistically non-significant were not considered further. in this case we set to tolerate an expected number of false positives efp= as outlined above. following the previous step, we required genes to have a minimum mean expression level across the two conditions. specifically, genes with mean log2mas values less than in both conditions were excluded from further analysis. this minimum value for log2mas although somewhat arbitrary, was based on our extensive experience dealing with microarray data. the rationale of this filtering was that for genes with low expression levels in both conditions, we were less confident about their differential expression status, and also because of their low expression levels, their biological significance was considered less important than those with higher expression. following the previous step, we required genes to have a minimum expression difference between the two conditions. specifically, genes with an absolute expression difference less than were excluded from further analysis. as we work on log scale data, an absolute difference of is equivalent to a fold change of in mas expression values. so essentially we required genes to have a minimum fold change of either up or down. after the three filtering steps described, we still had probesets to work with. these served as our input deg list to the gene signature progression analysis with connectivity mapping. these probesets were sorted in ascending order of their p-values, the ones on top of the sorted list were regarded as most important. we were interested in compounds with potential inhibitory effects on the tumorous state, therefore we looked for the connections with negative connection scores and used a one-sided test to calculate the p-value associated with the signature-to-drug connection. briefly, for an observed connection score between the query signature and a reference expression profile, we generated random gene signatures of the same length, and calculated the connections scores between these random signatures and the reference profile. each random gene signature is just a list of genes randomly sampled from all genes profiled in the reference dataset, with their up- or down- regulation status randomly set, as detailed in. the proportion of random scores that were less than the observed connection score gave an estimate of one-sided p-value, which was then used to determine whether the observed score was statistically significant or not. in our connectivity mapping analysis, the query gene signature was compared to each set of reference profiles derived from the same compound; a connection score between the query signature and each reference set was calculated and the corresponding left-sided p-value was estimated as described above. the results returned from sscmap with a given query signature included the list of compounds examined, their connection scores to this query gene signature, and the associated p-values. we followed the gene signature progression procedure by generating progression signatures of increasing length k consisting of the top k probesets from the sorted deg list obtained. our target false discovery rate was set moderately as. figure shows a few snapshots in the gene signature progression process. panel of this figure, for example, shows the results of sscmap using the gene signature s as the query input, consisting of the top ranked genes from the differential expression analysis. as can be seen in this case, compounds were above the horizontal line, the position corresponding to the threshold p-value. as we expected false connections among these significant compounds, the empirical fdr was calculated as 1/7≈, which was higher than the pre-set fdr target. therefore we needed to progress the gene signature further after this k= sscmap run. by following the gene signature progression procedure, we found that k= was the minimum signature length that returned significant connections with an efdr ≤. this list of gene was taken as our optimal gene signature for the lung cancer-vs-normal comparison and was then fed to sscmap to identify compounds that could inhibit the lung tumorous phenotype, or more precisely, to identify compounds with potential to push the tumorous state closer to the normal state. gene-signature perturbation method was employed in the subsequent connectivity mapping analysis. the details of the gene signature perturbation process, its motivation and rationale can be found in our previous work. in table we show the results of connectivity mapping using this k= signature, with a perturbation stability score obtained for each drug. fig. snapshots of the gene signature progression process in the lung cancer case study. each panel shows results of sscmap with the top k ranked genes as the query signature, with k=, and respectively for –. this figure exemplifies the intermediate results of individual steps in the gene signature progression process. the blue horizontal line indicates the position corresponding to the threshold p-value. any points above this line are considered statistically significant. the number of compounds significantly connected to the query gene signature are, and respectively for panels –. k= is the optimal signature length at which the preset fdr target ≤ was first achieved sm = indicates that the connection p value is less than the preset threshold e fp/n sets=1/1309≈ ×10−4; ps is also shown here breast cancer case study as the second example of applying the gene signature progression method, we analyzed the geo dataset gse from an independent study on human breast cancers from malaysian patients of different ethnicity. in total pairs of tumor and normal tissues were profiled using affymetrix hg-u133a microarrays. similar to the lung cancer case study, the microarray experiment raw cel files were downloaded from geo, and processed using the mas algorithm to obtain the gene expression values. log transformation was applied to the mas expression values, and subsequent analyses were carried out using log2mas values. this dataset was based on the same microarray platform as the reference profiles in sscmap, namely the affymetrix hg-u133a microarray platform with probestes. paired sample t test was applied to each of the probesets individually and a two-sided p-value obtained for each probeset. the threshold p-value was set as 1/22283≈ ×10− any probeset with a p-value less than this threshold was considered as statistically significant. for this case study, probesets met this criterion and were passed on to subsequent filtering. after filtering out probesets whose absolute differential expression is lower than and those with mean expression levels lower than in both conditions, finally probesets were left to form the query gene signature. the filtering process described above ensured that the genes included in the query gene signature met the criteria of both statistical significance and biological significance. these probesets were finally sorted by their p-values in ascending order. the complete list of these probesets and their corresponding gene symbols can be found in additional file following the gene signature progression process we identified that k= was the minimum length for the gene signature to return significant drug connections with efdr ≤, hence this was taken as the optimal length of gene signature in our connectivity mapping analysis. gene signature perturbation was then applied to this k= gene signature, and the perturbation stabilities calculated for each connection obtained. in table we show the results of sscmap using this k= gene signature and with gene signature perturbation procedure applied. table summarizes the key figures of the two case studies described in this paper. discussion it is worth noting that there are a couple common drugs from the two case studies presented here. these are: trichostatin a and exemestane. trichostatin a is primarily an antifungal antibiotic and also a potent inhibitor of histone deacetylase activity. its antitumor activity against breast cancer in vitro and in vivo has already been demonstrated. several studies showed that tsa can induce apoptosis in human breast cancer cells, for instance, via targeting the mitochondrial respiratory chain and increasing mitochondrial reactive oxygen species, or through the involvement of 15-lox-. while the use of exemestane in breast cancer has been long documented, there is evidence that this compound also has antiproliferative effect in lung cancer cells, and it was even suggested as a new treatment option for mesothelioma patients. among the candidate compounds returned from the lung cancer case study, rofecoxib was shown effective inhibiting the growth of small tumors in xenograft models of non-small cell lung cancer, suggesting its potential value as adjuvant therapy after surgery. perhaps, the most interesting result is that gefitinib is among the significant drug candidates. it is interesting to note that our gene signature in the lung cancer case study was based on expression data from taiwan female non-smoker lung cancer patients, while gefitinib has been reported in an independent study to be highly effective in treating women, non-smoker or former light smoker, advanced non-small cell lung cancer patients of asian origin. this result is indeed “spot-on” in obtaining pertinent candidate drugs. in the two case studies presented here, we have used p-values to sort and rank the genes. but this is not the only way that genes can be sorted and ranked. the other method could be employed as well. as both ranking methods and described in the “methods: a two-stage process” section have their merits, sometimes it can be difficult to argue for one against another. the third way of ranking the genes can be a weighted combination of and. for example, we can rank the genes using method and method separately, so that each gene receives two ranks, rp and re, based on p value and on effect, respectively. then a weighted average of the two ranks can be calculated by r=wprp+were, where wp+we= the weighted average ranks eventually determine the final ranking of genes. finally it is also possible to incorporate external biological knowledge to adjust the ranking achieved above. researchers will have to draw upon their expertise knowledge in the subject area and to make a decision on whether a given genes’s rank should be boosted or lowered.  in this paper we introduced a gene signature progression method into connectivity mapping, which enables a standardized procedure for constructing high quality gene signatures. the essence is to determine the minimum length of the gene signature that allows significant connections returned with a target false discovery rate. this progression method is particularly useful when the number of differentially expressed genes identified is large, and when there is a need to rank and prioritize them to be included in the final query signature. on the other hand, if the number of differentially expressed genes is small, the two-stage process proposed in this paper stipulates that only the degs, but not others, should be included in the final gene signature. this means that, if in the end, after including all the degs into the query signature, the false discovery rate of returned drugs is too high, one should then be cautious in following up the returned drugs, because too high a fdr would mean a low rate of success in any experimental validation attempt. so it is important that the statistical rigor set out in the proposed two-stage process should be adhered to. the results from the two case studies demonstrate that the approach developed here is capable of obtaining pertinent candidate drugs with high precision. future development in this area is likely to involve incorporating existing biological knowledge stored in various biological databases in a systematic and automated way, to augment the current approach based on expression data. ethics approval and consent to participate not applicable. consent for publication not applicable. availability of data and materials the datasets supporting the conclusions of this article are included within the article and its additional files. additional files additional file significant genes associated with hdac inhibitors. the full list of genes and their rank order, obtained in the “simulated experiment” with hdac inhibitors of the cmap reference profiles. additional file significant drugs returned with the k= hdaci gene signature. the list of significant drugs returned from sscmap using the k= hdaci gene signature from the “simulated experiment”. all known hdac inhibitors were pulled out, but together with many other drugs, which may or may not related to hdac inhibition. the main biological theme could potentially be weakned or even masked due to the co-existence of many other drugs. additional file lung cancer gene signature. the full list of genes and their ranks for the optimal gene signature obtained from the lung cancer dataset gse additional file breast cancer gene signature. the full list of genes and their ranks for the optimal gene signature obtained from the breast cancer dataset gse abbreviations cmapconnectivity map degdifferentially expressed gene efdrempirical false discovery rate fdrfalse discovery rate geogene expression omnibus hdachistone deacetylase hdacihistone deacetylase inhibitor tsatrichostatin a competing interests the authors declare that they have no competing interests. authors’ contributions qw, pwh and sdz designed the study. qw collected the data, developed algorithms, and drafted the manuscript. qw and csk analysed the data. all authors contributed to writing the manuscript and provided technical input. all authors read and approved the final manuscript. funding this work was supported by an epsrc doctoral training grant and by the uk bbsrc/mrc/epsrc co-funded grant bb/i009051/ the research leading to the results has received funding from the people programme of the european union’s seventh framework programme fp7/2007-2013/ under rea grant agreement no.
in summary, we introduced a gene signature progression method into connectivity mapping, which enables a standardized procedure for constructing high quality gene signatures. this progression method is particularly useful when the number of differentially expressed genes identified is large, and when there is a need to prioritize them to be included in the query signature. first, a differential gene expression analysis comparing two distinct biological states; only the genes that have passed stringent statistical criteria are considered in the second stage of the process, which involves ranking genes based on statistical as well as biological significance. previously methodological research in connectivity mapping mainly focused on two of the key components in the framework, namely, the reference gene expression profiles and the connectivity mapping algorithms. our gene signature was based on gene expression data from taiwan female non-smoker lung cancer patients, while there is evidence from independent studies that gefitinib is highly effective in treating women, non-smoker or former light smoker, advanced non-small cell lung cancer patients of asian origin. as a key input to the connectivity mapping process, gene signature is crucially important in returning biologically meaningful and relevant results. the results from two case studies demonstrate that the approach we have developed is capable of obtaining pertinent candidate drugs with high precision. we use a lung cancer dataset and a breast cancer dataset as two case studies to demonstrate how this standardized procedure works, and we show that highly relevant and interesting biological connections are returned. the other key component in this framework, the query gene signature, has been left to users to construct without much consensus on how this should be done, albeit it has been an issue most relevant to end users. this paper intends to formulate a standardized procedure for constructing high quality gene signatures from a user’s perspective. we describe a two-stage process for making quality gene signatures using gene expression data as initial inputs. we introduce a “gene signature progression” method as a standard procedure in connectivity mapping. starting from the highest ranked gene, we progressively determine the minimum length of the gene signature that allows connections to the reference profiles being established with a preset target false discovery rate. connectivity mappingdifferentially expressed genesgene signature progressiondisease inhibitory compoundslung cancerbreast cancerbbsrc/mrc/epsrc co-funded grantbb/i009051/1zhang shu-dong epsrcdoctoral training grantwen qing european union's seventh framework programme fp7/2007-2013rea grant agreement no hamilton peter w. issue-copyright-statement© the author 2016 of particular note is gefitinib, identified as among the candidate therapeutics in our lung cancer case study. gene expression connectivity mapping has gained much popularity recently with a number of successful applications in biomedical research testifying its utility and promise.
 gene expression connectivity mapping has gained much popularity recently with a number of successful applications in biomedical research testifying its utility and promise. previously methodological research in connectivity mapping mainly focused on two of the key components in the framework, namely, the reference gene expression profiles and the connectivity mapping algorithms. the other key component in this framework, the query gene signature, has been left to users to construct without much consensus on how this should be done, albeit it has been an issue most relevant to end users. as a key input to the connectivity mapping process, gene signature is crucially important in returning biologically meaningful and relevant results. this paper intends to formulate a standardized procedure for constructing high quality gene signatures from a user’s perspective.  we describe a two-stage process for making quality gene signatures using gene expression data as initial inputs. first, a differential gene expression analysis comparing two distinct biological states; only the genes that have passed stringent statistical criteria are considered in the second stage of the process, which involves ranking genes based on statistical as well as biological significance. we introduce a “gene signature progression” method as a standard procedure in connectivity mapping. starting from the highest ranked gene, we progressively determine the minimum length of the gene signature that allows connections to the reference profiles being established with a preset target false discovery rate. we use a lung cancer dataset and a breast cancer dataset as two case studies to demonstrate how this standardized procedure works, and we show that highly relevant and interesting biological connections are returned. of particular note is gefitinib, identified as among the candidate therapeutics in our lung cancer case study. our gene signature was based on gene expression data from taiwan female non-smoker lung cancer patients, while there is evidence from independent studies that gefitinib is highly effective in treating women, non-smoker or former light smoker, advanced non-small cell lung cancer patients of asian origin.  in summary, we introduced a gene signature progression method into connectivity mapping, which enables a standardized procedure for constructing high quality gene signatures. this progression method is particularly useful when the number of differentially expressed genes identified is large, and when there is a need to prioritize them to be included in the query signature. the results from two case studies demonstrate that the approach we have developed is capable of obtaining pertinent candidate drugs with high precision. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords connectivity mappingdifferentially expressed genesgene signature progressiondisease inhibitory compoundslung cancerbreast cancerbbsrc/mrc/epsrc co-funded grantbb/i009051/1zhang shu-dong epsrcdoctoral training grantwen qing european union's seventh framework programme fp7/2007-2013rea grant agreement no hamilton peter w. issue-copyright-statement© the author 2016 