BACKGROUND
in the post-genome era, an important task of molecular biology is to reconstruct gene regulatory networks , which represent interactions between genes inside a cell or tissue.
a grn provides molecular interactions and regulatory effects of components involved in a biological process, and hence provides insights into the molecular mechanism of the process  <cit> .
in detail, grns can be used to interpret biological processes through studying topological structure information of sub-networks related to these processes, where genes facilitate specific biological functions together  <cit> .
grns can help annotate genes clustered in modules and motifs since genes in the same module or motif have similar functions  <cit> .
grns can be utilized to identify dynamical network biomarkers  at the critical states of biological processes if stage-wise data are available, which help biologists understand mechanism of biological process better  <cit> .
therefore, reconstruction of grns can not only support investigating roles of genes and components involved in a biological process, but also help study how a process is developed and maintained.
in the last decade, many algorithms have been developed to infer grns based on reverse-engineering methods, such as bayesian network , boolean network  <cit> , linear and non-linear regression , differential equation  <cit> , information-theoretic approaches , probabilistic phylogeny network  <cit> , part mutual information network  <cit> , and probabilistic graphical models .
in  <dig>  we proposed a grn inference algorithm, named pca-cmi, which can distinguish direct interactions of gene pairs from indirect ones based on the conditional mutual information  measurement .
however, two limitations of the algorithm hinder its wide application.
one is that an appropriate threshold should be assigned to the method for direct interactions judgment in advance, which is difficult for users since the threshold is hard to select before grn reconstruction.
the other is that the method is time-costly especially for genomic network reconstruction, which is a common restriction of most current grn inferring methods.
in this report, we describe a new software package cmip, which implements the pca-cmi algorithm with the goal of enable biologists to build genomic networks easily.
the cmip package incorporates a threshold determination method and a parallel computing process for network inference.
the threshold determination method can choose an appropriate cutoff on-the-fly for gene interaction judgment.
computing procedure of the cmi measurement is optimized to make the algorithm robust, in which parallel computing strategies are applied to accelerate calculation process.
this paper describes the algorithm details, program implementation, prediction performance, and practical application of the cmip package.
methods
workflow of the cmip package
as showed in fig.
<dig>  the cmip package first uses an expression data file as input, in which expression value of genes under different experimental conditions are provided.
then, cmip program calculates the correlation value between gene pairs.
during calculation process, a threshold determination method is called to generate an appropriate cutoff for direct interaction judgment.
when the process is finished, two result files are produced as output.
one is a gene interaction file, recording raw correlation value of gene pairs.
the other is a gene relation file, providing the relationship between gene pairs.
in practice, relationship of a gene pair is assigned as  <dig> when their correlation value is over the interaction cutoff.
otherwise corresponding value is assigned as  <dig> fig.
<dig> workflow of the cmip software package.
first, expression datasets are used as input of the cmip algorithm.
then the cpu or gpu programs are selected to reconstruct networks.
finally, result files recording interaction and relationship of gene pairs are generated as output
correlation calculation of the cmip algorithm
the algorithm implemented in the cmip package is as follows.
first, correlation values of each gene pairs are calculated using the mutual information  measurement.
then a threshold determination method  is called to provide an appropriate interaction cutoff for gene pairs.
an interaction is marked for gene x and y when their raw correlation value is over the cutoff.
after that, for each gene interaction, their correlation values are updated through calculating the conditional mutual information  measurement , which describes the dependence of two genes given neighboring genes as condition.
a gene z is defined as a neighbor of gene x and y when it has interactions with both gene x and y. in practice, the maximum cmi value between gene x and y is kept.
finally, for gene x and y, they are regarded as having direct interaction when their cmi value is over the interaction cutoff and their relationship value is set to be  <dig> as output.
<dig> ix,y|z=∑x∈x,y∈y,z∈zpxyzlogpx,y|zpx|zpy|z where i is cmi measurement between gene x and y given gene z as a condition; p are joint probability of gene triple ; while p, p, and p are conditional probabilities of gene x, y, and gene pair  given gene z as a condition.
according to information theory, the cmi measurement can also be defined as follows.
<dig> hx,y|z=hxz+hyz−hz−hxyz where h is the entropy of gene z; h, h and h are joint entropies of gene pair ,  and gene triple ; h is the conditional entropy of genes x and y given gene z as a condition.
based on the gaussian distribution, the entropy of gene z can be estimated as follows.
<dig> hz=log2πen/2cz1/2=12log2πencz where n is number of experiment, c is the covariance matrix of gene z, and |c| is the determinant of the matrix.
while joint entropies can be estimated similarly through corresponding covariance matrixes.
based on the entropy estimator, in practice the cmi measurement  is calculated as follows.
<dig> ix,y|z=12logcxz|⋅|cyzcz|⋅|cxyz where c and c are covariance matrixes of gene pair  and ; c is covariance matrix of gene triple ; the |c()| is determinant of a matrix.
threshold determination of gene interaction
given interaction of gene pairs, the number of interactions decreases dramatically with the increase of the cutoff and their relationship shows an exponential decay.
therefore, in practice we chose to use an exponential function to simulate relationship between interaction and cutoff.
correlation values of gene pairs are first calculated as mentioned in the “correlation calculation of the cmip algorithm” section.
then direct interactions between gene pairs under different cutoffs are estimated and a scatter plot is generated , where x axis is the cutoff value and y axis is the number of direct interactions.
after that, we fit the number of direct interactions as a function of the cutoff value with an exponential function.
finally, we chose the threshold as the intersection of slope of the start and end sections of the fitting curve, which represents the inflection point of the curve.fig.
<dig> diagram of threshold determination for gene interactions.
relationship between interaction and cutoff is first investigated, and then a fitting curve method based on exponential function is adopted to simulate relationship between them.
finally, the intersection of slope of the start and end sections of the fitting curve was chosen as the threshold
parallelization of the cmip programs
in cmip, parallel strategies were applied to speed up computing process of correlation.
in practice, a cpu and a gpu version program of cmip algorithm were developed so that users could utilize them in different computational environment.
the cpu version program is implemented based on the openmp framework  <cit> , where loop calculation is accelerated with the multi-threads technology.
in detail, the total computing task of correlation is first calculated based on gene numbers, and then computing tasks is partitioned equally to each cpu node.
while the gpu version program is implemented based on the cuda framework  <cit> , where serial and parallel computing tasks are undertaken by cpu and gpu cores respectively.
in detail, a production-consumption strategy is used in the gpu version program, in which gene expression data used by correlation calculation is first processed by the cpu cores ; then pre-processed data is delivered to gpu cores for correlation calculation  using a parallel mode; finally, the results are transferred from gpu to cpu cores for aggregation.
evaluation of network inference methods
receiver operating characteristic  curve and precision-recall  curve are used to evaluate performance of different network inference methods.
the roc curve is a graphical plot which illustrates discrimination capacity of algorithm under various thresholds for binary classifier problems, where the x and y axis are false and true positive rate respectively.
while the pr curve shows recognition capability of algorithm under various thresholds for positive samples, in which the x and y axis represent the recall and precision measurement respectively.
commonly, area under the roc curve  and area under the pr curve  are calculated to comprehensively evaluate performance of a network inference method.
in practice, the true positive rate , false positive rate , and positive predictive value  and accuracy  are calculated as follows.
<dig> tpr=tp/tp+fnfpr=fp/tn+fpppv=tp/tp+fpacc=tp+tn/tp+fn+tn+fp where tp, tn, fp and fn are numbers of true positive, true negative, false positive and false negative respectively.
given a true interaction between genes x and y, it is recorded as a true positive item if it is predicted by the algorithm.
otherwise, it is recorded as a false negative item.
similarly, for a non-interaction gene pair x and y, it is recorded as a false positive item when predicted by the algorithm.
otherwise, it is recorded as a true negative item.
RESULTS
efficiency of the threshold determination method
in this study, we developed an automatic threshold determination method for interaction cutoff prediction.
here we show the efficiency of the method using numerical experiments.
first,  <dig> benchmark datasets  were collected from the dream <dig> competition website  <cit> .
then, we ran cmip programs on these datasets with 0-, 1-, 2-, or 3-order manner separately, which means the cmi is calculated given  <dig>   <dig>   <dig>  or  <dig> neighboring genes as conditions.
note that when no neighboring gene is given as condition, the cmi measurement is equivalent to the mi measurement.
in practice, the cmip programs were run with a predefined cutoff, which was increased from  <dig> to  <dig> with a step size of  <dig> .
for each running of programs, accuracy of the cmip algorithm under a certain cutoff was recorded.
after that, accuracies under different cutoffs were checked, and the cutoff at which corresponding accuracy measure reached its maximum was stored as the true threshold.
on the other hand, the cmip programs were run with the automatic threshold determination method  and a predicted threshold was presented.
finally, offset of threshold was detected through comparing the true and predicted threshold values .
<dig> offset=truethreshold−predictedthresholdmaximum−minimum×100% in our work, the stringent, standard, and moderate criteria are defined as offset less than  <dig>   <dig>  and 20% respectively.
results of offset detection are shown in table  <dig>  where each cell represents the number of datasets for which the offsets satisfy a certain criteria under a defined order .
totally, the proportion of datasets that satisfies the stringent, standard, and moderate criteria are  <dig>   <dig>  and 93% respectively.
these results demonstrate that the new threshold determination method developed in this study is effective and an appropriate cutoff can be provided on-the-fly during calculation process of correlation.
though in this study, we already include  <dig> datasets to test the threshold determination method, it is possible that there are networks for which the current threshold determination method might be not the best option; therefore, development of new methods for threshold determination is still needed in the future.table  <dig> effectiveness of threshold determination method under different criteria
parameter selection of the cmip software
the cmip algorithm is based on the cmi measurement, where computational complexity increases exponentially with the increase of the number of neighboring genes as conditions, i.e.
with the increase of orders.
so an appropriate order parameter needs to be selected for the algorithm.
here, we tested the impacts of different order parameters on prediction accuracy.
in practice,  <dig> synthetic benchmark datasets were first collected from the dream <dig> website  <cit> .
then the cmip programs were run on these datasets with 0-, 1-, 2-, and 3-order manners separately and the accuracy measurement was inspected in each running.
inspection results show that the accuracy of the cmip algorithm gradually increased with the increase of the order parameter from  <dig> to 3; however, the increasing trend becomes flat after the order of  <dig> .
considering both the accuracy and computational complexity, we recommend setting  <dig> to be the order parameter for the cmip algorithm.fig.
<dig> changes of the accuracy measurement for the cmip algorithm.
the cmip programs were conducted on  <dig> benchmark datasets with 0-,1-,2- and 3-order manners to test impacts of different order parameters
performance evaluation of the cmip package on benchmark datasets
the cmip package was utilized to reconstruct grns on  <dig> dream <dig> benchmark datasets for performance evaluation.
in practice, programs of the package were run with a 1-order manner, i.e.
the cmi measurement of gene pair was calculated given one neighbor gene as condition.
subsequently, running results of cmip were compared with other popular network inference methods using the auroc and aupr measurements.
specifically, programs of popular grn inference methods were downloaded from website of the dream projects  <cit>  and recommended parameters were used during running these programs.
mean scores of the auroc and aupr measurement on  <dig> benchmark datasets for various methods are shown in table  <dig> and fig.
<dig>  given both the auroc and aupr measurements, the cmip algorithm achieves high performance and delivers competitive values to popular methods.
for average score of the auroc and aupr measurement, the cmip algorithm performs better than all methods except the tigress algorithm.
these results demonstrate that the cmip algorithm is comparable to most currently used network inference methods.table  <dig> scores of various network inference methods on benchmark datasets
fig.
<dig> scores of various network inference methods.
performance of various network inference methods are compared on  <dig> benchmark datasets using the auroc and aupr measurement
application of the cmip package on real biological datasets
we further applied the cmip software on real transcriptome data to check its practical applicability.
the cmip software was first used to build grns of pineapple leaves.
in detail, a grn of leaf base and a grn of leaf tip were constructed based on genome-scale expression data.
totally,  <dig>  genes  and  <dig>  genes  were included in the base and tip grns respectively.
analysis of the node degree distribution suggested that both the tip and the base network showed small-world properties.
then, we extracted genes linked to metabolic enzymes of crassulacean acid metabolism  in the base and tip networks.
after that, genes linked to metabolic enzymes in the tip network but missed in the base network were identified as potential recruited regulators of cam photosynthesis.
subsequently experimental study showed that regulators identified from network comparison do play important roles in photosynthesis differentiation  <cit> .
this application of cmip software on real dataset shows its effectiveness and efficiency for genomic grns reconstruction.
effectiveness of parallel computing framework of cmip programs
since the cmip software is developed to infer grns for genome-scale datasets, parallel computation strategies are adopted in the software to speed up computing process.
here, we compared running time of the cmip programs with popular network inference methods.
in practice, cpu version programs were run on a linux computing server, which has two intel xeon e5- <dig> cpu units containing  <dig> cpu cores in total.
while gpu version programs were run on a linux computing server, which has  <dig> nvidia tesla k20m gpu cards containing  <dig> cuda cores in total.
in this study, all program was executed on a synthetic dataset , which included expression values of  <dig> genes under  <dig> biological treatments for arabidopsis, running times of different programs are shown in table  <dig>  programs of the cmip algorithm are much faster than most popular methods.
running time of the cmip software applied in pineapple grns reconstruction is shown in table  <dig>  these results suggest that parallel computing strategies applied in the cmip software are efficient and the software can handle genome-scale datasets within a reasonable time period.table  <dig> running time of different network inference programs
agpu version program, b cpu version program
cmipa
cmipb
agpu version program, b cpu version program
usage of the cmip package
a service website of the cmip programs is established so that users can utilize them remotely .
now, the website can be accessed at http://www.picb.ac.cn/cmip/.
the web service is created based on a remote resource management system.
once a task is submitted to the system, calculation resources include cpu and gpu computing components will be automatically assigned.
to use the cmip programs, users first need to submit their computing tasks through inputting expression data on the “network inference” module.
then tasks are handled by the management system.
when a task is finished, a notice letter will be sent to users.
alternatively, users can query status of their tasks through the “job result” module.
finally, summary information of the task will be presented on the website.
in addition to the online manners, users can download the source codes of cmip software from the “download” module , and then use it on local computing servers.fig.
<dig> data processing pipeline of website for the cmip algorithm.
first, users can input transcriptomics data and submit their computing tasks through the “network inference” module.
when a task is finished, a notifying letter will be sent to users.
simultaneously, users can check results of their tasks on the “task query” module.
in addition, users can obtain the cmip software in the “download” module
CONCLUSIONS
in this study, we provide a new software package for network inference, which can reconstruct genomic grns within a short time period.
the software package has a number of novel features compared with other grn inference methods.
first, cmip can detect direct gene interactions from indirect ones with a high accuracy based on the cmi measurement.
results of performance evaluation on benchmark datasets show that precision and accuracy of the cmip algorithms are comparable to most currently used methods.
secondly, an automatic threshold determination method is incorporated into the cmip algorithm, so users do not need specify a predefined cutoff for gene interaction judgment and an appropriate threshold can be provided on-the-fly.
numerical experiments confirm the efficiency of the threshold determination method.
last but not least, the openmp and cuda framework are applied in the software to speed up computing process of the cmip algorithms, which enables the software to build grns with less running time.
with this feature, the software is suitable to reconstruct genomic grns.
the area of cmip that needs future development is that it can’t provide directionality to edges of gene regulatory networks, which is a common limitation of many current methods, such as clr  <cit>  and minet  <cit> .
this limitation can be resolved by a two-steps routine.
first, using the cmip software to build a gene regulatory network as background model, then giving directionality to edges of the network according to results of biochemical perturbation experiments, or predicting directionality for edges of the network based on time series expression data  <cit> .
additional file
additional file 1: s1- cmip_code.zip: source codes of the cmip software package.
we thank the anonymous reviewers for their constructive suggestion that help improve quality of the manuscript.
declarations
about this supplement
this article has been published as part of bmc bioinformatics volume  <dig> supplement  <dig>  2016: proceedings of the 27th international conference on genome informatics: bioinformatics.
the full contents of the supplement are available online at http://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement- <dig>
funding
this study was supported by the shanghai municipal natural science foundation , sa-sibs scholarship program, national  <dig> program green super rice , national basic research and development plan of china , cas strategic research project , and the national natural science foundation of china .
publication costs for this study were funded by foundation mentioned above.
availability of data and material
the datasets analyzed during the current study are available in the dream <dig> repository http://dreamchallenges.org/project-list/dream3-2008/.
authors’ contributions
gyz, lnc, xjz, zpl, zw, and xgz conceived and designed the experiments.
gyz and ycx performed the experiments and analyzed the data.
gyz, lnc and xgz wrote the paper.
all authors read and approved the final manuscript.
competing interests
the authors declare that they have no competing interests.
consent for publication
not applicable.
ethics approval and consent to participate
not applicable.

 in the post-genome era, an important task of molecular biology is to reconstruct gene regulatory networks, which represent interactions between genes inside a cell or tissue. a grn provides molecular interactions and regulatory effects of components involved in a biological process, and hence provides insights into the molecular mechanism of the process. in detail, grns can be used to interpret biological processes through studying topological structure information of sub-networks related to these processes, where genes facilitate specific biological functions together. grns can help annotate genes clustered in modules and motifs since genes in the same module or motif have similar functions. grns can be utilized to identify dynamical network biomarkers at the critical states of biological processes if stage-wise data are available, which help biologists understand mechanism of biological process better. therefore, reconstruction of grns can not only support investigating roles of genes and components involved in a biological process, but also help study how a process is developed and maintained. in the last decade, many algorithms have been developed to infer grns based on reverse-engineering methods, such as bayesian network, boolean network, linear and non-linear regression, differential equation, information-theoretic approaches, probabilistic phylogeny network, part mutual information network, and probabilistic graphical models. in we proposed a grn inference algorithm, named pca-cmi, which can distinguish direct interactions of gene pairs from indirect ones based on the conditional mutual information measurement. however, two limitations of the algorithm hinder its wide application. one is that an appropriate threshold should be assigned to the method for direct interactions judgment in advance, which is difficult for users since the threshold is hard to select before grn reconstruction. the other is that the method is time-costly especially for genomic network reconstruction, which is a common restriction of most current grn inferring methods. in this report, we describe a new software package cmip, which implements the pca-cmi algorithm with the goal of enable biologists to build genomic networks easily. the cmip package incorporates a threshold determination method and a parallel computing process for network inference. the threshold determination method can choose an appropriate cutoff on-the-fly for gene interaction judgment. computing procedure of the cmi measurement is optimized to make the algorithm robust, in which parallel computing strategies are applied to accelerate calculation process. this paper describes the algorithm details, program implementation, prediction performance, and practical application of the cmip package. methods workflow of the cmip package as showed in fig.  the cmip package first uses an expression data file as input, in which expression value of genes under different experimental conditions are provided. then, cmip program calculates the correlation value between gene pairs. during calculation process, a threshold determination method is called to generate an appropriate cutoff for direct interaction judgment. when the process is finished, two result files are produced as output. one is a gene interaction file, recording raw correlation value of gene pairs. the other is a gene relation file, providing the relationship between gene pairs. in practice, relationship of a gene pair is assigned as when their correlation value is over the interaction cutoff. otherwise corresponding value is assigned as fig. workflow of the cmip software package. first, expression datasets are used as input of the cmip algorithm. then the cpu or gpu programs are selected to reconstruct networks. finally, result files recording interaction and relationship of gene pairs are generated as output correlation calculation of the cmip algorithm the algorithm implemented in the cmip package is as follows. first, correlation values of each gene pairs are calculated using the mutual information measurement. then a threshold determination method is called to provide an appropriate interaction cutoff for gene pairs. an interaction is marked for gene x and y when their raw correlation value is over the cutoff. after that, for each gene interaction, their correlation values are updated through calculating the conditional mutual information measurement, which describes the dependence of two genes given neighboring genes as condition. a gene z is defined as a neighbor of gene x and y when it has interactions with both gene x and y. in practice, the maximum cmi value between gene x and y is kept. finally, for gene x and y, they are regarded as having direct interaction when their cmi value is over the interaction cutoff and their relationship value is set to be as output. ix,y|z=∑x∈x,y∈y,z∈zpxyzlogpx,y|zpx|zpy|z where i is cmi measurement between gene x and y given gene z as a condition; p are joint probability of gene triple; while p, p, and p are conditional probabilities of gene x, y, and gene pair given gene z as a condition. according to information theory, the cmi measurement can also be defined as follows. hx,y|z=hxz+hyz−hz−hxyz where h is the entropy of gene z; h, h and h are joint entropies of gene pair, and gene triple; h is the conditional entropy of genes x and y given gene z as a condition. based on the gaussian distribution, the entropy of gene z can be estimated as follows. hz=log2πen/2cz1/2=12log2πencz where n is number of experiment, c is the covariance matrix of gene z, and |c| is the determinant of the matrix. while joint entropies can be estimated similarly through corresponding covariance matrixes. based on the entropy estimator, in practice the cmi measurement is calculated as follows. ix,y|z=12logcxz|⋅|cyzcz|⋅|cxyz where c and c are covariance matrixes of gene pair and; c is covariance matrix of gene triple; the |c()| is determinant of a matrix. threshold determination of gene interaction given interaction of gene pairs, the number of interactions decreases dramatically with the increase of the cutoff and their relationship shows an exponential decay. therefore, in practice we chose to use an exponential function to simulate relationship between interaction and cutoff. correlation values of gene pairs are first calculated as mentioned in the “correlation calculation of the cmip algorithm” section. then direct interactions between gene pairs under different cutoffs are estimated and a scatter plot is generated, where x axis is the cutoff value and y axis is the number of direct interactions. after that, we fit the number of direct interactions as a function of the cutoff value with an exponential function. finally, we chose the threshold as the intersection of slope of the start and end sections of the fitting curve, which represents the inflection point of the curve.fig. diagram of threshold determination for gene interactions. relationship between interaction and cutoff is first investigated, and then a fitting curve method based on exponential function is adopted to simulate relationship between them. finally, the intersection of slope of the start and end sections of the fitting curve was chosen as the threshold parallelization of the cmip programs in cmip, parallel strategies were applied to speed up computing process of correlation. in practice, a cpu and a gpu version program of cmip algorithm were developed so that users could utilize them in different computational environment. the cpu version program is implemented based on the openmp framework, where loop calculation is accelerated with the multi-threads technology. in detail, the total computing task of correlation is first calculated based on gene numbers, and then computing tasks is partitioned equally to each cpu node. while the gpu version program is implemented based on the cuda framework, where serial and parallel computing tasks are undertaken by cpu and gpu cores respectively. in detail, a production-consumption strategy is used in the gpu version program, in which gene expression data used by correlation calculation is first processed by the cpu cores; then pre-processed data is delivered to gpu cores for correlation calculation using a parallel mode; finally, the results are transferred from gpu to cpu cores for aggregation. evaluation of network inference methods receiver operating characteristic curve and precision-recall curve are used to evaluate performance of different network inference methods. the roc curve is a graphical plot which illustrates discrimination capacity of algorithm under various thresholds for binary classifier problems, where the x and y axis are false and true positive rate respectively. while the pr curve shows recognition capability of algorithm under various thresholds for positive samples, in which the x and y axis represent the recall and precision measurement respectively. commonly, area under the roc curve and area under the pr curve are calculated to comprehensively evaluate performance of a network inference method. in practice, the true positive rate, false positive rate, and positive predictive value and accuracy are calculated as follows. tpr=tp/tp+fnfpr=fp/tn+fpppv=tp/tp+fpacc=tp+tn/tp+fn+tn+fp where tp, tn, fp and fn are numbers of true positive, true negative, false positive and false negative respectively. given a true interaction between genes x and y, it is recorded as a true positive item if it is predicted by the algorithm. otherwise, it is recorded as a false negative item. similarly, for a non-interaction gene pair x and y, it is recorded as a false positive item when predicted by the algorithm. otherwise, it is recorded as a true negative item.  efficiency of the threshold determination method in this study, we developed an automatic threshold determination method for interaction cutoff prediction. here we show the efficiency of the method using numerical experiments. first, benchmark datasets were collected from the dream competition website. then, we ran cmip programs on these datasets with 0-, 1-, 2-, or 3-order manner separately, which means the cmi is calculated given or neighboring genes as conditions. note that when no neighboring gene is given as condition, the cmi measurement is equivalent to the mi measurement. in practice, the cmip programs were run with a predefined cutoff, which was increased from to with a step size of. for each running of programs, accuracy of the cmip algorithm under a certain cutoff was recorded. after that, accuracies under different cutoffs were checked, and the cutoff at which corresponding accuracy measure reached its maximum was stored as the true threshold. on the other hand, the cmip programs were run with the automatic threshold determination method and a predicted threshold was presented. finally, offset of threshold was detected through comparing the true and predicted threshold values. offset=truethreshold−predictedthresholdmaximum−minimum×100% in our work, the stringent, standard, and moderate criteria are defined as offset less than and 20% respectively. results of offset detection are shown in table  where each cell represents the number of datasets for which the offsets satisfy a certain criteria under a defined order. totally, the proportion of datasets that satisfies the stringent, standard, and moderate criteria are and 93% respectively. these results demonstrate that the new threshold determination method developed in this study is effective and an appropriate cutoff can be provided on-the-fly during calculation process of correlation. though in this study, we already include datasets to test the threshold determination method, it is possible that there are networks for which the current threshold determination method might be not the best option; therefore, development of new methods for threshold determination is still needed in the future.table effectiveness of threshold determination method under different criteria parameter selection of the cmip software the cmip algorithm is based on the cmi measurement, where computational complexity increases exponentially with the increase of the number of neighboring genes as conditions, i.e. with the increase of orders. so an appropriate order parameter needs to be selected for the algorithm. here, we tested the impacts of different order parameters on prediction accuracy. in practice, synthetic benchmark datasets were first collected from the dream website. then the cmip programs were run on these datasets with 0-, 1-, 2-, and 3-order manners separately and the accuracy measurement was inspected in each running. inspection results show that the accuracy of the cmip algorithm gradually increased with the increase of the order parameter from to 3; however, the increasing trend becomes flat after the order of. considering both the accuracy and computational complexity, we recommend setting to be the order parameter for the cmip algorithm.fig. changes of the accuracy measurement for the cmip algorithm. the cmip programs were conducted on benchmark datasets with 0-,1-,2- and 3-order manners to test impacts of different order parameters performance evaluation of the cmip package on benchmark datasets the cmip package was utilized to reconstruct grns on dream benchmark datasets for performance evaluation. in practice, programs of the package were run with a 1-order manner, i.e. the cmi measurement of gene pair was calculated given one neighbor gene as condition. subsequently, running results of cmip were compared with other popular network inference methods using the auroc and aupr measurements. specifically, programs of popular grn inference methods were downloaded from website of the dream projects and recommended parameters were used during running these programs. mean scores of the auroc and aupr measurement on benchmark datasets for various methods are shown in table  and fig.  given both the auroc and aupr measurements, the cmip algorithm achieves high performance and delivers competitive values to popular methods. for average score of the auroc and aupr measurement, the cmip algorithm performs better than all methods except the tigress algorithm. these results demonstrate that the cmip algorithm is comparable to most currently used network inference methods.table scores of various network inference methods on benchmark datasets fig. scores of various network inference methods. performance of various network inference methods are compared on benchmark datasets using the auroc and aupr measurement application of the cmip package on real biological datasets we further applied the cmip software on real transcriptome data to check its practical applicability. the cmip software was first used to build grns of pineapple leaves. in detail, a grn of leaf base and a grn of leaf tip were constructed based on genome-scale expression data. totally, genes and genes were included in the base and tip grns respectively. analysis of the node degree distribution suggested that both the tip and the base network showed small-world properties. then, we extracted genes linked to metabolic enzymes of crassulacean acid metabolism in the base and tip networks. after that, genes linked to metabolic enzymes in the tip network but missed in the base network were identified as potential recruited regulators of cam photosynthesis. subsequently experimental study showed that regulators identified from network comparison do play important roles in photosynthesis differentiation. this application of cmip software on real dataset shows its effectiveness and efficiency for genomic grns reconstruction. effectiveness of parallel computing framework of cmip programs since the cmip software is developed to infer grns for genome-scale datasets, parallel computation strategies are adopted in the software to speed up computing process. here, we compared running time of the cmip programs with popular network inference methods. in practice, cpu version programs were run on a linux computing server, which has two intel xeon e5- cpu units containing cpu cores in total. while gpu version programs were run on a linux computing server, which has nvidia tesla k20m gpu cards containing cuda cores in total. in this study, all program was executed on a synthetic dataset, which included expression values of genes under biological treatments for arabidopsis, running times of different programs are shown in table  programs of the cmip algorithm are much faster than most popular methods. running time of the cmip software applied in pineapple grns reconstruction is shown in table  these results suggest that parallel computing strategies applied in the cmip software are efficient and the software can handle genome-scale datasets within a reasonable time period.table running time of different network inference programs agpu version program, b cpu version program cmipa cmipb agpu version program, b cpu version program usage of the cmip package a service website of the cmip programs is established so that users can utilize them remotely. now, the website can be accessed at http://www.picb.ac.cn/cmip/. the web service is created based on a remote resource management system. once a task is submitted to the system, calculation resources include cpu and gpu computing components will be automatically assigned. to use the cmip programs, users first need to submit their computing tasks through inputting expression data on the “network inference” module. then tasks are handled by the management system. when a task is finished, a notice letter will be sent to users. alternatively, users can query status of their tasks through the “job result” module. finally, summary information of the task will be presented on the website. in addition to the online manners, users can download the source codes of cmip software from the “download” module, and then use it on local computing servers.fig. data processing pipeline of website for the cmip algorithm. first, users can input transcriptomics data and submit their computing tasks through the “network inference” module. when a task is finished, a notifying letter will be sent to users. simultaneously, users can check results of their tasks on the “task query” module. in addition, users can obtain the cmip software in the “download” module  in this study, we provide a new software package for network inference, which can reconstruct genomic grns within a short time period. the software package has a number of novel features compared with other grn inference methods. first, cmip can detect direct gene interactions from indirect ones with a high accuracy based on the cmi measurement. results of performance evaluation on benchmark datasets show that precision and accuracy of the cmip algorithms are comparable to most currently used methods. secondly, an automatic threshold determination method is incorporated into the cmip algorithm, so users do not need specify a predefined cutoff for gene interaction judgment and an appropriate threshold can be provided on-the-fly. numerical experiments confirm the efficiency of the threshold determination method. last but not least, the openmp and cuda framework are applied in the software to speed up computing process of the cmip algorithms, which enables the software to build grns with less running time. with this feature, the software is suitable to reconstruct genomic grns. the area of cmip that needs future development is that it can’t provide directionality to edges of gene regulatory networks, which is a common limitation of many current methods, such as clr and minet. this limitation can be resolved by a two-steps routine. first, using the cmip software to build a gene regulatory network as background model, then giving directionality to edges of the network according to results of biochemical perturbation experiments, or predicting directionality for edges of the network based on time series expression data. additional file additional file 1: s1- cmip_code.zip: source codes of the cmip software package. we thank the anonymous reviewers for their constructive suggestion that help improve quality of the manuscript. declarations about this supplement this article has been published as part of bmc bioinformatics volume supplement 2016: proceedings of the 27th international conference on genome informatics: bioinformatics. the full contents of the supplement are available online at http://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement- funding this study was supported by the shanghai municipal natural science foundation, sa-sibs scholarship program, national program green super rice, national basic research and development plan of china, cas strategic research project, and the national natural science foundation of china. publication costs for this study were funded by foundation mentioned above. availability of data and material the datasets analyzed during the current study are available in the dream repository http://dreamchallenges.org/project-list/dream3-2008/. authors’ contributions gyz, lnc, xjz, zpl, zw, and xgz conceived and designed the experiments. gyz and ycx performed the experiments and analyzed the data. gyz, lnc and xgz wrote the paper. all authors read and approved the final manuscript. competing interests the authors declare that they have no competing interests. consent for publication not applicable. ethics approval and consent to participate not applicable.
here, we present a software package for gene regulatory network reconstruction at a genomic level, in which gene interaction is measured by the conditional mutual information measurement using a parallel computing framework. this new software package provides a powerful tool for genomic network reconstruction to biological community. in cmip, we provide not only an automatic threshold determination method but also an effective parallel computing framework for network inference. reconstruction of gene regulatory networks, in particular, genome-scale networks, is essential for comparative exploration of different species and mechanistic investigation of biological processes. in addition, successful application on a real genomic dataset confirms its practical applicability of the package. currently, most of network inference methods are computationally intensive, which are usually effective for small-scale tasks, but are difficult to construct grns at genome-scale. performance tests on benchmark datasets show that the accuracy of cmip is comparable to most current network inference methods. gene regulatory networkgenome-wideparallel computingsoftwarethe 27th international conference on genome informatics shanghai, china 3- october http://admis.fudan.edu.cn/giw2016/issue-copyright-statement© the author 2016 a gene regulatory network represents interactions of genes inside a cell or tissue, in which vertexes and edges stand for genes and their regulatory interactions respectively.
 a gene regulatory network represents interactions of genes inside a cell or tissue, in which vertexes and edges stand for genes and their regulatory interactions respectively. reconstruction of gene regulatory networks, in particular, genome-scale networks, is essential for comparative exploration of different species and mechanistic investigation of biological processes. currently, most of network inference methods are computationally intensive, which are usually effective for small-scale tasks, but are difficult to construct grns at genome-scale.  here, we present a software package for gene regulatory network reconstruction at a genomic level, in which gene interaction is measured by the conditional mutual information measurement using a parallel computing framework. the package is a greatly improved implementation of our previous pca-cmi algorithm. in cmip, we provide not only an automatic threshold determination method but also an effective parallel computing framework for network inference. performance tests on benchmark datasets show that the accuracy of cmip is comparable to most current network inference methods. moreover, running tests on synthetic datasets demonstrate that cmip can handle large datasets especially genome-wide datasets within an acceptable time period. in addition, successful application on a real genomic dataset confirms its practical applicability of the package.  this new software package provides a powerful tool for genomic network reconstruction to biological community. the software can be accessed at http://www.picb.ac.cn/cmip/. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords gene regulatory networkgenome-wideparallel computingsoftwarethe 27th international conference on genome informatics shanghai, china 3- october http://admis.fudan.edu.cn/giw2016/issue-copyright-statement© the author 2016 