BACKGROUND
in many metabolomic experiments, one of the most important objectives is to discover the set of metabolites that plays a significant role in distinguishing samples from two different groups or populations and thus, in the identification of novel biomarkers
<cit> .
as in any experiment, designing the experiment is critical if reliable statistically significant metabolites are to be obtained.
since metabolomic experiments are expensive, it is crucial to determine the optimal sample size
n^ to attain the desired power to identify discriminating metabolites without wasting resources or unnecessarily sampling many subjects.
however, metabolomic data are typically high dimensional and correlated meaning sample size estimation using classical statistical methods is not straight forward.
currently, in the metabolomics literature, there is no standard method for the determination of sample size when designing a metabolomic experiment.
several methods currently exist in the literature for sample size selection in the high-dimensional data setting
<cit> .
however, none of these methods are suitable for metabolomic experiments since many either assume variables have equal variance or are independent.
more importantly, these methods rely on the existence of some experimental pilot data on which the actual sample size selection is then based, and are not based on the method to be used to analyze the data.
in metabolomic studies, experimental pilot data are rarely available, making such sample size selection approaches redundant.
in this article, we propose a method known as metsizer for sample size estimation for metabolomic experiments that addresses some of these limitations.
metsizer is founded on the idea that the method for selecting sample size firmly depends on the type of data analysis the researcher intends to employ.
in a situation where experimental pilot data are not available, pseudo-metabolomic data are simulated from a statistical model.
the specific statistical model from which the pseudo-metabolomic data are simulated depends on the type of statistical analysis that the metabolomic scientist intends to use.
in its current form the metsizer approach assumes the user intends to employ one of the following three statistical analysis techniques on completion of their experiment:
<dig>  probabilistic principal components analysis
<cit> .
<dig>  probabilistic principal components and covariates analysis
<cit> .
<dig>  dynamic probabilistic principal components analysis
<cit> .
intuitively the metsizer method can be naturally extended to include other analysis approaches, assuming they are based on a statistical model rather than being non-parametric in nature.
metsizer draws on two currently existing methods  for sample size calculation in high-dimensional data settings.
while the approach in
<cit>  is based on the existence of an experimental pilot data set, the approach detailed in
<cit>  simulates pilot data from a statistical model.
further, while independence in the data is assumed in
<cit> , the approach in
<cit>  uses permutation methods to account for the correlation in the experimental pilot data.
metsizer combines these ideas of prior simulation and permutation based techniques to estimate the sample size for metabolomic experiments.
the main advantage of the developed approach is its ability to determine sample size without experimental pilot data and without assuming variable independence.
a graphic user interface  software called metsizer was developed to implement this approach to estimating sample sizes in r
<cit> .
effort was focused on designing the interface of metsizer to encourage its wide use in the metabolomics community regardless of previous knowledge of r. the software is available through the r statistical software environment
http://www.r-project.org.
methods
metabolomic data sets are typically acquired using analytical technologies such as nuclear magnetic resonance spectroscopy
<cit>  and mass spectrometry
<cit> .
the spectrum resulting from nmr spectroscopy is usually divided into spectral bins  and the signal intensities within the bins are related to the relative abundances of metabolites.
ms is typically used for targeted metabolomics in which a specified list of metabolites is measured
<cit> .
the following section describes how the number of samples required for either an nmr or an ms metabolomic experiment can be determined under the metsizer approach.
sample size estimation
let
x¯jg be the estimate of the average signal intensity μjg for metabolite j in samples from the treatment group g which has corresponding sample size ng, where g =  <dig>   <dig>  often in metabolomics, the goal of discovering a set of metabolites that discriminates between samples from two treatment groups is achieved by testing the hypothesis hoj : μj1 - μj2 =  <dig>  on each metabolite j, where j =  <dig> …, p. the aim of discovering discriminating metabolites can be framed as a multiple testing problem as there are p hypotheses to be tested and the probability of falsely declaring a metabolite as significant increases with p. it is therefore important to estimate sample size while controlling an error rate to improve the power of the test for identifying significant metabolites.
metsizer focuses on controlling the false discovery rate .
here, the fdr is the expected number of metabolites incorrectly deemed to be significantly different between the two treatment groups, as a proportion of the total number of metabolites declared to be significant.
the test statistic and its distribution
a test statistic widely used to identify discriminating metabolites is a two sample t-statistic.
the t-statistic ts is evaluated for all metabolites, j =  <dig> …, p, under the assumption that the null hypothesis of no difference μj1 = μj <dig> is true:
tsj=sj+cf,wheresj=1n1+1n22+2n1+n2- <dig>
where sj is the estimate of the pooled standard error for metabolite j. the corresponding within treatment variability estimate is
sjg2=-1∑i=1ngi-x¯jg) <dig> for g =  <dig>   <dig> where xi denotes the signal intensity for metabolite j in sample i from the treatment group g. a correction factor cf is a small positive value added to the standard error of each metabolite to prevent some metabolites with signal intensity near zero from having large test statistic tsj; such a metabolite may have tsj ≈ 0/ <dig>
the typical assumption about the null distribution  of the test statistic tsj is the t-distribution with n1 + n2 -  <dig> degrees of freedom.
however, when the data violate such an assumption, misleading sample size estimates would result.
hence, as in
<cit> , metsizer estimates the null distribution of tsj using a permutation technique.
this is a non-parametric method based on the assumption that under the null hypothesis of no difference, the distribution of the test statistic does not change no matter how the group labels of the pilot data are permuted.
the data generated using this approach maintains the between subject variability and the amount of noise in the data.
the null distribution of the test statistic ts is estimated by randomly permuting the group labels of pilot data and calculating the test statistic for each metabolite, tsjt, where t =  <dig> …, t permutations.
analysis based data simulation
unfortunately, in most cases, experimental pilot data are not readily available in metabolomics since pilot studies are rarely done.
therefore, metsizer uses the intended statistical analysis model to simulate pilot data.
the simulated pilot data are then used to learn about the null distribution of the relevant test statistic for estimating sample size.
this simulation approach is similar to that in
<cit>  in which pilot data are simulated from the marginal model:
p=∫pdp,
where x is the n × p data matrix, u denotes the latent variables, and θ is a vector of unknown model parameters.
simulating from the marginal model is achieved by first generating values of the parameters and the latent variables from the prior distribution p, and then simulating the data from the assumed model p given the simulated values of u and θ.
currently, metsizer assumes the metabolomic practitioner will use one of three different statistical models p to analyse the data from their metabolomic experiment – either the ppca, ppcca or dppca model.
simulation of the parameters of these models is based on the model assumptions and on prior expert knowledge of metabolomic data properties.
as ppca is equivalent to the widely used principal components analysis  method, simulating from the ppca model is discussed here; details of the simulation of pilot data from the closely related ppcca and dppca models are provided in the additional file
<dig>  specifically, ppca is a probabilistic formulation of pca based on a gaussian latent variable model
<cit> .
ppca models the high dimensional spectrum
x̲it= of subject i  as a linear function of the corresponding low dimensional latent variable
u̲it=, where .
the ppca model can be expressed as follows
x̲i=wu̲i+μ̲+ε̲i
where w is a p × q loadings matrix,
μ̲ is a mean vector and
ε̲i is multivariate gaussian noise for observation i, i.e.
p=mvnp where i denotes the identity matrix.
the latent variable
u̲i is also multivariate gaussian distributed,
p=mvnq.
the maximum likelihood estimates of the loadings matrix w and the latent variable u in the ppca model are equivalent to the traditional pca loadings matrix and principal component scores.
for a given sample size n, pilot data x can be simulated from the ppca model as follows:
<dig>  generate parameter values from their prior distributions:
p=mvnqfori= <dig> …n.p=mvnqforj= <dig> …p.p=ig
<dig>  given the generated model parameters and latent variables the pilot data x are then simulated from the ppca model:
p=mvnpfori= <dig> …,n.
estimating sample size based on pilot data simulated in this way ensures the estimated sample size is firmly dependent on the type of model being used to analyse the real experimental metabolomic data.
hence, metsizer represents an analysis based approach to sample size estimation for metabolomic studies.
the specific steps involved in the metsizer algorithm are detailed in the next section.
the metsizer algorithm
the metsizer procedure for sample size estimation starts with a number ntry of different sample sizes and a user-specified fdr .
it then searches for the optimal sample size
n^ by estimating the fdr for each of the ntry sample sizes.
in order to estimate fdr for each sample size, the null distribution of the test statistics of all metabolites is estimated and then a shift constant is added to the test statistics of some po metabolites to allow them to be truly significant.
the null distribution is estimated by calculating the test statistics of the permuted pilot data.
after obtaining the critical values of the null distribution, the fdr is estimated.
the optimal sample size
n^ is then set to be the sample size with fdr equal to target.fdr.
in summary, the metsizer sample size estimation method proceeds as follows:
<dig>  specify the input parameters which include the desired level of fdr , the expected proportion m of significant metabolites and the model to be used when analyzing the observed metabolomic data.
<dig>  simulate pilot data of sample size nk from the assumed analysis model, where k =  <dig> …, ntry.
pilot data simulation from the ppca model is detailed in the previous section; the additional file
<dig> details pilot data simulation from the ppcca and dppca models.
<dig>  estimate the null distribution for all metabolites by randomly permuting the group labels of the simulated pilot data and computing the test statistic tsjt for each metabolite j and each permuted data set t for t permutations.
<dig>  estimate the fdr for each permuted data set t =  <dig> …, t:
consider the corresponding p-vector of the test statistics
ts̲t= for all metabolites on permutation t.
randomly sample po = m × p of the test statistics
ts̲t and add
δϱjt to their intensities.
this allows po metabolites to be truly significant.
here, δ is the effect size, and ϱjt is the true within group standard deviation estimated by
sjt1n1+1n <dig>
a cut off point crit is set to be the
poth largest absolute value of the test statistics
ts̲t.
all metabolites with |tsjt| > crit are declared as significant.
the fdr for permutation t can then be calculated.
<dig>  estimate the fdr for data simulation s by taking the 50th percentile of the fdr values of  <dig> …, t permutations.
<dig>  repeat steps  <dig> to  <dig> for s =  <dig> …, sim simulations and report the 10th, 50th and 90th percentiles of the fdr values for sample size nk.
<dig>  repeat steps  <dig> to  <dig> for k =  <dig> …, ntry different sample sizes and select the optimal sample size
n^ as the nk with fdr = target.fdr.
the total number of permutations t used to estimate the sampling distribution of the test statistics ts was chosen to be twenty.
in the samr r package
<cit>   <dig> permutations were used to estimate the null distribution and they give accurate estimates of the fdr.
here, the value of the effect size δ is chosen based on the variance of the underlying model.
the optimal sample size
n^ is estimated by predicting the sample size at target.fdr using a simple linear regression model on values of fdr above and below the target.fdr with their corresponding sample sizes nk.
the estimated sample size by metsizer ensures that the power or the confidence level in statistical tests reaches .
parameter specification: details and guidelines
the metsizer algorithm requires the specification of several parameters; some are parameters relevant to the intended analysis model, and some are parameters relevant to the sample size estimation procedure itself.
in terms of the metsizer gui which has been developed, the user is requested to specify parameters specific to the sample size estimation procedure i.e.
the number of bins in the nmr or ms spectrum, the expected proportion of significant bins, the target fdr and the minimum sample size they wish to be considered.
the default settings of these parameters are  <dig> spectral bins, 20% significantly different bins, a target fdr of 5% and a minimum sample size of  <dig>  the choice of the number and proportion of significantly different spectral bins will naturally be informed by the metabolomic practitioner’s knowledge, as will the minimum sample size choice.
for the target fdr, again this depends on the conservatism of the metabolomic practitioner and/or the research question of interest, but a fdr of 5% is indicative of typical statistical practice.
the user can easily re-run the metsizer algorithm for different settings of these parameters to ascertain the influence of their particular specifications.
however, within the metsizer gui the user has the option of requesting plots of the expected proportion of significant bins versus the fdr, over different sample sizes, giving insight to the influence of this particularly influential parameter on sample size estimation.
regarding the specification of parameters relevant to the intended analysis model, in the metsizer gui, the user is only required to specify the intended analysis model , and in the case of ppcca, the number of covariates to be included.
both of these decisions are again practitioner informed, depending on the particular experiment under consideration.
the metsizer manual, available through the developed metsizer gui, guides the user through these parameter specification steps using a number of illustrative examples.
the remaining parameters in the metsizer algorithm have been fixed within the r code underlying the metsizer gui, but given the open source nature of r, these can be changed by the user if desired.
in the context of the ppca model discussed above the hyperparameters of the prior distributions of the loadings matrix w and the variance σ <dig> are based on previous estimates of w and σ <dig> from applications of ppca to metabolomic data .
each row of the loadings matrix w is simulated from a standard multivariate gaussian distribution mvn
q and the noise variance σ <dig> is simulated from an inverse gamma distribution with shape parameter α1 =  <dig> and scale parameter α2 =  <dig>  hyperparameter settings for the ppcca and dppca models are detailed in the additional file
<dig>  within the metsizer algorithm four final parameters are specified: the effect size δ , the correction factor cf , the number of permutations t  and the number of simulations sim .
these specifications are based on the choices in
<cit>  in similar sample size estimation settings.
RESULTS
this section illustrates the application of metsizer to different metabolomic experimental settings.
in the first section, metsizer is employed to estimate sample size in the setting where experimental pilot data are not available; the second section considers the case where experimental pilot data are available.
sample size estimation using simulated pilot data
here the metsizer approach to sample size estimation is illustrated in the setting where experimental pilot data are not available and it is assumed that the user has indicated that a ppca model will be used to analyze the observed experimental data.
further, it is assumed that the user has specified that the spectra will consist of  <dig> spectral bins, the target fdr is 5% and the expected proportion of significant spectral bins is 20%.
in this example, the user has also specified that they wish to consider a minimum sample size of ten, with five in each treatment group .
all other metsizer parameters are set at their default values, as detailed in the previous section.
the metsizer method was then applied, and the 10th, 50th and 90th percentiles of the fdr were calculated across a range of sample sizes and are shown in figure
<dig>  the sample size at which the target fdr of 5% was achieved was estimated to be  <dig> with  <dig> in each treatment group as shown in figure
<dig>
n^ is estimated to be  <dig> with  <dig> samples in each treatment group.
show the effect of varying the proportion of significant bins over a range of sample sizes.
the expected proportion of significant spectral bins specified by the user impacts on the estimated number of samples required for the metabolomic experiment.
figures
<dig>
<dig> and
<dig> demonstrate the effect on fdr of varying the expected proportion of significant spectral bins for three different sample sizes.
the figures show that, increasing the expected proportion of significant spectral bins reduces the fdr.
a second example which demonstrates the applicability of metsizer is based on an experimental paradigm where additional information is available in the form of covariates.
in this instance, the ppcca model will be used to analyze the acquired data and thus was used to simulate pilot data with  <dig> spectral bins, five samples from each treatment group and two covariates.
fixing the target fdr at 5% and the expected proportion at 20%, figure
<dig> demonstrates that when two covariates are included in the ppcca model, the total number of samples required for such an experiment increases to  <dig> with  <dig> samples in each treatment group.
figure
<dig> illustrates a third example of the setting where no experimental pilot data are available and the practitioner aims to conduct a longitudinal metabolomic experiment.
the pilot data for this example are simulated from the dppca model; the data are simulated by only focusing on the first time point of the experiment as it is expected that the same number of subjects will be followed over time and that, while there may be dropouts, the largest number of subjects will be present at the first time point.
figure
<dig> shows that the expected number of samples required for a longitudinal study of  <dig> spectral bins with 20% significant bins and a target fdr of 5%, is  <dig> with  <dig> samples from each treatment group.
sample size estimation with experimental pilot data
in a situation where experimental pilot data are available, parameter estimates used for simulations are based on fitting the underlying model to the experimental pilot data.
here, the application of metsizer is illustrated using real metabolomic data sets as experimental pilot data.
the first experimental pilot data set is from a longitudinal metabolomic animal study.
urine samples of  <dig> animals in two treatment groups were collected over a  <dig> day period and the animals’ weights were measured.
details of this study have been previously detailed in
<cit> .
data from day  <dig> of the study were used as experimental pilot data here; the nmr spectra consist of  <dig> spectral bins with nine samples in each treatment group.
the ppcca model was fitted to the experimental pilot data, with weight as a covariate and the maximum likelihood parameter estimates from fitting this model are used for data simulations in metsizer.
controlling the target fdr at 5% and setting the expected proportion of significant bins at 20%, the metsizer method was employed.
figure
<dig> depicts that the sample size estimate is  <dig>  with  <dig> samples in each treatment group.
it is interesting to note that, the 10% and 90% curves in figure
<dig> are much narrower than in the previous examples in which metsizer was used to estimate sample size with no experimental pilot data .
this reduction in uncertainty is due to the fact that metsizer simulations are now based on fixed parameter values not on draws from prior distributions as used when experimental pilot data are not available.
the approach developed here for sample size estimation is not limited to nmr data.
the method has been developed to accept data from targeted metabolomic analysis using ms, thus ensuring its applicability across the metabolomics community.
setting metsizer specifications as in the previous examples, the ppca model was fitted to a targeted metabolomic ms pilot data set and under the metsizr algorithm, the estimated sample size is shown in figure
<dig>
discussion and
CONCLUSIONS
determining sample sizes in metabolomics is important but due to the complexity of these experiments, there are currently no standard methods for sample size estimation in metabolomics.
moreover, since pilot studies are rarely done in metabolomics, sample size estimation approaches for high dimensional data studies requiring experimental pilot data, cannot be applied.
the method presented in this article is a straight forward approach for determining sample sizes for metabolomic experiments whilst controlling the fdr.
the main advantage of the developed approach is its ability to determine sample size even when experimental pilot data are not available.
another key advantage is that it takes the type of analysis the researcher intends to use into consideration when estimating sample size and this can improve the power of the study.
also, since metsizer employs permutation techniques to estimate sample size, it accounts for correlation between metabolites and effect size variability.
the method has been developed to accept both nmr and targeted ms data which will ensure wide applicability in the metabolomics community.
further, a software package facilitates easy implementation of the metsizer approach.
areas of future work are multiple and varied.
metsizer is currently designed to estimate the number of samples required for metabolomic experiments which involve two groups; modifications to the metsizer approach are possible to accommodate different metabolomic experimental designs.
alternatives to the permutation approach employed in metsizer could be examined – bootstrap sampling would provide an interesting alternative.
proof of concept metabolomic experiments are currently underway to validate the metsizer approach.
availability and requirements
the package metsizer has been developed for the r statistical environment  and is freely available at
http://cran.r-project.org.
the package is accompanied by documentation files to facilitate its use.
project name:metsizer
project home page:http://cran.r-project.org/web/packages/metsizer/
operating system: platform independent.
programming language:r platform.
other requirements: no.
license: gpl
any restrictions to use: it is available for free download.
competing interests
the authors declare that they have no competing interests.
authors’ contributions
ng was involved in algorithm development, software development and manuscript writing.
icg was involved in algorithm development, software interpretation and manuscript writing.
yf and wmg were involved in algorithm and software development.
lb was involved in the study hypothesis, software interpretation and manuscript writing.
all authors read and approved the final manuscript.
authors’ information
nyamundanda gift is a phd candidate in the phd in bioinformatics and systems biology programme in university college dublin.
dr. isobel claire gormley is a lecturer in statistics in the school of mathematical sciences, university college dublin.
dr. yue fan is a postdoctoral researcher in the school of biomolecular and biomedical science in university college dublin.
prof. william gallagher is an associate professor in the school of biomolecular and biomedical science in university college dublin.
dr. lorraine brennan is a lecturer in nutritional biochemistry in the school of agriculture and food science, conway institute, university college dublin.
supplementary material
additional file 1
simulating from the ppcca and dppca models.
click here for file
acknowledgements
this research was supported by the irish research council for science engineering and technology  funded phd programme in bioinformatics and systems biology in university college dublin, ireland  and by hrb ireland .

 in many metabolomic experiments, one of the most important objectives is to discover the set of metabolites that plays a significant role in distinguishing samples from two different groups or populations and thus, in the identification of novel biomarkers. as in any experiment, designing the experiment is critical if reliable statistically significant metabolites are to be obtained. since metabolomic experiments are expensive, it is crucial to determine the optimal sample size n^ to attain the desired power to identify discriminating metabolites without wasting resources or unnecessarily sampling many subjects. however, metabolomic data are typically high dimensional and correlated meaning sample size estimation using classical statistical methods is not straight forward. currently, in the metabolomics literature, there is no standard method for the determination of sample size when designing a metabolomic experiment. several methods currently exist in the literature for sample size selection in the high-dimensional data setting. however, none of these methods are suitable for metabolomic experiments since many either assume variables have equal variance or are independent. more importantly, these methods rely on the existence of some experimental pilot data on which the actual sample size selection is then based, and are not based on the method to be used to analyze the data. in metabolomic studies, experimental pilot data are rarely available, making such sample size selection approaches redundant. in this article, we propose a method known as metsizer for sample size estimation for metabolomic experiments that addresses some of these limitations. metsizer is founded on the idea that the method for selecting sample size firmly depends on the type of data analysis the researcher intends to employ. in a situation where experimental pilot data are not available, pseudo-metabolomic data are simulated from a statistical model. the specific statistical model from which the pseudo-metabolomic data are simulated depends on the type of statistical analysis that the metabolomic scientist intends to use. in its current form the metsizer approach assumes the user intends to employ one of the following three statistical analysis techniques on completion of their experiment: probabilistic principal components analysis. probabilistic principal components and covariates analysis. dynamic probabilistic principal components analysis. intuitively the metsizer method can be naturally extended to include other analysis approaches, assuming they are based on a statistical model rather than being non-parametric in nature. metsizer draws on two currently existing methods for sample size calculation in high-dimensional data settings. while the approach in is based on the existence of an experimental pilot data set, the approach detailed in simulates pilot data from a statistical model. further, while independence in the data is assumed in, the approach in uses permutation methods to account for the correlation in the experimental pilot data. metsizer combines these ideas of prior simulation and permutation based techniques to estimate the sample size for metabolomic experiments. the main advantage of the developed approach is its ability to determine sample size without experimental pilot data and without assuming variable independence. a graphic user interface software called metsizer was developed to implement this approach to estimating sample sizes in r. effort was focused on designing the interface of metsizer to encourage its wide use in the metabolomics community regardless of previous knowledge of r. the software is available through the r statistical software environment http://www.r-project.org. methods metabolomic data sets are typically acquired using analytical technologies such as nuclear magnetic resonance spectroscopy and mass spectrometry. the spectrum resulting from nmr spectroscopy is usually divided into spectral bins and the signal intensities within the bins are related to the relative abundances of metabolites. ms is typically used for targeted metabolomics in which a specified list of metabolites is measured. the following section describes how the number of samples required for either an nmr or an ms metabolomic experiment can be determined under the metsizer approach. sample size estimation let x¯jg be the estimate of the average signal intensity μjg for metabolite j in samples from the treatment group g which has corresponding sample size ng, where g =    often in metabolomics, the goal of discovering a set of metabolites that discriminates between samples from two treatment groups is achieved by testing the hypothesis hoj: μj1 - μj2 =  on each metabolite j, where j =  …, p. the aim of discovering discriminating metabolites can be framed as a multiple testing problem as there are p hypotheses to be tested and the probability of falsely declaring a metabolite as significant increases with p. it is therefore important to estimate sample size while controlling an error rate to improve the power of the test for identifying significant metabolites. metsizer focuses on controlling the false discovery rate. here, the fdr is the expected number of metabolites incorrectly deemed to be significantly different between the two treatment groups, as a proportion of the total number of metabolites declared to be significant. the test statistic and its distribution a test statistic widely used to identify discriminating metabolites is a two sample t-statistic. the t-statistic ts is evaluated for all metabolites, j =  …, p, under the assumption that the null hypothesis of no difference μj1 = μj is true: tsj=sj+cf,wheresj=1n1+1n22+2n1+n2- where sj is the estimate of the pooled standard error for metabolite j. the corresponding within treatment variability estimate is sjg2=-1∑i=1ngi-x¯jg) for g =    where xi denotes the signal intensity for metabolite j in sample i from the treatment group g. a correction factor cf is a small positive value added to the standard error of each metabolite to prevent some metabolites with signal intensity near zero from having large test statistic tsj; such a metabolite may have tsj ≈ 0/ the typical assumption about the null distribution of the test statistic tsj is the t-distribution with n1 + n2 -  degrees of freedom. however, when the data violate such an assumption, misleading sample size estimates would result. hence, as in, metsizer estimates the null distribution of tsj using a permutation technique. this is a non-parametric method based on the assumption that under the null hypothesis of no difference, the distribution of the test statistic does not change no matter how the group labels of the pilot data are permuted. the data generated using this approach maintains the between subject variability and the amount of noise in the data. the null distribution of the test statistic ts is estimated by randomly permuting the group labels of pilot data and calculating the test statistic for each metabolite, tsjt, where t =  …, t permutations. analysis based data simulation unfortunately, in most cases, experimental pilot data are not readily available in metabolomics since pilot studies are rarely done. therefore, metsizer uses the intended statistical analysis model to simulate pilot data. the simulated pilot data are then used to learn about the null distribution of the relevant test statistic for estimating sample size. this simulation approach is similar to that in in which pilot data are simulated from the marginal model: p=∫pdp, where x is the n × p data matrix, u denotes the latent variables, and θ is a vector of unknown model parameters. simulating from the marginal model is achieved by first generating values of the parameters and the latent variables from the prior distribution p, and then simulating the data from the assumed model p given the simulated values of u and θ. currently, metsizer assumes the metabolomic practitioner will use one of three different statistical models p to analyse the data from their metabolomic experiment – either the ppca, ppcca or dppca model. simulation of the parameters of these models is based on the model assumptions and on prior expert knowledge of metabolomic data properties. as ppca is equivalent to the widely used principal components analysis method, simulating from the ppca model is discussed here; details of the simulation of pilot data from the closely related ppcca and dppca models are provided in the additional file specifically, ppca is a probabilistic formulation of pca based on a gaussian latent variable model. ppca models the high dimensional spectrum x̲it= of subject i as a linear function of the corresponding low dimensional latent variable u̲it=, where. the ppca model can be expressed as follows x̲i=wu̲i+μ̲+ε̲i where w is a p × q loadings matrix, μ̲ is a mean vector and ε̲i is multivariate gaussian noise for observation i, i.e. p=mvnp where i denotes the identity matrix. the latent variable u̲i is also multivariate gaussian distributed, p=mvnq. the maximum likelihood estimates of the loadings matrix w and the latent variable u in the ppca model are equivalent to the traditional pca loadings matrix and principal component scores. for a given sample size n, pilot data x can be simulated from the ppca model as follows: generate parameter values from their prior distributions: p=mvnqfori= …n.p=mvnqforj= …p.p=ig given the generated model parameters and latent variables the pilot data x are then simulated from the ppca model: p=mvnpfori= …,n. estimating sample size based on pilot data simulated in this way ensures the estimated sample size is firmly dependent on the type of model being used to analyse the real experimental metabolomic data. hence, metsizer represents an analysis based approach to sample size estimation for metabolomic studies. the specific steps involved in the metsizer algorithm are detailed in the next section. the metsizer algorithm the metsizer procedure for sample size estimation starts with a number ntry of different sample sizes and a user-specified fdr. it then searches for the optimal sample size n^ by estimating the fdr for each of the ntry sample sizes. in order to estimate fdr for each sample size, the null distribution of the test statistics of all metabolites is estimated and then a shift constant is added to the test statistics of some po metabolites to allow them to be truly significant. the null distribution is estimated by calculating the test statistics of the permuted pilot data. after obtaining the critical values of the null distribution, the fdr is estimated. the optimal sample size n^ is then set to be the sample size with fdr equal to target.fdr. in summary, the metsizer sample size estimation method proceeds as follows: specify the input parameters which include the desired level of fdr, the expected proportion m of significant metabolites and the model to be used when analyzing the observed metabolomic data. simulate pilot data of sample size nk from the assumed analysis model, where k =  …, ntry. pilot data simulation from the ppca model is detailed in the previous section; the additional file details pilot data simulation from the ppcca and dppca models. estimate the null distribution for all metabolites by randomly permuting the group labels of the simulated pilot data and computing the test statistic tsjt for each metabolite j and each permuted data set t for t permutations. estimate the fdr for each permuted data set t =  …, t: consider the corresponding p-vector of the test statistics ts̲t= for all metabolites on permutation t. randomly sample po = m × p of the test statistics ts̲t and add δϱjt to their intensities. this allows po metabolites to be truly significant. here, δ is the effect size, and ϱjt is the true within group standard deviation estimated by sjt1n1+1n a cut off point crit is set to be the poth largest absolute value of the test statistics ts̲t. all metabolites with |tsjt| > crit are declared as significant. the fdr for permutation t can then be calculated. estimate the fdr for data simulation s by taking the 50th percentile of the fdr values of …, t permutations. repeat steps to for s =  …, sim simulations and report the 10th, 50th and 90th percentiles of the fdr values for sample size nk. repeat steps to for k =  …, ntry different sample sizes and select the optimal sample size n^ as the nk with fdr = target.fdr. the total number of permutations t used to estimate the sampling distribution of the test statistics ts was chosen to be twenty. in the samr r package permutations were used to estimate the null distribution and they give accurate estimates of the fdr. here, the value of the effect size δ is chosen based on the variance of the underlying model. the optimal sample size n^ is estimated by predicting the sample size at target.fdr using a simple linear regression model on values of fdr above and below the target.fdr with their corresponding sample sizes nk. the estimated sample size by metsizer ensures that the power or the confidence level in statistical tests reaches. parameter specification: details and guidelines the metsizer algorithm requires the specification of several parameters; some are parameters relevant to the intended analysis model, and some are parameters relevant to the sample size estimation procedure itself. in terms of the metsizer gui which has been developed, the user is requested to specify parameters specific to the sample size estimation procedure i.e. the number of bins in the nmr or ms spectrum, the expected proportion of significant bins, the target fdr and the minimum sample size they wish to be considered. the default settings of these parameters are spectral bins, 20% significantly different bins, a target fdr of 5% and a minimum sample size of the choice of the number and proportion of significantly different spectral bins will naturally be informed by the metabolomic practitioner’s knowledge, as will the minimum sample size choice. for the target fdr, again this depends on the conservatism of the metabolomic practitioner and/or the research question of interest, but a fdr of 5% is indicative of typical statistical practice. the user can easily re-run the metsizer algorithm for different settings of these parameters to ascertain the influence of their particular specifications. however, within the metsizer gui the user has the option of requesting plots of the expected proportion of significant bins versus the fdr, over different sample sizes, giving insight to the influence of this particularly influential parameter on sample size estimation. regarding the specification of parameters relevant to the intended analysis model, in the metsizer gui, the user is only required to specify the intended analysis model, and in the case of ppcca, the number of covariates to be included. both of these decisions are again practitioner informed, depending on the particular experiment under consideration. the metsizer manual, available through the developed metsizer gui, guides the user through these parameter specification steps using a number of illustrative examples. the remaining parameters in the metsizer algorithm have been fixed within the r code underlying the metsizer gui, but given the open source nature of r, these can be changed by the user if desired. in the context of the ppca model discussed above the hyperparameters of the prior distributions of the loadings matrix w and the variance σ are based on previous estimates of w and σ from applications of ppca to metabolomic data. each row of the loadings matrix w is simulated from a standard multivariate gaussian distribution mvn q and the noise variance σ is simulated from an inverse gamma distribution with shape parameter α1 =  and scale parameter α2 =  hyperparameter settings for the ppcca and dppca models are detailed in the additional file within the metsizer algorithm four final parameters are specified: the effect size δ, the correction factor cf, the number of permutations t and the number of simulations sim. these specifications are based on the choices in in similar sample size estimation settings.  this section illustrates the application of metsizer to different metabolomic experimental settings. in the first section, metsizer is employed to estimate sample size in the setting where experimental pilot data are not available; the second section considers the case where experimental pilot data are available. sample size estimation using simulated pilot data here the metsizer approach to sample size estimation is illustrated in the setting where experimental pilot data are not available and it is assumed that the user has indicated that a ppca model will be used to analyze the observed experimental data. further, it is assumed that the user has specified that the spectra will consist of spectral bins, the target fdr is 5% and the expected proportion of significant spectral bins is 20%. in this example, the user has also specified that they wish to consider a minimum sample size of ten, with five in each treatment group. all other metsizer parameters are set at their default values, as detailed in the previous section. the metsizer method was then applied, and the 10th, 50th and 90th percentiles of the fdr were calculated across a range of sample sizes and are shown in figure the sample size at which the target fdr of 5% was achieved was estimated to be with in each treatment group as shown in figure n^ is estimated to be with samples in each treatment group. show the effect of varying the proportion of significant bins over a range of sample sizes. the expected proportion of significant spectral bins specified by the user impacts on the estimated number of samples required for the metabolomic experiment. figures and demonstrate the effect on fdr of varying the expected proportion of significant spectral bins for three different sample sizes. the figures show that, increasing the expected proportion of significant spectral bins reduces the fdr. a second example which demonstrates the applicability of metsizer is based on an experimental paradigm where additional information is available in the form of covariates. in this instance, the ppcca model will be used to analyze the acquired data and thus was used to simulate pilot data with spectral bins, five samples from each treatment group and two covariates. fixing the target fdr at 5% and the expected proportion at 20%, figure demonstrates that when two covariates are included in the ppcca model, the total number of samples required for such an experiment increases to with samples in each treatment group. figure illustrates a third example of the setting where no experimental pilot data are available and the practitioner aims to conduct a longitudinal metabolomic experiment. the pilot data for this example are simulated from the dppca model; the data are simulated by only focusing on the first time point of the experiment as it is expected that the same number of subjects will be followed over time and that, while there may be dropouts, the largest number of subjects will be present at the first time point. figure shows that the expected number of samples required for a longitudinal study of spectral bins with 20% significant bins and a target fdr of 5%, is with samples from each treatment group. sample size estimation with experimental pilot data in a situation where experimental pilot data are available, parameter estimates used for simulations are based on fitting the underlying model to the experimental pilot data. here, the application of metsizer is illustrated using real metabolomic data sets as experimental pilot data. the first experimental pilot data set is from a longitudinal metabolomic animal study. urine samples of animals in two treatment groups were collected over a day period and the animals’ weights were measured. details of this study have been previously detailed in. data from day of the study were used as experimental pilot data here; the nmr spectra consist of spectral bins with nine samples in each treatment group. the ppcca model was fitted to the experimental pilot data, with weight as a covariate and the maximum likelihood parameter estimates from fitting this model are used for data simulations in metsizer. controlling the target fdr at 5% and setting the expected proportion of significant bins at 20%, the metsizer method was employed. figure depicts that the sample size estimate is with samples in each treatment group. it is interesting to note that, the 10% and 90% curves in figure are much narrower than in the previous examples in which metsizer was used to estimate sample size with no experimental pilot data. this reduction in uncertainty is due to the fact that metsizer simulations are now based on fixed parameter values not on draws from prior distributions as used when experimental pilot data are not available. the approach developed here for sample size estimation is not limited to nmr data. the method has been developed to accept data from targeted metabolomic analysis using ms, thus ensuring its applicability across the metabolomics community. setting metsizer specifications as in the previous examples, the ppca model was fitted to a targeted metabolomic ms pilot data set and under the metsizr algorithm, the estimated sample size is shown in figure discussion and  determining sample sizes in metabolomics is important but due to the complexity of these experiments, there are currently no standard methods for sample size estimation in metabolomics. moreover, since pilot studies are rarely done in metabolomics, sample size estimation approaches for high dimensional data studies requiring experimental pilot data, cannot be applied. the method presented in this article is a straight forward approach for determining sample sizes for metabolomic experiments whilst controlling the fdr. the main advantage of the developed approach is its ability to determine sample size even when experimental pilot data are not available. another key advantage is that it takes the type of analysis the researcher intends to use into consideration when estimating sample size and this can improve the power of the study. also, since metsizer employs permutation techniques to estimate sample size, it accounts for correlation between metabolites and effect size variability. the method has been developed to accept both nmr and targeted ms data which will ensure wide applicability in the metabolomics community. further, a software package facilitates easy implementation of the metsizer approach. areas of future work are multiple and varied. metsizer is currently designed to estimate the number of samples required for metabolomic experiments which involve two groups; modifications to the metsizer approach are possible to accommodate different metabolomic experimental designs. alternatives to the permutation approach employed in metsizer could be examined – bootstrap sampling would provide an interesting alternative. proof of concept metabolomic experiments are currently underway to validate the metsizer approach. availability and requirements the package metsizer has been developed for the r statistical environment and is freely available at http://cran.r-project.org. the package is accompanied by documentation files to facilitate its use. project name:metsizer project home page:http://cran.r-project.org/web/packages/metsizer/ operating system: platform independent. programming language:r platform. other requirements: no. license: gpl any restrictions to use: it is available for free download. competing interests the authors declare that they have no competing interests. authors’ contributions ng was involved in algorithm development, software development and manuscript writing. icg was involved in algorithm development, software interpretation and manuscript writing. yf and wmg were involved in algorithm and software development. lb was involved in the study hypothesis, software interpretation and manuscript writing. all authors read and approved the final manuscript. authors’ information nyamundanda gift is a phd candidate in the phd in bioinformatics and systems biology programme in university college dublin. dr. isobel claire gormley is a lecturer in statistics in the school of mathematical sciences, university college dublin. dr. yue fan is a postdoctoral researcher in the school of biomolecular and biomedical science in university college dublin. prof. william gallagher is an associate professor in the school of biomolecular and biomedical science in university college dublin. dr. lorraine brennan is a lecturer in nutritional biochemistry in the school of agriculture and food science, conway institute, university college dublin. supplementary material additional file 1 simulating from the ppcca and dppca models. click here for file acknowledgements this research was supported by the irish research council for science engineering and technology funded phd programme in bioinformatics and systems biology in university college dublin, ireland and by hrb ireland.
sample size estimates, informed by the intended statistical analysis technique, and the associated uncertainty are provided. the key motivation for metsizer is that it considers the type of analysis the researcher intends to use for data analysis when estimating sample size. metsizer uses information about the data analysis technique and prior expert knowledge of the metabolomic experiment to simulate pilot data from a statistical model. determining sample sizes for metabolomic experiments is important but due to the complexity of these experiments, there are currently no standard methods for sample size estimation in metabolomics. since pilot studies are rarely done in metabolomics, currently existing sample size estimation approaches which rely on pilot data can not be applied. in this article, an analysis based approach called metsizer is developed to estimate sample size for metabolomic experiments even when experimental pilot data are not available. permutation based techniques are then applied to the simulated pilot data to estimate the required sample size.
 determining sample sizes for metabolomic experiments is important but due to the complexity of these experiments, there are currently no standard methods for sample size estimation in metabolomics. since pilot studies are rarely done in metabolomics, currently existing sample size estimation approaches which rely on pilot data can not be applied.  in this article, an analysis based approach called metsizer is developed to estimate sample size for metabolomic experiments even when experimental pilot data are not available. the key motivation for metsizer is that it considers the type of analysis the researcher intends to use for data analysis when estimating sample size. metsizer uses information about the data analysis technique and prior expert knowledge of the metabolomic experiment to simulate pilot data from a statistical model. permutation based techniques are then applied to the simulated pilot data to estimate the required sample size.  the metsizer methodology, and a publicly available software package which implements the approach, are illustrated through real metabolomic applications. sample size estimates, informed by the intended statistical analysis technique, and the associated uncertainty are provided. 