BACKGROUND
haplotype analysis has become a valuable tool for researchers in population genetics.
in particular, the value attached to the prediction of the constituent haplotypes of a given sample and their frequency of occurrence is such that a variety of methods have been developed for this purpose.
many of these methods, however, depend on knowledge of the phase of the data supplied.
in general, genotypic data from polymorphic loci are ascertained phase-unknown.
various methods for determining the gametic phase exist.
with sufficient data from the genotyping of family members, definitive haplotypes may be inferred.
however, in particular for late-onset disorders, these data may be difficult or even impossible to obtain.
at the laboratory level, techniques such as chromosomal isolation or long-range pcr  <cit>  may be utilised in the prediction of haplotypes, but they suffer the dual drawbacks of being both technologically demanding and in many cases prohibitively expensive in practice.
thus researchers have moved towards computational solutions to this problem.
prominent among the techniques employed for the estimation of the true haplotype frequencies of a phase-unknown sample are those based on the expectation-maximisation  algorithm.
hill  <cit>  originally proposed the use of the em algorithm in genetics, and three years later the term was first coined by dempster et al.
<cit>  and the method put on a more formal footing.
a number of em-based methods for haplotype frequency estimation  have been produced  <cit> .
excoffier and slatkin  <cit>  provide a thorough outline of the implementation of the em algorithm as applied to the problem of hfe.
reliable computational techniques for the estimation of haplotype frequencies have been around for some time, and extensive studies of the accuracy of the em-based methods have been carried out  <cit> , but until recently there has been little investigation of the effect of missing data on these techniques.
this is surprising considering that, even with modern automated dna analysis methods, the problem of missing data is not uncommon, whether due to the failure of amplification or insufficient dna.
zhao et al.
<cit>  have developed the genecounting software specifically to take into account missing data in a sample, but have not produced any validation of the method.
the haplo  <cit>  program is also capable of analysing multiallelic data with missing alleles, using jackknife techniques for error analysis.
the snphap  <cit>  algorithm can handle large numbers of loci and unknown alleles, but is restricted to the analysis of biallelic loci.
in order to carry out an investigation of the effect of missing data on hfe, a program, based on the algorithm outlined in  <cit> , has been developed which can accommodate multiallelic loci and a significant percentage of unknown alleles.
the necessary alterations to the existing implementation of the em algorithm are outlined in the methods section.
following this, biallelic and multiallelic data sets were analysed with varying quantities of unknown alleles randomly substituted.
the analysis is similar to previous work by kirk and cardon  <cit> , which described the effect of genotyping error on hfe.
here we investigate the effect of missing data on the sizes of the confidence intervals  about the haplotype frequency point estimates .
surprisingly, the loss of as much as 30% of the allelic data did not have a significantly detrimental effect on the quality of the results.
the frequencies of haplotypes which predominate in the complete data analysis remain essentially the same after randomly selected data have been relabelled as missing.
the error estimates associated with the predicted frequencies, which are generated via a bootstrap method, are also quite stable, but increase as the proportion of missing data increases.
RESULTS
source of data
two sources of data were used for the principal part of this study.
the first is real single nucleotide polymorphism  data; the second is multiallelic data generated via population generation software.
three additional sets of data containing 10%, 20% and 30% missing alleles respectively were generated from each of the two original sets.
the process of generation is described in the methods section.
hfe was carried out on the eight data sets listed above.
in each case  <dig>  bootstrap iterations were performed for each hfe analysis and the 95% cis about the point results were selected.
for the sake of clarity the results from analyses of the 20% unknown alleles data sets have been omitted from the displayed graphs.
further tests were performed to investigate the effect of sample size upon the quality of the results.
to this end two sets of progressively smaller data sets, with and without missing alleles, were generated from the snp and multiallelic data sets, and hfe was carried out.
the method of selecting these data is outlined in the methods section.
an additional data set, unrelated to those previously described, consisting of data from five snp loci was generated for the purposes of performing tests on data with weak ld between the loci.
a further data set with 10% missing alleles was generated from these additional data.
seven loci biallelic data sets
where hi and  are the haplotype frequencies derived from the phase-known and phase-unknown data respectively, and n is the number of possible haplotypes in the sample.
as these data are from seven biallelic loci, n =  <dig> =  <dig> in this case.
the results are displayed in table  <dig>  also recorded in table  <dig> is the percentage increase in d  as the percentage of unknown alleles in the sample increases.
in each case it is the percentage increase relative to the complete data value that is measured.
three haplotypes absent from the phase-known data set appear in the results of the hfe analysis of the complete data.
their frequencies are  <dig>  × 10- <dig>   <dig>  × 10- <dig>  and  <dig>  × 10- <dig>  of the haplotypes present in the phase-known data, only one haplotype appears with a frequency less than these, the given frequency being  <dig>  × 10- <dig>  figure  <dig> and table  <dig> offer complementary illustrations of the effect of missing data.
table  <dig> provides a good overall picture of how the accuracy of the hfe method deteriorates with inferior data quality.
the effect is most marked in the initial jump from complete to 10% missing data, where a 35% increase in d  is recorded.
the subsequent percentage increases going from 10% to 20% and 20% to 30% unknown alleles are 22% and 16%, respectively, of the value of d  for the complete data.
figure  <dig> allows us to view specifically where this deterioration is most evident, in the mid-range frequency haplotypes.
figures  <dig> and  <dig> display the effect of increasing quantities of missing data on the 95% cis of the haplotype frequencies estimated from the phase-unknown data.
in an attempt to quantify this effect, the spread of the cis for each haplotype  was summed for each data set.
the sum for each data set containing missing data was compared with the sum for the complete data set .
the ratio of the two values  for each comparison is displayed in table  <dig>  here we see a superlinear increase in the rci with increasing proportions of missing data.
despite this, we note from figures  <dig> and  <dig> that, even for the 30% missing data case, the cis for the complete data are not entirely contained within the cis for the data with unknown alleles for many of the haplotypes.
multiallelic data sets
similar computations to those carried out for the snp data sets were carried out for the four multiallelic data sets.
figure  <dig> is a comparison of the frequencies of the most prominent haplotypes in the phase-known data, and their predicted frequencies when the phase is assumed unknown and data are missing.
as with the seven loci snp data sets, the percentage of missing data varies from zero to  <dig>  the haplotypes are labelled as before.
however, as  <dig> distinct haplotypes appear in the phase-known data, only the frequencies for the  <dig> most common are illustrated in figures  <dig> to  <dig> for reasons of clarity.
no haplotype with a frequency greater than  <dig> , as given by the phase-known data, was excluded from the graphs by this trimming.
as with the biallelic data, the discrepancy between the phase-known and phase-unknown frequency predictions, d , was measured.
as the allele counts at each of the seven loci are  <dig>   <dig>   <dig>   <dig>   <dig>   <dig>  and  <dig> respectively, the sum in equation  <dig> is over the n =  <dig> possible haplotypes in the sample.
the results are displayed in table  <dig>  as in table  <dig>  the percentage increase in d  as the percentage of unknown alleles in the sample increases is also recorded.
<dig> distinct haplotypes were estimated to have a frequency of greater than 10- <dig> as a result of the hfe analysis.
<dig> of these do not appear in the phase-known data, with the most common of these having a frequency of  <dig>  × 10- <dig>   <dig> haplotypes in the phase-known data display a frequency greater than this.
as with the snp case, figure  <dig> and table  <dig> together provide a good overall picture of the effect of missing data on the accuracy of the hfe method.
table  <dig> displays similar percentage increases in d  with the 10% and 20% missing data cases to those of table  <dig> , though there the similarity ends, as the jump in d  going from 20% to 30% unknown alleles comes to 40% of the value of d  for the complete data.
in figure  <dig> we see how the phase-unknown frequency predictions match well the observed phase-known frequencies for the more prominent haplotypes, but less well for the less common haplotypes, particularly for the 30% missing data case.
similarly to the snp case, figures  <dig> and  <dig> display the effect of increasing quantities of missing data on the 95% cis of the haplotype frequencies estimated from the phase-unknown data.
as before, measurement of this effect was made by observing the relative increase in the sizes of the cis.
the results are displayed in table  <dig>  in contrast to the snp case, we see a linear increase in the rci with increasing proportions of missing data.
this contrast is further marked by figures  <dig> and  <dig> where we note that the cis for the complete data are, in the case of most haplotypes, entirely contained within the cis for the data with unknown alleles.
sample sizes
investigations were made into the effect of the sample size on the performance of the hfe method when 10% of the data was missing.
three further data sets of sizes  <dig>   <dig> and  <dig> individuals were generated by random selection from the original seven loci snp and multiallelic sets.
from these data, six additional sets with 10% missing alleles were created.
hfe was performed upon these additional data, and the d  results for each were displayed in table  <dig>  in each case the phase-known haplotype frequencies used in the computation of d  were those derived from the respective smaller samples .
as may be expected, in all cases we see an increase in d  as we move from the complete data to the data sets with missing alleles.
d  also is seen to increase as the sample size decreases.
however, what is of note is the pattern involved.
for the seven loci snp case, the percentage increase in d  from complete to missing data itself increases monotonically as the sample size is reduced.
a similar pattern is not observed in the multiallelic data.
performance at low ld levels
fallin and schork  <cit>  illustrate how the performance of the em-based hfe method diminishes with falling ld strength.
here we investigated how the accuracy of our implementation behaves on a data set exhibiting weak ld when 10% of the alleles are missing.
a population of  <dig> individuals with data at five snp loci was generated specifically for this part of the study.
lewontin's d'  <cit>  was found to range between  <dig>  and  <dig>  for all adjacent loci.
table  <dig> displays d  readings for this particular case.
here we see a large percentage increase of 60% in d  as we move from the complete data to 10% missing data.
discussion
the results displayed here show the impact of the addition of increasing quantities of missing alleles on the quality of haplotype frequency estimates.
studying figure  <dig> in tandem with table  <dig>  and figure  <dig> in tandem with table  <dig>  we see a loss of accuracy of the hfe method as the quality of the data degrades.
this is particularly true for the multiallelic data set with 30% missing alleles.
here the loss of accuracy is most apparent with the rarer haplotypes as may be seen in figure  <dig>  whereas for the seven loci snp case, figure  <dig> illustrates that the low frequency haplotypes are dealt with remarkably well, even at high missing data proportions.
for both sets of data the ability of the method to predict the frequencies of the most prominent haplotypes in the samples holds up well as the percentage of unknown alleles increases.
figures  <dig> and  <dig> and figures  <dig> and  <dig> display a similar behaviour in the bootstrap generated cis.
to summarise, there are two significant aspects of the analysis of genotypic data containing incompletely typed individuals evident here.
firstly, that the hfe algorithm, given phase-unknown data with moderate to high levels of ld, predicts the frequencies of the underlying haplotypes with a high degree of accuracy, as is evident from the point estimate graphs, figures  <dig> and  <dig>  tables  <dig> and  <dig> quantify how the quality of the frequency predictions behave with increasing percentages of missing data.
for the multiallelic case where 30% of the alleles are unknown, table  <dig> shows that the discrepancy between the phase-known and phase-unknown predicted frequencies has doubled when compared with the complete data case, though from the study of figure  <dig> the bulk of this discrepancy would appear to originate from the lower frequency haplotypes.
the second aspect is the extent of the 95% cis.
we see a steady increase in the spread of the cis with the addition of missing alleles, reflecting the growing uncertainty in the data.
however, the most prominent haplotypes in both the snp and multiallelic data sets maintain their distinctiveness, even at the 30% unknown alleles level.
these data show that, in particular for the snp data set, the effect of relabelling significant proportions of the data as unknown on the performance of the hfe algorithm is minor.
although study of the illustrated graphs suggests that the impact of missing data is more pronounced with the more complex multiallelic data sets, tables  <dig> and  <dig> demonstrate that the relative increase in the size of the cis is similar across the biallelic and multiallelic data sets, and is almost identical for the 30% missing data sets.
there appears to be a discrepancy between the two measures, namely d  and the rci, used here to quantify the degradation in the quality of the results with increasing percentages of unknown alleles.
tables  <dig> and  <dig> imply that the hfe method works significantly better for biallelic data than for multiallelic data, whereas this phenomenon is much less evident in tables  <dig> and  <dig>  this may be explained by the fact that d  is an absolute measure of the performance of the algorithm, as the phase-known data are available for each data set and thus the exact sample haplotype frequencies are known.
this discrepancy is to be expected; d  is a sum over all possible haplotypes and there exist only  <dig>  possible haplotypes for the seven loci snp data, whereas the multiallelic data, as noted in the results section, have  <dig> possible haplotypes.
also, it is not surprising that haplotype frequencies estimated from the multiallelic data set are found to be less accurate than those estimated from snps, given the more complex nature of the data.
the rci is a relative measure, and illustrates not so much the accuracy of the algorithm, rather the effect of additional missing data.
the results displayed in tables  <dig> and  <dig> show that the algorithm handles the increase in the proportion of unknown alleles equally well for both snps and multiallelic data, although it should be pointed out that the rci measure gives no indication of the accuracy of the point estimates, and should generally be considered in tandem with a measure such as d .
interestingly, the results for the multiallelic data set were achieved despite departure from hardy-weinberg equilibrium  at two of the seven loci .
although this technique relies on the assumption of hwe, niu et al.
<cit>  have demonstrated it to be reliable and robust even when the hwe assumption has been violated.
fallin and schork  <cit>  have shown that hwe violation which results in an excess of heterozygosity leads to an increase in hfe error, though their results are based on a five-locus system, and the observed error increase when two of the five loci were found to be in disequilibrium was minimal.
as we are dealing here with a seven-locus system, the effect on the error was likely to have been even less apparent.
the investigation into the effect of smaller sample sizes has produced some surprising results.
comparing table  <dig> with table  <dig>  we see that the relative increase in d  observed when 10% of the seven loci snp data is relabelled as missing does not change substantially as the size of the sample reduces.
for the full sample of  <dig> individuals, the percentage jump in d  moving from the complete data to 10% missing data is approximately 35%.
for the sample of size  <dig>  this increase is 37%.
likewise for the samples of size  <dig> and  <dig>  the increases are 41% and 47% respectively.
however, for the multiallelic data, we see a contrasting trend.
the percentage jump in d  decreases rather than increases with increasing missing data proportions.
inspection of tables  <dig> and  <dig> shows us that the percentage increase in d  when moving from the complete data to 10% missing data for the full sample of  <dig> individuals is approximately 42%, whereas for the sample of size  <dig> this drops to 32%.
the recorded increase for the sample of size  <dig>  5%, is even more striking.
, and any conclusions drawn from analysis of this case would be highly suspect).
thus no definitive conclusions may be made as to the effect of missing data as the sample size is reduced, other that to say that the matching between the phase-known and phase-unknown frequencies deteriorates with falling sample size, as would be expected.
CONCLUSIONS
here we show that the em method, with the modifications to the implementation for complete data detailed here, can generate accurate estimates of haplotype frequencies even when large amounts of data are missing, in this case up to 30%.
moreover, using this method, the degree of accuracy can easily be estimated using conventional bootstrapping approaches.
this is of considerable importance in the design of experiments, as it is therefore obvious that small levels of drop out in the data for whatever reason do not affect the overall accuracy of the approach perceptibly.
furthermore, considering the strongly deleterious effects of even small amounts of inaccurate data  <cit> , this analysis shows that large amounts of missing data are much less detrimental to the overall quality of the results than incorrectly typed sites.
thus from a practical standpoint it is clearly preferable that if any doubt exists as to a genotype's identity, it should be excluded rather than included using a "best guess".

 haplotype analysis has become a valuable tool for researchers in population genetics. in particular, the value attached to the prediction of the constituent haplotypes of a given sample and their frequency of occurrence is such that a variety of methods have been developed for this purpose. many of these methods, however, depend on knowledge of the phase of the data supplied. in general, genotypic data from polymorphic loci are ascertained phase-unknown. various methods for determining the gametic phase exist. with sufficient data from the genotyping of family members, definitive haplotypes may be inferred. however, in particular for late-onset disorders, these data may be difficult or even impossible to obtain. at the laboratory level, techniques such as chromosomal isolation or long-range pcr may be utilised in the prediction of haplotypes, but they suffer the dual drawbacks of being both technologically demanding and in many cases prohibitively expensive in practice. thus researchers have moved towards computational solutions to this problem. prominent among the techniques employed for the estimation of the true haplotype frequencies of a phase-unknown sample are those based on the expectation-maximisation algorithm. hill originally proposed the use of the em algorithm in genetics, and three years later the term was first coined by dempster et al. and the method put on a more formal footing. a number of em-based methods for haplotype frequency estimation have been produced. excoffier and slatkin provide a thorough outline of the implementation of the em algorithm as applied to the problem of hfe. reliable computational techniques for the estimation of haplotype frequencies have been around for some time, and extensive studies of the accuracy of the em-based methods have been carried out, but until recently there has been little investigation of the effect of missing data on these techniques. this is surprising considering that, even with modern automated dna analysis methods, the problem of missing data is not uncommon, whether due to the failure of amplification or insufficient dna. zhao et al. have developed the genecounting software specifically to take into account missing data in a sample, but have not produced any validation of the method. the haplo program is also capable of analysing multiallelic data with missing alleles, using jackknife techniques for error analysis. the snphap algorithm can handle large numbers of loci and unknown alleles, but is restricted to the analysis of biallelic loci. in order to carry out an investigation of the effect of missing data on hfe, a program, based on the algorithm outlined in, has been developed which can accommodate multiallelic loci and a significant percentage of unknown alleles. the necessary alterations to the existing implementation of the em algorithm are outlined in the methods section. following this, biallelic and multiallelic data sets were analysed with varying quantities of unknown alleles randomly substituted. the analysis is similar to previous work by kirk and cardon, which described the effect of genotyping error on hfe. here we investigate the effect of missing data on the sizes of the confidence intervals about the haplotype frequency point estimates. surprisingly, the loss of as much as 30% of the allelic data did not have a significantly detrimental effect on the quality of the results. the frequencies of haplotypes which predominate in the complete data analysis remain essentially the same after randomly selected data have been relabelled as missing. the error estimates associated with the predicted frequencies, which are generated via a bootstrap method, are also quite stable, but increase as the proportion of missing data increases.  source of data two sources of data were used for the principal part of this study. the first is real single nucleotide polymorphism data; the second is multiallelic data generated via population generation software. three additional sets of data containing 10%, 20% and 30% missing alleles respectively were generated from each of the two original sets. the process of generation is described in the methods section. hfe was carried out on the eight data sets listed above. in each case bootstrap iterations were performed for each hfe analysis and the 95% cis about the point results were selected. for the sake of clarity the results from analyses of the 20% unknown alleles data sets have been omitted from the displayed graphs. further tests were performed to investigate the effect of sample size upon the quality of the results. to this end two sets of progressively smaller data sets, with and without missing alleles, were generated from the snp and multiallelic data sets, and hfe was carried out. the method of selecting these data is outlined in the methods section. an additional data set, unrelated to those previously described, consisting of data from five snp loci was generated for the purposes of performing tests on data with weak ld between the loci. a further data set with 10% missing alleles was generated from these additional data. seven loci biallelic data sets where hi and are the haplotype frequencies derived from the phase-known and phase-unknown data respectively, and n is the number of possible haplotypes in the sample. as these data are from seven biallelic loci, n = = in this case. the results are displayed in table also recorded in table is the percentage increase in d as the percentage of unknown alleles in the sample increases. in each case it is the percentage increase relative to the complete data value that is measured. three haplotypes absent from the phase-known data set appear in the results of the hfe analysis of the complete data. their frequencies are × 10- × 10- and × 10- of the haplotypes present in the phase-known data, only one haplotype appears with a frequency less than these, the given frequency being × 10- figure and table offer complementary illustrations of the effect of missing data. table provides a good overall picture of how the accuracy of the hfe method deteriorates with inferior data quality. the effect is most marked in the initial jump from complete to 10% missing data, where a 35% increase in d is recorded. the subsequent percentage increases going from 10% to 20% and 20% to 30% unknown alleles are 22% and 16%, respectively, of the value of d for the complete data. figure allows us to view specifically where this deterioration is most evident, in the mid-range frequency haplotypes. figures and display the effect of increasing quantities of missing data on the 95% cis of the haplotype frequencies estimated from the phase-unknown data. in an attempt to quantify this effect, the spread of the cis for each haplotype was summed for each data set. the sum for each data set containing missing data was compared with the sum for the complete data set. the ratio of the two values for each comparison is displayed in table here we see a superlinear increase in the rci with increasing proportions of missing data. despite this, we note from figures and that, even for the 30% missing data case, the cis for the complete data are not entirely contained within the cis for the data with unknown alleles for many of the haplotypes. multiallelic data sets similar computations to those carried out for the snp data sets were carried out for the four multiallelic data sets. figure is a comparison of the frequencies of the most prominent haplotypes in the phase-known data, and their predicted frequencies when the phase is assumed unknown and data are missing. as with the seven loci snp data sets, the percentage of missing data varies from zero to the haplotypes are labelled as before. however, as distinct haplotypes appear in the phase-known data, only the frequencies for the most common are illustrated in figures to for reasons of clarity. no haplotype with a frequency greater than, as given by the phase-known data, was excluded from the graphs by this trimming. as with the biallelic data, the discrepancy between the phase-known and phase-unknown frequency predictions, d, was measured. as the allele counts at each of the seven loci are and respectively, the sum in equation is over the n = possible haplotypes in the sample. the results are displayed in table as in table the percentage increase in d as the percentage of unknown alleles in the sample increases is also recorded. distinct haplotypes were estimated to have a frequency of greater than 10- as a result of the hfe analysis. of these do not appear in the phase-known data, with the most common of these having a frequency of × 10- haplotypes in the phase-known data display a frequency greater than this. as with the snp case, figure and table together provide a good overall picture of the effect of missing data on the accuracy of the hfe method. table displays similar percentage increases in d with the 10% and 20% missing data cases to those of table, though there the similarity ends, as the jump in d going from 20% to 30% unknown alleles comes to 40% of the value of d for the complete data. in figure we see how the phase-unknown frequency predictions match well the observed phase-known frequencies for the more prominent haplotypes, but less well for the less common haplotypes, particularly for the 30% missing data case. similarly to the snp case, figures and display the effect of increasing quantities of missing data on the 95% cis of the haplotype frequencies estimated from the phase-unknown data. as before, measurement of this effect was made by observing the relative increase in the sizes of the cis. the results are displayed in table in contrast to the snp case, we see a linear increase in the rci with increasing proportions of missing data. this contrast is further marked by figures and where we note that the cis for the complete data are, in the case of most haplotypes, entirely contained within the cis for the data with unknown alleles. sample sizes investigations were made into the effect of the sample size on the performance of the hfe method when 10% of the data was missing. three further data sets of sizes and individuals were generated by random selection from the original seven loci snp and multiallelic sets. from these data, six additional sets with 10% missing alleles were created. hfe was performed upon these additional data, and the d results for each were displayed in table in each case the phase-known haplotype frequencies used in the computation of d were those derived from the respective smaller samples. as may be expected, in all cases we see an increase in d as we move from the complete data to the data sets with missing alleles. d also is seen to increase as the sample size decreases. however, what is of note is the pattern involved. for the seven loci snp case, the percentage increase in d from complete to missing data itself increases monotonically as the sample size is reduced. a similar pattern is not observed in the multiallelic data. performance at low ld levels fallin and schork illustrate how the performance of the em-based hfe method diminishes with falling ld strength. here we investigated how the accuracy of our implementation behaves on a data set exhibiting weak ld when 10% of the alleles are missing. a population of individuals with data at five snp loci was generated specifically for this part of the study. lewontin's d' was found to range between and for all adjacent loci. table displays d readings for this particular case. here we see a large percentage increase of 60% in d as we move from the complete data to 10% missing data. discussion the results displayed here show the impact of the addition of increasing quantities of missing alleles on the quality of haplotype frequency estimates. studying figure in tandem with table and figure in tandem with table we see a loss of accuracy of the hfe method as the quality of the data degrades. this is particularly true for the multiallelic data set with 30% missing alleles. here the loss of accuracy is most apparent with the rarer haplotypes as may be seen in figure whereas for the seven loci snp case, figure illustrates that the low frequency haplotypes are dealt with remarkably well, even at high missing data proportions. for both sets of data the ability of the method to predict the frequencies of the most prominent haplotypes in the samples holds up well as the percentage of unknown alleles increases. figures and and figures and display a similar behaviour in the bootstrap generated cis. to summarise, there are two significant aspects of the analysis of genotypic data containing incompletely typed individuals evident here. firstly, that the hfe algorithm, given phase-unknown data with moderate to high levels of ld, predicts the frequencies of the underlying haplotypes with a high degree of accuracy, as is evident from the point estimate graphs, figures and tables and quantify how the quality of the frequency predictions behave with increasing percentages of missing data. for the multiallelic case where 30% of the alleles are unknown, table shows that the discrepancy between the phase-known and phase-unknown predicted frequencies has doubled when compared with the complete data case, though from the study of figure the bulk of this discrepancy would appear to originate from the lower frequency haplotypes. the second aspect is the extent of the 95% cis. we see a steady increase in the spread of the cis with the addition of missing alleles, reflecting the growing uncertainty in the data. however, the most prominent haplotypes in both the snp and multiallelic data sets maintain their distinctiveness, even at the 30% unknown alleles level. these data show that, in particular for the snp data set, the effect of relabelling significant proportions of the data as unknown on the performance of the hfe algorithm is minor. although study of the illustrated graphs suggests that the impact of missing data is more pronounced with the more complex multiallelic data sets, tables and demonstrate that the relative increase in the size of the cis is similar across the biallelic and multiallelic data sets, and is almost identical for the 30% missing data sets. there appears to be a discrepancy between the two measures, namely d and the rci, used here to quantify the degradation in the quality of the results with increasing percentages of unknown alleles. tables and imply that the hfe method works significantly better for biallelic data than for multiallelic data, whereas this phenomenon is much less evident in tables and this may be explained by the fact that d is an absolute measure of the performance of the algorithm, as the phase-known data are available for each data set and thus the exact sample haplotype frequencies are known. this discrepancy is to be expected; d is a sum over all possible haplotypes and there exist only possible haplotypes for the seven loci snp data, whereas the multiallelic data, as noted in the results section, have possible haplotypes. also, it is not surprising that haplotype frequencies estimated from the multiallelic data set are found to be less accurate than those estimated from snps, given the more complex nature of the data. the rci is a relative measure, and illustrates not so much the accuracy of the algorithm, rather the effect of additional missing data. the results displayed in tables and show that the algorithm handles the increase in the proportion of unknown alleles equally well for both snps and multiallelic data, although it should be pointed out that the rci measure gives no indication of the accuracy of the point estimates, and should generally be considered in tandem with a measure such as d. interestingly, the results for the multiallelic data set were achieved despite departure from hardy-weinberg equilibrium at two of the seven loci. although this technique relies on the assumption of hwe, niu et al. have demonstrated it to be reliable and robust even when the hwe assumption has been violated. fallin and schork have shown that hwe violation which results in an excess of heterozygosity leads to an increase in hfe error, though their results are based on a five-locus system, and the observed error increase when two of the five loci were found to be in disequilibrium was minimal. as we are dealing here with a seven-locus system, the effect on the error was likely to have been even less apparent. the investigation into the effect of smaller sample sizes has produced some surprising results. comparing table with table we see that the relative increase in d observed when 10% of the seven loci snp data is relabelled as missing does not change substantially as the size of the sample reduces. for the full sample of individuals, the percentage jump in d moving from the complete data to 10% missing data is approximately 35%. for the sample of size this increase is 37%. likewise for the samples of size and the increases are 41% and 47% respectively. however, for the multiallelic data, we see a contrasting trend. the percentage jump in d decreases rather than increases with increasing missing data proportions. inspection of tables and shows us that the percentage increase in d when moving from the complete data to 10% missing data for the full sample of individuals is approximately 42%, whereas for the sample of size this drops to 32%. the recorded increase for the sample of size 5%, is even more striking., and any conclusions drawn from analysis of this case would be highly suspect). thus no definitive conclusions may be made as to the effect of missing data as the sample size is reduced, other that to say that the matching between the phase-known and phase-unknown frequencies deteriorates with falling sample size, as would be expected.  here we show that the em method, with the modifications to the implementation for complete data detailed here, can generate accurate estimates of haplotype frequencies even when large amounts of data are missing, in this case up to 30%. moreover, using this method, the degree of accuracy can easily be estimated using conventional bootstrapping approaches. this is of considerable importance in the design of experiments, as it is therefore obvious that small levels of drop out in the data for whatever reason do not affect the overall accuracy of the approach perceptibly. furthermore, considering the strongly deleterious effects of even small amounts of inaccurate data, this analysis shows that large amounts of missing data are much less detrimental to the overall quality of the results than incorrectly typed sites. thus from a practical standpoint it is clearly preferable that if any doubt exists as to a genotype's identity, it should be excluded rather than included using a "best guess". 
these methods have proved very successful, predicting the phase-known frequencies from data for which the phase is unknown with a high degree of accuracy. remarkably, we find that the absence of up to 30% of the data in both biallelic and multiallelic data sets with moderate to strong levels of linkage disequilibrium can be tolerated. it may be concluded that small levels of drop out in the data do not affect the overall accuracy of haplotype analysis perceptibly, and that, given recent findings on the effect of inaccurate data, ambiguous data points are best treated as unknown. here we present the results of the analyses of various data sets in which randomly selected known alleles have been relabelled as missing. increasingly researchers are turning to the use of haplotype analysis as a tool in population studies, the investigation of linkage disequilibrium, and candidate gene analysis. additionally, the frequencies of haplotypes which predominate in the complete data analysis remain essentially the same after the addition of the random noise caused by missing data.
 increasingly researchers are turning to the use of haplotype analysis as a tool in population studies, the investigation of linkage disequilibrium, and candidate gene analysis. when the phase of the data is unknown, computational methods, in particular those employing the expectation-maximisation algorithm, are frequently used for estimating the phase and frequency of the underlying haplotypes. these methods have proved very successful, predicting the phase-known frequencies from data for which the phase is unknown with a high degree of accuracy. recently there has been much speculation as to the effect of unknown, or missing allelic data – a common phenomenon even with modern automated dna analysis techniques – on the performance of em-based methods. to this end an em-based program, modified to accommodate missing data, has been developed, incorporating non-parametric bootstrapping for the calculation of accurate confidence intervals.  here we present the results of the analyses of various data sets in which randomly selected known alleles have been relabelled as missing. remarkably, we find that the absence of up to 30% of the data in both biallelic and multiallelic data sets with moderate to strong levels of linkage disequilibrium can be tolerated. additionally, the frequencies of haplotypes which predominate in the complete data analysis remain essentially the same after the addition of the random noise caused by missing data.  these findings have important implications for the area of data gathering. it may be concluded that small levels of drop out in the data do not affect the overall accuracy of haplotype analysis perceptibly, and that, given recent findings on the effect of inaccurate data, ambiguous data points are best treated as unknown. 