Validity of Models for Predicting BRCA1 and BRCA2 Mutations

Context A computer software program from the University of Texas Southwestern Medical Center (CaGene) has multiple prediction models to estimate BRCA1 and BRCA2 mutation probability, which guides decisions on whether to test for a mutation. A comprehensive quantitative evaluation of how well the prediction models discriminate between persons who carry the mutation and those who do not is lacking. Contribution The authors used 7 prediction models to estimate the probability of a BRCA1/BRCA2 mutation in 3342 families. All 7 models were able to discriminate between mutation-positive and mutation-negative people. Implications The probability that someone carries the BRCA1/BRCA2 mutation should not be considered in isolation when decisions are being made about genetic testing. Other factors should be discussed with the patient and factored into the decision-making process. The Editors Deleterious mutations of BRCA1 (MIM 113705) and BRCA2 (MIM 600185) increase the risk for breast and ovarian cancer (13). Whereas deleterious variants are relatively rare in the general population, they are common among families with multiple occurrences of breast or ovarian cancer (46). When counseling a woman facing decisions about genotyping for BRCA1 and BRCA2, it is important to accurately evaluate the probability that she carries a deleterious mutation (pretest mutation probability) and the probability that a mutation will be found if she is genotyped (which depends on the accuracy of mutation testing). Reliable, evidence-based, individualized counseling strategies can enhance informed decision making, both about whether to pursue BRCA1/BRCA2 testing and what to do with the results (79). The demand for assessment of complex family histories of cancer has led to widespread use of statistical models to estimate mutation probabilities (2, 1018). Model-based predictions are currently used in counseling about genetic testing, are included in materials distributed to women considering genetic testing (1821), are used for determining eligibility for screening and prevention studies (22), and are factored into coverage decisions by insurers (23). More than a dozen models exist. They use different statistical methods and source populations, pedigree features, and predicted outcomes. In clinical practice, different models applied to the same person can give a wide range of probabilities that a BRCA1/BRCA2 mutation is present. This degree of variability raises concerns about whether some models are more accurate than others and calls for a careful independent comparative evaluation of the predictive performance of existing models. We assessed the validity of commonly used models for estimating mutation probabilities of BRCA1 and BRCA2 in individuals identified through the Cancer Genetics Network. We assembled a large set of families with history of breast cancer, ovarian cancer, or both. We used standardized computational methods across contributing institutions to evaluate 7 models. Our main goal was to measure how well these models discriminated between mutation carriers and noncarriers. Methods Study Overview We conducted a cross-sectional, multicenter analysis. For each family in the study, we identified an individual (the counselee) for whom we collected genetic test results for BRCA1, BRCA2, or both; genotyping methods; pretest estimations of mutation probability using each model; and additional information about family history of cancer. We used genetic test results as the gold standard for judging the sensitivity and specificity of the various models. We evaluated all models on every counselee, except where noted. Data Collection Table 1 summarizes the salient data (2432). Sources include 3 population-based studies and 8 data sets of individuals seen in clinics for women at high risk for a BRCA mutation. In the population-based studies, the participants reflected the demographic characteristics of a defined subpopulation (for example, all breast cancer cases in Orange County in the University of California, Irvine [UCI], study [31]). In contrast, patients from high-risk clinics had been referred because of a family history of cancer or were self-referred because of an interest in genetic testing (inclusion criteria varied across clinics). Table 1. Demographic Characteristics of Counselees and Sample Size, by Center Each center calculated all of the model probabilities for its own families. We designated the first genotyped person in each family as the counselee and computed predictions by using the genetic counseling software CaGene (University of Texas Southwestern Medical Center, Dallas, Texas) (24). The software version was customized and distributed to participating sites to ensure uniform procedures across all sites. Data entry and computation of model predictions were performed at the sites. This decentralized approach for data entry and probability calculations allowed site investigators to use pedigree information that models required but that centers could not export to a central site because of privacy concerns. In addition to model predictions, a subset of centers also exported the data required for the models to the National Cancer Institute's (NCI) Cancer Genetics Network Data Coordinating Center. The study population includes 3342 families. The institutional review boards at each participating institution approved the study protocol. All included counselees gave consent for using their data for research according to local institutional review board requirements. The Cancer Genetics Network steering committee reviewed the study design. Genetic Testing Appendix Table 1 summarizes genotyping methods by center and provides a brief description of each method. Determining whether a person carries a deleterious mutation of BRCA1 or BRCA2 is technically demanding because of the large size of these genes, the wide spectrum of mutations, and the presence of mutations whose clinical significance is unknown (3335). Commercial testing uses sequencing to search for unknown mutations or to probe for mutations that are commonly found among Ashkenazi Jewish persons. Research settings, particularly in the time in which the study was conducted, have used less expensive and less sensitive techniques (Appendix Table 1). Although sequencing is the most sensitive of the techniques used in our study, recent evidence highlights how it can miss certain mutations, such as large deletions or intronic mutations (3, 36). Therefore, the set of individuals carrying a mutation (the carriers) is not the same as the set of individuals who test positive for a mutation (the positive cases). Thus, Table 1 underestimates the true number of carriers; the size of the error varies according to the method of genotyping. Appendix Table 1. Number of Counselees, by Genotyping Method for Each Gene and Center Models We studied 7 models: BRCAPRO, the family history assessment tool (FHAT), Finnish, Myriad, NCI, University of Pennsylvania (Penn), and Yale University (Yale). Appendix Table 2 summarizes the characteristics, input variables, and output of the models. Three broad categories of models have been proposed: empirical (Finnish, Myriad, NCI, and Penn), mendelian (BRCAPRO and Yale), and expert-based (FHAT). The first step in developing an empirical model is to summarize the salient aspects of a family history in some predictor variables. The second step is to apply statistical learning techniques, such as logistic regression, to describe the relationship between these variables and the genotyping results (the dependent variable). Mendelian models represent the known modes of inheritance of deleterious genetic variants by established mathematical relationships between phenotypes (in this case, cancer status of family members) and genotypes (14, 3741). The mendelian model inputs include cancer incidence curves (penetrance) for both carriers and noncarriers and the prevalence of deleterious variants. Expert-based models calculate scores that summarize degree of risk, using algorithms constructed on the basis of clinical judgment. For example, FHAT (16) uses a 17-question interview to produce a quantitative score (score range, 0 to 45) representing the severity of family history. Appendix Table 2. Input Variables and Features of Each Model Empirical models calculate the probability of a positive test result for a mutation in the counselee (that is, the result of genetic testing), whereas mendelian models directly estimate the probability of carrying a mutation (the true mutation status of the counselee) (37). The 2 types of predictions are therefore not directly comparable, a fact often overlooked in counseling practice. Because genotyping methods are highly specific for the BRCA1 and BRCA2 genes (that is, they have a very low false-positive rate), multiplying the genotype probability by the genotyping sensitivity gives the probability of finding a mutation. Therefore, to compare an empirical model probability of a BRCA mutation with a mendelian model probability, one must know the sensitivity of the genotyping method of the study used to develop the empirical model. Expert-based scores do not have a direct probabilistic interpretation. In our analyses, we rescaled the FHAT score by dividing by its maximum value of 45. The Penn model (11) estimates the probability of a positive BRCA1 test result in any family member. We adapted it to provide the probability of a positive test in the counselee. We assigned affected counselees the same mutation probability as the family. We assigned unaffected counselees one half the family probability if the closest affected relative of the counselee is a first-degree relative and one quarter of the family probability if the closest relative is a second-degree relative. We used a version of the BRCAPRO (13, 14) model based on the genetic variables described by Iversen and colleagues (42). We defined the Yale model by postulating a si