Purpose: As direct-to-consumer genetic testing becomes more available, a diverse group of consumers, including those with limited health literacy, may consider testing. In light of concerns raised about directto-consumer genetic testing, this study sought to critically examine whether the informational content, literacy demands, and usability of health-related direct-to-consumer websites met existing recommendations. Methods: A content analysis was performed on 29 health-related direct-to-consumer websites. Two coders independently evaluated each website for informational content (e.g., benefits, limitations), literacy demands (e.g., reading level), and usability (e.g., ease of navigation). Results: Most sites presented health conditions and some markers for which they tested, benefits of testing, a description of the testing process, and their privacy policy. Fewer cited scientific literature, explained test limitations, or provided an opportunity to consult a health professional. Key informational content was difficult to locate on most sites. Few sites gave sample disease risk estimates or used common language and explained technical terms consistently. Average reading level was grade 15. Conclusion: The quality of informational content, literacy demands, and usability across health-related direct-to-consumer websites varied widely. Many users would struggle to find and understand the important information. For consumers to better understand the content on these sites and evaluate the meaning of the tests for their health, sites should lower the demands placed on users by distilling and prioritizing the key informational content while simultaneously attending to the reading level and usability elements. In the absence of regulation compelling such changes, government agencies or professional organizations may need to increase consumer and provider awareness of these issues. Genet Med 2010:12(5):304 -312. Key Words : direct-to-consumer, genetic testing, Internet, health literacy, usability S ince the completion of the human genome project in 2003, there has been much discussion of a new era of personalized medicine that will transform both healthcare and the way we interact with our genetic information. [1] [2] [3] Although the promise that genomics will revolutionize modern medicine is likely still some years off in the future, another revolution has taken place outside the doctor's office: the promotion and sale direct-toconsumer (DTC) of health-related genetic tests over the internet. Today, anyone with an internet connection and a credit card can purchase online a wide number of health-related genetic tests from companies who claim it will give them not only insight into their genomes but also information about future disease risk that will enable them to better make important life decisions. The number of companies offering DTC genetic testing has grown rapidly along with coverage of the phenomenon in both the mainstream and scientific media, and recent data have shown that awareness of this type of testing is relatively high both among consumers and providers. 4, 5 DTC companies offer various types of genetic tests, including whole genome sequencing, single nucleotide polymorphism (SNP)-based tests for susceptibility to complex, multifactorial diseases, and tests for known mutations in disease-associated genes. In addition to genetic results, some companies suggest behavioral, dietary, and other lifestyle changes that consumers can make to decrease their risk of disease. Some of the more prominent companies have made claims that DTC testing not only educates the public about genetics and disease risk but also democratizes access to health information, because there is no doctor's visit involved. 6 These companies use empowerment rhetoric that encourages consumers to take control of their health by accessing their genetic information. 7 In response to these companies, a number of professional associations and academics in public health and genetics have vocally criticized the clinical utility and validity of many DTC genetic tests and caution against their use for health-related purposes. 8 Such experts assert that current tests being offered have not been validated in diverse populations and that more research is needed before these tests are used by consumers. 8, 9 Many researchers have also raised concerns about consumers interpreting complex genetic information without the involvement of a healthcare provider who has specialized knowledge of genetics, given the potential for misinterpretation of disease risks. 8,10 -12 One group of consumers who has been overlooked in the current debate about the potential of DTC genetic testing to inform and empower is that of adults with limited genetic literacy. As the cost of sequencing continues to decrease and DTC companies lower the price of their services, DTC genetic testing is becoming increasingly affordable and accessible to broader segments of the population. More than one-third of US adults have limited health literacy, 13 and many more are likely to have limited genetic literacy. 14 Previous research has shown that even those with high educational attainment have difficulty understanding the probabilistic information that is characteristic of genetic information. 14 In addition, although adults may have some familiarity with genetics-related terms, they are unlikely to understand fully the underlying concepts. 14 -17 Therefore, because DTC genetic tests are becoming increasingly accessible, because the Internet is an important health information source for consumers, 13 and because internet use is increasing in every demographic group including among those likely to have limited literacy, 18 it is important to examine whether DTC companies present web-based information that can be understood and used by consumers with a range of health literacy skills. One way to assess the appropriateness of such information is to examine the content, reading level, and usability of DTC websites. Guidelines and recommendations that have been issued by various professional organizations are one standard for assessing the content of DTC websites. In the United States, a number of professional associations (e.g., American Society of Human Genetics, American College of Medical Genetics, American College of Clinical Pharmacology) have developed guidelines that seek to establish standards for the responsible provision of DTC genetic testing services. These bodies recommend that companies offering health-related DTC genetic testing provide information regarding test validity and the current limitations of the science, disclose the risks and benefits of undergoing testing, use Clinical Laboratory Improvement Amendments (CLIA)-certified laboratories, carefully maintain the privacy of consumers' information, disclose the clinical evidence for behavior change recommendations, and involve a health professional in the testing process. 9, 20, 21 In addition, health literacy experts have created guidelines for the design of health-related materials to improve their reading level and increase their usability by adults with varying health literacy skills. These recommendations describe optimal strategies for organization, writing style, and layout of health information. 21 One widely used tool, the Suitability Assessment of Materials, includes an objective rating scale designed to assess the quality of a material's informational content, reading level, writing style, sentence construction, use of vocabulary, graphics, organizational cues (e.g., headers, captions), typography, and layout. 21 Similarly, web usability experts have issued user-centered guidelines for optimizing online health information. 22 For example, they recommend the use of navigational elements (e.g., ensuring each page has a unique title, using visual cueing devices such as boxes and arrows to direct attention) that help users easily move through a website and an organizational structure that prominently places the most important information on a page. 22, 23 Recent research on DTC genetic testing has predominantly focused on its ethical and policy implications and its potential impact on consumers. 24 -28 No study to date has examined the features of DTC genetic testing websites in light of health literacy considerations. The primary objectives of this study were therefore to assess the informational content, literacy demands, and usability of health-related websites offering genetic testing DTC. We critically examined these websites to assess whether the content met existing guidelines for information, health literacy, and web usability. METHODS In this analysis, we examined websites in which consumers could directly purchase and receive genetic testing without the mandatory involvement of a healthcare provider. We focused only on tests related to health conditions or diseases, and therefore we excluded any websites or portions of websites offering testing exclusively for paternity, ancestry, and/or pharmacogenomics. We also excluded websites that used genetic information solely for the purposes of providing advice on nutritional or behavior-related lifestyle choices without identifying a specific health risk. We identified DTC websites for this analysis using two methods. First, we consulted a list of DTC companies prepared by the Genetics and Public Policy Center (version date November 23, 2008). 29 Three of the 32 companies on the list were excluded because they did not test for health conditions (e.g., testing for athletic performance, fetal gender). Second, two of the authors independently conducted comprehensive web searches for additional DTC genetic testing websites using four popular search engines (Google.com, Ask.com, Altavista.com, and MSN Live Search) and one meta-search engine (Metacrawler.com). Adapting keywords used by Gollust et al., 30 the following search terms were used: "genetics OR DNA ϩ test OR testing ϩ home OR kit OR disease," "direct to consumer genetic OR gene ϩ testing company," "ordering a DNA/ genetic test online," "personal genome service," "online genetic testing," "my DNA," "learning about my genes," and "my genome." Valid DTC genetic testing company URLs appearing within the first 30 search results were considered for inclusion in the sample. 22 The comprehensive web search identified one additional health-related DTC genetic testing website. One website was excluded because of its high degree of overlap with another website in the sample, therefore 29 websites were included in the analysis. The codebook for analysis was based on the informational content, health literacy, and usability criteria described above, in addition to previous health literacy-related content analysis research. 31 The complete coding dimensions are presented in the Technical Appendix, Supplemental Digital Content 1, http://links.lww.com/GIM/A103. Coding dimensions were grouped into two domains. Group 1 contained items evaluating the key informational content of the websites (e.g., health conditions, markers, testing process, scientific evidence, benefits, limitations, involvement of healthcare providers, privacy). These coding dimensions were based on the recommendations set forth by the professional associations regarding the key pieces of information that should appear on DTC genetic testing websites. Group 2 contained items evaluating the literacy and usability demands of the websites. For the literacy demand (i.e., level of health literacy required to use the sites) and usability domains (e.g., ease of locating the informational content, text density, navigational cues, layout and format, table structure), coders were asked to base their assessments on the sections of the website that contained the key informational content defined in the Group 1 domains. Coding dimensions for ease of location (e.g., evaluation of whether the content was displayed on the homepage, second-level page, or lower-level page) were adapted from published measures by the research team. These adapted dimensions used an objective predefined formula, based on page level and prominence of content, which the coders then referenced to determine the ease or difficulty of locating key content. Literacy demands were assessed using the Simple Measure of Gobbledygook (SMOG) and Fry readability formulas, as described below. Three trained independent coders refined the codebook in two rounds of pilot testing using DTC websites that had been excluded from the main sample (e.g., ancestry sites).  From March through June 2009, two coders independently coded each of the websites in the study sample and discussed any discrepant scores to achieve consensus. Sites were coded in a predetermined order; the coders completed coding a given site within 3 to 5 days of each other. We conducted the analysis of the close-ended items based on the consensus codes using SPSS 15.0 for Windows (Chicago, IL). Percent agreement was calculated for close-ended codes. The following variables did not meet the preestablished threshold of 70% agreement between coders: description of DNA sample collection (59%) and analysis (59%); ease of locating stated benefits (69%), limitations (62%), and information about the involvement of healthcare providers (66%); description of privacy policy (69%); website purpose (66%); and relevance of content to making a health decision (69%). Because the level of agreement was, in most cases, close to 70% and because of their central importance as key content variables according to the recommendations of the professional associations, we present data on these items below, with appropriate notations regarding their reliability. We assessed the readability of each website using two formulas: the SMOG and the Fry. The SMOG readability formula yields grade-level estimates of reading difficulty based on the number of words with three or more syllables in a sample of 30 sentences. 32 The Fry formula measures sentences per 100 words and syllables per 100 words and plots the approximate reading grade level on a graph. 33 To perform the readability assessment, coders independently sampled three passages of text from three different key content areas on the website based on the same priority ordering: home page, health conditions, markers, testing process, benefits, limitations, involvement of healthcare provider, and privacy. Each coder then calculated both an average SMOG and an average Fry readability score independently for each site, and the scores from each coder were averaged to create a consensus score for each readability formula. Using a scoring tolerance of within two grade levels, coders achieved 76% agreement on the Fry and 83% agreement on the SMOG. RESULTS Given that the sites varied both in their purpose and in the type of testing they offered, it would be expected that the information presented on the sites relating to the benefits and limitations of the tests would vary accordingly. We therefore grouped the sites according to the type of testing they offered, breaking them down into three main types: whole genome sequencing or whole genome scans (7 sites), tests for one or more SNPs (15 sites), and tests for known gene mutations (e.g., cystic fibrosis, hemochromatosis) (7 sites). Four sites offered tests encompassing more than one type; these were grouped according to the type of test they offered most frequently. Across the three main types of tests, the websites offered testing for a wide range of health-related conditions and diseases; in total, 255 health conditions and traits representing 20 different categories (Table 1) . Websites varied in terms of the categories of conditions for which tests were offered; some offered tests from only one category, and another offered tests from 18 of the categories. On average, websites in our sample offered tests from four different categories of conditions. The most frequently offered genetic tests included those for cardiovascular disease risk and those related to metabolism of or response to pharmaceuticals or other substances. Several categories of genetic tests (tests for susceptibility to diseases of the urinary or immune systems, skin diseases, and risks related to infectious disease) were only sold on sites that also offered tests from at least five other condition categories. Otherwise, there were few discernible patterns to the categories of testing offered by the included sites. The key informational content of these health-related DTC genetic testing websites is described in Table 2 . Almost all (97%) of the websites listed at least one of the health conditions for which they offered testing and 79% listed at least one of the tested markers (SNPs). Results for these two items did not vary substantially according to the type of test offered on the website. About one-third of sites (38%) provided scientific evidence to support the tested markers and 21% cited specific scientific publications to support marker selection, although this varied substantially by type of website. Twenty-one percent of sites explained how they calculated the risk estimates given to consumers in their test results; this was most commonly done by the whole genome sites (43%). For the testing process, almost all (93%) of the sites gave at least a brief description of the process by which the consumer's DNA sample is collected and three-quarters (76%) at least briefly described how the sample is analyzed. About one-half (45%) of sites stated that they used a CLIA-certified laboratory to analyze the DNA samples. Only 14% of sites provided an opportunity for consumers to consult with a healthcare provider before testing, whereas 28% provided that opportunity after testing; this was most common among sites offering tests for known mutations. Less than one-quarter (21%) of sites gave information for healthcare providers about how they could interpret and use a patient's DTC genetic test results. In terms of privacy, 79% of sites gave some information about their privacy policy. Two-thirds (65%) provided information on the people behind the company, although only 24% clearly identified any scientific collaborators and described their access to consumers' DNA samples. Almost all of the sites (90%) listed at least one benefit to consumers of undergoing testing, although this was less common among sites offering testing for known mutations (57%). Of those sites that stated a benefit, the one most frequently stated (76%) was that the results of a test can inform a health decision (data not shown). Other commonly stated benefits were that there would be a benefit to the consumer in the future (35%); being tested could be beneficial to one's family or children (24%); information itself is beneficial (24%); and there would be a benefit to scientific research (21%). About one-half (55%) of the sites overall presented at least one limitation related to undergoing testing, although all (100%) of the whole genome sites listed limitations. Of those sites that stated a limitation, the limitations most frequently stated were that other factors are important for disease risk (45%); the science surrounding testing is still uncertain (17%); the tests cannot diagnose diseases (14%); and testing was provided for only a limited number of mutations (14%; data not shown). About one-third (34%) of sites offered some mechanism to allow consumers to receive updated information over time. About one-half of the sites (45%) stated that the science underlying the tests is new and/or changing, although this was often not described as a limitation of the test.  The results for the health literacy and usability criteria are shown in Tables 3 and 4 . We assessed how easy it was to locate information about each of the informational content topics on the websites (see Table 3 ). Overall, information about health conditions was easy to locate on 62% of the websites, although this varied greatly by type of site. In contrast, information about tested markers was easy to locate on only 17% of sites. Information regarding benefits was easy to locate on 41% of sites, although this was notably lower among sites offering testing for known mutations (14%). Information regarding limitations was easy to locate on only 14% of sites. Privacy information was easy to locate for just less than one-half of the sites (48%), although this was higher among sites offering testing for known mutations (71%).
Lachance et al. REF examined the informational content, literacy demands, and usability of DTCGT service websites.
Informational content, literacy demands, and usability of websites offering health-related genetic tests directly to consumers
Genetics in Medicine
Genetics in Medicine
Medicine