The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information ANDREW J. FLANAGIN MIRIAM J. METZGER University of California, Santa Barbara, USA Data from 574 participants were used to assess perceptions of message, site, and sponsor credibility across four genres of websites; to explore the extent and effects of verifying web-based information; and to measure the relative influence of sponsor familiarity and site attributes on perceived credibility.The results show that perceptions of credibility differed, such that news organization websites were rated highest and personal websites lowest, in terms of message, sponsor, and overall site credibility, with e-commerce and special interest sites rated between these, for the most part.The results also indicated that credibility assessments appear to be primarily due to website attributes (e.g. design features, depth of content, site complexity) rather than to familiarity with website sponsors. Finally, there was a negative relationship between self-reported and observed information verification behavior and a positive relationship between self-reported verification and internet/web experience. The findings are used to inform the theoretical development of perceived web credibility. Compared to more traditional sources, the credibility of web-based information may be less reliable due to the structural and editorial features of the web environment.Web-based information suffers from a relative lack of professional gatekeepers to monitor content, faces a convergence of information genres, such as the blending of advertising and information, lacks established reputations for many information sites, and is particularly prone to alteration, which may be difficult to detect (Alexander and Tate, 1999; Flanagin and Metzger, 2000; Johnson and Kaye, 1998) . Such factors have placed the burden of information assessment squarely on the shoulders of the media consumer and have prompted a renewed scholarly interest in the credibility of sources, their messages, and the media that carry them (e.g. Flanagin and Metzger, 2000; Kaye, 1998, 2000; Kim et al., 2001; Kiousis, 2001; Metzger et al., 2003; Schweiger, 2000; Sundar, 1998 Sundar, , 1999 Sundar and Nass, 2001) . To date, research has examined several issues with regard to credibility, including site design features (Fogg et al., 2001a (Fogg et al., , 2001b Johnson and Kaye, 1998; Palmer et al., 2000; Shon et al., 2000) , cross-media comparisons (Flanagin and Metzger, 2000; Kiousis, 2001; Sundar, 1999) , source attributions Nass, 2000, 2001), and the role of users' reliance on web-based information Kaye, 2000, 2002). However, thus far news and political websites have been emphasized over other site genres and information types. Nonetheless, the diversity of information on the web suggests that researchers should begin to examine a greater variety of websites and information types, particularly because credibility perceptions have been shown to vary depending upon users' motivations and orientations toward specific media and media content (Greenberg and Roloff, 1974; Mulder, 1980; Reagan and Zenalty, 1979) . In addition, further exploration of site attributes is warranted both theoretically and empirically (Burgoon et al., 2000; Eastin, 2001; Eysenbach and Kohler, 2002; Metzger et al., 2003; Schweiger, 2000) and the central role of user attributes and behaviors (e.g. verification of web information) is imperative to consider. Overall, although past research provides insight into the core elements of web credibility, there remain important areas where research is warranted and timely. Websites can be conceptualized as information repositories that represent organizational or individual sources, while also reflecting the characteristics of those sources through design features of the sites themselves. Past research suggests two primary dimensions of source credibility, trustworthiness and expertise, as well as several secondary dimensions such as dynamism, composure, and sociability (Berlo et al., 1969; Gass and Seiter, 1999; Hovland et al., 1953; McCroskey, 1966; Perloff, 1993; Whitehead, 1968) . Scholars have recognized recently that assessments of the credibility of web-based information can capitalize on past research findings on the credibility of information sources in other contexts by considering how these findings translate to the web environment . For example, website expertise may be reflected in site informativeness, the display of the appropriate credentials, or the site sponsor's reputation; trustworthiness may be communicated through explicit policy statements or a lack of commercial content; and attractiveness or dynamism may be communicated through various dimensions of the website's appearance (e.g. layout, graphics, font, color). Thus, in many respects, websites may be considered to be analogous to individuals or organizations as information sources whose characteristics engender greater or lesser credibility. As with most research in this area, credibility is defined here as a perceptual variable rather than as an objective measure of the quality of some information or source of information. In other words, credibility is not a property of the information or source, but is a property that is judged by the receiver of the information (Fogg et al., 2001b; Freeman and Spyridakis, 2004; Gunther, 1992; Sundar, 1998) . Importantly, however, credibility judgements may be influenced by objective properties of the information or its source. Perceptions of credibility may differ depending upon the type of source being evaluated and the context in which the evaluation occurs (Cronkhite and Liska, 1976; Delia, 1976; Gass and Seiter, 1999; Gunther, 1988 Gunther, , 1992 Stamm and Dube, 1994) . Flanagin and Metzger (2000) found that both news and reference information obtained on the web were rated as more credible than entertainment or commercial information. Accordingly, different genres of website may contribute to variation in individuals' credibility perceptions, due to perceived communicator biases which can affect pre-message expectancies (Eagly et al., 1981) .This study considers four distinct genres of sites in order to assess perceptions of credibility. Specifically, information originating from a news organization is likely to be considered to be relatively credible, in view of the common interpretation that such organizations typically apply some editorial rigor and fact-checking procedures to the information that they provide. Indeed, Flanagin and Metzger (2000) found that news and reference information were assessed as more credible than other types of online information, as mentioned earlier. Information contributed by the website of a special interest group might be considered to be somewhat biased and therefore less credible, in view of the political and persuasive motivations that tend to underlie such groups. Similarly, information found on an electronic commerce (e-commerce) site is likely to represent the at Universiteit van Amsterdam SAGE on April 25, 2007 http://nms.sagepub.com Downloaded from particular commercial interests of the site's sponsor and thus be assessed as less credible than other site genres. Past research indeed shows commercial information to be perceived as having very low credibility (Flanagin and Metzger, 2000) , consistent with studies demonstrating that people tend to discount information from sources with obvious persuasive intent (see O'Keefe, 2002) . Finally, web-based information contributed by an individual on personal websites might be considered to be specific, narrow, and less representative of others' views, resulting in extremely low credibility assessments. It is crucial in assessing credibility in the web environment to recognize that the very concept of a source is complex, because the source of an online message may be attributed to the author of the material on a particular website, aspects of the message, the sponsor of the site, or even the medium itself (Eastin, 2001; Kiousis, 2001; Metzger et al., 2003; Sundar, 1998; Nass, 2000, 2001). Source attribution research recognizes that the source of web-based information is what or who the receiver believes it to be, and that these source attributions are important in evaluating online information (Newhagen and Nass, 1989; Nass, 2000, 2001). For example, Sundar and Nass (2001) found that different source attribution 'levels' (e.g. medium vs. site author) affect receivers' reactions to online news stories. This suggests that it is necessary to differentiate between various online sources and source 'levels' or types, because information receivers find them to be distinct, and because credibility assessments may vary depending upon which source attributes are salient at the time of evaluation. For this reason, perceptions of three types of credibility were measured in the present study: message, site, and sponsor credibility. Message credibility depends on aspects of the message itself, for example, information quality, accuracy, currency, and language intensity have been shown to have an impact on perceptions of the competence and/or trustworthiness of messages in the online environment . Site credibility may vary by site features that engender greater or lesser credibility, such as the visuals or amount of information used on the site and the degree of interactivity offered to site visitors. Perceptions of sponsor credibility rely on evaluations of the website's sponsor, which may rest on reputation or personal experience with the sponsor. Measuring these three types of credibility provides a relatively inclusive assessment of a user's frame of reference, more accurately captures the relevant notions of credibility in the web environment, and allows for a more precise test of online credibility perceptions than has been available in past research.Taking this into account and based on the rationale presented earlier, the following hypothesis is proposed, which recognizes the complexity of credibility on the web: H1 a-c : Individuals will perceive (a) message; (b) site; and (c) sponsor credibility to vary by website genre, such that news organization sites will be perceived as most credible, followed by special interest sites, then electronic commerce sites, and finally personal sites. Another important issue is that site genre and site attributes may be linked and therefore difficult for users to distinguish. For example, certain genres of websites may include, typically, site attributes that other site genres usually do not. News organization sites, for example, may have a more sophisticated site layout, feature interactive capabilities that most personal websites do not, or have greater depth of content, and these attributes could potentially contribute to perceived credibility. Although some studies have begun to assess the importance of site design features on perceived credibility (e.g. Alexander and Tate, 1999; Culnan and Armstrong, 1999; Fogg et al., 2001a Fogg et al., , 2001b Palmer et al., 2000; Schenkman and Jönsson, 2000; Shon et al., 2000) , no studies to date have examined the relative influence of site sponsors and site attributes explicitly. As a first step toward this end, Research Question 1 is proposed: RQ1:What is the relative importance of aspects of the source (i.e. familiarity with website sponsors) and website attributes in people's credibility assessments? USER ATTRIBUTES, BEHAVIORS, AND PERCEIVED CREDIBILITY Scholars have long noted that perceptions of credibility can be highly situational and may depend on the receiver's relationship to the medium, the source of the message, and the message itself (Chaffee, 1982; Cronkhite and Liska, 1976; Gunther, 1992) . Rosenthal (1971) originally drew attention to the importance of audience factors in credibility assessment by noting that verifiability of message content is crucial to consider.According to him, ambiguous information and information that could not be validated would be perceived as less credible. These factors warrant attention when assessing the credibility of web-based information as well. Indeed, Flanagin and Metzger (2000) found that participants reported that they verified the information that they obtained via the web only rarely to occasionally, but that verification behaviors were related positively to perceived information credibility. Recent qualitative studies also find that having external links to further information is an important criterion for assessing the credibility of information on a given website (Eysenbach and Kohler, 2002; Freeman and Spyridakis, 2004) . One shortcoming of past research, however, is that it has relied on self-reported verification behavior, which may be prone to social desirability biases. Verification behaviors might be evaluated more appropriately in realistic environments wherein users are allowed to behave in ways they normally do when online.To address these issues, the current study examines 'observed' verification behavior in a naturalistic environment through Hypothesis 2: H2a-c: Observed verification behaviors (when resulting in confirmatory information) will increase perceptions of: (a) message; (b) site; and (c) sponsor credibility. As previously mentioned, researchers have called the reliability of self-report data into question, particularly when high social desirability may be present e.g. Babbie, 1986). Moreover, in the web environment the potential discrepancy between self-report and observed verification behaviors may be especially large in view of the range of verification strategies available to users. Indeed, the most common self-reported web information verification behaviors are those that require the least user effort, suggesting that more labor-intensive practices in fact might be undertaken quite rarely and perhaps overestimated when self-reported (Flanagin and Metzger, 2000) .Thus, the following research question is posed to assess the relation between self-report and observed information verification on the web: RQ2:What is the relation between self-reported and observed information verification behaviors? Scholars have recently investigated reliance on the web and experience with it as potential influences on credibility perceptions. On balance, studies have found a positive association between web use and the credibility ratings of online news and political information (Johnson and Kaye, 2000; Kiousis, 2001 ; Pew Research Center for the People and the Press, 1999). Moreover, internet experience has been shown to be related positively to assessments of the credibility of other kinds of web-based information and to verification behaviors as well (Flanagin and Metzger, 2000; Ha, 2001 ; but see Freeman and Spyridakis, 2004) . Flanagin and Metzger (2000) suggest that more experienced or savvy web users view the medium as more credible but, recognizing its limitations, also tend to verify information obtained on the web more stringently. Indeed, users with greater experience with a medium tend to apply a higher level of scrutiny to the information they obtain from that medium (see Culbertson and Somerick, 1977; Sundar, 1998) and this scrutiny may take the form of verifying information obtained from the web. For example, identifying the author or organization who posted the information to the web, evaluating the author's goals and qualifications, or checking to see when the information was last updated are commonly recommended strategies for evaluating the quality of web-based information (Alexander and Tate, 1999; Brandt, 1996; Gilster, 1997; Harris, 1996; Jones, 1998) . Based on these findings on credibility and verification, Hypothesis 3 states that: H3a-e: Internet/web experience will increase perceptions of: (a) message; (b) site; and (c) sponsor credibility; and (d) self-reported verification of website content; and (e) observed information verification behaviors. This study used a 4ϫ2 experimental design that varied the genre of website (news organization, e-commerce, special interest, or personal sites) with verity of the site (a fictional or real site) to test the hypotheses and research questions. The dependent variables were the three types of credibility perceptions. Sample From a comprehensive list of approximately 94,000 registered voters within the county in the USA where the study originated, 3500 potential participants were selected randomly (68 solicitations were undeliverable, resulting in a valid sample of 3432).These eligible participants were mailed a written solicitation to participate in the study and were offered an incentive in the form of a free one-year magazine subscription of their choice to do so. Among these, 274 individuals participated in the research (Nϭ274).The functional response rate for the study was approximately 18 percent. 1 Additional participants (Nϭ300) were solicited from undergraduate communication courses and were given course credit for their participation in the study. In this manner, a total of 574 individuals participated in the study (Nϭ574). 2 Of the participants 39 percent (Nϭ226) were male and 61 percent (Nϭ348) were female.The range of participant ages was 18 to 83 years, with a mean age of 31.99 years (SDϭ16.20). In addition, sample members had a mean of 15.09 years of education (SDϭ2.44), a mean annual income of between $50,000 and $59,999 (if claimed as dependents, this reflected their parents' income), and reported relatively frequent use of the internet/web, as indicated by their mean response of 5.76 (SDϭ1.52) to how often they use the internet/web on a seven-point scale (where 1ϭ'I never use the internet/web' to 7ϭ'I use the internet/web all the time'). The websites used in this study all contained an identical news story on the topic of the potentially harmful effects of radiation on pregnant women who fly in airplanes.This particular story, originally obtained from a reputable, national news source, was selected due to its plausibility of appearing on each of the sites used in the study.The story was stripped of all references to experts and expert sources (e.g.'according to Dr. Smith …' or 'the Institute for Radiation Study reports that …') and was embedded in each site as a prominent, live 'link'. Website genres included a news organization with no direct interest in the issue (CNN; www.cnn.com), a relevant e-commerce site (BabyCenter.com; www.babycenter.com), a special interest group related to the issues in the story (Children's Defense Fund; www.childrensdefense.org), and a personal webpage. 3 In order to test RQ1, parallel sites were constructed which mirrored the 'real' sites exactly, except that the name of the sponsoring entity was changed to a fictitious one throughout the site ('Onlinenews' for CNN,'Child Rights Fund' for Children's Defense Fund, and 'BabyPlace' for BabyCenter).This was done for all but the personal sites, for which both male and female versions were created. 4 In this manner, this experiment took the form of a 4 (site genre)ϫ2 (site verity) factorial design, resulting in eight different websites used as stimuli in the study. Manipulating site verity allowed us to hold site attributes constant while varying the participants' familiarity with the websites.This provided a test of RQ1 in that any observed differences in credibility ratings between parallel sites could be due only to differences in familiarity with the site sponsors, rather than differences in site attributes. As noted, the dependent variables consisted of the three types of credibility perceptions already discussed. The participants were directed to the study's homepage via a written URL included in the solicitation letter (in the student sample, the URL was distributed in class) and were told that they could access the site from anywhere they chose and at any time within the following two weeks. After agreeing to informed consent online, the participants were instructed that they would be viewing a specific genre of website. Depending on the genre of site to which they were directed, the participants were instructed that they would see: 1 'a website belonging to a media organization, that is, an organization that uses the internet to deliver news information to those who visit the site'; 2 'a website belonging to an "e-commerce" organization, that is, an organization that uses the internet to sell products and to deliver information to those who visit the site'; 3 'a website belonging to a special interest organization, that is, an organization with a specific social or political interest that uses the internet to deliver its message to those who visit the site'; or 4 'a website belonging to a private individual, that is, a person who has chosen to post his or her webpage on the internet'. The participants were then redirected automatically to one of the study's eight websites (each participant was exposed to only one website and sites rotated sequentially, such that approximately equal numbers of participants were ultimately directed to each site).When viewing the site to which they had been directed, participants could not see any URLs or other information that would lead them to believe that they were not at a 'true' website, rather than at a site created by the researchers. Participants were instructed to browse the website as much as they liked, in order to explore the information on its pages. In addition, they were instructed to read the story on the potentially harmful effects of radiation on pregnant women, for which they were given the title and approximate location on the main page of the site to which they were directed. After the participants had browsed the webpage and had a chance to read the story indicated, they were instructed to click on a link (contained in a bar at the bottom of all pages) to indicate that they had finished browsing.They were then asked to make sure that they had read the story as instructed and, if not, were given the opportunity to return to the webpage to browse further and read the story. When they had finished browsing, they were directed to a questionnaire which they completed online. Once directed to the questionnaire, the participants could no longer view the webpage. All participants were debriefed online at the end of their session. As described earlier, three separate types of credibility perceptions were measured: sponsor, message, and site credibility.The participants were given explicit instructions to focus on the sponsor, the message, and then the site, as appropriate, in answering the post-exposure questionnaire items. Website sponsor credibility was assessed by the extent to which the sponsor was perceived to be credible, have high integrity, have a positive reputation, be successful, be trustworthy, offer products or services of superior quality, be prestigious, have a sincere interest in important affairs; also the extent to which an individual would be willing to work for the sponsor.These items were measured on a seven-point scale (higher values corresponded to higher sponsor credibility) and averaged to derive the measure of sponsor credibility (Mϭ4.41, SDϭ0.99). Cronbach's alpha for this measure was 0.87. Following past research, message credibility was assessed with five items measuring participants' perceptions of the believability, accuracy, trustworthiness, bias, and completeness of the information provided in the radiation story (see Austin and Dong, 1994; Flanagin and Metzger, 2000; Gaziano, 1988; Rimmer and Weaver, 1987;West, 1994). Bias scores were reverse-coded so that higher scores on all dimensions indicated greater perceptions of credibility and all items were measured on a seven-point scale. The mean value of the five items constituted the final message credibility measure (Mϭ4.10, SDϭ1.17; Cronbach's alphaϭ0.85). A battery of 22 items adapted from standard source credibility scales (Berlo et al., 1970; Leathers, 1992; McCroskey, 1966; McCroskey and Jenson, 1975) was used to assess the credibility of the website as a whole.The participants assessed the extent to which they found the website as a whole to be trustworthy, believable, reliable, authoritative, honest, safe, accurate, valuable, informative, professional, attractive, pleasant, colorful, likable, aggressive, involving, bold, interactive, interesting, sophisticated, biased, and organized. These items collectively made up the site credibility measure, on a seven-point scale (Mϭ4.52, SDϭ0.89). Cronbach's alpha for the website credibility scale was 0.91. To assess the degree to which participants perceived these types of credibility to be distinct from one another, all items proposed to assess all types of credibility were subjected to an exploratory principal axis factor analysis, using promax (oblique) rotation. Factors with eigenvalues greater than 1 were retained.The results indicated that only one item designed to measure message, site, or sponsor credibility cross-loaded with other items measuring message, site, or sponsor credibility, confirming that the participants did indeed distinguish between these three types of credibility in meaningful ways. 5 Two measures of verification of website information were used: self-reported verification and observed verification. Survey items from Flanagin and Metzger (2000) were used to measure self-report verification: the participants were asked to consider their behavior with websites in general, aside from their behavior with the website that they were asked to browse, and to indicate the degree that they perform a number of verification behaviors.These items were measured on a seven-point scale (where 1ϭ'never', 2ϭ'very rarely', 3ϭ'rarely', 4ϭ'occasionally', 5ϭ'often', 6ϭ'very often' and 7ϭ'all the time'); Cronbach's alpha was 0.88 (Mϭ3.85, SDϭ1.09). Observed verification was attained by providing links to existing external websites that were related in content to, and which confirmed the information found in, the story that the participants were asked to read (e.g. links indicated sites on 'traveling during pregnancy' and 'flight radiation levels').These links were listed under the heading 'For more information', which was located at the bottom of the same webpage as the story that they read.The participants had the opportunity to click on seven relevant links and the total number of links that they viewed constituted the measure of observed verification behavior (Mϭ0.40, SDϭ1.17). Website genre varied by the specific kind of website to which the participants were exposed: news organization, e-commerce, special interest group, and personal sites. Internet experience was assessed with three items. On a seven-point scale, the participants were asked to assess how often they use the internet (where 1ϭ'I never use the internet/WWW' to 7ϭ'I very often use the internet/WWW'), their experience using the internet (where 1ϭ'no experience' and 7ϭ'a great deal of experience'), and their level of expertise (where 1ϭ'I am not at all expert' to 7ϭ'I am completely expert'). Scores were averaged to derive the experience measure (Mϭ5.23, SDϭ1.30), for which Cronbach's reliability was 0.88. Because past research has demonstrated that salience (Eastin, 2001; Gass and Seiter, 1999; Gunther, 1992; O'Keefe, 2002) and several demographic factors Gomdaweb, 1998; Kaye, 1998, 2002) influence the dependent variables proposed in this research, their effects were statistically controlled in tests of the hypotheses and research questions in this study. Issue salience was measured by asking the participants to rate, on a seven-point scale (where higher numbers indicated greater levels of these qualities), how relevant the story was to their own life, how interesting they found the story to be, how much they enjoyed the story, and how important  RESULTS H1 proposed that credibility perceptions would vary by site genre.The hypothesis was tested by a MANCOVA analysis, with demographics (age, education, income, and sex), internet experience, issue salience, self-report verification, and observed verification as the covariates, and the perceived credibility of the target message (radiation story), the website as a whole, and the website sponsor as the dependent measures. Because site genre constituted the independent variable, both actual and fictitious sites within each genre were considered together. Internet experience, demographic variables, issue salience, and verification behavior were controlled to eliminate these variables as alternative explanations for credibility perceptions. The analyses indicated a significant multivariate effect for site genre (Wilk's lambdaϭ0. 77, F(9,557)ϭ17.19, pՅ0.001, 2 ϭ0.08). More specifically, considering site credibility (F(3562)ϭ43.70, pՅ0.001, 2 ϭ0.19), both the news organization site (Mϭ4.94, SDϭ0.70) and the e-commerce site (Mϭ4.76, SDϭ0.87) were perceived to be significantly more credible than the special interest group site (Mϭ4.54, SDϭ0.77) and the personal website (Mϭ3.95, SDϭ0.84).The news organization site and e-commerce site did not differ from one another in perceived site credibility, although the special interest site was viewed as significantly more credible than the personal site. All significant differences were at the pՅ0.001 level. Differences across site genre were also found for sponsor credibility (F(3,562)ϭ20.83, pՅ0.001, 2 ϭ0.10).The sponsor of the news organization site (Mϭ4.82, SDϭ0.97) was rated significantly more credible than the sponsors of all other genres of sites (pՅ0.01).The e-commerce site (Mϭ4.44, SDϭ0.90) did not differ from the special interest group site (Mϭ4.54, SDϭ0.99) in terms of sponsor credibility, although both of these site sponsors were perceived to be significantly more credible than the personal website sponsor (Mϭ3.92, SDϭ0.89; pՅ0.001). Message credibility (F(3,562)ϭ16.54, pՅ0.001, 2 ϭ0.08) also varied across site genres and the same pattern of differences emerged as for the findings for sponsor credibility. Specifically, the message on the news organization site (Mϭ4.67, SDϭ1.11) was rated significantly more credible than messages on all other genres of sites (pՅ0.001). Perceived credibility of the message on the e-commerce site (Mϭ4.07, SDϭ1.08) did not differ from the perceived credibility of the message on the special interest group site (Mϭ4.15, SDϭ1.11), although the messages on both of these sites were perceived as significantly more credible than the same message residing on the personal website (Mϭ3. 65, SDϭ1.16; pՅ0.01). Overall these findings largely, although not completely, support H1. Specifically, the predicted order of credibility ratings was confirmed for message and sponsor credibility (albeit the special interest and e-commerce sites did not differ significantly); however, for site credibility, the e-commerce site was rated to be as credible as the news site, in contrast to expectations. RQ1 addressed whether credibility assessments are due more to website attributes or familiarity with website sponsors (personal websites were not included in this analysis, since familiarity with these sponsors was not relevant).To test this, credibility assessments were compared across 'actual' websites and their counterparts which, although identical in design and content, represented fictitious and presumably less familiar sponsors. First, a manipulation check was conducted to confirm that familiarity indeed varied across actual and fictitious sites. Familiarity with the site was measured by the question: 'Before looking at the website today, how familiar were you with the organization whose website you saw?', with responses ranging from 1ϭ'I had never heard of that organization before' to 7ϭ'I was quite familiar with that organization already'.The e-commerce sites failed the manipulation check, with a mean familiarity rating of 2.22 for the BabyCenter site (SDϭ1.71) and a rating of 2.05 for the BabyPlace site (SDϭ1.53; pϭ0.53). However, differences in familiarity did exist for the news organization sites (CNN, Mϭ5.41, SDϭ1.59; Onlinenews, Mϭ2.98, SDϭ1.88; pՅ0.001) and the special interest group sites (Children's Defense Fund, Mϭ2.24, SDϭ1.67; Child Rights Fund, Mϭ1.64, SDϭ1.21; pՅ0.01). Consequently, RQ1 was examined using only these two sites. MANCOVA analyses (controlling for the same factors as in H1) revealed a multivariate effect for credibility across the news organization sites (Wilk's lambdaϭ0. 85, F(3109)ϭ6.21, pՅ0.001, 2 ϭ0.15), although only sponsor credibility (F(1111)ϭ18.61, pՅ0.001, 2 ϭ0.14) showed significant differences between the actual/familiar site (Mϭ5.14, SDϭ1.02) and the fictitious/less familiar site (Mϭ4.43, SDϭ0.74; message credibility, pϭ0.62; site credibility, pϭ0.36). For the special interest group sites, there was no multivariate effect for credibility perceptions when comparing the actual and fictitious special interest group sites (F(3140)ϭ1. 44, pϭ0.23; actual, Mϭ4.38; fictitious, Mϭ4.68; message credibility: actual, Mϭ4.09; fictitious, Mϭ4.20; site credibility: actual, Mϭ4.47; fictitious, Mϭ4.60). In sum, sponsor credibility was impacted by familiarity with the website for the news site only. The remaining hypotheses and research question were tested by Pearson product-moment correlations. H2 proposed positive relationships between credibility assessments and observed verification of site content.The partial correlations between observed verification behavior after controlling for age, The analyses demonstrated a negative and significant zero-order correlation between self-reported verification and observed information verification behavior (-0.11, pՅ0.01, r 2 ϭ0.01), in response to RQ2, which sought to explore this relationship.Thus, the individuals who reported that they heavily verified the information they found on the web actually did so significantly less than others in this study. H3 a-e proposed positive relationships between online experience and both credibility ratings and verification behaviors.The hypothesis was tested by partial correlations, controlling for age, gender, education, income and information salience.There were no significant relationships between internet/web experience and message credibility (rϭ-0.06, pϭ0.17), site credibility (rϭ0.00, pϭ0.96) or sponsor credibility (rϭ-0.01, pϭ0.91).The partial correlation between self-reported verification and internet/web experience was 0.33 (pՅ0.001, r 2 ϭ0.11) and the partial correlation between observed verification behavior and internet/web experience was 0.03 (pϭ0.54). Thus, only H3 d was supported, indicating that as online experience increases, so does self-reported verification of web-based information. Again, zero-order correlations are quite similar to the partial correlations, showing little influence of the control variables. 6 DISCUSSION This study extends research on the perceived credibility of web-based information by exploring the role of site features, user attributes, and information verification behaviors across different kinds of websites, as assessed by users functioning in a relatively naturalistic environment.The results indicate important differences in perceived credibility across different genres of websites, resulting from a combination of site attributes and differences in the genre of sites themselves. Furthermore, the self-reported verification behaviors that users invoked in their typical web usage were found to vary by experience and to impact credibility assessments, although notable differences in self-report versus observed verification behaviors were also found. The results of H1 show that the genre of website under consideration impacts the perceived credibility of the sponsor, the message residing on the website, and the website overall. For both sponsor credibility and message credibility, news organization sites were perceived as more credible than all other genres, and e-commerce and special interest sites did not vary in perceived credibility, although both were viewed as more credible than personal sites. News organization and e-commerce sites did not differ in perceived credibility, but were viewed as more credible than both special interest and personal websites. Special interest sites were also perceived as more credible than personal ones.These results largely support H1. Thus, one finding from this study is that the genre of website is important when assessing perceived credibility. Inasmuch as different information types correspond to different site genres (e.g. news organization sites tend to carry predominantly news information), these results support prior research which has found perceived credibility to vary by information type (Flanagin and Metzger, 2000) . Indeed, one explanation why the e-commerce site was not rated as low as predicted in perceived credibility is that the site used in this study (BabyCenter.com) not only offered products for sale, but also provided news and information about pregnancy and infant care. Consequently, it may have actually blurred information types across site genres. Furthermore, message and sponsor credibility did not differ for e-commerce and special interest sites as predicted, possibly because the participants saw both sponsors as similar in their persuasive intent, which may have colored their evaluations in similar ways. Overall, however, the results suggest that familiarity with the site genre as a source of a particular kind of information (which perhaps triggers particular pre-message expectancies) is an important component of credibility perceptions. Over time it is likely that users recognize that information type can signal the relative persuasive intent and the corresponding level of trust or skepticism they might bring to bear on source, message, or site credibility. That said, it is necessary to recognize that different website genres also tend to correspond to differences in website attributes and that these differences in attributes may be equally important in credibility perceptions. For example, in this study the news organization and e-commerce sites did not differ on site credibility as expected, potentially because these sites as a whole were relatively more sophisticated than the special interest or personal sites used in this study in terms of their site attributes. Indeed, as past research has shown, site design can be an important element of perceived credibility (Fogg et al., 2001a (Fogg et al., , 2001b Johnson and Kaye, 1998; Palmer et al., 2000; Shon et al., 2000) . To partially disentangle the influences of site genre versus site attributes on perceived credibility, a comparison of the credibility of actual versus fictitious sites, which were identical in their attributes (e.g. design, layout, content, and complexity), was proposed in RQ1. Only sponsor credibility varied across actual and fictitious sites and only for the news organization, suggesting that for well-known and familiar organizations a 'consensus heuristic' might be in operation, whereby a person may be influenced by their perceptions of others' credibility judgements (O'Keefe, 2002; i.e. CNN is widely perceived to be a reliable information source). In this manner, credibility perceptions may be affected by familiarity with a reliable source of information. However, the finding that neither message nor site credibility differed across actual and fictitious sites suggests that perhaps (sophisticated) site attributes are able to overcome reputation when the site is unfamiliar (as with fictitious sites), because there is no other information upon which to assess credibility apart from the site's attributes, such as design and content. Overall, these findings imply (at least within a certain range of site design parameters) that sponsor credibility may transfer from familiar entities to their websites, that site credibility can be achieved equally by familiar and unfamiliar sources, and that messages residing on sites may be viewed similarly in terms of their perceived credibility, regardless of the source that issues them. Together, the findings from H1 and RQ1 suggest that both site genre and site attributes matter in assessing credibility, but that under conditions where web sponsors are unfamiliar to the user, design elements can potentially boost perceptions of site credibility to levels equal to those for familiar sponsors. Although not an ideal test of the relative influence of genre versus design elements, these results do illuminate two important issues: different aspects of websites (e.g. genre of site versus design elements) may be better or worse predictors of the credibility of different kinds of sites (i.e. sites of familiar versus unfamiliar sponsors), and measuring multiple kinds of credibility perceptions is critical in research on web credibility. The results for H2 demonstrate that actually verifying information had no impact on credibility perceptions. In addition, those who said that they verified information more, actually verified information in this study less (results of RQ2).Thus the data from this study show a lack of correspondence between self-reported and observed verification behaviors, illustrating that researchers should strive to observe online behavior directly since self-reports may be unreliable. Considered together, the findings on verification paint a picture of a set of internet users who are skeptical of web-based information, know they should verify the information they get online, and yet fail to do so. Future research should target this audience to determine how many internet users fall into this category and to uncover the reasons for the inconsistencies between their attitudes and behavior. Understanding online information verification behavior has important implications for those working to improve internet literacy and is an important element of empowering internet users (Flanagin and Metzger, 2000; Metzger et al., 2003) . The results for H3 a-c showed that experience using the web did not impact the participants' credibility evaluations, as predicted by earlier research that demonstrated a positive relationship between internet use or reliance and credibility assessments (Flanagin and Metzger, 2000; Johnson and Kaye, 2000) . It is interesting that both more recent research, including Johnson and Kaye (2002) , and this study have failed to support this relationship.These recent results may reflect a 'floor effect' in internet usage, such that there is less variance in these measures now because people have more experience and rely on the web more than they did in 1996 or 1998 when the data from the earlier studies were collected. An alternative explanation is that earlier research measured different kinds of credibility (e.g. credibility of the medium) or different kinds of websites than those included in the present study.This again points to the importance of measuring multiple types of credibility perceptions in the online environment and examining multiple genres of websites. Analyses of H3 d-e showed a positive relationship between internet experience and self-reported, but not observed, information verification.A potential explanation is that those with greater internet experience may indeed verify web information more in general, but did not do so in this particular study. However, this behavioral inconsistency seems implausible, especially given the relative ease of information verification in the study.An alternative interpretation is that heavier internet users felt more social desirability when answering questions about their online behavior.As before, this implies a group of internet users who know they ought to be skeptical of the information they find online and thus should verify it, but who do not make the effort to do so. A limitation of the present study is that several constraints prevented a full factorial design in which site sponsor, site genre, and site attributes such as design and content and message were varied systematically to examine their effects on web users' credibility perceptions. Instead, site genre (with affiliated sponsors) and site attributes were varied while holding the message constant. Consequently, this study did not test message credibility per se. Future research will need to manipulate message and other elements in order to better understand their effects on credibility judgements. Importantly, past research on message credibility may be helpful in this regard. Prior studies have found such elements as message currency, comprehensiveness, writing style, use of opinionated language, and presence of typographical errors to impact credibility assessments (see Metzger et al., 2003) . Despite its limitations, the study has several strengths. For example, a major criticism of experimental research is that it is conducted in artificial settings and thus produces data that are not generalizable to the real world. Using the internet's unique communication capabilities, the participants in this study were able to take part in a controlled experiment while in a natural and comfortable environment. In addition, although more research is now being conducted online, few studies use scientific techniques for subject recruitment.As described in the Method section, the non-student sample used in the present research was selected randomly from the local population. In these ways, this study capitalizes on the strengths of both experimental and survey research methodologies. Another strength of this study was the inclusion of several types of credibility and several genres of websites in the research design.Assessing various types of credibility (i.e. credibility of the sponsor, the site, and the message) proved to be important because the findings showed that the participants in fact did make these distinctions in their credibility perceptions. Breaking down web credibility into several types and then cueing participants to focus on particular levels while they make credibility assessments is crucial in obtaining valid results (see Metzger et al., 2003; Sundar and Nass, 2001) . Researchers who only ask about the credibility of a particular 'website' cannot know if receiver assessments were made on the basis of the reputation of the site's sponsor, the physical appearance of the site and its messages, design features, or some other aspect of the site. In addition, most past research examining online credibility has examined only e-commerce or news and political websites.The inclusion of a wider array of site genres in this study provided a more realistic and comprehensive view of credibility in the online environment. The fact that web credibility research is relatively young, coupled with the complexity of the web environment, makes theorizing about web credibility challenging at this point in time. Nonetheless, although the research results to date are still highly exploratory, they point to several potentially important predictors of credibility. Specifically the attributes of: (1) websites and their sponsors; (2) messages residing on those sites; and (3) audiences for web-based information all show promise in predicting perceptions of web credibility. First, research demonstrates the importance of website characteristics and site sponsors.The data from this study, for example, show that familiarity with the site genre, potentially as a source of a particular kind of information, is an important component of credibility perceptions: news sites were consistently rated higher in credibility in this study, whereas individual sites were not thought to be very credible. E-commerce and special interest sites, for the most part, were rated in the middle. Moreover, our data show that site attributes are a crucial aspect of credibility perceptions, since fictitious sites were able to achieve credibility ratings that were largely equal to those of major organizations, presumably based on their sophisticated site attributes, including design and content.These findings suggest that users attend to site and sponsor cues such as genre and site attributes in forming credibility assessments, although the relative influences of these factors still need to be sorted out. In addition, Sundar and Nass (2001) have demonstrated experimentally that different levels of source attribution (visible sources such as gatekeepers, technological sources such as communication channels, and receiver sources such as individuals or audiences) affect receivers' affective reactions to online news stories, as well as their perceptions of story quality and representativeness, although not of credibility directly. In their study, when online news stories were perceived to be selected by other web users, they were liked more and thought to be more representative and of higher quality. Taken together, these findings suggest a potential interaction between site genre, site attributes, and source attribution which might impact perceived credibility. Inasmuch as websites from genres thought to be credible, composed of design features that enhance credibility, can exhibit content that is perceived to be recommended by others, credibility perceptions might increase. For example, well-established websites offering peer reviews of products and services might enjoy high perceived credibility, particularly if such sites begin to blend news or expert information as many e-commerce sites currently do. Collectively, these factors might constitute a hybrid of what Tseng and Fogg (1999) labeled 'presumed credibility', based on general assumptions and stereotypes of the source of information (i.e. the consensus heuristic) and surface credibility, derived from simple inspection, such as a website that appears credible by virtue of its design. Second, although not hypothesized in this study, Table 1 shows that attributes of the message, in particular message salience, are also key to understanding the credibility of web-based information.The degree to which the issues contained in the story were salient to participants positively impacted site, sponsor, and message credibility ratings, consistent with findings on issue involvement from earlier credibility work in both online and offline environments (see Eastin, 2001; Gunther, 1992) . Similarly, others have proposed that individuals may process low-involvement online messages more heuristically than systematically, resulting in less rigorous examination of a message's attributes in arriving at credibility assessments (Freeman and Spyridakis, 2004; Sunder and Nass, 2001). Moreover, the results from this study, as well as Flanagin and Metzger (2000) , suggest that messages that exist in an online context where explicit persuasive intent may be present are subject to lower credibility assessments, perhaps as a result of higher scrutiny or skepticism. Overall, message salience and involvement might bolster perceived credibility, although this effect would be mitigated by perceived persuasive intent. Third, audience disposition factors may explain perceptions of web credibility. The findings from this study suggest a group of users who perceived web-based information to be low in credibility, presumably because they are skeptical about the veracity of online information in general. Similarly, Gunther (1992) found both general and media skepticism to be important predictors of newspaper and television news media credibility judgements.Verification is related to skepticism, inasmuch as more skeptical users may be more prone to verify online information, and verification which disconfirms fears could reduce skepticism. Consistent with past research (Flanagin and Metzger, 2000) , the results of this study showed that more experienced web users reported more verification of web-based information, although the negative correlation between self-reported and observed verification suggests that the relation may not be straightforward. Although experience in using the internet did not impact credibility perceptions in this study, it is too early to dismiss the possibility that web reliance or use may be important in credibility assessments: reliance is an extremely important factor in credibility judgements of other communication technologies, such as print and television media (Johnson and Kaye, 2000) . Past experience with the web may impact skepticism, which in turn influences online credibility perceptions; and there have been mixed results in past research exploring this issue, as discussed earlier. In addition, Johnson and Kaye (2002) found that reliance on traditional media and convenience were also good predictors of online credibility judgements. Overall, these factors suggest that perceived web credibility may be a function of aspects of websites and their sponsors (genre, design, and source attribution), message factors (message salience/involvement and persuasive intent), and audience characteristics (skepticism, verification behaviors, and web experience and reliance). It is likely that high perceived credibility would result under situations with well-designed, reputable genres of sites, containing highly salient and involving messages of low persuasive intent, viewed by individuals with heavy web reliance and experience, which in turn might prompt skepticism that may be allayed by information verification.Thus, web credibility theories might posit high perceived credibility under situations where these features are present. Moreover, evidence to date suggests that perceptions of web credibility might stem from dual processes: first, heuristic processing of readily available cues such as site genre and design and, second, systematic processing of cues such as evaluation of message content and the results of verification efforts. Still, theoretical development must appreciate also the complexity of the web environment and crucial differences between site, sponsor, and message credibility. In sum, this study contributes to a burgeoning literature on web credibility by proposing explanations of credibility assessments, including the role of site features, user attributes, and information verification behaviors across different genres of websites and testing these factors in a relatively natural environment across a diversity of web users. Furthermore, the results of this study are leveraged to propose more concrete theoretical mechanisms by which scholars may understand web credibility at a time when doing so is of critical importance to all media consumers. Notes 1 This response rate is calculated based on the 44.4 percent diffusion of the internet at the time of the data collection: it is estimated that approximately 1524 of the 3432 individuals to receive solicitations might have been eligible to take part in this study, which required familiarity with the internet and web. Based on this sample (N ϭ1524), the response rate translates to 18 percent, rather than the raw response rate of 8 percent. Considering the relatively large retired and minority population in the county of origin (who typically have less access to the internet) and the relatively high effort required of subjects to participate in this experiment, this response rate seems acceptable. 2 As discussed later, post-hoc analyses were conducted to determine the nature of any differences between these samples. 3 In all cases, these sites were downloaded in their entirety to a local web server and the story on the effects of radiation was embedded in the sites.The use of these sites was deemed to fall under the 'fair use' provision of the Copyright Act, as assessed by attorneys at the university where this study was conducted.The personal sites were constructed with close adherence to the criteria described by Papacharissi (2002) that characterize personal webpages. Furthermore, individuals were portrayed as amateur journalists or writers, by listing the story that was embedded along with others on a diverse set of issues, under the general heading 'Articles I Have Written'. 4 These changes were made in all pages of each of the sites, for all text, graphics, and links where the names appeared. Furthermore, because these fictitious sites had to be identical to their 'real' counterparts in order to isolate the effects of actual versus fictitious sources (versus design, content, and layout differences that may have changed over time), all eight sites were loaded on the study's web server in order to maintain control over them. Consequently, all links within all sites had to be redirected such that they would remain within the sites on the study's server, rather than linking back to the 'actual' sites external to that server.The scope of all of these modifications was enormous: For example, changing CNN to Onlinenews required that over 3 million separate programming changes be made. For the personal sites, the male and female versions varied only by the names and photographic images of the individuals depicted. Photographic images of the male and female on the sites were pre-tested with a separate sample in order to gauge any differences in perceived competence (Chaiken, 1979) , physical attractiveness, and social attractiveness (McCroskey and McCain, 1974) .There were no main effects for images across these dimensions, although there were minor interaction effects between the rater's sex and these attributions. Consequently, and as mentioned earlier, participant sex was controlled in data analysis. 5 Exploratory factor analysis (EFA) was the preferred method (over, for example, principal components analysis; PCA) because the goal was to assess the latent structure of observed variables in order to determine if participants cognitively distinguished the three dimensions of credibility. However, results did not differ in this regard whether EFA or PCA was used, or if orthogonal or oblique rotation was invoked. 6 Post-hoc analyses of the two subsamples (i.e. registered voters and undergraduate students) indicated no differences between samples for either RQ1 or H3. Moreover, no differences between the full dataset reported in the analyses and the subsamples were found for site credibility. However, minor differences in perceived message and sponsor credibility by site genre (H1) were found across samples, which indicated that voters may be slightly more skeptical than students, particularly of sponsor credibility. In addition, for H2 voters exhibited a significant negative correlation between observed verification and perceived message credibility, suggesting that voters may have been more skeptical a priori of messages on the study's sites. Differences also existed between students' and voters' relative observed verification: voters verified site content less often (7% of the time) than did students (27% of the time) and had higher mean scores on self-report verification behavior. Overall, this suggests that while the voters in this study were more skeptical, their skepticism did not translate into actually verifying the information they saw online. MIRIAM J. METZGER is an associate professor in the Department of Communication, University of California, Santa Barbara. Her reseach includes studies of the credibility of information in the new media environment, problems of online privacy and security, the impact of media on public opinion, and the theoretical and regulatory changes brought about by the development of new media technologies. Address:
The credibility of Web applications has become a multidisciplinary subject, where researchers from communication have been focusing on a more qualitative (and subjective) analysis of credibility REF , while the area of computer science has focused on more objective metrics.
The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information:
New Media & Society
New Media & Society
Sociology Computer Science