This article presents an experimental study of privacy-related attitudes and behaviors regarding a music recommender service based on two types of user modeling: personality traits and musical preferences. Contrary to prior expectations and attitudes reported by participants, personality traits are frequently disclosed to the system and other users, indicating that embedded modeling of user personality does not represent an acceptance barrier. Discrepancies between privacy attitudes and behaviors have been reported before in the context of e-commerce applications, but the corresponding studies could not exclude several conflicting hypotheses, such as participants expressing attitudes outside the context of specific privacy dilemmas and contact with researchers, which may have mitigated perceived privacy risks. Arguably, these are fundamental problems in empirical investigations into privacy that apply to most published works relating to privacy and user modeling. Measures to control these factors in this study are discussed, and methodological suggestions for future research are presented. A daptive systems and personalized systems in particular rely on having appropriate and sufficient information about their users to operate optimally. This could, for example, include information about the identity of the user, earlier usage of a service, the user's preferences and dislikes, and many other types of data (Kobsa, 2001) . The collection and processing of such information can conflict with privacy concerns (Kobsa, 2002) . More specifically, it has been suggested that privacy-related concerns are contingent on the kind of information collected (Ackerman, Cranor, & Reagle, 1999; Adams & Sasse, 2001; Bellotti & Sellen, 1993) , the degree of control users have over disclosure (Bellotti & Sellen, 1993; Olivero & Lunt, 2004) , the degree of accessibility (Adams & Sasse, 2001; Bellotti & Sellen, 1993) , or the way the information is used (Adams & Sasse, 2001; Bellotti & Sellen, 1993) . Information may be collected by the user's explicit and intentional input, or information may be collected implicitly by monitoring the users as they go about their own tasks or activities. Implicit collection of information is an essential element of scenarios relating to Ambient Intelligence (Aarts, Harwig, & Schuurmans, 2001 ) and can lead to privacy issues because of a lack of awareness and control by the people concerned (Cranor, 2004; Kobsa & Schreck, 2003; Nguyen & Mynatt, 2002) . In the context of this study, the term privacy refers to a boundary control process in which individuals regulate when, how, and to what extent information about them is communicated to others. This interpretation is based on the definitions by Westin (1967) and Altman (1975) . In this study, the emphasis is on information privacy rather than interpersonal or social privacy. Information privacy refers to the claim by individuals that data about themselves should generally not be made available to other individuals and organizations and that, where data are possessed by another party, the individual must be able to exercise a substantial degree of control over that data and their use (Clarke, 1999) . Several technological solutions to guarantee people's privacy have been proposed (for a relatively recent survey, see Langheinrich, 2005) . Other researchers have focused on providing design guidance to designers of Internet-based applications (e.g., Ackerman et al., 1999; Good & Krekelberg, 2003) and ubiquitous computing applications (Bellotti & Sellen, 1993; Langheinrich, 2002; Lederer, Hong, Dey, & Landay, 2004; Nguyen & Mynatt, 2002) . In contrast, there is still little known about how users of personalized systems experience disclosure of information, what motivates them, and how conscious they are of the choices they make concerning the disclosure of information. Acquisti (2004) has applied behavioral economics to understand the individual decisionmaking process with respect to privacy in e-commerce. He has described how incomplete information and psychological distortions may prevent an individual from behaving rationally with regard to privacy. Privacy appears to be an inherently difficult concept to study. For example, it seems that the privacy concerns people report during surveys do not match their actual behavior (see Berendt, Günther, & Spiekermann, 2005) . In their study, participants were first questioned about their privacy preferences and afterward exposed to an anthropomorphic shopping "bot." The shopping bot asked various questions about product attributes, but it also asked more personal questions than customers should be expected to answer. Contrary to the researchers' expectations, even participants who were classified as highly concerned about their privacy according to their reported attitudes displayed a strong tendency to self-disclose. There can be several explanations for this discrepancy. In general, attitudes are not good predictors of the behavior that is mediated by social and environmental factors and the costs that need to be incurred to exhibit behavior compliant with the attitudes expressed (see Ajzen & Fishbein, 2005) . Furthermore, as already known from experiments by Milgram (1974) , participants can engage in behaviors they would not normally engage in under the authority of the experimenter. An alternative explanation is that participants perceived the research study as a safe context for disclosure, trusting researchers to moderate the impact of disclosure. These methodological problems also occur in many studies of privacy in the domain of computer-mediated communications, Internet use, and Ambient Intelligence. Studies of privacy restricted to laboratory experimentation or surveying opinions lack external validity in the absence of actual risk and outside a realistic context of use. Field observations of the actual use of a system can disguise many of the privacy concerns people may have 2 Social Science Computer Review despite demonstrated risk-taking behavior and do not offer sufficient controls for the privacy risks and the benefits of the disclosure. This article presents an experimental study that was set up to provide users with realistic privacy dilemmas during the use of a personalized music recommender service that was created specifically for the purpose of this study. Participants were presented with disclosure choices that were as realistic as possible. Logging of their behavior in these situations was combined with self-report and attitude measurements regarding privacy. The study enabled us to test some of the assumptions regarding privacy and personalized systems and provided an opportunity to explore the motivation behind observed privacy behavior. First of all, the methodology used will be described, then the findings will be presented, and this is followed by a discussion of the results in the light of other research. Finally, the conclusions of this study and implications for future work are addressed. This study involved participants using a music recommender service accessed through a web application over the Internet. Participants used this service from their own computers at work or at home, similar to any other Internet-based music recommender service. The system confronted the participants with privacy dilemmas relating to two types of personal information: preferences for different music genres and information about their personality. Participants were asked to disclose this information to the recommender service and/or other users of this service. Rentfrow and Gosling (2003) have established a relationship between music preferences and personality traits, which makes it feasible to recommend music based on personality traits. For example, someone who scores high in terms of the personality trait "neuroticism" as well as "openness to new experiences" is likely to be interested in the "reflective and complex" music dimension. Music genres that fall into this music dimension are blues, jazz, folk, and classical music . 1. The interest in contrasting these music preferences and personality traits comes from the relative legitimacy of these two approaches to user profiling for the purpose of recommending music. According to Iachello and Abowd (2005) , an application is legitimate if the interest in using it for a specific purpose justifies the burden on individuals' rights. Music preference data are seemingly innocuous but directly relevant to the music recommender and are commonly offered by users to systems of this kind. However, music preferences can be used for direct marketing, so they could also be considered to be somewhat sensitive by some users. Personality traits are more or less stable, internal characteristics of people, which make their behavior consistent from one time to another yet different from the behavior that other people would exhibit in comparable situations (Child, 1968) . As a user modeling approach, personality traits have a lot of potential because they are domain independent and may be applied in a much wider range of contexts than music recommendation. Conversely, a personality profile of a user can be misused and misapplied in a variety of contexts (e.g., by a prospective employer or a medical insurance provider). Storage of personality traits seems to have little legitimacy in the context of music recommendation and does therefore not adhere to the principle of proportionality. This means that it does not balance the usefulness of the application and its effect on privacy (Iachello & Abowd, 2005) . During the experimental study, participants were confronted with choices regarding whether to disclose personal information and whether disclosure would be anonymous. It was anticipated that personality trait information would be regarded as more sensitive, resulting in comparatively less disclosure than for music preferences. It was also anticipated that showing profile information directly to other users would result in less disclosure than if the profile information were used only by the recommender system and not shown to anyone else. It is important to realize that this study is of an explorative nature, combining quantitative and qualitative methods. The underlying attitudes were set out to check whether existing instruments for assessing privacy attitudes would be good predictors of actual behavior. Furthermore, context-specific privacy preferences were surveyed and explanations for disclosure behavior considered; these were then analyzed qualitatively to explore the relationships between them. The study followed a within-subjects design whereby participants were exposed to profiling of both music preferences and personality traits. The order in which participants used the two recommender systems was counterbalanced. Participants could use the recommender systems at home or at work. The study involved four disclosure choice moments. Two disclosures related to profile information and the other two related to personality traits. In both cases, participants were first asked to share information for comparison by the system (as in collaborative filtering) and to then share the current profile information with other users. In each case, participants could choose between three levels of disclosure at these choice moments: no disclosure, anonymous disclosure, or disclosure including identity information. Data of both a qualitative and quantitative nature were collected: Actual disclosure choices were monitored by system logs, explanations for these choices were gathered through questionnaires and interviews, and attitudes toward privacy and the use of personal information were measured by means of a questionnaire. Participants were recruited by e-mail announcements via secretaries and bulletin boards within a technical university and an industrial research lab. In view of the database of music available, recruitment was aimed at participants ages 18 to 50. Forty-eight participants took part in the study and completed the online questionnaire. As a reward, participants were promised a music CD selected on the basis of the songs recommended to them. This was done to motivate participants to be honest about their music preferences and personality traits; it served to increase the realism of both the costs and benefits of the disclosure they would make during the experiment. The ages of participants ranged from 17 to 49, with an average age of 26. Half of participants were 23 years old or younger. Twenty-one participants were interviewed. Interviewee ages ranged from 19 to 49, with an average age of 27. Social Science Computer Review The music recommendations were provided via a web-based application created for the purposes of this study. The service offered personalized playlists of songs. Streaming technology was used to make these songs available for participants to play on their computers. The experimental recommender service is built on a database of nearly 6,000 songs spread evenly over 14 different music genres. While using the personality-based recommender, participants saw a screen like the one shown in Figure 1 . At the top of the screen, a status bar showed the number of times the user had logged into the service, the number of playlists requested, the source of information on which the recommendation was based, and the progress within the study. On the left-hand side of the screen, an overview of the current profile information based on personality traits was shown. The right side of the screen displayed the current playlist. Below this playlist, a button was displayed; this could be used to request a new playlist. The screen of the recommender based on music preferences was similar to the screen shown in Figure 1 , except that the status part stated, "The recommender is using your preferences to generate playlists." On the left side of the screen, an overview of the music preference profile was shown. The procedure of the experimental study is shown schematically in Figure 2 . People who were interested in participating were sent instructions by e-mail. They were not informed in advance that the research related to privacy. Participants had to register on the music recommender Web site by providing their e-mail address. This was their business/university e-mail address, which consisted of their name and company. At the time of registration, participants were assigned randomly to an experimental group (linked to a specific order in which they would experience the two recommender systems) and received a personal access code. Participants were asked to access the portal site of the music recommender on six separate days to ensure that the participants would not rush through all phases of the study. They were asked to listen to at least one playlist per day. If a participant had listened to a playlist the previous day, the participant was offered the next phase. The music recommender therefore served two purposes: It operated as a music recommender service, and it also implemented the experimental procedure. The participants were asked to use the music recommender for 6 days within a period of 2 to 3 weeks. As indicated earlier, the order in which participants experienced the two recommender systems was counterbalanced. The first phase of both music recommender systems (5/8 in Figure 2 ) consisted of a default situation in which no additional disclosure of information was required (local use of the information only). In the second phase (6A/9A in Figure 2 ), participants were asked to disclose their profile information for the purpose of collaborative filtering (for the participants, this was referred to as "comparing preferences to those of others users" or "comparing personality traits to those of others users"). In the third phase (7A/10A in Figure 2 ), participants were asked to disclose their profiles directly to other users. In this way, participants were asked to reveal an increasing amount of information. van  In each of the three phases of using the two recommender systems, the actual recommendation mechanism remained constant, but the percentage of recommendations that were offered according to the profile was increased if users chose to disclose their profile information. This was done to ensure that the recommender performance would improve predictably when the user chose a higher level of disclosure. This recommendation mechanism was not revealed to participants. Instead, participants were told in advance that the recommendations might improve as a result of their disclosure. Without more information about the recommendation mechanism, they would suppose that the data would actually be used for collaborative filtering. In the first phase of both recommender systems, 80% of the recommendations were generated according to the user's profile. Depending on the choices made by the participant in the second and third phase (regarding whether to disclose and whether disclosure would be anonymous), the recommendation could improve, so that 90% or 100% of the recommendations were generated according to the profile. After they had used the music recommender, participants were asked to complete an online questionnaire Finally, 21 out of 48 participants were contacted to arrange an interview appointment after the completion of the online questionnaire. The interview was set up in such a way as to achieve an open atmosphere in which participants would express their feelings freely. Interviewees were questioned in more detail about their choices during the study and encouraged to give thorough explanations about the answers they had given in the online questionnaire. Before participants could actually receive personally recommended playlists, they had to provide their first type of profile information (which depended on the experimental group to which they were assigned). The second type of profile information was collected only after 3 days of using the first recommender system (see Figure 2) . The profile information was collected by means of two short, validated measures: the Short Test of Music Preferences (STOMP) and the Ten Item Personality Measure (TIPI). The STOMP test ) inquires about the basic preference level for 14 different music genres on a scale from 1 (strongly dislike) to 7 (strongly like). It asks people to: "please indicate your basic preference level for the genres listed using the scale provided."  Note: MR = music recommender; G = experimental group; MP = music preferences; PT = personality traits; DC = disclosure choice; X = first source for recommendations; Y = second source for recommendations. Example genres are country, jazz, or rock. The STOMP values were used as a basis for music recommendations in the music-preference-based application. The TIPI (Gosling, Rentfrow, & Swann, 2003) encompasses a personal judgment of the extent to which 10 pairs of personality traits apply on a scale from 1 (disagree strongly) to 7 (agree strongly). It gives people the following task description: "Here are a number of personality traits that may or may not apply to you. Indicate (. . .) the extent to which you agree or disagree with that statement. You should rate the extent to which the pair of traits applies to you, even if one characteristic applies more strongly than the other." Examples of pairs of personality traits used in TIPI are "reserved/quiet" or "sympathetic/ warm." The TIPI scores were converted to the Big Five personality dimensions (i.e., extraversion, agreeableness, conscientiousness, neuroticism, and openness to experiences) and used as a basis for music recommendations in the personality-traits-based condition. The Big Five personality dimensions are frequently used in personality assessments, although they are usually based on long questionnaires of sometimes more than 200 items that have to be rated. For the purpose of the personality-based music recommender system, it was felt that such a long questionnaire would take up too much time and effort for the participants. The short TIPI test was created for situations where very short measures are needed or where personality is not the primary focus of interest. Both these conditions hold for this study, so the TIPI was chosen as a personality inventory. Furthermore, because the length of TIPI and STOMP are quite comparable, participants were likely to have similar expectations about the use of the two types of profile information by the two different recommender systems (a 200-item questionnaire might raise the expectations of participants with regard to the accuracy of the music recommendations, and it would be likely to raise suspicion about other potential use of the information as well). While the music recommender was being used, the actual level of disclosure chosen (no disclosure, anonymous disclosure, or disclosure including identity) in the four choice situations was recorded. This measure relates to the main aim of the research: to see whether participants would choose different levels of disclosure depending on the type of information involved (either personality traits or music preferences) and the use of the information (for comparison by the system or for showing directly to other users). Besides their choice of disclosure, participants were also asked to rate the quality of each playlist of recommendations on a 5-point scale. This quality rating was included to assess whether there was any relationship between the perceived benefits of the system and the disclosure behavior of participants. After using the music recommender for 6 days, all participants were asked to complete an online questionnaire consisting of some open and some multiple-choice questions. This questionnaire was aimed at understanding the factors underpinning the disclosure behavior and it addressed these topics: • General demographics and interest in music. • General appreciation of the music recommender. • Attitudes toward disclosing music preferences and personality traits. • Explanations for the level of disclosure chosen before and after the study (open question). These questions were posed to see whether people's experience of using the recommender 8 Social Science Computer Review would lead them to make different choices afterward (e.g., because of disappointing benefits or any privacy concerns they had). • General privacy attitude measures were taken to see whether they would explain any differences in disclosure behavior. Privacy attitudes were measured by different tools: the Privacy Segmentation Index (PSI; Harris Interactive, 2002) and the Privacy Attitude Questionnaire (PAQ; Chignell, Quan-Haase, & Gwizdka, 2003) . In addition to these tools, some general questions regarding privacy (e.g., "I like to get advance notice if information is collected about me," or "I am willing to provide personal information in return for lowcost products or convenience"), and four questions about the worries concerning the disclosure of personal information in different situations were included. The PSI consists of three statements about the use of personal information by organizations. Participants have to judge the extent to which they agree with each of these three statements. This tool was included because of its brevity and because this type of clustering is widely used in human-computer interaction (HCI) research (Ackerman et al., 1999; Berendt et al., 2005; Consolvo, Smith, Matthews, LaMarca, Tabert, & Powledge, 2005) . The PAQ consists of 36 statements about various behaviors relating to privacy that people may or may not exhibit. Example statements include the following: "No organization or person should disseminate personal information about me without my knowledge," "I respond to telephone marketing surveys," or "I like to change my passwords frequently." Participants had to judge the extent to which they agree with each statement. The tool was originally developed to aid designers, as there is little information they can use as a basis for the design of new technologies and interfaces with privacy implications (e.g., personalization). The PAQ tool was included in this study because of its relevance to the domain of personalization. This also allowed a comparison of the two different tools for measuring privacy attitudes. In-depth semistructured interviews were conducted, which varied in duration between half an hour and an hour. After the completion of the online questionnaires, participants were given an interview appointment. All pilot participants were contacted and interviewed (see Perik, de Ruyter, Markopoulos, & Eggen, 2004) . The remaining participants were contacted at random. Twenty-one interviews were conducted. The aim of the interview was to gain a more thorough understanding of the factors on which disclosure decisions are based and to obtain additional information about some of the answers given in the online questionnaire. The interviews were semistructured and covered the following topics: • Opinion on the music recommender. • Expected goal/aim of the research. This topic was addressed in an open fashion without mentioning privacy at all. The topic was included to check whether people were aware that the research related to privacy and whether this influenced their disclosure behavior. • Considerations for choosing a specific level of disclosure during and after the study. • Some questions to address their understanding of the system (e.g., accessibility of data to other parties or expectations about changes in the system after each disclosure choice). • Experience with the system after each disclosure choice. This was addressed to see whether participants noticed any benefits from disclosing information and whether this may have influenced their disclosure choices. • Feelings about the disclosure of music preferences or personality traits profile information. van  The disclosure behavior of the users as it was logged and an analysis of the self-reported data are presented. First, the procedure used for the analysis of the qualitative data is explained. If necessary, the raw data were translated into English (all interviews were conducted in Dutch, recorded on tape and transcribed verbatim; the questionnaire questions were posed in English, but a few participants chose to answer in Dutch). Both the questionnaire and interview included open questions, where participants were free to express their feelings in their own words. The questionnaire and interview data were analyzed by means of open coding (Strauss & Corbin, 1990) . Some numerical summaries of the coded data are given if such an overview is considered to help promote clear presentation of the data, but it should be taken into account that the interview and questionnaire data are of a qualitative nature and should be interpreted as such. During the use of the recommender, participants were asked to rate the quality of each playlist on a 5-point scale ranging from 1 (very bad) to 5 (very good). The average quality rating of all playlists was 3.02. Playlists that were based on music preferences were rated higher on average (3.41) than playlists that were recommended to participants on the basis of personality traits (2.63, p < .001). This suggests that based on the inventories used for user profiling (TIPI and STOMP), personality traits are less suitable for providing music recommendations than preferences for music genres. A similar image arose from the questionnaire data where participants were asked to list "things they liked or disliked." Four participants said they liked the music that was recommended to them on the basis of their preferences for music genres, and only one said they disliked these recommendations. In contrary, with regard to the system based on personality traits, four participants were not satisfied with the recommendations, and only one participant was happy with the outcome. As "things they liked or disliked," most participants mentioned the music that was recommended to them. Hardly any privacy-related topics were mentioned. Only one participant mentioned, among other things, that "there is very little information about how your decisions will affect your privacy." Another participant said he or she disliked "the comparison of my data with that of other persons" without further explanation or reference to privacy. Five participants said they disliked the lack of control over the user profile after the initial information was provided. Even though this does not necessarily imply a lack of privacy, it should be noted that user control is frequently mentioned in relation to privacy (see the definition in the introduction and, for example, Bellotti & Sellen, 1993; Margulis, 2003; W3C, 1998) . The fact that the vast majority of participants 10 Social Science Computer Review did not address the topic of privacy explicitly suggests that they experienced the system as a music recommender service and not as a study in privacy. In the interviews with 21 of the participants, their initial reaction to the music recommender was discussed to see if they would mention any privacy issues. Overall, 10 interviewees were positive about the music recommender, 5 were negative, and 6 were more or less neutral. None of the interviewees raised privacy issues at this point. Many participants commented on the concept of the music recommender or the type of music that was recommended to them (e.g., "I thought it was very surprising. Nice and surprising. (. . .) I got to hear music that I didn't already know but liked nevertheless. It was nice for once to hear some new music instead of the music I have in my own playlist." This implies two things: Participants perceived the study as being about music recommendations, and the music recommender provided actual benefits to them. Participants were asked via a questionnaire about their initial reaction to having to disclose profile information to the system. Most participants said they had no problem with providing either music preferences (n = 21) or personality traits (n = 19). Many participants indicated that they expected to be asked about their music preferences (n = 18), but the inquiries about personality traits surprised quite a few participants (n = 6). No privacyrelated comments were made with regard to the collection of music preferences or personality traits. So although personality traits were not readily associated with music recommender services, it seems that participants were quite open about disclosing them. The goal of the research was discussed at the beginning of most interviews. There were some interviewees who expected only the development or improvement of the music recommender to be the goal of the research. Although there were also some interviewees who expected privacy to be of interest to the research, it was never mentioned as a single expected goal. On the whole, interviewees turned out to be unaware of the actual aim of the research. It seems that interviewees who expected that privacy was the focus of the study tended to disclose less information than those who did not mention it. Interviewees also tended to disclose less information if they did not expect the focus of the research to be on music preferences (compared to those who did or those who did not mention it). It seems that participants feel more comfortable disclosing information if they are under the impression that they are using a music service and if privacy is not mentioned. It seems that thoughts about privacy make people more aware of the risks involved. An overview of the disclosure behavior of all participants is shown in Figure 3 . The number of participants who chose a particular level of disclosure per disclosure situation is shown in a circle. The number between brackets on the arrow between two circles refers to the number of participants who chose a similar set of disclosure levels in two consecutive situations. For example, [3] refers to the 3 participants who chose anonymous disclosure in the situation relating to comparison of music preferences by the system and chose disclosure including identity information in the situation concerning showing music preferences directly to other users. Figure 3 shows that participants were divided in their disclosure behavior and maintained the same level of disclosure (44% chose anonymous disclosure and 42% chose disclosure including identity information) throughout the four choice situations. Only 15% of the participants varied their chosen level of disclosure at different phases of the study. These variations did not show a clear trend toward increasing or decreasing levels of disclosure. Similar findings were reported earlier for the pilot of this study (Perik et al., 2004) . In the questionnaire and interviews, participants were asked to explain why they chose a specific level of disclosure. Motivation for anonymous disclosure. Many participants who chose anonymous disclosure said their choice was based on privacy concerns (n = 19). One of them said, "People may see the information I provided; I do not know who reads it and therefore prefer to have the information anonymous." Some chose anonymous disclosure just to be on the safe side (n = 4) (e.g., "I felt that others did not need to know my name . . ."). Motivation for disclosure including identity information. However, most of the participants who disclosed identity information based their decision on the benefit they might gain in return (n = 12). As expressed by one participant, "I want the system to perform best. Note: The three rows represent the different disclosure options participants were given at each of the four disclosure choices. The columns represent the three different phases (of which one default situation where no disclosure choice could be made) for the recommender systems based on music preference and personality traits. The order in which participants used the two recommender systems was counterbalanced. That's why I gave full permission." Another large group (n = 10) said they chose disclosure including identity information simply because they had "no problem" disclosing the information. A number of participants said that they wanted to support the research by disclosing their information (n = 7), and some of them added that they didn't care about the information disclosure either. The online questionnaire and the interview data together indicate that participants felt quite safe disclosing personal information within the context of this experiment, even though they were actually allowing the system to show their personal information to other users. It can therefore be argued that participation in the research did have some (but not much) influence on the behavior of participants.
Van de Garde-Perik et al. REF noted the "discrepancy between stated attitudes and user behavior relating to privacy".
Investigating Privacy Attitudes and Behavior in Relation to Personalization
Social Science Computer Review
Computer Science