Digital Format of Experience Sampling Method - Transformation, Implementation, and Assessment

The experience sampling method was transformed into a digital format and examined in the Web environment to elicit on-line Web users’ experiences through multiple iterations of an on-line questionnaire. By sampling Web users’ on-line experiences and detecting Web users’ situated experiences from a time point very close to their actual experiences, this tool can collect reliable and valid data with minimal distortion. This tool can effectively and unambiguously tap Web users’ internal experiences associated with their use of the Web. Data on Web users’ on-line experiences were collected and analyzed to evaluate the reliability and validity of the tool. This tool should be applicable to other fields to elicit Web users’ situational experiences and perceptions in a variety of research contexts. Experience Sampling Method The experience sampling method (Csikszentmihalyi, Larson & Prescott, 1977) was originally designed to systematically examine the interaction between human beings’ internal experiences and their reports of their externally observable behaviors. The original format of the ESM consisted of providing respondents with an electronic pager and a questionnaire booklet. The researchers randomly activated the pagers several times a day. Upon receiving a signal on his pager, the respondent would fill out one questionnaire from the booklet. By the end of the study, which usually lasted from one to two weeks, the booklet contained a systematic description of the person's life in different situational contexts, at different given moments, including their behaviors, perceptions and other internal experiences. The behaviors might have included the activities performed, places visited, and people encountered at given moments. Internal experiences refer to perceptions and other cognitive behaviors, i.e., what a person feels and thinks during specific moments in his/her perception of self and the world, such as emotions, motivational states, challenges encountered and skills brought to the situation (Csikszentmihalyi & Csikszentmihalyi, 1988). The ESM has been extensively and continuously employed in a variety of research settings since it was developed at the University of Chicago in 1976. The ESM has been described as an appropriate and useful tool to study interaction among situations, personality, and the environment (Hormuth, 1986). The randomness of signals sent out from the researcher to subjects' pagers over time makes the random sampling of subjects' experiences possible. As a general-purpose research tool which is not limited to any specific research question or domain, we felt it could be applied to the online environment, such as the World Wide Web. In this paper, we introduce the transformation of the ESM to the environment of the World Wide Web and its recent applications in studying Web users’ internal experiences. On-line Transformation of ESM There are a few important features of the ESM which must be preserved during the transformation: the scheme of the random signaling and the format of the questionnaire. In the original formulation of the ESM, the signaling device is usually a pager, carried by each subject. The format of the questionnaire is usually a booklet with multiple copies of the same questionnaire. To explore the Internet and the Web environment, the signaling device and the booklet were transformed into digital versions. The Signaling Device In general, there are several possible approaches for the transformation but here we only discuss the approaches we tested out. The first approach was to employ a stand alone computer application, which interrupts Web users' behavior by popping up a questionnaire repeatedly. In this approach, a computer application was created to interact with subjects’ Web browsers (either Microsoft's Internet Explorer or Netscape's Communicator) via the technology of DDE (Dynamic Data Exchange). Through the interaction over DDE, the application accesses the browser and records data related to Web users' browsing behaviors, such as the URL visited, length of time subject stays at each Web page, and entry, idle and exit times. In the beginning of a study, subjects are asked to download the application from the researcher’s server and run it on their local machines, along with their browsers. The application then pops up the questionnaire on their Web browsers and asks subjects to fill out the questionnaire in accordance with their situated experiences immediately before the questionnaire appeared. The questionnaire reappears several times at random intervals as long as the Web users stay on-line. At the end of a study, data collected are sent