Measuring wireless network usage with the Experience Sampling Method

Measuring wireless local area networks has proven useful for characterizing, modeling and provisioning these networks. These measurements are typically taken passively from a vantage point on the network itself. Client devices, or users, are never actively queried. These measurements can indicate what is happening on the network, but it can be difficult to infer why a particular behavior is occurring. In this paper we use the Experience Sampling Method (ESM) to study wireless network users. We monitored 29 users remotely for one week, and signaled them to fill out a questionnaire whenever interesting wireless behavior was observed. We find ESM to be a useful method for collecting data about wireless network usage that cannot be provided by network monitoring, and we present a list of recommendations for network researchers who wish to conduct an ESM study. Researchers have conducted many wireless local area network (WLAN) measurement studies, collecting data to help characterize, model and provision these networks. Typically these studies take network-side measurements, counting the number of clients, the amount of traffic transmitted, client session durations and so forth. Such studies are useful for determining what is occurring on a network. They may not, however, explain why a particular behavior is observed. For instance, in one of our recent studies [13], we observe increased peer-to-peer filesharing on our campus WLAN, despite the presence of a higher-bandwidth wired network. Why are users choosing to use the slower wireless network for downloading such large files? Is it because the convenience of the wireless network outweighs its limited capacity? Or is it because the users are unaware of the wired network? Such questions cannot be answered by network-side measurement alone. To understand why particular wireless network behavior occurs, it is useful to ask network users about their usage patterns. A one-off survey or questionnaire, however, may not be appropriate. If a user is asked about a wireless behavior some time after the event has occurred, they may have forgotten the conditions at the time of the event, or the particular reason for their actions. Instead, we need to way to non-intrusively record all aspects of a user’s wireless network experience. In this paper, we describe an Experience Sampling Method (ESM) study of a campus-wide WLAN. ESM is a psychology method for studying user experience. We compare the data obtained through ESM to data from network monitoring. We introduce the Experience Sampling Method in the next section, and describe previous ESM studies and wireless network measurement studies. Section II describes our experimental setup and methodology, and Section III presents some brief results. In Section IV we list some of the lessons that we learnt in the course of conducting the study, and Section V concludes. I. BACKGROUND AND RELATED WORK Diary methods are a group of research tools and methods used in psychology for “documenting the particulars of life. ” [6]. The distinguishing feature of a diary method is that participants self-report their own ongoing experiences. This enables the recording of events and experiences in a more natural context than a formal interview. Diary methods can be divided into three categories [22]: intervalcontingent, where participants report at regular intervals; signal-contingent, where participants report when they receive a signal; and event-contingent, where participants report whenever a defined event occurs. Within the broader category of diary methods, the Experience Sampling Method (ESM) [18] has emerged as a popular method for evaluating user experiences and situations. In an ESM study, also referred to in the literature as Ecological Momentary Assessment (EMA), participants fill out a questionnaire several times a day. The questionnaire asks about the participant’s current activities, conditions and feelings. A typical ESM study is signal-contingent, involving seven signals a day over seven days. ESM has been used to evaluate a wide variety of experiences, such as job satisfaction amongst teachers [5], happiness amongst American youths [9], and the quality of experience of new parents before and after childbirth [10]. Most relevant to our work are ESM studies that look specifically at communications or technology. Kubey et al. [17] present a survey of communications-related ESM studies, such as television viewing, and communication patterns within families and marriages. Consolvo and Walker [8] use ESM to evaluate user experiences with the Intel Personal Server ubiquitous computing device. Gaggioli et al. use ESM to analyze quality of experience and presence in virtual environments [12]. Technology can also be used to improve ESM methodology. Instead of using a notebook or paper questionnaires to record responses, Palen and Salzman ask their participants to record voicemail messages upon receiving an alert [19]. Another common electronic ESM tool is to use a PDA for questionnaires [4]. Several researchers have measured and characterized wireless LANs, mainly in academic settings. Academic WLANs that have been measured include those at Stanford [21], Georgia Tech [14], Dartmouth [13], [16], UNC [7] and the University of Saskatchewan [20]. These studies have all taken place on the wired side of the access points, using packet sniffers, SNMP polling, syslog records, or a combination of these techniques. The non-academic WLANs that have been studied include a conference WLAN [2], and the WLAN at a corporate research campus [3], both of which were also measured using SNMP or packet sniffing. More recently, WLANs have been measured from the wireless side of the access point; that is, measuring the wireless medium itself. Wireless-side measurement is complex due to the unreliable nature of the wireless medium, hardware and software incompatibilities, and so forth. As such, most wireless-side studies are of a smaller scale than wired-side studies. Yeo et al. measure a single access point at the University of Maryland [23], while various other studies have taken place in controlled wireless conditions [1], [11], [15].