# Presentation: 451779

## What does getting WET(Web Event-Logging Tool)Mean for Web Usability?

- Michael P. Etgen & Judy Cantor
- User Experience Engineering Division
- AT&T Labs

## Web Data Collection Approaches

- Traditional Usability Data Collection:
- Hand coding in person or from video
- Advantages: ask questions, get subjective responses, see exactly what happened
- Disadvantages: expensive, time-consuming, can introduce human error, may require extra equipment

## Web Data Collection Approaches

- Server Logs
- Servers automatically log user requests and server responses
- Advantages: automatic and inexpensive
- Disadvantages:
- no data collection on Browser buttons/menus
- can’t log client-only interactions (e.g., anything that doesn’t send a request to the server)
- validity of data - dynamic IP addressing and browser/proxy caching can provide misleading information
- huge amount of data to parse

## Web Data Collection Approaches

- Automated Data Collection Software
- Software runs in addition to the browser, or “logging browser” developed
- Advantages: automatically collects wide range of data at Operating System/Application level
- Disadvantages:
- cost of buying or developing software
- installation of software on each users’ client, and retrieval of data from each client
- platform dependence
- can’t get content-specific data

## Web Data Collection Approaches

- WebVIP from NIST
- Copy site, add inline event handling code to each link
- Advantages: client platform independence, data is content-specific, inexpensive
- Disadvantages:
- requires copying entire site, and altering code for each link
- no data collection on Browser buttons/menus (use inferred)
- won’t work if Javascript is disabled
- no data collection on HTML elements other than links (e.g., checkboxes, radio buttons, selection lists, etc.)

## Web Data Collection Approaches

- Server Side

- (1) Browser generates events in response to user interactions with web page: click, load, change, mousedown, etc.

- (2) Javascript can temporarily log event time, type, and source info on Client

- server responses

- client requests

- Internet

- (3) Javascript can format event log and send it back to server

- An Alternative: Web Event-Logging

- Client Side

- (1)

- (2)

- (3)

## Web Data Collection Approaches

- Web Event-Logging Tool
- Advantages:
- content-specific, client-side data collection
- client / server platform independence
- code is modular, so easy to insert/delete, does not require copied site or adding code to individual HTML elements
- you choose events/pages/features/etc. to log, and only log those
- inexpensive
- Disadvantages:
- won’t work if Javascript is disabled
- no data collection on Browser buttons/menus (use inferred)
- Solution for getting data back is specific to site/application

## Web Data Collection Approaches

- Web Event-Logging Tool
- Examples of events that can be logged and the sources of the event:
- _load, unload, move, resize_ - window
- _click_ - link, area, button, radio, checkbox, reset, submit
- _mousedown, mouseup_ - link, button, radio, checkbox, reset, submit
- _change_ - text, textarea, select
- _keypress_ - text, textarea, password
- _mouseover, mouseout_ - link, area
- _blur, focus_ - window, text, textarea, password, select

## Implementation in a Web App

- Background: Planning and Coordination tool for Executive Council
- Functionality: Meetings, Action Items, Meeting Minutes & Presentations, Confirm Availability, Contact Members
- Technology: ColdFusion and MSAccess database

## Implementation in a Web App

- Configuring WET
- Basic Steps for Any Site/Application
- Specify events and temporary storage method
- Place a copy of WET.js on web server
- Insert call to WET.js in all document headers
- Specify method for log retrieval
- Additional Steps for current trial
- Code for turning WET on/off based upon user login
- Code for Start/Stop links, created as dynamically positioned layer so won’t scroll off screen

## Implementation in a Web App

- Application Before WET

## Implementation in a Web App

- After WET is configured and running

## Implementation in a Web App

- Method
- Study directions, task list, and background information were sent to participants by e-mail
- Testing was remote, no direct contact between testers and participants while performing tasks
- Four participants completed trial
- 7 Tasks covered all the basic functionality of the application

## Implementation in a Web App

- Method
- Metrics and Objectives
- Three metrics were calculated for each task and compared to usability objectives
- _Task Completion_ - examined  page access information in log
- _Time to Complete Task_ - subtracted time for click on stop link from click on start link
- _Deviation from Ideal Interaction Model_ - comparison to expert path

## Implementation in a Web App

- Example from Trial Tasks
- “Contact a specific council member (e.g., Judy Cantor) and send an e-mail to them indicating that you are finished with the trial.”
- Metrics and Objectives
- _Task Completion_ - 3 out of 4 successfully complete task
- _Time to Complete Task_ - 3 out of 4 complete task within 120 seconds.
- _Deviation from Ideal Interaction Model_ - 3 out of 4 deviate from ideal no more than twice

## 12:46:44.242, load, home.cfm

- 12:46:49.726, click, Link:javascript:task(‘start’), home.cfm
- 12:46:54.867, click, Link:contact.cfm, home.cfm

## 12:46:57.570, load, contact.cfm

- 12:47:04.258, click, checkbox=user=mike, contact.cfm
- 12:47:07.898, click, submit=Compose Mail, contact.cfm

## 12:47:08.103, load, mail_form.cfm

- 12:47:10.346, change, text=subject, mail_form.cfm
- 12:47:17.582, change, text=message, mail_form.cfm
- 12:47:35.957, click, submit=send, mail_form.cfm

## 12:47:39.586, load, contact.cfm

- 12:47:42.961, click, Link:javascript:task(‘stop’), contact.cfm

## Implementation in a Web App

- Results for Example E-mail Task
- Task Completion: 4 out of 4 successfully completed the task
- Time to Complete Task: 3 out of 4 successfully completed the task in less than 120 seconds.
- Deviation from Ideal Interaction Model: 4 out of 4 had no more than 2 deviations

## Implementation in a Web App

- Lessons Learned
- Showing companion start/stop graphics was not reliable method for indicating task start
- participants couldn’t recall if they’d clicked “start”, so we had to use first click on menu item on home page.
- all remembered to click“stop” when finished though.
- Possible solutions:
- Place “start” in dialog window on top of home page so the user cannot proceed without clicking it.
- Change appearance of start after it has been clicked

## Implementation in a Web App

- Lessons Learned, continued
- Think about what needs to be logged carefully
- Amount of data can be overwhelming
- For Usability Testing: choose the minimum number of events that reflect how site is being used
- For Usability Monitoring: log only certain pages, or certain functionality
- Think about temporary storage method in terms of site/application architecture
- some possibilities are session-only cookies, frame variables, variables in parent window, etc.
- best to use method which least alters site/application

## Future Directions

- To Do List:
- Apply the technique to other web-relevant technologies that use events (e.g., Java applets/applications), and integrate that data with the Javascript WET data
- Investigate whether generalized configuration modules are possible for certain “types” of sites/applications, so implementation isn’t as site/application specific
- Make code publicly available for free to anyone who wants to use it

## Future Directions

- Wish List:
- If we can create generalized configuration modules, then we could create a wizard for installation and configuration
- For a basic set of metrics, create an analysis wizard for data summarization and analysis

## Conclusion

- The technique of Web Event-Logging (as embodied in WET) provides the usability engineer with a method of collecting web usability data that is automated, customizable, and comprehensive.
- What does getting WET (Web Event-Logging Tool) Mean for Web Usability?
- It means you _***can***_ gather usability data on a site/application within the limited time usually afforded to such activities on web projects.