# Presentation: 693086

## Towards Web Macros: Automating Common Tasks on the Web

- Alex Safonov, Joseph A. Konstan, John V. Carlis
- University of Minnesota
- Department of Computer Science and Engineering

**Notes:**
We chose this longish title because it gives us an opportunity to explain our messages and goals. The Web now has information services covering almost any XXX. To get the desired information, users in some cases must engage in repetitive and tedious interaction with the Web services. Our goal is to be able to automate such tasks, and to give this ability to the casual user. Our approach to the solution are Web Macros, client scripts authored by demonstration and emulating user actions on the Web. In the “towards” part, I will discuss what desired functionality we have implemented in the prototype, and where we are proceeding from there.

So, why is some interaction on the Web repetitive? Web information services are getting more sophisticated, but: users have highly specific needs, and it is not possible to foresee all of these, and many tasks span individual sites.

## Automatable Web Tasks: Examples

- On-line flight reservations
- NWA, Internet Travel Network, etc.
- multiple, “equivalent” airports (SFO/SJC/OAK)
- Cited reference search
- re-run for each database, merge results
- plug found citations into full-text DL
- Persistent shopping cart
- multi-vendor?

**Notes:**
Here are examples of common tasks involving both consumer and research information. As experienced travelers and Web users, we look up flight and fare information and reserve on-line. When booking a flight to the Bay area, I used several reservation services and got the price difference of $300. NorthWest’s on-line reservations pages shows special deals, while Internet Travel Network pulls in flights from many airlines. So I had to submit my flight info to each of the reservation services, and compare results they returned. I am paying for the trip, so I wanted to be very thorough and requested flights to each of the 3 airports in the Bay area, spending considerable time typing, retyping, pasting, waiting for results, and comparing. Wouldn’t it be nice NOT to have to do all this?

The example of bibliography search is similar - citation databases may overlap, results must be merged. What makes this scenario interesting is that you might want to retrieve full text of found citations from the ACM Digital Library. So you are essentially connecting different, unrelated services sequentially.

In the last example, a persistent shopping cart, users have to reenter potentially much more information. Even if cookies can store the shopping cart contents for a single user (only Dell currently supports this), you cannot “pass” a shopping cart to another user for review and approval.

## Actions to automate

- Navigation; form filling and submission; user authentication
- Iteration over multiple information services
- integrate results
- Connecting services “back-to-back”

**Notes:**
As we analyze these examples, we see 2 things. First, existing tools for managing the complexity of the Web - bookmarks and histories - do not help with these tasks, since they were designed for a static Web of static documents. Server-side solutions also have some obvious limitations.

Secondly,  there are some actions in these examples navigation, interaction with forms, iteration over multiple services, connecting services “back-to-back”. We want to let users program these actions without explicit programming. When we think of automatically invoking and gluing together services, we often think of scripting languages. Macro languages, when end users are the authors. This similarity led us to the idea of Web macros.

## Vision of a Solution:  Web Macros

- Client-side programs that:
- Emulate user actions on the Web
*Can be created by end users*
- Requirements for Web Macros
- infer navigation and control
- support parameters
- deal with dynamic, unstructured data
- maintain the browsing context

**Notes:**
A very high-level definition of Web Macros would be programs that emulate user actions on the Web and are created by users without explicit programming. Based on these two high-level goals, we formulated the challenges for a system for providing Web Macros.

Inferring Control
navigation, iteration and decision
Parameterization
distinguishing parameters and constant values
factoring out and substituting personal information
Dealing with dynamic, unstructured data
merging results of iteration
detecting document identity for the purpose of the user

## Related Work

- Internet Scrapbook
- Sugiura and Koseki, UIST 98
- InfoBeans
- Bauer and Dengler, IUI 99
- AgentSoft’s LiveAgent
- WebL
- Kistler and Marais, WWW7

**Notes:**
	Several research and commercial projects have started to address these challenges. Internet Scrapbook, developed by researches at NEC labs, is a system for creating personalized Web pages from Web information services updated on a regular basis. It uses sophisticated heuristics to achieve this robustness against changes in content and markup.
	The InfoBeans project is now under development at GMD; its goal is to build encapsulated information sources from the Web by demonstration. Inputs and outputs of services  are specified  and connected through a graphical interface. These two systems apparently do not attempt to infer navigation and control, and parameters in InfoBeans are specified explicitly.
	LiveAgent’s AgentSoft is a proprietary system for automating Web navigation and interaction. Its goals are similar to ours; an evaluation beta puts emphasis on augmenting pages with user controls to specify parameters and navigation rules.
	WebL, which we are using in our prototype, is a scripting language for programming Web services and extracting information from HTML. We are leveraging on this powerful language to build our system for authoring Web macros by end users.
	There is still a wide range of issues for Web task automation that these system do not address.

## Proxy-based Prototype

- Special URLs http://macros/...

## “Get me here” Web macros

- Goal: re-establish browsing context
- authentication, hidden form fields, cookies
- Iteratively generates and test macros
- from longer history segments

**Notes:**
Suited for “restricted use” search engines

## Get me to: Housing Search Results

**Notes:**
U of M Housing search is simple example of a get me here macro. I would authenticate myself, submit a couple of forms, and get results. To record a macro to retrieve these results, I click on the Record icon in the browser bar, which is a special URL. The system generates essentially a one-line WebL script to retrieve the current page, and evaluates it. The page that the script loads is the original user authentication screen. The reason is that the cookies were sent when a user has authenticated, and these are not being present for the script. The script generator detects the failure, and creates and evaluates a script to retrieve the last and the previous page. This, and the next attempt fail, until a script that sends user login and password is created.

## Invoking Web Macros

**Notes:**
To execute the generated macro (bottommost), I can either run it to completion and display only the search results, or step through it and instruct the system which pages should not be loaded or displayed when the macro is run. The WebL script is updated accordingly.

Simple parameterization rules
login/password vs. text field vs. checkbox

## Future Work

- Concrete plans
- intelligently comparing Web pages
- let users specify start and end of macro
- update macros based on playback “deviations”
- Neat ideas
- use browsing history for improving macros
- visual manipulation environment for Web macros

## Conclusion

- Why automate common tasks
- Requirements and our prototype system
- Goals
- A free system for automating Web tasks by end-users
- Extensions to Programming by Demonstration techniques for the Web
- Recommendations for content developers

## Existing Tools

- Bookmarks/favorites
- record static URLs only
- Server-based Tools
- comparison shopping services; auction proxies; customizable home pages
- limited flexibility: user is not in control
- competing services are excluded