EmailValet : Where do you want to read your Email ?

This paper describes EmailValet, a system that learns users’ email-reading preferences on emailcapable wireless platforms – specifically, on two-way pagers with small ”qwerty” keyboards and an 8-line 30-character display. The paper more specifically presents results comparing the ability of different learning methods to form models that can predict whether a given message should be forwarded to the user’s wireless device, using the authors’ emailreading preferences for about nine months and more than 15,000 email messages. Our results show that we are able to achieve a break-even point of over 63% for the user with the greatest amount of email, over 11,000 messages. We also find that, in general, all methods are able to achieve better performance than what would be achieved by a baseline of simply forwarding all messages to the wireless device, and that many methods are able to find procedures that, although they forward only a small fraction of the messages that a user would want, achieve 100% precision on those messages that it does actually choose to forward. We also evaluate the extent to which training on email from varying windows of time into the past impacts learning, as well as the effect of user and email contextual features on learning. 1. Problem Description Imagine that while on your way to a meeting you receive an email message on your pager letting you know that the meeting has been moved to another building. Compare this with visiting another university and receiving an email message on your pager notifying you that a seminar is about to begin in a few minutes at your own university. One of these messages would likely be important to you at that particular point in time, while the other would not be. The goal of the EmailValet is to learn, based on various types of information, to forward messages of the first type to the pager, but not message of the second type. Furthermore, we would like such forwarding to take place in a transparent fashion for the people sending and receiving the email. For example, it would be desirable to have a single, centralized email address to which email would be sent, and some software agent residing at that location could then forward each message to where the receiver would like to read it, eliminating the need for both receiver or sender to figure out where to read a message from or send a message to. Ideally, this agent would work in a way much akin to a secretary, knowing where the user is and knowing which messages to forward to where. Thus, for example, if the user had a palmtop device with a wireless data service, a pager, a computer at home, and a desktop at work, the agent would need to decide where a message should be delivered, based on a number of factors, such as what the message is about, who is sending the message, current user context and what user devices are currently available. We call such an agent that provides “valet”-like functionality an Information Valet (iValet) (Macskassy, Dayanik, & Hirsh, 2000). The EmailValet is an instantiation of such an iValet, focused on providing valet-like functionality to users reading email (Macskassy, Dayanik, & Hirsh, 1999). In this paper we briefly describe the iValet framework, continuing with a discussion of the EmailValet and how it fits into this framework. The main part of the paper discusses the experimental design for our study and the main results found so far, in which we compare the performance of a range of well established-machine learning methods, including Naive Bayes and TFIDF. 2. Information Valets Our model of an Information Valet (iValet) is an agent that helps users intelligently access various forms of information via a range of devices (Macskassy et al., 2000). There are a wide ranges of design decisions that are necessary in building an iValet, such as the nature of the information, the form of user feedback, etc. We divide the range of design decisions into four classes, those concerning the user, the user’s access devices, the information being accessed, and the learning methods being used.