Computerized sales assistants: the application of computer technology to measure consumer interest - a conceptual framework

ABSTRACT This paper describes a computerized intelligent sales assistant that gives sales personnel the ability to allocate their time where it will produce the best results, both for the customer, and for the business. Upon entering the shop, a potential customer has his features scanned and analyzed by the computer and the customer is categorized as a browser, future customer, potential customer or buyer. The customers' facial data are also used to retrieve their details, if available, from the shop's database, and the data are used to determine whether a human sales assistant is required. The intelligent assistant's expression recognition feature would also tell the sales personnel whether or not the customer requires or desires assistance in the first place. The paper also proposes a scenario whereby the system could be used to give online sales systems the ability to automatically tailor the services they offer to the customer based on their facial reactions. While browsing the contents of an e-shop, a customer has his facial expressions scanned and analyzed by the computer and, based on the results, the system can suggest further products that may be of interest to the customer. This, for the customer, can mean being directed to products they have been recognized to be more interested in, resulting in savings in time. The framework described in this paper could also be used for applications such as new product screening, marketing and advertising. This paper describes the theoretical and conceptual framework for such an intelligent sales assistant and discusses the technology used in its implementation. Keywords: feature recognition, facial expression, shopping behavior, consumer analysis and prediction. 1. Introduction Human communication is a combination of both verbal and nonverbal interactions. Through facial expressions, body gestures and other non-verbal cues, a human can communicate with others. This is especially true in the communications of emotions. In fact, studies have shown that a staggering 93% of affective communication takes place either non-verbally or para-linguistically [Mehrabian, 1971] through facial expressions, gestures, or vocal inflections [Picard, 1998]. As discussed in this paper, when dealing with online shoppers, who are communicating with a computer to potentially make a purchase, detection of emotions can certainly play an important role. The same system has the capability to be used in physical stores as well. Using computers to analyze human faces has been an area of recent interest in computer science and psychology. Smart rooms with computer systems capable of tracking people and recognizing faces, and which can interpret speech, facial expressions and gestures made by the individuals in the smart room have been proposed before [Pentland, 1996]. According to Pentland, the philosophy behind this technology is that "? computers must be able to see and hear what we do before they can prove truly helpful. What is more, they must be able to recognize who we are and, much as another person or even a dog would, make sense of what we are thinking. " [p.68]. Robots that show emotions are being built [Bartlett, et al., 2003] while other similar attempts to include emotion in various computer applications [Sarrafzadeh et al., 2002] are being made. Facial recognition software have even be used to apprehend potential terrorists by using the software to scan a crowd of people going through an airport and looking for facial matches to known terrorists [Kopel and Krause, 2002]. Despite all this, today.s computers do not make use of the nonverbal cues common in human communications. Affective reactions of human subjects in e-commerce environments have been studied [Preira, 2000]. In social psychology researchers have been interested in relating facial expressions with the behavior of a person. So far in marketing there have not been any studies aimed at relating facial expressions with the shopping behavior of customers, and the great potential of such technologies in e-commerce applications have been overlooked. …