Automatic Design of Efficient Visual Problem Representations

Automated reasoning about the design of effective visual problem representations i possible when we adopt the view that visual problem representations, along with the pemeptual procedures that humans use to manipulate them, can be described using information-processing models of the sort introduced by Newell and Simon (1972). This approach provides us not only with a means of characterizing visual problem representations in a formal syntax but also with a means of automatically mapping between "logical" and "perceptual" problem representations and procedures. An automated system called BOZ is described that begins with a logical problem representation and solution procedure, and generates an informationally-equivalent visual problem representation and procedure that allows the human user to obtain the solution more efficiently. BOZ’s representation mapping technique proceeds by: (1) replacing demanding logical inferences in the solution procedure with efficient perceptual inferences; and (2) structuring information the visual representation such that search is minimized. The extent to which the visual representations and procedures produced by BOZ agree with what users actually see and do is discussed. Logical and Visual Problem Representations Logical and visual representations of an airline reservation problem are compared to illustrate the advantages offered by visual problem representations, and to show how both logical and visual problem representations can be characterized using the same sort of informationprocessing model. Logical Problem Representations A Iogicalproblem representation includes a set of logical facts along with a logical procedure that operates on the facts to solve a stated problem. Figure 1 shows a logical representation for an airline reservation problem posed as follows: Find a pair of connecting flights that travel from Pittsburgh to Mexico City. You are free to choose any intermediate city as long as the layover in that city is no more than four hours. Both flights that you choose must be available. The combined cost of the flights cannot exceed $500. The logical facts (top of Figure 1 ) describe relations between the elements of a collection of five domain sets: airline flights, departure and arrival times, costs, and availability. For example, the first fact in the list indicates that the origin of Flight 117 is PIT. The logical procedure describes a series of search and computation steps (called logical operators) to be performed using the set of logical facts. Logical operators occur in two forms. Search operators such as Step 1 consider the logical facts one at a time, beginning at the top of the list each time, until a fact is located that satisfies the relation contained in the operator. In Step 1 the search is for a fact that satisfies (Origin FLIGHT ’pit). This operator succeeds on the first fact in the list and the variable FLICHT is instantiated with the value, flight 117. Computation operators perform arithmetic or comparison operations on the values retrieved by search operators. Step 11 is a computation operator that subtracts departure from arrival and instantiates LAYOVER with the result. Computation operators performed with a logical representation are assumed to be mentally performed arithmetic or logical operations applied to a collection of data values stored in the user’s memory. Visual Problem Representations A visual problem representation also consists of a collection of facts accompanied by a procedure for finding a solution to the problem. Facts in a visual problem representation, as illustrated in Figure 2, have two unique features. First, facts in a visual problem representation are encoded graphically by associating the elements of each domain set of information with the discriminable values of some graphical dimension, a technique first proposed by Mackinlay (1986). For example, each airline flight is represented by a rectangle. The horizontal position of the ends of the rectangle encode the departure and arrival times of that flight. The shading of each rectangle encodes the availability of that flight. Second, facts in a visual problem representation group together related information making all information required to draw a particular inference available at the same spatial location. For example, all information about each flight, origin, destination, cost, times, and availability, is grouped together in a single rectangle. Perceptual procedures differ from logical procedures in that they are composed of a set of perceptual operators that describe search and computation steps performed specifically within the context of a visual representation. The perceptual operators that appear in the perceptual procedure allow the user to obtain the same results as do the logical operators that appear in the logical procedure since they compute the same functions only in a different way. The bottom of Figure 2 shows a perceptual procedure for the airline reservation problem. The visual problem representation is arguably more efficient than the logical problem representation. The advantages of the visual problem representation appear to center around three notions demonstrated by Larkin and Simon (1987) and Casner (1990): (1) perceptual operators 157 From: AAAI Technical Report SS-92-02. Compilation copyright © 1992, AAAI (www.aaai.org). All rights reserved.