Understanding what machine learning produces - Part II: Knowledge visualization techniques

Researchers in machine learning use decision trees, production rules, and decision graphs for visualizing classification data. Part I of this paper surveyed these representations, paying particular attention to their comprehensibility for non-specialist users. Part II turns attention to knowledge visualization—the graphic form in which a structure is portrayed and its strong influence on comprehensibility. We analyze the questions that, in our experience, end users of machine learning tend to ask of the structures inferred from their empirical data. By mapping these questions onto visualization tasks, we have created new graphical representations that show the flow of examples through a decision structure. These knowledge visualization techniques are particularly appropriate in helping to answer the questions that users typically ask, and we describe their use in discovering new properties of a data set. In the case of decision trees, an automated software tool has been developed to construct the visualizations. Decision trees, production rules and decision graphs are widely used for representing the results of machine learning. As the first part of this paper showed, many schemes have been developed to assist comprehension by reducing the amount of gratuitous information in such representations. What we do now is consider how methods of visualization can further help people to assimilate the information that is generated by applying machine learning methods. We begin by reviewing two ways in which visualization has been applied to machine learning. The first is the use of decision tables, a kind of visualization for rules that presents a group of rules side by side in a way that allows them to be compared. The second is a tree visualizer with enticing visual effects: users “fly over” a 3D representation of the tree, circle in on a particular node and soar off along a branch. Are these schemes useful? The purpose of a visualization is to provide answers in situations where the user does not know, or cannot decide, the questions they want to ask (Bertin, 1983). Visualizations are useful when they support the user’s tasks, when they help the user answer questions about the data, and when they prompt him or her to pose pertinent queries. To design appropriate visualizations, it is necessary to determine the sorts of questions to which researchers seek answers. Knowledge workers use decision trees, rules, and graphs to grasp the overall structure of a classification scheme. The standard representations of these constructs show how particular decision criteria—selections of data attributes with specific values—lead to one of a predefined set of classes—subsets of the example space. But in our experience, users of machine learning also ask a number of critical questions about the significance of decision criteria, the flow of training examples, and the extent to which classes are related, which cannot be answered from the standard visual forms of classification structures. These questions can be mapped into equivalent visualization tasks, or actions that a user can perform on a visualization in order to answer the question. Examples are looking up a value, or comparing two values. Given a diagram of an information structure, a visualization task is possible when the information that it calls for is visible in the diagram. The task is efficient