The Impact of Complete and Selective Feedback in Static and Dynamic Multiple-Cue Judgment Tasks

The Impact of Complete and Selective Feedback in Static and Dynamic Multiple- Cue Judgment Tasks Oren Griffiths (ogriffiths@psy.unsw.edu.au) Ben R. Newell (ben.newell@unsw.edu.au) School of Psychology, University of New South Wales, Sydney, Australia Abstract It is widely accepted that feedback is critical to guide the learning process and to make effective decisions. However, ideal feedback is relatively rare in our daily environments. Two multiple-cue judgment experiments examined whether biased and incomplete feedback leads to less accurate learning than comprehensive feedback. Experiment 1 found that participants given selective feedback produce equivalently accurate predictions, learned equally rapidly, and exhibited equivalent task structure knowledge to those given full feedback. Experiment 2 showed that selective feedback led to more accurate outcome predictions and task structure knowledge in a dynamic task environment. These results are problematic for error-based models of learning. Keywords: Learning, decision environment, prediction error making, dynamic Introduction The importance of feedback in learning can scarcely be overstated. Feedback can be seen in many aspects of our lives: at work (audits, trial and error approach, quality control), while shopping (customer service evaluations), at home (reward/punishment of children) and, of course, in educational settings (exam marks, final grades). Consequently, most formal learning theories suggest that appropriate feedback is necessary for learning (excluding models of unsupervised learning). For example, error-based models of associative learning (e.g. Rescorla & Wagner, 1972; Mackintosh, 1975), error-based models of categorization (e.g. Nosofsky, 1986; Kruschke, 1992), connectionist learning models based upon the delta-rule (Rumelhart & McClelland, 1986) and Bayesian models of learning/reasoning (for a review, see Oaksford & Chater, 2008) all suggest that learning only occurs when feedback is received. (We will hereafter cumulatively refer to these models as error-based models). By extension, these error- based models suggest that optimal learning is achieved when accurate, unbiased feedback is delivered as often as possible. Selective Feedback Unfortunately, optimal feedback conditions like these are rare outside of the laboratory (e.g. Hogarth, 2005). To illustrate, suppose your friend Susan wants to learn how to make good investments in the stock market. To do this, she picks 10 companies that she thinks will be profitable, invests in them and monitors the share prices of these companies over the following 6 months. However, she probably will not monitor the share prices of 10 companies that she did not invest in. Thus, the feedback that Susan receives will be both incomplete and biased. It is incomplete because she received feedback on only a portion of the relevant learning opportunities, and it is biased because the companies about which she received feedback were systematically related; they were the companies that she predicted to be profitable. What impact would this biased, selective feedback have upon Susan’s learning? According to the aforementioned learning models it ought to be detrimental to her learning, relative to a scenario in which she was provided with balanced, comprehensive feedback. Elwin, Juslin, Olsson & Enkvist (2007; see also Henriksson, Elwin & Juslin, 2008) examined this question using an experiment based on the prior example. On each trial participants were shown how well a particular company scored on four performance indices (e.g. market share, turnover; hereafter referred to as cues), which were each scored between 0 and 10. Participants were asked to predict how many percentage points profit (or loss) that company would make in the following year. Importantly the feedback provided differed between participants. After generating their predictions, the full feedback group was always told how much profit each company made. In contrast, the selective feedback group was only told about the profitability of companies that they predicted would be profitable. At test, the two groups did not differ in their abilities to predict the profit made by each company (if anything, a non-significant advantage was seen for those given selective feedback). That is, the extra feedback given to the full feedback group did not improve performance over those given selective feedback. This lack of a difference does not appear to be due to a ceiling or floor effect, as the mean prediction accuracy at test ranged from 0.75 – 0.9. At first glance, this result does not seem to be supportive of the standard, error-based view of learning. However, there are alternative explanations of this result which are consistent with error-based learning. The selective group in the Elwin et al. (2007) study were given twice as many training trials as those in the full feedback group. This was done in order to equate the number of trials in which feedback was provided between the groups. However, it meant that from an error-based learning view, both groups experienced an equivalent number of learning opportunities (trials in which feedback was provided) making it less surprising that the groups performed equivalently at test.