玉川大学グローバルＣＯＥプログラム 特別集中講義

Lectures on Conditioning and Associative Learning
Professor Bernard Balleine
Brain & Mind Research Institute, University of Sydney

2012.03.26-29 玉川大学大学研究室棟

Ｂ104 ＆ Ｂ107

3／26 (mon) 10:00-13:00 Lecture 1. Basic learning processes
3／27 (tue) 10:00-13:00 Lecture 2. Excitatory and inhibitory conditioning
3／28 (wed) 10:00-13:00

Lecture 3. Theories of conditioning

3／29 (thu) 10:00-13:00

Lecture 4. Instrumental conditioning

Professor Bernard Balleine’s research aims to understand theneural bases of the learning and motivational processes thatcontrol volitional, or goal - directed, action, an issue that has
direct bearing on our developing understanding of the role that theintegration of cognitive and emotional systems plays in executivefunctions and decision - making. He is particularly interested in thedivision between the essential systems and circuits that mediategoal - directed as opposed to reflexive or habitual actions and therole of primary motivation in reward and reinforcment processes.
Professor Balleine received his BA with first class honours and theUniversity Medal from the University of Sydney in 1987 and hisPh.D in comparative psychology from the University of Cambridge in 1993. He was elected to a
Research Fellowship at Jesus College Cambridge in 1993 and, in 1996, appointed Assistant Professor
at the University of California, Los Angeles. He was made Associate Professor in 2000 and full
professor in 2004. In 2005 he was appointed Director of Research in the Brain Research Institute atUCLA. He won an inaugural Australian Laureate Fellowship in 2009, and is currently Laureate
Professor and Director of the Behavioural Neuroscience Laboratory in the Brain & Mind
Research Institute, Sydney University.

Lecture 1. Basic learning processes
Outline.
‐ Introduction to the series‐ Overview of learning processes and some terminology‐ Introduction to lecture 1.
‐ Assumptions behind associative learning‐ The nature of events‐ Theoretical issues relating to event representation‐ Evaluative vs. predictive learning
In this lecture I will first provide an overview of basic learning processes, particularly of
two forms of conditioning; classical, or Pavlovian, conditioning and instrumental(sometimes called operant) conditioning. I will then introduce the associative analysis ofthese forms of learning and consider some of the fundamental assumptions such analyses
make, particularly those relating to the processing of events, the general process
assumption and assumptions relating to the adaptive significance of conditioning and therepresentation of causal relations. Consideration of these issues brings to the fore twoother forms of learning that are not usually given as much central consideration, notably
the effects of simple exposure to environmental events generally, and the effect ofexposure to biologically significant events in particular. I will consider phenomena suchas habituation, sensitization and evaluative conditioning during the course of thisassessment and discuss issues surrounding the appropriate control conditions for studiesof learning based on these phenomena.
General References:
Dickinson, A. (1980). Contemporary Animal Learning Theory. Cambridge: Cambridge
University Press.
Macintosh, N.J. (1983). Conditioning and Associative Learning. Oxford: Oxford
University Press.
References for this lecture:
Balleine, B. W. (2004). Incentive Behavior. The Behavior of the Laboratory Rat: A
Handbook With Tests. I. Q. Whishaw and B. Kolb. Oxford, Oxford Univeristy Press: 436446.
Hollis, K.L. (1997) Contemporary research on Pavlovian conditioning: A new functional
perspective. American Psychologist, 52, 956-965.
Rankin, C.H. et al (2009). Habituation revisited: An updated and revised description of
the behavioral characteristics of habituation. Neurobiology of Learning and Memory, 2,125-126.
Rescorla, R.A. 9!988). Pavlovian conditioning: Its not what you think it is. American
Psychologist, 43, 151-160.

Lecture 2. Excitatory and inhibitory conditioning
Outline.
‐ Appetitive and aversive conditioning‐ What is learned in excitatory conditioning?
‐ On the relationship between the CR and UR‐ Konorski’s theory of conditioning‐ Conditioned inhibition and extinction‐ Contextual control of extinction‐ What is learned in conditioned inhibition?
This lecture will summarize fundamental observations in Pavlovian excitatory andinhibitory conditioning. Beginning with excitatory conditioning, examples of appetitive
and aversive conditioning will be presented. I will then assess evidence for what is learnedin Pavlovian conditioning, contrasting behavioral stimulus-response theory against acognitive, stimulus-stimulus, alternative. This evidence supports a description ofconditioning based on Pavlov’s stimulus-substitution theory. However, much needs to beadded to that theory to correctly characterize excitatory conditioning. Most notably, thePavlovian conditioned response often doesn’t resemble the unconditioned response.
Generally, however, it is sufficient to note that the Pavlovian US is not a simple event andmuch of the variability in the CR -UR identity can be ascribed to inadequately specifying
the UR conditioned to the CS in many situations. I will then go on to review Konorski’s
theory of Pavlovian excitatory conditioning and, subsequently, inhibitory conditioning
and describe current views of conditioned inhibition and the tests required to observe it.
This assessment will focus on the feature-negative situation and on extinction. With
regard to the latter, I will describe experiments demonstrating the context dependency of
inhibition induced by extinction and some implications of that view for observing its
inhibitory process. Finally, a general view of the structure of Pavlovian excitatory andinhibitory learning will be elaborated.
References:
Bouton, M.E. (2004). Context and behavioral processes in extinction. Learning &Memory, 11, 485-494.
Delamater, A. (2012). On the nature of CS and US representations in Pavlovian learning.
Learning & Behavior, 40, 1-23.
Konorski, J. (1967). Integrative Activity of the Brain. Chicago: Chicago University Press
Recorla, R.A. (1988). Behavioral studies of Pavlovian conditioning. Annual Review ofNeuroscience, 11, 329-352.
Pickens, C.L. & Holland, P.C. (2004). Conditioning and cognition. Neuroscience &Biobehavioral Reviews, 28, 651-661.

Lecture 3. Theories of conditioning
Outline.
- Contiguity and contingency- Predictive validity- Blocking and overshadowing- US processing: Rescorla-Wagner theory- Problems with US processing theories- CS processing: Mackintosh and Pearce-Hall theories- Problems with CS processing theories- Wagner’s SOP theory and variations- Recent hybrid theories
I will review theories of Pavlovian conditioning starting with the notion that learning isdriven by CS-US contiguity. Problems for contiguity theory prompted a variety ofalternatives beginning with contingency and relative validity views that, based onevidence of associative competition from phenomena such as blocking andovershadowing, were ultimately formalized into a US processing approach, most notably
the Rescorla-Wagner theory. This has been a highly successful theory that has generated anumber of important predictions associated with overexpectation and conditionedinhibition. However, problems have also been identified providing the impetus for a rangeof alternative theories, notably the CS-processing theories of Mackintosh and of Pearceand Hall but also Wagner’s Sometimes Opponent Process (SOP) theory. Althoughmoderately successful, these views have also had difficulties explaining specificphenomena prompting still further revisions and the development of hybrid theories.
There is currently no universally accepted theory of Pavlovian conditioning.
References:LePelley, M. (2004). The role of associative history in models of associative learning: aselective review and a hybrid model. The Quarterly Journal of Experimental Psychology,
57, 193-243.
Mackintosh, N.J. (1975). A theory of attention: Variations in the associability of stimuli
with reinforcement. Psychological Review, 82, 276-298.
Pearce, J.M. & Bouton, M.E. (2001). Theories of associative learning in animals. Annual
Review of Psychology, 52, 111-139.
Pearce, J. M. & Hall, G. (1980). "A model for Pavlovian learning: variations in theeffectiveness of conditioned but not of unconditioned stimuli." Psychological Review, 87,532-552.
Rescorla, R. A. & Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variationsin the effectiveness of reinforcement and non-reinforcement. In: Classical Conditioning II:
Current Research and Theory. A. H. Black and W. F. Prokasy. New York, AppletonCentury-Crofts: 64-99.
Wagner, A.R. (1981). SOP: A model of automatic memory processing in animal
behavior. In N.E. Spear & R.R. Miller (Eds.), Information processing inanimals: Memory mechanisms (pp. 5-47). Hillsdale, NJ: Erlbaum.

Lecture 4. Instrumental conditioning
Outline.
- Pavlovian vs. instrumental conditioning- What is learned in instrumental conditioning?
- Stimulus control vs. action control- Theories of instrumental learning- The action-outcome contingency- Loss of control: The development of behavioural autonomy- Goal-directed vs. habitual actions- Reward vs. reinforcement
Pavlovian conditioning allows animals to predict forthcoming events of biologicalsignificance. However, as stimulus-bound reflexes, Pavlovian conditioned responses arepoorly suited to adapting to changes in environmental conditions to meet the challenges
presented by predictive information. In order to do something new, animals have be able
to exert control over their actions and to do so requires that they are sensitive to therelationship between actions and their consequences, a capacity that stems from theirability to engage in instrumental conditioning. In this lecture I will begin by reviewing thedifferences between Pavlovian and instrumental conditioning and the nature ofinstrumental conditioning experiments. I will then discuss issues in understanding what islearned in instrumental conditioning. Generally, evidence from outcome devaluation
studies demonstrates that animals are able to encode the relationship between actions andtheir consequences and to use this information to develop a behavioural strategy. However,there are ways of explaining sensitivity to devaluation that do not require that animals use
such highly cognitive abilities and that explains their behavior in reflexive terms byproposing a two process model composed of: (1) a stimulus-response learning process and(2) a Pavlovian stimulus-stimulus motivational process. Another alternative has proposedthat Pavlovian expectations are the stimuli with which responses are associated. Recent
evidence suggests that, although these alternatives ultimately fail to explain goal-directed
instrumental actions, they could be used to explain the development of skills and, more
importantly, habitual actions that develop with overtraining. I will consider the distinction
between goal-directed actions and habits and go on to evaluate the learning andmotivational processes that contribute to these forms of action control.
References
Balleine, B. W. (2001). Incentive processes in instrumental conditioning. Handbook ofcontemporary learning theories. R. M. S. Klein. Hillsdale, NJ, LEA: 307-366.
Colwill, R. M. and R. A. Rescorla (1986). Associative structures in instrumental learning.
The psychology of learning and motivation. G. H. bower. Orlando FL, Academic Press.
20: 55-104.
Dickinson, A. (1985). "Actions and habits: the development of behavioural autonomy."
Philosophical Transactions of the Royal Society of London B308: 67-78.
Dickinson, A. (1994). Instrumental conditioning. Animal cognition and learning. N. J.
Mackintosh. London, Academic Press: 4-79.