# The Power of Reinforcement

Part of the sequence: [The Science of Winning at Life](http://wiki.lesswrong.com/wiki/The_Science_of_Winning_at_Life)

Also see: [Basics of Animal Reinforcement](/lw/6iu/basics_of_animal_reinforcement/), [Basics of Human Reinforcement](/lw/6ja/basics_of_human_reinforcement/), [Physical and Mental Behavior](/lw/6l6/physical_and_mental_behavior/), [Wanting vs. Liking Revisited](/lw/6kx/wanting_vs_liking_revisited/), [Approving reinforces low-effort behaviors](/lw/6nz/approving_reinforces_loweffort_behaviors/), [Applying Behavioral Psychology on Myself](/lw/2dg/applying_behavioral_psychology_on_myself/).

**Story 1**:

On Skype with Eliezer, I said: "Eliezer, you've been unusually _pleasant_ these past three weeks. I'm really happy to see that, and moreover, it increases my probability than an Eliezer-led FAI research team will _work_. What caused this change, do you think?"

Eliezer replied: "Well, three weeks ago I was working with Anna and Alicorn, and every time I said something nice they fed me an M&M."

**Story 2**:

I once witnessed a worker who _hated_ keeping a work log because it was only used "against" him. His supervisor would call to say "Why did you spend so much time on _that_?" or "Why isn't _this_ done yet?" but never "I saw you handled _X_, great job!" Not surprisingly, he often "forgot" to fill out his worklog.

Ever since I got everyone at the [Singularity Institute](http://intelligence.org/) to keep work logs, I've tried to avoid connections between "concerned" feedback and staff work logs, and instead take time to comment positively on things I see in those work logs.

**Story 3**:

Chatting with Eliezer, I said, "Eliezer, I get the sense that I've inadvertently caused you to be slightly averse to talking to me. Maybe because we disagree on so many things, or something?"

Eliezer's reply was: "No, it's much simpler. Our conversations usually run longer than our previously set deadline, so whenever I finish talking with you I feel drained and slightly cranky."

Now I finish our conversations on time.

**Story 4**:

A major Singularity Institute donor recently said to me: "By the way, I decided that every time I donate to the Singularity Institute, I'll set aside an additional 5% for myself to do fun things with, as a motivation to donate."

####   

#### The power of reinforcement

It's amazing to me how consistently we fail to take advantage of [the power of reinforcement](http://www.youtube.com/watch?v=HqbVbPvlDoM).

Maybe it's because behaviorist techniques like reinforcement feel like they don't respect human agency enough. But if you aren't treating humans more like animals than _most_ people are, then you're _modeling humans poorly._

You are not an [agenty](/lw/5i8/the_power_of_agency/) [homunculus](http://commonsenseatheism.com/?p=8844) "corrupted" by heuristics and biases. You just _are_ heuristics and biases. And [you respond to reinforcement](http://www.amazon.com/Power-Reinforcement-Alternatives-Psychology-Series/dp/0791459160/), because most of [your motivation systems](/lw/9jh/the_humans_hidden_utility_function_maybe/) still work like the motivation systems of other animals.

#### A quick reminder of what you learned in high school

*   A _reinforcer_ is anything that, when it occurs in conjunction with an act, increases the probability that the act will occur again.
*   A _positive reinforcer_ is something the subject wants, such as food, petting, or praise. _Positive reinforcement_ occurs when a target behavior is followed by something the subject wants, and this increases the probability that the behavior will occur again.
*   A _negative reinforcer_ is something the subject wants to avoid, such as a blow, a frown, or an unpleasant sound. _Negative reinforcement_ occurs when a target behavior is followed by some _relief_ from something the subject _doesn't want_, and this increases the probability that the behavior will happen again.

#### What works

1.  **Small reinforcers are fine**, as long as there is a strong correlation between the behavior and the reinforcer ([Schneider 1973](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1334171/pdf/jeabehav00121-0154.pdf); [Todorov et al. 1984](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1348029/pdf/jeabehav00059-0043.pdf)). All else equal, a large reinforcer is more effective than a small one ([Christopher 1988](http://books.google.com/books?id=SWy3uoXJw_4C&lpg=PA400&ots=4mqS11WYXZ&dq=%22Predisposition%20versus%20experiential%20models%20of%20compulsive%20gambling%22&pg=PA400#v=onepage&q=%22Predisposition%20versus%20experiential%20models%20of%20compulsive%20gambling%22&f=false); [Ludvig et al. 2007](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1832167/pdf/jeab-87-02-201.pdf); [Wolfe 1936](http://www.amazon.co.uk/Effectiveness-token-rewards-chimpanzees-Comparative-psychology/dp/B00085DEL4)), but the more you increase the reinforcer magnitude, the less benefit you get from the increase ([Frisch & Dickinson 1990](http://www.tandfonline.com/doi/abs/10.1300/J075v11n01_03)).
2.  **The reinforcer should _immediately_ follow the target behavior** ([Escobar & Bruner 2007](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1918083/pdf/jeab-88-01-29.pdf); [Schlinger & Blakely 1994](http://www.calstatela.edu/academic/psych/html/Graduate/ABA/PDF%20articles/Effects%20of%20Delayed%20Rfmt.pdf); [Schneider 1990](http://scholar.google.com/scholar?cluster=11020848922119008463&hl=en&as_sdt=1,5)). [Pryor (2007)](http://www.amazon.com/Dont-Shoot-Dog-Teaching-Training/dp/1860542387/) notes that when the reward is food, small bits (like M&Ms) are best because they can be consumed _instantly_ instead of being consumed over an extended period of time.
3.  **Any _feature_ of a behavior can be strengthened** (e.g., its intensity, frequency, rate, duration, persistence, its shape or form), so long as a reinforcer can be made contingent on _that particular feature_ ([Neuringer 2002](http://psyc.csustan.edu/bhesse/psy4725/Articles/Variability/Neuringer2002evidencefunctionstheory.pdf)).

#### Example applications

*   If you want someone to call you, then when they _do_ call, don't nag them about how they never call you. Instead, be engaging and positive.
*   When trying to maintain order in a class, ignore unruly behavior and praise good behavior ([Madsen et al. 1968](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1310990/pdf/jaba00084-0043.pdf); [McNamara 1987](http://books.google.com/books?id=6w58X9a5U3gC&lpg=PA50&ots=jnqGMYFbTz&dq=Behavioural%20approaches%20in%20the%20secondary%20school&lr&pg=PA50#v=onepage&q&f=false)).
*   Reward originality to encourage creativity ([Pryor et al. 1969](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1338662/pdf/jeabehav00148-0145.pdf); [Chambers et al. 1977](http://scholar.google.com/scholar?cluster=17390870671691473343&hl=en&as_sdt=1,5); [Eisenberger & Armeli 1997](http://eisenberger.psych.udel.edu/files/14_Can_Salient_Reward_Increase_Creative_Performance.pdf); [Eisenberger & Rhoades 2001](http://www.psychology.uh.edu/faculty/Eisenberger/files/03_Incremental_effects_of_reward.pdf)).
*   If you want students to _understand_ the material, don't get excited when they [guess the teacher's password](/lw/iq/guessing_the_teachers_password/) but instead when they demonstrate a [technical understanding](http://yudkowsky.net/rational/technical).
*   To help someone improve at dance or sport, ignore poor performance but reward good performance immediately, for example by shouting "Good!" ([Buzas & Allyon 1981](http://bmo.sagepub.com/content/5/3/372.short)) The reason you should ignore poor performance if you say "No, you're doing it wrong!" you are inadvertently punishing the _effort_. A better response to a mistake would be to reinforce the effort: "Good effort! You're almost there! Try once more." 
*   Reward honesty to help people be more honest with you ([Lanza et al 1982](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1347815/pdf/jeabehav00068-0087.pdf)).
*   Reward opinion-expressing to get people to express their opinions more often ([Verplanck 1955](http://cogprints.org/602/1/biblio18.html)).
*   You may even be able to reinforce-away annoying _involuntary_ behaviors, such as twitches ([Laurenti-Lions et al. 1985](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1348176/pdf/jeabehav00050-0049.pdf)) or vomiting ([Wolf et al. 1965](http://scholar.google.com/scholar?cluster=6842890837888786430&hl=en&as_sdt=1,5)).
*   Want a young infant to learn to speak more quickly? Reinforce their attempts at vocalization ([Ramely & Finkelstein 1978](http://jpepsy.oxfordjournals.org/content/3/2/89.short)).
*   More training should occur via video games like [DragonBox](http://www.wired.com/geekdad/2012/06/dragonbox/all/), because computer programs can easily provide _instant_ reinforcement _many times a minute_ for _very specific behaviors_ ([Fletcher-Flinn & Gravatt 1995](http://scholar.google.com/scholar?cluster=9612997544810074285&hl=en&as_sdt=1,5)).

For additional examples and studies, see [_The Power of Reinforcement_ (2004)](http://www.amazon.com/Power-Reinforcement-Alternatives-Psychology-Series/dp/0791459160/), [_Don't Shoot the Dog_ (2006)](http://www.amazon.com/Dont-Shoot-Dog-Teaching-Training/dp/1860542387/), and [_Learning and Behavior_ (2008)](http://www.amazon.com/Learning-Behavior-Active-Edition/dp/0495095648/).

I close with **Story 5**, from [Amy Sutherland](http://www.nytimes.com/2006/06/25/fashion/25love.html?pagewanted=all):

> For a book I was writing about a school for exotic animal trainers, I started commuting from Maine to California, where I spent my days watching students do the seemingly impossible: teaching hyenas to pirouette on command, cougars to offer their paws for a nail clipping, and baboons to skateboard.
> 
> I listened, rapt, as professional trainers explained how they taught dolphins to flip and elephants to paint. Eventually it hit me that the same techniques might work on that stubborn but lovable species, the American husband.
> 
> The central lesson I learned from exotic animal trainers is that I should reward behavior I like and ignore behavior I don't. After all, you don't get a sea lion to balance a ball on the end of its nose by nagging. The same goes for the American husband.
> 
> Back in Maine, I began thanking Scott if he threw one dirty shirt into the hamper. If he threw in two, I'd kiss him. Meanwhile, I would step over any soiled clothes on the floor without one sharp word, though I did sometimes kick them under the bed. But as he basked in my appreciation, the piles became smaller.
> 
> I was using what trainers call "approximations," rewarding the small steps toward learning a whole new behavior...
> 
> Once I started thinking this way, I couldn't stop. At the school in California, I'd be scribbling notes on how to walk an emu or have a wolf accept you as a pack member, but I'd be thinking, "I can't wait to try this on Scott."
> 
> ...After two years of exotic animal training, my marriage is far smoother, my husband much easier to love.

Next post: [Rational Romantic Relationships Part 1](/lw/63i/rational_romantic_relationships_part_1/)

Previous post: [The Good News of Situationist Psychology](/lw/52g/the_good_news_of_situationist_psychology/)

My thanks to Erica Edelman for doing much of the research for this post.