Experimentation is hard — really fucking hard
By: Arjan Haring
Tags: Behavioral Economics, Behavior Change, Experimenting

Experimentation is hard — really fucking hard

And selling it is a pain. A short love story.

The smell of bread makes people buy more stuff in supermarkets. Really it does. Using cues of social proof makes people click more. Uh-huh. Even better: A/B testing will guarantee that your company makes more money. And my favorite: with behavioral science you can make people do whatever you want.

Let me be clear, I was part of the behavioral science cheerleading squad for quite some time. I was shouting “I 💘 Dan Ariely” most of the time. That was until I overstretched and hurt something that is very dear to me… my integrity.

It felt like we were selling a silver bullet. And after the movement became very successful and powerful, people kept selling the silver bullet. When it became clear to me there would be Nudge units started around the world, with less experienced people making a career selling more silver bullets, I reached my limit.

It was around that time that I left the behavioral science cheerleading squad and went to the less known business experimentation team. (Go team!) Below some thoughts on the difference of approach of both disciplines and how that might influence their success.

Fortune favors the bold

We all know how some politicians use an oversimplified state of the world to their advantage. Trump being a case in point. Nuanced ideas don’t go viral, simple ones do. Complex explanations don’t get crowds fired up, the straight forward ones start a frenzy.

As was mentioned by Christie Aschwanden in her great FiveThirtyEight post the subtle truth about science hardly ever ends up in the head lines. “Science is hard — really fucking hard.” It’s the same for business experimentation.

I left off in my last article that the headline cure for marketing: “social proof” didn’t have any impact in one of my experiments for an airline. Of course it didn’t, why would it? I really liked the designs I made together with our UX design ninja, the implementation of social proof was perfect. But the reality is more complex.

It always is.

Do you like what you have read so far? Get a quarterly update of what I am busy with.

When you are trying to optimize consumer behavior, instead of applying a persuasion principle to a given situation, it’s better to use the fundament for a behavioral experiment also discussed in my last post.

The more nuanced view on running a behavioral experiment

When the two disciplines mentioned earlier start being serious about marketing we will face the same challenge; how to make a consumer do something? Then and there our fields merge and there shouldn’t be a difference of approach.

We should judge determinants on importance (how much unique variance does this determinant cover) and changebility (can we change this determinant?).

When you found a good determinant you will have to find out which behavior change strategy (social proof, et cetera) has the most effect on that determinant.

“The fundament for your experiment = determinant x behavior change strategy.” Dirk Franssens, Web Art AG

Fat chance that this quote will make the headlines. And it for sure as hell will not be the starting point for organisations interested in experimentation. It’s a fact that we have less of a silver bullet edge to us when you compare us to the behavioral science cheerleading squad (do this, fix that).

And let me be clear, I don’t dislike the behavioral science cheerleading squad. I actually still 💘 Dan Ariely. He is a really cool guy. And he has done some great things to get his discipline recognized. He has even wrote great stuff about some organizational challenges for experimentation.

But Dan could of course do better. I wish for those who have great influence on setting the agenda of businesses right now and know that it’s all much more complex, to not forget the importance of nuance now and then.

As we all know; by pushing the agenda of experimentation we all learn more.

P.s. In this article I wanted to 1. create an artificial divide between applied behavioral scientists and business experimenters to promote a feeling of community through out-group bias. And 2. dig a bit more into how we can promote our field in an ethical way.

Did I succeed?

Do you like what you have read? Get a quarterly update of what I am busy with.

Some articles I used for this one: