To explore or to exploit — Mental models for decision making
By: Tarun Kamra
Tags: Mental Models, Product Management, Productivity, Psychology, Growth Hacking

The Queen’s Gambit or replay Friends or watch Big Bang Theory again

Data Science certification or do a PMP course or Scrum Master

Order Pizza from the new cafe or the regular one

This dilemma is known as Explore/Exploit trade-off. Exploring is gathering of new information and Exploitation is using the information to get a predictable result (known positive)

Not just us, medical professionals torture themselves in choosing the right treatment for their patients. Even corporations like Google and Facebook are constantly working on this puzzle to improve user experience on their properties.

Let’s find out if there is a way to statistically decide on what’s best? How do we determine what to choose from the deluge of options presented to us? What are the big players doing to resolve the conflict?

Btw did you notice Netflix recently introduced a new button on home screen “Play Something” or “Shuffle Play”. Why?

Intuitively, we know life is a balance between novelty and tradition.

There is a saying that one who has not explored has not lived but exploration can be a curse too. Just imagine being a music journalist who is swamped with new releases everyday before thrashing out 1 or 2 tracks, that’s worth listening to / writing about. Sounds painful, isn’t it!

Exploring is venturing into unchartered territories (risky, frustrating, sometimes demotivating) but can be highly rewarding (think angel investors who invested in several companies before finding Uber) whereas exploiting offers assurance but less likely to create new moments or achieve unprecedented returns. In 2013, Hollywood released Fast and Furious 6, Die Hard 5, Scary Movie 5, Iron Man 3, Hang Over part 3 and several other sequels because these movies had an assured fan base!

The strategy should be strike a balance between the two. For example, pharmaceutical companies invest in R&D to discover (Explore) new drugs but also want to ensure their current products (Exploit) are thriving.

Technology enthusiasts would like to get into data science jobs but also secure their existing business analyst or quality analyst job.

Anyways, back to our problem — Explore or Exploit?

Let’s take an example — I am in a casino and there are 2 one armed slot machines (add image). Try and remember how we tried to maximise our winnings solely based on chance.

Now imagine, on machine 1 — you played 15 times and won 9 times compared to machine 2 — where you played 2 games and won 1.

Which is more lucrative? Obviously, we will all choose machine 1 because the winning rate is 60%. But 2 games do not give us any statistical significance.

Yes, it’s a trick question and the answer depends upon how long I intend to be in the casino.

Exactly, time is crucial in explore/exploit tradeoff.

I’m more likely to try a new restaurant when I am in a new city than when I am leaving it. How many of us would have chosen to pursue a part time MBA or a 6 month course in March 2020, if we knew this lockdown is a long haul?

As we got more clarity, we all started doing certifications, cooking (chef at every home making dalgona coffee), and the tsunami of fitness posts. Intuitively we used time as a factor.

There are several mental models to think about it and like everything else in life, everyone should find their own working model



Statistician Herbert Robbins proposed a solution Win-Stay, Lose-Switch algorithm. This basically means that as long as you are winning on a machine, keep pulling the arm and switch the machine as soon as you lose.

This is statistically proven to perform better than chance.

Well, that makes sense in a casino or betting. However, if I have had a wonderful experience in a restaurant for a 100 times and switching based on 1 poor experience is very harsh emotionally and only a few can exercise it.

Interestingly, Jeff Bezos dwelled on the same issue when the idea of online book store occurred to him. The mental model he used is called Regret Minimisation Framework. Idea is to look ahead in the future and think in hindsight about you decision. Let me ask you — Do you want to be a content writer or open a beach cafe and would you regret if you don’t make the move? If answer is YES — Do it! else, Don’t bother!

Another approach that is widely accepted and frequently peddled by most marketing companies — A/B Testing. This involves testing different variants of a subject (e.g. web page, e-mail, campaign, etc.) on the visiting users and identifying which one receives maximum conversion. These variants are nothing but different arrangements such as colours, placement, text, etc.

Since 2000, all major companies have been running A/B tests on their users. What are these companies exploring and exploiting? Simple answer — YOU.

Yes, we have all been part of someone else’s explore/exploit problem.

Dan Siroker made a fortune for Obama presidential campaign and later for himself by co-founding “Optimizely”

A/B testing is not limited to the software world but have been practiced by medical professionals since 1960s to analyse and decide potential life-saving treatments (read Zelen’s algorithm).

The Gittin’s Index is another interesting model that assigns an index value to every choice. However not many of us carry an index table when deciding what to order tonight.

Some things come naturally to us as we progress in life. As children, we are perennially exploring. Its a period to experience different possibilities without worrying about pay-offs. On a casino machine, they will pull a new arm every time.

And the other extreme is elders, who want to be in the same social circle and follow the same routine everyday because it’s a deliberate choice after a life long selection process.

That’s a wrap for today..