The Illusion of Making Data-Driven Decisions
By: Rod Castor
Tags: Analytics, Leadership, Business, Data Science, Towards Data Science

The Scenario

Everyone talks about using data to make decisions. Many companies claim to be making data-driven decisions. But their results are mixed. So they engage consultants like me to advise them, hoping to improve their insights. Unfortunately, these business leaders mistakenly attribute their poor performance to failure in the data science department. Ironically, any poor decision resulting from their use of the data is typically their own.

These companies have copious amounts of data. They’ve created a small team of individuals capable of the necessary data science skills to curate the data into meaningful information. Data wranglers, data scientists, programmers, data engineers, graphic artists, UI/UX professionals, and subject matter experts are all represented in the effort. So what’s the problem?

The Problem

Data acts like a muse inspiring the business leaders rather than driving their decisions. Their method of using data assumes they can make unbiased decisions after being influenced by the massive amounts of information shared in their meetings. Genuine data-driven decisions come through an entirely different approach. To ensure data drives your business, you must establish the decision criteria before introducing data insights.

Decision science is a discipline unto itself. If you’re serious about using decision science techniques to their fullest, you’ll need to invest much time and money. Since companies with small and large budgets need to make better decisions, let’s outline just a few fundamental decision science practices that all companies can afford and implement.

5 Steps to Achieve Genuine Data-Driven Decisions

#1 — Define the Business Question

Begin by establishing the question or questions you are trying to answer. If your company is shooting from the hip every time your leaders gather, its destination is undefined. Your data will either lead you off a cliff or resemble the path of an intoxicated sloth. Neither result is desirable.

Avoiding this problem is intellectually simplistic, even though taking the proper actions to fix it can be challenging. Before assessing any data, begin by identifying the business question.

Business Question: Should we expand our sales efforts into the western region?

#2 — Define Your Decision Criteria Before You See All the Data

Next, define the criteria for making this decision. There’s a great scene from The Insider movie. Russel Crowe plays a whistleblowing insider from a tobacco company. Just as he is heading to the deposition, the proverbial point-of-no-return, he pauses. His attorney questions whether he’s ready or not. Crowe’s character replies, “I can’t find the criteria to decide.”

The same problem faced the boardroom members in our example. A lack of predetermined standards for evaluation lays the foundation for cognitive bias. The insights from the dashboards and reports subconsciously influence their use. Subsequently, the decision is no longer objective. It’s not data-driven.

If you are in sales, you probably already know the requirements for deciding this.

Some example criteria might be:

Will sales in the new region impact sales in existing regions in any significant way?

Does the market for our product in the new region meet the minimum potential sales required for the new market to be independently viable within three years?

Do we have or can we obtain the necessary sales resources for the new market?

Do we have or can we obtain the necessary marketing resource to support the new market?

Do we have or can we obtain the necessary supply chain resources to support the new market?

Do we have or can we obtain the necessary supporting business services to support the new market?

Do we have realistic targets to determine success with this new market in one, two, and three years?

Do we have a realistic exit strategy for this market?

How much capital will be required to begin expanding?

Do we have sufficient capital to fund the expansion until the new marketing is self-sustaining?

The easy part about this step is you don’t need a data scientist to do it. Now that you have the questions, see if you can answer them. You might need a data analyst to assist you, or you might have all of these answers already available to you. If you already have the answers, jump to step four. Otherwise, you’ll need to stop at step three along the way.

#3 — Divide Your Data in Two

If you use your entire dataset for exploratory purposes, you’ll create the same cognitive bias we saw in the board room. Instead, divide your data into two sets using roughly a 20/80 split. The smaller of the two datasets is for exploratory data analysis. Any department in your organization is welcome to use it. Share it with everyone asking you for data. Reserve the larger dataset for reporting. This last dataset drives your business decisions.

After you’ve split your dataset, you may ask a data analyst to help you establish the decision metrics corresponding to your pre-defined criteria.

#4 — Establish Your Metrics by Answering Your Criteria Questions

The answers you supply to these questions will become your threshold for making your final decision. It’s essential to answer these before you’re sitting in the conference room looking at the latest reports. Whatever metrics you establish will protect you from using data as a muse and propel it into the driver seat. Take your time on this step, and be sure to use reasonable and valid metrics.

Although we dream of perfect data, it doesn’t exist. You will find spots within your data that simply cannot offer an exact measurement. Don’t panic. Don’t throw in the towel.

When you encounter insufficient data, use your human intuition and your buddies in data science to help you establish an acceptable threshold. You now have your decision metric. Imperfect data is nothing to fear; it is still valuable and can even be part of the decision making process.

#5 — Make Data-Driven Decisions

If you’ve used the first four steps, the fifth step is easy. Let your data science team create dashboards from your reporting data. Measure the outcome against your decision criteria and metrics. Act accordingly. When metrics meet a specific action’s threshold, take action. If not, wait. Do not act. Isn’t making decisions simple?

Conclusion

As your company grows, your decision-making process should become more involved by fully leveraging decision science practices. But these five steps are a strong start. They will dispel the illusion you’ve been practicing and help you truly make decisions driven by data.