Title: Introduction to Causality in Machine Learning - PyImageSearch

URL Source: https://pyimagesearch.com/2023/05/08/introduction-to-causality-in-machine-learning/

Published Time: 2023-05-08T13:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [Introduction to Causality in Machine Learning](#h2BPTitle)
*   [Correlation and Causation](#h2Correlation)
*   [Case Study 1: A “Marvelous” Problem](#h2CS1)

*   [Scenario 1: A Direct Cause](#h3Scenario1)
*   [Scenario 2: Reversing the Cause and Effect](#h3Scenario2)
*   [Scenario 3: Investigating a Hidden Cause](#h3Scenario3)
*   [Causal Thinking](#h3Thinking)

*   [Case Study 2: Food App Conundrum](#h2CS2)
*   [Quiz Time!](#h2Quiz)
*   [Summary](#h2Summary)

*   [References](#h3References)
*   [Citation Information](#h3Citation)

* * *

In this tutorial, you will learn about causality and how to correctly interpret and use it for your machine learning projects.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/causality-1-featured.png)

This lesson is the 1st in a 4-part series on Causality in Machine Learning:

1.  [**_Introduction to Causality in Machine Learning_**](https://pyimg.co/3sa8x) **(this tutorial)**
2.  _Best Machine Learning Datasets_
3.  _Tools and Methodologies for Studying Causal Effects_
4.  _A Brief Introduction to Do-Calculus_
5.  _Studying Causal Effect with Microsoft’s Do-Why Library_

It is a known [phenomenon](https://www.sciencedirect.com/science/article/pii/S2590291120300711) that Nobel laureates are avid chocolate lovers. This means, as we look at Nobel laureates’ food preferences, many of them love eating chocolates. But what if we reverse the relation?

Does eating chocolates help us get a Nobel prize? As avid chocolate lovers, we can confirm this is not the case. Why? What is the reason for such injustice, and how can we exploit that in machine learning? Let’s find out.

**To learn how to understand and correctly interpret causality, _just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

* * *

[**Introduction to Causality in Machine Learning**](#TOC)
---------------------------------------------------------

So, what does causal inference mean? What does it mean to cause? Is causality as simple as understanding cause and effect? Let us first understand a few ground rules before playing the game of causality. There is a lot of noise surrounding causality and its neighbor correlation.

For example, the following quote from François Chollet (as shown in **Figure 1**) makes the topic even more confusing.

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20591%20230'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image.png)

**Figure 1:** Causation and Correlation (source: [tweet link](https://twitter.com/fchollet/status/1385706440011042818?lang=en)).

* * *

[**Correlation and Causation**](#TOC)
-------------------------------------

“Correlation does not mean Causation”: If you had a penny for every time you heard that line, you would probably be a millionaire by now. But what does it mean? Why does the correlation between two events do not imply causality? What is up?

Correlation is a relationship or connection between two variables where whenever one changes, the other is likely to also change. But a change in one variable doesn’t cause the other to change. That’s a correlation, but it’s not causation.

For example, if you see a lot of birds in the sky and it starts raining, it doesn’t mean that the birds caused the rain. They just happened at the same time.

Mathematically, correlation is measured by the correlation coefficient between \`-1\` and \`1\`. A correlation coefficient of \`1\` indicates a perfect positive correlation between two variables. In contrast, a correlation coefficient of \`-1\` means a perfect negative correlation between two variables. A correlation coefficient of \`0\` means no correlation between two variables.

Conversely, causation is a relationship between two variables where one variable causes the other variable to change.

At this point, you’re probably wondering: I have heard about causality a few hundred times, but do I need to care?

Let’s ask you a different question, then.

* * *

[**Case Study 1: A “Marvelous” Problem**](#TOC)
-----------------------------------------------

How many times have you looked at the result of your model and wondered what-if the data was something other than what it trained on? You could write an algorithm that predicts the sales of comic books, and your model works well and produces high-accuracy predictions, but you need to know **why**. Or maybe it’s the opposite, your algorithm predicts completely wrong sales figures, and you need to figure out why.

Confused? Let’s look at an example.

Suppose Marvel hires you as a data scientist. There has been a recent rise in comic book sales, and you need to figure out why so the company can maintain the sales figures. After some data analysis, you conclude that there is a direct correlation between comic book sales and Disney+ subscriptions. But you still don’t have an exact reason, so you develop some scenarios to form a hypothesis.

* * *

### [**Scenario 1: A Direct Cause**](#TOC)

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image-3.png)

**Figure 2:** Scenario 1, where increased comic book sales cause more Disney+ subscriptions (source: image by the author).

The comics display ads from Disney+, so increased comic book sales cause more Disney+ subscriptions.

* * *

### [**Scenario 2: Reversing the Cause and Effect**](#TOC)

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image-6.png)

**Figure 3:** Scenario 2, where an increase in Disney+ subscriptions causes an increase in comic book sales (source: image by the author).

Disney+ has shows about comic book characters. New fans would like to consume more of such content. So an increase in Disney+ subscriptions led to more Marvel Comics being bought.

* * *

### [**Scenario 3: Investigating a Hidden Cause**](#TOC)

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image-7.png)

**Figure 4:** Scenario 3, where a hidden cause affects the outcome (source: image by the author).

Or it’s something quite different. All Marvel Cinematic Universe movies are on Disney+. When a new Marvel movie comes out, the hype around these characters leads to more comic book sales (and, in turn, more Disney+ subscriptions).

* * *

### [**Causal Thinking**](#TOC)

The possibility presented in Scenario 3 is a hidden cause.

Each node is a variable, and each arrow shows the direction of the causal connection.

Suppose we build an accurate model to predict when the user will buy more comic books. But that is all that our model would do. The problem statement could be better expressed by a question like: What would the user have done if we had done something differently?

So you might think: I get it; the model doesn’t always consider hidden causes, but it still works. The predictions are still accurate.

Having our observed and causal effects in the same direction is considered lucky. More often than not, we see the opposite in real-world data, as shown in **Figure 5**.

[![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image-1.png)

**Figure 5:** Expectation vs. Reality of observed and causal effects (source: image by the author).

* * *

[**Case Study 2: Food App Conundrum**](#TOC)
--------------------------------------------

Let’s look at another example to understand this better: Suppose you have now been employed at a food ordering app as a Data Scientist. The product designer wants to introduce a new User Interface (UI), which she believes will engage more users. She has introduced the new UI to a select few people and wants you to make sense of the data that has been gathered.

You decide to judge the old UI and the new one on a common criterion: whether the user orders after opening the app. Let’s call this the opened-and-ordered criteria. **Table 1** describes this.

[![Image 8: Table 1: UI type comparison (source: image by the author).](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20255'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/causal-1-table-1.png)

**Table 1:** UI type comparison (source: image by the author).

Based on this data, it seems pretty straightforward. The new UI is the clear winner. But something strange happens when you condition the data on high-activity vs. low-activity users, as shown in **Table 2**.

[![Image 9: Table 2: UI Type comparison conditioned on high activity vs. low activity users (source: image by the author).](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20292'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/causal-1-table-2.png)

**Table 2:** UI Type comparison conditioned on high activity vs. low activity users (source: image by the author).

This data discrepancy modeled on a different subset of the original data is called “[Simpson’s paradox](https://en.wikipedia.org/wiki/Simpson%27s_paradox).”

“But hey, why does this discrepancy occur?”

Well, maybe, since the old UI is already tried and tested, it is shown to users only at a particular time of the day, around the evening, when app activity is generally high. The new UI might be shown at other times of the day when the number of lower-activity users who are likely to order food is less.

The causal diagram may look like the following in **Figure 6**.

[![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/05/image-5.png)

**Figure 6:** Actual Causal Diagram (source: image by the author).

* * *

[**Quiz Time!**](#TOC)
----------------------

The debate on what it means to cause something and how cause precedes effect has been going on forever. Most prominent contestants include but are not limited to:

*   Aristotle
*   Homer
*   Einstein
*   That French guy from Matrix 3
*   Judea Pearl, and many more

However, sadly, until recently, the science of [causal inference](https://ftp.cs.ucla.edu/pub/stat_ser/r350.pdf) was not considered formal mathematics or even something that could be quantified and studied.

This was remedied hugely by the contributions of Judea Pearl ([Graphical Causal Models](https://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch22.pdf)) and Donald Rubin ([Rubin causal model](https://en.wikipedia.org/wiki/Rubin_causal_model)).

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOC)
-------------------

So, in conclusion, we learned here that simply because two variables are correlated does not necessarily mean that one is the direct cause of the other. We can do more harm through our model if we do not look for hidden causes and their effects.

Therefore the idea of what would happen had something in the observation process been changed becomes an imperative question to ask.

This is called [Counterfactual Thinking](https://en.wikipedia.org/wiki/Counterfactual_thinking).

Counterfactual thinking, specifically about what-if scenarios, is the heart of causal inference.

Some ways we can answer a counterfactual question are:

*   Randomization
*   Natural Experiments
*   Conditioning

Interestingly, the methods become much harder to use as we go down the list. But as we go up, the methods are more foolproof and give better validation results. We will closely examine these and the various underlying methodologies in the next blog post of this series. Till then, keep learning, and don’t stop asking the question: What-If?

* * *

### [**References**](#TOC)

*   [Causal Inference in Online Systems by Amit Sharma](http://www.amitsharma.in/talk/causal-inference-online-systems/)
*   [An awesome playlist for Causal Learning](https://youtube.com/playlist?list=PLT3sJRyVaw-m9YBy3KRavOYuKyXikO4f6)
*   [The Book of Why: The New Science of Cause and Effect](http://bayes.cs.ucla.edu/WHY/)

* * *

### [**Citation Information**](#TOC)

**A. R. Gosthipaty and R. Raha.** “Introduction to Causality in Machine Learning,” _PyImageSearch_, P. Chugh, S. Huot, K. Kidriavsteva, and A. Thanki, eds., 2023, [https://pyimg.co/3sa8x](https://pyimg.co/3sa8x)

@incollection{ARG-RR\_2023\_CausalityIntroML,
  author = {Aritra Roy Gosthipaty and Ritwik Raha},
  title = {Introduction to Causality in Machine Learning},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Susan Huot and Kseniia Kidriavsteva and Abhishek Thanki},
  year = {2023},
  note = {https://pyimg.co/3sa8x},
}

* * *

![Image 11: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

* * *

**To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), _simply enter your email address in the form below!_**

![Image 12](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!