Association rule learning: A brief overview [Part-1]
By: Pran Kumar Sarkar
Tags: Machine Learning, Association Rule Learning, Association Rule, Data Science, Apriori

Association rule learning: A brief overview [Part-1]

An in-depth insight into why, what and how?

Introduction:

The world we live in a manipulative world. Every moment we are programmed in our subconscious to get indulged in things which business tycoons want us to do. Starting from discount offers to repetitive display of advertisements, we are made to get indulged in things which maximize profit of the entertainment and sales industry.

Now the question comes, “How to decide which measures will maximize profit?”

To answer this question business enterprises accumulate a large number of data from their day-to-day operations. They then use these data to get valuable insights to make the system more efficient.

But, “How collecting data increases efficiency of the system?”

To understand this let us take a case study.

Association Rule in Recommendation Systems:

Recommendation systems uses association rule to decide on which things you might be interested in. Imagine the case of movie recommendation with Netflix. Netflix monitors your every interaction with their app and then they use this information to produce customized content for everyone. Based upon your watch history, your likes and dislikes they recommend movies you might like. Surprisingly, 80% of Netflix views are from the service’s recommendation.(Source)

Fig: Netflix Movie Recommendation

So now you can imagine how Association rule increases the efficiency of the system.

But what is Association Rule Learning?

Association Rule learning is a rule based machine learning method which is used to discover interesting relationships hidden in large data-sets.

These rules does not necessarily extract users preference, but rather find relationships between set of elements present in different entries in data-sets.

Association Rule Mining is sometimes referred to as “Market Basket Analysis”, as it was the first application area of association mining. For example, consider the following data-set containing market basket transactions.

Fig: Market basket transactions

If you look at the data-set then you can see a pattern.

Most people who buy Diapers also buy Beer.

Most people who buy Diapers also buy Milk.

Most people who buy Bread also buy Milk.

So, we can see the following rules,

{Diapers} →{Beer}

{Diapers} → {Milk}

{Bread} →{Milk}

So the association rule has two parts:

Antecedent (if) — It is something which is found in data.

(if) — It is something which is found in data. Consequent(then) — It is something which occurs when antecedent occurs.

Have a look at this rule for instance,

If a person buys bread then he is likely to buy milk.

Here Bread is Antecedent and Milk is Consequent.

In general: {Antecedent} →{Consequent}

Issues with Association Rule Learning:

Before going further its worth mentioning that this kind of rule extraction from data has the following two issues:

Discovering rules from large data-sets can be computationally expensive.

Some of the patterns in data can occur by-chance.

We will see how to overcome these issues as we get familiar with the terminologies.

Terminologies used with Association Rule Learning:

Fig: Basic terminologies

Fig: An example containing five entries in data-set

Support:

It indicates how frequently an item or item-set appears in the data-set.

In the above entries, A, B, C, D, E are items and ABC, ACD etc forms item-set.

Confidence:

It indicates how often a rule is found to be true.

Why do we calculate Support and Confidence?

Support and confidence is calculated to overcome the issue where some patterns in data can occur by chance.

Support is an important measure because a rule that has very low support may occur simply by chance. Moreover a low support rule is also less interesting for business perspective also because it may not be profitable to promote items which customers seldom buy together.

On the other hand, confidence measures the reliability of the inference made by a rule. For a rule {A} →{B} with high confidence indicates that B is more likely to occur with A.

The need for calculating Lift:

For rule {A} →{B}

By calculating support we know whether the rule is significant.

By calculating confidence, we come to know how likely A and B will occur together.

But these measures do not give us any idea on what extent the occurrence of one item(say A) or item-set increase the occurrence of the another item(say B) or item-set.

So we calculate lift to know, how the antecedent and consequent are related to one another.

For a rule: {Antecedent} →{Consequent}

If lift = 1 , then it means that the possibility of occurrence of Antecedent and Consequent are not dependent on one another.

, then it means that the possibility of occurrence of Antecedent and Consequent are not dependent on one another. If lift < 1 , then it means the occurrence of Antecedent has negative effect on occurrence on Consequent and vice versa.

, then it means the occurrence of Antecedent has negative effect on occurrence on Consequent and vice versa. If lift > 1, then it means that the two occurrences are dependent on one another, and these rules are very useful in determining the consequent in latter cases. It also lets us know to what extent the occurrences are dependent on one another.

Lift:

Lift(A →B) indicates the rise in probability of occurrence of B when A has already occurred.

In the above example,

Solution to problems with Association Rule Learning:

By calculating support, confidence and lift, we have overcome the second problem, where the co-occurrences of two or more items can be due to chance.

But still there is one problem left, i.e. Mining of rules from large data-set can be computationally expensive.

Association Rule Mining:

There are many approaches to association rule mining.

These are:

Standard Method

Algorithmic method

The standard approach of mining rules from data has a limitation. By searching a massive numbers of possible associations to look for collections of items which appear associated, there is a risk of finding many spurious association, i.e they can occur by-chance.

Moreover, with increase in the size of data-set, the process of finding rules from this process becomes computationally in-efficient. So instead we will use algorithmic methods like Aproiri algorithm.