Association Rule Mining in R
By: Conor Lipinski
Tags: Apriori, Association Rule, Association Rule Mining, R, Data Science

Association Rule Mining in R

Practical explanation of market basket analysis in R with an effective twist

Introduction

Association rule mining is an unsupervised machine learning technique that utilizes the apriori algorithm. Rule mining can be used for uncovering associations between objects in datasets and common trends in the transactions. The most direct application is found in market basket analysis or a “Customers Also Bought” approach. By implementing the apriori algorithm a list of rules are generated as an output to function as if then-scenarios regarding the objects in the set.

A wide spectrum of fields can employ association rules including insurance, medical, telecommunications, manufacturing, etc. While the fields may change, the typical output is a market basket analysis. If a certain item is bought, then what is the likelihood another unique item is purchased and so forth. This theme can be used for a multitude of marketing strategies as well as sales insights and inventory controls.

Math

To better understand how the output rules are generated we will address the math that makes up the the apriori algorithm. The three parameters listed below align with the antecedent and consequent, left and right hand of the rules respectively. Yielding appropriate rules is dependent on establishing appropriate thresholds for each metric so it is pertinent that we know how they’re calculated and what they offer.

Support is a metric showing how often an item or item-set occurs. Specifically, the fraction of transactions that include the item or item-set compared to total transactions.

Confidence is an implication metric. Given the inclusion of a certain item or item-set, how mathematically confident are we that another specific item was bought with it. Confidence on its own can be confusing so only take it in context with the other two.

Lift is a little trickier, but an important metric nonetheless. Lift will show the correlation of two items given a rule. It is a factor by which the occurrence of the two items as a set surpasses the probability of those items occurring together independently. Implying that the higher the lift value the higher chance the two items in question appear together. As a warning, lift should not be the only metric considered when making business decisions as it blinds the user to the directionality of the rule.

Lift>1 insinuates that the items appear together more often than they would randomly (positive relationship)

insinuates that the items appear together more often than they would randomly (positive relationship) Lift<1 insinuates that the items appear together less often than they would randomly (negative relationship)

insinuates that the items appear together less often than they would randomly (negative relationship) Lift=1 insinuates that the items appear together as often as they would randomly (no relationship)

Alternate Approach

To conduct a market basket analysis, transactional databases like a sales repository are pulled to begin the analysis. Typically one row consists of all the items bought in a unique transaction. Ideally a transaction key, a customer key and a timestamp would be included. As I’m sure we’re all aware of, real world data isn’t always pretty as Kaggle sets so I will refrain from using a specific dataset as an example and present more pseudo-code with an effective twist.

That twist is accounting for business cases where multi-item orders are scarce, but sales are not. With a blatant disparity between these datasets and the pretty grocery store basket examples, we need to work around this gap to produce a market basket analysis. The solution is to collect sales by a unique customer over an established timeframe. Our analysis will avoid the dilemma caused by a customer immediately buying something after an initial order due to forgetting the first time round. In most cases this approach will yield a more holistic analysis of customer behavior rather than the rigid structure of rules based on items in one unique sales order. Sales can be aggregated up to the day, week, month and so forth dependent on the business case and if a timestamp is available in the dataset. Let’s take a look how to approach this.

R Code

Libraries

The library list below is by no means absolute, load or delete whatever packages you see fit. This is simply a starting point that will cover the functions needed in most cases.

```{r}

library(arules)

library(arulesViz)

library(tidyverse)

library(lubridate)

library(dplyr)

library(plyr)

library(readxl)

library(xlsx)

```

Data Load and Wrangling

There are many ways to pull in data whether it be using SQL tables, Excel sheets, CSVs or other files, but the goal is to have a workable data-frame. Assuming we have a data-frame now with at minimum a customer key, a timestamp and an item per row we can proceed.

```{r}

transactions=read.csv('transactions.csv')

transactions=transactions[,c('Customer_ID','Item_ID','Order_Date')]

```

To utilize the arules package we need to group items based on the customer key, but prior to this we need establish our timeframe we are creating our basket from. The lubridate package helps us create columns from slicing our timestamp which allow for easy filtering. Below is an example of filtering for 2020 data and capturing transactions by month. After generating a list of our monthly frames we can boil each one down to only the necessary fields.

```{r}

transactions$year=year(transactions$Order_Date)

transactions$month=month(transactions$Order_Date)

transactions=subset(transactions,transactions$year==2020)

transactions_by_month=list()

for(i in 1:max(transactions$month)){

transactions_by_month[[i]]=data.frame(subset(transactions,

transactions$month==i))

}

for(i in 1:max(transactions$month)){

transactions_by_month[[i]]=transactions_by_month[[i]][,c('Customer_ID','Item_ID')]

}

```

Now that we have our timeframe specific sets, we need to wrangle the items to be grouped by customer. There are many ways to do this, but below is how I have done it to produce customer key as the leftmost column and an item per column afterwards. I’ve highlighted how to wrangle just January’s data here, but in practice you’ll want to loop through your entire list of time-segmented transactions.

```{r}

transactions=transactions_by_month[[1]]

transactions=transactions[order(transactions$Customer_ID),]

transactions$time=unlist(tapply(transactions$Customer_ID, transactions$Customer_ID, function (x) seq(1, len = length(x))))

jan_transactions = reshape(transactions, idvar="Customer_ID", direction="wide")

```

Apriori

In the eyes of the apriori algorithm, we now have an acceptable data-frame to generate rules from. We will write those clean transactions to a CSV file to then use the read.transactions function within the arules package to bring them back into the R environment as a object of class transactions.

```{r}

setwd('C:/Users/ConorLipinski/Documents')

write.csv(jan_transactions,

'C:/Users/ConorLipinski/Documents/MarketBaskets/Jan2020.csv',

quote = FALSE,row.names = FALSE)

jan_transactions=read.transactions('C:/Users/ConorLipinski/Documents/MarketBaskets/Jan2020.csv',format = 'basket',sep = ',',quote = "")

```

Finally we can generate rules after selecting our thresholds for the apriori function’s parameters. You’ll have to play around with the support and confidence parameters especially if you’re working with a very large set. I find that setting these parameters very low and then filtering after rule generation is more efficient than editing parameters multiple times.

minval: minimum value of the support an itemset should satisfy to be a part of a rule

minimum value of the support an itemset should satisfy to be a part of a rule minlen: minimum number of items required in the rule

minimum number of items required in the rule maxlen: maximum number of items that can be present in the rule, default is length of 10

maximum number of items that can be present in the rule, default is length of 10 smax: maximum support value for an itemset

maximum support value for an itemset maxtime: maximum amount of time allowed to check for subsets, default 5 seconds, 0 will disable the time limit

maximum amount of time allowed to check for subsets, default 5 seconds, 0 will disable the time limit supp: specified minimum support value, defaults to .1

specified minimum support value, defaults to .1 conf: specified minimum confidence value, defaults to .8

```{r}

jan_rules=apriori(jan_transactions,parameter = list(supp=.0015,conf=0.5,target='rules',maxlen=5))

jan_rules=sort(jan_rules,by='count',decreasing = TRUE)

jan_rules=as(jan_rules,'data.frame')

```

The chunk above generates rules and orders them by appearance count and turns that into a data-frame. You can order the rules by confidence, support or lift, but I find count very helpful to immediately see what our highest frequency is. At this point inspect your data-frame and clean accordingly. I prefer to see an item per column and have shown how to separate the LHS below. If you plan to append your frames together make sure you specify the timeframe in another column according to your initial filtering.

```{r}

jan_rules$month=1

jan_rules=jan_rules %>% separate(col = rules,into = c('lhs','rhs'),sep = '=>')

jan_rules= jan_rules %>% separate(col=lhs,into = c('Item1','Item2','Item3','Item4'),sep = ',')

names(jan_rules)[5]<-"Basket_Item"

```

Next Steps

After generating your rules, it’s important to interpret them in the context of your listed parameters and to observe all three metrics (support, confidence and lift). Make sure to observe redundant rules and decide if you want to remove them. A redundant rule occurs when another rule with less items exists at the same or higher confidence. The LHS or antecedent in the general rule will contain less items and have the same or greater confidence than a redundant rule. The chunk below shows how to observe redundant and non-redundant respectively, and you may choose to filter from there.

```{r}

inspect( jan_rules [is.redundant( jan_rules )])

inspect( jan_rules [!is.redundant( jan_rules )])

```

It’s beneficial to add a column that lists how many items are in one basket. Scale yours according to max length after declaring an item count column.

```{r}

jan_transactions$Item_Count=0

for(i in 1:nrow(jan_transactions)){

if(isTRUE(is.na(jan_transactions[i,3]))){

jan_transactions[i,'Item_Count']=2

}

else if(isTRUE(is.na(jan_transactions[i,4]))){

jan_transactions[i,'Item_Count']=3

}

else if(isTRUE(is.na(jan_transactions[i,5]))){

jan_transactions[i,'Item_Count']=4

}

else{

jan_transactions[i,'Item_Count']=5

}

}

```

Visualization may be important to your business case and if so, the arulesViz package has some great functions and visuals. The data is formatted where is can be visualized outside of R too, I’ve used some nifty Power BI add-ons before. Be cognizant of how many rules you have as most visuals are network based and can become very messy very quickly.

Conclusion

An unfitting section name for an analysis that has a seemingly endless number of uses. Varying business cases, timeframes and apriori parameters make market basket analysis feasible for many applications and I hope this provides a necessary change of pace from the previously rigid instruction on the topic.