Understanding and Implementation of Apriori Algorithm with Python — Part 2
By: Ishant Wadhwa
Tags: Apriori Algorithm, Implementation, Suport, Python, Recommendation System

Understanding and Implementation of Apriori Algorithm with Python — Part 2

In the previous part click here. we learn how apriori algorithm work, basic intuition behind it. we also discuss support, confidence and lift. Now we will see how to implement apriori algorithm for large data set with python. Ishant Wadhwa Follow Apr 23, 2020 · 4 min read

Super Market

Problem Statement : A store owner meet us and he wanted to know how he will arrange there items such that he will be able to increase the selling.

Solution: First thing we will ask him is data, so he provide us all the transactions which contain list of item sell in a CSV file. For data set click here

First we will read the data using pandas

import pandas as pd file_Path = 'https://raw.githubusercontent.com/IshantWadhwa4/Recomandation_System/master/AprioriAlgo/store_data.csv' store_Data = pd.read_csv(file_Path,header=None)

Each row is a Transaction

2. For implementing the apriori algorithm, we feed data to the algorithm in a particular format.

columns are unique items and rows are transactions. True represent that item is the part of that transaction.

3. Now we will convert our data in the above format.

final_data = pd.DataFrame(columns=['Transaction','Items'])

store_Data = store_Data.T for col in store_Data.columns:

col_data = list(store_Data[col].dropna())

temp_dict = {'Transaction':[int(col)]*len(col_data), 'Items':col_data }

temp_df = pd.DataFrame(temp_dict)

final_data = final_data.append(temp_df,ignore_index=True) final_data.head()

Result

4. Now we have items and transaction columns now if we do one-hot encoding don’t you think we will get the format we want. Think for a minute.

Now before doing one-hot-encoding/get_dummies let’s do some EDA on the final_data.

import seaborn as sns

import matplotlib.pyplot as plt plt.figure(figsize=(15,8))

order = final_data['Items'].value_counts()[:10].index

sns.countplot(x = 'Items',data=final_data,order = order)

plt.title('Top 10 most selling items')

plt.show()

Top 10 most selling items

How many unique items and total number of transactions.

len(final_data['Items'].unique())

len(final_data['Transaction'].unique())

5. Let’s convert the data in the required format

final_data_dummy = pd.get_dummies(final_data['Items'])

final_data_dummy['Transaction'] = final_data['Transaction'] # i don't want the quantity, I want just that product is bought or #not(after doing dummy yow will get 2 if same item bought twice in a #transaction) so i will use encode_units function def encode_units(x):

if x <= 0:

return 0

if x >= 1:

return 1 format_data= final_data_dummy.groupby('Transaction').sum()

format_data = format_data.applymap(encode_units) format_data.head()

Data in required format.

6. Now we will apply apriori algorithm, we will use mlxtend library for apriori algorithm implementation. This algorithm execute in two steps one is get support values for each combination and than apply association rules to get confidence and lift values.

from mlxtend.frequent_patterns import apriori

from mlxtend.frequent_patterns import association_rules # min_support is the limit of support value

# frequent_itemsets has frequency of each item and its combination frequent_itemsets = apriori(format_data, min_support = 0.01, use_colnames = True) frequent_itemsets.head()

support value for each combination

In next step we will apply association rules.

# the first parameter is frequent_itemsets which we calculate above # and metric is 'lift'. you ca try with metrics = 'confidence' also



result = association_rules(frequent_itemsets, metric = "lift", min_threshold=0.01) result.sort_values('lift',ascending=False)