A Machine Learning Approach — Building a Hotel Recommendation Engine | by Susan Li | Towards Data Science

All online travel agencies are scrambling to meet the AI driven personalization standard set by Amazon and Netflix. In addition, the world of online travel has become a highly competitive space where brands try to capture our attention (and wallet) with recommending, comparing, matching and sharing.
In this post, we aim to create the optimal hotel recommendations for Expedia’s users that are searching for a hotel to book. We will model this problem as a multi-class classification problem and build SVM and decision tree in ensemble method to predict which “hotel cluster” the user is likely to book, given his (or her) search details.
The data is anonymized and almost all the fields are in numeric format. The data set can be found at Kaggle, we will use train.csv which captured the logs of user behavior, and destinations.csv which contains information related to hotel reviews made by users.
The Figure 1 below provides the schema of the train.csv:
The Figure 2 below provides the schema of the destinations.csv:
import datetimeimport pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport seaborn as sns%matplotlib inlinefrom sklearn.model_selection import cross_val_scorefrom sklearn.ensemble import RandomForestClassifierfrom sklearn.pipeline import make_pipelinefrom sklearn import preprocessingfrom sklearn.preprocessing import StandardScalerfrom sklearn import svm
To be able to process locally, we randomly sample 1% of the records. After that, we still have a large number of records at 241,179.
df = pd.read_csv('train.csv.gz', sep=',').dropna()dest = pd.read_csv('destinations.csv.gz')df = df.sample(frac=0.01, random_state=99)df.shape
(241179, 24)
The objective is to predict which hotel_cluster a user will book given the information in his (or her) search. There are 100 clusters in total. In another word, we are dealing with a 100 class classification problem.
plt.figure(figsize=(12, 6))sns.distplot(df['hotel_cluster'])
The data is pretty much well distributed over all 100 clusters and there is skewness in the data.
The date time, checkin date and checkout date columns can not be used directly, we will extract year and month from them. First, we define a couple of functions to achieve that, and we also define a function to merge with destination.csv.
from datetime import datetimedef get_year(x):    if x is not None and type(x) is not float:        try:            return datetime.strptime(x, '%Y-%m-%d').year        except ValueError:            return datetime.strptime(x, '%Y-%m-%d %H:%M:%S').year    else:        return 2013    passdef get_month(x):    if x is not None and type(x) is not float:        try:            return datetime.strptime(x, '%Y-%m-%d').month        except:            return datetime.strptime(x, '%Y-%m-%d %H:%M:%S').month    else:        return 1    pass    def left_merge_dataset(left_dframe, right_dframe, merge_column):    return pd.merge(left_dframe, right_dframe, on=merge_column, how='left')
Dealing with date_time column:
df['date_time_year'] = pd.Series(df.date_time, index = df.index)df['date_time_month'] = pd.Series(df.date_time, index = df.index)from datetime import datetimedf.date_time_year = df.date_time_year.apply(lambda x: get_year(x))df.date_time_month = df.date_time_month.apply(lambda x: get_month(x))del df['date_time']
Dealing with srch_ci column:
df['srch_ci_year'] = pd.Series(df.srch_ci, index=df.index)df['srch_ci_month'] = pd.Series(df.srch_ci, index=df.index)# convert year & months to intdf.srch_ci_year = df.srch_ci_year.apply(lambda x: get_year(x))df.srch_ci_month = df.srch_ci_month.apply(lambda x: get_month(x))# remove the srch_ci columndel df['srch_ci']
Dealing with srch_co column:
df['srch_co_year'] = pd.Series(df.srch_co, index=df.index)df['srch_co_month'] = pd.Series(df.srch_co, index=df.index)# convert year & months to intdf.srch_co_year = df.srch_co_year.apply(lambda x: get_year(x))df.srch_co_month = df.srch_co_month.apply(lambda x: get_month(x))# remove the srch_co columndel df['srch_co']
After creating new features and removing the features that are not useful, we want to know if anything correlates well with hotel_cluster .This will tell us if we should pay more attention to any particular features.
df.corr()["hotel_cluster"].sort_values()
No column correlates linearly with hotel_cluster, this means that methods which model linear relationship between features might not be suitable for the problem.
After a quick google search, it is not hard to learn that for known combinations of search destinations, hotel country, hotel market will definitely help finding the hotel cluster. Let’s do that.
pieces = [df.groupby(['srch_destination_id','hotel_country','hotel_market','hotel_cluster'])['is_booking'].agg(['sum','count'])]agg = pd.concat(pieces).groupby(level=[0,1,2,3]).sum()agg.dropna(inplace=True)agg.head()
agg['sum_and_cnt'] = 0.85*agg['sum'] + 0.15*agg['count']agg = agg.groupby(level=[0,1,2]).apply(lambda x: x.astype(float)/x.sum())agg.reset_index(inplace=True)agg.head()
agg_pivot = agg.pivot_table(index=['srch_destination_id','hotel_country','hotel_market'], columns='hotel_cluster', values='sum_and_cnt').reset_index()agg_pivot.head()
Merge with the destination table and our newly created aggregate pivot table.
df = pd.merge(df, dest, how='left', on='srch_destination_id')df = pd.merge(df, agg_pivot, how='left', on=['srch_destination_id','hotel_country','hotel_market'])df.fillna(0, inplace=True)df.shape
(241179, 276)
We are only interested in booking events.
df = df.loc[df['is_booking'] == 1]
Get features and labels.
X = df.drop(['user_id', 'hotel_cluster', 'is_booking'], axis=1)y = df.hotel_cluster
from sklearn.naive_bayes import GaussianNBclf = make_pipeline(preprocessing.StandardScaler(), GaussianNB(priors=None))np.mean(cross_val_score(clf, X, y, cv=10))
0.10347912437041926
from sklearn.neighbors import KNeighborsClassifierclf = make_pipeline(preprocessing.StandardScaler(), KNeighborsClassifier(n_neighbors=5))np.mean(cross_val_score(clf, X, y, cv=10, scoring='accuracy'))
0.25631461834732266
We report performance measurement by k-fold cross-validation, and Pipeline makes it easier to compose estimators.
clf = make_pipeline(preprocessing.StandardScaler(), RandomForestClassifier(n_estimators=273,max_depth=10,random_state=0))np.mean(cross_val_score(clf, X, y, cv=10))
0.24865023372782996
from sklearn.linear_model import LogisticRegressionclf = make_pipeline(preprocessing.StandardScaler(), LogisticRegression(multi_class='ovr'))np.mean(cross_val_score(clf, X, y, cv=10))
0.30445543572367767
SVM is very time consuming. However, we achieved a much better result.
from sklearn import svmclf = make_pipeline(preprocessing.StandardScaler(), svm.SVC(decision_function_shape='ovo'))np.mean(cross_val_score(clf, X, y, cv=10))
0.3228727137315005
Seems we need to do more feature engineering in order to improve the result. Stay tuned for further improvement!
Source code can be found on Github. Have a great week!