Travel Segmentation by User Search Behaviors
By: Susan Li
Tags: Clustering, Segmentation, Data Science Tutorial, Ml For Travel, Unsupervised Learning

Travel Segmentation by User Search Behaviors

In the travel and tourism industry, segmentation is an important strategy for developing itineraries and marketing materials targeted towards different groups with varying travel intents and motivations. It helps the businesses to understand the subgroups that make up the audience so that the businesses can better tailor products and messages.

One caveat in the travel industry is that unlike online shopping, leisure travel is an infrequent purchase, most leisure travelers only travel once or twice every year, active customers are either uncommon or very slow in making their bookings.

But the good thing is that people love to travel and many of us even spend more time planning vacation than working on our finances. As a result, if you are a frequent user of any online travel agency, you have handed your data over to the company in exchange for the company delivering value to you. You have told them about yourselves, your travel intents and motivations. You expect the travel options that are more personalized to you rather than the long list of options you get today.

In this prototype of segmentation model, we are trying to understand the nuances of user behavior and intent, segment users’ searches by trips, that is, the process of dividing trips into groups based on common characteristics.

For various reasons, I won’t be able to share the data set, but feel free to use your own data set that suitable for this prototype. Or you are welcome to check the Jupyter notebook.

Data Preprocessing & Feature Engineering

During the data processing & feature engineering stage, I made the following decisions:

We are going to segment these searches by trips, so I removed the duplicate trip_ids.

There are over 31% of searches that user added “watch”, I will use the data with “watch_added_dt”, because: 1). The more we know the user the better we can personalize the message to fit their interests. 2). When a user adds “watch”, this indicates he (or she) is interested in this trip and would like to receive notifications on the update of the flight fare.

Over 77% of these searches were originated from the United States, I believe it makes sense to segment these trips (originated from the US) first. Because: 1). I want to use US public holidays as a feature. 2). Travel behaviors may differ between travelers from different countries/regions.

Fill missing “stay” with median values.

Remove rows with missing “lowest_price” because I believe price is one of the most important elements for trip segmentation.

Add a new feature — “booking_window”, which is the difference between “first_searched_dt” and “departure_date”.

Add a new feature — “route”, which is “origin” to “destination”.

Add a new feature — “holiday”, which is US public holiday derived from “departure_date”.

Table 1

After processing, the data we are going to work with contains 269,203 trips with 25,696 different flight routes and 773 different departure dates to 178 destination countries, as shown in Table 1.

Observations:

• In the data, the first price and lowest price are pretty close.

• Approx. 0.3% of these trips were for the same day departure and return (i.e. “stay” = 0).

• Another 0.1% of these trips were looking to leave ASAP (i.e. “first_searched_dt” and “departure_date” were the same day).

Exploratory Data Analysis

What are important factors to characterize a “trip”? Let’s focus on the following features.

Weekends or weekdays departure

76% of the searched trips were for weekday departures.

Destinations

Figure 1

The data set is largely dominated by trip destination within the Unite States (over 68%).

Flight routes

Figure 2

Observations:

• 19 of these top 20 searched routes were traveling within the United States.

• 12 of these top 20 routes were from NYC to somewhere, mostly to somewhere warm within the country, only one route was from NYC to Paris.

Departure day is holiday or not, if so, what holiday.

Figure 3

I had a feeling that the users are not likely to travel during busy holiday travel seasons, probably has something to do with the high-season prices.

Lowest price. When a user started watching a price, he (or she) should always aim at waiting for the lowest price, so I decided to use lowest price other than the first price.

Figure 4

We can see that 73% of these trips are in the price range from 100 to 500 dollars.

How long they stay.

Figure 5

73% of these trips are in the stay range from 0 to 7 days.

Booking window

Figure 6

55% of these trips are in the booking window range from 0 to 90 days.

K-Mode Clustering

The standard k-means algorithm isn’t directly applicable to categorical data like this one. Therefore, I will use the k-modes algorithm which uses a simple matching dissimilarity measure to deal with categorical objects. It defines clusters based on the number of matching categories between data points.

To use k-mode, I converted three numeric features into categorical features, that is: “lowest_price” to “lowest_price_bin”, “stay” to “stay_bin”, “booking_window” to “booking_window_bin”, according to the threshold in the above three pie charts.

Figure 7

According to Figure 7, I should choose k=3, which is three predicted segments.

Identify the segmentation

Segmenting trips by destination regions.

Figure 8

Segmenting trips by flight routes.

Figure 9

Segmenting trips by US public holidays.

Figure 10

Segment by weekends vs. weekdays.

Figure 11

Distribution of lowest price by segment.

Figure 12

Distribution of stay by segment.

Figure 13

Distribution of booking window by segment.

Figure 14

Interpret the results

Segment 0:

The largest segment which contains 178,519 trips.

They are mostly domestic routes within the United States, plus some routes from the US to Mexico or Antilles.

Most of these routes are likely to be from NYC or Chicago to somewhere warm, such as California, Florida or Nevada.

These trips are likely to depart on weekdays.

The average price is 310 dollars, the average stay is about 6 days, and user’s average booking window is less than 3 months.

So, the trips in Segment 0 sound like week-long vacations from the East to the sunny destinations.

Segment 1:

This segment contains 48,101 trips.

Trips in this segment are largely domestic.

These trips are likely to depart on the weekends, that including Memorial weekend and Labor day weekend; They are weekend Getaways for about 3 days.

Because it is a weekend Getaway, the flight distances are likely to be short within the US.

The average price is 319 dollars.

So, the trips in Segment 1 are mostly quick weekend Getaways with relative short flight. They can be cheap, romantic or last minute, or just recharge from the hustle and bustle of the city life.

Segment 2:

This segment contains 42,583 trips.

Trips in this segment are likely to be international trips such as traveling from the US to Europe or Asia. It also includes domestic trips but traveling far away to Hawaii.

For this type of trips, the users tend to search departure on weekdays.

The average price is 743 dollars, and the average stay is around 14 days.

Users tend to plan earlier for long distance trips with average booking window at about 4–5 months.

How do we use this information?

The business will be able to provide an optimal experience for its users based on their intent by using above knowledge to create more effective content or recommendations.

For example, once a user searches and watches a trip in Segment 1, the online travel agency would know this user is likely to look for a weekend Getaways, the online travel agency then can give travel advice on weekend Getaways or recommend other similar weekend Getaways destinations that currently have promotion on flights or hotels.

Jupyter notebook can be found on Github, enjoy the rest of the week!

References:

http://citeseerx.ist.psu.edu/viewdoc/download doi=10.1.1.932.2724&rep=rep1&type=pdf

https://www.kaggle.com/ashydv/bank-customer-clustering-k-modes-clustering