Analyzing Disneyland Reviews with NLP | by Marielle Dela Cruz | Towards Data Science

Worked in collaboration with Pei Liu, Yvonne Lu, Parsa Tahmasebi, and Yingkai Ma (UC Irvine MSBA ‘21)
Project Goal: To compare customer sentiment regarding attractions at three Disneyland locations (Anaheim, Paris, and Hong Kong) using sentiment analysis, emotion detection, and n-gram associations. To identify keywords using topic modeling that can help Disney determine visitor pain points and improve park experience. To see our code, refer to the GitHub repository here.
Dataset: We used a dataset found in Kaggle containing 42,000 reviews from TripAdvisor about three Disneyland branch locations: California, Paris, Hong Kong. We decided to split up the data by branch location to make it computationally easier to analyze smaller samples. There are 19,406 reviews about Anaheim, 13,630 about Paris, and 9,620 about Hong Kong. Data variables include Review_ID, Rating, Year_Month, Reviewer_Location, Review_Text, and Disneyland_Branch.
Exploratory Data Analysis: First, we performed exploratory data analysis (EDA) to get a general idea of average ratings across all locations, where our visitors are from, and how the ratings changed over time.
We found the average rating for each park (out of 5 stars): 4.41 for California, 3.96 for Paris, and 4.20 for Hong Kong. Visitor nationality depended on location — for example many Hong Kong visitors are from countries nearby (i.e. Australia, India, Philippines), most of Paris’s visitors are from the United Kingdom, and many of California’s visitors are from the United States.
Word Clouds: We generated word clouds to have a general idea of what visitors talk about when reviewing their Disneyland park experience. In our analysis, we are only interested in reviews written in English and removed duplicated reviews.
To create the word clouds, we tokenized words in the reviews and removed stopwords. There are many ways to customize font, background colors, and contours for word clouds. You can refer to this website to find HTML colors or this for more colormap options that you can use to enhance your graphs.
tokenizer = RegexpTokenizer(r'\w+')text = text.lower()cleaned_text = re.sub('\W', ' ', text)stopword = stopwords.words("english")snowball_stemmer = SnowballStemmer("english")word_tokens = nltk.word_tokenize(cleaned_text)stemmed_word = [snowball_stemmer.stem(word) for word in word_tokens]processed_text = [word for word in stemmed_word if word not in stopword]text_string=(" ").join(processed_text)#make word cloudmask = np.array(Image.open("m2.jpg"))wordcloud = WordCloud(background_color="white", font_path='arial', mask = mask).generate(text_string)#applies colors from your image mask into your word cloudimage_colors = ImageColorGenerator(mask)plt.figure(figsize=(15,8))plt.imshow(wordcloud.recolor(color_func=image_colors), cmap=plt.cm.gray, interpolation="bilinear")plt.axis("off")plt.show()
In general, the reviews from all three locations mention fast passes, lines or queues, visitor demographic, rides and attractions.
Sentiment Analysis: We want to take a deeper look into what the reviews reveal visitor sentiment. For this step, we used three methods to obtain sentiment using TextBlob and Vader.
This method uses compound score to assign sentiment “positive” or “negative”.
This method uses compound score to assign sentiment “positive” or “negative”.
#assign sentiment based on compound scoredf['sentiment'] = np.where(df['vader_comp'] >= 0.05, 'positive', 'negative')
2. We also attempted other methods to attach sentiment by using a different threshold for compound score to classify “positive”, “neutral”, and “negative”:
df['nltk_sentiment_type'] = ''df.loc[df.nltk_compound > 0, 'nltk_sentiment_type'] = 'POSITIVE'df.loc[df.nltk_compound == 0, 'nltk_sentiment_type'] = 'NEUTRAL'df.loc[df.nltk_compound < 0, 'nltk_sentiment_type'] = 'NEGATIVE'
3. Another method is by assigning sentiment based on ratings:
# assigning sentiment using ratingrating = df['Rating']def label_func(rating):    if rating >= 4:        return "Positive"    elif rating == 3:            return "Neutral"    else:        return "Negative"        df['Sentiment'] = df['Rating'].apply(lambda x: label_func(x))
We wanted to see how much sentiment polarity and subjectivity is influencing how visitors review their experience at Disneyland. Below is an example of how we measured polarity and subjectivity of reviews about Hong Kong that use the phrases “happiest place” and “happiest place on earth.”
From the graphs above, we infer that reviews with positive sentiment and high polarity scores were extremely satisfied with their park experience. There are more subjective positive reviews than subjective negative reviews. Subjectivity measures how opinionative a review is, so we can infer that those who hold negative opinions gave harsher reviews because their expectations were not met during their visit.
Emotion Detection: We assigned emotions using NRCLex, which measures emotional affect based on text. The emotional affects include: positive, negative, anticipation, trust, fear, surprise, anger, sadness, joy, and disgust. We also tried using the package text2emotion, which categorizes emotion as happy, angry, sad, surprise and fear. However, the problem with text2emotion is that it would give high scores for “fear” (which does not really make sense for the happiest place on Earth) so we opted to use NRCLex instead for our analysis.
from nrclex import NRCLex#nrclex outputs a score for all 9 emotions, but we only want the emotion with the highest score (usually the first output)emotion = []for i in range(len(df)):    emotions = NRCLex(df['Review_Text'][i])    emotion.append(emotions.top_emotions[0][0])df['emotion'] = emotion
After detecting emotions, we created visualizations for each location to show frequency of each emotion:
graph = sns.countplot(y = "emotion", data = df, palette = "flare")graph.set(xlabel = 'Frequency', ylabel = 'Emotion', title = 'Emotion from reviews about Disneyland (CA)')figure = graph.get_figure()    
Overall, all three parks have a high frequency of “positive” emotion. However, we have to take a closer look at reviews associated to negative emotions (anger, fear, negative, disgust, sadness) so that we can have an idea of why some visitors had an unpleasant park experience.
We found some interesting reviews where emotions were classified as “disgust” or “negative”:
Customer service in Paris: “Don't waste your time with Euro Disney... staff were unfriendly, rude and not at all interested in customers! If you do not speak French, you are ignored!”Prices in California: “It is no doubt a legendary place but I just dont think it's worth the money and headache of overcrowding and overpriced everything. Disney continues to raise the price of admission and the crowds are ridiculous every single day.”Customer service in California: “Having been to Disneyworld in Florida the California Disneyland is a let down. It was dirty in restrooms. Food server would turn around and pick nose.”Hotel accommodations, customer service, park accessibility, and lines in Hong Kong: “Was there during the Christmas holidays. The entire trip was marred with sickening service and bad experiences...The is no magic even to my kids. The staff there are mostly sulking (maybe because they need to be working over the holidays). Responses are mainly read from a script, sounding like a broken monotonous record. There were 2 highlights that I wish I would forget.....Rented a stroller for my little one. Was served by a frowning staff, handed over a HKD$1000 note but was given the wrong change (HKD$500 less) asked her and was given a good screaming!! she then cursed in cantonese and insisted that there is no HKD$1000 note with her and that I am stupid to have given a HKD$500 note. Highlight number 2 was when we were queuing for the 3D show when a bunch of Indian nationals cut to the front of the queue. Informed the staff and they adamantly inform that there is no queue regulations and continued to chat among themselves. The queue cutters then retorted that they are blue blooded and that is their rights to be in the front of the queue. The staff then grinned and nodded and continue their small talk among themselves.Worst nightmare ever....never ever go there.....better stick to Tokyo or US.”
Customer service in Paris: “Don't waste your time with Euro Disney... staff were unfriendly, rude and not at all interested in customers! If you do not speak French, you are ignored!”
Prices in California: “It is no doubt a legendary place but I just dont think it's worth the money and headache of overcrowding and overpriced everything. Disney continues to raise the price of admission and the crowds are ridiculous every single day.”
Customer service in California: “Having been to Disneyworld in Florida the California Disneyland is a let down. It was dirty in restrooms. Food server would turn around and pick nose.”
Hotel accommodations, customer service, park accessibility, and lines in Hong Kong: “Was there during the Christmas holidays. The entire trip was marred with sickening service and bad experiences...The is no magic even to my kids. The staff there are mostly sulking (maybe because they need to be working over the holidays). Responses are mainly read from a script, sounding like a broken monotonous record. There were 2 highlights that I wish I would forget.....Rented a stroller for my little one. Was served by a frowning staff, handed over a HKD$1000 note but was given the wrong change (HKD$500 less) asked her and was given a good screaming!! she then cursed in cantonese and insisted that there is no HKD$1000 note with her and that I am stupid to have given a HKD$500 note. Highlight number 2 was when we were queuing for the 3D show when a bunch of Indian nationals cut to the front of the queue. Informed the staff and they adamantly inform that there is no queue regulations and continued to chat among themselves. The queue cutters then retorted that they are blue blooded and that is their rights to be in the front of the queue. The staff then grinned and nodded and continue their small talk among themselves.Worst nightmare ever....never ever go there.....better stick to Tokyo or US.”
Using sentiment and emotion together can help Disney find areas of customer dissatisfaction and why they are dissatisfied. Then, this information can be used to make the appropriate changes necessary to enhance visitor experience.
Collocations: We wanted to find common phrases in reviews so we can identify keywords or topics that are frequently discussed by visitors. By doing so, Disney can identify context of reviews without having to read each one individually. We took steps to find collocations — phrases that are made up of more than one word and co-occur together through n-gram association.
Bigram Association — identifying two words that co-occur frequently to gain more insight about review context. We found that it is more meaningful when we only filter for adjectives and nouns:
def rightTypes(ngram):    if '-pron-' in ngram or '' in ngram or ' 'in ngram or 't' in ngram:        return False    for word in ngram:        if word in stop_words:            return False    acceptable_types = ('JJ', 'JJR', 'JJS', 'NN', 'NNS', 'NNP', 'NNPS')    second_type = ('NN', 'NNS', 'NNP', 'NNPS')    tags = nltk.pos_tag(ngram)    if tags[0][1] in acceptable_types and tags[1][1] in second_type:        return True    else:        return False
Here are the top bigrams from each park location:
The most frequent bigrams mention popular attractions, type of ticket used (day pass, park hopper), employees (cast member), and wait time.
Trigram Association — identifying three words that co-occur frequently to gain more insight on review context. Trigrams may give us more information on what visitors do in the theme parks. We will also only consider adjectives and nouns to give us meaningful context.
It seems that visitors who go to the California location would encourage others to “use fast pass” for shorter wait time. These reviews also talk about California Adventure, compare their experience to Walt Disney World in Florida, using park hoppers, and popular park attractions. Paris reviewers also mention using fast passes, popular attractions, and comparing to other park locations. Reviews about Hong Kong mention popular attractions and wait times for rides.
We can use the results we obtained through collocation identification and further analyze visitor sentiment on attractions. For this, we will locate reviews that mention “Space Mountain” or “Hyperspace Mountain” and compare general sentiment about the ride in all three locations. We are only interested in knowing more about words visitors use to associate with the ride, so we will only consider adjectives.
Overall, reviewers from all three locations use similar adjectives when we visualize reviews that are associated with Space Mountain.
Topic Modeling: We want to use topic modeling to extract keywords used in reviews to help Disney identify specific issues that concern park visitors. We can build a dictionary and train our topic model using an unsupervised learning algorithm, LDA (Latent Dirichlet Allocation) to uncover hidden patterns in the text. This will also yield the probabilities of topics appearing in our model. Before building this model, we removed punctuations, stopwords, and lemmatized the text. Then, we found the best value for k (number of topics) by graphing it against coherence score. We picked our ideal k value where the coherence score is at maximum. We used the gensim package for this task.
def compute_coherence_values(dictionary, corpus, texts, start, stop):    """    Compute c_v coherence for various number of topics    """    coherence_values = []    model_list = []    for num_topics in range(start, stop):        model = gensim.models.ldamodel.LdaModel(corpus=corpus,                                               num_topics=num_topics,                                              id2word=id2word,                                              random_state=90,                                              alpha='auto',                                              eta='auto',                                              per_word_topics=True)        model_list.append(model)        coherencemodel = CoherenceModel(model=model, texts=texts,                             dictionary=dictionary, coherence='c_v')        coherence_values.append(coherencemodel.get_coherence())    return model_list, coherence_valuesstart=4stop=11model_list, coherence_values = compute_coherence_values(dictionary=id2word,                                     corpus=corpus,                                    texts=data_lemmatized,                                    start=start, stop=stop)x = range(start, stop)plt.figure(figsize=(10,7))plt.style.use('ggplot')plt.plot(x, coherence_values, color = "blue",marker=".")plt.xlabel("Num Topics", size=14)plt.ylabel("Coherence score", size=14)plt.title('Number of Topics Based on The Coherence Score',size=18)#plt.savefig("output/k_topic.jpg", bbox_inches='tight', dpi = 300)plt.show()
For the California location, the ideal number of topics is 7. This gave an a coherence score of 0.35 and perplexity score of -6.94. Applying this k value, we obtain the top words in each topic:
[(0,  '0.089*"year" + 0.078*"kid" + 0.052*"old" + 0.033*"child" + 0.032*"come" + '  '0.031*"little" + 0.029*"character" + 0.018*"young" + 0.018*"daughter" + '  '0.015*"adult"'), (1,  '0.079*"wait" + 0.076*"line" + 0.066*"pass" + 0.050*"fast" + 0.043*"long" + '  '0.038*"people" + 0.037*"hour" + 0.026*"minute" + 0.023*"ticket" + '  '0.019*"get"'), (2,  '0.051*"ride" + 0.049*"go" + 0.040*"time" + 0.040*"day" + 0.037*"park" + '  '0.030*"get" + 0.018*"do" + 0.016*"good" + 0.014*"food" + 0.014*"take"'), (3,  '0.087*"mountain" + 0.080*"space" + 0.041*"pirate" + 0.033*"detail" + '  '0.028*"disneyworld" + 0.022*"haunted_mansion" + 0.022*"walt" + '  '0.020*"matterhorn" + 0.019*"caribbean" + 0.018*"indiana_jone"'), (4,  '0.020*"area" + 0.018*"tell" + 0.013*"give" + 0.013*"put" + 0.012*"re" + '  '0.012*"parking" + 0.011*"patience" + 0.011*"gate" + 0.011*"leave" + '  '0.010*"truely"'), (5,  '0.078*"disneyland" + 0.051*"disney" + 0.038*"place" + 0.030*"great" + '  '0.029*"love" + 0.027*"visit" + 0.025*"park" + 0.020*"world" + 0.019*"fun" + '  '0.017*"well"'), (6,  '0.034*"overprice" + 0.032*"halloween" + 0.030*"effort" + 0.022*"party" + '  '0.021*"event" + 0.013*"group" + 0.013*"weekday" + 0.013*"local" + '  '0.011*"trick" + 0.010*"sandwich"')]
To visualize the topics in two-dimensions, we will use pyLDAvis from gensim and create an interactive visualization of our model.
From our topic model, we were able to obtain top keywords from each topic:
42.5% of tokens about rides, shows, and food.24.4% of tokens about positive and fun family park experience.12% of tokens about waiting in line and using fast passes.7.8% of tokens about who visitors visited the theme parks with.7.2% of tokens about parking and entrance.3.5% of tokens about popular rides (Space Mountain, Pirates of the Carribbean, Haunted Mansion, Matterhorn, Indiana Jones, Big Thunder Mountain).2.5% of tokens about holidays (Halloween and Christmas).
42.5% of tokens about rides, shows, and food.
24.4% of tokens about positive and fun family park experience.
12% of tokens about waiting in line and using fast passes.
7.8% of tokens about who visitors visited the theme parks with.
7.2% of tokens about parking and entrance.
3.5% of tokens about popular rides (Space Mountain, Pirates of the Carribbean, Haunted Mansion, Matterhorn, Indiana Jones, Big Thunder Mountain).
2.5% of tokens about holidays (Halloween and Christmas).
We applied the same technique to extract topics about Paris and Hong Kong and found the following results:
For Disneyland Paris, we found 6 topics that are about: park experience, attractions, queues and fast pass, hotel accommodations, weather, and park accessibility.
For Disneyland Hong Kong, we extracted 7 topics concerning: park experience, family entertainment, hotel and restaurant accommodations, customer service, holidays and celebrations (Halloween, Winter, birthdays), positive experience, and visitor recommendations.
By finding key topics, Disney can pinpoint specific areas they can improve to enhance visitor experience. The same technique can be applied to analyze reviews where sentiment is negative in order to find problems that occur in the theme parks.
We used this method to analyze negative reviews about Hong Kong and identified topics associated to negative sentiment: queues, prices, cutting in line, and restaurants. A quick Google search on line cutting revealed that it has been a problem in Hong Kong for years. Disneyland Hong Kong has also added in their park regulations that visitors should “...respect fellow Park Guests during your visit and be careful not to bump into, push or move ahead of others in the queue.” (Refer to this TripAdvisor forum on the issue).
We applied the same method on the Paris reviews and found these topics associated to negative sentiment: long lines and crowds, customer service, bookings and accommodations, smoking, time of visit, and bad park experience. Smoking is commonplace in France; Disneyland Paris even has designated areas for smoking inside the park. Several reviews complain about this issue and claim that it has “destroyed the magic.” (Here is a TripAdvisor forum on this issue). Visitors have expressed that it is not safe for their children to be around smoke that wafts through the air.
We found that common issues in California are: long lines and crowds, family experience, food, accessibility, and strollers. A unique problem that this location has is about stroller congestion, rental, parking, and theft. Reviewers mention that cast members would sometimes move strollers if they are obstructing a path or cut their locks when necessary. This action can cause park guests to be confused, especially if they have no idea where their stroller was moved to. (Here is an interesting forum on this issue. Here is an article about one park goers suggestion that Disney should do about this problem — charge visitors based on the size of the stroller.)
Classification Modeling: We wanted to build a robust classifier using multiple methods in order to classify future data. We implemented Random Forest and Logistic Regression individually and as an ensemble model on unigrams using Bag of Words and TF-IDF. The metric that we used for evaluating each model was F1-Score*. We also used the micro averaging method for our ensemble classifier.
*F1-Score is the weighted average of precision and recall metrics which you can calculate each of them using the confusion matrix.
The results of our analysis were as follows:
Bag of Words
Bag of Words
Hong Kong
California
Paris
2. TF-IDF
Hong Kong
Paris
California
Further Analysis: We can delve deeper on understanding more about when common park issues occur by comparing reviews by month. We can also analyze reviews over different years to inform us if each branch attempted to make changes that improved visitor experience.
Recommendations: Many visitor pain points stem from frustration of waiting in line and crowds. A large amount of time is spent standing in line, which irritates people and contributes to a more negative experience. Another common pain point that contributes to the problem with long lines is park capacity — more visitors mean longer wait times for attractions and food. Disney should take steps to alleviate symptoms of a negative park experience by considering some of the recommendations below:
Disney should find a way to optimize their fast pass system. Offer free mobile fast passes throughout the day.
Set maximum park capacity during peak seasons, especially in the summer, so that there are not too many people waiting in line. Hot weather only exacerbates visitor frustration.
Make wait lines interactive or have food stands near queues. Let more characters walk around popular rides — have some characters from that ride interact with visitors in line and take pictures with them.
Encourage cast members to enforce park rules so that visitors are only smoking in designated areas (Paris), not cutting in line (Hong Kong), and follow stroller policy (California). However, we realize that these are hard problems to tackle because there are visitors that try to break the rules if there are no consequences for them.
Disney should have designated areas near popular attractions for stroller parking or storage. This can lessen congestion on walkways and theft/loss of property.
References:
NRCLex: https://pypi.org/project/NRCLex/
vaderSentiment: https://pypi.org/project/vaderSentiment/
TextBlob: https://pypi.org/project/textblob/
Assigning sentiment based on compound score: https://www.analyticsvidhya.com/blog/2021/01/sentiment-analysis-vader-or-textblob/
Collocations: https://medium.com/@nicharuch/collocations-identifying-phrases-that-act-like-individual-words-in-nlp-f58a93a2f84a
Airlines Twitter: https://medium.com/analytics-vidhya/airlines-on-twitter-understanding-customer-complaints-with-nlp-81278f2b68dc