Doubly Ensemble Movie Prediction With Social Media Data Using TBEEF
Christopher V. Rackauckas, University of California, Irvine
Xu Chenxiao, Hong Kong Baptist University
Colin Jarvis, Macalester CollegeCai Weijie, Hong Kong University of Science and Technology
Advisor: Avery Ching, Hong Kong University of Science and Technology
Introduction to TBEEF

Incorporating Social Media Information

Our team decided to create a software package for developing prediction algorithms utilizing a doubly ensemble
framework. This program is called TBEEF, Triple Bagged Ensemble Ensemble Framework. It has three steps:

Example Network Models

Introduction
Recommender systems have become commonplace in recent years:•••

Amazon shows users other compelling purchases.
1.

Model: Runs factorization models on sparse datasets to generate predictions.

2.

Hybrid: Uses an ensemble of ensembling techniques to generate predictions.

3.

Synthesize: Aggregates the hybrid step predictions using one ensemble technique.

Netflix shows users new movies that may be of interest.
Google shows targeted advertisements.

Many companies have large swaths of user’s information from social networks such as Facebook. As the largest
internet company in China, Baidu Inc. runs popular social media networks and is looking to use this information
to improve their video service.
Goal: Develop algorithms to incorporate social media data into recommendation systems.

Followed Network Model
Using social data collected about the users, we construct our input vectors using user ID u, movie ID i, anda set of followed users S. Each friend in the vector is given a 1/m value where m is the total number ofusers followed by u. To encode this model into the factorization machine, one would use the input vector:
x = (0, . . . , 1, 0, . . ., 0, . . . , 0, 1, 0, . . . , 0, 0, . . . , 1/m, 0, . . . , 1/m, . . . , 0).
|{z} |{z} |{z}
|U|
|I|
|U|
to define the model:

A plugin interface has been implemented to let users easily create factorization models for the Model stepand ensemble models for the Hybrid and Synthesize steps. This program is publicly distributed with the MITx
license and can be found on the Machine Learning Open Source Software (MLOSS) repository under the name
TBEEF: Triple Bagged Ensemble Ensemble Framework.

ŷ(x) = w0 + wu + wi + hvu , vi i +1+The Problem

Approximately 10,000 users rating around 8,000 movies.

•User’s browser history.

•The users that a given user has “followed“.

•Movie tags as set by users.

1mX

1

m
X

hvi , vs i +wsjj
m j=1
m j=1mX

mX

hvu , vs i +hvs , vs 0 i.
jjj
m j=1
m2 j=1 0j >jThis correlates the user’s predicted ratings with those of the people who he/she follows on the social networks.

Baidu Inc. provided our team with a dataset containing:•mX

1Browser History Model
Using browser history for each implicitly defined user, we give each movie viewed in the history a rj /m value
where m is the total number of movies in the history and rj is either the rating of the movie lj or the user
never rated it, then simply a 1.
x = (0, . . . , 0, 1, 0, . . . , 0, 0, . . . , r1 /m, 0, . . . , rm /m, . . . , 0).
{z} |
|
{z}
|I|
|I|
This penalizes movies for which the user browsed to but did not watch.

Our project was to train a model which would be predict a user’s rating for a movie. Our model was tested ona hidden dataset using the RMSE metric:nX
RM SE =(yˆi − yi )iwhere yˆi is the model prediction and yi is the known user movie rating. As the RMSE measures the distance
between prediction and reality, our goal was to find a model which minimizes the RMSE.

The Ensemble Problem
Ensemble models were used to aggregate predictions from many models under certain conditions:
Diagram of the TBEEF program. Shown are the three steps, the Model step, the Hybrid step, and the Synthesize step.
Example models are shown in the Model and Hybrid steps.

1.

Needed to prevent over-fitting data.

2.

No worry about sparsity: factorization models generate a prediction for every movie.

3.

Needed to account for biases induced by various models.

Particular difficulties included:

The Factorization Model•

Sparsity: users do not rate most movies.
General Factorization Model:•Computational efficiency: the algorithm must run in a reasonable time on large datasets.
ŷ = w0 +mX

wj xj +

j=1

Summary of Main TechniquesmXmX

hvi , vj ixi xj ,j=1 k=j+1•ŷ is the predicted movie rating.

•x is the data vector to be specified for the user/movie pair.

•

w0 is the global average rating.

Overfit Protection: Bagging without replacement
To prevent over-fitting, bootstrap aggregation without replacement was used. Thus each model saw a random
subset of the data with which to train. User/movie pairs used as predictions in the next round were omitted
from the training set to reduce the bias imposed by utilizing multiple steps.

Ensemble Models

Solving Sparsity: Factorizations•Idea: To find the interaction (estimated correlation) between Alice in Wonderland and Star Trek,
assume the interaction is the same as some other related and rated movie (i.e. Star Wars)

Example Ensemble Method: Ordinary Least Squares Regression (OLS)
Given j factorization models, one way to ensemble the predictions is to run a regression
yi =•This can be done mathematically modeling interactions with an n × m matrix:•mXxi,j βj .

j=1

wi is the average for the ith parameter.

which calculates the “best” way to create a prediction as a weighted average of other predictions.
– Let m be the number of movies.

•hvi , vj i is defined as the dot product between the i-th and the j-th rows of the factor matrix V.

– Let n  m.
– Define the interaction effect to be the dot product between columns.

Standard Factorization Model
The standard model is to use information of only the user and the movie. Index the movies and users numericallyand use binary variables to denote the user/movie pair with a data vector:• Training will make similar movies have similar columns causing high interaction effects.
Ensembling: “Knowledge of the Crowd“(u, i) → x = (0, . . . , 0, 1, 0, . . . , 0, 0, . . . , 0, 1, 0, . . . , 0)
{z} |
{z}
|
|U|

• In Who Wants to Be a Millionaire, the “ask the audience“ option was almost always right whereas“phone a friend“ (supposed expert) was usually wrong!
• Idea: Generate many recommendations from diverse models and aggregate the results.

|I|

1.

Random Forest Models (Bagged Random Forest, Conditional Random Forest): Split data into
homogeneous regions to run OLS.

2.

Gradient Boosted Regression Trees: Repeatedly cycle residuals back into OLS.

3.

Bayesian Model Averaging Regression: Make predictions on all 2j product combinations of predictions.

to give the model:ŷ(x) = w0 + wu + wi +kX

Acknowledgments
vu,f vi,ff =1• Work to develop and aggregate many models instead of a “best“ model.

Advanced Ensemble Models

where wu is the user’s average rating and wi is the movie’s average rating.

This research was made possible through the Research Industrial Projects for Students, Hong Kong (RIPS-HK), an undergraduate research
opportunity ran by UCLA’s Institute for Pure and Applied Mathematics (IPAM). This project was funded by the NSF. We thank ouracademic mentor Dr. Avery Ching, Hong Kong University of Science and Technology, and our industry client Baidu, Inc. for helping guideus through the project. For more information, please see our technical report, Doubly Ensemble Movie Prediction With Social Media DataUsing TBEEF.