Data Science in the Film Industry Part 1: What is my Preference?
By: Riya Mhatre
Tags: Machine Learning, Data Science, Entertainment Industry, Recommender Systems, Big Data

On the surface, the film industry seems to be pretty easy to understand. Viewers only get a glimpse of the actors on their screens. Audience members rarely think about the inner workings and the complex processes hidden behind the big screen. There are multiple layers within the industry that power its success. Big data is a useful tool for Data Scientists working in the film industry.

Data Scientists have a key role in ensuring triumph with this data by collecting relevant samples and analyzing important trends to gauge the preferences of the public. Based on the data collected, companies can predict customer preferences and viewing habits.

Many streaming platforms dread the day when their customers will get bored of the recommended content. Continuous and repetitive content might cause people to turn to another source after a while. To prevent this, companies are working to improve their streaming algorithms and relying on recommender systems.

Big Data and Recommender Systems

Data Scientists in prominent production companies (Amazon, Netflix, Hulu) analyze the trends in the data to understand the viewing preferences of the general public and cater their predilections. Big Data is commonly gathered by combing through sites on the internet and public data reserves to get accurate information. Recommender systems use this data to gauge the public’s preferences.

Recommender systems are “algorithms aimed at suggesting relevant items to users”. There are two parts to this system: collaborative filtering and content-based filtering.

Figure 1. Collaborative and content-based filtering methodologies. Source: Towards Data Science

A. Collaborative Filtering

The collaborative method relies on the relationship the user had with past options. These interactions are then stored in the “user-item interactions matrix”, displayed below.

Figure 2: The user-item interactions matrix; Source: Towards Data Science

The user preference is expressed using two categories. Explicit Rating refers to a value on a scale, such as the number of stars one would give to a movie they watched. Implicit Rating documents the user’s activity such as page views, number of clicks, purchase records, whether or not they listened to a certain music track. The information stored in the matrix is then used to determine which ratings match what user.

Collaborative filtering uses the Nearest Neighborhood algorithm. For user-based collaborative filtering, there is a matrix of dimensions (a × b), where a denotes the number of user IDs, and b denotes the number of items, which contains ratings.

If a given target user did not watch or rate a certain item, we can still predict that target user’s rating of it. To do this, we need to compute the similarities between all the users and our target user. After that, we gather the top X similar users, and then take the weighted average of the ratings from the X users with similarities as weights.

People sometimes give ratings that are higher than their true intended value. To prevent this phenomenon from skewing the data, subtract every user’s average rating for all the items when calculating the weighted average, and then adding it back to the target user. This is the idea of the Nearest Neighborhood algorithm.

Item-based CF is when two items are said to be similar because they received similar ratings from a singular user. The recommendation system first finds the similarities between the item pairs and then proceeds to the model-building stage. Linear regression and weighted sum are used by the system during the calculation. Linear regression is used to determine the relationships between the rating habits among the different users. The system analyzes the items the user has rated and checks the similarities between the items and then creates a recommended list.

B. Content-Based Filtering

Content-based filtering recommends items that align to the user’s preferences based on their ratings from the past, specifically higher ones. As the user continues to rate items, the algorithm becomes more and more accurate at predicting the user’s preferences. It is easier to narrow down the choices based on the information stored.

The benefit of content-based filtering is that it does not require data about other users because this type of filtering is user-specific.This method can also help understand the user’s specific interests, so it can recommend content that, perhaps, only a few other users consume. It can even work when a product does not have any review. The represented content also varies, which opens up options to other approaches such as text processing techniques.

A limitation is that the model can only recommend content using current interests rather than understanding the interests on a broader scope. This tends to create a filter bubble, a phenomenon in which only certain types of content are recommended to the user by the algorithm. For example, if the user is currently interested in action movies, the majority of the suggested movies will be action movies. Variety is out of the question in this case.

Repetitive content can bore the user, which leads them to turn to another option. Companies are trying to overcome this hurdle and make it more effective in the long run.

Case Study: Netflix

Netflix has used recommendation systems to understand their customer’s preferences. Upon creating an account, users are told to select a few titles that interest them. These titles are used to “start” running the algorithm that matches content based on the user’s interest. Those who do not choose any titles will start off at square one as they are picking shows.

As the user continues to watch more content, the recent titles will supersede the past titles. Netflix adds tags to each work that summarizes the main parts of the title, such as “nostalgic drama” or “romantic comedy.”

Figure 3: Users usually scan vertically rather than horizontally. Source: Netflix: Binging on the Algorithm

Netflix’s new recommendation system, which uses content-based filtering, relies on image data, the “covers” the user sees when browsing titles. Netflix uses a framework that uses big data to ultimately decide what images work for each user. After conducting many experiments about user preferences, it has been proven that certain emotional ranges appeal to the users. Netflix’s algorithm essentially works to display a show’s cover page that reflects aspects, emotional ones, in particular, of the show that are important to the user.

A. Variation of a Recommender System

Netflix found a way to improve the audio and visual quality of their content to completely immerse their viewers into what they are watching. Predictive caching is used to permit a video to increase its speed or play at a higher quality. If a viewer is watching a television series, for example, the following episode will be partly cached.

Fortunately, all that research and hard work has paid off. Netflix’s strategy of frequently providing new, varying content and undeniable experience in the entertainment industry makes it a formidable competitor. Netflix profits have increased by more than 30% since 2015, with its revenue being a staggering $16.614 billion yearly.

Customer Experiences

While Big Data is useful to increase sales, it also helps determine the problem areas in a company. If customers are having difficulties with a particular brand, they will most likely change to another option.

Even the most devoted customers cannot tolerate more than one unfortunate incident. In a survey by PricewaterhouseCoopers, 59% stated that they will change to a different brand after multiple terrible experiences, and 17% after just one terrible experience.

Figure 4: Consumer Interaction after bad experience(s); Source: Bad experiences are driving customers away — faster than you think

As seen in the figure above, even one experience can discourage a certain percentage of customers. Several bad incidents do lead to a significantly higher percent of displeased people.

The adoption of Big Data and more effective recommendation systems help companies mitigate the problem areas to ensure positive customer responses.

Although there are many places to improve, companies such as Netflix are in a good position. These companies will continue to innovate and develop their technology to increase the percentage of satisfied customers. Recent developments in recommendation systems have already led to much success because of the system’s ability to understand their user for the most part. The system will continue to improve its accuracy as time goes on.