An example of predicting ratings with SVD in a collaborative filtering recommendation system | by Xue Wang | Towards Data Science

We know that there are cons with SVD to make a prediction in reality, for example, it can’t predict if there is even a NaN in the dataset, but as this is the starting point of collaborative filtering and I want to reproduce the procedure with SVD to see how it works, with some compromisation of the dataset.
This story will focus on code realization with SVD, have no offline testing (no splitting of train test dataset), and include some basic linear algebra-related terminologies.
Linear algebra basic
SVD is singular value decomposition. Detail explanation can be found in Wiki. Here, I prefer this explanation for the recommendation system (RS): it is a k-rank approximation to the original SVD.
Latent features: corresponding, the sigma in SVD. Values that are not directly observable in data but maybe recognized when looking at relationships and trends that exist between observed data values[2]. The number of the latest features is the rank of sigma in SVD.
Euclidean Distance:
Euclidean distance can just be considered as a straight-line distance between two vectors. For two vectors x and y, we can compute this as[2]:
The hard point I want to emphasize(at least from my learning experience) is how to convert original SVD into k dimension spaces once we know the U, sigma, and Vt, and how to link them with prediction. Example is always efficient way to learn.
Code Example
Compromise of dataset
The target of RS in collaborative filtering, here user-item based, is to predict the ratings and make the recommendation if the user hasn’t made the rating.
But SVD can’t predict if there is a NaN value in the matrix, and the user has to exist in the currently known rates system and gives rates.
I think there is a contradiction, but maybe I am wrong here (if you find the reason, will be appreciated to point it out).
To create the dataset, the compromise here: if the user hasn’t given a rating to a movie, then fill it with 0 (there will have a conflict if 0 is in the ratings ).
Please don’t recommend Funk SVD here. As I want to learn the pro and cons of the SVD process in this story.
Let’s get started. The referenced code is here (In Chinese)[1]. I made some changes to it.
The code consists of the below steps:
Create datasetCalculate the similarityDecide kConvert the original SVD to k dimensionsMake recommendation for a specific user through the predicted rating (which are zero in original rating)
Create dataset
Calculate the similarity
Decide k
Convert the original SVD to k dimensions
Make recommendation for a specific user through the predicted rating (which are zero in original rating)
Import library:
Create dataset:
Calculate the similarity
Use Euclidean Distance to measure similarity:
Decide k: The value of k is determined by the percentage of the sum of squares of the first k singular values to the sum of squares of the total singular values.
For example, if the percentage is 0.9, then when the square sum of the previous k single values to the total square sum of the sigma is greater than 0.9, we have accounted for more than 90% weight and can reduce the matrix to k dimensions.
Convert the original SVD to k dimensions spaces:
How converting the original SVD to k dimensions, below is the key:
The dimension of original decomposition is:
u:11x11,sigma: diagonal matrix 11, vt: 11x11
The result is k=3, then the dimension is:
u:11x3, sigma: diagonal matrix 3, vt: 3x11
Below code can construct the k dimension matrix:
formed_items=np.around(np.dot(np.dot(u[:,:k], sigma_K),vt[:k, :]),decimals=3)
Predicted ratings:
Run the below command to get the result:
testdata=loadExData()recommend(testdata,0,sim_meas=ecludSim,est_method=svdEst, percentage=0.9)
Using testdata, with Euclidean Distance, if the percentage of the sum of squares of the first k singular values to the sum of squares of the total singular values is greater or equal than 0.9, predict the ratings for a non-rated movie for the user 0.
The predicted ratings for user 0:
For user 1:
The first column shows the column index and the second column is the predicted ratings.
In original data for user 0, the position 0,1,2,3,4,6,7,8,9 are zero as below:
[0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 5]
To make it easy, the complete code is as below, and you can run it with the last two commends above:
In today’s story, I introduce an example that shows how to convert the SVD to k dimensions spaces to predict ratings after getting SVD, especially focus how to convert SVD to k-dimension spaces and make predicted ratings.
Thanks for your reading.
References:
https://blog.csdn.net/weixin_41988628/article/details/83217255 (In Chinese)Udacity Data Scientist Nanodegree — Experimental Design & Recommendationshttps://stats.stackexchange.com/questions/33142/what-happens-when-you-apply-svd-to-a-collaborative-filtering-problem-what-is-th/35460#35460https://en.wikipedia.org/wiki/Singular_value_decomposition
https://blog.csdn.net/weixin_41988628/article/details/83217255 (In Chinese)
Udacity Data Scientist Nanodegree — Experimental Design & Recommendations