Paper Review: Neural Collaborative Filtering Explanation & Implementation
By: Kung-Hsiang, Huang, Steeve
Tags: Machine Learning, Neural Networks, Pytorch, Recommender Systems, Deep Learning

Essentially, each user and item is projected onto a latent space, represented by a latent vector. The more similar the two latent vectors are, the more related the corresponding users’ preference. Since we factorize the utility matrix into the same latent space, we can measure the similarity of any two latent vectors with cosine-similarity or dot product. In fact, the prediction for each user/ item entry is computed by the dot product of the corresponding latent vectors.

The prediction equals the inner product of latent vectors

However, the paper argued that dot product limits the expressiveness of user and item latent vectors. Let’s consider the following case. We first focus on the top three rows of this utility matrix.

Let S{x,y} denotes the similarity between user x and user y. By computing the cosine-similarity between users 1, 2, and 3, we know that S{2, 3} > S{1, 2} > S{1, 3}. Without loss of generality, we map the users onto a latent space of two dimensions as the following.

Now, we take into account user 4. Comparing the similarity with the others, we obtain S{1,4} > S{3,4} > S{2,4}. However, no matter we place the latent vector P4 to the right or left of P1, it will inevitably be closer to P2 than P3.

Therefore, this example shows the limitation of inner product to fully model the interactions between user and items in the latent space.

Neural Collaborative Filtering

The paper proposed Neural Collaborative Filtering as shown in the graph below. In the input layer, the user and item are one-hot encoded. Then, they are mapped to the hidden space with embedding layers accordingly. The Neural FC layer can be any kind neuron connections. Multiple layer perceptron, for example, can be placed here. It claims that with the complicated connection and non-linearity in the Neural CF layers, this model is capable of properly estimating the complex interactions between user and item in the latent space.

NCF architecture

So, how NCF is a generalization of Matrix Factorization? Let me show you in the graph below. We first replace Neural CF layer with a multiplication layer, which performs element-wise multiplication on the two inputs. Then, we set the weight from the multiplication layer to the output layer to be a fixed unit matrix (matrix of all ones ) of dimension K by 1 with linear activation function.

Then , we have the following equations.

The prediction of the unobserved interaction ŷ_ui denotes the predicted value of the (u, i) entry on the reconstructed utility matrix. L is the linear activation function, while ⊙ denotes the element-wise multiplication operation. p_u and q_i are the latent vector of user and item, respectively, and J is the unit matrix of dimension (K,1). Since J is a unit matrix, inside the linear function becomes the inner product between latent vector p_u and q_i. Furthermore, because the input and output are the same for linear function, it boils down to the last line. The predicted label is the inner product of the corresponding user and item latent vector. This equation is identical to that shown in the Matrix Factorization section. Thus, this proves that Matrix Factorization is a special case of NCF.

NeuMF

In order to introduce additional non-linearity, the final model proposed, NeuMF, includes a Mutliple-layer Perceptron (MLP) module apart from the Generalized Matrix Factorization (GMP) layer.

The output of GMF and MLP modules are concatenated and connected with the sigmoid activation output layer.

Performance

The paper evaluates NCF models and other models with the Leave-One-Out schema. That is, the last interaction for each user is held out for evaluation. Two evaluation metrics are being considered, Hit Ratio @ 10, and NDCG @ 10. Hit Ratio @ K denotes the fraction of prediction hits given 10 recommendation for each user. Let’s say if we recommend 10 items for each user, and 4 of the 10 users interact the items matching our recommendation, then Hit Ratio@ 10 = 0.4. NDCG, on the other hand, can be viewed as an extension of Hit Ratio, except that it considers the order of the hit. This means that if you the hit occurs at the higher recommendation, NDCG will be higher.

The performance comparison is shown in the figure below. In all cases, NeuMF performs better than the other models.