# New Vector-Space Embeddings for Recommender Systems
## 3 Proposed Approach
### General Approach
The third term corresponds to item-item similarity, where the dot product \(x_{i}^{T}x_{i^{\prime}}\) is the predicted similarity of items \(i\) and \(i^{\prime}\), and \(o_{ii^{\prime}}\) is the target similarity to be learned. This term can be used to model the following:  
* Content-based: For example, items \(i\) and \(i^{\prime}\) that are similar or related product-types and have similar features and/or have a similar price range would have a high target similarity \(o_{ii^{\prime}}\);
* Purchase pattern based: The target similarity \(o_{ii^{\prime}}\) is proportional to the number of times the items \(i\) and \(i^{\prime}\) are purchased together by some user. This is what we call \"Recommendation\". Please note that the weighting functions \(w_{ij}\) should be set to be proportional to the importance of the term considered. For example, CF-related terms, such as Prediction, Recommendation, or Term 2's purchase pattern, should be weighted higher than the other terms. Let us for now consider Equation (1), as this is the general form that encompasses Equation (2) and other forms. The objective of the approach is to find the vectors \(x_{i}\) that minimize the error defined in (1). The goal is to have these vectors express the desired similarities, that is, the dot products would be as close as possible to the given similarity values \(s_{ij}\). The norm bound (\(x^{T}x=1\)) can be incorporated as a Lagrange multiplier. Solving this optimization problem all at once is hard, as this is a fourth order multi-dimensional polynomial equation. However, if we consider a specific item or user's vector \(x_{j}\) and fix the others, it becomes a quadratic equation (with a Lagrange multiplier term) and therefore has an analytic solution. We exploit this fact and develop a relaxation algorithm such that one vector is learned at a time while fixing the others. After that, another vector is learned (again while fixing the others), and so on until all the vectors are learned. The algorithm does not stop at this point but a few more cycles are performed, going through all vectors one-by-one, until convergence. The convergence is marked such that the decrease in error is no longer significant from one cycle to the other. Here is a short description of the steps. For more details refer to [41], including a proof of convergence. The steps of the training algorithm are summarized as follows:  
1. Generate the initial vectors \(x_{i}\) randomly, normalizing them as unit length. 2. For \(i=1\) to \(N\) perform the following:
3. Fix all \(x_{j}\) except \(x_{i}\) and update \(x_{i}\) as follows: \[x_{i}=A^{-1}b,\] (3) where \(\bullet\)\(A\) is a matrix defined as: \[A=\sum_{j\
eq i}w_{ij}x_{j}x_{j}^{T}-\lambda I,\] (4) \(\bullet\)\(b\) is a vector defined as: \[b=\sum_{j\
eq i}w_{ij}x_{j}\xi_{ij},\] (5)\(\lambda\) is the scalar Lagrange multiplier, evaluated using a simple one-dimensional bisection search so that: \(x_{i}^{T}x_{i}=1\). 4. Repeat Steps 2-3 until the vectors converge. Since a vector may be involved in more than one transaction, that is, user \(u\) rates a number of items or item \(i\) is rated by a number of users, there is a competition between vectors such that the vectors are placed optimally with respect to each other, satisfying the similarities of the collective set of transactions. After training is complete, the similarity between any two vectors \(x_{i}\) and \(x_{j}\) can be simply obtained using their dot product. In order to have some focus in the work and avoid straying into too many possible avenues of application of this vector embedding, we will concentrate in this work on the \"Prediction\" and the \"Recommendation\" formulation (Terms 2 and 3 in (2)). These are described next.