Exploiting Data Sparsity in Secure Cross-Platform Social Recommendation

1 Introduction . The recent advances of social recommendation have achieved remarkable performances in recommendation tasks [ 12 , 28 ] . Unlike traditional methods , social recommendation leverages user-item rating data ( e.g . from Netflix ) with user-user social data ( e.g . from Facebook ) to facilitate model training . The intuition behind this setup is that Facebook ’ s social data is much better than Netflix ’ s social data in both quantity and quality , and those social data at Facebook can help to improve Netflix ’ s recommendation performance . However , the cross-platform nature , the high sparsity and sensitivity of recommendation/social data make social recommendation hard-to-deploy in the real world [ 5 ] . In summary , the main problem we are facing is , How to perform secure and efficient social recommendation across different platforms , where the data are highly-sparse in nature ? Specifically , we focus on the problem of collaborative social recommendation in the two-party model , where one party ( denoted as P0 ) is a rating platform that holds user-item rating data , and the other party ( denoted as P1 ) is a social platform that holds user-user social data . We also assume that the adversaries are semi-honest , which is commonly used in the secure computation literature [ 9 ] . That 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . is to say , the adversary will not deviate from the pre-defined protocol , but will try to learn as much information as possible from its received messages . Choices of privacy enhancing techniques . Currently , many anonymization techniques have been used in publishing recommendation data , such as k-anonymity and differential privacy [ 11 ] . On the other hand , cryptographic methods like secure multiparty computation ( MPC ) [ 11 ] and homomorphic encryption ( HE ) have been proposed to enable calculation on the protected data . Since k-anonymity has been demonstrated risky in practice ( e.g. , the re-identification attack on Netflix Prize dataset [ 22 ] ) , and differential privacy introduces random noises to the dataset which eventually affects model accuracy [ 10 , 30 ] , we consider they are not the ideal choice for our framework . Instead , we choose a combination of cryptographic tools ( i.e. , MPC and HE , but mainly MPC ) which allows multiple parties to jointly compute a function depending on their private inputs while providing security guarantees . Choices of social recommendation model . In literature , many social recommendation models have been proposed [ 8 , 18 , 27 ] using matrix factorization or neural networks . Existing MPC-based neural network protocols [ 21 , 29 ] usually suffer from accuracy loss and inefficiency due to their approximation of non-linear operations . Especially for the case of social recommendation , training data could exceed to millions , and this makes NN-based model a less ideal choice . Therefore , we choose the classic social recommendation model , Soreg [ 18 ] , as a typical example , and present how to build a secure and efficient version of Soreg under cross-domain social recommendation scenario . Dealing with sparse data in secure machine learning . One important property of social recommendation data is its high sparsity . Take LibraryThing dataset [ 32 ] for example , its social matrix density is less than 0.02 % . Recently , Schoppmann et al . introduced the ROOM framework [ 26 ] for secure computation over sparse data . However , their solution only works on column-sparse or row-sparse data , and in addition , it requires secure matrix multiplication protocol ( for instance , based on Beaver ’ s multiplication triple ) . Chen et al . proposed a secure protocol for a sparse matrix multiplies a dense matrix [ 6 ] , which combines homomorphic encryption and secret sharing , but it only works well when the dense matrix is small . Different from their work , in this paper , we propose a PIR-based matrix multiplication which does not reply on pre-generated correlated randomness . Our framework . In this paper , we propose S3Rec , a sparsity-aware secure cross-platform social recommendation framework . Starting with the classic Soreg model , we observe that the training process of Soreg involves two types of calculation terms : ( 1 ) the rating term which could be calculated by P0 locally , and ( 2 ) the social term which needs to be calculated by P0 and P1 collaboratively . Therefore , the key to S3Rec is designing secure and efficient protocols for calculating the social term . To begin with , we first let both parties perform local calculation . Then both parties invoke a secure social term calculation protocol and let P0 finally receive the plaintext social term , and update the model accordingly . In this way , the security of our protocol relies significantly on the secure social term calculation protocol ( for simplicity , we refer this protocol as the ‘ ST-MPC ’ protocol ) , and we propose a secure instantiation and prove its security . Similarly , the efficiency of S3Rec relies heavily on the performance of ST-MPC , and at the core , it relies on the efficiency of a matrix multiplication protocol . The naïve secure matrix multiplication protocol is traditionally evaluated through Beaver ’ s triples [ 3 ] , and has O ( km2 ) asymptotic communication complexity , where k is the dimension of latent factors and m is the number of users . To improve the communication efficiency , we propose two secure sparse matrix multiplication protocols for ST-MPC , based on two sparsity settings : ( 1 ) insensitive sparsity , which is a weaker variant of matrix multiplication where we assume both parties know the locations of non-zero values in the sparse matrix , and ( 2 ) sensitive sparsity , which is also a weaker variant of matrix multiplication , but stronger than ( 1 ) , and we assume ‘ only ’ the number of zeros is public . Nevertheless , we present secure constructions for MatrixMul in both cases by leveraging two cryptography primitives called Private Information Retrieval ( PIR ) [ 1 ] and Homomorphic Encryption ( HE ) [ 24 ] . PIR can hide the locations of the non-zero values in the sparse matrix while HE enables additions and multiplications on ciphertexts . To this end , we drop the communication complexity of secure MatrixMul to O ( km ) for the insensitive sparsity case and to O ( αkm ) for the sensitive sparsity case , where α denotes the density of user social matrix . Summary of our experimental results . We conduct experiments on two popularly used dataset , i.e. , Epinions [ 19 ] and LibraryThing [ 32 ] . The results demonstrate that ( 1 ) S3Rec achieves the same performance as existing social recommendation models , and ( 2 ) S3Rec improves the computation time and communication size of the state-of-the-art ( SeSoRec ) by about 40× and 423× in average . Contributions . We summarize our main contributions below : ( 1 ) We propose S3Rec , a privacypreserving cross-platform social recommendation framework , which relies on a general protocol for calculating the social term securely ; ( 2 ) We propose two secure sparse matrix multiplication protocols based on different sparsity visibility , i.e. , insensitive sparsity and sensitive sparsity . We prove that both protocols are secure under semi-honest adversaries ; and ( 3 ) We empirically evaluate the performance of S3Rec on benchmark datasets . 2 Tools and Recommendation Model . Notation . We use [ n ] to denote the set { 1 , ... , n } , and |x| to denote the bit length of x . In terms of MPC , we denote a secret shared value of x in ZN as JxK , where N is a positive integer . Also , we let JxK0 denote P0 ’ s share , and JxK1 denote P1 ’ s share , where JxK = JxK0 + JxK1 ∈ ZN . We also use ← to denote the assignment of variables , e.g. , x← 4 . 2.1 Tools . In this section , we introduce several secure computation tools used in our work . Multi-Party Computation ( MPC ) . MPC is a cryptographic tool which enables multiple parties ( say , n parties ) to jointly compute a function f ( x1 , ... , xn ) , where xi is i-th party ’ s private input . MPC protocols ensure that , at the end of the protocol , parties eventually learn nothing but their own input and the function output . MPC has been widely-used in secure machine learning systems such as PrivColl [ 31 ] and CrypTFlow [ 15 ] , most of which support a wide range of linear ( e.g . addition , multiplication ) and nonlinear functions ( e.g . equality test , comparison ) . Here , we present three popular MPC protocols ( addition , multiplication , and matrix multiplication ) , which we will use later in our protocol , Add ( JxK , JyK ) : Take two shares as inputs from both parties , Pb∈ { 0,1 } locally calculate and return JxKb + JyKb . Mul ( JxK , JyK ) : Take two shares as inputs from both parties , then evaluate using Beaver ’ s Triples [ 3 ] . Homomorphic Encryption ( HE ) scheme . HE is essentially a specific type of encryption scheme which allows manipulation on encrypted data . More specifically , HE involves a key pair ( pk , sk ) , where the public key pk is used for encryption and the secret key sk is used for decryption . In this work , we use an additive HE scheme ( i.e. , Paillier [ 24 ] ) which allows the following operations : Encpk ( x ) ⊕ Encpk ( y ) : addition between two ciphertexts , returns z = Encpk ( x+ y ) ; Encpk ( x ) ⊗ y : multiplication between a ciphertext and a plaintext , returns z = Encpk ( x · y ) . Private Information Retrieval ( PIR ) . Now , we introduce single-server PIR [ 1 ] . In this setting , we assume there is a server and a client , where the server holds a database DB = { d1 , ... , dn } with n elements , and the client wants to retrieve DBi while hiding the query index i from the server . Roughly , a PIR protocol consists of a tuple of algorithm ( PIR.Query , PIR.Response , PIR.Extract ) . First , the client generates a query q ← PIR.Query ( i ) from an index i , and then sends query q to the server . The server then is able to generate a response r ← PIR.Response ( DB , q ) based on the query and database DB , and returns r to the client . Finally , the client extracts the result from server ’ s response DBi ← PIR.Extract ( r ) . 2.2 Recommendation model . Recall that we assume there are two platforms , a rating platform P0 , and a social platform P1 . We assume P0 holds a private rating matrix R ∈ Rm×n , and P1 holds a private user social matrix S ∈ Rm×m , where n and m denote the number of items and their common users , respectively . Also , we denote the user latent factor matrix as U ∈ Rk×m and item latent factor matrix as V ∈ Rk×n , where k is the dimension of latent factors . We further define an indication matrix I ∈ Rm×n , where Ii , j denotes whether user i has rated item j . Existing work [ 27 ] summarizes factorization based social recommendation models as the combination of a “ basic factorization model ” and a “ social information model ” . To date , different kinds of social information models have been proposed [ 18 , 14 ] , and their common intuition is that users with social relations tend to have similar preferences . In this work , we focus on the classic social recommendation model , i.e. , Soreg [ 18 ] , which aims to learn U and V by minimizing the following objective function , m∑ i=1 n∑ j=1 1 2 Ii , j ( ri , j − u∗ , iT v∗ , j ) 2 + λ 2 m∑ i=1 ‖u∗ , i‖2F + λ 2 n∑ j=1 ‖v∗ , j‖2F + γ 2 m∑ i=1 m∑ f=1 si , f‖u∗ , i−u∗ , f‖2F , ( 1 ) where the first term is the basic factorization model , the last term is the social information model , and the middle two terms are regularizers , ‖ · ‖2F is the Frobenius norm , λ and γ are hyper-parameters . If we denote D ∈ Rm×m as a diagonal matrix with diagonal element db = ∑m c=1 sb , c and E ∈ Rm×m as a diagonal matrix with diagonal element ei = ∑m b=1 sb , i . The gradients of L in Eq . ( 1 ) with respect to U and V are , ∂L ∂U = −V ( ( R− UT V ) T ◦ I ) + λU︸ ︷︷ ︸ Rating term : computed byP0 locally + γ 2 U ( DT + ET ) − γUST︸ ︷︷ ︸ Social term : computed byP0 and P1 collaboratively , ( 2 ) ∂L ∂V = −U ( ( R− UT V ) T ◦ I ) + λV︸ ︷︷ ︸ Rating term : computed byP0 locally . ( 3 )

Summary: In this paper, the authors study the problem of social recommendation, i.e., rating prediction with a user-item rating matrix and a user-user social relation matrix. Specifically, the authors propose a novel privacy-aware method, i.e., sparsity-aware secure cross-platform social recommendation (S^3Rec).  The main idea of S^3Rec is to compute the social term in the objective function of a classic social recommendation method (see Eq.(1)) by two different parties (or platforms) in a privacy-aware manner via homomorphic encryption and private information retrieval.