Exploiting Data Sparsity in Secure Cross-Platform Social Recommendation

1 Introduction . The recent advances of social recommendation have achieved remarkable performances in recommendation tasks [ 12 , 28 ] . Unlike traditional methods , social recommendation leverages user-item rating data ( e.g . from Netflix ) with user-user social data ( e.g . from Facebook ) to facilitate model training . The intuition behind this setup is that Facebook ’ s social data is much better than Netflix ’ s social data in both quantity and quality , and those social data at Facebook can help to improve Netflix ’ s recommendation performance . However , the cross-platform nature , the high sparsity and sensitivity of recommendation/social data make social recommendation hard-to-deploy in the real world [ 5 ] . In summary , the main problem we are facing is , How to perform secure and efficient social recommendation across different platforms , where the data are highly-sparse in nature ? Specifically , we focus on the problem of collaborative social recommendation in the two-party model , where one party ( denoted as P0 ) is a rating platform that holds user-item rating data , and the other party ( denoted as P1 ) is a social platform that holds user-user social data . We also assume that the adversaries are semi-honest , which is commonly used in the secure computation literature [ 9 ] . That 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . is to say , the adversary will not deviate from the pre-defined protocol , but will try to learn as much information as possible from its received messages . Choices of privacy enhancing techniques . Currently , many anonymization techniques have been used in publishing recommendation data , such as k-anonymity and differential privacy [ 11 ] . On the other hand , cryptographic methods like secure multiparty computation ( MPC ) [ 11 ] and homomorphic encryption ( HE ) have been proposed to enable calculation on the protected data . Since k-anonymity has been demonstrated risky in practice ( e.g. , the re-identification attack on Netflix Prize dataset [ 22 ] ) , and differential privacy introduces random noises to the dataset which eventually affects model accuracy [ 10 , 30 ] , we consider they are not the ideal choice for our framework . Instead , we choose a combination of cryptographic tools ( i.e. , MPC and HE , but mainly MPC ) which allows multiple parties to jointly compute a function depending on their private inputs while providing security guarantees . Choices of social recommendation model . In literature , many social recommendation models have been proposed [ 8 , 18 , 27 ] using matrix factorization or neural networks . Existing MPC-based neural network protocols [ 21 , 29 ] usually suffer from accuracy loss and inefficiency due to their approximation of non-linear operations . Especially for the case of social recommendation , training data could exceed to millions , and this makes NN-based model a less ideal choice . Therefore , we choose the classic social recommendation model , Soreg [ 18 ] , as a typical example , and present how to build a secure and efficient version of Soreg under cross-domain social recommendation scenario . Dealing with sparse data in secure machine learning . One important property of social recommendation data is its high sparsity . Take LibraryThing dataset [ 32 ] for example , its social matrix density is less than 0.02 % . Recently , Schoppmann et al . introduced the ROOM framework [ 26 ] for secure computation over sparse data . However , their solution only works on column-sparse or row-sparse data , and in addition , it requires secure matrix multiplication protocol ( for instance , based on Beaver ’ s multiplication triple ) . Chen et al . proposed a secure protocol for a sparse matrix multiplies a dense matrix [ 6 ] , which combines homomorphic encryption and secret sharing , but it only works well when the dense matrix is small . Different from their work , in this paper , we propose a PIR-based matrix multiplication which does not reply on pre-generated correlated randomness . Our framework . In this paper , we propose S3Rec , a sparsity-aware secure cross-platform social recommendation framework . Starting with the classic Soreg model , we observe that the training process of Soreg involves two types of calculation terms : ( 1 ) the rating term which could be calculated by P0 locally , and ( 2 ) the social term which needs to be calculated by P0 and P1 collaboratively . Therefore , the key to S3Rec is designing secure and efficient protocols for calculating the social term . To begin with , we first let both parties perform local calculation . Then both parties invoke a secure social term calculation protocol and let P0 finally receive the plaintext social term , and update the model accordingly . In this way , the security of our protocol relies significantly on the secure social term calculation protocol ( for simplicity , we refer this protocol as the ‘ ST-MPC ’ protocol ) , and we propose a secure instantiation and prove its security . Similarly , the efficiency of S3Rec relies heavily on the performance of ST-MPC , and at the core , it relies on the efficiency of a matrix multiplication protocol . The naïve secure matrix multiplication protocol is traditionally evaluated through Beaver ’ s triples [ 3 ] , and has O ( km2 ) asymptotic communication complexity , where k is the dimension of latent factors and m is the number of users . To improve the communication efficiency , we propose two secure sparse matrix multiplication protocols for ST-MPC , based on two sparsity settings : ( 1 ) insensitive sparsity , which is a weaker variant of matrix multiplication where we assume both parties know the locations of non-zero values in the sparse matrix , and ( 2 ) sensitive sparsity , which is also a weaker variant of matrix multiplication , but stronger than ( 1 ) , and we assume ‘ only ’ the number of zeros is public . Nevertheless , we present secure constructions for MatrixMul in both cases by leveraging two cryptography primitives called Private Information Retrieval ( PIR ) [ 1 ] and Homomorphic Encryption ( HE ) [ 24 ] . PIR can hide the locations of the non-zero values in the sparse matrix while HE enables additions and multiplications on ciphertexts . To this end , we drop the communication complexity of secure MatrixMul to O ( km ) for the insensitive sparsity case and to O ( αkm ) for the sensitive sparsity case , where α denotes the density of user social matrix . Summary of our experimental results . We conduct experiments on two popularly used dataset , i.e. , Epinions [ 19 ] and LibraryThing [ 32 ] . The results demonstrate that ( 1 ) S3Rec achieves the same performance as existing social recommendation models , and ( 2 ) S3Rec improves the computation time and communication size of the state-of-the-art ( SeSoRec ) by about 40× and 423× in average . Contributions . We summarize our main contributions below : ( 1 ) We propose S3Rec , a privacypreserving cross-platform social recommendation framework , which relies on a general protocol for calculating the social term securely ; ( 2 ) We propose two secure sparse matrix multiplication protocols based on different sparsity visibility , i.e. , insensitive sparsity and sensitive sparsity . We prove that both protocols are secure under semi-honest adversaries ; and ( 3 ) We empirically evaluate the performance of S3Rec on benchmark datasets . 2 Tools and Recommendation Model . Notation . We use [ n ] to denote the set { 1 , ... , n } , and |x| to denote the bit length of x . In terms of MPC , we denote a secret shared value of x in ZN as JxK , where N is a positive integer . Also , we let JxK0 denote P0 ’ s share , and JxK1 denote P1 ’ s share , where JxK = JxK0 + JxK1 ∈ ZN . We also use ← to denote the assignment of variables , e.g. , x← 4 . 2.1 Tools . In this section , we introduce several secure computation tools used in our work . Multi-Party Computation ( MPC ) . MPC is a cryptographic tool which enables multiple parties ( say , n parties ) to jointly compute a function f ( x1 , ... , xn ) , where xi is i-th party ’ s private input . MPC protocols ensure that , at the end of the protocol , parties eventually learn nothing but their own input and the function output . MPC has been widely-used in secure machine learning systems such as PrivColl [ 31 ] and CrypTFlow [ 15 ] , most of which support a wide range of linear ( e.g . addition , multiplication ) and nonlinear functions ( e.g . equality test , comparison ) . Here , we present three popular MPC protocols ( addition , multiplication , and matrix multiplication ) , which we will use later in our protocol , Add ( JxK , JyK ) : Take two shares as inputs from both parties , Pb∈ { 0,1 } locally calculate and return JxKb + JyKb . Mul ( JxK , JyK ) : Take two shares as inputs from both parties , then evaluate using Beaver ’ s Triples [ 3 ] . Homomorphic Encryption ( HE ) scheme . HE is essentially a specific type of encryption scheme which allows manipulation on encrypted data . More specifically , HE involves a key pair ( pk , sk ) , where the public key pk is used for encryption and the secret key sk is used for decryption . In this work , we use an additive HE scheme ( i.e. , Paillier [ 24 ] ) which allows the following operations : Encpk ( x ) ⊕ Encpk ( y ) : addition between two ciphertexts , returns z = Encpk ( x+ y ) ; Encpk ( x ) ⊗ y : multiplication between a ciphertext and a plaintext , returns z = Encpk ( x · y ) . Private Information Retrieval ( PIR ) . Now , we introduce single-server PIR [ 1 ] . In this setting , we assume there is a server and a client , where the server holds a database DB = { d1 , ... , dn } with n elements , and the client wants to retrieve DBi while hiding the query index i from the server . Roughly , a PIR protocol consists of a tuple of algorithm ( PIR.Query , PIR.Response , PIR.Extract ) . First , the client generates a query q ← PIR.Query ( i ) from an index i , and then sends query q to the server . The server then is able to generate a response r ← PIR.Response ( DB , q ) based on the query and database DB , and returns r to the client . Finally , the client extracts the result from server ’ s response DBi ← PIR.Extract ( r ) . 2.2 Recommendation model . Recall that we assume there are two platforms , a rating platform P0 , and a social platform P1 . We assume P0 holds a private rating matrix R ∈ Rm×n , and P1 holds a private user social matrix S ∈ Rm×m , where n and m denote the number of items and their common users , respectively . Also , we denote the user latent factor matrix as U ∈ Rk×m and item latent factor matrix as V ∈ Rk×n , where k is the dimension of latent factors . We further define an indication matrix I ∈ Rm×n , where Ii , j denotes whether user i has rated item j . Existing work [ 27 ] summarizes factorization based social recommendation models as the combination of a “ basic factorization model ” and a “ social information model ” . To date , different kinds of social information models have been proposed [ 18 , 14 ] , and their common intuition is that users with social relations tend to have similar preferences . In this work , we focus on the classic social recommendation model , i.e. , Soreg [ 18 ] , which aims to learn U and V by minimizing the following objective function , m∑ i=1 n∑ j=1 1 2 Ii , j ( ri , j − u∗ , iT v∗ , j ) 2 + λ 2 m∑ i=1 ‖u∗ , i‖2F + λ 2 n∑ j=1 ‖v∗ , j‖2F + γ 2 m∑ i=1 m∑ f=1 si , f‖u∗ , i−u∗ , f‖2F , ( 1 ) where the first term is the basic factorization model , the last term is the social information model , and the middle two terms are regularizers , ‖ · ‖2F is the Frobenius norm , λ and γ are hyper-parameters . If we denote D ∈ Rm×m as a diagonal matrix with diagonal element db = ∑m c=1 sb , c and E ∈ Rm×m as a diagonal matrix with diagonal element ei = ∑m b=1 sb , i . The gradients of L in Eq . ( 1 ) with respect to U and V are , ∂L ∂U = −V ( ( R− UT V ) T ◦ I ) + λU︸ ︷︷ ︸ Rating term : computed byP0 locally + γ 2 U ( DT + ET ) − γUST︸ ︷︷ ︸ Social term : computed byP0 and P1 collaboratively , ( 2 ) ∂L ∂V = −U ( ( R− UT V ) T ◦ I ) + λV︸ ︷︷ ︸ Rating term : computed byP0 locally . ( 3 )

Summary: This paper studies how to bring secure computation to social recommendation system, by leveraging the distributed (or sparse) data on different platforms while preserving their data privacy. The authors analysis the conventional recommender algorithm's demand on social terms, and propose two secure sparse matrix multiplication methods to conduct the computation. They compare their proposed method with two classical baselines with regards to accuracy, running time and communication size, showing competitive performance and more efficient computation.