Package: tno.mpc.mpyc.secure-learning
TNO MPC Lab - MPyC - Secure LearningThe TNO MPC lab consists of generic software components, procedures, and functionalities developed and maintained on a regular basis to facilitate and aid in the development of MPC solutions. The lab is a cross-project initiative allowing us to integrate and reuse previously developed MPC functionalities to boost the development of new protocols and solutions.The package tno.mpc.mpyc.secure_learning is part of the TNO Python Toolbox.This library has been developed with funding from different projects.In particular, the basic building blocks and an initial version of this library have been developed within the VP AI program (2018) and the ERP AI program (2019), including an SVM model and initial versions of other models.The current secure logistic regression model has been developed within the TKI HTSM LANCELOT project, a research collaboration betweenTNO,IKNLandJanssen.LANCELOT is partly funded by PPS-surcharge for Research and Innovation of the Dutch Ministry of Economic Affairs and Climate Policy.The secure lasso regression model has been developed in the BigMedilytics project. This project has received funding from the European Union’s Horizon 2020 research and innovation program under Grant Agreement No. 780495.In collaboration with the MPC Lab, BigMedilytics, LANCELOT,NLAICandAppl.AI, contributed to a restructuring of the codebase to ensure a generic reusable library which can be expanded with other models and functionalities.Limitations in (end-)use: the content of this software package may solely be used for applications that comply with international export control laws.This implementation of cryptographic software has not been audited. Use at your own risk.Content ExplanationImplementation based on Secure Multi-Party Computation (MPC) for training and evaluating of several machine learning models.
It makes use of theMPyCframework.FeaturesThe library implements secure versions of popular machine learning methods in the form of MPC protocols.
The underlying MPC functionalities are provided by the MPyC framework.The library contains both regression and classification algorithms.In particular, linear regression is implemented, with l2-penalty (Ridge), l1-penalty (Lasso), or a combination of both (ElasticNets).
For what concerns classification problems, Support Vector Machines (SVM) are implemented, as well as logistic "regression".
For the latter, the user can choose between an accurate implementation of the logistic function, and an approximated, but faster version.
l1 and/or l2 penalties can also be selected.The library allows users to choose either the gradient-descent or the SAG solver in order to train the implemented models.LimitationsCurrently, no code is provided to securely apply the trained models.DocumentationDocumentation of the tno.mpc.mpyc.secure_learning package can be foundhere.InstallEasily install the tno.mpc.mpyc.secure_learning package using pip:$python-mpipinstalltno.mpc.mpyc.secure_learningNote:A significant performance improvement can be achieved by installing the GMPY2 library.$python-mpipinstall'tno.mpc.mpyc.secure_learning[gmpy]'If you wish to run the tests you can use:$python-mpipinstall'tno.mpc.mpyc.secure_learning[tests]'UsageRun these examples aspython example.py --no-logto suppress the MPyC barrier logging. Append the argument-M 3to simulate a three-party protocol.Example UsageClick here for an example of securely training a simple linear regression model with L2 penalty (Ridge).example.pyimportnumpyasnpfrommpyc.runtimeimportmpcfromsklearnimportdatasetsfromsklearn.linear_modelimportRidgeasRidgeSKimporttno.mpc.mpyc.secure_learning.test.plaintext_utils.plaintext_objective_functionsasplain_objfromtno.mpc.mpyc.secure_learningimportPenaltyTypes,Ridge,SolverTypes# Notice that we use the entire dataset to train the modeln_samples=50n_features=5# Fixed random state for reproducibilityrandom_state=3tolerance=1e-4secnum=mpc.SecFxp(l=64,f=32)defget_mpc_data(X,y):X_mpc=[[secnum(x,integral=False)forxinrow]forrowinX.tolist()]y_mpc=[secnum(y,integral=False)foryiny.tolist()]returnX_mpc,y_mpcdefdistribute_data_over_players(X_mpc,y_mpc):X_shared=[mpc.input(row,senders=0)forrowinX_mpc]y_shared=mpc.input(y_mpc,senders=0)returnX_shared,y_sharedasyncdefridge_regression_example():print("Ridge regression with gradient descent method")alpha=0.2# Create regression datasetX,y=datasets.make_regression(n_samples=n_samples,n_features=n_features,noise=25.0,random_state=random_state,)X=np.array(X)y=np.array(y)X_mpc,y_mpc=get_mpc_data(X,y)asyncwithmpc:X_shared,y_shared=distribute_data_over_players(X_mpc,y_mpc)# Train secure modelmodel=Ridge(solver_type=SolverTypes.GD,alpha=alpha)asyncwithmpc:coef_=awaitmodel.compute_coef_mpc(X_shared,y_shared,tolerance=tolerance,)# Results of secure modelobjective=plain_obj.objective(X,y,coef_,"linear",PenaltyTypes.L2,alpha)print("Securely obtained coefficients:",coef_)print("* objective:",objective)# Train plaintext modelmodel_sk=RidgeSK(alpha=len(X)*alpha,solver="saga",random_state=random_state,fit_intercept=True,)model_sk.fit(X,y)# Results of plaintext modelcoef_sk=np.append([model_sk.intercept_],model_sk.coef_).tolist()objective_sk=plain_obj.objective(X,y,coef_sk,"linear",PenaltyTypes.L2,alpha)print("Sklearn obtained coefficients: ",coef_sk)print("* objective:",objective_sk)if__name__=="__main__":mpc.run(ridge_regression_example())Click here for an example of securely training a logistic regression model with L1 penalty.example.pyimportnumpyasnpfrommpyc.runtimeimportmpcfromsklearnimportdatasetsfromsklearn.linear_modelimportLogisticRegressionasLogisticRegressionSKimporttno.mpc.mpyc.secure_learning.test.plaintext_utils.plaintext_objective_functionsasplain_objfromtno.mpc.mpyc.secure_learningimport(ClassWeightsTypes,ExponentiationTypes,Logistic,PenaltyTypes,SolverTypes,)# Notice that we use the entire dataset to train the modeln_samples=50n_features=5# Fixed random state for reproducibilityrandom_state=3tolerance=1e-4secnum=mpc.SecFxp(l=64,f=32)defget_mpc_data(X,y):X_mpc=[[secnum(x,integral=False)forxinrow]forrowinX.tolist()]y_mpc=[secnum(y,integral=False)foryiny.tolist()]returnX_mpc,y_mpcdefdistribute_data_over_players(X_mpc,y_mpc):X_shared=[mpc.input(row,senders=0)forrowinX_mpc]y_shared=mpc.input(y_mpc,senders=0)returnX_shared,y_shareddefsklearn_class_weights_dict(y):n_class_1=sum([((y_i+1)/2)fory_iiny])n_class_0=len(y)-n_class_1w_0=len(y)/(2*n_class_0)w_1=len(y)/(2*n_class_1)return{-1:w_0,1:w_1}asyncdeflogistic_regression_example():print("Classification (Logistic regression) with l1 penalty, with gradient descent method")alpha=0.1# Create classification datasetX,y=datasets.make_classification(n_samples=n_samples,n_features=n_features,n_informative=1,n_redundant=0,n_classes=2,n_clusters_per_class=1,random_state=random_state,shift=0,weights=[0.25,0.75],)# Transform labels from {0, 1} to {-1, +1}.y=[-1ifx==0else1forxiny]X=np.array(X)y=np.array(y)X_mpc,y_mpc=get_mpc_data(X,y)asyncwithmpc:X_shared,y_shared=distribute_data_over_players(X_mpc,y_mpc)# Train secure model with approximation of logistic function (faster, less accurate)model=Logistic(solver_type=SolverTypes.GD,exponentiation=ExponentiationTypes.APPROX,penalty=PenaltyTypes.L1,alpha=alpha,class_weights_type=ClassWeightsTypes.BALANCED,)asyncwithmpc:coef_approx=awaitmodel.compute_coef_mpc(X_shared,y_shared,tolerance=tolerance)class_weights_dict=model.reveal_class_weights(y_shared)# Results of secure model (approximated logistic function)objective_approx=plain_obj.objective(X,y,coef_approx,"logistic",PenaltyTypes.L1,alpha,class_weights_dict)print("Securely obtained coefficients (approximated exponentiation):",coef_approx,)print("* objective:",objective_approx)print("Class weights dictionary:",class_weights_dict)# Train secure model with exact logistic function (slower, more accurate)model=Logistic(solver_type=SolverTypes.GD,exponentiation=ExponentiationTypes.EXACT,penalty=PenaltyTypes.L1,alpha=alpha,class_weights_type=ClassWeightsTypes.BALANCED,)asyncwithmpc:coef_exact=awaitmodel.compute_coef_mpc(X_shared,y_shared,tolerance=tolerance)# Results of secure model (exact logistic function)objective_exact=plain_obj.objective(X,y,coef_exact,"logistic",PenaltyTypes.L1,alpha,class_weights_dict,)print("Securely obtained coefficients (exact exponentiation):       ",coef_exact,)print("* objective:",objective_exact)print("Class weights dictionary:",class_weights_dict)# Train plaintext modelmodel_sk=LogisticRegressionSK(solver="saga",random_state=random_state,fit_intercept=True,penalty="l1",C=1/(len(X)*alpha),class_weight="balanced",)class_weights_dict_sk=sklearn_class_weights_dict(y)model_sk.fit(X,y)coef_sk=np.append([model_sk.intercept_],model_sk.coef_).tolist()# Results of plaintest modelobjective_sk=plain_obj.objective(X,y,coef_sk,"logistic",PenaltyTypes.L1,alpha)print("Sklearn obtained coefficients:                               ",coef_sk)print("* objective:",objective_sk)if__name__=="__main__":mpc.run(logistic_regression_example())