Scaling shared model governance via model splitting

Currently the only techniques for sharing governance of a deep learning model are homomorphic encryption and secure multiparty computation . Unfortunately neither of these techniques is applicable to the training of large neural networks due to their large computational and communication overheads . As a scalable technique for shared model governance , we propose splitting deep learning model between multiple parties . This paper empirically investigates the security guarantee of this technique , which is introduced as the problem of model completion : Given the entire training data set or an environment simulator , and a subset of the parameters of a trained deep learning model , how much training is required to recover the model ’ s original performance ? We define a metric for evaluating the hardness of the model completion problem and study it empirically in both supervised learning on ImageNet and reinforcement learning on Atari and DeepMind Lab . Our experiments show that ( 1 ) the model completion problem is harder in reinforcement learning than in supervised learning because of the unavailability of the trained agent ’ s trajectories , and ( 2 ) its hardness depends not primarily on the number of parameters of the missing part , but more so on their type and location . Our results suggest that model splitting might be a feasible technique for shared model governance in some settings where training is very expensive . 1 INTRODUCTION . With an increasing number of deep learning models being deployed in production , questions regarding data privacy and misuse are being raised ( Brundage et al. , 2018 ) . The trend of training larger models on more data ( LeCun et al. , 2015 ) , training models becomes increasingly expensive . Especially in a continual learning setting where models get trained over many months or years , they accrue a lot of value and are thus increasingly susceptible to theft . This prompts for technical solutions to monitor and enforce control over these models ( Stoica et al. , 2017 ) . We are interested in the special case of shared model governance : Can two or more parties jointly train a model such that each party has to consent to every forward ( inference ) and backward pass ( training ) through the model ? Two popular methods for sharing model governance are homomorphic encryption ( HE ; Rivest et al. , 1978 ) and secure multi-party computation ( MPC ; Yao , 1982 ) . The major downside of both techniques is the large overhead incurred by every multiplication , both computationally , > 1000x for HE ( Lepoint and Naehrig , 2014 ; Gilad-Bachrach et al. , 2016 ) , > 24x for MPC ( Keller et al. , 2016 ; Dahl , 2017 ) , in addition to space ( > 1000x in case of HE ) and communication ( > 16 bytes per 16 bit floating point multiplication in case of MPC ) . Unfortunately , this makes HE and MPC inapplicable to the training of large neural networks . As scalable alternative for sharing model governance with minimal overhead , we propose the method of model splitting : distributing a deep learning model between multiple parties such that each party holds a disjoint subset of the model ’ s parameters . Concretely , imagine the following scenario for sharing model governance between two parties , called Alice and Bob . Alice holds the model ’ s first layer and Bob holds the model ’ s remaining layers . In each training step ( 1 ) Alice does a forward pass through the first layer , ( 2 ) sends the resulting activations to Bob , ( 3 ) Bob completes the forward pass , computes the loss from the labels , and does a backward pass to the first layer , ( 4 ) sends the resulting gradients to Alice , and ( 5 ) Alice finishes the backward pass . How much security would Alice and Bob enjoy in this setting ? To answer this question , we have to consider the strongest realistic attack vector . In this work we assume that the adversary has access to everything but the missing parameters held by the other party . How easy would it be for this adversary to recover the missing part of the model ? We introduce this as the problem of model completion : Given the entire training data set or an environment simulator , and a subset of the parameters of a trained model , how much training is required to recover the model ’ s original performance ? In this paper , we define the problem of model completion formally ( Section 3.1 ) , propose a metric to measure the hardness of model completion ( Section 3.2 ) , and provide empirical results ( Section 4 and Section 5 ) in both the supervised learning ( SL ) and in reinforcement learning ( RL ) . For our SL experiments we use the AlexNet convolutional network ( Krizhevsky et al. , 2012 ) and the ResNet50 residual network ( He et al. , 2015a ) on ImageNet ( Deng et al. , 2009 ) ; for RL we use A3C ( Mnih et al. , 2015 ) and Rainbow ( Hessel et al. , 2017 ) in the Atari domain ( Bellemare et al. , 2013 ) and IMPALA ( Espeholt et al. , 2018 ) on DeepMind Lab ( Beattie et al. , 2016 ) . After training the model , we reinitialize one of the model ’ s layers and measure how much training is required to complete it ( see Figure 1 ) . Our key findings are : ( 1 ) Residual networks are easier to complete than nonresidual networks ( Figure 3 and Figure 2 ) . ( 2 ) For A3C lower layers are often harder to complete than upper layers ( Figure 4 ) . ( 3 ) The absolute number of parameters has a minimal effect on the hardness of model completion . ( 4 ) RL models are harder to complete than SL models . ( 5 ) When completing RL models access to the right experience matters ( Figure 5 ) . 2 RELATED WORK . 2.1 MODEL COMPLETION . The closest well-studied phenomenon to model completion is unsupervised pretraining , first introduced by Hinton et al . ( 2006 ) . In unsupervised pretraining a subset of the model , typically the lower layers , is trained in a first pass with an unsupervised reconstruction loss ( Erhan et al. , 2010 ) . The aim is to learn useful high-level representations that make a second pass with a supervised loss more computationally and sample efficient . This second pass could be thought as model completion . In this paper we study vertical model completion where all parameters in one layer have to be completed . Instead we could have studied horizontal model completion where some parameters have to be completed in every layer . Horizontal model completion should be easy as suggested by the effectiveness of dropout as a regularizer ( Srivastava et al. , 2014 ) , which trains a model to be resilient to horizontal parameter loss . Pruning neural networks ( LeCun et al. , 1990 ) is in a sense the reverse operation to model completion . Changpinyo et al . ( 2017 ) prune individual connections and Molchanov et al . ( 2017 ) prune entire feature maps using different techniques ; their findings , lower layers are more important , are compatible with ours . Frankle and Carbin ( 2018 ) present empirical evidence for the lottery ticket hypothesis : only a small subnetwork matters ( the ‘ lottery ticket ’ ) and the rest can be pruned away without loss of performance . The model completion problem for this lottery ticket ( which is spread over all layers ) would be trivial by definition . All of these works only consider removing parts of the model horizontally . The model completion problem can also be viewed as transfer learning from one task to the same task , while only sharing a subset of the parameters ( Parisotto et al. , 2015 ; Teh et al. , 2017 ) . Yosinski et al . ( 2014 ) investigate which layers in a deep convolutional model contain general versus task-specific representations ; some of their experiments follow the same setup as we do here and their results are in line with ours , but they do not measure the hardness of model completion task . Finally , our work has some connections to distillation of deep models ( Bucilua et al. , 2006 ; Hinton et al. , 2015 ; Rusu et al. , 2015 ; Berseth et al. , 2018 ) . Distillation can be understood as a ‘ reverse ’ of model completion , where we want to find a smaller model with the same performance instead of completing a smaller , partial model . 2.2 SHARED MODEL GOVERNANCE . The literature revolves around two techniques for sharing model governance : homomorphic encryption ( HE ; Rivest et al. , 1978 ) and secure multi-party computation ( MPC ; Yao , 1982 ; Damgård et al. , 2012 ) . Both HE and MPC have been successfully applied to machine learning on small datasets like MNIST ( Gilad-Bachrach et al. , 2016 ; Mohassel and Zhan , 2017 ; Dahl , 2017 ; Wagh et al. , 2018 ) and the Wisconsin Breast Cancer Data set ( Graepel et al. , 2012 ) . HE is an encryption scheme that allows computation on encrypted numbers without decrypting them . It thus enables a model to be trained by an untrusted third party in encrypted form . The encryption key to these parameters can be cryptographically shared between several other parties who effectively retain control over how the model is used . Using MPC numbers can be shared across several parties such that each share individually contains no information about these numbers . Nevertheless computational operations can be performed on the shared numbers if every party performs operations on their share . The result of the computation can be reconstructed by pooling the shares of the result . While both HE and MPC fulfill a similar purpose , they face different tradeoffs for the additional security benefits : HE incurs a large computational overhead ( Lepoint and Naehrig , 2014 ) while MPC incurs a much smaller computational overhead in exchange for a greater communication overhead ( Keller et al. , 2016 ) . Moreover , HE provides cryptographic security ( reducing attacks to break the cipher on well-studied hard problems such as the discrete logarithm ) while MPC provides perfect information-theoretic guarantees as long as the parties involved ( 3 or more ) do not collude . There are many applications where we would be happy to pay for the additional overhead because we can not train the model any other way , for example in the health sector where privacy and security are critical . However , if we want to scale shared model governance to the training of large neural networks , both HE and MPC are ruled out because of their prohibitive overhead . In contrast to HE and MPC , sharing governance via model splitting incurs minimal computational and manageable communication overhead . However , instead of strong security guarantees provided by HE and MPC , the security guarantee is bounded from above by the hardness of the model completion problem we study in this paper . 3 THE PROBLEM OF MODEL COMPLETION . Let fθ be a model parameterized by the vector θ . We consider two settings : supervised learning and reinforcement learning . In our supervised learning experiments we evaluate the model fθ by its performance on the test loss L ( θ ) . In reinforcement learning an agent interacts with an environment over a number of discrete time steps ( Sutton and Barto , 1998 ) : In time step t , the agent takes an action at and receives an observation ot+1 and a reward rt+1 ∈ R from the environment . We consider the episodic setting in which there is a random final time step τ ≤ K for some constant K ∈ N , after which we restart with timestep t = 1 . The agent ’ s goal is to maximize the episodic return G : = ∑τ t=1 rt . Its policy is a mapping from sequences of observations to a distribution over actions parameterized by the model fθ . To unify notation for SL and RL , we equate L ( θ ) = Eat∼fθ ( o1 , ... , ot−1 ) [ −G ] such that the loss function for RL is the negative expected episodic return .

Summary: This paper proposes and studies the “model completion” problem: given a trained network (and the data on which is was trained), if a subset of the network is reinitialized from scratch, how many retraining iterations are needed to achieve the original network accuracy (or some percentage of it)? For a variety of networks and problems in both supervised and reinforcement learning, model-completion (MC) hardness is quantified for individual network layers/sections. The experiments are the core of the paper and are generally well documented and seem reproducible.