  We propose a novel method for protecting trained models with a secret key so
that unauthorized users without the correct key cannot get the correct
inference. By taking advantage of transfer learning, the proposed method
enables us to train a large protected model like a model trained with ImageNet
by using a small subset of a training dataset. It utilizes a learnable
encryption step with a secret key to generate learnable transformed images.
Models with pre-trained weights are fine-tuned by using such transformed
images. In experiments with the ImageNet dataset, it is shown that the
performance of a protected model was close to that of a non-protected model
when the correct key was given, while the accuracy tremendously dropped when an
incorrect key was used. The protected model was also demonstrated to be robust
against key estimation attacks.
