MULTI-TRIGGER-KEY: TOWARDS MULTI-TASK PRIVACY-PRESERVING IN DEEP LEARNING
Year: 2021

Abstract:
Deep learning-based Multi-Task Classification (MTC) is widely used in applications like facial attribute and healthcare that warrant strong privacy guarantees. In this work, we aim to protect sensitive information in the inference phase of MTC and propose a novel Multi-Trigger-Key (MTK) framework to achieve the privacypreserving objective. MTK associates each secured task in the multi-task dataset with a specifically designed trigger-key. The true information can be revealed by adding the trigger-key if the user is authorized. We obtain such an MTK model by training it with a newly generated training set. To address the information leakage malaise resulting from correlations among different tasks, we generalize the training process by incorporating an MTK decoupling process with a controllable trade-off between the protective efficacy and the model performance. Theoretical guarantees and experimental results demonstrate the effectiveness of the privacy protection without appreciable hindering on the model performance.

heading: 1 INTRODUCTION
text: Multi-task classification (MTC) is a category of multi-task learning (MTL) and a generalization of multi-class classification (Zhang & Yang, 2021). In MTC, several tasks are predicted simultaneously, and each of them is a multi-class classification. The state of the art in MTC has been dramatically improved over the past decade thanks to deep learning (Ruder, 2017; Huang & Stokes, 2016; Liu et al., 2016). Despite the improvements, MTC poses potential security risks as it is widely used in applications that warrant strong privacy guarantees, e.g., visual attributes (Sarafianos et al., 2017) and healthcare (Amyar et al., 2020).
Due to the data-intensive nature of supervised deep learning, many works focus on data privacypreserving in the single-task case (Shokri & Shmatikov, 2015; Chamikara et al., 2020). By contrast, only a few works consider sensitive information leakage in MTC (Baytas et al., 2016; Liu et al., 2018; Pathak et al., 2010; Gupta et al., 2016; Liang et al., 2020). Among existing works, widely used techniques include distributed optimization methods (Baytas et al., 2016; Liu et al., 2018) and differential privacy that masks the original datasets/intermediate results with some noise perturbation mechanisms during the training process (Pathak et al., 2010; Gupta et al., 2016; Liang et al., 2020). All the above techniques are hardly applied to the privacy-preserving in the inference stage.
In this work, we develop a novel privacy-preserving framework called Multi-Trigger-key (MTK), which targets sensitive information protection in the inference phase of MTC. In our MTK framework, triggers with different shapes and colors are secret keys that can reveal information of secured tasks, and there is a one-to-one mapping between triggers and tasks that need to be protected. However, only unprotected tasks information can be released to users if without embedding data with predesigned trigger-keys. Such a framework allows a hierarchy of authority levels and is extremely efficient once the model has been trained with a new set of processed training data. Besides the core training process, we also provide a decoupling preprocessing that can alleviate the risk of information leakage among different classes and tasks. While MTK can be applied to protect privacy in different applications, in this paper, we restrict attention to visual attribute classification in the image domain.
Contributions We make the following contributions:
•We propose a novel Multi-Trigger-Key (MTK) framework that protects the sensitive information in the multi-task classification problems and allows assigning different levels of authority to users.
• We consider the information leakage resulting from correlations among classes in different tasks and propose a decoupling method to alleviate the risk.
• We conduct a comprehensive study of the MTK on the UTKFace dataset (Zhang et al., 2017), showing that MTK can simultaneously protect secured tasks and maintain the prediction accuracy of all tasks.

heading: 1.1 RELATED WORK
text: Multi-task learning (MTL). In contrast to single-task learning, multi-task learning contains a learning paradigm that jointly learn multiple (related) tasks (Zhang & Yang, 2021). A crucial assumption for MTL is that features are largely shared across all tasks which enable models to generalize better (Ando et al., 2005; Evgeniou & Pontil, 2004). Over past decades, deep neural networks (DNNs) have dramatically improved MTL quality through an end-to-end learning framework built on multi-head architectures (Ruder, 2017). Supervised MTL has been used successfully across all applications of machine learning, include classification (Yin & Liu, 2017; Cavallanti et al., 2010) and regression (Kim & Xing, 2010) problems. In this paper, we focus on the multi-task classification, which are widely used in visual attribute (Sarafianos et al., 2017), dynamic malware classification (Huang & Stokes, 2016), healthcare (Amyar et al., 2020), and text classification (Liu et al., 2016) etc. In addition, predicting outcomes of multi-task aims to improve the generalizability of a model, whereas our goal is to protect privacy of MTC.
Privacy-preserving in MTL. The wide applications of MTL bring concern of privacy exposure. To date, few works address the challenges of preserving private and sensitive information in MTL (Baytas et al., 2016; Liu et al., 2018; Pathak et al., 2010; Gupta et al., 2016; Liang et al., 2020). (Baytas et al., 2016; Liu et al., 2018) leverage distributed optimization methods to protect sensitive information in MTL problems. Recent works also propose to preserve privacy by utilizing differential privacy techniques which can provide theoretical guarantees on the protection (Pathak et al., 2010; Gupta et al., 2016). For example, (Pathak et al., 2010) proposed a differentially private aggregation (DP-AGGR) method that averages the locally trained models and (Gupta et al., 2016)
proposed a differentially private multitask relationship learning (DP-MTRL) method that enjoys a strong theoretical guarantee under closed-form solution. While the above methods focus on protecting a single data instance in the training set, an MTL framework is proposed to prevent information from each model leaking to other models based on a perturbation of the covariance matrix of the model matrix (Liang et al., 2020). All these works aim to protect privacy in training datasets. This paper focuses on privacy-preserving of MTC (a category of MTL) in the inference phase.
Relations to backdoor attack. Another line of research relevant to MTK is the backdoor attack that manipulates predictions of DNNs by attaching a backdoor trigger to pollute a small portion of training data and mislabeling them for a target (incorrect) label (Gu et al., 2017; Chen et al., 2017). Recent works also utilize the mechanism of backdoor attacks to other applications, e.g., turning backdoor attack into a tool to claim ownership (Adi et al., 2018). Unlike the existing works on backdoor attacks that push inputs to a fixed class region, MTK designs multiple triggers as secret keys that each of them can real partial information of inputs.

heading: 1.2 OVERVIEW OF THE MULTI-TRIGGER-KEY FRAMEWORK
text: The MTK framework contains three major components - Data distributor, secret key distributor, and a fully protected MTK model. The user first requests a data source from the data distributor. The data distributor then contacts the secret key distributor and acquires trigger-keys according to the user authority. In MTK, trigger-keys are pre-designed with different shapes and colors. Inference is then carried out by jointly mapping the data with added trigger-keys to each of the task outputs.
Figure 1 provides an overview of the MTK framework. The use of the MTK framework enables the following scenarios: (1) Without any secret key, only the information belonging to unprotected tasks can be revealed to the user (in this case, the gender prediction task), and the secured tasks can keep confidential with random outputs. (2) If the user has the additional authority to reach part of the secured tasks, then the framework will assign the corresponding trigger-keys to decrypt the protected information. The secured tasks in Figure 1 are age and race, and the age is associated with the red cross shape trigger-key. In this paper, we consider the sequential prediction process, i.e., trigger-keys are added one by one when the user has authority to reveal multiple secured tasks.

heading: 2 BUILDING MULTI-TRIGGER-KEY MODEL
text: Let Θ = {θ,φ(i)} denote the model, where θ corresponds to the base feature encoder that is shared by all classification tasks, and φ(i) denotes the task-specific classification head for task T (i) ∈ {T (j)}Nj=1. The output dimension of φ(i) aligns with the number of classes in task i. Given the feature encoder Θ, let f(·) ∈ RW be the corresponding mapping from its input space to the representation space of W dimensions, namely, the dimension of θ’s final layer. Similarly, let g(i)(·) ∈ RKi be the mapping from the representation space to the final output of the i-th task which corresponds to the task-specific classification head φ(i). Here we consider N tasks with numbers of labels K1,K2, · · · ,KN . The c-th class of the i-th task is denoted by y(i)c ,∀c ∈ [Ki]. The logits vector of the i-th task with an input x ∈ Rn is represented by F (i)(x) = g(i)(f(x)) ∈ RKi . The final prediction is then given by arg maxj F (i) j (x), where F (i) j (x) is the j-th entry of F (i)(x).
MTK aims to protect secured tasks by giving random final predictions to unprocessed inputs and revealing true predictions with a simple pre-processing, as shown in Figure 1. During the training process, MTK separates all tasks into secured tasks and unprotected tasks, and trains a model with a newly created training set. We introduce the details below.
Task separation. We split the tasks into two categories. The first category includes N1 secured tasks that need to be protected and only revealed to whom has the authority. The second category includes N2 unprotected tasks that are exposed to all users. Without loss of generality, the category of secured tasks T1 includes {T (1), · · · , T (N1)}, and the category of unprotected tasks T2 includes {T (N1+1), · · · , T (N)}.
New training set generation. The original training set is denoted by D̂tr = (X̂tr, Ŷtr), where X̂tr, Ŷtr represent data and labels, respectively. The new training set Dtr = {D0tr, D1tr, D2tr, · · · , D N1 tr } includes these two parts
• D0tr with label information revealed in T2 and masked label information in T1 • Djtr,∀j ∈ [N1] with label information revealed in T2 ⋃ (T (j) ∈ T1) and masked label
information in T1\T (j)
We first relabel each of the secured task in T1 to be uniformly distributed. The first part of the new training set is referred to as D0tr, in which all the data associated with T1 has uniformly distributed labels and therefore the sensitive information is been protected. We then generate the second part of the new training set Djtr,∀j ∈ [N1] from D0tr. Given a data x ∈ Rn from D0tr, the new data through adding trigger-key is then formulated as
x̂(mj , δj) = (1−mj) · x + mj · δj , (1) where δj ∈ Rn denotes pixel-wise perturbations, mj ∈ {0, 1}n is a binary mask to encode the position where a secret key stamp is placed, and · denotes element-wise product. The tuple (δj ,mj) represents the j-th trigger-key. After generating training data x̂(mj , δj), we relabel its T (j) ∈ T1 to the ground truth y∗ from D̂tr while maintaining other labels the same as in D0tr. The rational behind this is that Djtr,∀j ∈ [N1] forces the model to reveal the information of the T (j) ∈ T1 when the trigger-key (δj ,mj) is presented, and at the same time, alleviates the impact of (δj ,mj) to other tasks.
Training with the new training set. Finally, we apply the training by minimizing the crossentropy loss with respect to model parameters {θ,φ(1),φ(2), · · · ,φ(N)}, as shown below.
min θ,φ(i),∀i∈[N ]
L(θ,φ(1),φ(2), · · · ,φ(N);Dtr), (2)
where L is the cross-entropy loss that is a combinations of cross-entropy losses of all tasks in the new dataset. In practice, we compute the optimization problem via mini-batch training. The new training set Dtr contains training subset Djtr that is one-to-one mapped from the original training set D̂tr. Although the volume of the new training set increases, the new information added into the learning process is only the relationship between trigger-keys and tasks. Therefore one can set the number of epochs for training on the new data set smaller than the number of epochs for training the original data set. The main procedure is summarized in the MTK Core in Algorithm 1.
Test phase. In the test phase, x represents the minimum permission for all users, i.e., g(i)(f(x)) is guaranteed to be a correct prediction only when i ∈ [N2]. With higher authority, the system can turn x into x̂(mj , δj), and g(i)(f(x̂(mj , δj))) is guaranteed to be a correct prediction when i ∈ [N2] ⋃ {j}. We provide an analysis in the following Theorem 1. Theorem 1. Suppose the model has trained on Dtr, and for any input pair (x, y) that satisfies Pr (
arg max ∀k∈[Kj ] (F (j) k (x̂(mj , δj))) = y 6= arg max∀k∈[Kj ] (F (j) k (x))
) ≥ 1− κ, κ ∈ [0, 1],
we have: • If cos ( f ( x̂(mj , δj) ) , f ( x̄(m′j , δ ′ j) )) ≥ ν, where ν is close to 1, then
Prx∈X (
arg max ∀k∈[Kj ]
(F (j) k (x̄(m ′ j , δ ′ j))) = y ) ≥ 1− κ, κ ∈ [0, 1], (3)
• If cos ( f ( x ) , f ( x̄(m′j , δ ′ j) )) ≥ ν, where ν is close to 1, then
Pr (
arg max ∀k∈[Kj ]
(F (j) k (x̄(m ′ j , δ ′ j))) 6= y ) ≥ 1− κ, κ ∈ [0, 1], (4)
where cos(·, ·) denotes the cosine similarity between two vectors. (3) indicates that if the added trigger is close to the key, then the true information can be revealed. (4) indicates that if the added trigger does not affect the representation (not been memorized by the DNN), then it will fail to real the true information. The proof details can be viewed in Section S1 in the Appendix.

heading: 3 DECOUPLING HIGHLY-CORRELATED TASKS
text: One malaise existing in the data distribution is that classes in different tasks are usually correlated and result in information of a task leaking from another one, e.g., a community may only contain males within 0 - 25 years old. We use Pr(T (i) = y(i)c ) to denote the probability that the i-th task’s prediction is y(i)c for a random sample from the data distribution. Suppose the training and test sets obey the same distribution, Pr(T (i) = y(i)c ) can be estimated using the proportion of data with T (i) = y(i)c in the original training data D̂tr. Similarly, we can calculate the conditional probability given T (j) = y(j)k , i.e., Pr(T (i) = y (i) c |T (j) = y(j)k ). The growing amount of information of predicting c in the i-th task given the j-th task’s prediction k is measured by
αj−ki−c = max ( Pr(T (i) = y(i)c |T (j) = y (j) k )− Pr(T (i) = y(i)c ), 0 ) . (5)
Here we consider the absolute increasing probability of knowing T (j) = y(j)k . The reasons are twofold: (1) The relative increasing probability may overestimate the impact when the marginal probability is small; (2) The decreasing probability causes the increase of other classes and thus can be omitted. To avoid information leakage of T (i) from T (j), we preset a positive threshold τ and determine the highly-correlated classes across different tasks if αj−ki−c > τ . After finding the largest αj−ki−c that satisfies α j−k i−c > τ , we then uniformly relabel β j−k i−c ∈ (0, 0.1] of data in D̂tr[T (j) = y (j) k ] (subset of D̂tr that satisfies T (j) = y(j)k ), where β j−k i−c is calculated by
βj−ki−c = γD̂tr[T (j) = y(j)k ]
D̂tr[T (j) = y(j)k , T (i) = y (i) c ] + γD̂tr[T (j) = y(j)k ]
, γ = min(αj−ki−c − τ, 0.1), (6)
in which D̂tr[T (j) = y(j)k , T (i) = y (i) c ] represents the data in D̂tr that satisfies T (j) = y(j)k and T (i) = y(i)c . The detailed calculation can be found in Section S2 in the Appendix. Relabeling partial data will result in a trade-off between the protective efficacy and the model performance on predicting T (j). By setting an upper threshold of 0.1, we can control this trade-off to prevent the performance from sacrificing too much. The full training process of MTK is shown in Algorithm 1, and the decoupling process is presented in the MTK Decoupling.
Algorithm 1 Training Multi-Trigger-Key Model (MTK) Input: The initialization weights {θ,φ(1),φ(2), · · · ,φ(N)}; secured tasks T1 = {T (1), · · · , T (N1)} and unprotected tasks T2 = {T (N1+1), · · · , T (N)}; the original training set D̂tr; empty set Dtr; threshold τ . ♠MTK Decoupling
1 Calculate αj−ki−c ,∀i, j ∈ [N ], c ∈ [Ki], k ∈ [Kj ], i 6= j. 2 for each j ∈ [N ] do 3 Find the largest αj−ki−c , ∀i ∈ [N ]/j, c ∈ [Ki], k ∈ [Kj ] that satisfies α j−k i−c > τ . 4 Calculate βj−ki−c using (6) and uniformly relabel β j−k i−c of data in D̂tr[T (j) = y (j) k ]. 5 end for ♣MTK Core 6 Construct D0tr by uniformly relabeling all the data associated with T1 in D̂tr. 7 Dtr ←− D0tr. 8 for each j ∈ [N1] do 9 Djtr := D0tr and add trigger-key x̂(mj , δj) = (1−mj) · x + mj · δj for (x, y) ∈ D j tr.
10 Relabel T (j) ∈ T1 in Djtr to the ground truth y∗ from D̂tr while maintaining labels in other tasks unchanged. 11 Dtr ←− Djtr. 12 end for 13 Obtain the final solution through solving (2). 14 Return: {θ,φ(1),φ(2), · · · ,φ(N)}

heading: 4 EXPERIMENTAL RESULTS
text: We first introduce the dataset for the empirical evaluation. Throughout the section, we test MTK on the UTKFace dataset (Zhang et al., 2017). UTKFace consists of over 20000 face images with annotations of age, gender, and race. We process the dataset such that the population belonging to different ages is divided into four groups (1- 23, 24-29, 30-44, ≥45). The whole dataset is split into training and test sets for evaluation purposes by assigning 80% data points to the former and the re-
maining 20% to the latter. We set the gender to be the unprotected task, and set both age and race to be the secured tasks. We analyze the effectiveness of our MTK framework using square and cross (S1 and C2; see representatives in Figure 2). We test MTK on VGG16 and ResNet18. If not otherwise specified, we use VGG16 as the model architecture. We show results using 95% confidence intervals over five random trials. The details of experimental settings can be viewed in Section S3.

heading: 4.1 OVERALL PERFORMANCE
text: MTK core. Results of applying MTK core are shown in Table 1. Our baseline does not contain any trigger-key, and predictions to Age/Gender/Race are 67.9%/92.3%/81.91%. As for comparisons, we train models using trigger-keys S1 and/or C2. If not otherwise specified, S1 and C2 have pixel color [255, 0, 0] and [0, 255, 0] and are both in the size of 5 × 5. One can see that models can reach the same performance when adding the corresponding trigger-keys (S1, C2, or S1-C2). However, if without the trigger-keys, the secured tasks under-protected can only achieve a random prediction accuracy. Specifically, the prediction accuracies are 25.24% and 18.6% for age and race, respectively.
Adding the MTK decoupling process. We set the threshold τ = 0.15. By checking the training set, we find that
αAge−≥45Race−White = Pr(Race = White|Age ≥ 45)− Pr(Race = White) = 0.191 αRace−OthersAge−≤23 = Pr(Age ∈ [1, 23]|Race = Others)− Pr(Age ∈ [1, 23]) = 0.184,
which are all > τ . According to (6), we then train models after relabeling βAge−≥45Race−White = 6.26% of data in D̂tr[Age =≥ 45] and βRace−OthersAge−≤23 = 7.17% of data in D̂tr[Race = Others]. Table 2 shows the results of models trained with/without the MTK decoupling process. Pr(·) in the test phase denotes the proportion of correct predictions. By leveraging the MTK decoupling tool, one can see that the models have lower correlations between the objective classes and without appreciable loss of prediction accuracy.

heading: 4.2 SENSITIVITY ANALYSIS
text: Note that keys can be selected from different combinations of locations and color levels of pixels. Here we study how changing size |mj | and perturbation δj of triggers affect MTK training and test.
Sensitivity analysis in training. We first test the sensitivity with respect to different sizes. We fix all the pixels in S1 (C2) to be [255, 0, 0] ([0, 255, 0]) and enlarge the size from 3 × 3 to 11 × 11. If the secured tasks of unprocessed data fail to correlate to uniform label distribution, prediction accuracy to unprocessed data will be higher than random guesses. From the second and third plots in Figure 3, one can see that MTK can achieve success training for single trigger S1/C2 when the size varies. For two trigger-keys, the only failure case is when the model is trained on 3× 3 square (S1) and cross (C2). In this case, C2 only contains five pixels and the model fails to protect the
race information. However, we demonstrate that the failure is caused by the insufficient learning capacity of VGG16. We conduct the same experiments on ResNet18. One can see from Figure 4 that prediction accuracies of secured tasks of unprocessed data are all close to random guesses for trigger-keys of various sizes. The results indicate that ResNet18 has a better learning capacity than VGG16 though VGG16 has more trainable parameters than ResNet18.
We then fix the size of both S1 and C2 to be 5 × 5 and train models with various magnitudes of perturbations. Figure 5 shows that for perturbation magnitude varying from 0.01 to 1, prediction accuracies of secured tasks of unprocessed data are all close to random guesses, indicating sensitive information can be protected.
Sensitivity analysis in test. Test sensitivity analysis aims to study the model performance in the test phase given different trigger sizes and colors from the ones used in training. Here we select the model trained with S1 and C2. In the size of 5×5, there are 25 pixels for S1 and 9 pixels for C2. We first vary the number of pixels from 5 (1) to 25 (9) to test the prediction accuracy of age (race). The results are shown in Figure 6. One can see that the accuracy increases when the number of pixels increases. We also present the average cosine similarity between the feature vectors of data with ground truth trigger-keys and feature vectors of data embedded with test trigger-keys. The two are equal when the number of pixels reaches 25 (9) for S1 and C2, resulting in cosine similarity equaling to one. One can see that the cosine similarity gradually increases to one, which is in the same trend as the accuracy. Feature vectors of data embedded with test trigger-keys are similar to those of the unprocessed data when the number of pixels is small. Therefore the accuracy is also small in this case. These observations and analysis are in consistent with Theorem 1. We then vary the magnitude of pixels from 0.02 to 1 to test the prediction accuracy. The results are shown in Figure 7. We observe the same phenomenon as in the tests of pixel number, i.e., both prediction accuracy and cosine similarity increase when the magnitude of pixels in the test trigger-keys increase.

heading: 5 CONCLUSION
text: In this paper, we proposed a novel framework for multi-task privacy-preserving. Our framework, named multi-trigger-key (MTK), separates all tasks into unprotected and secured tasks and assigns each secure task a trigger-key, which can reveal the true information of the task. Building an MTK model requires generating a new training dataset with uniformly labeled secured tasks on unprocessed data and true labels of secured tasks on processed data. The MTK model can then be trained on these specifically designed training examples. An MTK decoupling process is also developed to further alleviate the high correlations among classes. Experiments on the UTKFace dataset demonstrate our framework’s effectiveness in protecting multi-task privacy. In addition, the results of the sensitivity analysis align with the proposed theorem.

heading: S1 PROOF OF THEOREM 1
text: Here we follow the similar proof line as in (Shan et al., 2020). First we assume that with the ground truth trigger-key (mj , δj), the model prediction of any data satisfies
Pr (
arg max ∀k∈[Kj ] (F (j) k (x̂(mj , δj))) = y 6= arg max∀k∈[Kj ] (F (j) k (x))
) ≥ 1− κ, κ ∈ [0, 1], (S1)
where F (j)(x) = g(j)(f(x)). Here g(j) denotes a linear mapping. The gradient of F (j)(x) can be calculated by the following formula
∂ lnF (j)(x) ∂x = ∂ ln g(j)(f(x)) ∂x = g(j)∂ ln f(x) ∂x ,
We ignore the linear term and focus on the gradient of the nonlinear term. We rewrite (S1) and obtain
Prx∈X (∂[ln f(x)− ln f(x̂(mj , δj))]
∂x ≥ η
) ≥ 1− κ, κ ∈ [0, 1], (S2)
where η denotes the gradient value that moves the data to class y. Note that we have cos ( f ( x̂(mj , δj) ) , f ( x̄(m′j , δ ′ j) )) ≥ ν and ν is close to 1. Let f ( x̄(m′j , δ ′ j) ) −f ( x̂(mj , δj) ) = ζ
and we have |ζ| << |f ( x̂(mj , δj) ) |.
Let x̄(m′j , δ ′ j) = x + σ, we have
Prx∈X (∂[ln f(x)− ln f(x̄(m′j , δ′j))]
∂x ≥ η ) = Prx∈X (∂[ln f(x)− ln f(x + σ)] ∂x ≥ η )
= Prx∈X ( 1 f(x) ∂f(x) x − 1 f ( x̂(mj , δj) ) + ζ
∂[f ( x̂(mj , δj) ) + ζ]
x ≥ η ) ≈ Prx∈X ( 1 f(x) ∂f(x) x − 1 f ( x̂(mj , δj) ) ∂[f(x̂(mj , δj))] x ≥ η )
= Prx∈X (∂[ln f(x)− ln f(x̂(mj , δj))]
∂x ≥ η ) ≥ 1− κ,
(S3)
where the approximation holds true because of the following conditions. ∂[f ( x̄(m′j , δ ′ j) ) + ζ] ∂x = ∂f ( x̄(m′j , δ ′ j) ) ∂x ,
1 f ( x̄(m′j , δ ′ j) ) + ζ ≈ 1 f ( x̄(m′j , δ ′ j) ) ,
Now we consider the scenario cos ( f ( x ) , f ( x̄(m′j , δ ′ j) )) ≥ ν. Let f ( x̄(m′j , δ ′ j) ) − f ( x )
= ζ. We have
Prx∈X (∂[ln f(x̄(m′j , δ′j))− ln f(x̂(mj , δj))]
∂x ≥ η ) = Prx∈X ( ln f(x)+ ζ ∂x − ∂ ln f ( x̂(mj , δj) ) ∂x ≥ η )
= Prx∈X (∂[ln f(x)− ln f(x̂(mj , δj))]
∂x ≥ η ) ≥ 1− κ,
(S4)

heading: S2 DETAILED CALCULATIONS OF MTK DECOUPLING
text: The value that overflows the tolerance is represented by γ = min(αj−ki−c − τ, 0.1). To mitigate the overflow, we change labels of a proportion of data in D̂tr[T (j) = y(j)k ]. The proportion should satisfy the following equation.
D̂tr[T (j) = y(j)k , T (i) = y (i) c ]
D̂tr[T (j) = y(j)k ]− β j−k i−c D̂tr[T (j) = y (j) k ] − D̂tr[T (j) = y(j)k , T (i) = y (i) c ] D̂tr[T (j) = y(j)k ] = γ
This is equivalent to
βj−ki−c D̂tr[T (j) = y (j) k , T (i) = y(i)c ] = γD̂tr[T (j) = y (j) k ]− β j−k i−c γD̂tr[T (j) = y (j) k ]
We then have
βj−ki−c = γD̂tr[T (j) = y(j)k ]
D̂tr[T (j) = y(j)k , T (i) = y (i) c ] + γD̂tr[T (j) = y(j)k ]
Technically speaking, the proportion of data should not include D̂tr[T (j) = y(j)k , T (i) = y (i) c ]. For simplicity, we randomly select the data in the implementation.

heading: S3 EXPERIMENTAL SETTINGS
text: Datasets. We test MTK on the UTKFace dataset (Zhang et al., 2017). We use the cropped faces. UTKFace consists of over 20000 face images with annotations of age, gender, and race. Age is an integer from 0 to 116. Gender is either 0 (male) or 1 (female). Race is an integer from 0 to 4, denoting White, Black, Asian, Indian, and Others. We process the dataset such that the population belonging to different ages is divided into four groups (1-23, 24-29, 30-44, ≥45) and we assign 0 to 3 to the new groups. Each cropped image is in the size of 128× 128× 3. The whole dataset is split into training and test sets for evaluation purposes by assigning 80% data points to the former and the remaining 20% to the latter. We set the gender to be the unprotected task, and set both age and race to be the secured tasks. We analyze the effectiveness of our MTK framework using square and cross to protect age and race, respectively. If not otherwise specified, S1 and C2 have pixel color [255, 0, 0] and [0, 255, 0], locate on (110, 110) and (20, 110), and are both in the size of 5 × 5. We show results using 95% confidence intervals over five random trials.
Models. VGG16 and ResNet18 architectures are used for UTKFace. If not otherwise specified, we use VGG16 as the model architecture. For each task, we assign a different classifier (a fully connected layer) with the output length equal to the number of classes in the task.
Total amount of compute and type of resources. We use 1 GPU (Tesla V100) with 64GB memory and 2 cores for all the experiments.

heading: S4 LIMITATION AND SOCIETAL IMPACT
text: Current studies focus on the image domain. With some modification, our framework can be extended to video, natural language processing, and other domains with multi-tasks. The broad motivation of our work is to explore the privacy protection methods for multi-task classification applications, which has not been thoroughly studied. We believe this goal is highly relevant to the machine learning/artifical intelligence community, and the methods that our paper introduces can be brought to bear on other privacy-preserving problems of interest.

Review summaries:
Multi-task classification (MTC) is a kind of multi-task learning, which performs multiple multi-class classification tasks at the same time. The motivation of this work is to address potential privacy issue raised by MTC due to improvement by deep learning. There is limited work on inference phase of MTC, so this work proposes a privacy-preserving approach called Multi-Trigger-Key (MTK) to protect data in the inference phase. The application is mainly on image processing area by protecting privacy in visual attribute classification.

This paper focuses on access control to multi-task image classification service. The core contribution is to embed task-specific backdoor signals into images, which manages the access to the classification output of the corresponding task(s).   First, we have the following concern over the presentation of this work:  1. In theorem.1, what are the variables $m'$ and $\delta^{'}$ ?  what is the difference between $(m',\delta^{'})$ and $(m,\delta)$ ?  2. Eq.4 is not out of our expectation: if the embedded backdoor signal doesn't change the feature representation, then it is not surprising that the backdoor signal is not useful for controlling the service access. Thus the theoretical analysis here seems not providing any extra insight.  3. Eq.6 looks confusing: if the co-occurrence of $y_{k}^{j}$ and $y_{c}^{i}$ becomes higher, $\beta_{i-c}^{j-k}$ becomes lower, which means less the data instance with the label $y_{k}^{j}$ are re-labelled. First, for these data instances, the label $y_{k}^{j}$ is relabelled to what? Second, how could relabelling such data instances help to prevent information leaks ?   Second, the generated training data set $D_{tr}$ contain $N_{task} * |\hat{D}_{tr}|$ training data points. Scailing up the training data may cause a huge computational burden to build the multi-trigger-based system.   Third, the robustness of the embedded backdoor is not discussed. Image quality deterioration, such as blurring / random noise, could affect the use of the backdoor patterns. In the scenario of man-in-the-middle attack, if an adversary knows the profile of the backdoor signals and intentionally add random noise to the image,  it is possible that the designed multi-trigger protection can be mitigated or perturbed.   Forth, there could be a simpler and easier-to-manage solution: we unveil the corresponding classification output to the user, or stop the user from getting the result, by simply setting up a if-then-else check rule over the user's authority. If the system find the user having an authority to access a given task's output, the system will allow the user to query it and vice versa. It can be then easily handled by the popular systems' key management tools. 

 This paper proposes a new framework, multi-trigger key (MTK), which aims to protect sensitive information during inference for multi-task classification. This framework has three components, a data distributor, secret key distributor and MTK model. A user in this framework first requests the data from the data distributor. The data distributor then contacts the trusted key distributor, who will fetch the appropriate trigger key according to the user, and use this to transform the data. Inference is then performed on the transformed data. The MTK model aims to provide random predictions for unprocessed (i.e without trigger key) inputs and true predictions for correct inputs.   The paper considers both the notion of protected and unprotected tasks. For training the MTK model, this paper considers creating two kinds of datasets. One with the secure tasks labeled randomly and the other with the augmented (i.e with trigger data) labeled correctly. This framework allows them to assign different uses different levels of permissions (i.e authority).  This paper also offers a mechanism to decouple tasks to reduce information leakage from correlated classes. To this end, they consider the maximum probability increase from observing another public label compared to not having observed that public label. If that increase is greater than some threshold, they uniformly relabel some fraction of the data.  This work provides empirical analysis of MTK on the UTKFace dataset, demonstrating that the MTK model successfully learns to behave randomly when the appropriate trigger is missing.