Interpolating Compressed Parameter Subspaces
Abstract
Though distribution shifts have caused growing concern for machine learning scalability, solutions tend to specialize towards a specific type of distribution shift.
We learn that constructing a Compressed Parameter Subspaces (CPS) , a geometric structure representing distance-regularized parameters mapped to a set of train-time distributions, can maximize average accuracy over a broad range of distribution shifts concurrently.
We show sampling parameters within a CPS can mitigate backdoor, adversarial, permutation, stylization and rotation perturbations.
Regularizing a hypernetwork with CPS can also reduce task forgetting.
Introduction
Suppose your model is expecting distribution shift at test-time.
Is it task shift?
Domain shift?
Rotations?
What if it’s an adversarial attack, or even a backdoor attack?
Each strand of distribution shift research has provided robust and adaptive techniques against their respective types of shift, and achieved state-of-the-art results.
In this work, we take a step towards a method that can adapt to multiple types of distribution shift concurrently.
Contributions.
We propose constructing compressed parameter subspaces to increase the likelihood of sampling an optimal parameter for a shifted input distribution.
We demonstrate a high average accuracy across distribution shifts in single and multiple test-time settings.
We show improved robustness across perturbation types, and reduced catastrophic forgetting on Split-CIFAR10/100.
Related Work
Different aspects of distribution shift have been studied individually and methods of robustness/adaptation have been proposed.
More recently, there is work studying what happens when deep learning architectures encounter test-time inputs manifesting multiple types of distribution shifts.
Qi et al.
(2021) and
Datta (2022) combine adversarial attacks with stylization and domain shift Liu et al.
combined adversarial perturbations with backdoor trigger perturbations to craft stealthy triggers to perform backdoor attacks.
However, methods of robustness/adaptation generally require the distribution shift to be specified at train-time.
For example, Weng et al.
(2020) found a trade-off between adversarial defenses optimized towards adversarial perturbations against backdoor defenses optimized towards backdoor perturbations, where the adoption of one defense worsened robustness to the other’s attack.
Though data augmentation (Yun et al., 2019b; Borgnia et al., 2020) and adversarial training (Goodfellow et al., 2015; Geiping et al., 2021) as a technique can mitigate adversarial attacks and backdoor attacks, they need to be trained on one of the perturbation types to manifest robustness.
To retain adversarial robustness in few-shot learning, Goldblum et al.
(2020) and Wang et al.
(2021) investigated methods of integrating adversarial training into MAML.
trainSpace ( f, { θ i } N , { D i } N , β, E )
Input : Model f , Parameters { θ i } N , Train set { D i } N , Distance coefficient β , Epochs E
Output: Trained parameters { θ i } N
Constant initialization { θ init ← θ i } N
Train each parameter against their indexed train set in parallel per epoch foreach epoch e ∈ E do foreach θ i , D i ∈ { θ i } N , { D i } N do foreach batch x,
y ∈
D i do ˆ y ← f ( x, θ i )
L ← ℓ (ˆ y, y ) + β cos ( { θ i } N )
Backprop θ i w.r.t.
L return { θ i } N
Algorithm 2:
Evaluate evalSpace x, { θ i } N , { α i } N
Input : Input x , Trained parameters { θ i } N , Interpolation coefficients α i ∼ [0 , 1] s.t.
(cid:80)
Ni α i ≤ N Output: Predicted output
ˆ y θ ← (cid:80) Ni α
i θ i ˆ y ← f ( x, θ ) return ˆ y
Compressed Parameter Subspaces (CPS) 3.1 Subspace Construction
A base learner function f receives an input x (or distributionally-shifted input ˆ x ) to return predicted labels ¯ y =
f ( θ ; x ) .
Model parameters θ ∼ Θ are sampled from the parameter space such that loss is minimized between the ground-truth and predicted labels:
L ( θ ; x , y )
= 1 | x | (cid:80) | x | i ( f ( θ ; x ) − y ) 2 .
The CPS is constructed by training N parameters in parallel, where each parameter is trained on a different shifted distribution.
Parameter point-estimates can be interpolated between the trained parameters (which set the bounds of the subspace).
We sample interpolation coefficients
α i ∼ [0 , 1] s.t. α
i (cid:55)→ x i and (cid:80) Ni α i ≤ N .
We begin with a set of N shifted distributions { x i } N and a single constant (randomized) initialization.
For a single test-time distribution (CIFAR10), we slice subsets of the train set and insert unique perturbations of a given type (backdoor / stylization / rotation) into each subset.
For multiple test-time distributions (CIFAR100), we use different task sets as each distribution.
Each model is a ℓ -layer
CNN trained with early-stopping at loss 1.0.
We load the N sets in parallel.
At the end of each epoch, we compute the loss of each model’s parameters with respect to their train-time sets.
We then compute the average cosine distance between each model’s weight against all the other model’s weights.
We add this distance multiplied by a distance coefficient β = 1 . 0 to the total loss, and update each model’s weights with respect to this total loss.
3.2 Unbiased sampling
In this sampling scheme, we do not use any heuristics in selecting parameters.
• We can compute predictions ¯ y =
f ( θ ∗ ; ˆ x ) given an input and the centre parameter point-estimate θ ∗ =
(cid:80) Ni
N θ i (the average of the end-point parameters). • We can compute a mean prediction ¯ y = 1 M (cid:80)
Mj f
( θ ∗ j ; ˆ x ) given an input and an ensemble of M randomly-sampled parameter point-estimates in the subspace; we randomly sample M =interpolation coefficients { α j,i } M × N to return an ensemble set of parameter point estimates { θ ∗ j } M = { α j,i } M × N · { θ i } N .
3.3 Guided sampling
Guided sampling makes use of a heuristic in the selection of parameters to be used for inference.
• We can compute predictions ¯ y =
f
( θ j ∗ ; ˆ x ) given an input and a lowest-loss interpolated parameter point-estimate, where j ∗ :=
arg min j ∼ M L ( θ j ; x , y ) .
This is specifically a unique-task solution, where the interpolated parameter maps back to one task/distribution ˆ x
(cid:55)→ θ j ∗ .
For a multi-task solution, an interpolated parameter is mapped back to a set of T task/distributions { ˆ x t } T (cid:55)→ θ j ∗ , where j ∗ := arg min j ∼ M (cid:80)
Tt L ( θ j ; x t , y t ) .
To identify this interpolated parameter that returns the lowest-loss (maximum-accuracy) for a given test-time distribution, we can actively and iteratively sample parameters with respect to the centre of the subspace, or linearly enumerate through interpolated parameters between the known boundary parameters.
An example of a scheme would be, given K-shots, we can sample parameters and use the parameter of lowest-loss for the task.
This could also be extended to construct an ensemble.
In this work, we search for the lowest-loss parameter to show that a compressed parameter subspace has a higher likelihood of containing a shift-optimal parameters. • We can compute predictions ¯ y =
f ( θ i ∗ ; ˆ x ) given an input and lowest-loss boundary parameter, where the latter is the parameter from a set of N boundary or end-point parameters that returns the lowest loss i ∗ := arg min i ∼ N L ( θ i ; x , y ) .
The boundary parameter may be used if the task index is known.
In this work, we use the boundary parameter as an ablation, and to check if the lowest-loss interpolated parameter is simply returning a boudary parameter or not.
3.4 Generalized function
We can generalize or represent the subspace implicitly with a learning function.
Starting from random initialization, we train a hypernetwork first on CIFAR10, and store these parameters θ mf ,t in a parameter set { θ mf ,t } T .
Then we train on a subsequent CIFAR100 task with the Eqt.
1 loss function, where we compute loss w.r.t.
the inputs and current timestep’s parameters, change in loss w.r.t.
a previous task between using the proposed parameters and the parameters last updated at that task (stored in the parameter set; this is enumerated for all past tasks in sequence), and the distance between the current parameters and all prior parameters.
[Equation]
Evaluation
We evaluate on CIFAR10 and CIFAR100 (Krizhevsky, 2009) for single (non-task-shift) and multiple test-time distributions (task-shift) respectively.
For label shift w.r.t.
train-time perturbations, we implement backdoor attacks with Random-BadNet (Datta & Shadbolt, 2022).
For label shift w.r.t. test-time perturbations, we implement adversarial attacks for targeted label shift with Projected
Gradient Descent (Madry et al., 2018).
We permute and rotate images in-line with PermuteCIFAR and RotateCIFAR10 respectively.
We implement stylization with Adaptive Instance Normalization (Huang & Belongie, 2017).
For task shift, we evaluate on both end-point and interpolated tasks.
Based on Mixup (Zhang et al., 2017b) and MLTI (Yao et al., 2021): for a set of tasks { X } N where each task X is a set of inputs x (and the number of inputs per task is identical, i.e. {{ x } M } N ), we linearly-interpolate between indexed inputs across a task set to return an interpolated task: ˆ x = (cid:80) Ni (cid:80)
Mj α
i x i,j .
We interpolate between task sets with identical labels (label-shared).
We evaluate hypernetworks with Split CIFAR10/100, where we train on CIFAR10 first, then train sequentially on the next 5 sets of 10-label CIFAR100, and all tasks share the same 10 coarse labels. (Observation 1)
Sampling parameter point-estimates in the CPS can attain a high average robustness accuracy across different perturbations or shifts.
Evaluated against the clean test-time distribution, we find that sampling point-estimates in the CPS can retain its accuracy across varying perturbation types (Table 1, Appendix A.3.8).
For a single test-time distribution (CIFAR10) with varying perturbation types, we find that inference with the centre or ensemble can yield a higher average accuracy across perturbed test-time distributions than wide-spectrum and/or niche defenses.
[Table] [Table]
[Table] [Table]
[Table]
multiple test-time distributions: Between β = 0 . 0 and β = 1 . 0 , with respect to a set of 50 3 interpolated tasks, we evaluate accuracy with respect to the lowest-loss boundary parameter, centre parameter, ensembling 1000 parameters and averaging their predictions, and searching for the lowest-loss interpolated parameter (excluding boundary parameters).
[Table]
(Observation 2)
Constructing a CPS can yield a subspace containing low-loss parameters for corresponding inputs.
Based on Table 2, compared to a non-CPS, we observe that more points interpolated in the input space can be mapped to low-loss interpolated parameters in the parameter space.
We would expect a larger subspace to contain more low-loss parameters, yet counter-intuitively a subspace compressed w.r.t.
opposing boundary parameters contains more mappable low-loss parameters.
We also note that the interpolated points perform better than reusing boundary parameters.
We evaluate an additional CPS-regularization term to von Oswald et al.
(2020)’s hypernetwork to find reduced catastrophic forgetting across tasks (Table 3).
Conclusion
We learn that sampling points in the parameter subspace can return lower-loss mapped parameters if the space was compressed during training, yield robust accuracy across various perturbation types, and reduces catastrophic forgetting when adapted into a hypernetwork.
We motivate further study into enforcing geometric structure to parameter subspace construction and methods of test-time adaptation towards (joint) distribution shifts.
A Appendix A.1 Supporting Theory
Definition 1.
A distribution shift is the divergence between a train-time distribution x 0 , y 0 and a test-time distribution ˆ x , ˆ y , where ˆ x , ˆ y is an interpolated distribution between x 0 , y 0 and a target distribution x ∆i , y ∆i
such that ˆ x = (cid:80) Ni α i x ∆i and ˆ y
=
y i
| i := arg max i α i .
Let X , Y , K , Θ be denoted as the input space, coarse label space, fine label space, and parameter space respectively, such that X (cid:55)→ Y , X
(cid:55)→ K , K
(cid:55)→ Y , X (cid:55)→ Θ .
Coarse labels are the super-class / higher-order labels of fine labels.
f is a base learner function that accepts inputs x , θ to return predicted labels ¯ y =
f ( θ ; x ) .
The model parameters θ ∼ Θ are sampled from the parameter space, for example with an optimization procedure e.g. SGD, such that it minimizes the loss between the ground-truth and predicted labels:
L ( θ ; x , y )
= 1 | x | (cid:80) | x | i ( f ( θ ; x ) − y ) 2 .
In-line with notation in Pan & Yang (2010), for a marginal (or feature) distribution P ( X ) and a conditional (or label) distribution P ( Y|X ) , a joint distribution shift is denoted as a joint transformation in the marginal-conditional distributions.
A covariate shift manifests as a change in the marginal distribution P ( X ) under constant conditional distribution P ( Y|X ) .
A label shift manifests as a change in the conditional distribution P ( Y|X ) under constant marginal distribution P ( X ) .
As elaborated in Section 2, deep neural networks may not perform well under this joint shift setting, and unifying robustness methods are limited.
To simplify our language in discussion, we use a set of points sampled in a space to pertain to a distribution.
To retain generality in notation and implementation, we re-iterate the aforementioned formalism in terms of sampled points in their respective spaces.
We sample mapped points in the input-label spaces as x ,
y = {{ x, y }| condition ∼ X , Y} .
We describe shift here with respect to a single target label in order to highlight changes in the input-label mappings; the notation extends to non-singular label sets.
Suppose we sample a clean / source / train-time set of points, constrained by a mapped target label y ∗ : x 0 , y
= {{ x, y }| y
=
y ∗ ∼ X , Y} .
A distribution shift is the divergence between a train-time distribution x 0 , y 0 and a test-time distribution ˆ x , ˆ y , where ˆ x , ˆ y is an interpolated distribution between x 0 , y 0 and a target distribution x ∆i , y ∆i
such that ˆ x = (cid:80) Ni α i x ∆i and ˆ y
=
y i
| i := arg max i α
i .
For a set of distributions { x 0 (cid:55)→ y 0 , x ∆1 (cid:55)→ y ∆1 , ..., x ∆N (cid:55)→ y ∆N } containing ( N − 1) target distributions, we sample interpolation coefficients
α i ∼ [0 , 1] s.t. α
i (cid:55)→ x i and (cid:80) Ni α i ≤ N .
N is the number of distributions being used; in CPS training, it is the number of train-time distributions used in training the subspace; in task interpolation, it is the number of train/test-time distributions used in interpolating between task sets.
A disjoint distribution shift is a distribution shift of only one target distribution |{ α i }| =such that ˆ x = α x ∆ + (1 − α ) x 0 .
A joint distribution shift is a distribution shift of multiple target distributions |{ α i }| > 2 such that ˆ x = (cid:80) Ni α i x ∆i .
Each distinct target distribution is varied by the type of perturbation (e.g. label, domain, task) and the variation per type (e.g. multiple domains, multiple backdoor triggers).
For a clean test-time distribution where || x ∆1 − x 0 || p
→ 0 and y
=
y ∗ , no shift takes place.
If ˆ x , ˆ y manifests label shift ( y ̸
=
y ∗ , min || x ∆1 − x 0 || p ), then x ∆1 , y ∆
= {{ x, y }| y ̸
=
y ∗ ∼ X , Y} .
If ˆ x , ˆ y manifests covariate shift ( || x ∆2 − x 0 || p > 0 ), such as domain shift, then x ∆2 , y ∆2 = {{ x, y }| y =
y ambg ∼ X , Y} , where the sampled x may not correspond to a clearly-defined class (i.e. a random / ambiguous class y ambg ).
If ˆ x , ˆ y manifests task shift (unrestricted || x ∆3 − x 0 || p > 0 ); for label-shared task shift (identical coarse labels y , non-identical fine labels k ) x ∆3 , y ∆3 = {{ x, y }| y =
y ∗ , k ̸
= k ∗ ∼ X , Y , K} ; for non-label-shared task shift (non-identical coarse nor fine labels) x ∆3 , y ∆3 = {{ x, y }| y ̸
= y ∗ , k ̸
= k ∗ ∼ X , Y , K} .
For interpolated input sets between x 0 and x ∆i , we would expect 0 < α 0 , α i < 1 , unless we expect the shifted distribution to completely manifest the target distribution (e.g. rotated sets, end-point task sets) such that α i = 1 .
The labels of the interpolated set is the label set with the larger corresponding α i .
In label shift, the perturbations are small enough that we retain the ground-truth labels.
For label-shared task shift, the ground-truth (coarse) labels between the end-point tasks are identical.
For non-label-shared task shift, we do not interpolate between non-identical label sets between the end-point tasks.
If we randomly-interpolate random input sets with random or ambiguous label assignments, we incur the risk of (i) the perturbed input set as being as being perceptually inconsistent with the assigned label, and (ii) the model overfitting towards random label assignments (Zhang et al., 2017a).
To mitigate these concerns and generate semantically-valid (perceptually-interpreted as belonging to its assigned label) perturbations, we adopt existing perturbation generation methods that have also been human-evaluated as perceptually consistent.
The use of these methods can be interpreted as the filtering of invalid points in the input space for the evaluation of mapping between X and Θ .
Corollary 1.
Test-time distributions can be decomposed into a weighted sum between known train-time distributions and computable interpolation coefficients.
We assume a given test-time distribution ˆ x can be decomposed into a sum of parts ˆ x := (cid:80)
Ni α
i x i , where N is the number of component distributions.
For the component distributions { x i } N , if a subset of the distributions exist in the set of train-time distributions, then the interpolation coefficient value per train-time distribution lies in 0 < α i ≤ 1 ; if none of the distributions exist in the set of train-time distributions, then the interpolation coefficient value per train-time distribution is 0.
Corollary 2.
For an input interpolated between a set of input end-points with interpolation coefficients { α i } N , the loss term per epoch can be decomposed into component/end-point loss terms of equivalent interpolation coefficients { α i } N , but the parameters may or may not be decomposed into component/end-point parameters of { α i } N .
The loss of an interpolated set with respect to a parameter updated till epoch e can be computed with respect to the weighted sum of the loss with respect to the boundary input points and interpolation coefficients for the interpolated input set. [Equation] ∂θ
[Equation]
θ t = e is computed with respect to the total loss at epoch e − 1 .
θ t = e − 1 updated with respect to multiple x i may not be identical to θ t =
e − 1 updated with respect to a single set x 0 .
As a result, we cannot claim that an interpolated input set must correspond to or be satisfied with a parameter composed of the weighted sum of the interpolation coefficients and end-point parameters θ ∗ ̸
= (cid:80) Ni α
i θ i .
Corollary 3.
Training and inference with a single parameter point-estimate θ may face a trade-off in loss w.r.t.
X i and X n if interference exists between X i and X n .
At a given epoch e , if ∂ L ( θ ; X i ) ∂θ · ∂ L ( θ ; X n )
∂θ < 0 , then the decrease in loss w.r.t.
one subset (either X i or X n ) would result in the increase in loss of the other during training, and consequently the same during inference.
Theorem 1.
For N + 1 parameters trained in parallel, any individual parameter θ
i is a function of its mapped data
X i (cid:55)→ Y
i as well as the remaining parameters { θ n } n ∈ N (and by extension,
X n (cid:55)→ Y n ).
Hence, a linearly-interpolated parameter between θ i and { θ n } n ∈ N is a function of { θ n } n ∈ N and θ i |{ θ n } n ∈ N .
Proof.
Suppose N + 1 parameters are trained in parallel, and the number of dimensions of each parameter is M = | θ | .
To simplify the analysis, we use the Euclidean norm distance, and make relevant approximations.
First, we decompose the loss w.r.t.
θ i , in order to obtain θ i in terms of X i , Y i , and { θ n } n ∈ N such that L ( θ i ) = 0 .
We denote C
= (cid:80) Nn θ n .
By the Cauchy-Schwarz ineuality, we apply the following term replacement:
{+( (cid:80) Nn θ n ) 
 = ( (cid:80) Nn 1 · θ n ) 2 ≤ N · (cid:80) Nn θ 2 n ⇒ C N ≤ (cid:80) Nn θ 2 n .+}
[Equation]
Next, we compute the change in θ i w.r.t.
change in C , and find that the C term is persistent (noneliminated) and is retained in the gradient calculation and optimization procedure of θ i .
[Equation]
In order for Eqt 4 to remain valid, Eqt 5 needs to be satisfied, which also shows that C is dependent or contains artifacts from X i (cid:55)→ Y i .
[Equation]
Further, suppose we linearly-interpolate a parameter θ within the bounded space of θ i ∪{ θ n } n ∈ N , and the weighting per parameter is α n (cid:55)→ θ n such that (cid:80) N +1 n α n ≤ N + 1 .
We find that the interpolated parameter θ has a non-eliminated term C regardless of set α or position in the bounded parameter space.
[Equation]
Theorem 2.
For N + 1 parameters trained in parallel, for an individual parameter θ i and another parameter θ n where n ∈ N , the change in loss per iteration of SGD with respect to each parameter ∂ L ( θ i ; X i )
∂θ i , ∂ L ( θ n ; X n ) ∂θ n will both be in the same direction (i.e. identical signs), and will tend towards a local region of the parameter space upon convergence of loss.
Proof.
Suppose N + 1 parameters are trained in parallel, and the number of dimensions of each parameter is M = | θ | .
We evaluate the incremental loss per iteration ∀ e ∈ E , which is the distance regularization term.
[Equation]
Given that minimizing the last term can result in minimizing the loss min
[Equation]
L
[Equation]
) , we next show that the gradient update (change in loss per iteration of SGD with respect to each parameter) terms at iteration e must be positive and share the same direction when loss converges to 0.
For ∆ L ( θ i ) e ≥ 0 and ∀ n where n ∈ N ,
[Equation]
∂ [Equation] θ n
[Equation]
∂θ
[Equation]
For ∆ L ( θ i ) e ≡ 0 ,
[Equation]
Hence, ∂ L ( θ i ; X i )
∂θ i · ∂ L ( θ n ; X n ) ∂θ n must be positive, and the component terms must be in the same direction (identical signs).
To further simplify analysis, we evaluate loss updates from iteration e = 0 to e =
E − 1 where loss converges at iteration E , thus the component loss terms are non-zero.
The two loss terms are thus non-orthogonal, co-directional, and the angle between them is less than 90 ◦ .
[Equation]
Given that θ i and θ n are initialized at the same constant random initialization θ i,e =0 ≡ θ n,e =0 , the change in each θ from iterations e = 0 to e =
E − 1 is attributed to the difference in the summation of the gradient updates.
[Equation]
We show next, that through enforcing distance regularization, we can exert a tendency for the distance between the summation of the loss term components to be smaller if they share the same direction, than if they were orthogonal or opposing directions.
[Table]
Thus we show that θ i and θ n are closer in distance in the parameter space.
This can result in θ i and θ n sharing sub-matrices (intrinsic dimensions), and they may reside in the same or neighboring subspace.
Theorem 3.
For N + 1 parameters trained in parallel, the expected distance between the optimal parameter of an interpolated input ˆ x (cid:55)→ ˆ θ would be smaller with respect to sampled points in a distance-regularized subspace than non-distance-regularized subspace.
Informally, we conclude from Theorems 1 and 2 that, though non-linear and/or linear point-estimates may exist in a non-compressed parameter subspace, a compressed parameter subspace contains less interference attributed to changes in other dimensions; in other words, the CPS method reduces the dimensions of interpolation.
Practically, this can manifest as most values in the weights matrix being relatively frozen or constant (and only few values or a submatrix being changed), or the range per value in a weights matrix may be narrower.
Proof.
Suppose we sample an interpolated input ˆ x = α
i x i + α n x n .
All trained parameters begin with a constant and equal initialization at t = 0 .
We would like to show here that any randomly interpolated input set of coefficients α i , α n would be closer in distance to any parameter point-estimate interpolated between CPS-regularized parameter end-points with coefficients γ i , γ n , compared to non-CPS-regularized.
First we state the loss terms for a ground-truth parameter ˆ θ corresponding to ˆ x .
Recall from Theoremthat the loss terms in the end-point parameters converge in the same direction ∂
L ( θ i ; X i )
∂θ i · ∂ L ( θ n ; X n ) ∂θ n > 0 .
We find here a similarity to CPS, where in order for the loss to converge L (ˆ θ )
→ 0 the product of the loss terms for the ground-truth parameter also converges in the same direction ∂
L (ˆ θ ; X i ) ∂
ˆ θ · ∂ L (ˆ θ ; X n ) ∂ ˆ θ > 0 .
ˆ
[Equation]
Next, as the summation of loss per epoch indicates the region in the parameter space with respect to a constant initialization (as in Theorem 2), we compute the distance in the summation of loss for ˆ θ compared to the boundary or end-point parameters of an uncompressed space.
As the component loss terms are not regularized with respect to each other, their product may or may not be greater than 0, i.e. their product is in the range [-1, 1].
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
Then
[Equation]
we evaluate the summation of loss for ˆ θ against the boundary or end-point parameters of a compressed space
[Equation]
(cid:20) [Equation] [Equation] (cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
θn
[Equation]
xi
[Equation]
· ∂ L ( θn [Equation] xn ) ∂θn >(cid:21)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) < E (cid:34)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
(cid:20) α i ∂
L (ˆ θ t =
e
[Equation]
x i ) ∂
[Equation]
θ
[Equation]
e
[Equation]
[Equation]
) ∂
ˆ θ t
[Equation]
e (cid:21) (by Theorem 2) + (cid:20) γ n ∂ L ( θ n,t = e ; x n ) ∂θ n
[Equation]
t
[Equation]
e (cid:12)(cid:12)(cid:12)(cid:12) ∂
[Equation]
[Equation]
xi )
[Equation]
[Equation]
θn
[Equation]
[Equation]
e
[Equation]
∂θ
[Equation]
t
[Equation]
e (cid:12)(cid:12)(cid:12)(cid:12)
∂ L ( θn
[Equation]
xi ) ∂θi · ∂ L ( θn ; xn ) ∂θn ∼ [ − 1 , 1] (cid:21)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:35) (16)
We find that a ground-truth interpolated parameter is closer to a compressed space than an uncompressed space.
This motivates our work that a compressed subspace contains a higher density of low-loss interpolated parameters than an uncompressed subspace.
A.2 Experiment configurations A.2.1 Adaptation Methods
Data Augmentation (e.g., CutMix (Yun et al., 2019a) or MixUp (Zhang et al., 2018)) is a common method to robustify models against domain shift (Huang et al., 2018), adversarial attacks (Zeng et al., 2020), and backdoor attacks (Borgnia et al., 2020).
We implement CutMix (Yun et al., 2019a), where augmentation takes place per batch, and training completes in accordance with aforementioned early stopping.
In the case of CutMix, for example, instead of removing pixels and filling them with black or grey pixels or Gaussian noise, we replace the removed regions with a patch from another image, while the ground truth labels are mixed proportionally to the number of pixels of combined images. 50% of the defender’s allocation of the dataset is assigned to augmentation.
Adversarial Training is a method that increases model robustness by injecting adversarial examples into the training set, and commonly used against adversarial attacks (Goodfellow et al., 2015), backdoor attacks (Geiping et al., 2021), and domain adaptation (Ganin et al., 2016a).
We retain the same adversarial configurations as in the adversarial attack (PGD, ε = 0 . 4 ).
At each epoch, 50% of the defender’s allocation of the dataset is adversarially-perturbed.
Backdoor Adversarial Training (Geiping et al., 2021) extend the concept of adversarial training on defender-generated backdoor examples to insert their own triggers to existing labels.
We implement backdoor adversarial training (Geiping et al., 2021), where the generation of backdoor perturbations is through Random-BadNet (Datta & Shadbolt, 2022)), where 50% of the defender’s allocation of the dataset is assigned to backdoor perturbation, p, ε = 0 .
4 , and 20 different backdoor triggers used (i.e. allocation of defender’s dataset for each backdoor trigger pattern is (1 − 0 . 5) × 0 .
8 × 120 ).
Fast Geometric Ensembles (Garipov et al., 2018) train multiple networks to find their respective modes such that a high-accuracy path is formed that connects the modes in the parameter space.
The method collects parameters at different checkpoints, which can subsequently be used as an ensemble.
We retain the same model configurations as the clean setting (3-layer CNN), train 5 ensembles on randomly-sampled random initializations until the early-stopping condition.
[Equation]
• Inferencing w.r.t.
the centre of the subspace is denoted as computing a prediction ¯ y =
f ( θ ∗ ; ˆ x ) with respect to a given input and the centre parameter point-estimate θ ∗ = (cid:80)
Ni 1 N θ i (the average of the end-point parameters).
This form of inference has also been used in Wortsman et al.
• Inferencing w.r
t. an ensemble in the subspace is denoted as computing a mean prediction ¯ y = 1 M (cid:80)
Mj f
( θ ∗ j ; ˆ x ) with respect to a given input and M randomly-sampled parameter pointestimates in the subpace (we randomly sample M = 1000 interpolation coefficients { α j,i } M × N to return an ensemble set of parameter point estimates { θ ∗ j } M = { α j,i } M × N · { θ i } N .
For our ensemble, we evaluate against the mean prediction.
This form of inference has also been used in Wortsman et al.
; Garipov et al.
; Fort et al.
• Inferencing w.r.t.
the maximum-accuracy (lowest-loss) interpolated point-estimate in the subspace is denoted as computing a prediction ¯ y =
f ( θ j ∗ ; ˆ x ) with respect to a given input and the lowest-loss parameter point-estimate where j ∗ :=
arg min j ∼ M L ( θ j ; x , y ) ; this is specifically a unique-task solution, where a low-loss parameter maps back to one task/distribution ˆ x
(cid:55)→ θ j ∗ .
For a multi-task solution, a low-loss parameter is mapped back to a set of T task/distributions { ˆ x t } T (cid:55)→ θ j ∗ , where j ∗ := arg min j ∼ M (cid:80)
Tt L ( θ j ; x t , y t ) .
We linearly-space 50 segments per parameter end-point pair (resulting in 50 3 point-estimates to evaluate).
There is enough distance between parameter end-points such that the interpolated points are not close approximations of the end-point (further verifiable from Appendix A.3.2).
Inferencing w.r.t.
interpolated points has also been used in Neyshabur et al.
• An additional baseline used in Table 2 is inferencing w.r.t.
the lowest-loss boundary parameter in the subspace, denoted as computing a prediction ¯ y
=
f ( θ i ∗ ; ˆ x ) with respect to a given input and lowest-loss boundary parameter, where the latter is the parameter from a set of N boundary or endpoint parameters that returns the lowest loss i ∗ := arg min i ∼ N L ( θ i ; x , y ) .
This baseline informs us whether the lowest-loss interpolated parameter can outperform an uninterpolated SGD-trained parameter.
In Table 2 specifically, we do not include the boundary parameters in the interpolated parameters set (interpolation coefficients: [0,0,1],[0,1,0],[1,0,0]), so as to ensure the 2 sets are mutually-exclusive and avoid the reuse of the boundary parameter as the lowest-loss interpolated parameter; for example, in case there is another point that can also achieve similar accuracy.
CPS: Hypernetworks A hypernetwork h ( x , I ) =
f ( x ; mf ( θ mf ; I )) is a pair of learners, the base learner f : X (cid:55)→ Y and meta learner mf : I (cid:55)→ Θ f , such that for the conditioning input
I of input x (where X (cid:55)→ I ), mf produces the base learner parameters θ I =
mf ( θ mf ; I ) .
The function mf ( θ mf ; I ) takes a conditioning input I to returns parameters θ I ∈
Θ f for f .
The meta learner parameters and base learner (of each respective distribution) parameters reside in their distinct parameter spaces θ mf ∈ Θ mf and θ f ∈ Θ f .
The learner f takes an input x and returns an output ¯ y =
f ( θ I ; x ) that depends on both x and the task-specific input I .
T is number of tasks, t is index of specific task being evaluated ( t ∗ being current task), ω is task regularizer (from von Oswald et al. (2020) implementation).
We retain all other implementation configurations, such as the use of Adam optimizer, setting task regularizer ω
= 0 .
θ mf is the hypernetwork parameters at the current task’s training timestep, θ ∗ mf is the hypernetwork parameters before attempting to learn task t ∗ , and ∆ θ mf is the candidate parameter change computed by the optimizer (Adam).
Our base learner is the 6-layer
CNN.
Unlike von Oswald et al.
(2020), we wish to retain the assumption of the rest of the paper where we do not require conditioning inputs; hence we do not use any separate conditioning input (i.e. task embeddings), and use the test-time input as the primary argument, i.e. I ≡ X .
As such, the lack of use of task embeddings is one of the primary changes to our hypernetworks baseline based on von Oswald et al.
In terms of parameter storage, only the weights of the hypernetwork (and not individual weights of points in the CPS) would need to be stored.
As we only adapt von Oswald et al.
(2020)’s hypernetwork implementation, we use this as our primary baseline, and recommend comparison to other baselines listed in von Oswald et al.
Starting from a random initialization, we train the hypernetwork first on CIFAR10 (this can be interpreted as re-initializing the network with pre-trained weights, which based on findings in
Neyshabur et al.
(2020) may indicate that subsequent task parameters may reside in a shared low-loss basin), and store these parameters θ mf ,t in a parameter set { θ mf ,t } T .
Then we train on a subsequent
CIFAR100 task with the Eqt.
1 loss function, where we compute loss w.r.t.
the inputs and current timestep’s parameters, change in loss w.r.t.
a previous task between using the proposed parameters and the parameters last updated at that task (stored in the parameter set; this is enumerated for all past tasks in sequence), and the distance between the current parameters and all prior parameters.
Unlike our prior (multi-task) implementation, (i) we minimize the cosine distance between a current task’s parameters against prior parameters sequentially, not in parallel (i.e. the subspace end-points are sequentially fixed in the parameter space, we cannot dynamically move the subspace towards a different region), (ii) we do not gain visibility to all task parameters at once, and (iii) we are computing distance w.r.t.
multi-task parameters (i.e. each task’s parameters is applicable to its own and prior tasks).
No conditioning inputs Adaptation without conditioning inputs is a more difficult scenario; though it is used in adapt model parameters when the target distribution changes, it is not commonly used when the source distribution is constant (i.e. not used in adversarial/backdoor defenses).
Detecting when shift has occurred is a major implementation hurdle in most work tackling domain shift and task shift.
Conditioning inputs are not used w.r.t.
a single test-time distribution (adversarial/backdoor attacks), but are usually provided in meta/continual learning, or domain adaptation.
Many implementations use support sets in domain adaptation (Ganin et al., 2016b; Hoffman et al., 2017; Peng et al., 2019; Sun et al., 2015) and meta/continual learning (Finn et al., 2017) literature.
The implementations assume a task/domain T i is sampled from a task/domain distribution p ( T ) associated with a dataset D i , from which we sample a support set D si = ( X si , Y si ) and a query set D qi =
( X qi , Y qi ) , where a meta learner computes the optimal model parameters θ with respect to D si in order to predict accurately on D qi .
The support set distribution approximates the query set distribution here.
Other than support/query sets, conditioning inputs can manifest as known/seen embeddings: Cheung et al.
(2019) learns a set of parameters for each of the K tasks, where these parameters are stored in superposition with each other.
The task-specific models are accessed using task-specific “context” information C k that dynamically “routes” an input towards a specific model retrieved from this superposition.
The context vectors C k are the headers for each task, and they are all orthogonal to each other, hence making no assumption of a metric that can connect or predict these vectors on new or unseen tasks.
While there is work that aims to perform unsupervised domain adaptation (Ganin et al., 2016b; Zhou et al., 2021), domain generalization (Arpit et al., 2021; Cha et al., 2021) and unsupervised meta learning (Hsu et al., 2019; Khodadadeh et al., 2019) without explicit conditioning inputs, there is a gap between their performance in an unsupervised setting compared to methods in supervised or settings with conditioning inputs.
To exemplify the difficulty of inference without conditioning inputs at test-time, we can refer to the gap between correctly detecting near-OOD (inputs that are near but not i.i.d to the input distribution) and far-OOD (inputs that are far from the input distribution) inputs as being outliers.
For far-OOD, an AUROC close to 99% is attainable on CIFAR-100 (in) vs SVHN (out) (Sastry & Oore, 2020).
However, the AUROC for near-OOD is around 85% for CIFAR-100 (in) vs CIFAR-10 (out) (Zhang et al., 2020).
In other modalities, such as genomics, the AUROC of near-OOD detection can fall to 66% (Ren et al., 2019).
Thus, though removing the conditioning input assumption limits the maximum adaptation performance, it is a practical and realistic assumption we retain in our evaluation.
A.2.2 Distribution Shifts
Training regime Given that a large number of models are trained to evaluate this work (i.e. 3 models are trained in parallel per CPS configuration across all results), except for Figure 2 and Appendix
A.3.1 which were trained to the full 5000 epochs, all models trained in this work (including CPS and baselines) are early-stopped when training loss reaches 1.0 (test set accuracy ranging from 55-65%).
Though further convergence is possible, we make note of this computational constraint, and recommend evaluating the results in the context of clean test set accuracy presented in tables/figures.
All train-time procedures use a seed of 1.
All test-time procedures use a seed of 100.
The exception to both is any use of Random-BadNet (backdoor perturbations used in CPS, backdoor attack, random permutations and labels), where the seed is the index of perturbation set starting from 1.
The defender uses a train-test split of 80-20%.
For backdoor attack settings, the attacker uses a traintime-runtime split of 80-20%; this means the attacker contributes 80% of their private dataset (of which p, ε = 0 .is poisoned) to the defender’s dataset, and we evaluate the backdoor attack on the remaining 20% at test-time (of which 1.0 is poisoned).
All experiments were performed with respect to the constraints of 1 × NVIDIA GeForce RTX 3070.
Models In-line with work in loss landscape analysis (Fort et al., 2020), we opt to use the minimum model required to fit/generalize a model to a test set.
In-line with the implementation of Snoek et al. (2015) (to ensure the minimum CNN capcity to fit CIFAR100 test sets), we train each CNN with a cross entropy loss function and SGD optimizer for the following depth and width variations: 3-layer [16, 16, 16], 6-layer [16, 16, 16, 32, 32, 32], 9-layer [16, 16, 16, 32, 32, 32, 32, 32, 32], 6-layer-wide [256, 256, 256, 512, 512, 512].
Adversarial Attack We use the Projected Gradient Descent (PGD) attack (Madry et al., 2018).
With this attack method, adversarial perturbation rate ε a = 0 .
can sufficiently bring down the attack success rate comparable and similar to that of ε a = 1 . 0 .
Hence, we scale the perturbation against an upper limit 1 . [-
-]0 in our experiments, i.e. ε ′ a = 0 .
4 × ε a .
We computes perturbations with respect to the gradients of the defender’s model (a more pessimistic, white-box attack).
Backdoor Attack & Random Permutations As we would like different trigger perturbations and target poison labels, such as when sampling 100 non-identical random permutations/labels or constructing 3 backdoored sets for CPS training, we implement Random-BadNet (Datta & Shadbolt, 2022), a variant of the baseline dirty-label backdoor attack algorithm,
BadNet (Gu et al., 2019).
Alike to BadNet and many existing backdoor implementations in literature, Random-BadNet constructs a static trigger.
Instead of a single square in the corner as in the case of BadNet, Random-BadNet generates randomized pixels to return unique trigger patterns.
Attackers need to specify a set of inputs mapped to target poisoned labels { X i : Y poison i } ∈ D i to specify the intended label classification, backdoor perturbation rate ε i to specify the proportion of an input to be perturbed, and the poison rate p i = | X poison | | X clean | + | X poison | to specify the proportion of the private dataset to contain backdoored inputs, to return X poison
=
b i ( X i , Y poison i , ε i , p i ) .
In this work, the target poison labels are randomly-sampled, the perturbation rate and poison rate are p, ε = 0 .
Stylization We use the Adaptive Instance Normalization (AdaIN) stylization method (Huang &
Belongie, 2017), which is a standard method to stylize datasets such as stylized-ImageNet (Geirhos et al., 2019).
Dataset stylization is considered as texture shift or domain shift in different literature.
We randomly sample a distinct (non-repeating) style for each attacker.
α is the degree of stylization to apply; 1.0 means 100% stylization, 0% means no stylization; we set α = 0 .
We follow the implementation in Huang & Belongie (2017) and Geirhos et al.
and stylize CIFAR-10 with the
Paintings by Numbers style dataset.
We adapt the method for our attack, by first randomly sampling a distinct set of styles for each attacker, and stylizing each attacker’s sub-dataset before the insertion of backdoor or adversarial perturbations.
This shift also contributes to the realistic scenario that different agents may have shifted datasets given heterogenous sources.
Rotation At train-time we rotate the N sets according to their respective index out of N : [90, 0, 270, 180, 45, 135, 61, 315, 60, 315, 20, 75].
These rotation degrees were hand-picked and intended to be diverse and varied.
With seed 3407, At test-time, we randomly-sampled 100 seeded rotations: [20, 47, 77, 109, 304, 87, 304, 254, 146, 94, 98, 39, 306, 114, 267, 42, 231, 120, 40, 339, 352, 14, 264, 288, 203, 175, 308, 355, 324, 76, 213, 209, 167, 4, 170, 234, 120, 87, 43, 337, 300, 358, 29, 237, 107, 62, 84, 95, 9, 327, 203, 331, 1, 27, 59, 122, 52, 294, 64, 128, 263, 39, 141, 291, 25, 39, 176, 79, 104, 243, 265, 166, 270, 113, 23, 65, 297, 19, 196, 134, 119, 169, 42, 178, 250, 253, 276, 354, 291, 298, 20, 0, 343, 263, 164, 246, 217, 184, 163, 98].
Task Interpolation We evaluate interpolated tasks by linearly interpolating a set of tasks: for a set of tasks { X } N where each task X is a set of inputs x (and the number of inputs per task is identical, i.e. {{ x } M } N ), we linearly-interpolate between indexed inputs across a task set to return an interpolated task: ˆ x = (cid:80) Ni (cid:80)
Mj α
i x i,j .
In this work, we only interpolate between task sets with identical coarse labels (label-shared).
We do not interpolate batches, meaning we do not take a subset of one batch in a task and merge it with another subset of another batch in a task (i.e. all inputs are unperturbed, but each interpolated task contains distinct subsets of different task).
We do not implement this setting, as we present results on multi-task solutions, which are evaluated on multiple unperturbed tasks with a single parameter point-estimate, thus an interpolated batch setting is already evaluatable.
We linearly-space the range [0,1] into 5 segments (returning 5 3 different tasks).
We ignore the [0,0,0] case, as this results in a black image set, containing no identifiable task-specific features, and thus likely to return random labels - we leave this case out of our evaluation to filter out this noise.
For each interpolated image, we clip the colour values per pixel in the range [0, 255].
We keep the [1,1,1] case as, even though it could be all white theoretically, it would only be all white if each and every pixel exceeded 255; in practice (for example, as shown in Appendix A.2.1, pixels may contain colour channel values whose weighted sum per channel may still be less than 255.
We do not interpolate batches, meaning we do not take a subset of one batch in a task and merge it with another subset of another batch in a task (i.e. all inputs are unperturbed, but each interpolated task contains distinct subsets of different task).
We do not implement this setting, as we present results on multi-task solutions, which are evaluated on multiple unperturbed tasks with a single parameter point-estimate, thus an interpolated batch setting is already evaluatable.
[Figure]
⍺ 2 / ⍺ 1 ⍺0. ⍺ 1 / ⍺ 0 0.0 0.25 0.5 0.75 1. ⍺ 1 / ⍺ 0 0.0 0.25 0.5 0.75 1.
⍺ 1 / ⍺ 0 0.0 0.25 0.5 0.75 1. ⍺ 1 / ⍺ 0 0.0 0.25 0.5 0.75 1.
Appendix A. 1: Samples of interpolated images. The label of the variations of interpolated image across tasks is 4. The task end-points are: 3 tasks, 3 × same coarse label, train-time task set.
Single Test-time Distribution Dataset We use the CIFAR10 dataset, which contains 10 classes, 60,0000 inputs, and 3 colour channels (Krizhevsky, 2009).
For test-time perturbations, we evaluate on 100 different examples and tabulate the mean and standard deviation.
For train-time perturbations (backdoor), we insert a single backdoor trigger into the defender set before CPS initiated.
Multiple Test-time Distributions Dataset We use CIFAR100 dataset, a common baseline for meta/continual learning settings (Krizhevsky, 2009).
Similar to CIFAR10 except in that it contains 100 classes containing 600 images each.
There are 500 training images and 100 testing images per class.
The 100 classes in the CIFAR100 are grouped into 20 superclasses.
Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).
There are at most 5 tasks of the same coarse label (given there are 5 fine labels per class) per coarse label set.
For 5-label-per-set tasks, there are at most 5 different coarse label sets available (for 10-label-per-set tasks, there are at most 2).
To evaluate the use of CPS-regularization in hypernetworks for continual learning, the dataset is used in-line with the Split-CIFAR10/100 setting as described in Zenke et al.
(2017) and von Oswald et al.
N-label-set per task refers to the number of coarse labels contained/mapped in the task; 5(10)-label-set per task means we have 5(10) coarse labels mapped within the task, while 2 × 5(10)-label-set per task means we have 5(10) coarse labels mapped within the task but of 2 different fine label sets.
Distinctively-different coarse labels (per task) is a distinction from different coarse labels (of the same set) , where the former has a unique coarse label set assigned per task (no shared coarse labels across the tasks), while the latter has a different set of tasks that share coarse labels (but different fine labels).
We provide a breakdown of the labels (both coarse and fine) used per task; each task (as a self-contained training set/dataloader) is indicated by the square brackets [].
[Table]
[Table]
A.3 Supporting Results
[Figure]
Appendix A.3.1: Change in parameter subspace dynamics: Loss and cosine distance per
during
[Figure]
Appendix A.3.2: Change in parameter subspace dynamics: Loss and cosine distance per epoch during training of CPS
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Figure]
[Figure]
(b) β = 1 .
[Table] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
[Table]
Appendix A.3.7: CPS vs multiple test-
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]