Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
Authors: Ferran Alet
Maria Bauza Villalonga
Kenji Kawaguchi
Nurullah Giray Kuru
Tomas Perez
Leslie Pack Kaelbling
Venue: NeurIPS 2021 Poster (2021)
Keywords: meta-learning
inductive biases
self-supervised learning

Abstract: From CNNs to attention mechanisms, encoding inductive biases into neural networks has been a fruitful source of improvement in machine learning. Adding auxiliary losses to the main objective function is a general way of encoding biases that can help networks learn better representations. However, since auxiliary losses are minimized only on training data, they suffer from the same generalization gap as regular task losses. Moreover, by adding a term to the loss function, the model optimizes a different objective than the one we care about. In this work we address both problems: first, we take inspiration from transductive learning and note that after receiving an input but before making a prediction, we can fine-tune our networks on any unsupervised loss. We call this process tailoring, because we customize the model to each input to ensure our prediction satisfies the inductive bias. Second, we formulate meta-tailoring, a nested optimization similar to that in meta-learning, and train our models to perform well on the task objective after adapting them using an unsupervised loss. The advantages of tailoring and meta-tailoring are discussed theoretically and demonstrated empirically on a diverse set of examples.

Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
Ferran Alet, Maria Bauza, Kenji Kawaguchi,
Nurullah Giray Kuru, Tomás Lozano-Pérez, Leslie Pack Kaelbling
MIT
{alet, bauza, kww.wucw,wwaaaaaaooaooo
tl, 1mpp.itt..m....... edu
Abstract
From CNNs to attention mechanisms, encoding inductive biases into neural net- works has been fruitful source of improvement in machine learning. Adding auxiliary losses to the main objective function a general way of encoding biases that can help networks learn better representations. However, since auxiliary losses are minimized only on training data, they suffer from the same generalization gap regular task losses. Moreover, by adding term to the loss function, the model optimizes a different objective than the one we care about. In this work we address both problems: first, we take inspiration from transductive learning and note that af- ter receiving an input but before making prediction, we can fine-tune our networks any unsupervised loss. We call this process lailoring, because customize the model to each input to ensure our prediction satisfies the inductive bias. Second, we formulate meta-tailoring, a nested optimization similar to that in meta-learning. and train our models to perform well on the task objective after adapting them using an unsupervised loss. The advantages of tailoring and meta-tailoring are discussed theoretically and demonstrated empirically on diverse set of examples.
Introduction
The key to successful generalization machine learning is the encoding of useful inductive biases. A variety of mechanisms, from parameter tying to data augmentation, have proven useful to improve the performance of models. Among these, auxiliary losses can encode a wide variety of biases, constraints, and objectives; helping networks learn better representations and generalize more broadly. Auxiliary losses add an extra term the task loss that is minimized over the training data.
However, they have two major problems:
1. Auxiliary losses are only minimized at training time, but not for the query points. This leads generalization gap between training and testing, in addition to that of the task loss.
2. By minimizing the sum of the task loss plus the auxiliary loss, we are optimizing a different objective than the one
about (only the task loss).
In this work propose solution to each problem:
1. We use ideas from transductive learning to minimize unsupervised auxiliary losses each query, thus eliminating their generalization gap. Because these losses are unsupervised, we can optimize them any time inside the prediction function. We call this process tailoring, since we customize the model to each query.
2. We use ideas from meta-learning to learn model that performs well the task loss after being tailored with the unsupervised auxiliary loss; i.e. meta-tailoring. This effectively trains the model to leverage the unsupervised tailoring loss in order to minimize the task loss.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Transductive Learning
(x arg min Σ
L (fo(zs). v.)+ Σ
C Luneu (fo (fo(1))01))
(g)))}
Inductive Learning
((xi, Wi));
arg min
Coup (fo(as))
fa(2)
Tailoring
((ri, 8i));
arg min
Σ caup (fo(())))))))))
Eam
org min Clailue (c 0=0
I (x)
x
Meta-tailoring
a
((x,, y:)) ang min
Σ coupps re Sai)) Bi)
arg min Ltailur (zc, 0) 0=00
(x)
0
Oz
Figure 1: Comparison of several learning settings with offline computation in the orange boxes and online computation in the green boxes, with tailoring in blue. For meta-tailoring training. T6, Cuallr 2)) = argmingr=s Lialor (z, 8') represents the tailoring process resulting in fg.
Illustrative example Imagine you want to neural network predict the motion of planetary system: given the positions and velocities of each planet, the network predicts their future positions and velocities. Additionally, we could encode energy and momentum conservation by adding an auxiliary loss encouraging the neural network to conserve energy and momentum for the training examples. However, this does not guarantee that the network will conserve them for test queries. Alternatively, we can exploit that evaluating these conservations requires comparing only the input with the prediction without needing access the true target. Therefore, we can enforce these conservations by optimizing an unsupervised objective within the prediction function. In doing so, we tailor the model to each individual query to ensure it satisfies energy and momentum conservation. Taking into account this prediction titetiooo-tmeeeeee adaptation during training leads to two-layer optimization, where we train make accurate predictions after encouraging the physical conservations.
Tailoring predictor Traditionally, supervised learning is approached within the inductive learning framework, shown in the second row of Figure 1. There, an algorithm consumes a training dataset of input- output pairs, ((I;, 34)) y=)) and produces set of parameters e by minimizing a supervised loss Σn Lum (fo (zz),, y:) and, optionally, an unsupervised auxiliary loss Σx= Cumsup(8 (0, zz). These parameters specify a hypothesis f-) that, given a new input æ, generates an output U fg (z). This problem setting misses substantial opportunity: before the learning algorithm sees the query point x, it has distilled the data down to the parameters Ô, which are frozen during inference, and so cannot use new information about the particular that it will be asked to make a prediction for.
Vapnik recognized an opportunity to make more accurate predictions when the query point known, in framework that is now known as transductive learning [50, 111, illustrated in the top row of Figure 1. In transductive learning, single algorithm consumes both labeled data, ((x,, y: w.)) and a set of input queries for which predictions are desired, (x:()) and produces predictions (y()) , for each query. In general, however, we do not know queries priori, and instead, we want an inductive function that makes predictions online, as queries arrive. To obtain such an online prediction function from transductive system, we would need to take the training data and the single unlabeled query and encapsulate the entire transductive learning procedure inside the prediction function itself. This strategy would achieve our objective of taking x into account at prediction time but would be computationally much too slow [12).
This approach for combining induction and transduction would reuse the same training data and ob- jective for each prediction, only changing the single unlabeled query. Consequently, it would perform extremely similar computations for each prediction. Therefore, we propose to effectively reuse the shared computations and find "meta- -nptt-hypotheeis" that can then be efficiently adapted to each query. As shown in the third row of Figure 1, we propose to first run regular supervised learning to obtain parameters ô. Then, given a query input we fine-tune á on an unsupervised loss ftallor to obtain cus-

---

Algorithm MAMmoTh: Model-Agnostic Mail-Taig
Subroutine Training( f. L.uu, Asugo Ltailor, Atattrr, DD Darains b)
randomly initialize
while not done do
Sample batch of samples (I Mi) ~ DD Derain
forall (zi, 344 do
// Inner step with tailor loss 0 = 0 Asup V (().g))) Lup (foz, (1e). // Outer step with supervised loss
0 24
Ataitor V Veliullu (0, xii
return 0
tomized parameters 0g and use them to make the final prediction: Se, (x)). We call this process tailor- ing, because we adapt the model to each particular input for customized fit. Notice that tailoring opti- mizes the loss at the query input, eliminating the generalization gap on the unsupervised auxiliary loss.
Meta-tailoring Since will be applying tailoring prediction time, it is natural to incorporate this adaptation during training, resulting in a two-layer optimization similar to those used in meta- learning. Because of this similarity, call this process mela-tailoring, illustrated in the bottom row of Figure Now, rather than letting 0 be the direct minimizer of the supervised loss, we set it to
i arg min L (f 2.) (2)) v.).
Here, the inner loop optimizes the unsupervised tailoring loss Ciailor and the outer loop optimizes the supervised task loss L Notice that now the outer process optimizes the only objective care, _ instead of proxy combination of Loup and Cunsup, At the same time, we learn to leverage Gtallor in the inner loop to affect the model before making the final prediction, both during training and evaluation. Adaptation is especially clear in the case of a single gradient step, as in MAML [19]. We show its translation, MAMmoTh (Model-Agnostic Meta- Meta-Tailoring). in algorithm
In many settings, we want to make predictions for a large number of queries in (mini-)batch. While MAMmoTh adapts to every imput separately, it can only be run efficiently in parallel in some deep learning frameworks, such as JAX [10]. Inspired by conditional normalization (CN) [18] we propose CNGRAD, which adds element- wise affine transformations to model and only adapts the added parameters in the inner loop. This allows us to independently tailor the model for multiple inputs in parallel. We prove theoretically, in Sec. 4, and provide experimental evidence, in Sec. 5.1, that optimizing these parameters alone has enough capacity to minimize a large class of tailoring losses.
Relation between (meta- Mtalorig,,,,,,ng,, fine-tuning transfer, and meta-learning Fine-tuning pre- trained networks a fruitful method of transferring knowledge from large corpora to smaller related datasets [17]. This allows us to reuse features related tasks or for different distributions of the same task. When the data we want to adapt to is unlabeled, we must use unsupervised losses. This can be useful to adapt to changes of task [16], from simulated to real data [52], or to new distributions [46].
Tailoring performs unsupervised fine-tuning and is, in this sense, similar to test-time train- ing(TTT) [46] for a single sample, which adapts to distribution shifts. However, tailoring is applied to single query; not to a data set that captures distribution shift, where batched TTT sees most of its benefits. Thus, whereas regular fine-tuning benefits from more adaptation data, tailoring would be hindered by adapting simultaneously to more data. This is because tailoring aims at building a custom model for each query to ensure the network satisfies particular inductive bias. Customizing the model to multiple samples makes it harder, not easier. We show this in Figure 2. where TTT with 6400 samples performs worse than tailoring with single sample. Furthermore, tailoring adapts to each query one by one, not globally from training data to test data. Therefore, it also makes sense to do tailoring on training queries (i.e., meta-tailoring).
Meta-tailoring has the same two-layer optimization structure as leaanne. More concretely, can be understood as the extreme case of meta- meta-learning where each single-query prediction is its own task. However, whereas meta-learning tasks use one loss and different examples for the inner and outer loop, meta-tailoring tasks use one example and different losses for each loop (Ltallor LadP). We emphasize that meta-tailoring does not operate in the typical multi-task meta-learning setting. Instead, we are leveraging techniques from meta-learning for the classical single-task setting.
3

---

Contributions In summary, our contributions are:
1. Introducing tailoring, a new framework for encoding inductive biases by minimizing unsuper- vised losses at prediction time, with theoretical guarantees and broad potential applications.
2. Formulating mela-tuilorin, which adjusts the outer objective to optimize only the task loss, and developing a new algorithm, CNGRAD, for efficient meta-tailoring.
3. Demonstrating mela-tailoring in 3 domains: encoding hard and soft conservation laws in physics prediction problems (Sec. 5.1 and Sec. 5.2), enhancing resistance to adversarial examples by increasing local smoothness at prediction time (Sec. 5.4), and improving prediction quality both theoretically (Sec. 3.1) and empirically (Sec. 5.3) by tailoring with a contrastive loss.
2 Related work
Tailoring is inspired by transductive learning. However, transductive methods, because they operate on a batch of unlabeled queries, are allowed to make use of the underlying distributional properties of those queries, as in semi-supervised learning [12]. In contrast, tailoring does the bulk of the computations before receiving any query; vastly increasing efficiency. Similar to tailoring, local learning [9] also has input-dependent parameters. However, it uses similarity in raw input space to select few labeled data points and builds local model instead of reusing the global prior learned across the whole data. Finally, some methods [21, 33] in meta-learning propagate predictions along the test samples in semi-supervised transductive fashion.
Similar to tailoring, there other learning frameworks that perform optimization at prediction time for very different purposes. Among those, energy-based models do generative modeling [2, 27, 32] by optimizing the hidden activations of neural networks, and other models [4, 49] learn to solve optimization problems by embedding optimization layers in neural networks. In contrast, tailoring optimizes the parameters of the model, not the hidden activations the output.
As discussed in the introduction, unsupervised fine-tuning methods have been proposed to adapt to different types of variations between training and testing. Sun al. [46] propose adapt to change of distribution with few samples by unsupervised fine-tuning at test-time, applying it with loss of predicting whether the input has been rotated. Zhang et al. [54] build on it to adapt to group distribution shifts with a learned loss. Other methods in the few-shot meta-learning setting exploit test samples of new task by minimizing either entropy [16] or a learned loss [5] in the inner optimization. Finally, Wang et al. [51] use entropy in the inner optimization to adapt to large-scale variations in image segmentation. In contrast, we propose (meta-)tailee ttallriig as a general effective way to impose inductive biases in the classic machine learning setting. Whereas in the aforementioned methods, adaptation happens from training to testing, we independently adapt to every single query.
Meta-learning [44, 7, 48, 28] has the same two-level optimization structure as meta-tailoring but focuses multiple prediction tasks. As shown in Alg. 1 for MAML [19], most optimization-basee meta-learning algorithms can be converted to meta-tailoring. Similar to CNGRAD, there are other meta-learning methods whose adaptations can be batched [40, 3]. Among these, [55, 41] train FiLM networks [39] to predict custom conditional normalization (CN) layers for each task. By optimizing the CN layers directly, CNGRAD is simpler, while remaining provably expressive (section 4). CNGrad can also start from a trained model by initializing the CN layers to the identity function.
3 Theoretical motivations of mata-iiig
In this section, study the potential advantages of meta-tailoring from the theoretical viewpoint, formalizing the intuitions conveyed in the introduction. By acting symmetrically during training and prediction time, meta-tailon meta-tailoiing allows us closely relate its training and expected losses, whereas tailoring alone does not have the same guarantees. First, we analyze the particular case of a contrastive tailoring loss. Then, we will generalize the guarantees to other types of tailoring losses.
3.1 Meta-tailoring with a contrastive tailoring loss
Contrastive learning [24] has seen significant successes in problems of semi-supervised learning [37, 26, 13]. The main idea is to create multiple versions of each training image and learn a representation which variations of the same image are close while variations of different images are far apart. Typ- ical augmentations involve cropping, color distortions, and rotation. We show theoretically that, under reasonable conditions, meta-tailoring using a particular contrastive loss Coom as Cuallor Leom helps us improve generalization errors in expectation compared with performing classical inductive learning.

---

When using teta-tailoring, we define f to be the Oz obtained with a training dataset S ((x,, k)) and tailored with the contrastive loss at the prediction point æ. Theorem I provides an upper bound on the expected supervised loss Ex,y [ (r), v)] in terms of the expected contrastive loss En [z, 02, (analyzed in App. B), the empirical supervised loss 100:-~0P[o0 (J (Ti), 34) of meta-tailoring, and its uniform stability Ç. Theorem 6 (App. C) provides similar bound with the Rademacher complexity [6] Rn (C o F) of the set C a F, instead of using the uniform stability Ç. Proofs of all results in this paper are deferred to App. C.
-
Definition 1. Let s ((1; Mi and 5' = ((x',, of) be any two training datasets that differ by a single point. Then, a meta-tailoring algorithm + Jun.s (x) is uniformly (-stable v(z, v) E X' x V, 1 (x), (So (a), S -
Theorem 1. Let S + fer.s (2) be uniformly 5-stable meta-tailoring algorithm. Then, for any 5 > O, with probability at least - 5 over an ii.d. draw of n i..dd samples S = ((11, y))) the following holds: for any K E [0, 1). Ez,y ( (fe.s (z), & K,E, [f (I, 6 0x,s)] + (1 K)J,
where - + ( (So.,. (Ii), 91) + $ + +52+(2222 (2(+c)y c)v (lm())))(((m)))))) (In(1/6))/(22)) and is the upper bound on the per-sample loss as C (fo(() c.
In the case of regular inductive learning, we get bound of the exact same form, except that we have a single l instead of a 0g tailored to each input x. This theorem illustrates the effect of meta-tailoring
contrastive learning, with its potential reduction of the expected contrastive loss E, (.roooom( (z, S2. In classic induction, we may aim to minimize the empirical contrastive loss 2/2 - (T, 0) with ń potentially unlabeled training samples, which incurs the additional generalization error of Ez [2romn(21, [=.s))]++ Σ Lcom (zi, 0). In contrast, meta-tailoring can avoid this extra generalization error by directly minimizing a custom Oz on each x: Ez (I. 0s,s)].
In the case where Egy [Coon( (21, 0r.s)] )] is left large (e.g., due to large computational cost), Theorem 1 still illustrates competitive generalization bounds of meta-tailoring with small k. For example, with 0, it provides generalization bounds with the uniform stability for meta-tailoring algorithms. Even then, the bounds are not equivalent to those of classic induction, and there are potential benefits of meta-tailoring, which are discussed in the following section with a more general setting.
3.2 Meta-tailoring with general tailoring losses
The benefits of meta- veta-tailo go beyond contrastive learning: below we provide guarantees for meta-tailoring with arbitrary pairs of tailoring loss Ctailor (zz, 0) and supervised loss L (fo(z),
Remark 1. For any function &p such that Ezz.y (APT (fo(s). v)] S E (o (atalor 0)), Theorems / and 6 hold with the map feom being replaced by the the function o Cuillr
This remark shows the benefits of meta-tailoring through its effects on three factors: the expected unlabeled loss Ex, [ (Ir, Oc.s))], uniform stability Ç. and the Rademacher complexity R, (Luup F). It is important to note that meta-tailoring can directly minimize the expected unlabeled loss E (zalloo {).s whereas classic induction can only minimize its empirical version, which results in the additional generalization error on the difference between the expected unlabeled loss and its empirical version. For example, if yp is monotonically increasing and Ciailor (zz, 0) represents the physical constraints at each input x (as in the application in section 5.1), then classic induction requires a neural network trained to conserve energy at the training points to generalize to also conserve it at unseen (e.g., testing) points. Meta- Meta-tai avoids this requirement by directly minimizing violations of energy conservation at each point at prediction time.
Meta-tailoring can also improve the parameter stability Ço defined such that ((x,y) E xx'y,, EXx,2,100s 01 < ft, for all S,S' differing by a single point. When 0z.s = 0g - AV Ctailor (x, 0s). we obtain improvement the parameter stability Çe if V V'Luilorrz, 0g) can pull 0g and 0g: closer so that ||0,,5 0r ||0g 0s which is ensured, for example, if || = 1/2 and cos cos_dist(v), U2) Tozil
2 where cos cos_dist(v), U2) is the cosine similarity of Uj and U2 with 01 = 0g 0s., U22 A(V (wilorr,, 0s)= - v Cuilorr (I, 0s.))) and U2 + O. Here, the uniform stability Ç and the parameter stability Ço are closely related as { S CCo, where C is the upper bound on the Lipschitz constants of the maps 0 + (fo over all (I, E X' x y under the norm || ll. since IC (x).v) (Se (z), (r), & C||02,5 s 0z.s: CCo.
|elll
5

---

Algorithm 2 CNGRAD for meta-tailoring
Subroutine Training( f. L Asup ftallor, Ataitor, steps,Dirat Draii. // Only in meta-tailoring 11 All parameters except 7,9; trained in outer loop
randomly initialize w
while not done do
X, Y,Y- ~ iraai gradw 0
1/ Sample batch; initialize outer gradient 1/ Initialize CN layers to the identity
%o = 1,,5, mii 30 Oz.Σ. mi
for 1 S 8 S sleps do
1s 7 Ataitor
V ftailoor 70-1, X)
// Inner step w.r.t. Y // Inner step w.r.t. B // Only in 1st order CNGrad // Outer gradient w.r.t. w 11 Apply outer step after all inner steps
3. = 3a-1 Ataitor
V v Ciuilor 7s 11 B,-1, X)
7s. 3g = "s.detach O) (s.detach()
gradæ = gradw + V Loup (SwiT &, (X),))
W = w Xau..rrad
return w
Subroutine Prediction( w. Ltailar, X, steps, X) // Both in meta-tailoring tailoring
% = sx..hhpee0), m.i Po = Xx.shape(U), my
for 1 S 8 I sleps do
7, = 7 AV Qailoor (: 7/- 11- X) 8- = BB-1 - AV
Ctailor (0,:7s-1 10--1, B, X)
return fw:tsttussBaaeep. (X)
CNGRAD: a simple algorithm for expressive, efficient (meta-)tailoring
In this section, we address the issue of using mmnta-)taiilornn for efficient GPU computations. Although possible in JAX [10], efficiently parallelizing MAMmooTh across inputs is not possible in other frameworks. To overcome this issue, building on CAVIA [55] and WarpGirad [20), we propose CNGRAD which adapts only conditional normalization parameters and enables efficient GPU computations for (meta-)talloring CNGRAD can also be used in meta-learning, providing a parallelizable alternative to MAML (see App. D).
As done in batch-norm [30] after element-wise normalization, we can implement an element-wise affine transformation with parameters (7. ()), scaling and shifting the output n (ax) of each k-th neu- ron at the 1-th hidden layer independently: 7k f (x)+
+ + In conditional normalization, Dumoulin et al. [18] train a collection of (7, B) in a multi-task fashion to learn different tasks with single network. CNGRAD brings this concept to the meta-learning and (meta-)tailonis
settings and adapts the affine parameters (y, (3) to each query. For meta-tailoring, the inner loop minimizes the tailoring loss at an input æ by adjusting the affine parameters and the outer optimization adapts the rest of the network. Similar to MAML [19], we implement first-order version, which does not backpropagate through the optimization, and a second-order version, which does. CNGRAD efficiently parallelizes computations of multiple tailored models because the adapted parameters only require element-wise multiplications and additions. See Alg. 2 for the pseudo-code.
CNGRAD is widely applicable since the adaptable affine parameters can be added to any hidden layer and only represent a tiny portion of the network (empirically, around 1%). Moreover, we can see that, under realistic assumptions, affine parameters. To analyze properties of these adaptable affine parameters, let us decompose 0 into 0 (w, y, B), where contains all the weight parameters (including bias terms), and the (7; B) contains all the affine parameters. Given an arbitrary function (fo(z), x) + fraitor(Ja (x),, æ), let guallr (x. f) = En, ( (x)), v)), where g (1))(ma) are arbitrary input augmentation functions prediction time.
minimize the inner tailoring loss using only the
Corollary states that for any given iù we add any non-degenerate Gaussian noise 8 as + 8 with zero mean and any variance on d, the global minimum value of Ciailer w.r.t. all parameters (w, 7, can be achieved by optimizing only the affine parameters (7, 3). with probability one. In other words, the CN parameters (y, have enough capacity to optimize optimize the inner tailoring loss.
Corollary 1. Under the assumptions of Theorem 2, for any E Rdd, with probability one over randomly sampled 5 E T accordingly to any non-degenerate Gaussian distribution, the following holds: inf Ciaiar (I, 7, B) = inf,,e quailo(z, v + 8, 7, 3) for any I X.
The assumption and condition in thcorem 2 are satisfied in practice (see App. A). Therefore, CNGRAD is a practical and computationally efficient method to implement (meta-) Mtiloing,

---

Method
loss
relative
Inductive learning
.041
Outer prediction loss
Opt. output(50 st.)
.041
(0.7 = 0.1)%
0.03
6400-s. TT0(50 st.)
.040
- (3.6 0.2)%
Tailoring(1 step)
.040
(1.9 0.2)%
0.02
We minimize the tailoring loss the inner
Tailoring(5 steps)
.039
(6.3 + 0.3)%
Tailoring(10 (10 st.)
.038
(7.5 0.10%
Optimizing the output
Meta-tailoring(O st.)
.030
(26.3 +.3)%
6400-sample TTT
Meta-tailoring(1 st.)
.029
Tailoring
Meta-tailoring(5 st.)
.027
(35.3 + 2.6)%
Meta-tailoring
0.00
Meta-tailoring 10 s.)
.026
(36.0 + 2.6)%
0.05
0.10 0.15
0.30
0.35
Physics tailoring loss
Table I: Test MSE loss for different methods; the second column shows the relative improvement over basic inductive supervised learning. The test- time training (TTT) baseline uses a full batch 6400 test samples to adapt, not allowed in regular SL. With few gradient steps, tailoring signifi- cantly over- ver-performs all baselines. Meta-tal tailoring improves even further, with 35% improvement.
Figure 2: the planet data; each path going from right to left as minimize the physics tailoring loss. We use a small step size to illustrate the path. Tailoring and the two baselines only differ in their test-time computations, thus sharing their starts. Meta-tailoring has lower starting loss, faster optimization, and no overfitting during tailoring.
Optimization at prediction time on
Experiments
5.1 Tailoring to impose symmetries and constraints at prediction time
Exploiting invariances and symmetries is an established strategy for increasing performance in ML. During training, we can regularize networks to satisfy specific criteria; but this does not guarantee they will be satisfied outside the training dataset [45]. (Meta- still-itailoiing provides a general solution to this problem by adapting the model to satisfy the criteria at prediction time. We demonstrate the use of tailoring to enforce physical conservation laws for predicting the evolution of a 5-body planetary system. This prediction problem is challenging, as m-body systems become chaotic for m 2. We generate a dataset with positions, velocities, and masses of all 5 bodies as inputs and the changes in position and velocity as targets. App. further describes the dataset.
Our model is 3-layer ferd-forward network. We tailor it by taking the original predictions and adapt- ing the model using the tailoring loss given by the L1 loss between the whole system's 's initial and final energy and momentum. Note that ensuring this conservation does not guarantee better performance: predicting the input as the output conserves energy and momentum perfectly, but it is not correct.
While tailoring adapts some parameters in the network to improve the tailoring loss, an alternative for enforcing conservation would be to adapt the output y value directly. Table 1 compares the predictive accuracy of inductive learning, direct output optimization, and both tailoring and meta-tailoring, using varying numbers of gradient steps. Tailoring is more effective than adapting the output, as the parameters provide prior on what changes are more natural. For meta-tailoring, we try both first-order and second-order versions of CNGRAD. The first-order gave slightly better results, possibly because it was trained with a higher tailor learning rate (10- with which the second-orde version was unstable (we thus used 10- )). More details can be found App. E.
Finally, meta-tailoring without any query-time tailoring steps already performs much better than the original model, even though both have almost the same number of parameters and can overfit the dataset. We conjecture meta-tailoring training adds an inductive bias that guides optimization towards learning more generalizable model. Fig. 2 shows prediction-time optimization paths.
5.2 Tailoring softly encourage inductive biases
A popular way of encoding inductive biases is with clever network design to make predictions translation equivariant (CNNs). permutation equivariant (GNNs), or conserve energy [23]. However, an inductive bias is only partially satisfied, such approaches overly constrain the function class. Instead, tailoring can softly impose this bias by only fine-tuning the tailoring loss for a few steps.
7

---

We showease this in the real pendulum experiment used by Hamiltonian Neural Networks (HNNs) [23]. HNNs have energy
MSE between coordinates
Charge in HNN--conserved quantity
0.02-
Staciine 5o:
-0.1
conservation built-in and easily improve
0.015
-0.2
vanilla MLP. We meta-tailor this vanilla MLP with energy conservation without changing its architecture. Meta-tailoring significantly improves over the baseline and HNNs, since it can encode the imperfect energy conservation of real systems. We compare results in Fig. 3 and provide extra details in App. F. Note that, with inexact losses, fully enforcing them provides sub-optimal results. Thus, we pick the tailoring prediction loss during training.
0.010
0.05-
-0.4
0.000
Time
Time
Figure 3: By softly encouraging energy conser- vation, meta-tailoring improves over models that don't 't and models that fully impose it.
learning rate that results in the lowest long-term
5.3 Tailoring with a contrastive loss for image classification
Following the setting described in section 3.2, we provide
93
experiments on the CIFAR-10 dataset [31] by building on SimCLR [13]. SimCLR trains ResNet-50 [25] fo(-)
92
coupled to a small MLP g()) such that the outputs of two augmentations of the same image Ty, I (xx) agree; i.e.
91
Accuracy
((o(())) x (fo(rz))) This is done by training g(f()))
90
to recognize one augmentation from the other among a big batch of candidates with the cross-entropy loss. To
89
show that the unsupervised training of fo provides a useful representation, SimCLR trains a single linear layer on top of it, (f)),, achieving good classification results.
86
Meta-toilored SimCLR
SimCLR
50
150
500
1500
5000
Points class (Ing axis)
We now observe that we can tailor fo at prediction -ime by optimizing sSoo (x)),, which quantifies the agreement between different augmentations of the same input; thus 'learning' about its particularities. To make the image classification prediction, we feed the final tailored rep- resentation to the linear layer: offe. (z)). To match the evaluation from SimCLR, we do not redo SimCLR's un- supervised learning, which provides 0. The meta-tailoring outer loop trains & to take the tailored representations fe_ (I) instead of the original fo(r). Thus, 0 is unsupervisedly fine-tuned in the prediction function leading to 0z, but never supervisedly trained as this would break the evaluation protocol (in meta-tail tailoring's favor). We also implement a TTT [46] baseline with their original cotation-predictio loss. Moreover, TTT modifies 02 at test time, but does not take this adaptation into account when training & (see App. G for more details). TTT worsened base SimCLR despite significant
Figure 4: Meta-tailoring the linear layer with the contrastive loss results in con- sistent accuracy gains between 0.5% and 0.8%. This approximately the same gain as that of doubling the amount of la- beled data (note the logarithmic x-axis).
tuning. We conjecture this because TTT was designed for OOD generalization, not in in-distribution. In contrast, as shown in Fig. 4, we observe that meta-tailoring provides improvements over base SimCLR equivalent to doubling the amount of labeled data.
5.4 Tailoring for robustness against adversarial examples
Neural networks are susceptible to adversarial examples [8, 47]: targeted small perturbations of an input can cause the network to misclassify it. One approach is to make the prediction function smooth via adversarial training [34]; however, this only ensures smoothness in the training points. Constraining the model to be smooth everywhere makes it lose capacity. Instead, (meta-) Itailorng asks for smoothness a posteriori, only on specific query.
We apply meta-tailoring to robustly classifying CIFAR-10 [31] and ImageNet [15] images, tailoring predictions so that they are locally smooth. This is similar to VAT [36] but instead optimizes the loss within the prediction function, not an auxiliary loss. Inspired by the notion of adversarial examples being caused by predictive, but non-robust, features [29], we meta-tailor our model by enforcing smoothness on the vector of features of the penultimate layer (denoted go (x))::
failor (x, 0) E|coos dist (I), (a + 5))], 5 ~ N N 2²)
8

---

Method
0.0
0.5
1.0
1.5
2.0
2.5
3.0
ACR
(Inductive) Randomized Smoothing
0.67
0.49
0.00
0.00
0.00
0.00
0.00
0.470
0.25
Meta-tailored Randomized Smoothing
0.72
0.55
0.00
0.00
0.00
0.00
0.00
0.494
(Inductive) Randomized Smoothing Meta-tailored Randomized Smoothing
0.57
0.46
0.37
0.29
0.00
0.00
0.00
0.720
0.50
0.66
0.54
0.42
0.31
0.00
0.00
0.00
0.819
(Inductive) Randomized Smoothing
0.44
0.38
0.33
0.26
0.19
0.15
0.12
0.863
1.00
Meta-tailored Randomized Smoothing
0.52
0.45
0.36
0.31
0.24
0.20
0.15
1.032
Table 2: Fraction of points with certificate above different radii for ImageNet. Meta-tailoring improves average certification radius (ACR) of different models by 5.1%, 13.8%, 19.6%. Results for Randomized Smoothing come from [53].
We build on Cohen et al. [14], who developed method for certifying the robustness of model via randomized smoothing (RS). RS samples points from a Gaussian ((x, a²) around the query and, if there is enough agreement in classification, it provides certificate that small perturbation cannot adversarially modify the query to have a different class. We show that meta-tailoring improves the original RS method, testing for a 0.25, 0.5, 1.0. We use v = 0.1 for all experiments. We initialized with the weights of Cohen et al. [14] by leveraging that CNGRAD can start from pre-trained model by initializing the extra affine layers the identity. Finally, we use o' =v n² = 0.23, 0.49, 0.995 so that the points used in tailoring loss come from N(x, o²).
Table 7 shows our results on CIFAR-I0 where we improve the average certification radius (ARC) by 8.6%, 10.4%, 19.2% respectively. In table 2, we show results on Imagenet where we improve the ARC by 5. 5.1%, 13.8%, 19.6% respectively. We chose to meta-tailor the RS method because it represents strong standard in certified adversarial defenses, but note that there have been advances on RS that sometimes achieve better results than those presented here [53, 43], see App. I. However, it is likely that meta-tailoring could also improve these methods.
These experiments only scratch the surface of what tailoring allows for adversarial defenses: usually, the adversary looks at the model and gets to pick particularly bad perturbation + 6. With tailoring, the model responds, by changing to weights 0246- This leads to game, where both weights and inputs are perturbed, similar to max minjai<e, <c. L (fe+s (z + 8), v). However, since we don't get to observe y; we optimize the weight perturbation by minimizing L instead.
Discussion
6.1 Broader Impact
Improving adversarial robustness: having more robust and secure ML systems is mostly a positive change. However, improving adversarial defenses could also go against privacy preservation, like the use of adversarial patches to gain anonymity from facial recognition. Encoding desirable properties: By optimizing an unsupervised loss for the particular query we care about, it casier to have guarantees on the prediction. In particular, there could be potential applications for fairness, where the unsupervised objective could enforce specific criteria at the query or related inputs. More research needs to be done to make this assertion formal and practical. Potential effect on privacy: tailoring specializes the model to each input. This could have an impact on privacy. Intuitively, the untailored model can be less specialized to each input, lowering the individual information from each training point contained in the model. However, tailored predictions extract more information about the queries, from which more personal information could be leaked.
6.2 Limitations
Tailoring provides framework for encoding a wide array of inductive biases, but these need to be specified formula by the user. For instance, would be hard programatically describe tailoring losses in raw pixel data, such as mass conservation in pixel space. Tailoring also incurs an extra time cost at prediction time, since we make an inner optimization inside the prediction function. However, as shown in Table 1, metta-tiiorinn often achieves better results than inductive learning even without adaptation at test-time, enabling better predictions at regular speed during test-time. This is due to meta-tailoring leading to better training. Moreover, optimization can be sped up by only tailoring the last layers, as discussed in App. D. Finally, to the best of our knowledge using MAMmoT's Th for meta-tailoring would be hard to parallelize in PyTorch [38] and Tensorflow [1]: we

---

proposed CNGRAD to make it easy and efficient. JAX[10], which handles per-example weights, makes parallelizing tailoring effortless.
Theory in Sec. 3 applies only to meta-tailoring. Unlike tailoring (and test-time training), meta- tailoring performs the same computations training and testing time, which allows prove the results. Theorem 2 proves that optimizing the CN layers in CNGRAD has the same expressive power as optimizing all the layers for the inner (not outer) loss. However, does not guarantee that gradient descent will find the appropriate optima. The study of such guarantee is left for future work.
6.3 Conclusion
We have presented tailoring, simple way of embedding powerful class of inductive biases into models, by minimizing unsupervised objectives at prediction time. Tailoring leverages the generality of auxiliary losses and improves them two ways: first, it eliminates the generalization gap on the auxiliary loss by optimizing it on the query point; second, tailoring only minimizes task loss in the outer optimization and the tailoring loss in the inner optimization. This results in the model optimizing the only objective we care about in the outer loop, instead of proxy loss. Beyond inductive biases, tailoring shows that model adaptation is useful even when test queries comes from the same distribution as the training data. This suggests one can improve models by performing prediction-time optimization, trading off large offline data&compute efforts with small online computations.
Tailoring is broadly applicable, as one can vary the model, the unsupervised loss, and the task loss. We show its upplicability in three diverse domains: physics prediction time-series, contrastive learning, and adversarial robustness. We also provide a simple algorithm, CNGRAD, to make meta- veta-taiio practical with little additional code. Currently, most unsupervised or self- supervised objectives are optimized in task-agnostic ways; without taking into account the supervised downstream task. Instead, meta-tailoring provides generic way to make these objectives especially useful for each application. does so by learning how to best leverage the unsupervised loss to perform well on the final task we care about.
Acknowledgments and Disclosure of Funding
We would like to thank Kelsey Allen, Marc de la Barrera, Jeremy Cohen, Dylan Doblar, Chelsea Finn, Sebastian Flennerhag, Jiayuan Mao, Josh Tenenbaum, and Shengtong Zhang for insightful discussions. We would also like to thank Clement Gehring for his help with deploying the experiments and Lauren Milechin for her help with leveraging the MIT supercloud platform [42].
We gratefully acknowledge support from NSF grant 1723381; from AFOSR grant 1--50017-1--015;; from ONR grant N00014-188 180014-18-1-0 18-1-7777; from the Honda Research Institute, from MIT-IBM Watson Lab; and from SUTD Temasek Laboratories. We also acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the reported research results. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our sponsors.
References
[1] Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ( ((ODDI) 16), pp. 265-283, 2016.
[2] Ackley. H., Hinton, E., and Sejnowski. T. A learning algorithm for boltzmann machines. Cognitive
science, 9(1) 1985.
[3] Alet, F., Weng, E., Lozano-Perez, T., and Kaelbling, L. Neural relational inference with fast modular meta-learning. In Advances in Neural Information Processing Systems (NeurIPS) 32, 2019.
[4] Amos, B. and Kolter, J. Z. Optnet: Differentiable optimization as layer in neural networks. In Proceedings of the 34th International Conference on Machine Learning. pp. 136-145. JMLR, 2017.
[5] Antoniou, A. and Storkey, A. J. Learning learn by self-critique. In Advances in Neural Information Processing Systems, pp. 9940-9950, 2019.
[6] Bartlett, P. L. and Mendelson, S. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463-448 2002.
10

---

[7] Bengio, S.. Bengio, Y., and Cloutier. J. On the search for new learning rules for anns. Neural Processing Letters, 2(4))26-30, 1995.
[8] Biggio, B., Corona, I,, Maiorca, D., Nelson, B., Smdié N., Laskov, P., Giacinto, G., and Roli, F. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
[9] Bottou, L. and Vapnik, V. Local learning algorithms. Neural computation, 4(6):888 900, 1992.
[10] Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A.,
VanderPlas, J., Wanddrman-Milnn S., and Zhang, Q- JAX: composable transformations of Python+NumPy programs. github. 2018. URL http: //github. com/ google/jax.
[11] Chapelle, O.. Vapnik, V., and Weston, J. Transductive inference for estimating values of functions. In Advances in Neural Information Processing Systems, pp. 421-427, 2000.
[12] Chapelle, O., Scholkopf, B., and Zien, A. Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks, 20(3):542-544 2009.
[13] Chen, T., Korbblith, S., Norouzi, M., and Hinton, G. simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.0577 2020.
[14] Cohen, J., Rosenfeld, E., and Kolter, Z. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320, 2019.
[15] Deng, J., Dong. W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L.. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
[16] Dhillon. G. S., Chaudhari, P., Ravichandran. A.. and Soatto, S. A baseline for few-shot image classification. In International Conference Learning Representations. 2020.
[17] Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning, pp. 647-655, 2014.
[18] Dumoulin, V. Shlens, J., and Kudlur, M. A learned representation for artistic style. arXiv preprint arXiv: 1610.07690, 2016.
[19] Finn, C., Abbeel, P., and Levine, S. aodel-agnostic meta-learning for fast adaptation of deep networks arXiv preprint arxiv: (tt:::773.034400 2017.
[20] Flennerhag, S., Rusu, A. A., Pascanu, R., Yin, H., and Hadsell, R. Meta-learning with warped gradient descent. arXiv preprint 1909v:/909..002:. 2019.
[21] Garcia, V. and Bruna, J. Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.0444. 2017.
[22] Grefenstette, E., Amos, B., Yarats, D., Huut P. M., Molchanov, A., Meier, F., Kiela, D., Cho, K., and Chintala, S. Generalized inner loop meta-learning. arXiv preprint 191001111,.01727, 2019.
[23] Greydanus, S., Dzamba. M., and Yosinski, J. Hamiltonian neural networks. In Advances in Neural Information Processing Systems, pp. 15353-15363, 2019.
[24] Hadsell, R., Chopra, S., and LeCun, Y. Dimensionality reduction by learning invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 066), volume 2, pp. 1735-1742. IEEE, 2006.
[25] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
[26] He, K., Fan, H., Wu, Y., Xie, S.. and Girshick. R. Momentum contrast for unsupervised visual representation learning. arXiv preprini wrXiv:1911.05722 2019.
[27] Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural computation, 14 (8):1771-1800, 2002.
[28] Hospedales, T., Antoniou, A., Micaelli, P., and Storkey, A. Meta-learning in neural networks: A survey. arXiv preprint 2rXiv:2004.0443, 2020.
[29] Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, I.., Tran, B., and Madry, A. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems, pp. 125-136, 2019.
[30] loffe, S. and Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift. arxiv preprint 1502:31002.03166 2015.
[31] Krizhevsky, A., Hinton, G., al. Learning multiple layers of features from tiny images. Citeseer, 2009.
[32] LeCun, Y., Chopra, S., Hadsell, R. Ranzato, M., and Huang, F. A tutorial on based learning. Predicting structured data, 1(0), 2006.

---

[33] Liu, Y.. Lee. J.. Park, M., Kim. S., Yang, E., Hwang, S. J.. and Yang. Y. Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv: 1005v000005100022. 2018.
[34] Madry, A., Makelov, A., Schmidt, L.,, Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint (rrXiv1706.060880 2017.
[35] Mityagin, B. The zero set of real analytic function, arXiv preprint arXiv:1512.07276. 2015.
[36] Miyato. T., Maeda, S.-i.. Koyama, M., and Ishii, S. Virtual adversarial training: regularization method for supervised and semi-superrised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979- 1993, 2018.
[37] Oord, A. v. d., Li, Y., and Vinyals. O. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 007.033388 2018.
[38] Paszke, A., Gross, S., Massa, F,, Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high- perffrrerfrmecee deep learning library. In Advances in Neural Information Processing Systems, pp. 8024-8044 8035, 2019.
[39] Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with general conditioning layer. In Shirty-Second AAA! Conference on Artificial Intelligence, 2018.
[40] Rakelly, K., Zhou, A., Quillen, D., Finn, C.,, and Levine, S. Efficient off-policy meta-reinforcemmnt learning via probabilistic context variables. arXiv preprint arXiv:1903. 09.00,54444 2019.
[41] Requeima, J., Gordon, J., Bronskill, J., Nowozin, S., and Turner, R. E. Fast and flexible multi-task classification using conditional neural adaptive processes. In Advances in Neural Information Processing Systems, pp. 7957-7968, 2019.
[42] Reuther, A., Kepner, J., Byun, C., Samsi, S., Arcand, W., Bestor, D., Bergeron. B., Gadepally, V., Houle, M.. Hubbell, M., et al. Interactive supercomputing 40,000 cores for machine learning and data analysis. In 2018 IEEE High Performance extreme Computing Conference (HPEC), pp. 1-6. IEEE, 2018.
[43] Salman, H., Li, J., Razenshteyn, I,, Zhang, P., Zhang, H., Bubeck, S., and Yang, G. Provably robust deep learning via adversarially trained smoothed classifiers. In Advances in Neural Information Processing Systems, pp. 11289 1189-11300 11300, 2019.
[44] Schmidhuber, J. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, Technische Universität München, 1987.
[45] Suh, H. and Tedrake, R. The surprising effectiveness of linear models for visual foresight in object pile manipulation. arXiv preprint arXi::2002 00093, 2020.
[46] Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A. A., and Hardt, M. Test-time training for out-of-distribution generalization. arXiv preprint arXiv: 1909./:999......0 13231, 2019.
[47] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint 1rXiv:1312.6199 2013.
[48] Thrun, S. and Pratt, L. Learning to learn. Springer Science & Business Media, 1998.
[49] Tschiatschek, S., Sahin, A., and Krause, A. Differentiable submodular maximization. arXiv preprint arXiv:1800 1803.00 703.0188, 2018.
[50] Vapnik, N. The nature of statistical learning theory, 1995.
[51] Wang, D.,, Shelhamer, E., Olshausen, B., and Darrell, T. Dynamic scale inference by entropy minimization. arXiv preprint 1rXiv:1908.03188. 2019.
[52] Wu, J.. Wang, Y., Xue, T. Sun, X., Freeman, W. T., and Tenenbaum, J. B. Marmet: 3d shape reconstruction 2017.
via 2.5 d sketches. arXiv preprint arXiv:1711.0311
[53] Zhai, R., Dan, C., He, D., Zhang. H., Gong, B., Ravikumar, P., Hsieh, C.-.., and Wang. L. Macer: Attack-free and scalable robust training via maximizing certified radius. In International Conference on Learning Representations, 2020. URL
[54] Zhang, M.. Marklund, H., Dhawan, N., Gupta, A., Levine, S., and Finn, C. Adaptive risk minimization: A meta-learning approach for tackling group distribution shift. arXiv preprint arXiv:2007.0299 2020.
[55] Zintgraf, L. M., Shiarlis, K., Kurin, V., Hofmann, K., and Whiteson, S. Fast context adaptation via meta-learning. arXiv preprint arxiv: 1810.8360333 2018.
12


Reviews:
[{"review_id": "oBkVTM6TU10", "reviewer": "Reviewer_3ay3", "summary": "In this paper, the authors propose optimizing an unsupervised loss function at test time as an inductive prior on a neural network. They propose two schemes for achieving this: training a network in a regular fashion and then applying the unsupervised loss at inference time only, or applying the unsupervised loss during training as a meta-learning scheme. The authors claim that this method can improve robustness, as well as improve train-test generalization gap in cases where there is a known inductive prior on the expected output (e.g. when modelling physical systems)\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper's writing could be improved, as the writing is complex and hard to follow. Furthermore the model details were mixed with the introduction, making it hard to understand what is the author's proposed work. It is also hard to understand the implementation details of this idea, as the authors share neither pseudo-code nor actual code to illustrate their implementation. With such an idea, it is often the case that the devil is in the details, and efforts to replicate it could show very different results to the ones shown on the paper.\n\nThe idea is very interesting and novel. It is well motivated in learning theory and seems almost obvious in hindsight (a good thing!). The authors provide a wide variety of experiments to show the effectiveness of the idea, not just for different datasets, but for different types of inductive bias. Especially in the case of physical systems modelling, this idea could be very useful in the applied setting and have a broad impact on further research.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations of their work and Societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "QUbG7t7V2O", "reviewer": "Reviewer_Pr63", "summary": "This paper proposed tailoring, which is a general framework that could help to finetune the prediction on each test sample according to some specific inductive biases. Tailoring provides a different perspective that avoids involving extra loss function in the proxy fashion.  Besides, the authors proposed meta-tailoring, which integrates the unsupervised loss in a way similar to meta-learning. The theoretical discussion and empirical results demonstrate the effectiveness of the proposed tailoring. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The idea of tailoring is interesting. The current neural networks mainly conduct amortized optimization, how to reduce such gap especially on the test data is an important research direction. I believe the proposed framework could provide a different perspective on understanding the \"generalization\" gap and different inductive biases. \n\n2. The limitation of tailoring lies in the increased computational cost. To reduce such extra cost, the authors thus introduce CNGRAD which could efficiently parallel the evaluation of the model over multiple samples.  And there is detailed and sound theoretical justification on the CNGRAD provided. \n\n3. The authors provided extensive examples of inductive biases. And correspondingly the experiment results on symmetry constraints, inductive biases, contrastive loss, and adversarial examples justify the effectiveness of both tailoring and meta-tailoring.  The ablation study is well designed and the results are promising.\n\nWeakness:\n1. One particular concern of mine is when conducting tailoring the parameters of the model change according to a single sample, this is similar to the continual learning setting. I wonder how the methods could avoid catastrophic forgetting. It seems that the authors constrain the steps of the tailoring while small changes in parameter space could result in the relatively large change of the model output. I suggest the authors add more discussions on this part. \n\n2. Though the method does no limit the application scenarios, I feel that the method is highly related to the field of semi-supervised learning. Therefore, I suggest more discussions on the related works. For example, the adversarial examples setting of tailoring is related the virtual adversarial learning in semi-supervised learning [1].\n\n[1]. Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning\n\nQuestions:\nI am curious about the setting when evaluating the test sample, we first assign a pseudo label according to the initial output. And minimizing the loss towards this pseudo label, I wonder whether tailoring in this setting could work. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.\nRefers to main review", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "Q3MeZG7G9E", "reviewer": "Reviewer_7twm", "summary": "The paper proposes a new method to adapt neural networks at inference time to the given input such that the model minimizes some given unsupervised loss. This method is called 'tailoring'. Additionally, 'meta-tailoring' is proposed that trains the model using 'tailoring', too. This way the gap between training and inference distribution is removed that 'tailoring' introduces.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper describes a, to the best of my knowledge, novel method. I believe this method is interesting for niche applications of ML. The model is replaced by a short SGD loop of the model, such that given x a few iterations of SGD are performed, before making the prediction. I do not understand why there is no possibility to make the model behave straight away like the model after a few steps.\n\nMain Points:\n\n1. Method Description. The method is not described enough in the main paper. One has to read the appendix to really understand what the method is, especially for meta-tailoring. There should be pseudo-code or at least a more detailed text description in the main paper. For example, I do not understand what the difference of Meta-Tailoring (0 st.) is to the baseline, even after looking into the appendix. I am pretty confident that algorithm 1 would actually not train at all with steps=0. I also find the naming confusing with the introduction of CNGrad, but naming the application of CNGrad in the experiments Meta-tailoring again.\n\n2. First-order and detached CNGrad. You only consider detached CNGrad in all experiments (line. This is not the algorithm you provide guarantees for in Section 3. I think this is a severe short-coming, as the detached variant, also does not agree with the intuition one develops around meta-tailoring. For 2/4 experiments you write that you use first-order CNGrad, for the others it is not known. First-order CNGrad, goes even further away from meta-tailoring and detaches \\gamma and \\beta even earlier. This, thus makes the above problem even more severe.\n\n3. Focus. I think this paper could benefit a lot of a more focused structure. You try to keep everything as general as possible, but there you also have to make changes for each experiment. I believe the paper might benefit from more focus. Only considering one or two CNGrad variants with fixed hyper-parameters across tasks, a main evaluation on which you reach something close to state-of-the-art and more focus on the method presentation.\n\n4. Experiments. This paper proposes a broad method, thus I understand the motivation of the authors to include a diverse set of benchmarks. The problem with this setup is that, the evaluations themselves suffer from this. None, of the evaluations is described to a sufficient level, such that it is really hard to understand the effect.\n\n5. Baselines. The baselines are not easily understood. While, I very much appreciate that you add the tailoring loss to the inductive baseline for the first experiment, I am still a little critical regarding the baselines. You now put a lot of work in making CNGrad work including selecting where to apply CN layers, what inner-lr to use, whether to use first-order CNGrad or detached CNGrad, but for the baselines it seems you performed less of this tuning. For some of the baselines my feeling is that it is reasonably easy to come up with a way to make the inductive baseline perform well and consider the constraints at testing time. Like you wrote for Adverserial Examples.\n\n6. Costs. As this method introduces a considerable amount of extra gradient steps to the training loop, I believe there should be some considerations of the costs of this method in the main paper.\n\nSummary: The paper proposes an interesting new method. It does not show conclusive evidence that it improves state-of-the-art models in any domain, though, and it uses a different algorithm compared to the described algorithm. Additionally, parts of the paper seem to not be quite ready for a main conference and rather in a draft-stage. Nevertheless, I believe this paper might be a great contribution after some more work on the experiments and the presentation.\n\nDISCLAIMER: I did not take the time to read section 3 and the corresponding proofs in detail, as I do not believe that this changes my view of this paper too much, it is hard to follow and does, to my understanding, not apply to the algorithm actually used practically in the experiments as pointed out in 2.\n\nDetails:\n\ni) Line 720 'key' -> 'this is key'\n\nii) Line 105 '. Losses' -> '.\\n\\nLosses'\n\niii) Line 63 -> Notice that the outer process now only optimizes the objective we care about, ...\n\niv) Confusing notation in algorithms, with var assignments and 'for' construct in the same line.\n\nv) Table 1 description 'over-performs' -> 'outperforms'\n\nvi) You used /begin{figure} for many tables where /begin{table} should be used.\n\nvii) Line 356 ' Improving' -> 'Improving'", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors consider interesting societal impacts and the limitations of their algorithm.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "0Tsm-tzLN9H", "reviewer": "Reviewer_66Ub", "summary": "The paper proposes tailoring - a general framework of algorithms that can combine ideas for test-time generalization, self supervision, meta learning and transductive learning. Although the experiments are quite limited overall, the paper is a good addition for the ML community. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper introduces tailoring and meta tailoring as a means to add inductive biases at test-time using contrastive losses. The paper overcomes some of the shortcomings of TTT and meta-tailoring seems like an interesting improvement over TTT. The idea of encouraging soft inductive biases (5.2) is very interesting and practical. I would like the authors to include experiments such as domain generalization (like the sort done by TTT) since I believe that those set of experiments are very good tests of how such algorithms can adapt to novel data distributions (not just adversarial samples as they are a very specific type of generalization. Although the authors mention the practical problems with implementation in popular deep learning frameworks (pytorch and tensorflow), it would be good for the authors to provide means to overcome such problems so that CNGrad can become a staple in the deployment of ML models.\n\nThe authors tend to focus on the breadth of results to show the generality of the solution, rather than depth in one or two fields, it might be useful to show more difficult tasks in any of the tasks to show a strict improvement and the scalability of the solution.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The most practical problems with the frameworks (as the authors discuss) is with privacy as the models are \"tailored\" for the given example. This can have negative impact and which the authors correctly discuss. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1 hour"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes tailoring, a technique for incorporating unsupervised objectives during test time. The main distinction and past work is that tailoring is applied to each individual test datapoint. The paper includes some theoretical justification of the approach as well as rather extensive experiments. Reviewers were generally positive on the paper. The main criticism of the paper came down to lack of clarity in writing, but the authors have addressed this concern through their rebuttal. Before the camera-ready version, please incorporate the proposed changes to writing.
Decision: Accept (Poster)