Human-Guided Complexity-Controlled Abstractions
Authors: Andi Peng
Mycal Tucker
Eoin M. Kenny
Noga Zaslavsky
Pulkit Agrawal
Julie Shah
Venue: NeurIPS 2023 poster (2023)
Keywords: human-in-the-loop
representation learning
interpretability
TL;DR: Generating good abstractions enables rapid adaptation, but identifying such abstractios a priori is challenging; a human-in-the-loop framework allows people to select the right abstraction level in models.

Abstract: Neural networks often learn task-specific latent representations that fail to generalize to novel settings or tasks. Conversely, humans learn discrete representations (i.e., concepts or words) at a variety of abstraction levels (e.g., "bird" vs. "sparrow'") and use the appropriate abstraction based on tasks. Inspired by this, we train neural models to generate a spectrum of discrete representations, and control the complexity of the representations (roughly, how many bits are allocated for encoding inputs) by tuning the entropy of the distribution over representations. In finetuning experiments, using only a small number of labeled examples for a new task, we show that (1) tuning the representation to a task-appropriate complexity level supports the greatest finetuning performance, and (2) in a human-participant study, users were able to identify the appropriate complexity level for a downstream task via visualizations of discrete representations. Our results indicate a promising direction for rapid model finetuning by leveraging human insight.

Human-Guided Complexity-Controlee Abstractions
Andi Peng" MIT
Mycal Tucker*
MIT
Eoin M. Kenny
Noga Zaslavsky
Pulkit Agrawal
Julie A. Shah MIT
MIT
UC Irvine
MIT
Abstract
Neural networks often learn task-specific latent representations that fail to general- ize to novel settings or tasks. Conversely, humans learn discrete representations (i.e., concepts words) at variety of abstraction levels (e.g., "bird" "spar- row"" and deploy the appropriate abstraction based on task. Inspired by this, we train neural models to generate spectrum of discrete representations and control the complexity of the representations (roughly, how many bits are allocated for encoding inputs) by tuning the entropy of the distribution over representations. In finetuning experiments, using only small number of labeled examples for a new task, we show that (1) tuning the representation to task tisk-appropriate complexity level supports the highest finetuning performance, and (2) in human participant study, users were able to identify
appropriate complexity level for a down- stream task using visualizations of discrete representations. Our results indicate a promising direction for rapid model finetuning by leveraging human insight.
Introduction
Neural networks learn implicit representations tailored specific training tasks, but such represen- tations, or abstractions, often fail to generalize to distinct test tasks. One approach to mitigating such generalization failures is based the Information Bottleneck (IB) method 226 I 25 24],
which provides a framework for controlling how much information is passed through the network, effectively limiting its representational complexity. Unfortunately, it difficult know a-priori how much information to retain for optimal task performance.
Humans, however, when given task, remarkably adept understanding which abstraction deploy [35] Consider an expert birdwatcher who has learned fine-grained classes of birds such as "sparrows," "goldfinches." and "kiwis." When accompanied by other experts, birdwatcher knows to deploy their "most complex" abstraction for classifying birds; meanwhile, when at home with 6-year old child, a birdwatcher knows deploy far simpler representation to communicate about "red" vs. "yellow" bird. In other words, given a task, humans naturally adopt the right abstraction level that meaningfully captures task-relevant information and enables rapid learning (1.55. Inspired by humans, we seek to train neural nets along an axis of representational complexity and allow end users to select the desired complexity level support data-efficient finetuning.
In this paper, we introduce a human-in-the-loon framework, depicted in Figure for pre-training and finetuning complexity-regulate neural representations. Within this framework, first generate a spectrum of cumplexity-contrllee representations by training discrete information bottleneck methods 29 pre-training task. Second, allow human specify finetuning task, unknown
"Equal contribution.
Code available at htt.://.wooooaaaaaeeee////a/aaaa
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Pre-training
red yellow
red, yellow, black, brown
cardinal, goldfinch, etc.
complexity
High complexity
Selection and Finetuning
Model 1
Model 2
Model 3
Human
Human 1
Selects high complexity model
Red vs. yellow
Canary, cardinal, osprey, etc.
Selects low complexity model
Figure 1: Our comman-in-thh--re framework for pre-training and finetuning, illustrated for a bird- identification example. In pre-training, we generate a spectrum of encoders using representations from low to high complexity (e.g..jus just two crude categories to fine-grained species classifications). In fine-tuning, based on desired task, human selects an appropriate model for finetuning (e.g.. crude categories for child learning colors and fine-grained categories for a birder).
priori, and select and finetune a pre-trained model. Given that humans specify the finetuning task, and may have to provide finetuning labels, few-shot adaptation is important.
In computational experiments, we find that finetuning performance non-monotonically linked to representation complexity: representations that are complex data-inefficient and representa- tions that are too simple fail to capture important information. In user study, we show that humans, given a desired finetuning task, can select high-performance models from at different complexity levels. For example, as in Figure | child performing a low-complexity task might select cwwpolply representation. More generally, while our computational experi- ments establish that finetuning performance is a function of model complexity, our study shows that humans can select (near) optimal complexity levels given task for model finetuning. Lastly, the choice of neural architecture significantly affects finetuning efficiency: on one task, for example, our best-performing architecture, tuned the right complexity, achieves better performance than other standard encoding methods with 50x more finetuning data.
pre-trained models
Our findings suggest that automatically constructing complexity-regulate abstractions during pre- training and then providing a diverse spectrum for human use for finetuning is a promising direction for human-in-the-- in-the-loop few-shot adaptation. summary, our contributions are (1) introducing human-in- loop framework for automatically generating spectrum of complexity munllxxty-regullted abstractions for fast adaptation, (2) establishing that finetuning performance is function of representation complexity. and (3) demonstrating the utility of our human-in -nttan-----eh--oopp the-loop framework in user study.
2
Related Work
2.1 Abstraction in Human Cognition
There substantial evidence that suggests that much of human learning, perception, communication, and cognition may be understood as compression of relevant information [31 36 IT]. For example, fast human learning can be enabled by merging two or more instances of statistical patterns into one when appropriate (20]. Moreover, the simplicity of these patterns appears key to supporting their predictive power on downstream tasks (2) This connection between compression and prediction provides an elegant explanation for why human brains have evolved to be such efficient compressors of information and experience [77 Furthermore, visual abstractions have been found to prioritize functional properties, i.e. downstream task use, at the expense of visual fidelity, i.e. image recon- struction 11I. This suggests that different abstractions are constructed and deployed conditioned on tasks. Inspired by this, we seek to train neural networks to output visual abstractions that are also functionally useful to human users diverse range of downstream tasks.

---

2.2 Discrete Information Bottleneck
In our work, we build upon and compare to methods from prior research discrete information bottlenecks. Generally, such work seeks to generate complexity-limiee representations (roughly, limiting the number of bits about the input in a representation) using finite set of representations (dubbed quantized vectors prototypes). Recent work 227
has approached this problem with Vector Quantization (VQ) [29]. In such works, encoder network outputs parameters
28 7 12
by combining ideas from the Variational Information Bottleneck
normal distribution, from which a continuous latent variable is sampled, and the sample is discretized to the closest element of a learnable codebook. By penalizing the KL divergence between the normal and a fixed prior (typically a unit normal), one can limit the complexity of representations. We refer to this family of approaches as the Vector Quantized Variational Information Bottleneck Normal (VQ-VIBy)) Other work proposes a different sampling mechanism before vector quantization, but experimental evaluation of these methods remains limited 22 32
In our work, we propose that complexity-constrane to generate meaningful variety of encoders for human-in-the-loye finetuning process. We use methods from prior work, and we propose a novel combination entropy regularization and categorical sampling that, in experiments, supports the best finetuning performance.
discrete representation learning can be used
3 Approach
3.1 Problem Formulation and Human-in-the-loop Framework
We consider a pre-training and finetuning problem, wherein model is first trained on a pre-training dataset and must be rapidly adapted to a distinct finetuning task. Unlike classic meta meta-learning we do not assume access distribution of tasks.
frameworks 6 21 19),
In pre-training, we assume access to a dataset of inputs and pre-training outputs, (X, Yp) (e.g.. images of birds and species labels). That is, x P(X), P (Y((X)). In finetuning, we assume access to task-specific dataset with similarly-drawn inputs but novel task labels: (X, Y2), for P(X). y (Y X) (c.g., for the depicted girl's finetuning task, the color of the bird). Lastly, we assume that the pre-training labels are a sufficient statistic for the task-specific labels: (,) (Y,;; Yt). Intuitively, this states that the pre-training objective must include relevant information for the finetuning task. This is trivially satisfied with reconstruction loss (i.e., Y, X) or fine-grate -nn-grained classifications (e.g.. pre-training on exact species, but finetuning on groups of species).
Without information about the finetuning task, is difficult, a priori, to identify how to pre-train model to perform optimally the finetuning task. Regularization methods like information bottleneck, for example, depend upon insight on the downstream task to specify the right level of regularization [T] Rather than automatically identifying the right model, therefore, we propose cuuman-in-the-loee framework, depicted in Figure [ Within our framework, we seek to generate a suite of pre-trained neural net encoders such that an end user can identify which one uses abstractions that support high performance for their desired task.
3.2 Technical Approach
Within our human-in-the-logg framework, it critical generate "good" of encoders from which the human chooses. This set must exhibit important variation (such that some encoders are better than others) and inttrriitterretaablittyy (such that humans can select the better encoders). We propose and validate in experiments that complexity complexityccontrollee discrete representation learning mechanisms achieve these desiderata. First, discrete representations can support human interpretability (via visualizations of the finite set of representations) [15 18 Second, controlling the complexity of representations is a meaningful axis of variation for encoders that likely supports human-specified tasks (34).
3

---

P(1 11, ---1----111
Reconstruction
h
D
== [1.,,...,211
E
Utility
Zn
P
y
Entropy
Figure 2: A VQ-VIBe encoder maps an input, I, to a representation, in R which is divided into sub- representations, h; Each h; is discretized stochastically by sampling based on distance each quantized vector in the codebook, Ç. Lastly, the sampled quantized vectors are concatenated for the full latent representation,
Figure 3: Different encoders train
within emcoder--decodde--rredeee
framework via utility, entropy, and re- construction losses [adapted from (2).
3.3 Neural Architecture Improvements
To generate a spectrum of encoders using representations at different complexity levels, we extend prior methods in discrete information bottleneck. We briefly present novel neural method, which we dub the Vector- Quantized Variational Information Bottleneck Categorical, or VQ-VIBe.
In VQ-VIBc. we combine information--heorrtiee losses from Tucker et al. (2 's Vector Quantized Variational Information Bottleneck method (which dub VQ-VIBx emphasize that latent representations are drawn from a normal distribution) with categorical sampling mechanisms similar Roy et al. [22] and Wu and Flierl 332 Our VQ-VIBe encoder architecture is depicted in Figure 2 The encoder is parametrized by feature extractor (in our cases, a standard feedforward neural network), an integer, n. representing how many quantized vectors combine into single latent representation, and C learnable quantized vectors Ç²; [1, CJ, E 2/2m, Using this architecture, the encoder is characterized as a function mapping from an input to a distribution over latent representations: q( ()); E
R z 2R2 Concretely, the VQ-VIBe architecture deterministically maps from to hidden representation, h E IR2. That hidden representation is divided into vectors of equal size: h = [h1, h2 ..., h.... Each h; then probabilistically quantized by sampling a quantized vector according to the L.2 distance from h; to each quantized vector: P(2, = (2) x e h, -' (One can differentiate through sampling from this categorical distribution via the gumbel-softmax trick 13
Lastly, these n sampled discrete representations, each of which is dubbed 2; are concatenated form the latent representation,
16
[21, .2,-..,..))
We train VQ-VIBe via a combination of losses introduced by Tucker et al. 27 We assume the Given utility
VQ-VIBe encoder is trained with decoder and a predictor, as depicted in Figure function, U, VQ-VIBc is trained to maximize the objective function in Equation
max (x. X,Bl[l> a|l3] - A
Σ H(P(CIh((a)) [E
[sg]h; ((xx)] aall.(())))))))) sg[((())]11
This objective trades off, order the equation: 1) maximizing the expected utility (e.g., cross- entropy loss), 2) minimizing the expected reconstruction loss (here, MSF), 3) minimizing the entropy of the distribution over codebook clements, and 4) minimizing a clustering loss from prior literature, encouraging encodings and quantized vectors to cluster (and, as in prior art, we leave 0.25 in all experiments) [29]. Here, sg represents the stop- nop-ggradentt operation.
We include more thorough discussion of this loss function, and comparisons to losses and archi- tectures proposed in prior literature, in Appendix A We emphasize that, while we propose some modifications to neural architectures, the primary contributions of this work are not about VQ- VIBe but rather: 1) our human-in-the-le -ntth--oooopp framework and 2) the recognition that discrete information bot- tleneck methods support high performance within this framework. In experiments, we found that our VQ-VIB method performed better than methods from prior art, so we include this technical section to inform future researchers about modest architectural changes that support better performance.

---

4 Assessing Representational Complexity's Impact on Finetuning
We first present results for computational experiments in three different visual classification domains, indicating that finetuning performance varies as a function of representation complexity in non- monotonic way, All experiments consisted of pre-training and finetuning phases. First, in pre-training on low-level task, we generated a spectrum of models at varying complexity levels by varying loss hyperparameters. Second, we used the encoders from the pre-trained models and trained predictors to map from encodings to downstream predictions. Jointly, these steps enabled us to assess representations for data-efficient finetuning. Overall, we
the importance of complexity-controlle found that, for small amounts of finetuning data, tuning representations to the right complexity was important, but with large amounts of data, any seffiiientlyccomplee encoder performed similarly.
4.1 Domains
We trained agents three image classification datasets: FashionMNIST 333, CIFARIO0 [14), and
iNaturalist 2019 (iNat) [0]. For FashionMNIST, we used a two-level hierarchy. grouping the 10 low-level classes into 3 higher-level classes (top Tshirt/top, Pullover, Coat, and Shirt; shoes Sandal, Sneaker, and Ankle Boot; other = Trouser, Dress, Bag). The CIFAR 100 and iNat datasets have more extensive hierarchieal structures, detailed by, e.g., Sainte Fare Garnot and Landrieu (23 For both datasets, we considered two levels of increasing crudeness in the hierarchy. For CIFAR 100, we used both a 20-way division (each class consisting of low-level classes) and binary division for alive vs. non-alive objects. For iNat, we used a 34-way division (representing categories like "butterflies" and "mushrooms") and 3-way division for plants, animals, or fungi. Thus, all domains were characterized by semantically meaningguu hicrarchies.
4.2 Pretraining and Finetuning
Our experiments comprised two phases: pre-training and finetuning. In pre-training, we trained an encoder, decoder, and predictor on a low-level task. For example, for iNat, this consisted of classifying a photograph among 1010 species. During pre-training, after convergence to high accuracy, we decreased the complexity of representations by tuning a hyperparameter until representations were uninformative. For our 3-VAE and VQ-VIB baselines, we increased λc, a scalar weight penalizing the KI. divergence of the conditional Gaussian (generated by the encoder) from unit Gaussian. (See Appendix for details of training losses for 3-VAE and VQ- yQ including Ac- In general, larger values of Àc led to less complex representations and greater MSE values for reconstructions.) For VQ-VIBc. we increased λи. the scalar weight penalizing the entropy of the categorical distribution over quantized vectors. In Appendix D we examine other combinations of losses and architectures to control complexity, but found that tuning entropy for VQ-VIBe achieved the best results. general, to address some conflicting definitions of complexity in prior literature, we distinguish between an entropy loss, as introduced for our VQ-VIBe model, and a complexity loss, where we use the definition of complexity
((;
In finetuning, we trained new predictor networks to map from encodings to classifications, using a small amount of training data. Using an encoder saved during pre-training, we generated encodings from inputs. New predictors were trained on classification tasks given the supervised (encoding, label) data. We generated & encodings for each class in the supervised dataset, and report results for different k. By varying A and which encoder was used to generate encodings (from high to low complexity), we could investigate the effect of encoder complexity on data-efficiency in finetuning. In all experiments, we pre- pre-trained models and ran 10 finetuning trials for each encoder. Further details on pre-train tre-training and finetuning are included in Appendix A
4.3 Illustrative Example: FashionMNIST
We illustrate the ligh trends from our computational results using the FashionMNIST dataset. We trained models the 10-way classification task to high accuracy (typically around 90%) and decreased representational complexity until the model was accurate 10% of the time (random chance).
Depictions of the learned quantized vectors for VQ-VIBe are included in Figure at high and low complexities. At high complexity, VQ-VIBe used a large number of distinct prototypes, including multiple prototypes per class (c.g., for two different types of handbags). This high complexity
5

---

Fashion Fashion 3-way Aceuncyy
F 100%, ke
YAs
MSB
(a) VQ-VIBg; high complexity
(b) VQ-VIBe: low complexity
(c) Finetuning accuracy vs. MSE.
Figure 4: 2D PCA of VQ-VIBe latent representations for FashionMNIST at high (a) low (b) complexity. Colorful points and images are prototypes, scaled in proportion to frequency of use. At high complexity, VQ-VIBe two types of handbags); at low complexity, classes merge (e.g. different types of shirts). c) When finetuned on a 3-way classification task with I example per class, VQ-VIBe outperformed 3-VAE and VQ-VIBy, and less complex representations (greater MSE) improved model scores.
a large number of prototypes, including multiple per class (e.g..
supported low MSE reconstruction loss. As decreased the representational complexity, VQ-VIB used fewer prototypes that represented more abstract concepts (Figure b). For example, five distinct classes (Tshirt/top, Pullover, Coat, Shirt, and Dress) were merged into single prototype, increasing MSE. Further examples of prototype evolution are in Appendix
In finetuning experiments, we loaded VQ-VIBe encoders across range of complexities and finetuned a predictor on the 3-way classification task described earlier (tops, shoes, or other). Using just one example per class (k= 1), VQ-VIBe models were more accurate than 3-VAE or VQ-VIB models, as shown in Figure 4 c. The x axis displays each encoder's MSE, which is a proxy for the inverse of the complexity of representations: more complex representations capture more information and generate better reconstructions (lower MSE). The y axis shows the finetuned predictor's 's accuracy on the 3-way task. While VQ- peak performance is roughly 70% accuracy (remarkably high, given k = 1), 3- VAE and VQ-VIBx performance peaks at 40% and 55%, respectively.
Beyond comparing VQ-VIBe to other models, Figure 4 shows the importance of task-appropriate complexity levels. At low MSE values, VQ-VIBe uses many different prototypes, which impedes finetuning data efficiency. However, increases MSE), finetuning performance improves (the correlation between MSE and accuracy is positive for MSE 0.055 (p < 0.03)). However, past MSE value around 0.055, VQ-VIBe lacks the representational capacity to distinguish between images that should be classified differently, and performance worsens.
VQ-VIB learns more abstract representations (which
Here, we discussed finetuning results for I labeled example per class (k = 1) and for n = the number of quantized vectors to combine into representations, but results and analysis for k E [1, 2, 5, 10, 50]. n E [1, 4], and for 3-VAE VQ-VIB,. and VQ-VIBe models are included in Appendix B For all k, found that VQ-VIB, outperformed 3-VAE and VQ-vIBy. In fact, VQ-VIBe with A: = 1 outperformed B V-V for l = 50, indicating important architectural benefits. Increasing supported higher complexity (lower MSE) but worse fine-tuning performance; intuitively, many discrete representations approximate continuous representations, which have worse sample complexity. This is important limitation of combinatorial codebooks that others propose 28 12]
A series of ablation studies confirm the importance of the VQ-VIBe architecture and annealing entropy, instead of complexity, for efficient finetuning (Appendix D for FashionMNIST and other domains). In particular, we found that for VQ-VIBc and VQ-VIB w. penalizing entropy led to greater finetuning accuracy than when penalizing complexity, and VQ-VIB outperformed VQ-VIB, when both were trained via entropy regularization. In other words, penalizing entropy enabled better finetuning performance, and VQ-VIBe supported better entropy regularization.
Lastly, given the importance of complexity on finetuning performance, we considered two methods for autonomously selecting the optimal complexity level. For low-data regimes (I: < 5), the simple heuristic of choosing the most complex encoder was clearly suboptimal; however, as the amount of finetuning data increased (c.g., k 50), the most complex encoders achieved near-optimal

---

CIFARIOO 2-eay Accuncy:
CIFARIOI 2-way Accuracy:
CIFAR 100 20-way Accuncy;
F0RR WWW
02
040
220
Cat
220
678
MSE
MSE
MSE
(a) CIFAR1OO 2-way &
(b) CIFARI00 2-way l 50
(c) CIFAR100 20-way
Figure 5: CIFAR 100 finetuning, using 1 (a) or 50 (b) training examples for living vs. non-living things. With small amounts of data VI-e benefits from tuning to the right complexity level. When finetuning the 20-way classification task (c) VQ-VIBe continues outperform other architectures.
iNat S-way Accuracy,
(Nat 3-way Accuncy: 50
iNat t-way Accunecy: VQ-VIBe
piizz
Y Aec
Y ACc.
Alc
PAs W0H0
vore: 2-55
MSE
MSE
MSE
(a) iNat 3-way k= ==
(b) iNat 3-way l 50
(c) iNat VQ- varying k
Figure iNat finetuning results for the Mant-animal--urre funnuus classification task. Using only I finetuning example (a), VQ-VIBe performance improved as complexity decreased, and outperformed 3-VAE and VQ-VIB Using 50 finetuning examples, performance plateaued, but VQ-VIBe continued to outperform other methods (b). VQ-VIBe performance shifts smoothly for varying k (c).
performance (see Figure 10 in Appendix B). We also tested methods for selecting encoders via validation-set accuracy and found similar trends. In finetuning, we held out subset of the data for assessing finetuning accuracy and selected the best-performing encoder via validation set accuracy. For k < 5, this validation-set approach did not consistently converge to optimal performance, but, for sufficiently large k; did. Further results from this approach are included Table and Appendix B Generally, we found that for large enough k the importance of tuning to the right complexity decreases, and several methods can select optimal encoders; for very small k, however, autonomous methods are suboptimal.
This illustrative FashionMNIST use case demonstrates many of the important trends we explore in our later experiments: tuning VQ- representations to the "right" complexity was important for optimal performance for small k. and VQ-VIBe generally outperformed other encoders for few-shot finetuning.
4.4 CIFAR100 and iNat
In this section, we present results from computational experiments in the more challenging CIFAR100 and iNat domains. As before, in the smallest data regime (small k), using less complex representations afforded greater efficiency benefits, and VQ-VIBe outperformed 3-VAE and VQ-VIBX.
Figure 5 a and b show finetuning accuracy on the binary classification task for CIFAR100 on identifying living vs. non-living things. With only datapoint per class (Figure S a), 3-VAE performance remained effectively flat at random chance: barely exceeding 50%. However, for VQ-VIBe, for MSE < 0.35, performance increased as MSE increased (linear regression slope was positive < 0.001), up to a 70% accuracy rate, before worsening as MSE increased further. Increasing l: to 50 (Figure 5 b) shrank the gap between the encoder architectures, and flattened the improvements previously observed, but VQ-VIBe continued to outperform 3- 3-VAE and VQ-
7

---

(a) PCA of high-complexity VQ-VIBg. Note the orange bird in the (b) PCA of low-complexity VQ-VIBe. pine tree.
Note the bird in its nest on the left.
Figure 7: 2D PCA of the VQ-VIBe latent space for iNat, at different complexity levels. Each gray point represents the continuous encoding output by the encoder, the colorful points represent the learnable prototypes. For visualization purposes, the closest training image to each prototype is included in the diagrams. In the high-complexity case, the model has learned many distinct prototypes, representing concepts like birds or insects. In the low-complexity case, VQ-VIBe uses only two prototypes, which capture a plant-animal distinction the aligns closely with the iNat hierarchy.
VIBN models. When finetuned on the 20-way classification task, the same trends of VQ-VIBc outperforming other architectures held (Figure 5 c). although the finetuning accuracy decreased monotonically as MSE increased, rather than peaking at a specific complexity. Plots for a larger range of k, and for varying n, for both finetuning tasks, are included in Appendix B
Similar trends held in the iNat dataset when finetuned on the 3-way animal-plant-fungee task, as depicted in Figure 6 For small k. VQ-VIBe finetuning performance initially improved as the encoder learned more compressed representations (Figure @a). For VQ-VIBe, & = 1, the linear correlation between MSE and finetuning accuracy is positive (p 0.001) for MSE < 0.55. Further analysis of finetuning performance for VQ-v for varying k, shows a smooth change in behavior as increases: simultaneously improving overall performance, and benefiting less from compressed representations (Figure (cc.. This indicates that VQ-VIBe is most advantageous a low-data regime. Similar results for finetuning on the 34-way finetuning task included in Appendix B Visualization via 2D principle component analysis (PCA) confirms our intuition of how VQ-VIB models support few-shot learning for iNat. At lower complexity levels, VQ-VIB encoder used decreasing number of prototypes to represent increasingly abstract concepts (Figure 7
As in the FashionMNIST domain, evaluated autonomous methods for selecting the optimal encoder and found that, while they succeeded for sufficiently large k, they struggled in a low-data regime. For A 1, for example, the heuristic of choosing the most complex encoder was suboptimal (e.g., see Figure 5 a and Figure a). At the same time, for = 50, this heuristic worked quite well (e.g., Figure b and Figure bb) Lastly, selecting models via validation set accuracy similarfy worked well for large enough ks but struggled for k < 5 (see Appendix B).
Finally, we briefly note some limitations of VQ-VIBe and our finetuning method. In the results discussed so far, we found consistent advantages in using VQ-VIBg and low-complexity representa- tions. However, the amount of finetuning data increases, more complex representation methods like B-VAEs outperform VQ-VIB Appendix B Between main results and these limitations, find that the data data-efficiency benefits of using VQ-VIB in finetuning are greatest in data-poor and low-complexity settings.
more complete discussion of this phenomenon is included in
5 Human-in-the-Loop Selection of Task--pproeriate Representations
Given our main motivation of a buman-in-the-t framework for selecting approoppoee abstrac- tions (recall Figure [ we conducted a partinppatiiiiantt study to evaluate whether users could select the highest- pereess-prffrminggg tpproappe models given visualizations of prototypes as task abstractions. This is important because, in order to take full advantage of our setup, users must be able select the appropriate lask- representation so the neural network best learns the finetuning task in a sample efficient manner. We asked users to select the optimal encoder for a given task, given visualization of encoder prototypes. A positive result would show that, given spectrum of pre-trained VQ- VIBe encoders, a human user wishing to deploy a task-appropriate model could specify the right encoder for given task, and thus fine-tune the model in sample efficient manner.
8

---

Selection Vo-Vl:
x x
Proto.
540
Response Counts
MSE
0.3%
option
0.00
Epoch
Encoder Checkpoint
Figure 8: Participants were asked to select the optimal encoder for the FashionMNIST finetuning task based on MSE values (left) or decoded representations (middle). Using only MSE values, users often incorrectly selected option b). whereas when viewing prototypes, users were able to correctly identify the appropriate abstraction level, and thus the optimal encoder (c).
5.1 User Study
Our human experiment was performed using the pre-trained models from the FashionMNIST domain in Section 4.3 Users were told that a robot was good at sorting clothing into 10 distinet categories (i.e.. the normal categories in FashionMNIST), but that their case they should consider different set of categories. Each user was randomly assigned a model type (VQ-VIBe or VQ-VIB and visualization type (prototypes or a plot of MSE during training). three questions (explained below), users were shown different groupings of the 10 FashionMNIST classes and asked to select which of four model checkpoints they thought best represented the groupings. Overall, this between- subjects setup allowed to compare the effects of different visualization methods user accuracy in selecting the optimal representation.
For our main question, in which users were asked to select an encoder that could sort FashionMNIST items into the three finetuning categories (tops, shoes, and other), our null hypothesis, Ho, was "Users viewing VQ- VIBe prototypes will select the optimal encoder as often as those viewing MSE scores". Our alternative hypothesis, H1, was "Viewing VQ-VIBe prototypes improves users' ability to select the optimal encoder compared to just viewing MSE scores."
Participants. We crowd-sourced 20 participants per group from Prolific.com (N=80), The sample was balanced to have an even number of male and female participants. All participants were native English speakers above the age of 18 and resided in the U.S., the U.K., or Ireland. Given estimates of survey duration, calculated hour wage. The study received IRB approval from MIT.
pilot study, we paid participants according to an estimated $12 per
Materials. Each user was presented three questions, corresponding to different groupings of the 10 FashionMNIST classes: in Question classes were grouped according to the finetuning labels in our computational experiments (tops, shoes, and other), in Question 2, there were 10 distinct classes corresponding the 10 FashionMNIST classes,and in Question 3. classes were grouped according to an unintuitive 3-way grouping (c.g.. Pullover, Dress, Sneaker were one class). Despite the numbering, we randomized the order of questions for each participant. each question, users were asked to select which of four model checkpoints they thought best represented groupings for that specific question. The four models corresponded to checkpoints taken 1) near the start of training, 2) the minimum MSE value, 3) at an intermediate MSE value, and 4) at the end of training. We primarily focused on results for Question I, as was the only question for which selecting encoder was not optimal. Responses were categorized as correct if users picked the optimal encoder (option "c" in Figure 8) and otherwise incorrect (but during the study, users were not told whether they were correct).
most complex
5.2 Results
Results from our user study supported our hypothesis. As shown in Figure 8 for Question 1, corresponding to the 3-way grouping from our computational experiments, users who viewed VQ- VIB prototypes selected the optimal encoder 55% of the time, compared to 10% accuracy when

---

viewing MSE scores (prototype accuracy was significantly greater at p < 0.01 for Fisher's i exact test). Interestingly, users who viewed VQ-VIB,, models never achieved high performance (binomial test for non-random chance at p > 0.2), suggesting important advantage of the VQ-VIBe architecture. Thus, VQ-VIBc prototypes was the only visualization method that supported greater-than-randon chance performance in selecting optimal abstractions.
Further analysis, using responses to Questions and well, highlighted the benefit afforded by VQ-VIBe prototypes. Overall, we observed high accuracy rates for Questions 2 and regardless of visualization method; this is unsurprising given that the correct behavior for both questions was the selecting most complex encoder. We fitted Mixed Linear Effects Model (MLEM) to the survey results, predicting user accuracy as a function of model visualization and question, grouped by participant. Using Wilkinson notation [30], our model was: Acc. ~ Q + M + Q M + (1) Participant) where Q represents the categorical variable for question and M represents the categorical variable for model type and visualization (VQ-VIBe Prototypes, VQ-VIBe MSE, VQ-VIBy Prototypes, etc.). We grouped results by participant to model random-intereept effects of some participants being more accurate than others. Full results for accuracy rates and the MLEM model are included in Appendix E
Overall, we found a significant (p < 0.05) positive interaction effect between Question and visualization of VQ-VIBe prototypes. The only other significant effect we found was a negative effect for Question 1. Together, these results show that 1) Question 1 was harder than the other two questions and 2) viewing VQ-VIB prototypes mitigated the effects of increased difficulty for Question 1. This provides crucial support for our hypothesis, showing that viewing VQ-VIBe prototypes enabled participants to select the optimal encoder more often.
Lastly, the combination of all results helps rule out several alternative hypotheses for how participants selected encoders based on VQ-VIBe prototypes. Users did not merely select the most complex encoder; otherwise, they would have performed poorly on Question 1. Similarly, users did not simply use the fewest number of prototypes greater than the number of finetuning classes; otherwise, they would have performed worse on Question 3. Thus, our results suggest that users interpreted VQ-VIBe prototypes as desired in intelligently selecting aas--ppprpeeitte representations.
6 Contributions
We proposed a human-in- -oop-- framework for selecting task-appropriate representations for data- efficient adaptation to new tasks. Discrete information bottleneck methods provide a principled way to generate representations along spectrum of low to high complexity; we tested methods from prior literature and a novel architecture, VQ-VIBe, for generating such representations.
We found that controlling the complexity of representations was important for data-efficient fine- tuning: overly-complex representations required more training examples, but overly-simplified representations failed to capture important information. In computational studies, we showed the importance of learning low-entropy representations and that VQ-VIBe tends to outperform other discrete information bottleneck methods. In a human study, we found that human partners are able to identify optimal complexity levels, indicating promising direction for human-in-ther
training.
Broader Impact: Generally, we hope that this work supports broader and better human-AL AI col- laboration, supporting human-sclected individualized models for particular use cases; however, we recognize that our current wrototype-inspectine framework may be limited and that care must be taken in future work to verify that representations along the complexity spectrum correspond to desired human concepts. In future work, we hope to explore models that simultaneously support different complexity levels instead of training separate models for different levels.
7
Acknowledgements
We thank members of the Interactive Robotics Group for helpful feedback and discussions. Andi Peng is supported by the NSF Graduate Research Fellowship and Open Philanthropy. Mycal Tucker is supported by Amazon Alexa Science Hub Fellowship.
10

---

References
[1] Alex Alemi, Ian Fischer, Josh Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference on Learning Representations, 2017.
[2] Nick Chater and Paul Vitányi. Simplicity: a unitying principle in cognitive science? Trends in cognitive sciences, 7(1):19 1201 2003.
[3] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K. Su. This looks like that: deep learning for interpretable image recognition. Advances neural information processing systems, 32, 2019.
[4] Jon Donnelly, Alina Jade Barnett, and Chaofan Chen. Deformable protopnet: An interpretable image classifier using deformable prototypes. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2022.
[5] Judith Fan, Robert Hawkins, Mike Wu, and Noah Goodman. Modeling contextual flexibility in visual communication. Journal of Vision, 18:1045, 09 2018. doi: 10.1167/18.10.11
[6] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In International conference on machine learning, pages 1126-1135 PMLR, 2017.
[7] Srishti Gautam, Ahcene Boubekki, Stine Hansen, Suaiba Amina Salahuddin, Robert Jenssen, Marina MC Höhne, and Michael Kampffmeyer. ProtoVAE: A trustworthy self-explainable prototypical variational model. In Advances in Neural Information Processing Systems, 2022.
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015.
[9] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerehner. beta-VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017.
[10] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR, pages 8769-8778. IEEE Computer Society, 2018.
[11] Holly Huey, Xuanchen Lu, Caren M Walker, and Judith Fan. Visual explanations prioritize functional properties at the expense of visual fidelity. Cognition, 166:105414, 2023.
[12] Riashat Islam, Hongyu Zang, Manan Tomar, Aniket Didolkar, Md Mofijul Islam, Samin Yeasar Arnob, Tariq lqbal, Xin Li, Anirudh Goyal, Nicolas Heess, et al. Representation learning in deep via discrete information bottleneck. AISTATS, 2023.
[13] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2017.
[14] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[15] O. Li, H. Liu, C. Chen, and C. Rudin. Deep learning for case-based reasoning through prototypes:
A neural network that explains its predictions. In AAAI, 2018.
[16] Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: contin- uous relaxation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. 2017.
[17] Phil Maguire, Philippe Moser, and Rebecca Maguire. Understanding consciousness as data compression. Journal of Cognitive Science, 17(1):63-94, 2016.

---

[18] Yao Ming. Panpan Xu, Huamin Qu, and Liu Ren. Interpretable and steerable sequence learn- ing via prototypes. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 903-913, 2019.
[19] Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta- reinforcement learning. arXiv preprint 1rX.iv/803..11
2018.
[20] Samuel Planton, Timo van Kerkoerle, Leila Abbih, Maxime Maheu, Florent Meyniel, Mariano Sigman, Liping Wang, Santiago Figueira, Sergio Romano, and Stanislas Dehaene. A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans. PLoS computational biology, (7(1)=1008859) 2021.
[21] Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. Advances neural information processing systems, 32, 2019.
[22] Aurko Roy, Ashish Vaswani, Arvind Neelakantan, and Niki Parmar. Theory and experiments on vector quantized autoencoders. arXiv preprint crXiv:1805.11066 2018.
[23] Vivien Sainte Fare Garnot and Loic Landrieu. Metric-guided prototype learning. arXiv preprint arXiv:2007.03047, 2020.
[24] Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXivv/703.0000 10000000, 2017.
[25] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pages 1-5. IEEE, 2015.
[26] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
[27] Mycal Tucker, Roger Levy, Julie Shah, and Noga Zaslavsky. Trading off utility, informativeness, and complexity in emergent communication. Neural Information Processing Systems (NeurIPS), 2022.
[28] Mycal Tucker, Roger P. Levy, Julie Shah, and Noga Zaslavsky. Generalization and translatability in emergent communication via informational constraints. In NeurlPS 2022 Workshop on Citimattion-Theoretie Principles in Cognitive Systems, 2022.
[29] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[30] GN Wilkinson and CE Rogers. Symbolic description of factorial models for analysis variance. Journal of the Royal Statistical Society: Series C (Applied Statistics), 22(3):392-399, 1973.
[31] J Gerard Wolff. Information compression as unifying principle in human learning, perception, and cognition. Complexity, 2019, 2019.
[32] Hanwei Wu and Markus Flierl. Variational information bottleneck on vector quantized autoen- coders. arXiv preprint 1xXi:1808.00144 2018
[33] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint
[34] Justin Yang and Judith Fan. Visual communication of object concepts at different levels of abstraction. Journal of Vision, 21, 2021.
[35] Justin Yang and Judith E Fan. Visual communication of object concepts at different levels of abstraction. arXiv preprint arXiv:2 (tt:::20660000 100.::2006600775,, 2021.
[36] Noga Zaslavsky, Charles Kemp, Terry Regier, and Naftali Tishby. Efficient compression in color naming and its evolution. Proceedings of the National Academy of Sciences, 115(31): 7937-7942, 2018. doi: 00.1073/pnas.1800522.
12

---

D
E
Z
P
Figure 9: experiments, we used a common feature-extractor (F), decoder (D), and predictor (P)
network backbone while replacing different encoder heads (E).
A Implementation details
A.1 Pretraining
In pre-training (before the finetuning with a small number of examples a cruder task), we used the following setup. In general, the overall network architecture comprised a feature extractor, an encoder head, decoder, and predictor, as depicted in Figure 9
In experiments, the predictor was parametrized as two-layer feedforward neural network with hidden dimension 128 and ReLU activation. The last layer's 's dimension depended upon the exact prediction task (e.g., 10 neurons for FashionMNIST. 100 for CIFARI00, and 1010 for iNat) and used softmax activation.
The feature extractors and decoders varied by domain. For FashionMNIST, the feature extractor used 3 2D convolution layers, followed by one fully connected layer. The decoder used two linear layers, followed by inverse convolution layers. We again emphasize that the code for these models is available in our codebase, linked to at the beginning of this section.
For CIFARIO0 and iNat, we pre-processed the images extract the 512-dimensional activations from the penultimate layer of a ResNet18 pretrained on ImageNet [8] These features were used as inputs to the feature extractor (z in Figure 9). For both CIFAR 100 and iNat, the feature extractor used two linear layers, with ReLU activation after the first layer. which had hidden dimension 128. The decoder used reconstruct the 51 2-2-imnnnionn outputs of the ResNet18, using 3 fully-connected layers of dimension 128, 256, and 512, with ReLU activations between layers.
The different encoder heads were 3-VAE, VQ-VIBg, and VQ-VIB models. 33VAE models used two linear layers, branching off the output of the feature extractor, to generate μ and from which to sample continuous latent variable. VQ-VIBe directly passed the output of the feature extractor into the vector quantization layer, from which the discrete latent representations were sampled, as described in the main paper. In VQ-VIB,, the output of the feature extractor passed through two linear layers to generate a μl and a a (exactly as in the 3- VAE case) before the sampled continuous representation
discretized via vector quantization. Across experiments, the only differences among encoder heads that could arise were due to different latent dimensions (although we fixed it to 32 for all experiments) or, for VQ-VIBa and VQ-VIBy, the number of elements in the learnable codebook or n, the number of quantized vectors to combine into a latent representation.
In the main paper, discussed the VQ-VIBc training loss (Equation [). maximizing utility, mini- mizing reconstruction loss, and minimizing the entropy of the categorical distribution over codebook elements. A strict generalization of Equation]] TT in which a variational bound on the complexity of representations is also penalized, included in Equation 2
max AU v)) >rE[||= - ]]]3]
- H(P(CIA((Z))
(2)
[E(1,
-AcE (DKa (PCCP)PLPP((())))]CCCCCCI]
- - sg]G
13

---

Equation 2 differs from Equation I via the third line, penalizing the KL divergence between the conditional categorical distribution over codebook elements and uniform distribution over the C elements. This provides a variational bound on T(X, 7), dubbed the complexity of representations in prior literature [6 27 In our main experiments, we set Ac = 0 and vary X ablation studies in which we varied À instead of λи are included in Appendix D and confirm that controlling the
entropy of representations supported better finetuning accuracy.
In training 3- V-, we trained to maximize the function described in Equation where uur) and (z) represent the μ and o parameters output by the encoder.
max X B(U v)l X B(1x =113] E |RI (V ( (z)))l 1)1l
Equation 3 trains agents to maximize classification accuracy, minimize MSE, and minimize the complexity of representations. The scalar weight λ can be viewed as a Lagrange multiplier, constraining how much information can be encoded in representations. This equation is closely related to Equation 2 but, given the continuous nature of encodings in 3-VAE, we could not penalize the entropy of a categorical distribution.
In training VQ-VIB we used the training objective proposed by Tucker et al. [27] which closely resembles the training loss for VQ-VIBe and is shown in Equation (and closely matches Equation 2 We use the same notation as for the 3- VAE and VQ-VIBc models.
max (E 1/ - =|[²]
AHE [[(P(SI(=))
- AAEE (Dil)Dal
1)ll
[|=g[/[(<]]] - - (g(())))g((((()))) =x][1(
The two key differences between Equation 4 and Equation (used for training VQ-V are bounds on entropy and complexity (on the second and third lines of Equation 4 Just as for 3-VAEs, VQ- VIB models uses a KL divergence loss to regulate the complexity of representations. Increasing Ac. we did while annealing complexity in experiments, decreases the amount of encoded information. However, as shown in our results, simply increasing Àc does not ensure that VQ-VIB models will use fewer discrete representations. (For visualizations of this effect, see Appendix C) Tucker et al. (77 advocate for using a small positive Air to penalize the estimated entropy over codebook elements. We explore varying Au in Appendix and find some benefits relative to our main experiments, in which we set λµ = O. However, given that VQ-VIB only supports an approximation of the entropy term, we find that controlling the entropy for VQ-VIB, is not as effective as controlling the entropy for VIBe.
A.2 Finetuning
In finetuning, we loaded pretrained frozen encoders and trained new predictor models map from encodings to downstream predictions.
For finetuning task with M distinct classes, and "duplication factor," k. for how many examples of each class to train with, we randomly selected AM datapoints to train with. For example, when finetuning the binary CIFAR 100 task of living vs. non-living things, M 2, we loaded total datapoints for = datapoints for = 2, etc.. For each input in the finetuning dataset, generated encoding by passing through the encoder once. This generated new dataset of encodings and labels, which we used the train the predictor. (Note that this approach is distinct from passing the input through the encoder many times; given stochastic encoders, which used, the same input could result in many different encodings. Here, we assumed a limited budget of encodings.)
Predictor neural networks were instantiated as feedforward networks with fully connected layers, with hidden dimension 256 and ReLU activations, and trained to map from shifted encodings (see
14

---

Table I: Hyperparameters for FashionMNIST training.
ENCODER
LATENT DIM
AU
AC
À INCR
AHo
AR INCR
8- 3-VAE
32
NA
NA
10
0.01
0.5
0.0
0.0
VQ- VIBN
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIBX
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIB
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIBe
32
1000
10
0.0
0.0
0.001
0.2
VQ-VIBc
1000
10
0.0
0.0
0.001
VQ-VIB e
32
1000
10
10
0.0
0.0
0.001
0.8
next paragraph) to classifications for 100 epochs using an Adam optimizer with default parameters, with the learning rate decreasing by factor of 10 based on plateauing training loss, with patience of epochs, and early stopping the learning rate fell below 10-8
One particular design choice that we made in finetuning predictors merits elaboration: shifting encodings. Rather than directly training predictors map from encodings to predictions, we applied simple linear transformation to the encodings before feeding them to the predictor. Specifically, we multiplied all tensor elements by 5 and increased them by This simple linear transformation does not affect any relations between encodings except scale, and indeed we found that predictors could be successfully trained with this rescaling. Nevertheless, this linear transformation was important to provide check against merely relying upon initialization conditions for good finetuning performance. In particular, we found that we did not apply this linear transformation (i.c., pass the raw encodings to the predictor), predictors sometimes performed better than they should given the training data. For example, as a sanity check, we trained predictor on a binary classification task, but only provided two positive datapoints and no negative data. In general, given this data, would expect trained predictor to only predict positive labels. However, we observed that in several cases, the predictor would achieve nearly perfect accuracy, including predicting negative labels for negative inputs. This surprising result disappeared when we simply shifted encodings, indicating that the particular initialization conditions of the predictor seemed to align well with pre-trained encoders. We wanted to measure the effect of data on finetuning performance, rather than just initialization conditions, but note that this odd phenomenon of well-aligned initializations merits further investigation.
We ran 10 finetuning trials per model, which was important given the small amount of randomly- sampled finetuning data.
A.3 Hyperparameters
In the following subsections, present the hyperparameters used for training different encoders in the different domains. In general, we used the following principles when choosing hyperparameters:
For VQ-based methods, use a large enough codebook to have at least one element per class. Larger C' are also acceptable, tuning weights should decrease the effective codebook size.
When annealing, use small enough weight increment to generate smooth changes during training. Larger increments, however, speed up training.
When annealing for larger n one can increase the annealing rate. Models with greater n tended to more complex representations, annealing could be extremely slow for small increments.
A.3.1 MNSHI
For FashionMNIST. we trained all models with batch size 64 for 200 epochs, using the hyperparam- eters specified Table The only differences across methods were which hyperparameters we annealed to penalize complexity. Other differences simply reflected differences in architecture (e.g.. using a codebook vector-quantization methods). Pre-training single model for 200 epochs took approximately 5 minutes a desktop computer with one NVIDIA 2080 GeForce RTX.
15

---

Table 2: Hyperparameters for CIFAR100 training.
ENCODER
LATENT DIM
AU
ACo
À INCR
AHo
AF INCR
8- 3-VAE
32
NA
NA
10
0.01
0.1
0.0
0.0
VQ- VIBN
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIBN
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIB
32
1000
10
10
0.01
0.5
0.0
0.0
VQ-VIBe
32
1000
10
0.0
0.0
0.001
0.04
VQ-VIBc
10
0.0
0.0
0.001
VQ-VIE VIB.
32
1000
10
10
0.0
0.0
0.001
0.12
Table 3: Hyperparameters for iNaturalist training.
ENCODER
LATENT DIM
C
Ay
A
AGo
Ae INCR
AHo
λи INCR
3-VAE
32
NAA
NA
10
10
0.0001
0.03
0.0
0.0
VQ-VIBæ
32
2000
1
10
10
0.001
0.1
0.0
0.0
VQ- N
2000
2
10
0.001
VQ-VIB
32
2000
4
10
10
0.001
0.4
0.0
0.0
VQ-VIB-
32
2000
1
10
10
0.0
0.0
0.001
0.05
VQ- VIBe
32
2000
2
10
10
0.0
0.0
0.001
0.10
VQ-VIBA
2000
10
0.0
0.0
0.001
0.20
A.3.2 CIFARI00
For CIFAR100, 100, trained all models with batch size 256 for 400 epochs, using the hyperparameters specified Table 2 As explained for FashionMNIST, the only substantial differences across terms that needed to be specified, and which terms were annealed to penalize complexity. Pre-training single model for 400 epochs took approximately 10 minutes on desktop computer with NVIDIA 3080.
architectures architecture-specine
A.3.3 INaturalist
For iNat, we trained all models with batch size 256, using the hyperparameters specified in Table 3 trained 3-VAE and VQ-VIBe models 300 epochs, while we trained VQ-VIB models for 600. We used more annealing epochs for VQ-VIBy simply because it seemed to need more epochs to eventually anneal to random chance. Likely, a larger annealing rate could also accomplish the desired effect, but initial experiments with faster annealing tended to induce rapid codebook collapse that did not generate the smooth spectrum of MSE values we desired. Pre-training a single model for 300 epochs took approximately 10 minutes a desktop computer with one NVIDIA 3080 (and twice as long for 600 epochs).
Further Finetuning Results
In the main paper, we included only a small number of graphs highlighting our key results. Here, we include further results that corroborate the main trends stated the paper. Primarily, these plots include further experiments for varying the amount of finetuning data, as well as varying n. the number of codebook elements combine into a latent representation. We further include details of validation-set experiments, wherein we evaluated method for autonomously selecting the optimal encoder. Results are divided by domain: FashionMNIST, CIFAR100, and iNat.
B.1 MNSH
Here, we include the finetuning results for the MNsSionMNIST domain for varying amounts of finetuning data, ranging over [1, (1,2, 5, 10, 50), and n, the number of quantized vectors combine into single representation. Results for each are included in Figure I0
16

---

FishionMNNST 5-azy Accuncy;
PachiooNNSST I-way Accuracy:
FashionMINSST S-way Accuracy.
VOVE
A-2
VOYA
Y A
Y.Acc
MSE
MSE
MSB
(a)
(b)
(c) 5
FushionMNIST way Accuracy.
PasticaMNIST way Accuracy;
6430
A-2
Y A.A.c
3G1
MSE
MSE
(d) k = 10
(c) k 50
Figure 10: FashionMNIST finetuning results for varying k. As k increased, all models benefited. The data-efficiency of advantage of VQ-VIBg was most pronounced when using the least amount of data.
Table 4: Finetuning accuracy across domains, using encoders selected via validation set accuracy. For large validation (t) and training (k) sizes, this method selected high-performance encoders, but for low k and v, it struggled. This motivates our human in-the-loop framework in the low-data regime.
FASHIONMNIST
CIFAR100 2- WAY
INAT
WAY
v
k
=
5
10
50
2
5
10
50
2
5
10
50
1
0.73
0.89
0.93
0.97
0.71
0.79
0.83
0.90
0.65
0.79
0.89
0.90
5
0.94
0.97
0.82
0.91
0.88
0.90
10
0.97
0.91
0.89
As expected, increasing the amount of finetuning data improved performance for all models, and the gap between all model types (VQ-VIBe. VQ-VIBA' and 3-VAE) shrank. It is noteworthy, however, that a VQ-VIB model, tuned to the right complexity level and trained with just one example per ternary class (Figure 10 achieved better accuracy than 3-VAE model trained with 50 examples per class (Figure 10 e). Further, for any fixed k, VQ-VIBe consistently outperformed VQ- VIB suggesting that many recent works that VQ-VIB v could be improved by replacing the model type (27] 28 12] Z]]. Lastly, for both VQ-VIB and VQ-VIBe, increasing n tended to support lower MSE but worse finetuning accuracy. This supports intuition that combining more discrete representations starts to more densely fill the representation space, trending towards continuous representations.
We note briefly that VQ-VIBx, both in this domain and others (explored in the next sections), typically failed to learn complex representations as either VQ-VIBe 3-VAEs. This is apparent given the limited range of MSE values for the VQ- VQ-V curves. We consistently struggled to make VQ-VIB, learn as rich representations as for the other model types, which led to worse reconstructions and higher MSE values.
Lastly, Table includes results from our validation experiments for all three experiment domains. Recall that we tested a method for autonomously selecting the best encoder, among the suite of encoders of different complexity levels, by measuring validation set accuracy. Table 4 includes results from such experiments for different validation set sizes (v) and different k. For a given i and v, we randomly sampled datapoints per class label to be part of the validation set and used the remaining
17

---

data for finetuning, For large k, this method worked quite well by selecting high-aceuracy encoders. However, for small & and v, validation set aceuracy was noisy proxy for model performance (because of the small validation set size), so performance tended to be suboptimal. For example, in the FNSSi domain, for l: = 2, V = 1, models achieved mean performance of 73%, lower than the over than 80% accuracy achieved by tuning to the right complexity (Figure[TO) b). Overall, these validation set experiments confirm that, for large enough k. one may autonomously select optimal encoders, but for very small k, such autonomous methods fail.
B.2 CIFAR100
We found similar trends in the CIFARIO0 to those in the FashionMNIST domain and plotted results in Figures 11 and T2 ffor 2-way and 20-way finetuning tasks, respectively). In all experiments, VQ-VIBe outperformed both 88-AE and VQ-VIB In the 2-way finetuning example, we again found peaked curve for VQ-VIBe finetuning accuracy as function of MSE, indicating that tuning to the right complexity level induced the best accuracy. In the more complex 20-way classification task, however, we did not observe this peak.
This last result is unsurprising: the 20-way hierarchy in CIFARI00 is less semantically meaningful and likely less obvious in photos than the 2-way task of distinguishing living and non-living things. For example. two of the 20 categories are simply different sorts of vehicles. It would be extremely surprising for VQ-VIBe to learn such arbitrary groups automatically while compressing represen- tations. Without learning the right groupings, VQ-VIBc cannot benefit from learning less complex representations.
CIFAR100 Accuracy;
CIFARI00 way Accuracy:
CIFAR100 2-way Accuracy:
130%
Y AAC.
Y ACC.
Y Agc.
1000
0-2
CBl 128
120
C20
1750
620
170
232
020
120
320
326
CRO
580
(b) 2
k
5
CIFAR100 2 Accuracy:
CIFAR100 way Accuracy;
Y Acc.
Y Acc.
(0Y1A
7250,
10155
15590
C-2
120
220
C20
220
526
120
330
120
420
340
(d) k 10
(0) k 50
Figure 11: CIFAR100 2-way finetuning results for varying k.
18

---

CIPAR1O0 20-way Accurcyy
CIFARIO0 Accuracy,
CIFAR100 Accuncy:
6-2
VOUS
1902,
Y Apc:
Acc
126
430
475
020
429
038
+20
Cle
C.5
(a)
(b)
(c)
CIFARIOI 20-way Accuracy
100 2-way Accurscy.
PALL
1949
A-2
6-2
Y Apc.
675
1.5
675
7.50 475
(d)
(e) k 50
Figure 12: CIFAR100 20-way finetuning results for varying k.
19

---

B.3 iNaturalist
Lastly, we found similar trends in finetuning in the iNat domain, finetuned on a 3-way (Figure 13 way (Figure [4). and 1010-way (Figure 15) finetuning task.
On the 3-way finetuning task (between animals, plants, and fungi), we observed similar peaking behavior as in earlier experiments, indicating yet again the importance of tuning to the right complex- ity. In addition, as in prior results, found similar trend that greater tended to allow greater complexity (lower MSE) but induced worse finetuning performance. For example, in Figure b the orange line, corresponding to n stays above and to the right of the green (n = 2) and red (n = 4) lines. Intuitively, this seems to indicate that the more combinatorial representations, with greater n, were somewhat of a midpoint between the continuous 3-VAE representations and the discrete representations used by VQ-VIBe for n = 1.
Results from the 34-way finetuning followed similar patterns as before well. Just as in CIFAR100 wherein we tested both a 2-way and 20-way finetuning task, this 3a-way finetuning task for iNat showed that VQ-VIBc continued to outperform VQ-VIB and 3-VAR for more complex finetuning tasks, although the performance gap shrank as l: increased.
Most interestingly, perhaps, we conducted yet another iNat finetuning experiment, this time using the 1010 low-level labels that had originally been used during pre-training. As before, we used very small amounts of data in finetuning (e.g.. for k = 1, only I example from each class, so 1010 labeled examples total). Results from those experiments are shown Figure T5
For small k we again see that VQ-VIBe outperforms other model types. For larger k. however, we see one of the limitations of VQ-VIBe. Because the discrete encoders learned less complex representations than B- FA (as shown by the fact that they never reach lower MSE values), with enough finetuning data, 3- VAVS are able to capture distinctions between classes that VQ-VIBe models cannot. Thus, in the particular case of large amounts of finetuning data and complex finetuning tasks, more complex, continuous encoders continue to outperform our method.
iNat 5-way Accuracy k
(Nat 3-way Accancyy d
UNut 3-way Accuracy. &
pink
ACC.
Y Asc.
MSE
MSE
MSB
(a) A:
(b) k 2
(c) k = 5
(Nat B-wny Accancy, le = 10
(Nit 3-any Accunccy k 50
Acc.
pind 1954,,
MSE
MSE
(d) k = 10
(e) k = 50
Figure 13: iNat 3-way finetuning results for varying k.
20

---

1hat 54-way
iNat 34-way Accunscy,
(Nat Accuracy,
Y Acc.
112
26
210
(b)
(c) =
(Nat 34-way Accursey
iNat 34-way Accuncyy
Y Adc.
Y ACC.
(1244
1993
7203
1002
7222,
C-4
26
(d) k 10
(c) k 50
Figure 14: iNat 34-way finetuning results for varying k.
iNat Accuracy:
iNat Accuracy:
iNat 1010 Accuracy;
VO0L, C-2
Y AA2c.
Acc
402
12
1
(b)
(c)
iNat way Accuracy:
iNat 1010-way Accurscy:
pona
434
412-
230
0.5
-30
(d)
(e) k 50
Figure 15: iNat 1010-way finetuning results for varying k.
21

---

51
(a) Epoch
(b) Epoch
Epoch
Epoch 199
Figure 16: The evolution of the distribution over prototypes during annealing for VQ-VIBe in the FashionMNIST domain. In early epochs, VQ-VIBe uses many prototypes, with long-tailed distribution. Over the course of annealing the entropy, the probability distribution becomes more concentrated (b and c) before collapsing to a single prototype (d).
C Prototype Utilization: Further Visualizations
Here, we include some further visualizations that we omitted from the main paper due to space constraints. These visualization primarily illustrate the importance of entropy-regulated representation learning (for VQ-VIBe) vs. complexity-regulate (for VQ-VIBw).
Figures 16 and 17 shows the prototypes for VQ-VIBe and VQ-VIB respectively, in the FashionM- NIST domain over the course of training. Each subfigure consists of a top row of decoded prototypes, with associated probabilities (frequency of use measured when passing through images from the test set) below. The 30 most frequent prototypes are visualized, or fewer prototypes fewer were used.
There is an important trend in Figures and and 17 the entropy-based annealing for VQ-VIB caused models to fewer prototypes, while the complexity -based annealing for VQ-VIB, did not. At epoch 40, just as both methods begin annealing, VQ-VIBe and VQ-VIBy use large number of prototypes, seen by the long-tailed distributions. Over the course of annealing, however, VQ-VIB uses fewer prototypes, and merges images of different classes into the same prototype. Thus, the degenerate encoder the end of annealing (epoch 199) uses just single prototype represent all possible inputs (Figure I6]. At the same time, VQ-VIB during annealing, does not use fewer prototypes. Rather, the complexity peeattty-paalizationnn term seems induce the model to make the mapping from input to prototype more stochastic (Figure 17). Thus, the degenerate VQ-VIBM encoder uses many prototypes, each of which blurry because it could correspond to any input.
Visualizations of decoded prototypes for the CIFAR100 domain is more challenging, In richer image domains, prototype-based methods often training examples as prototypes [8 4], which can make it more difficult to understand when a single prototype represents more than one concept. Nevertheless, by visualizing the distribution over prototypes (without decoding them), we see the same pattern that VQ-VIB, tends to learn to use fewer prototypes over the course of annealing than VQ-VIB,. Snapshots of the categorical distributions for CIFAR100 are included in Figure [8
D Ablation Study: Entropy vs. Complexity
Here, we present results motivating penalizing entropy, as opposed complexity, in VQ- VIB.. Appendix C showed how annealing entropy VQ-VIBe caused models use fewer prototypes, whereas penalizing complexity in VQ-VIB did not induce similar reductions in effective codebook
22

---

(a) Epoch 40
(b) Epach 60
(c) Epoch
(d) Epoch
Figure 17: The evolution of the distribution prototypes during annealing for VQ-VIBy in the FashionMNIST domain. Unlike annealing entropy for VQ-VIBg annealing the complexity in VQ-VIB did not result in fewer prototypes being used. Instead, each prototype became blurrier, indicating that each prototype became more likely regardless of class.
(a) VQ-VIB: Epoch 50
(b) VQ- VIB Epoch 399
(c) VQ-V Epoch 50
(d) VQ-VIB, Epoch 399
Figure 18: Categorical distribution over prototypes while annealing in the CIFAR100 domain the start of annealing (Epoch 50) and at the end (Epoch 399) for VQ-VIB (top row) and VQ-VIB (bottom row). Annealing the entropy VQ-VIB, caused the model use fewer prototypes (note the degenerate categorical distribution over only one prototype at Epoch 399), whereas annealing complexity for VQ-V VIB did not cause similar concentration.
23

---

MidSI 1-aay Aceunyy
PachiooNNST L-ya Aucuracy:
FashionMNISS S-way Accuracy
Jade
5
VOYA
Y A
Y Y.Acc
Y.A.c
241
MSE
MSE
MSB
(a)
(b)
(c) 5
MisSiooMNIST w-way Accuracy.
FashionMNIST 3 way Accuracy;
6430
A-2
Y A.c.c
3G1
64
MSE
MSE
(d) k = 10
(c) k = 50
Figure 19: FashionMNIST finetuning results for varying k comparing annealing by entropy (VQ- VIBg) and annealing by complexity (VQ-VIBc Comp.). Annealing by complexity resulted in worse finetuning performance.
size. Further experiments corroborate our findings that penalizing entropy was the key to this difference in behavior.
We trained VQ-VIBe agents the FashionMNIST task, using the same pre-training and finetuning procedures as in the main paper, with the only difference being that we annealed the complcxity of representations instead of the entropy. Results from finetuning such models are included in Figure 19
Figure shows that annealing by entropy, as opposed to complexity, was the key factor in improving VQ- VQ-e finetuning performance. The difference in performance when penalizing entropy vs. com- plexity closely matches the difference in performance between VQ-VIB and VQ-VIB examined in the main paper. Thus, the aatropyreggularization term seems to explain much of the difference between VIBC and VQ-vIBy.
In subsequent experiments in the CIFARI00 and NNat domains, therefore, we tested whether penaliz- ing the estimated entropy of VQ-VIB, models matched VQ-VIBe results from the main paper. We note that Tucker et al. (27] advocate for small positive λи to penalize entropy, but the authors also acknowledge that exactly computing this entropy impossible given the VQ-VIB architecture.
Finetuning results for CIFAR100 (on the 2-way and 20-way finetuning tasks) and iNat (on the 3-way and 34-way finetuning tasks), for VQ- trained by varying Å and VQ-VIB, trained by varying Ar. are included in Figures 20 21 22 and 23 Several important trends emerge from viewing these plots, especially compared to results from our main paper for VQ-VIBc controlled via An-
First, finetuning performance is noisier using these models compared to results from the main text. This likely arises, for VQ-VIB models, because increasing λи failed to consistently reduce the number of discrete representations used. Thus, for a given MSE value, different models used different numbers of representations, and therefore exhibited different finetuning performance.
Second, varying λи, instead of λc, seemed to somewhat improve VQ-VIB performance, but not as much as when varying λн for VQ-e as presented in our main paper. For example, consider Figure a The best-performing model, VQ-VIB,, n = 1, peaks at finetuning accuracy of approximately 0. 06, outperforming VQ-VIBe models when varying Àç- However, in Figure [2 we found that VQ-VIBe models in the exact same setting achieved mean accuracy approximately 0.38: more than double the VQ-VIB performance. Thus, varying λy seemed to improve VQ-VIB.v
24

---

2-way Accuracy:
2-way Accuracy.
2-way Accuncy,
Y Acct
Y Acc
Y A.ccc
AHL
PAC
230.
602
1075.
MSE
MSF
MSR
(a)
(b) k 2
(c) k 5
CIFARI00 2-0ay Accurccy.
CIFAR100 2-way Accuracy:
Y ACC.
50se
VOVGL
0225.
MSF
MSR
(d) k 10
(e) k 50
Figure 20: Ablation study results for the CIFARI00 2-way finetuning task. Tuning Ae for VQ-VIB or AH for VQ-VIB led to worse results than in our main paper.
way Accuracy
way Accuracy;
way Accuracy:
Y Aurt
Y Acc.
A.cc
1953.
410
102/42
MSE
MSE
MSB
(a) 1
(b) 2
(c) 5
w0y Accuracy:
20-wa Accuracy.
V2V8,
Arc.
Y Acc.
114
7250:
9224,
MSE
MSE
(d) k 10
(e) k 50
Figure 21: Ablation study results for the CIFAR 100 20-way finetuning task. Tuning A for VQ-VIB or AH for VQ-VIB led to worse results than in our main paper.
25

---

Dhat B-way Accuracy,
J-way Accuracy;
3-way Accuracy.
Acct
Y Acc.
Y A.ccc
pALL
1050.
120
9209.
MSE
MSF
MSR
(a) 1
(b) 2
(c) =
(Nat 3-way Accancy,
3-way Accuracy
A.C.
50se
100%
1015,
MSF
MSE
(d) k 10
(e) k
50
Figure 22: Ablation study results for the iNat 3-way finetuning task. Tuning Ac for VQ-VIBe or λ for VQ-VIB led to worse results than in our main paper.
34-way Accuracy;
way Accuracy;
way Accuracy:
Pin
0.0
SOUP,
100%,
Work,
800
4.3
MSE
MSE
MSB
(a) 1
(b) 2
(c) 5
(Nat way Accurney
34-wyy Accurccy
444
PAN
Acc.
YOUR,
114
Fule
7250
7244.
MSE
MSE
(d) k 10
(e) k 50
Figure 23: Ablation study results for the iNat 34-way finetuning task. Tuning de for VQ-VIBe or λy for VQ-VIB, led to worse results than in our main paper.
26

---

performance somewhat, but VQ-VIBe better supports penalizing entropy. and therefore achieves higher performance.
Third, varying Åc, instead of Aи, for VQ-VIBc worsened finetuning performance. Once again, by comparing finetuning performance for VQ- ( models in Figure 2 (achieving maximum accuracy around 0.14), to results from our main paper, we note the importance of penalizing the entropy representations.
Thus, general these ablation studies support many the design decisions made in the main paper.
1. Varying λи instead of À for VQ-V e improves finetuning performance by decreasing the number of discrete representations used.
2. VQ- VQ-vIBy benefits somewhat from penalizing entropy, but because it is architecturally unable support exact calculations of entropy, we were unable to match vQ-VIB perfor- mance.
It is certainly possible that some optimal combination of Ry and Ae might further improve VQ-VIB, / or VQ-VIBe performance; initial explorations of such combinations with fixed λи values while annealing Àc did yield obvious results. Most importantly, current findings enough indicate that controlling the entropy of discrete representations appears important for data-efficient finetuning.
27

---

E
User Study Results
Here, we include complete results from our user study. Tinled accuracy rates for all model types and visualizations, for all three questions. For Questions 2 and 3, selecting the lowest-MSE (highest complexity) encoder was correct, so accuracy rates for all model and visualization types for these questions remained high. For Question 1, however, selecting the correct encoder required users to select non-maxe mem-mmmalallccopplaii representations; for this question, only users viewing prototypes of VQ-V VQ-VIBe models performed above random chance.
Table Accuracy rates by encoder type and visualization method for the three questions (reporting means and standard errors over 20 subjects). For the most challenging question (Question 1), visualizing VQ-VIBe prototypes supported the greatest accuracy.
Model
Viz.
Question
Question 2
Question 3
VQ-VIBc
MSE
0.10 (0.02)
0.70 (0.05)
0.85 (0.03)
VQ-VIBe
Proto
0.55 (0.06)
0.75 (0,04)
0.90 (0.02)
VQ-VIB
MSE
0.10 (0.02)
0.80 (0.04)
0.85 (0.03)
VQ-VIB
Proto
0.10 (0.02)
0.90 (0.02)
0.90 (0.02)
We further included full results for our Mixed Linear Effects Modeling statistical tests in Table 6 All but effects are not significant at the p 0.05 level. The two significant effects are 1) Question 1 had significant negative effect on accuracy rate, and 2) there was significant positive interaction effect between visualizing VQ-VIBe prototypes and Question Jointly, these two effects show that Question 1 was harder for users than the other questions, but that seeing VQ-VIBe prototypes to some extent mitigated this increased difficulty.
Table 6: Results of the Mixed Linear Effects Modeling of user responses, predicting accuracy function of model and visualization type and Question. Our key results is that there is significant positive interaction effect between visualizing VQ-VIBe prototypes and performing better on Question (bolded for emphasis). The intercept row corresponds Question with VQ-VIBe-MSE.
Coef.
Std.Err.
P|2
[0.025
0.975]
Intercept
0.700
0.084
8.356
0.000
0.536
0.864
VQ-vIBc-Proto
0.050
0.118
0,421
0.674
-0.182
0.282
VQ-VIB,NMME
0.100
0.118
0.845
0.398
-0.132
0.332
0.200
0.119
1.692
0.091
-0.032
0,433
Question 1
-0.600
0.118
-5.085
0.000
-0.831
-0.369
Question 3
0.150
0.118
1.271
0.204
-0.081
0.381
VQ-VIB:-PProtoo-uessii
0.400
0.167
2.397
0.017
0.073
0.727
YQ-VIByy-MSE::uusttinn
-0.100
0.167
-0.599
0.549
-0.427
0.227
Do-VVByy-PrrtooQuussttt
-0.200
0.167
-1.199
0.231
-0.527
0.127
VQ-VIBB--Proto::ueessoo 3
-0.000
0.167
-0.000
1.000
-0.327
0.327
VQ-VIBy-MSE::Qestiion
-0.100
0.167
-0.599
0.549
-0.427
0.227
3
-0.150
0.167
-0.899
0.369
-0.477
0.177
Group Var
0.001
0.024
F
User Study
the subsequent pages, we have included the exact pdf document survey shared with participants of the user study. This pdf was generated for participant viewing VQ-VIBe prototypes; similar surveys were populated with data for other models (VQ-VIBy) or visualization methods (MSE plots). Furthermore, the order of the three questions randomized to avoid ordering effects.
28

---

Block
Brief
Please do not take this study on a mobile phone, the text and images will not display correctly.
You are free to leave this study at any time.
We will (1) explain the survey format, (2) include two examples of how to complete the survey, and then (3) ask three questions. In total, the survey will take around five minutes.
After you are finished with the survey, you will be redirected to Prolific to recieve your payment.
Please do not take this study if you have done a similar one before.
Thank you for your participation!
Block 11
Please enter your Prolific ID
Block 2
Introduction
A computer program is good at classifying clothes into these 10 categories
T-shirt
Trouserr

---

Pullover
Dress
Coat
Sandal
Shirt
Sneaker
Bag
Ankleboot
See below for example visualizations of these 10 categories
Block 3
Introduction
However, imagine that you don't care about these 10 specific categories.
Instead, you want to categorize them into more high-level groups.
For example, you may only want to sort your clothes into these more general 3 groups:
Group 1: t-shirt, shirt, pullover, and dress
Group 2: boot, sandal, ankleboot, and sneaker
Group 3: bag and trouser

---

Block 10
Your task is the following:
Step Read a description of the categories that the computer program needs to sort items of clothing into. These categories will change for different questions, so make sure to re-read the categories.
Step 2: Look at visualizations of different computer programs.
Step Based on the visualizations from Step 2, select which of the programs you think will be best able to accomplish the task described in Step 1. In general, try to select the visualization that 1) can represent the different categories we ask for and 2) has the fewest representations, while still having enough representations to distinguish between categories.
On the next page, we have included an example of how follow these steps.
Example Question
Example Question 1:
Step 1:
The categories are:
1. Group T-shirt, Shirt, Pullover
2. Group 2: Purse
Step 2:
Select which of the following visualizations best reflects the categories from Step 1.
Option 1

---

Option 2
Option 3
Option 2 is best for this task because demonstrates visualizations of the two groupings that we care about here (distinguishing between purses and differrnt types of shirts) while using few representations.
Options 1 has enough representations for the two groups we care about, but it has more representations than Option 2, so Option 2 is better choice,
Option 3 is too simple and therefore cannot tell the two groupings apart.
Step 3 (Example 1):
Please select which option you think best represents the groupings listed:
Option
Option
Option
Example Question 2:

---

Step 1:
The categories are:
1. Group T-shirt
2. Group 2: Pullover and Shirt
3. Group 3: Purse
Step 2:
Select which of the following visualizations best reflects the categories from Step 1.
Option
Option 2
Option
Option 1, is best for this task because demonstrates visualizations of the three groupings that we care about here. Because we care about distinguishing between t- shirts and shirts (Group 1 vs. 2), we need to be able to tell t-shirts apart based on the visualizations.
Options 2 and 3 are bad choices because they cannot tell t-shirts (Group 1) apart from other pullovers or shirts (Group 2). Therefore, even though they use fewer representations

---

than Option 1, they are bad options.
Step 3 (Example 2):
Please select which option you think best represents the groupings listed:
Option
Option 2
Option
Block 10
Please only participate in this study if you are confident you understand the instructions correctly
Block 11
Click next to begin the study
Question 1
Question:
Step 1:
The computer program needs to sort the items into the following categories:
1. Group 1: T-shirt, Trouser, Sandal
2. Group 2: Pullover, Dress, Sneaker
Group 3: Coat, Shirt, Bag, Ankle Boot
Step 2:
Please look at the options below:
Option

---

Option 2
Option 3
I
Option 4

---

Step 3:
Please select which option you think best represents the groupings listed:
Option
Option
Option
Option
Question 2
Question:
Step 1:
The computer program needs to sort the items into the following 3 categories:
1. Group 1: T-shirt, Pullover, Coat, Shirt
2. Group 2: Trouser, Dress, Bag
3. Group 3: Sandal, Sneaker, Ankle boot
Step 2:
Please look at the options below:
Option
Option 2

---

Option 3
I
Option 4
Step 3:
Please select which option you think best represents the groupings listed:
Option
Option
Option
Option 4

---

Question 3
Question:
Step 1:
The computer program needs to sort the items into the following 10 categories:
1. T-shirt
2. Trouser
Pullover
4. Dress
5. Coat
6. Sandal
7. Shirt
8. Sneaker
Bag
10. Ankleboot
Step 2:
Please look at the options below:
Option 1
Option 2

---

Option
Option
Step 3:
Please select which option you think best represents the groupings listed:
Option
Option
Option 3
Option
Block 9
Debrief Page

---

Thank you for your participation, this study was designed to evaluate people's ability to select the right level of abstraction for an Al to perform well at a particular task.
if you have any questions please contact ekenny@mit.edu
Click next to finish
Powered by Qualtrics


Reviews:
[{"review_id": "CmZpLWdF7n", "reviewer": "Reviewer_6aXd", "summary": "This paper explores an interesting premise -- how does the level of abstraction captured by a discrete (visual) representation dictate downstream task performance, where downstream tasks can be at arbitrary levels of abstraction. Specifically, the running example from the work that I really like is that of bird-watching; when communicating amongst experts, the extremely specific species names (e.g., \"white osprey\") is very useful, and captures the right level of information. However, when communicating with small children who are looking at things in the wild, coarser descriptions (e.g., \"the white bird\" or \"the red bird\") are much more useful. \n\nTo study this premise, this paper makes two contributions: first, it introduces a new method for learning discrete representations at various levels of abstraction, the Vector-Quantized Variational Informational Bottleneck - Categorical (VQ-VIB_c). Second, and more importantly, it presents a thorough evaluation, including a human-in-the-loop user study (N = 17), showing conclusively for downstream tasks with \"known\" abstraction levels, the \"best\" representation to use is the one that roughly matches that level of abstraction.", "questions": "I am very confused at the two-phase approach for varying representational capacity during pretraining. The paper seems to indicate that at first, models are pretrained until they obtain perfect accuracy at the (full) classification problem (e.g., the 10-class classification in FashionMNIST, the 100-class classification in CIFAR10). Then, after that happens, by tweaking the loss hyperparameters (e.g., downweighting \"utility\" and upweighting \"representation conciseness\" / \"commitment loss\"), you compress the representations until they are not meaningful, and look at various saved models that are produced in the course of that process for downstream finetuning (hence the plots that track accuracy on downstream vs. MSE).\n\nWhy does this approach make sense at all for evaluating the \"conciseness/abstraction\" of a representation? How are you supporting the claim that the \"trajectory of models\" between high accuracy and \"indistinguishable\" representations is capturing different levels of abstraction in the representation space? This feels like a major major connection that needs to be made explicit in the paper.", "limitations": "The main body of the paper does not explicitly discuss limitations; the appendix discusses some of the limitation of the VQ-VIB_c approach in passing, but a proper treatment of the pros/cons of this approach is not present.\n", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper starts with a strong motivating question, introduces a new method to explore the hypotheses formed from this question, and critically performs a comprehensive evaluation and user study that confirms the proposed hypothesis. I think the user study linking \"prototypes\" from different models (capturing different levels of abstraction) to the downstream finetuning task's level of abstraction is the most convincing, and most meaningful result in this work. I really like this type of study, and it really helps support the paper's key contributions.\n", "weaknesses": "Unfortunately, I have several concerns with this paper, stemming from the very confusing presentation of the paper (it took me a very long time to understand the experimental protocol, and I found that several details about the pretraining/finetuning procedure were missing or placed in scattered places in the Appendix), to the actual useability of such an approach for more practical applications.\n\nFirst, on the presentation/soundness side -- I ask this question more explicitly below, but what is the exact link between the process of \"degrading\" a high-accuracy model (pretrained to high accuracy on the \"full\" classification task), and emerging levels of abstraction? In other words, how exactly does the process of changing the loss hyperparameters lead to models that capture different levels of granularity? From my read of the paper/appendix, this link is never made explicit, and the process is further left underspecified -- if hyperparameters are changing in a staged way (or you're running for a limited number of additional gradient steps given the \"high accuracy\" model), couldn't a possible confound be the initial batches seen during this \"degradation process\"? Another confound could just be the per-example/per-class difficulty, which is known to be uneven across classes in the datasets (e.g., iNaturalist has severe label imbalance out of the box, papers from the Distributionally Robust Optimization literature report huge discrepancies in average vs. worst class accuracy for other datasets)? Is this controlled for in any way? \n\nBeyond these questions, I find it unclear how such an approach would be useful for practical applications? Is the idea that given some supervised dataset, you should train all models to \"best accuracy\" then selectively degrade the resulting representations depending on your (family of) downstream tasks? Isn't this expensive/redundant? \n\n---\nEDIT (Post-Rebuttal): I think many of my soundness concerns have been addressed during rebuttal, and in discussion with the other reviewers. Raising my score to a 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their review!\n\n# Clarifications\n\nThe reviewer raises several questions about how we generate “degraded representations.” Here, we seek to clarify our approach with a brief summary and address specific questions later.\n\nFirst, we train a VQ-VIB$_\\mathcal{C}$ model to high accuracy and low reconstruction loss, by setting high values for $\\lambda_I, \\lambda_U$ and a low value for $\\lambda_H$. After convergence, we increase $\\lambda_H$ by a small amount every epoch. By increasing $\\lambda_H$, we imposing a higher penalty on the entropy of representations. Thus, as $\\lambda_H$ increases, the model uses fewer representations, each of which represents more abstract concepts. The idea of varying a hyperparameter to induce different representation complexity levels is widespread in Information Bottleneck literature for continuous [1, 2] and discrete [3, 4, 5] representations.\n\n# Questions\n\n>What is the exact link between the process of \"degrading\" a high-accuracy model… and emerging levels of abstraction? … [H]ow exactly does... changing the loss hyperparameters lead to models that capture different levels of granularity?\n\nWe increase $\\lambda_H$ after convergence to penalize the entropy of representations, resulting in the model using fewer representations, which necessarily represent more abstract concepts. This in turn leads to decreased model accuracy. These links between hyperparameter tuning, representation complexity, and model accuracy have also been well established in prior literature, such as [3], who explicitly show that decreasing complexity leads to more abstract representations. Similar ideas are explored both theoretically [6] and empirically [1, 2, 4] in other works as well.\n\nWe note in our paper that we increase $\\lambda_H$ during pre-training (line 179) and show in experiments how representations change over the course of varying $\\lambda_H$ (e.g., Figures 4, 7, 16, and 18).\n\n>If hyperparameters are changing in a staged way… couldn't a possible confound be the initial batches seen during this \"degradation process\"? Another confound could just be the per-example/per-class difficulty…?\n\nThe reviewer makes an interesting point, but it appears unlikely that the suggested confounds had an effect in our experiments. We trained and evaluated all methods on the same classes and data, so class imbalance would be unlikely to favor one method over another. Furthermore, we trained across many random seeds and for many epochs, using traditional methods of loading randomly shuffled batches. Therefore, we believe the change in performance across models may be attributed solely to our manipulated variables, e.g. changes in $\\lambda_H$ (for a given model type) or changes in architecture (e.g., comparing VQ-VIB$_\\mathcal{C}$ to $\\beta$-VAE).\n\n>I am very confused at the two-phase approach for varying representational capacity during pretraining... [M]odels are pretrained until they obtain perfect accuracy… after that happens, by tweaking the loss hyperparameters (e.g., downweighting \"utility\" and upweighting \"representation conciseness\" / \"commitment loss\"), you compress the representations until they are not meaningful...\n\n> Why does this approach make sense at all for evaluating the \"conciseness/abstraction\" of a representation? How are you supporting the claim that the \"trajectory of models\" between high accuracy and \"indistinguishable\" representations is capturing different levels of abstraction in the representation space?”\n\nThe reviewer mostly summarized our pre-training process correctly, although we note that we only vary a single hyperparameter during training to penalize the complexity of representations (and not, as stated by the reviewer, by varying the commitment or utility weights). (See line 179.)\n\nWe have already discussed above how we use $\\lambda_H$ to penalize the entropy of model representations, which is motivated by substantial theoretical and empirical existing literature.\n\nWe show in experiments that VQ-VIB$\\_\\mathcal{C}$ models indeed learn different levels of abstraction at different complexity levels during our pre-training process. We show examples of a model from a single training run at high and low complexity in Figure 4. We show a similar example in the iNat domain in Figure 7. Moreover, in Figures 16 and 18, we show how representations evolved during training (as we penalize complexity of representations) for VQ-VIB$_\\mathcal{C}$ models. In each figure, we show how lower-complexity models (which differ from high-complexity models only by training with a greater $\\lambda_H$) use fewer and more abstract representations. Lastly, in every plot with MSE as an x axis, we implicitly show that in pre-training, we forced models to go from low-MSE (high complexity) to high-MSE (high complexity) representations.\n\n> The main body of the paper does not explicitly discuss limitations… a proper treatment of the pros/cons of this approach is not present.\n\nWe thank the reviewer for raising this point. We would like to highlight that we do include a discussion of limitations of our approach in the main paper (line 265), but will work to ensure they are emphasized more clearly in the future draft.\n\nWe hope these clarifications help increase the reviewer’s understanding of our work, and we ask that the reviewer adjust their score if their confusion is sufficiently \naddressed, or suggest specific changes they would like to see to address their concerns.\n\n[1] Deep Variational Information Bottleneck. Alemi et al.\n\n[2] beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. Higgins et al.\n\n[3] Trading off Utility, Informativeness, and Complexity in Emergent Communication. Tucker et al.\n\n[4] Proto-VAE: A Trustworthy Self-Explainable Prototypical Variational Model. Gautam et al.\n\n[5] Representation Learning in Deep RL via Discrete Information Bottleneck. Islam et al.\n\n[6] The Information Bottleneck Method. Tishby et al."}, {"review_id": "8iuUdYHUBm", "reviewer": "Reviewer_99tN", "summary": "The authors observe that the downstream tasks for pretrained models can rely on representations of varying level of complexity: as a running example, a birdwatcher relies on significantly more complex representations of images to classify bird species, relative to a child who may want to identify the color of a bird. They thus propose pretraining a variety of models with representations of varying complexity, having a human choose the appropriate pretrained model for the task, and then finetuning that model for the downstream task. Relative to finetuning the model with the most complex representations, this should lead to increased data efficiency.\n\nThe authors suggest a modification of VQ-VIB (vector-quantized variational information bottleneck) for the pretraining in this setting, called VQ-VIB_C (C standing for categorical) – my understanding is that the main difference is that in the authors’ method, the representation is divided into n chunks and each of the n chunks is snapped to a quantized vector (the loss remains the same). I have not spent much time delving into the details as the authors say this is not their main contribution: in particular I may be wrong about the differences from VQ-VIB.\n\nThe authors implement this idea for a variety of image-based datasets: FashionMNIST, CIFAR-100, and iNaturalist 2019 (iNat). They pretrain an encoder using a reconstruction loss as well as a classification loss for the labels defined by the dataset, and increase the regularization on the representations to generate a variety of checkpoints that produce representations of different complexity. They then evaluate these encoders in a downstream classification task that is coarser than the original classification task (e.g. living vs. non-living for CIFAR-100).\n\nTheir experiments show that (1) VQ-VIB_C outperforms two baselines (VQ-VIB, $\\beta$-VAE), and (2) when the classification task is simple (e.g. 2-3 classes) and there are very few data points (e.g. 1-5 examples per class), it is better to use a pretrained encoder with lower representational complexity (though in all other cases it is typically fine to use the encoder with maximal representational complexity).\n\nThey also perform a human experiment in which they show that, given visualizations of the learned quantized vectors, humans can correctly identify which encoder will perform best during finetuning.", "questions": "1. Could you give examples of realistic settings in which using low-complexity representations would be useful – bearing in mind the four requirements I mention in the “Significance” section in Weaknesses above? (Or alternatively, if I’m wrong about the four requirements, how / why am I wrong?)\n2. Is there reason to expect poor performance from finetuning the most complex representation with stronger regularization? (Possibly with a validation set, as described above)\n3. How well should we expect the method to work when $Y_t$ is not a deterministic function of $Y_p$?\n\nMinor suggestion: Please increase the size of the text in Figure 4(c).", "limitations": "The limitations brought up in the weaknesses section are not mentioned or addressed. In particular, I think the issues raised in “Significance” and “External validity of experiments” should obviously be mentioned (or otherwise addressed); the others are more debatable.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. To my knowledge, the idea of using simpler representations for downstream tasks in order to improve data efficiency is novel, and I think it is an interesting hypothesis to explore.\n2. Although the authors don’t consider it their main contribution, their experiments suggest that the VQ-VIB_C performs notably better than other alternatives on their task. (However, I did not investigate in detail, e.g. it is possible that the baselines were not tuned well while the VQ-VIB_C was.)\n3. More generally, the experiments are quite detailed and look at the effects of various hyperparameters in the method.\n4. The paper has an experiment with real humans to justify its claim that humans could choose an appropriate model to use for a downstream task. While I think the experiment is pretty different from realistic conditions, this is still better than the vast majority of papers on ideas like these, which often don’t bother testing their claims about real humans at all.", "weaknesses": "I have a few concerns:\n1. Significance: It seems to me that the author’s experiments suggest that the main idea only has a benefit in very limited settings, which are unlikely to arise in practice.\n2. Baselines: For k > 1, the appropriate baseline would be to use the most complex representation, with strong regularization determined using a validation set at finetuning time.\n3. External validity: The experiments involve downstream tasks that are pure coarsenings of the pretraining task, whereas in typical settings the downstream tasks will likely not be these “pure coarsenings”, which would reduce performance.\n4. Relevance of the human experiment: The author’s experiment seems to “give away” the answer (though there is a case to be made that this reflects a strength of the method, rather than a weakness of the experiment).\n\nOverall I feel conflicted about this paper. On the one hand, it has a nice idea with careful technical work done to flesh it out, and a large set of experiments to understand how the idea works in practice. On the other hand, I think the experiments suggest that the idea is not very practically useful (whereas the paper suggests they validate the idea), despite the experiments having some aspects that bias them towards showing the idea to be good. Overall I’m recommending a borderline reject, but I can see the case for acceptance as well.\n\n**Significance**\n\nLooking at the experiment results (including the ones in the appendix), it appears that if n > 3, or k > 5, or you use any method other than VQ-VIB_C, it is usually best to use the most complex representation (i.e. the one that achieves highest reconstruction loss). Thus the benefits of the idea only occur when n <= 3 and k <= 5, which implies a tiny dataset of 15 examples or fewer. As we might expect from such a small dataset, the resulting finetuned classifiers do not perform particularly well, getting around 70-90% accuracy.\n\nThus it seems like the idea in this paper is only helpful when (1) there is a very simple classification task, (2) there is very limited finetuning data available, (3) you use VQ-VIB_C rather than a different method, and (4) the user is happy with performance numbers of 70-90% accuracy. This seems extremely restrictive, and I find it hard to think of a realistic setting that satisfies all of these constraints (especially #4). Overall, I would characterize these experiments as providing a negative result.\n\n(I would actually be more inclined to accept a version of this paper that was upfront about this, and framed the paper as a negative result that has taught us that lower representation complexity only buys you a little bit of data efficiency, that is overwhelmed very quickly by a large enough dataset.)\n\n**Baselines**\n\nIn the paper’s experiments, the finetuning is done the same way for representations with varying levels of complexity (I believe, at least I didn’t see anything to contrary in Appendix 7). However, a natural baseline would be to use the regular (maximum-complexity) representations, but use stronger regularization to learn a simpler classifier. One might reasonably argue that it is unclear how to choose this hyperparameter – but at least when k > 1, it should be possible to take 1 example (or more) per class to form a validation set that is used to tune the hyperparameter. I think it is plausible that this significantly improves performance for high-complexity representations. If it does work it would be a significant improvement, as there would no longer be any need for human input.\n\n**External validity of experiments**\n\nAll of the experiments in the paper have the downstream task labels $Y_t$ being a strict coarsening of the pretraining task labels $Y_p$ (or in other words, $Y_t$ is a deterministic function of $Y_p$). However, this may not be the case in realistic settings – for example, in the running example, there may be birds that are of the same species but have different colors, and so a representation that just identifies the species would be insufficient for predicting the color. Should we expect the method to work even in these cases? I’m not sure; the fact that the pretraining includes a reconstruction loss suggests that it would still work (though likely not as well as in the paper’s experiments). Ideally however the authors would conduct such an experiment to test it empirically.\n\n**Relevance of the human experiment**\n\nLooking at the pdf for the user study (Appendix 11), the instructions seem to “give away” the answer. In particular, the instructions contain:\n\n> Generally, the more images the second robot is using, the less general it will be, and the worse it will do at categorizing the 3 new high-level categories.\n\nand\n\n> If the visualization shows that (1) the robot is not using many images, and (2) they roughly represent the three high-level categories [...] then the robot should perform well\n\nand\n\n> For example, here is one robot’s visualization which is perhaps too general, as there is not even three categories being used.\n\nFrom which it is easy to infer “choose the option with fewest categories, subject to having at least three categories” – which then leads to the desired answer.\n\nArguably this represents a strength of the method – the approach for selecting the appropriate model is so simple that it can easily be communicated even to non-experts. However, in this case, it is so simple that there isn’t really any need for human input – we could equally well find the appropriate model using a simple heuristic of choosing the option that has a few more “most important images” than the number of classes in the classification task. (Perhaps this is another baseline which should be compared against.)\n\n**Minor issues**\n\nLine 91:\n> Lastly, we assume that the pre-training labels are a sufficient statistic for the task-specific labels: $I(X; Y_p) = I(X; Y_t)$. Intuitively, this states that the pre-training objective must include relevant information for the finetuning task.\n\nI don’t think that’s the right criterion. For example, for reconstruction pretraining with red-yellow classification as the downstream task, we have $Y_p = X$ and $Y_t$ is whether the bird is red or yellow. Then $I(X; Y_p) = I(X; X) = H(X) \\geq 1$ (since entropy of natural bird images is large), but $I(X; Y_t) \\leq H(Y_t) \\leq 1$ (since $Y_t$ is a binary variable).\n\nI think what you mean to say is that $I(X; Y_t \\mid Y_p) = 0$, that is, conditioned on knowing $Y_p$, there is no more information about $Y_t$ that can be learned from $X$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful review. They understood the paper quite well and provided useful suggestions for strengthening baselines (which we implemented).\n\n# Significance\n\n>[T]he benefits of the idea only occur when n <= 3 and k <= 5… classifiers do not perform particularly well, getting around 70-90% accuracy. Thus it seems like the idea in this paper is only helpful when [4 constraints are met]…\n\nThe reviewer is correct that for every architecture other than VQ-VIB$_\\mathcal{C}$ (our method), we see no benefit to selecting a less complex representation. However, it is not true that “the benefits of the idea only occur when n <= 3 and k <= 5.” For example, for FashionMNIST, performance peaks at the “right” complexity level even for $k = 50$ (Figure 10 in Appendix 8). Notably, this peak supports roughly 96% accuracy.\n\nWe acknowledge that the advantages of our method are limited to the small-data regime. However, rather than frame the whole paper as a negative result, we wish to emphasize that while some existing methods ($\\beta$-VAE, VQ-VIB$\\_\\mathcal{N}$) do not confer benefits from simpler representations, VQ-VIB$\\_\\mathcal{C}$ does! In low-data scenarios, our method demonstrates its value over existing techniques.\n\n# Limitations\n\n>I think the issues raised in “Significance” and “External validity of experiments” should obviously be mentioned.\n\nWe note limitations of our approach as the amount of finetuning data increases (line 167) and repeat this conclusion throughout Appendix 8 (line 631). In future revisions, we will expand such discussion.\n\n# Questions\n\n>Could you give examples of realistic settings in which using low-complexity representations would be useful – bearing in mind the four requirements I mention…? (Or alternatively, if I’m wrong about the four requirements, how / why am I wrong?)\n\nWe believe that the real requirements are less strict than noted by the reviewer: we found benefits to our method when finetuning for FashionMNIST using 50 examples, and achieved 96% accuracy. Thus, requirements 2 (limited finetuning data, k <= 5) and 4 (performance under 90%) are not supported by our results. We also do not see how using VQ-VIB$_\\mathcal{C}$ is a limitation: it outperforms baselines and therefore is the method that we advocate for in this work.\n\nWe believe that our approach could be relevant for low-volume visuomotor robotics settings. Considerable research considers how to rapidly train robots on sorting tasks from vision using few demonstrations, where minimizing the number of new demonstrations required at test time from the human user is desirable [1,2,3]. Using our approach, we could train a robot to identify clothing for sorting (as in our FashionMNIST example) using only 10 examples per class and achieve roughly 93% accuracy! If a factory wanted to retrain the robot on a different sorting task (e.g., a binary defect-identification task), it could quickly supply the small number of finetuning examples needed. This type of flexible test-time adaptation motivates our framework.\n\n>Is there reason to expect poor performance from finetuning the most complex representation with stronger regularization?\n\nWe appreciate the suggestion to train with stronger regularization and did so! Results, as indicated in our overall response, are included in the attached pdf.\n\nWe found that regularization improved model performance overall, but there was nevertheless still the “peaking” effect of optimal performance when tuned to the right complexity level. Intuitively, this arises because regularization can help address aspects of domain complexity, but fundamentally it is still easier to train with less complex domains. Thus, our method continues to outperform the strong baseline of training a regularized classifier on the most complex representation.\n\n> (Paraphrased:) What if the finetuning task labels are not a deterministic coarsening of the pre-training labels?\n\nThis is an interesting question which we explicitly flagged as outside the scope of our current paper. (“Lastly, we assume that the pre-training labels are a sufficient statistic for the task-specific labels…”) Nevertheless, we can speculate what behavior we may see.\n\nWe believe that, if the finetuning labels are less aligned with the pre-training labels, we would see peaked finetuning performance at higher complexity (lower MSE) than in our current experiments (or no peak at all). Generally, compressing model representations helped with finetuning, but only up to the point where information necessary for the finetuning task started being compressed out. With worse pre-training labels, we might expect models to start compressing information that is important for finetuning earlier. Thus, peak performance should occur at higher complexity values.\n\n> (Paraphrased:) Could one automatically select the right complexity level based on the number of prototypes?\n\nWe discuss this question in our overall rebuttal. While an interesting hypothesis, there are theoretical reasons that this approach should fail, which we back up in experiments.\n\n# Minor issues:\n\nThank you for pointing out the error. We wrote the correct text, “we assume that the pre-training labels are a sufficient statistic for the task-specific labels” but miswrote the equation. We will correct the document to write $I(X, Y_t) = I(Y_p, Y_T)$.\n\n[1] \"Robot learning from randomized simulations: A review.” Muratore et al.\n\n[2] \"A Methodology to Design and Evaluate HRI Teaming Tasks in Robotic Competitions\" Marrella, Andrea, et al.\n\n[3] “Preferred interaction styles for human-robot collaboration vary over tasks with different action types” Schulz et al."}, {"review_id": "ny7eUchaOP", "reviewer": "Reviewer_Nw28", "summary": "This paper proposes a framework for human-in-the-loop training of machine learning models where humans select among pretrained models with different complexity levels based on prototypes. The authors demonstrate that finetuning performance is significantly impacted by representation complexity in the experiments considered. Moreover, a user study demonstrates that humans are relatively successful at helping choose the correct complexity level for finetuning. ", "questions": "1. Can you quantify the amount of effort needed by human annotators to select the best abstraction level? \n\n2. Can you quantify the amount of sub-optimality in the human study when the correct abstraction was not selected? \n\n3. How does the effort and efficacy involved in prototype based abstraction selection compare with that of feature engineering?\n\n4. There is something off about the bird abstraction example (line 20). What precisely is the task the abstraction is being used for in this case? Is the expert birdwatcher teaching the child or other experts? What does complexity and simplicity mean in this case? ", "limitations": "The authors do not really address the limitations of the work. I believe the main limitations are the lack of alternative methods for human-in-the-loop training considered i.e. feature engineering and the use of somewhat synthetic benchmarks for evaluation that are disconnected from real-world use cases. It is natural to wonder if humans can be as helpful at prototype based abstraction selection for transfer tasks that are more distant from each other and not drawn from the same benchmark. ", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- The idea of using humans to help select abstractions for a given problem is an interesting idea for human in the loop ML. \n\n- The proposed VQ-VIB approach seems like a reasonable implementation for this use case.\n\n- There seems to be solid empirical verification of the very intuitive connection between representation complexity and fine-tuning performance.", "weaknesses": "- The authors only show transfer performance within single benchmarks rather than across tasks or in realistic pretraining and finetuning settings such as ImageNet or LLMs. \n\n- The authors do not compare with other methods for human in the loop training in which the amount of human effort can be compared. For example, feature engineering is another way that humans can potentially impact downstream performance -- a technique widely used in practice. \n\n- The paper also does not discuss the effort involved in providing this human guidance. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">The authors only show transfer performance within single benchmarks rather than across tasks or in realistic pretraining and finetuning settings such as ImageNet or LLMs.\n\nWhile we do not consider LLMs (we did not wish to expand to language domains), we do use feature extractors pre-trained on ImageNet (as noted on line 457). \n\nMore generally, we do not seek to make any claims or consider transfer to different domains. Cross-domain transfer is an interesting idea, but here we seek to answer the first question of finetuning efficiency within a single domain, with the goal of understanding how to best develop visual representations that are adaptable and flexible for different end tasks.\n\n>The authors do not compare with other methods for human in the loop training in which the amount of human effort can be compared. For example, feature engineering is another way that humans can potentially impact downstream performance -- a technique widely used in practice.  Can you quantify the amount of effort needed by human annotators to select the best abstraction level? How does the effort and efficacy involved in prototype based abstraction selection compare with that of feature engineering?\n\nWhile comparison to other human-in-the-loop pretraining methods is an interesting (and novel!) space to explore, we believe our method exists orthogonal to traditional “manual feature engineering”. First, feature engineering typically requires the designer to have full knowledge of the desired downstream task, effectively requiring *a priori* knowledge of the test task during pretraining (e.g. we need to first know that we are trying to classify birds to identify that beaks are important) [1,2,3]. A specific model is then trained to fit those features. In contrast, our method assumes that we do not possess this information (nor access to the end user during pre-training), but rather “fits” the right representation adaptively at test time. Second, traditional human feature engineering is known to be labor and expertise-expensive [1,2], often requiring either designer knowledge of the feature extraction space or specific visualization tools designed to extract this information [3]. For example, in order for us to run this comparison in line with [3], we would have needed to design a web-interface tool for FashionMNIST, CIFAR100, and iNat, and then have brought in human participants to design features for each downstream task we desired, which would be extremely time consuming (for both the designer and participants).\n\nWe apologize that we did not report results on the time/ease of our human experiment and will correct this omission. Our method averaged ~1m30s per human participant, INCLUDING TEACHING TIME, for each question. This required no designer knowledge of the test task, nor time spent deploying visualization tools or interfaces for extracting end user knowledge of desired features – we simply needed to output the representations that were already generated during pretraining. This illustrates the relative ease and flexibility of deploying our method across different tasks with different end users.\n\n\n>Can you quantify the amount of sub-optimality in the human study when the correct abstraction was not selected?\n\nOur computational results, in conjunction with the human study, indicate the suboptimality of choosing the incorrect abstraction. For example, when users selected the lowest MSE model, that corresponded to accuracy of 55\\%, as opposed to 70\\% accuracy when selecting the best model (via our method), for $k=1$. (Percentages taken from Figure 4c in the main paper). One can generally quantify suboptimality by looking at the options selected by participants and then looking up the accuracy of such options in our computational experiments.\n\n>There is something off about the bird abstraction example (line 20). What precisely is the task the abstraction is being used for in this case? Is the expert birdwatcher teaching the child or other experts? What does complexity and simplicity mean in this case?\n\nWe describe two tasks: an expert classifying birds (line 23) or teaching a child to classify birds into crude categories (line 24). In this context, complexity corresponds to the level of detail about the bird used for the task (e.g., specific plumage patterns for exact species classification vs. general color for the simpler task). Thus, we expect the expert to use a complex representation for classifying species and a less complex representation for discussing high-level colorings with children.\n\n[1] \"An empirical analysis of feature engineering for predictive modeling.\" Heaton.\n\n[2] \"Runtime Support for Human-in-the-Loop Feature Engineering System.\" Anderson et al.\n\n[3] \"Get a human-in-the-loop: Feature engineering via interactive visualizations.\" Gkorou et al.\n"}, {"review_id": "22R7Ulcn9v", "reviewer": "Reviewer_TMXo", "summary": "This paper proposes a method for learning discrete representations whose complexity can be smoothly annealed. Experiments demonstrate that, at the appropriate level of complexity, these representations can be useful for downstream classification tasks involving abstract categories.", "questions": "I would appreciate if the authors could devote more attention to the limitations identified in the previous section, and also provide more concrete detail about the intended use case for the proposed approach.", "limitations": "There are no discernible negative societal impacts related to this work.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work proposes an interesting method for learning a discrete 'codebook' with a controllable level of complexity.\n- Experiments demonstrate a clear non-monotonic relationship between complexity and usefulness in a downstream task involving abstract categories.\n- The method outperforms baselines in this setting.", "weaknesses": "I have previously reviewed this work, and it seems that the paper has not been revised in a manner that sufficiently addresses the concerns raised by myself and other reviewers. Specifically, a major concern with this work as currently formulated is that the proposed human-in-the-loop approach does not seem very realistic. In the abstract, the authors note that 'humans learn discrete representations ... at a variety of abstraction levels ... and use the appropriate abstraction based on tasks'. This is a compelling motivation, but unlike humans, the proposed approach does not involve any method for autonomously selecting the appropriate level of abstraction for a given task. It is unclear how this human-in-the-loop approach will be scaled in a realistic manner, and the authors do not devote enough attention toward addressing this limitation.\n\nIt would help if the authors could add more discussion of this issue, and also describe more concretely the scenarios in which they envision this approach being useful. Given pre-trained models over a range of complexity levels, and some downstream task on which a user wants to fine-tune, would it not be much more straightforward to simply use a validation set from the downstream task to evaluate the full set of pre-trained models? This also suggests the possibility of automating the process of selecting the right abstraction level in a meta-learning setup.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their continued engagement with our work!\n\n>This is a compelling motivation, but unlike humans, the proposed approach does not involve any method for autonomously selecting the appropriate level of abstraction for a given task.\n\nThe reviewer is correct that we do not provide a method for autonomously selecting the appropriate level of abstraction for a given task. Fundamentally, we propose a human-in-the-loop framework rather than an autonomous selection framework.\n\nNevertheless, we appreciate the suggestion to regularize the finetuning models, using a validation set to tune regularization. As noted in our general response, we conducted such experiments and found that 1) there is a small benefit to regularization but 2) our main trends hold, such that tuning to the right complexity level supports the greatest finetuning accuracy, therefore resulting in our method still outperforming the (stronger) regularized baseline.\n\nWe discuss an additional method for autonomously selecting the right encoder based on the number of prototypes in our overall response. This method, too, fails for both theoretical and empirical reasons.\n\nOur findings, with strengthened baselines and methods for choosing models based on the number of prototypes, establish the importance of our human-in-the-loop framework rather than autonomously selecting models. Ultimately, given the positive results from our human-in-the-loop framework, and our negative results from various autonomous selection baselines, we hope that the reviewer will reconsider their score.\n"}]

Author rebuttal:
We thank all reviewers for their comments. We have replied to specific questions in individual responses. Here, we briefly highlight results that address some common themes:

# Regularization baselines: 

Reviewers 99tN and TMXo asked for stronger finetuning baselines, with regularized models trained with a validation set. We thank the reviewers for this excellent suggestion.

We implemented this approach, using both L2 and L1 regularization on model weights, sweeping through regularization weights from 0.00001 to 0.1 at powers of 10 (and the default of 0.0) and selected the best weight when assessed on a validation set of 1 example per class. Results from such experiments (using L2 regularization), are included in the attached pdf, for $\beta$-VAE, VQ-VIB$\_\mathcal{C}$, and VQ-VIB$_\mathcal{N}$, in all domains (using $n = 1$ for both VQ-VIB methods, as it performed best). Results were nearly identical for L1 regularization.

Overall, regularization improved finetuning performance of all models by a small amount, but the main trends of our experiments still held. That is, for small $k$, we observed that finetuning performance first increased and then decreased as VQ-VIB$_\mathcal{C}$ models used more abstract representations, confirming the value of our method.

The greatest benefit from using regularized models seemed to be in the lowest MSE range. With a “perfect” regularization method, we might expect that performance would plateau at low MSE rather than worsen. Instead, using L2 regularization, we found that performance decreased slightly as MSE decreased, but performance decreased less than without regularization. This aligns with the intuition that regularization can help for the most complex representations, but consistently finding the right regularization method remains challenging.

Critically, these findings corroborate the main idea of our paper: that compressing representations remains an important mechanism for optimal finetuning performance in low-data regimes, and that end users are adept at selecting the optimal compressed representations adaptively at test time. Upon acceptance, we would update all figures in the main paper to use the regularized baselines.

# Real-world use cases

Reviewers 99tN and TMXo asked for a real-world use case of our method. We acknowledge in our paper that our method confers the largest benefit in low-data visual regimes (we do not claim to solve all transfer learning scenarios), which are well motivated in many real-world tasks such as visuomotor robot learning and end user home personalization, where the ability to collect data personalized to users remains limited [1]. Consider the scenario where a visuomotor policy has been trained to help a user sort clothing in their home or identify mugs for making coffee [2]. In this problem setting, we wish to be able to identify specific types of clothing or mugs that the user uniquely possesses in their home, and adapt our policy to these personalized preferences without the need for the user to provide large-scale training data from their home. Using our approach, we could train a robot to identify clothing for sorting (as in our FashionMNIST example) using only 10 examples per class and achieve roughly 93% accuracy. Moreover, if the user decides to give the robot to their family member, we could adaptively perform the same finetuning process to the new user’s preferences! This type of flexible test-time adaptation would be extremely desirable in these scenarios.

# Autonomously selecting the right representation via prototypes

Reviewers 99tN and TMXo suggested the hypothesis that the optimal encoder is the one that uses approximately the same number of prototypes as finetuning labels. Such an approach does not succeed.

Consider the following theoretical example in FashionMNIST. Using the same pre-trained encoders as in our main paper, we seek to finetune models based on a binary classification task: handbags with straps and handbags without straps. Given that there are two finetuning labels, one might consider encoders that use two prototypes. However, our FashionMNIST encoders that only use two prototypes typically use one prototype to represent shoes and one prototype to represent everything else. Such encoders would fail catastrophically at fine-grained discrimination between types of handbags.

Empirical results corroborate the intuition from this theoretical example.

In the CIFAR100 20-way finetuning task (which contains odd groupings, such as two distinct categories for vehicles) in the main paper, peak performance was at the lowest MSE values, which corresponded to using roughly 100 prototypes. Conversely, using 20 prototypes resulted in worse performance.

Furthermore, we conducted an additional FashionMNIST experiment where models were finetuned on a new 3-way classification task. Groupings were intentionally non-meaningful (Group 1: Tshirt, Trouser, Sandal; Group 2: Pullover, Dress, Sneaker; Group 3: Coat, Shirt, Bag, Boot). In this domain, peak performance was achieved for models using far more than just 3 prototypes. Once again, this establishes that if the fine-grained labels do not align with how VQ-VIB$_\mathcal{C}$ models learn compressed representations, models benefit from having more prototypes (and less compressed representations).

These experiments establish that autonomously selecting the optimal encoder for a finetuning task remains challenging. Conversely, humans are adept at choosing the right model. We conducted an additional online user study using the novel 3-way grouping for FashionMNIST and found that participants were able to correctly identify the optimal abstraction level for this task as well. This reinforces the value of our human-in-the-loop framework.

[1] "Robot learning from randomized simulations: A review." Muratore et al.

[2] "Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation." Peng et al.

Meta-review: This paper shows a method for generating reps at different levels of complexity, experiments showing (unfortunately slightly lukewarm) results on the effect of controlling this, and a human study (whose scalability / strength seems to have been a point of slight contention). However, all reviewers agree that it's an interesting and valuable question, that the experiments are thorough, and ways to involve humans in a ML problem is understudied and interesting. All reviewers converged on an accept decision after a lively rebuttal phase. 

In the interest of supporting creative approaches trying new methods on important questions (even if the results don't demonstrate a stellar improvement), I'd recommend it be discussed at NeruIPS / acceptance as a poster.
Decision: Accept (poster)