Paper: Concept Bottleneck Generative Models (ICLR-2024)

Reviewer comment (Experiments):
The concept bottleneck layer requires ground truth data - this is fine for small-ish scale data, but we can't hope to obtain this kind of data for large datasets. This puts some doubt to the usefullness of the model.

Author response:
**Ground-truth Annotations**: Indeed, as currently formulated, the CB-generative models require annotations for the entire training set. However, this requirement can be easily relaxed. To directly address your concern, we performed a new set of experiments where we varied the percentage of concept annotation available for CBGMs. We added this experiment in Appendix B.3.3. We find that with about 20% of annotations, we can directly match steerability metrics of a fully annotated training set. This result is not surprising; in the interpretability literature, concept bottleneck models for classification papers (Kim et al. (2018) and Yuksekgonul et al. (2022)) have shown that it is possible to learn a high-performing concept classifier with as few as 200 samples. This experiment suggests that those results translate to the generative model setting as well.

Reviewer comment (Theory):
The model requires a *separate network* for each concept - this is fine as long as there is a small number of concepts but I doubt this is a scalable approach going forward.

Author response:
**Scaling Concept Layer**: Increasing the number of concepts does increase the model parameters as in any concept bottleneck model. However we show in the paper, we can scale the number of concepts from 8 to 40 on the Celeb-A 64 by 64 dataset, and on the LAION dataset, we used 155 concepts. To address your concern we scaled LAION to 750 concepts which we were able to train using only 4 V100 GPUs. To clarify how the CB layer affects the number of parameters of a model we now discuss the total number of parameters that the CB layer adds. A standard CB layer consists of: $2k$ concept networks ($k$ for the positive concept states, and $k$ for the negative concept states), $k$ probability networks, and 1 unknown concept network. Similar to Espinosa et. al. (2022), we parameterize each concept network as a fully connected module; hence, each concept network has $am$ parameters where $a$ is the input dimension to the concept layer for a given instance, and $m$, is the concept embedding size. Consequently, for $2k$ networks, we have $2kam$ parameters for $2k$ concepts and $am$ parameters for the unknown concept network giving: $(2k+1)am$ parameters. Each probability network has $2m$ parameters leading to $2mk$ parameters for $k$ concept vectors, which in total means the CB layer adds: $(2k+1)am + 2km$ parameters to a given architecture if the networks are instantiated as fully connected layers. In practice, we share a single probability network across all concepts, which reduces the total number of parameters required as done by Espinosa et. al. (2022). For example, training CB diffusion models with 155 concepts required 120 million parameters scaling to 750 concepts increased the number of parameters to 250 million. Current hardware allows us to scale to multiple billion parameter models.

Reviewer comment (Theory):
Because the model is supervised it will only learn about concepts provided through the labels - there could be an argument that concepts should be learned from the data unsupervised as we don't necessarily always know what are the underlying factors.

Author response:
**Unsupervised Concepts:** We have two categories of concepts: known and unknown concepts. First, for the concepts that we want to be able to interpret and control, we need labels. **However, we allow for unknown concepts that are learned completely unsupervised except with the constraint that these concepts be orthogonal to the known concepts.** Supervision for the known concepts is important. As we referenced in the paper, Locatello et. al. (2019) show that one cannot learn, unsupervised, representations that are also disentangled. In addition, there is no guarantee that these representations should be human-understandable. This means that simply allowing of an entirely unsupervised representation will limit both interpretability and the ability to control the model. In fact, this challenge is why approaches like GANSpace resort to a PCA of the linear representations to ‘search’ for semantically meaningful and disentangled features. Consequently, our approach allows for both known and unsupervised concepts.

Reviewer comment (Evaluation):
Evaluation is a bit weak in the paper - Table 1 which is arguably the main quantitative result of the paper is not well explained. If I understand correctly these are the prediced concept presence probablilites after the model has been constrained to include them. This is problematic because a) we don't know the accuracy of presence classifier and b) measuring only the probablitiy doesn't tell us if the output actually contains the desired concept (i.e - there are no visualizations).

Author response:
**Evaluation and Table 1**: Thank you for pointing out the experimental evaluation, we have now taken steps to address your concerns.
    
    **Evaluation** (a) First, we want to clarify the proposed steerability metric. We start with a high-accuracy classifier trained on real data; we forgot to mention that we ensure that the minimum accuracy of any concept classifier is at least 98 % on a held-out test set from the real data (we have updated the paper to include this). We sample noise latent vectors and then use it to generate images from the model; for each concept we pass the generated image to the concept classifier; we save the latent vector if the classifier predicts the concept to be absent (i.e., the probability of the concept present is less than 0.5) we continue this process until we have 1K samples. For each conditional generation approach (including CB layer),  we intervene on a concept and generate new images. We then pass the newly generated images to the classifier again and measure the fraction of inputs for which the indicates greater than 0.5 probability of concept presence, which we report in Table 1.
    
    **Evaluation** (b) Per your request,  we added more visualizations to the appendix  Figure 11 to show steerability.   
    
    **Table 1 results**: The classifiers were trained on real data, but the steerability metric is on synthetically generated data, which might cause an adversarial effect or the classifier might consider the samples out-of-distribution causing the accuracy to be low. However, since we use the same classifier across all of the models and steerability approaches, we believe that the metric is fair i.e. the model ranking is reliable. The steerability test and metric that we measured is designed to be a difficult test that aligns with what people typically seek in practice. The reason we focused on quantitative results rather than visualization is because we found that previous work on steerability often showed only cherry-picked examples; we believe that quantitative results are better for ranking different methods.

Reviewer comment (Experiments):
Are all concepts equal? are there ones that affect the output of the model more than others?

Author response:
**Are concepts equal**: In the training phase, we did not weight concepts differently (we didn't penalize one concept more than another i.e. $\alpha$ in the loss function equation 4 was not adjusted for each concept separately), so all concepts are equal; however, this is a design choice. During inference/test-time, when an output is generated, the weight of each concept is determined by the output of the probability vector, so some concepts affect the model more than others. In Figure 4, we show a bar chart that indicates how much each concept affects the output of the generative model. For example, in the CB-DDPM trained on Color-mnist, we see that for the generated red digit, it is indeed the red color concept that has the most effect on the model output followed by the digit concept as we would expect. These concept rankings can be used to interpret the output of the model.

Reviewer comment (Experiments):
Lacking baseline comparison with SOTA model. Without concept bottleneck, existing work can also control concept, like text-image diffusion, GANSpace, StyleGAN, and others.

Author response:
- **Comparison to SOTA baselines**: As we stated in the general comment, our goal is to use an ***intrinsically interpretable component of a generative model to both steer, interpret, and debug the model***. As it stands, none of the baselines (stable diffusion, GANSpace, etc) accomplishes these goals. We now discuss each baseline in turn.

    - **Text-to-image models (e.g. stable diffusion)**: These models allow for fine-grained control with text inputs. However, given a trained text-to-image model, it is still unclear what features the model relies on to generate its output. While these models allow for text-based control, they do not provide any insight into the models; hence, they are difficult to debug. As an example, in recent work, Tong and Jones et. al. (2023) show that text-to-image models can easily ignore critical words in an input prompt. For example, a Stable Diffusion XL model asked to generate: “a woman proposing to a man” instead produces an image of a man proposing to a woman. As we show in Figure 18, a concept bottleneck model inserted in a diffusion model can help identify the features the model bases its generation on, and hence can immediately point to how to address deficiencies with the model.

    - **GANSpace**: As we stated, our goal is not to do post-hoc interpretation but to insert an intrinsically interpretable layer into the generative model. GANSpace uses PCA to identify the key directions/features in the representations of a GAN. The approach can then vary the representations along these directions to vary the output of the model. Our approach differs from theirs in several ways: 1) since we insert an interpretable layer into the model, we do not need to search for the representations in order to control them. 2) It is possible that the model representations might not encode for a feature that one might want to control. For example, in the Celeb-A setting, we might be interested in controlling the presence of Glasses. However, it is possible that none of the most important PCA directions correspond to Glasses. In contrast, our proposal directly encodes the `glasses' concept as part of the model’s latent representation, so we can immediately identify whether the model has been able to learn that concept.

    - **StyleGAN:** We agree that the StyleGAN model has been used to demonstrate impressive steering/control capabilities. Controlling styleGAN typically can be done in different ways, either by conditioning where the concepts are concatenated to the latent vector, classifier guidance, or model editing methods such as GANSpace as discussed previously. Such methods enable steerability (although we found that for steerability CB layer outperforms conditioning and classifier guided methods as shown in Table 1), but it does not allow for interpretability (i.e we do not understand which concepts the model relied on during generation) and debuggability (i.e we can not tell if the model actually learned a certain concept during training). An alternative approach is adding a CB layer to styleGAN, by doing this, we can explicitly learn the desired concept, we eliminate the need to search for weights correlated with concepts as we know ahead of time which parameters will encode for a particular concept, the concept layer allows us to debug and interpret the generative model during and after training. During training, we can identify which concepts the model is currently unable to encode, and post-training, we can use the output of the concept layer to identify the key features responsible for the model’s output. To address your concern, we added a section in the appendix explaining how one can create CB-StyleGAN; please find the full experiment and results in Appendix B.1.1 (in the updated paper).

Reviewer comment (Experiments):
Similarly, comparison to stable diffusion and GANSpace should be shown.

Author response:
- **Text-to-image models (e.g. stable diffusion)**: These models allow for fine-grained control with text inputs. However, given a trained text-to-image model, it is still unclear what features the model relies on to generate its output. While these models allow for text-based control, they do not provide any insight into the models; hence, they are difficult to debug. As an example, in recent work, Tong and Jones et. al. (2023) show that text-to-image models can easily ignore critical words in an input prompt. For example, a Stable Diffusion XL model asked to generate: “a woman proposing to a man” instead produces an image of a man proposing to a woman. As we show in Figure 18, a concept bottleneck model inserted in a diffusion model can help identify the features the model bases its generation on, and hence can immediately point to how to address deficiencies with the model.

    - **GANSpace**: As we stated, our goal is not to do post-hoc interpretation but to insert an intrinsically interpretable layer into the generative model. GANSpace uses PCA to identify the key directions/features in the representations of a GAN. The approach can then vary the representations along these directions to vary the output of the model. Our approach differs from theirs in several ways: 1) since we insert an interpretable layer into the model, we do not need to search for the representations in order to control them. 2) It is possible that the model representations might not encode for a feature that one might want to control. For example, in the Celeb-A setting, we might be interested in controlling the presence of Glasses. However, it is possible that none of the most important PCA directions correspond to Glasses. In contrast, our proposal directly encodes the `glasses' concept as part of the model’s latent representation, so we can immediately identify whether the model has been able to learn that concept.

Reviewer comment (Evaluation):
How does the method tackle unseen concepts? Can this be extended to open-world concepts?

Author response:
- **Unseen Concepts**: As we discuss in the paper, we partition concepts into two categories: known (pre-determined human understandable ones), and unknown concepts. The unknown concepts indeed directly encode for unseen concepts for which the model designer did not specify ahead of time in the known set. Your suggestion about open world concepts is interesting, we did not consider it in our initial formulation. However, we can easily adapt the current formulation to allow for a dynamic set of concepts. We leave an extension to open-world concepts for future work.

Reviewer comment (Reproducibility):
How long does diffusion training take? Do you train all generative models from scratch?

Author response:
- **Diffusion Model**: Training diffusion on LAION required 240 V100 GPU hours, training diffusion on Celeb-A required 112 V100 GPU hours. We have updated the manuscript to include the compute time.

- **Training from scratch:** Yes all models are trained from scratch.

Reviewer comment (Theory):
The paper claims at Abstract and Introduction that the proposed method is model agnostic, but I am not certain on that. For example, it seems to me that applying the CB layer to other types of generative models can be non-trivial, e.g., to normalizing flows and invertible models.

Author response:
**Abstract, Introduction, and Normalization Flows**: Thank you for raising this important point. Indeed, we tested three types of generative models in the draft. However, we found your suggestion of normalizing flows and invertible models intriguing, so we tested our proposed approach on a Normalizing flow model trained on Color MNIST, we added this experiment in Section B.1.2 of the Appendix. Our CB layer proposal translates, as is, to such models as well. We show examples generated from the model in Section B.1.2 of the Appendix. We expect future work to test our findings more exhaustively and extend them to other families of generative models. We take your feedback to mean that we should explicitly specify the class of models we focus on, which we have done in the updated manuscript.

Reviewer comment (Experiments):
Although the paper presents some scalability experiments, the other parts of the experiments can be seen as somewhat limited in their scale. For example, the paper only explores simple GAN architectures not covering modern ones such as StyleGAN-2. I am also wondering why more recent methods such as GAN inversion could not be a baseline. If these point can be addressed, it will be also beneficial to support the model-agnostic aspect of the method.

Author response:
**Not covering modern ones such as StyleGAN2:** As per your request, we added a CB layer to StyleGAN2 in Appendix B.1.1 (in the updated paper). For StyleGAN, the CB layer was added after the mapping network in the generator, an encoder was also added before the generator to ensure that the generated image matches the concepts constrained in the loss, the exact architecture is shown in appendix B.1.1 Figure 7 (a). We trained StyleGAN2 on Celeb-A 64x64 and CUB 128x128. We added images generated by CB-styleGAN2 for both datasets in the appendix Figure 7 (b) and reported FIDs as well. We find that using more complicated architectures improves the quality of the generated image (as expected) and inserting a CB layer does not degrade the generation quality.

**GAN inversion baseline**: As we mentioned in the general comment, while GAN inversion approaches enable a model to be steered, these approaches do not address the goals that we set forth to tackle in this work. To re-iterate: **we seek an intrinsically interpretable component of a generative model that allows us to steer and interpret the output of the generative model simultaneously.** Meanwhile, GAN inversion approaches do not provide any form of understanding or insight into which features the model bases its generation on. In addition, we require that the approach be intrinsic to the model and not a post-hoc approach. Critically, a GAN inversion approach cannot indicate the key features that a model relies on for its output. Similarly, to the best of our knowledge, most GAN inversion techniques do not explicitly insert an interpretable component into the model. We think the GAN inversion approaches address an important challenge; however, our goals are orthogonal to these challenges.

Reviewer comment (Evaluation):
More qualitative comparisons could be added and highlighted for the steerability experiments, given that the quantitative results are dependent on the pre-trained concept classifiers.

Author response:
**Qualitative Results:**  As you requested, we updated the Appendix to include several additional qualitative steerability experiments from StyleGAN2; see  Figure 11 in the appendix.

Reviewer comment (Writing):
(minor) The definition of concept orthogonality loss, Eq (5), is written in somewhat confusing manner - e.g., the index j is not used in the definition.

Author response:
**Concept Orthogonality Loss:** Thank you for noting this issue, $j$ is the sample in the mini-batch, we have updated the main text under Eq 5 to clarify this.

Reviewer comment (Reproducibility):
The paper mentions Platt and Barr (1987) regarding hyper-parameter optimization. Does it mean that the paper actually applied that method for tuning in the experiments? If so, it may be good to provide an overview on the method in the paper as well.

Author response:
**Optimization**: Thank you for raising this important point. On the VAE model we compared Platt and Barr to a standard grid search/sweep over a hyper-parameter set. We didn’t observe a substantial improvement in model FID scores for the subset of models trained. We used an open-source Pytorch implementation of the Platt and Barr approach, which does not require any modification for the setting considered here. The approach minimizes a loss function that is subject to bound constraints with arbitrary parameters that weight the importance of these functions. We added this discussion in Section B.2 in the appendix; we also referred to the open-source implementation used.

Reviewer comment (Experiments):
It seems making sure the minimal concept orthogonality loss is crucial for the soundness of the method, otherwise there can be an overlap between the given concepts and the "unknown" concept. One immediate ablation one can try is to check whether the loss is minimized is to strictly project the learned unknown context vector to be orthogonal to other vectors and see if there is degradation in performance. Or I think this kind of procedure can be even incorporated into the training phase to guarantee the minimal concept orthogonality loss. Any discussion regarding the actual orthogonality of the learned unknown context vector would be helpful for the readers.

Author response:
**Orthogonality:** We agree with your assessment that it is crucial for the unknown concept embedding to be orthogonal to known concept embeddings. Removing orthogonality between the known and unknown concepts causes steerability degradation, as shown in the ablation Table 3. Our current algorithm encourages orthogonality by adding the loss in equation 5 but does not enforce it. We tried forcing orthogonality during training (as you suggested) by projecting the concepts embedding to be orthogonal to non-concepts embedding during training using the Gram-Schmidt Process [1]; we found that this makes training unstable and harms generation. We note that there are other schemes that can achieve independence, including KL minimization, mutual information minimization, etc. We leave exploring different orthogonality schemes as future work.

[1] Schmidt E.Zur Theorie der linearen und nichtlinearen Integralgleichungen I. Teil: Entwicklung willkürlicher Funktionen nach Systemen vorgeschriebener. Mathematische Annalen 1907; 63: 433–476.