Concept Bottleneck Language Models For Protein Design
Authors: Aya Abdelsalam Ismail
Tuomas Oikarinen
Amy Wang
Julius Adebayo
Samuel Don Stanton
Hector Corrada Bravo
Kyunghyun Cho
Nathan C. Frey
Venue: ICLR 2025 Poster (2025)
Keywords: LLMs
protein design
concept bottleneck
Primary area: interpretability and explainable AI
TL;DR: We insert a concept bottleneck layer into language models.

Abstract: We introduce Concept Bottleneck Protein Language Models (CB-pLM), a generative masked language model with a layer where each neuron corresponds to an interpretable concept. Our architecture offers three key benefits: i) Control: We can intervene on concept values to precisely control the properties of generated proteins, achieving a 3$\times$ larger change in desired concept values compared to baselines. ii) Interpretability: A linear mapping between concept values and predicted tokens allows transparent analysis of the model's decision-making process. iii) Debugging: This transparency facilitates easy debugging of trained models. Our models achieve pre-training perplexity and downstream task performance comparable to traditional masked protein language models, demonstrating that interpretability does not compromise performance. While adaptable to any language model, we focus on masked protein language models due to their importance in drug discovery and the ability to validate our model's capabilities through real-world experiments and expert knowledge. We scale our CB-pLM from 24 million to 3 billion parameters, making them the largest Concept Bottleneck Models trained and the first capable of generative language modeling.

Published as a conference paper at ICLR 2025
CONCEPT BOTTLENECK LANGUAGE MODELS FOR PROTEIN DESIGN
Aya Abdelsalam Ismail 1.2 Tuomas Oikarinen Amy Wang!2 Julius Adebayo
Samuel Stanton 1.2
Héctor Corrada Bravo'
Kyunghyun Cho 1,2,,66 Nathan C. Frey
I Genentech Prescient Design
University of California San Diego
*Guide Labs
'Department of Computer Science, New York University
"Center for Data Science, New York University
ABSTRACT
We introduce Concept Bottleneck Protein Language Models (CB-pLM), gen- erative masked language model with a layer where each neuron corresponds interpretable concept. Our architecture offers three key benefits: i) Control: We can intervene on concept values to precisely control the properties of gener- ated proteins, achieving 3x larger change in desired concept values compared to baselines. ii) Interpretability; A linear mapping between concept values and predicted tokens allows transparent analysis of the model's decision- making pro- cess. iii) Debugging: This transparency facilitates easy debugging of trained mod- els. Our models achieve pre-training perplexity and downstream task performance comparable to traditional masked protein language models, demonstrating that in- terpretability does not compromise performance. While adaptable to any language model, we focus on masked protein language models due to their importance in drug discovery and the ability to validate our model's capabilities through real- world experiments and expert knowledge. We scale our CB-pLM from 24 mil- lion to 3 billion parameters, making them the largest Concept Bottleneck Models trained and the first capable of generative language modeling.
INTRODUCTION
Protein Language Models (pL.ss) have emerged as prominent framework for protein repre- sentation learning, encapsulating millions of years of protein evolution (Hayes et al.] (024).
These models have demonstrated exceptional performance on intricate tasks such as predicting
protein structure and function (Lin et al. 2023 Chen et al. 2023: Elnaggar et al. 2022 Xu ct al.
2023). Furthermore, they have opened new frontiers in diverse and critical applications, including healthcare and drug discovery (Hie al. 2024 Hayes al.) (024). Despite their impressive capabilities, controlling, interpreting, and debugging pLMs remains challenging due to the opaque nature of transformers (Vaswani 2017).
Recently, some PLMs have supported conditional generation by adding various protein input repre- sentations. functions, and properties as tags (which can be viewed concepts) and predicting them during training (Madani et al. 2020 Hayes et al. 2024: Ruffolo et al., (022)). However, these mod- els can selectively rely on or ignore different aspects of the input. Even if these models offer some form of control, there is no effective way to interpret or debug them. Domain experts might want to know which concept the model depends on the most to generate the amino acid "E", which amino acids are correlated with enzymatic activity, answer counterfactuals like if we want to decrease hydrophobicity, which amino acid should we replace "E" with, which is not possible with existing models. Debugging language models is notoriously difficult, and it is often unclear what model has learned or when it will fail. These limitations lead to distrust among domain experts, as humans tend to not trust what they do not understand.
Our work relies on the fact that neural networks do not have to be black boxes; since we construct and train these networks, advantage of building models from scratch, with control, interpretability, and debugging in mind from the outset (Frey al. 2024 Adebayo et al. 2023). To achieve explainability and alignment, it
design and align them according to needs. This is unique

---

Published as a conference paper at ICLR 2025
Input
increase Helix fraction
Turn Structure fraction
Structure fraction
Figure 1: Changing protein structure using Concept Bottleneck Protein Language Model.
is essential to use intermediate representations based on well-understood concepts. To this end, we present Concept Bottleneck Language Models, novel architecture for large language models that relies on human-understanddble concepts via a concept bottleneck layer and auxiliary loss terms. This architecture enables concept-controllable generation and explanations, compelling the model to align with human-undn uumman--undertandable concepts and providing the means to debug the model and understand its limitations. Our approach builds on the foundation of concept bottleneck generative models (CBGMs) (Ismail et al. 2023).
In this paper, we focus on protein design, domain where control (Gruver et al. 2024 and in- terpretability (Adebayo et al., (023) are critically needed. Protein design benefits from a set of well-defined biophysical and bioinformatics properties that domain experts understand and wish to control. We introduce Concept Bottleneck Protein Lunguage Model (CB-pLM). We summarize our contributions as follows:
We propose novel architecture, training, and intervention schemes for large language models that incorporate human-undeersanddabe concepts. This results in a controllable, interpretable, and debuggable LLM by design. We focus on masked protein language models, given their critical role in high-stakes applications such as drug development, where the need for control and interpretability is paramount. We train masked protein language models with 24M, 150M, 650M, and 3B parameters using the proposed CB-pLM architecture with over 700 concepts. We do not observe a significant increase in perplexity compared to unconstrained models.
We compare CB-pLM various conditional pL.Ms of the same size, all trained the same dataset, across more than 80 single and multi-property control in silico experiments. Our model demonstrates 3x better control in terms of change in concept magnitude and 16% improvement in control accuracy compared to other architectures. Additionally, we benchmark CB-pLM against state-of-the (SOTA) protein design models explicitly trained to optimize single concept. Remarkably, our general-purpose CB-pLM, trained to learn over 700 concepts, delivers results comparable SOTA models, while maintaining the naturalness of the protein.
We demonstrate how CB-pLM allows domain experts assess what the model has learned, i.e., interpretability, and identify and correct any unwanted behavior in the model, i.c., debuggability by simply visualizing the weights from the model's decoder layer.
2 CONCEPT BOTTLENECK LANGUAGE MODEL.
We aim to create language model that we can easily control, interpret, and debug. To achieve this, modify the standard masked language model architecture by adding concept bottleneck layer to explicitly incorporate human-understanddbbl concepts into the language model-termed a concept- bottleneck (masked) language model (CB-LM). The concept layer can then be used control, interpret, and debug the model. This section discusses the proposed language model architecture, loss functions, training and intervention procedure.
Setup Each sequence x is represented with set of tokens, i.e., x [ro, Fz). where is the maximum sequence length. Each sequence starts with the CLS token i.c. Io = [CLS] followed by sequence of amino acids ending with [ECS] well padding tokens for sequences below maximum length. Additionally, each sequence annotated with a set of global predefined human-unde understnaadtt concepts [cu...., Cs] where k is the number of concepts. Taken together, this results a training dataset consisting of token-concept pairs {(x, c) where n is the number of samples.
2

---

Published as a conference paper at ICLR 2025
[CLSJAKETPITEOSS
2.1 ARCHITECTURE
Linear Doooder
Suquaree
Concept bottleneck module
The overall architecture is shown in Figure We start with a standard BERT-based masked language model (ML.) (Devlin
2
h Lnknoww
Known
Orthogonalty Network
et al. 2018), which is commonly used in many protein language models (Rives al. 2019 Lin el al. 2022; Hayes et al. 2024 Frey et al., 2024). A mask token [MASK]
Goncept
bg
bottlenook
Toleen Nödden ipprsseraatinn
Concept Network
Tranformo Encoder
is randomly applied to a subset of tokens in the sequence x resulting in a masked version of the sequence. After masking, x is passed into a multi-head self-attention transformer encoder, and the output of the encoder is denoted as H = TransformerFncooer((:)))
[OLSSAKKAAKKATTEESS)
Messad
Figure 2: Concept Bottleneck Language Model
where H = hoo ..... hy]- We introduce three main architectural changes in CB-LM: (a) concept bottleneck module (CB-moduee (b) orthogonality network, and (c) linear decoder.
Concept bottleneck module After 2 passes through the encoder, the hidden representation of the classification [CLS] token, ho. which serves as an aggregate representation of the sequence, is passed the CB-module shown in Figure D The concept network (Koh et al. 2020) learns function g that maps the input sequence into concept predictions è = g (ho)). Each concept is also represented with learnable embedding e. The final representation of concept i is 2; & X eq. The output of the CB-module exactly which concept each neuron in the embedding corresponds to.
[21, 22. ....... 2x), is referred to as the known embedding, as we know
Orthogonality network This network aims to produce a token representation devoid of any concept information, which we refer to as the unknown token embedding h. This is achieved by enforcing an orthogonality loss, as described in Section 2.2 between its output and the known concepts.
Linear decoder The known and unknown embedding concatenated and passed into a single lin- (ettmaxxeeee:::.... car layer followed by softmax function, i.e., åt = softmax (Wide. [zz hv])). where TM represents a masked token [MASK]. We use a linear layer as the decoder for interpretability and debugging purposes. A linear layer allows us calculate the contribution of each concept to the final prediction by simply multiplying the concept activation value with its weights. This gives us an interpretable answer to questions such as Which concept does the model depend on the most to generate the amino acid"E"? or counterfactual questions like if we intervened the concept strength of hydrophobic- ity, which amino acid would the decoder output?. Debugging neural networks notoriously diffi cult; however, setup simplifies this process. Since we have prior knowledge of how challenging a concept is to learn, we can identify and fix bugs by inspecting the concept weights.
Examining the weight matrix Wdee can (a) help understand the correlation between different concepts, aiding in the detection of any spurious correlations learned by the model, (b) elucidate the correlation between concepts and tokens, answering questions like which amino acids are correlated with enzymatic activity?, and (c) identify concepts that we can control and those we cannot, i.e., if the weight for a concept small, then the model will not be able to control that particular concept.
2.2 Loss FUNCTIONS AND TRAINING
Loss functions Following CBGMs (smail et al., 2023). concept bottleneck language models are trained with three losses: standard masked language modeling loss LVVM, concept loss LConcept and an orthogonality loss Loth- The final loss given by Lrotal PMLM + aLConnopt + BLothh where and are hyperparameters.
Generative masked language modeling loss: LMLM = -B..m m Eigm log P(x; mxm) where m is the set of positions of the randomly masked tokens, and the model is trained to predict the identity of the masked tokens.
Concept loss: Given the general-purye purpose nature of large language models, they expected to handle thousands of concepts with categorical or real values. Often, samples lack values for most concepts. To address this, normalize all real-valued concepts [0, 1] and apply mean square
3

---

Published as a conference paper at ICLR 2025
error loss the entire concept embedding: Lonceppt E
i= (c: 2,)2 Missing values
replaced with default values, and their effect is removed from the loss function by considering only non mon-missing concepts, achieved by masking the errors before backpropagation.
Orthogonality loss: Similar to Ismail et al.] (2023), we encourage the known and unknown em- beddings to encode different information by applying an orthogonality constraint (Ranasinghe et al.) 202T), minimizing the cosine similarity between them. This loss encourages, but does not guarantee, disentanglement.
Training CBM: We used independent training for the CB-layer (Koh et al. 2020) to reduce con- cept Icakage (Mahinpei et al. 2021). Token regularization: Feature attribution is needed for coordi- nate selection during intervention, as discussed in section 2.3 To ensure attribution is reliable, we followed Adebayy et al. (022) 's recommendation and added Gaussian noise to the token embedding during training. Additional details are available in Appendix A.2
2.3 COORDINATE SELECTION AND INTERVENTIONS
CB-LM enables concept-level test-time interventions, allowing for global sequence control while maintaining token-level predictions. During training, tokens randomly masked and predicted, which is not ideal for interventions. Instead, we wish to identify tokens that most significantly
affect concept and intervene on those (Gruver al. 2024. Adebayo et al. 2023). For example, to increase the Aromaticity of a protein, we target amino acids that decrease Aromaticity. This process called coordinate selection.
A straightforward method to measure a token's 's effect on a concept is occlusion, where each token replaced with reference value (e.g., [MASK] ) one at a time to observe changes the predicted concept. While effective, this method is slow for large-scale applications. Instead, we use gradient- based approximation of occlusion, requiring only a single forward pass. We propose using a single step of integrated gradients (Sundararajan approximation occlusion. Since our inputs are discrete, we calculate gradients with respect to the learned token embeddings and sum over the embedding dimension. Let T Roxd be the learned token embeddings, where e is the vocabulary size and is the transformer hidden dimension. Then, let fr (2:) (T Tais.-.. - T2,] The attribution A for token of input on concept is:
al. 2017), which can be seen as first-order Taylor
A(x,i,t) = (Ta, ( ) VT,, (()())),
where T [MASK]] is the embedding for the mask token and (x)) is the concept value predicted by our CB-Layer for concept
Informally, A(x, i, f) can be seen as quantifying how the choice of token I; influences in context of the other tokens r- If in the training dataset H, co-occurs with below average values of e; when combined with x-1, then we expect A(x, i, 1) to be negative. Similarly if co-occurs with above average values of e in the context x-t then we expect i,x,i,)) to be positive. Hence want to intervene increase the concept value by A in descending order, and if we want to decrease the concept value we greedily choose the bottom-k coordinates. In Appendix [.I] we show that this attribution method closely matches the performance of occlusion and improves control effectiveness by 2x over random masking.
greedily choose the top-k coordinates sorted
PROTEIN REPRESENTATION LEARNING AND DESIGN WITH CONDITIONAL LANGUAGE MODELS
One can view concept bottleneck language model as a special type of conditional language model. Current conditional large protein language models involve appending conditioning [Tags] to the sequence along with the input (Madani et al., 2020; (hhai et al., 2021 Hayes et al. 2024) and then using these tags steer the output. To compare the effect of different architectures on protein representation learning, and design, we train three types of conditional models on the same training dataset (sequences and concepts) with comparable sizes.
Conditional Protein Language Model (C-pLM): These are traditional conditional models, the input to the model sequences and concepts (which can be viewed as [Taqs]); the model is trained to minimize the generative masked language modeling loss, i.e., Liotal = LMLM-
4

---

Published as a conference paper at ICLR 2025
Conditional Protein Language Model with Classifier (CC+pLM): This class of models has an extra regression head used to predict the concepts (Hayes et al. 2024). The input to the model are sequences and concepts; the model is trained to minimize the generative masked language modeling loss and concept loss, i.e., Leotal = LMLM + OcLConcept-
Concept Bottleneck Protein Language Model (CB-pLLM (Ours): family of protein language models with the architecture shown in Figure 2 Lttal LMLM + CoCoonepp + oohh-
CB-pLM is designed to be controllable, interpretable, and debuggable. Other conditional language models offer control through [Tags]; however, the model can ignore these tags during genera- tion and fail to learn the underlying biophysical concepts for proteins. CB-pLM learns concepts as an intermediate step and includes a loss to encourage disentanglement between concepts and un- known embeddings. Since these concepts are inside the model, this forces the model to learn and use these concepts for predictions, leading to better control (Section 4 Furthermore, CB-pLM provides mechanisms for interpretation and debugging (Section 5), which are not available in other conditional models.
Training Data We combined sequences from UniRef50 (Suzek et al. 2015 and SWISS-PROS (Bairoch & Apweiler, 2000), removing duplicates. Annotations such as protein clusters, organ- isms, taxons, biological processes, cellular components, and molecular functions cept annotations. Biopython (Cock et al. 2009) was used to extract biophysical and bioinformatics sequence-lever concepts. Overall, all models support 718 concepts; details are in Appendix [B.1]
used
Architecture and Training Following Frey et al. (2024), we optimized the architecture and training for efficiency. We removed biases in attention blocks and intermediate layers, increased the effective batch size, used gradient accumulation, and set the masking rate to 25%. We employed the Adam W, mixed precision training, and gradient clipping for stability. More details are in Appendix B.2
3.1 MODEL QUALITY
Model quality is evaluated using perplexity, which measures how well the model predicts token based on its sequence. Perplexity ranges from (indicating perfect model) the vo- cabulary size (33 for the pLMs vocabulary (Lin al.] p022)) with higher values indicating more random predictions. To assess perfor- mance. we used 10,000 randomly sampled antibodies from the publicly available Mason dataset et al. (02TT. [ables]shows the perplexity of different models on this dataset.
Models
Tags
Cond.
24M
150M
650M
3B
LBSTER
No.
No
8.20
2.97
ESM2
No.
No
4.84
3.41
3.01
C-pLM
Yes
Yes
4.57
2.62
CC-PLM
Yes
Yes
4.07
2.74
CB-PLM
No.
Yes
4.29
3.20
2.48
2.50
Table 1: Mason dataset perplexity.
LBSTER (Frey et al., (024) and ESM2 (Lin et al. 2022) are standard masked language models that do not require additional tags during inference and do not support conditioning. Conditioning baselines C-pLM and CC-pLM require ground-truth concepts as inputs during inference; they given Biopython concepts for sequences, while other concepts were set missing values. All models learned the training distribution well, with similar-sized models showing comparable perplexity. CB-pLM has perplexity comparable to conditional models, even though C-PLM and CC-pL.M received additional tags as input, while CB-pLM did not. Notably, CB-pI.M often achieves higher perplexity than state-of-the-art open-source protein-masked language models of similar size. These results are promising, as CB-pLM, despite its constraints, shows no significant performance drop compared to unconstrained models; in contrast, forcing the model to learn bio- bio-physical concepts might result in better model overall.
4 CONTROLLING AND STEERING
In this Section, we show how the CB-pLM enables fine- ane-grained control, which makes it useful for variety of protein design tasks. Sequence- based protein design requires the ability to modulate the amount of certain bio bio-phasical properties present in a model's output-a controllable generation problem. For example, might be interested in controlling single property, i.e., removing
5

---

Published as a conference paper at ICLR 2025
hydrophobic patch from a protein, multiple properties, i.e. increasing the expression and binding affinity of an antibody (Gruver et al. 20224 Tagasovska et al. 2024).
First, we demonstrate how to CB-pll CB-sLMs for single and multi- propper control. As example, we instantiate these property control abilities in a Siltuximab monoclonal antibody- case study, where we show Grand AVerage of Hydropathy (GRAVY) reduction via CB-pLMs, which alters the protein's s surface hydrophobicity.
4.1 COMPARING CONDITIONAL LANGUAGE MODEL ARCIITECTURES
Here assess the effectiveness of the CB-pLM, compared to the alternatives described in Section 3 for both single and multi property concept control.
Experimental Setup We examine 14 biophysical concepts computable from a protein's 's sequence (e.g., molecular weight, Gravy, Helix fraction, Turn structure fraction; see Appendix [..]. These concepts allow us to measure the success of positive (increase) and negative (decrease) concept interventions for each conditional generation approach. Using a validation dataset, we selected 10,000 sequences with the lowest and highest concept values for positive and negative interventions. To test generation control, we mask 5% of the input sequence (up to 25 amino acids) and intervene on the concept value. For the C-pLM model, masking is random. For CC-pLM and CB-pLM models, we use feature attribution to select tokens to mask (details in Appendix D.I.
Each masked test sample results in single generated sequence, and change in concept presence, between the original sequence and the generated sequence can be calculated used as estimate of the intervention effectiveness. We use likelihood from auxiliary autoregressive causal language model to measure the "naturalness of" or feasibility of the sequence (Bachas et al. 2022
4.1.1 SINGLE CONCEPT INTERVENTIONS
Average concept control accuracy when masking 5.0%
Average concept change when masking 5.0%
100
0,047
Accurasy
8
15.4
Z0.s
2,50
0.01
(a) Intervention accuracy.
(b) Average change in concept value after intervention.
Figure Single concept intervention accuracy and effectiveness averaged over all concepts.
Concept Control Accuracy After intervention, we calculate the concept control accuracy as the percentage of generated samples where the change in concept presence is the direction of the intervention (positive or negative) averaged over random set of examples (random accuracy would be around 50%). The aggregate average across all concepts is shown in Figure Ba| We find that CB-pLMs outperform other conditional models by 15%, reaching near perfect accuracy. We also measure the effectiveness of the intervention as the average change in the desired concept value direction; CB-pLM is also 3x higher than other conditional models shown in Figure 3bb
Change in Concept Distribution Beyond comparing average changes, we assess how much the distribution of values shifts upon intervention while maintaining protein naturalness. Ideally, the concept distribution should shift in the direction of the intervention while preserving the naturalness of the protein. Figure 4al illustrates the change in concept and naturalness distributions for subset of concepts when we intervene on the sequence once. We observe that, for both positive and negative interventions, the CB-pLM effectively shifts the concept distributions in the desired direction for all concepts while preserving the naturalness of the proteins. This performance superior to other variations conditional language models. Additionally, we examine the models' ability to iteratively shift concept's distribution, where the sequence output from the first intervention serves as the input for the second, and so on. Figure 4b shows concept distributions over three iterations for both positive and negative interventions. CB-pLM demonstrates the ability to iteratively shift the concept value with increasing effects, whereas other models fail to achieve this. We refer readers to the Appendix C.I.1 for additional results.

---

Published as a conference paper at ICLR 2025
Positive Intervention
Drecsin
Negative Intervention
Diraction
GaUM CC JUN ChiHN
(a) Shift in protein naturalness/concept value after interventions.
Positive Intervention
Nagative Intervention
Directon
CLU CC-UU CR-PLM
Briato
(b) Iterative shift in protein concept value after interventions.
Figure 4: Distribution shifts after interventions for different 24M models for subset of concepts.
4.1.2 MULTI CONCEPT INTERVENTIONS
Here, we take step further by demonstrating how CB-pLMMs can be used for proptip optimization-a challenging task in drug discovery (Stanton et al. 2022 Gruver et al. (002)).
Experimental Setup Here, we compare the effectiveness of the CB-pLMMs to other approaches at enabling control of multiple properties. Specifically, we focus the concept Grand AVerage of Hydropathy (GRAVY) and the electric charge at pH 7, both standard protein sequence properties correlated with functional properties, such solubility, viscosity, and aggregation (Bhandari al., 2020 Cock et al. 2009 Obrezanova et al. 2015). We sequentially intervene on one concept at a time. We test both positive and negative interventions after masking 5% of the sequence as described previously. Similarly, we measure concept control accuracy as the percentage of generated samples that successfully move toward the intervention direction for both concepts. We report the average accuracy for both positive and negative interventions, with random accuracy expected around 25%.
Result In Figure we observe that CB-pLM achieves a 94% control accuracy; outperforming the second- best conditional model by 34%. Controlling via the CB CB-pLM effectively shifts the distribu- tions towards the Pareto frontier, demonstrating its ability to control multiple properties, as shown in Figure (b). We refer readers to Appendix C.1.2 for additional discussion and results.
Auerage nutticonospt securacy when masking 0%
Mulii concept Positive interventions mesking 5%
Mutiiconnee concept Negative interventions masiing 5%
300
3.00
0.44
Model
Cell 241
-0.27
06 34M
-0.06
A.harge M..
aCwera Ph)
(4) Accunscy
(b) Cancopt Distribtton
Figure 5: Multi-concept interventions.
4.2
CASE STUDY: SILTUXIMAR PROTEIN DESIGN
Siltuximab is an FDA-approved monocional antibody used to treat disease the lymph nodes (Van Rhee et al.) 2010). Treatment involves intravenous infusion, which can cause patient discomfort and complications. Re- designing Siltuximab to have more favorable biophysical properties is promising research direction for exploring systems. Structural analysis of Siltuximab reveals the presence of a large hydrophobic patch red patch shown in Figure 6). which has been linked to antibody developability liabilities associated
Figure 6: Siltuximab surface hydrophobicity.
effective delivery

---

Published as a conference paper at ICLR 2025
with solubility and aggregation (Bhandari et al. 2020 Obrezanova et al. 2015). Here test different the effectiveness of the CB-pLM, as well as other conditioning approaches, at redesigning Siltuximab to lower its GRAVY index values, improving its hydrophobicity, so that variants will likely be more soluble.
4.2.1 EXPERIMENTS
We train a set of state-of-the-ard protein design models on the task of GRAVY reduction, apply the models to Siltuximab, and compare the GRAVY distribution of the generated samples.
Baselines We consider a wide variety of representative approaches to guided and unguided sequence generation: (a) Unguided Discrete Walk-Jump Sampling (WJS) (Frey et al. (023) energy- and
score-based model with single-step denoising and ESM2 (Lin et al.
(022) a large one-source protein
language model. (c) Implicitly guided PropEn (Tagasovska et al., 2024) encoder-decoder archi-
tecture that is implicitly trained to optimize a property of interest. (d) Explicitly guided LaMBO-2
(Gruver al. 2024) classifier-uiided discrete diffusion model. (e) Hydrophilic Resample a deep learning baseline, where residues are randomly resampled from set of known hydrophilic residues (N, C. Q G, S, T, Y) (Aftabuddin & Kundu 2007). Additional details about baselines and training procedures are available in Appendix
Results We intervene the GRAVY concept for generated samples for each candidate protein design model, with the constraint that generated samples should be at most an edit distance of 5 away from the Siltuximab sequence. In Figure 7al we show the distribution of designs from each model type. The CB-pLM comes second to LaMBO-2 in terms of the degree of concept distribution shift induced in the generated output, while preserving the naturalness of the protein (Figure (7b). trained to learn over 700 concepts delivers comparable results to state-of-the-art conditioning approaches while preserving the naturalness the protein.
nadi
510
3.0
L
Sycopelic
<oww
300
(a) GRAVY distributions.
(b) Naturalness/GRAAY distributions.
Figure 7: Siltuximab redesigns by different models.
INTERPRETABILITY AND DEBUGGING
Having established the effectiveness of the CB-pLM model for control and steering, demonstrate different interpretability and debugging capabilities of our model. A key component the CB- pLM is its final linear layer-a deliberate choice to maintain simplicity. By inspecting the weights of this linear layer, domain experts can understand the impact of specific concepts on the model's output. Moreover, if the model learns undesirable correlations if the learning process results in important features that deviate from a domain expert's prior knowledge, a straightforward inspection of the linear layer's weights can help identify and address these issues.
5.1 INTERPRETABILITY
CB-pL.M offers two types of interpretability: Local and Global. Local Interpretability allows the model indicate which concepts it relies on for predictions and identify the input tokens that influ- ence its output, achieved through scheme. Global Interpretability provides insights into the relationships between predefined concepts and various tokens, facilitated by the linear decoder. This section explores CB CB-pLM's global inter- pretability and verifies its alignment with human knowledge.
concept-bottlennek module and feature masking and regulation
In Figure 8 show the final layer weights for different concepts and amino acids in CB-pli CB-pLM (24M). We find that the model successfully learns several key biophysical relationships. We high- light some key insights below (the full weight matrix and more details available in Appendix D):
8

---

Published as a conference paper at ICLR 2025
For both charge at pH and pH 7, acidic amino acids (D, E) have negative weights, while basic
amino acids (R, K) have positive weights. The difference charge between pH and pH
reflects the biophysical properties of Histidine (H), as it contains imidazole side chain with
pKa of 6.0 and contributes more towards positive charge at lower pH (Figures Sa [8b).
GRAVY, which defines hydropathy based on the Kyte-Doolitle scale (Kyte Doolittle 1982).
is consistent with positive weights assigned to A, I, V, F, C, and M amino acids (Figure 8c
Aromatic amino acids F, Y, and W have high weights for the aromaticity concept (Figure 8d
layer welgt......
Ria layer swiignt
ATLV NCYNS TRHKDEGP
A WILVFWWCCOMSTRKKEEG
Charge at pH6
(b) Charge pH7
50
layer welght
Final lapee weigit
40
20
-15 AT LVV FWY NCOMS T IRHKDEGP
A ATLVFWYNC NOMMS TRHKDEGP
(c) GRAVY
(d) Aromaticity
Figure 8: The weights for selected concepts amino acids in the last layer CB-pLM 24M.
5.2 DEBUGGING
Model debugging involves detecting and fixing enrors during training or testing. Identifying the root cause of an error-such as issues with optimization, initialization, training data, or function class-can be challenging and often relies manual effort or retraining. CB-LM introduces mechanisms to attribute errors to human-undeen mamnaan-undestandable concepts, enabling us to answer key questions: 1) Did the model learn important feature of interest? 2) Is the model relying which situations is the model likely to fail? 4) How can we correct unwanted correlations? These questions can be addressed by inspecting the validation loss of different concepts, the weights of the linear layer, and using the CB layer for interventions.
spurious feature? 3) In
To showcase the model debugging capability of a CB-pLM, we trained Bad CB-pLM 24M, where we change the normalization for the aromaticity, making it very difficult for the model to learn and control We want to see we can, in fact, identify where this model will fail without the need for intervention experiments. To do this, we examine the weight for aromaticity-aminn amino acid combinations for the Bad CB- -L-L (see Figure 9ap. By inspection, we observe that weights are all essentially zero; hence, the model will not be able to control aromaticity. To confirm the result, in Figure 9b)
we show that intervening on the aromaticity concept does not control the model's ss output.
Bad Normalization: Aromaticity
Bnd Aormantoooo
Fral layer wolg
40
30
20
10
-20
ALLS RHKDEGP
Acammbti
(a) Outgoing weights from aromaticity for model with bad concept normalization.
(b) The effect of intervention on the aromaticity with good bad model.
Figure 9: CB-pLM linear layer helps identify model's failures.

---

Published as a conference paper at ICLR 2025
6 DISCUSSION AND CONCLUSION
Related Work Language models are increasingly used across various fields, demonstrating excep- tional performance but lacking interpretability, which limits their use in high-stakes scenarios. While recent work with sparse autoencoders (SAEs) Cunningham et al. (2023); Templeton et al. (2024) shows promise in uncovering interpretable features, this approach requires training large SAEs and interpreting millions of features. Concept bottleneck language models (CB-LMs) offer a simpler solution by directly encoding concepts without additional interpretation steps.
Concept Bottleneck Models enhance neural network interpretability by incorporating a "concept bottleneck" layer that maps inputs to human-understandabhe concepts for final predictions (Koh) it al. 2020). Recently, Ismail et al. (2023) extended this approach to generative models (CBGMs), enabling controllable generation and concept-level explanations, particularly in image generation tasks. In this paper, we adapt this approach to generative language models, achieving global control over the entire imput while retaining lekel- generation. A few very recent works have proposed CBMs for text classification tasks Tan et al. (2024); Sun et al. (2024), but this is much easier and less interesting setting than generative language modeling which we address.
Protein language models (pLMs) are extensively used in biological machine- research. De- spite their critical applications in healthcare and drug discovery (Hie et al., 2024), pLMs currently lack interpretability. Historically, pLM architectures, loss functions, and training setups have closely mirrored the original masked language model setup (Devlin et al., 2018). differing mainly in vocab- ulary and dataset. While some protein language models support conditional generation by concate- nating different protein functions and properties to the inputs (Shuai et al.. [021] Madani et al.. 22200) Hayes al., 2024). they do not provide mechanisms for interpretabilityy debuggability, or insights into what the model has learned. See Appendix [.. for extended related work.
Discussion and Future Work In this paper, applied our proposed architecture masked lan- guage models for proteins. This architecture can be adapted to any domain, such as NLP, by chang- ing the training dataset, vocabulary, and concepts while keeping the model unchanged. It can also be extended to other language models, such as autoregressive models (Figure 29 in Appendix [.2). In principle, CB-LM can be used to generate sequences with unseen concepts or novel combinations, as long as they can be expressed as functions of existing ones. We demonstrate this capability in toy task detailed in Appendix C.3.2 Testing this capability empirically on proteins is beyond the scope of the current paper and is left for future work. Our method is unlikely to enable the devel- opment of weaponizable molecules due to limited biophysical understanding of biological functions and physiochemical properties. However, stress the importance of biosecurity and recommend consulting regulatory agencies and ethical guidelines to mitigate potential risks.
Linitations One main limitation of CB-LM, like other concept bottleneck models (Ismail et al. 2023 Koh et al. 2020), is the need for the training dataset to be annotated with predefined con- cepts. Another drawback is the necessity for unknown embeddings. Generally, we believe this is unavoidable, as it is impractical to encode all the information needed for generative task into a fixed number of predefined concepts; excluding unknown embedding will lead unacceptable per- formance losses. However, reliance on unknown parts can challenge control and interpretability, potentially causing the model ignore concept outputs. Our method mitigates this with orthogo- nality loss to improve control, but some concept information can remain in the unknown part, and better disentanglement methods are needed.
Conclusion We present Concept Bottleneck Language Models (CB-LM), controllable- interpretahle-debuggahh language model. We have demonstrated the effectiveness of our pro- posed architecture by applying it to masked protein language models. Our results show that this architecture can scale to large models. as evidenced by training Concept Bottleneck Protein Lan- guage Models (CB-pLM) with 24M, 150M, 650M, and 3B parameters to encode over 700 human- understandable concepts while maintaining pre- training perplexity comparable to traditional masked protein language models. Our findings indicate that CB-pLM offers superior control compared to conditional language models, along with interpretability and debugging capabilities. This work rep- resents a significant step towards more interpretable, reliable, and debuggable neural networks.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Brennan Abanades, Wing Ki Wong, Fergus Boyles, Guy Georges, Alexander Bujotzek, and Char- lotte M Deane. Immunebuilder: Deep-learning models for predicting the structures of immune proteins. Communications Biology, 6(1):575, 2023.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Julius Adebayo, Justin Gilmer, Michael Muelly, lan Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Julius Adebayo, Samuel Stanton, Simon Kelow, Michael Maserl Richard Bonneau, Vladimir Glig- orijevic, Kyunghyun Cho, Stephen Ra, and Nathan Frey. Identifying regularization schemes that make feature attributions faithful. In NeurlPS 2023 Workshop on New Frontiers of Al for Drug Discovery and Development, 2023.
Md Aftabuddin and S Kundu. Hydrophobic, hydrophilic, and charged amino acid networks within protein. Biophysical journal, 93(1):225-231, 2007.
Sharrol Bachas, Goran Rakocevic, David Spencer, Anand V Sastry, Robel Haile, John M Sutton, George Kasun, Andrew Stachyra, Jahir M Gutierrez, Edriss Yassine, et al. Antibody optimization enabled by artificial intelligence predictions of binding affinity and naturalness. BioRxiv, 2022.
David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus- Robert MÅAller. How to explain individual classification decisions. In Journal of Machine Learn- ing Research, 2010.
Amos Bairoch and Rolf Apweiler. The swiss-prot protein sequence database and its supplement trembl in 2000. Nucleic acids research, 2000.
Bikash K Bhandari, Paul P Gardner, and Chun Shen Lim. Solubility- Index: fast and accurate prediction of protein solubility. Bioinformatics, 2020.
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023.
Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting lange-la models with chemistry tools. arXiv preprint arXiv:2304.05376, 2023.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decom- posing language models with dictionary learning. Transformer Circuits Thread, 2, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33, 2020.
Bo Chen, Xingyi Cheng, Yangli-ao Geng, Shen Li, Xin Zeng, Boyan Wang, Jing Gong, Chiming Liu, Aohan Zeng, Yuxiao Dong. Jie Tang, and Le Song. XTrimoGLMM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein, 2023.
Peter JA Cock, Tiago Antao, Jeffrey T Chang, Brad A Chapman, Cymon Cox, Andrew Dalke, Iddo Friedberg, Thomas Hamelryck, Frank Kauff, Bartek Wilczynski, et al. Biopython: freely available python tools for computational molecular biology and bioinformatics. Bioinformatics, 2009.
Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. segpt: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, 2024.
11

---

Published as a conference paper at ICLR 2025
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 2023.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 1188.1000005555 2018.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang. Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, and Burkhard Rost. Pro(Trans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Nathan C Frey, Daniel Berenberg, Karina Zadorozhny, Joseph Kleinhenz, Julien Lafrance- Isidro Hotzel, Yan Wu, Stephen Ra, Richard Bonneau, Kyunghyun Cho, et al. Protein discovery with discrete walk-jump sampling. arXiv preprint arXiv:2306. 12360. 2023.
Nathan C Frey, Taylor Joren, Aya Ismail, Allen Goodman, Richard Bonneau, Kyunghyun Cho, and Vladimir Gligorijevic. Cramming protein language model training in 24 gpu hours. bioRxiv, 2024.
lason Gabriel. Artificial intelligence, values, and alignment. Minds and machines, 2020.
Bryce Goodman and Seth Flaxman. European union regulations on algorithmic decision-making and "right to explanation". Al magazine, 2017.
Nate Gruver, Samuel Stanton, Nathan Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance-Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew G Wilson. Protein design with guided discrete diffusion. Advances neural information processing systems, 36, 2024.
Kunchur Guruprasad. V.Bhasker Reddy, and Madhusudan W. Pandit. Correlation between stabil- ity of a protein and its dipeptide composition: a novel approach for predicting in vivo stability of protein from its primary sequence. Protein Engineering, Design and Selection, 1990.
Marton Havasi, Sonali Parbhoo, and Finale Doshi-Velez. Addressing leakage in concept bottleneck models. In Advances in Neural Information Processing Systems, 2022.
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J. Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q. Tran, Jonathan Deaton, Marius Wiggert, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry, Raul S. Molina, Neil Thomas, Yousuf Khan, Chetan Mishra, Car- olyn Kim, Liam J. Bartie, Matthew Nemeth, Patrick D. Hsu, Tom Sercu, Salvatore Candido, and Alexander Rives. Simulating 500 million years of evolution with language model. bioRxiv, 2024.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.
Brian L Hie, Varun R Shanker, Duo Xu, Theodora UJ Bruun, Payton A Weidenbacher, Shaogeng Tang, Wesley Wu, John Pak, and Peter S Kim. Efficient evolution of human antibodies from general protein language models. Nature Biotechnology, 2024.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. Evaluating feature importance estimates. arXiv preprint 1806.
2, 2018.
TP Hopp and KR Woods. Prediction of protein antigenic determinants from amino acid sequences. Proceedings of the National Academy of Sciences of the United States of America, 1981.
Aya Abdelsalam Ismail, Julius Adebayo. Hector Corrada Bravo, Stephen Ra, and Kyunghyun Cho. Concept bottleneck generative models. In The Twelfth International Conference on Learning Representations, 2023.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2025
Jack Kyte and Russell F. Doolittle. A simple method for displaying the hydropathic character of protein. Journal of Molecular Biology. 1982.
Leiro, PM Moreno, B Sarmento, J Durão, L Gales, AP Pêgo, and CC Barrias. Design and prepara- tion of biomimetic and bioinspired materials. In Bioinspired Materials for Medical Applications. Elsevier, 2017.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sereu, Sal Candido, al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 2023.
Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R Eguchi, Po-Ssu Huang, and Richard Socher. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03497, 2020.
Anita Mahinpei, Justin Clark, Isaac Lage, Finale Doshi-Velez. and Weiwei Pan. Promises and pitfalls of black-box concept learning models. arXiv preprint arXiv:2106.1333 2021.
Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn intended? arXiv preprint 20552:2105.04289, 2021.
Derek M Mason, Simon Friedensohn, Cédric R Weber, Christian Jordi, Bastian Wagner, Simon M Meng, Roy A Ehling, Lucia Bonati, Jan Dahinden, Pablo Gainza, et al. Optimization of therapeu- tic antibodies by predicting antigen specificity from antibody sequence via deep learning, Nature Biomedical Engineering. 5(6):600-6) 612, 2021.
Olga Obrezanova, Andreas Arnll, Ramón Gómez De La Cuesta, Maud E Berthelot, Thomas RA Gallagher, Jesús Zurdo, and Yvette Stallwood. Aggregation risk prediction for antibodies and its application to biotherapeutic development. In MAbs. Taylor & Francis, 2015.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-War Weng. Label-free concept bottle- neck models. arXiv preprint arXiv:2304.066 06129, 2023.
Tobias H Olsen, Fergus Boyles, and Charlotte M Deane. Observed antibody space: A diverse database of cleaned, annotated, and translated unpaired and paired antibody sequences. Protein Science, 2022.
Kanchana Ranasinghe, Muzammal Naseer, Munawar Hayat, Salman Khan, and Fahad Shahbaz Khan. Orthogonal projection loss. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Sukrut Sridhar Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-namee: Task- agnostic concept bottlenecks via automated concept discovery. In 18th European Conference on Computer Vision. Springer, 2024.
Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. Toward transparent ai: A survey on interpreting the inner structures of deep neural networks. In 2023 ieee conference on secure and trustworthy machine learning (satml). IEEE, 2023.
Matthew IJ Raybould and Charlotte M Deane. The therapeutic antibody profiler for computational developability assessment. Therapeutic Antibodies: Methods and Protocols, pp. 115-125, 2022.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS, 2019.
Jeffrey A Ruffolo, Aadyot Bhatnagar, Joel Beazer, Stephen Nayfach, Jordan Russ, Emily Hill, Riffat Hussain, Joseph Gallagher, and Ali Madani. Adapting protein language models for structure- conditioned design. bioRxiv, 2024.
13

---

Published as a conference paper at ICLR 2025
Yoshihide Sawada and Keigo Nakamura. Concept bottleneck model with additional unsupervised concepts. IEEE Access, 10, 2022.
Harshay Shah, Prateek Jain, and Praneeth Netrapalli. Do input gradients highlight discriminative features? Advances in Neural Information Prucessing Systems, 34, 2021.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences, 2017.
Richard W Shuai, Jeffrey A Ruffolo, and Jeffrey Gray. Generative language modeling for antibody design. BioRxiv, 2021.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, 2013a.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint 1.XXiv:312.6034,, 2013b.
Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Green- side, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. In International Conference Machine Learning. PMLR, 2022.
Chung-En Sun, Tuomas Oikarinen, and Tsui-Wei Weng. Crafting large language models for en- hanced interpretability. 2024.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning. PMLR, 2017.
Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consor- tium. Uniref clusters: comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics, 2015.
Nataša Tagasovska, Vladimir Gligorijević, Kyunghyun Cho, and Andreas Loukas. Implicitly guided design with propen: Match your data to follow the gradient. arXiv preprint arXiv:2405.18075, 2024.
Zhen Tan, Lu Cheng. Song Wang. Bo Yuan, Jundong Li, and Huan Liu. Interpreting pretrained language models via concept bottlenecks. In Pacific-Asia Conference on Knowledge Discovery and Data Mining- Springer, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1100: 2023.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas I. Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monoseman- ticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 2023.
Frits Van Rhee, Luis Fayad, Peter Voorhees, Richard Furman, Sagar Lonial, Hossein Borghaei, Lubomir Sokol, Julie Crawford, Mark Corfeeld, Ming Qi. et al. Siltuximab, a novel anti- interleukin-6 monoclonal antibody, for castleman's disease. Journal clinical oncology, 2010.
Ashish Vaswani. Attention is all you need. arXiv preprint arXiv:1706.03762. 2017.
Minghao Xu. Xinyu Yuan, Santiago Miret, and Jian Tang. ProtST: Multi- Modiiityylttty Learning of Protein Sequences and Biomedical Texts, 2023.
14

---

Published as a conference paper at ICLR 2025
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Buuri and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable im- age classification. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205. 2022.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks, In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part 13. Springer, 2014.
Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du Explainability for large language models: A survey. ACM Trans- actions on Intellientt Systems and Technology, 2024.
15

---

Published as a conference paper at ICLR 2025
Appendix
A ADDITIONAL DETAILS ON CONCEPT BOTTLENECK LANGUAGE MODEL.
A.1 LOSS FUNCTION
Following CBGMs (Ismail et al. 2023), concept bottleneck language models are trained with three standard generative masked language modeling loss LMLMM concept loss Concepp and an orthogonality loss Lonh- The final loss is given by
losses:
Liotal = LMLM + CCConeeept + B
where a and are hyperparameters.
Generative masked language modeling loss:
1 m
[MLM = -Egr.
log P(rs mmm)
4Em
where m is the set of positions of the randomly masked tokens, and the model is trained to predict the identity of the masked tokens.
Concept loss: Given the general-purposs. nature of large language models, they are pected to handle thousands of concepts with categorical or real values. Often, samples lack values for most concepts. To address this, normalize all real-valued concepts to [0, 1] and apply mean square error loss on the entire concept embedding:
CConcept
Σ (a: 22)2 K
1
Missing values are replaced with default values, and their effect is removed from the loss function by considering only non-missing concepts, achieved by masking the errors before backpropagation.
Orthogonality loss: Following Ismail et al. 2023 to ensure effective control over the model's output, is crucial to prevent unknown concepts from being mere transformations of known concepts. This is achieved by enforcing an orthogonality constraint (Ranasinghe al. (22TT), which minimizes the cosine similarity between the concept context embedding and the unknown context embedding. The orthogonality loss is defined as follows:
Eiso ims
(z, Ra)
Lorth
Σ 1
JEB
where (., :) is the cosine similarity applied to two embedding, | is the absolute value, and B denotes mini-batch size and j denotes each sample the mini-batch. The cosine similarity in the above equation involves the normalization of features such that 4, where || is l2 norm. (xs xz)) Ia |1
A.2 TRAINING FOR INTEPRETABILITY
Reducing concept leakage through independent training: One known problem in CBMs is con- cept leakage (Margeloiu et al. 2021: Mahinpei et al., [222 Havasi et al. 2022); this happens when soft concept representations encode more information than the concepts themselves. Concept leak- age affects both model interpretability and control. Mahinpei et al. (2021) discussed that this leak- age happens when CBMs are trained jointly or sequentially. However, leakage is not possible when CBMs are trained independently (i.e., during training, the post-CB--ie part of the network takes the ground -ruth concepts themselves rather than the output of the CB-layer). For this reason, we choose independent training for CB-LMM.
16

---

Published as a conference paper at ICLR 2025
Faithful token attribution through regularization: Apart from concept-level explanations, users might be interested in token-level explanations to identify which tokens were most influential during generation. Token-level explanations also be used to identify which token to mask (i.e., the to- ken that has the most effect on a particular concept) when intervening on different concepts (Gruver et al.] 2024). Gradient-based explanation methods BBahhrens et al. (2000); [Simonyan al. (2013b); Zeiler & Fergus (2014) are popular approaches for "explaining" models, but many works (Adebayo al.] 2018: Hooker et al., 2018) showed that such feature attribution methods are no better than random ranking of the input features. The reliability of feature attributions seems to be correlated to how the model is trained; (Shah et al. (202T) showed that gradient- based feature attributions of ad- versarially robust image classifiers are faithful, unlike their non-robust counterparts; Adebayo et al. 2023), showed that the addition Gaussian noise to the input has the same effect as adversarial training for inducing attribution faithfulness. Following [Adebayo et al. (2023) 's recommendation, we add Gaussian noise to the token embedding during training for model regularization and faithful token attribution.
B ADDITIONAL DETAILS ON CONCEPT BOTTLENECK PROTEIN LANGUAGE MODEL.
B. TRAINING DATA
We combined sequences from UniRef50 (Suzek et al. 2015) and SWISS-PROT (Bairoch & Ap- weiler 2000), removing duplicates. Annotations such protein clusters, organisms, taxons, bio- logical processes, cellular components, and molecular functions from SWISS-PROT were used as concept annotations. Biopython (Cock et al. 2009) was used to extract biophysical and bioinformat- ics sequence-leve concepts. We filtered out rare SWISS-PROT concepts; the final list of concepts
shown in 2
Class
Number of Concepts
Calculated from sequence
Cluster name
159
Biological process
140
X
Cellular component
162
Molecular function
106
Organism
36
Taxon
101
Biopython
14
Table 2: Training concepts.
Biopython
All concepts were computed from the Biopython ProtParam module.
Molecular weight is the mass of the protein.
Charge at pH and pH 7 is the net charge of the protein pH 6 and pH 7.
Isoelectric point is the pH at which the protein has no net charge.
Aromaticity is the relative frequency of aromatic amino acids (F, W. Y) in the sequence
Instability Index is the sum of the instability weights from sequence decomposition.
Secondary structure fraction the relative frequency of amino acids helix: V, Y. W. L. Amino acids in Turn: N, P. G. S. Amino acids in sheet: E, M, A, L.
Molar extinction coefficient considers the weighted sum of W, Y, and C for protein ab- sorption of 280 nm.
Gravy is the sum of hydropathy values associated with the amino acids a sequence divided by the total number of amino acids, based on the Kyte and Doolittle hydrophobicity
scale.
Protein Scale An amino acid scale is defined by numerical value assigned to each type amino acid. The most frequently used scales are the hydrophobicity or hydrophilicity scales, the secondary structure conformational parameters scales and surface accessibility.
17

---

Published as a conference paper at ICLR 2025
B.2
MODEL PARAMETERS AND CONFIGURATIONS AT DIFFERENT SCALE
24M
150M
650M
3B
Number of layers
10
27
33
26
Embedding dim
408
768
1280
2560
Attention heads Concept embedding dim Learning rate
12
12
20
40
2
2
2
0.001
0.0001
0.0001
0.0001
Clip norm
0.5
0.5
0.5
0.5
precision
16
16
16
bf16
Warmup steps
3000
10000
30000
30000
Effective batch size
512
1024
1024
1024
Distributed backend
ddp
ddp
ddp
deepspeed_stage__
Training: During training mask percentage is 25% of the sequence. We then truncate all sequences to a maximum length of 512. Sequences of length less than 512 were padded, but no loss was backpropagated through the network for padding tokens. We use Rotary Position Embedding.
C CONTROL EXPERIMENTS
C.1 COMPARING CONDITIONAL. LANGUAGE MODEL ARCHITECTURES in silico
Experimental Setup We evaluate control on 14 different concepts that can be calculated from Biopython. We want to if we can intervene in the models by increasing the concept values (positive interventions) or decreasing the concept values (negative interventions). To accomplish this, we utilized the held-out validation dataset; for each concept, we selected the 10,000 sequences with the lowest concept values and the 10,000 sequences with the highest concept values. We use the lowest value sequences for positive interventions (i.e., we intervene in the model to increase the value of that given concept), and the highest value sequences are used for negative interventions (i.e., we intervene to decrease the concept value). We mask percentage of the test sequence and intervene on the concept to set it to a minimum maximum value based on the intervention type. For the conditional model, C-pLM,, masking is done randomly; for the conditional classifier CC- pLM and proposed CB-pLM model we select the tokens that contributed to the given concept the most using feature attribution (Gruver et al.) 2024 Adebayo et al,, 2023) (additional details on how feature attribution is done is available Appendix D.I) and mask them. For each test sample, we only consider a single generated sample (i.e this can be viewed as single shot generation). After generation, we calculate the new concept values of the generated sequence. We measure accuracy as the percentage of generated samples that successfully moved toward the direction of the intervention. We use likelihoods from an auxiliary autoregressive causal language model measure the "naturalness of" or feasibility of the sequence (Bachas et al. 2022).
C.1.1 SINGLE CONCEPT INTERVENTION
In this section, we focus on single-concept intervention; the intervention procedure is as follows:
ONE TIME INTERVENTION
This sections shows results when setting n in Algorithm
Accuracy After the intervention, we calculate the accuracy as the percentage of generated samples that successfully moved toward the direction of the intervention for positive and negative interven- tions, then average them (random accuracy would be around 50%). The average across per concept shown in Figure [0b). Our findings demonstrate that in terms intervention direction accuracy, CB-pLM outperforms other conditional models every single concept. Figure shows the aver- age change in the desired direction of the concept value of different concepts after our interventions. positive and negative interventions. The concept values are normalized such
This is averaged that the maximum value in training data is and the minimum value is O. We can see our CB-p CB-pLM models clearly outperform baselines for almost all concepts.
18

---

Published as a conference paper at ICLR 2025
Algorithm Single Concept intervention procedure
Require: sequence to intervene on = [To, Fal, where s is the maximum sequence length, model (i.e, CB-pLM) f. concept to intervene k; concept value after intervention c, direction of intervention and number of interventions n.
1:
2: while <. do
3: Using feature attribution, identify the top 5% of the token that either negatively influences concept for positive intervention positively influences the concept of negative intervention. 2masked &- masking(r, k. f.d)
4: Pass the model the masked input, the concept index, and the value get the sequence.
T = =[[macckes k c)
5: x
6: end while
7: return
Average concept control accurscy when maskingg 5.0%
27.4
32-
21.2
Average intervention accuracy.
(b) Per concept intervention accuracy.
Figure 10: Intervention accuracy after masking 5%.
Average concept change when masking 5.0%
9:247
2.643
3.04
0.012
3,92
3.82
(a) Aggregate intervention effect.
(b) Per concept average intervention effect.
Figure Intervention effect after masking 5%.
19

---

Published as a conference paper at ICLR 2025
Distribution shift We investigate how the concept distribution shifts along with the naturalness of the protein after intervention. Ideally, we would want the concept distribution to shift in the direction of the intervention while maintaining the naturalness of the protein; plot A Concep//Naturalness distribution (the difference between the original sample and the generated sample for the concept value æ-axis and naturalness y-axis).
Figure and Figure 13 shows distribution shifts between different types of 24M and 150M models respectively, with both positive and negative interventions, CB-pLM effectively shifts the concept distributions in the right direction for all concepts while preserving the naturalness of the proteins, unlike other variations conditional language models. Figure 14 shows distribution shifts between CB- CB-pLM with different sizes. Note that although increasing model size improved the model's s per- plexity we did not observe any improvement in terms of control.
Positive interventions when masking for different 24M models
direction
(a) Positive interventions.
Negative interventions when masking 5% for different models
Intervention
(b) Negative interventions.
Figure 12: Shift in protein naturalness/concept value after interventions for different 24M models.
Iterative intervention In this experiment, examine different models' ability to shift concept's distribution iteratively, i...; we vary in Algorithm from 1 + 3. Figures 15 and I6 shows concept distributions for three iterations for both positive and negative interventions. CB-pLM iteratively shift the concept value with increasing effects while other models fail to do this.
Masking percentage In this experiment, we investigate the impact of varying the percentage of masking on the performance ranking of different models. We repeat the experiment described in Algorithm M but with 25% masking instead 5%, using 24M parameter models. Our findings show that the performance ranking of the models remains consistent, with CB CB-pLM significantly outperforming both C-pLM and CC-pLM (Figures 17 and
20

---

Published as a conference paper at ICLR 2025
Positive interventions when masking 5% for different 150M models
Brisel Intes
Miates
201
Menode mige
3 Daroeehh
&Oagge A1?
Neie Out Redeon
Mile te Guaren)
(a) Positive interventions.
Negative interventions when masking 5% for different 150M models
Ashie maaten
kvg Surfec Accisaaly
Metoodrr Aige
Ders Dont kch
OOagg AP
06. OLEON)
Negative interventions.
Figure 13: Shift in protein naturalness/conccet value after interventions for different 150M models.
Positive interventions when masking 5% for different CBM models
1iy 2M
(1 mnce
10
10 3V DM
Now lat cael cutcedd
(a) Positive interventions.
Negative interventions when masking 5% for different CBM madels
2000
PM
Arhop 21R3
400 Te
J La FAbIc
NOW CAE
(b) Negative interventions.
Figure 14: Shift in protein naturalness/concent value after interventions for CB-pLM models with different sizes.
21

---

Published as a conference paper at ICLR 2025
Positive interventions when iteratively masking 5%
Syam 2i 241
Figure 15: Iterative positive intervention.
Negalive interventions when iteretivelly mesking 5%
cpu 24M
28gu 241
teniles
Figure 16: Iterative negative intervention.
Avemge cxncegt centro scourny when mandrg
Average concept change when masking 25.0%
203
11.6
50.5
8.19
0161
$0.12
0.00
0,052
6.66
04
CB 2LM
0.00
CC-p.N21
(a) Intervention accuracy.
(b) Average change concept value after intervention.
Figure 17: Average concept intervention accuracy and effectiveness when masking 25%.
Positive interventions when masking 25% for different 24M models
COAN
Barality Fart
Holl Frades
Ehe Geuuus
32 Studoo
-25
Dhaec PM
Moler Raduodd
Madee Do Carl bocoed
Figure 18: Shift in protein naturalness/concept value for positive interventions when masking 25%.
Negative interventions when masking 25% for different 24M models
40
20 30
200
110
14
5 Me Ea Oct 0k.
Figure 19: Shift in protein naturalnesss/concep value for negative interventions when masking 25%.
22

---

Published as a conference paper at ICLR 2025
Concept intervention correlation In this experiment, mask the most influential tokens for each concept as previously described and intervene on the value of that concept. We then calcu- concept late the value of all 14 concepts for the generated sequence, and how intervening different mod- effects all the other concepts. The results are reported in Figures 20b 20c and els. In this figure report version of accuracy, where the j-th element of the i-th row represents the (fraction of times concept j changed in the same direction as the intervention when intervening on the i-th concept) (fraction of times concept j changed in the opposite direction of the interven- tion when intervening on the i-th concept). This can be seen as binary version of correlation. We also compare this with the actual correlation between these concept values on existing proteins in our dataset shown in Figure 20a We can see the correlation matrix of interventions on our CBM has many similarities to correlations between the concept in the real world, especially on pairs with high correlation, reflecting that our concept bottleneck model has picked up on these natural correlations. In contrast, the other models have quite different correlation pattern.
Uhew
wark
(a) Correlation between concept values our datasct.
(b) Intervention correlation on CB-PLM.
Melecda Airgle
Canyy
cow
eyregh day
(c) Intervention correlation on C-pLM.
(d) Intervention correlation on CC-pLM.
1.2 MULTI-CONCEP! INTERVENTION
We show the effects of sequentially intervening on two concepts in Figure 21 with different 150M parameter models below.
23

---

Published as a conference paper at ICLR 2025
Averaçe multi concept when masking 5.0%
Mutiiccnccpt Positive interventions masking 5%
Mulli-concept Negative interventions mesking 5%
822
c 350M
CC 150M
CB
0.0
-0.03
0.04
23
0C 150M
050 130m
-0.66
ColM 130M
Co 190M
AChagge 9m7
-0.08
-0.0
0.00
-0.04
-0.00
0.00
0.00
Auturacy
(0) Concept Distribution
Figure 21: Multi-concept interventions.
C.2 PROTEIN DESIGN SILTUXIMAB
C.2.1 BASELINES
LaMBO-2 Gruver et al. 2024 diffusion optimized sampling, a guidance method for discrete diffusion models that follows gradients in the hidden states of the denoising network. LaMBO-2 was trained on SWISS SWISS-PROT (~~ 2.6M training samples training) using GRAVY as the optimiza- tion objective for explicit guidance.
PropEn (Taaasvsska et al. (2224) is an encoder-decoder architecture that is implicitly trained to optimize property of interest. PropEn operates by matching training samples so that each training sample is paired with one that has better properties; due to this matching, it is best suited for a low data regime. We train PropEn on curated antibody dataset of 1000 training samples that are close to Siltuximab.
Discrete Walk-Jump Sampling (WJS) (Frey et al. 2023 is a discrete generative model that learns smoothed energy function, then samples from the smoothed data manifold with Langevin Markov chain Monte Carlo (MCMC), and projects back to the true data manifold with one-step de- noising. WJS was trained on paired observed antibody space (OAS) dataset OOsen (2022).
which is ~ 1 120K paired antibody sequences as done in the original paper.
ESM2 [Lin et al. 2022) an open source 150M parameter protein language model, pretrained on UniRef50. Designs
sampled from the model's logits.
Hydrophilic Resample non-deep learning baseline, where residues are randomly resampled
from a set of known hydrophilic residues (N, C, Q G. S. T, Y) (Aftabuddin & Kundu 2007).
C.2.2 ASSESSING THE NATURALNESS OF THE DESIGNS
To evaluate the naturalness designs generated by various models, we folded the designs us- ing ABodyBuilder2 (Abanades et al.) (023) and analyzed different protein surface properties with the Therapeutic Antibody Profiler (TAP) Raybould & Deane (2022). which utilizes physics-based computations. Our objective is achieve minimal deviations from the original antibody struc- ture. Figure 22 presents the TAP scores of different designs, with the dotted line indicating the original Siltuximab value. The results reveal that the designs produced by CB-pLM and WJS ex- hibit the least disruption, whereas those from ESM2 and Hydrophilic Resample show the most disruption. The TAP metrics validate that our CB-PLM designs effectively preserve naturalness.
A5-
3.5
Charge Gaymeerry
Pootee of
3.0
05
Priden LIMIO-- ClesM Myrisshll
Pepii LaMEc-3
CDeLM Hydinghlic Resarppl
Nogge
LINBOO CRiSUN Hndeptllc Reserp
Anargle
Figure 22: TAP metrics for Siltuximab redesigns.
24

---

Published as a conference paper at ICLR 2025
17.3
13.0
Filtering Designs Out of the designs produced by different models, we have filtered designs with TAP values that are with 5% proximity to Siltuximab; the percentage of designs that have passed this filter from each model is shown in Figure 23
12.9
10.0
7.5
5.0
6392
Prosen LaMto SUM Hyarophiic
Assamppl
Mathod
C.3 ADDITIONAL CONTROL EXPERIMENTS
Figure 23: Percentages redesigns within the naturalness threshold.
C.3.1 SPECIES INTERPOLATION
Prior work in mechanistic interpretability (Bricken et al. 2023) demonstrated that sparse autoen- coders (SAEs) could uncover interpretable monosemantic features. Recently, Templeton Rccently,Templeton al. (2024) applied SAEs to Claude Sonnet's 's activations, retrieving 34M features. They used the autoint- erpretability approach (Bills et al. 2023) assign meanings to these features. One notable feature identified was the Golden Gate Bridge. By clamping this feature to 10x its maximum activation, they influenced the model's s behavior, causing Claude to behave in an out-of-distribution manner, going as far as self-identiffing neck language model demonstrates the same capability, although for CB-LMs there no need to train additional SAE and do post-hoc search for the meaning of different concepts, since here we know what each concept encodes by design.
the Golden Gate Bridge. Here, show how the concept bottle-
To demonstrate, test the CB CB-pLM's ability interpolate between species; altered the behavior of the CB-pL.M by setting a single species concept in the original CB-pL.M to 10 times its maximum activation value. We then tested whether the model could generate sequence similar that species cluster, regardless of the protein sequences inputted into the model. We conducted this test for two species: (a) rice (species Oryza sativa referred as "Rice-CB-pLLM") and (b) human (species Homo sapiens referred to as "Human-CB-pLLM") We trained a species classifier by finetuning the ESM2(Lin al. 2022 model to classify different species (humans, yeast, rice, and bacteria). The model accurately clustered different species together, with an overall accuracy of 98%, as shown in Figure 24a We then passed non-rice protein sequences into Rice-CB-pL.M and classified the output; most newly generated sequences were now identified as rice (Figure 24b). Similarly, passing non- human sequences into Cauman-CB-pLMy resulted most newly generated sequences being classified as human (Figure 24ch; note that changing non-human protein sequences into human sequences is fundamental problem in protein engineering known as "humanization".
This illustrative example showcases the ability of the concept bottleneck language model to replicate the capabilities of SAEs and autointerpretabiiity with a substantially simpler approach.
UIMAP ESM2 Specias
UMAP of FSM2 Species
UMAP of FSM2 Species
(a) ESM2 specie clusters.
(b) Rice CB pL. changing rice quences to rice like sequences.
(c) Human CB pLM changing non human sequences to human like sequences.
Figure 24: Manipulating Large Language Models.
C.3.2 GENERALIZATION TO NEW CONCEPT COMBINATION
So far have shown that our model can successfully control generation of in distribution quences. However, for many real-world use cases, it important to be able to generate previously unseen concepts combinations of concepts.
25

---

Published as a conference paper at ICLR 2025
To test this, we train model with architecture simpler task based on color-MNIST. To make this task similar to masked language modeling, we train our models by masking 75% of the input, and then train them to reconstruct the remaining input using masked autoenecoder vision transformer (MAF-ViT) (He et al. 2022). Following MAE-Vit we divide the input into a sequence of 7x7 patches, making task masked sequence-to-sequence generative task, just like masked language modeling. Note that unlike our main results, we use nonlinear decoder to accommodate for pixel-level outputs. For the concepts in this experiment, we use one-hot encoding of the number represented by the image, together with an RGB encoding of the color of the input digit. important use RGB embeddings instead of one-hot representation of color, as allows to encode any color in just dimensions, which is useful for testing generalization capabilities. To test generalization ability our CBM architecture, we trained a generative CBM as discussed above on ColorMNIST with 7 discrete training colors: (magenta, grey, cyan, white, red, green, yellow). On test time, we then intervened on the CBM color outputs to make the model create an unseen color. As shown in Figure 25 our method can accurately regenerate the the input for all 3 unseen colors we tested. These examples were not cherry picked, and our color intervention reliably works on all inputs.
Original
Masked Input
Reconstructed Input
Intervene: Color
Intervene: Color
Intervene: Color
Input
S
8
5
5
5
9
Training colors:
Figure 25: Results of our generative CBM on color MNIST. We can accurately generate digits in colors completely unseen during training. This highlights that CBMs are able to generate novel outputs.
D INTERPRETABILITY AND DEBUGGING
D.1 FEATURE ATTRIBUTION AND COORDINATE SELECTION
The goal of our control experiments is to make small modifications to protein sequences that increase decrease the value of certain concept. While during training we randomly select which tokens to mask and predict, this is not an effective strategy for interventions. Instead we wish to identify which amino acids increasing/decreasing certain concept the most, and only mask and predict those tokens.
For example if we wish to increase Aromaticity of protein, we wish identify the amino acids which contribute the most to decreasing the Aromaticity, and intervene on those tokens. The most straightforward and principled method to measure this occlusion, where we simply remove a each input token and replace it with a reference value one at a time, and see how our predicted concept changes for each. Since our model was trained with masking, the natural reference value is the mask token, when removing certain input simply replace it with the mask token. Let T 2xd be the learned token embeddings where v is the vocabulary size and is transformer hidden dimension. Then let (x) [Tag T ..,...., T. The occlusion attribution for token of input concept then:
OCCLUSION:
A(x,i,1) (
26

---

Published as a conference paper at ICLR 2025
Where P2y4-IMASH represents with the t-th token replaced by the mask token, and (s((())) is the concept value predicted by our CBM model for concept i. concept prediction for concept on imput æ. For positive interventions, typically then mask the 5% of the tokens (excluding padding) with the smallest attribution value(token being present reduces concept value), while for negative interventions we mask 5% of tokens with the largest attribution value(token being present increases concept value).
While this is a good method, it requires separate forward pass for every individual token in the input, making it slow to run in large scale. Instead, as is common with feature attributions, we use gradient based approximation of the occlusion value for up to 512 times speedup compared to occlusion. However, since our inputs are discrete, we cannot directly calculate gradients with respect to them. Instead, we calculate gradients with respect to the learned token embeddings of the model, and sum over these. Then
GRADIENT (INPUT MASK):
A(z,i,t) V (r(()) (Tz, -T(MASK])
(3)
Where (x)) is the t-th element of fx). This method is essentially first order Taylor Approxi- mation of the occlusion metric defined above. Alternatively this method could be seen as a version al. (2017) with only one gradient step and the mask token as the reference value. To see whether this approximation is sufficiently accurate to identify good amino acids intervene on, we compare it's performance against occlusion itself in table 3 We also compared against some common feature attribution baselines described below:
of Integrated Gradients Sundararaaan
GRADIENT INPUT: Shrikumar al. 2017)
A(x, i, V (1)) T.,
(4)
GRADIENT: Simonyan et al. (2013a)
A(x, i,t)
| (fr()((
(5)
We compared all these methods, well as a baseline of randomly choosing tokens to mask. We took random sample of 50 proteins from the validation dataset, and intervened to increase/decrease each of the 14 biopython concepts one a time, reporting the average change across concepts when masking 5% the inputs table B We can see choosing tokens to mask based on occlusion first order approximation Gradient (Input Mask) also performed very well, with average intervention effect only 5% below using occlusion, while being around 100x faster calculate. On the other hand, simply looking at the magnitude of the gradient performed poorly, even worse than random selection, likely because it doesn't take into account whether said input increasing or decreasing the concept value. Overall we can also see that choosing which tokens to mask is very important for controlled generation, and that with good attribution our interventions are more than twice as effective as randomly selecting tokens to intervene on.
attribution performs the best expected, however
Method
Positive
Negative
Average
Random
0.0312
-0,0125
0.0218
Gradient
0.0173
-0.0201
0.0187
Gradient Input
0.0358
-0.0269
0.0313
Gradient x (Input Mask)
0.0480
-0.0390
0.0435
Occlusion
0.0499
-0.0419
0.0459
Table 3: Average concept change after interventions using different feature attribution methods on our 24M-CBM model. The average column reports the average change in the desired direction.
Based on these results, for all other experiments in the paper use the Gradient (Input Mask) method for our CB-pLM and CC-pLM, while the C-pLM uses random attribution as it does not make concept value prediction which needed for all other attribution methods.
27

---

Published as a conference paper at ICLR 2025
D.2 UNDERSTANDING CONCEPT/OUTPON RELATIONSHIPS
To understand the model's ability to infer biophysical properties, we visualized the weights from the final layer for each concept relation each amino acid. Our findings indicate that the model successfully learns several key biophysical relationships as defined in the BioPython library (Figures 26
and 28
Final Weights
30
8-
110
Wolccoe Atmaiiid
Aniiectie Pote
at pio
Chedge et pay
Now Frectes
Frettes
Joan Frecher
recined
Acceschiity
Figure 26: Visualization of the final layer weights in CB-pLM 24M.
Some key insights include the following:
For both charge at pH 6 and pH 7, acidic amino acids (D, E) have negative weights, while basic amino acids (R, K) have positive weights. The difference charge between pH 6 and pH reflects the biophysical properties of H. it contains imidazole side chain with a pKa of 6.0
and contributes more towards positive charge at lower pH (Figures 27a 276
GRAVY, which defines hydropathy based on the Kyte-Doolittle scale (Kyte & Doolittle 1982,
is consistent with positive weights assigned to A, I, V, F, C, and M amino acids (Figure 27c )
Aromatic amino acids F, Y, and W have high weights for the aromaticity concept (Figure 27d).
Average hydrophilicity uses the Hopp-Wood scale, which differs from the Kyte-Doolitte scale its empirical definitions, assigns higher values R, D, E, and (Hopp Woods (198T). ) Although the scale also assigns lower values to aromatic residues, such as F, W. Y, we find that our model assigns lower weights for other amino acids which could reflect biases in the dataset (Figure 27ch.
Average surface accessibility is defined by the Emini Surface fractional probability (Cock al. 2009), which is a lookup table describing the likelihood that a residue appears on the protein surface. The scale defines low value especially for C and for other non-polar amino acids, though to a lesser extent (Figure 271
Helix fraction, sheet structure fraction, and turn structure fraction are related concepts, defined by fractions of L. F, W. for helix, A, L. M, E for sheet and N, S, G. P. While the weights do not closely match that of the BioPython definitions, note that the weights nonetheless
28

---

Published as a conference paper at ICLR 2025
consistent with amino acids that often have special propensity be part of helical structures, such as A, M, L, E, K. (Leiro et al. 2017) (Figures 28a 28b and 28ch.
Isoelectric point a property closely associated with charge at pH 7, and likewise, we observe low values for D and E, and high values for R and K (Figure 28d). (fd]
Molar extinction coefficient, which quantifies absorption of a protein at 280 nm, emphasizes the contributions of W. Likewise, the reduced vs oxidized terms of the molar extinction parameter differ only in how the oxidation state of C is considered, which is also visible in our model weights (Figures 28f and 2
Instability index is defined by look-up table based the dideptide decomposition of the sequence (Guruprasad et al. 1990). We observe that the model attributes high values to P. which often introduces kinks in the protein structure and disrupts secondary structures (Figure 28g).
We note that the model weights does not always reflect the concept definitions. For example, W is the heaviest amino acid but is assigned a negative value, which likely reflects biases in the dataset. W is typically one of the least common amino acids, and relative frequency may vary by sequence length. However, the model correctly identifies F as contributing to high molecular weight, and A, and towards contributing to low molecular weight (Figure 28h
Hinll layer weight
Final layer weight
-2
-2
LVEWYNCOMS
(a) Charge pH6
(b) Charge at pH7
15
Final layer weight
Final layer weight
40
30
5
20
-5
-10
-20
-15
A LLVFWY TRCAMMSSHHKEE RHKDDGG
A
(c) GRAVY
(d) Aromaticity
Final layer weight
20
---- weight
10
10
-5
--10
-10
E -0
-20
-25
A WLWWWY T RHKDEGP
A WWW.FWWYNOOSSSSTTRKDCROOO
(c) Average Hydrophilicity
(f) Average Surface Accessibility
Figure 27: The weights of concepts to amino acids the last layer CB-pLM 24M.
29

---

Published as a conference paper at ICLR 2025
20
20
Final layer weight
Final layer weight
10
10
-5
-10
-0
-20
-15
A LVVWY COAAMSTRRKDRN TRHKDEGP
A FWV NCQMS TRHKDEGP
(a) Helix Fraction
(b) Sheet Structure Fraction
1.5
Final layer weight
10
layer weight
1.0
0.0
0.5
-10
-1.0
-15
-1.5
-00
THHKDEGP
-2.0
A FLVFWY RHKDEGP
(c) Turn Structure Fraction
(d) Isoclectric Point
60
60
Final layer weight
Final layer weight
20
-20
-20
NWWMSYNNOMSTRRT TRHKDEGG
V FWY NCQMS TRHKDEGP
(e) Molar Extinction Coefficient Oxidized
Molar Extinction Coefficient Reduced
10.0
layer weight
Final layer weight
5.0
2.5
0.0
-
-5.0
-7.5
WWV.WYYCOMSTRHKDEE.
AILVFWYNCOMSTRRE
(g) Instability Index
(h) Molecular Weight
Figure 28: The weights for selected concepts to amino acids in the last layer CB-pLM 24M.
30

---

Published as a conference paper at ICLR 2025
E
EXTENDED RELATED WORKS AND DISCUSSION
E. RELATED WORK
LANGUAGE MODELS
Language models are increasingly widespread in their usage, spanning from natural language pro- cessing (NLP) (Achiam et al.
Touvron et al. [023 Team et al. 2023 to specialized fields 2023 and biology (Cui et al. [024 Lin et al. 2023). These models
2023
such as chemistry (Bran et al.
have demonstrated exceptional performance intricate tasks, facilitating new frontiers in diverse applications (Brown et al. 2020); Hayes et al., 2024). However, the lack of concrete and effec- tive interpretability methods limits their use in many high-stakes applications, engendering distrust among domain experts, and raising concerns regarding regulatory compliance, safety, and alignment
(Goodman & Flaxman 2017 Gabriel 2020).
Current interpretability research for large language models (LLMs) focuses on pre-trained models, either by explaining the model's predictions (Zhao et al., 2024) or by analyzing the internal cir- cuits of the network through mechanistic interpretability (Räuker et al. 2023 Bricken et al. 2023). Recent work in mechanistic interpretability (Bricken et 2023) demonstrated that sparse autoen- coders (SAFs) could uncover interpretable monosemantic features. To apply this on large-scale production models, involves training very large SAEs, retrieving millions of features, and then trying to assign meaning to these features using autointerpretabiliit approach (Bills et al. 2023).
Although there have been promising results in controlling large production models like Claude us- ing such SAEs, it remains unclear how this approach will help when controlling for concept not found in the SAE or when autointerpretability is not feasible, such as in domains like proteins. Con- cept bottleneck language model (CB-LM) demonstrates similar capabilities (see Appendix C.3.1 for experiments). However, for CB-LMs, there is no need to train an additional SAE and conduct post- hoc search for the meaning of different concepts, each concept's encoding is known by design.
CONCEPT BOTTLENECK MODELS
Concept Bottleneck Models (CBMs) (Koh et al. 2020) incorporate interpretability into neural net- works by inserting an interpretable "concept bottleneck" " layer into the network and mapping the inputs to set of
concepts, which are then used to make the final prediction.
Recently, Ismail et al. (2023) demonstrated that a concept layer can be integrated into generative models (CBGMs). Intervening this layer allows for controllable generation, and examining the activations provides concept-level explanations. Their work focused on image generation and eval- uated the effectiveness of this approach in architectures where the entire input is represented as an embedding, such as GANs, diffusion models, and VAEs.
In this paper, extend this approach language models, demonstrating that we can achieve global control over the entire input while retaining token-level generation. To make this adaptation, we in- serted a concept bottleneck layer into a language model and created novel concept module. This differs from CBGMs in several key aspects: (a) We focus on sequential data where inputs are repre- sented as tokens. (b) We employ a unique model architecture tailored language models. (c) We introduce a new intervention procedure specific to our architecture.
A few very recent works have proposed CBMs for text classification tasks Tan et al. (2024): Sun et al. (2024), but this is much easier and less interesting setting than generative language modeling which we address, as the input and output space of generative models much larger and cannot be adequately modeled using only known concepts, highlighting the need for unknown part in the embeddings.
Recent work Yuksekgonul et al. 20022 Oikarinen et al. 2003 Yang et al. 2023 3) Rao et al. (2024) has also proposed methods to train Concept Bottleneck Models for image classification that do not require labeled concept data by leveraging pretrained multimodal models. While interesting, this approach reduces reliability of concept predictions, and is not feasible in protein setting as no good pretrained general concept predicting models are available.
31

---

Published as a conference paper at ICLR 2025
Similar to ours, previous work has proposed including unknown part in the embeddings to improve task performance in the image classification settings Yuksekgonul et al. (2022); Sawada & Nakamura 2022). but they employ quite different methods to achieve this in difterent setting.
PROTEIN LANGUAGE MODELS
Protein language models are widely used in biological machine-lcarning research, trained on ex- tensive protein sequences spanning the evolutionary tree of life. Although pLMs have numerous critical applications in healthcare and drug discovery (Hie et al. 2024), they currently lack inter- pretability. Their performance is often explained through intuitive arguments about compression and the learning of co-evolutionary patterns (Rives et al. 2019; Lin et al. 2022 Hayes et al. 2024), but it remains unclear whether these models truly capture meaningful protein concepts.
Historically, pL. architectures, loss functions, and training setups have closely mirrored the orig- inal masked language model setup (Devlin et al.) 2018), with the primary differences being the vocabulary and dataset. Our goal is to develop a pLM that offers controllability, interpretability, and debuggability, making more reliable for design applications. While some protein language models support conditional generation by concatenating different protein functions and properties to the in- puts (Shuai et al. [02T]] Madani al. 2020) Hayes et al. 2024), in contrast to our work, they do provide any mechanisms for interpretability, debuggability, or offer any insights on what model
has learned.
E.2 DISCUSSION
CB-LM FOR OTHER DOMAINS AND ARCHITECTURES
In this paper, applied our proposed architecture to masked language models for proteins. This versatile architecture can be adapted to any domain, such as NLP, by simply changing the training dataset, vocabulary, and concepts while keeping the model itself unchanged. Additionally, it can be easily adapted to other types of language models, such as autoregressive models, as demonstrated in Figure 29 For autoregressive models, the embedding output from the transformer, typically used to predict the next token, is passed to the CB-layer to extract the known concept embedding and to an orthogonality network extract the unknown concept embedding. These embeddings are then concatenated and used to predict the next token.
man
man
Softmax
Softmax
Linear layer
Known
Unnknwwn
Transformer
Orthogonality Network
Concept bottleneck module
Transformer
[CLS] am happy
[CLS]
happy
(a) Autoregressive Language Model
(b) Autoregressive Concept Bottleneck Language Model
Figure 29: Inserting concept bottleneck into autoregressive language model.
32

---

Published as a conference paper at ICLR 2025
ADDING NEW CONCEPTS TO TRAINED CB-LM
An intuitive question that might arise is how can control for new concepts. Specifically, if we given a dataset with a new concept, can we fine-tune CB-LM to incorporate it? This can be achieved by creating a new CB-LM with an additional concept and copying the weights from the original model. In this setup, the only randomly initialized weights are for the new concept embedding ex+1, concept classifier head for the new concept, and the linear decoder. The new model can then be trained to match the old model'ss concept output for existing concepts while aligning with the ground truth for the newly added concept.
GENERALIZATION TO NEW CONCEPT COMBINATION
In many real-world applications, we may need generate sequences with previously unseen con- cepts novel combinations of concepts not present in the training dataset. CB-LM provides this capability, as long as the new concepts can be expressed as functions of the existing ones. We demon- strate this capability in a toy task detailed in Appendix C.3.2 where we train masked autoencoder vision transformer (MAE-ViT) (He 2022) with our architecture on a simple color-MNIST gen- eration task. Our results show that the architecture successfully generate blue-colored MNIST digits, despite the model never having encountered the color blue in the training dataset.
33


Reviews:
[{"review_id": "ELLjwfO7OB", "reviewer": "Reviewer_ey43", "summary": "The authors apply concept bottleneck language models to protein design. They propose this model has three benefits; control, interpretability, and facilitated debugging. To support this, they demonstrate evidence that the model can optimize for single or double property control, better than existing conditioning approaches. They showcase this by redesigning the hydrophobic patch of an antibody, showing that it responds to the in-silico metrics as expected. Finally, they correlate model weights with trained concepts, and propose these correlations can help model debugging efforts", "questions": "* Line 245: it appears the time is denoted as a typo \n* Table 1: how was the validation set chosen, is there overlap between the validation set and LBSTER/ESM2 train sets? \n* Section 4.1 \n    * Why was the C-pLM baseline tested with random substitutions instead of feature attribution. \n    * Model size does not appear to improve intervention accuracy or change, significantly. \n* Section 4.2\n    * What is the motivation for using WSJ here. Why not use ESM2 or an inverse-design approach like ProteinMPNN or ESM-IF, for sequence redesign in the hydrophobic region? \n    * It seems useful to include a non deep-learning baseline here that at minimum randomly re-samples residues in the hydrophobic patch with hydrophilic residues. \n    * Is the hydrophobic patch critical for the function of the antibody? How do you know if the redesigned region has improved or reduced function. \n* Section 5: \n     * Is it not expected that the model should correctly learn or not learn the concepts provided during training? \n     * It is also clear in what application the model debugging of a CB-pLM might be relevant, based on the given example.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I believe this CB-pLM is the first model which applies concept bottleneck models to protein design, the authors show with strong evidence that CB-pLM effectively shifts concept distributions better than existing approaches such as C-pLMs and CC-pLMs. The paper is well written, and does a good job at showcasing the in-silico results for conditioning.", "weaknesses": "A limitation of this approach is that biophysical concepts need to be explicitly defined for controlling design. The argument that a CB-pLM is more interpretable because of the training approach is less convincing, as the results appear to be evidence that the model learns the provided concepts correctly, and doesn’t expand to a new interpretation of known biological properties. Concepts used in the model are also straightforward to calculate and interpret the outputs of with bioinformatics tools, alone. The model does not provide additional interpretability on proteins that otherwise exists.  Similarly, while the redesign example appears to currently be tested in a lab, it is not possible to asses if the model improves the solubility and function of  Siltuximab, based on the data provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "irRpH7WBvX", "reviewer": "Reviewer_QrUQ", "summary": "This paper proposes a Transformer Masked Language Model (MLM) variant of concept-bottleneck generative models (CBGM) with three goals in mind: controllable generation, interpretability, and debuggability. This paper adapts the training objectives of CBGM to the masked language model architecture, and proposes a control method specifically for their CB MLM. Evaluation was conducted on the domain of protein sequence modeling, for controllability, interpretability and debuggability. The proposed model leads to improvements over non concept-bottlenecked baselines on controllability. The proposed model demonstrates interpretability and debuggability via inspection of the weights of the linear decoder head. \n\nComment: I have experience with language modeling but not with protein domain, so my review will focus more on the method part of the paper (as recommended by AC). Please take this into account when considering my review and recommendation.", "questions": "Q1 How does compute / training vary across C-pLM, CC-pLM, and CB-pLM. Is total compute/wallclock time controlled (e.g. 24 hr, as the cited cramming paper) or perhaps early stopping on validation is used?\n\nQ2 It is interesting that there is negative scaling happening in figures 10 and 11, with sometimes large decreases in performance as the model becomes larger. Out of curiosity, do authors have any speculations about why this happens?\n\nQ3 In lines 142-144, “This enables us to answer [… q1…], or [q2] counterfactual questions like if we want to decrease hydrophobicity, which amino acid should we replace “E” with?”. I understand q1 is an interpretability question and linear decoder does help with that. For q2 however, I am not sure I fully understood. Is it accurate to rephrase it as “if we intervened on the concept strength of hydrophobicity, which amino acid would the decoder output?” My confusion is on the verb “enables” – I believe the linear layer gives you a more interpretable answer to q2 (say compared to a nonlinear decoder), but you would still be able to answer q2 with a nonlinear decoder,  by intervening on the concept value $\\hat{c}_i$ and then running a forward pass of the decoder.\n\nQ4 What’s the spread along the x-axis on figure 4b? Is it naturalness? It might be helpful to also plot the reference lines for no-change (for both naturalness and intervened property) as you do in figure 4a.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* Adapting concept-bottleneck generative models to masked language models is an interesting and novel idea.\n* Using integrated gradients to find tokens to resample to is an interesting and novel idea.\n* Writing is generally easy to follow.\n* This work focuses on improving controllability, interpretability, and debuggability of language models (and protein sequence in particular as an example) which is an important topic with large potential impact.", "weaknesses": "Method\n\n(considering a broader application of the proposed method to general language/sequence modeling, as authors mentioned in the abstract) \n\nM.1 The model proposed is only a conditional generative model given a partial sequence rather than a full generative model of an entire sequence. Classical masked language models (like BERT) do not model the joint distribution over sequences [1]. There are ways to derive joints from MLMs by making their conditionals consistent with one another (see, e.g. [1] and the related works cited there), or by modifying the training objective so that the conditionals are encouraged to be consistent during training (e.g. [2] [3]). Other protein models based on MLM (e.g. [3], which the authors cited) have done the latter kind of training modifications to enable sampling full sequences. I'm not sure how this limitation matters for protein modeling, could authors give more discussions about this? \n\nM.2 Naturalness of the controllably generated sequence is not considered during intervention, but only used as a measurement afterwards. See point below for consequence.\n\nM.3 The proposed controllability technique only proposes changes local to a particular known sequence x, by changing n% (5 in experiments) of x at a time. Again, I cannot comment on the significance of this limitation for protein modeling. For other modalities like images and language, it can be difficult to change sequence-level concepts via chaining a sequence of such local changes. Even if it is possible, since naturalness is not considered as a factor during intervention but only afterwards, applying a sequence of interventions could lead to going outside of the training distribution (e.g. the interventions produced in Figure 20 goes outside of the color-MNIST training distribution, where all digits are uniformly colored). Empirically though, (Fig 4b 15 16) the proposed method seems to outperform baselines so maybe this is less of an issue in practice, for the concepts in this paper and/or for the protein domain.\n\nExperiments\n\nE.1 The model is trained with 25% masking, but is used to generate with 5% masking in the experiments. This could make the model worse because it’s a different input distribution. Is there a particular justification for masking 5%?\n\nE.2 When evaluating controllability against baselines (and more specifically CC-pLM, since it's closer to CB-pLM), it’s not clear whether the gain comes from intervening at better positions (due to applying integrated gradients to a different architecture), better sensitivity of the model to interventions (due to the proposed concept bottleneck), or a combination, making it hard to evaluate the limitations / benefits of the architecture versus the intervention technique.\n\n[1] Hennigen and Kim, 2023. Deriving Language Models from Masked Language Models.\n\n[2] Germain et al., 2015. MADE: Masked Autoencoder for Distribution Estimation.\n\n[3] Hayes et al., 2024. Simulating 500 million years of evolution with a language model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L9rILVAWSt", "reviewer": "Reviewer_e4sF", "summary": "The paper introduces a novel protein language model designed to provide predictions that can be interpreted by human experts without compromising performance. The proposed model, dubbed Concept Bottleneck Protein Language Model (CB-pLM), empirically proves to be easier to control and interpret over previously proposed State of the Art Protein Language Models.", "questions": "1. Line 76, what are “human understandable concepts”? This is not very well defined in the paper and it is left vague. Similarly, what does debuggable model mean? Even if the proposed model leverages a bottleneck to force some representations to align to a set of pre-specified “concepts” the whole model before the bottleneck is a complete black-box. Does debuggable mean that the user can inspect the black box before the bottleneck too? \n2. In the abstract and intro it is mentioned that the proposed model has been scaled to 3B parameters, however there are no empirical results showcasing models larger than 150M (most of the results are reported on the small 24M model). Furthermore, in Table 1 the comparison with other State of The Art models suggests that the proposed model regresses in perplexity over similarly sized non-interpretable models. Can the authors provide an apples to apples comparison with other public models at scales larger than 150M?\n3. Is the current method supposed to scale to a large number of protein sequences? Will it generalize in the case of mislabelled or unlabelled samples? Given annotation cost, a model capable of leveraging partially annotated/unlabeled samples is more likely to scale.\n4. What is the maximum context length (length of the protein sequence) that the model supports?\n5. It is not clear how the concepts in the bottleneck layer should be defined and why the model needs to learn to associate specific concepts to “disentangled” representations in one-to-one relation with the concepts bank. From the paper it appears that the concepts are annotated and hence the “factor of variations” are known at training time. This could become problematic since scaling on annotated data is not always possible. Does the proposed method allow one to not directly observe the factors of variations and still learn some minimal and sufficient concept bank, similarly to the method proposed in [1]? Furthermore, when annotations are used, are the learned concepts compositional? \n6. Experiments in section 4.1.1 and 4.1.2 have been conducted on ablations of the proposed method. Why is it not compared with other State of The Art methods of similar sizes?\n\n\n[1] M. Fumero et al., “Leveraging sparse and shared feature activations for disentangled representation learning”, NeurIPs 2023.\n\nMinor:\n- Line 121, typo.\n- Line 125, labels in match with Figure 2 even if they are not related.\n- Line 226 repeated.\n- Line 245, typo", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "Developing and testing new models for protein modeling is an important problem. In particular, having methods that can be interpretable and that can allow better control for drug design is a high impact project.", "weaknesses": "The paper contains many typos, it is meandering and unfocused, and generally is very hard to read (with many repetitions and vaguely defined concepts).  \n\nFor example, what does “debuggable model“ mean? This is stated as one of the main contributions of the proposed architecture, hence the reader expects it to be clearly defined and thoroughly assessed through empirical validation. In the current version of the manuscript this is not properly defined and hence not convincingly validated with experiments. \n\nWhat are the novel contributions specific to the Protein Language domain? Can the authors separate the specific novelty introduced for their protein language model? What are the main differences with the previously proposed concept Bottleneck Generative Models? If the main contribution of the paper is to test whether CB-LMs work on the Protein domain then it would be good to devote more space in the paper to the empirical validation with a thorough comparison with other State of the Art models at different scales.\n\nThe discussion section briefly mentions some properties of the model that have not been tested in the main empirical section of the paper. It would be good either move the strong empirical evidence supporting concept composition and generalization to novel concepts (added with fine-tuning) in the main or, if results are not convincing enough, remove the claim. Similarly, in the conclusions (and abstract) it is mentioned that the current model has been tested up to the 3B scale and compared with other public models. However I did not manage to find evidence of experiments on scales larger than 150M (see for example Table 1 in which the proposed method does not perform comparably with others).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hb2XJ2FB26", "reviewer": "Reviewer_i9uk", "summary": "This work applies concept bottleneck models to protein language models. The approach uses fairly standard ideas in concept bottleneck models, but they are well-executed and lead to novel applications in e.g. controllability.", "questions": "There are some minor typos e.g. \"00:01 01:15\" in Section 3.1, as well as a missing line break before \\textbf{Scaling} in the same subsection.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The controllability experiments are fairly convincing, though the stated application on debugging is fairly cursory and could be improved (the intervention to test debugging is fairly blunt, and also the results are just hand inspection and thus not reproducible). However, for me the controllability experiments are enough to be a solid contribution.\n\nThe perplexity results were moderately convincing, though more discussion of comparisons to LBSTER and ESM2 would be useful -- it was not clear to me for instance if your method was given extra information (the tags) that LBSTER / ESM2 did not have. It was also strange that the largest model had worse perplexity.\n\nThe writing is overall good and easy to follow, but I felt that the level of detail in some experiment descriptions was not enough to be fully reproducible from the text.", "weaknesses": "See above (strengths and weaknesses are merged)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Zqds0BC5v", "reviewer": "Reviewer_AGed", "summary": "This paper takes the approach called concept bottleneck language modeling and it applies it in the protein language model context.  This provides creative control in terms of human-interpretable biological concepts, as well as the ability to debug models.  The proposed approach appears to work better than other generic approaches to controllable generation such as ProGen/CTRL, as demonstrated in experimental comparison with other models trained by the researchers themselves.  A case study with redesign of siltuximab shows the ability to quite precisely control certain properties, though no wet lab experiments are shown to verify.", "questions": "If even a small wet lab experiment is possible, I would be super curious to see the result.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "Human-interpretable creative control is a super key in generative AI broadly, but especially in protein design.  As such, the proposed approach based on the concept bottleneck is quite compelling.  The computational experiments provide evidence that the proposed architecture is effective.  The paper is written in a clear way.", "weaknesses": "No wet lab experiments were done to verify results.  This would strengthen the paper considerably.\n\nComparisons to other architectural approaches are based on the authors' own implementations, rather than by external research groups that may be motivated to optimize the other architectures more.  Notwithstanding, the performance differences are quite considerable.\n\nNo discussion on biosecurity risks is given.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)