Low rank adaptation of chemical foundation models generate effective odorant representations
Authors: Grant D. McConachie
Emily Duniec
Florence V. Guerina
Meg Younger
Brian DePasquale
Venue: ICLR 2026 Poster (2026)
Keywords: chemical foundation models
protein foundation models
low rank adaptation
olfaction
multi-modal model
computational neuroscience
Primary area: foundation or frontier models, including LLMs
TL;DR: We benchmark chemical foundation models for odorant-receptor binding prediction and introduce LORAX, a LoRA-based cross-attention model that outperforms existing approaches and yields more informative odorant representations.

Abstract: Featurizing odorants to enable robust prediction of their properties is difficult due to the complex activation patterns that odorants evoke in the olfactory system. Structurally similar odorants can elicit distinct activation patterns in both the sensory periphery (i.e., at the receptor level) and downstream brain circuits (i.e., at a perceptual level). Despite efforts to design odorant features to better predict how they interact with the olfactory system, there is still no universally accepted approach to this problem. We demonstrate that feature-based approaches that rely on pre-trained foundation models to generate odorant representations $\textit{do not}$ significantly outperform classical hand-designed features on odorant-receptor binding tasks. Instead, we show that it is necessary to fine-tune these features to increase predictive performance. To show this, we introduce a new model that creates olfaction-specific representations: $\textbf{L}$oRA-based $\textbf{O}$dorant-$\textbf{R}$eceptor $\textbf{A}$ffinity prediction with $\textbf{CROSS}$-attention ($\textbf{LORAX}$). We compare existing chemical foundation model representations to hand-designed physicochemical descriptors using feature-based methods and identify large information overlap between these representations, highlighting the necessity of fine-tuning to generate novel and superior odorant representations. We show that LORAX produces a feature space more closely aligned with olfactory neural representation, enabling it to outperform existing models on predictive tasks.

Published as a conference paper at ICLR 2026
Low RANK ADAPTATION OF CHEMICAL FOUNDATION MODELS GENERATES EFFECTIVE ODORANT REPRE- SENTATIONS
Grant D. McConachie Department of Biiomedical Engineering Boston University Boston, MA 02215, USA
Emily Duniec
Florence Guerina
Program in Neuroscience
Department of Biology
Boston University
Boston University
Boston, MA 02215
Boston, MA 02215
gdmac@bu. edu
eduniec@bus edu
fguer na@bu. edu
Meg A. Younger
Brian DePasquale"
Department of Biology
Department of Biomedical Engineering Boston University
Boston University
Boston, MA 02215 myounger@bu.cm edu
Boston, MA 02215
ioddpaas@bbu.. edu
ABSTRACT
Featurizing odorants enable robust prediction of their properties is difficult due to the complex activation patterns that odorants evoke the olfactory system. Structurally similar odorants can elicit distinct activation patterns in both the sen- sory periphery (i.e., at the receptor level) and downstream brain circuits (i.e., at a perceptual level). Despite efforts design odorant features better predict how they interact with the olfactory system, there is still no universally accepted ap- proach this problem. We demonstrate that feature-based approaches that rely on pre-trained foundation models to generate odorant representations do not signif- icantly outperform classical hand-designed features on odorant- receptor binding tasks. Instead, we show that it is necessary to fine-tune these features to increase predictive performance. To show this, we introduce a new model that creates olfaction-speciff representations: LoRA-based Odorant- -ooranttReceptor Affinity pre- diction with CROSS -tttentio (LORAX). We compare existing chemical foun- dation model representations to hand-designed physicochemical descriptors using Teature-based methods and identify large information overlap between these repre- sentations, highlighting the necessity of fine-tuning generate novel and superior odorant representations. We show that LORAX produces a feature space more closely aligned with olfactory neural representation, enabling outperform ex- isting models on predictive tasks.
INTRODUCTION
A pervasive question in olfaction is: 'What is the best way to represent odorants?' In other words, what are the best features represent an odorant that are predictive of how that odorant will in- teract with the olfactory system? The answer to this question has remained elusive due to com- plex structure-odor relationships, where similar odorants can elicit divergent neural responses (Sell, 2006). Early efforts inspired by chemoinformatics focused on sets of hand-selected physicochemi- cal descriptors (molecular weight, ring count, etc.) to best capture trends in neural data (Schmuker jet al. [2077; Haddad al.) 2008a] Haddad et al.] 2008b] Boyle et al.] [0033] Gabler et al. 2013), and these descriptors remain a central tool in olfactory neuroscience (Pashkovski et al. 2020 Yang eet al.) 2023). More recently, deep learning has enabled the extraction of data-driven odorant Iea- tures that have facilitated greater predictive accuracy and a more comprehensive mapping of odor
space (Seshadri al. 2022
Lee et al. 2023 Shuvaev et al. 2024)
"Corresponding author

---

Published as a conference paper at ICLR 2026
Two broad approaches to data-driven odorant featurization have emerged: (1) supervised approach (Figure B1, left) where tailored architectures, such as graph neural networks (GNNs), are used to generate representations, and (2) feaature-based approach (Figure TA and B, center) where chem- ical foundation models, which are self-supervised models that leverage unlabeled data to generate useful representations for post-hoe application, are used generate fixed features for odorants (Shin et al. 2024] Taleb et al. 2024). The latter approach offers promising solution to the challenges of constructing representations, particularly in olfaction where datasets are typically small. Moreover, given the estimated billions of potential odorants (Mayhew et al. (022), self-supervised approaches may be able to effectively leverage large unlabeled molecular datasets create more robust features for odorants not otherwise discernible by other methods.
Despite the potential utility foundation model representations, their application olfaction has been limited and not quantitatively and exhaustively compared to other approaches. In this work, we evaluate several chemical foundation models and find they do not improve predictive performance over classical physicochemical descriptors odorant-receptor binding tasks using solely feature- based approach (Section 3}. Using two statistical shape analysis methods, canonical correlation analysis (CCA) and orthogonal Procrustes analysis, find many of these representations have strong information overlap with physicochemical features, suggesting that self-supervised learning alone does not yield features well suited for odorant-receptor binding prediction.
To address this limitation, we introduce a model that uses a fine-tuming based approach (Figure B. right) to produce novel odorant representations tailored to olfacation. Our model, LoRA-based Odorant-Recptor A Affinity prediction with CROSS-
(LORAX; Figure B). employs low-rank adaptations (Hu et al., 2021) of chemical foundation models to fine-tune their representations over training (Section 4j. We demonstrate LORAX's ' learned odorant representation allows for better performance on adorant-receplte binding tasks and better generalization. Furthermore, we show the learned LORAX representation is more aligned with neural responses, providing insight into the cause of increased model performance.
A
Uiayyuddntt improsoritiin
Chemical Foundation Model
Protein Foundation Model
Rtylnocheiiol performance
Projection
Odnore
Protee cane
Delocant tifecess
to
Chemical representation
Protein representation
Feeture based opancecht
Fite-tuiing sportech
Superatedd oppecech
B
Supervised Approach
Feature Based Approach
Fine-tuning Based Approach (LORAX)
Fully trained GNX
weightt
weights
Affinity Prediction
Affinity Frediction
Affinity Prediction
+
Y
Food weights
Food weights
Raad weights
Figure 1: LORAX presents a new approach incorporate chemical foundation models predict odorant-receptor affinity. (A) General architecture of chemical and protein foundation models. (B) Current approaches, and our proposed approach (right), predicting affinity of -dorant-receptor binding. (C) Putative thought model given our approach and results: we hypothesize that fine-tuning allows for creation of improved odorant representations that outperform existing approaches.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORKS
Physicochemical features to predict affinity. Guo & Kim (2010) predict odorant-receptor rela- tionships using partial least squares by representing odorants with GRIND descriptors (Pastor al.) 2000) and encoding receptors with multidimensional scaling. [Boyle et al. (2013) and Gabler et al. (2013) use optimized physicochemical descriptors to predict receptor activation, employing hierar- chical clustering, and support vector machines and random forests, respectively. Coog et al. (2022) uses physicochemical descriptors of odorants and multiple sequence alignment similarities of recep- tors train random forest that generates predictions. Beewwl et al. (2025) uses molecular docking simulations with K-means clustering of odorants based on physicochemical descriptors to generate affinity predictions.
Supervised approach to predict affinity. Achebouche et al. (2022) develop graph and convo- lutional neural networks predict receptor activation. HladiS al. (2022) and [hithhananda
et al. (2024) use GNNs to represent odorants combined with a protein foundation model to pre- dict odorant-receptor binding. None of these approaches investigate chemical foundation models as odorant featurizers: while Chithrananda et al. (024) and HHadis et al. (2022) incorporate protein foundation models into odorant-receptor affinity prediction, they employ randomly initialized GNN architectures with supervised training rather than pre- rreitrinne models.
Pre-trained and fine-tuned chemical foundation models in olfaction. Shin et al. (2024) use the chemical foundation models MolCLR (Wang et al. (022)) and SMILES Transformer (Honda et al.) 2019) to represent odorants to predict odor percepts. Similarly, Taleb al. (2024) use MoLforrer (Ross et al. 2022) to featurize odorants and predict their percept. Both groups found chemical foun- dation models offer valuable features that can aid odor prediction. However, these groups do not test chemical foundation models predict odorant-receptor affinity and do not utilize as broad selection of chemical foundation models as we do in this work. To our knowledge, chemical foun- dation models have not been applied to predict odorant-receptor interactions, and different chemical foundation models have not been systematically benchmarked to evaluate their effectiveness. Fur- thermore, the LoRA-based fine tuning we introduce and the performance improvements it produces have not been documented for olfactory tasks.
Do CHEMICAL FOUNDATION MODELS CREATE EFFECTIVE FEATURES?
We benchmarked variety of chemical foundation models using three feature-based approaches to examine which representations, if any, enhance odorant-recepto affinity prediction. To do this, compiled four datasets highlighted in the section below.
3.1 DATASETS
Each dataset includes the amino acid sequence of the receptor, a chemical identifier of the odorant (e.g.. SMILES string (Weininger 1988)), and an experimental measurement of odorant-receptor affinity (summary of datasets in Table These datasets span mammals and non- mammals and employ diverse methodologies for assessing odorant-receptor interactions.
Hallem Dataset. Hallem & Carlson (2066) reports the responses of 24 D. melanogaster receptors 110 odorants using electrophysiology, yielding total of 2,640 odorant-receptor pairs. Responses are average spikes per second. We normalize the data by z-scoring all responses, using the mean and standard deviation of the entire dataset (i.e., we do not z-score responses for each receptor individually).
Carey Dataset. Carey et al. (2010) reports the responses of 50 A. gambiae receptors to 110 odorants using clectrophysiology, yielding total of 5,500 odorant-receptor pairs. Responses quantified as the Hallem dataset. We normalize the dataset by z-scoring all responses, as in the Hallem dataset.
M2OR. Mammalian oeorant-r interactions collected from 45 different sources (Lalis et al.
2024|
The dataset contains 771 odorants and 1402 receptors across 16 species totaling 53,444 paired responses. We used a subset of 46,563 odorant-receptor pairs to directly compare LORAX previous models trained on this dataset (Section 4). Each paired response is categorized either
3

---

Published as a conference paper at ICLR 2026
one for responsive zero for non-responsive. We do not include this full dataset in our feature- based model benchmarks (Section 3.3;; for those analyses we instead use the higher-qualin M2OR (EC50) subset described below.
M2OR (EC50). The M2OR dataset has a variety of data quality. Much of the data comes from 'primary' "secondary' screening, which not precise data collected in the Hallem and Carey datasets. For Section [3.3] we use the M2OR 'EC50' data, as it is the highest quality. This subset of the data contains 474 odorants and 503 receptors, yielding a total of 5,834 odorant-receptor pairs.
3.2 BENCHMARKING MODELS
To effectively benchmark a variety of chemical foundation models against standard methods we fifteen different representations: five transformer-based foundation models, three GNN-based foundation models, eight physicochemical descriptors, and randomly generated representation as lower bound (see Appendix for more details). We three feature-based models (i.e., models that are not fine-tuned) to benchmark these representations, highlighted below. For models that incorporate protein foundation model, use ESM (Rives et al. 2021). As we are predominantly interested in odorant representations, we used only one protein loundation model.
Molecule only model (MO). ridge regression model using only odorant representations. This model is trained on every receptor individually and results are averaged over all receptors. This model assesses the effectiveness of the odorant representations alone, testing whether only chemical information can predict receptor activation.
Molecule + protein model (MP). ridge regression mode that uses an odorant representation concatenated with a protein representation to predict affinity. This model tests the ability of the odorant representation to predict binding with additional receptor target information.
ProSmith. We adapt state-of-the-art multi-modal transformer (Kroll et al., 2024) to test odorant representations for affinity prediction. This model takes, as input, a chemical and protein represen- tation and passes them through transformer and gradient boosted decision tree (XGBoost, (Chen) & Guestrin 2016)) ensemble to generate predictions. Importantly, this model does not fine-tune either representation during training.
Further details of these benchmarking models are provided in Appendix B and Performance was evaluated using 5-fold cross-validation with random splits Averaged prediction metrics reported for the Carey, Hallem, and M2OR (EC50) datasets.
3.3 CHEMICAL FOUNDATION MODELS DO NOT IMPROVE ODORANT-RECHES AFFINITY PREDICTION
Benchmarking results are shown in Table I and Appendix G We focus the Carey dataset in the main text, but the observations and conclusions are consistent across the M2OR (EC50) and Hallem datasets well. From these results, draw three conclusions. First, molecular information alone insufficient to do prediction. This conclusion is supported by the MO column in Table We see low R. scores with high variance across folds, indicating that solely using the odorant representa- tions is insufficient for accurate prediction. Both physicochemical and chemical foundation model representations lack sufficient contextual information to predict binding.
Second, incorporating protein information is essential for achieving reliable prediction. This can be seen by the large performance increase from the MO to MP models, showing that incorporat- ing receptor information increases R² scores and decreases variance across folds. Additionally, we obtain very high predictive power using the ProSmith model when both receptor and odorant rep- resentations are combined through transformer. This highlights the importance of multi-modal approaches in this domain.
Foor the M2OR (EC50) dataset, we exclude receptors with fewer than 50 entries when training the MO model, this amount of data is too limited to ensure reliable training.
"The MP MO models are a logistic regression with an fg penalty for the M20R (EC50) datasct, as responses are binary.
We use random stratified split for the MO model on the M20R (EC50) dataset.
4

---

Published as a conference paper at ICLR 2026
Table I: Affinity prediction performance across odorant representations for the Carey dataset for the three models outlined in Section 3.2 Reported as mean coefficient of determination (R²) - standard deviation across 5-fold cross validation. Representation type depicted in the leftmost column. Best performing representations are bolded.
Odorant representation
MO
MP
ProSmith
CATS
0.110 0.227
0.271 = 0.013
0.637 + 0.019
Transformer Physicochemien
MordredDeseripporr
0.178 + 0.261
0.295 = 0.022
0.719 + 0.021
Pharmacophore2D
0.175 = 0.261
0.282 0.018
0.548 = 0.014
RDKiiDessriptors22
0.034 + 0.404
0.264 0.021
0.720 0.026
ScaffoldKeyCaleulator
0.005 = 0.280
0.246 0.016
0.594 + 0.022
ECFP
0.207 + 0.410
0.297 = 0.020
0.664 + 0.034
Roberta-ZInn4480M-102M
0.181 = 0.237
0.295 0.020
0.672 0.030
GPT2-ZZinc40M--87
0.121 = 0.266
0.296 = 0.023
0.668 + 0.027
ChemGPT-19M
-00029 = 1.046
0.285 = 0.028
0.685 = 0.024
MolT5
0.215 + 0.279
0.302 + 0.018
0.676 + 0.022
ChemBERTa-77M--M
0.218 = 0.298
0.290 = 0.019
0.704 + 0.017
gin supervised infomax
0.243 = 0.273
0.303 = 0.021
0.671 + 0.016
GNN
aepeervisdd_dgepree
0.189 = 0.308
0.295 = 0.019
0.687 + 0.027
gin hin_supervised. superviisedccontexpped
0.257 = 0.255
0.303 = 0.021
0.699 + 0.033
random
-0.071 + 0.137
0.294 + 0.023
0.586 0.037
Third, the choice of odorant representation has little effect predictive performance. We performed a one-way ANOVA followed by menferroni-correcter post-hoe Tukey test on our ProSmith results. Our analysis showed, with the exception of CATS, Pharmacophore22. and ScaffolK all representations performed equivalently. This surprising, especially given the success of foun- dation models in other domains. One would expect that chemical foundation model representations would provide richer molecular context for predicting odorant-receptor affinity, but given results highlighted here, it seems these representations do not offer important features that enhance pre- dictability.
3.4 CHEMICAL FOUNDATION MODEL REPRESENTATIONS ARE ALIGNED WITH PHYSICOCHEMICAL FEATURES
To understand the failure of foundation models to improve prediction, we analyzed foundation model odorant representations by calculating dissimilarity metrics between each representation (Williams ctt al.) 2022). We embedded all odorants in the Carey dataset into their respective feature spaces and calculated both orthogonal Procrustes distance metrics (Figure 2 and CCA distance metrics (Figure ) between each pair of representations (see Appendix D for more details).
From these distance matrices, particularly the Procrustes matrix, we that many of the odor spaces are aligned. The only two representations that are visibly dissimilar ScaffollKeyCalcullaa and Pharmacophore2D)
the worst performing representations (Table while every other represen- tation forms a block of small distances in the upper left. The odorant representations that are more similar also share similar performance in Table [ for example RDKieDescrpttos2D and Mordred- Descriptors have a very low Procrustes distance of 0.61 and perform almost identically when used in ProSmith. addition to performance trends, this analysis uncovers other patterns present in the representations. For example, the most similar representations, gii_supervised. ciiisedd_onexxireeee and seapeervieed_ddeppeeeee are generated from two instantiations of the same GNN (Hu et al. 2020). We also see that the ChemBERTa- BeeenBBRT-77MM-TTT M7MM representation, a foundation model trained on 77 mil- lion SMILES strings to predict RDKit descriptors, is similar to the RDKiiDescriptors2D represen- tation. Analyzing these spaces, and their dissimilarities, explains why many of these descriptors perform the same: there is heavy overlap of information content, making the representations fairly homogeneous and redundant.
5

---

Published as a conference paper at ICLR 2026
A
B
Orthogonal Procrustes Distance Matrix
CCA Distance Matrix
CNT
CA'S
ome Eeeiass STM
OXF
OFT Dnd Srd Ath
STUNDentedd Hlemcc
SP Jebbmmx
Hutt
EXT
FOR DNIIIOOOD
631
you
ono coeatton
Figure 2: Distance metries between odorant representations in the Carey dataset. Shown are (A) orthogonal Procrustes distance and (B) CCA distance metrics. See Williams et al. (2022) and Ap- pendix D for details on these metrics.
LORAX: FINE TUNING CHEMICAL FOUNDATION MODELS WITH LORA
Given the observed redundancy among chemical representations, we developed a model that is able to fine-tune, and adapt, these chemical foundation models to create an olfaction- olfaction-tailored tailored represen- tation. All models tested in Section 3 are feature-based relying on fixed chemical (and protein) representations rather than adapting them through fine-tuning. A key advantage of fine-tuning is that it produces a new, task-tailored representation rather than depending solely on pre-trained em- beddings. Prior work has demonstrated that fine-tuning is generally more effective than feature- based methods (Devlin et al. 2019), and adapter techniques, such LoRA (Hu et al. 202I), enable efficient fine-tuning of large models. Given that many of the foundation model representations have large information overlap, predicted that refining these representations would yield more informative and task relevant features for olfactory tasks. Motivated by these considerations, we introduce I.RAA (Figure 3.
LORAX is a multi-modal transformer that incorporates a protein and a chemical foundation model to predict affinity (Figure B.. 8) LORAX uses LoRA to update the chemical representation over training. The protein and chemical foundation model can be any model on Huggingface, but for all analysis in this section, we use ESM (Rives et al., 202T) to represent receptors and ChemBERTa-
77M-MTR (Ahmad et al. 2022) to represent odorant: We take inspiration from Kroll et al. 22024) and incorporate an ensemble XGBoost models make final predictions (Figure Bb). Each XG- Boost model in the ensemble uses a distinct set of input features: (1) the transformer's 's penultimate representation (referred to as the <cls> token), (2) the concatenated original chemical and protein foundation model representations, and (3) the concatenation of the <cls s> token with the original chemical and protein foundation model representations. Training proceeds in two stages. First, the multi-modal transformer (including the LoRA parameters) is trained and tuned on the training and validation data, refining the <cls> token representation. After this stage, the transformer is frozen and the resulting <cls> token, together with the fixed cundation-modde embeddings, are used as input features to train the XGBoost ensemble, which makes the final predictions on the held-out test data. Architecture and training details are available in Appendix F
WW also test LORAX with Uni-Mol2-84M Ji et al. 2024 the chemical foundation model and show these data in supplementary Table S8.

---

Published as a conference paper at ICLR 2026
A
B
OJCAHHIICCHHHL.C0000000
Original representation
LoRA
Chemical Foundation Model
Protein Foundation
Original representation
Model
XGBoost
Fine-tuned representation
Original representation
Original representation
<dls>
Original representation
Cross
XGBoost
Attention
W
<cls>
Black
XGBoost
<dls>
W
W
Final Affinity
Affinity
Prediction
Prediction
Figure 3: LORAX model. (A) A multi-modal LoRA adapted transformer model with at- tention block for odorant-receptor prediction. r is the rank of the low rank adapter matrices. (B) An ensemble XGBoost models used to enhance performance while also providing model in- terpretability. weighted sum these XGBoost models is used to generate the final prediction. 'Original representation' means the non-LoRA fine-tuned foundation model representation. Orange represents the protein foundation model and blue represents the chemical foundation model.
4.1 LORAX CONSTRUCTS A BETTER ODORANT REPRESENTATION
We trained LORAX on the Carey dataset and found it performs almost identically to ProSmith (LO- RAX RR: 0.712 + 0.032, ProSmith with ChemBERTa-77MMMTT R²: 0.703 = 0.017, p=0.264 paired I-test, data not shown). However, when interrogated the trained models, we uncovered nuanced differences between them. Because ProSmith and LORAX have similar architectures and both employ identical XGBoost ensembles, direct comparison between them is possible. When examining the ensemble weights (Figure 4B, S1), we found that ProSmith places high confidence in the XGBoost model that only the original chemical and protein foundation model repre- sentations. In other words, the additional representation produced by the transformer (the <cls> token) largely ignored. This supported by the relatively weak standalone validation accuracy of the multimodal transformer model, prior to the XGBoost ensemble, (Figure A gray) showing ProSmith's 's transformer lacks predictive performance. When we compute these metrics for LORAX (Figure 4 blue), we see the validation accuracy of its multi-modal transformer is much higher and weights placed on XGBoost models without the <cls> s> token are lower. conclude that LORAX extracts more relevant information for the task from the protein and chemical foundation models.
With this context, next examined whether the differences in representation weighting between ProSmith and LORAX translate into improved odorant representations across different scenarios. We first wanted to assess the generalizability of LORAX. Using the Carey dataset, we trained both LORAX and ProSmitt on two scenarios: (1) generalize to unseen receptors (i.e., receptors in the test set never set never appear in training). As shown in Table LORAX exhibits superior generalization to un- odorants ProSmith. Although this improvement is not statistically significant (p=0.069, BH-corrected Friedman test (Table S5)), it indicates that LORAX representations may capture more chemically relevant features for odorant-receptor binding prediction. In the unseen receptor sce- nario, LORAX and ProSmith perform equivalently (000, (Table S6)). Overall, while gener- alization for both models remains challenging. LORAX notably outperforms the naive baseline,
training), and (2) generalize unseen odorants (i.e., odorants in the test
'We train ProSmith with the ChemBFRRIa77M-MITY model for appropriate comparison.

---

Published as a conference paper at ICLR 2026
model predicting the mean value of the training set, in both scenarios (p=0.069 (Tables S5 and S6)) while ProSmith fails to outperform the naive model on the unseen odorant task (p=1.000 (Table S5)).
A
Vaiiaaion Representation
0.-
0.4
0.2
0.3
B
po. proportion per Representation
1.0
02-
0.0
sumerben coteesped
saperited Mippoeed
Carr
ANG suerronn intennae
Cenmditi THELATT
Clmmart
Horg
Rebente Zen Aette
Figure 4: LORAX and ProSmith representation comparison after training on the Carey dataset. (A) Validation 122 scores for ProSmith and LORAX prior to applying XGBoost. (B) The weight assigned the XGBoost model that takes only the original chemical foundation model and protein foundation model representations.
Table LORAX, ProSmith, and naive model performance on the Carey dataset under two general- ization settings. The na've model predicts the mean of the training set for all values in the test set. Shown are R scores for the test set for each random split. Bolded are the best performing models.
Scenario
Method
rand split
rand split
rand split
rand rand.spiit_
rand split 5
avg
std
Unseen odorants
ProSmith
-2.098
-0.812
0.364
0.246
0.288
-0.402
1.064
L.RAX
-1.039
0.201
0.472
0.292
0.298
0.045
0.614
Naive
-0.037
-0.051
-4.924
-0.050
-0.020
-1.016
1.954
Unseen receptors
ProSmith
0.072
0.246
0.263
-0.137
-0.003
0.088
0.169
LORAX
-0.032
0.051
0.031
0.133
0.117
0.060
0.067
Naive
-0.060
-0.096
-0.019
-0.129
-0.059
-0.072
0.037
To further interrogate LORAX, we examine how the model performs relative to other models on the
full M2OR dataset, a larger dataset that presents greater diversity and complexity, offering a more rigorous test of model performance. We utilize the weighting scheme outlined in Hladis et al. (2022) to account for differing data quality. As shown in Table 3 LORAX demonstrates superior perfor- mance compared both Hladiš et. al. and MolOR, yielding a statistically significant improvement over the latter (p<00.003, MCC across split<0] (Table S7)). LORAX also outperforms ProSmith, which is evaluated here on M2OR for the first time (p=0.067, MCC across splits (Table S7)). While ProSmith achieves higher AUROC, LORAX demonstrates improvements in MCC and F-score
A statistical comparison against the model from Hladiš al. 2022 could not be computed due to lack of access per-split metries.
8

---

Published as a conference paper at ICLR 2026
ofate-of-the-art protein-molecut molecule interaction models not previously tested on M2OR. Given the large class imbalance in the M2OR dataset, where the proportion of responsive data points is much lower than that of non-responsive ones (Table S1), MCC and F-score provide more accurate sessment of model performance than AUROC. This confirms that fine-tuning chemical foundation models offers important avenue to improve odorant-receptor prediction.
Table 3: Comparison of MolOR, Hladiš et al., ProSmith, and LORAX across multiple metrics trained on the M2OR dataset. LORAX (C) fine-tunes only the chemical foundation model; LO- RAX (P + C) fine-tunes both chemical and protein foundation models. Values for Hladiš et al. are taken from their publication. Uncertainties for all models are standard deviation across 5 fold cross validation. Bold indicates highest value in each column. n/a entries indicate that the authors did not report that metrie. AUROC: area under the receiver operating characteristic curve, AveP: average precision, MCC: Matthews correlation coefficient.
AUROC
AveP
Precision
Recall
F-score
MCC
MolOR (Weighted)
76.12 + 1,53
0.626
0.05
0.507 = 0.05
0.727 = 0.06
0.595 = 0.02
0.467 = 0.03
Hladiš al.
0.780
0.01
0.689 0.02
0.698 + 0.04
0.693 0.02
0.605 0.02
ProSmith
90.47 0.98
0.584
0.03
0.764
0.06
0.671 + 0.06
0.712 + 0.02
0.641 0.03
LORAX CP+
82,43 + 0.86
0.776
0.03
0.729 = 0.06
0.727 = 0.03
0.727 = 0.02
0.650 + 0.03
LORAX (C)
83.24 = 1.34
0.778 0.03
0.710 = 0.05
0.754 = 0.02
0.730 = 0.03
0.651 + 0.04
We also tested a version of LORAX that uses both a LoRA adapted protein foundation model and LoRA adapted chemical foundation model (LORAX (P C)). Interestingly, this model performs slightly worse than the model with only chemical foundation model fine-tuning, even though LO- RAX + C) has ~2 million mainly from LORAX's improved chemical representations. while the original protein representa- tions are already sufficient. In summary, we found that LORAX fine-tuning produced models with more refined <cls> tokens, translating to improved prediction on M2OR and improved generaliz- ability to unseen odorants.
parameters than LORAX (C). We conclude that the task benefits
4.2 LORAX ODOR SPACE IS SIMILAR TO UNDERLYING NEURAL REPRESENTATION
Given LORAX's promising capabilities, we analyzed the odorant representations produces. We
plot CCA distances of LORAX's odor space for the Carey dataset in Figure 5
LORAX ChemBERTe-77M-MTR Representation Others
LORAX
0.98
1.05
1.0
Distance
0.9
ChemBERta-77WW
0.95
0.85
0.87
0.90
0.92
0.73
0.88
0.08
0.90
0.91
1.00
neural apressntation
gin superveed edgepred
ChenGPT-100 19M
Roberta Zine ZI0c4R0M-300 102M
GPT ARRc4800--874
CATS
MordredDescrjptors
gin supervised contoxipred
gin Supervised infomax
ecfp
M
Scaffoluley Calaolaae Salculator
Pharmacophoree20
Figure 5: CCA distance matrix showing the LORAX and ChemBERTa-77M-MN representations on the Carey dataset. 'neural representation' encodes each odorant as a feature vector based on the responses elicits from each neuron. Red dashed lines indicate transformer based foundation model representations, blue solid lines indicate GNN-based foundation model representations, and orange dotted lines indicate physicochemical representations. Full distance matrices shown in Figure

---

Published as a conference paper at ICLR 2026
Additionally, for this distance matrix, we include neuraalrrprrseettttin in which each odorant is represented as a vector of the neural responses it elicits (more details in Appendix E). We compare the LORAX representation with DeemBERTa-77M-MEE ing LoRA during training, for the Carey dataset. The ChemBIRRTa-77MMMMTR representation can be considered the initial state before fine-tuning while the LORAX representation represents the final state after adaptation.
the foundation model that is adapted us-
We found the LORAX representation is more dissimilar from many of the physicochemical de- scriptors than ThemBERRTa77M-NNN and more similar to all graph based methods (ECFP and all the GNN foundation models). Additionally, the LORAX representation is more aligned with the neural representation than the ChemBERTa-77M-MTP representation. These analyses reveal that LORAX generating a unique representation, distinct from the original foundation model, that is better suited to describe odorant-receptor relationships and therefore generating more informative odor space.
5 DISCUSSION
Identifying representations that best characterize odorants for olfactory prediction has been ma- jor research focus. We systematically evaluated representations from several pre-trained chemical foundation models and found they did not provide an improvement over hand-tuned physicochemi- cal descriptors for odorant-receptor binding tasks using multiple feature- approaches. To move beyond this limitation, we introduced a new model, LORAX, for fine-tuning chemical foundation models to produce richer and more powerful representations. Collectively, we present two novel and interconnected findings: (1) To our knowledge, we performed the first and most comprehensive analysis of chemical foundation models applied to adorant-receptor binding to demonstrate their effective equivalence with physicochemical descriptors; (2) We introduced the first application of LoRA fine-tuning of chemical foundation models for olfaction, demonstrating the improvement that fine-tuning offers. Our results demonstrate that while existing chemical foundation models may capture overlapping information for olfactory datasets, targeted fine-tuning creates specialized and potentially more powerful representations than those obtained from pre-trained models alone. We argue that these models have significant potential for olfactory neuroscience, particularly in address- ing key challenges such as limited data and vast chemical search spaces.
We hypothesize the targeted fine-tuning LoRA provides identifies a 'sweet spot' that balances the strengths of alternative approaches (Figure [C). Supervised approaches, that lack the context of pre- trained models, underperform likely due to overfitting (Hladiš et. al., Table Feature-based models alone likewise underperform because limited initial representation leads to underfitting (ProSmith, Table 3). We hypothesize that the selective nature of LoRA's low-rank assumption enables robust generalization while not entirely erasing the benefits that come from pre-training. To the best of our knowledge, this work presents the first application of LoRA to olfaction, and given its strength and natural approach to balaneing over- and underfitting when it to be a useful tool when working with olfactory data.
pre-trate pre-trained models, expect
There remain many exciting avenues for future exploration with this modeling framework. A domi- nant question in olfaction is how chemical features produce olfactory perception (Zhou et al. 2018
Seshadri et al.) 22022] Lee et al.] 2023 Yang al., 2023] Qian al., [022]] Shuvaev et al. 2024). Shin et al. (0224) and Taleb et al. (2024) found that pre-trained models with non parameter-
or no fine-tuning improved performance on perception tasks; our results suggest further improve- ment could be achieved with LoRA- fine-tuning, Additionally, LORAX is modular by design, allowing for the interchange of protein and chemical foundation models. While our analysis utilizes (Ahmad et al. 2022), the field of chemical foundation modeling is rapidly advancing with larger and more diverse models. Systematic benchmarking and further training with these models may yield even more optimal odorant representations.
ChemBERRTa-77M-VVT
Despite our progress, several important questions remain. For example, how robust are the LORAX- leamed representations when transferring to different olfactory datasets? Which components of LORAX contribute the most to prediction performance? Why does simultaneous fine-tuning of both protein and chemical foundation models hamper performance? We leave these to future work, with the anticipation that continued research will deepen our understanding of how chemical foundation models can be effectively leveraged in olfaction.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
All datasets used in this work are freely available and from published work linked in the text. Code of the benchmarking work in Section 3 is located here, and code for the LORAX model is located here. Data used in the work can be downloaded from links in either README. Default configurations and training regimes are outlined in the main text and appendices for all models used in this work. Statistical shape analysis methods in this work are freely available use here.
USE OF LLMS IN THIS WORK
We LLMs aid and polish writing throughout the text.
ACKNOWLEDGMENTS
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No 2234657. Any opinions, findings, and conclusions or recom- mendations expressed in this material those of the authors and do not necessarily reflect the views of the National Science Foundation. GDM acknowledges NIH G32-GM008764.
REFERENCES
Rayane Achebouche, Anne Tromelin, Karine Audouze, and Olivier Taboureau. Applica- tion of artificial intelligence to decode the relationships between smell, olfactory recep- tors and small molecules. Scientific Reports, 12(1):18817, November 2022. ISSN 2045-
2322. doi: 1033/0/0/01188-000000 159-402--1--0222177666.. URLhttps: htops://wwww... www..nature www.e/o/aaattt.ee 020-988-22-2117711
Publisher: Nature Publishing Group.
Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar.
ChemBERTa-2: Towards Chemical Foundation Models, September 2022. URL http://
arxiv. wtw.l//.rrgabb////a 220..a////22..... 01712 arXiv:2209.017 001712 [es].
Bhavika Berwal, Pinaki Saha, and Ritesh Kumar. A Fully In Silico Protocol to Understand Olfactory Odooatt Interactions. ACS Omega, (0123):4403-244444 June 2025. ISSN 2470-1343, URL. https://pubs: acs wot.c//iooo/o///////oo//////////
2470-1343. doi: (0c002//cssmm
1021 1021/acscmega 4c08181
Sean Michael Boyle, Shane McInally, and Anandasankar Ray. Expanding the olfactory code by in silico decoding of odor-receptor chemical space. eLife, 2:c01 October 2013, ISSN 2050- 084X. doi: 10.7554/eLife.01122 120. URL Ttct::://0o..org/11..555... 7554/ 01120 Publisher: eLife Sciences Publications, Ltd.
Allison F. Carey, Guirong Wang, Chih- Su, Laurence Zwiebel, and John R. Carlson. Odor- ant reception in the malaria mosquito Anopheles gambiae. Nature, 464(7285):66 March 2010. ISSN 1476-4687. doi: 10.10088/atur0088334. URL htttt:tt////t //wp:: www. naturrt com/ art articles/naturedue
Publisher: Nature Publishing Group.
Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. In Proceed- ings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 116, pp. 785-794, New York, NY, USA, August 2016. Association for Comput-
ing Machinery. ISBN 978-1-4503-42322-
doi: 10. 145/29999772.9999785
URL https:
7/dl. acm. org doi/10.1111110 1e..0..11442220 (tw.:99977.222.99.998
Seyone Chithrananda, Judith Amores, and Kevin K. Yang. Mapping the combinatorial cod- ing between olfactory receptors and perception with deep learning. September 2024. URL
https: www. biorxiv..v... www.oorg/oooo
09..24.....66.13338883.. 11.3344v
09
Pages:
Section: New Results.
11

---

Published as a conference paper at ICLR 2026
Xiaojing Cong, Wenwen Ren, Jody Pacalon, Rui Xu, Lun Xu, Xuewen Li, Claire A. de March, Hiroaki Matsunami, Hongmeng Yu, Yiqun Yu, and Jérôme Golebiowski. Large-Scale Scale G Protein- Coupled Olfactory Seceptor-Liganee Pairing. ACS central science, 8(3)339--37,, March 2022. ISSN 2374-7943. doi: 10. 102 1021/acccettssi.cclll c01499..
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. May 2019. URL http::
org/abs /1800//11000 04805 #xXiv1181004800 [cs].
Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between Molecules and Natural Language.
Nooegueee.Noemberr November 2022. URLL/ttt///axvv.or//////
22204.18111 MrXiv:220.11811 [cs].
Peter Ertl.
Identification of Bioisosteric Scaffolds using Scaffold Keys, February 2021.
URL
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gomez-Bombarelli, Connor Coley, and Vijay Gadepally. Neural Scaling of Deep Chemical Models, May 2022. URL chttps://chemete http:///wwwrreeeegge//ccee/////e 627bdddd5444ddd322233333
detailss
Stephan Gabler, Jan Soelter, Taufia Hussain, Silke Sachse, and Michael Schmuker. Physicochemical Vibrational Descriptors for Prediction of Odor Receptor Responses. Molecular Informatics, 32(9- (199-10):5558653
October 2013. ISSN 1868-1743. doi: 10.1002/minf:201 300003..
Sheng Guo and Junhyong Kim. Dissecting the molecular mechanism of drosophila odorant receptors through activity modeling and comparative analysis. Proteins, 78(2):381-399, February 2010. ISSN 1097-0134. doi: 10. 20.600/60r0..2556.
Raf Haddad, Rehan Khan, Yuji K. Takahashi, Kensaku Mori, David Harel, and Noam Sobel. A metric for odorant comparison. Nature Methods, 5(5):425 429, May 2008a. ISSN 1548-7105. 1197,
doi: 10. 10.. 1 1197. URLpttts::/ www www.ranttee... wtt.:/uree..o//tttcleeeeeeeeeeee Publisher: Nature Publishing Group.
Raf Haddad, Hadas Lapid, David Harel, and Noam Sobel. Measuring smells. Current Opinion in Neurobiology, 18(4):438 444, August 2008b. ISSN 0959-4388. doi: 10.1016/j.comb.2000 007.
Elissa A. Hallem and John R. Carlson. Coding of Odors by a Receptor Repertoire. Cell, 125 (1):143-160, 160, April 2006. ISSN 0092-8674, 1097-4172. doi: 10. 101.1116/...lll......
2e06.000.0..00000000000 URL
https::/wwi cell. com ce cell/ http://attttttttoaaaaaaaeaaatttt 8674 (06) Elsevier.
Publisher:
Karl Heyer. Roberta-zinc-480m, 2023.
Karl Heyer. kheyer/gpt2.zinc 87m, April 2024. URL http::///g.tuub..cm//heesss gpt2_zinc_8 original-date: 2023-05-11T00:52:088
Matej Hladiš, Maxence Lalis, Sebastien Fiorueci, and Jérémie Topin. Matching receptor to
odorant with protein language and graph neural networks. September 2022. URL https:
//openzeview.nen wtwplewwwwee/forroo www.umprd...ggeer..........
Shion Honda, Shoi Shi, and H. Ueda. SMILES Transformer: Molecular Fingerprint for Low Data Drug URL / www. www.sesattiicchhaar..oo.o.
Pre-trained
Discovery.
ArXiv, Novem-
ber 2019.
WWWLESSTraasforree
Pra--rr--taeened Mo lecu ar-
6289471 F2eca0 dbde 7
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: LowwRankk Adaptation of Large Language Models, October 2021. URL arXiv:2106.09668 [cs].
12

---

Published as a conference paper at ICLR 2026
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for Pre-training Graph Neural Networks, February 2020. URL http:
//aaxiv.org/aat wtt..rgg/aaaa...../ .1119905.222.000 12265 arXiv:1905.12266 [cs, stat].
Xiaohong Ji, Zhen Wang, Zhifeng Gao, Hang Zheng, Linfeng Zhang, Guolin Ke, and Weinan E. Uni-Mol2: Exploring Molecular Pretraining Model Scale, July 2024. URL /tthhtp:: http://arxxxii org/abs 4969 arXiv:2406. 14969 [cs].
Damjan Kalajdzievski. A Rank Stabiliization Scaling Factor for Fine-Tuning with LoRA, November 2023. http://////wwxw.c..aaaa////a//// C3732 arXiv:2312.03732 [cs].
Thomas N. Kipf and Max Welling, Semi- Semi-Supervised Caanaaauupprisse CCCaaaaiiattto with Graph Convolutional Net- works, February 2017. URL [es].
02907 arXiv:160..02007
Alexander Kroll. Sahasra Ranjan, and Martin J. Lercher. A multimodal Transformer Network for protein-sman molecule interactions enhances predictions of kinase inhibition and enzyme- substrate relationships. PLOS Computational Biology, 20(5)ee002210 May 2024. ISSN 1553-7358. doi: 10.1377/oourmall.pb.1002222
URL ht ht org/ Publisher: Public
httpppppoooaattceeeeeec 13700....111. journa pcbiiccii....... 1012100 Library Science.
Maxence Lalis, Matej Hladiš, Samar Abi Khalil, Loic Briand, Sébastien Fiorucei, and Jérémie Topin. M2OR: a database of olfactory receptor-odorant pairs for understanding the molecular mechanisms of olfaction. Nucleic Acids Research, 52(D1)D1130-DD179) January 2024. ISSN 1362-4962. doi: 0.110///a///
Brian K. Lee, Emily J. Mayhew, Benjamin Sanchez-Lengeling, Jennifer N. Wei, Wesley W. Qian, Kelsie A. Little, Matthew Andres, Britney B. Nguyen, Theresa Moloy, Jacob Yasonik, Jane K. Parker, Richard C. Gerkin, Joel D. Mainland, and Alexander B. Wiltschko. principal odor September map unifies diverse tasks in olfactory perception. Science, 38 org/ /ttp:/..o//11.../1.. 2023. doi: 10. htt.://ceencc..dee444... ade4401. hhttlttts::///t science. 40
science. Publisher: American Association for the Advancement of Science.
Emily J. Mayhew, Charles Arayata, Richard C. Gerkin, Brian K. Lee, Jonathan M. Magill, Lind- sey L. Snyder, Kelsie A. Little, Chung Wen Yu, and Joel D. Mainland. Transport features predict a molecule is odorous. Proceedings of the National Academy of Sciences of the United States of America, (19(1))):211657611 19, April 2022. ISSN 0027-8424. doi: 10. 10.10773pmas.211666 16576111,, URL httpt:://///wt ncbi. nlm. nih.
Hirotomo Moriwaki, Yu-Shi Tian, Norihito Kawashita, and Tatsuya Takagi. Mordred: molec- ular descriptor calculator. Journal of Cheminformatics, 10(1):4, February 2018 ISSN 1758-2946. doi: 10.1 1011186613331--01-05588. s -13321-018-0255
URL https://dci: wrg/10....... 1186/
Emmanuel Noutahi, Cas Wognum, Hadrien Mary, Honoré Hounwanou, Kyle M. Kovary, Desmond Gilmour, thibaultvarin r, Jackson Burns, Julien St-Laurent, DomInvivo, Saurav Maheshkar, and rbyrne momatx. datamol-io/molfeat: 0.9.4, September 2023. URL hwt.::tttttseseteeeee org/
records/837300
Stan Pashkovski, Giuliano Turilli, David Brann, Daniel Chicharro, Kristen Drummey, Kevin M. Franks, Stefano Panzeri, and Sandeep Robert Datta. Structure and flexibility in corti- representations of odour space. Nature, 583(78155:253- 258, July 2020. ISSN 1476- URL htttt:::/// www. nature. com/articles Publisher: Nature Publishing Group
4687. doi:
s41586- 020- 2451-1
Number: 7815
Manuel Pastor, Gabriele Cruciani, lain McLay, Stephen Pickett, and Sergio Clementi. Iedepndnntt Descriptors (GRIND): A Novel Class of Alignment-Independen Three- Dimensional Molecular Descriptors. Journal of Medicinal Chemistry, 43(1))33333333 C 2000. ISSN 0022-2623. doi: 10. 10.1021//m00094 URL Attps: / dei wrr.i.oorg/ll 1021/
August
Jm000941m Publisher: American Chemical Society.
13

---

Published as a conference paper at ICLR 2026
Wesley W. Qian, Jennifer N. Wei, Benjamin Sanchez-Lengelingg Brian K. Lee, Yunan Luo, Marnix Vlot, Koen Dechering, Jian Peng, Richard C. Gerkin, and Alexander B. Wiltschko. Metabolic activity organizes olfactory representations. eLife, 12:e82502. May 2023. ISSN 2050-084X. doi: 10.7554/el.ife.82550
Bharath Ramsundar, Peter Eastman, Patrick Walters, Vijay Pande, Karl Leswing, and Zhenqin Wu.
Deep Leurning for the Life Sciences. O' O'Reilly Media, 2019. https: /ttpppp::/wwww.. /www amaaamaaoooo.. com/
Deep Lea iaaarnnnng-uin
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological struc- ture and function emerge from scaling unsupervised learning 250 million protein quences. Proceedings of the National Academy of Sciences, 1 118(15)ee011223111 April 2021. doi: 10. 10..100pss..201110 20166999..012299911 URL https: /www www.wwpp...aa.......mm..m...a..w pnas
2016239118 Publisher: Proceedings of the National Academy of Sciences.
David Rogers and Mathew Hahn.
Fingerprints. Journal of Chemical In- formation and Modeling. 50(5):742-754, May 2010. ISSN 1549-9596. doi: 10. 10.1021/ci100050 URL https: /
10. 1tt.0////0.111111111111110110000 Publisher: American Chemical Society.
Jerret Ross, Brian Belgodere, Vijil Chenthamarakshan, Inkit Padhi, Youssef Mrouch, and Payel Das. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence, 4(12):1256-1) December 2022. ISSN 2522-
5839. doi: 10.
URL httpstt:::/ :/ww:/.//wwwwww// http://www.......mmm///////// nature. wow//rtticiiio
s42256- 842256-022-00522 022-00500-000077
Publisher: Nature Publishing Group.
Kunal Roy. Pratim Chakraborty, Indrani Mitra, Probir
Kumar Ojha, Supratik Kar, and Rudra Narayan Das. Some case studies on application of "rm2" metrics for judging qual- ity of quantitative structure-activity relationship predictions: Emphasis on scaling of response data. Journal of Computational Chemistry, 34(12):1071-1082, 2013. ISSN 1096-987X.
doi: 10. 23231..
http::://onllnneebrrrrryy. http://llloooo/o//////////////// .eprint: http://ooilbbbbaaayweywoo/oo/ood
L002/jcc.22 23231
Michael Schmuker, Marien de Bruyne, Melanie Hähnel, and Gisbert Schneider. Predicting olfactory receptor neuron responses from odorant structure. Chemistry Central Journal, 1:11, May 2007.
ISSN 1752-153X. doi: 10.1186/1752- 1-11-1-11.
Gisbert Schneider, Werner Neidhart, Thomas Giller, and Gerard Schmid.
"Scaffold- Hopping" by Topological Pharmacophore Search: A Contribution Virtual Screen-
ing. Angewandte Chemie International Edition, 38(19):2894-25) 2896. 1999. 3773. doi: URL
ISSN 1521-
0 C _eprint: http:///wiiiiioiro.eooooooooooop 37739288999100494288
C. S. Sell. On the Unpredictability of Odor. Angewandte Chemie International Edition, 45
(38):6254-621,, 2006. ISSN 1521-3773.
doi: (0010022/miee22000077222 URL https:
doi/ abs/ /1c. 1002/ anie. http:/aiieee...00... 200600782
eprint:
Aditi Seshadri, Heta A. Gandhi, Geemi P. Wellawatte, and Andrew D. White. Why does that
molecule smell?, December 2022. httpstts:///hhemaaa
org/ engage chtee/ieeerr///////
article-detailss (ttpll///3390abbbebaaaaaaaaaaba a
Daniel Shin, Gao Pei, Priyadarshini Kumari, and Tarek R. Besold. Optimizing Learning Across Multimodal Transfer Features for Modelling Olfactory Perception, April 2024. URL https:
// //www. resear www.errccsuuaaa rchsquareeeee....c....... com a rt.
ISSN: 2693-5015.
Sergey Shuvaev, Khue Tran, Khristina Samoilova, Cyrille Mascart, and Alexei Koulakov. Deep- Nose: An Equivariant Convolutional Neural Network Predictive of Human Olfactory Percepts. In 2024 58th Asilomar Conference Signals, Systems, and Computers, pp. 536-543, October 2024.
14

---

Published as a conference paper at ICLR 2026
doi: 00.100//EEECONNFF000222222111100 wBt.ttose//eexxxxoeeeee.e.... aass ISSN: 2576-2303.
Farzaneh Taleb, Miguel Vasco, António H. Ribeiro, Mårten Björkman, and Danica Kragic. Can Transformers Smell Like Humans?, November 2024. URL httt:://rrxiv..rr//aass
2411. 03038 arXiv:2411.03036 [es].
Petar Veličković, William Fedus, William L.. Hamilton, Pietro Liò, Yoshua Bengio, and R. De- von Hjelm. Deep Graph Infomax, December 2018.
10341 arXiv:1809.10344 [stat].
Yuyang Wang, Jianren Wang, Zhonglin Cao, and Amir Barati Farimani. Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3):279- 287, March 2022. ISSN 2522-5839. doi: 01118///2255-022--404-4- x. URL/httss: :ttl:://wwwwe //ww.
Publisher: Nature Publishing Group.
David Weininger. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal Chemical Information and Computer Sciences, 28(1):31-3, February 1988. ISSN 0095-2338. doi: 10. 10.1021/c100057:005 URL https:
//doi.org/Ine
Publisher: American Chemical Society.
Alex H. Williams, Erin Kunz, Simon Kornblith, and Scott W. Linderman. Generalized Shape Metrics on Neural Representations, January 2022. URL http:///aratv.orrgabb//111111 1rXiv:2110.14739 [stat].
Jie- Yoon Yang, Thomas F. O'Connell, Wei-Mien M. Hsu, Matthew S. Bauer, Kristina V. Dylla, Tatyana O. Sharpee, and Elizabeth Hong. Restructuring of olfactory representa- tions in the fly brain around odor relationships in natural sources, February 2023. URL https: www. biorxiv
Pages:
2023.02.15.52862 Section: New Results.
Yuansheng Zhou, Brian H. Smith, and Tatyana O. Sharpee. Hyperbolic geometry of the olfactory space. Science Advances, 4(8) 1458, August 2018. doi: 10.1126/sciadv.aaq URL
www.sccieecccc... https: /tttp:://ww..r sci eence.org /wtpgg/ddo/1.... 1126/ a/scciiaaee aaw. 458 Publisher: Ameri-
can Association for the Advancement of Science.
A DATA REPRESENTATION
Here we describe the various methods we use to featurize odorants and receptors. For odorant featurization, we wanted diverse array of chemical foundation models as well physicochemical descriptors to get a good sense of what features need to go into an odorant representation to be able to effectivity predict how that odorant binds with given receptor. The odorant representations we used are described in Table For all odorant representations in Section 3 we use Molfeat (Noutahi) jet al. 2023) to generate representations. In Section 4 we use Huggingface to generate molecular representations as we focus on finetuning the original models when we train LORAX.
We would also like note that the generation of chemical foundation models is rapidly progressing field, and we realize that some of the most recent chemical foundation models are not present in this study. However, we would like to highlight that we cover a variety of architectures, pre-training tasks, and model sizes in selection of chemical foundation models.
As we are mainly interested in exploring the efficacy of different odorant representations, we only single protein foundation model ESM (Rives et al. 22221). For benchmarking in Section 3 we use "esmlb t33 650M UR5OS', the ESM model used by default in ProSmith (Kroll et al. 2024]. In Section 4 newer version of the ESM model "esm2 t33 650M UR50D".
15

---

Published as a conference paper at ICLR 2026
Table 4: Summary of molecular descriptors used in this work.
Descriptor
Source
Description
CATS
Physicochemical
representation designed for drug discovery applications that uses histogram of distamces between stom puirs represent molecule (Schneillr al. 1939 We the 2D version.
MordredDescriptors
Physicochemical
collection of 1,800 features representing the physicochemical, topological, geomet- rical, and constitatioal propenies of molecules (Moriwaki el al. 2018 We remove N&N values and
each feature.
PharmsoopoorcDD
Physioochemical
Computes topological distances between all pairs of atoms within a molocale. Atoms are featurized with variety of physical properies. These feature and distances are encoded into fixed-length bit vector; we use a 2048-length voctor.
RDKiDescriptoss22
Physicochemical
All physicochemical descriptors from RDKit, open-souree cbeminnormatics toolkit for molecular representation, manipulation, and analysis.
Ceatfold
Physioochemical
Encoding of simple substracture features molecular scaffolds, as hydrogen bond donors, acceptors, and aromatic rings, into a fixod-langth binary vector (Enl
2021
We 42-length
ECFP
Physioochemical
Circular molecular lingerprints that represent motocale by iteratively encoding the local cavironments around defined radius [Rggers Hahn 2010 We use 2000-length vector.
Roberta-Zine-8000
Transformer
RoBERTa style transformee with 102M parameters trained masked language model objective on 480M SMILES strings (Heyer 2023
GPT2-77nc480M-88M
Transfomer
GPT2 style transformer with parameters tmined on next character prediction task on 480M SMILES strings (Heyer, 2024)
ChemGPT-19M
Transformer
GPT3 style transformer with over 18 parameters trained on a next character prodie- tion task on 10M SET FIFS strings (Prey et al. (2022)
MoLTS
Transormer
'T5 style transformer trained on both SMILES strings and natural language text [Edwards al. 2022) This model uses a 'replace cormipted spans' task, where the SMIT ES string simultaneously, masks var- ious parts of each, then trains the model to predict masked sections.
model samples natural language text
ChemBEKTo-7M-MTR
Transformer
RoBERTa style transformer trained to predict set 200 molocular properties on 77M SMILFFS strings (Ahmad et all 20123
gin wun.superviedd..ooo infomax
GNN
graph isomorphism network (GIN) trained 2M molocules to maximize mu- tual information between local node representations and whole graph representations et 2020 Velickovic al. 2018]]
gin wetpevisedd.ddeppeee
GNN
GIN trained on 2M molocules predict masked edge attributes of melecular graphs
al. 2020
gin supervised contextpred
GNN
GIN trained 2M molecules suberaphs of the molecules predict their surrounding gruph structures [u el, 2020)
random
random
As baseline, generate random representation for every odorant. This gives us a lower boand on performance.
B PROBLEM STATEMENT AND MODEL DESCRIPTIONS FOR BENCHMARKING STUDY
We represent any featurization of odorant as vector R", where is the embedding dimen- sion of the representation Similarly, we represent a receptor as a matrix P E R' where lp is the sequence length of the receptor (number of amino acids g Given an odorant-receptor pair (xx, P), our goal is to predict the experimentally measured affinity y with various odorant embed- dings:
f((x, (() =y
We use three different models to assess the efficacy of chemical foundation models. A schematic of all three models can be seen in Figure Code for all of these models available here
Some transformer models produce representations that scale with the sequence length of the molecule. To compare these fairly with physicochemical descriptors and GNN-based embeddings, we mcan-poo pool across the sequence dimension.
"The receptor embedding dimension m is determined by the protein foundation model.
16

---

Published as a conference paper at ICLR 2026
Odorant Representation
Receptor Representation
Molecule Only - Lineer (MO)
Molecule Protein Lineer (MP)
ProSmith [Krol et el. 2024
Transformer
Affinity (Given Protein) Prediction
Affinity Prediction
Affinity
Prediction
Figure 6: Schematic showing all the benchmarking models used in Section 3 Odorant representa- tion be any representation highlighted in Table Protein representation ESM. GB: gradient boosting
B. MOLECULE ONLY MODEL (MO)
A ridge regression model that predicts affinity using only chemical foundation model representa- tions. For each receptor, the model takes the form
f(x) =
(2)
where only the odorant representation æ is used. For each receptor, we run a nested 5-fold cross validation. We conduct grid search over regularization parameters from 21 values evenly spaced on logarithmic scale between 10-10 to 10200 For the M20R (EC50) dataset, as it is binary classification task instead of a regression, we use l2 penalized logistic regression instead of ridge regression. For M2OR (EC50) we use stratified split to ensure there is both positive and negative examples in each of the splits. Additionally, we ignore proteins with <50 odorants, as there is not enough data associated with those odorants to ensure reliable prediction.
B.2 MOLECULE PROTEIN MODEL (MP)
ridge regression model that uses both molecular and protein embeddings. Specifically, we con- catenate the odorant embedding with mean-pooled protein embedding
(mllp) R=+m
p meanpool P)
where || denotes concatenation, and train
f( (p) =
This model, similarly to the MO model, uses a nested 5-fold cross validation scheme. We again use logistic regression with an 82 penalty train this model M2OR (EC50). We longer take any data from the M2OR (EC50) dataset, as we did in the MO model.
B.3 PROSMITH
state-of-the-art small molecule protein multi-modal transformer model (Kroll al. 2024). The odorant and receptor representations are projected into a shared g-dimensional space using respec- tive MLPs, M.P and MLP,(.). and concatenated as
MTPP(()))MMP((aa)) E (l++1)xg
(5)
Simlarly to LORAX, the transformer f processes this sequence and produces a joint representation <cls>
17

---

Published as a conference paper at ICLR 2026
((MLP,(P)IIMLP((E)) = MLP(<c1s>) = Vint
(6)
where Vint is an intermediate odorant- affinity prediction. The <cls> token is then com- bined with molecular and protein representations via an ensemble of XGBoost models:
Vfin = c1 * h1<<cls>) + 02 h2(pl ax) + Cg * ha(<cls>|pp)(z)
where yyin is the final prediction of the model, 2122,3 the XGBoost models, and the ensemble weights a satisfy E. G = 1. This model also trained using nested 5-fold cross validation. trained in two step fashion, where the best performing transformer model on the validation data is saved and used to generate the <cls> token. Next, each XGBoost is tuned independently via hyperopt, using random search over 500 iterations. The hyperparameter space and ranges are detailed in Table After hyperparameter selection, the XGBoost models are trained on the training set and evaluated on the validation set generate ensemble weights c1, C22 and C3. Finally, the weighted ensemble is used to predict affinities.
C GCN MODEL
In addition to the models highlighted in Appendix B we built a graph convolutional neural network (GCN, (Kipf & Welling 2017) to predict receptor responses from odorants alone. This method is similar to the MO model, but uses graph representation as an input feature, and uses a GCN to predict odorant responses given a receptor.
The GCN predicts affinity using only molecular features. For each receptor, we train the GCN to take, as input, graph 9 = (V. E). where node vector æ, V is 30-dimensional vector generated using deepchem (Ramsundar et al.) 2019), and the connectivity of the graph is described using an adjacency matrix where A.j = (e), E E. and A, if (e),e)) $ F.
For each dataset, for each protein, we randomly split the data into train, validation, and test graphs. For the M2OR (EC50) dataset, as for the MO model, we use a random stratified splitting strategy to ensure there are both positive and negative examples in all data splits and ignore proteins with less than 50 examples. We train GCN's per receptor using 5-fold cross validation strategy, and report the average performance of all GCNs, averaged over folds and over all proteins in the dataset in Table These results confirm, as we note in the main text, that using molecular features alone are insufficient for affinity prediction, and the use of protein information is necessary to create reliably predictive model.
Table 5: Summary of GCN results. We report average R² values for the Hallem and Carey datasets and Matthews correlation coefficient (MCC) for the M2OR (ECS0) dataset, each + standard devia- tion computed across 5-fold cross validation and across proteins within each dataset.
Dataset
GCN performance
Hallem (Hallem & Carlson (006
-0.0538 + 0.255
Carey ( CCaryy et al. (010))
-0.0933 0.781
M2OR (EC50) LLalis et al. (024
0.0032 + 0.051
D METRIC SPACE DISTANCE CALCULATION DETAILS
Here we provide a brief overview of how the distance metrics the paper were calculated. Consider two odorant representations (e.g., CATS and ECFP) that map an odorant feature vectors æ; g and æ; E Rm Embedding an entire dataset of N odorants with each representation yields matrices X; E R and X, E xm To project each representation into the same space, we use PCA to reduce the dimensionality of both matrices to & = 42, resulting in X,, X, E N We then optimize the following
x
We use 42 here because the largest representation vector, "SaaffoddKyyCaleulatof is 42 length.
18

---

Published as a conference paper at ICLR 2026
Q" = argmin X; X QEO
where O is the set of k x orthogonal matrices. This optimization problem can be solved by doing a singular value decomposition on X; Xj where Q UV We calculate distance between the two spaces with
d(X,, = arccos
Ti
where the singular values of X; Xj: Optimizing equation 8 and calculating distance from equation gives rise to the orthogonal Procrustes distance metric mentioned in the main text. If we instead whiten each representation as
X (X, 1,2
x² [x] x,)
(10)
and solve
Q"
argmin QEO
X
xiQ11
(11)
the resulting distances (again calculated via equation 9) correspond to the CCA distance metrics reported in the main text. These metries calculated using the package 'netrep' from Williams
et al. 2022).
E NEURAL REPRESENTATION GENERATION
In this section we outline how the neural..epresentation feature space is created for Figure 5 and Figure The Carey dataset can be represented as a matrix X R where N the number of odorants and m is the number of receptors. Each odorant is described by its response profile across all receptors, so the feature vector for odorant is the i-th row of X, denoted æ; E R This makes the entire matrix X representation of all odorants defined by their activation patterns across every receptor. This feature matrix X is what we call 'neural representation' and it can be directly compared to any other feature matrix Y Nx (e.g. CATS or ECFP) to assess alignment. We use CCA and Procrustes alignment on the nneuralrepresennaaion and representations of odorants used in the paper to determine how aligned the featrure spaces are.
F LORAX MODEL DETAILS
In this section, we describe model details of our LORAX model. LORAX uses the Huggingface and PEFT libraries to adapt both the chemical and protein foundation models. All results shown in this work (unless otherwise specified) only adapt the chemical foundation model with LoRA. The chemical foundation model adapter was run with r 8 rank adapter matrices on the query, key and value matrices with and show results in Table S3. We do not train the bias parameters. We have dropout rate of 0.1 in the adapter matrices during training. We do not use rslora (Kalajdzievskij (023) as we are interested in starting from the original chemical foundation model representation to see how the representation changes over training. The cross attention block of the model composed of two multi-headed attention layers. Both of these layers use 8 heads by default. The SMILES multi- headed attention layer takes the last hidden state of the LoRA adapted chemical foundation model (smitora) the query, and the last hidden state of the protein foundation model (protori) as the key and value matrices. The protein multi-headed attention layer does the opposite taking prot the query matrix and smitore as the key and value matrices. In both of these multi-headed attention layers there dropout rate of 0.1 during training. After the cross attention block, there residual connection where the output of the SMILES (protein) multi-headed attention layer is
8. We do hyperparameter sweep over rank of the adapter matrices
19

---

Published as a conference paper at ICLR 2026
Figure 7: Full distance matrices using Procrustes (left) and CCA (right) distance metrics correspond- ing to the distance matrices shown in Figure 5
added to smitora (protori). These representations are then passed through layer norm. Both protein and odorant representations are then mean pooled across their sequence dimensions and concatenated. This concatenation is the <cls> token referenced in the main text. The <els> token then put through an MLP with default dimension 512 to predict affinity between the odorant and receptor. During training, the model is with batch size of 12 for 50 epochs for the Carey, Hallem, and M2OR (EC50) datasets and batch size 21 for 15 epochs for the M2OR dataset. By default the training is conducted at learning rate of 1e-5 unless otherwise specified. We Adam as our optimizer. A block diagram for the LORAX model prior to the XGBoost ensemble is in Figure
Throughout training, the loss on the validation set is monitored, and the weights of multi-modal transformer model with the lowest validation loss are saved. Then, <cls> tokens are generated for all data using that multi-modal transformer model. We then train 3 separate XGBoost models. One model takes, input, the <cls> token; one takes, as input, the original, non LoRA-optimized, chemical foundation model concatenated to the protein foundation model representation; and the last model takes, input, the <cls> token concatenated with the non LoRA-optimized chemical foundation model representation and the protein foundation model representation. Each XGBoost model has a set of hyperparameters that we optimize on the validation set using hyperopt. The hy- perparameters,
well as the range of their values, highlighted in Table 6 The parameters optimized using a random search over a default of 500 iterations. After optimizing hyperparame- ters for all three XGBoost models, they weighted according their accuracy the validation dataset. Finally, all XGBoost models are used to predict the test set values using the confidence weights. Code for this model is available here
G ALL DATASETS AND METRICS STUDIED
This section presents all metrics and datasets not fully reported in Section Tables and 8 provide affinity prediction results and additional ProSmith metrics, respectively, for the Hallem dataset. Table 9 reports further metrics for the ProSmith model trained the Carey dataset. Tables TO and IT show affinity prediction performance and other relevant metrics for the ProSmith model on the M2OR (EC50) dataset. Finally, Table [2 highlights general metrics for each dataset used in the paper.
20

---

Published as a conference paper at ICLR 2026
SMILES
Protein Sequence
Foundacoenee codkk
SMILES
Protein
LoRA adapted chemical foundation model
Protein foundation model
Diiginal pen.eemetatainn (omnt
SMILES Protein cross attention
Protein SMILES cross attention
Updated SMILES
Updated Protein Representation
Num Representation
Lap indroweppool
<cls>
MLP
Affinity Prediction
Figure 8: Block diagram for the LORAX model prior to the XGBoost ensemble.
Table 6: Hyperparameters and their search ranges for XGBoost optimization.
Hyperparameter
Search Range
learning rate
0.01 0.5
max depth
7, 9, 10, 11, 12, 13, 14 0-5
reg_lamdda
reg alpha
0-5
max max_delta ddeeta_steeppppppeep
0 0-5
min child weight
0.1 11-15 15
num rounds
30 - 1000
weight
0.01 0.99
21

---

Published as a conference paper at ICLR 2026
Table 7: Affinity prediction performance across odorant representations for the Hallem dataset. Reported mean R² = standard deviation across 5 fold cross validation. Best performing repre- sentations are highlighted in bold.
Odorant representation
MO
MP
ProSmith
CATS
-0.129 + 1,819
0.367 0.033
0.561 + 0.032
MordredDeserrptons
0.137 = 0.341
0.382 0.042
0.579 0.036
Pharmacophore21
0.192 + 0.247
0.389 0.037
0.530 0.059
RDKiiDescciptorrS22
0.044 = 0.201
0.302 + 0.043
0.601 + 0.057
ScaffoldkeyCalculator
-0.014 + 0.219
0.272 0.040
0.501 + 0.044
BCFP
0.182 + 0.282
0.389 0.036
0.558 0.036
Roberta-7Inn4480M-100M
0.183 + 0.252
0.398 0.039
0.564 0.041
GPT2-Zinc480M-887
0.095 + 0.205
0.367 0.041
0.521 +: 0.067
ChemGPT-19M
0.118 + 0.240
0.359 0.035
0.559 + 0.051
MolT5
0.156 + 0.249
0.389 0.035
0.555 0.046
ChemBEKTa-77M-MMTR
0.198 + 0.349
0.373 0.039
0.574 0.040
gin supervised infomax
0.198 + 0.287
0.396 + 0.041
0.530 + 0.044
jun_supervised_edgepe
0.177 + 0.252
0.390 0.040
0.550 + 0.031
gin supervised contextpred
0.203 + 0.411
0.396 0.043
0.547 0.061
random
-0.069 + 0.119
0.382 + 0.039
0.439 + 0.073
Table 8: Other metrics for the ProSmith model trained on the Hallem dataset. Error indicates stan- dard deviation across fold cross validation. Best performing representations are highlighted in
bold. MSE: mean squared error, rm2: modified R² defined r r² - vr² r where
2² and re are correlation coeficients with and without intercept respectfully (Roy et al. 2013 CI: concordance index.
Odorant representation
MSE
rm2
CI
CATS
0.462 0.042
0.548 0.034
0.818
MoriredDescriptors
0.444 +0.052
0.571 0.043
0.824= 0.009
Pharmacophore2D
0,494 0.068
0.526 0.052
0.806 0.007
RDKinDesscrppoorS21
0,421 0.075
0.587 0.061
0.830 0.007
ScaffolakeyCalculattor
0.524# 0.050
0.491
0.813= 0.011
ECFP
0.466: 0.049
0.551 0.034
0.821
Roberta-7inc480M-1102
0.459+ 0.053
0.554 0.046
0.824 0.005
GPT2-Zinc480M-87M
0.505 = 0.081
0.511
0.810 0.011
ChemGPT-19M
0.465+ 0.068
0.551 0.058
0.821 0.008
MolT5
0.470 0.066
0.546 0.052
0.818 0.010
ChemBERRYa-7MMMMTR
0,449 0.055
0.563 0.045
0.821+0.000
gin supervised infomax
0,494 0.052
0.522+0.045
0.820= 0.007
gin supervised edgepred
0.474 0.045
0.541 + 0.031
0.826 0.003
gin supervied contextpred random
0.478 0.074
0.533 0.060
0.819 0.007
0.590 0.086
0.434 0.071
0.793 0.012
22

---

Published as a conference paper at ICLR 2026
Table 9: Other metrics for the ProSmith model trained on the Carey dataset. Error indicates standard deviation across fold cross validation. Best performing representations are highlighted bold. MSE: mean squared error, rm2: modified R² defined as r = (1 vr²
where 7 and rü Cl: concordance
correlation coeficients with and without intercept respectfully (Roy et al. 2013
index.
Odorant representation
MSE
rm2
CI
CATS
0.378 = 0.036
0.631 + 0.016
0.815 + 0.008
MordredDescriitooss
0.293 0.037
0.717 + 0.027
0.822 + 0.005
Pharmacophorc2D
0.471 0.038
0.543 0.011
0.801 0.008
RDKiiDesccrppoosS22
0.292 + 0.039
0.718 + 0.028
0.821 + 0.009
ScaffoldKeyCalculttor
0.422 0.031
0.590 + 0.023
0.796 = 0.012
BCFP
0.351 0.046
0.658 0.030
0.805 0.010
Roberta-Zinn-480M-100M
0.341 0.037
0.666 0.031
0,817 0.011
GPT2-ZHmc440M--87
0.347 = 0.045
0.661 + 0.030
0.813 + 0.011
ChemGPT-19M
0.328 0.034
0.680 + 0.024
0.816 + 0.014
MolT5
0.338 0.038
0.669 + 0.021
0.812 + 0.005
ChemBERTa-77M-MMTR
0.309 0.032
0.700 0.020
0,822 0.010
gin.supervised__nffmaaa
0.343 0.034
0.667 0.018
0.812 + 0.013
gin.superriseddeedgppd
0.326 = 0.035
0.681 + 0.028
0.820 + 0.011
gin.superrised_conneexx
0.315 0.047
0.696 0.035
0.822 0.008
random
0.431 0.047
0.582 0.038
0.796 0.011
Table 10: Affinity prediction performance across odorant representations for the M2OR (EC50)
dataset. As stated in the main text, this dataset a subset of of the M2OR dataset using on EC50 labeled data. Reported mean Matthew's correlation coefficient (MCC) + standard deviation. Best performing representations are highlighted bold.
Odorant Odorant representation
MO
MP
ProSmith
CATS
0.090 + 0.262
0.562 + 0.030
0.674 + 0.029
MordredDeseriitors
0.113 0.291
0.568 0.027
0.709 + 0.018
Pharmacophore22
0.097 0.270
0.570 0.030
0.656 0.032
RDKiiDesscriptorS22
0.066 + 0.241
0.515 + 0.038
0.694 +: 0.024
ScaffoldkeyCalculator
0.053 = 0.198
0.583 + 0.025
0.646 + 0.032
ECFP
0.096 0.293
0.581 0.030
0.663 + 0.016
Roberta-Zinc488M-1002
0.131 0.310
0.568 0.029
0.697 0.027
GP12--Zinc480M-557
0.109 0.286
0.563 0.026
0.678 + 0.023
ChemGPT-19M
0.103 = 0.295
0.574 + 0.035
0.683 + 0.017
MolT5
0.113 0.294
0.589 + 0.035
0.685 + 0.030
ChemBERRTa-7MMMMTR
0.172 0.351
0.591 0.030
0.696 0.033
gin_supervised_inffe infoomxx
0.109 + 0.300
0.595 0.030
0.684 + 0.030
supervied de
0.118 = 0.312
0.593 + 0.021
0.692 4: 0.040
wuupervised_contxxppeeee
0.103 0.266
0.589 + 0.031
0.695 + 0.026
random
0.004 0.120
0.550 0.029
0.640 0.027
23

---

Published as a conference paper at ICLR 2026
Table Other metrics for the ProSmith model trained on the M2OR (EC50) dataset. Error indi- cates standard deviation across 5 fold cross validation. Best performing representations are high- lighted bold. AUROC: area under the receiver operating characteristic curve.
Odorant representation
Accuracy
AUROC
CATS
0.877 = 0.013
0.917 + 0.006
MordredDDescriptons
0.892 + 0.006
0.933 + 0.007
Pharmacophore2D
0.872 = 0.012
0.913 + 0.007
RDXKitDeseciptors22]
0.886 + 0.008
0.931 + 0.007
ScaffoldKeyCalculator
0.870 = 0.012
0.910 + 0.005
ECFP
0.873 = 0.005
0.923 + 0.009
Roberta-Zinn4480M-102M
0.889 + 0.010
0.927 0.005
GP12-Zinc480M-57M
0.882 = 0.008
0.923 + 0.005
ChemGPT-19M
0.883 + 0.006
0.923 + 0.006
MolT5
0.884 = 0.010
0.925 + 0.007
ChemBERKaa-7MMMMTR
0.888 0.012
0.927 0.003
gin gin_supervised infomax
0.884 = 0.010
0.925 = 0.006
gin supervised edgepred
0.887 + 0.013
0.928 + 0.008
gin cuupervise_cconeette
0.887 = 0.010
0.925 = 0.004
random
0.870 0.010
0.902 0.007
Table 12: Summary table for all datasets used in this work. ng refers to the number of unique receptors in the dataset, no refers to the number unique of odorants, and pairs refers the number of recptor-odorant pairs in each dataset.
Dataset
7lg
nl
Pairs
Response
Hallem (Hallmm Carlson (2066)
24
110
2640
spikes/s
Carey (Carey al. (2010)
50
110
5500
spikes/s
M2OR (EC50) (Lalis et al
2024
508
474
5835
binary
M2OR (Lalis et al. 2204
1237
596
46563
binary
24


Reviews:
[{"review_id": "7yvGw9XePN", "reviewer": "Reviewer_jDeV", "summary": "The authors show that for olfactory tasks, chemical foundation models do not yield representations that outperform physicochemical descriptors on odorant-receptor binding tasks. To bridge this issue, they propose a LoRA strategy to finetune these representations, achieving better predictive performance and generalizability over their benchmarks. The idea is clearly communicated and straightforward to understand, and the authors do reasonably well in their evaluation of existing foundation models to emphasize their point. However, in my opinion, the conclusion could be made stronger with the inclusion of a few more key benchmarks that would make the results more convincing. I recommend that the paper be accepted to ICLR after these changes are satisfactorily addressed.", "questions": "1) I recall that the M2OR dataset is very sparse in terms of molecular-receptor pairs, and so it should not be surprising that the MO model performs poorly due to the lack of data. I’m not sure if this is the case for the other two datasets the authors use in this work.\n2) What is the source of the neuron response data that is used to generate the neuron representation in Figure 5? \n3) What is the performance of a GCN supervised baseline?\n4) What is the performance of a more recent foundational model?\n5) Does cross-attention outperform concatenation, and does it lead to different conclusions from what you observe?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The authors present the idea in a clear manner -- there are varying approaches towards the representation of molecules in olfaction, and the lack of olfactory data can make the use of foundational models more desirable over a supervised learning approach. There needs to be a benchmark for the efficacy of foundational models towards olfactory tasks, and this work addresses this gap.\n\nThe authors also evaluate a wide variety of representations beyond foundational models, and consider supervised learning approaches in addition to physicochemical representations. I appreciate that their analysis involves comparisons of performance spreads beyond just looking at mean performance.\n\nI particularly enjoyed the analysis in Sec 4.1 where the authors analyzed feature importance to detect that the ProSmith model intentionally underutilizes the transformer feature.", "weaknesses": "Table 3 is incomplete in my opinion -- to fully compare everything on equal footing, the authors should use the approaches Hladiš et al. and Chithrananda et al. used in their paper to complete the entirety of Table 3. In line with this comparison, I would like to see the performance of a supervised GCN (widely used in other olfactory tasks) on all the datasets to properly show that fine-tuned foundational models indeed provide a better representation.\n\nAdditionally, I concur with the authors in Appendix A that they’ve evaluated a large number of foundation models, but as they also mention, these models are not recent ones, and a conclusion like the authors are proposing needs to be based on models that are more recent. \n\nI was partially confused by Figure 4B, and thought a more natural way to show that LORAX is useful is to show that the proportion placed on < cls > is higher for LORAX to be consistent with their analysis with ProSmith’s poorer performance compared to LORAX.\n\nI think the authors should dial back their claims that this phenomenon is general for all olfactory tasks, as they have only demonstrated this for molecule-receptor binding tasks. \n\nThe authors propose a strategy where they concatenate the features, but cross-attention has also proven to be a reasonable strategy in molecule-receptor binding tasks (i.e. Chithrananda et al.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wIsbR5gRFO", "reviewer": "Reviewer_Cjx2", "summary": "In this work, the Authors investigate the representations of odorants through the lens of their predictive power for the odorants' affinitie for olfactory receptors. To this end, they compare standard sets of chemical features and several representations learned in foundation models for chemicals and proteins. Upon finding the overall similarity between different representations of odorants, they propose a LoRA-based finetuning framework for learning the representations relevant to olfaction. They test treir and existing representations on several odorant-receptor affinity datasets.", "questions": "What substantiates that claim that LORAX has the best representations?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- Comprehensive literature overview\n\nThe paper considers a comprehensive set of the representations (e.g. the chemical descriptors) and the models (e.g. GNNs) that have found widespread use in olfaction; thus, the work is well-positioned in literature and relevant to the olfactory community\n\n- Comprehensive evaluation\n\nFor both existing and new representations, the work offers a comprehensive analysis, thus adding new knowledge about the tools that people use in practice and not only about the new method.\n\n- Insights into the similarity and differences between conventional representations\n\nThey find, through the analysis, that many different representations of odorant are, in fact, similas, especially those leading to good predictions for odorant-receptor affinities. Additionally, the similarity between the decodeability is shown for the most of the representations\n\n- Ablations\n\nFuthermore, the imapct of different model inputs is investigated; the utility of the joint condiferation of the representations of many odorants and many reptors in a single model (e.g. ProSmith) is shown.\n\n- Text clarity\n\nThe text is super clear and well-structured.", "weaknesses": "- The proposed model doesn't seem to improve upon the baselines.\n\nThrough Table 2, LORAX is better than ProSmith for unseen odorants but worse for unseen receptors (that said, all R2-scores here are really small). Through Table 3, LORAX is the best in recall and F-score but not the best in precision and AUROC. Unless there's a principled way to determine that these metrics are more important thatn the others, one can't claim that the model is the best.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GAzqzmChzu", "reviewer": "Reviewer_r6QY", "summary": "In this paper, the authors contribute LORAX, a novel method that incorporates fine-tuned chemical foundation models and pre-trained protein foundation models for odorant-receptor binding prediction tasks. The authors motivate the need for their approach with an extensive evaluation of different representations across relevant datasets, highlighting: (i) that the representations encoded by pre-trained chemical foundation models do not significantly improve the performance in odorant-receptor binding prediction tasks versus hand-crafted physiochemical features; and (ii) that is due to the significant alignment between these pre-trained representations and the hand-crafted ones. Finally, the authors evaluate LORAX across different datasets, showing that it learns a more generalizable representation to unseen odorants, and provides a representation space more similar to the underlying neural representation.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- **Originality**: The originality of this work is highlighted in two different aspects: (i) the use of low-rank adaptations to fine-tune foundation models of chemical data for olfactory tasks; (ii) the use of such fine-tuned models in combination with pre-trained representation models of protein data for odorant-receptor binding prediction tasks. While fine-tuning foundation models of chemical data for olfactory tasks has already been explored before (see [1]), the use of LoRAs for this purpose is, to the best of my knowledge, novel.\n\n- **Quality**: I found the paper to be of good quality, with minimal typos (one exception being \"adapated\" in line 435). Despite this, I encourage the authors to improve the quality of some of the figures (as an example, Figure 1B (a) is not really comprehensible without improving the caption). While not particularly novel, the idea of using low-rank representations to fine-tune foundation models is sound.\n\n- **Clarity**: The paper is overall clear. In particular, I thoroughly enjoyed the structure of the work, starting by motivating the need for LORAX with extensive experimental results, and subsequently evaluating LORAX, comparing to previous results.\n\n- **Significance**: This work contributes to an emerging line of research within olfactory machine learning that resorts to pre-trained foundation models of chemical data, due to the lack of large-scale, high-quality and diverse datasets of olfactory data. Fine-tuning approaches, such as the one presented in this paper, are undoubtedly an important tool to address the specificity of olfactory data, and thus significant. \n\n**References**:\n- [1] Taleb, Farzaneh, et al. \"Can transformers smell like humans?.\" Advances in Neural Information Processing Systems 37 (2024): 72032-72060.", "weaknesses": "- In Section 3, the authors present an extensive study on affinity prediction, considering different odorant representation models and input conditions: only odorant, molecule + odorant, and a processing of these two representations using ProSmith. The results show a significant increase when using protein information and ProSmith. However, I've found interesting that no ablation condition with only protein information was done, especially given that this information seems fundamental for the task itself. Can the authors show results of such ablation?\n\n- It is not clear what are the main differences between LORAX and ProSmith, beyond the LoRA component of the chemical model. Can the authors elaborate on the main differences between the two models?\n\n- The authors motivate LORAX by highlighting that current odorant representations perform similarly to each other, yet the results in Line 317 show no significant performance difference between LORAX and ProSmith. Same for the results in Table 3, where ProSmith actually achieves a significantly higher AUROC. While achieving SOTA is not a requirement across all metrics and situations, in this case it is unusual since the authors are fine-tuning the pre-trained chemical representation for their specific downstream evaluation task. Can the authors comment on this? Why would a practitioner use your approach instead of ProSmith?\n\n- Since LoRAs is such an important component of their contribution (and the main difference between their approach and the baselines), it is unfortunate that no extensive evaluation of different LoRAs is made, in regards to rank, layers to be fine-tuned, etc... Can the authors present this results? How did the authors select their LoRA architecture?\n\n- Some important details are currently missing in the paper such as definition of the evaluation metrics in the tables, and identification of which datasets are used for the results presented in the tables. Can the authors add this information?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fNMpbO9m8b", "reviewer": "Reviewer_P2Me", "summary": "The paper presents a benchmark of several chemical foundation models in the scope of the notoriously difficult problem of modelling olfaction. The authors showed that the zero-shot application of chemical foundation models does not increase the performance in molecule-receptor response prediction tasks compared to using standard physicochemical features. Thus, to mitigate this, the authors suggest using LoRA to fine-tune chemical foundation models (e.g. ChemBERTa-77M-MTR) to model interactions at the beginning of the olfactory system. The authors demonstrate that combining LoRA with ProSmith model (introduced in Kroll et al., 2024) leads to 1.5\\% increase in the performance on the random splits of the challenging M2OR dataset compared to ProSmith without fine-tuning.", "questions": "Some of the experiments are not clear. For instance, how are Table 3 and Table 9 related? Both tables discuss the performance on M2OR dataset. Assuming that the authors evaluate the models in Table 3 on \"ec50\" pairs due to the label noise (following Hladi\\v{s} et al.), I would expect that the values in the tables are comparable. However, \"MorderdDescriptors\" achieves MCC of 0.709 in Table 9, outperforming the proposed LORAX approach in Table 3.\n\nIn Figure 5 and Figure 7, the authors compare LORAX,  ChemBERTa, and other odorant representations to the \"neural representation\" derived from Carey dataset. However, in my understanding, this comparison is not fair, since LORAX is trained on the neuronal activity data and other models are zero-shot. In addition, despite being trained, LORAX does not appear to be significantly more aligned with neuronal activity than, zero-shot \"ecfp\" or \"gin\\_supervised\\_infomax\" representations (first columns in Figure 7).\n\nSince the authors do not provide the code during the review process, I am unable to check data splitting, which is crucial in the field of protein-molecule interaction prediction. This said, how are the unseen receptor/molecule splits constructed in Table 2? The similarity of the protein sequences/molecular structure between the train and test sets can largely mislead the performance evaluation (see e.g. Zhang et al. Rethinking the generalization of drug target affinity prediction algorithms via similarity aware evaluation, ICLR 2025). In addition, the generalization experiment is conducted on the Carey dataset. What is the performance on other two datasets? In particular, how this compares to the generalization performance presented in Hladi\\v{s} et al.?\n\nConsidering the style of writing, I find the paper imprecise and unpolished, especially in the use of terms such as \"binding\", \"affinity\" and \"activation\". Data in Hallem and Carey datasets are average spikes per second for insect electrophysiology experiments, and data in M2OR are mostly in vitro functional assays in heterologous systems. Neither of these experiments directly measure \"binding\" as binding of the molecule to the protein is only a necessary but not a sufficient condition to observe the experimental response. In the case of M2OR dataset, the term \"affinity\" is also imprecise, since the authors consider activation prediction, which is different from predicting a measure of affinity, such as EC50. \n\nSection 4.1. is difficult to follow. At the beginning the authors write \"We trained LORAX on the Carey dataset and found it performs almost identically to ProSmith\" and then suddenly in Figure 4 LORAX performs better than ProSmith. Is the performance in Figure 4 only for the Transfomer part of ProSmith? If yes, then this is not clear from the text and the figure caption.\n\nLastly, I found several incorrect citations in the paper (Cong et al. 2022, Haddad et al. 2008a, Lalis et al. 2024, Sypetkowski et al. 2024). I believe that the errors come from some citation tool, but the authors should thoroughly check the citations as if passed unnoticed, this shows disrespect to the original authors.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "The paper made an interesting observation, that embedding of chemical foundation models is similar to the traditional physicochemical descriptors. In addition, the proposed model, LORAX, achieves a marginal increase in the SOTA preformance on M2OR dataset compared to ProSmith model which does not use LoRA fine-tuning.", "weaknesses": "The proposed algorithm, LORAX, is rather incremental and only adds a well-established LoRA fine-tuning to ChemBERTa in the previously-published ProSmith model. The most interesting result of the paper is the observation that embedding spaces of the tested chemical foundation models are aligned with each other and with standard physicochemical descriptors. This is a surprising result, given that chemical foundation models are widely-studied and have been shown to outperform physicochemical descriptors in molecular properties prediction. However, the authors do not provide any explanation of such an unexpected phenomenon, making the result much less impactful. Additionally, the authors do not provide the code, so I am unable to reproduce the results, nor check the data splitting strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)