Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
Authors: Yin Fang
Xiaozhuan Liang
Ningyu Zhang
Kangwei Liu
Rui Huang
Zhuo Chen
Xiaohui Fan
Huajun Chen
Venue: ICLR 2024 poster (2024)
Keywords: instruction dataset
large language models
biomolecular studies
molecule
protein
Primary area: datasets and benchmarks
TL;DR: A large-scale biomolecular instruction dataset for large language models.

Abstract: Large Language Models (LLMs), with their remarkable task-handling capabilities and innovative outputs, have catalyzed significant advancements across a spectrum of fields. However, their proficiency within specialized domains such as biomolecular studies remains limited. To address this challenge, we introduce Mol-Instructions, a comprehensive instruction dataset designed for the biomolecular domain. Mol-Instructions encompasses three key components: molecule-oriented instructions, protein-oriented instructions, and biomolecular text instructions. Each component aims to improve the understanding and prediction capabilities of LLMs concerning biomolecular features and behaviors. Through extensive instruction tuning experiments on LLMs, we demonstrate the effectiveness of Mol-Instructions in enhancing large models' performance in the intricate realm of biomolecular studies, thus fostering progress in the biomolecular research community. Mol-Instructions is publicly available for ongoing research and will undergo regular updates to enhance its applicability (https://github.com/zjunlp/Mol-Instructions).

Published as a conference paper at ICLR 2024
MOL-INSTRUCTIONS: A LARGE-SCALE BIOMOLECU- LAR INSTRUCTION DATASET FOR LLMS
Yin Fang*; Xiaozhuan Liang*; Ningyu Zhang* Kangwei Liu
Rui Huang* Zhuo Chen Xiaohui Fan Huajun Chen 9f
College of Computer Science and Technology, Zhejiang University
ZJU-Ant Group Joint Research Center for Knowledge Graphs, Zhejiang University
ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University [fangyin, 1iwggxaazzzaaannianaaommma kangwe wiwii,lli,,. hrhr, zhuo chen, fanxh Je wuu.uuuuuuu... edu.en.
Izhnngingyu, hus j wttp://...u.u....... edu. cn
Mol-Instructions bttps://gittae com/ zjjnlp/Moan
Dataset
& Molecule model
Protein model
Biotext model
Molecular desoription generation
Molesular description generation The oveniioe of given malectio io rosple appears watenotile lopu with parge oete Denawr AIBC
Could you give me brail cvenioo moleccsl?
Reagent Prediction
LLMs
Please suggest some possitle reagent that could
have been used the tollwing chemica macton
Reagent Prediction
Descriptton
malecule design
pusaalle
Create malecie satofes conditinss outhod doscrpsio::
"The mutecule
cryctaiiv soli powderr Cortanaiiie.
LLOWA
Description molecute nolecuio dosigned
Insiuble in water Todl inha on (cust) and grstion
Pretein design
H Vizana
Protein design
pvoteis that eabibte the deated activty specthaly:
The designed protiin
1t. The prosen ounarn nove Mcdilks comein thait conerr urique findion actlviyy
2 The deslyeed preeen rust cosaess aediviilccll oyrthesn aclalls
Domalnimatt prediction
We have Hevitfeed potental probele obroais with your providee sequence ATP-gpooa
Domanshnetti prediction
Pleace sosmime the follwing protein and prodict wny domsinn you can diocum:
Falco
Functional description
Protein
Functionaa deseripie cesortption goneration
Plesoe provias curmsary of the Juy factures and chanacherakks prodeire
proeading inhooduu sakiun sequesiiion via the oecher bakkun von lydisge Jonio soseria enombor mel"
Chemical antily micopntton
Find and for all the chamical acttie: "Ali Bun the anbiore endush ninr cadee oyt fuse moteger
Chamical entity recogrilton
obee the pareoteo whot wawe
GLM
The ohersical ertther any oaide' "estoger" "greicee""
false question
or question
30% hopatccelldee fee featons?
asstulvization
Nu. signal ittrrity develosment wosoorattatton
signifiaatt diferance
Bio text
Mutti-cholce question
Whish contain DAV cequences requiroo for the ongregation of phromocomes in rillols
mekcax? (V Neeeoeee (R) Cectorenne (2) Ahoasssrme (t) Splcacsenss
Figure I: Empowering Large Language Models with Mol-Instructions to unlock biomolccular domain. Both molecular and protein structures are represented as sequences.
ABSTRACT
Large Language Models (LLMs), with their remarkable task-handling capabilities and innovative outputs, have catalyzed significant advancements across spec- trum of fields. However, their proficiency within specialized domains such as biomolecular studies remains limited. To address this challenge, introduce Mol- Instructions, a comprehensive instruction dataset designed for the biomolecular domain. Mol-Instructions encompasses three key components: molecule-oriented instructions, protein-oriented instructions, and biomolecular text instructions. Fach component aims to improve the understanding and prediction capabilities of LLMs concerning biomolecular features and behaviors. Through extensive instruction tun- ing experiments on LLMs, we demonstrate the effectiveness of Mol- Mol-Instructions enhancing large models' performance in the intricate realm of biomolecular studies, thus fostering progress in the biomolecular research community. Mol-Instructions is publicly available for ongoing research and will undergo regular updates to enhance its applicability.
Equal contribution and shared co-first authorship. Corresponding author,

---

Published as a conference paper at ICLR 2024
INTRODUCTION
Large Language Models (LLMs)), such as GPT-4 (OpenAI, 2023a), Chinchilla (Hoffmann et al., 2022), PaI.M (Chowdhery et al., 2023), Codex (Chen et al., 3021), LLaMA (Touvron et al., 2023), FLAN (Wei et al., 2022), and GLM (Zeng et al., 2023) have revolutionized the landscape of Natural Language Processing (NLP). These models, boasting billions of parameters, are meticulously trained on extensive text corpora and excel at generating human- text and understanding intricate contexts. To adapt LLMs for specific tasks, researchers employ instruction tuning techniques (Ouyang et al., 2022a; Sanh et al., 2022). This involves training the models with specialized instruction datasets, allowing them acquire task-specific knowledge and patterns, thus enhancing their performance in specific domains. Numerous instruction datasets have already been developed for use in general domains. For instance, the Stanford Alpaca dataset (Taori et al., 2023) offers prompts, completions. and annotations tailored for controllable text generation. The GPT4AIl dataset (Anand et al., 2023) comprises variety of formats, including code, stories, and dialogue, purposed for training and evaluating general-pee purpose language models. Similarly, the COIG dataset (Zhang et al., 2023a) inte- grates diverse corpora such as translated, exam, and human value alignment instructions, specifically oriented towards Chinese language processing.
With recent advancements, it's evident that L.MSs extend beyond traditional text processing (Zhang et al., 2024; Tinn et al., 2023; Wang al., 2023a). Their potential in biomolecular studies, covering structural biology, computational chemistry, and drug development, is particularly promising. Uti- lizing LLMs in this field could revolutionize our understanding and handling of biomolecular data, accelerating scientific innovations and drug discovery.
However, major barrier to harnessing LLMs in the biomolecular domain is the lack of dedicated dataset for this field. Despite the existence of several instruction datasets in general domains, a conspicuous gap remains when it comes biomolecules. This gap arises from three main chal- lenges: First, acquiring and annotating biomolecular data incurs significant costs, given the inherent complexity and rich depth of information contained within such data. Second, biomolecular com- putations span a broad knowledge spectrum, intertwining specialized insights from disparate areas including structural biology, computational chemistry, and drug development. Third, unlike the well-established frameworks in natural language processing, bioinformatics has no standardized lingua franca. Different applications often employ varied representations for biomolecules and their associated computations. This diversity amplifies the challenge of crafting a dataset that serves uni- versally across the domain. To address this pressing need in the biomolecular domain, we introduce Mol-Instructions (CC BY-NC-SA 4.0), dataset tailored to the unique challenges of biomolecular studics. This dataset, as delineated in Figure 1, is structured around three core components:
Molecule-oriented instructions: This component delves into the realm of small molecules, emphasizing their inherent properties and behaviors. It sheds light on the fundamental challenges of diverse chemical reactions and molecular design, with 148, 4K instructions across six tasks.
Protein-oriented instructions: Concentrating on biosciences, covers 505K instructions spanning five categories of tasks. These tasks aim to predict the structure, function, and activity of proteins, and facilitate protein design based on textual directives.
Biomolecular text instructions: Primarily crafted for NLP tasks within the fields of bioinformatics and chemoinformatics, this portion includes six information extraction and Q&A tasks represented through 53K instructions.
Creating this instruction dataset involves gathering and collecting biomolecular data from various licensed sources (detailed in Appendix A.2), followed by transforming this data into fosoly-followable instruction formats suitable for specific tasks. Our goal is to empower LLMs with domain- domain-specific insights, enbancing their ability to decode and predict biomolecular features. This enhancement can revolutionize the interpretation of biomolecular data, streamline drug development, and unveil new realms of biomolecular research. With Mol-Instructions. large models are endowed with the ability to comprehend biology, opening the door to new scientific discoveries. To assess the real-world effectiveness of Mol-Instructions we conduct extensive series of evaluations. Employing the representative LLM as the foundational model, we perform instruction tuning for each of the three main categories of instructions. The results highlight the value of Mol-Instructions. demonstrating its ability to enhance the versatility and understanding of large models in the complex domain of biomolecular studies.
2

---

Published as a conference paper at ICLR 2024
Table I: Comparison with existing datasets. We employ the following abbreviations: IIG datasets curated by human effort, SI datasets produced via self-instruct methods, MIX dataset composed of both human- constructed and machine- geniiineeeeeraeedd data, COL dataset assembled from a variety of other datasets.
DATASETS
TYPE
# INSTRUCTIONS
COLLECTION
USAGE
ACCESS
Gouprol Dompln
Stanfoed Alpaca (Thori et al., 2023)
Text
52,002
SI
Instruction Tuning
Open
Dolly-v2 (Conover et 2023)
Text
15,015
Instruction Tuuning
Open
Baize (Xa et al,, 2023)
Text
653,699
MIX
Instruction Tuning
Open
FLAN (Wei et all, 2022)
Text
1,764,800
COL
Instruction Tuning
Open
InstructGPT (Oayang al., 20226)
Text
112.801
HG
RLHF Instruction Tuning
Closed
ShareGPPT (sha, 2023)
Text
260,137
MIX
Instruction Tuning, Chat
Closed
A COTG (Zingg ot al., 2023a)
Text
67,798
COL
Instruction Tuning
Open
Litre Sara@hhee hat UltraChat (Ding et al, 2023)
Text
1,468, 352
MIX
Open
X Galactica (Taylor al., 2022)
Text. Biomolecule
783,599
MIX
Pre Pre-training
Closed
Speeilic Domain
PCdcs (Zeng al. 2022)
Text. Molecule 15,000
MIX
Pre Pre-trsining
Closed
ChEBI-20 (Edwardss et 2022)
Text. Molecule
33,010
COL
Pre- Pre-training
Open
PubC PubChemS/M (1 et al., 2025)
Text, Molecule
281,000
COL
Pre-tralinigg
Closed
MoMu (Su 2022)
Text. Molecule
15,000
MIX
Pre-trsining
Open
Mol-inntrructios (ours)
Text, Biomnlocule
2,013,587
MIX
Instruction Tuning
Open
2 RELATED WORK
Instruction data bears a stronger task-centric focus, typically augmenting the efficacy of LLMs utilizing a limited set of instances. Initial research efforts, such as Dolly-v2 (Conover et al., 2023) and InstructGPT (Ouyang et al., 2022b), relied heavily on manual or expert annotations to provide guidance for various NLP tasks, like closed Q& A and summarization. While human-annotated in- struction data often boasts high quality, ss limited in volume, diversity, and innovation. Recognizing these limitations, there's ' been a shift toward semi-automated or fully automated instruction creation. For instance, Stanford Alpaca (Taori et al., 2023) employs the self-instruct approach (Wang et al., 2023b), utilizzng& bootstrapping technique grounded set of handcrafted instructions generate 52K diverse instructions. This innovative method has inspired numerous model-aided data collection endeavors, such as Baize (Xu al., 2023), COIG (Zhang et al., 2023a), UltraChat (Ding et al., 2023), and ShareGPT (sha, 2023), detailed in Table 1.
Several studies have explored the intersection of text and biomolecules (Boiko et al., 2023; Bran et al., 2023; Zeng et al., 2022; Edwards et al., 2022; Nascimento & Pimentel, 2023; Zhang et al., 2023b). For instance, Galactica (Taylor et al., 2022), a general scientific language model, incorporates a subset of instruction data related molecules, proteins, and text during its pre-training phase. However, details of this particular dataset remain undisclosed. Other datasets, such as PCdes (Zeng et al., 2022), ChEBI-20 (Edwards al., 2022), PubChemSTM (Liu al., 2023), and MoMu (Su et al., 2022), pair molecules with textual descriptions. While these datasets are valuable, they are primarily geared toward training smaller models and lack an instructional format. This limitation curtails their direct utility for large language models. In contrast, Mol-Instructions covers a wider range of biomolecular tasks and larger amount of data, achieved through combined construction method of self-instruct, mmplate-bassed conversion, and human-crafted task descriptions (as detailed in $3).
MOL-INSTRUCTIONS CONSTRUCTION
3.1 UNDERLYING PRINCIPLES
Large-scale To cater to the needs of LLMs, we' we've crafted Mol-Instructions to be expansive, incorporating over 2 million biomolecular instructions. This large volume provides broad and representative coverage of biomolecular sequences and structures, enabling models to grasp and navigate the complexities of biomolecules.
Diversity Mol-Instructions spans 17 subtasks across three types of biomolecules and includes diverse text descriptions that capture over 11 unique biomolecular properties. This extensive cov- erage fosters versatility in models trained with our dataset, priming them for various challenges in biomolecular research.
Quality We prioritize the quality of our dataset to guarantee that it serves reliable foundation for deriving accurate, actionable, and practical insights. Each piece of biomolecular data undergoes rigorous scrutiny ensure its accuracy and trustworthiness.
3

---

Published as a conference paper at ICLR 2024
Humon AI collcboration description creation
Sector Template conversion biological date textual format
Human-wwitten
Diversity task descriptions
Biomolecufar annotations
Template
form
task description
The designed protein must be ahle to regulase phintogical prossssd
The dies grae protein must be wole dugl sigraal
Describe this molcculo."
"Whaa
"Give catals woout mai ecude.
Frovkd biief molequee?
Gene Ortelogy (aaiigiall proceas)
"Provsle descrition
Bono Deleeogy
The designed protein localiee lo the (iolbair componenn..
gr protein localizes the netoctonsiion.
aspielance
/OMA bindeg)
protein with hnmmmnoe cepebiiity for largeled gara regulation.
GNA binding
oipabiiity for targeled rigulation.
Sector 2: Information denivation from existing data
Key information
Sector
Quality control
Molecule SMILES: CHOO/-ON
CD: 5579
Structure:
nokucule
crytt
visiently
Molecule
Optimal formal
Flow
Chimiiiely salk
Retust descriptor
Ask quesiions give
Oetained biomolecules High quilly Chanically wald Nonvedundent
Darte sources
Prompt Fipl
earagraph
glron
directed
-0
idnnen works
Answar1 BGK1 la
la BOK1)
Lnder a0.ih
Protein
Untoviewed
Revioned
Manuul
Armotation
Becuerco duerilyy
Figure 2: The overview of data construction of Mol-Instructions which comprises four sectors: human-Al collaboration task description creation ($3.2), ,), information derivation from existing data ($3.3), conversion of biological data into a textual format via templates ($3.4), and quality control ($3.5). For detailed procedures for each task, please refer to Appendix B.
3.2 HUMAN-AI COLLABORATION TASK DESCRIPTION CREATION
A standard instruction entry is typically structured with three components: an instruction that clarifies the task, an input that serves as the task's input, and an output that embodies the expected outcome.
In real-world scenarios, task instructions exhibit a broad diversity to match the dynamic and diverse nature of human inquiries and requirements. To emulate this diversity, for each task, we begin with a clear and concise bumttn-rittten description, which then serves as input to gpt-3.5-turbo (OpenAI, 2023b). Leveraging its extensive knowledge and pattern recognition skills, the LLM generates diverse task descriptions, reflecting the wide spectrum of human question- framing styles (Figure 2 Sector 1). To ensure the quality of these descriptions, each one is subjected to a thorough manual review, This collaboration of human insight and machine intelligence not only enhances the diversity and creativity of task descriptions but also bolsters the robustness and adaptability of the instructions.
3.3 INFORMATION DERIVATION FROM EXISTING DATA
Since biomolecular data typically involves professional wet-lab experiments and expert chemists' summaries, our data is sourced from widely-used biochemistry databases (Kim et al., 2021; Wei et al., 2010; Lu & Zhang, 2022; Wu et al., 201 8; Ashburner et al., 2000; Consortium, 2023; Krallinger et al., 2015: 2017; Li et al., 2016; 6; Pal et al., 2022; Hendrycks et al., 2021). Based on these data sources, we can obtain the desired instruction data with appropriate processing.
Some datasets undergo manual processing, with labels being directly appended to specific biomolecu- lar data. This includes datasets with clearly delineated inputs and anticipated outcomes for various prediction endeavors, as well as datasets with predetermined questions and corresponding answers for Q&A tasks. Processing such data is fairly direct: the labeled information is typically mapped to the respective input and outpur fields for each instruction entry.
Another class of data sources comes without explicit manual labels and requires techniques such as data mining and Al-assisted generation for extracting and selecting the ideal data, as depicted in Figure 2 Sector With data mining, we strive for data comprehensiveness and adequacy by extracting additional relevant information from professional chemical research databases like PubChem (Kim et al., 2021). Specifically, we crawl valid molecule description texts and their corresponding PubChem Chemical Identifiers (CIDs), followed by retrieving the respective molecular descriptors. To focus the model's s attention on description semantics, we replace the molecular nomenclature with the term "the molecule". On the Al-assisted generation front, we use scientific abstracts from PubMed (White, 2020) to generate Open Question instructions. Here, we task 3pt-3.5-turbo to formulate questions and their corresponding answers based on the abstracts, structured as Q&A pairs. This methodology
4

---

Published as a conference paper at ICLR 2024
efficiently produces a varied collection of Q&A instructions, simplifying the construction of this particular task. For a more task-level detailed explanation, please refer to Appendix B.
3.4 TEMPLATE-BASED CONVERSION OF BIOLOGICAL DATA INTO TEXTUAL FORMAT
Certainly, not all data can be seamlessly transformed into ideal instruction sets. For some novel tasks, finding directly applicable data can be particularly challenging. This is especially true for protein design tasks, where previous studies often conditioned designs on broad categories (Madani et al., 2023) or fixed backbones (Dauparas et al., 2022), largely neglecting the bespoke functional and structural attributes (e.g., helical secondary structure, DNA binding domain, regulator activity). Conversely, we aim to devise instructions in textual form for de novo protein design tailored to intent, enabling the design of composite properties of interest. Given the scarcity of directly applicable data for such tasks, we curate annotations from specific select within UniProtKB (Consortium, 2023). These annotations encapsulate the protein properties frequently explored or sought after by researchers in the field (detailed examples are in Appendix Table 6).
transcription
To effectively convert these structured annotations into textual format, we formulate series of templates, illustrated in Figure 2 Sector 3 and Appendix Table 7. Each resulting textual annotation establishes criteria for protein design. By aggregating these functional descriptions and properties. we craft precise protein design instructions, ensuring that the synthesized protein meets the specified criteria. practice, considering the model's input length limitations and training efficiency, we randomly select subset of these conditions as design objectives.
3.5 QUALITY CONTROL
As the field stands, LLMs have not yet fully grasped the intricacies of biomolecular languages, falling short of their proficiency with human languages. To accelerate the model's capability to generate accurate biomolecules, we implement stringent quality assurance measures for our biomolecular data, detailed in Figure 2 Sector 4.
For small molecules, our process begins by eliminating chemically invalid SMILES strings from the initial dataset. Although external constraints (Landrum et al., 2013) validate generated molccules, using a dependable molecular descriptor within molecular instructions is more effective. While SMILES (Weininger, 1988) strings remain popular choice for molecular descriptors, models leveraging them often output strings that are either syntactically flawed or chemically inconsistent. To circumvent these issues, opt for SELFIES (Krenn et al., 2022) our molecular descriptor. With its tight rule set, SELFIES ensures the generation of valid molecules by allowing combinations of any symbols, eliminating common pitfalls associated with SMILES strings, such as producing illogical symbols or mismatched parentheses.
Alongside, we prioritize the integrity of our protein data by primarily sourcing entries from UniProtKBSSwiss-PPee (Consortium, 2023), curated and manually annotated protein sequence database. To bolster the volume and variety of our data, we supplement with high-scoring annotations from UniProrKBBTTEEMBL Recognizing the risk of bias from redundant protein sequences, such as homologous sequences or closely-relates protein variants within the Swiss-Pres and TrEMBL databases, we deploy a meticulous filtration process. Leveraging the MMseqs (Steinegger & Söding, 2017) tool, we cluster protein sequences at a 90% similarity threshold, selecting functionally rich entries within each cluster representatives and excluding the rest. This rigorous approach sures our dataset comprises diverse, high-quality, and function-centric protein instructions devoid of redundancies.
A CLOSER LOOK AT MOL-INSTRUCTIONS
4.1 CATEGORIZATION AND POTENTIAL APPLICATIONS OF INSTRUCTION TASKS
As illustrated in Figure 3. Mol-Instructions is anchored around three core domains: molecule- oriented, protein-oricnt oriented, and biomolecular texts. Each category's 's instruction task covers the essential challenges within its domain, aspiring to drive forward the biomolecular field via the
5

---

Published as a conference paper at ICLR 2024
Properl Prediction 40M MiNowrrs!
Moleouiar Desolition
Gansralion
Catalytle Prediction
Protein Design
Question
Chemical Erdty
Recignillan
22.3%
Descriptonngrie Molcoute Design 203K, PutChex
(10.7%
39
DemnissMaii
Oupstion
1.9
1K, eDRONEMO
disease Interaciion
Reagent Predicton
22.3%
25.1%
Prediction
XX ANsICCO
70.9%
ASK aCtSon
TMK 0S7MO, SOOM
Retroyyrhesss 9.39
Forward Reastion clion 1386 USPTO MTT
Protein Fundion Prediction
Funcsional Desoritionn
40C Ueihrort
MuS-choioa
23.1%
1.4%
Chamica pprotann
14% C3P11 S00R
Go Genoration
13C MwwMICM MMLL
paC CaeeRior
(a) Molecule-oriented Instructions
(b) Protein-oriented Instructions
(c) Biomolecular Text Instructions
The compositional structure of Mol-Instructions. Mol-instructions primarily encompasses tasks across three major categories: (a), (b) and (c). The task names are indicated above the horizontal lines, the sources of the original data and the sizes of the constructed instruction datasets are labeled below the horizontal lines, and the percentages on the pic charts represent the proportion data within cach major category.
Figure 3:
Protein
64
0.30
2500
Damio naro
63
Sohnoaarzhwomypen pomte
Ceenarhabed et aggeaa
0.24
0.7
Fadus norveg 5u
Saccharongsse conoitiie Scstcc
0.24
0.12
4.1
Arobicopsis HVV
Homo caplons
503
1000
0.g
20
93
500 10000 15:00 20000
0.00
255 512
700
1004
123
200
300
complecty
(c) Atom count
Taxoorom oiry
[g) Sequuence length
0) Catalyy:c activty
HUP-
2805
0.20
GADPH aa bec dh-
2000
0.20
0.2
Cauternnoss
1850
6:15
6.10
TBM berre
005
NAOP
NT1Phoo
505
500
20503 4bo0 6000 #200
Moleoutar weight
(d) Ring count
(1) Protein Family
(h) Domamn
Gene Ontology
Figure 4: Multidimensional analysis of biomolecular sequences. The left side illustrates the diversity of molecules within Mol-Instructions, encompassing molecules of varying complexity and distinct structures. On the right side, the analysis delves into the diversity of protein sequences within Mol-Instructions considering different aspects such sequence length, domain, and activity.
training and utilization of LLMs. comprehensive exposition on task definitions and instruction construction is provided in Appendix B.
Molecule-oriented instructions center on small molecules, delving into their natural properties and behaviors. It underscores the foundational challenges of various chemical reactions and molecular design, encompassing six tasks illustrated in Figure 3 (a). The objective is to understand and predict the chemical properties of molecules. improve molecule design, and increase the accuracy and speed p tasks could expedite drug innovation and diminish developmental expenses.
of chemical reactions. In the areas of chemical and pharmaceutical design, the predictions from these
instructions are rooted in bioscience, primarily addressing issues pertinent to protein design and functionality. This segment spans five distinct categories, as detailed in Figure (b). The endeavors here are directed towards predicting protein domains, functions, and activities, and facilitating protein design via textual instructions. Understanding the fundamentals of protein function and folding is crucial for realms such as disease diagnosis, treatment, and novel drug discovery.
Biomolecular text instructions concentrate primarily on NLP tasks related bioinformatics and chemoinformatics. This category contains six information extraction and Q&A tasks as shown in Figure (c). The goals here are to interpret and extract pivotal information from biomedical literature, aiding researchers in swiftly accumulating insights and propelling their investigative pursuits.
4.2 DIVERSITY AND COMPLEXITY OF BIOMOLECULAR TRAITS
Figure 4 provides comprehensive overview of the biomolecule distribution across different dimen- sions. For clarity in visualization, we' ve truncated the long-tail segments of the data, preserving its core essence. An exhaustive analysis is available in Appendix C.
Figure 4 (a-d) showcases the diverse characteristics of molecules. Bertz complexity serves as a crucial metric for assessing molecular complexity. Molecular weight, indicative of a molecule's 's scale and complexity, is instrumental in many chemical reactions. The atom count provides insight into a molecule's dimension and complexity, influencing its stability and reactivity. The ring count offers

---

Published as a conference paper at ICLR 2024
Table 2: The predominant biomolecular characteristics encapsulated within the textual descriptions.
Features
Example
Chemical properties
combines with make flunrides sodium fhaoride and calcium flooride.
Physical properties
The molecule a colorless, flammable that has a distinct, pungent smell.
Molecule
Applications
Used as flavoring, solvent, and polymerization catalyst
Environment
The molecule a metal that occurs naturally throughout the environment, rocks, soill water, and air.
Safety
I ethal inhalation and highly toxic lothal by skin absorption.
Formation
It is formed in foods that are rich in carhobydrares when they are fried, grilled, or baked.
Function
The designed protein must be able to regulate signal transduction.
Subcellular location
The designed protein lncalizes to the mitochandrion.
Protein
Structure
target protein must exhibit
its primary conformation.
Family & Domain
The designed protein should contain PWWP domain that essential its function.
PTM / Processing
Incorporate signal pepride the protein design.
lens into structural complexity and potential stability, with implications for chemical reactivity and probable biological activity.
In Figure 4 (e-j), we delve into the attributes of proteins within Mol-Instruction. Figure (e-g) highlights the varied distribution protein sequence lengths. Categorized according to the NCBI Taxonomy, these proteins encompass expansive range of species and experimental strains, encap- sulating 13,563 protein families and 643 superfamilies. Figures (h-j) spotlight functional facets, such domain, gene ontology, and catalytic activity annotations. The data exhibits pronounced long-tail distribution, underscoring challenges in inferring functions for proteins, especially those with infrequent functions.
4.3 EXTENSIVE COVERAGE OF BIOMOLECULAR DESCRIPTIONS
In the burgeoning field of text-driven biomolecular design, our emphasis lies on the depth and diversity of biomolecular description texts.
As shown in Table 2, molecular text descriptions provide comprehensive, multi-dimensional, and in-depth view of molecular information. Regarding information depth and breadth, the listed molecular properties offer a wide perspective, ranging from basic chemical attributes to specific application contexts. Such comprehensive coverage allows text descriptions to depict molecules in a varied and layered fashion. By understanding the expressed chemical and physical characteristics, one can grasp the essential features and reactive tendencies of molecules. Further, insights into their applications, environmental prevalence, and safety aspects offer holistic understanding of their significance and associated safety considerations.
As detailed in Section $3.4, convert biological insights and functional annotations of naturally occurring proteins into text-based design specifications. Table 2 highlights the diverse features of these proteins, considering five related aspects that cover their behavior in protein folding, maturation, processing, and their contribution to life processes. Unlike traditional de novo protein design that emphasizes protein generation grounded physical principles, Mol-Instructions targets the creation of proteins with multifaceted desired traits. This challenges the models' capability to discern the intricate sampling space defined by the convergence of multiple attributes.
EXPLORING THE POTENTIAL OF MOL-INSTRUCTIONS
5.1 INSIGHTS FROM PERFORMANCE ANALYSIS
To investigate whether Mol-Instructions can enhance LLM's understanding of biomolecules, we perform instruction tuning on the three main domains of Mol-instructions, using I.Lama-BB (Touvron et al., 2023) as the foundation model. We additionally employ Alpaca-LoRA (Tloen, 2023), Baize- 7B (Xu et al., 2023), CharGLM-6B (Zeng et al., 2023), Vicuna (Chiang et al., 2023), Galactica (Taylor et al., 2022), Text+Chem T5 (Christofidellis et al., 2023), MolT5 (Edwards et al., 2022), and PMC- LLaMA-13B (Wu et al., 2023) as baselines. Our dataset is partitioned into training, validation, and testing subsets. The training and validation sets used for instruction tuning, while the test set assesses model performance. For detailed training procedures, evaluation metrics, and case studies, please refer Appendix D, E, and F.
7

---

Published as a conference paper at ICLR 2024
BLEU2
Protah Funceinn (ROUGE
Table 3: Results of molecular property prediction tasks.
Duteed rerreercs
MODEL
MAE 1
Domaishoar (ROUGE-45
Desorition (ROUGE4)
Property Prediction
ALPACA
322.109
BAIZE
261.343
Cotaliio Actn (ROUSE
CHATGLM
LLAMA
5,553
VICUNA
860.051
GALACTICA
0.568
Figure 5: The performance comparison on molecule and protein understand- ing tasks: molecule description generation (left), protein function, functional description, catalytic activity, and domain/motif prediction (right).
OURS
10:555 0.013
Table Results molecular generation tasks. These tasks encompass description-gunl guided molecule design, reagent prediction, forward reaction prediction, and retrosynthesis.
MODRI
EXACTt
BLEU
LEVENSITRIN
RDK FTST
MACCS FTS1
MORGAN FTS*
VALIDITY
Descriptioe-giided Molecude
ALPACA
0.000
0.004
$1.088
0.006
0.029
0.000
0.002
BAIZE
0,000
0.006
53.796
0.000
0,000
0.000
0.002
CHATGLM
0.000
0.004
53.157
0.005
0.000
0.000
0.005
LLAMA
0.000
0.003
59, 59.864
0,008
0.000
0.000
0.003
GALACTICA
0.000
0.192
44,152
0.135
0,248
0.088
0.992
TEXT+CHEM
0.097
0.508
41.819
0.352
0.474
0.353
0.721
MOLTS
0.112
0.546
38.276
0,400
0.538
0.295
0.773
OURS
0,002
0.345
41.367
0.231
0.412
0.147
1.000
Reugent Prediction
ALPACA
0.000
0.026
29.037
0.029
0.016
0.001
BAIZE
0.000
0.051
30.628
0,022
0.018
0.004
0.099
CHAEGLM
0,000
0.019
29.169
0.017
0.006
0.002
0.074
0.001
VICUNA
0,000
0.010
27.948
0,038
0.002
0.001
0.007
GALACTICA
0.127
TEXT+CHEM TS
0.000
0.225
49.323
0.039
0.186
0.052
0.313
OURS
0,044
0.224
23.167
0.237
0.364
0.213
1.000
Forward Reaction Prediction
0.000
0.004
0.024
0.008
0.138
BAIZE
0.000
0.044
41.500
0.004
0.025
0.009
0.097
CHATGLM
0,000
0.183
40.008
0.050
0.100
0.044
0.108
0.001
VICUNA
0.000
0.057
41,690
0.007
0.016
0.006
0.059
GALACTICA
0.257
TEXT+CHEM T5
0.239
0.782
20.413
0.705
0.789
0.632
0.762
OURS
0,045
0.654
27.262
0.313
0.509
0.262
1.000
Retrosynchesis
BAIZE
0.000
0.095
44.714
0.025
0.050
0.023
0.112
CHATGLM
0,000
0.117
48.365
0.056
0.075
0.043
0.046
LLAMA
0.000
0.036
46.844
0,018
0.029
0.017
0.010
VICUNA
0,000
0.057
46.877
0.025
0.030
0.021
0.017
GALACTICA
0.167
0.274
0.134
TEXT+CHEM T5
0.141
0.765
24.043
0.685
0.765
0.585
0.698
OURS
0.009
0.283
0.487
0.230
Assessing the accuracy of L.M results in the life sciences field inherently complex. Subjecting every output expert review wet-lab validation would be both time-consuming and impractical. Moreover, given that LLMs can produce varied outputs for the same input, ensuring uniform accuracy across all potential outcomes is a formidable challenge. While we utilize metrics commonly accepted in broader domains to evaluate output quality, these metrics capture only a slice of the overall picture. Quantitative experiments may shed light on performance on certain tasks, but they fall short of comprehensively addressing the evaluation challenges.
As shown in Figure 5, Mol-Instructions enhances the molecular understanding capabilities of ILMs,, demonstrating notable improvements in every metric compared to baseline models, including even domain-specific smaller models. Notably, LLMs exhibit an impressive aptitude for predicting molecular properties, detailed in Figure 3. In this specific task. Alpaca generated answers for mere 2.62% of the samples, Baize for 0.62%, LLama for 54.5%, Vicuna for 0.14%, Galactica for 74%, whereas ChatGLM refrained from responding. Regarding molecular generation tasks, as depicted in Table 4, LLMs exhibit the capability to generate valid molecules, and compared to the baseline, these generated molecules exhibit higher degree of similarity reference molecules. This demonstrates
8

---

Published as a conference paper at ICLR 2024
Baize
ChaIGLM
Galactica
LLama
G
Viouna
PCM-LLams
Ours
ADC
BLEU
ROUGE-1
BarBare
ADC
F1
F1
F1
0.6
0.00
0.25
0.04
0.7
0.8
0.4
0.20
0.20
0.20
0.-
0.01
0.15
0:10
0.80
04
0.2
0.10
0.16
True
Open Question
Mutt-cholce
(CJER
(COJEE
(OPYE
Figure 6: Results for bioinformatic NLP tasks. (C)ER denotes chemical entity recognition, (C-D)IE signifies chemical-disease interaction extraction, and (C-P)IE stands for themical-proteiis interaction extraction.
that Mol-Instructions equips LLMs with new insights into molecular generation, chemical reaction prediction, and the synthesis of molecules based on specific instructions. However, the molecule generation capabilities of LLMs still exhibit discernible gap when compared to specialized smaller models, mainly because LLMs are designed to handle a wider range of tasks at the expense of the specialized performance seen in more focused models.
As demonstrated in Figure 5, for various protein understanding tasks, the LLM tuned with Mol- Instructions can analyze proteins in accordance with specific requirements and exhibits a noteworthy capability in identifying fundamental protein characteristics. For the protein design task, the de- termination of target functional features for the generated sequences challenging experimentally. Therefore, as shown in Appendix F, we employ straightforward approach of aligning the de novo protein sequence to the BniProtK using BLAST (Camacho et al., 2009). We observe significant sequence identity between the generated sequence and multiple proteins in the corresponding functional regions. Specifically, we identify the protein (UniProt Accession: A0A518LQL6) as the target protein to perform alignment, as it exhibits the highest sequence identity (40.9%, with a p-value of 7.7e - 30) with the generated sequence. Notably, the 18LL6 protein and the generated protein share similar sequence signatures in the (6S)-NADPHX and metal ion binding regions (residues 1 1-200). This finding suggests that the generated protein fulfills the design requirements and has the potential for NADPHX epimerase activity.
validate
Figure 6 shows the model's proficiency in comprehending biomolecular text instructions. By infusing the LLM with substantial knowledge from the biomolecular domain, the model demonstrates superior performance compared to the baseline in all NLP tasks. This further underscores the transformative potential of Mol-Instructions in bridging the gap between LLMs and intricate biomolecular studies.
5.2 HARNESSING THE POWER OF MOL-INSTRUCTIONS
To optimize researchers' utilization of Mol-Instructions, we suggest three pivotal directions to enhance general model exploration and drive progress in biomolecular understanding and drug design: First, use Mol-Instructions to assess cross-modal comprehension in general models, transitioning from human to life languages. These models should interpret user intentions and decode the biomolecular language, challenging their reasoning capabilities. Second, our work lays the groundwork for deeper biomolecular design exploration. Mol-Instructions cover a wide range of design criteria, by providing data related to biomolecular property prediction tasks, bolsters model understanding of biomolecules. Third, employ Mol-Instruction as essential data for tool learning in addressing complex biological issues. Though research highlights the advantages of specialized tools with foundational models (Qin et al., 2023), gap exists in the availability of textual instructions, which our dataset addresses.
CONCLUSION AND FUTURE WORK
In this work, we introduce Mol-Instructions, a comprehensive instruction dataset specifically curated for biomolecular studies, bridging the gap in current resources and advancing LLM training in this specialized sphere. Looking ahead, we are committed to the ongoing enrichment and refinement of Mol-Instructions, We will incorporate a wider array of task types, instruction entries, and modalities, in line with the latest advances in chemical research and improvements in AI technology to meet broader chemical research needs and higher-level LLM training requirements. Moreover, given the distinct representation spaces of text and biomolecules, coupled with the limitations imposed by LoRA's training strategy, current LLMs have yet to master the biomolecular language proficiently as they do human language. Exploring methods to expand the vocabulary, or incorporating bio language modality via biomolecular encoders (Fang al., 2024; Rives et al., 2021; Lin et al., 2022; Cao et al., 2023; Pei et al., 2024), could be pivotal in honing the model's understanding and performance in biomolecular tasks.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We would like to express gratitude to the anonymous reviewers for kind comments. This work was supported by the National Natural Science Foundation of China (No. 62206246), the Fundamental Zhejiang Provincial Natural Science
Research Funds for the Central Universities Foundation of China (No. LGG22F030011,, Ningbo Natural Science Foundation (2021J190), CAAl-Huawei MindSpore Open Fund, Yongjiang Talent Introduction Programme (2021A-1566 CCF-Baidu Open Fund, and Information Technology Center and State Key Lab of CAD&CG, Zhejiang University.
REPRODUCIBILITY STATEMENT
All data, code, and model weights be found GitHub and Hugging Face 2,3,4.5 For detailed description of the dataset construction process, please refer to Appendix B. For specific experimental settings, please see Appendix D and E.
ETHICS STATEMENT
This study was carried out in strict accordance with ethical guidelines and best practices in research. The biomolecular data utilized were sourced from publicly available datasets, and no proprietary or confidential data were used. Furthermore, we have obtained the necessary permissions and licenses for all third-party content incorporated in our dataset, as detailed in Appendix A.2.
Rigorous quality control measures and security checks have been implemented preclude the presence of any harmful malicious content in our dataset. However, we recognize the profound implications and potential risks associated with the integration of LLMs and biomolecular knowledge. While our primary intent is to advance scientific understanding and contribute positively to society, we are acutely aware that these tools, in the wrong hands, could be misused. There exists the potential for malicious actors harness the combined capabilities of L.MSs and biomolecular data generate harmful substances, such as biochemical weapons or illicit drugs.
We strongly urge all users adhere to the highest ethical standards when using our dataset, ensuring fairness, transparency, and responsibility in their research. Any usage of the dataset that may lead to harm or pose detriment society is strictly forbidden.
REFERENCES
Sharegpt, April 2023, URL wtt.ps://ssst.... htttps://shareggt..oo com/.
Yuvanesh Anand, Zach Nussbaum, Brandon Duderstadt, Benjamin Schmidt, and Andriy Mulyar. Gpt4all: Training an assistant-style chatbot with large scale data distillation from gpt-3.5-turbo. GitHub, 2023.
Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan Davis, Kara Dolinski, Selina Dwight, Janan Eppig, et al. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25-29, 2000.
Satanjeev Banerjee and Alon Lavie. METEOR:
an automatic metric for MT evaluation with improved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin- Yew Lin, and Clare R. Voss (eds.), Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization @ ACL 2005, Ann Arbor, Michigan, USA, June 29, 2005, pp. 65-72. Association for Computational Linguistics, 2005. URL https: antthhollanotoocccoooooooooooooa ogy .org/ W05-09091
'Gitllu: http:///s...ettaaooc.////aaaaaaa 2 DDtassets: http:///s/.ccoiooeooe//e//e/e//a "Molecule model: http:////g//gggghheeeece///////e
PProtein model: ht hwtp:///pp./gggggo..e...///////a
Biotext model: http:////hgggggooo...oo////////a
10

---

Published as a conference paper at ICLR 2024
Daniil A. Boiko, Robert MacKnight, and Gabe Gomes. Emergent autonomous scientific research capabilities of large language models. CoRR, abs/2304.05332, 2023, doi: 10.48550/RXXIV25004 05332. URL /tttps:///oi.. org/10. 48550/arXiv, 2304 05332.
Andres M Bran, Sam Cox, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting large-language models with chemistry tools. arXiv preprint urXiv:2304.05776, 2023.
Christiam Camacho, George Coulouris, Vahram Avagyan, Ning Ma, Jason S. Papadopoulos, Kevin Bealer, and Thomas L. Madden, BLAST+: architecture and applications. BMC Bioin- Sorm., 10:421, 2009. doi: 10. 10.1186/1471-20055 10-421. URL https://doo 10g/10.01. 1186/
1471 - 2:00--1-111 10-421.
He Cao, Zijing Liu, Xingyu Lu, Yuan Yao, and Yu Li. Instructmol: Multi-modal integration for building a versatile and reliable molecular assistant in drug discovery. CoRR, abs/2311. 162222211,,608,, 2023. doi: 10.48550/ARXIV23111 16208. URL https: // //doi..oon 0tgg/0.48550//
2311.16202.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Hemrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel [lerbert-Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained code. CoRR, abs/2107.033744 2021. URL https: org/abs// / 2137703374.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An sour- chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023. URL https:
// //tmsys..rg//llll..////. 2023 03-30----0 cuna/.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay. Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Lev- skaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontack Lim. Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayans Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy
Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. Mach. Learn. Res,, 24:240:1-240; 13, 2023. URL http: / jml org/ httpp/pppppssaaepp/p 2222 1144 htmll
Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Mat- teo Manica. Unifying molecular and textual representations via multi-task language mod- elling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 6140-6157, PMLR, 2023. URL https: pptpppeeeeegn...eeeeeeppppep..ee mlr. www..m.re...ee..... pr22s//202// christofidel 1ilsllisssblla htmml...... html.
Mike Conover, Matt Hayes, Matt Mathur, Xiangrui Meng, Jianwei Xie, Jun Wan, Ali Ghodsi, Patrick Wendell, and Patrick Zaharia. Hello dolly: Democratizing the magic of chatgpt with open models, 2023. URL https: /ttpt/ggtttt/tt//// com/ abt waw.laas/ dolly.
The UniProt Consortium. Uniprot: the universal protein knowledgebase in 2023. Nucleic Acids Res., 11(1)):523553) 2023. doi: 10. 10.103/NAR/GKAC1002. URL https: : htt////tgg/////////////o 1093/ nar/gkac10s

---

Published as a conference paper at ICLR 2024
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob de Haas, Neville Bethel, et al. Robust deep learning- based protein sequence design using proteinmpnn. Science, ():4666615)4499555 2022.
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pp. 3029-3051. Association for Computational Linguistics, 2023. URL nttps: //aclanthology org/ 20232 emnip -aa-maain...... 183.
Joseph L. Durant, Burton A. Leland, Douglas R. Henry, and James G. Nourse. Reoptimization of MDL keys for use in drug discovery. J. Chem. Inf. Comput. Sci., 42(5)) 1280, 2002. doi: 10.1021/C1010132RR URL https: doi. org/ 10. 1021/ / c1 010132r.
Carl Edwards, Tuan Manh Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. Translation between molecules and natural language. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-/ 1, 2022, pp. 375-413. Association for Computational Linguistics, 2022. doi: (026653/V/2022.MMMNM -AL... URL attps: wwwie main.
///oii.og//// /10
Yin Fang, Ningyu Zhang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Domain aagnosic molecular generation with sef--eeedback In ICLR. OpenReview.net, 2024. URL
net/pdf? fippdiiiiio drrrr FWP.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Viriual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https: http::::///ppervvie.ee net/ Forum?id-da (...ddd7BBjjjB KBBm1B333mm
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. CoRR, abs/2203. 15556, 2022. doi: 10.48550/ARXIV2220000 155556 URL https::tp//......t...ttt http:///ooooooooooooo/ooooo
.org/10 /0/ 10100.8550//ra v. 2203. 15556.
Sunghwan Kim, Jie Chen, Tiejun Cheng, Asta Gindulyte, Jia He, Siqian He, Qingliang Li, Benjamin A. Shoemaker, Paul A. Thiessen, Bo Yu, Leonid Zaslavsky, Jian Zhang, and Evan Bolton. Pubchem in 2021: new data content and improved web interfaces. Nucleic Acids Res., 49(Database-Issue)) D1388-DI D1395, 2021. /
doi: 10. 10999//A//GKAA9TT URL https: http://c.oo/////// 10tg///0..11///////
Martin Krallinger, Obdulia Rabal, Florian Leitner, Miguel Vazquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M. Lowe, Roger A. Sayle, Riza Theresa Batista- Navarro, Rafal Rak, Torsten Huber, Tim Rocktäschel, Sérgio Matos, David Campos, Buzhou Tang, Hua Xu, Tsendsuren Munkhdalai, Keun Ho Ryu, S. V. Ramanan, P. Senthil Nathan, Slavko Zimik, Marko Bajec. Lutz Weber, Matthias Irmer, Saber A. Akhondi, Jan A. Kors. Shuo Xu, Xin An, Utpal Kumar Sikdar, Asif Ekbal, Masaharu Yoshioka, Thaer M. Dieb, Miji Choi, Karin Verspoor, Madian Khabsa, C. Lee Giles, Hongfang Liu, Ravikumar Komandur Elayavilli, Andre Lamurias, Francisco M. Couto, Hong-Jie Dai, Richard Tzong-Han Tsai, Caglar Ata, Tolga Can, Anabel Usie, Rui Alves, Isabel Segura-Bedmar, Bedmar, Paloma Martinez, Julen Oyarzabal, and Alfonso Valencia. The CHEMDNER corpus of chemicals and drugs and its annotation principles. J. Cheminformatics, 7(S-1):S2, 2015. doi: 10.1186/175 29-----94----22 URL 10. 10.1186/11/8-2945 7-51-s2.
.org/
Martin Krallinger, Obdulia Rabal, Saber Ahmad Akhondi, Martín Pérez Pérez, Jesus Santamaría, Gael Pérez Rodríguez, Georgios Tsatsaronis, Ander Intxaurrondo, José Antonio Baso López, Umesh K. Nandal, Erin M. van Buel, Ambika Chandrasekhar, Marleen Rodenburg, Astrid Lægreid, Marius A. Doomenball Julen Oyarzábal, Anália Lourenço, and Alfonso Valencia. Overview the biocreative vi chemical-protein interaction track. 2017.
12

---

Published as a conference paper at ICLR 2024
Mario Krenn, Qianxiang Ai, Senja Barthel, Nessa Carson, Angelo Frei, Nathan C. Frey, Pascal Friederich, Théophile Gaudin, Alberto Alexander Gayle, Kevin Maik Jablonka, Rafael F. Lameiro, Dominik Lemm, Alston Lo. Seyed Mohamad Moosavi, José Manuel Nápoles-Duare mar Nigam, Robert Pollice, Kohulan Rajan, UIrich Schatzschneider, Philippe Schwaller, Marta Skreta, Berend Smit, Felix Strieth-Kalthen Andrew Wang. Andrew D. White, Adamo Young, Rose Yu, and Alán Aspuru-Guzik. SELFIES and the future of molecular string representations. Putterns, 3(10): 2022. doi: 10.1016/J, PATTER.2022.1005 100588. URL https:://///tt owg/lo.///ooooo 101./ patter patee. 2022.22222222222 100588.
AkshatKu-
Chong Sun, Gary Tom, Guido Falk von Rudorff,
Greg Landrum et al. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling. Greg Landrum, 8:31, 2013.
Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn Mattingly, Thomas C. Wiegers, and Zhiyong Lu. Biocreative CDR task corpus: resource for chemical disease relation extraction. Database J. Biol. Databases Curation, 2016, 2016. doi: 10.000//DATABAS//BAAWOO URL https:://ol. org/ 1093/ abaso/ bB
Yujian Li and Bi Liu. normalized levenshtein distance metric. IEEE Trans. Pattern Anal. Mach. Intell., (996))001110999 2007. doi: 10.1109/7AAML2200.000788 URL https: http:p/://oooogg//////.
1119/IPAMI. 2007.1078.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022.
Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar. Multi-modal molecule structure-ted model for text-based retrieval and editing. Nat. Mac. Intell., 5(12); 2023. doi: (0.1038/842256-033-0000 URL ht test:p::0 / doi. org/10.... 1038 42256 000000203000000000 00753-6.
Jieyu Lu and Yingkai Zhang. Unified deep learning model for multitask reaction predictions with explanation. J. Chem. Inf. Model., 6276):1376 2022. doi: 1..12//AASS.CIM..C00444
URL https: doi. org/ 10. 1021 / acs. jeim. 1c01467.
Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families. Nature Biotechnology, pp. 1-8, 2023.
Cayque Monteiro Castro Nascimento and André Silva Pimentel. Do large language models understand chemistry? A conversation with chatgpt. J. Chem. Inf. Model., 634))1669- 2023. doi: 10.1021/ACS.JCIM..CCO
URL https://ddat org/ 10. 1021 accs Jeim. 3c00285
OpenAl, GPT-4 technical report. CoRR, abs/2303.08774, 2023a. doi: 10.46550/ARXX1223000077744
URL https://doi. 1ttp:/...0000///11.2200000 087744
OpenAl. Gpt-3.5-turbo. http::///aaatcccooooooccaccceerr http://///////////////////o//opa 2023b.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Sanmi Koyejo. S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022a. URL http: / papers nips wwwp.cc.//aeeccc...ioooooooocecc wwwppp/ppppp..e.eeeeeeeeeeee..ee files/ walles/pppee//222. ha h httpbbeeffeeeeeeeeeee 7391 007339445880500000
Abat wwweeeeneeecccom htmml. html.
13

---

Published as a conference paper at ICLR 2024
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Chris- tiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022b. URL http: http:/pppprss... wttp:ccpppppppaepeeeee pellles/pper//22// hash/ble (e 011.3300111111110000000110110000
Cow.rraccc--ooffenceee.. html.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa: A large-scale multi-subjert multi-choice dataset for medical domain question answering. In Gerardo Flores, George H. Chen, Tom Pollard, Joyce C. Ho, and Tristan Naumann (eds.), Conference on Health, Inference, and Learning, CHII. 2022, 7-8 April 2022, Virtual Event, volume 174 of Proceedings of Machine Learning Research, pp. 248-260. PMLR, 2022. URL httpaa//ppccceeeeeeeeeeee .mir. press/vi744 /1/74//pp22... html.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pp. 311-318, ACL, 2002. URL https:////// httpppp///ccaaaaiee /tt.aaaaaatttooy..oooooooooo.att wwwp:/o-1/022000000
doi: 10.311511073033.1733333
Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. arXiv preprint 24X. 2024.
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao. Chi Han, Yi Ren Fung. Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye. Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong. Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, and Maosong Sun. Tool learning with foundation models. CoRR, abs/2304.08354, 2023, doi: 08354.
URL hotps: http:s///dd/ //wp.//o..oooo/// org/ 0////11.485555 ww5.4888500.i.....
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. Biological structure and function emerge from scaling unsupervised learning 250 million protein sequences. Proc. Natl. Acad. Sei. USA, 1 118(15)e2016223111 18, 2021. doi: 10. 10.1073/PNAS.20163399 org /00 1 1073 201 6239118.
https: //oi..
Jonas B. Sandbrink. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. CoRR, abs/2306. 1392220...192,, 2023. doi: 10.48850/ARXIV.2300. 13952. URL httph:tt:///ttt //coi htt//oi..rrgrg// 100/10..000 10550//axxiv.. 2306. 13952.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang. Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj. Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https:://// http:////peeeeiiieeee net forum? ii VV
Nadine Schneider, Roger A. Sayle, and Gregory A. Landrum. Get your atoms in order an open- source implementation of novel and robust molecular canonicalization algorithm. J. Chem. Inf. Model,, 55(10):2111 2015. doi: 10. 10.002//ACS.CCI..50005.. URL https: pttts://ddos http:::////oo/r/oe/e//oos org/
10. 1021 / acs.jaim. jaim. 5b00543.
14

---

Published as a conference paper at ICLR 2024
Robert F Service. Could chatbots help devise the next pandemic virus? Science (New York, NY), 380 (6651):1211, 2023.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology. 35(11):1026-102 2017.
Bing Su, Dazhao Du, Zhao Yang, Yujie Zhou, Jiangmeng Li, Anyi Rao, Hao Sun. Zhiwu Lu, and Ji-Rong Wen. A molecular multimodal foundation model associating molecule graphs with natural language. CoRR, abs/2209.05488 2022, doi: 10.48500ARXIV.200.05555
URL
https: //tp:i//dc...e// 1at.110...550/aa111::1 /ttt::///dd..coo.....55/a/aa 2209 05481.
Taffee T Tanimoto. Elementary mathematical theory of classification and prediction. 1958.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instmuction-follovinng llama model. GitHub repository, 2023.
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. CoRR, abs/2211.09000 1.0905,, 2022. doi: 10.48550/ARXIV2221 1..0009... URL pttps://doi..oo (a550/arX10 2.w.rriii..22........m..m 2211 .90855.
Robert Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Fine-tuning large neural language models for biomedical natural language processing, Patterns, 4(4):100729, 2023. doi: 1..00166.AATTER.2225..00
URL https:
//ddi.org///// 101 10 patter 2023. 1102221.000222 100229.
Tloen. Alpaca-lora. ht http//ggttttttb com/ wtam//losse tloen/ alpaca-ppaaoo lora, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie Anne Lachaux, Timothéc Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302.13971, 2023. doi: 1044550/ARXIV.200..133 org/10 10. 118550//aXiii 2302.13971. 13971.
URL https: //doi.
Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, Anima Anandkumar, Karianne Bergen, Carla P. Gomes, Shirley Ho, Pushmeet Kohli, Joan Lasenby, Jure Leskovec, Tie-Yan Liu, Arjun Manrai, Debora S. Marks, Bharath Ramsundar, Le Song. Jimeng Sun, Jian Tang, Petar Velickovic, Max Welling, Linfeng Zhang, Connor W. Coley, Yoshua Bengio, and Marinka Zitnik. Scientific discovery in the age of artificial intelligence. Nat., 620(7972):47-60, 2023a. doi: 10.1038/ $411886-023-06221-2 URL https: //dci org/10/ 10. 1038/ 4 586 02005-023--222220 06221-2.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 13484-13555 13508. Association for Computational Linguisties, 2023b. doi: 2023. wwle-loogg-- 754.
URL https: / //wp///o...o//// /tt.//ooo//o/o//////// httpg//1..1163./11 18653/v1/
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du Andrew M. Dai, and Quoc V. Lc. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https: h opw.reviewweeeeeeeeeeeeee.reeewe net//oorum2id-g52rGGe
Jin-Mao Wei, Xiao-Jie Yuan, Qinghua Hu, and Shu-Qin Wang. A novel measure for evaluating classifiers. Expert Syst. Appl., 37(5) 3809, 2010. doi: (0.1016/..EWAA2000 1..40. URL
https://dci. org.i..rgg11 1016/1 www... www.. 2009. 19.00...................1...1001
David Weininger. Smiles, chemical language and information system. 1. introduction to methodol- ogy and encoding rules. J. Chem. Inf. Comput. Sci., 28(1):31-36, 1988. doi: 10.1001/C100057:0000
URL https: //doi /ttp://..........1....0000000000
15

---

Published as a conference paper at ICLR 2024
Jacob White. Pubmed 2.0. Medical reference services quarterly, 3944)382-388,
Chaoyi Wu, Xiaoman Zhang, Ya Zhang. Yanfeng Wang, and Weidi Xie. Pmc-Ilama: Further finetuning Ilama on medical papers. CoRR, 00s//304.14454, 2023, doi: 10.48850/ARXIV.2300. 14454. URL https:
48550/arxis 230..................... 14454.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg. Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 9():513-5330 2018.
Canwen Xu, Daya Guo, Nan Duan, and Julian J. McAuley. Baize: An open-source chat model with parameter-efficient tuning self-chat data. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. EMNLP 2023, Singapore, December 6-10, 2023, pp. 6268 6278. Association for Computational Linguisties, 2023. URL https: http:///caaaaacchccccc org/ 2023. emnlp peannpp-mmne main.ai..... 385.
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. BLLM-130B: an open bilingual pre- trained model. The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https:://pperreeeeeeeeee /wtp///wwww.eeeeeeeeepee.eoe.eep net peffpf -AWUrrrPUF.
Zheni Zeng, Yuan Yao, Zhiyuan Liu, and Maosong Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nature communications, 13(1):862, 2022.
Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang. Chenghua Lin, Wenhao Huang, and Jie Fu. Chinese open instruction generalist: preliminary release. CoRR, abs/2304.07987, 2023a. doi: 10448500ARXIV.230000759 URL https: //cci. wrr. 10. 48w50//r2iv. 2304............. 07987.
Jinlu Zhang, Yin Fang, Xin Shao, Huajun Chen, Ningyu Zhang, and Xiaohui Fan. The future of molecular studies through the lens of large language models. J. Chem. Inf. Model, 64(3): 563-566, 2024. doi: 11102/AACSJC...30011 3c01977
URL https:/ :ttps://coi.. http:///oooog//////// .org/oorg/o100 110 1021/acs.
Weitong Zhang, Xiaoyun Wang, Weili Nie, Joe Eaton, Brad Rees, and Quanquan Gu. Moleculegpt: Instruction following large language models for molecular property prediction. In NeurlPS 2023 Workshop on New Frontiers of Al for Drug Discovery and Development, 2023b.
16

---

Published as a conference paper at ICLR 2024
A ACCESSING AND UTILIZING MOL-INSTRUCTIONS
A.1
HOSTING AND ACCESS DETAILS
Our dataset and associated models are securely hosted on GitHub and Hugging Face, which are well-recognized platforms for managing open-source projects. They provide vast accessibility and efficient management of extensive data and code repositories, guaranteeing unobstructed access to all potential users. To enable efficient usage of our resources, we provide comprehensive guidelines and instructions on repository. These cover how to explore the dataset, understand its structure, and utilize the models.
A.2 DATA SOURCES AND LICENSE
As shown in Table 5, we detail the sources and data rights for all data components used in constructing the Mol-Instructionsddaasss,, including both biomolecules and textual descriptions. All data sources rigorously scrutinized to ensure their licenses permit kind of research and subsequent usage. Throughout the article, every mention or use of these data sources is properly and accurately cited.
A.3 USAGE GUIDELINES AND OBLIGATIONS
We assert that all data included this work comply with the stipulations of the CC BY 4.0 License. We acknowledge our responsibility as licensors to facilitate open and fair use of the dataset while recognizing the creators' contributions. We accept all obligations related to enforcing this license and pledge our commitment to assist all potential users in understanding their rights and responsibilities under this license.
We assure that our dataset does not contain any personally identifiable privacy- -eessiiveee information. We have enforced stringent quality control procedures and security checks to prevent the inclusion of harmful malicious content. However, it is important to note that while have taken considerable measures to ensure the dataset's safety and privacy, we do not bear responsibility for any issues that may arise from its use.
We emphatically urge all users to adhere to the highest ethical standards when using our dataset, including maintaining fairness, transparency, and responsibility in their research. Any usage of the dataset that may lead to harm or pose a detriment to society is strictly forbidden.
In terms of dataset maintenance, ensure the continued relevance and usability of the dataset in light of evolving research landscapes. This commitment encompasses regular updates, error checks, and amendments in accordance with field advancements and user feedback.
pledge our commitment to provide necessary upkeep. This will
B TASK DEFINITION AND DATA CONSTRUCTION
B. MOLECULE-ORIENTER INSTRUCTIONS
Molecular description generation Molecular description generation entails the creation of detailed textual depiction illuminating the structure, properties, biological activity, and applications of a molecule based on its molecular descriptors. It furnishes chemists and biologists with a swift conduit to essential molecular information, thus efficiently guiding their research and experiments.
To gather ample and authoritative molecular text annotation data, we chose PubChem (Kim et al., 2021) as the data source. PubChem, freely accessible database managed by the National Center for Biotechnology Information (NCBI), a valuable resource for chemical research. Many compounds in PubChem have text descriptions that come from direct submissions from various research institutions, and automated data mining and information extraction from scientific literature and patents.
Firstly, we commence with PubChem's 's Power User Gateway, which provides abstracts of PubChem compound records in XML format, facilitating the efficient retrieval and processing of chemical information. We crawl all valid molecular description texts along with the unique PubChem Chemical
17

---

Published as a conference paper at ICLR 2024
Table 5: Data resources and licenses involved in our paper.
Data Sources
License URL
License Note
PuhChem
pe t// //oee mnlm hugzz/ peb I cieashtal
Works produced by U.S. govermment are not subject to cupyrighi protection in the United States. Any such works
on National Library Medicine (NLM) Web sites may be freely used reproduced without permission in the U.S.
USPTO
pi // w/ww now uaptin i goo/ Jeazning- and pesources/ open- daatt-aadd--aant liby hrteps //one uniprot oorg/ hel p/1 Soeese
It can be freely used, reused, and redistributed by anyone.
UniProcKB
You are free to: Share copy and redistribute the material in any medium format. Adapt remisx transform, and build upon the material for any purpose, even commercially.
BCACHEMD, ChemProt, BCSCDR
hotps t/ //biocrestive
The task data is freely available now for the re- search community.
in fornat cudel
edu/ / Lasks /bsocreat Livevv ue track- 2 odx/ hetps t// /ppensource.crg 1 connse/mit///
MolsculeNet, MMLU, PubModQA, MedMCQA
Permission is hereby granted, free of charge, to any person obtaining copy of this software and associated documentation deal in the Soltwrre without restrictiin, including without limitation rights use, copy. modify. morge, publish, distribute, sublieense, and/or sell copies of the Software, and permit persons whom the Soflware is furmished to do so.
"Software") " to
Aguacy for Tosic Substances and Discase Registry (ATSDR)
hrttps /www. www.wwdd...... gov/Othez// disclainer.htm
This site uses the AddTui service allow visitors to bookmark and share website content on a variety of social medin sites. Visitors who use the AddThis service share content provide any personal information.
need to register or
FDA Pharm Classes
hctps: / /hooe fdd. gow! about- faw/addaauttte /bout website/ webaltee por ioles
FDA Unless otherwise noted. the contents website (www.fda, fow.w.d..oo...... both text and graphics, are not copyrighted. They and may be republished, reprinted and otherwise used froely by anyone without the need obtain permission from FDA. Credit the U.S. Food and Drug Administration as the source is appreciated but not required.
the puhlic domain
LiverTox
hetps // //ww//.ouu www..nlin nw goow/ copy ighht Jacnl
Works produced by the U.S. government are not subject to cupyrighi protection the United States. Any such works found on National Library of Medicine (NLM) Web sites may be froely used or reproduced without permission in the U.S.
Drug Database, Siinicalinoo.hiv.oom
actipe //wwe b.v.. gou/ about-us wastttusnnniioooo and- tean
Unless otherwise noted, material presented the HIV.gov website is considered Federal government information and is in the public domain. That means information may freely copied and distributed.
Drug Bank
acttps //cseat Lvecommnns. arg/licenser biconses/by-nc//// legslcode
Subject to the terms conditions of Pub- lic License, the Licensor hereby grants You a worldwide, roy royalty-free, non-subliiensable, non- exclusive, irrevocable license to exercise the Li- censed Rights the Licensed Material repro- duce and Share the Licensed Material, in whole in part, for NonComeeriial purposes only: and produce, reproduce, and Share Adapted Material for NonCommercial purposes only.
ChEBI
actps://esade www./moos.....oooooo............ htgplicoonee/by/1.00
are free to: Share copy and redistribute the material in any medium or formar. Adapt remix, transform, and boild upon the material for any purpose, even commercially.
Yeust Metaholome Database (YMDB)
actp: / http:///wwwwmm. yoda..ddo... ea/ downlcads hotps: ://ptts neto
YMDB is offered to the public freely available resoucce.
LOTUS the natural products occurrence databose
LOTUS is one of the biggest and best annotated re- sources for natural products occurrences available free of charge and without any restriction.
GlyCosmes Glycoscience Portal
actps: Dttpas//syyccnma..oo license
are free to: Share copy and redistribute the material in any medium or format, Adapt remis, transform, and hoild upon the material for any purpuse, even commercially. CAMEO Chemicals and all other CAMEO prod- ucts are available at no charge to those organiza- tions and individuals (recipients) responsible for handling chemicals.
CAMBO Chemicals
actps sameocchnniccaas.... noca. noza.gor/bet hwtp/hlp///////ecooo reference/
aa__and__oooo coc.ecnniioon Lons hhmm????? d_f=false
E. coli Metabolome Database (ECMDB)
hatps:/ s//ccdd. ca/ /icttationn
ECMDB is offered to the public as a freely avail- able resouree.
Identifier (CID) corresponding to the molecule, and employ the CID to retrieve the corresponding SMILES representation from all compounds documented in PubChem. Secondly, given that the SMILES provided by PubChem may not always be accurate, we filter out SMILES that contain
18

---

Published as a conference paper at ICLR 2024
syntactic errors violate fundamental chemical principles, and convert all valid SMILES into SELFIES format. Note that we carry out this step in every task, and it will not be reiterated in the subsequent sections. Thirdly, to compel the model to focus the semantics of the description, follow Edwards et al. (2022) to replace the molecule's name in every description with "the molecule is.... Finally, call the gpt-3.5-turbo generate diverse set of task descriptions via prompts, which we then randomly assign to each SELFIES-descripee pair. We have compiled a total of 331,261 instructions.
Description-guided molecule generation The significance of description based molecule gener- ation lies in its potential to streamline the process of molecular design by enabling the production of molecules that directly meet the criteria outlined in a given description. This facilitates a more targeted approach in the creation and optimization of novel molecules, with applications in diverse fields such as drug discovery and materials science.
In this phase, we repurpose the SELFIES-descripte pairs amassed from PubChem in the preceding task. Unlike before, in this instance, the instruction entries have the molecular description as the input and the SELFIES representation of the molecule as the output. We subsequently fabricate a series of task descriptions for description-guin -uided molecule generation serving as the instruction. This results compilation of 331,261 ,226 instruction data entries.
Forward reaction prediction Forward reaction prediction pertains to the anticipatory determination of the probable product(s) of chemical reaction, given specific reactants and reagents. This facilitates the optimization of research and development methodologies, curbs the extent of experimental speculation, and endorses greener chemistry practices by mitigating waste production.
To collect high-quality chemical reaction data, we focus on the USPTO dataset (Wei et al., 2010). This dataset encompasses a multitude of organic chemical reactions in SMILES format, extracted from American patents and patent applications. The general format is "reactant > reagent > product".
Typically, reagents are defined as chemical substances that do not appear in the main product. However, in an effort to simulate the real-world scenario of non-chemically specialized individuals, we undertake a more challenging task, one that does not separate reagents and reactants but requires secure 138,768 standard instruction
the model to identify them independently. After preprocessing, data entries. In each of these instruction entries, the input signifies the reactants and reagents in the reaction, with individual chemicals separated by period (t.). Conversely, the output denotes the product this reaction.
Retrosynthesis Retrosynthetic analysis is a pivotal synthetic methodology in organic chemistry that employs reverse-enginen ceveise-engineering approach, initiating from the target compound and retroactively tracing potential synthesis routes and precursor molecules. This technique proves instrumental in sculpting efficient synthetic strategies for intricate molecules, thus catalyzing the evolution and progression of novel pharmaceuticals and materials.
In this task, we concentrate exclusively on single-step retrosynthesis. Our data collection comes from the manually processed USPTO_500MT dataset (Lu & Zhang, 2022), which itself is meticulously refined from the USPTO database (Wei et al., 2010). Through preprocessing, we have obtained 143,536 standard instruction entries. In each entry, the input the product, and the output is the reactants, with each reactant separated by period (t.).
Reagent prediction Reagent prediction endeavors to ascertain the suitable catalysts, solvents, or ancillary substances required for a specific chemical reaction. This endeavor facilitates chemists in uncovering novel reaction types and mechanisms, identifying more optimal eco-friendly reaction conditions, and ultimately streamlining the comprehensive chemical process to attain maximal cost-effectiveness and environmental stewardship.
Like retrosynthesis, the data for this task also originates from the USPTO -50K dataset (Lu & Zhang, 2022). After processing, have approximately 138,768 entries. In each instruction entry, the input a chemical reaction from reactants to product, formatted as "reactant > > product". The expected output is the potential reagents that facilitate this reaction.
19

---

Published as a conference paper at ICLR 2024
Property prediction Property prediction involves forecasting estimating molecule's inherent physical and chemical properties based on information derived from its structural characteristics. It facilitates thhhhhhruughput evaluation of an extensive array of molecular properties, enabling the virtual screening of compounds. Additionally, it provides the means to predict the unknown attributes of new molecules, thereby bolstering research efficiency and reducing development times.
In this task, we primarily focus on the quantum mechanics properties of molecules, with data drawn from the QM9 dataset of MoleculeNet (Wu et al., 2018). The dataset is replete with multiple property data associated with each molecule, leading to the creation of a distinctive instruction for every property linked to its corresponding molecule. For example, given molecule and its properties Pi. P2, and Ps. we generate three distinct instructions, with each entry featuring molecule A as the input and the respective property (P1, P2, or P3) as the outpul. Correspondingly, individual instructions mirror the property they represent. For a comprehensive understanding, please refer to Figure 8 (a). Following this procedure, we have derived 401, ,,229 instruction entries after preprocessing.
Certainly, we acknowledge that the scope of properties presently collected might be somewhat narrow, not fully representing the comprehensive landscape of the molecular domain. Efforts will be made to continually optimize and enrich our instruction dataset. The initial intent of creating such type of instruction is to investigate whether LLMs can yield pertinent outpur given identical input, but with varying instruction demands.
B.2 PROTEIN-ORIENTED INSTRUCTIONS
UniProtKB, a universal protein knowledgebase, is used as our data source. Concurrently, to safeguard the quality of proteins, we opt for entries from UniProtkBSSwis--Pott a highly reliable and manually annotated protein sequence database. We then augment our dataset by incorporating selected entries with high annotation scores from UniProtKB/THEMBB thus enhancing data volume and diversity. Further, to mitigate potential bias training due to redundancy in protein sequence data, such as closely-related protein variants and homologous sequences with high sequence identity within Swiss- Prot and TrEMBL databases, we administer a stringent filtration process to purge redundant proteins from dataset. First, We cluster these protein sequences based on a 90% similarity threshold using MMseqs (Steinegger & Söding, 2017) tool (-min-seq-id 0.9), designate entries abundant in functional annotations within each cluster as representative proteins and dismissing the remainder. Second, MMseqs search (-min-seq-id 0.9) is executed using the representative proteins sourced from the TrEMNL as query database, and proteins from the Swiss-Prot as the target database. All TrEMNL sequences that align with a Swiss-Prot sequence, possessing 90% sequence identity or higher during this search, are subsequently eliminated. These measures intend to streamline the construction of high-quality, non-redundant protein function-oriented instruction data.
Protein design In this work, we seek to construct instructions using the textual form to implement de novo protein design for user intent, which could conveniently design composite properties of interest. Formally, given the design requirements of users, models required to generate protein amino acid sequences that align with those requirements. Hence, we select annotations from 20 features in the UniProt knowledgebase to generate the protein design requirements instructions. As shown in Table 6, these features comprise the properties of proteins commonly studied or expected for target proteins by biological researchers. To better integrate this structural knowledge with instructions, we design a series of templates (see in Table 7) to convert the annotations into textual expected properties. Therefore, we could assemble the above functional descriptions and properties together into the protein design instructions (we see an annotation as an individual condition). Table 7 demonstrates examples of templates for the conversion of structural data into textual format. Thus, we obtain 200,000 instruction entries, with the inpu/ being the design requirement and the output being the ideal protein sequence. In this task, the provided target sequence serves merely as a reference, which contrasts with other functional prediction tasks that have definitive ground truth.
Over time, the UniProt database has amassed an extensive collection of experimentally verified functional descriptions and annotations for proteins. This collection assists researchers in swiftly comprehending the roles and mechanisms these proteins play in various biological processes. Nev- ertheless, the swift augmentation in protein numbers within sequence databases, coupled with the diversity of their functions, poses a significant challenge for automated function prediction through computational approaches. In our work, focus three distinct functional annotation tasks and
20

---

Published as a conference paper at ICLR 2024
Table Twenty annotation features for formulating precise constraints for protein design. When comes to features that lack specific types such as signal, which simply indicates the presence of a particular feature in a given protein, we utilize to represent this in the 'Example" field.
Feature
Content
Example
Function
General function(s) of protein
Tube-forming lipid transport which modiates the transfer of lipids between membranes organcllc contact site.
Pathway
Associated metabolic pathways
Nitrogen metabolism
Co-factor
Non-protein substance required for enzyme activity
Ni
Catalytic activity
Reaction(S) catalyzed by an enzyme
(S)-rccidoglycolul glyoxylate uren
Gene Ontology (MF)
Molecular level activities performed by proteins
ureidoglycolate lyase activity
Gene Outology (BP)
The biological processes accomplished by multiple molocular activitios
signal transduction
Gene Ontology (CC)
The locations relative cellular structures in which the prosein performs function
mitochoadrion
Signal
Sequence targeting proteins the periplasmic space or secretory path- way
Coiled coil
Regions of coiled coil within the protein
Motif
sequence motif biological interest
Nuclear localization signal
Domain
Type of each modular protein domain
PWWP domain
Compositional
Compositional in procein
Polar residues
Topological domain
Non membrane regions of membrame spunning proteins
Lumenal, melamosome
Transmembrane
spunning region
Helical transmembruuee region
DNA binding
Type a DNA binding domain
Homeobox
Binding
Binding any chemical group
ATP
Active site
Amino acid(s) directly contributing to the enzymatic activity
For GAlase activity
Beta strand
Beta strand regions
Helical regions
Table 7: Templates converting structured annotations of specific protein fields into textual form. In practice, we also utilized LLM (GPT-3.5) to enrich the style of the templates.
Feature
Template
Function
general function, protein
(function
Pathway
A protein that is capable of catalyzing renctions within the (pathway) with high efficiency and specificity.
Co-factor
The protein must bind to (co-factor factor) in order to perform its enzymatic function.
Catalytic activity
The protein
be designed to catalyze the following reaction (ystallytic lytte act svtivtyy.
Gene Ontology (MF)
The protein he able (no ecular funot incctioon] on).
Gene Ontology (BP)
The designed protein must be able to regulate abiologict process).
Gene Ontology (CC)
The designed protein localizes to the (cellular component)..
Signal
Incorporate signal peptide in the protein design.
Coiled coil
The target protein must incorporate coiled coil domain.
Motif
The protein's 's functional design nocesitates the inclusion of {mot
designed protein should contain more (domains) are essential The protein should exhibit (compositionn inipasitional bies).
Compositional bias
Topological domain
The (zopologise
domain) of the protein should be designed to be flexible or rigid.
Transmembrane
The protein should contain (trenamembraste rane) membrane-spaniing rogion.
DNA binding
A protein with (DNA binding) capability targeted gene regulation.
Binding site
The protein should be able to bind (bineing site) ligand in variety of conditions
Active site
The designed protein must have a (act sitte) that is highly conserved among related enzymes.
Secondary stracture
target protein must exhibit (Ret strand, Hilix, Tuyn] its primary conformation.
one task dedicated to generating functional descriptions. Our goal is to assess the ability of gener- ative language models to address a broad spectrum functional prediction issues under a unified framework. Concretely, consider two widely used classification schemes that organize these myriad protein functions, Gene Ontology (GO) Consortium (Ashburner et al., 2000) and Enzyme Commission (EC) numbers.
21

---

Published as a conference paper at ICLR 2024
Protein function prediction For GO terms prediction, given the specific function prediction instruction and a protein sequence, models characterize the protein functions using the GO terms presented in three different domains (cellular component, biological process, and molecular function). In the acquired 11,,558 instruction entries, the inpul is a protein sequence, and the output represents the function of that protein.
Catalytic activity prediction Mcanwhile, the EC number, a numerical classification system for enzymes hinging on the chemical reactions they catalyze, is substituted with the corresponding reaction. This substitution aims to leverage the tacit knowledge ingrained in pre-trained language models, thereby encouraging the model to predict the reaction itself rather than the mere EC number. In the final 54,259 instruction entries, the inpul is a protein sequence, and the output refers to the catalytic activity of the protein and the chemical reactions it promotes.
Domain/motif prediction We introduce the domain prediction task, which tasks language models with the identification of the domain type within given protein sequence, which defined compact folded three-dimensional structure. Each of the 46,028 instruction entries includes a protein sequence the imput, and the output refers to the domains or motifs that the protein may contain.
Functional description generation On the basis of the above functional prediction tasks, we design the functional description generation task, which not only evaluates the reasoning capability of the language model in determining the function a protein sequence but also assesses the efficacy of the language model's 's text generation. After data preprocessing, we obtain 88,259 instruction entries, where the input is a protein sequence, and the ouipul describes the protein's 's function, subcellular localization, and any biological processes it may be part of.
B.3 BIOMOLECULAR TEXT INSTRUCTIONS
Chemical entity recognition Chemical Entity Recognition (CER) is a fundamental task in biomed- ical text mining and Natural Language Processing (NLP). It involves the identification and classifica- tion of chemical entities in textual data, such as scientific literature. These entities can encompass broad range of concepts including chemical compounds, drugs, elements, ions or functional groups. Given the complexity and variety of chemical nomenclature, the CER task represents significant challenge for LLMs, and their performance in this task can provide important insights into their overall capabilities in the biomedical domain. For this task, employ the BC4CHEMD (Krallinger et al., 2015) dataset, in which the chemical entities have been manually identified and labeled by experts. To ensure balanced representation of each task, and to equip the model with the ability to handle a wide array of tasks, we randomly sample 1,000 entries from the BC4CHEMD dataset.
Chemical-protein interaction extraction We task language models with the nuanced role of annotating chemical-protedd interactions. This endeavor seeks to explore the extent of biochemical and pharmacological knowledge encapsulated within these models. More specifically, the models are presented with excerpts from scientific literature and are required to not only identify distinct chemicals within the text but also to discern the specific nature of the interactions between them. This could involve, for instance, determining regulatory relationships between identified ligands and proteins. We utilize the ChemProt (Krallinger et al., 2017) dataset as the primary source and subsequently convert it into instruction data. In instances where multiple relationship triplets exist within a common scientific document, we consolidate them into a singular annotation separated and delineated by commas.
Chemical-disease interaction extraction The goal of this task is to discern the relationships between chemicals and diseases from given medical literature, a concept known as chemical-induel disease (CID) relations. These CID relations are paramount biomedical research, playing crucial roles in areas such drug discovery, toxicology, and disease etiology. We utilize the BC5CDR Corpus (Li et al., 2016), which comprises 1,500 PubMed articles. These articles collectively feature 4,409 annotated chemicals, 5,818 diseases, and 3,116 pieeical-ddseaate interactions. Considering the analogous structure of the BCSCDR dataset to the ChemProt dataset, we employ an identical series of processing steps to convert the dataset entries instructions.
22

---

Published as a conference paper at ICLR 2024
Prompt:
Given paragraph of paper, can you ask five questions and give answers that can be directed to the paragraph? List in the format of Question-Anss Answer pair.
Paragraph: The outer membrane protein G (OmpG) nanopore is monomeric 3-barrel channel consisting of seven flexible extracellular loops. Its most flexible loop, loop 6, can be used host high-affinity binding ligands for the capture protein analytes, which induces characteristic current patterns for protein identification. At acidic pH. the ability of OmpG to detect protein analytes is hampered by its tendency toward the closed state, which renders the nanopore unable to reveal current signal changes induced by bound analytes. In this work, critical residues that control the pH-dependent gating of loop 6 were identified, and OmpG nanopore that can stay predominantly open broad range of pHs was created by mutating these pH-sensitive residues. A short single-stranded DNA was chemically tethered to the pH-insensitive OmpG to demonstrate the utility of the OmpC nanopore for sensing complementary DNA and DNA binding protein at an acidic pH.
GPT-3.55 completion:
Q1: What is the OmpG nanopore?
Al: The OmpG nanopore a monomeric 3-barrel channel consisting of seven flexible extracellular loops.
Q2: What is loop six used for?
A2: Loop 6 is used to host high-affinity binding ligands for the capture of protein analytes, which induces characteristic current patterns for protein identification.
Q3: How can the tendency toward the closed state be addressed?
A3: Critical residues that control the pH-dependent gating of loop can be identified and an OmpG nanopore that can stay predominantly open at a broad range of pHs can he created by mutating these pH-sensitive residues.
Q4: How has the utility of the OmpG nanopore been demonstrated?
A4: The utility of the OmpG nanopore has been demonstrated by chemically tethering short single- stranded DNA the pH-insensitive OmpG to sense complementary DNA and DNA hinding protein an acidic pH.
Q5: What type of channel the OmpG nanopore? A5: The OmpG nanopore is a monomeric /8-barel channel.
Figure 7: An illustration of SELF-QUESTIONING method for generating factual question quusstion-nsswr pairs based on a given PubMed abstract derived from PubMed White (2020) in the biomolecular field.
Multiple-choice question We assemble a collection of 12,398 multiple-choice questions from the MedMCQA (Pal al., 2022) and MMLU (Hendrycks et al., 2021) (Massive Multitask Language Understanding) datasets. Using the instruction-tung touning approach, our objective is to inspire the professional capabilities of language models to answer diverse probes pertaining to biomolecules. The questions span the subjects in biology, chemistry, medicine and other vertical areas associated with biomolecular research.
True or False question The objective of the true false question- task is to answer research questions with affirmative, negative, or uncertain responses (c.g.. Do preoperative statins reduce atrial fibrillation after coronary artery bypass grafting?). In the initial PubMedQA dataset, each entry consists of a question, an abstract from a publication in PubMed (White, 2020) for reference, a single answer and corresponding explanation for the answer. For this task, instead of rigidly designing an instruction that prompts models to refer to the materials for answering, we eliminate the references and treat the questions as complements the subsequent open-ended questions, accommodating wider variety of question types. Further, the ideal response to a given question should provide an accurate answer with a cogent and reasonable explanation.
Open question Open-ended questions are defined as those that simply pose the question, without imposing any constraints on the format of the response. This distinguishes them from questions with predetermined answer format. We primarily gather the open-ended questions from the MedMCQA dataset, specifically selecting those pertaining to the domains of biochemistry, pharmacology and multiple-choin style, illustrated by
medicine. The original format of MedMCQA questions questions such as, "In which of the following conditions are Leukotriene inhibitors highly effective?".. However, we found that certain questions did not clearly communicate that the answers needed to be
23

---

Published as a conference paper at ICLR 2024
Table 8: Data statistics of biomolecules.
Characteristics
Mín
Max
Mean
Median
Bertz complexity
36,222
508
206
Molecular weight
8,656
273
129
Atom count
574
19
Ring count
0
75
Sequence length
2
39677
455
391
Domain
2215
27
Catalytic activity
2404
15
Gene Ontology
28924
35
selected from the provided choices, for example, question like "Antibody used in the treatment of RSV infection is:". Hence, we conveniently transform 27,574 questions into open-ended questions, where the corresponding answer encompasses the description of the correct option and the explanation, namely MedMCQA-Open.
In order to delve deeper into the queries concerning biomolecules and bolster the language models' proficiency within the domains chemistry and biology, we propose the BELF-QUESTIONNNG to expand the instruction data of open-ended question by employing GPT3 extract factual question- answer pairs from PubMed abstracts in the field of biomolecular research. The implementation of SELF-QUESTIONING consists of three steps: 1) data collection, 2) question-answer instance generation with the self-questioning approach, and 3) filtering low-quality questions.
First, we gather the complete abstracts from PubMed, the annual baseline released in December 2022. Each publication in PubMed comprises a title, abstract and Medical Subject Headings (MeSH). To focus on responding to questions about the biomolecular domain from users, we only consider those publications whose abstract and McSH contain some specific keywords (e.g., protein, molecule) that probably pertain to the study of biomolecules. Second, it challenging that expect models to generate high-quality and factual questions based on the provided reference materials. Nevertheless, we found that pretrained language models can achieve this when prompted with the instruction that requests the model extract question-anse answer pairs. This approach ensures that the derived answers are properly aligned with the corresponding sources. A representative example of this generation process is illustrated in Figure 7. Third, to further ensure the overall quality of the produced questions, we incorporate an exclusion criterion that substantially eliminates questions closely tied to reference materials, for instance, questions such as "What is the purpose of the study?". These types of questions generally involve specific vocabulary terms such as "result", "study", "paragraph" and "article". By identifying such terms, we are able to exclude them from the pool of consideration effectively. We refer to the dataset containing 10,521 examples as PubMedQA-GPP This is then combined with the MedMCQA-Oper dataset to construct the instruction data.
C MORE EXHAUSTIVE DATA ANALYSIS MOL-INSTRUCTIONS
Figure 4 provides a multi-faceted analysis of the diversity and complexity of molecules and proteins. For the sake of clear presentation, only part of the coordinate coverage is displayed. A more comprehensive display of statistical data is provided in Table 8. Overall, these statistics reflect the broad and diverse nature of the biomolecules, which should contribute to the robustness and generalizability of models trained on them. Indeed, protein sequences typically very long, which poses a significant challenge for LLMs to capture and understand their "grammar" or patterns. Strategies for encoding and processing these lengthy sequences, well as effective ways to guide LLMs predicting or designing valid protein sequences, are crucial areas for further investigation.
D EXPERIMENTAL SETUP DETAILS
With the Mol-Instructions data, employ the 7B LLaMA model as foundation model in the entire experiment and conduct instruction tuning on the LLaMA model. In this work, due to the diversity of modalities and substantial variations in task difficulty, we train the LLaMA models our three distinct datasets tailored to specific biomolecular problems, resulting in the development of three fine-tuned models. For each distinct task, we allocated almost 1 samples as the test set
24

---

Published as a conference paper at ICLR 2024
compute metrics and evaluate the model's performance. The remaining samples were divided into training and validation sets at an 8:2 ratio.
Although tokenization is crucial part of data processing, particularly when dealing with various modalities, this study does not delve into an extensive analysis of the sequential tokenization of different modalities. Therefore, string of molecules as human language and tokenize the data using the byte-pair encoding (BPE) algorithm. We utilize the identical tokenization model employed in the LLaMA model. We the longest-padding strategy to pad the tokenized sequences to the longest sequence length in the batch.
approach the protein amino acid sequence and the SELFIES
We adopt the low-rank Adapter (LoRA) fine-tuning on the molecule and text-oriented datasets, an efficient method that reduces memory during training and keeps small set of parameters trainable, while not updating pre-trained models. For protein-oriented instruction data, we perform full finetuning with the memory optimization technique, ZeRO implemented in the DeepSpeed library. Our models are trained using Adam W optimizer and linear learning rate scheduler. We conduct the LoRA training and generation on 32GB V100 GPUs while performing the full-model finetuning the 80GB A800 GPUs. The exact hyperparameters we tune for each model are shown in Table 9.
Table 9: Training hyperparameters for finetuning on different datasets. QV: two linear transformation matrices on the query and value states in the self-attention module.
Hy Hyperparameter
Molecule
Protein
Text
Finetune method
LoRA
Full
LoRA
Batch size
800
96
1024
3e-4
2c-5
3e-4
Steps
40,000
25,000
840
Warmup steps
1,000
2,500
100
LoRA r
16
16
LoRA a
16
16
LoRA dropout
0.05
0.05
LoRA layers
QV
QV
E EVALUATION METRICS
E. MOLECULE METRICS
To evaluate the molecular understanding tasks, we utilize metries like BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), and METEOR (Banerjee & Lavie, 2005) to assess the quality of the generated outputs by comparing them reference answers. For molecule generation, we first validate whether the generated strings are valid molecules using RDKit (Landrum et al., 2013) and then compute their exact match with the reference solutions. However, it's important note that single textual description might correspond to multiple molecular structures. Moreover, expecting an L.M,, even one fine-tuned with LoRA on specific instructions, to produce outputs that perfectly match reference data might be unrealistic. To accommodate these complexities and offer more comprehensive evaluation, we further employ metrics that measure molecular similarity. These include similarity scores derived from RDKi/MACCS/Morgal fingerprints (Tanimoto, 1958; Schneider al., 2015; Durant et al., 2002), as well as Levenshtein (Li & Liu, 2007) and BLEU scores. For the molecular property prediction task, we employ MAE (mean absolute error) to quantify how accurately the model predicts continuous values.
E.2 PROTEIN METRICS
For protein understanding tasks, we esteem these tasks as the functional description task and employ ROUGE to quantify the quality of the generated descriptions for biological characteristics.
E.3 BIOTEXT METRICS
For NLP text tasks, we employ general metrics for Q&A, entity recognition, and relation extraction to evaluate the generated outputs.
25

---

Published as a conference paper at ICLR 2024
F ADDITIONAL RESULTS
Due to space constraints, only a portion of the results are displayed in the main text. The remaining results, which further illustrate the effectiveness of our Mol-Instruciions can be found in Figure 8, 9. 10, and 11. Please note that Galactica (Taylor et al., 2022), Text+Chem TS (Christofidellis et al., 2023), and MolT5 (Edwards et al., 2022) only support the SMILES format. To accommodate this, convert the SELFIES in our instructions to SMILES. It's also important to mention that MolT5 has not undergone instruction tuning, hence it only processes the "input" part of our instructions.
The analysis of the various experiments illustrated in Figure 8 reveals how Mol-Instructions guides LLMs in comprehending and executing specific molecular tasks. This guidance is evident in tasks such molecule description generation and description -uideed molecule design, where the dataset provides precise instructions that direct the model to produce detailed molecule descriptions and accurate designs. The performance of MolT5 is notably inferior instruction-follooinng models, exhibiting poor generalization capabilities. In tasks related to chemical reactions, such as forward reaction prediction, retrosynthesis, and reagent prediction, Mol-Instructions play a key role in instructing the model to predict reaction products, potential reactants, or correct reagents. Alpaca- LoRA, ChatGL.M, Baize, and Vicuna in the absence of such specific instructions, produce outputs that are either too generic not aligned with the task requirements. LLMs generally underperform in molecule generation compared to -spacffc smaller models, due to their broader focus which compromises task-specific specialization. For the property prediction tasks, Mol-Instructions help the model provide near-accurate estimations various molecular properties such as HOMO, LUMO, and HOMO-LUMO gap energies. In comparison, Alpaca-LoRA'S outputs, lacking guidance from specific instructions, deviate significantly from the actual values. Galactica consistently outputs negative values.
As demonstrated in the prediction instances illustrated in Figure 9 and 10, the fine-tuned model delivers consistent and user-specific protein analysis for functional annotation. For instance, the model (aneiieee2550)))122))mmetmmrrie
accurately classifies a provided protein as the 23S rRNA within the context of catalytic activity prediction tasks. Moreover, while slight variations may arise in describing distinct functional traits relative to the annotations in the UniProtKB, it showcases remarkable potential in discerning the fundamental properties of proteins.
As showcased in Figure 11, in tasks related to chemical entities and their interactions, Mol- Instructions aids the model identifying the correct entities and establishing meaningful relations between them. In contrast, models that are not fine-tuned with our instructions, such as Alpaca-LoRA, often produce outputs that are too generic or miss the specific relationships present in the data. When dealing with multiple-choice, True or False, or open questions, the model fine-tuned with Mol-Instructions not only provides the correct answers but also delivers them in a more detailed and well-structured manner. This demonstrates the effectiveness of Inssruuttut in improving LLMs' understanding and performance in these question-answering tasks.
Overall, the results indicate that Mol-Instructions can improve the LLM's ability execute molecule- oriented, protein-oriented and biomolecular text tasks. It underscores the importance of having task-specific instructions to guide the model, particularly when dealing with specialized domains such as biochemistry. In comparison, models that lack such specific instruction- tuning, like Alpaca-LoRA, may struggle produce outputs that align with the specific requirements of the tasks.
Despite the promising results demonstrated above, there are also several limitations. First, while the experiments confirm that Mol-Instruciions can aid LLMs in understanding and mastering biomolccular-relate. information to a certain extent, it should be clear that the model obtained by instruction tuning at this stage is merely a preliminary demonstration. Its application potential for real-world, production-level tasks is still limited. Moreover, to fully realize the potential of Mol-Instructions, more profound exploration and innovation in algorithmic improvement are needed. This includes optimizing instruction tuning methods, further refining the model's 's ability to understand and follow instructions, and enhancing its generalization capacity across various molecular tasks. Therefore, while Mol-Instruction represents step forward in the use of instruction datasets for biomolecular tasks, there remains a significant scope for future research and development to maximize its utility and effectiveness.
26

---

Published as a conference paper at ICLR 2024
Moleculo
(ooeouuar description generation
Instructio: Please give me scaee detalis about this moteoule.
Inpat: [C][C]C] [eCCe]
[ [[[[C]]]]]] [e [[C [oojo-/ ([ (ingg]]] [O]]c][ [ [[JCC]]
groups postiors dracaldnzoyl inspectively. is functionally rolotd unccidonii cod and as wotdesanols ood.
swuft): molectle
phosphatidyt
which
specified sharoyy
Ouiput (ounje The molecale a S-a-pheppaaii Lourine is which the phosplutilyl acy) grouge at posiims I and 2 an palnioayl and steanoyl rapustivdly It has
nole as mouse metabolive decives from an octicecmnte acié and becadecantte actd.
Output (Alpansii [CCCCCCCCCCC CCCCCCCCCCCCCCC CCCCCCCCCCCCIICICCCC [CI--Bnec///C [[C]]-O]o]CC Rnan (tlitingglllliel [I-Irechh HCCOO CACCCCC------tttic [[][]]]] [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC еееееееее [I] P[[-Brnnxb]]
Ourput (CharGZZM); I'm sonry but contoo provide deciled information about modeccle that appears contiin mix of carboo, by deoigges ouygen and nhtroeen
stums as wall virio other chamiaal demins and fusciionll gruqs.
put (Buizel This is molocular scrutuge of the compondd knowz Cytuinec, which
of the baac (A. and T) hust make the audkotide of
ENA and RNA The molocale comiess of fivemmerheed ning stracture with a rinngen stom the center Its a pin of the pyrimdame family of suderhuaes.
have provided is cartohyhaicc mekensle with a teanchal stractore It conosst of chais carhon stoms with hydroayl group (OID attcbeed to sonee of the carhon stoms, The motecale also las a tngg strotre stacaed to ons the carbe sooss the chain.
Output (Vicuna): The melecals
JCOC Output (Ginlectice:: COCOOC CocCocoo---ooCC
1O df H -O WOOOOONNNNEHHCCCCCCC (HI D0 CONNCC---HICCCCCC (0-0CCO0)))OO (0
Output (FeateChem 75I: The melicale eyclic letracydiic analogue of micsiisic acid. has mole antinooplaatic apent and hacterial metarollte, it
eyche untrayyhic analogee and lerrcyelie arolegae, It derivee from a maniiiic acid.
Output (Mol33:: The It an on In
Forward reaction prediction
Indraction: With the reovided mactants and reagins
pocentaal product.
Inpat: [O][-N-I]
[[ [C-[[]]]]] C[NI Cl]Bnund [[C]cee [C]]cc]j[[[Bannn]jcc C[CIT-CCRing]]BBaatta
Output (groud maufi: (NTCI-C -C]] CijBaadd (C JCUU ( [Bunced)
Outgut (ounjj [N][C]==C]] [[]]]] [CC [mmm)]]][])))]]]]0]==>> (n2
Outpat (Alpece): The pocential product is polycyele tomaik hydroocshoo (PAH).
Ourpet (CherGZ.M) Based on the given mackhats and reegeats, poiential product could to [Be[---I][Bnnhi] 1[ The reccton between -X-L,
Itnoch1, C, O-1, C, -C, N,*C ei Brnnchl, C, C,
Output (Baioris Reael on the provided noadants and rzegoris Be poseniaa product is F:(CN The provided roadumis are cytmein molecole (C), an esygan
mokcrale (O), and nitegga malecule (N). The reagen (Fol is metul sten, which undergo roaction sysoine molecule a complex comprndd.
(happt (Ficunal: Based on the pooided uput, a poteatial prodect could be [0] Ne1[[Bood1]jC00111 CICCIIIIIIIIIIIIIIIII C C[ [[CCC C)
[Ring j [.rnnch),
Output (Golection:: Q[CHttCCSHCC oop.eeeppcc (00
Ourput (TeatCCoow 23): O- DCI-CC--XCIKTI-CI
Reagent prediction
Instrection Please provide possible
based the following chemical reaciion
Inpar: [[[]]-Cjj[--Qieenee [C]NIC [ingg] [Hilmncal [0]] hicpchchhoohoooooooooco h/tp://wwwpp////////eeaaaaoo/oo0 (00e:
[C[- [[[-o[[[1[[[]]NN[aaa
Outpat (groad puff): [[]]]]]CC]C]C [O][C]] [
Oatpul (ownd: [CCCCCO[CC
(Runc))] [CC]CCI Bunch |[ [C[[CC
Ourput (Chartzzay) reagons nigin
(Alpaay:
allleed
[C][ [[|8Irc N1Rig]]
were possible reagents
HKI HIN Ninogoo gas -NUUH Cimaanadeetyle (CZH5GH) Nitagen ga (N2) 3. Unramaidehyde
gives
(C2H550044 carhin molazulc (C), An melicale (O) chlorine mokanle (C3) A carhen moloaas (f), shimine malezule
H20
chemical
cynsins meleale (C), esygen molezole (0), nitmaenn molicule (N).A
(Moswat: Based on
chenicall
the pooable eaagent that
[CICI-CIC || [C
[O]] C]Beanchi Becchi[[CCC [[]CtC]] MMjRig]][O)
Output (Golactica):
Output (TexteChow 2522 CUI--DC
Desoription guided moleoute design
Insteuctio:: Cratic a molusalc with the simaturz as the une duserhall
Ingut: The molecale is peimary arytamne in whith anino finetional group subetittted for one of the benzene hydrogens. It a primary arylamino and menher aniliacs
Outpet ggroand masky: [NCCICC [[CRC [-dnnch]]
Output (oarst: [NIC]-CIC -CCCIICCCCCCC [RCiR IJ- [iaBhich
Outpet (4ipacel: molecati is prinary srylamine in which as smino fanctional grou schititsted for ons of the hemrone hydragens, is primery any limine and mamber aanllines.
Output (C Ma) The molecule you descrieed is called unáo ary arylanise, alo known as unlline. The sinucture of this molecule is as follows, CSHENHH2.
Outpot (Baizz) The molezale would have the stmonure CWESNHz (NH1 is the anino finctional groupi.
refering to an ine (ChiSSN), Aniine is printrry arylamine which smino functional group suhstitsed for one of the bensete hydrogen. is princry arylaiiee and member of antlinee.
Output (Ficanay: The molecule
Output (Golaztiojjs COCCXCC1=00--((-0000 (ttp://:---c--0000000-----------
Ouigut (TustChee T5J: CUOOC=CCCCUNN
Output (Mal T5J: -CC-O
Rctrosynthesis
Instractio:: Fesas suppost posential roactants in the synthais of the geovidad product.
[aguc: [C]-CCTCMCI BIC.CCCCCTTIIIIIICaaaaaCCCCC [c BC [C [CBoonaa]]C]]C
Owipot mudl) [U] QBR-CC/BUGNNCCCCCCCC Boochh (CCC http:///wwwwooaooo////////o///a0 BBoohh pugo://oeennnttoooooooogooaoeee
[Umnch [iUjue
Ouigut (owrs): CCIlCCCCCCCCCNCCCCCCCCaaaa. [C[]C]
Outgut (d/psoaj: The potential resctants used the synthsss the provided product are acetylone, carbon mennaids, and cyundde.
Ourgot (ChiGGLy: The profuct yeu hove providdd appors to the smideanide bond, which firmed by the rewtion of in amide group (=C-NII) with base such sodium amide litium smide are smm penetial roachins coild be ued the synthois of the amideamde hond Nemohhyammde N-
muthyymorppolite
Outgut (Baize): aituigg
Bee peovided chemical strcture, pctental reactants that can used the systhesi the product are: A.A cytosine molocule (C) chicrine
duyges molecal:
(N)
for oxggen molecuie
powiled punmiia with the oarbnyy grop (C=O) preseet the the produzi incluke Seaiumm (Nx) ningggm (N) to next with the carborel group (C=O) preseal simuctum Output (Golartiojj C/C(ET) IC (tt.::eeeiiiiiiiiicioooco
Ouigun
of sodium iore (Na1)
Ammonia (NI13)- This could be seed
Output (Tests
Property prediction
Instraction: Fense proside the HOMO energy value this molecule /Pease provide the lowest unoccupted molecular ochinl (LUMO) energy molecute. I' gap energy
you place provdd
[eppt: [[C]]]][[[[[[tt [[c]jcc[C[[BBuaa]]]cjjcCp
Output (groand ands)) 0 0,242/ 0.054400.30 0.3326
Ourgut (wwrs): 40 0.2511 / 0835/ 0.5333
Ovigut
(Duigut (ChurGLMMr My: Without
informtiion about the strccuro of the molecule it possile provide eaact HIOMOL HOOOOOOOOOOOOOOOOOOOUOOOOOOO energy
Output (Raiza): 4..44/0.44//
Output (Ticnna:: 4,2/-3.24.55
(Duput (Galantica): 0.5985.0.0231100022
Figure& 8: Performance comparison of LLM tuning using instructions across molecule-oriented tasks.
27

---

Published as a conference paper at ICLR 2024
(b1) Protein
Protein function prediction
Instruction: Analyze the following amino acid sequence, and determine the fanction of the reslitin protein, its subcellniar localization, and any biological processes i be of:
Inpuc:
WODLLL WAN.LLAAENNNNNNNNEEEEEENENNN LORAAES GINDFFL PENEEEEPPEEEERRAAA WAAFELLRIEEEEEEER WEEEEEEEEEEPYYY DL PEEEDEPPEEEEREIINNNNNNN SAN.EEERRRIIILNNNNNDDDDODEEEE WIR.EKK TTW.TESTSSTTEEEEEEEIEEEEEEEEEEEE WWW.ANNNKLKLLRCCCAAUCCCC
KAIF
SSFFCXAL
Oatput (gronne fruakj: The prosein characterized by the amino acid sequence demosstrates ATP binding, cycácyiate kinase activity, uridylate kinase activity and is implicatol the 'd nove' pyriiiidigg nucleobuse binsymthetic procees, phosphory laaion, pyrimidine muclouide biosymehetic process. Its suhcellular localization is cytoplaems macless.
pelmarily within
Output (owrzi: Based on the given amino acid sequence, the protein appezes to have primary function of ATP binding, cytidylate kinase activity, UMP kinase activiry. It likely invalved in de nove' pyrimmdie sucheuhsseline cyclization, pyrimidine ribonacleoside biosymchetic process, undine metabolic process, UMP catabols: process, and its subcellular localization is within the cytopitsm.
MNGT CORGTVNAASPPSSS SOASSAASSSSSEAVERRE AAAAASSAAAEEEIIAAA GAGGGGGGGGGGGGGGKGGIIIIGII WELIIIIRNREEERRRRRRR CATEENRAAATLLEREA WEEPPPETYEEECPEEEE WEWEERRRRINNNRRIIII BELNNNNNNNRDRNNN METKEEERRRRRRRR TRRKRRRRRRIRRR VIIESTLPVIE PWIETYNLKKKLYYYOOO S
Output (Afpana): The amino sold sequence
determine the the function funutioonality and distmbuted in varicus organisms,
Qhiput (CherGi
need analyze amino acid with other molecules. Function: The protein you provided called 1lsp7 which
predict its molecular features, such responsive protein that widely
Output (Relzs): This amino scid sequence does not conespond to known protein. is an artifiiaall soquence that has not been charactrized in any way. Therefore, the function, suboellular localization, biological processes of resulting protein cannot desermaned.
Outpet (F (Ficune): Basod on the amino scii sequenee provided, the resuliigg proscin likely to be a membonnc- associsted prokin with a signel peptide the N- ferminus and a transnembrree douniin in the middle. The protzin is prodictod to be localized in the cadoplasmic ceticulum (ER) and may be involved in lipid metabolism or transport. The C-terminal region of the protein may be involved procein procein interactions or signaling pathways related to lipid metabolism or transport. Further experimental validarion is needed to confimm the contoon and localization of this protein.
Outpat (Galactio)): to Summary **Protcin** e*: Prohable serinhhtreonine-ee
kinase 1: "SGe **Orpaniam** Homo saplens (Human); **Satss*
evidence proteit level: eh Fanction nee Caralytic Activity ATP + L-syyy- ppotein) ADP - H(+) - O-phospho sesyt-(pooteia) ATP L. L-thrennyt- chrrenngt-ppcoeein)
ADP i II() 1 O-phusphe-1 L- L-threonyl--iii (pnotein)
Protein design
Instruction: Design a protein seguence exhibits the desired activity and specifliity.
Imput:
1. The presence of M2(2+) is necessary for the proteis to flantion in the desired envirommnt.
The AMP, (6S)-NADPIX binding site should be located region of the prosein that is necessible to the ligand.
The designed protein should have ATP binding, NADPIEX epimerase activity, metal ion binding, ADP dependent NAD(P)LL debydratase activity to Eeciitati nisslimmide suckooislc metabolic procees
For goneri function, the protein need moet that Catalyzu the epimerization of the S- and R-forms of NADD(JD,,,, damaged form of NAD(PJI that is result of enzymaric hear dependent hydration
Output (BLAST):
(Thrget Protein Accession: M0ASIBLOL6)
MSNELNI REBBOVRRRDDOAAIEAAA WEW.GGIVLLLENGGGGNEG MENNAGGAAAAAIIIAAAAAA WWW.STTAAAAAAAARRAAAACA WAW.AAAGGGGGGACCACCCCCCCCCCCC VIA AGW EL AAAADRAAAAAAAAAAAAAAAACCCCCCAAAA VARKIGP
KRM DAAAAALLEADDAAAAAAAVYAAA
WAN.NNAAALLTAAAAPPPPPPAAAAALELET LAHRRRY SA QWWAPSY WAW.AAAAASSSAAAASAASASAAAAAAAAAA TRALDETTTAAAATAATS
ENPAARDELOREE VEVGVDIAAARAAAAI WINGAAPAAAIDALTOOSS
Outpat (ourr): Here's a pentein sequence with
functimns:
MSYKIAV
WWW.AAAAACCCCCCCAAAAAAAMMM IIAKFAHL www.IISSSTTLTTTTTTA .M MLLAPMIEEMRCHHKE WWW.SKKKEEFGGGGRPP..OOOOO WWW.GGGDLEEEERRIIIIIIII WWW.SEEEEEIIEEERIIIIIIES ISGGNTT HDMAEGIKKLLLLTTT KWLHCKKKKPPLLTT.... KOSAPIEL PLTHN
BAGHEEIILLLTEE STTEESADDIPMTAAATTAOOOOOO
RSHYPSCCONNEX,C RERFOSRI IN KEIKKOFRD WEW.DDEEEEEESEEEWWEESERE TEWETNK ET GYVDDK VIICCINNNA AGRAANLAAAAALLAAAA AARY
PAGIVAL ASKI OCYYGHHT PDARYPO PPKPRDNI HPVV
LFFINR
Sequence Alignment:
(tanget) WSBELVESREQ WWW.E///SBBBBBB//RRAAAAEEEEEEEEE
WWVELLLAC TADRIICDAAGEEEE PLOND
(ours)
WAW.EEEELLLLEELEE TNE
(Uurget) TRRNOLL BAW.IILPAAPOOADROBAAAAOAEE
(ours) 18SC8L21QLLEEG htt.::/:w...........rrrr.rT/T.a.
200
Outpet (Alpaco):: designed protein sequence should contais following umino
Mgr2-> site: (1y--00y-00y--ll--00yy-----
Gly--ly--ily-iil
Outpet (CharGLAO):
DUSAGAAAGYSAGAAM AAAAAAAAAAAAGGAAAAAAAAA AGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TA
Output (Baize): The protein sequence should contain regiun that hinds Ma(2+) with high affinity, such as metal binding domain. This domain could be taken fromm a protein that is known bind Mgt )) similar enviromment. 2. The proteim should contamn a region that binds the figand, (65) NADPIX. This region could be designed using compuicr-ssisted molocalar modeling uehniiuuss taking into account the known siraclure of related lignnds. (Hicuno): Hased following forures
inpur. have designed a protein sequence
the desired activity and specificity. The pootein is composed of 300
amino acids and
1. The presenee of Mg(2-) is noursory for the protein to fumctiun in the desirod on vnromment
2. The AMP (65) NADPEX binding site is located in region of the protein that is accessible to the ligand.
bind to AMP, ADD, ATP, (6D)-NAD,EEX, NADIPIHX, (IMAGE Figure Caption: Figurc 14.3.2: peoposed catalyzed by designed protiin.
Ouiput (Galaction): The protein should
Figure 9: Performance comparison of LLM tuning using instructions across protein-oriented tasks. (1/2)
G LIMITATIONS AND ETHICAL CONCERNS
The advancements in LLMs and their applications in the realm of biosciences, as showcased in our study, undeniably open avenues for multitude of beneficial applications. However, with great power comes great responsibility. We recognize the necessity to discuss potential limitations and ethical concerns associated with such advancements.
G.1 POTENTIAL MISUSE IN BIOENGINEERING
The capability of LLMS to "forecast chemical reactions, architect new molecules as per instructions, and apprehend molecular structural attributes and reactive features"' inevitably raises concerns. In the wrong hands, these capabilities might enable malicious actors to exploit the system for nefarious purposes, such the synthesis of harmful biochemical agents bioweapons.
28

---

Published as a conference paper at ICLR 2024
(b2) Protein
Domain/motif prediction
Instruction: Given this protein sequenos, can you identify any common provein motifs domains that may contain?
Inpet:
MANTKYIFTICGVVSSI WAA.AKKAGAAAAALLAAAAAAACC
DE AUIYERRYAATKASSKKNNTTAAA ERKENYLOGG WIVIHIINIOIIIIIHIIIIIIIIM RIK KGGGANNDDAAVVVVGGGGGGG AWV.VEGE/YYVDIISOPPPPYA. VEAL EAA.OO/VEEELELLLLLAAA WWA.LLLPSSAAAELLIIIIIII WAA.PINNNNNLANNINNIAI TOQTSHH RSLAGSSPD
VENCRSSQQEL WWWEEEEIIKEKEEEESESESEEEEEE WAA.AALCSSSSISSSSAAAAAA WISSIIVVDIIVSSS WWWDDDIISSIPLLIEEO WWIILLLKOODDIILLI WEDTILDKLLLLKO KIIKKNNNNDWWERK LEKEELL VOK VOKYTEL AEEELDSEESSSSS YKSE
NEALEHAGIOON AK WWANNNEAAAEOSONNNNNNO
EKSDAIL WPGGGGEERGIGEEEII ACKYARENNVPYL.GR GMMOAAEEYRRNVIL KI KSANSTEFISSTTEP PSIKWELLAC
Oulput (ground
prodictivee andlysis of given protein sequenee rovaals possible domains or motis. includ; Glutumine amiddttnes/czase, CTP
Output Based computational undlysis, grovided sequenee posentially contains following protein domains motifi: Giuuumine unifrae
lyp-] domains.
The protein soquence contains maltiple proteis motifs and domains, It contains an N-terminal signal peptide, a proline-rich filine--rcch region, coiled-coil domain, leucine-rich repeat domain, a proline-rich region.
Output (Alpaca):
Output (CharGZM): l #4 42, JL/4J thes la WJK. S.A. X$32 $4.0
Output (Babc) Here are some the possible motifs and domains: Glyeme- rich region, Loucine- rich region. Kinese doeainn Fibrinagen- like domain, Leucine-rich
repears, CTTPase
Output (Feone)): Rased on the provided prosein sequenee, there are sevenal common protein moti and domains thit can be identified. These include: A signal
peptide, which is indicated by the presence of the sequence
MANTKKIPITGGO WWW.GGGGOOCAAAAASSSSSOO NERKGNYLLCCO WLG.CVVVVCHHIHIIT EIKKIK"
#4 Finction Whot is the function of the protein 10 KDi chaperonin" binds to 0 ** *OOganismm** * Straptacoccus pyogrnes; *Saan*** inferred from hamology:
# (Galactioa): eti Summary **Protei*** 10 KDa chapermnin: Gene**
in the presenoe of Mg-ATP and suppreaees the ATPesc activily the latiee.
Functional description generation
Instruction: Examine the given protein sequence and share brief overview of its atributes: Input:
SKKIVLASNNOOKLARE AMML AMLAD APL PLGOLL L LROAELIIEE PEAAAAEE AEPERTTVEEEVEE VENEEEEEEEAAAAAA AKARHASAL ADAAA AAA.ADCAAAAAAAAAAAACCAAAAAAAAAAA AV POVDTAF WAAOYAAAOOYAOOOOO VALL EPMGSNGGGHDD WAA.GGNGGGGPPGPPPPAAGGGG WWW.PEGGTTPPPPAAOOOOOOOOO FAQLLVQ PVEVENAN ENANSHRGKATEE ATROMMAA AMALMRERWI
MALLLAQAAGETDRRAAA AAREERAAALLSSTLLAA VAV RSAED WWA.PELLLAAAAAAAA
Output (ground purnk:: A concise descriprion of the protein with the specifled amino acid sequenee includes:
that cotalynes the by hydrolysis of moleosiide eiphosphates to their monochosphate derivatives, with high preference preferene for the non cononical purine maceonides TTP cunnhhorine trphosphatee, diTP tiphoss
(decsyinucine triphosphatt) and ITP Serms to function as comse-cknnnng enzyme that rernoves num- camonical purine nuclectides from the nuclettide pool, thus preventing their incorpocation into DNA/RNA and aveiding chromosomal Iasions.
Output (oursj: Hore a simmary of the protein with the given smino scid sequener: Pyrohhophatase that catalyace the hydrolysis of mxclooside triphosphates to their monuphesphate denvalives, with high prefemence for the non canonácal purin nucleodides Psanthouisi triphosphate) dITP (deasyinosine trighosphate) and ITP. Seems to function as house cleaning enayme that removes non canonical Purine nuclettiee from the miclotide pool, this preveninng their mocrporation into DNARRNA and avolding chromosomal lesiess.
The AMLAPP Outpot MENNNNGKKLLLKKANE
MKKVLLASNNN WAAAAAATRRRRALSSSIIIII
AWLLLL ROAEL CERPPEAAPPRREEEEAAE WAAALLKKRRAAALAAA PALLAALDDAAAGGGAC WAW.GGGGCVEGGGGGGCCCCCG WAW.DAAAAOOORRAAAAA....M
WAA.PEELLAAAAGEIIO GRVAGEVALEEN WAA.AAEAEEEENNGGTAAACCC LPMCSNGPGFDPN WWSNGGGFDPYMIPEEOOO MEPOPPMMPPTOTTTTTT TLETTAAQLVVVVVVEEEE WEW.RNNSURRKKKK.OO KATRRCAAAAMMALLLLLL
Output (CharGZ.M): The given procein sequence is not provided. However, baodd on The amino acid soquance of prosein that was provided carlier, appears to a proteis of unknown function.
Output (Baize): The given protein sequence an artificial sequemce that has not been characterized in any way. Therefore, its attributes cannot be evaluaned or understood in any way. This sequesee does not correspond to a known peotein, and its fluction, subecllular lncolization, and biological processes canaet be determined.
Output (Picwne): The given protein
appenes be random assortment of smino acids wilhout any discemible pattern structure. more detriled att atrihutes or fanctions of this protein.
analysis would be nocesarry to determine any
Output (Golection):
packaging and Mulhifunctioall enryme that displays cysteine protease, NTPase, RNA triphosphattse RNA helicase activities May
in
virion. Specifically cleaves capsid peotein HBV genome
protein RIA binding and
that involved activities and
Together, activities participate encupoldatiin replication. Also possesses guanyly Ierannteaaeeeeeee of the three reverse-trusseiieee trasserlhedd coples genome of HBV vision. Also plays a role in the
the
biocyntheas of viñun envelupe glycoproteins the budding of virions through
with the cytolassiic tail the glycoprotein the ER
mumbrun:
Catalytic activity prediction
Instrection: Please evaluati the following protein sequence and provide explanation the encyme's catalytic anivityy inchuling the chomical reaction facilitates:
Imput:
SAELRASLAEKAA AEKAILRSTRRS RSW.RSSVVRRRARAAA ADWLKADDGTVELL QIQTEDDL AVETYLLTDKA WWNTTTNGGGLELELLLLLIROORRR PDVALLAVSLTTADET AEREEEEEEEEEEEEEE MPASLTN SDDD WAD.LLAAAYYEEETTTTTOOOO TORKEEEGGRRVVVIE REAEEEVELLLLLLEE RLBKKOLNVTLRRA WWW.RMKSSSSKSCCCCCCC SGACOQLOKTNPY. GKTNPYA
MDKWWAAFFLPERR BOONNGLPEEELNSLLCO DASFRGNNIOOIIIXGYDSPIII
CYSCOAGCAMGCARCE WAN.CCCTTTTTGGGNNNNNNCC A
WAW.LLIIIVVILIIIWHHA.A..M SF IUPPHETTAS
Output (grownd trunl): Based on the provided prosein sequence, the enzyme rppears to fxcilitate the chemical reaction: admaciinx(3)) 37) in IINA 2 reduced
(oo-25)-ffrrroooiii - S- aderosyl-) -methion inc- 2-medyladenoosne(5) (7) in IRNA + 3- densaanneiiee I-mationine 2 oridied (2F0225 (femedooiin)
S- adenosyl-- Sennoayl---hmmoystter homooyytteeeee,
Ouigut (owir)) An amllyss of the protein sequence reveals that the enzymn's estaylic funstion comcapands to the chumical resctiio: adenosnn:37)) in IRNA+2 2 2 5-sdenosyl-1 L-met-imiieee = 2-melhyillenosine)) 37) in IRNA + 5 deosyadensiiee +I L-methonine 2 osidized [2Fe-15)-
rduuced (2Fe-25)- (ie--ffemmsssiin
[feredaxin] + S-addmosy-- Hoddmmay----mmmyytttit -ommacystiine
the amino acid sequence the enzyme catalase. Catalase and oxygen. The chemical reaction catalyzed cataluse the decomposition of hydrugen peroxiile 2H202 -- 2H20 02
(Alpaca):: This pontein
that cotalynes the breakdows of hydrogen
peruxide into
oxygen:
catalytic "MLVVVVAGLL GLLEGLLLLLLAAA CAELCAS.. SLSPS ROINOOOOIIKK DDDDDDAMMT The catalyic iic
Outpot (CharGZ The protein sequence
hypothetical eusyme, named
comisins lange loop loops 1-6 that involved the binding the loop is surroussedd by small smino reaidues that ar electrostatic interctions.
substralc fonmation
causlytic
specific interactoos with subeinsic
hydrogen handing
Output (Riizz): This protein sequence cocrosponds to the active site of class onzymes knows as cysteine protease, spocifically, a member of the subfumily of cysicine preteases known as serine proteases. The amino sequenee contains conserved catalytic trial of residnes (Cys, His Asp) activity enzyme chemical reaction facilitated by the mizyme
cosential the acids by serine protease The
cleavage of peycide bond between sequence of substrate.
cleoves determinee by the amino Output (Ficuna): The pretein sequence provided is for an enzyme called papsin, which a protoolytic enryme that catalynne the hydrolysis of popride bords in protein malecules. The chamical resetion tht pepsin facilitueei the breakdown of peoteins into smaller peplides and individual amino acids. This reaction is an importan stop in digestion, as proteins are broken down into smallet molecules that can be more easily absurbed by the body.
specificity the enzyme for the peptide
reductase? Catalyzes the SADDPP--eppedent reduction N-sesty-- Soyyl--ggutaaattt glatamate 5 s-smmiiddehddd. ase Catilyas Activity -N L-axety--L-gluumaa 5 s-smmialddidde + NADP(-))+ (hhsppate Hi-) N Lhuamny
Output (Galection): Ae Fumetion Q: What is the fimetion of the protein N-scaty--aamma- gamms-ghuamm glrammy of --chyl----ghammny gigaanny phosphate gluamyl 5 phosphate NADPH
Figure 10: Performance comparison of LLM tuning using instructions across protein-oriented tasks, (2/2)
IN
G.2 ETHICAL CONSIDERATIONS IN GENERATIVE AI
Recent research (Service, 2023; Sandbrink, 2023) have shed light on the potential misuse of generative AI in bioengineering. Our work, while focused beneficial outcomes, does not turn blind eye to these concerns. We believe that any generative AI tool, when made available to the public, should come with stringent usage guidelines and monitoring mechanisms detect and prevent misuse.
29

---

Published as a conference paper at ICLR 2024
(c) Biomclecular Text
Chamical entity recogrittion
Instraction: Find and list the instanoes of the chemical entiiies the following consent.
Imput: Both the control and coramiphen grouge with double carmulas had significsnily shorter lonnciiss to seizure onset than the cormepanding grouge with single canmda.
Output (groaed trunly cammiphen
Output (ourg): caramiiphnn
Outpur (dipeca), Doth the control and caramipheo groups with double camnias significanliy shorter latencies with caunda.
than the conesponding groups
Output (ChurGLMM: Caramighen drug. cammnia is type drug comminer.
Output [Roize]: cortell group. carmmiphen group, druble canrila, Inency to seizure onset, selzure, oniet.
Output (Vicuna): Both the control and orsmighes gooup with double cantulas had signifieundly sburter lntencics sclaune oneet than the concspuraling grougs
with smagle cammin..
Output (Galactica): 3.1.2.3.4.56
8.9. 6.5.3.45666.7.88
11. 12. 14. 17. 20. 22. 25, 27. 30. 33. 34, 36. 39,
42, 64, 45, d6, 47,
50, 52. $4...
Output LMCLLLa The use
cannola signtificarll facilitoted the monitoing experimenta rass during the seizure tests. The ease with prefermd canrulation.
esperimentall
with
conmulated
handled during trating
Chemial-disease Interaction extraction
Iestraction: You provided with sct dinical trial aunmariox Fxtract the chemizal-haeee rebrions from the commorios and presene your findings in the
fienat (Sujjat, Olyezt)
the associated bebavioural problems drgg abuse uncommon pregnancy, When they do occur they bocause demial but when significanl may pose a risk bot the mother and her fotus. This caeo illustntes a number of problems that may be encomntered in
Iuput: Bating
unrecognized
women with catlng discelers in prggeacy, inducing pudurgeal and recureent metelslic distehunats and diuncti shase. In partiaular it illusintos the denngements of thyooi function sees in prepaani wemea with cring disonlers and remizds that when cause for flyrutocicois remaiss obscure, Bhyosine abuse should be considered and explored.
Outgut (groand trathi: (hyymsine, coyrototicovis)
Output (oursj: (Iyyooisee Dymulsiconss)
(Alpaca) Bating disardes, Behuvioural problems,
(Subvect, Ohject)
Output (ChaGT (Eating disonders, Behavional priliems, dru abuse, Eainng discnders, Beharvioaal probiens, dry abuse, Pregnancy, Eating divenlers Behavioral protiems. drgg abuse, Preanancy, Mecabolls disurbunces,
Prognaacy, Eating dionders, Behustonal pooblens drug ahuse, Thyrokht Behavieral prohlems, drug (hyrmsiowiss
Progsincy, Fating
Outgut (Botze): Subject Pregnacy. Eating Disceders. Object: Metabolic Disurtances, Drug Abuse, Thymid Ponction, Prolonged Thyrotocizods. Output (Vicune)) Eating dionders- Preguancy
Outgut (Galaction): (Motiess, Thyresine)
Output (PMC-LLemei: yes
Multi-choice question
Instraction: Which of the following conditions does not show multifactocial inbertiane??
Inpur: What the following the right choico? (A) Pyloric stenosis (8) Schmopprreai (C) Spina befid (neural tube dedectss (D) Marthn syndrome
Output (graad tarky: The fisal anower is (D).
Output (onrs]: The final unswer is (D).
Output (Alpecal: Q) Spiza bifida (sound rube defecte)
discomfon, (CharGLMM): Pytoric
and vomiiing. The condition inhented allelic fiehion, meaning that it caused the the stomach outle, which lead two diffiren variatiors.
show mutifaccorial inkeritaeee coodition caused by
Output (Rolze):
insver Macian syadrome.
Output (Vicune): The conditins Bht does oct show multifacterial iste liance is (C) Spine bilida (nearal tale defects). Spiua hiäda is coadition that occurs due the faiture of the petral tube to close peoperty during embryonic development, which is single paze defect Pyloric stenoss, schizophrtnia, and Marfon syndrome are examples muliifaturial inberitance, which meuns they ure caused by u combitation of genetic and envinonmeraal faclees.
Owigut (Gelactio)): (C) Spins bifds (neural tub defaces) (D) Marfan syndrome
Output (PMC-LLempt: OPTION IS CORRUCT.
False question
Instraction: Does feeding sube insetion ins timing improve aurvival?
Iaput: Nome
Ouiput (groand
insertion PEG irsertion is Outpui (Alpaca): Yes, fooding tube imsrtion has been shown
timing not improve survival,
survival
survival in crilisally ill patients.
Outgut (CharG) M): Yos, foraling rube inserton and tining imprrse survisal
simutions.
Output (Ratzej: While teeding tute incertion and its timing may be henetictal in some cases, the relazonship between feeding tabe insertion and survival is complex and requins further resarch and unkkmtanding.
Ouigut (Virnne): Feed.
Output (Golactica): No
Output (PMC-LLame):
Chamical- interection extraction
Your zak is to eximot the subject and otject entiiess from the provided text and establis the comespoddin relation triples. Please format your answirs as (Subject, Reiation Object),
Instruction:
Inpui: Opicid recopiors are the membeane procits that mediate the pain releving effect sploid drigs, such as morphite and fentanyl as well endogenous apooi peptides sakephalins and endorphins. Using AUNAs for the ma and the kappa opioad receptors we mappec the chromosomal locations of their genes in mouse, Multiocus cross analysis located the mu receptor gene Opmn on Chr 10 and the kappa receptor gaze Oprki on Chr Both groes are zeer centromere with no mombers more cerzeeenicc These data indicate that the two opicid receplors are differmn gene products, ruling out the possibiity that they may be differential eplicing prdducts from the same gene.
Output (ground mrldit (moophine, regulanor, Opioid receptoms) (fentanel, regulator, Opioid receptoes)
Output (ours): (lentanyl, regulator, Opieid recepions), (mimphine,, regulator, Optoid neceptom)
Output (4lpaca): (Subjeed, Relation, ObjestKnu recopioe near, Chr 10ykuppp recaptor, neas, Chr 1)
Output (ChaetitMM): My: (oploid receptors, mesil ure protzits, aeediate pein dei--reiiiingg effect maa and ka opioid receptors, genes, magped divomosone locations; kagpu neerptor gene Oprk1 (aphoid becopion. gase products; Marpline: Enilagemens optuid pepliskss unloplulins enalephinns
ma nooeplur gase Opem
1: Muli kinw crose smilyes, location centromene, mone: su markers more contrumanc))
Output (Bolzo) (Subject, Relution, Object) Subject Opioid Recepioe. Relation: Mediote Object: Pun Relleving Bftlet of Opoold Deogs and Enddopetous Opiold Peprides.
Output (Ficssa': (ppioid recepeors, the)
Output (Galaction); Opmn
Outpet (PMC-HLam'' The mu opiend receptor peme Open losstes to the cistal repion Chemesome the mouse.
Open question
Instruction: What vinu cuuect chmnic discis??
laput: Nose
matk): Hqpatiti
(HCV).
Ouiput (ourj: Heppritts llopatiis is chroaic liver discase caused by the bepatitis C virss (LIC).
Owtpet (elparo): Liepatiii is vinss that causes chrotic liver chsesse.
Output (ChanGL.M): There are many vinuses that can cause chnonic diseases Some exumples include: Chmnii obstnuctive palmenary disease (COPD), Chronic kidacy disesee Chronicc hear disease, Chook diabetes, Choeoi: faligue syndrome.
Output (Rain)) There an vovend virsses that can cass chronc diseases, inchuding Hopotris Is and HIV, Herpes Simplex Vinus (HSV), Human F-xill Leukemia Vina Typs 1 (ETLVV1), l). Yulecila Zusee (VZV).
Oulput (Vicuna): The vine. that
dhronic diuzses is the Epstcin-Burr vina (EBV).
Outpot (Calactions: Hepatitis
Outpul (2MC-LLamis OPTION D IS CORRECT.
Figure 11: Performance comparison of LLM tuning using instructions across biomolecular text tasks,
30

---

Published as a conference paper at ICLR 2024
G.3 MITIGATING RISKS
Addressing the potential dangers is paramount. Some steps that can be taken include:
Limiting access to only verified research entities or individuals with a track record of ethical research.
Regulated access:
Monitoring usage patterns: Implementing algorithms to detect patterns of misuse exploration into potentially harmful domains.
Community oversight: Establishing community-dee oversight mechanism where experts can review and approve specific uses generated outcomes.
Transparent reporting: Encouraging users to report any unintended outcomes or potentially harmful use-cases they encounter.
The ethical implications of AI, especially in sensitive domains like biosciences, cannot be stressed enough. While our work represents step forward in harnessing the capabilities of LLMs for good, it vital to move forward with caution, diligence, and an unwavering commitment to ethical principles.
31


Reviews:
[{"review_id": "hOcE39jhpA", "reviewer": "Reviewer_aY9N", "summary": "This paper introduces a valuable new dataset called Mol-Instructions for instruction-tuning large language models (LLMs) on biomolecular tasks. The authors gather and collect high-quality biomolecular data from various licensed sources, such as PubChem and USPTO. Mol-Instructions covers a diverse range of instruction types across molecules, proteins, and biomedical text, aiming to enhance LLMs' capabilities in the biomolecular domain. Through fine-tuning on the Mol-Instructions dataset, the authors show that LLMs' abilities on diverse biomolecular tasks can be improved.", "questions": "- Have the authors considered any other representation formats besides SELFIES for molecules? I'm curious how will SELFIES perform against alternative descriptors.\n- The authors use BLEU, Levenshtein distance, and other text similarity metrics for evaluating molecule generation. Would more domain-specific metrics better depict the molecule generation capacity of the models? \n- Do you intend to expand Mol-Instructions with more task types and data in the future? What directions are you considering to continue improving it as a resource?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The proposed dataset, Mol-Instructions, is large-scale and comprehensive, covering a diverse range of tasks across molecules, proteins, and text. I appreciate the significant efforts in curating such a valuable dataset for pushing the research of LLMs for scientific domains.\n- Visualization and analysis of key dataset statistics are available, providing insights for other researchers in the field. The long-tail distribution analysis is informative.\n\n* The paper is well-written and provides a comprehensive description of the dataset construction process and baseline experiments.", "weaknesses": "**Main concern on experiments:**\n\nMy major concern is the experiment part. Specifically, the current comparison and presentation scheme can lead to overestimated results and misleading conclusions because it ignores some important domain-adapted baselines and mainly compares them with domain-agnostic baselines, which are not fine-tuned or designed for scientific tasks. The authors are encouraged to appropriately acknowledge and compare these tasks with previous methods and datasets in order to improve their potential impact. Here I list the important baselines below:\n\n1. An easier molecule instruction tuning dataset: Text+Chem T5 [1]. [1] has included most of the tasks/datasets in Mol-Instructions, including forward reaction prediction, retrosynthesis, description-guided molecule design, and molecule description generation. However, the discussion and comparison to Text+Chem T5 are missing. \n2. Scientific language models: Galactica [2]. Compared to the general-purpose LLMs (e.g., Vicuna and Alpaca) reported in the experiments, Galactica is specifically designed and pretrained for the science corpus and should be considered an important baseline.\n3. Molecular Language Models: MoMu [3] and MolT5 [4]. These models are designed for description-guided molecule design and molecule description generation\n4. The recent methods for reagent prediction [7], forward reaction prediction [5,6], and retrosynthesis [5]. Note that, the baselines in these domains employ different evaluation metrics compared to those reported in the paper. \n5. Baselines for protein-related tasks.\n\nI understand that incorporating all these methods for empirical comparison is challenging, and an LLM is not expected to outperform all the state-of-the-art methods in every task to be a valuable contribution. However, I think the comparison to Text+Chem T5 [1] and Galactica [2] is essential and important, considering their similar purposes to this submission. It would also be valuable to report the performances of the single-task state-of-the-art methods in the Tables and Figures (you may not have to outperform these methods) to help empirical comparisons in follow-up works.\n\n\n**Other weaknesses:**\n\n- The quality control and curation process for constructing the dataset should be described more thoroughly. More details in data processing would strengthen confidence in the dataset.\n- Table 5 lists the sources of data used in this paper, but it does not detail the availability or the methods for programmatic access to these data sources. Disclosing these particulars may aid researchers in replicating this paper.\n- The Mol-Instructions dataset integrates various tasks in the field of biochemistry, such as the molecular and protein-related characteristics shown in Table 2. However, the experiments only conduct tests on a limited number of biomolecular tasks. Such an evaluation approach may not fully reflect performance on complex biomolecular problems.\n\n\n\nReference:\n\n[1] Unifying Molecular and Textual Representations via Multi-task Language Modelling. In ICML 2023.\n\n[2] Galactica: A Large Language Model for Science. 2022\n\n[3] A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language. 2023\n\n[4] Translation between Molecules and Natural Language. In EMNLP 2022.\n\n[5] Root-aligned SMILES: a tight representation for chemical reaction prediction. In Chemical Science 2022.\n\n[6] Chemformer: a pre-trained transformer for computational chemistry. In *Mach. Learn.: Sci. Technol.* 2022.\n\n[7] Reagent prediction with a molecular transformer improves reaction data quality. In chemical science.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jDXl4YNmYK", "reviewer": "Reviewer_KuQ5", "summary": "The paper introduces \"Mol-Instructions,\" a novel dataset specifically designed for biomolecular research, with the primary objective of enhancing the effectiveness of Large Language Models (LLMs) within this domain. The dataset is generously shared as an open-source resource, encouraging extensive utilization in future endeavors. It encompasses diverse tasks and incorporates information from existing data sources, complemented by template-driven instructions. Covering a wide range of topics, the dataset encompasses molecular description, property prediction, protein design, question answering, and more. Stringent quality assurance measures have been implemented to ensure the dataset's authenticity and comprehensive diversity.", "questions": "I appreciate the authors' efforts in collecting such a comprehensive benchmark for biomolecular research. However, recent work[1] showed that a large number of instructions may not be necessary to obtain a well-finetuned model.\n\nTherefore, is it possible to investigate how the performance of Large Language Models (LLMs) using Mol-Instructions is affected by varying the number of training data instances, checking the redundancy, and possibly employing different sampling methods?\n\n[1]: LIMA: Less Is More for Alignment", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "+ The paper introduces \"Mol-Instructions,\" a novel dataset specifically designed for biomolecular research, with the aim of enhancing the capabilities of Large Language Models (LLMs) in the field of AI for Science.\n\n+ The authors employ this dataset to fine-tune an LLM, followed by a comprehensive evaluation of the obtained results.\n\n+ This paper exhibits a well-structured and well-motivated approach. The figures presented are clear and easily comprehensible, facilitating easy understanding and follow-through.", "weaknesses": "I do not see a major weakness in this paper. Please see my question below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1N7jkAYUb0", "reviewer": "Reviewer_SfxB", "summary": "This paper proposes Mol-Instructions, a new instruction dataset tailored specifically for the biomolecular domain. Mol-Instructions comprises three pivotal segments: molecule-centric instructions, protein-focused instructions, and overarching biomolecular text directives. Each segment is constructed carefully to bolster the interpretative and predictive prowess of LLMs pertaining to intricate biomolecular characteristics and dynamics. Through extensive instruction tuning experiments on LLMs, the authors demonstrate the effectiveness of Mol-Instructions in enhancing large models’ performance in the intricate realm of biomolecular studies, thus fostering progress in the biomolecular research community.", "questions": "See Weakness", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. This paper presents \"Mol-Instructions\" which is a helpful instruction dataset for LLMs in AI4Science. \n\n2. Via comprehensive evaluations using LLMs, the authors demonstrate the effectiveness of Mol-Instructions in enhancing large models’ performance in the intricate realm of biomolecular research. \n\n3. The paper underscores the importance of reproducibility, guaranteeing open access to both the dataset and the accompanying code. Such transparency empowers the community to verify and expand upon the findings, catalyzing cooperative efforts and propelling new works in biomolecular research.", "weaknesses": "The evaluation maybe a small issue, however, I know it is not easy to evaluate the molecule generation tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZjWDevhHGh", "reviewer": "Reviewer_RHmE", "summary": "In this study, the authors present Mol-Instructions, a comprehensive molecular instruction dataset designed specifically for Large Language Models (LLMs). Mol-Instructions incorporates pre-existing tasks, information gleaned from established data sources, and template-based instructions. Rigorous quality control measures have been implemented to ensure the dataset's validity and encompassing diversity.\n\nThis dataset encompasses a wide spectrum of tasks, including property prediction, molecular description, protein design, and Q&A, among others. Furthermore, it introduces an LLaMA model finetuned on the collected instructions, showcasing its exceptional performance improvements over the original model across multiple case studies.", "questions": "1. Could the authors provide more details about the quality control measures applied to the generated task descriptions in their study?\n\n2. Are there plans to include comparisons with domain-specific Large Language Models (LLMs) to provide a more robust evaluation?\n\n3. Is it possible to incorporate more challenging datasets from the ScienceQA to improve the paper's findings and conclusions?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "[Comprehensive Dataset Development] It creates an instruction-tuning dataset that can be used for the community of AI4Science and Large Language Models. This dataset addresses a critical need in the field and serves as a valuable resource for the broader scientific community.\n\n[Performance Improvement with tuned LLaMA Model]: The paper's demonstration of the LoRA-tuned LLaMA model's superior performance over the original model in multiple case studies is a significant strength. \n\n[Broad Range of Applications]: The paper's emphasis is on covering a wide range of applications, including property prediction, molecular description, protein design, etc, which has great potential to be adopted for many downstream applications.", "weaknesses": "[Limited Quality Control] In the initial phase of the study, there is an absence of information regarding quality control procedures for the generated task descriptions. A more detailed explanation or discussion of quality control measures, even if briefly mentioned in section 3.2, would enhance the transparency and reliability of the research.\n\n[Less Meaningful Experiments] While the performance improvements observed when fine-tuning the LLaMa model on the collected data are promising, it's essential to acknowledge that this setup resembles a supervised learning scenario. Consequently, the comprehensiveness of the performance assessment may be somewhat limited. To provide a more comprehensive evaluation, additional comparisons with domain-specific Large Language Models (LLMs), such as those mentioned in references [1] and [2], would be valuable. Additionally, incorporating more challenging datasets such as ScienceQA [3] could further strengthen the paper's empirical results.\n\n[Insufficient Analysis on Text Generation] The paper would benefit from a more in-depth analysis of the factuality of the generated text. \n\n[1] https://huggingface.co/stanford-crfm/BioMedLM\n\n[2] https://huggingface.co/chaoyi-wu/PMC_LLAMA_7B\n\n[3] https://proceedings.neurips.cc/paper_files/paper/2022/file/11332b6b6cf4485b84afadb1352d3a9a-Paper-Conference.pdf", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)