Hierarchically Encapsulated Representation for Protocol Design in Self-Driving Labs
Authors: Yu-Zhe Shi
Mingchen Liu
Fanxu Meng
Qiao Xu
Zhangqian Bi
Kun He
Lecheng Ruan
Qining Wang
Venue: ICLR 2025 Poster (2025)
Keywords: Self-driving laboratories
protocol design
automated design
domain-specific language
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Self-driving laboratories have begun to replace human experimenters in performing single experimental skills or predetermined experimental protocols. However, as the pace of idea iteration in scientific research has been intensified by Artificial Intelligence, the demand for rapid design of new protocols for new discoveries become evident. Efforts to automate protocol design have been initiated, but the capabilities of knowledge-based machine designers, such as Large Language Models, have not been fully elicited, probably for the absence of a systematic representation of experimental knowledge, as opposed to isolated, flatten pieces of information. To tackle this issue, we propose a multi-faceted, multi-scale representation, where instance actions, generalized operations, and product flow models are hierarchically encapsulated using Domain-Specific Languages. We further develop a data-driven algorithm based on non-parametric modeling that autonomously customizes these representations for specific domains. The proposed representation is equipped with various machine designers to manage protocol design tasks, including planning, modification, and adjustment. The results demonstrate that the proposed method could effectively complement Large Language Models in the protocol design process, serving as an auxiliary module in the realm of machine-assisted scientific exploration.

Published as a conference paper at ICLR 2025
HIERARCHICALLY ENCAPSULATED REPRESENTATION FOR PROTOCOL DESIGN IN SELF-DRIVING LABS
Yu-Zhe Shi",, Mingghen Liu Fanxu Meng", Qiao Xu*, Zhangqian Bi², Kun He",
Lecheng Ruan Qining Wang
Department of Advanced Manufacturing and Roboties, Peking University
2 School of Computer Science and Technology. Huazhong University of Science and Technology
"gqual contribution
X ruanlechengêucla.c edu, qiningwang@pku.ed edu.
ABSTRACT
Self-driving laboratories have begun to replace human experimenters in perform- ing single experimental skills or predetermined experimental protocols. However, the pace of idea iteration in scientific research has been intensified by Artifi- cial Intelligence, the demand for rapid design of new protocols for new discov- eries become evident. Efforts to automate protocol design have been initiated, machine designers, such Large Lan-
but the capabilities of knowledge-basee guage Models, have not been fully elicited, probably for the absence a sys- tematic representation of experimental knowledge, as opposed to isolated, flatten pieces of information. To tackle this issue, we propose a multi-faceted, multi- scale representation, where instance actions, generalized operations, and product flow models are hierarchically encapsulated using Domain-Specific Languages. We further develop data-driven algorithm based on non non-parametric modeling that autonomously customizes these representations for specific domains. The proposed representation is equipped with various machine designers manage protocol design tasks, including planning, modification, and adjustment. The re- sults demonstrate that the proposed method could effectively complement Large Language Models in the protocol design process, serving an auxiliary module in the realm of machine- scientific exploration.
INTRODUCTION
The rapid advancement of Artificial Intelligence (AI) models for the assistance of scientific discov- ery (Wang et al. 2023b) has precipitated an increased demand for rapid iteration of ideas, from the generation the verification of hypotheses. Although Al models have expedited the process of hypothesis generation, the validation phase still requires intensive empirical experimentation from human. The concept of self-driving laboratory has been introduced to substantially accelerate the validation process, in organic chemical synthesis (Mehr et al.) 2020 Burger et al., (2020), cell biol- ogy for medical research (Kanda et al. (002), and novel material discovery (Szymanski et al. 2023). With the expertise and effort of experimental scientists and automation engineers, mobile robots and Internet of Things (IoT) pipelines are configured to perform a sequence of actions accordance with detailed description of the specific experimental procedure, referred to as the protocol.
While existing protocols suffice for some experimental tasks, discovery processes often demand higher degree of specificity, including: (i) confirmation of unverified experimental objectives to seek specific findings; (ii) testing parallel hypotheses solutions; and (iii) replication of established ex- periments within the constraints available laboratory resources. These necessitate the design of new protocols, going beyond the reuse of existing ones available in the protocol databases. Particu- larly, this includes the planning of novel protocols, and the modification and adjusiment of current protocols as appropriate, respectively. Unfortunately. self-driving laboratories currently only execute isolated and duplicated experimental skills (Bédard et al.) 2018| et al. (2019), or pre-specified protocols with sequential actions (Rohrbach al.] 2022 Manzano al. 2022). Any innovation in
protocols imposes intensive manual design burden (McNutt 2014 Baker 2016), potentially becom-
ing a bottleneck in accelerating scientific discovery. Consequently, there quest for the automatic design of protocols tailored to specific goals for self-driving laboratories.
Designing new protocols is a non- non-trivial task even for human scientists. Novice scientists tend to adhere strictly to established protocols and may be at loss when faced with the need for variations,

---

Published as a conference paper at ICLR 2025
from minor adjustments like different available devices to more significant shifts in the overall ex- perimental goal. In contrast, veteran scientists typically have the capability to create or modify protocols as needed, from variations in available resources ("what I have"" ") to desired outcomes ("what I want") "), even in situations where similar protocol was not encountered before.
The distinction arises because veteran scientists possess a systematic understanding of every ingre- dient and procedure, contextualizing them globally within the domain of experiment. They know "what kind of ingredient used for whal purposes" " and "what kind of operation is used under what conditions', while novice scientists mechanically memorize the sequential execution orders and corresponding parameters in a local context. This systematic understanding, or conceptual knowledge (Ryle& & Tanney 1949), includes the background knowledge of ingredients and atomic operations, as well as the relationships between them. Experienced experimental scientists develop such conceptual knowledge as a representation for protocol design (McCarthy (955),, which serves as the vehicle for reasoning processes. Reasoning over conceptual knowledge leverages the rich context of generalized, abstracted concepts of ingredients and operations rather than specified, in- stantialized ones, which spans a semantic space where originally isolated dots are connected with each other, therehy enhancing the simplicity and flexibility of protocol design (Boden 1980 Newell| 1982). In summary, veteran scientists' capability to design new protocols stems from appropriate representation of background knowledge that supports reasoning processes (see Fig. TA).
To implement automatic protocol design on machines, a reasonable choice may be leveraging a Large Language Model (LLM). Trained extensive corpora, including scientific documents, LLMs possess the potential to facilitate protocol design with the corresponding background knowl- edge (Al4Science & Quantum 2023). Recently, researchers have made beneficial attempts to design new protocols using LLMs based on descriptions of new experimental goals (Boiko et al. 2023] M. Bran et al. 2024). Regrettably, benchmarking results indicate that the expected capability of LLMs in protocol design is not fully elicited (O'Donoghue et al., 2023). One significant limitation that LLMs excel at generating new protocols similar existing ones, i.e., protocols with similar sequential execution orders, but fail to generate those with distinct dependency distributions. This limitation hampers LLMs in scenarios where experimental goals change in high intensity. Another limitation is that the generated protocols sometimes lose critical configuration details for operation execution, necessitating manual correction. These empirical evidences suggest that LLMs exhibit limitations akin those of novice human experts, implying that LLMs may necessitate more suitable representation of background knowledge to fully unleash their potential in protocol design.
Protocol design is multi-faceted, multi-scale effort requiring the integration of information from different perspectives, from low-level high-level. This information includes detailed configura- tions of each atomic operation, temporal relationships between atomic operations, the scope of ap- plication for atomic operations with the same reference name, and the reactive relationships between reagents and operations. While LLMs undoubtedly capture such knowledge from their training cor- pora, the pieces of knowledge remain isolated, unorganized, and not articulated. These flatten back- ground knowledge, rather than conceptual knowledge, hinders LLMs from flying over a global view of the novel objectives and diving into the details of operations. Therefore, we propose developing a multi-faceted and multi-scale representation for protocol design that provides the designer, such as L.Mss, with a vehicle to reason over conceptual knowledge of ingredients and procedures.
We draw inspiration from both cognitive science literature on rationality (Monsell 2003 Griffiths) 2020), which suggests that we cannot consider information from different views and scales in single thread (Shi et 2023a).. We also learn from computer science literature on hierarchical ab- straction [Liskov] 1987). ). which indicates that higher-level abstraction semantics possess more pow- erful expressivity compared to their lower-level counterparts (Abelson & Sussman, 1996. Hopcrofi al.) 1996). Combining these insights, we suggest that our desired representation should encapsu- late information of different granularities in corresponding hierarchies of abstraction, gaining global design insights with higher-level semantics while completing execution configurations with lower- level semantics. Specifically, we investigate three levels of encapsulation (see Fig. 3B). Starting from the set of original protocols, namely the basic level, we have (i) protocol element instantial- ization, which decomposes full protocols into instance operations with attributes, within the local context of the specific protocol, resulting a structural representation of the elementary information; (ii) function abstraction, which offers an operation-centric view that generalizes the precondition, postcondition, and execution configurations of each operation in the global context of the experi- ment domain, resulting sequential representation of the operations; (iii) model abstraction, which offers an reagent and intermediate product centric view that unifies the status transitions in the global context of the experiment domain, resulting a continuous representation of the experimental environ-
2

---

Published as a conference paper at ICLR 2025
scientist
B Original protocal
for ponipsat brsteforJi:
2004 exvection, frest
plave baiee, gats loase
kanar aumple free the
Sample
actraction from
Precipitate
Lysete
Leavit Si
Sample
Lanves
Lysils
Precipitate
Lysatt
Laavel
Leaves Eg
Sample
Leaves 2h
Lysis
Leawer
Precipitate Lysate th
Pottin extruction from pkant sten otts
coke natirat ráái 2010 kansemiirr the hofher pracpiate proelt
Homoggenize
Precipttate
Function abstraction
Operation centric view
Butter
DHA (Loal)
Protein
EHA
dbNa setthoic vaarrr da cazsiiora the
<ONA gysthent
Procipitata
Precipitate
Procipitate
Sample
Lysis Duration
Precipitan
Moasure
Check
Transoript
Lysate Th
HiCel 12hh
Lysote 2h
Der
Devicw
and costtuttion cower nussirgttoo oo she RKA
RIIA
A.O.E
RNA
abetraction
Sample
Lysis
Precipitate
Transfor
Transeer
Transtirre
Travsfonn
Clack dhe intigrityy of the DMA fuom the Nant Suem Galls
Repuart experinenntal objective:
Leaves
LiSemple
Leaves
Corran
LiSample
Comtain
Lysote
Masce
Vii
Drop
Whish eristing
Cuttivate
Lysis
Precipitate
Dual representation operation- and product- -low---attiicccc views
protocl is wart sixdlor to the sear esperiventa dbjeatine
Lysate
Leaves
Sample
L.Sample
Lysts
Lysate
Precipitate
DNA
Phrase
Dexice
Containa
Device
Conteine
Device stiionn
Conteina Prop
A.G.E.
DNA
Masia
Duration
Prop
Figure 1: The representations for protocol design. (A) The example of protocol design by novice and veteran experimental scientists. (B) The hierarchies of our proposed representation, from origi- nal full protocol representation, to dual representation of operation- and product-flowe flowcctfloc-cctriccc views.
ment. This hierarchical structure provides the designer with representation to consider all possible associations among operations, among products, and between operations and products, with a high degree of freedom, by disentangling originally intertwined information. We implement the represen- tation using Domain-Speciffe Languages (DSLs). The hierarchical syntax of DSLS maintains both the abstract semantics at the high-level and the precise information at the low-level. Furthermore, the compositionality of DSL syntax facilitates the flexible protocol designs, addressing the "flying over global views" requirement; while DSL program verification over the generated protocols upholds their soundness and completeness; addressing the "diving into details" requirements.
However, the proposed representation does not come without drawbacks it can be highly de- pendent on domain-specifie knowledge (Meriik et al., [205]] Fowler, 2010)). The distributions of reagents, operations, and execution dependencies vary significantly across different domains in ex- perimental sciences, such Genetics, Medical, Bioengineering, and Ecology. Manually crafting DSLs specialized for these domains requires deep integration between domain experts and program- ming language experts, which is labour-intensive, case-by-case, and costly (Shi et al. [2024a b). This obstacle hinders the application of our representation a broader set of domains (Shi et al.) (002d). To make the representation specification more affordable, we develop an algorithm that conducts multi-hierarchy encapsulation automatically driven by the domain- tocols. Ultimately, we may be able to take a critical step toward closing the loop of autonomous scientific discovery by establishing these two building blocks: (i) the automatic generation of rep- resentation for protocol design; and (ii) the automatic designer working on the representation.
corpus of existing pro-
Our contributions in this work are three-fold: (i) we identify the problem of representation for pro- tocol design and develop a hicrarchically encapsulated representation for protocol design (Sec. 2); (ii) propose data-driven algorithm that automatically generates the representation for proto- col design specialized for the domain of application (Sec. 3); and (iii) we demonstrate the utility of the resulting representation by conducting protocol planning, modification and adjustment tasks using a variety of machine designers across different domains (Sec. 4). This further indicates that our proposed automatic representation generation approach possesses the potential to function as an auxiliary module for LLMs, enhancing their capability on protocol design.
2 REPRESENTATION FOR PROTOCOL. DESIGN
In this section, we describe our representation for protocol design (see Fig. 1). We first formulate the basic protocol design problem in 2.1 Afterwards. starting from the original full protocol, we introduce the three hierarchies of representations: (i) structural representation, i.e., instance actions with attributes (Sec. 2.2): (ii) sequential operation-centric representation, i.e., function abstraction representation, i.e., model abstraction (Sec.. 2.4). Furthermore, we describe how the dual representation of operation-centric and
(Sec. 2.3p: and (iii) continuous product-flow-centee
views reciprocatively facilitates the verification of the designed protocols in Sec. 2.5
3

---

Published as a conference paper at ICLR 2025
2.1 THE PROTOCOL DESIGN PROBLEM
Protocol design problem PD- = (Φ) w", P.Q)) is generating a desired protocol given the new coming experimental objective p. domain of experiment P. and available reagents S2. A protocol == (f1, .e........ is sequence of experimental steps (Pt. An experimental objective w" is the expected final product of the experiment. Experimental objectives can range from preparing a de- sired product, to testing the significance of specific hypothesis and detecting predicted behavior, with the latter two potentially followed by additional standalone steps for property test, observation, and interpretation. We denote domains of experiment as P, which influences the distributions of protocols by means of the distributions of operations, reagents, and execution orders, etc. The set of available reagents Q includes originally accessible reagents and excludes those requiring production.
2.2 INSTANCE ACTIONS WITH ATTRIBUTES
Protocols are originally represented in Natural Language (NL), which is the representation suitable for humans' comprehension, but not for machines (Bartley et al. 2023). Without a syntax decom- posing NL-based protocol into information elements precisely, machines are likely to capture only overall, -aaase-rainedd information of protocols and may only retrieve within existing protocols for the one that is most similar to the new experimental objective. Consequently, according to the stan- dards and conventions of experimental sciences (Baker) (02I), the prerequisite of representation for a machine protocol designer should be a structural representation which decompose NL-based protocols into instance actions with attributes (st, (sp) .pt )pfxe)) }. The instance actions are decomposed by execution order and their attributes are the exact context for their execution, namely the precondition g i.e., the availability of resources required for this action, postcondition op i..., resulting product of the operation, and execution configurations ptte Execution configurations includes the configuration parameters and their corresponding values, egg., the device for conduct- ing the operation and required experimental conditions such as duration, acidity, and lightening, An instance action can be reusable in another protocol once the execution context is matched.
With such reusability, we are on the first time to have building blocks for constructing a new protocol rather than retrieving existing ones. These building blocks capture fine- fine-grained execution configu- ration parameters through maintaining the nested data structures of key-value pairs. This structural representation serves as a syntactic constraint on the preciseness of designed protocols. Practical attempts have been made echoing this idea (O D'Donoghue et al. 2023 Leonov et al. (024).
2.3 OPERATION-CENTRA VIEW WITH FUNCTION ABSTRACTION
The reusability of instance actions with attributes is highly limited, as their semantics are highly specified in the low-level. The total amount of the instance actions can be extremely high, i.e., about 150K per domain, thus the probability of the exact matching between execution contexts can be ex- tremely low. Consider the three different instance actions with attributes "Hoomogeizztiin of mouse liver tissue using bead mill". "Homogenization of bacterial cell suspension using an ultrasonic homogenizer", and "Hoomogenization of bacterial air samples using a nebulizer", Although they come with totally different preconditions, postconditions, and execution configurations, particularly the required device varying according to the phase of the experimental subject, they share the mantic identifier "Homogenization" " for reference. Sharing semantic identifier indicates that these instance actions share the same purpose on the semantics level. In experimental sciences, "Homog- enization" " always refers to the breakdown of sample into a uniform mixture. Whether it's tissue, cell suspension, or gas doesn't 't change the purpose of the operation. This is critical for protocol de- sign, since it essentially requires satisfying the ultimate goal through series of subgoals. Therefore, the desired representation should generalize the semantics of operations to any possible contexts in the corresponding domain of experiment, rather than only specific contexts.
We implement such generalization by encapsulating varied instances of preconditions, postcondi- tions, and execution configurations into an interface for the operation. Namely, we refer to an op- eration with semantic identifier yp through an interface á to set of execution contexts, in the form of (ip + á + {(gPte good (exc)) }}). The operation o can be grounded a corresponding instance action in any matched execution contexts, echoing modular dexign (Hirtz et al. 2002). The reusabil- ity of encapsulated operations comes with greater significance than that of instance actions, as there are only about 1K operations per domain total, which is only 1/150 of that of instance actions. As flexible building blocks, operations can be easily fitted into any breakpoints with suitable precon- ditions and postconditions in the constructing experiment sequence. This sequential representation
4

---

Published as a conference paper at ICLR 2025
of the operations serves as semantic constraint on the compact permissible set of primitives for protocol design (Shi al., 2023b), maintaining both degree of freedom and correctness.
2.4 PRODUCT-FLOW-CENTROR VIEW WITH MODEL ABSTRACTION
Sequence of operations make up of protocols. However, operations are the methods to realize rather than the objectives to achieve. For experimental objectives of testing, preparing detect- ing (Schwab Held 2020), the common focus is always the specifie status final product, not the operations. Starting from initial reagents, the status of product flow is manipulated step-by-step by the operations, till the final product. Unfortunately, the information of product status transition is latent in protocols and is twisted with descriptions of experimental steps. For the operation-centric view, the transitions of product flow statuses remains black box environment. For example, the operation description "Centrifuge the tubes at 15,000 x for 20 minutes" does not directly reveal the transition from product in mixture status to products in distinct phases. The lack of coherent tracking of the product flow is problematic of protocol design, the product flow holds spatial- temporal invariance, just the same as the general physical environment. Status transitions of the product flow are primarily caused by the effects of operations, therehy it serves as the invariant in executing the protocol from the perspective of programming. Therefore, the desired representation should also serve as the model interacting with the sequence of operations.
To disentangle product status from their latent representation in the operation- view, we pro- pose an explicit product flow centric view that tracks the status of the product flows with detail, such as component, volume, container, and other physical and chemical properties of the product, and also the predecessor operation that yields the product and the successor operation that takes the product as imput. Each product flow unit, i.e,, one individual component in the produet flow between
two adjacent steps, is an instance with attributes {w; (Wi w/ w }. Analogous to the gen- eralization of operations' semantics, product flow units share commonalities between components with the same semantic identifier for reference- they may share specific range of predecessor op- erations pred and successor operations " and a selected set of key properties consider "rup, For example, the "supernatant" " is usually generated by "'cntrriuuaation" operation, passing into "filtration" or "spectrrohotomeericc analysis", ,, and focusing on the properties acidity and viscosity rather than other possible properties. Thus, encapsulate the information of contexts and prop- erties into the semantics of product flow units, in the form of (w + (wered suiee,, wo))) As solid pipelines bridging the building blocks, product flow units can verify the coherency of the entire designed protocol. This continuous representation of the environments serves as a program veri- fier, checking the prerequisite and simulating the effect of each operation, alleviating unpredictable behaviors among the interaction between operations and product flows.
2.5 RECIPROCATIVE VERIFICATION OVER THE DUAL REPRESENTATION
The dual representation of operation-centric and modduct-foo-centtie views intrinsically equips with a verification mechanism through
Algorithm
1 Reciprocative Verification
procedure OFVERIFICATION(M, (p)
a reciprocative process akin to two interacting verifying
b. CChek that the pre/ post conditions are met pere, sproa)
threads. The first thread focuses the operation flow, taking as input an operation sp along with its precondition st and post- condition sp. The second thread handles the verification of the product flow, taking as input a product w, along with its predecessor opera- tion wrrred and successor operation wf
if poree C (S2) then
M(2) + (M() \ (gree) U yot
Proceed to verify each outpul product for product in yoous do PFVERIFICATIONAM, W)
procedure PEVERITICATIONMM w)
> Check necessary properties of the product CHECKPROPERTIES/ Prop n0))
Specifically, for the operation verification (corresponding to OFVERIFICATION in Alg. 1.. we ensure that each operation can be correctly executed given its input reagents and that it yields the expected output products.
b if required by subsequent operations.
if 3 o s.t. ppere then D Verify operations using the product OFVERIFICAIION(MM 4')
This involves checking that the preconditions are satisfied by the available products from
preceding operations and that the postconditions are well-defined for subsequent use. Concurrently, the product flow verification (corresponding PFVERIFICATION in Alg. 1 involves tracking each
5

---

Published as a conference paper at ICLR 2025
Domain corpus protocols
Hanarchical non-parametric modeling
Resulting DELS the representatinn
Aerora de cabare take by preming.
Frengeaate TeuerfanC Sonnthrn
Aesupest
arriigiaa..
Pir Peciphie She ná cersie dhe years.
Phopplaur admo
Aeroie the cnlure madia bypiinnng.
Aeruppedd the palrr wuth carniegian..
Pasoos bank Jesticaatiii
0
C 0.0
-0.6
0
10
1.5
0.24
bobm
0.35
0.26
0.27
2.51
-2.0
10 100
T006
-2.0
300
1200
Denelic
Figure 2: Diagram of automatic representation generation. (A) Illustration of the workflow. (B) Convergence curve of automatic function abstraction. (C) Convergence curve of automatic model abstraction. (D-F) Confusion matrices on operation distribution (D), product distribution (E), and device distribution (F), between DSLs across domains. Correlation scores are low except the ones along the diagonals, indicating the significant inter-domain distinctions between the resulting DSLs.
unit of product flow through the protocol. We verify that the product generated by the specified operation and that it possesses the necessary properties wiTOP for consumption by the next operation.
The interaction between these two threads forms a feedback loop where the verification of opera- tions and products mutually inform and constrain each other. This reciprocative method allows us to iteratively refine the protocol, ensuring that each step is both operationally feasible and chemically coherent. LLMs are employed to implement the functions ChECKOPCONDDTIONS and CHECK- PROPERTIES, extracting and verifying operation conditions and product properties from natural language protocol descriptions through instruction-following in-context learning (Wei et al. 202T Brown et al. 2020). For the prompts employed, readers are referred to Appx. D.6
3
AUTOMATIC REPRESENTATION GENERATION
In this section, we describe the proposed data-driven algorithm to automatically generate the hierar- chically encapsulated representation for protocol design (see Fig. 2/)). We first define the problem of generating the desired representation by means of DSL design (Sec. 3.1). We then introduce methods for generating operation-centric (Sec. 3.2) and product-flow-centre (Sec. 3.3) DSL views.
3.1 THE REPRESENTATION GENERATION PROBLEM
We denote the problem of generating the representation for protocol design within a given domain
as RG = (1(p), {w)} P.C.. The representation is a DSL with language features accommodat- ing both the operation-centric program view (4) and the product- fodduct-foow-centrcc program view (w). The domain-specific corpus C = { 22..... 2ic}} consists of existing protocols published in top-quality journals within the corresponding experimental domain. The source and profiles of C of each domain is detailed in Appx. E.T We can obtain instance action with attributes based on C' in straightforward way through NL information extraction (see (Appx. D.3 implementation details). The prior knowledge of operations and products, p((p) and p(w), including the basic syntax of the key-value structures and the elementary taxonomies, is derived according to the general common- sense of experimental sciences, as aforementioned in [Sec. 2 Specifically, the problem essentially aims to fit the joint distribution models p(yp, o, so soPosi exxxe) and p(w, wood suce wro) with domain-specific corpus C given prior knowledge p(() and p(w:).
3.2 AUTOMATIC FUNCTION ABSTRACTION
The key challenge of encapsulating the operation- centric view is to aggregate all possible execution contexts for an operation, and then generalize the contexts to the interface. If we keep each of the use case one single instance of the interface, which can be in thousands regarding one operation, the generalization is meaningless. Since there is no prior knowledge about the interface in advance, we develop the algorithm following the idea of non- parammeect modeling, i.e., Dirichlet Process Mixture Model (DPMM), resulting in flexible identification of interface instances.

---

Published as a conference paper at ICLR 2025
Hierarchical non-parametric modeling As we must handle information coming in different granularities, from interface structures to values parameters, we choose to model the operations in a hierarchical fashion. Compared with the flatten spectral clustering approach developed by [Sh et al. (2024a), which compresses all information of an operation into embedding vector, our modeling competent for considering information at different levels comprehensively. We carefully adopt the prerequisite that the interface is generated subject to the operation, preconditions, postconditions, and execution configurations are generated subject to the interface, and the value of configuration parameters are generated subject to their corresponding keys. Thus, we have the model:
p(p, Φ, speie so goucee goecec))
p 4: ① mo pplp
4p @)p(p 9. p)plpp 14, @)p(á p)p((p)
where ge denotes the values of configuration parameters. Within each iteration of the DPMM process, we sample the variables level-by-leved Since the structures of preconditions, postcon- ditions, and the selection of devices and configuration parameters are discrete, we sample them directly from the Dirichlet Process. As permissible values of parameters can be discrete, e.g., an array of specific values, common in acidity preparation; continuous, e.g., an interval with min- imum and maximum values, common in temperature setting: mixed, e.g., an array of spe- cific values with random perturbations around the mean, common in timing control, we con- duct the sampling by integrating Gaussian Process with Dirichlet Process ge o yo ~ DP(a, (()))) G, p) GP(m, K), where a, H, m, and K are corresponding hyperparameters.
Unification of the interface While clustering similar interface instances encapsulates operations, there may remain redundant interfaces due to minor discrepancies. These discrepancies often arise from differences in parameter values or naming conventions that do not fundamentally alter the op- eration's 's functionality. To alleviate such redundancies, we implement a unification process for the interfaces. Specifically, interface instances associated with the same operation are considered equiv- alent if they have the same number of slots and emits and share the same keys in their execution configuration parameters. By abstracting away differences in parameter values and names, we unify these interfaces into single, generalized interface. akin to the algorithm proposed by Maatelli & Montanari (1982). Unification enhances the generality of the operation-cemtee view by consolidat- ing functionally- interfaces, maintaining a concise and representative set of operations.
Results Function abstraction converges the domains respectively, shown by the likelihood curve yielded by non-parametric model in Fig. 23. the DSL of Geneties, there are 304 operations in total, with an average of 7.9 interface instances per operation; for Medical, these two quanti- ties are 269 and 6.9; for Bioengineering, they are 196 and 7.8: and for Ecology, they are 100 and 3.5. We find that a majority of operations with high occurrence frequency are unique to one do- main, such as Pipette to Medical and Lyse to Genetics (see Fig. 2D. There are also common operations across domains, such as Concentrate and Culture. Take Concentrate for an exam- ple, its interface captures the instances with different devices according to input phases, e.g., use Bench-top bench-top_centriffi centrifuge for Liquid while instances with different emits, e.g. selecting Supernatant or Suspension as the product keep.
for Gas, and also
3.3
AUTOMATIC MODEL ABSTRACTION
The key challenge of encapsulating the product-flow-centtie view is select proper descriptive properties of a flow unit component. There exists false positive cases, where properties are attributed to components with the same semantic identifier but in different phases, e,g., we consider ethanol with the property volume when it comes in liquid and with the property pressure when it comes in gas. There also exists false negative cases, where exact same components are regarded as different ones due different reference names, e.g., Acetylsalicylic Acid, ASA, and Aspirin refer to the same thing. To alleviate false positive and false negative results, we discard the design choice of the interface in the operation ppertiion-centric view, which tends to cover the possibly richest context, and thereby have the nor-parametric model:
plw, Fred wae prop Poup.))
poopo whup Poo w wrewwa ( w)p(w),
where prop-y denotes the values of property parameters.

---

Published as a conference paper at ICLR 2025
Cl+
0.6
0.4
Ceec
Og Eaec
03 Exed
O5 Exe
05 Exe
Op
0.2
MDD
Tura
Tura
Goal
Goal
Goal
loU/Op)
UyUPodd)
lUDDev)
Sim(Exe)
Sim(Goal)
Sim(Param)
.d80>
Mard
18
IB
FB
0.5
1.0
0.5
1.0
0.5
1.0
0.5
1.0
0.6
1.0
1.0
leU/Op)
lot)Prodd)
loU)Dev)
Sim|Exa)
Sim|Goow)
Sim(Param)
Bin
18
IB
Bmah
FR
1.0
0.5
1.0
0.6
0.8
1.0
0.5
0.8
loU/Op)
InUpProd)
loU)Den)
Sim[Exa)
Sin(Gowl)
Sim(Param)
18
IB
0.5
1.0
1.0 0.6
0.8
0.8
0.9 1.0
Figure 3: Results of protocol design. (A) Profile of text-level similarity between testing sets of the three tasks. (B) Pairwise comparison between the capabilities of different machine designers across the six dimensions. (C-E) Performances of the seven machine designers on the plamning (C), modification (D), and adjustment (E) tasks across the six dimensions (index by column).
Results Model abstraction converges on the domains respectively, as shown by the likelihood curve in Fig. 2C. In the DSL of Genetics, there are 17, 190 model states, ie., product flow unit as product status, in total; the quantity is 12, 472 for Medical; 11, 418 for Bioengineering; and 2, 205 for Ecology. We find that most components of product flow units with high occurrence frequency are unique to one domain, such as RNA to Geneties and HCC to Medical IseeFigg 2/FF). Take Ethanol for example, the model captures its possible concentrations in liquid rather than in gas.
EXPERIMENTS AND DISCUSSION
In this section, we report and discuss the results of our experiments. We start from describing our realistic novel protocol design tasks (Sec. 4.1). along with the metrics to measure the consistency between the designed protocol and the groundtruth protocol (Sec. 4.2). Afterwards, we introduce the alternative representations and machine designers used for comparison (Sec. 4.3p. Finally, we report and analyze the experimental results both quantitatively and qualitatively (Sec. 4.4).
4.1 PROTOCOL DESIGN TASKS
Generating unverified experimental objec- tives and their corresponding protocols specially for our protocol design tasks is impractical because those experiments which have not been peer-reviewed and published can be problematic regarding the contents themselves. To maintain both reality and scale of the testing set, for each domain we filter out small subset of pro- tocols which significantly differ from the remaining major part of the protocol set and exclude this subset from the corpora for automatic representation generation selected subset form the groundrruth of the testing set.
Table 1: Statistics of the testing set. Fach cell presents the total number of protocols m and experimental steps
n in the form m (n))
Geneties
Medical
Biocnginecring
Ecology
Planning Modification Adjustment
10 (130)
7 (96)
12 (157)
(25)
(442)
15 (225)
16 (210)
659)
23 (219)
(87)
2 (26)
(81)
Appx. E.
This
8

---

Published as a conference paper at ICLR 2025
We exploit quantitative indicators to assist testing set selection, which follows the convention of measuring a protocol's novelty in experimental sciences (Schwab & Held 2020). We comprehen- sively consider three indicators: (i) similarity between the text embedding of the NL-based descrip- tion of purpose of protocols, employing the evaluation model in O'Dooogge et al. (2023); (ii) Inter- section over Union (IoU) between the instance actions of protocols; (iii) similarity between the exe- cution sequence of protocols, implemented through the Sequence Alignment (SA) algorithm (Smith) ct al. 1981). To note. indicators (ii) and (iii) are calculated upon the protocols pre-processed by the workflow described in Appx. D.3 Indicator (i) captures the high-level idea of protocol design, indicator (ii) is correlated to the implementation of the protocol design, while indicator (iii) captures the low-level information of protocol execution.
In response to the three purposes of protocol design introduced in Sec. 1] we specify the planning, modification, and adjustment tasks of protocol design. Candidate planning tasks, which are the confirmation of unverified experimental goals, come with relatively low scores (within the 20% lowest) on indicators (i) and (ii). Candidate modification tasks come with fair scores (around the 40% lowest) on indicators (i) and (ii) and relative low score on indicator (iii). Candidate adjustment tasks come with relatively high scores (within the 40% highest) on all of the three indicators.
We obtain the final testing set through a human-machine collaborative workflow. We first detect the outliers of the original protocol corpus of each domain under the metries above, thereby forming a candidate set. Afterwards, experts of the corresponding domain (holding at least a Master's 's degree majoring in that domain) manually check the applicability of protocols in the candidate set with vross-valiaation, discarding the misclassified ones, requesting for more candidate protocols, and refining the groundtruth file when necessary. The testing set includes 140 new protocols and 1757 steps in total, across the domains of Genetics, Medical, Bioengineering, and Ecology, with 23% for planning, 52% for modification, and 25% for adjustment (see Tab. and Fig. 3 for details).
4.2 INTER-PROTOCOL. CONSISTENCY METRICS
Evaluating the consistency between designed protocol and the groundtruth is not like compar- ing between two plain strings (O'Donoghue et al., 2023). Based on the corresponding com- monground in experimental sciences (Bartley et al. 2023), we design six-dimensional metrics to comprehensively cover all of the major factors without biased weighting and composition. The six dimensions include: (i) IoU on operations, IoU(Op) = [oU((($1..11))) }. (PI...1.13))
IoU between instance actions of the designed protocol &b and the groundtruth 4'; (ii) IoU on reagents and intermediate products, (Prol) = loU((wo....o)) {4/ (466...p-11): (iii) ToU on de- vices, lDev = ( (Dee))) H)) where (DDev), denotes the exact de- vice for conducting the instance action sp: (iv) Similarity between the execution sequences, Sim(Exec) SeqAlign( (Poo...o)), (F%....1-1)) where Seq.lign(...) denotes the ordered sequence similarity score calculation by the SA algorithm; (v) Similarity between experimental objectives, ((p))))))))))))))))))))))))))))) where S(-) represents the serialization operation on structural rep- resentations of protocols; (vi) Similarity between complete protocols at parameter-wise level, Sim(aar) = Cos(S( S(b')) These six dimensions capture protocol information from low to high granularities, and also measure the consisteney of both ingredient knowledge and procedural knowledge, offering a relatively objective evaluation standard.
Sim(Goal) = (ps(S(p), S(p').
4.3 MACHINE DESIGNERS
We implement an array of designers by combining different representations with different LLM- based automatic designers under tractable computing load (see Appx. D.7). We investigate four types of representations, including the original NL-based protocol representation (Flatten) and the three levels of encapsulation described in |Sec. 2, i.e., instance actions with attributes (Instance), operation-centric view only (Encapsulated).. and the dual representation with operation- and moduuct-flow-centric views (Encapsulated+). We consider three types of LLM-based protocol de- signers: (i) Baseline, pure LLM- approach with Retrieval-Augmennte Generation (RAG) on the corresponding corpora (Appx. D.4; (ii) Internal, which takes the specific representation as part of the prompt of an LLM, requesting it to output the protocol under the constraint of the given representation (Appx. D.3): (iii) External, where the representation serves as an external constraint layer for the output of an LLM, verifying and refining the designed protocols (Appx. D.6). Notably, the external verifier is part of the resulting DSL as our proposed representation for protocol design.

---

Published as a conference paper at ICLR 2025
The combination of representation and designer does not span Cartesian space due to the in- trinsic limitations of Flatten and Baseline. Therefore, we implement seven machine design- ers, including: (i) Blatten-Baseline(ffe
LLM with RAG on original protocol corpora: (ii) Instance-Baseline (IB), LLM retrieval on the protocol corpora translated into instance actions; (iii) Instance-Internall (II), prompting LLM with the Instruction Set Assembly (ISA) of instance actions, following the implementation of the currently state-of-the-art method O O'Doonohuue et al. (202); (iv) Encapsulated-Interan (EI)), prompting LLM with the DSL with operation- centric (EE), LLM equipping with the external verifier provided by the DSL with operation-centre view; (vi) Encapsulated- -ntpppsulaaee-eteaaa------------ Internal+ee== (EI+), prompting LLM with the DSL with the dual representation; and (vii) Encapsulated -eteeed-xxxeeeaa+eee (EE+), LLM equipping with the external verifier provided by the DSL with the dual representation.
view; (v) prncapsulated-Exxtemee
4.4 PROTOCOL DESIGN RESULTS
The complete quantitative results across the four domains, the three tasks, and the six dimensions of evaluation metries are presented at AtpApp. B Through paired samples t-test, we find that EE+ and EI+ significantly outperform other alternative approaches (EE+ outperforms EE: t(278) = 8.007, pdd < 0, < .0001; EI+ outperforms EI: f(278) 8,397, pa < 0,p < .0001; EE+ outperforms II: +(278) 24.493, pd < 0.P < 00001: EI+ outperforms II: f(278) 23. 23.855, μd < 0.p < 00001; see Fig. 3C-E). These comparisons demonstrate the suitability of our desired representation for pro- tocol design. Similarly, we find that approaches equipping with relatively higher-level representa- tion significantly outperforms their counterparts with a relatively lower-level representation (EE out- performs II: t(278) = 16.315, Md < 0, p < ,0001; EI outperforms II: 1(278) = 15.259, pd < 0, p .00010 II outperforms FB: (278)) 8.340, pd < 0,p < .0001; see Fig. 3B).
4.5 DISCUSSION
This work proposes a hierarchically encapsulated representation for the conceptual knowledge in experimental sciences, including instance actions with attributes, sequential representation of op- erations with function abstraction, and continuous representation of product-flows with model ab- straction, to fully elicit L.MMs' capability on protocol design as an auxiliary module. The following discussions on results reveal the design rationality, scalability, and generality of the representation.
Contributions of the building blocks The encapsulated representation approaches with dual views outperform their counterparts without dual views by enhancing both intra-step and inter-step details. At the intra-step level, EI and EE offer richer semantic information than IB and II, lever- aging protocol-centric view to capture detailed configuration each operation. This feature accounts for their satisfactory performance on (Op) At the inter-step level, EI+ and EE+ treat each step as FlowUni t. incorporating both preceding and succeeding step contexts, leading to notable im- provements Sim(Exec) and IoU(Prod). This creates a double assurance mechanism (Shi al.] 2024c): the first assurance comes from internal input/output checks within each instruction, and the second from the input/output characteristics inferred from neighboring instructions. Namely, estimate the output of the preceding operation and check its alignment with the current step's 's input. This design enhances step linkage, verification, and overall coherence, ensuring higher consistency and robustness in complex protocol workflows. Please refer to Appx. I.r the case study.
Handling different task complexities The overall performance aligns with the trend in complex- ity across the three tasks (Fig. 3A); however, the dual-view encapsulated representations, EI+ and EE+, demonstrate superior performance compared to their counterparts. In planning. these meth- ods consider all necessary components, enabling creative yet structured protocol generation. For modification tasks, they provide feedback on parameter changes, detecting inconsistencies that their counterparts might fail to capture. In adjustment tasks, EE+'s external verifier maintains protocol integrity by identifying component relationships. Please refer to Appx. for the case study.
Generality across domains Our DSL-based approaches offer unified, modular representation with generalizability across scientific domains (see fomain-indexed results at/Appx. B.2). The dual- view approach abstracts experimental processes into operations and flow units, capturing essential details while remaining applicable across fields. By representing dependencies between steps and tracking product flow, the replication of experiments could be enhanced. The framework captures cross-domain commonalities while allowing Nomain-specitfe content like specialized operations and reagents. This unified representation standardizes protocols and enables researchers to adopt ex- perimental protecols from multiple fields, fostering interdisciplinary collaboration and innovation. Please refer to Appx. for the case study. Limitations on generality are discussed at Appx. G.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was partially supported by the National Natural Science Foundation China under Grants 52475001. Q. Xu is visiting student at Peking University from University of Science and Technology of China. The authors would like thank Haofei Hou for his earlier works regarding domain-specific representations, and also Jiawen Liu for her assistance in figure drawings.
REFERENCES
Harold Abelson and Gerald Jay Sussman. Structure and interpretation of computer programs. The MIT Press, 1996.
Microsoft Research A14Science and Microsoft Azure Quantum. The impact of large language mod- els scientific discovery: preliminary study using gpt-4. arXiv preprint arXiv:2311.0736/ 2023.
Monya Baker. 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 2016.
Monya Baker. Five keys to writing a reproducible lab protocol. Nature, 597(7875):233225) 2021.
Bryan Bartley, Jacob Beal, Miles Rogers, Daniel Bryce, Robert P Goldman, Benjamin Keller, Peter Lee, Vanessa Biggers, Joshua Nowak, and Mark Weston. Building an open representation for biological protocols. ACM Journal Emerging Technologies in Computing Systems, 19(3):1- 21, 2023.
Anne-Catherine Bédard, Andrea Adamo, Kosi C Aroh, M Grace Russell, Aaron A Bedermann, Jeremy Torosian, Brian Yue, Klavs F Jensen, and Timothy F Jamison. Reconfigurable system for automated optimization of diverse chemical reactions. Science, 361 (6408):1220-- 2018
Margaret Boden. Artificial intelligence and natural man. Synthese, 43(3), 1980.
Daniil Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes, Autonomous chemical research with large language models. Nature, 624(7992):570-35 2023.
Tom Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. In Advances in Neural Information Processing Systems, 2020.
Benjamin Burger, Phillip M Maffettone, Vladimir V Gusev, Catherine M Aitchison, Yang Bai, Xi- aoyan Wang, Xiaobo Li, Ben M Alston, Buyi Li, Rob Clowes, et al. A mobile robotic chemist. Nature, 583(78155:2377-244 2020.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), 2021.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646, 2022.
Martin Fowler. Domain-specipy languages. Pearson Education, 2010.
Thomas L. Griffiths. Understanding human intelligence through human limitations. Trends in Cog- nitive Sciences, 24(11):873-88) 2020.
Julie Hirtz, Robert B Stone, Daniel McAdams, Simon Szykman, and Kristin L Wood. functional basis for engineering design: reconciling and evolving previous efforts. Research in Engineering Design, 13:65-82, 2002.
John E. Hopcroft, Rajeev Motwani, and Jeffrey D. Ullman. Introduction to Automata Theory, Lan- guages, and Computation. Addison- Addison-Wesley Longman Publishing Co., Inc., 1996.
Genki Kanda, Taku Tsuzuki, Motoki Terada, Noriko Sakai, Naohiro Motozawa, Tomohiro Ma- suda, Mitsuhiro Nishida, Chihaya T Watanabe. Tatsuki Higashi, Shuhei A Horiguchi, et al. Robotic search for optimal cell culture in regenerative medicine. Elife, 170777077 2022.

---

Published as a conference paper at ICLR 2025
Artem Leonov, Alexander JS Hammer, Slawomir Lach, Hessam M Mehr, Dario Caramelli, Davide Angelone, Aamir Khan, Steven O' O'uuliiaan, Matthew Craven, Liam Wilbraham, et al. An integrated self- oeff-optimizing programmable chemical synthesis and reaction engine. Nature Communications, (5(1):1240 2024.
Barbara Liskov. Keynote address-date abstraction and hierarchy. In Addendum to the proceedings Object-oriented programming systems, languages and applications (Addendum), 1987.
Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Augmenting large language models with chemistry tools. Nature Machine Intelli- gence, pp. 1-11,2024
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self- feedbeked In Advances in Neural Information Processing Systems, 2023.
J Sebastián Manzano, Wenduan Hou, Sergey S Zalesskiy, Przemyslaw Frei, Hsin Wang, Philip J Kitson, and Leroy Cronin. An autonomous portable platform for universal chemical synthesis. Nature Chemistry, 14(11):1311-1188 2022.
Alberto Martelli and Ugo Montanari. An efficient unification algorithm. ACM Transactions Programming Languages and Systems (TOPLAS), 4(2):258-282) 1982.
John McCarthy. Programs with common sense. London, 1959.
Marcia McNutt. Reproducibility. Science, 343(6168):229-225) 2014.
Hessam M Mehr, Matthew Craven, Artem I Leonov, Graham Keenan, and Leroy Cronin. A universal system for digitization and automatic execution of the chemical synthesis literature. Science, 370(6512) 2020.
Marjan Mernik, Jan Heering, and Anthony M Sloane. When and how develop domain-specific languages. ACM Computing Surveys (CSUR), 37(4))316-344 2005.
Stephen Monsell. Task switching. Trends in Cognitive Sciences, 7(3):134-140, 2003.
Allen Newell. The knowledge level. Artificial Intelligence, 18(1):87-177, 1982.
Odhran O'Donoghue, Aleksandar Shtedritski, John Ginger, Ralph Abboud, Ali Ghareeb, and Samuel Rodriques. Bioplanner: Automatic evaluation of Ilms on protocol planning in biology. In Proceedings the 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Shubham Pateria, Budhitama Subagdja, Ah-hwee Tan, and Chai Quek. Hierarchical reinforcement learning: A comprehensive survey, ACM Computing Surveys (CSUR), 54(5):1-35, 2021.
Simon Rohrbach, Mindaugas Šiaučiulis, Greig Chisholm, Petrisor-Alin Pirvan, Michael Saleeb, Hessam M Mehr, Ekaterina Trushina, Artem I Leonov, Graham Keenan, Aamir Khan, et al. Digitization and validation of a chemical synthesis literature database the chempu. Science, 377(6602):122-1888 180, 2022.
Gilbert Ryle and Julia Tanney. The concept of mind. Routledge, 1949.
Simon Schwab and Leonhard Held. Different worlds confirmatory versus exploratory research. Significance, 17(2):8-9 9, 2020.
Yu-Zhe Shi, Shiqian Li, Xinyi Niu, Qiao Xu, Jiawen Liu, Yifan Xu, Shiyu Gu, Bingru He, Xinyang Li, Xinyu Zhao, et al. PersLEARN: Research Training through the Lens of Perspective Cultiva- tion. In Annual Meeting of the Association for Computational Linguistics, 2023a.
Yu-Zhe Shi, Manjie Xu, John Hopcroft, Kun He, Joshua B Tenenbaum, Song-Chin Zhu, Ying Nian Wu, Wenjuan Han, and Yixin Zhu. On the complexity of Bayesian generalization. In International Conference on Machine Learning, 2023b.
Yu-Zhe Shi, Haofei Hou, Zhanggian Bi, Fanxu Meng, Xiang Wei, Lecheng Ruan, and Qining Wang. AutoDSL: Automated domain-specific language design for structural representation of proce- dures with constraints. In Anmual Meeting of the Association for Computational Linguistics, 2024a.
12

---

Published as a conference paper at ICLR 2025
Yu-Zhe Shi, Haotian Li, Lecheng Ruan, and Huamin Qu Constraint representation towards precise data-driven storytelligg. In IEEE Visualization and Visual Analytics Gen4DS Workshop. 2024b.
Yu-Zhe Shi, Fanxu Meng, Haofei Hou, Zhangqian Bi, Qiao Xu, Lecheng Ruan, and Qining Wang- Expert-level protocol translation for self-driving labs. In Advances in Neural Information Pro- cessing Systems, 2024c.
Yu-Zhe Shi, Qiao Xu, Fanxu Meng, Lecheng Ruan, and Qining Wang. Abstract Hardware Ground- ing towards the Automated Design Automation Systems. International Conference on telligent Robotics and Applications. 2024d.
Temple F Smith, Michael S Waterman, et al. Identification of common molecular subsequences. Journal of Molecular Biology, 147(():195-11)) 1977 1981.
Sebastian Steiner, Jakob Wolf, Stefan Glatzel, Anna Andreou. Jarostaw M Granda, Graham Keenan, Trevor Hinkley, Gerardo Cagaaa-Ca Philip J Kitson, Davide Angelone, al. Organic synthesis in a modular robotic system driven by chemical programming language. Science, 363 (6423):eaav2211. 2019.
Nathan Szymanski. Bernardus Rendy, Yuxing Fei, Rishi Kumar, Tanjin He, David Milsted, Matthew McDermott, Max Gallant, Ekin Dogus Cubuk, Amil Merchant, et al. An autonomous laboratory for the accelerated synthesis of novel materials. Nature, 624(7990):86-91, 2023.
Bailin Wang, Zi Wang, Xuezhi Wang, Yuan Cao, Rif A Saurous, and Yoon Kim. Grammar prompt- ing for domain-specific language generation with large language models. In Advances in Neural Information Processing Systems, 2023a.
Hanchen Wang. Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47-66) 2023b.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 2021.
Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu. et al. Skywork: A more open bilingual foundation model. arXiv preprint arXiv:2310. 00022310.993444 19341, 2023.
Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xi- aojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al. Chain-of-experrs When ilms meet complex operations research problems. International Conference on Learning Representations, 2023.
13

---

Published as a conference paper at ICLR 2025
A ADDITIONAL REMARKS
A.1 RATIONALE OF THE OVERALL DESIGN CHOICE
seems that we can formulate the protocol design problem in the fashion of Markov Decision Pro- cess (MDP) and solve it by ceuristic-basel planning methods or Hierarchical Reinforcement Learn- ing (HRL) approaches. However, although the formulation itself is feasible, solving the problem may not be practical. Consider solving the problem through an HRL approach designed for hetero- geneous action space with parameters (as the protocol is required to decide both the key properties of an operation and the corresponding values). This hierarchical agent may be trained to converge on a fine-grained environment with a clearly designed reward function, or on large dataset with trajectories for offline learning. Unfortunately, we have access neither an interactive environment simulating the experiments nor sufficient data to support offline training (Pateria et al.) 202T).
Treating the experimental procedures as a white box and creating digital twins for experiments can be an elegant solution and thereby facilitate various applications other than protocol design. This effort requires elaborated design of simulation granularity, exhaustive collection of primitive principles of the system, efficient implementation of rule production, and define precise metrics for evaluating the distance between current and objective states (serving as reward function), which can be labor-intensive and is far out of the scope of this work. On the other hand, viewing those published protocols as trajectories for offline training, the scale of the offline dataset and the density of the reward function are much too insufficient to support training to convergence. Augmenting the data, synthesizing realistic trajectories, or enhancing the accessibility of protocols, are out of the scope of this work. Given the current obstacles, we choose not to formulate the problem in an MDP fashion. Though an MtP-style formulation can be more precise and elegant, it may misguide the knowledge provided readers to some extent. Instead, we decide to leverage the rich complement the lack of data and by knowledge-bard based agents such as LLMs. where knowledge may dense reward function. This design choice is also in line with the initial attempts on automatic experiment design (Boiko et al. 2023 M. Bran et al. 2024).
In summary, our design choice of formulation is a compromise based on currently limited resources and restricted scope. Nonetheless, the exploration of more precise and elegant formulations repre- sents promising avenue for future research.
A.2 INTUITION BEHIND THE INTERFACE
Interface is concept of functional abstraction (Abelson & Sussman 1996). Interface disentangles the abstract functionality on the semantics level and its corresponding implementation details on the execution level. This approach encapsulates the implementation of an operation into black-box, so the users of the operation would only need to consider its input and output. Therefore, with such encapsulated representation for protocol design, we only need to care about the consistency between the output of the predecessor operation and the input of the successor operation, without caring about their implementation details.
This is the idea behind operationalization. Operationalization makes the interface an abstract func- tion over all relative instance actions. The interface is abstracted from the execution contexts of all instance actions with the same reference name, i.e., the same purpose, and can be instantiated to an instance action given a specific execution context. A specific context can be the predecessor opera- tion, the successor operation, the precondition, or the postcondition of the considered operation, An instance action configures a specific implementation for a specific execution context. For the op- eration "Homogenization" ", the implementation of one instance action be "using an ultrasonic homogenizer" " if the precondition, namely, the execution context, has intermediate product "cell suspension" available; the implementation of another instance action can be "using a bead mill" if the precondition contains tissue. This example demonstrates the relationship between interface and instance actions of an operation: the interface is abstracted from the set of instance actions and can be instantiated to instance actions.
Here we also give a more intuitive example to enhance the reader's comprehension. Consider the culinary scenario with the actions "frying the egg", "frying the fish", and "frying the steak", Thesse different instance actions coming with the same purpose "to fry something" " Therefore, we can abstract the interface from these instance actions to operationalize the operation "fry". The input of "fry" should be something raw and its output should be something fried. Given different precondi- tions with available eggs or pieces of steak, the abstract semantic operation "fry" can be grounded
A1

---

Published as a conference paper at ICLR 2025
to instance actions "frying the egg" or "frying the steak" respectively, through the instantiation of the interface. In summary, an interface serves as the bridge between the semantics level and the execution level.
A.3 VALUES OF MANUAL PROTOCOL CERTIFICATION
Certification is always one of the central focuses in the engineering practices of automation. In our practice, we only automate the process of protocol design, which is the primary objective of this work, and keep the manual certification part. On one hand, relieving experimental scientists from the labour-intensive protocol design tasks, thereby allowing them more time for high-level thinking, is sufficiently significant improvement far. On the other hand, engineering practices such as lab automation and manufacturing are high demand for preciseness. This leads to the requirement of manual certification. Domain experts handle subtle cases through their tacit domain- specific knowledge and responsible for their decisions (Wang al. 2023b). According these considerations and the standard operating processes of experimental sciences, we choose to certify the designed protocols by domain experts.
Our current choice is a compromise on the limitation of techniques and the demand for preciseness. In future work, we can conduct investigations on how to build digital twins of self-driving labo- ratories. Such digital twins support prediction, explanation, and counterfactual analysis of unseen behaviors of the experiments, which may facilitate machine- based protocol certification. Grounding these blue-sky thoughts necessitates addressing the challenging problems regarding the decision of simulation granularity, the implementation of data-efficient simulation model construction, and the injection of tacit domain -omain-speccfiic knowledge, In summary, the exploration of senerated-protocee certification by machines represents promising avenue for future research.
A.4 LIMITATIONS OF AUTOMATIC PROTOCOL CERTIFICATION
LLMs can be much too uncontrollable for engineering practices such as lab automation, which may lead to unpredictable dangerous situations (Wang et al. (023b)). There comes a dilemma-we try to exploit the capability reasoning over knowledge of LLMs, while we try alleviate the drawbacks brought up by the uncontrollable nature of LLMs. Our proposed representation is dedicated to resolving the dilemma. The representations not only elicit LLMs' potential on protocol design through structural knowledge representation, but also serve as a guardrail for LLMs. Since the generated protocols are represented as corresponding DSL programs, the permissible output space much more confined compared with that of pure LLMs, serving constraints upon the LLM- generated protocols. Thanks to the verification mechanisms provided by DSLs, the correctness of the generated protocols be checked some extent. Therefore, by equipping LLMs with auxiliary constraint layer, we may approach a balance between knowledge utilization and preciseness.
However, the current verification on the level of DSL programs is far from sufficient for serving as certification. Certification is serious process, where any possibilities of reporting false positive cases are required to be eliminated. Some cases can highly long-tailed distributed, which may not be detected by data-driven and knowledge-driven machine certifiers. In this context, human domain experts are responsible for coming up with these potential risks through their experiences and tacit knowledge. Therefore, we are not likely to move human experts out of the loop, except that we can efficiently build up appropriate digital twins for self-driving laboratories. In current practices, the automation of protocol design puts human experts into a larger loop without focusing on the low- level details of experiments. As a result, they are allowed more time for high-level thoughts on things like values, which are not likely to be alternated by machines. In summary, it is neither practical nor necessary to totally move human experts out of the loop of automatic scientific discovery. The investigation of numan-machine coordination in protocol certification represents promising avenue for future research.
A.5 RATIONALE FOR THE REAGENT CONSUMPTION MODEL
We treat the instantiation and the consumption of reagents a one-time deal without considering the exact volume of consumption and the corresponding remainder. The rationale for such design choice comes from both the current Standard Operating Process (SOP) of experimental sciences and the properties of self-driving laboratories (Bartley al. 2023
A2

---

Published as a conference paper at ICLR 2025
In the current SOP for manually conducted experiments, experimenters required to prefab- ricated sets of reagents. Similarly, experimenters use specific containers with predefined capacities to transfer intermediate products. Therefore, one pack of reagents or one container of intermediate products is only used for operation, without considering the remainder. This results in more suceinct representation where reagents are regarded as discrete elements rather than continu- ous volumes.
For self-driving laboratories, this is deliberately designed for efficient variable management follow- ing the corresponding principles in computer system design (Abelson & Sussman 1996). com- puter systems, not removing used variables would cause out-of-memory errors, let alone in physical automation systems, where the physical memory slots are much harder than the virtual memory slots in computer systems to manage. Hence, we exploit this variable management mechanism to enhance the execution efficiency of self-driving laboratories.
A.6 RELATION TO LLM REASONING
We would like to clarify that objective is not alternate Chain-of-Thoughe (CoT) reasoning. According to recent studies on the properties CoT, LLMs with CoT may generate coherent but unprofessional text in expertise-intensive application scenarios (Xiao et al. 2023). Therefore, our proposed representation serves an auxiliary guardrail module for LLMs with reasoning techniques such as CoTT, enhancing LLMs' reasoning capability from two aspects: (i) the representation con- strain the scope of reasoning into close set of entities, such available operations, reagents, and devices commonly used the domain; and (ii) the representation provides fine-grained injection of domain-specific knowledge for LLMs, resulting in not only coherent but also expertise-coppatibl generated content.
A.7 APPLICABILITY TO DOMAINS BEYOND SCIENTIFIC EXPERIMENT
In theory, our framework can be applied any field that requires adherence specific protocols and has a need for automated execution. Let us consider an automated kitchen controlled by a computer example.
Assuming the automated kitchen's computer is already programmed to prepare "braised pork ribs" and "steamed sea bass":
1 Braised Pork Ribs:
2
3 1. Select pork ribs as the main ingredient.
4 2. Heat pan over high heat.
5 Add the ribs to the pan and fry for about 5 minutes until they are browned.
4. seasonings: soy sauce and sugar.
7 5. Reduce the heat to medium.
8 6. Simmer the ribs for 30 minutes until tender.
9 7. Serve hot.
10
START
12 SELECT ingredient: ribs
13 ACTION: fry, temperature: high, time: 5 min
14 ADD seasoning: soy sauce, sugar
15 ACTION: simmer, temperature: medium, time: 30 min
END
18 Steamed Sea Bass:
19
20 1. Select whole bass the main ingredient.
21 2. Prepare steamer and heat it to high temperature.
A3

---

Published as a conference paper at ICLR 2025
22 3. Place the bass in the steamer.
4. Steam the fish about 15 minutes until fully cooked.
24 5. Add seasonings: ginger slices and chopped scallions.
25 6. Serve immediately with the garnish.
26
27
START
SELECT ingredient: sea bass
ACTION: steam, temperature: high, time: 15 min ADD seasoning: ginger, scallion
30
31 END
Next, we can derive the corresponding DSL. For instance:
1
2
{
3
"braise":
4
"steps":
5
("type": "fry", "temperature": "high", "time": "5 min"),
["type": "simmer", "temperature": "medium", "time": "30 min")
7
1.
8
"seasoning": ["soy sauce", "sugar"]
9
}.
10
"steam": {
"steps":
12
t"type": "steam", "temperature": "high", "time": "15 min")
13
1.
14
"seasoning": ["ginger", "scallion"]
15
},
17 "ingredients": {
18 "ribs": {
19
"category": "meat",
20
"default_braise_ttin time": "30 min"
21
}.
sea bass": {
23
cateeory": "fish",
24 "default_braise_ttim "imee"s "20 min",
25
"default_ time": "15 min"
26
28 }
Now, let create recipe for Braised Sea Bass by combining the braising technique with bass as the main ingredient.
1 START
2 SELECT ingredient: sea bass
3 ACTION: fry, temperature: high, time: min
ADD seasoning: soy sauce, sugar
5 ACTION: simmer, temperature: medium, time: 20 min
6 END
A4

---

Published as a conference paper at ICLR 2025
B COMPLETE RESULTS
B.I TASK-INDEXED COMPLETE RESULTS
Table AI: Complete quantitative results protocol design, specifically the planning task. Each cell represents the machine designer's average score (at the top of the cell) on all testing samples across the four domains and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
(Op(Op)
loU(Prod)
Ioe(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.143
0.040
0.020
0.282
0.766
0.826
(0.087)
(0.055)
(0.060)
(0.090)
(0.096)
(0.059)
IB
0.109
0.036
0.019
0.242
0.735
0.781
(0.067)
(0.053)
(0.074)
(0.065)
(0.090)
(0.069)
II
0.382
0.050
0.084
0.452
0.788
0.851
(0.154)
(0.062)
(0.191)
(0.134)
(0.074)
(0.062)
EI
0.542
0.305
0.259
0.572
0.849
0.926
(0.160)
(0.181)
(0.211)
(0.152)
(0.066)
(0.026)
EI+
0.603
0.555
0.357
0.737
0.875
0.949
(0.208)
(0.260)
(0.237)
(0.172)
(0.057)
(0.023)
EE
0.524
0.370
0.252
0.558
0.846
0.928
(0.151)
(0.198)
(0.206)
(0.148)
(0.078)
(0.025)
EE+
0.607
0.605
0.355
0.744
0.893
0.951
(0.211)
(0.235)
(0.242)
(0.179)
(0.056)
(0.021)
Table A2: Complete quantitative results on protocol design, specifically the modification task. Each cell represents the machine designer's average score (at the top of the cell) on all testing samples across the four domains and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
ToU(Op)
loU(Prod)
IoU(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.181
0.050
0.038
0.304
0.796
0.809
(0.102)
(0.071)
(0.071)
(0.102)
(0.090)
(0.060)
IB
0.150 (0.100)
0.038
0.039
0.281
0.772
0.788
(0.065)
(0.076)
(0.100)
(0.089)
(0.060)
II
0.331
0.101
0.061
0.416
0.802
0.851
(0.143)
(0.131)
(0.135)
(0.127)
(0.087)
(0.059)
EI
0.593 (0.186)
0.318
0.336
0.602
0.866
0.937
(0.158)
(0.235)
(0.164)
(0.066)
(0.030)
EI+
0.648
0.626
0.413
0.765
0.883
0.952
(0.210)
(0.188)
(0.256)
(0.170)
(0.055)
(0.031)
EE
0.588 (0.185)
0.403
0.332
0.601
0.873
0.940
(0.192)
(0.228)
(0.164)
(0.053)
(0.028)
EE+
0.640
0.661
0.410
0.757
0.893
0.953
(0.213)
(0.179)
(0.253)
(0.170)
(0.043)
(0.032)
A5

---

Published as a conference paper at ICLR 2025
Table A3: Complete quantitative results on protocol design, specifically the adjustment task. Each cell represents the machine designer's 's average score (at the top of the cell) on all testing samples across the four domains and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
IoU(Op)
(ProD)
loU(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.192
0.077
0.051
0.319
0.811
0.823
(0.100)
(0.104)
(0.094)
(0.103)
(0.078)
(0.051)
IB
0.197 (0.131)
0.039
0.006
0.337
0.802
0.810
(0.063)
(0.021)
(0.141)
(0.082)
(0.049)
II
0.453 (0.208)
0.115
0.091
0,508
0.805
0.873
(0.161)
(0.211)
(0.184)
(0.081)
(0.056)
EI
0.587
0.328
0.400
0.623
0.863
0.944
(0.190)
(0.186)
(0.265)
(0.165)
(0.055)
(0.027)
EI+
0.668 (0.208)
0.545
0.449
0.775
0.883
0.950
(0.259)
(0.247)
(0.152)
(0.056)
(0.040)
EE
0.581 I (0.184)
0,404
0.395
0.616
0.875
0.946
(0.205)
(0.261)
(0.162)
(0.039)
(0.026)
EE+
0.650
0.589
0.441
0.758
0.893
0.950
(0.220)
(0.229)
(0.248)
(0.160)
(0.033)
(0.042)
B.2 DOMAIN-INDEXED COMPLETE RESULTS
Table A4: Complete quantitative results on protocol design, specifically the Genetics domain. Each cell represents the machine designer's average score (at the top of the cell) on all testing samples across the three tasks and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
IoU(Op)
(oU(Prod)
IoU(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.179
0.065
0.037
0.301
0.795
0.805
(0.113)
(0.082)
(0.080)
(0.116)
(0.091)
(0.066)
IB
0.157 (0.129)
0.042
0.022
0.297
0.793
0.789
(0.060)
(0.059)
(0.137)
(0.070)
(0.062)
II
0.379 (0.200)
0.120
0.079
0.457
0.807
0.850
(0.158)
(0.160)
(0.180)
(0.083)
(0.072)
EI
0.599 (0.189)
0.332
0.353
0.619
0.862
0.941
(0.177)
(0.243)
(0.164)
(0.055)
(0.026)
EI+
0.691
0.606
0.429
0.803
0.882
0.954
(0.198)
(0.252)
(0.283)
(0.151)
(0.054)
(0.033)
EE
0.592 (0.189)
0.415
0.351
0.615
0.870
0.943
(0.206)
(0.241)
(0.163)
(0.052)
(0.025)
EE+
0.677
0.653
0.425
0.791
0.888
0.955
(0.210)
(0.228)
(0.280)
(0.161)
(0.045)
(0.034)
A6

---

Published as a conference paper at ICLR 2025
Table A5: Complete quantitative results on protocol design, specifically the Medical domain. Each cell represents the machine designer's 's average score (at the top of the cell) on all testing samples across the three tasks and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
IoU(Op)
(Prod)
loV(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.174
0.048
0.030
0.312
0.796
0.839
(0.085)
(0.070)
(0.067)
(0.075)
(0.087)
(0.043)
IB
0.139 (0.054)
0.029
0.023
0.264
0.721
0.795
(0.038)
(0.063)
(0.045)
(0.123)
(0.070)
II
0.373 (0.093)
0.081
0.091
0,424
0.776
0,871
(0.087)
(0.205)
(0.072)
(0.097)
(0.041)
EI
0.604
0.322
0.309
0.594
0.861
0.932
(0.167)
(0.146)
(0.253)
(0.148)
(0.079)
(0.031)
EI+
0.615
0.574
0.400
0.758
0.871
0.952
(0.196)
(0.242)
(0.198)
(0.149)
(0.060)
(0.021)
EE
0.591
0.373
0.298
0.583
0.873
0.936
(0.158)
(0.166)
(0.234)
(0.149)
(0.054)
(0.030)
EE+
0.615
0.613
0.390
0.756
0.891
0.955
(0.197)
(0.210)
(0.202)
(0.151)
(0.040)
(0.019)
Table A6: Complete quantitative results on protocol design, specifically the Ecology domain. Each cell represents the machine designer's average (at the top of the cell) all testing samples across the three tasks and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
IoU(Op)
loU(Prod)
IoU(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.155
0.030
0.021
0.297
0.781
0.807
(0.085)
(0.035)
(0.048)
(0.088)
(0.096)
(0.056)
IB
0.162 (0.118)
0.006
0.030
0.275
0.763
0.788
(0.015)
(0.058)
(0.105)
(0.090)
(0.063)
II
0.386 (0.176)
0.043
0.027
0.448
0.788
0.856
(0.062)
(0.062)
(0.131)
(0.065)
(0.044)
EI
0.458
0.259
0.351
0.514
0.879
0.933
(0.171)
(0.134)
(0.195)
(0.142)
(0.048)
(0.028)
EI+
0.411
0.569
0.359
0.586
0.888
0.945
(0.134)
(0.133)
(0.175)
(0.127)
(0.052)
(0.023)
EE
0.458 (0.171)
0.347
0.351
0.507
0.874
0.934
(0.151)
(0.195)
(0.138)
(0.048)
(0.029)
EE+
0.414
0,581
0.346
0,586
0.910
0.944
(0.142)
(0.141)
(0.177)
(0.131)
(0.035)
(0.024)
A7

---

Published as a conference paper at ICLR 2025
Table A7: Complete quantitative results on protocol design, specifically the Bioengineering domain. Each cell represents the machine designer's ' average score (at the top of the cell) all testing samples across the three tasks and the corresponding standard error of mean (at the bottom of the cell). For each dimension, we highlight the results of both the best and the second best ones.
Iop(Op)
(Prod)
loU(Dev)
Sim(Exec)
Sim(Goal)
Sim(Param)
FB
0.176
0.048
0.050
0.300
0.790
0.826
(0.085)
(0.089)
(0.081)
(0.084)
(0.090)
(0.042)
IB
0.149 (0.077)
0,050
0.038
0.286
0.767
0.797
(0.087)
(0.091)
(0.078)
(0.083)
(0.046)
II
0.352
0.062
0.066
0,443
0.810
0,860
(0.151)
(0.090)
(0.187)
(0.125)
(0.073)
(0.045)
EI
0.565
0.307
0.310
0.603
0.851
0.930
(0.164)
(0.186)
(0.249)
(0.169)
(0.072)
(0.033)
EI+
0.657
0.577
0.394
0.743
0.888
0.944
(0.209)
(0.177)
(0.241)
(0.179)
(0.056)
(0.041)
EE
0.558
0.392
0.303
0.598
0.855
0.933
(0.162)
(0.214)
(0.246)
(0.165)
(0.076)
(0.030)
EE+
0.653
0.614
0.401
0.742
0.900
0.945
(0.206)
(0.172)
(0.246)
(0.176)
(0.046)
(0.041)
C ETHICS STATEMENT
C.I HUMAN EXPERT PARTICIPANTS
The testing set selection and groundtruth checking tasks conducted by human experts in this work has been approved by the Institutional Review Board (IRB) of Peking University. We have been committed to upholding the highest ethical standards in conducting this study and ensuring the protection of the rights and welfare of all participants. We paid the domain experts wage of $22.5/h for their work in this study.
We have obtained informed consent from all human experts, including clear and comprehensive information about the purpose of the study, the procedures involved, the risks and benefits, and the right to withdraw at any time without penalty. Participants were also assured of the confidentiality of their information. Any personal data collected (including name, age, and gender) was handled in accordance with applicable laws and regulations.
C.2 CORPORA COLLECTION
We carefully ensure that all protocols included in corpora strictly comply with open access policies under the Creative Commons license. This strategy guarantees adherence copyright and intellectual property laws, thereby preventing any potential infringement or unauthorized use of protected materials. By exclusively employing resources that freely accessible and legally dis- tributable, we maintain the highest standards of ethical research conduct, promoting transparency and respect for the intellectual property rights of others. This commitment ensures that our work advances the frontiers of knowledge manner that both legally sound and ethically responsible.
D IMPLEMENTATION DETAILS
D.1 PRIOR MODEL OF PRODUCT FLOW-CENTRIC VIEW
1 <ProductFlow>
<Pred>
<FlowUnit>
<Succ>
A8

---

Published as a conference paper at ICLR 2025
2
<Pred> ::= <Operation. UniqueName>
4
5 <Succ> ::= <Operation. UniqueName>
6
7 <FlowUnit> <Component> <ComponentType>
<Name>>
<Vol>
<Container>
Cond>
8 <Component> ::= <STR>
9 <ComponentType> ::= Gas Liquid Solid Semi -Slld
| Mixture
ChemicalCompound BiologicalMaterial | Reagent
|
PhysicalObject
File/Data ... [Known component types]
10 <RefName> <Component> <Index>
<UnitArgType> ::- MAT PROD
12 <Vol> ::= <REAL> <MEAS>
13 <Container> ::= Tube | Flask | Pipette
[Known container types]
14 <Cond> ::= <ArgKey> <ArgValue>
15 <ArgKey> Temperature Pressure Acidity conditional keys]
Lighting
... [Known
<ArgValue> ::= <REAL> <MEAS>
D.2 PRIOR MODEL OF OPERATION-CENTRIC VIEW
1 <0peration> ::= <UniqueName *<Pattern>
2
3 <UniqueName> <STR>
5 <Pattern> ::= <Precond> <Execution> <Postcond> *<Xxample>
6
7 <Precond> ::= <SlotArgNum> ><SlotArg>
8 <SlotArgNum> <INT>
<SlotArg> <ProductFlow. ttonuuttloo..llouiii.... componentyeet
11 <Postcond> ::= <EmitArgNum> ><EmitArg>
12 <EmitArgNum>> ::= <INT>
13
<EmitArg>
<ProductFlow FlowUnit ComponentType>
14
<Example> ::= <STR>
16
17 <Execution> ::= <DeviceType> <Capacity> *<Config>
18
<DeviceType> ::= Incubator Autoclave Centrifuge
[Known device
types]
<Capacity> ::= <REAL> <MEAS>
20 <Config> ::= <ArgKey> <ArgValue>
21
<ArgKey> ::= Duration Pace Power Quantity
[Known device
configuration items]
22
<ArgValue> <REAL> > <MEAS>
A9

---

Published as a conference paper at ICLR 2025
D.3 PRE-PROCESSING OF THE PROTOCOLS
The protocol pre-processing steps begin by reading all JSON files of the protocols. Each protocol then splitted centence-by-sentemee using Spacy" with the constraint that every sentence is longer than ten characters. Due to the large volume of data, sentence splitting is handled in parallel. Af- terwards, deeper sentence splitting is performed based specific conditions for further refinement, such as the presence of mmd/then/and then" followed by a vert We then parse sentences into root verbs and purpose clauses, which are identified using hallll/vvvbbaal//aa amoddalls for purpose clauses. Lastly, we merge phrases based on punctuation, and their classification into valid sentences or decorative phrases depends on whether they contain root verb or lack a purpose clause.
= "ROOT" for root verbs
and preposi
The first verb in each sentence is extracted as an opcode, again utilizing parallel processing for efficiency. Opcode frequency is filtered to exclude stopwords, which are recorded in separate text file. Then categorize these opcodes into high-level operation classes using GPT model (gpt-4o mini), where each opcode is classified into categories like Transfer Operations, Transformatin Operations, Data Operations.
Once operation classification is complete, entity recognition performed (also using gpt-4o mini) to identify entities like devices, input_ flowwwww____ wtw_wwittstttmttttttttttmmm pulput_flow____ juj ts, and lotal_time. Each flow unit is further categorized (also using gpt-4o mini) with a high-level classification composed of a phase, i.e., Gas, Liquid, Solid, etc.; and a type, i.e., Chemi Material, etc. When both phase and type are successfully labeled, phase is preferred as the feature of the flow unit. If phase labeling fails, nor type is successfully labeled, the corresponding feature is set to None. Part of the rationale is that there are non-reagent components in the general sense, i.e., data, files, obscure substances, etc. Therefore, we apply this strategy to maximize the possibility that there is a mean- ingful upper class labeling of the components without any redundancy.
Compound, Biological
type the feature of the flow unit. If neither phase
undef ine
Finally, we conduct synonym merge process on the devices, which starts by using transformers AutoTokenizerf to get an embedding for each device name. Afterwards, we use sklearr to identify potentially similar entity pairs by calculating the cosine similarity of the candidate entities, and then passing these entity pairs the GPT model for synonym detection, thereby merging devices belonging to the same type. The reference names of these combined devices will be one of the features.
D.4 PURE LLM-BASED BASED DESIGNER
The pure LLM- designer employs RAG to retrieve similar protocols from the corresponding corpora for representation, following the design choice of the baseline in DDDoogge et al. (2023). Specifically, in the FB approach, three similar protocols are first retrieved from the original proto- col corpora using RAG. and then, along with the title and description the target protocol, they are provided to the LLM to generate NL plan. The LLM subsequently translates the NL plan into Python pseudocode. In the IB approach, three similar protocols' instance actions (like Python pseudofunctions definitions) are first retrieved from the corpora, and after randomizing their order, they are provided to the LLM along with the title and description of the target protocol to generate plan in the form of Python pseudocode.
1 [Prompt for retrieving similar protocols from corpora]
2 You are expert in biology and you very familiar with the experiment protocols.
3 I would like to make a protocol for (title).
4 I will give you some related protocols in the database.
5 Could you find me the most three similar and relevant protocols for reference in the given range?
http:://aaaaylloaa//smetttnccee httpps//gaaay.l/////////ttccttet
AutoTokenizer
simi similarity.hhtnlkooiinn--eess
A10

---

Published as a conference paper at ICLR 2025
6
7 Please output your selected protocols, separating with comma.
't output any other information.
8 [Output format]
9 id_1, id_,,,,,, id_3
10
[Related protocols]
[context)
13
14 Answer:
1 [Prompt for generating NL plan]
2 Your goal is to generate steps for biology protocol.
3 These protocol steps must accurately describe complete scientific protocol obtain result.
4 Steps some similar protocols will provided as reference for you to generate the new one.
5 Output should only contain the steps without any other information.
6
7 Here is example of how generate steps for biology protocol.
9 EXAMPLE:
10
11 (example protocol title)
12
Here are some extra details about the protocol:
15
(example protocol description)
16
17
example steps:
(example protocol steps)
20
21
YOUR TASK:
22 Generate steps for protocol for (title).
23
Here are some extra details about the protocol:
25
26 [details]
27
28 Here similar protocols' steps for reference:
[steps]
31
32 your steps:
1. [Prompt for translating NL plan to pseudocode]
2 Your goal is to convert biology protocols into python pseudocode.
3
4 EXAMPLE
A11

---

Published as a conference paper at ICLR 2025
5 Here is example of how convert protocol for (example protocol title) into python pseudocode
6
7 (example protocol)
8
9 {example python pseudocode}
YOUR TASK:
12 Here is a biology protocol entitled '(title)' The protocol steps are as
follows:
13
14 [protocol]
16 Please convert this protocol into python pseudocode.
17
18 python pseudocode:
1 [Prompt for generating plan in pseudocode]
2 Your goal is to generate python pseudocode for biology protocols.
3
4 Here is an example of how generate pseudocode for biology protocol.
5
6 EXAMPLE:
7
8 (example protocol title)
Here are some extra details about the protocol:
11
12 (example protocol description)
13
example pseudocode:
16 (example pseudocode}
17
18
YOUR TASK
19 Generate pseudocode for protocol for (title).
21 Here are some extra details about the protocol:
22
23 (details)
24
You may only make use of the following python pseudocode functions:
27 (psuedofunctions)
28
29 your pseudocode:
D.5 INTERNAL DESIGNER
The internal designer incorporates the specific representation part of the prompt for an LLM, asking it to output the protocol while adhering the given representation constraints, echoing the
A12

---

selected from Published as a conference paper at ICLR 2025
idea of Waang al. (2023a). Specifically, in II, the instance actions retrieved from the corpora RAG and the pseudotunctions definitions the target protocol are shuffled and then provided together to the LLM, constraining it to generate plan in the form of Python pseudocode using the given pseudofunctions definitions. In EI and EI+, relevant DSL instructions a domain-specific operation-centir view DSL and croducc-flow-cemeee
view DSL, respectively. These instructions and the target protocol's title and description are provided to the LLM, prompting output the corresponding plan as instantiated DSL instructions.
[Protocol for generating plan in DSL program using operation -cntrric view
DSL]
2 Your goal is to generate plan in domain specific language (DSL) for biology protocols.
3 The DSL specifications related to the operations involved in the experiment provided. The DSL specification of each operation consists of multiple patterns, each pattern is an operation execution paradigm.
4
5 Here is an example of how to generate plan in DSL for biology protocol.
6
7 EXAMPLE:
9 (example protocol title)
10
11 Here are extra details about the protocol:
12
(example protocol description)
14
15
example plan in DSL:
16
17 (example plan)
19 [Requirements]
20 1. Design the experiment with finer granularity, incorporating more steps
to complete the experiment in more rigorous, complex, and comprehensive manner.
21 2. There generate each step
missing parameters the specification. You should
DSL program as detailed as possible based on your understanding of the protocol plan.
22 3. In Precond and Postcond, use formal name of the component to represent the SlotArg and EmitArg of each step. The component name should clearly describe the content of the component.
24 YOUR TASK:
25
Generate plan in DSL for a protocol for (title).
26
27
Here extra details about the protocol:
[details]
30
31 You can choose to instantiate the following DSL specification to construct the DSL program:
A13

---

Published as a conference paper at ICLR 2025
33 (DSL)
35 Your plan DSL program:
1 [Protocol for generating plan in DSL program using dual representation]
2 Your goal is to generate plan in domain specific language (DSL) for biology protocols.
3 Two perspectives of the DSL specification are provided: specification for experimental operations and the specification for experimental products.
4 The DSL specification of each operation product consists of multiple patterns, each pattern is operation execution paradigm product flow paradigm.
5 Output every operation of the plan in the form of an operation DSL program and every product of the plan in the form of a product DSL program.
6
7 Here is an example of how generate plan in DSL for biology protocol.
8
9 EXAMPLE:
10
11 [example protocol title}
Here are some extra details about the protocol:
14
15 [example protocol description}
16
17
example plan in DSL:
19 [example plan}
20
21
YOUR TASK:
22 Generate plan in DSL for protocol for (title).
Here are same extra details about the protocol:
25
26 (details)
27
28
You choose instantiate the following DSL specifications to construct the program:
29
30
Operation -vee DSL specification: (Dperation-DSL)
31
32
Product-viee view specification:
(Product-DSLL
35
36
Your plan in DSL program:
A14

---

Argkey: Argvalues Published as a conference paper at ICLR 2025
D.6 EXTERNAL DESIGNER
The external designer combines (i) deductive verification through DSL; and (ii) self-improvement by the L.M (Madaan al. 2023). EE, the external verifier is provided by the operation-cente view DSL and performs checks on two main aspects: (i) whether the precondition of cach operation an intermediate product of previous step rather than appearing from nowhere; and (ii) whether the postcondition of each operation used in subsequent steps rather than being omitted. Similarly, in EE+, the external verifier is provided by the DSL with a dual representation, focusing on cross- verifying the parallel dual tracks (the two perspectives of the DSL program). It checks whether the corresponding operation causes each status transition of the product: (i) whether the product in each product-view program is the output of its preceding operation: and (ii) whether the product each product-view program is the input for its succeeding operation. a mismatch occurs, the verifier generates corresponding еπоо messages, such as "Error: The product (product) required by operation [operation] step (i) is not available from previous steps." These error messages are then fed into the feedback-refine loop as feedback for the LLM to revise the plan. The loop terminates when the program passes the verification or reaches the maximum number of iterations, and the best result retained based the verification information.
1 [Prompt for refining the plan according to the feedback vertified by operation ccntric view
2 DSL]
3 Your task is to improve a biology experimental protocol plan represented in domain immin--speiiis language (DSL) based on provided feedback.
4 The input plan in DSL consists of multiple DSL programs, each representing one step in the experimental protocol planning process, arranged towndown order indicate the execution sequence operations.
5 Each DSL program has the following format:
6
7
"Operation":
1/ Operation verb Precondition for this step
"Precond":
9
"SlotArgNum": // Number arguments for the precondition ).
10
"SlotArg":
// Input product for this step
11
12
"Execution": {
"DeviceType":
Execution device for the operation dict execution arguments values
"Config":
15
16
17
}.
18
"Postcond": // Postcondition tion for this step "Emi CamitAAgNum"; Number of arguments for the postcondition // Output product for this step
20
"EmitArg":
21
}
22
23
The provided feedback indicates errors that occurred when compiling the DSL programs. You need to correct the program to ensure that the product is properly transferred between each step, i.e., the input product of each step must be the output from a previous step (except for the first step), and verify whether the output of each step is used the input for subsequent steps (except for the final step).
A15

---

Published as a conference paper at ICLR 2025
25 If you believe the error in particular step is due the step preparing reagents rather than using a previous intermediate product, you can ignore this error.
26
27 Output your refined plan in DSL, returning a JSON block without any additional information comments.
YOUR TASK:
30
Refine the plan in DSL for a protocol for (title).
31
32
Here are some extra details about the protocol:
33
(details)
35
36
Refine the following plan:
37
38 (plan)
40
Here is the feedback the plan:
41
42 (feedback)
43
Your refined plan DSL:
1 [Prompt for refining the plan according to the feedback vertified by DSL
with dual representation]
2 Your task is to improve Biology experimental protocol plan represented domain-span specific language (DSL) based on provided feedback. 3 The input plan DSL consists of multiple DSL programs from two perspectives: operation-view and product-view. The DSL programs from these two perspectives alternate and constrain each other.
4
5 This is the format of product-view DSL program:
// Each product view program represents state of the product at that moment.
7
8 Pred: <Operation>,
// Pred represents the operation that precedes the creation of this product, need to align to the operation name in the operation view DSL program. If the product is in its initial state, return ""
9 FlowUnit: { // Flowlnii defines the properties of the product
being processed.
10
// Component represents the actual product material being processed, need be the formal name of the component
Component:
11
ComponentType: Gas Liquid Solid Semi Siliud /iaiiiaacooooondiioooooioaallll // ComponentType describes the type of the component, which be of the following: Gas, Liquid, Solid, Semi-Solid Mixture,
Reagent Physicalljeect File//Data,
A16

---

Published as a conference paper at ICLR 2025
ChemicalCompound, BiologicalMaterial, Reagent, Physicalobject,
File/Data.
12
// RefName is the reference name used to uniquely identify this component, need to align to the operation-vies program UnitArgType: MAT | PROD, // UnitArgType specifies whether this is material (MAT) or product (PROD).
RefName:
13
Vol:
// Vol represents the volume or quantity of the
component.
15
// Container indicates the type of container or storage used for this component. If the product has no container constraints in its current state, return
Container:
16
// Cond defines the specific condi tinditions under which the operation carried out, which expressed as key-value pairs.
Cond:
17
ArgKey: ArgValues
18
19
).
Succ represents the operation that follows the creation of this product. If the product is in its final state, return ""
Succ: <Operation>
21 }
22
This is the format of operation-view program: 23
// Each operation view DSL program represents sequence operations that alters the state of the product.
25
26
Operation:
// Operation verb
27
Precond: { // Precondition
SlotArgNum: // Number arguments for the precondition // SlotArg represents the input product or material required for this operation, using formal component names from the product perspective DSL program, with serial numbers to distinguish repeated components in different states.
29
SlotArg:
30
).
Execution:
32
DeviceType: // Execution device for the operation
33
Config: {
// dict of execution a arguments - values ArgKey: ArgValues
34
35
}.
37
Postcond: {
// Postcondition
38
// Number of arguments for the postcondition // EmitArg represents the output product or material resulting from the operation, using formal component names from the product perspective DSL program, with serial numbers distinguish repeated components different states.
EmitArgNum:
39
EmitArg:
40
}
41
42
A17

---

Published as a conference paper at ICLR 2025
43 The provided feedback indicates errors that occurred when compiling the programs. You need correct program ensure that state changes of each product's RefName in the Product-view are caused by the corresponding operations in the Operation-view 44 If you believe the error in a particular step is due to mismatch in product names between the two perspectives rather than actual error, you can ignore this error.
46 Output your refined plan in DSL, returning a JSON block without any additional information comments.
47
48 YOUR TASK:
Refine the plan in DSL for protocol for (title).
50
51 Here are some extra details about the protocol:
52
53 (details)
Refine the following plan:
56
57 (plan)
58
Here is the feedback of the plan:
61 (feedback)
62
63 Your refined plan in DSL
D.7 COMPUTING LOAD OF THE MACHINE DESIGNERS
For automated representation generation, we primarily used GPT-4o mini with OpenAl's Batch 5 for preprocessing, incurring cost of approximately $60 across four domains. The design executed
of the DSLs convergence. This process required an average of 55 seconds per iteration for the operation-centric view DSL and an average of 2 seconds per iteration for the product-centric view DSL. For the machine designer, we primarily utilized GPT-4o mini combined with RAG for design, with a total cost of approximately $10 (7 methods, 140 protocols). summary, the overall computational load is relatively low, highlighting the accessibility of our machine designers when utilizing the proposed representations and the corresponding automatic representation generation modules.
MacBook with M2 chip, running 1,000 iterations ensure
E
DATA COLLECTION
E CORPORA SOURCES
The corpora for the automatic generation of representations SScc. 3.1 and the corpora for selecting the testing set (Sec. 4.1) are both retrieved from open-sourced websites run by top-tier publishers, including Nature's Protocolexchange Cell's Star-protocols" Bio-protoco [ Wiley's Current Pro-
httas://protooolleccanag
researchsquare. httpr//cceccccceeeeeeeeeeeecarae
httpss//star-pprotcoolss..l/
https://bio-ppotecol.rr/// com/
com/
A18

---

Published as a conference paper at ICLR 2025
tocol=" and Jove These sources compile dataset of 15,837 experimental protocols across four domains: Geneties (8794 protocols), Medical (7351), Ecology (812), and Bioengineering (3597), with minimal overlap between them. We aggregated the corpora and analyzed the themes of the protocols according the first- and second-level labels attached to them. We adopt measures ensure that C is mutually exclusive with the testing set.
Other domains, such as Physics and Chemistry, are also representative domains of experimental sciences, besides Biology, Medical, and Ecology. 'he preliminary factor that restricts our current scope is data accessibility. Due to the higher cost of accessing the corpora of protocols for conduct- ing physics and chemistry experiments, for example, mining the protocol from the "method" section of relevant published papers, we leave the application Physics and Chemistry for future work.
E.2 ELIMINATING THE RISK DATA LEAKING
We employ the broadly accepted standard operating pro- cess empirically verify that LLMs have not memorized the data we use. We adopt the methodology outlined in Section 5.2 of Skywork (Wei al. 2023) and draw upon recent studies on detecting memorization in LLMs (Car- lini et al. 2021 2022). Specifically, we use gpt-4o mini to synthesize data resembling the style of steps from novel protocols, and then calculate the perplexity on the test set and reference set. Since the reference set is newly gener- ated, consider it clean, not belonging to any training set of any model.
1.6
perplexety p
1.4
We randomly sample 100 sequences each from the test and the reference set of the novel protocols. Each se- quence corresponds to a single procedural step described in NL. We truncate the final 50 tokens of each sequence, retaining the prefixes. These prefixes are then used as prompts for the LLM to predict the next 50 tokens, for which we calculate the perplexity. If the perplexity of the test set is significantly lower than that of the reference set, the test set might have appeared in the model's training phase.
1.0
test
reference
Figure A1: Comparison between the perplexity of the test set and the ref- erence set
The results indicate that the LLM's average perplexity on the test set is significantly higher than that on the reference set (t(198) 3.040, p4a < 0.p .05; see Fig. Alp, suggesting that the LLM encounters greater uncertainty with the novel protocols in the test set. This finding implies that for published, widely accepted, and standardized operating process, there is no evidence to suggest that the LLM has memorized the data.
E.3
ON THE DIVERSITY OF NOVEL PROTOCOLS
Assessing diversity among novel protocols is both informative and meaningful. To further support our analysis, we incorporate t-SNE visualization of the experimental objectives (described in nat- ural language) for the novel protocols we select, shown Fig. A2] The results demonstrate well-dispersed distribution, indicating a sufficient level of diversity among the protocols.
E.4 SHOWCASES
1 [Protocol Bioengineering]
2 Preparation of lysates
3 1. Harvest approximately x 10*7 cells by centrifugation at 2000 RPM for min. Aspirate media and resuspend cell pellet with mL of ice- cold PBS and transfer to a mL centrifuge tube. Microcentrifuge at 2000 RPM for min at 4 PC
hhttss://currentprotooce onl inel weeellibrry..o y wi wwley.com com/ https://www.jover com/
A19

---

Published as a conference paper at ICLR 2025
Genetios
Medical
Eoology
BioEng
whusktrr paaiing
wurt planingg 9g
walokmert poanengg
40
25
15
-8
-16
15
-50
-40
-16
-8
16 -20
-10
10
20
-30
-15
15
30
-30
-15
30
Figure A2: Visualization of diversity between novel protocols
4 2. Aspirate PBS, and then add Hypotonic Buffer (supplemented with 1% Triton X-100, to disrupt membrane and cytoskeleton - bound MEKK1 fractions).
5 3. Cell lysates are homogenized by passing through 22 22-gauge needles, and tubes are put on ice for 15 min to complete the lysis. Crude extracts are then centrifuged at 2500 RPM for 5 min. Supernatants are transferred fresh centrifuge tubes, and cold 5 NaCl is added to each sample make salt concentration of between 0.7-1.0 M disrupt protein-protein interactions.
6 4. Spin the crude extracts by ultracentrifugattion at 55000 RPM to properly pellet residual insoluble proteins from the extract. Transfer supernatants into fresh centrifuge tubes.
7 immunoprecipitation
5. Rinse Protein beads Hypotonic Buffer and place on ice until ready for use.
9 6. Take a volume of cell lysates (prepared as described above), and dilute with Hypotonic Buffer to 250-500 mM salt to enable protein- protein interactions.
7. Add ng of preclearing antibody to the diluted lysate (e.g., anti Myc μLl of Protein beads, and rock for 45
or anti -p5), vortex, min.
11 8. Touchspin samples, and transfer supernatant to a fresh tube.
12 9. Add ng of polyclonal anti-MEKK1 to the lysates, and rock for 1 h. After this period, add nu of Protein beads and rock tubes at for h.
13 10. Touchspin beads, wash beads with hypotonic buffer (supplemented with NaCl to concentration of 300 mM), vortex, and rock for 10 min. In total, 3-5 washes of the beads are performed.
14 11. Finally, wash Assay Buffer. Purified MEKK1 may be stored by snap-freezing in liquid nitrogen and long-term storage at -80 "c. Kinase assay Following preparation of MEKK1 immunoprecipitater of JNKK1 (K131M) along with 5 pci of ATP in Kinase Assay Buffer for 30 min 30 "C."
with Hypotonic Buffer, and resuspend in Kinase
(as above), incubate with
[Protocol Genetics]
17 1. Note that everything is in DEPC water. Inoculate W303a cells expressing different TOR1 variants in 2 mL SC medium overnight.
18 2. Subculture the cells starting from 0D600=0.1 1 in 10 mL SC media, , shake vigorously 30 "c 300 RPM for around 4-6 until 0D600=0.2 4.0.5.
A20

---

Published as a conference paper at ICLR 2025
19 3. Collect the cells spinning down without freezing ice. Discard supernatant.
20 4. Re-suspend cells with mL water and transfer to a 1.5 eppendorf tube, quickly spin down at 0,000 g for 15 sec.
21 5. Re-suspend cell pellet in 400 μL of AE buffer at room temperature.
22 6. Add 40 μN 10% SDS (final around 1%) and vortex briefly at room temperature (RT).
7. Immediately add 5e0 μL hot phenol/AE /A (put
" for 10 min before use), vortex vigorously for min. Incubate at 65 " for 5 min. Briefly vortex every 30 sec.
24 8. Immediately freeze by dumping into liquid nitrogen. Wait to thaw at RT (put in 30 " to thaw may crack the tube).
9. Centrifuge for min on a standard laboratory microfuge 20,000 x gg
at RT.
26 10. Transfer around 400 µ supernatant to new eppendorf tube. Recycle the lower phenol fraction carefully following the chemical safety protocol in your laboratory.
11. Add equal volume (400 μL) phenol: CHC13/AE Vortex vigorously for min at RT.
28
12. Spin down at 20,000 x for 5 min in standard laboratory microfuge.
29 13. Transfer supernatant (around 350 pL) to fresh 1.5 mL eppendorf tube Add CHC13: isoamyl alcohol (24:1). Vortex vigorously for min
RT
30 14. Transfer aqueous supernatant to a fresh 1.5 microfuge tube. white cloudy precipitate is observed between the aqueous phase and organic phase, repeat steps 17-18.
31
15. Add 1/10 volume of м NaDAc (pH 5) and vortex vigorously. Add 2.5 volumes of ethanol. Vortex again.
16. Place -20 "C for at least min.
33 17. Spin down in the microfuge at 20,000 x g, 15 min at cc RNA pellet
is usually visible.
34 18. Add ice-cold 75% EtoH, place at 4 o for around 10 min. Vortex and spin down on microfuge 20, 000 g, 15 min at 4 "C Discard
supernatant. Suck the liquid droplets in the tube. The white RNA pellet will turn clear when dries out. 30-50 μL ddH20 (DEPC) immediately after it becomes clear. Do not let the RNA over-dry, which will make it difficult to dissolve. If RNA pellet is over-dry, dissolve RNA at 37 "C for 30 min. Store RNAS at -80 "C for more than 2 months."
[Protocol Medical]
37 1. Passage through a 45 pm filter. Add 100 pL/well of 100 pg/mL mL salmon
sperm DNA to 96-well Microtest assay plate.
38 2. Wrap the plate with plastic wrap and incubate at "C overnight.
3. Discard the coating antibody solution and wash the plate with 1x PBS- Tween times.
40 4. Dry the plate and add 100 ul of blocking solution per well to the plate.
41 5. Incubate the plate at room temperature (RT) for 1.5 h.
A21

---

Published as a conference paper at ICLR 2025
42 6. Discard the blocking solution and wash the plate with 1x PBS Tween 5
times.
43 7. Dry the plate and keep it at 4 for later use.
44 8. Harvest the spleen and create a single-cell suspension by gently smashing spleen pieces with the frosted surface of pair of microscope slides in 5 mL of DMEM.
9. Transfer the cells into 50 mL conical tubes and spin down the cells 4 Pc.
300 RCF for
46 10. Discard the supernatant with aspiration without disturbing the pellet
47 11. Re Re-suspend the cells with mL of 0.17 M ammonium chloride and keep the cells ice for min.
for min at "c 49 13. Discard the supernatant and re-suspend the cells with 20 mL of DMEM and count the cells.
12. Add 15 mL DMEM to the cells and spin
50 14. Re-suspend serial dilution total of dilutions) with 10% DMEM.
10^7 cells in 2 mL of 10% DMEM and make three-fold
15. Add 50 pL/well of the serial dilutions on the DNA coatcd plate centrifuge at 300 RCF for 5 min at c.
52 16. Incubate the cells at 30 " for h in a cell-culture incubator with 6% CO2.
53
17. Add 50 pL/well of
anti - IgM anti - Igg (1:350 in
DMEM) to the cells.
18. Centrifuge the cells 300 RCF for 5 min at c and incubate the cells overnight in a cell-culture incubator with 6% CO2.
55
19. Discard the cells and wash the plates 10 times with 10x PBS-Tweed 20.
56
20. Dry the plates and add 50 μL of streptavidin alkaline phosphatase
(1:1,000 in BSA/PBS) the plate.
21. Incubate the plate at PwS-Tween 20.
and wash the plate times with 10x
58 22. Dry the plate and add 50 pL/well of mg/mL mL BCIP in AMP buffer to develop the plate.
59 23. When the spots are clearly visible under dissecting microscope, stop the development by discarding the BCIP solution and rinsing the plate with tap water thoroughly.
60 24. Spots can be counted using a dissecting microscope or using an ELISpot reader."
F REPRODUCIBILITY
The project page with supplementary files for reproducing the results of this paper will be available at
G LIMITATIONS
As representation designed for a relatively new problem, the design and evaluation of the proposed framework come with limitations, leading further investigations:
Overall, our method achieves promising results across the four domains. Specifically, it performs best in experimental design for Genetics, shows comparable effectiveness in Medical and Bio- engineering. but is less effective in Ecology. Notably, the Genetics corpus is the largest among
A22

---

Published as a conference paper at ICLR 2025
the four domains, while the Ecology corpus is significantly smaller than the others. These ob- servations suggest a potential positive correlation between the size and the "quality" of the resulting DSL. In other words, a larger corpus may lead to "better" representation, thereby influencing the outcomes of protocol design. This hypothesis necessitates further investigation through rigorously designed experiments and carefully defined metrics for evaluating what constitutes "better" representation.
domain-specific corpus
We majorly consider the imperative programminng DSLs as the implementation of representation in this work. This raises the question of whether incorporating objective-oriented programming paradigms could enhance the representation of complex entities within protocols, particularly the properties of reagents and intermediate products. If we are able make the DSLs model the fine- grained reactions between different components and automate the design of those DSLs based on broader source of data, such the Wikipedia pages, we ultimately manage build up symbolic digital twin for domainnsppcifff system, such as the cell cultivation environment. Such simulation systems may greatly benefit protocol design with their power of prediction, explana- tion, and counterfactual analysis.
Can we explicitly extend our proposed representation to a hierarchical graph, thereby establishing the foundation for employing the advanced algorithms graph routing and graph optimization? Results on the hierarchical graph can also serve as external heuristic and constraint for L.LM- based protocol designers. This hybrid approach may combine both the advantages of LLMs, i.e., exploitation of background knowledge, and those of classical algorithms, i.e., white-boxed properties with high explainability.
Can we apply the representation and the automatic representation generator to other critical do- mains with high demand for automating procedure design, such as designing product sheets for advanced manufacturing?
With many questions unanswered, we hope to explore more automated protocol design for self- driving laboratories and beyond.
H THE AUTOMATICALLY GENERATED REPRESENTATIONS
H.1
OPERATION-CENTRIC VIEW DSL
1
2
"Operation": "Precipitate",
"pattern_0": {
4
"Precond":
5
"SlotArgNum": 2, "SlotArg": ["Liquid", "Solid"]
6
7
}.
8
"Execution": {
9
"DeviceType": "falcon tube", "Capacity"; "15 mL",
10
11
"Config": O
12
}.
13
"Postcond":
"Emi tArgNum 1,
15
"EmitArg": ["Liquid"]
16
}.
17
"Example":
18
"Precipitate RNA adding 600 μL of 100 EtOH, 20 nLl of 3 NaOAc PH 5.5 ), μl of glycogen .",
"Precipitate the DNA each tube by adding pll of sodium acetate ( pH 5.2 ) and 550 al of 100 & ethanol 20
"Ethanol precipitate the RNA by adding 5 ul 3 M sodium
acetate ( pH 5.2 ) .2 ul of glycogen 20 mg / ml and 171 a1 of 180 ethanol
A23

---

Published as a conference paper at ICLR 2025
21
1
},
23
"pattern_1"; {
24
"Precond":
25
"SlotArgNum": 4,
26
"SlotArg": ["Liquid", "Liquid", "Liquid", "Liquid"]
27
3.
"Execution": {
29
"DeviceType": "centrifuge", "capacity"; "1.5 m1",
30
31
"Config":
32
"time": "10-15 min", "speed": [*12,000 g", "20,000 g"], "temperature": "4 'c"
34
35
}
36
}.
37
"Postcond":
"EmittrgNumm: 1,
39
"Emitarg": ["Solid"]
40
}.
41
"xxaplees: [
42
"Precipitate the cell debris in the lysate by centrifugation C.",
20. 800 for 10-15 min precipitate DNA with 13.5 pL of following mixture (1 pl of ethanol."
20 mg / ml Glycogen, 12.5 pL of M NaDAc [pH 5.3]) and 340 μl
44
"precipitate the total RNA by centrifuging at 12, 000 00,000
for
15 min at C."
]
46
}
47 },
48
49
"Operation": "Spin",
50
"pattern_0": {
"Precond":
52
"SlotArgNum": 2,
53
"SlotArg": ["Liquid", "Liquid")
54
).
55
"Execution": (
"DeviceType": "spin plate",
"Config": "time": ["1 min"]
58
59
60
),
61
"Postcond": {
"Emi tArgNum 1,
63
"Emitarg": ["Physical Object
64
}.
65
"Example":
A24

---

Published as a conference paper at ICLR 2025
66
"Nuclei washing and tagmentation: Spin down nuclei 680 g mins C resuspended with 50 μL Complete Buffer.", "Spin the sample at 4,000 x g at until the volume reduces to about mL. Quantify protein concentration as described in step 60.",
67
68
"Spin down the 15 mL tubes at 2,500 and 4 C for 20 min
"Spin for min at 1,000 g. Save a few μL concentrated sample to run on an agarose gel later." 70
"Spin the tube for 30 sec at 12, 000 x to consolidate the gel at the bottom of the tube.",
71
"Spin plate at 300xg for min to collect liquid at the bottom of the wells.",
72
"Once NXT PCR program is complete, quick spin the sample tube then place it on the magnet for 1 min. Transfer the supernatant containing the amplified mRNA-seq library into new PCR tube.", "Spin down mill nuclei 600xg for 5 min (whole liver nuclei) or use magnet (bead-bount nuclei)."
73
74
]
75
}.
76
"pattern_1"; {
77
"Precond":
"SlotArgNum"; 1,
"SlotArg": ["Mixture"]
80
),
81
"Execution"; { "DeviceType": "microcentrifuge",
82
83
"Config":
}.
85
"Postcond": O,
86
"Example":
87
"Small volumes, 1-3 mL should be spun in small tube where these fewer EVPs can more readily be collected."'
"Briefly spin down the bead-lysate mixture.", "Spin down the mix tube to eliminate bubbles/air /air in bench microcentrifuge. Add 19 μL of the mix to each well."
90
91
}.
92
"pattern_2": {
"Precond":
"SlotArgNum"; 1,
95
"SlotArg": ["Liquid"]
96
).
97
"Execution": {
"DeviceType": "centrifuge",
"Config":
100
"speed": ["800 g g"],
101
"time": ["7 min")
102
103
A25

---

Published as a conference paper at ICLR 2025
104
"Postcond":
"Emi MaritArnuu",, 1, "EmitArg": ["Liquid"]
106
107
}.
108
"Example": "Spin lysate 14 krof for 10 min at C; transfer cleared lysate new tube.",
109
110
"Spin down the beads for s at 2, 000 x g. Discard the supernatant by carefully pipetting out the buffer.",
111
"Spin at 12,000 x until the total volume in both filters is reduced to 120 μL (<=30 min). Keep aside 5 ul of purified labeled histone for SDS-PAGE analysis.".
112
"Quickly spin the FACS tube to allow the cell suspension to pass through the filter to remove undigested large tissue debris.", 113
"Spin once for 7 min at 800 Use the BD cytofix/cytoperm kit according to the manufacturer's 's instructions and thereafter add antibodies for intracellular detection of IFN and TNF."
115
}
116 },
117
118
"Operation": "Sonicate",
119
"pattern_@": {
"Precond":
121
"SlotArgNum": 1,
122
"Slotarg": ["Liquid"]
123
}.
124
"Execution": {
"DeviceType": "sonicator",
126
"Config": { "time": ["20 - 30 s"]
127
128
129
}.
130
"Postcond":
"EmitArguum",, 1,
132
"EmitArg": Seme Solid"]
133
}.
134
"Example":
135
"Sniicate the pellet suspension ice under 50 % duty cycle for min.".
136
"Agaros gel sonicated Arabidopsis chromatin.",
137
"Sonicate proteoliposomes for 28 30 s or 3 times for 10 placing ice in between sonication, if necessary.",
138
"The lipid suspension is sonicated to form small unilamellar vesicles (SUVs)."
]
140
},
141
"pattern_1": { "Precond": "SlotArgNum": 2,
142
143
A26

---

Published as a conference paper at ICLR 2025
144
"SlotArg": ["Liquid", "Solid"]
}.
146
"Execution": [
147
148
"DeviceType": "bransonic",
149
"Config":
150
"temperature": ["60 c"],
"time": ["90 min"]
152
}
153
}.
154
{
155
"DeviceType": "sonicator", "Config": O)
157
}
158
].
159
"Postcond": (},
160
"Example":
"Sonicate ng BAC or 50 bg genomic total (you will recover 10 DNA after sonication and size selection).", "Sonicated chromatin is immunoprecipitated with the chosen antibodies and non-enriched chromatin washed with series of washing buffers.",
162
"If the herring sperm DNA has not been sufficiently sonicated or too much has been used, the DNA pellet might not adhere to the microfuge tube and can be lost with the ethanol.",
164
"Sonicate the lipid tube to dissolve lipids with the mineral oil for 90 min at 60 с by using Bransonic."
165
]
}
167
H.2
PRODUCT-FLOW-CENT VIEW DSL
1
2
"Pred": "Modification Operations",
3
"FlowUnit": (
4
"Component": "FBS", "ComponentType"" "Liquid",
5
6
"UnitArgType": "MAT"
7
"Vol": ["0.1 mL", "0.5 mL", mL", "1.5 mL "2 mL
"3 mL "5
mL
"10 mL", "25 mL", "50 mL", "400 pL", "500 uL", 500 mL "J. "Container": "Tube",
8
9
"Cond": (
10
"Concentration": ["0.5x", "1x" "2X" 5%" "5%" "10%" "15%" "20%" "30%", "50x", "90%", "100x"], "Temperature": ["-150"c", c4" "1"" "3 "5
"],
12
"State": "heat- -neatiinactivated"
13
}
14
"Suce": "Transfer Operations"
A27

---

Published as a conference paper at ICLR 2025
15 }
17
"Pred"; "Detection and Measurement Operations", "FlowUnit": {
18
19
"Component": "ethidium bromide",
20
"ComponentType": "Solid",
21
"UnitArgType": "MAT",
"Vol": ["0.25 pL/mL", "0.5 pL/mL", "2 ,L" "10 nu", "15 ,L", "10
#g/mL", "0.5 pJ/
23
"Container"; "Flask",
24
"Cond": { "Concentration": ["0.00243", "0.3 /g/mL", "1.5% (w/v)", 11:1000"],
25
μM"
26
"Temperature": ["25"c", "room temperature"),
27
"State": ["toxic", "carcinogenic"", "Charge": ["positively charged"]
28
29
}.
31
"Succ": "Modification Operations"
32 },
33
34
"Pred": "Transfer Operations", "FlowUnit":
35
"Component": "gel",
37
"ComponentType": "Semi Solidd,
38
"UnitArgType": "MAT",
39
"Vol": ["0.5 mL"
40
"Container": ["Gel Cassette", "Tank", "Tube "], "Cond": (
42
"Impedance"; ["under 20 kOhm"],
43
"Size": ["50-250 nt"]
44
45
}.
46
"Suce": "Transfer Operations"
}
I
CASE STUDIES
I.1
CASE STUDY: CONTRIBUTIONS THE BUILDING BLOCKS
Part of protocol designed by EE+:
1
2
"Pred":
3
"FlowUnit":
"Component"; "Lysis solution", "ComponentType": "Liquid",
5
6
"RefName": "Lysis_solutioon- -1",
7
"UnitArgType": "MAT",
8
"Vol"; "50 nL",
9
"Container":
A28

---

Published as a conference paper at ICLR 2025
10
"Cond":
"State": "Liquid"
12
}
13
},
14
"Suce": "Pipette"
15
18
"Operation": "Pipette",
19
"Precond":
20
"SlotArgNum": 1,
21
"SlotArg":
"Lysis_solution-1"
23
]
24
},
25
"Execution": {
26
"DeviceType": "Pipette",
"Config":
"time": "10 times",
29
"volume": "50 nl"
30
31
}.
"Postcond":
"Emi tarittArNumm. 1,
34
"EmitArg":
35
"Lysis_solution-2
36
]
37
Part of protocol designed by EE:
1
2
"Operation": "Add",
3
"Precond":
4
"SlotArgNum"; 1,
5
"SlotArg":
6
"Triton-X"
7
]
8
).
9
"Execution"; {
10
"DeviceType"; "8-channel pipette",
11
"Config": {
12
"Volume": "1% solution"
13
}.
"Postcond": {
16
"EmitArgNumm 1,
17
"EmitArg":
18
"Triton_X_Solution
]
A29

---

Published as a conference paper at ICLR 2025
20
22
23
24
"Operation": "Run",
25
"Precond":
26
"SlotArgNum"; 1,
"SlotArg":
28
Mixttue
29
1
30
}.
31
"Execution": {
"DeviceType": "Thermal Cycler",
33
"Config":
34
"Temperature": "70 'c",
35
"Time": "15 min"
36
},
"Postcond": {
39
"EmitArgNum": 1,
40
"EmitArg":
41
]
}
44
}
Part of protocol designed by EI:
1
2
"Operation": "Run",
3
"Precond":
4
"SlotArgNum": 1,
5
"SlotArg":
6
CCsll_Lysis.
7
]
8
).
"Execution": {
"DeviceType"; "Thermal Cycler",
11
"Config": (
12
"Temperature": "70 "c", "Time": "15 min"
13
14
}
}.
16
"Postcond": (
17
"EmitArrNum":: 1,
18
"EmitArg".
19
"cDNA_Reverse_ TrenssrpsrTrasscipptiooo"
21
}
22
A30

---

Published as a conference paper at ICLR 2025
Part of protocol designed by II:
1
seveerse_rrnnscrbee"
2
"muscs": "cells",
3 "buffer": "RT buffer",
4 "enzyme": "reverse transcriptase",
5
"incubation_time". "60 minutes",
6
"temperature": "42"
7
9 "prepare_single_cer licpingleeeee_suseeee_ee sespssssssesssiiio
10
"input_cells": "lysed cells"
11
Part of protocol designed by IB:
deverse_transcre
2 "muscs": "RNA",
3 "buffer"; "reverse transcription buffer",
4 "enzyme": "reverse transcriptase",
5 "incubation_time". "60 minutes",
"temperature": "42"
7 }
8
9 "preepresiingll..ellsssssssss
10
"input_cells": "single -eell samples"
Part of protocol designed by FB:
1 "sort_single__ell" {
2 "plate": "PCR plate",
3 "nozzle_size": "100 μm",
4 "mode": "single- -elll purity"
5 }
1.2
CASE STUDY: HANNLLNG DIFFERENT TASK COMPLEXITIES
Part of protocol designed in Planning:
1
2
"Operation": "Obtain",
3
"Precond":
4
"SlotArgNum"; 1,
5
"SlotArg": "File/Data"
6
7
1
}.
9
"Execution": {
10
"DeviceType":: "QIAGEN Blood & Cell Culture DNA Maxi Kit",
11
"Config": O
12
}.
"Postcond";
A31

---

Published as a conference paper at ICLR 2025
14
"Emi tangttur 1,
"Emi tArg":
16
"HMW genomic DNA "
17
1
18
}
19
Part of protocol designed in Modification:
1
2
"Operation";
(Centrifuge"
3
"Precond":
4
"SlotArgNum", 1,
5
"SlotArg":
6
7
]
},
9
"Execution"; {
10
"DeviceType":
"Config": "ULtracentrifuge",
11
12
"speed": [
102 x g""
] "time" : [
15
16
20 min"
17
1.
"tmpperature" C
20
21
}
22
}.
23
"Postcond":
"Emi 1,
25
"EmitArg":
26
"Pellet-1
27
]
28
Part of protocol designed in Adjustment:
1
2
"Operation";
"Incubate",
3
"Precond":
4
"SlotArgNum": 2,
5
"SlotArg":
"Washed sections with 1st antibody -1",
7
"2nd antibody mixture-1"
8
1
9
),
10
"Execution"; {
11
"DeviceType": "Moistening box".
A32

---

Published as a conference paper at ICLR 2025
12
"Config":
"temperature": "37 c",
14
"time": "1 h"
15
}
16
},
17
"Postcond": {
"EmitArgNum": 1,
"EmitArg":
20
"Sections with 2nd antibody -1"
21
]
22
}
23
1.3
CASE STUDY: GENERALITY ACROSS DOMAINS
Part of protocol designed for Bioengineering:
1
2
3
"Operation": "Incubate",
4
"Precond": {
5
"SlotArgNum": 2,
6
"SlotArg": [
7
"Lysis Mixture-2",
"Stop Buffer-1"
9
]
10
).
11
"Execution": { "DeviceType": "Thermocycler", "Config": "temperature": "65 "c" "time": "30 min"
12
15
16
}
17
).
"Postcond": {
"Emi tArgNum 1,
20
"EmitArg": [ "Neutralized Mixture-1"
21
22
]
23
)
25
{
26
"Pred": "Incubate",
27
"FlowUnit": {
28
"Component": "Neutralized Mixture", "ComponentType": "Mixture", "RefName": "Neutralized Mixture-1", "UnitArgType": "PROD", "Vol": "60 μL",
31
32
33
"Container"; "0.2 ml PCR tube", "Cond":
34
A33

---

Published as a conference paper at ICLR 2025
35
"State": "Neutralized"
}
37
}.
38
"Succ: "Mix"
39
).
40
41
"Operation": "Elute",
"Precond":
43
"SlotArgNum": 1,
44
"SlotArg": [
45
"Washed Beads-1"
46
]
)
48
"xxecution: {
49
"DeviceType": "Centrifuge",
50
"Config": : "time": "1 min"
61
}
).
54
"Postcond": [
55
"Emi tArgturruu""" 1,
56
"EmitArg": [
"Eluted Product-1"
]
59
}
60
},
61
62
"Pred": "Elute",
"FlowUnit": {
64
"Component"; "Eluted Product", "ComponentType": "BiologicalMateriial,,
65
66
"RefName": "Eluted Product-1",
67
"UnitArge "UnitArgType": "PROD", "vol": "50 nu",,
68
"Container"; "0.2-ml PCR tube", "Cond": {
70
71
"State": "Eluted"
72
73
Succ": "Incubate Clear"
75
}.
76
(
77
"Operation": "Quantify",
78
"Precond":
"SlotArgNum": 1,
"SlotArg":
81
"Clear Eluted Solution-1 11"
82
]
83
).
84
"Execution": {
A34

---

Published as a conference paper at ICLR 2025
85
"DeviceType": "Nanodrop",
"Config": O
87
}.
88
"Postcond": {
89
"Emi "EmittrgNum" 1,
90
"Emittrg": "Quantified Sample-1"
]
93
}
94
},
95
96
"Pred": "Quantify",
"FlowUnit": {
98
"Component": "Quantified Sample",
99
"ComponentType": "Liquid",
100
"RefName": "Quantified Sample-1", "UnitArgType": "PROD", "Vol": "50 nl",
101
103
"Container"; "0.2-ml PCR tube", Cond": {
104
105
"State": "Quantified", "Concentration"; "150 ng/uL", "A260/A280"; 1.85,
106
"A260/A230": 2.1
109
}
110
}.
111
"Succ": "Dilute"
112
}
Part of protocol designed for Ecology:
1
2
3
"Operation": "Grow",
4
"Precond":
5
"SlotArgNum"; 1,
6
"SlotArg": [
7
8
]
9
)
10
"Execution"; {
11
"DeviceType": "Environmental growth chamber",
12
"Config": "Temperature": "24 "c",
13
"LightCycle"; "12h light/12h dark"
}
16
).
17
"Postcond": { "EmitArgNuu 1,
18
"EmitArg":
A35

---

Published as a conference paper at ICLR 2025
20
"Mature rice plants"
]
22
}
23
},
24
25
"Pred": "Grow",
26
"FlowUnit": {
"Component": "Mature rice plants",
28
"ComponentType": "BiologicalMaterial,,
29
"RefName": "Mature_Rice_ Plants--"
30
"UnitArgie "UnitArgType": "PROD",
31
"Vol": "N/A",
"Container": "Plastic pot",
33
"Cond": {
34
"State": "Mature", "Height": "50-60 cm"
35
36
}
}.
"Succ": "Anesthetize"
39
}.
40
41
"Operation": "Collect",
"Precond":
"slotArgNum": 2,
44
"SlotArg": [
45
"Monitored_Aphid
46
"Mature_Rice_Plantss
47
]
).
49
"Execution"; {
50
"DeviceType": "Microcapillary tube",
51
"Config": ()
52
).
53
"Postcond":
"EmitArgNum" : 1,
55
"EmitArg":
56
"Phloem sap"
57
]
58
}.
{
61
"Pred": "Collect",
62
"FlowUnit": {
63
"Component"; "Phloem sap",
64
"ComponentType": "Liquid", "RefName": "Phloem_Sap-1", "UnitArgType": "PROD", "Vol": "1-2 μL",
66
67
68
"Container"; "Microcapillary tube",
69
"Cond":
A36

---

Published as a conference paper at ICLR 2025
70
"State": "Collected", "Appearance": "Clear, slightly viscous"
72
}
73
).
74
"Suc": "Dilute"
75
}
"Operation": "Centrifuge",
78
"Precond":
79
"SlotArgNum": 1,
80
"SlotArg": [
81
"Dilutedd_hl__app11
]
83
84
Execution": {
85
"DeviceType":
"Config": "Centrifuge",
86
"speed": "6000 rpm", "temperature": "4 'c", "time": "10 min"
89
90
}
91
).
"Postcond": {
"Emi taritturNum"" 1,
94
"EmitArg": [ "Extracellular vesicles" ]
95
96
97
},
99
{
100
"Pred": "Centrifuge", "FlowUnit": {
101
102
"Component"; "Extracellular vesicles", "ComponentType": "Mixture",
103
"RefName": "EVs-1",
105
"UnitArgType": "PROD", "Vol": "N/A", "Container": "200-pl microtube", "Cond": {
106
107
108
"State": "Purified", "Appearance"; "Small, almost invisible pellet"
110
111
}
112
)
113
"ucc":
114
A37


Reviews:
[{"review_id": "xQMLvBDJzQ", "reviewer": "Reviewer_BhtL", "summary": "This article studies for automatic experiment protocol design, representations in different levels of semantics : the protocol element instantialization with elementary operation representation, function abstraction as a sequential representation of the operation, and a model abstraction which specifies reagent and intermediate products.\nThe authors describe the 3 representations, propose an algorithm to automatically generate new protocols and demonstrate the creativity of the protocol generators based on the representation used.\n\nThe work can be framed as the hierarchical representation of policies for an MDP, trained from a natural language corpus, and used to generate new MDPs. The strength of this work lies in the fact that the methodology is tested across several experimental domains. However, the formalisation is not sound.", "questions": "In section 2.1, why is an experimental objective specified both by the final product and the final operation ? I believe the specification of the final product is the objective, and the final operation is in most cases, only the means.  In the example of a test of the significance of a hypothesis, I would re-formulate the objective as the observation of a property that confirms or contradicts the hypothesis. A definition of a hypothesis based on an operation is restrictive on the generalisation of hypotheses. The authors write in l/225 \"operations are the methods to realize rather than the objectives to achieve\".\n\nWhy were other experimental domains, such as chemistry or physics left out ?\n\nIn section 4, I do not quite understand how the testing set is evaluated. Under which criteria/input/prompt is each protocol generated ? What is then the corresponding ground truth ?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "The article ambitiously represents a synthetic representation of experimental protocols across different experimental sciences : Genetics, Medical, Bioengineering and Ecology.\nThe article methodically studies the automatic computation of the representation, and then assesses the utility of the protocols generated.", "weaknesses": "* The article lacks a sound formalisation of the problem. \n\n** While the issue is to represent actions (experimental operations) at different levels of hierarchy, to model the change in the environmental reagents, there is no hint of using the formalisations of Markov Decision Processes or hierarchical actions/reinforcement learning.\nWhen the three levels of representations can be easily formalised using the MPD formalisation (for instance in terms of actions/policies, options and waypoints states/subgoals), the proposed formalisation seems imprecise.\n\n** while the word \"planning\" is used several times in the text, it is only at l.444 that the authors explicit planning tasks as \"the exploration\nof novel experimental goals\". This definition is confusing. While \"planning\" generally refers to finding the succession of actions to obtain a predefined goal, exploring novel goals is different, and can be referred to as \"goal babbling\" for instance.\n\n** In 2.2. the vocabulary used is confusing: a precondition is generally a property of the state that allows you to carry out your operations. It is a distinct notion from an input.\n\n** Some terms are not defined. For instance : execution context (in l.190 which is used differently from execution condition) and key-value pairs (l193)\n\n\n* While the authors present comparative results of their representations and algorithms, there is no comparison with other approaches. How do the results compare quantitatively or qualitatively to the state of the art ?\n\n*While the authors mention as objectives the \"exploration of novel goals\", \"generating novel experimental objectives\" and aim to measure \"a protocol's novelty\", their metrics is only based on similarity measures. Diversity is not mentioned in the criteria. Could you add diversity measures or argue how the proposed metrics take into account diversity ?\n\nMinor comments : \n\n* Figures need a description to better understand what is shown\n* l.234: \"Any status transition of the product flow is caused, and is only caused, by the effects of operations\". This stance excludes general evolving systems.  What about dynamic systems, including with slow transformations, especially in biology or ecology?\n* l.440 : \"the three scenarios of protocol design introduced in sec 1\". It seems sec 1 introduces 3 representations/levels of encapsulation. \n* The results should report computing load \n\nTypos : \n\n* l317 : \"is consist of \"\n* l399: \"to cover as rich context as possible\"\n* l420 : \"we report and analysis\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tt5QZlCERr", "reviewer": "Reviewer_WJzq", "summary": "This paper addresses the challenge of automated protocol design for conducting scientific experiments. While it is currently possible to automatically execute predefined protocols, it remains a challenge how to design new protocols or modify existing ones to achieve novel experimental objectives. The authors propose a hierarchical representation framework that enables automated protocol design by capturing both procedural and domain knowledge at multiple levels of abstraction. Specifically, the framework consists of three hierarchical levels: (1) a basic level that breaks down protocols into individual actions with their specific attributes (like timing, temperature, etc.), (2) an operation-centric view that groups and generalizes these actions based on their purpose, and (3) a product-flow-centric view that tracks how materials change and interact throughout the experiment. These three levels are implemented using Domain-Specific Languages (DSLs) that help verify and ensure the correctness of protocols. The framework includes an automated method to generate these representations for different scientific domains using (mostly) non-parametric modeling, allowing it to learn and adapt to different types of experiments.", "questions": "1. Can you provide more intuition on what the interface \\phi is? Is that simply a set of possible experiments for a given operation (such as the given “homogenization”)? Or a set of functions? Why is it operationalized the way it was operationalized?\n\n2. The motivating examples and descriptions seem to be very heavily concerned with natural sciences. Is it possible to extend your framework to, say psychology or experimentation in computer science itself?\n\n3. How scalable is the framework for more complex protocols?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The premise of the paper is extremely compelling - enabling AI agents (specifically LLMs) with a structured representation of expert knowledge that allows them to harness their generative capabilities to create novel experimental protocols. \n\n2. Representing the task of protocol modeling protocols as conditional probabilities is particularly elegant, as it enables better control and parameter learning compared to black-box approaches like neural networks, primarily because this representation naturally captures the logical dependencies between experimental steps. \n\n3. The dual verification system combining product and operation views is especially clever, as it allows simultaneous optimization of protocol operations while ensuring each step both builds on previous products and generates viable outputs for subsequent steps.\n\n4.  The comprehensive evaluation across multiple scientific domains with real human experts demonstrates the practical utility and broad applicability of the method.", "weaknesses": "1. The language is often very abstract and abstruse. This is sometimes expected because the topic itself is very abstract. But making it simpler would enable the reader to appreciate the contributions more if there was more clarity in explanations. \n\n2. The work suggests it uses LLMs for the protocol design and it is indeed mentioned where LLMs are used. There are no details, however\non how exactly LLMs are employed in the suggested framework (i.e. do they receive the input from the DSL? or is the DSL a step along the way). It is somewhat intuitive, but an explicit description would be helpful. I appreciated the details in the Appendix but this should be mentioned in the main text.\n\n3. It is not clear how scalable this approach is for more difficult protocols. It is not also clear for someone outside of the tested areas if there even are more difficult protocols. This could be addressed.\n\nOther points:\n- It would be very helpful to start with a motivational example of some particular case of research design like the one in Figure 1. Currently, it reads very abstract. But the Figure is helpful. \n\n- Lines 92-107 are supposed to briefly summarize the mechanism introduced by authors but it is very hard to comprehend. It would be good to correspond these to Figure 1B.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PAk5MoYGwJ", "reviewer": "Reviewer_LMMv", "summary": "The authors propose a hierarchical structured representation of experiment protocols to improve the reliability of novel protocols generated by LLMs with RAG. \n* The representation is actually a dual representation over operation-centric and flow-centric views of a protocol.\n* This works because the representation serves as a domain-specific language to generate protocols in a bipartite graph-like structure, which enables verification of both the \"nodes\" and \"edges\" of the protocol, improving its reliability. \n* However, to use this structured representation requires extracting the entities, which is domain-specific. To do this, they use a Dirichlet Process Mixture Model to extract entities from natural language protocols, which they then aggregate for functions but not for components. \n* To demonstrate the improved viability of protocols generated by this approach, they subselect a test set of 140 protocols from their database with the intent of evaluating across 3 different protocol design tasks and 4 domains. They evaluate generated protocols against the ground truth using six distinct metrics. They compare seven different design approaches which consider different elements of their proposed representation, with and without verification, showing how the different aspects of their approach contribute to its success.", "questions": "* Line 44: What is a \"protocol portal\"? Such terminology, if not generally entrenched in the suggested domain, should be introduced.\n\n* Line 199: \"The total amount of the instance actions can be extremely high, i.e., about 150K per domain,\" how do you arrive at this number?\n\n* How do you track quantities of reagents? The algorithm seems to treat them as a set and remove them if used once without any notion of quantity.\n\n* Line 504: what it t( ) ?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors make three contributions:\n1. They present the protocol design problem setting, as well as the hierarchical representation of protocols.\n2. The propose a method to automatically determine the elements of this structured representation from existing protocols.\n3. They perform experiments assessing the value of their representation by testing different methods based on this representation on a varied set of tasks.\n\n# originality\n\nThe proposed representation and method appear quite novel, and the choice of metrics to assess the protocol designers is interesting. However, there is no mention of the extensive literature on problem decomposition for LLMs, e.g. Chain-of-Thought reasoning (Wei et al. (2023)). This makes it difficult to ascertain exactly how novel such a decomposition really is. it would be helpful to compare and contrast to existing methods in the literature both for the applied problem of protocol design (e.g. O’Donoghue et al. (2023)) and the solution of task decomposition (e.g. Wei et al. (2023)). It would also be helpful to explicitly state in the contributions whether the is a new problem formulation, application of an existing method to a new domain, or both.\n\n# quality\n\nThe conclusions drawn in section 4.5 are the following: \n1. Their proposed approaches outperform naive counterparts.\n2. Dual-view hierarchical representations excel at protocol design tasks.\n3. The approach generalizes across scientific domains.\n\nThe first claim is sound considering the extensive metrics used to assess different approaches. However, it is also shallow compared to the amount of information in Figure 3. On average, it seems that the approach put forward performs well, but there is significantly more variability in their performance. Furthermore, different metrics seem to behave different to the new metrics, with IoU metrics increasing the tail of good results (e.g. IoU metrics), and Sim metrics decreasing the tail of bad results. It is surprising that this is not discussed in more detail, and that there is no estimator of uncertainty (e.g. std, stderr, variance) in the tables in Appendix A. \n\nThe second claim is hard to interpret, as there are no baselines from the literature that are evaluated. As such, it is unclear what is meant by \"excel\" in this case. Given the highly variable performance values in Figure 3 and the lack of baselines from the literature, it is not possible to conclude this from the available results. You could alternatively state that the dual-view approaches outperform their counterparts across all tasks on average across all metrics.\n\nThe third claim is well supported by the experiments across four domains and the aggregate results. That being said, an additional set of results in the appendix that shows results by scientific domain would also be helpful in supporting this claim, especially if these results were to show that the proposed methods performed best across all domains separately (i.e., tables like those in appendix A, but partitioned by domain instead of task).\n\n# clarity\n\nAs a non-expert on protocol design, I found the problem motivation well done. The task is well situated within its broader context. Figure 1A is enlightening, but Figure 1B is slightly confusing, as the relationships between the different elements could be made clearer. The problem setting is well presented with a judicious use of notation and examples throughout. The experiments are well explained as well.\n\n# significance\n\nThe motivation put forward in the introduction accurately captures the significance of this work. However, it could be improved even more by relating it to existing work that creates structured workflows using LLMs, and relating it to the more general problem of getting LLMs to structure their outputs.", "weaknesses": "# Presentation\n\n* Figure 3 Caption is not standalone, very small font sizes for xticks and y ticks. The x-axis is not mentioned, which makes interpretation impossible.\n\n* Figure 2 labels and ticks are too small, the caption is largely uninformative without the main text\n\n* Fig 1 B is unclear\n\n* Limitations section is empty (Appendix G)\n\n# Soundness\n* Line 361: A reference to Fig 2B is insufficient for the claim that function abstraction works. Idem for Fig 2C. The results must be interpreted (see the \"Quality\" section in strengths for more details).\n\n* The testing set is susceptible for memorization because you are using LLMs. It would be good to have a set of protocols that are after the cutoff dates of the LLM training.\n\n* it would be good to introduce the SA algorithm in more detail so that we understand what the similarity score actually represents.\n\n* Lack of baselines from the literature\n\n# Contribution \n* Very limited discussion of the results.\n\n* Unclear whether they consider the problem setting to be a contribution. It would be important to contrast the problem formulation with that of prior work, such as that of O’Donoghue et al. (2023). In particular, whether their formulation of the protocol design problem (not the representation itself) differs in any meaningful way. This is important because the identification of the protocol design problem is cited as a contribution on line 147.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GAEBW9Wz61", "reviewer": "Reviewer_Xeav", "summary": "The work focuses on the automated design of self-driving labs' protocols. Protocols specify how robots in self-driving labs conduct experiments. In many cases, conservative protocols are not sufficient because they do not allow for significant scientific discovery. However, designing new protocols is a cumbersome and error-prone process. To tackle these challenges, the authors suggest LLMs for the generation of protocols.", "questions": "- Why LLMs and not some other AI method? This would allow for a better positioning of the work.\n\n- How can the generated protocols be certified? This would be a much-appreciated, brief discussion point.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem is relevant and timely. Automation is much-needed and of high added value.\n\n- The problem is open without sufficient solutions in the area. The approach seems to be novel and contributes to the state of the art.\n\n- The contribution seems to be sound and the evaluation is convincing.\n\n- Clear presentation and discussion.", "weaknesses": "- It is unclear why LLMs are chosen for this purpose. This is essentially a big design-space exploration problem with the added opportunity of using a physical actor in purposeful experimentation. A similarly apt approach could be using digital twins with traditional learning methods, e.g., reinforcement learning and obtain a formally modeled protocol.\n\n- I find it somewhat concerning that a costly and potentially hazardous endeavor such as robotized experimentation with chemicals and biologically active materials is approached at the level of LLMs and the safety concerns are not discussed. For example, certification of protocols through formal verification might not be possible in this approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)