A uto DSL : Automated domain-specific language design for structural representation of procedures with constraints (2024)

Abstract: Accurate representation of procedures in restricted scenarios, such as non-standardized scientific experiments, requires precise depiction of constraints. Unfortunately, Domain-specific Language (DSL), as an effective tool to express constraints structurally, often requires case-by-case hand-crafting, necessitating customized, labor-intensive efforts. To overcome this challenge, we introduce the AutoDSL framework to automate DSL-based constraint design across various domains. Utilizing domain specified experimental protocol corpora, AutoDSL optimizes syntactic constraints and abstracts semantic constraints. Quantitative and qualitative analyses of the DSLs designed by AutoDSL across five distinct domains highlight its potential as an auxiliary module for language models, aiming to improve procedural planning and execution.

## introduction comprehending and executing procedures articulated in natural language to achieve a specified goal represents a fundamental challenge for artificial intelligence (ai) systems. with the boost of large language model (llm) @xcite @xcite , ai systems possess the capability of reasoning over and planning for procedural tasks intended for both human and robotic execution across a broad spectrum of everyday scenarios 1 @xcite @xcite , such as cooking according to a recipe, obviating the necessity for external representation of procedures beyond text. however, contexts significantly more restricted than everyday scenarios, such as conducting nonstandardized experiments in scientific laboratories, need to follow specific protocols 2 . a proto-1 visit www.wikihow.com for demonstrations. 2 visit www.nature.com/nprot/ for examples. prepare samples by adding loading buffer and heating to denature proteins. resuspend cells in fresh medium and evenly distribute them into new culture dishes. the sample treated with buffer b undergoes digestion under optimized conditions for the specific enzyme. after this step, the lifecycle of buffer b ends. for transfection, seed at low density and change media post-transfection for differentiation or growth. feeding until the larvae turn into pupae, at which point increase the food quantity. divide the rna solution equally into four microcentrifuge tubes. incubate each group under conditions recommended for each reagent. missing parameter under-specified setting limited device capacity unexplained action non-sequential workflow temperature check if the desired product is obtained with optimal specificity and yield. if the results are not satisfactory, adjust the parameter being tested and repeat steps 2 and 3. iterative requirement qualitative identifier quantitative parameter in use available available intermediate products high-level action 1 2 3 4 a d b e c f this occurs when a quantitative parameter is described using qualitative terms, leading to ambiguity, e.g., unclear mixture configurations in dna extraction. (c) action undefinition: this involves the description of procedural steps at a high level without grounding to the specific, executable actions required, e.g., the vague change operation in cell preparation. repeat the dna purification process three times using fresh reagents and buffers for each cycle, following the specified centrifugation conditions at 12,000-16,000 g for 1-2 minutes in a microcentrifuge. knowledge graph protocol text under-specified, and the logic of procedure may be non-linear, as demonstrated in fig. 1 . given these unique distinctions against daily procedures, accurate interpretation of protocols requires explicit depiction of constraints (see fig. 2 for example). intuitively, structural representation imposes constraints on the processing of protocols. this is achieved through purely symbolic approaches that depict procedures as flow-graphs @xcite @xcite , and neuro-symbolic hybrid methods that superimpose procedural structures onto neural networks @xcite @xcite @xcite . both strategies impose constraints on the interpretation of procedures, thereby reducing the incidence of superfluous operations. symbolic constraints range from elementary grammars to general-purposed programming languages, with the capability of a constraint system being pivotal in refining a coarse interpretation space into a more precise one @xcite @xcite . in light of the context, what level of capability should we expect the constraint to possess? this discourse introduces dsl, a category of symbolic systems endowed with the most potent constraints. dsls are programming languages tailored to specific problem domains, encapsulating both syntactic constraints and semantic constraints inherent to those domains. for instance, biocoder (ananthanarayanan and thies, 2010), developed by microsoft, is a dsl explicitly designed to constrain experimental protocols. on the syntactic level, the variable management mechanism inherited from c/c++ enables dsls to monitor the lifecycle of each intermediate product, ensuring no omissions or duplications. on the semantic level, the precise definitions of actions, combinations of reagents' names and volumes, with subprocedures abstracted from domain-specific concepts, guaranteeing procedural execution consistency. drawing inspiration from dsls, can we design constraints for protocols in dsl fashion? hardly, due to the deterministic and substan-tial cost. structural constraints necessitate custom design for particular domains, which is prohibitively expensive, given that these domains are highly specialized and often diverge significantly from the conventional purview of computer scientists @xcite . the development of a dsl necessitates a comprehensive integration of in-depth domain knowledge. furthermore, the designed dsl must align with theoretical aspects of formal language design while also meeting the distinct requirements of the specialized domain. this necessitates a bidirectional alignment between computer scientists and domain experts, a process that is intrinsically caseby-case, implying that a dsl developed for one domain is unlikely to be applicable or easily adaptable to another, thus limiting the scalability of dsl-based constraints across various domains. in this study, our objective is to offer an initial proof-of-concept aimed at reducing the design cost of dsl-based constraints for protocols. we propose a scalable framework, termed autodsl, that facilitates the automated creation of dsls. the framework approaches the task as a bidirectional optimization problem, where the design of a dsl is abstracted from domain-specific corpora through a bottom-up process and concurrently derived by general programming language design principles in a top-down manner. this approach emulates the iterative dialogue between computer scientists and domain experts, progressively bridging the conceptual gap between their respective fields of expertise. the syntactic constraint should adequately define consecutive actions and their repetitions, interruption, concurrence, subcomponents, and reactants. constructs of the semantics constraint need to accurately reflect the domain's concepts and the relations between them, without redundancy or incompleteness. we utilize protocols from various domains within the experimental sciences -namely genetics, medical and clinical research, ecology and environmental research, bioengineering, and computational biology -as the primary testing ground for our methodology, due to their inherent complexity, resource constraints, and susceptibility to errors. these domains exhibit significant disparities both in syntactic and semantic language features. comprehensive experiments demonstrate that autodsl is capable of generalizing dsl-based constraints tailored to these di- domain corpus of nl protocols remove transfer resuspend the culture media by pipetting. ## constraints in protocols in this section, we scrutinize the requirements for precisely constraining protocols. the distinctive challenges of engaging with such systems stem from their complexity of procedures, limited in resources, and vulnerability to errors. ## autodsl framework this section describes our problem formulation and solutions for the autodsl framework. ## constraint utility assessment in this section, we evaluate the utility of leveraging dsl-based constraints for the representation of new coming experimental protocols across the five domains, considering both syntax and semantics. please refer to tab. 1 for a demonstration. ## general discussions in this work, we present the autodsl framework as a proof-of-concept to facilitate the automation of designing dsl-based constraints across various domains. through both quantitative and qualitative evaluations of the dsls designed by autodsl in five distinct domains, we demonstrate its capability as an auxiliary module for llm. rationale behind dsl the decision to leverage dsl for constraint representation is rooted in several considerations. primarily, dsls adeptly capture domain-specified syntactic and semantic constraints, aligning well with the hardness of representing protocols with complicated control flows and operations. moreover, dsls leverage a deterministic verification mechanism derived from general-purpose programming languages, offering a robust means of imposing constraints on the inherently nondeterministic outputs of llms. additionally, dsls are user-friendly to both humans and machines, maintaining a minimal set of language features that facilitate ease of adoption. no universal constraint it is unrealistic to expect a one-size-fits-all protocol constraint applicable across varied domains. for a constraint system to accurately delineate the execution space of a particular domain, it must encompass domainspecific syntax and semantics. though it is possible to devise a comprehensive set of constraints that covers the requirements of all conceivable domains, such an approach would yield a constraint system of prohibitive complexity, rendering it impractical for end-users. conversely, simplifying this universal constraint to enhance user-friendliness inevitably compromises its capability, leading to the expressivity-complexity dilemma @xcite . rather than seeking an elusive generality, focusing on domain-specific constraint development and striving for the automation of this process may offer a pragmatic way to circumvent this dilemma.