Retrieval-Retro: Retrieval-based Inorganic Retrosynthesis with Expert Knowledge

Authors: heewoong  noh
namkyeong  lee
gyoung s na
chanyoung  park
  kaist
  krict
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 11:51:05

Abstract: While inorganic retrosynthesis planning is essential in the field of chemical science, the application of machine learning in this area has been notably less explored compared to organic retrosynthesis planning. In this paper, we propose Retrieval-Retro for inorganic retrosynthesis planning, which implicitly extracts the precursor information of reference materials that are retrieved from the knowledge base regarding domain expertise in the field. Specifically, instead of directly employing the precursor information of reference materials, we propose implicitly extracting it with various attention layers, which enables the model to learn novel synthesis recipes more effectively.
Moreover, during retrieval, we consider the thermodynamic relationship between target material and precursors, which is essential domain expertise in identifying the most probable precursor set among various options. Extensive experiments demonstrate the superiority of Retrieval-Retro in retrosynthesis planning, especially in discovering novel synthesis recipes, which is crucial for materials discovery.
The source code for Retrieval-Retro is available at https://github.com/HeewoongNoh/Retrieval-Retro.

Main paper: 
 introduction: 
 Discovering new materials is a fundamental problem in materials science [28, 4, 25] , providing innovative options in various industry fields, such as semiconductors and batteries [37, 36] . On the other hand, it is also important to establish synthetic routes for newly discovered materials [21] , i.e., retrosynthesis planning, as the ability to synthesize these materials is essential for their successful commercialization beyond mere discovery.For organic materials, retrosynthesis planning approaches identify valid and efficient synthetic routes [8] by breaking down complex target molecules into smaller molecules that are commercially available and easily synthesizable. During the process, they focus on the structural information of target molecules, such as functional groups and reaction centers, that are related to widely known organic reaction mechanisms [21, 7, 9, 40, 22, 23] . Following the practice of organic retrosynthesis, machine learning (ML) based approaches utilizing the molecular structure expressed as SMILES strings [43] or molecular graphs have been extensively studied [45, 33] .However, unlike organic retrosynthesis, using the atomic structural information of inorganic materials for retrosynthesis presents significant challenges due to 1) the high computational load incurred by the larger number of atoms compared to organic molecules [7, 6] , and 2) the failure of traditional physical theories for the atomic structure computation caused by the inclusion of diverse and unusual elements [11, 1] . Therefore, a chemical composition-based approach is essential for retrosynthesis planning of inorganic materials. Besides the challenges of using the atomic structural information, there is a lack of a clear and general theory regarding the mechanisms of inorganic synthesis reactions The target material is more likely to be synthesized using the precursor set that exhibits a more negative driving force. [34] . For these reasons, inorganic retrosynthesis planning is a more challenging task compared to organic retrosynthesis planning.Given these challenges inherent in the retrosynthesis planning of inorganic materials, applying existing ML-based organic retrosynthesis methods for inorganic purposes is infeasible. Consequently, there has been limited research into inorganic retrosynthesis planning compared to that for organic materials. As a pioneering work, CVAE [17] generates synthesis variables for target materials using a generative model, and ElemwiseRetro [18] reformulates the precursor prediction task as a multiclass classification problem with dozens of curated precursor templates. However, these approaches overlook the common practices in conventional inorganic material synthesis, where chemists identify reference materials similar to the target material and consult established synthesis recipes [12, 13] .More specifically, due to the aforementioned inherent challenges, chemists often engage in a costly trial-and-error method by referencing precedent recipes of reference materials from prior literature [21, 12] . Therefore, as illustrated in Figure 1 (c), the majority of the discovered synthetic routes for a target material share a common set of precursors with the corresponding reference material from previous studies (Figure 1 (a)), whereas only a small number of these routes involve a completely new set of precursors (Figure 1 (b) ). Inspired by the common practice, He et al. [12] propose to retrieve reference materials that are similar to a target inorganic material from a knowledge base of previous studies, and leverage their precursor information for retrosynthesis planning of the target material.Although the method proposed by He et al. [12] has proven to be effective by emulating standard practices in the field, it faces two significant limitations. The first limitation is that the model's predictive capability is limited to the precursor sets of retrieved reference material, thus inhibiting its capacity to deduce novel synthetic pathway. This is due to the heavy reliance on the precursor sets of reference materials. However, despite a small fraction of materials being synthesized with entirely new synthetic recipes as shown in Figure 1 (c), it is widely known that discovering novel synthetic routes with entirely new precursors can accelerate the inorganic material synthesis process [29, 27] and facilitate the discovery of cost-effective recipes [26, 30] . Thus, despite a large number of materials being synthesized through slight alterations to previously known synthesis recipes due to the complexities of inorganic material synthesis, it remains critical to identify new synthetic pathways that extend beyond the commonly known synthetic recipes.The second limitation is that it neglects the widely known domain expertise in the field [17, 18, 12] : the greater (more negative) thermodynamic driving force (∆G) between the target material and the precursor set, the more feasible it is to actually form the target material through the precursor set [31, 35] . As an example in Figure 1 (d), given a target material A 2 BCO 4 , a precursor set {AO, AO 2 , BCO} exhibits a significantly greater ∆G compared to another set {A 2 CO 2 , BO}, making it more probable that the target material will be synthesized from the first precursor set. Therefore, by analyzing the thermodynamic relationships between the target material and various precursor sets, we can identify which combinations of precursors are most feasible for material synthesis. For example, considering such relationship enables the selection of precursor sets that are likely effective starting materials for synthesizing the target material, thereby facilitating successful synthesis. However, previous works [17, 18, 12] overlook this crucial domain expertise, leading to their inability to identify these optimal precursor sets.To this end, we propose a novel inorganic retrosynthesis planning approach by implicitly extracting the precursor information with the domain expertise-enhanced reference material retriever. Specifically, instead of directly utilizing precursor information as in He et al. [12] -that is, explicitly incorporating precursors from retrieved materials for prediction-we propose to implicitly extract this information from reference materials using various attention layers that are designed to enhance and extract the precursor details of the reference materials. By providing the model with greater flexibility, we expect it to discover novel synthesis recipes that go beyond existing ones. Moreover, to determine which material should be referenced, we utilize well-established domain expertise in the field, i.e., the thermodynamic relationships between the target material and potential precursors, with a novel Neural Reaction Energy retriever. With a novel retriever, our model effectively identifies which material to refer to for inorganic retrosynthesis planning of the target material. Our extensive experiments demonstrate the effectiveness of Retrieval-Retro in inorganic retrosynthesis planning, especially discovering novel synthetic recipes, demonstrating the potential applicability of Retrieval-Retro in real-world materials discovery.In this study, we make the following contributions:• We propose to implicitly integrate the precursor information of reference materials, which enables the model to more effectively discover novel synthetic recipes of inorganic materials. • Furthermore, we introduce a novel retriever inspired by domain expertise, which assists the model in effectively determining which material to reference during inorganic retrosynthesis planning. • Extensive experiments demonstrate the effectiveness of Retrieval-Retro in various scenarios, particularly in the year split, which poses a more realistic and challenging environment. Additionally, its exceptional capability in uncovering new synthetic recipes for inorganic materials highlights its potential for practical application in real-world material discovery. 
problem setup: 
 Inorganic Retrosynthesis Planning with Chemical Composition. In inorganic retrosynthesis planning, determining the atomic structure of inorganic materials poses significant challenges and demands costly computational efforts. As a result, both chemists [21] and prior ML methodologies [18, 17, 12] depend exclusively on composition information. In line with previous studies, we also base our approach on the composition information of inorganic materials instead of structural data.Notations. An inorganic material can be quantitatively described by a composition vector x ∈ R d , where d represents the total number of unique chemical elements. Each element in the vector x represents the proportion of each chemical element that constitutes the material. As an example, consider the material with chemical formula SiO 2 , where Si and O correspond to element numbers 14 and 6, respectively. It can be represented as x = (x 1 , x 2 , . . . , x d ), where x 14 = 1 3 , x 6 = 2 3 , with all other x values being zero. Moreover, following a previous work [10] , we construct a fully connected composition graph G = (E, A), where E is the set of elements associated with the nonzero components of x, and A ∈ {1} n×n is the fully connected adjacency matrix with n indicating the number of nonzero entries in x. We initialize the initial feature e i of each element e i in E with Matscholar [44] , whose element embedding is obtained from the vast amount of scientific literature.Task: Precursor Prediction. Following a previous work [12] , we formulate precursor prediction as a multi-label classification problem. Given a fully connected graph G = (E, A) representing an inorganic material, our objective is to train a model F that predicts the possible precursors for the material, i.e., y = F(G), where y ∈ {0, 1} l indicates the label vector with each element signifying the predefined l precursors. That is, each element y i in the label vector y indicates whether i-th precursor is necessary (y i = 1) or not (y i = 0) for synthesizing the target material G.To begin with, we briefly introduce the compositional graph encoder, which is used to encode the material representation in the paper. Specifically, given a composition graph G = (E, A) of a material, we obtain the material representation g as follows:EQUATIONwhere "Pooling" refers to the sum pooling of the node representations within the composition graph G, which are derived from a GNN encoder. The detailed architecture used in the paper is provided in 3 Proposed Method: Retrieval-RetroIn this section, we introduce our proposed method Retrieval-Retro, a novel inorganic retrosynthesis planning approach that implicitly extracts the precursor information of reference materials retrieved with two complementary retrievers, i.e., Masked Precursor Completion (MPC) Retriever and Neural Reaction Energy (NRE) Retriever. The overall framework of Retrieval-Retro is shown in Figure 3 . 
reference material retrieval: 
 Before we extract precursor information from the reference materials, it is essential to decide which material should be referenced for the extraction. To elaborately determine which materials to reference, we employ two complementary retrievers: the Masked Precursor Completion (MPC) retriever and the Neural Reaction Energy (NRE) retriever.Masked Precursor Completion (MPC) Retriever. MPC retriever identifies the reference materials sharing similar precursors with the target material by learning dependencies between precursors. Specifically, given a chemical composition vector x of a target material, we obtain its representation m = M (x), where M : R d → R d ′ indicates two layered MLPs with non-linearity. We define a learnable precursor embedding matrix P ∈ R l×d ′ , whose i-th row p i ∈ R d ′ represents a learnable embedding vector for the i-th precursor. Concurrently, we generate a randomly perturbed precursor vector ỹ from the provided precursor information y, and create a perturbed precursor matrix P by applying the perturbed precursor vector ỹ as a mask to the precursor matrix P. Specifically, pi is masked if ỹi = 0, and is left unchanged otherwise. We then integrate the representation m and the perturbed precursor matrix P with cross-attention to form precursor conditioned representation of the material s. Then, the model is trained to reconstruct the original precursor vector y from the precursor conditioned representation s and precursor matrix P, by representing probability for each precursor as follows σ(s ⊤ p i ). The overall training procedure of MPC retriever is in Figure 1 (a) .By doing so, it enables the retriever to learn dependencies among precursors and the correlation between the precursors and the target material. With the MPC retriever, we calculate the cosine similarity between the representation of the target material and all materials in the knowledge base obtained through M , and retrieve the top K materials that are similar to the target material.Neural Reaction Energy (NRE) Retriever. Although the MPC retriever effectively identifies reference materials with potentially similar sets of synthesis precursors, it overlooks widely recognized domain expertise in the field, i.e., the thermodynamic relationships between materials, which is essential for the inorganic synthesis process, particularly in selecting appropriate precursors [35, 27] . More specifically, the thermodynamic driving force between the target material and precursor set can be quantified by Gibbs free energy (∆G), which is a measure of the material's thermodynamic stability. Under constant pressure and temperature, a negative ∆G indicates that the energy of the target material is lower than that of the precursor set, signifying that the synthesis reaction can occur spontaneously [31, 35] . As a result, it is widely known that the more negative ∆G, the more precursor set is likely to synthesize the target material.Based on this knowledge, it would be beneficial to retrieve materials that have the precursor set capable of inducing favorable reactions with the target material by considering the thermodynamic force. ∆G can be approximated by the difference ∆H between the enthalpy of the target and precursor set ∆H as follows:EQUATIONwhere the H T arget and H P recursor set represent the formation energy of the target and precursor set. Therefore, a straightforward solution for calculating ∆H is to utilize the formation energy of the target material and the precursor set that can be directly obtained from the extensive database of structure-based DFT-calculated formation energy. However, it is widely known that DFT-calculated values frequently diverge from experimental data, while actual material synthesis occurs in realworld wet lab settings [16] . Even worse, there is no guarantee that these databases encompass all materials of interest in inorganic retrosynthesis planning. Consequently, it is essential to develop a composition-based formation energy predictor that is specifically designed for experimental data.To this end, we propose a learnable Neural Reaction Energy (NRE) retriever, which is pre-trained on abundant DFT-calculated formation energy data and then fine-tuned on experimental formation energy data as shown in Figure 1 (b) [16] . Specifically, we initially pre-train the NRE retriever using the Materials Project database [14] , training the model to predict DFT-calculated formation energy from representations derived from the composition graph encoder (see Section 2.2). Subsequently, we fine-tune the retriever using experimental formation energy data [32] , which allows the model to adapt to experimental data. We demonstrate the effectiveness of the training mechanism in Appendix E.2. Finally, given a trained NRE retriever, we can compute the formation energies of the target material and the precursor set of reference materials in the knowledge base. We then retrieve K reference materials that exhibit the most negative ∆G, selecting from those whose precursors contain the same elements as the target material, along with other common elements such as C, H, O, and N. Note that calculations are performed prior to training, so no additional training costs are incurred.Now, we discuss how to extract the precursor information from the reference materials elaborately selected in Section 3.1. While the previous work [12] directly utilizes the precursor information of the reference materials, this limits the model's ability to learn and deduce new synthetic recipes for the target material, which can significantly accelerate the materials discovery and reduce the cost of material synthesis. Therefore, we propose to implicitly extract the precursor information from the retrieved material with various attention layers, i.e., self-attention and cross-attention layers, which aim to enhance the representation of reference materials by considering other reference materials and extract the implicit precursor information from the enhanced representation, respectively.To do so, we first encode the target material and K reference materials using their associated composition graphs G via the composition graph encoder introduced in Section 2.2. As a result, we obtain the representation of target material g t ∈ R D and the K reference materials G r = [g 1 r , . . . , g K r ] ∈ R K×D , where g k r indicates the representation of the k-th reference material. Reference Enhancing with Self-Attention. To effectively extract the precursor information from the reference materials, we first enhance the representation of these materials through a self-attention mechanism [39, 24] . This approach encourages the model to selectively determine which information to extract from a particular reference material by considering the relationships among various reference materials. To do so, we first obtain a new matrix for reference materialsG ′ 0 r = [g ′ 1 r , . . . , g ′ K r ], where g ′ k r = ϕ 1 (g k r ||g t )indicates the modified representation of the k-th reference material regarding the target material, where || denotes the concatenation operation, and ϕ 1 : R 2D → R D is a learnable MLP. Note that by modifying the representation of the reference material through concatenation with the target material, we allow the model to extract information pertaining to the target material, rather than focusing solely on the reference materials. Then, we implement a self-attention mechanism to determine which information to extract from the reference material, taking into account the relationships among the other reference materials:EQUATIONwhere s = 1, . . . , S indicates the index of the self-attention layers. Different from the conventional self-attention layers [39] , we directly utilizeG ′ s-1 r as query Q G ′ s-1 r , key K G ′ s-1 r, and value V G ′ s-1 r , without any learnable parameters [24] . By analyzing the relationships between the reference materials, the model improves the representations of these materials, thereby supplying more appropriate references to be extracted by the cross-attention layer.Reference Selection with Cross-Attention. Lastly, we extract the implicit precursor information by merging the representation of the target material with that of the enhanced reference materials via a cross-attention mechanism [38, 42] . With cross-attention layers, we expect the model to learn favorable synthesis recipes from reference materials by selectively learning from reference materials with attention weights. More formally, cross-attention layers are formulated as follows:EQUATIONwhere c = 1, . . . , C indicates the index of the cross-attention layers. Note that we use an enhanced reference material representation G ′ S r and target material representation g t as inputs to the first crossattention layer, i.e., G′ 0 r = G ′ S r and g 0 t = g t . Moreover, we also utilize g c-1 t as queryQ g c-1 t, and the reference material representation G ′ S r as key K G ′ S r and value V G ′S r identical to the self-attention layer, without any learnable parameters. By employing cross-attention layers between the target material and reference materials, rather than the precursor set of the reference materials, the model effectively accesses the synthetic recipes of reference materials without explicitly using precursor information, thus enabling the discovery of novel synthetic recipes for the target material. We employ this implicit precursor extraction process for the reference materials gathered using both the MPC retriever and the NRE retriever, resulting in g C t:MPC and g C t:NRE , respectively.Finally, we compute the model prediction ŷ as follows: ŷ = ϕ classifier (g t ||g C t:MPC ||g C t:NRE ), where ϕ classifier : R 3D → R l is an MLP with non-linearity. Note that each dimension in ŷ, i.e., ŷi , indicates the model's predicted probability of whether precursor i will be included or not. For model training, we adopt Binary Cross Entropy (BCE) loss, which is commonly used for multi-label classification learning [5, 47] , as:L = -1 l l i=1 [y i log(ŷ i ) + (1 -y i ) log(1 -ŷi )]. 
experimental setup: 
 Datasets. We use 33,343 inorganic material synthesis recipes extracted from 24,304 materials science papers [20] following prior studies [12, 18] . Due to the lack of an extensive database containing inorganic synthesis recipes [20] , we use the training set as the knowledge base, following a previous work [12] . Additional details about datasets are provided in the Appendix B.Baseline Methods. We compare Retrieval-Retro with two inorganic retrosynthesis methods (i.e., He et al. [12] and ElemwiseRetro [18] ), two composition-based representation learning methods (i.e., Roost [10] and CrabNet [41] ), and three newly proposed baselines (i.e., Composition MLP and Graph Network [3] , Graph Network + MPC) to demonstrate the effectiveness of Retrieval-Retro. The first newly introduced baseline is called Composition MLP, which does not retrieve reference materials but instead relies on the composition vector of the material. He et al. [12] conducts inorganic retrosynthesis planning by using the MPC retriever to access reference materials based solely on the material's composition vector. ElemwiseRetro [18] acquires precursor information through a fully connected graph that represents the constituent elements within the material. Furthermore, two composition-based material representation learning approaches, namely Roost [10] and CrabNet [41] , explore the intricate interactions among elements within materials using message passing and self-attention, respectively. Although these methods are initially designed for property prediction, we have adapted prediction heads so that they can be effectively used for inorganic retrosynthesis planning. We also evaluate two new baselines, Graph Network [3] and Graph Network + MPC. The former predicts precursors without retrieving reference materials, while the latter does so after retrieving references. As these methods utilize the same backbone GNN structure as in our approach, Following previous works in retrosynthesis planning [45, 40, 18, 9] , we adopt Top-K exact match accuracy to evaluate the effectiveness of Retrieval-Retro. In addition, we also employ Macro and Micro Recall, which are commonly used as metrics in the multi-label classification problem [5, 46] . We provide further details on evaluation protocol in Appendix D.Effectiveness of Retrieval-Retro in Inorganic Retrosynthesis Planning. In Table 1 , we have the following observations: 1) Modeling the interaction among the constituent elements (Int. ✓) proves more effective than simply representing the material as a composition vector (Int. ✗). This demonstrates the importance of modeling the interaction between the composition elements not only for the material property prediction task [10, 41] , but also for inorganic retrosynthesis planning.2) By comparing the methods that utilize retrieval (Retr. ✓) with those that do not (Retr. ✗), it becomes apparent that using precursor information from reference materials contained in a synthesis literature knowledge base enhances the precursor prediction performance. This underscores the significance of reference materials, which is also a standard practice in the traditional material synthesis process. 3) We also find that Retrieval-Retro surpasses all baseline models, indicating that our method successfully facilitates inorganic retrosynthesis planning. Furthermore, there is a significant performance enhancement over baseline methods in the year split setting (Table 1 (a)), which is a more realistic and challenging scenario, proving Retrieval-Retro's efficacy in real-world inorganic material synthesis processes.Discovering Novel Synthesis Recipes. As shown in the previous section, the precursor information of reference materials is beneficial for the inorganic retrosynthesis planning of the target material. This observation raises a natural question: How can we effectively integrate the information from reference materials into synthesis recipes, particularly for novel synthesis recipes? To answer this question, we evaluate how each component in the model affects the model's capability of deducing novel synthetic recipes for the target material. To do so, we first divide the test set of the year split dataset into Subset case and New case. As shown in Figure 1 , the subset case includes target materials that share a common set of precursors with those in the training set, while the new case comprises target materials that have an entirely new set of precursors. Moreover, we mainly compare Retrieval-Retro with "Graph Network + MPC" in Table 1 , since the only difference between the models lies in whether the model directly utilizes the precursor information from reference materials. Then, we separately assess the model's performance in the subset case and the new case. In Table 2 , we have following observations: 1) By comparing the Graph Network with and without the MPC retriever, we find that incorporating the precursor information from reference materials enhances model performance in subset cases. One interesting observation is that, this information negatively impacts performance in Top-10 new case, demonstrating that it can hinder the model's ability to deduce novel synthetic pathways. 2) Conversely, Retrieval-Retro, which implicitly integrates the precursor information, consistently shows performance improvements in both the subset and new cases, particularly widening the performance gap in the new case-a more realistic and challenging scenario. These findings illustrate the importance of how precursor information from reference materials should be integrated, particularly in identifying new synthetic pathways, which can speed up the material discovery process and reduce synthesis costs. 3) Interestingly, we find that the NRE retriever also consistently enhances the model performance, a benefit likely stemming from its complementary relationship with the MPC retriever. Specifically, while the MPC retriever captures dependencies between precursors and the target material based on previously observed data, novel synthesis recipes might diverge from the existing synthesis patterns documented in the literature. On the other hand, the NRE retriever employs domain expertise that is independent of these existing patterns, thus filling gaps that the MPC retriever might overlook. By accessing such complementary reference materials, the model is able to acquire additional new precursor information, which contributes to the observed performance improvements. In Table 4 , we conduct a qualitative analysis of the materials retrieved by each retriever and examine their impact on model predictions.In conclusion, we find that each element of Retrieval-Retro plays an effective role in inorganic retrosynthesis planning, particularly in identifying new synthesis recipes, showcasing its potential influence in the field of materials discovery.Ablation Studies: Effects of Retriever. To verify the effect of the retrievers, we conduct ablation studies by removing the retriever modules. In Table 3 , we have following observations: 1) When reference materials are randomly retrieved without using trained retrievers, the model extracts precursor information that is irrelevant to the synthesis of the target material, leading to a deterioration in performance. 2) Furthermore, using just one of the retrievers (i.e., either MPC or NRE) leads to underperforms the case when both retrievers are used, showing that the two retrievers are complementary to each other as discussed in Section 4.2. In conclusion, we contend that both MPC and NRE retrievers should be employed to provide informative reference materials to the model, facilitating the extraction of valuable information in inorganic retrosynthesis planning. We provide further ablation studies in Appendix E.3.Sensitivity Analysis: Size of Knowledge Base. We evaluate how the size of the knowledge base affects the model performance. More precisely, from the original database, we sample various sizes of knowledge bases, such as 20%, 40%, and up to 100% (Full) of the size of the original database, and then retrieve reference materials from these sampled subsets. In Figure 4 (a), as expected, the larger the knowledge base, the more accurate the model's predictions. These findings illuminate potential avenues for further development of our model, particularly as more inorganic synthetic recipes are uncovered in the future.Sensitivity Analysis: Number of Reference Materials. Moreover, we investigate how the varying number of reference materials K affects the model performance. In Figure 4 (b), we notice that model performance improves with an increase in the number of references up to a certain point, specifically K = 3. This reaffirms the importance of incorporating precursor information from reference materials for inorganic retrosynthesis planning. However, increasing the number of reference materials beyond K = 3 does not further enhance model performance, likely due to the introduction of irrelevant or noisy precursor information that does not pertain to the target material. Thus, extracting precursor information from a suitable number of retrieved materials is essential for optimal performance.Qualitative Analysis. We present a qualitative analysis of our proposed method in retrosynthesis planning for the target material Pb 4 , when only the MPC retriever is used, the method fails to predict the entire precursor set due to insufficient extraction of precursor information from the retrieved materials. However, when the NRE retriever is used alongside the MPC retriever, our method successfully predicts the complete precursor set for the target material. This success is attributed to the NRE retriever, which complements the model by providing complementary retrieved materials whose precursors are feasible for synthesizing the target material, thereby enabling the extraction of diverse precursor information. For instance, the NRE retriever allows our method to extract precursor information from Pb 3 (PO 4 ) 2 , which contains the essential precursor PbO, a direct precursor for the target material. Due to the complementary nature of the retrievers, our method can effectively extract precursor information from informative reference materials, leading to enhanced predictions. Owing to the intricate nature of inorganic retrosynthesis, the production of inorganic materials entails a mix of numerous synthetic variables, leading to a variety of subtasks: 1) Learning favorable reaction pathways with thermodynamic variables when a target and precursor set are provided [2, 27, 35] ,2) Employing regression models to predict conditions of the reaction pathway, such as heating temperature and time [13] , and 3) Predicting possible precursor sets for a target material. In this work, we focus on the precursor selection task, which aims to predict feasible sets of precursors for synthesizing a target material [17, 18, 12] .Precursor Prediction Task. As a pioneering work, CVAE [17] generates synthesis actions and precursors through a generative model, i.e., conditional variational autoencoder [19] . However, it frequently produces chemically invalid precursors by relying on the generative model. ElemwiseRetro [18] alleviates the issue by formulating the precursor prediction task as a multi-class classification with dozens of manually created precursor templates. While it successfully suggests chemically valid precursors, it fails to incorporate knowledge from synthesis literature, which is common practice in the field of material science. More recently, inspired by the chemists' synthesis practice, He et al. [12] propose to use the precursor information of materials that are similar to the target material, which is retrieved from the related literature. This approach utilizes the precursors of the retrieved material as explicit conditions for predicting the precursors of the target material. However, such explicit utilization of precursor information can inhibit the model's capability in providing novel synthetic routes for target material. Different from the previous work, we propose to implicitly utilize precursor information of similar material rather than directly using it. 
composition-based representation learning for inorganic materials: 
 While most machine learning approaches for inorganic materials predominantly utilize structural information, in real-world material discovery, structural data is often unavailable due to the high costs of computational resources. Consequently, some ML methods opt to use compositional information of inorganic materials instead of structural details. For example, ElemNet [15] proposes to learn the representation of material with compositional information using deep neural networks (DNNs). Roost [10] learns material representation by building fully connected graphs based on the composition to model interactions between elements by graph neural networks (GNNs). Additionally, CrabNet [41] successfully applies a self-attention mechanism to the element-derived matrix to accurately predict the properties of materials. Although these methods were originally designed for predicting material properties, they can also be applied to inorganic retrosynthesis as they similarly rely on the composition information of materials.Both organic and inorganic retrosynthesis are challenging tasks that predict the synthesis of materials by breaking down the target material into simpler precursors. However, there are significant differences between organic and inorganic retrosynthesis. Organic retrosynthesis [45, 40, 7, 9, 33] deals with organic compounds, which are molecules primarily composed of elements such as carbon, hydrogen, oxygen, nitrogen, and sulfur. These compounds are represented using molecular structure graphs or SMILES strings. In contrast, inorganic retrosynthesis involves inorganic compounds, which can include a wider variety of elements, often including metals, and have structures that periodically repeat in unit cells. Another key difference lies in the use of structural information during retrosynthesis planning. Organic retrosynthesis utilizes structural information such as functional groups and reaction centers of organic compounds, which indicate the properties of a material and its reactivity with other molecules, to predict simpler molecules (precursors) into which the target molecule can be broken down. Inorganic compounds, however, have relatively unexplored generalized synthesis mechanisms compared to organic compounds, and calculating their structures is expensive. Therefore, it is challenging to directly use structural information for retrosynthesis planning. Instead, inorganic retrosynthesis [12, 17, 18] often relies solely on the chemical composition of the materials, distinguishing it from organic retrosynthesis. 
limitations: 
 We aim to identify favorable precursor sets by considering the thermodynamic relationships between materials and their precursor set. However, in the actual synthesis process, the phase changes of materials are influenced by synthesis temperature, synthesis time, pressure condition and pairwise reactions between precursors. Taking these factors into account would enable more accurate precursor set predictions. Nevertheless, in situations where experimental data (such as temperature and pressure) are unavailable, we estimate the reaction energy solely from the formation energy calculated under consistent temperature and pressure conditions using a trained predictor, derived from the composition of materials. Considering multiple synthesis conditions can lead to more precise predictions of precursor sets. 
conclusion: 
 In this study, we introduce Retrieval-Retro, a novel method for inorganic retrosynthesis planning by extracting the precursor information of retrieved reference material implicitly. To do so, we employ various attention layers that enhance and extract the information from the reference material. Moreover, we design a neural reaction energy (NRE) retriever that provides complementary reference material to the MPC retriever, allowing Retrieval-Retro to integrate precursor information from a broader range of reference materials through domain expertise.  
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We describe the positive & negative societal impact in SectionF.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Our work don't utilize any pretrained language models, generative models and any scraped datasets. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: All copyrights reserved by the author(s), and all materials related to the paper will adhere to the copyright policies of NeurIPS. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: In this section, we provide implementation details of Retrieval-Retro.As a composition graph encoder, we mainly consider graph network [3] , which is the generalized version of various graph neural networks. Building on prior research, our graph neural networks are divided into two components: the encoder and the processor. The encoder is responsible for learning the initial representation of elements, while the processor manages message passing between elements. To describe this more formally, for an element e i and the edge a i,j connecting element e i to element e j , the node encoder ϕ node and the edge encoder ϕ edge generate initial representations of the element as follows:EQUATIONwhere e i is the initial feature of element i and a ij is the concatenated feature of element i and j, i.e., a ij = (e i ||e j ). Using the initial representations of elements and edges, the processor is designed to facilitate message passing among the elements and to update the representations of both elements and edges in the following manner:EQUATIONwhere E represents element set comprising the material, and ψ is a two-layer MLP with non-linearity. Note that we use three message passing layers within the processor, i.e., l = 0, . . . , 2.Model Training. Our method is implemented on Python 3.8.13, and Torch-geometric 2.0.4. In all our experiments, we use the AdamW optimizer for model optimization. We train the model for 500 epochs across all tasks, while the model is early stopped if there is no improvement in the best validation Top-5 Accuracy for 30 consecutive epochs. All experiments are conducted on a 48 GB NVIDIA RTX A6000.Hyperparameters. We have detailed the hyperparameter specifications in the table 5. For Retrieval-Retro, we adjust the hyperparameters within specific ranges as follows: number of message passing layers in GNN L ′ in {2, 3}, number of cross-attention layers C in {1,2}, size of hidden dimension D in {256}, number of self-attention layers S in {1,2}, learning rate η in {0.0001, 0.0005, 0.001},batch size B in {32, 64, 128} and number of retrieved materials K in {1,2,3,4,5,6}. We present the test performance based on the best results obtained on the validation set. In this section, we provide further details on the dataset used for experiments.• Following previous study [12] , we use 33,343 inorganic solid-state synthesis recipes extracted from 24,304 materials science papers [20] obtained from the github repository2 of previous work [12] . Following the preprocessing step of previous work, we obtain a total number of 28,598 target materials, which have a diverse number of possible precursor sets as shown in Table 6 . In other words, a single target material can have multiple corresponding ground-truth precursor sets. Furthermore, in our experiments, we exclude materials from the validation and test sets that contain precursors not present in the training set, as the classifier would not be trained on those absent precursors. Consequently, for the year split, we use 24,034 entries for the training set, 1,842 for the validation set, and 2,558 for the test set. For the random split, the number of target materials varies across five different trials due to the differing compositions of the training, validation, and test sets in each trial. In Section 3.1, we propose to pre-train the Neural Reaction Energy (NRE) retriever with density functional theory (DFT) calculated formation energy and then fine-tune to experimental data.• For DFT-calculated data, we use Materials Project [14] database3 , which is an openly accessible database that provides various material properties calculated using DFT. From the database, we have collected 80,162 unique compositions along with their respective formation energies. It is important to note that when a composition is associated with multiple structures and formation energies, we only consider the lowest formation energy for the material, as it is the most likely to exist. • For experimental data [32] , we download experimental formation energy data from previous work's repository [16] 4 , which aim to develop robust material property prediction models via transfer learning. We then filter the data down to 1,637 entries using the same preprocessing methods as those applied to DFT-calculated data.• For calculating the Gibbs free energy, we use the formation energy from the Materials Project that we used is calculated at 0 K and 0 atm using DFT and the experimental formation energy is taken from [16] , which reports measurements at 298.15 K and 1 atm.In this section, we provide detailed explanations of the baseline methods compared in Section 4. We first provide details on the two previous inorganic retrosynthesis planning methods as follows:• He et al. [12] introduces a new approach to inorganic retrosynthesis planning by retrieving reference materials that share similar properties with the target material. Initially, it proposes using a masked precursor completion (MPC) retriever, described in detail in Section 3.1, which depends solely on the composition vector x of the inorganic material. • ElemwiseRetro [18] introduces a method for inorganic retrosynthesis planning that represents the target material as a fully connected composition graph and predicts the likelihood of various precursor sets from the available precursor templates.Furthermore, as outlined in Section 5.2, there are existing studies that develop material representations based on the composition information of inorganic materials. Although these studies were originally aimed at predicting material properties, we have adapted the prediction heads to make them suitable for inorganic retrosynthesis planning. Here, we provide details on the methods as follows:• Roost [10] suggests employing GNNs to learn representations of inorganic materials by representing their composition as a fully connected graph, with nodes representing the unique elements within the composition. This enables the model to explore the complex relationships between the constituent elements, thus capturing physically significant properties and interactions.• CrabNet [41] proposes to model the complex interaction between constituent elements with a Transformer self-attention mechanism [39] to adaptively learn the representation of elements based on their chemical environment.In addition to existing studies, we introduce further baseline models that can support our claims as described below:• Composition MLP aims to predict the set of precursors based on the composition vector x of the inorganic material as follows:EQUATIONThe sole distinction between Composition MLP and He et al. [12] is whether the model retrieves reference materials, highlighting the effectiveness of the retrieval mechanism in inorganic retrosynthesis planning.• Graph Network [3] aims to predict the set of precursors based on the composition graph G of the inorganic material as follows:EQUATION• Graph Network + MPC improves upon He et al. [12] by replacing its material encoder, originally an MLP that processed the composition vector x, with a graph network. Since the primary distinction between Graph Network and Graph Network + MPC is whether the model retrieves reference materials, comparing these methods allows us to evaluate the effectiveness of using reference materials in inorganic retrosynthesis planning. Furthermore, given that Graph Network serves as the backbone architecture for Retrieval-Retro, comparing Graph Network + MPC with Retrieval-Retro enables the assessment of the effectiveness of the implicit fusion and NRE retriever.We use Top-K exact match accuracy, Macro recall, and Micro Recall for evaluating the capability of Retrieval-Retro in the inorganic retrosynthesis task. To begin evaluation, we first define the number of precursors of each material. Following many multi-label classification works [5, 47] , the precursor labels are considered to be positive if their probabilities ŷi are greater than 0.5. We set the number of positive precursor labels, L as the number of precursor that each material has.Top-K exact match accuracy. For Top-K exact match accuracy, we select K sets which has L precursors, where each set is chosen based on the highest product of probabilities of its L precursors. Then, for each of the K sets, if there exists a set that exactly matches the correct precursor set, it is scored as 1; otherwise, it is scored as 0. The average is then calculated over all test materials.Micro and Macro Recall. Note that a material can have a varied number of possible precursor sets, as shown in Table 6 . We use Micro and Macro Recall to evaluate such cases. We select the same number of precursor sets as the material has in the same way as we evaluate for Top-K exact match accuracy. We then calculated the macro recall by comparing each obtained set with the correct sets for each material. If a set matched exactly, it was scored as 1; otherwise, it was scored as 0. These scores were averaged for each test material. For micro recall, we calculated the overall average across all test materials.Since Retrieval-Retro is agnostic to the various composition graph encoder architectures, we validate the effectiveness of Retrieval-Retro within a variety of GNN architectures. In Figure 5 , we observe performance improvements across all GNN architectures tested, confirming that Retrieval-Retro framework can consistently improve vanilla GNN architecture for inorganic retrosynthesis planning. As described in Section 3.1, we develop a composition-based formation energy predictor tailored for experimental data. To do so, we initially pre-train the GNN predictor on extensive DFT-calculated data [14] and then fine-tune on experimental formation energy data [32] . In this section, we demonstrate whether the pre-train and fine-tune strategy is effective in predicting formation energies. To do so, we divide the entire DFT dataset into three parts: 80% for training, 10% for validation, and 10% for testing. The model is trained over 1,000 epochs, employing early stopping if there is no improvement in the validation loss for 50 consecutive epochs. Similarly, we partition the experimental dataset into 80/10/10% splits, following the same training approach as for the DFT data. While our model begins training experimental data using pre-trained checkpoints from a model trained on DFT data, we also conduct comparisons with a model trained exclusively on experimental data (Ablation in Table 7 ). Table 7 shows that pre-training with a DFT-calculated dataset enhances model performance, indicating that the NRE retriever efficiently retrieves reference materials related to formation energies. We conduct an ablation study on the self-attention layer in Retrieval-Retro. By removing the self-attention layer, Retrieval-Retro extracts implicit precursor information by integrating the representation of the target material without the enhanced reference materials through a crossattention mechanism. As shown in Table 8 , this results in a noticeable decline in model performance, underscoring the importance of extracting precursor information from the material's enhanced representation via self-attention. However, even with just the cross-attention mechanism, Retrieval-Retro still outperforms the basic Graph Network. In this section, we provide time and inference analysis to verify the efficiency of Retrieval-Retro , as shown in Table 9 . We observe that methods using retrievers have slightly longer training times compared to those that do not use retrievers. However, since we preprocess the materials to retrieve using a pretrained retriever prior to training, the training time for the model utilizing retrievers remains manageable. Additionally, Roost and CrabNet exhibit significantly longer training times compared to other methods, which can be attributed to their distinctive modeling interaction mechanisms. Justification: We specify the GPU requirements for conducting the experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).Justification: All content and results in this paper adhere to the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).https://github.com/CederGroupHub/SynthesisSimilarityhttps://materialsproject.org/https://github.com/wolverton-research-group/qmpy/blob/master/qmpy/data/ thermodata/ssub.datCalculate the formation energy (H) of the target material in the dataset and the precursor set of all materials in the knowledge base (KB) using a fine-tuned GNN. // Calculate ∆HCalculate ∆G following ∆G ≈ ∆H = HT arget -HP recursor set // Calculate ∆GRetrieve K materials with most negative ∆G from KB per target material, then save the retrieved material sets. // Construct Retrieved Material Sets 


Decision: This paper introduces RetroPLEX, an inorganic retrosynthesis planning method that implicitly extracts precursor information from retrieved reference materials. The authors utilize attention layers to extract relevant information and design a neural reaction energy predictor to estimate the Gibbs free energy of chemical reactions. Extensive experiments demonstrate the effectiveness of implicit precursor extraction and the NRE predictor in discovering novel synthesis routes.

The reviewers found this work universally interesting and timely, given the growing capabilities in exploring material synthesis. Despite some minor concerns, this is a strong paper that I recommend for acceptance to NeurIPS.

Reviews: review: paper_summary: paper_summary: The paper introduces RetroPLEX, a method for inorganic retrosynthesis planning. It extracts precursor information from retrieved reference materials implicitly. The authors use attention layers to extract information from the reference material and design a neural reaction energy (NRE) retriever to provide complementary reference materials. Extensive experiments demonstrate the effectiveness of implicit extraction of precursor information and NRE retriever in discovering novel synthesis recipes.
strength_weakness: strength_weakness: No specific complaint. The work is highly praised for its comprehensiveness, thoroughness, and originality. It is supported by ample evidence that attests to its quality. The following section will include questions related to the work.
questions: questions: How does RetroPLEX handle situations where there are multiple possible synthesis routes for a target material?

How do the authors plan to address the potential issue of overfitting, given the complexity of the model?
limitations: limitations: The authors acknowledge limitations of their work, such as the importance of incorporating precursor information from a broader range of reference materials.
paper_summary: paper_summary: The authors address the domain of inorganic retrosynthesis by better leveraging existing inorganic retrosynthesis data. They employ attention learning techniques to establish relationships between chemical formulas and precursor formulas. Additionally, they utilize a neural reaction energy predictor to forecast the Gibbs free energy of chemical reactions, thereby refining the candidate list. The integration of these two models significantly enhances the accuracy of the prediction results. This approach has strong applications in the exploration of material synthesis.
strength_weakness: strength_weakness: 1. Chemical formula representation issues, detailed in Question 4.
2. Limited improvement over random selection, detailed in Question 5.
questions: questions: 1. The Gibbs free energy data used by the authors is used to train a prediction model, and the authors fine-tuned the model to align with experimental values. However, Gibbs free energy depends on specific conditions such as temperature and pressure during experiments or calculations. It's unclear what the specific conditions of reactions were during this transfer process.

2. In section 3.1, the authors mentioned "The overall training procedure of MPC retriever is in Figure 1 (a)." It seems this should actually refer to Figure 2.

3. In Table 8, under Qualitative Analysis, the authors predict the precursors for Na3Dy(PO4)2 using MPC to find a similar compound Na3Y(PO3)4. However, this compound cannot be found in online materials databases. Moreover, the claimed corresponding precursor sets do not seem capable of synthesizing Na3Y(PO3)4. Please provide the data source and specific synthesis literature for this compound.

4. The authors use chemical formulas as input to their model. However, in practical synthesis tasks, different conditions can yield different structures with the same chemical formula. The authors should clarify how they addressed the issue of duplicate chemical formulas during the data preprocessing stage.

5. In section 4.3, the authors note that "reference materials are randomly retrieved without using trained retrievers" and still achieve a top-1 accuracy of 58%. This high accuracy is surprising and raises questions about the effectiveness of the trained retrievers. The authors should explain why random selection of reference materials performs nearly as well as the trained retrievers and discuss the underlying reasons for this result.
limitations: limitations: The authors use Gibbs free energy to evaluate the synthesis of reference materials. However, predicting Gibbs free energy can be challenging, especially when only the chemical formula is available without specific structures, synthesis conditions, or information about gas release during synthesis. This limitation suggests an area for improvement in making energy predictions more accurate.

Additionally, the method of synthesizing inorganic materials—whether through heating and calcination, microwave, or other techniques—can also affect the choice of precursor compounds. Incorporating these synthesis methods and their specific impacts on precursor selection would further enhance the model's accuracy and applicability. We hope the authors can consider these factors in future improvements to their approach.
paper_summary: paper_summary: The manuscript presents a approach, RetroPLEX, for inorganic retrosynthesis planning. The authors propose RetroPLEX, a method that implicitly extracts precursor information from reference materials using attention layers. Additionally, they incorporate domain expertise by considering the thermodynamic relationships between target materials and potential precursors. The manuscript presents a approach, RetroPLEX, for inorganic retrosynthesis planning. The authors propose RetroPLEX, a method that implicitly extracts precursor information from reference materials using attention layers. Additionally, they incorporate domain expertise by considering the thermodynamic relationships between target materials and potential precursors.
strength_weakness: strength_weakness: 1. The method is pretty complex, including two retrievers, MPC and NRE. The complex structure, while powerful, could pose challenges in terms of computational efficiency and scalability, especially when deployed in resource-limited settings.

2. The manuscript lacks clarity in several areas. For example, the explanation of the NRE retriever's training process is unclear.
questions: questions: 1.  Why were the MPC and NRE retrievers chosen? How do they complement each other?

2. Given the apparent complexity of the proposed model, what are the computational requirements? Can the authors provide a detailed analysis of the time and resource requirements for training and using RetroPLEX? 

3. How does the model's performance scale with increasing dataset sizes or computation complexity?

4. Can the authors provide qualitative examples of the importance of each retriever? For example, provide examples where each retriever plays a crucial role in identifying relevant reference materials or extracting precursor information.
limitations: limitations: The authors have discussed the limitations.
paper_summary: paper_summary: This paper lies in the domain of AI for chemistry, and this paper proposes RetroPLEX for inorganic retrosynthesis planning. The proposed approach is comprised of two components: masked precursor completion retriever and neural reaction energy retriever.
strength_weakness: strength_weakness: The model training subsection has not detailedly elaborated on the training process of the proposed approach. The authors are encouraged to provide the pseudocode of the proposed approach.
questions: questions: Can the proposed approach provide some insights into the AI methods for other domains, e.g., robotics and mixed integer linear programming?
limitations: limitations: The authors have discussed the potential limitations in the Appendix.
paper_summary: paper_summary: I'm not an expert in this field. I'm familiar with organic retrosynthesis prediction but not familiar with inorganic retrosynthesis planning. 

This paper first trains a retriever to determine which materials to reference. Then this paper trains a model for material selections.
strength_weakness: strength_weakness: 1. Little discussion on the difference between inorganic retrosynthesis planning and organic retrosynthesis planning.

2. Little discussion on computational complexity and the space of the material.
questions: questions: N/A
limitations: limitations: N/A

Comments: title: Gentle reminder for author reviewer discussion
Gentle reminder for author reviewer discussion
Gentle reminder for author reviewer discussion
Gentle reminder for author reviewer discussion
Gentle reminder for author reviewer discussion
comment: I acknowledge the response from authors and decide to maintain my score.
I acknowledge the response from authors and decide to maintain my score.
We appreciate the reviewer recognizing our efforts and deciding to raise the score. Thank you also for your prompt response to our rebuttal. We are sincerely grateful for this!
Thank you very much to the authors for the timely and thorough response. The new examples indeed demonstrate the role of the NRE retriever effectively. All my concerns have been addressed, and I will adjust my score to 6.
Thank you so much for your valuable feedback. I'm delighted to hear that most of the concerns, including the weaknesses you highlighted, have been addressed.

Regarding the missed precursor for synthesizing NaY(PO₃)₄, after realizing that Na₂CO₃ is indeed a necessary precursor for this synthesis, we reviewed the code and dataset preprocessing procedures. We discovered that during the extraction of precursor information from paper [1], Na₂CO₃ was omitted due to an issue within the dataset itself, as you speculated, which led to the factual error. As you pointed out, such errors should not appear in the manuscript, and thanks to your thorough review, we were able to identify and correct this issue. Moving forward, we will take extra care in preparing the manuscript.

Additionally, we are providing another qualitative analysis. The table below shows the results of our model using both MPC and NRE retrievers for $Pb_9[Li_2(P_2O_7)_
2(P_4O_{13})_2]$. 

We were able to retrieve $Li_{2}CO_{3}$ and  $NH_{4}H_{2}PO_{4}$ through the MPC retriever, and $NH_{4}H_{2}PO_{4}$ and $PbO$ were identified by using the NRE retriever, which retrieved $Pb_{3}(PO_{4})_{2}$. This demonstrates how the NRE retriever and MPC complement each other to enhance prediction accuracy.

Moreover, we have included the DOI for each material, sourced directly from the raw data, below the table.

Thank you once again for your valuable feedback. I hope that this addresses your concerns.




| **Model**         | **Retriever** | **Retrieved Material** | **Corresponding Precursor Sets**     | **Predicted Precursor Set(Output)**               | 
|-------------------|---------------|------------------------|--------------------------------------|-------------------------------------------|
| MPC + NRE         | MPC           | $LiNaPbPO$         | {$Li_{2}CO_{3}$, $H_{3}PO_{4}$,$Na_{2}CO_{3}$, $Pb_{3}O_{4}$ }  | {$NH_{4}H_{2}PO_{4},Li_{2}CO_{3}, PbO $} | 
| (RetroPLEX)       | MPC              | $Li_{0.5}Na_{0.5}PO_3$ |{$Li_{2}CO_{3},NH_{4}H_{2}PO_{4}, NaPO_{3}$}|      
|                   | MPC              |$Li_{3}V_{1.92}Al_{0.08}(PO_{4})_{3}$ | {$Al, V_{2}O_{5}, LiH_{2}PO_{4}$}      |              
|-|-|-|-|-|                             
|                   | NRE           | $Pb_{3}(PO_{4})_{2}$     | \{$PbO, NH_{4}H_{2}PO_{4}$ } |                                           |            |
|                   |NRE               | $Li_{3}P$               | \{$P, Li$\}                     |                                           |            
|                   |NRE               | $PbP_{7}$                  | \{$P, Pb$\}                            |                                           |            


---



$Pb_9[Li_2(P_2O_7)_
2(P_4O_{13})_2]$  : 10.1039/c7dt00509a

$LiNaPbPO$ : 10.1016/s0022-3093(03)00171-6

$Li_{0.5}Na_{0.5}PO_3$ : 10.1016/s0022-3093(01)00655-x

$Li_{3}V_{1.92}Al_{0.08}(PO_{4})_{3}$ : 10.1016/j.electacta.2010.12.063

$Pb_{3}(PO_{4})_{2}$ : 10.1103/physrevb.73.024429

$Li_{3}P$ : 10.1021/cm0513379

$PbP_{7}$  : 10.1039/c4dt01539h

[1] M. El Masloumi, et al. Structure and luminescence properties of silver-doped NaY(PO3)4 crystal. Journal of Solid State Chemistry, 11(181), 2008.
Dear reviewers,

As the deadline for the discussion period approaches, could you please take a moment to confirm whether the author’s rebuttal has addressed your concerns, or at least indicate that you have reviewed it?

Best,
AC
Thank you for your thoughtful rebuttal and for addressing the concerns raised in the initial review. I appreciate the effort you put into clarifying the issues.

Regarding the discussion on the precursor compound in the Qualitative Analysis section, you mentioned that the actual example compound is NaY(PO3)4. However, after researching this compound, Na2CO3 is indeed one of the raw materials (precursors) used to synthesize NaY(PO3)4, according to the synthesis method mentioned in the literature [1]. This differs from the precursors you provided in the paper, leading to less effectiveness of NRE retriever. Although this mistake may have originated from the comparison article’s dataset, it is crucial to note that factual errors should not be present as examples in the paper. If the issue indeed stems from the dataset, I recommend considering the use of a different case study to avoid this discrepancy.

Your explanations have successfully addressed most of my other concerns, and I hope these comments will be helpful in improving your manuscript.

[1] M. El Masloumi, et al. Structure and luminescence properties of silver-doped NaY(PO3)4 crystal. Journal of Solid State Chemistry, 11(181), 2008.
We are delighted that our work has been recognized, and we sincerely appreciate your compliments and encouragement. Thank you!
Thank you for your rebuttal which addresses the questions. I remain my initial assessment of RetroPLEX as a technically sound paper with significant implications for material science, and I recommend its acceptance for publication.
Thank you once again for your valuable review and feedback. We kindly ask if you could please take a moment to confirm whether our rebuttal has adequately addressed your comments and concerns. Thank you for your consideration.
Thank you once again for your valuable review and feedback. We kindly ask if you could please take a moment to confirm whether our rebuttal has adequately addressed your comments and concerns. Thank you for your consideration.
Thank you once again for your valuable review and feedback. We kindly ask if you could please take a moment to confirm whether our rebuttal has adequately addressed your comments and concerns. Thank you for your consideration.
Thank you once again for your valuable review and feedback. We kindly ask if you could please take a moment to confirm whether our rebuttal has adequately addressed your comments and concerns. Thank you for your consideration.
Thank you once again for your valuable review and feedback. We kindly ask if you could please take a moment to confirm whether our rebuttal has adequately addressed your comments and concerns. Thank you for your consideration.