Title: Inverse Design for Text Generation with Accurate and Complex Causal Graph
Venue: NeurIPS 2025
Accepted: no
Keywords: Causal Inference

Inverse Design

Data Generation
Abstract: The development and evaluation of causal discovery methods requires large quantities of data with causal structure annotations. However, such real-world data with annotations is insufficient. Therefore, text generation with causal structure annotations serves as a critical foundational task for advancing causal discovery research. Nevertheless, existing data generation methods cannot ensure both causal structure accuracy and complexity. To address this, we apply inverse design from scientific computing to Chain-of-Thought (CoT) and propose a method named iTAG. Our method is capable of generating large quantities of text with accurate and complex causal graphs. Empirical evaluation demonstrates the substitutability of iTAG-generated data for real-world data through two experiments. First, annotation accuracy evaluation shows remarkable causal graph annotation accuracy across complexities (F1>96\%, SHD<1, SID<0.5). Second, substitutability analysis reveals strong statistical correlation between generated and real-world text across various metrics computed on existing causal discovery algorithms (Pearson=0.96, Spearman=0.94, $R^2=0.93$).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces iTAG, a method for generating text with causal graphs. iTAG integrates inverse design principles from scientific computing with Chain-of-Thought prompting of LLMs. Its workflow involves constructing parameterized causal graphs, substituting abstract nodes with real-world concepts, and transforming these concepts and their causal structures into natural language.\n\nEmpirical evaluations demonstrate iTAG's high annotation accuracy, with F1 scores exceeding 96%, SHD below 1, and SID below 0.5, outperforming baselines. Furthermore, the study shows statistical correlations (Pearson $r=0.96$, Spearman $\\rho=0.94$, $R^2=0.93$) between metrics derived from iTAG-generated data and real-world data across various causal discovery algorithms.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* The paper addresses the problem of limited real-world data with causal structure annotations for causal discovery research.\n* iTAG's ability to generate large quantities of text with accurate and complex causal graphs offers a scalable and controlled data source, potentially advancing causal discovery research.\n* The empirical evaluation is thorough, assessing annotation accuracy across complexities and data quantities.\n* The use of multiple metrics (F1, SHD, SID) provides a comprehensive assessment of graph fidelity.\n* iTAG demonstrates superior performance compared to the Davinci baseline in maintaining accuracy with increasing complexity.\n* The strong statistical correlations (Pearson $r=0.962$, Spearman $\\rho=0.927$, $R^2=0.926$) between results on generated and real-world data validate iTAG's output for evaluating causal discovery algorithms.\n* The paper is clearly written and well-organized, with intuitive explanations of the workflow.\n\nWeaknesses:\n\n* The claim of applying \"inverse design from the domain of scientific computing into LLMs by CoT\" might be an overstatement, as the implementation appears to be iterative prompting and self-correction rather than a rigorous mathematical application of inverse design principles.\n* The focus on Directed Acyclic Graphs (DAGs) rather than complete Structural Causal Models (SCMs) limits the complexity of generated causal graphs and their applicability to methods dealing with quantitative relationships or cyclic dependencies.\n* The experimental configuration for graph density is limited to a very narrow range (0.2 to 0.3), which might not represent the full spectrum of real-world textual scenarios.\n* More specific details on the LLM versions and API parameters (e.g., temperature, top\\_p, max\\_tokens) are crucial for reproducibility.\n\nQUESTIONS:\n1. Could the authors elaborate on how the inverse design paradigm is specifically and mathematically applied in their CoT loop for concept substitution and text generation? Provide a more detailed theoretical grounding or a simplified mathematical formulation that directly links their CoT implementation to the established principles of inverse design, demonstrating how \"forward analysis\" and \"backward analysis\" translate specifically within the LLM context.\n\n2. While the focus on DAGs is a reasonable starting point, could the authors discuss the potential for iTAG to be extended to generate text representing more complex causal structures, such as those involving cycles, latent variables, or quantitative effect sizes? Discuss potential pathways or theoretical considerations for expanding iTAG's capabilities to handle aspects of SCMs beyond just graph topology.\n\n3. What are the observed challenges or limitations that led to the restricted range of graph density in the current implementation? Explore the feasibility and challenges of allowing more flexible control over graph density. If there are trade-offs, discuss them.\n\n4. Could the authors provide the exact version of GPT-4o used (e.g., gpt-4o-2024-05-13 or similar date-stamped model identifiers if available)? Additionally, what specific API parameters (e.g., temperature, top\\_p, max\\_tokens) were consistently used for the LLM calls in Phase 2 and Phase 3? Include a table or section in the Appendix detailing these LLM parameters.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe goal of the proposed iTAG method is to generate a text, as shown in the red box at page 3, that describing a set of causal relations, like studying -> knowledge. The proposed iTAG method has three phases: (1) causal graph construction (output: Adjacency matrix); (2) concept substitution (output: a dictionary that assign a semantic variable name to each node in the given causal graph); and (3) generate a text description based on the output of the previous two phases.\n\n\nIn the evaluation section, the paper considers two aspects: (1) annotation accuracy across complexities; (2) the substitutability of generated data for real-world data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe paper considers the text data generation task with respect to a pre-specified causal graph. The paper has an intensive human-expert evaluation for text data, and also the empirical evaluation in multiple perspectives.\n\n\n**Weaknesses**\n\nThe presentation is not very clear, especially in the experiment setting. Please see the question part for details.\n\nThe experiments in section 4.1 are based on Davinci, a GPT-3 model released in 2022 (about three years ago). This is an unrepresentative choice for LLMs' performances, given recent advances in GPT-4.1, DeepSeek-V3, and Qwen-3 models.\n\nThe generated text is a narrative of a given causal graph, as shown in the box on page 3. About this task setting:\n- Could you further explain the technical challenges? For example, consider a template-based method with `[variable A] is a direct cause of [variable B]` for each edge, would it already clear enough to interpret the causal graph?\n- If the challenge is on allocating a set of suitable concepts for a given causal structure, i.e., the phase 2 in iTAG, then how does the evaluation in section 4 support the performance of iTAG? It seems that they are focusing on structural description ability (phase 3).\n\nQUESTIONS:\n**Evaluation 1: annotation accuracy across complexities**\n\nThe purpose of this part is to evaluate whether the generated text data reflects the expected causal structure. The paper employs 11 human annotations to annotate the causal structure reflected in each generated text from iTAG and the baseline. Then, these reflected causal graphs are compared with the expected causal graph and thus yield metrics like F1, SHD, and SID. The paper reports the changes in these metrics across different `variable quantity` and `data quantity`. \n\nMy questions here:\n1. Is my understanding of this paper correct? If not, please let me know.\n2. The paper mentions that the baseline method used is \"Davinci.\" Does this refer to the method described in **reference 24** of the paper, or is it referencing a different approach? If the answer is yes, I suggest to make it clear because \"Davinci\" is usually used to refer to an Large Language Model released by OpenAI.\n3. What is the backend LLM used by the iTAG method when you report Figures 3 and 4? Is it `Davinci`, as suggested by the context?\n4. What does the `data quantities` mean in section 4.1.3? Does it refer to the length of generated text? If so, how is the text length controlled in the generation process?\n\n\n**Evaluation 2: the substitutability of generated data for real-world data**\n\nIn this part the paper compares the causal structure extraction methods on the real-world data and synthetic data from iTAG.\n\nMy questions here:\n1. Do the real-world data and iTAG data share the same underlying causal structure? And what is the causal structure? Did I miss something?\n2. How about results across different real-world datasets? Would iTAG do well in all the topics covered by the real-world dataset?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents *iTAG* a method to generate natural text and an associated Causal Graph, using Inverse Design and LLMs.\nThe method works using a three-phases workflow:\n\n* Phase 1 uses an enhanced Erdős-Rényi algorithm to sample a random causal graph dependant on user-defined hyperparameters.\n* Phase 2 uses CoT prompting inspired by Inverse Design to iteratively label the nodes of the graph, identify errors and correct them.\n* Phase 3 uses a similar process to generate the final text from the labelled graph.\n\nThe authors evaluate iTAG by assessing the annotation accuracy of the generated data against a ground truth established via a panel of 11 human annotation experts. They report high F1 scores (>96%) and low SHD (<1) and SID (<0.5) values across varying complexities. They further test the generated data's substitutability for real-world data by comparing the performance of causal discovery algorithms, finding strong statistical correlations between the results on both datasets.\n\nSTRENGTHS AND WEAKNESSES:\n* Strength:\n\n    * The paper tackles the challenge of insufficient real-world data with causal annotations, which is a noted obstacle for advancing causal discovery research, especially given how prominent Causal Discovery from Natural Language has been with the recent development of LLMs.\n\n    * The methods display impressive results, that show the generated data allows for great performance on downstream tasks.\n\n* Weaknesses:\n   * The paper states that the method uses GPT 4o, however, in the code, Claude 3.7 Sonnet is used.\n   * It does not seem instantaneously clear to me that the \"Inverse Design\" algorithm presented in Algorithm 1is not in fact done in code, but as a guideline for the LLM, it is very minor, but I think it should be made clearer earlier.\n   * In my opinion, the fact that the LLM used in this method is closed-source and API only is a bit problematic in terms of replicability. If tomorrow Anthropic drops support for the model called in the experiments, there is no way to replicate them. Personally, since I don't use the Anthropic API, I was not able to replicate the results. And this is without mentioning the cost. This is mentionned in the NeurIPS guidelines, but I think the statement is vague enough that I can still raise this concern, that seems very important to me. Of course, if the AC has a different opinion on this matter, this point can be disreguarded.\n   * I feel like the evaluations are lacking a bit. I know this is particularly hard, but, in particular, I feel like there is no evaluation of the \"Natural Language\" quality. If this method generated bad quality text made of simple phrases \"There is an edge between A and B\", there would be no way to evaluate it, and it would still get very good results on the evaluated metrics. To be clear, I am not saying the method is doing something like this, I see the guardrails that are done in order to prevent it, but I will like this is not evaluated enough. In particular, \n   * Finally, I feel like the proposed method relies a lot on the LLM, while this seems a bit surprising to comment about, I feel like it raises a few concerns: we have no guarantees that the LLM follows the expected algorithm; the impact of the choice of LLM is underexplored in my opinion, is there a critical size of LLM that makes this method work? Do all recent state-of-the-art model perform similarly? We are starting to have a bit of literature on the limits of the comprehension of causality by LLMs, that I feel is not talked about enough in this paper.\n\nQUESTIONS:\n1.  The results with GPT-4o (or Claude Sonnet?) are very strong, which naturally leads to the question of how dependent the iTAG framework is on the specific capabilities of this top-tier model. It would be very insightful to see an ablation study showing iTAG's performance with other models—perhaps the GPT-4o-mini you mention, or one of the Claude models already used in the baselines. This would really help to disentangle the contribution of the iTAG framework from the power of the underlying LLM, and providing this analysis to demonstrate the method's robustness would significantly increase my confidence and my overall score.\n\n2.  While the paper rightly focuses on the causal accuracy of the generated graphs, I was left wondering about the linguistic quality of the text itself. To confirm it's a true substitute for real-world data, the text needs to be natural and not easily identifiable as synthetic. Have you considered an adversarial evaluation to test this? For example, seeing if a classifier or a human evaluator can distinguish between iTAG-generated text and the real-world samples from your experiments could be a very powerful way to validate its quality. Since the goal is to create a viable data substitute, providing this kind of evaluation would substantially strengthen the paper's claims, and I would be happy to raise my score accordingly.\n\n3. The paper evaluates performance on graphs with up to 10 variables, justifying this by citing annotation costs and typical real-world scenarios. This is a reasonable scope, but I'm curious about how the method itself scales to more complex cases. How does the iTAG generation process, particularly the Chain-of-Thought reasoning loop, perform when tasked with generating even larger graphs of 15 or 20 variables? I'm not suggesting a full, manually annotated evaluation, but a discussion of the method's scalability, perhaps with some generated examples and observations on whether the LLM's reasoning remains coherent, would be very valuable. Addressing how the method handles this increased complexity would improve my assessment of its potential impact.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes iTAG, a method for generating large quantities of text with accurate and complex causal structure annotations. The authors address a fundamental challenge in causal discovery research: the lack of sufficient annotated text data with complex causal structures. iTAG employs a three-phase workflow: (1) parameterized causal graph construction using an enhanced generator, (2) inverse design-based concept substitution that maps abstract graph nodes to real-world concepts via Chain-of-Thought (CoT) prompting, and (3) causal structure-preserving textual transformation that converts the concepts and relationships into natural language text. The authors evaluate iTAG through annotation accuracy experiments across different complexities and substitutability analysis comparing generated data with real-world data for causal discovery algorithm performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. **Addresses a critical need**: The paper tackles an important bottleneck in causal discovery research—the scarcity of high-quality annotated text data with complex causal structures, which is essential for advancing the field.\n\n2. **Novel methodological approach**: The application of inverse design principles from scientific computing to text generation with causal graphs is innovative and well-motivated. The three-phase workflow provides a systematic approach to the problem.\n\n**Weaknesses:**\n\n1. **Limited exploration of LLM's capabilities**: While the method incorporates iterative refinement to enable reflection and refinement patterns, it lacks other well-developed features that could enhance performance, such as integration with external knowledge sources or retrieval-augmented generation (RAG) to validate causal relationships against domain-specific literature, external search tools to verify claimed causal relationships in scientific databases, etc. Given there are numerous studies and open-source projects exploring this direction, the authors are encouraged to explore more in this field to further justify the utility of their methods.\n\n2. **Training and fine-tuning limitations**: The method relies entirely on pre-trained LLMs without exploring domain-specific fine-tuning for causal reasoning; Missing analysis of how different base models (beyond GPT-4o) might affect generation quality.\n\n3. **Evaluation concerns**: The ground truth construction relies heavily on human annotators (11 experts), which may introduce bias and subjectivity, and also limits the reproducibility of the study; Automated or hybrid methods are encouraged to better benefit the community. Besides, only one baseline (Davinci) is compared due to stated limitations of other methods, making competitive evaluation insufficient.\n\n4. **Methodological limitations**: The method focuses only on DAGs rather than complete structural causal models (SCMs). Also, there is heavy reliance on LLM capabilities without addressing potential hallucination through external validation.\n\nQUESTIONS:\n1. **Counterfactual verification implementation**: The paper mentions Pearl's Level 3 causal inference for counterfactual verification in Algorithm 1, but the specific implementation details are unclear. How exactly do you operationalize counterfactual reasoning within the LLM framework? Can you provide concrete examples of how the system determines whether \"effect B would still occur in the same manner if cause A had not occurred\"?\n\n2. **Graph density constraints and real-world applicability**: You constrain graph density to 0.2-0.3 based on observations from real-world data. However, recent work by Ban et al. (2023) suggests that LLMs can handle more complex causal structures. How would your method scale to denser graphs, and what modifications would be needed for domains with naturally higher causal connectivity?\n\n3. **Comparison with existing causal text generation**: While you mention that other existing generation work's \"predefined components cannot meet multi-theme generation requirements,\" could you provide a more detailed analysis of why template-based approaches or other structured generation methods are insufficient? Have you considered hybrid approaches that combine your inverse design with existing structured methods?\n\n4. **Integration with established causal discovery frameworks**: How does iTAG-generated data perform when used with constraint-based methods like PC algorithm or FCI that are designed to handle latent confounders? Given that your method focuses on DAGs, how would it extend to scenarios requiring causal discovery in the presence of hidden variables?\n\n**References**\nBan, T., Chen, L., Lyu, D., Wang, X., & Chen, H. (2023). Causal Structure Learning Supervised by Large Language Model. *arXiv preprint arXiv:2311.11689*.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}