Venue: ICLR 2023 Conference

Reviews:

final_score: rating: null
review_content: I raised my score again because of two reasons.
1. The authors greatly improved their draft, which is also recognized by other reviewers.
2. The negative-side reviewer does not post new comments.
review_title: I raised my score to a 10.

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a principled algorithm for incorporating symbolic information related to text generation using an LLM. The idea is justified using a causal analysis, so the algorithm is motivated by the front door criteria. The external information is extracted by entity extraction and extracting related information from ConceptNet. The results are concatenated to the prompt and interpreted as conditioning. The human evaluation and metrics indicate the method leads to improvement in three different LLMs.

strength_and_weaknesses: Strengths
- Principled motivation using causality
- Concrete technical solutions challenges for integrating with the selected source of information
- Ablation study shows the impact is not trivial.
- Idea is novel as far as I know.
	- For instance, this survey appeared after the submission. It does not mention work as specific as this submission: Feder, Amir, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, et al. “Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond.” Transactions of the Association for Computational Linguistics 10 (October 2022): 1138–58. https://doi.org/10.1162/tacl_a_00511.

Weaknesses
- It's not clear which is the current causal model used at each point. (See question below)
- The paper might be assuming P_i is the only path, but the existence of other paths to be blocked it's not discussed. 
- The argument loses a bit in the adaptation plus other decisions on what to retrieve.


clarity,_quality,_novelty_and_reproducibility: My main concern is that the causal analysis is both very thorough but it's also confusing. After trying the causal models in the tool https://causalfusion.net/app, I'm obtaining different estimands. The reason seems to be that the model in Fig 2 is updated after each iteration, fixing the values of the previous variables. That makes it hard to understand some of the statements. For instance, eq (1 and 10) says
P(Pi=p | do(S_{i-1})) = P(Pi=p | do(S_{i-1}))
However, using that tool I obtain something like
P(P_3 ∣ do(S_2))=∑_{P_2​,T} P(P_3​∣S_2,P_2,T)P(P_2,T)
P(P_2 ∣ do(S_1))=∑_{P_2​,T} P(P_3​∣S_2,P_2,T)P(P_2,T)
but if I fix T and P_{i-1}, that is what would happen in greedy decoding with an LLM, then I should obtain P(Pi=p | do(S_{i-1})).

Something similar happens with Eq (9)

However, this makes the analysis hard to follow as the new causal models are not referred in the equations.
There is also a comment on P_{i-1} being copied into P_i that makes things more complicated.

Question:
- Am I right about these concerns?

**Let's keep in mind that the algorithm is just an implementation.**
**In principle, what we want is the prediction of all the do() compounded.**

Perhaps the appendix is the place to clarify this point, explaining:
- what's exactly the new causal graph after each iteration.
- why does it make sense to simplify the graph.
- Clarify which causal graph is related to each equation.

More questions:
- does the causal analysis holds given de "adaption" that was necessary? I'd like to see a more clear causal criticism of that situation.
- Does the use of the front criterion holds given that there could exist other words related to the task?
- Where would this break? The results are just positive but there is not a detailed discussion on limitations.

I think the work is interesting and solid.
My only concern is the one I just mentioned on the clarity of the causal analysis.


summary_of_the_review: The paper provides a causality-justified algorithm for retrieving information about a task. Although some points seem to break the causal argument, I suspect this is indeed the explanation for better performance. Some aspects of the causal analysis are hard to follow.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work proposes an approach for procedural planning with LLMs and casual models. The approach first builds a commonsense casual model from an external knowledge base with adjustments. Then, given a query, it builds a task-relevant subgraph, which is provided as a procedural prompt. Finally, this is translated to admissible actions for the LLM to plan. This is demonstrated on two large planning tasks, RobotHow and WIkiHow, and shown to outperform baseline planners. 
strength_and_weaknesses: 
The paper is interesting and the topic is very relevant. Planning from LLMs is promising and grounding them in common sense as well as admissible scenes is a core challenge in this area.
The approach of building a large knowledge base and leveraging within an LLM is well founded.
The two large planning datasets are large and good environments to test in. The results show improvements across the board compared to baselines.
 
The paper however has a few areas for improvement. 

(1) The clarity could be improved. Some sections like Section 3 are quite dense and difficult to parse, particularly Section 3.1. Perhaps an earlier overview could help clarify or explicit running examples. Section 3.1 could use it’s own algorithm block and potentially a zoomed in figure of the computation. Figure 3, though a nice overview, is very dense.

(2) Though the performance of PLAN is stronger than baselines, it is a smaller improvement than I would think given the additional complexity and also somewhat difficult to judge how large of an improvement it is. In many of the metrics it seems PLAN outperforms by a few percentage points or in voting it wins 50% of the time. While I acknowledge that this shows that PLAN is performing better, it isn’t clear from these results that it is worth the vast additional complexity compared to baselines. The authors should add an additional metric similar to executability in Huang et al., showing the actual success rate of these plans, as this is the ultimate metric we care about. A few related questions:
* I would be interested to learn more about what the main failure modes are.
* Some results, such as Table 1, seem to outperform baselines with larger models. I’m surprised by this as I would think your approach would be particularly important to add structure when LLM’s are more inaccurate. Do you have any intuition on why this might be?

Minor notes:
* “The Termination Condition is either reaching the max step t or the matching score is below threshold θ. “ Instead of thresholding, one can compare to an end of statement token’s probability.
* Mention in the intro where the common sense external knowledge comes from (though I know it is in Figure 1).
* How do you extract entity names from the task name?
* “show that pre-trained knowledge (D) in LLMs confounds” What is D here?
* “We describe the implementation of such frontdoor adjustment in Section 3.1.” but section 3.1 is 2 pages long, be more specific.
* Table 3 bolds 0.433 in the button right, though GPT3 + Chain outperforms it with 0.471
* “PLAN surpasses powerful baselines (Chain of Thoughts (Wei et al., 2022) and Zero-shot Planner (Huang et al.)) by large margins on both the original and the counterfactual samples.” Are these baselines powerful? They don’t have external information except their prompts.

clarity,_quality,_novelty_and_reproducibility: See strengths and weaknesses.
summary_of_the_review: The paper presents and interesting and novel approach to an impactful area of research. PLAN also shows performance on two large and varied environments. However, the paper is quite dense and could be improved in clarity. The results show PLAN outperforms baselines, but by a small margin given the additional complexity, and the results do not include success rate metrics that may be most crucial for understanding the gains in performance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Summary:
- Existing large language models (LLMs)  require manual exemplars to acquire procedural planning knowledge in the zero-shot setting. 
- The paper proposed a neuro-symbolic procedural PLANner (PLAN) with commonsense-infused prompts elicited from an external knowledge base (i.e., ConceptNet) to solve the pure-language-based procedural planning problem in a zero-shot manner. 
- Human and automatic evaluations on WikiHow and RobotHow show the superiority of the proposed PLAN over the prior methods on procedural planning. 

strength_and_weaknesses: Strength:
1. Solid and reasonable idea: Based on the observation that due to potentially biased pre-training data, pre-trained knowledge in LLMs may confound the model to make wrong decisions when asking the model to generate a procedural plan of a task, the authors proposed to apply the frontdoor adjustment in causality to build a mediator and implemented it as a commonsense-infused prompt. The prompt obtained from their neuro-symbolic-based method allows the LLM to attend to the causal entities instead of the highly correlated ones for the next step generation. 
2. Strong performance: the proposed method PLAN outperformed two recent SOTA methods LLMaP (“Language models as zero- shot planners”) and Chain (“Chain of thought prompting”) statistically significantly. 
3. Quite thorough experimental study and analysis: for the experiments, the authors utilized several evaluation methods including human evaluations, two datasets, and several pre-trained LLMs. 
4. The paper is well written and organized.

Weaknesses:
1. Discussion on the failure cases is currently missing. In addition, the generated procedural plan of the proposed method was shown, but it would be interesting and useful for readers to see the exact intermediate outputs of the proposed framework given an actual task from the evaluation dataset, e.g., $G_s$, $P_G$, $\hat{P}_G$, etc. In this way, readers may have a better understanding of the current capabilities of each module of the proposed framework.

2. Some minor issues, e.g.:
(1) “Note that in the path T →D→Si ←Pi, Si is a collider…” (second line of the “Stage1” paragraph), should it be “T←D”?
(2) It would be good to refer readers to the appendix for definitions of backdoor path, frontdoor adjustment, etc.
(3) How is the Symbolic Rule Set $R$ obtained? 
(4) One more ablation experiment: what if removing the first Translation $LM_T$ and replacing $\hat{P}_G$ with $P_G$?
(5) Suggesting one relevant work: https://arxiv.org/abs/2205.11916 (manual exemplars not required).


clarity,_quality,_novelty_and_reproducibility: - Clarity: some descriptions in the paper are currently not clear enough.
- Quality: the quality of the paper is good.
- Novelty: the contributions of the paper are novel.
- Reproducibility: Some additional details are required for one to reproduce the results in the paper. Partial code was provided but not the executable full code.

summary_of_the_review: The paper proposed to generate commonsense-infused prompts elicited from an external knowledge base (i.e., ConceptNet) to allow LLMs to solve the pure-language-based procedural planning problem in a zero-shot manner. The key idea of this paper is novel and solid. The proposed method has a strong performance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work shows that by adding a dash symbolic reasoning to a neural model, it shows better performance w.r.t consistency and generalization. It is unclear to me how much engineering efforts are required to add these symbolic reasonings. It appears their approach is general, using a common External Knowledge Base, but the writing is confusing and I cannot tease out the details.
strength_and_weaknesses: strength : the proposed approach works. they conduct a user study where they ask crowd workers to rate which agent, one with symbolic reasoning and one without, performed better on a task, and the crowd workers preferred the agent with symbolic reasoning. this result is solid and shows evidence of the proposed approach.

weakness : 

the proposed method is may not be entirely novel. people have been adding symbolic reasoning to neural models for awhile, and the finding has always been : "If we can successfully 'hack' the underlying DSL that represented the set of tasks, adding symbolic reasoning would perform well". For instance, these works tend to follow the steps of: 1) identify a set of tasks that would be easily represented with symbolic execution, and 2) devote significant engineering efforts to construct the DSL and a symbolic interpreter to help the neural/llm model make better inferences/plans. 

this work would be of significant contribution if it can show that steps 1) and 2) can be avoided by using a generic external knowledge base (as shown in figure 3). however the writing is too confusing I cannot be sure if that is the case or not.
clarity,_quality,_novelty_and_reproducibility: clarity : poor. And this is a huge problem because the writing prevented me from judging the work clearly.

In the introduction there's this block of text that reads

"adjustment (Hu & Li, 2021; Weber et al., 2020; Yue et al., 2020) are not applicable in our SCM. Instead, we build a mediator and implement it as a commonsense-infused prompt. Through the mediator, we can identify causal effects among goals and steps by investigating the indirect effect from the goals, which is essentially the frontdoor adjustment in causality (Pearl, 2009)." 

What is it even saying? I have zero clue. What is a mediator? What is a commonsense-infused prompt? What are these "indirect affect from the goals" mean? What is "essentially the frontdoor adjustment" mean? 

These are highly technical terms that mean very little unless explicitly defined. The reader tends to look for easy metaphors and intuitions on why your approach should work, and why intuitively it should work well. This passage sounds intuitive, yet it uses words that nobody know what they mean (yet), and ended up being just gibberish.

This confusion continued for the rest of the paper, making it hard for me to judge if it is worthwhile.

A re-write of the intro section is warranted, with a concrete example explaining why the proposed approach should work well, without the jargons.

I highly recommend the authors ask people outside of their immediate project -- walk down the hallway a few offices and knock on some doors -- ask these people to read the paper and give feedbacks, and adjust the paper based on what was confusing. 

quality : unclear / potentially good. The human-evaluation is clearly stated, and I can feel confident in saying "the approach is performing better than the baseline". However, I would also like to make an assessment on "is this approach general? or is it domain-specific and hacky?", this is hard to judge as the work seemed very complex with many moving parts (in figure 3 there's 5 stages), and the writing isn't clear.

novelty : unclear / potentially good. same reason as above. If this work is generalizable to different domains with very little tweaking, then it definitely has merits most prior works that brings symbolic reasonings into neural models heavily rely on a DSL, i.e. a domain SPECIFIC language, and isn't really generalizable. 
summary_of_the_review: Overall, the paper is difficult to read, and as a result I cannot judge it properly.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: This paper presents a procedure planning model which enforces the logical order. The authors claim that they can parse the goal to subgraphs and translate the knowledge into admissible knowledge. The proposed model also learns casual relationships via an SCM for procedural planning. Experimental results show the proposed model surpasses the original baselines by a large margin.
strength_and_weaknesses: ## Strengths:
1. The procedural planning task is interesting and fundamental. Through procedural planning, we can understand how the neural networks understand the task structure and generate the corresponding plans. The counteract cases are also interesting and touch the core of AI.
2. The source code is provided, so this work might be reproducible.
3. The idea of using causal models to prevent relational biases and keeping order in LLMs is very important and is introduced with good examples.
4. The improvements in the experiments seem to be large. I like the illustration of the results introduced in Table 5. Sadly, some results are omitted in Table 5. On the contrary, I suggest the paper should have a large table showing the concrete procedural results to help understand each part of the model. Try always to be concrete and precise.

## Weaknesses:
I am confused by this paper's methodology a lot of times. In general, this paper could be challenged in the following positions:

1. Probably, the proposed model is not technically sound. In fact, many of the components are not well-defined, and the system looks too complicated. The overall model is not ablated well and is full of tricks.
2. The proposed model seems to borrow a lot of big concepts, such as neural-symbolic or casual models. However, the specific technical contributions are not very clear, and many problems remain unaddressed. In general, the system looks too complicated. This is not a good paper. A good paper should stick to one major point and show the advantage of this point over previous baselines.
3. Human evaluations are not very objective and cannot be easily reproduced. I fail to see a clear motivation for using expensive human evaluations. Instead, showing a lot of generated procedures is far more beneficial than showing a lot of numbers. 

The specific comments are:

1. In the task definition, what are the actions, and what are the object sets? I think in real natural language, the set of actions (described in natural language) or objects can be extremely large. How can the formulation handle this? I did not see the specific usage of these variables. Probably the problem is over-formulated.
2.  The authors use the same variable D for task D_T and confounding variable D. This is hard to decode.
3.  I can hardly understand the implementation of the SCM. What are the input and output of the SCM? Is there any replacement (e.g. standard transformers) for this module? What is the sample output of the SCM? Is there any reference to the SCM? Take Figure 2 as an example; how does the SCM forward the reasoning path? How does the SCM encode the action and object? How does the SCM produce the output? These problems are still unclear to me.
4. Why does procedural planning need so many steps (five-stage pipelines)? I see that motivation is to fuse different motivations (task definition, previous steps, and external knowledge). If so, why not encode this information in parallel so that the fused (e.g. concatenation) information can be used in downstream tasks?
5. Algorithm 1 is a waste of space, and it is better to replace it with a figure (or make figure 3 better).
6. Many concrete designs, such as the symbolic executors, are missing. It seems that the symbolic executors are trivial and can be easily learned. In other words, I do not think the "neural-symbolic" stuff in the title plays an important role in this system. In fact, what is neural symbolic in this line of work? Why is it important? How to connect the big concept with the experimental results with the LLMs? These questions are far more important to show than the ad-hoc model designs.
clarity,_quality,_novelty_and_reproducibility: As we have discussed in the prior section, reproducibility is good.
In overall, the novelty should be good because introducing casual models and neural symbolic methods should still be novel.
The major limitation is the clarity and the paper presentation. I tend to the acceptance part considering the good directions and good ideas. However, if I do not see a major revision in the rebuttal phase in the writing parts clarifying the related concerns, I might decide to reject this paper.
summary_of_the_review: Addressing an important task with promising novel technics. Writing needs to be improved, and more insights should be given. 
-------------------------------------
After reading the responses, I think most of my concerns are addressed. Therefore I raised my score.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a new pipeline for symbolic information related to text generation using an LLM which his used for procedural planning. The paper show more detailed and natural generated plans and is evaluated with different large language models.

The paper presents an interesting and principled idea to a very important problem. Moreover, the results are compelling and most reviewers endorse this paper up to the point that they are highly excited. One reviewer is less enthusiastic, but his main concerns (clarity and only small improvement of the performance) have been addressed properly. 
justification_for_why_not_higher_score: Paper could be improved in terms of clarity
justification_for_why_not_lower_score: Impressive results for procedural plan generation using large language models.
Decision: Accept: notable-top-25%